Chache stored_event_count (on Issue and Projet)

"possibly expensive" turned out to be "actually expensive". On 'emu', with 1.5M
events, the counts take 85 and 154 ms for Project and Issue respectively;
bottlenecking our digestion to ~3 events/s.

Note: this is single-issue, single-project (presumably, the cost would be lower
for more spread-out cases)

Note on indexes: Event already has indexes for both Project & Issue (though as
the first item in a multi-column index). Without checking further: that appears
to not "magically solve counting".

This commit also optimizes the .count() on the issue-detail event list (via
Paginator).

This commit also slightly changes the value passed as `stored_event_count` to
be used for `get_random_irrelevance` to be the post-evication value. That won't
matter much in practice, but is slightly more correct IMHO.
This commit is contained in:
Klaas van Schelven
2025-02-06 16:24:25 +01:00
parent d0f2a0c686
commit 615d2da4c8
15 changed files with 150 additions and 12 deletions

View File

@@ -103,8 +103,10 @@ def _make_message_body():
"project_count": Project.objects.count(),
"team_count": Team.objects.count(),
# event-counts [per some interval (e.g. 24 hours)] is a possible future enhancement. If that turns out to be
# expensive, one thing to consider is pulling _make_message_body() out of the immediate_atomic() block.
# event-counts [per some interval (e.g. 24 hours)] is a possible future enhancement. We've already seen that
# such counts are expensive though, but if _make_message_body() is pulled out of the immediate_atomic()
# block (which is OK, since there is no need for some kind of 'transactional consistency' to register this
# simple metadata fact), then it might be OK to add some more expensive queries here.
},
}