Chache stored_event_count (on Issue and Projet)

"possibly expensive" turned out to be "actually expensive". On 'emu', with 1.5M
events, the counts take 85 and 154 ms for Project and Issue respectively;
bottlenecking our digestion to ~3 events/s.

Note: this is single-issue, single-project (presumably, the cost would be lower
for more spread-out cases)

Note on indexes: Event already has indexes for both Project & Issue (though as
the first item in a multi-column index). Without checking further: that appears
to not "magically solve counting".

This commit also optimizes the .count() on the issue-detail event list (via
Paginator).

This commit also slightly changes the value passed as `stored_event_count` to
be used for `get_random_irrelevance` to be the post-evication value. That won't
matter much in practice, but is slightly more correct IMHO.
This commit is contained in:
Klaas van Schelven
2025-02-06 16:24:25 +01:00
parent d0f2a0c686
commit 615d2da4c8
15 changed files with 150 additions and 12 deletions

View File

@@ -0,0 +1,18 @@
# Generated by Django 4.2.18 on 2025-02-06 10:04
from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
("projects", "0003_project_projects_pr_name_11d782_idx"),
]
operations = [
migrations.AddField(
model_name="project",
name="stored_event_count",
field=models.IntegerField(default=0, editable=False),
),
]

View File

@@ -0,0 +1,23 @@
from django.db import migrations
def unfill_stored_event_count(apps, schema_editor):
Project = apps.get_model("projects", "Project")
Project.objects.all().update(stored_event_count=0)
def fill_stored_event_count(apps, schema_editor):
Project = apps.get_model("projects", "Project")
for project in Project.objects.all():
project.stored_event_count = project.event_set.count()
project.save()
class Migration(migrations.Migration):
dependencies = [
("projects", "0010_project_stored_event_count"),
]
operations = [
migrations.RunPython(fill_stored_event_count, unfill_stored_event_count),
]

View File

@@ -52,6 +52,7 @@ class Project(models.Model):
# denormalized/cached/counted fields below
has_releases = models.BooleanField(editable=False, default=False)
digested_event_count = models.PositiveIntegerField(null=False, blank=False, default=0, editable=False)
stored_event_count = models.IntegerField(blank=False, null=False, default=0, editable=False)
# alerting conditions
alert_on_new_issue = models.BooleanField(default=True)