jlengrand/bugsink
1
0
Fork 0
mirror of https://github.com/jlengrand/bugsink.git synced 2026-03-10 08:01:17 +00:00
Code Issues Packages Projects Releases Wiki Activity

Semi-manual squash-migrations

Browse Source 
## Goal

Reduce the number of migrations for _fresh installs_ of Bugsink. This implies: squash as
broadly as possible.

## How?

"throw-away-and-rerun". In particular, for a given app:

* throw away the migrations from some starting point up until and including the last one.
* run "makemigrations" for that app. Django will see what's missing and just redo it
* rename to 000n_b_squashed or similar.
* manually set a `replaces` list on the migration to the just-removed migrations
* manually check dependencies; check that they are:
    * as low as possible, e.g. an FK should only depend on existence. this reduces the
      risk of circular dependencies.
    * pointing to "original migrations", i.e. not to a just-created squashed migration.
      because the squashed migrations "contain a lot" they increase the risk of circular
      dependencies.
* restore (git checkout) the thrown-away migration

## Further tips:

* "Some starting point" is often not 0000, but some higher number (see e.g. the outcome
  in the present commit). Leaving the migrations for creation of base models (Event,
  Issue, Project) in place saves you from a lot of circular dependency problems.
* Move db.sqlite3 out of the way to avoid superfluous warnings.

## RunPython worries

I grepped for RunPython in the replaced migrations, with the following results:

* phonehome's create_installation_id was copied-over to the squashed migration.
* all others where ignored, because:
    * they "do something with events", i.e. only when events are present will they have
      an effect. This means they are no-ops for _new installs_.
    * for existing installs, for any given app, they will only be missed (replaced) when
      the first replaced migration is not yet executed.

I used the following command (reading from the bottom) to establish that this means only
people that did a fresh install after 8ad6059722 (June 14, 2024), but before
c01d332e18 (July 16) _and then never did any upgrades_ would be affected. There are no
such people.

git log --name-only \
    events/migrations/0004_event_irrelevance_for_retention.py \
    issues/migrations/0004_rename_event_count_issue_digested_event_count.py \
    phonehome/migrations/0001_initial.py \
    projects/migrations/0002_initial.py \
    teams/migrations/0001_initial.py

Note that the above observation still be true for the next squashmigration (assuming
squashing starting at the same starting migrations).

## Cleanup of the replaced migrations

Django says:

> Once you’ve squashed your migration, you should then commit it alongside the
> migrations it replaces and distribute this change to all running instances of your
> application, making sure that they run migrate to store the change in their database.

Given that I'm not in control of all running instances of my application, this means the
cleanup must not happen "too soon", and only after announcing a migration path ("update
to version X before updating to version Y").

## Roads not taken

Q: Why not just do squashmigrations? A: It didn't work reliably (for me), presumably b/c
of the high number of strongly interdependant apps in combination with some RunPython.

Seen after I was mostly done, not explored seriously (yet):

* https://github.com/3YOURMIND/django-replace-migrations
* https://pypi.org/project/django-squash/
* https://django-extensions.readthedocs.io/en/latest/delete_squashed_migrations.html
This commit is contained in:
Klaas van Schelven
2025-02-03 15:17:18 +01:00
parent 0ec809cbb3
commit 0b42d3ff1e
5 changed files with 378 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

97
events/migrations/0004_b_squashed.py Normal file
View File

@@ -0,0 +1,97 @@
# Generated by Django 4.2.18 on 2025-02-03 13:17
from django.db import migrations, models
import django.db.models.deletion
import django.utils.timezone
class Migration(migrations.Migration):
replaces = [
('events', '0004_event_irrelevance_for_retention'),
('events', '0005_event_events_even_project_abe572_idx'),
('events', '0006_event_never_evict'),
('events', '0007_set_never_evict'),
('events', '0008_remove_event_events_even_project_abe572_idx_and_more'),
('events', '0009_event_events_even_issue_i_90497b_idx'),
('events', '0010_rename_ingest_order_event_digest_order_and_more'),
('events', '0011_remove_event_events_even_project_adcdee_idx_and_more'),
('events', '0012_event_ingested_at'),
('events', '0013_harmonize_foogested_at'),
('events', '0014_event_grouping'),
('events', '0015_set_event_grouping'),
]
dependencies = [
("events", "0003_initial"), # the previous migration
("issues", "0001_initial"), # This defines the Grouping model, which the below FKs to
# This is what Django auto-detected, but I think it's too restrictive:
# ("issues", "0007_alter_turningpoint_options"),
# Additionally, Django auto-detected the below, presumably because we add a (project, event_id) unique
# constraint. However, we don't actually depend on this migration when adding that constraint, so we don't need
# to depend on it here.
# ("projects", "0002_b_squashed_initial"),
]
operations = [
migrations.RenameField(
model_name="event",
old_name="ingest_order",
new_name="digest_order",
),
migrations.RenameField(
model_name="event",
old_name="server_side_timestamp",
new_name="digested_at",
),
migrations.AlterUniqueTogether(
name="event",
unique_together={("project", "event_id"), ("issue", "digest_order")},
),
migrations.AddField(
model_name="event",
name="grouping",
field=models.ForeignKey(
null=True,
on_delete=django.db.models.deletion.SET_NULL,
to="issues.grouping",
),
),
migrations.AddField(
model_name="event",
name="ingested_at",
field=models.DateTimeField(default=django.utils.timezone.now),
preserve_default=False,
),
migrations.AddField(
model_name="event",
name="irrelevance_for_retention",
field=models.PositiveIntegerField(default=0),
preserve_default=False,
),
migrations.AddField(
model_name="event",
name="never_evict",
field=models.BooleanField(default=False),
),
migrations.AddIndex(
model_name="event",
index=models.Index(
fields=[
"project",
"never_evict",
"digested_at",
"irrelevance_for_retention",
],
name="events_even_project_ac6fc7_idx",
),
),
migrations.AddIndex(
model_name="event",
index=models.Index(
fields=["issue", "digested_at"], name="events_even_issue_i_b18956_idx"
),
),
]

44
issues/migrations/0004_b_squashed.py Normal file
View File

@@ -0,0 +1,44 @@
# Generated by Django 4.2.18 on 2025-02-03 13:29
from django.db import migrations, models
class Migration(migrations.Migration):
replaces = [
('issues', '0004_rename_event_count_issue_digested_event_count'),
('issues', '0005_rename_ingest_order_issue_digest_order_and_more'),
('issues', '0006_issue_next_unmute_check'),
('issues', '0007_alter_turningpoint_options'),
]
dependencies = [
("projects", "0002_b_squashed_initial"),
("issues", "0003_alter_turningpoint_triggering_event"),
]
operations = [
migrations.AlterModelOptions(
name="turningpoint",
options={"ordering": ["-timestamp", "-id"]},
),
migrations.RenameField(
model_name="issue",
old_name="ingest_order",
new_name="digest_order",
),
migrations.RenameField(
model_name="issue",
old_name="event_count",
new_name="digested_event_count",
),
migrations.AlterUniqueTogether(
name="issue",
unique_together={("project", "digest_order")},
),
migrations.AddField(
model_name="issue",
name="next_unmute_check",
field=models.PositiveIntegerField(default=0),
),
]

61
phonehome/migrations/0001_b_squashed_initial.py Normal file
View File

@@ -0,0 +1,61 @@
# Generated by Django 4.2.18 on 2025-02-03 12:06
from django.db import migrations, models
import uuid
def create_installation_id(apps, schema_editor):
Installation = apps.get_model("phonehome", "Installation")
Installation.objects.create() # id is implied (it's a uuid)
class Migration(migrations.Migration):
replaces = [
('phonehome', '0001_initial'),
('phonehome', '0002_create_installation_id'),
('phonehome', '0003_outboundmessage'),
('phonehome', '0004_installation_created_at'),
('phonehome', '0005_installation_silence_email_system_warning'),
]
initial = True
dependencies = []
operations = [
migrations.CreateModel(
name="Installation",
fields=[
(
"id",
models.UUIDField(
default=uuid.uuid4,
editable=False,
primary_key=True,
serialize=False,
),
),
("created_at", models.DateTimeField(auto_now_add=True)),
("silence_email_system_warning", models.BooleanField(default=False)),
],
),
migrations.RunPython(create_installation_id),
migrations.CreateModel(
name="OutboundMessage",
fields=[
(
"id",
models.BigAutoField(
auto_created=True,
primary_key=True,
serialize=False,
verbose_name="ID",
),
),
("attempted_at", models.DateTimeField(auto_now_add=True)),
("sent_at", models.DateTimeField(null=True)),
("message", models.TextField()),
],
),
]

87
projects/migrations/0002_b_squashed_initial.py Normal file
View File

@@ -0,0 +1,87 @@
# Generated by Django 4.2.18 on 2025-02-03 13:08
from django.conf import settings
from django.db import migrations, models
import django.db.models.deletion
class Migration(migrations.Migration):
replaces = [
('projects', '0002_initial'),
('projects', '0003_project_retention_max_event_count'),
('projects', '0004_project_quota_exceeded_until'),
('projects', '0005_project_ingested_event_count'),
('projects', '0006_initial_ingested_count_value'),
('projects', '0007_rename_ingested_event_count_project_digested_event_count'),
('projects', '0008_project_next_quota_check'),
('projects', '0009_alter_project_visibility'),
]
dependencies = [
migrations.swappable_dependency(settings.AUTH_USER_MODEL),
("teams", "0001_initial"), # defines the Team model, which we have a FK to
("projects", "0001_initial"), # this is the previous migration
]
operations = [
migrations.AddField(
model_name="project",
name="digested_event_count",
field=models.PositiveIntegerField(default=0, editable=False),
),
migrations.AddField(
model_name="project",
name="next_quota_check",
field=models.PositiveIntegerField(default=0),
),
migrations.AddField(
model_name="project",
name="quota_exceeded_until",
field=models.DateTimeField(blank=True, null=True),
),
migrations.AddField(
model_name="project",
name="retention_max_event_count",
field=models.PositiveIntegerField(default=10000),
),
migrations.AddField(
model_name="project",
name="team",
field=models.ForeignKey(
null=True, on_delete=django.db.models.deletion.SET_NULL, to="teams.team"
),
),
migrations.AddField(
model_name="project",
name="users",
field=models.ManyToManyField(
blank=True,
through="projects.ProjectMembership",
to=settings.AUTH_USER_MODEL,
),
),
migrations.AddField(
model_name="projectmembership",
name="user",
field=models.ForeignKey(
default=1, # unused; this migration is run against no-rows
on_delete=django.db.models.deletion.CASCADE,
to=settings.AUTH_USER_MODEL,
),
preserve_default=False,
),
migrations.AlterField(
model_name="project",
name="visibility",
field=models.IntegerField(
choices=[(1, "Joinable"), (10, "Discoverable"), (99, "Team Members")],
default=99,
help_text="Which users can see this project and its issues?",
),
),
migrations.AlterUniqueTogether(
name="projectmembership",
unique_together={("project", "user")},
),
]

89
teams/migrations/0001_b_squashed_initial.py Normal file
View File

@@ -0,0 +1,89 @@
# Generated by Django 4.2.18 on 2025-02-03 12:06
from django.conf import settings
from django.db import migrations, models
import django.db.models.deletion
import uuid
class Migration(migrations.Migration):
replaces = [
('teams', '0001_initial'),
('teams', '0002_initial'),
('teams', '0003_alter_team_visibility'),
('teams', '0004_remove_team_slug'),
]
initial = True
dependencies = [
migrations.swappable_dependency(settings.AUTH_USER_MODEL), # Defines AUTH_USER_MODEL, which we use
]
operations = [
migrations.CreateModel(
name="Team",
fields=[
(
"id",
models.UUIDField(
default=uuid.uuid4,
editable=False,
primary_key=True,
serialize=False,
),
),
("name", models.CharField(max_length=255, unique=True)),
(
"visibility",
models.IntegerField(
choices=[(1, "Joinable"), (10, "Discoverable"), (99, "Hidden")],
default=10,
help_text="Which users can see this team and its issues?",
),
),
],
),
migrations.CreateModel(
name="TeamMembership",
fields=[
(
"id",
models.BigAutoField(
auto_created=True,
primary_key=True,
serialize=False,
verbose_name="ID",
),
),
(
"send_email_alerts",
models.BooleanField(blank=True, default=None, null=True),
),
(
"role",
models.IntegerField(
choices=[(0, "Member"), (1, "Admin")], default=0
),
),
("accepted", models.BooleanField(default=False)),
(
"team",
models.ForeignKey(
on_delete=django.db.models.deletion.CASCADE, to="teams.team"
),
),
(
"user",
models.ForeignKey(
on_delete=django.db.models.deletion.CASCADE,
to=settings.AUTH_USER_MODEL,
),
),
],
options={
"unique_together": {("team", "user")},
},
),
]