* As per the spec 'takes precendence'
* Also fixes the reported bug on Laravel, which apparently doesn't send event_id
as part of the event payload.
* Fixes the envelope tests (they were doing nothing when I moved the
data samples around recently)
* Adds a 'no event_id in data, but yes in envelope' test to that test.
* Adds handling to send_json such that we can send envelopes when the event_id
is missing from the event data.
Now that we have eviction, events may disappear. Deal with it:
* event-specific 404 that still allows for navigation
* first/last buttons
* navigation to prev/next when prev/next is not just 1 step away
* don't use HttpRedirect for "lookup based" views
in principle, the thing you were looking for might go missing in-between
drawback: these URLs are not "stable"
* issue.event_count to digested_event_count
* event.ingest_order to event.digest_order
* issue.ingest_order to digest_order
This is generally more correct/explicit, and is also in preparation
of doing work on-digest (which may or may not happen)
The direct cause for this was the following observation: there was no mechanism
in place to safeguard counted events across evictions, i.e. the following order
of events was not accounted for:
* ingest/digest a bunch of events (PCs correctly updated)
* eviction (PC still correct)
* server/snappea restart (PC reloaded, but based on new events. not correct).
I though about various approaches to fix this (e.g. snapshotting) but in the end
such approaches added even more complexity to the PC mechanism. I decided to first
check how non-performant the SQL route would be, and this PoC seems to say: just
go SQL.
There's also a small semantic change (probably in the direction of what you'd
expect), namely: the periods are no longer 'calendar' periods.
Unscientifically (n=1, changing circumstances), this improved times like so when the max was 10k:
* 573.56ms EVICT; down to 8813, max irr. from 15 to 13 in 171ms+402ms and 5+4 queries (pre-index)
* 229.34ms EVICT; down to 7643, max irr. from 15 to 12 in 7ms+222ms and 5+7 queries (post-index)
The order of the index was chosen because we have 3 types of queries in our algo:
* on Project -> irrelevance <= amount of work
* on Project, timestamp -> irrelevance <= observed irrelevances
* on Project, timestamp, irrelevance -> deletion
for performance, but also fixes:
* not just the 'last frame' but the 'last relevant frame' (in-app)
* truncation is properly done (matching the DB size, and for each of the fields)
