164 Commits

Author SHA1 Message Date
Klaas van Schelven
b618575e83 Quota settings: per-month on project level; per-smaller on installation level
and make it configurable on in the templates & docker
2026-01-07 15:41:58 +01:00
Klaas van Schelven
bd35c1efdb Installation quota: pick up on settings-changes
like 7b7cd66dfb but at the installation-level
2026-01-07 15:18:29 +01:00
Klaas van Schelven
1c0d163f92 Express per hour more clear as '1 hour'
now that this info is used in the message string
2026-01-07 14:45:02 +01:00
Klaas van Schelven
7b7cd66dfb Project quota: pick up on settings-changes 2026-01-07 14:02:43 +01:00
Klaas van Schelven
802768f63d Remove 'matter of taste' comment and max()
the amount of extra mental overhead this introduces is not offset by the
gain in correctness
2026-01-07 11:40:54 +01:00
Klaas van Schelven
1d261b4b7c Setting & check for site-wide per-month event ingestion maximum 2026-01-07 11:29:49 +01:00
Klaas van Schelven
fb5424db22 Renames/comment rewrites for understanding (pure refactoring) 2026-01-07 09:23:53 +01:00
Klaas van Schelven
e6c163c674 When minidumps feature is turned off, don't 500 when sent
Fix #293
2026-01-06 19:33:07 +01:00
Klaas van Schelven
5cf185388e Fix exception for unsupported envelope items
See #293
2026-01-06 16:23:45 +01:00
Klaas van Schelven
b7f6331cc6 Fix never_evict for the "conditional ummute" case
The key phrase from our codebase was:

> # .save() will be called by the caller of this function

But this wasn't the case for the conditionally called path.

Adds data-fixing migration too.

Fix #292
2026-01-06 11:07:53 +01:00
Klaas van Schelven
4fe8bd3fad ingest ParseError: don't raise a 500; make this the SDK's problem (400)
500 was just fine when I was still calibrating the envelope parsing, but AFAICT it's
correct now, so I'd rather not get notified about 'remaining problems' (which are SDK problems)
2025-12-09 15:58:48 +01:00
Klaas van Schelven
ec6d480cd1 Cleanup: harmonize import-style for exceptions 2025-12-09 15:56:02 +01:00
Klaas van Schelven
bee889f2cd Raise 413 for the 'content too large' case 2025-11-26 12:45:22 +01:00
Klaas van Schelven
4137565de9 Note about (crashpad/minidump) guid 2025-11-16 19:54:13 +01:00
Klaas van Schelven
8283b80b35 Minidump API Endpoint: custom/extra fields support 2025-11-16 09:29:20 +01:00
Klaas van Schelven
661d83bd93 minidumps: FEATURE flag 2025-11-15 13:33:49 +01:00
Klaas van Schelven
ab065a6329 api_catch_all: header-based
rather than try-and-recover, just look at the headers and show body/POST etc.
this avoids hard-to-reason about situations where either of those won't work
because the other has already been executed; in combination with reasoning
about max size usage the explicit solution is simply easier to reason about.

further:

* makes api_catch_all one of the content_encoding-ready views.
* implement a max length for the ingest api view
2025-11-11 15:25:51 +01:00
Klaas van Schelven
937df4cbb8 minidump endpoint: support content encoding
adds readline() method to GeneratorReader (ChatGPT-generated; eyeballed for
correctness) to match the Django FILES/POST handling expectations.
2025-11-11 13:50:07 +01:00
Klaas van Schelven
54c96eb680 Minidump upload: more explicit errors (and logging) 2025-11-11 09:48:00 +01:00
Klaas van Schelven
690a92a1f9 Merge branch 'main' into minidumps 2025-11-09 21:56:18 +01:00
Klaas van Schelven
a6ead89ca8 Remove event.debug_info
basically unused
2025-11-09 20:58:39 +01:00
Klaas van Schelven
cb8f913cbe Document 2 TODOs 2025-11-05 13:23:31 +01:00
Klaas van Schelven
d807ea2c50 Minidump: via envelope interface
See #82
2025-11-05 11:10:14 +01:00
Klaas van Schelven
d945b39259 Dead code removal
this was inlined in 7f831f52d4
2025-11-05 09:27:10 +01:00
Klaas van Schelven
31596a9b44 /store/ endpoint: non-immediate digestion
this makes this consistent with the work we did in the previous commit
at the price of being slightly more inefficient.

but it's a deprecated endpoint anyway
2025-11-05 09:20:35 +01:00
Klaas van Schelven
7f831f52d4 Remove DIGEST_IMMEDIATELY option
Although DIGEST_IMMEDIATELY=True is theoretically a nice thing to
have, the upkeep is not worth it now that we're about to introduce
minidump ingestion.

The only thing that you're saving is the round-trip via the filesystem,
but performance of that is negligable, and if you're configuring
DIGEST_IMMEDIATELY you're actually _not_ in the performance-critical path
anyway.

Getting rid of it _also_ harmonizes/reduces the number of paths to test.

It's approximately 1% of our installed base.
2025-11-05 09:03:17 +01:00
Klaas van Schelven
391e22bcf0 parser: event_output_stream closing pushed in
in preparation of the minidump handling through the envelope path,
which probably requires dealing with the whole envelope as a whole.

"theoretically" this might be less efficient, but [a] see the notes
at the top of the parser on how we think about streaming parsing and
[b] the inefficiencies are in the "immediate" path anyway (which has
the assumtion that it's not in high-performance envs).

"blind commit" (tests not run against this commit).
2025-11-04 14:51:21 +01:00
Klaas van Schelven
e7aad45db2 Minidumps: PoC for minidump 'endpoint'
See #82
2025-11-04 10:47:04 +01:00
Klaas van Schelven
b0b2573d17 Releases API
Fix #191
See #146
2025-09-11 09:55:15 +02:00
Klaas van Schelven
829cea1a80 detection of a new release through an event ⇏ triggering of a TurningPoint
This more exactly expresses semantics by itself, and is also in preparation of
creating releases through the API (which have no triggering event)

See #146
2025-09-11 09:55:09 +02:00
Klaas van Schelven
4ad3c5efcf Hardening of Temporary-Directory Usage
Defends against certain forms of local privilege escalation, i.e.
understood to be defense in depth rather than a security issue given
the recommended ways of deploying (docker container or in a single-use
single-server)

Fix #174

See https://github.com/python/cpython/pull/23901
2025-08-30 15:10:50 +02:00
Klaas van Schelven
4bf2c1c522 envelope event_id check: on-parse 2025-07-29 15:22:34 +02:00
Klaas van Schelven
6b8d912e1a Store remote_addr on the event
Fix #165
2025-07-25 21:54:32 +02:00
Klaas van Schelven
7b340fd8ff Hide in-progress deletions of Project & Issue from the UI
I've done a full grep on Issue.objects, Project.objects and get_object_or_404
equivelents, and applying some common sense. The goal: avoid having
confusing/half-broken pages in the UI.

On index-usage: I've decided not to update the indexes. The assumption is:
`is_deleted` items will be a tiny minority of items in general, making the
cost/benefit analysis not turn out favorably (just scanning them out as a final
step is more efficient).  Also: sqlite is able to use the correct index without
adding a special one, proof:

```
EXPLAIN QUERY PLAN SELECT [..] WHERE ("issues_issue"."project_id" = 1 AND "issues_issue"."is_muted" = (0) AND "issues_issue"."is_resolved" = (0)) ORDER BY "issues_issue"."last_seen" DESC LIMIT 250;
QUERY PLAN
`--SEARCH issues_issue USING INDEX issue_list_open (project_id=? AND is_resolved=? AND is_muted=?)

EXPLAIN QUERY PLAN SELECT [..] WHERE ("issues_issue"."project_id" = 1 AND "issues_issue"."is_muted" = (0) AND "issues_issue"."is_resolved" = (0) AND "issues_issue"."is_deleted" = 0) ORDER BY "issues_issue"."last_seen" DESC LIMIT 250;
QUERY PLAN
`--SEARCH issues_issue USING INDEX issue_list_open (project_id=? AND is_resolved=? AND is_muted=?)
```

See #139 for the 0/1 notation in the above.

(Project-indexes: not an issue, the scale is "below relevance for indexes")
2025-07-07 10:27:36 +02:00
Klaas van Schelven
6b9e4d8011 Project.delete_deferred(): first version (WIP)
Implemented using a batch-wise dependency-scanner in delayed
(snappea) style.

* no real point-of-entry in the (regular, non-admin) UI yet.
* no hiding of Projects which are delete-in-progress from the UI

* lack of DRY
* some unnessary work (needed in the Issue-context, but not here)
  is still being done.

See #50
2025-07-03 21:01:28 +02:00
Klaas van Schelven
49e6700d4a Grouping.grouping_key: hash it for the index 2025-05-06 11:32:19 +02:00
Klaas van Schelven
cc2b22f08e Digest: check Grouping.exists only once (save a query) 2025-05-06 10:30:51 +02:00
Klaas van Schelven
50343c0b2c Document further thoughts on store-envelopes in DB 2025-04-12 21:28:04 +02:00
Klaas van Schelven
9b1911aded Fix issue.stored_event_count for eviction/retention 2025-03-31 14:51:58 +02:00
Klaas van Schelven
7a19e2d277 Tags; deducing tags; search on tags; WIP 2025-02-27 13:12:49 +01:00
Klaas van Schelven
757ee31bed Support for CORS
Tested in-browser with:

```
function main() {
    $.ajax({
        type: "POST",
        url: "http://bugsink:8000/api/1/store/",
        headers: {
            "Content-Type": "application/json",
            "X-Sentry-Auth": "Sentry sentry_key=a2df4cd647dc4b7a8a81b78a3601eba1, sentry_version=7, sentry_client=bugsink/0.0.1",
        },
        data: JSON.stringify({foo: "Bar"}),
        success: function(data) {
            console.log(data);
        }
    });
}
```
2025-02-20 14:43:10 +01:00
Klaas van Schelven
615d2da4c8 Chache stored_event_count (on Issue and Projet)
"possibly expensive" turned out to be "actually expensive". On 'emu', with 1.5M
events, the counts take 85 and 154 ms for Project and Issue respectively;
bottlenecking our digestion to ~3 events/s.

Note: this is single-issue, single-project (presumably, the cost would be lower
for more spread-out cases)

Note on indexes: Event already has indexes for both Project & Issue (though as
the first item in a multi-column index). Without checking further: that appears
to not "magically solve counting".

This commit also optimizes the .count() on the issue-detail event list (via
Paginator).

This commit also slightly changes the value passed as `stored_event_count` to
be used for `get_random_irrelevance` to be the post-evication value. That won't
matter much in practice, but is slightly more correct IMHO.
2025-02-06 16:24:25 +01:00
Klaas van Schelven
19bb91b636 Fix: set digested_at time correctly 2025-01-31 16:23:09 +01:00
Klaas van Schelven
9ee623de6b Add Event.grouping field and fill it
An event always has a single (automatically calculated) Grouping associated with it.
We add this info to the Event model (we'll soon display it in the UI, and as per the
now-removed comment it's simply the consistent thing to do)
2025-01-31 16:16:07 +01:00
Klaas van Schelven
c42aa9118a Describe role of Grouping and how it relates to Issue
third time's a charm (5e5b53abed, 48307daa0f)
2025-01-31 15:25:18 +01:00
Klaas van Schelven
672bbde4ba Fix DeprecationWarning ('warn') 2025-01-31 13:18:02 +01:00
Klaas van Schelven
452ac806b7 Raise from None
See https://www.bugsink.com/blog/using-raise-from-none-in-python/
in this case: PermissionDenied so clearly implies 'not found in db' that
the details of not finding are truly immaterial
2025-01-29 09:09:05 +01:00
Klaas van Schelven
59372aba33 First version of multi-tenant setup (EE) 2025-01-29 09:04:19 +01:00
Klaas van Schelven
7727bc6168 Show dsn-as-understood when failing to authenticate in error message
status code for PermissionDenied rather than 404 also
2025-01-22 22:15:53 +01:00
Klaas van Schelven
505cae4b18 fix: don't raise non-existing exception
the previously referred type is from the DRF, but we don't use that
2025-01-17 17:04:28 +01:00