Parser audit against MALFORMED-300

We run your parser over the same 300-case corpus, with the same harness and the same grading spec that produced the public leaderboard, and publish a private report page for you.

What the report contains

Your score per failure category — 12 categories, 25 cases each.
Every case id your parser fails, with the input, the expected value and what you returned.
Whether your parser invents values for the 25 unrecoverable cases — the cases where the model produced nothing recoverable and the only correct behaviour is to refuse.
Where you land against the 21 parsers already measured on the leaderboard (11 Python, 10 JavaScript).

What we need from you

Two fields, both collected at checkout:

field	what to put in it
`package`	pip/npm name or public git URL
`entrypoint`	function to call, e.g. mylib:parse

Requires a parser that installs and runs from a public package or public git URL on Python 3.12 or Node 20, with no credentials and no network access at run time.

Turnaround

The report is published at a private URL within 48 hours. If the code cannot be installed and run on a clean machine, the report says exactly that and the payment is refunded.

The corpus you are measured on

300 cases: 275 recoverable, 25 unrecoverable. SHA-256 of the corpus file ded36c275f9ff481a9e264f9e900ccbb16da9cf430ff6fe8ffc23b9c02acb88a. Grading is the published spec — an unrecoverable case passes only by refusing, and returning {} for it is a failure, not a partial credit. For scale, on this corpus the Python standard library's json.loads scores 25/300, the strongest Python entry scores 282/300 and the strongest JavaScript entry scores 205/300. Twelve of the cases, one per category, and the scorer itself are free and CC0, so you can see the format before you buy anything.

Buy the audit

Parser audit — €49

Payment processed by Stripe. Prices include VAT where applicable. After payment you are sent to a page carrying your audit id; keep that URL, it is where the report is announced.