Round 2
What changed, and what was wrong
Between rounds this project was audited against itself, harshly, on the assumption that anything it asserted rather than demonstrated would be found by somebody else first. Most of what follows is not a new feature. It is a defect that was shipping.
Each entry says what was wrong before it says what was done, and links to where you can check the claim. The two that are still open are at the top, because a list of fixes that quietly omits the unfinished work is the kind of thing this project spent the rest of its time removing.
This page is in English only, for the same reason /honesty is: the Hindi elsewhere on this site has not been read by a native speaker, and a page whose job is to be believed is the wrong place for an unreviewed translation.
Still open
Both of these are measurements nobody has taken yet. Neither can be closed by writing code, and neither may be filled in with an estimate.
The sixty-second claim is not proven yet, and the runs that exist do not yet answer it. There are now enough of them for the word median, and they arrived in two groups too far apart for one to mean anything — so the site shows both groups and prints no median. What is missing is not more runs but the reason for the gap: the runs recorded before 8 September were stored as bare durations, with no note of when they happened or whether the person corrected anything before sending.
The harness that records them was broken for the whole of Round 1, which is why there were none at all to argue about; it works now, so the number is being earned rather than asserted. Every run is kept, including the slow ones: the distribution contains a run that came in over sixty seconds, and it stays in and is counted against the claim. Runs from 8 September carry their provenance, so the next time the sample splits the site can say why. Filling the rest with scripted runs would be the exact fabrication this project refuses everywhere else.
See the runs, and how many there areThe comparison against cybercrime.gov.in is still uncounted. Nobody has opened the live portal and counted the fields it asks for before it accepts anything.
The documentary route was tried and failed — the portal's own manuals either cover the helpline route or are image-only. So the column stays empty and says why, rather than carrying an estimate.
See the empty column
Fixed
In the order they would cost a reader confidence, worst first.
The two pages that exist to state the project's limits were the ones overstating. The honesty page called the sixty-second claim “fully demonstrable” while the evidence page said it should not be read as proven; described the portal as a “fourteen-minute form”, a duration nobody has timed; and said no human run-throughs had been recorded while the evidence page was displaying two of them. The evidence page opened by calling the claim “measured against the same task on the live portal” — a comparison it refuses to make four sections further down its own page, because nobody has counted the portal.
All four removed. Both pages now say the sixty seconds is measured rather than asserted and point at the runs and their sample size; the portal is described by what it asks for instead of how long it takes; and the run entry no longer restates a count it cannot keep current. Of everything found this round these were the most expensive, because these are the pages a reader checks to find out whether the rest is posture.
Read them and check them against each otherThe most important path in the product had never been measured. The interrupt — a supporting feature — had a rigorous evaluation, 22 labelled cases and five disclosed limits. Extraction, which is the thing a person actually comes here to do, had none, and the screenshot path that the front page leads with had never been run once. The project measured its second-most important feature and not its most important one.
There is now an extraction evaluation over generated screenshots and text in English, Hinglish, Hindi and an unpunctuated dictation transcript. Across three passes: 74 of 75 fields correct, and the screenshot path 40 of 40 with nothing invented — including three images where a field was cut out entirely, so any value returned for it could not have been read.
The numbers and their limitsOne value got through wrong, and it is the one that matters. Given a dictation transcript saying “fastcart dot pay at samplebank”, the model returned fastcart@samplebank — a different account, and a perfectly well-formed handle. The server checks shape, so it had nothing to object to, and the value went into the packet.
Nothing automatic, deliberately. Shape validation catches malformed values and cannot catch plausible ones; that is now a measured limit stated on the honesty page rather than a suspicion. The mitigation that exists is the confirm screen, where the handle is shown back and can be corrected before sending. A guard built in three days against a single observed case would either fire constantly or never.
Read the limitThe judging harness did not work. Pressing “Start the run” at /judge began a stopwatch over a frame that never loaded — a clock counting up against an empty box. It had been shipping that way since the page was written, and it is the page the submission offers reviewers as the guided run.
The iframe was rendered only once a run started, so the ref was still null in the click handler that set its source and the frame kept the blank placeholder from its markup. It is now always mounted and driven by an effect. This is also why no run had ever been recorded: none could finish.
Time a run yourselfThe front page asserted a number the evidence page refuses to guess. It opened with “about fifty facts”, while /evidence says in as many words that guessing the portal's field count would discredit everything next to it.
Removed from the landing page, the summary and the video script. The sentence now describes what the portal does without counting it. A figure goes back only when a person has counted one.
Read the claim and its limitsThe front page then did the same thing again, with its own number. When the first real runs landed, /evidence learned that a handful of runs is not a distribution and started showing a caveat below five. The landing tile had only ever been taught about the case of exactly one — so at two runs it went back to reading “Median time to dispatch”, while the page one tap away said two runs “is not yet a distribution”. The same figure, described more confidently on the screen a reader reaches first.
The threshold and the labelling rule moved into one module that both pages read, so they cannot drift apart again. Below five runs neither of them uses the word median, and neither does the tile at zero.
Compare the twoThe stopwatch behind the headline claim did not start when the person started. Tapping “Add a screenshot” opened the file picker without starting the clock — the clock started when the file came back. So the time spent hunting for the debit alert in a gallery, on the input path the product leads with and the video opens on, was outside the measurement. Dictation had the same shape: tapping the button was free, and only the finished transcript started it.
Both now start the clock on the tap. The measured times get longer and the sixty-second claim gets harder to meet, which is the correct direction — the provider's own comment says timing from page load would flatter the number, and this was flattering it in the same way one screen further in. It also means the five runs already on record are undercounted by an unknown amount, most likely the three fastest ones, and /evidence says so next to them rather than leaving the split unexplained.
Read what the runs do and do not showThen it happened a third time, and having enough runs is what caused it. The fifth real run landed, the sample cleared the threshold above, and both pages started reporting “Median 12.4s” over runs of 7.7, 9.4, 12.4, 49.5 and 64.5 seconds. Those are two groups with a thirty-seven-second hole in the middle, and the median was sitting in the hole — describing neither group, and quietly underselling the sixty-second claim by a factor of five while appearing to support it. Every rule the site had was about how many runs there were. None was about whether they were the same thing.
The rule is now in the code rather than in a reader's judgement: if the widest gap between neighbouring runs is wider than every other gap put together, with at least two runs each side, the sample is two groups and neither page prints a median for it. The tile is removed rather than relabelled — the same treatment the unsourceable recovery percentage got. Both groups are shown with their ranges, and the page states plainly that it cannot yet account for the gap, because the runs were recorded as bare numbers. That last part is the actual defect: the site could publish a distribution it had no way to explain.
See both groupsThe honesty page was unreachable. /honesty and /judge were linked from nowhere — not the header, not the footer, not the landing page. A disclosure nobody can navigate to is not a disclosure.
Both are in the footer on every screen and on the landing page.
What is real and what is notThe interrupt could not fire on the product's most likely input. A screenshot of a debit alert carries no evidence about whether the caller is still on the line, and there was nowhere to say so afterwards — so a screenshot-only report could never trip the stop screen. The route built for exactly this had no caller outside the test script.
The confirm screen now takes an optional sentence and triages it through the same gate. It cannot block the send button: dispatch before it answers and the packet goes.
Try it on a demo caseThe measured false-positive rate described a code path the product never ran. The number came from one route; the report flow calls a different one, with a different prompt, asking the same question alongside nine other fields. The two were assumed equivalent because both are built from one shared paragraph.
The evaluation now scores both and leads with the one a user actually hits. They agreed on all 22 cases with identical verdicts, so the old claim turns out to have been true — it had just never been tested, which is a different thing, and the honesty page now says so in those words.
The numbers and their limitsNo demo could show the anti-hallucination layer working. Every demo case's blanks came from the model declining to read a field — which demonstrates the prompt behaving, not the server refusing a confident wrong answer. The strongest thing in the codebase was the one thing a reviewer could not watch happen.
A demo case now returns an eleven-digit reference at high confidence. The server refuses it on shape, the confirm screen shows the rejected value, and the receipt lists the field as sent blank. A test fails if that case stops being refused.
Run “A confident misread”A misread timestamp could not be corrected. The time control appeared only when the model failed to read one — so if it read one wrongly, the field that drives the meter and that a bank reads first was stuck.
The control is always shown, with what was read stated in words above it.
Start a reportThe receipt counted a blank as sent. An unread payment method is stored as “unknown”, which is not a value a bank can use, but the count treated it as one — so an entirely empty packet reported one of nine fields sent.
Both the confirm screen and the receipt now treat those placeholders as the holes they are. On the screen whose argument is that it counts its holes honestly, that count should not have been the generous one.
The comparison table hid its own column on a phone. It is 32rem across at its narrowest and a 360px screen gives it 20.5, so it scrolled sideways inside its own container — and the column that went over the edge was Golden Hour's. The page's central comparison needed a swipe to see half of, on the width this design commits to.
The rows stack below the small breakpoint, so both columns are on screen at once, and stay a table above it where both fit anyway. The comparison is the argument; it should not have needed a gesture.
See it at phone widthThe clock counted in English on Hindi screens. The meter's units came from the arithmetic module, which is English by design because it is the tested one.
The units are looked up from the copy dictionary instead. The Hindi is still unreviewed and still says so.