Every night a pipeline of mine pulls SEC filings, insider transactions, options activity and news for a few hundred companies, scores them, clusters the signals, and has a model write a short summary of what changed. It does the part that used to need a person reading filings.
It ran unattended for months. Nothing crashed, no error rate moved, and the summaries were fluent and specific the entire time some of them were wrong.
The interesting part is not that models hallucinate. It is that a pipeline can be correct in every line of code, protected by every guardrail you would think to add, and still publish false statements every night with no signal anywhere that something is off.
Three mechanisms did most of the damage. None of them is a model failure.
1. The grounding rule that preserved a lie
Each nightly prompt includes a static, human-written background note on the company, context I typed in at some point and forgot. The prompt also carries the guardrail everyone building on models converges to within a week: only cite numbers present in the input, do not invent figures.
That rule is right. In this configuration it also turns stale facts into fresh-looking claims.
One infrastructure company’s note read “$39.2B backlog. JPMorgan Overweight, $515 target.” True when I wrote it. By the time I audited, the backlog was $48.5 billion and the stock traded near $660, well above the price target still being quoted as forward-looking.
Every summary that company produced, every night across the window I checked, faithfully repeated “$39.2B backlog; JPMorgan OW $515.”
The model did nothing wrong. It was told to ground itself in the provided numbers and did exactly that. The direction of the effect is what surprised me: a less constrained model, drawing on general knowledge, might have hedged or contradicted the stale figure. The anti-hallucination rule is what guaranteed the stale number survived intact, night after night, in confident prose.
Grounding buys fidelity to the input. It buys nothing about whether the input is still true, and by construction it removes the one mechanism that might have caught the problem. Anything injected as ground truth needs a date on it, and the model needs permission to discount what is old.
My first theory was drift: each night’s summary feeding the next, degrading like a photocopy of a photocopy. The prompt does feed the previous summary, so it was plausible. But the numbers were not degrading. They were pinned exactly to the static field, and a drift fix would have changed nothing.
2. A category code that bundles opposite events
The SEC’s Form 8-K reports material corporate events, tagged by item number. Item 5.02 reads:
Departure of Directors or Certain Officers; Election of Directors; Appointment of Certain Officers; Compensatory Arrangements of Certain Officers.
One number. A CEO getting fired, a CEO getting hired, and a routine compensation adjustment all file under it.
My ingest saw 5.02, emitted an executive-transition event, and left the
departure type null because nothing had read the actual filing. Downstream the
scorer received a transition with an unknown departure type and returned
bearish, confidence 50. Since the caller never populated departure type, that
was the only branch that could ever run.
Live, over 90 days: 151 executive-transition events, 151 of them bearish, zero neutral and zero bullish, across 119 different companies. A company announcing a strong new CFO scored identically to one whose CFO walked out.
The general form of this is not specific to filings. Any time you key off someone else’s enumeration, you inherit their reasons for drawing the boundaries where they did. The SEC groups those three events because they are all disclosures about officers, which is a filing-burden category rather than a sentiment category. It was never designed to carry the distinction I needed, and no amount of downstream scoring sophistication recovers information the taxonomy already threw away. Either the parse reads the document, or the system admits it does not know.
3. An identifier without its referent
The clustering step asks a small, cheap model to compress several signals into one readable sentence. The prompt passed the ticker symbol and the signal lines. It did not pass the company name.
So the model filled in the blank. Live in the alerts table:
“Stacks (STX) is showing consistent bullish sentiment…”
STX is Seagate Technology. Other rows invented “Stackwise.” One row said “Seagate Technology,” correctly.
That inconsistency is what makes it hard to catch. An unanchored identifier does not fail cleanly. It fails at whatever rate the model happens to recall the mapping, which presents as flakiness rather than as a missing input. Had it been wrong every time I would have caught it in a day. Being right some of the time is what let it survive.
It is also invisible in code review. The prompt template looks complete. Every variable in it is populated, and every value is correct. The defect is a variable that isn’t there, and there is no linter for the field you forgot to interpolate.
The number I noticed was not the number that was wrong
The audit started from one symptom: a stock showing a strongly negative composite score, day after day, that didn’t match its actual news. The obvious suspect was the Item 5.02 bug above, which is real, universe-wide, and exactly the kind of thing that produces spurious bearishness.
It was the wrong explanation. Decomposing that score:
- News-sentiment signals: 23 of them, average confidence 75, roughly 300 to 375 of the negative total after per-type caps. The dominant term by a wide margin.
- Executive-transition signals: 2, at 50 each, about 100. A quarter.
- Insider trades (64) and unusual options (21): all classified neutral, contributing nothing in either direction.
The alert’s own generated text said it outright: a bearish shift in news coverage. Fixing the bug I had correctly identified would have moved the number by a quarter and left the anomaly sitting there looking almost as wrong.
A composite score is a sum, and the term you can name is not necessarily the one that dominates it. Two of the four inputs were contributing zero, which I would never have guessed and which is arguably a bigger design problem than the bug I went in looking for. If you can’t decompose the number, you are picking fixes by feel.
What the three have in common
In every case the output stayed well-formed.
A stale backlog figure is a valid number in a fluent sentence. A wrong company name is a real company in correct grammar. A universally bearish signal is a plausible reading of a real filing. There is no exception, no null, no malformed response, no latency spike, nothing a monitor could page on. The system’s confidence was identical on the days it was right and the days it wasn’t.
Conventional software mostly fails by stopping. This kind fails by continuing, producing exactly the artifact you asked for from a premise that stopped being true in March. The engineering problem is not making the model more accurate, since the model was faithfully executing every instruction it was given. It is that a pipeline like this has no natural place where wrongness becomes visible, so you have to build one deliberately, and it has to check the inputs rather than the outputs.
I don’t have a clean answer yet. Date-stamping injected facts, passing referents alongside identifiers, and decomposing any score before trusting it cover the three specific holes above. Monitoring a system that fails by producing fluent text is the part I’m still working on.