The most dangerous cell in a research spreadsheet is the one that looks fine. We think the difference between “nothing filed”, “zero” and “we failed to collect it” deserves more care than most desks give it.
Every research desk that builds datasets from public portals runs into the same quiet problem. A cell is empty, or shows a dash, or shows zero. The spreadsheet treats all three almost the same way. The analysis that follows does not.
A simple analogy helps. A bakery that writes “0” in its sales log on a day it was shut for a holiday has not sold zero loaves; it did not trade. If it writes “0” because the till crashed and nobody entered the day, the log is not recording zero either; it is recording a failure. Averaging the month with those zeros makes sales look worse than they were.
Public market and regulatory data has the same three cases:
| What the cell shows | What it might really mean | What goes wrong if you treat it as zero |
|---|---|---|
| 0 | A real zero: no complaints, no outflows | Nothing; this one is correct |
| Blank or “--” | The entity filed nothing for that month | Averages, totals and rankings are pulled down |
| Blank | Your collection failed: a timeout, a renamed column, a missed page | A data failure is reported as a fact about the market |
The first is information. The second is information of a different kind. The third is not information at all, and it is the one most likely to go unnoticed, because the sheet still looks complete.
Most of this is not carelessness. It comes from ordinary steps in manual collection:
None of these raise an error. That is the problem. A broken process that fails loudly gets fixed the same day. One that fails silently runs into a model for a month.
Whether the collection is manual or automated, we would hold the dataset to a few rules:
These rules are the reason Terminal X works the way it does. Its agents normalise values on the way in, so “12.4%”, “(3.2)” and “--” become 12.4, −3.2 and blank. Names are matched by registration number or ISIN rather than by text. Empty filings are marked as “no data”, not zero. Each run shows a live count of done, pending, errors and “no data filed”, errors retry on their own, and every record keeps its source, fetch time, run and previous version. When a source changes its layout, the change is reported rather than swallowed.
The detail on detecting layout changes is in our schema change detection playbook.
Before a number goes into a model or a client note, it is worth asking one thing: if this were missing, would I know? If the honest answer is no, the dataset needs a status column more than it needs another chart. More on building reliable research datasets is in the Terminal X guides.
This post is about research data workflows, not investment advice. For any investment decision, confirm with your adviser.
Because a blank can mean the entity filed nothing, or that your collection failed. Treating either as zero distorts averages, totals and rankings without any warning.
Empty filings are marked as “no data”, not zero, and each run shows counts of done, pending, errors and “no data filed”. Every record keeps its source, fetch time, run and previous version.
No. It is about how research data is collected and cleaned. For investment decisions, confirm with your adviser.
See it on your own data. Terminal X — Filings and market data, collected. Book a 30-minute working session with an engineer.
