# Feed contract for importers (schema_version 1)

Producer: `earnings_dates.py`, run by `run.sh` from the ec2-user crontab on the plant box at 04:50 and 14:50
Pacific, Monday to Friday. Each run rewrites all three outputs.

| output | local path (plant box) | URL |
|---|---|---|
| JSON (preferred for import) | `/var/www/apps.inncommand.com/earnings-dates/earnings_dates.json` | https://apps.inncommand.com/earnings-dates/earnings_dates.json |
| CSV (same content, flat) | `/var/www/apps.inncommand.com/earnings-dates/earnings_dates.csv` | https://apps.inncommand.com/earnings-dates/earnings_dates.csv |
| page | `/var/www/apps.inncommand.com/earnings-dates/index.html` | https://apps.inncommand.com/earnings-dates/ |

An importer on the same box should read the local file. Files are written to a temp name and renamed into
place, so a reader never sees a partial file. Read the whole file each time.

## Trigger instead of polling

After every successful refresh `run.sh` executes `/etc/earnings-dates/post_run.sh` if that file exists and
is executable, passing the JSON path as `$1`. Its output goes to `run.log` in the feed directory. Put the
importer (or a one-line call to it) there. It runs as `ec2-user`.

Do not place scripts, config or credentials anywhere under `/var/www/apps.inncommand.com/`. That tree is
served over HTTPS as plain files with no authentication.

## Document shape (JSON)

```
{
  "schema_version": 1,
  "generated_at": "2026-09-08T23:28:14-07:00",      ISO 8601 with offset
  "as_of_date": "2026-09-08",                       the date the estimate was made for
  "universe": "...",                                 human text
  "source": "...",
  "enums": { "status": [...], "confidence": [...], "date_kind": [...], "bell": [...] },
  "symbols": [ <record>, ... ],                      one record per (symbol, fiscal_quarter)
  "errors":  [ {"symbol": "...", "company": "...", "error": "..."} ]
}
```

## Record fields

| field | type | meaning |
|---|---|---|
| `symbol` | string | SEC-style ticker: `BRK-B`, `GOOGL`. Use with `fiscal_quarter` as the import key. |
| `symbols_all` | string[] (CSV: `;`-joined) | every share class of the issuer, e.g. `["GOOG","GOOGL"]` |
| `cik` | int | SEC Central Index Key, the stable issuer id |
| `company` | string | SEC registrant name |
| `fiscal_quarter` | string | `Q3 FY2027`; `fiscal_year` and `fiscal_quarter_num` carry the same as ints (JSON only) |
| `period_end` | date | fiscal period end being reported (projected for the target quarter) |
| `status` | enum | `estimated`; `reported` (Google Finance's scheduled date within 21 days, or a single news report); `confirmed` (the company's own announcement found, two independent reports agreeing, or a manual entry in `confirmed_dates.csv`). CSV also uses `error`. |
| `confidence` | enum | `high` (top share >= 85%), `medium` (>= 70%), `low` (several days shown), `reported`, `confirmed` |
| `date_kind` | enum | `single`: one day; `range`: several plausible days, see `candidates` |
| `primary_date` | date | most likely day; `primary_weekday` is its `Mon`..`Sun` |
| `date_min`, `date_max` | date | envelope of the listed candidate days; equal to `primary_date` when `single` |
| `candidates` | array (CSV: `candidate_dates`, `candidate_weekdays`, `candidate_shares`, `;`-joined, same order) | `{date, weekday, share, holiday_adjusted}` sorted by share desc; shares are fractions of history weight and sum to <= 1 because days under 10% are dropped. Days are exact, not a contiguous span: NVDA may list two Wednesdays a week apart. |
| `bell` | enum | `BMO` before 09:30 ET, `AMC` after 16:00 ET, `weekend` (Berkshire, Saturday morning), `intraday` (rare) |
| `bell_share` | number 0..1 | history weight behind the bell call |
| `last_release` | object (CSV: `last_release_date`, `last_release_bell`, `last_release_quarter`) | the company's most recent actual release from EDGAR; `ten_q_pending` true when the 8-K is in but the 10-Q is not yet |
| `based_on` | string | `filing history` for the next quarter; `estimate for Q..` for chained quarters when the feed is run with `--quarters 2` |
| `confirmation` | object or null (CSV: `confirmation_method`, `confirmation_source`, `confirmation_url`) | present when `status` is `reported` or `confirmed`: `{status, method, source, url, headline, found_at}`. `method` is `press release`, `news report`, `vendor schedule` or `manual`. When set, `primary_date`, `date_min`, `date_max`, `candidates` and `bell` carry the confirmed values. |
| `vendor_schedule` | object or null (CSV: `vendor_date`, `vendor_time_et`, `vendor_bell`) | Google Finance's scheduled event for the quarter, fetched once a day for every symbol: `{source, date, time_et, bell, fiscal_quarter, period_end, fetched_at, url}`. Informational until it is within 21 days, then it drives `status = reported` unless a company announcement overrides it. Google does not flag whether its date is company-confirmed. |
| `estimate` | object or null (JSON only) | the model's own estimate as it stood before a confirmation replaced it: `{primary_date, date_min, date_max, candidates, bell, bell_share, confidence}`. Use it to score the estimator. |
| `pattern` | string | human summary of offsets and weekdays used |
| `note` | string | empty, or a warning such as all candidate days being past |
| `generated_at`, `schema_version` | | CSV only (top-level in JSON) |

All dates are `YYYY-MM-DD` on the US trading calendar. Field names and enum values are stable within a
schema version; fields may be added without a bump; a change of meaning bumps `schema_version`.

## Import semantics

* Key: `(symbol, fiscal_quarter)`. Upsert on every run; a row's dates and shares move as filings arrive.
* When a company reports, the next run emits the following quarter for that symbol. The previous row stays
  in the database as history and its `last_release` on the new row tells you what actually happened, which
  lets you score the estimate.
* A row with `status = confirmed` carries the company's announced date; treat it as authoritative. A row with
  `status = reported` carries a vendor or press date that is very likely right but not company-sourced.
* The confirmation step runs once a day per symbol: Google Finance for every symbol, and the press-release
  search once a release is within 30 days of the earliest estimated day (`--confirm-window`). Results persist
  in `confirm_state.json`; automatic confirmations are also listed in `confirmed_auto.csv`; the search log is
  `confirm.log`.
* `errors` lists symbols the producer could not estimate; keep the previous row rather than deleting it.

Suggested MariaDB table (the plant box runs MariaDB 11.4):

```sql
CREATE TABLE IF NOT EXISTS earnings_estimates (
  symbol            VARCHAR(12)  NOT NULL,
  fiscal_quarter    VARCHAR(12)  NOT NULL,
  cik               INT UNSIGNED NOT NULL,
  company           VARCHAR(100) NOT NULL,
  period_end        DATE         NOT NULL,
  status            ENUM('estimated','reported','confirmed','error') NOT NULL,
  confidence        ENUM('high','medium','low','reported','confirmed') NULL,
  date_kind         ENUM('single','range') NULL,
  primary_date      DATE NULL,
  date_min          DATE NULL,
  date_max          DATE NULL,
  candidates_json   JSON NULL,
  bell              ENUM('BMO','AMC','weekend','intraday') NULL,
  bell_share        DECIMAL(5,4) NULL,
  last_release_date DATE NULL,
  last_release_bell VARCHAR(10) NULL,
  based_on          VARCHAR(40) NULL,
  note              VARCHAR(255) NULL,
  confirmation_method VARCHAR(20) NULL,
  confirmation_source VARCHAR(120) NULL,
  confirmation_url  VARCHAR(500) NULL,
  vendor_date       DATE NULL,
  vendor_time_et    CHAR(5) NULL,
  vendor_bell       VARCHAR(10) NULL,
  estimate_json     JSON NULL,
  generated_at      DATETIME NOT NULL,
  schema_version    TINYINT UNSIGNED NOT NULL,
  imported_at       TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
  PRIMARY KEY (symbol, fiscal_quarter),
  KEY idx_primary_date (primary_date),
  KEY idx_range (date_min, date_max)
);
```

Upsert: `INSERT ... ON DUPLICATE KEY UPDATE` every column except the key. Read the JSON with any JSON
library; `candidates` can be stored verbatim as `candidates_json`.

## Sanity checks an importer should make

* `schema_version == 1`, otherwise stop and alert.
* `generated_at` newer than the last import (the file is only rewritten by the cron runs).
* `len(symbols) >= 25` (30 expected); fewer means a data-source problem, see `errors` and `run.log`.