Workflow / New indexer
Build offline. Prove the backfill. Then join live.
A new protocol decoder, security rule, or derived index should not begin at today's ledger and ignore the history already retained. Stargazer's target workflow replays every new module against the local archive, validates the result in isolation, catches it up to the live watermark, and only then promotes it.
01 / Lifecycle
One path from first fixture to live operation.
- 01
Define Planned Version the module, its accepted facts, output schema, and supported code hashes.
- 02
Fixture In progress Capture representative real ledgers and exact expected outputs.
- 03
Backfill Planned Replay the local archive into isolated staging storage with delivery disabled.
- 04
Validate Planned Check continuity, idempotency, parser coverage, errors, and output invariants.
- 05
Catch up Planned Process the gap between the archive high-water mark and the current live checkpoint.
- 06
Publish Planned Atomically promote the materialization and enable the same version on the live tail.
02 / Module contract
Version behavior, not just code.
Each indexing module should declare enough information to reproduce a finding later:
| Field | Purpose |
|---|---|
module_id | Stable identity for a protocol decoder, generic fact extractor, or rule. |
module_version | Immutable behavior version attached to every derived row. |
schema_version | Expected storage contract and migration boundary. |
recognized_wasm | Exact code hashes a protocol-specific decoder understands. |
input_facts | Normalized chain facts required before the module can run. |
coverage | Network, ledger range, contracts, and known exclusions for the materialization. |
An unknown Wasm hash must continue through generic indexing and create a coverage finding. It must not be decoded silently with the closest known interface.
03 / Fixtures and benchmarks
Start with difficult, real ledgers.
- Choose representative history. Include high-transaction ledgers, failed invocations, fee bumps, deep authorization trees, upgrades, and restorations.
- Pin the raw evidence. Record ledger hashes, archive paths, transaction hashes, and checksums outside generated databases.
- Assert semantic outputs. Test exact wallet-contract links, protocol attribution, executable transitions, and known negative cases.
- Measure the cost. Compare ledgers per second, decoded bytes, database growth, write amplification, and peak memory.
- Replay twice. The second pass must leave counts and materialized state unchanged.
04 / Isolated backfill
Never mutate the active projection in place.
Staging can be a new database, a versioned table set, or a separate storage generation. The key property is that readers see either the complete old generation or the complete validated new generation—never a partially rebuilt mixture.
05 / Promotion gate
What must pass before live activation.
- the full target ledger range is contiguous and hash-chain consistent;
- all fixtures match their exact expected facts and findings;
- duplicate replay changes no counts, versions, or alert identities;
- parser errors and unsupported code hashes are counted and reviewable;
- archive and live source outputs agree over an overlapping sample;
- the new projection covers the live high-water mark or a documented handoff ledger;
- read queries and indexes meet their bounded latency and memory budgets;
- historical delivery remains disabled throughout replay.
06 / Alert semantics
Historical detection is not a new incident.
A backfill may discover an old upgrade, control change, or incident exposure. That finding belongs
in the evidence database and can appear in a retrospective report, but it must carry historical
provenance and delivery_enabled = false. Only findings produced after the promoted live
handoff are eligible for normal notification policy.