Isaac Mineo

Private research · In development

Arbor v2

A private prediction-market research system with traceable data, independent validation and guarded execution.

For A private research system with one operator. It is shown here for its engineering, not as a product anyone can use.

Read the source notes

Research system · Notes reviewed 20 September 2026

Architecture · no product screens

From raw file to readable data

  1. Discover
  2. RAW
  3. Normalize to SILVER
  4. Manifest and cutoff
  5. Union of builds
  6. Derive GOLD
  7. Certify
  8. Read by requirement
  9. Bind to a job
Every step, explained below

Reproducible data. Explicit permissions. A record of why.

The problem

A research result can become misleading when its dataset changes underneath it, repeated observations are mistaken for independent outcomes, or a test uses information that was not available at decision time. A running process is also not proof that a system performed the intended action safely.

The goal

Arbor v2 is a separate-generation rebuild focused on research integrity, reproducible datasets and tightly governed execution boundaries. Its value in this portfolio is the engineering: source lineage, validation, durable work and auditable decisions.

The product

A private research system has no screens to show. These are its architecture and two short pieces of its code.

From raw file to readable data

  1. DiscoverOne definition of every source. A format the reader cannot open is kept with a reason, not dropped.
  2. RAWSource bytes stay where they are, recorded by fingerprint rather than copied.
  3. Normalize to SILVERA bounded-memory pass writes typed Parquet through DuckDB. Malformed input is counted and blocks completeness.
  4. Manifest and cutoffEach build records its inputs, code version and source cutoff, and an open cutoff is labelled open.
  5. Union of buildsThe dataset is the read-time union of immutable builds. A build without a manifest is invisible.
  6. Derive GOLDDecision-time rows for every market that could have been evaluated, including ones that could not be traded.
  7. CertifyPublished, complete and eligible are separate statuses. A certificate binds one exact build and its required gates.
  8. Read by requirementAn uncertified build, an uncovered use or too few independent events is refused with a named error.
  9. Bind to a jobA job's identity includes its inputs, so changed data is a different job.

01Architecture

Architecture: how a raw file becomes data a study may read.

Every check before an order

  1. Decision written firstThe cascade appends exactly one terminal row, fired, blocked or skipped, before it returns.
  2. Attribution at birthAn order intent cannot exist without its trace, so the venue's own record links back to the decision.
  3. Book-wide pre-flightHalted scope, concentration and free cash, in order. A structural test makes this the only caller of the executor.
  4. Decision matches intentRefused unless the decision fired and its trace, market and mode match.
  5. Shadow and paper stopIn these modes a refusal row is written and nothing reaches the network.
  6. Exposure capsHard ceilings refuse rather than resize. An unreconciled snapshot is an error, never zero.
  7. Two-key live gateA launch flag and an owner-written, expiring control, both always evaluated. Either missing means dry run.
  8. Claim before the wireThe attempt is journaled before the network call, so a retry cannot place twice.
  9. Single order capabilityOnly the executor may import the order client; a structural test fails the build otherwise.
  10. Terminal row, alwaysPlaced, refused or blocked: every exit writes a row with a reason.

02Architecture

Architecture: every check an order passes before the network, in order.

execution/executor.py

@dataclassclass Executor:    """The single order path.    ⛔⛔⛔ EVERY GUARD IS A REQUIRED FIELD WITH NO DEFAULT, and that is the whole design. An    executor missing its caps, its journal or its live gate **cannot be constructed** — so there is    no unguarded path to forget to use.    ...    """    client: OrderClient    mode: Mode    caps: Caps    journal: OrderJournal    live: LiveDecision    session_start: datetime
Every guard is a required field with no default, so an executor missing its caps, its journal or its live decision cannot be constructed. The docstring is shortened where marked.

Excerpt — trimmed for reading, not a full file.

03Code

Code: an executor without its guards cannot be constructed.

store/canonical/reader.py

    if min_events:        available = dict(            dataset.query(                gold_root,                "universe",                "SELECT venue_series, count(DISTINCT event_id) FROM d GROUP BY venue_series",                build_ids=selected,            )        )        if family and available.get(family, 0) < min_events:            raise NoCanonicalDataError(                f"⛔ {family} has {available.get(family, 0)} independent events in gold, below the "                f"{min_events} this study asked for. That is a COVERAGE fact, not a null result — "                "reporting n=0 as a finding is what this refusal exists to prevent."            )
Coverage is counted in independent events, not rows. A study that asks for more than the certified data holds gets a named error instead of an empty result that could be reported as a finding.

Excerpt — trimmed for reading, not a full file.

04Code

Code: too little coverage is a named failure, not an empty result.

  1. Flow01
  2. Flow02
  3. Code03
  4. Code04

How it is used

  1. CaptureScheduled collectors append public market and feed snapshots, unchanged.
  2. Build and certifySources become typed SILVER and decision-time GOLD, and a certificate binds one exact build.
  3. Read by requirementA study asks for a family, a window and a minimum of independent events, never a file.
  4. ValidateAn independent validator is the only writer of an approved verdict.
  5. Execute, or refuseOne executor, every guard required, and a journal row for every exit.

01 · System

How it fits together

Research and execution are separate authorities. The validator's verdict is a boundary, not a function call.

Select a part to see what it is responsible for and what it connects to.

Every part and connection, as text
Scheduled capture Worker

A timer-driven collector that appends raw snapshots of public data to disk.

→ RAW: appended files

RAW Data

Source bytes, discovered and fingerprinted, never copied or rewritten.

→ Canonical build: fingerprinted sources

Canonical build Worker

Normalizes to SILVER, derives GOLD, and records every unreadable or malformed input.

Trade-off: Malformed input blocks completeness instead of being skipped quietly.

→ SILVER: typed build + manifest

→ Quality certificate: gate run

SILVER Data

Lossless typed Parquet builds, unioned and de-duplicated at read time.

→ GOLD: decision-time rows

GOLD Data

Versioned decision-time rows, including markets that were not executable, with a reason.

→ Canonical reader: certified builds

Quality certificate Gate

Certifies one exact build for named uses; a copied, stale or partial certificate fails closed.

→ GOLD: bound to one build

Canonical reader Service

The only way research reaches data: by requirement, never by file path.

Trade-off: A study can be refused for coverage, which is slower than reading whatever is there.

→ Research jobs: rows meeting a requirement

Research jobs Worker

Bounded studies keyed on their inputs, in a durable SQLite/WAL job ledger.

→ Independent validator: proposed finding

Independent validator Gate

The sole writer of an approved verdict, separate from whoever proposed the finding.

→ Executor: approved verdict (authority boundary) (Authority boundary)

Executor Gate

The single order path. Every guard is a required field, and live action needs two independent keys.

Trade-off: The default is no live action at three separate layers.

→ Journal: claim, placed or refused

Journal Data

A claim before the network and a terminal row for every exit, including refusals.

Python modules and typed contracts separate canonical data, research, validation, portfolio decisions, execution and accounting. The data plane uses versioned RAW, SILVER and GOLD layers with DuckDB and Parquet. Research reads a selected, eligible canonical dataset through a shared reader rather than naming a growing raw file. A durable SQLite/WAL job framework records work and lineage. Execution has a single capability-bearing boundary with required guards and a journal.

02

What it does

  • Canonical, reproducible inputs

    The reader asks for a dataset capability and minimum independent-event coverage. A completed build is not automatically a quality-certified build.

  • Independent outcomes, not inflated rows

    The coverage model distinguishes repeated market observations from independent outcomes. Sample volume alone is not evidence of predictive reliability.

  • One execution authority

    The executor requires a matching decision and intent, attribution, duplicate protection, exposure checks and an explicit live decision. Refusals are also terminal facts to record.

  • Default to no live action

    Two independent authorization conditions are required for live execution. Missing, invalid or expired owner authorization leaves the system in dry-run mode.

03

Built with, and why

Core

Modules with typed contracts between research, validation and execution.

Python · Typed contracts

Data

Reproducible, versioned datasets that a study can name exactly.

DuckDB · Parquet · Versioned canonical datasets

Operations

Durable work, a record of every order decision, and tests that enforce the boundaries.

SQLite/WAL jobs · Execution journal · Structural boundary tests

04

Decisions and trade-offs

Preserving research provenance

Approach

Canonical readers use eligible, versioned datasets and distinguish independent outcomes from repeated observations. Quality certification is a separate assertion from successful file creation.

Trade-off

A strict evidence boundary can block a study rather than return a convenient partial result. That friction is useful when the alternative is silently changing the meaning of the experiment.

Making authority impossible to forget

Approach

Execution guards are required constructor dependencies rather than optional conventions. A single order capability and structural checks constrain which modules can reach that boundary.

Trade-off

Safety also depends on deployment, input integrity and tests. A well-designed gate does not by itself show that every runtime or historical caller goes through it.

05

Check the work

Source notes →The product, architecture, decisions and limits this page is written from.

Ask about Arbor v2

Answers quote the source notes and cite them.