Core data model
From the RepoWiki wiki
| Also known as | Person, Contributor, Manifest, Feature, Revision, Citation, Claim, Architecture |
|---|---|
| Files | 47 |
| Lines of code | 6,547 |
| Languages | TypeScript |
| Entry points | packages/core/src/index.tspackages/core/src/test-fixtures.ts |
| First commit | 30 September 2026 |
| Last commit | 8 October 2026 |
| Main contributors | Sean May (100%) |
| Revision | 5b50749 |
Core data model is the set of TypeScript schema definitions in packages/core/src that define the structure of wiki articles, features, revisions, people and project metadata. The model specifies valid claim structure, citations, feature lineage, and person facts; every other part of the system validates against it and exports data conforming to it.
Overview
Every claim carries an id, text with markdown and link tokens, a kind (fact, limitation or history), citations from code or commits, a supports array for lead claims, a staleness marker, and a hook flag for Main Page candidates.[1] A revision is one version of a feature page, storing its id, feature id, git sha, commit date, generation date, parent revision id, reason (build, update or manifest-change), model name, token usage and an infobox with file counts and languages.[2]
The Architecture article (the project's own page at /special/about/) stores the project title, feature edges with import and call counts, sections named lead, purpose, layers, request-paths, dependencies and infrastructure, and a basis of feature page ids it summarizes.[3] A feature page carries cite links as code citations (file, line range, sha, optional symbol, content hash) or commit citations (sha, subject, optional PR number).[4]
A person revision stores the id, person id, git sha, sections (lead, chronicle, areas), and claim violations are checked to ensure chronicle claims cite commits and area claims cite exactly one feature.[5] The core data model defines six kinds of schema: claims and sections, features and the manifest, revisions, the Architecture article, people and people pages, and the export format that bundles everything together.[1][6][7][8][9] A feature has a permanent lowercase kebab-case id (at most 64 characters), a title, aliases, a status (active, redirect, disambiguation or retired) and a lineage of events (create, rename, merge, split or retire).[10]
The manifest lists all features, their ids and titles, maps files and symbols to features with weighted membership, and validates that all references exist and are consistent.[11] Person schemas hold identity facts (id, name, kind), activity counts (commits, lines added and deleted), featured work, pull requests, and a people snapshot at each sha listing all people, their redirects, and excluded people's anonymous activity.[12]
How it works
The model uses Zod, a TypeScript schema validation library, to define types as z.object() chains with refinements that check cross-field rules; safeParse() returns success or an array of typed issues, while parse() throws on failure.[13][14][15] ClaimId is a non-empty string; claim text is at most 2,000 UTF-16 code units; the claimAnchor() function generates an HTML id from a claim id by prepending claim- if the id matches the regex [A-Za-z0-9_-]{1,64}, or returns null otherwise.[16]
Manifest.safeParse() checks that all feature ids are unique, all membership entries point to active features (not redirects), weights are between 0 (exclusive) and 1 (inclusive), and all redirect and split targets exist and form no cycles.[17] Architecture validates that its id is architecture-<sha12>-<n> where sha12 matches the first 12 characters of its sha field; its basis array is sorted without repeats; its edges are sorted heaviest-first (by import + call count); and no edge repeats or joins a feature to itself.[15] ArchitectureClaim extends Claim with a pages array (at most 3 feature ids); lead claims must support at least one body claim and carry no citations or pages; body claims need either a citation or a page; request-path claims must have a code or commit citation.[18]
PersonName must be a cleaned, non-empty string where cleanPersonName() has removed all control, invisible, bidi and filler characters, collapsed whitespace, trimmed it and cut it to 120 code points.[19] FeatureId is lowercase kebab-case at most 64 characters; Feature validates that its lineage starts with create, that merges and splits have matching status, that a redirect does not target itself, and that disambiguation targets are unique and do not include the feature itself.[20][21]
The llmsTxt.ts module renders the wiki as a text file for LLM consumption: llmsTxtLine() escapes markdown and block markers in a max-length line; plainClaimText() strips link tokens to plain words; renderLlmsTxt() outputs the title, summary, pages with lead snippets, the About article, people, and a link to the JSON export.[22][23]
Data flow
flowchart LR n3["packages/core/src/claim.ts"] n5["packages/core/src/feature.ts"] n12["packages/core/src/person.ts"] n14["packages/core/src/primitives.ts"] n15["packages/core/src/revision.ts"] n4["packages/core/src/export.ts"] n6["packages/core/src/index.ts"] n3 -->|"uses primitives"| n14 n5 -->|"uses primitives"| n14 n15 -->|"uses feature ids"| n5 n15 -->|"uses primitives"| n14 n12 -->|"uses claim structure"| n3 n12 -->|"uses primitives"| n14 n4 -->|"bundles features"| n5 n4 -->|"bundles revisions"| n15 n4 -->|"includes people"| n12 n6 -->|"exports claims"| n3 n6 -->|"produces export"| n4
The export format (WikiExport) bundles the manifest, all feature page revisions (current only), the Architecture chain, all person revisions (current only), and inflight pull request snapshots; it is serialized to JSON and served at export.json.[24] Citations bind claims to source code (file, line range, sha, symbol) or commits (sha, subject, PR); each code citation includes a content hash so readers can warn when code at those lines has changed.[4]
Feature data flows from the repository into a manifest with file-to-feature membership weights; revisions store the current state of feature pages and are chained by parent id; Architecture articles summarize the feature graph at each sha.[25][2][26] Person data flows from git commits into a people snapshot holding all people, their commits, lines and pull requests; person revisions are chained like feature revisions and store the narrative text at each sha; the export carries only current revisions.[27][9]
History
- People facts, snapshot and narrative revision schemas were added on 7 October 2026.[28][29][30]
- The people registry, snapshot and blame cache were added to the store schema as migration 10 on 7 October 2026.[31]
- The export format was extended to include people and person pages on 7 October 2026.[32]
- The llms.txt renderer was added on 4 October 2026, rendering the wiki as a text file for LLM consumption.[33]
- Token usage tracking was added to the export on 3 October 2026.[34]
- The Architecture article schema was given a title field and Purpose and features section on 2 October 2026.[35]
- The Ask schemas and claimAnchor function were added on 5 October 2026.[36]
- GitHub snapshot schemas and inflight data structures were added on 6 October 2026.[37][38]
- Email sanitization was progressively hardened from 7 October 2026 onwards to catch addresses in various disguises.[39][40][41]
Known limitations
A TODO comment notes that claim text longer than the maximum is truncated rather than paged.[42]
See also
References
-
^ a b
packages/core/src/claim.ts:L19-31@5b50749(Claim) -
^ a b
packages/core/src/revision.ts:L25-36@5b50749(Revision) -
^
packages/core/src/architecture.ts:L106-171@5b50749 -
^ a b
packages/core/src/citation.ts:L1-29@5b50749 -
^
packages/core/src/person.ts:L316-370@5b50749 -
^
packages/core/src/feature.ts:L31-38@5b50749(Feature) -
^
packages/core/src/manifest.ts:L18-23@5b50749(Manifest) -
^
packages/core/src/architecture.ts:L111-132@5b50749(Architecture) -
^ a b
packages/core/src/person.ts:L325-336@5b50749(PersonRevision) -
^
packages/core/src/feature.ts:L4-29@5b50749 -
^
packages/core/src/manifest.ts:L6-22@5b50749 -
^
packages/core/src/person.ts:L128-225@5b50749 -
^
packages/core/src/claim.ts:L1@5b50749 -
^
packages/core/src/feature.ts:L1@5b50749 -
^ a b
packages/core/src/architecture.ts:L111-171@5b50749(Architecture) -
^
packages/core/src/claim.ts:L1-42@5b50749 -
^
packages/core/src/manifest.ts:L18-128@5b50749 -
^
packages/core/src/architecture.ts:L28-74@5b50749 -
^
packages/core/src/person.ts:L47-59@5b50749 -
^
packages/core/src/feature.ts:L7-19@5b50749 -
^
packages/core/src/feature.ts:L39-72@5b50749(Feature) -
^
packages/core/src/llms-txt.ts:L25-50@5b50749 -
^
packages/core/src/llms-txt.ts:L105-149@5b50749(renderLlmsTxt) -
^
packages/core/src/llms-txt.ts:L51-110@5b50749 -
^
packages/core/src/manifest.ts:L18-22@5b50749(Manifest) -
^
packages/core/src/architecture.ts:L106-120@5b50749 -
^
packages/core/src/person.ts:L205-214@5b50749(PeopleSnapshot) -
^ Commit
e753f59: "feat(core): add the People facts and snapshot schemas, and clean author names" (PR #557) -
^ Commit
5a15b2a: "feat(core): add the person narrative revision and its claim rules" (PR #558) -
^ Commit
c2e552a: "feat(core): add WikiExport.people, the people role and run kind, and contributorsOf" (PR #559) -
^ Commit
67855e8: "feat(store): add migration 10 with the People registry, snapshot, narrative revisions and blame cache" (PR #564) -
^ Commit
c49adc3: "feat(store): export People and list person pages in llms.txt, with no email anywhere" (PR #571) -
^ Commit
7495b6a: "feat(core): render the wiki's llms.txt" (PR #333) -
^ Commit
6843cb1: "feat(core): carry each run's token totals in the export" (PR #305) -
^ Commit
f19b58a: "feat(core): title the project's article with its name and add Purpose and features" (PR #290) -
^ Commit
5d17382: "feat(core): add the Ask schemas, claimAnchor and the ask role" (PR #434) -
^ Commit
8085be3: "feat(core): add the GitHub snapshot schemas, neutralise GitHub text, and add the inflight role" (PR #487) -
^ Commit
45702f6: "feat(core): add the derived work-in-flight snapshot and WikiExport.inflight" (PR #488) -
^ Commit
5bac6a9: "fix(core): find addresses in any disguise and clean names to visible text" (PR #592) -
^ Commit
cf241a1: "fix(core): count only feature-id targets as feature links, and check revision ids exactly" (PR #593) -
^ Commit
7f8e7c2: "fix(core): catch an address with a zero-width joiner or non-joiner inside it" (PR #597) -
^
packages/core/src/test-fixtures.ts:L44@5b50749(INGEST_PY)
This page was last edited on 8 October 2026, at commit 5b50749.