It is designed to be authored by people, generated by agents, exchanged across organizations, and consumed by both. The format is intentionally minimal: a directory of markdown files with YAML frontmatter. There is no schema registry, no central authority, and no required tooling. If you can cat a file, you can read OKF; if you can git clone a repo, you can ship it.
OKF is a universal, vendor-neutral format for representing knowledge
as plain markdown files with YAML frontmatter. It is not tied to any
particular agent, framework, model provider, or serving system. The
goal is simple:
- Anyone can produce OKF — humans authoring by hand, agents built on
any framework (Google ADK, LangChain, custom), export pipelines from
existing catalogs (Dataplex, Unity Catalog, Collibra, ...), or scripts
walking a database.- Anyone can serve and consume OKF — a static file server, a
knowledge-management UI (Obsidian, Notion, MkDocs), an LLM loading
files into context, a search index, or a graph viewer like the one
bundled in this repo.The agent below is a proof of concept demonstrating one way to
produce OKF bundles automatically. The format itself is the
contribution; this agent and the visualizer exist to make the format
tangible at both ends — production and consumption.See OKF in practice — three ready-to-browse bundles produced by this
agent, checked intobundles/:
bundles/ga4/— GA4 e-commerce dataset
(viz.html)bundles/stackoverflow/— Stack Overflow
public dataset (viz.html)bundles/crypto_bitcoin/— Bitcoin
blocks/transactions (viz.html)bundles/acme_retail/— Acme Retail
(viz.html)
OKF represents catalog knowledge as plain markdown files with YAML frontmatter, organized in a directory hierarchy. That choice unlocks a few properties that are hard to get from a service-owned metadata store:
cat a concept; an LLM can ingest it verbatim into context.type, resource, tags, generated, status); use the markdown body for the prose, schemas, and example queries that LLMs and humans actually read.sources with per-source credibility signals), who produced and confirmed it (generated, verified, from which consumers derive a trust tier), and whether it is still current (status, stale_after) — so an agent-maintained corpus stays trustable without any bespoke runtime.index.md files let an agent or human navigate the hierarchy one level at a time instead of loading the entire bundle into context.The net effect is that reference agents, consumption agents, and humans collaborate on the same artifacts in the same way they already collaborate on source code.
python3.13 -m venv .venv
.venv/bin/pip install --index-url https://pypi.org/simple/ -e .[dev]
gcloud auth application-default login plus a project for billing (gcloud config set project <id>). Public datasets are readable, but the caller's project is billed for query bytes.GEMINI_API_KEY (AI Studio) or use Vertex AI by setting GOOGLE_GENAI_USE_VERTEXAI=true, GOOGLE_CLOUD_PROJECT=<id>, and GOOGLE_CLOUD_LOCATION=<region>.The reference agent runs in two passes. The BQ pass writes one OKFdoc per concept the source advertises, using BigQuery metadata alone. The web pass then runs the LLM as its own crawler: it receives a list of seed URLs (provided via --web-seed or --web-seed-file), fetches the seeds via the fetch_url tool, and decides which outbound links are worth following based on whether they look like authoritative documentation for the existing concepts. For each page it fetches, the agent chooses to (a) enrich one or more existing concept docs, (b) mint a standalone references/<slug> doc, or (c) skip. A hard --web-max-pages cap and a same-domain allowed-hosts filter (configurable via --web-allowed-host) are enforced inside the tool, so the agent cannot overrun. Use --no-web to skip the web pass.
Minimum invocation — point at a BigQuery dataset and a bundle output directory. Seeds for the web pass are explicit; omit them (or pass --no-web) to run BQ-only:
.venv/bin/python -m reference_agent enrich \
--source bq \
--dataset <project>.<dataset> \
--web-seed-file <path/to/seeds.txt> \
--out ./bundles/<name>
Iterate on a single concept by adding --concept <type>/<name> (e.g. --concept tables/events_); repeatable.
Each sample pairs a recipe (samples/<name>/, with the seed URLs and exact enrich command) with the produced bundle (bundles/<name>/) that the recipe generated. Open the recipe to reproduce; open the bundle to browse the result directly.
bitcoin-etl pipeline. Exercises cross-tableThe visualize subcommand renders any OKF bundle as a self-contained interactive HTML file — one file, no backend, no install on the viewing side. Open it in any modern browser, share it as an artifact, host it on a static file server, or commit it next to the bundle (as
this repo does).
The viewer is itself a proof-of-concept consumer of OKF, mirroring the way the reference agent is a proof-of-concept producer. OKF bundles can be consumed by anything that reads markdown; this is just one shape.
[…](/path/to/concept.md) links rewired to navigate within the viewer instead of following the path..venv/bin/python -m reference_agent visualize --bundle ./bundles/<name>
That writes bundles/<name>/viz.html. Flags:
| Flag | Default | Description |
|---|---|---|
--bundle |
(required) | Bundle root directory. |
--out |
<bundle>/viz.html |
Output HTML path. |
--name |
bundle directory name | Display name shown in the viewer header. |
Example, writing the output somewhere else and overriding the header:
.venv/bin/python -m reference_agent visualize \
--bundle ./bundles/crypto_bitcoin \
--out /tmp/btc.html \
--name "Bitcoin OKF"
The HTML embeds the bundle as a JSON blob and uses Cytoscape.js for the graph and marked for in-browser markdown rendering, both loaded from a CDN. No data leaves the page; the bundle is parsed once at generation time and serialized into the file.
.venv/bin/pytest
Last modified 11 September 2026