Build a connector
Add a connector for a tool you use. Follow this page from top to bottom, or hand it to a coding agent. A definitions connector needs no change to the app: only its folder, two list lines and a test fixture.
A connector is a manifest (data) and a runtime (code) in a folder of its own. A definitions connector says which metrics and dimensions exist. A content connector reads documents that become suggestions. This page covers definitions; content connectors follow the same steps, with Markdown in Git as their example.
Hand it to a coding agent
Paste this into your agent, in a clone of the repository:
Build a definitions connector for <tool> in this repository.
Find the documentation page called Build a connector and follow it from top to bottom. Copy the template, do not invent. Use invented fixtures only, and make no live calls to the tool in a test. Change nothing outside the places the page names. Say which checks you ran, and show me what failed.The contract in one screen
- Definitions stay canonical. A definition is the closed list of fields below. Never a calculation, a query or a data row.
- Definitions only flag. A change you report flags the items that rely on it, and a failed read changes nothing.
- Secrets stay in the store. They reach your runtime only, toward the settings they were given for. Never log or return one.
- Domains are explicit. You pass the domain the tool names, and an admin maps the rest. A hint never creates a domain.
- Read only. Reach declared hosts only, and say the least access you need first.
- Your output is untrusted. Every message is checked, and unknown keys are refused.
- No vendor in the core. Your tool’s name lives in your folder, nowhere else.
A definition has these fields (the schema):
| Field | What you put there |
|---|---|
ref | <refPrefix>:<group>.<name>, stable for good. The prefix belongs to the tool |
kind | metric or dimension |
name | <group>.<name>, the ref without its prefix |
title | The tool’s label, or null |
descriptionCopy | descriptionCopy(text).copy: a copy of at most 500 characters, or null |
valueType | The tool’s own word for the type, or null when it says none |
visibleGroups | Always null |
areaHint | The domain id the tool names, from domainHint, or null |
domainSource | source with a hint, none without one |
modelFile | The file that defines it, relative to the folder you read, or null |
synonyms | synonymsOf(value): the other names the tool lists |
baseName | The member’s own name when the tool joins it under a prefix, or leave it out |
variant | For a metric with filters: filterVariant(filters) and the shape’s mark |
builtFrom | For a metric: the refs of its inputs, and columnsOf(expression), or none |
family | declaredFamily(meta): the family its meta.footnote declares, or none |
changeMark | meaningMark(meaning): the SHA-256 of what it means |
A read yields three kinds of message:
warning, with a code and a target;definition, one for each definition;end, once, with aversion, whether it iscomplete, and acursor.
A failure throws a SourceError with a code, a target and a class:
| Class | Who fixes it | Example |
|---|---|---|
config | An admin | A refused token |
transient | Nobody: it is tried again later | A timeout |
source | Whoever owns the tool’s content | A broken model |
A code is one of the service’s own, or yours, worded in the manifest and sent as <id>.<code>.
Decide before you write
- Id, tool word and ref prefix. The id is your folder’s name and stays for good.
toolis how people say the tool. The ref prefix is the tool’s, for good: a second connector for the same tool writes the same prefix. - Where the input comes from. A file in a Git repository (
ctx.git), a folder on the server (ctx.folder), or an https address or API (ctx.fetch). - What to read. Prefer the tool’s own built output to its source files, and never run the tool. A built file usually lives where a pipeline uploads it, so offer object storage first.
- Which routes to offer. dbt MetricFlow reads one file from an address, an object store, Git or a server folder, with one
radiofield andvisibleIfon the rest. - What may read it unchanged. A file read over https may have an ETag or a date. Send them back, and end with
unchanged: trueon a304, and the last read is kept. Put your mapping’s version in the cursor. - Whether an address is a secret. A signed link is a credential. Keep it in a secret slot and declare its host with
{ fromSecret: <slot> }. Name only its host in a step. - Whether it can read a folder on the server. If
readworks fromctx.folderwith no settings, declarecapabilities.serverFolder: true. Leave it out for an API or a live service. - Whether a definition links to its file. If your read names each definition’s
modelFileand the tool’s files have a web view, add an address setting for the model folder and name it incapabilities.modelLinks. A definition’s page then opens its file. Without both, it links nowhere. - What a group is. The part of a name before the last dot. Admins map each group to a domain, and a name without a dot cannot be listed.
- What is left out. Join keys, inputs people never query, hidden members and saved queries.
- Whether same-named dimensions are one.
mergeDimensions: 'by-name'when views share their dimensions, as in Cube.'never'when each group’s is its own, as in dbt. - Least access and hosts. The access lines an admin reads before giving anything, and every host you reach.
Steps
-
Copy the template. Copy the template folder to a new folder beside it, named after your id. It is a working connector that passes the kit, and each step below changes it into yours.
-
Fill in the manifest. Each field has a comment saying what goes there. Change every value, and word every warning code you send. The service does not start with a bad manifest, and the error names each problem.
-
Write the mapping. It is a pure function from the tool’s format to definitions and warnings, with no Git, no folder and no clock. Check the input at its edge with
zod/mini, use the shared helpers, and follow the mapping rules. -
Write the runtime around it.
checkreaches the input and says what it found, step by step, in your tool’s words.readgets the input, maps it, and yields the warnings, then the definitions, thenend. -
Handle failures. Throw a
SourceError.ctx.fetchthrows one itself for a host it refuses or cannot reach, and for an answer over its cap: only a status is yours to turn into one. -
Use only what the context hands you. Lint refuses
fetch,process, files and the rest: see the rules for connector code. A parser you would not run in the service goes in a worker,ctx.worker, as Cube’s compiler does. -
Record the fixture in a folder of your id under the test fixtures.
-
List the connector twice. Add its manifest to the web app’s list, and
defineConnector(manifest, runtime)to the service’s list. A test fails when a folder is not listed, or when the two lists differ. -
Name it in two tests. Add it to the lists in the platform test, twice, and in the sources test.
-
Add a
describeConnectorblock to the conformance test, with the hooks of the kit. Copy the template’s block, at the end of that file. -
Write the README, with the manifest’s
titleas its heading, the sections the review checklist names and no TODO left. A test refuses a listed connector’s README otherwise. -
Run the checks.
Terminal window npx vitest run --project server connectors -u # once, to write the expected file; then read itnpx vitest run --project server connectorsnpm testnpm run typechecknpm run lintnpm run test:integrationWith the README not written yet, the kit’s README test fails on the first run, and the file is still written.
Nothing else in the app changes: the Integrations cell, the connect form and every word on screen come from the manifest.
Two small things are optional:
- A logo: add the tool’s to
logosin the integrations model, keyed by your id. Without one, the cell shows the manifest’sicon. - Words: the lead of its page and the name of its domain setting go in
words, in the same file.
Mapping rules
Every definitions connector follows these, and the template does each of them.
- Refs and groups. The same member gives the same ref on every read, whatever the order of the input. Send each ref once. When two members of a group can have one name, keep one and warn with your own code. A ref never moves to another member.
- Titles and descriptions. The tool’s label and description as they are, or null.
descriptionCopycuts a long description at its last whole word inside 500 characters, with an ellipsis, and you warn with your owndescription_cut, worded atinfo. - Domains.
domainHint(value, config.domainNames)reads the tool’s value through the source’s domain names. A member’s own valid domain wins over its group’s. - Domain warnings. A group that names no domain warns
missing_domain. A value that is not a domain id warnsinvalid_domainand gives no hint. If the tool copies a group’s value onto its members, warn once, on the group. - Synonyms.
synonymsOftrims them, keeps each once whatever its case, and keeps at most 20. - Value types. The tool’s own word, from a list you know. Anything else is null.
- Variants. When the tool gives a metric’s filters,
filterVariant(filters)leaves out the literal each compares and returns those literals. Send them as the metric’svariant, with the mark of its calculation with those filters in its place, and the ref of the dimension each literal is compared with when you read it. A tool that gives no calculation sends none: its variants stay metrics. - What a metric is built from. Send the refs of the definitions it takes as
builtFrom.refs, and the columns its own calculation reads, fromcolumnsOf(expression), asbuiltFrom.columns. Names only: never the expression. Leave it out for a type you do not know. - A declared family. When the tool keeps a
metaper member, read Footnote’s one convention as written:- Call
declaredFamily(meta, dimensionRef). It readsmeta.footnote.family,meta.footnote.byandmeta.footnote.family_description. - Send the result as the member’s
family. - For
invalid, send none and warninvalid_familywith the member’s name. - Leave
footnoteout of the change mark: a family changes no meaning. - The core checks the families across members and warns for one that does not hold.
- Call
- Records built on records. Hash an input once and put its digest in what is built on it. Sort warnings as well as definitions.
Unknown, not guessed
Your connector only says what the tool says. Never invent a group, a domain or a name to fill a gap.
| When | Do |
|---|---|
| The tool names no domain | areaHint: null, domainSource: 'none', and warn missing_domain |
| It names something that is not a domain | Warn invalid_domain, and give no hint |
| You cannot tell a value type | Send null, never a default such as string |
| A record depends on what the input lacks | Leave it out, warn with your own code, and end with complete: false |
A partial read keeps everything already known, so a broken input never flags an item or loses a ref.
Marks are meaning
The change mark is the SHA-256 of what the definition means, built with meaningMark.
| Put in | Leave out |
|---|---|
| The calculation, with the meanings of what it builds on | The domain and the synonyms |
| The type | File paths, ordering and formatting |
| The title and the description | Anything that names the environment, such as a database |
If the tool has two shapes for one meaning, normalise them before you hash: a migration in the tool must not flag every item.
meaningMark leaves out empty values, so an absent key and an empty one hash the same. When you change how you compute marks, bump markVersion: the first read after it stores the new marks without flagging any item, and tells the owners once.
Nothing that is a calculation crosses
A definition is a closed list of fields, but text fields are yours to fill. Never put SQL, a filter or an expression in title, descriptionCopy, valueType or a warning target. Give the kit every piece of SQL in your fixture as calculation.
Words on screen come from your manifest
Every word that names the tool comes from tool. If you find yourself writing the tool’s name outside your folder, stop: that is a bug in the core to fix, not a line to add.
Fixtures
- Invented only. No real names, addresses, data, warehouses or credentials. A key the tests need is invented when they start.
- Rebuildable. When the input is a tool’s output,
REBUILD.mdgives the exact commands and versions, and how environment names were made neutral. A hand-written input says so. - Planted. Keep the tool’s calculations in the fixture on purpose: they are what the kit checks never crosses. A secret the tests use is planted the same way.
- Every way in. A connector that reads its input by several routes runs the kit once per route, against the same
definitions.json. - Expected.
definitions.jsonis the read of the fixture, as the kit writes it. A change to the mapping shows in it, in review. - No live calls. A test Git host serves a repository built from the fixture, and a fake of an API runs on the same computer.
The kit
describeConnector runs the conformance kit through the service, as Integrations does. Its tests switch on by what the manifest declares and the hooks you give:
| Hook | What the kit proves |
|---|---|
folder | The manifest is valid, and its README is there and names its title |
good | The form takes it, the check passes, refs are prefixed and once, marks are stable |
refused | A secret the tool refuses is for an admin to fix |
unreachable | A host that cannot be reached fails, and says so |
invalid | Each setting the form must refuse is refused on its field |
serverFolder | A folder holding the good input gives the same refs and marks |
partial | Input with a record it cannot read gives a partial read with a warning |
calculation | None of these strings is in any definition or warning |
expected | The read of good gives exactly definitions.json |
reformat | The same meaning in another form gives the same refs and marks |
meaningChange | One changed calculation moves exactly the refs you name |
wrongFormat | Input of another format is refused, or fails with a worded config or source code |
planted | The secret is in no check, record or warning |
To read a failure, the test’s name says the rule, and the assertion names the ref or code that broke it.
- A mark that moves under
reformatmeans the mark hashes form, not meaning. - Too many refs under
meaningChangemeans an input’s label or description is in the marks built on it. - Too few means an input’s calculation is not.
Common mistakes
- Naming a member without a dot.
- Hashing the tool’s raw payload, so that every formatting change flags items.
- Taking a domain from a key the tool drops in its output.
- Putting a guessed value type such as
stringon a dimension the tool calls onlycategorical. - Treating a measure, an entity or a join key as a definition.
- Reading a file over the Git cap of 20 MB and failing silently: it is
unreadable_file. - Fetching a file with no
maxBytesand no cap of its own: 20 MB is the most, not a size to aim for. - Putting a signed link in a setting, or its path in a warning’s target. Settings go in the history log.
- Giving a
visibleIffield arequiredwith no thought for stored sources, which have no value for it. - Committing a fixture built against a real warehouse or with real names.
- Writing a test that calls the tool’s live API.
- Adding the tool’s name to web copy.
Review checklist
- Copied from the template, the folder is listed twice, and
describeConnectorhas every hook above.serverFolderis there when the manifest declares it. - The mapping is pure, and
definitions.jsonreads right. - The fixture is invented,
REBUILD.mdrebuilds it, and environment names are neutral. - Every warning code is worded in the manifest, and no screen word outside the folder names the tool.
- The access lines say the least access, and egress names every host.
- The README has its sections: what it reads, setup in the tool, settings, warnings, limits, and what its marks see.
- No vendor name was added to the service, the web app or the domain code.
Worked examples
- The template reads an invented tool’s metrics file from Git, with every mapping rule and every kit hook.
- dbt MetricFlow, built by following this guide, is the example for a real tool: a pure mapping of a built file with two shapes for one meaning, its fixture and its own tests.
For the full contract, see the connector contract. Connectors are built in only for now: packages are a design, not built.