Reference

CLI

arka has three commands: attack runs the corpus and appends to a JSONL log, score recomputes a scorecard from a log, and report renders one as a static HTML page. No API keys, no network, no configuration.

Usage

arka --help

arka attack --demo [--shielded] [--out <path>] [--report <path>]
arka attack --agent <path> [--shielded] [--out <path>] [--report <path>]
arka attack --repo <git-url> --entry <path> [--shielded] [--out <path>]
arka score <results.jsonl> [--agent-name <name>]
arka report <results.jsonl> --out <report.html> [--agent-name <name>]

arka attack

Runs all 16 vectors against an agent and appends one result per vector to the log. Pick exactly one agent source:

--demo: Uses the bundled zero-config reference agent. This is the path judges can run with nothing installed beyond the repo.
--agent <path>: Loads a real agent module exporting an AgentUnderTest, as a default export, a named agent export, or a factory.
--repo <git-url> --entry <path>: Shallow-clones a public repo and loads --entry relative to the clone root, the same way --agent does.
--shielded: Wraps the agent in @heyarka/shield before attacking it. Run the same agent twice, once with and once without, for the A/B.
--out <path>: JSONL log path. Defaults to reports/results.jsonl.
--report <path>: Also render an HTML report card after the run.

--repo executes code you did not write

It clones a repository and imports a module from it. Only point it at agents you own or have written permission to test. Disclose findings privately first and publish only aggregate or anonymized results unless the owner consents.

The A/B in two commands

Terminal

node packages/cli/dist/bin.js attack --demo
# grade C · 31.3% of attacks changed the order

node packages/cli/dist/bin.js attack --demo --shielded
# grade B · 12.5% of attacks changed the order

Both runs append to the same log by default. The shielded agent’s name carries a +shield suffix, which is what keeps the two separable when the log is scored.

arka score

Recomputes a scorecard from an existing log and prints it. Pure aggregation. It never re-runs the agent, so scoring an old log is always reproducible.

Terminal

arka score reports/results.jsonl --agent-name heyarka-demo-agent

--agent-name is required when a log holds more than one agent. Without it the command fails rather than averaging unrelated runs into one misleading number.

arka report

Renders a scorecard as a standalone HTML report card. The judge-facing artifact. Self-contained, with no external assets.

Terminal

arka report reports/results.jsonl --out reports/card.html