Documentation

HeyArka

An adversarial evaluation harness for LLM trading agents. It runs a corpus of 16 published attacks against an agent, scores what changed, and ships a shield that blocks most of them. Everything here runs offline with no API keys.

What problem this solves

A trading agent that reads the news has an untrusted input channel. The literature has already shown what that costs: arXiv:2601.13082 documented that Unicode homoglyph substitution and hidden-text clauses in headlines are imperceptible to a human reader but reliably flip LLM sentiment, and proposed no defense. arXiv:2601.13770 showed that standard LLMs appear profitable in backtests partly by recalling memorized outcomes rather than predicting.

HeyArka is the missing half: it reproduces those attacks against a specific agent, quantifies the damage, and provides the layer that stops them.

The measured result

Against the bundled reference agent, the full corpus produces this. reproducible on any machine with Node 20 by running pnpm attack:

Unshielded: Grade C · 31.3% of attacks changed the order · 25.0% breached the agent’s own risk contract.
Shielded: Grade B · 12.5% of attacks changed the order · 0.0% risk violations.

The shield does not reach zero, and the docs do not claim it does. It eliminates every sanitizable vector (homoglyphs, hidden text, tool hijacking) and stops none of the semantic traps, because a well-formed lie is indistinguishable from a well-formed fact at the character level. What the shield does and does not stop is written out honestly.

The packages

@heyarka/core: The attack corpus, the runner that fans it out against an agent, and the scoring math. No I/O, no network, just pure functions, which is why the same code runs in the CLI and in the browser-facing Desk.
@heyarka/shield: The hardening layer: sanitizer, corroboration gate, point-in-time guard, and a deterministic risk contract. Wraps any agent and returns one with the same shape.
@heyarka/cli: The arka binary: attack, score, and report. Runs the whole corpus offline with no API keys.
@heyarka/mcp: An MCP server exposing the corpus, the shield pipeline, and the scorer as tools for Claude, Cursor, and any other MCP client.
@heyarka/canary: The live A/B: two Bitget Demo paper accounts, one shielded and one not, trading the same real news feed.

192 tests across 6 packages.

Rules this project enforces in code

Demo trading only

The canary sends the Bitget paper-trading header unconditionally. There is deliberately no live mode. Not a flag, not a config option. A canary that could accidentally target live funds is not something this project ships.

Credentials never enter the log

API keys are read from environment variables only. They are never logged, never written to the JSONL audit trail, and never printed.

Consent before you point this at anyone

arka attack --repo clones and executes code from a git URL. Only use it on agents you own or have written permission to test. Disclose privately first; publish aggregate results only.

Where to go next

/docs/quickstart: Three commands to a graded report card, offline.
/docs/corpus: All 16 attack vectors, generated from the corpus source itself.
/docs/shield: The four hardening layers, and what each one cannot do.
/docs/scorecard: How each of the six metrics is computed and graded.
/docs/cli: Every arka command and flag.
/docs/mcp: The five MCP tools and how to wire them up.