Neuromancing 101

Building agents today feels a little like sorcery: the stronger your craft, the better your summons, and the more of them you can call up at once without the whole thing collapsing into noise. Projects like Mythos, XBow, and RunSybil are S-tier wizards. Their workflows are defined tightly enough that the agents underneath can run autonomously and still return accurate, high quality work.

This post is about the methodology I use to hack at scale, and the system I built it on: Kiwi (cyberpunk edgerunners reference), and specifically its protocols (workflows) feature. The best way I know to learn a concept, and to stumble into novel ones, is to pick a real problem you face every day and build a solution you actually use.

It started with one app, by hand#

I wanted my own Cortana. The first real use-case was date planning. My girlfriend and I are foodies, and the time-consuming loop (which I’m not complaining btw lol, I just saw a faster way to do it) of finding spots she will like, reading menus, and checking reservations could have been reduced to a set of API calls, but the judgement and planning required AI. So I reverse engineered Beli to build an interface my agent could drive directly.

That took about a week, most of it spent guiding my agent to find the small logic bugs and bypasses that let the agent talk to Beli the way the app does. Somewhere in that week the real question showed up. Every app I wanted to add to my assistant would need the same treatment, and each one is different. At the speed I was going, it would take months. The reversing process is not fully deterministic. But every app shares the same high level shape, and that shape never changes. And most importantly, I already know how to reverse engineer systems effectively, so why not just develop a high level flow using the steps I would take, and see what an AI could do? Since I know what a good result looks like, I would be the initial judge, and eventually hand off evaluation to another LLM.

The insight: a fixed shape with unpredictable contents#

Pull back far enough and every app reversing job runs through the same handful of steps, even though what each step finds is impossible to predict in advance.

• Device level controls. Anti debug, anti tamper, anti reversing. What you find here decides what has to happen before any on device dynamic analysis: disabling root checks, hiding Frida, defeating custom certificate pinning. The specific bypasses change from app to app, but you always check first.
• Hardcoded credentials. This step always happens. The method shifts per app. A generic secrets scanner gets you partway, but someone still has to triage the results, throw out the false positives, and catch the ones the scanner missed because they were base64 encoded or assembled at runtime. I am not reviewing five hundred regex hits by hand. An agent will, without complaint.
• Remote attack surface. Every IP, domain, subdomain, API endpoint, GraphQL operation, and socket the app talks to. There might be zero. There might be hundreds. Each one has its own required parameters, request shape, and auth, and each result changes what you do next.

That last property is the whole game. These are deterministic phases with non-deterministic outputs, and the next move depends on the output. That is precisely the kind of work an AI is good at and a traditional scanner is not.

Protocols: the workflow as a graph#

In Kiwi, that fixed shape lives in a protocol. A protocol is a definition file, written to a config.yaml, organized as steps that contain phases. Steps are performed in series, phases are performed in parallel. The reversing workflow I’ll be using as an example is the android-apk-reversing protocol, and it has three steps:

1. Setup. Decompile the APK into a clean corpus and record the basics: the real package name, the framework, and how the bundle was normalized.
2. Recon. Four phases that run at the same time against that one corpus: anti-* controls, credential sweep, OTA firmware analysis, and remote surface enumeration.
3. Review. A curation pass that turns everything the recon phases wrote into a single connected knowledge base.

Each phase runs as its own independent LangGraph thread, with its own message stream and its own LangSmith trace, all pointed at one shared on-disk workspace. A thin coordinator launches a thread per phase, joins the step, and advances only when every phase of that step has completed. If a phase fails, the run halts on it. I reply on that phase’s own thread, and the coordinator retries it, inheriting the files the rest of the run already produced.

%%{init: {'theme':'base','themeVariables':{'fontFamily':'monospace','lineColor':'#7d8597','edgeLabelBackground':'#ffffff'}}}%%
flowchart TB
	APK["APK upload"]:::io
	SETUP["Setup<br>decompile to corpus"]:::setup

	subgraph RECON["Recon: 4 phases, one LangGraph thread each, in parallel"]
		direction LR
		P1["anti-* controls"]:::recon
		P2["credential sweep"]:::recon
		P3["OTA firmware"]:::recon
		P4["remote surface"]:::recon
	end

	CORPUS[("Decompiled corpus")]:::store
	VAULT[("Shared Obsidian vault")]:::vault
	CURATE["Review<br>wiki-curator"]:::curate

	APK --> SETUP --> RECON --> CURATE
	SETUP -.->|writes| CORPUS
	P1 & P2 & P3 & P4 -.->|read| CORPUS
	P1 & P2 & P3 & P4 -.->|read + write notes| VAULT
	CURATE -.->|connect, dedup, flag| VAULT

	classDef io fill:#eef4ff,stroke:#5b8fd6,color:#0b2545;
	classDef setup fill:#cfe3ff,stroke:#5b8fd6,color:#0b2545;
	classDef recon fill:#9cc3ff,stroke:#5b8fd6,color:#0b2545;
	classDef curate fill:#cfe3ff,stroke:#5b8fd6,color:#0b2545;
	classDef store fill:#fff3bf,stroke:#d4a72c,color:#3a2f0b;
	classDef vault fill:#ffe8cc,stroke:#e8842b,color:#3a2410;

The shared memory every summons can read#

Running one agent per task in a shared workspace gets you concurrency, but on its own it has a real flaw. The artifacts an agent leaves behind show its outputs and nothing else. They do not show the methodology, what was examined, what was skipped, the dead ends, or the shortcuts. Those are exactly the details the other agents need so they do not repeat each other’s work or walk into the same wall.

There is a cost problem hiding here too. The cost of riding an agent to dig deeper and deeper is it will exhaust its context. You can compact, but compaction throws away detail you may need later. And without a way to share what they learn, two agents might (ineffectively) do the same research from opposite ends. Example: The remote surface phase notices a hardcoded token riding along on every API call and works out what it is for. The credential sweep phase finds that same token in the code and, with no shared memory, starts the identical investigation from scratch. At scale that duplicated context is wasted money.

This is where Karpathy’s LLM wiki idea comes in. Every run provisions a blank Obsidian vault, shared by all phases. As each phase works, it writes atomic, wikilinked notes into the vault, and it is told to read what is already there and enrich it rather than start cold. So when the credential sweep finds that token, the answer is already waiting as a note, and it moves on.

%%{init: {'theme':'base','themeVariables':{'fontFamily':'monospace','lineColor':'#7d8597','edgeLabelBackground':'#ffffff'}}}%%
flowchart TB
	A["remote-surface phase<br>sees token X on every API call"]:::recon
	B["credential-sweep phase<br>later finds token X in code"]:::recon
	VAULT[("Shared vault")]:::vault
	CUR["wiki-curator<br>connect, dedup, flag"]:::curate

	A -->|"writes note: what X is, where it goes"| VAULT
	VAULT -->|"B reads the note, skips the re-research"| B
	VAULT --> CUR
	CUR -->|"cross-links + contradiction flags"| VAULT

	classDef recon fill:#9cc3ff,stroke:#5b8fd6,color:#0b2545;
	classDef vault fill:#ffe8cc,stroke:#e8842b,color:#3a2410;
	classDef curate fill:#cfe3ff,stroke:#5b8fd6,color:#0b2545;

One rule has to be enforced for this idea to work: a note has to state the finding in its body, not just point at a file:lineor another artifact. A note whose body is only a list of file references is a defect. The whole point is that a reader, human or agent, gets the answer from the note without opening the source.

This screenshot is a real example of my workflow reverse engineering the app that controls my air purifier. As sub-agents perform their tasks, they fill up the knowledge base (visualized on by the graph in the right pane). I love love love visualizing cool data, so I had to include that in my workflow. The nodes are real notes and findings discovered through the agent’s research. Additionally, the knowledge doubles as a valuable deliverable. At the end, the knowledge base can be exported and then pipelined into real reports for bug bounties, responsible disclosure, and SDK builds.

Keeping the knowledge base knowledgeable#

A vault that five phases write into concurrently, without seeing each other’s notes, drifts. You get islands of notes that link only among themselves, orphans, duplicates, and the occasional flat contradiction. Left alone, it stops being useful right when it grows large enough to matter.

So the Review step is a dedicated curation pass run by a wiki-curator agent. It is deliberately domain agnostic and works as a map-reduce loop:

• A CLI tool, find-disconnected-notes.py, computes the connected components of the vault’s wikilink graph, treated as undirected, and surfaces the islands that simple orphan detection cannot see.
• The curator dispatches one vault-reconciler subagent per island. Each one proposes earned connections: real cross-links, shared parent links, genuine merges.
• The curator is the single writer. It applies those proposals serially so concurrent edits never collide.
• Contradictions and stale claims are flagged in place with a [!curator] callout on both notes and reported back, never auto resolved. Resolving a contradiction is a judgment call for a domain specialist, not something to paper over.

This pass is expensive, so it earns a few optimizations:

• Only review what changed. An incremental scope (--since for a timestamp, --since-git for a ref) re-digests just the components that contain a changed note. On a vault of hundreds of notes, this is the difference between cheap and painful.
• Delegate and digest. Subagents do the per island reading and hand back digests. The lead decides where to look next and keeps the big picture.
• It scales. The same loop holds up as the vault grows.
• Domain flavors. Curating a security assessment wiki is not the same job as curating a finance wiki or a pile of personal notes, so the methodology supports curate-<domain> variants over a shared generic core.

Binding each agent to one objective#

The architecture only holds together when every phase is handed a single, clear objective. A few principles do most of that work:

• One task per agent. Credential sweep is one phase. The anti-* controls pass is another. Mixing them muddies both.
• Resolve your own ambiguity. These phases run unattended, so they are told never to stop and ask. When a choice is unclear, they make the call and lean toward the more thorough result. There is no operator sitting there to answer.
• Deliverables up front. Each phase’s opening directive carries an explicit checklist of what it must produce, even though the actual contents depend entirely on the app. Fix the what, free the how, again.

The lead agent has a set of scoped deliverables, all sharing a theme. This allows it to fan-out work in the mission of a set of tightly scoped objectives, so the quality of the work is better, and the research is a much deeper dive than a set of varied objectives (breadth vs depth).

Load-on-demand tooling#

Context is the budget you are always fighting, so I keep the system prompt lean. Only high frequency skills, like writing to Obsidian, live in the always loaded skills/ directory and cost tokens on every turn. Everything niche lives in a separate skill-library/ that costs nothing until it is needed. I implemented the skill-library idea after reading this Praetorian blog that was super insightful:

Deterministic AI Orchestration: A Platform Architecture for Autonomous Development

Agents reach it through a skill_search tool that matches by exact name, keyword, or regex and returns a path. The agent then reads the skill on demand. The library holds the specialized playbooks:

• reverse-engineer-android-app for unpacking and decompiling
• assess-android-app-hardening for the anti-* controls pass
• find-hardcoded-secrets for the credential sweep
• analyze-graphql-api for large GraphQL surfaces
• curate-knowledge-base for the curation methodology

alongside my own personal ones, like restaurant selection for the original Beli use case. A skill is only ever pulled into context when an agent actually reaches for it.

Me-in-the-loop#

The workflow does not run fully end to end on its own.

• Dynamic analysis is still mine. Agents cannot reliably drive an Android device beyond adb, so capturing a realistic, diverse set of requests still means me, by hand, clicking through the app for ten or twenty minutes. I hand the captures back, and the workflow resumes from there.
• Hard questions still come back. When a phase hits something it genuinely cannot resolve on its own, it surfaces to me.

Still, it is a fraction of what the work used to cost.

The payoff#

The first app, Beli, took about a week to reverse by hand. Most of that week was not really about Beli. It was about discovering the workflow, the shape that turned out to generalize.

Once the protocol existed, the numbers changed. I reversed five apps in about two hours, in parallel, and that figure includes the full security assessment of each one and the SDK work needed for my assistant to actually interface with them. The per app human cost dropped to the ten or twenty minutes of clicking around to capture requests. My original workflow was also expensive but I was able to cut the cost down from an average of 12$ per app reversed to just 8$ per app, so about 25% context efficiency gain, and I further reduced costs by switching from OpenAI’s ChatGPT 5.5 to Qwen3.7 Max model. When the harness is good enough, the model makes marginal difference (as long as its a good comparable model to something like Opus 4.7 and ChatGPT 5.5, I wouldn’t try this was a shitter model).

The difficult part is in defining the work precisely enough that the agents you summon can carry it without you. And this workflow system is very easily used: I just define a new workflow in YAML, and I can automate just about any task. So currently I have workflows for:

• Reversing APKs
• Reversing Firmware
• Vuln research for web apps
• WIP: Vuln research for binary exploitation (kernels, firmware, etc)

And of course, my original date planning workflow 😼