Section 0

Building AI Agent Harness

From Chatbots to Agent Harnesses

Today's Mission

Where Agents Shine

Any domain with structured data, repeatable workflows, and too much for humans alone.

If the work is tool-heavy, repeatable, and data-rich — an agent can do it. Today we focus on cybersecurity as our lens, but the harness pattern applies everywhere.

What is an LLM?

A Large Language Model predicts and generates language based on patterns learned from vast amounts of data.

OpenAI

GPT

Claude

Gemini

Llama

Mistral

Mistral

The Traditional Chatbot

The Problem with Chatbots

Adding Tools

What is a Tool?

A tool is a capability the AI can invoke — a function with inputs and outputs.

Tool Calling

User asks: "What is the hostname of IP 8.8.8.8?"

AI decides a tool is useful → generates structured arguments → system executes → result returned

Tool Call Example

User asks: "What's the weather in Tokyo?"

From Tool Calling to Agents (Agentic Framework)

What is an AI Agent?

An AI Agent is an AI system that can interpret a goal, decide what actions to take, use tools, observe results, and continue working toward an outcome.

ReAct: Reasoning + Acting

The agent interleaves reasoning ("I think...") with acting ("I'll use...") and observes the result before deciding the next step.

The Agent Loop

Agent vs Workflow

Workflows follow predefined paths. Agents dynamically decide the next action based on what they discover.

Agent Examples

Agent Examples (Beyond Cyber)

Evolution Summary

The Problem with Bare Agents

LLM + tools + loop sounds powerful. In production, it breaks fast. Here's why — and what fixes it.

The Brain Analogy

A brain can't do anything on its own. It needs a body.

LLM → Agent → Harness: The Stack

Each layer adds capability. Skip one and things break.

What the Harness Does

An Agent Harness is the runtime and control layer that connects an AI model to tools, context, memory, state, permissions, execution logic, safety mechanisms, and observability.

9 components. Every production harness has all of them.

1. While Loop

The agent's heartbeat. Without a loop, an LLM is just a chatbot — one response per prompt and stop.

The loop is the only difference between an agent and a chatbot.

2. Context Management

Everything the model sees, assembled fresh every loop iteration. Get this wrong and the agent goes blind.

3. Tools

Functions the agent can call to act on the world. Every agent needs universal primitives. Cybersecurity agents add specialized tools on top.

PRIMITIVES · universal

CYBERSECURITY TOOLS · specialized

Agent Skills

Reusable, composable capability bundles. Tool + Instructions + Schema = Skill.

MCP: Model Context Protocol

Standardized protocol for connecting AI agents to tools and data sources.

MCP servers expose tools, resources, and prompts. Agents discover and call them through a unified client. Like USB-C for AI tools.

4. Sub-Agents

The main agent can spawn specialized sub-agents for subtasks. Each sub-agent has its own context, tools, and loop. They can run sequentially or in parallel.

Sub-agents isolate context and can fan out in parallel — faster than serial execution.

5. Built-in Skills

Pre-bundled utilities the harness ships with. The agent gets them out of the box — no custom code needed.

6. Session Persistence

What the agent remembers across runs. Short-term: this conversation. Long-term: past investigations and learned patterns.

7. System Prompt Assembly

The system prompt isn't a static string. The harness assembles it dynamically from multiple sources before every LLM call.

Same agent, different context → different effective system prompt.

8. Lifecycle Hooks

The harness gives you places to attach your own code. It automatically calls your code at the right moment.

The harness does the heavy lifting. You just decide what extra things should happen at each step.

9. Guardrails & Approvals

Safety checks at every stage. Block dangerous actions. Require human approval for critical decisions.

Trace: "Investigate 8.8.8.8"

One investigation. Every layer engaged.

The Complete Harness

9 components. One production-ready agent system.

The agent decides what to do. The harness controls how it operates — safely, reliably, and measurably.

Examples of Agent Harnesses

Production harnesses in the wild — all nine components, different implementations.

Section 4

CyberSec Investigator

Our Running Example

Meet CyberSec Investigator

Investigate suspicious IP addresses, domains, and URLs — and produce evidence-based security reports.

Agent Architecture

Investigation Example

How Does the Agent Decide?

Agent Trace

Live Demo

Behind the Scenes

What happened inside the harness?

Your Mission

Starter Architecture

Agent {
  model: LLM,
  tools: [dns, whois, reputation],
  instructions: "investigate domains",
  loop: while (task not done) {
    think();
    act();
    observe();
  }
}

Tool 1: DNS Lookup

Tool 2: WHOIS Lookup

Tool 3: Reputation Lookup

Build the Agent Loop

function agentLoop(goal, tools) {
  let context = { goal, history: [] };
  let done = false;
  let maxIterations = 10;

  while (!done && maxIterations-- > 0) {
    let decision = model.decide(context);

    if (decision.type === "tool_call") {
      let result = tools[decision.tool](decision.args);
      context.history.push({ tool, result });
    } else if (decision.type === "final") {
      done = true;
      return decision.answer;
    }
  }
}

Run Your Agent

Add a New Tool

Extend your agent with another capability:

Test Your Agent

Agent Attack Surface

Prompt Injection

Tool Abuse

Excessive Agency

A poorly designed agent has too many capabilities:

Least Privilege

Data Leakage

Memory Poisoning

Agent-to-Agent Risk

Secure Agent Architecture

The 8 Principles

1. Right Tools for the job
2. Reliable Tools that work consistently
3. Least Privilege permissions
4. Good Context management
5. Observability into every action
6. Human Approval for risky actions
7. Failure Handling at every level
8. Evaluation to measure and improve

Design for Failure

Agent Observability

Agent Evaluation

Demo vs Product

Agent Design Challenge

You are the architect. Design an AI SOC Analyst.

Share Your Design

Key Takeaways

The model is only one part of the agent.

The harness determines what the agent can do,
how it behaves, what it can access,
and how safely it operates.

Frameworks for Building Agents

Choosing the right foundation for your security agent:

The Cybersecurity Toolset

Tools the security harness provides to the agent:

Choosing the LLM Engine

Agent Sandbox Architecture

The Agent Loop & State Control

Defending the Harness

The Future: Multi-Agent Swarms

Next Steps: Implementation