Build Your Own Agent Framework

Master AI Agents by Building One

Author

Xiaoquan Kong

Published

December 29, 2025

Preface

You’ve used agent frameworks — LangGraph, OpenAI Agents SDK, Google ADK, or whatever’s trending this month. But when something breaks, you’re staring at stack traces a mile deep, with no idea what went wrong.

The abstractions that made “getting started” easy now make debugging impossible. You want to customize behavior, but the framework fights you at every turn. And next month, there’s a new framework everyone’s switching to.

This book takes a different path: instead of teaching you to use a framework, we’ll build one together — from an empty file to a production-ready system.

What You’ll Learn

By the end of this book, you’ll be able to:

  • Explain exactly how AI agents work — no magic, no hand-waving
  • Build a complete agent framework in pure Python
  • Customize any behavior because you wrote the code
  • Evaluate new frameworks critically, knowing what’s under the hood

Who This Book Is For

This book is for developers who:

  • Want to deeply understand how AI agents work
  • Are frustrated by “magic” in existing frameworks
  • Need to customize agent behavior beyond what frameworks allow
  • Want to build their own tools and integrations

The AgentSilex Framework

Throughout this book, we’ll build AgentSilex — a minimal, transparent, and hackable agent framework.

About the Author

Xiaoquan Kong is a Google Developer Expert in Machine Learning and author of Conversational AI with Rasa (Packt, 2021). He spent a decade building conversational AI systems at Alibaba and other leading tech companies, serving millions of users. He holds a Master’s in AI from Duke University. He wrote this book because he was tired of debugging frameworks he couldn’t understand.

How to Read This Book

The book follows the actual development history of AgentSilex:

Part Chapters Code Version What You’ll Build
I: The Minimal Agent 1-4 v0.1 Working single agent
II: Multi-Agent Systems 5-7 v0.2-v0.4 Handoffs, context, composition
III: Production Features 8-12 v0.5-v0.9 Observability, MCP, streaming, structured output

Each chapter corresponds to a code version in the code/ directory:

# See the code at any checkpoint
cd code/v0.1  # After Part I
cd code/v0.4  # After Part II
cd code/v0.9  # Complete framework

Each version is a complete, runnable project with its own dependencies.

Prerequisites

  • Python 3.12+
  • Basic understanding of LLMs and APIs
  • An OpenAI API key (or other LLM provider)

Getting Started

pip install agentsilex
export OPENAI_API_KEY="sk-..."

You’re ready to go. We’ll write our first agent in Chapter 1.

Conventions

Note

Code Reference: Points to source files in code/vX.X/ directories.

Tip

Checkpoint: Marks a working milestone. Each version is a complete project you can run.

Let’s build something together.