Could I build a complete AI agent layer with open-source software?

Earlier this year I started an experiment with a fairly broad question: how much of a complete AI agent layer could I put together from open-source software? I didn’t have a particular production workload in mind. I wanted to connect the pieces, run them locally and see what was involved before choosing a real problem to put through it.

This is a look back at the design I explored between roughly mid-September and late October 2026. The local proof of concept eventually ran end to end, but the initial question was about the shape of the system as much as the individual tools.

More than a model call

An agent needs a way to reach a model, but that is only one part of the setup. It also needs an execution environment, access to tools and a way to understand what happened during a run. I wanted those parts to work together as a layer that could support more than one agent, instead of wiring each example directly to its own model and services.

I split the experiment into four areas. LiteLLM would provide a common point for model access and management. An agent platform would handle configuration and execution. A tool gateway would aggregate MCP tools and control access to them. Finally, Langfuse, Prometheus and Grafana would give me traces and operational views of the system.

The separation gave me a practical way to reason about the boundaries. An agent could ask for a model and a collection of tools without needing to know every upstream service’s details. The gateway could enforce access and handle upstream credentials separately. That also gave me a place to inspect tool calls when something needed investigation.

Use what already exists

I wanted to assemble existing capabilities wherever they were a reasonable fit. LiteLLM already addressed much of the model gateway problem, and agent frameworks could handle parts of agent construction that I didn’t want to recreate. The observability stack could provide the traces and dashboards instead of me starting those from scratch.

That still left real software to write. The repository contains a custom MCP gateway and an agent control plane, including configuration, execution and interfaces for invoking agents. The aim was to keep custom code focused on the gaps between the components, not to pretend those gaps didn’t exist.

What would count as a useful result?

For this first pass, I wanted a local setup that could bring the services up together, run simple agents, make tool calls and let me inspect what happened. A successful demonstration would tell me that the components could be connected and exercised on one machine. It wouldn’t tell me how the platform would behave under a real workload, with real upstreams or sustained traffic.

That’s where the experiment sits now. I got the local stack working and was surprised by how smoothly the pieces fitted together. I’m pausing it until I have a use case that can put the design under meaningful pressure. In the next post I’ll look at how the components were connected and where the custom work ended up.