I wanted to see how much of a complete AI agent layer I could put together from open-source software, and how much custom code I would still need. I built a local proof of concept spanning model access, agents, tools and observability. The stack worked end to end, and I’m pausing until I have a real use case to stress-test it.
I started this experiment around mid-September 2026 with a question: could existing open-source components provide a practical, complete layer for building and running AI agents? I wanted to assemble capabilities where they already existed, while keeping the custom code focused on the gaps between them.
The design covered four connected areas. LiteLLM brought model access and LLM operations together. Agent frameworks and a custom control plane handled agent creation, configuration and execution. A custom MCP gateway aggregated tools and controlled access to upstream services. Langfuse, Prometheus and Grafana supplied traces, metrics and dashboards.
The local proof of concept ran end to end on one machine. I created simple agents that looked up information and called tools, and I could inspect dashboards and MCP calls. I was surprised by how smoothly the pieces fitted together and how quickly I could create agents. LiteLLM also made model and cost management easier to handle in one place.
There was still substantial custom work in the gateway and control plane. The local environment uses stand-ins for some providers, identity and upstream services, so running it successfully does not establish production behaviour. I have not yet tested it against a real workload or stressed it under realistic usage. The project is paused while I wait for a use case that can exercise it meaningfully.
Current status
The local proof of concept works. Further work is on hold until there’s a real use case that can put it under meaningful load.
The local proof of concept works. Further work is on hold until there’s a real use case that can put it under meaningful load.
A concrete end-to-end milestone: the repository history records a math connector and agent.
A local-stack milestone. The repository has a metrics note dated September 14; figures need verification before quoting.
Work begins on the experiment.
The initial idea: test how much of a complete AI agent layer could be assembled from open-source software while keeping custom code to a minimum.
Project history
The local proof of concept connects agents, models, tools and observability. It is a useful demonstration, but it still needs a real workload and stress testing.
Connecting LiteLLM, an agent control plane, an MCP gateway and observability showed me where existing tools saved work—and where custom platform code was still needed.
I wanted to find out how much of an end-to-end agent platform I could assemble from existing open-source components, and what I would still have to build myself.