The AI agent platform works end to end. Now I need a real workload.

I managed to run the full AI Layer proof of concept end to end on one machine. I created simple agents that looked up information and called tools, then inspected dashboards, traces and MCP calls. Seeing the whole path work was the result I was looking for from the initial experiment.

This is a retrospective account of work done intermittently between roughly mid-September and late October 2026. The project is paused, not finished: I want a real use case before I try to draw conclusions about how well the platform performs in practice.

Starting the stack together

The repository’s ./scripts/init.sh provides a one-command local startup path. It builds the custom services, starts the Compose stack, prepares local credentials and a local model, and checks that the services are healthy. It also checks aggregated tools, makes a model completion through an alias and confirms that agents have registered. An optional --verify runs the acceptance script.

That startup work matters because a platform made of several services can appear healthy while a connection between them is missing. The script goes beyond checking that a model gateway answers: it makes a local model call, which can reveal a missing model behind an otherwise healthy gateway. It also retries the tool and agent checks while upstream synchronisation and seeding finish.

What I could exercise

LiteLLM brought model access together; the agent platform supplied configuration and execution; and the custom MCP gateway aggregated and controlled tool access. Langfuse gave me traces, with Prometheus and Grafana providing operational metrics and dashboards. I could follow a basic agent run through those pieces and inspect its tool calls.

The repository includes examples of configuration changes being reflected in an agent’s tool contract without rebuilding the agent. It also includes small dashboard, app and script demonstrations that use the gateway. These made the connections tangible: the tool layer wasn’t only something an agent could call from a test.

I was surprised by how well the open-source components fitted together, and how quickly I could create agents using the frameworks. I didn’t hit an open-source limitation in this experiment. That is an observation about this proof of concept, not a conclusion that those components have no limitations.

What the demonstration doesn’t prove

Several dependencies are local stand-ins, including provider, identity and upstream service examples. They are useful for exercising integration paths without external accounts, but they cannot reproduce the behaviour of production systems. The repository records tests and dated performance figures; I haven’t freshly rerun those measurements for this write-up, and they don’t substitute for a workload.

There has been no realistic stress test. I don’t yet know how the system will behave with real traffic, provider failures, rate limits, changing data or the demands of an actual user. The custom gateway and control plane also need to be judged against the maintenance they require, not just whether the demo runs.

For now I’m leaving the project paused. The local result gives me a useful starting point; the next meaningful step is to find a real use case, run it through the layer and let that experience show me what needs to change.