03Deep dive · office3d
Turning agent traces into a live 3D office
The same traces I use to debug the system also draw it: an isometric office where every figure is a live conversation. How I got 5,000 draw calls down to 8 with instanced meshes.
The problem
Every conversation the specialists handle is already recorded as a trace: every turn, every tool call, every decision. That's where I debug the system from. One day it hit me that the same data could be drawn — an isometric office where each figure is a conversation happening, and you watch the specialists work. It's what I show in a demo and what I use to find bugs, out of the same code.
Constraints
- 01Truth, not theatre. Every figure has to be a real conversation out of the traces; a scripted animation is something a CTO spots immediately.
- 02It has to survive a full room. When the office fills up, drawing naively turns into thousands of draw calls and a slideshow.
- 03Public without leaking. The same scene runs on my portfolio, and there it reads from a filtered, opt-in source — never the internal one, with customer names and revenue.
- 04Interactive: orbit, zoom, and click a figure to see what that conversation did.
Architecture
agent_traces (Postgres)
the source of truth, already written by the running agents
scene loader
groups traces by conversation → post, speech bubble, outcome
React Three Fiber
dynamic({ ssr: false }) — the WebGL bundle never touches other pages
two sources, one component
logged-in reads /api/vitrine/* · public reads /api/showcase/* (allowlist + rate limit)
The audience of already-served customers is the rendering trap: a full day is thousands of figures, and drawn one by one that's thousands of draw calls. An InstancedMesh collapses them into a handful — the crowd went from around 5,000 draw calls to 8, one per mesh type, with every position written into a single buffer.
The hard part
3D traps don't show up in a unit test. They show up the moment you orbit the camera.
drei's <Html> labels carry a z-index in the millions and were rendering over a modal that was supposed to be in front, so they had to learn to hide whenever the transcript modal opens. That same <Html> was swallowing clicks meant for the figure behind it. The section signs, pinned to the back wall, slid sideways as the camera turned, because they sat at a different depth from the desks they labelled — and no fixed x-offset gets that right at every angle; I fixed it by anchoring each sign at the depth centre of its own section. And the floor fought the rugs for z until I pulled the coordinates apart. None of this is something I'd have predicted from reading documentation.
Trade-offs
- WebGL is a real bundle. I keep it off every page that doesn't draw the scene, with a client-only dynamic import. The cost is only paid when the office actually shows up.
- Instancing trades flexibility for frames. Every customer in the crowd shares one material; styling them one by one would mean giving up the instanced path. For a crowd, I'll take the frames.
- The scene reads near real time by polling, not websockets. Simpler, cache-friendly, and a few seconds of lag is invisible in an ambient view.
What I'd do differently
I'd separate the office's set from its actors more cleanly. Right now the room geometry and the trace-driven figures live closer together than they should, so touching the scenery risks the movement. With a static set and data-driven actors on top of it, I'd grow the environment towards the walkable office I want, without threatening the orbit, the zoom and the walking — which is what makes people remember this screen.