Utariga — a procedural island city that Claude and I both edit

Utariga is a workflow exploration for gen AI worlds. It's a low res procedural island city in Three.js that Claude and I both edit. About 300 buildings, each one a JSON spec regenerated from one seed. I take buildings into Blender with Claude and the dcc-mcp, light them in Cycles or Octane, and bring the results back into the city.

The newest work is baked lighting: per-light lightmaps and lightprobes from Octane, on meshes and on splats, and sim ready assets for a Genesis robot sim. Most everything is deep linkable, so a link carries the camera, the render and the light mix.

Disclaimer: this is a workflow/tool exploration. Some assets and splats are low quality to save tokens, and it hasn't had an optimization pass.

Joey's Apartment at night with Octane baked lightmaps, characters lit by lightprobes
Joey's Apartment with an Octane lightmap. The characters are lit by lightprobes made from that lightmap.

1. Baked lightmaps and lightprobes

Lightmaps are baked in Octane, one AOV per light: sun, ambient, and a channel per light ID. In the browser the channels are summed, so every light can be remixed live with sliders, a Kelvin gel per light, MIDI faders, or the mix= URL param.

Joey's Apartment → bld_910c2a Octane lightmap, probe lit characters

Characters lit by probes baked from the lightmap. Some are from past projects, some are open source avatars, motion from Mixamo. 23 s

In the live app

Ostraka → bld_5920g9 artist compound · per-fixture light channels

Ostraka is a sculptor's walled compound that Claude built: a long studio, a kiln, a maquette archive and a helix stair. Its lights are split into channels for the fire glow, the side room, the courtyard lights and the sphere lights, so each group can be mixed on its own.

Ostraka interior, a pink glowing sphere light on the floor, mixer panel open Ostraka at dusk, a green neon pin and two glowing fire tables, mixer panel open
The sphere lights, and the neon pin and fire tables at dusk. Both are the 16M relightable splat on the live site.
Remixing the light channels at dusk. 48 s
The sphere lights, then dropping the sun and ambient to night. 49 s

In the live app

2. The same lighting on splats

The splats aren't trained from photos. They're generated in the browser from the same mesh, with Three.js's mesh sampler, so the normal, color and lighting at each point are already known.

Joey's Apartment as an 8M splat, with a splat avatar lit by the probes. 15 s

In the live app

The first version of this was the Kip House, loosely modeled after the house in Berkeley, with a Cycles lightmap transferred to the splat.

Kip House, splats made in the browser lit by Cycles.

3. Sim ready assets: the Panda sim

A Franka Panda arm on Joey's desk picks up the mug, sets it down and sweeps the bottle off the desk. The sim runs in Genesis and plays back in the browser, lit by the room's lightmap and probes.

The Panda sim in Joey's Apartment. The arm and the props it moves are probe lit. 19 s

In the live app

Cake sim (WIP)

A procedural cake made with Geometry Nodes in Blender, with zones for crust, crumb, filling and frosting, cut by a Genesis MPM sim. It works in Blender. Getting it into the world, on the desk next to the Panda, is WIP.

Genesis MPM cutting, rendered with Cycles in the Blender viewport, sped up 2x. 33 s

4. Buildings to Blender and back

Any building exports as a GLB from the browser and goes into a live Blender session through the MCP. Claude models, set dresses and lights the interior from plain language art direction. I clean up its mistakes, edit layouts, add lights and cameras, and send it back. The GLB re-binds to the same lot, and Blender cameras come back as shots stored in building space, so they move with the building.

The stock procedural interior: a correctly-shaped empty room The same room after the Blender authoring pass, furnished and lit
Joey's Apartment before and after the Blender pass. The brief was a small coastal bachelor apartment with surfboards, skateboards and a pelican on a bicycle.
The Spire going out to Blender and coming back, 923 objects. 26 s

In the live app

5. Renders and compares on the authored camera

Cycles and Octane renders publish from Blender back into the app, keyed by building and camera name. Opening one puts the camera where the Blender camera was and overlays the render with a blend slider and a wipe. The link carries the time of day and the post settings.

The villa kitchen rendered in real time by three.js The same villa kitchen camera rendered in Cycles
Same camera, real-time Three.js and Cycles.

In the live app

6. Imagine and Lightmatch

The ✨ Imagine panel sends the current frame and a prompt to Flux Kontext dev in local ComfyUI, or Qwen Image Edit on a Hugging Face Space. The result is published as a render on that camera, so it opens in the same blend/wipe compare.

Qwen Image Edit making the beach restaurant photoreal, then blending back to the live scene. 49 s

Lightmatch tunes the real-time lights to match a target image. It runs Nelder–Mead over six multipliers (exposure, ambient, point/spot, area, warmth, emissive), one evaluation per frame. The loss compares image statistics, not pixels, since the AI targets change the content too. Doing it at a few times of day gives a light curve that is saved per building.

Solving, one evaluation per frame. 30 s

7. Marble splats from panoramas

The app renders a 360° panorama and a depth plate from a point in a room and sends both to World Labs' Marble, which returns a splat. It lands back in the scene aligned. The depth is encoded from percentile bounds so a bit of city through a window doesn't eat the depth range, and the heading is solved locally since Marble doesn't report it. Marble 1.1 is much better than 1.0.

Panorama and depth to Marble, and back as a target on the same camera. 1 min 9 s

8. Splat to object

Click an object in a splat and the app sends the click points to SAM 3 in local ComfyUI for a mask, then SAM 3D Objects makes a GLB that's placed back in the scene. Texture baking doesn't fit on a 24 GB 4090, so the GLBs are vertex colored, about a million triangles, and need retopology.

Checking the segmentation against the live room before building. 22 s

9. Editing the city live

Roads and buildings are editable in the browser. Drag a road and the terrain, pedestrians and street addresses follow it. Move or raise a building and the ground follows the footprint. Edit a building's JSON and the city rebakes around it.

Editing a road in the running city. 23 s

10. Buildings and stories Claude made

I had Claude pick buildings with no interior and give them the full treatment: a name, a backstory, the Blender pass, Cycles cameras, relights, solved light grades and a narrated tour. It likes bars, old shops and spiral staircases. The in-game walk tests caught float precision gaps at stair and floor seams that Blender's raycasts missed, and those checks went back into the build scripts.

Cycles render: 24 neon tubes in spectrum order on a rack Cycles render: dark gallery lit by a salvaged MOTEL sign
Argonaut Neon, a neon sign workshop, in Cycles.

Hidden features and known issues

Hidden features

Known issues

How it's built

Three.js for rendering, React for the UI, Vite to build. An Express server with a JSON file fallback. Post is pmndrs postprocessing with N8AO, bloom, SMAA and an experimental SSGI. Splats render with Spark.

Blender authoring is Claude driving a live Blender session through an MCP gateway. Lightmaps and renders come from Cycles and Octane. Sims run in Genesis. Image generation is Flux Kontext dev in ComfyUI or Qwen Image Edit on a Hugging Face Space. Marble splats from World Labs. Object extraction is SAM 3 and SAM 3D Objects in local ComfyUI.

Elsewhere