[micahtyong]

Summary

Every Tinker call that reaches Spindle passes through the frontend before it reaches a trainer. Along the way the frontend talks to Modal several times. It looks up which trainer owns the model, records the request, and later stores the result where the client can poll for it. Each of those is a round trip to Modal's key-value store or function API, roughly nine per request. Those services are nearest us-east: one round trip takes about 25 ms from us-east, 75 ms from us-west, 112 ms from eu-west and 260 ms from ap-southeast.

The frontend server only asked Modal for a routing region, which steers requests, not a placement region, which decides where the container runs. So Modal was free to start the frontend anywhere, and over one day of testing its containers came up in us-east, us-west, the UK and ap-southeast. A client and a trainer sitting in us-east then paid an intercontinental round trip on every call.

This change places the frontend server in platform.modal.region when the recipe sets one, and in us-east otherwise. Only the frontend is placed. It runs on CPU, so this has no effect on where trainers and samplers are scheduled; those stay unpinned unless the recipe sets a region, as before.

E2E results

The workload is a 4B LoRA classifier on 4×H100: each step runs two forward calls on a frozen adapter, one forward_backward on the trained adapter, then optim_step, one after another, as the stock Tinker SDK client does. Each call carries 128 examples of about 1,631 tokens with 5 candidate labels per position. 16 steady-state steps, client and trainer in us-east.

"Time outside the trainer" is the client's step time minus the time the trainer spent in forward, backward and optimizer work. It is everything the request path adds.

Before WandB / After WandB

Median per step	Before	After
Time outside the trainer	11.27 s	8.95 s
Frontend handling of one forward / forward_backward submit	0.71–1.16 s	0.27–0.28 s

In the before run Modal happened to place the frontend in us-west. The worst case we saw was a frontend in ap-southeast: a step took 57 s instead of about 31 s, each submit took 3.6 s, and some 4 MB requests from the frontend to the trainer timed out and were retried.

🤖 Generated with Claude Code

[micahtyong]

each training request makes about nine round trips from the frontend to Modal's key-value store and function API, and those services are nearest us-east (one round trip is ~25 ms from us-east, 75 ms from us-west, 112 ms from eu-west, 260 ms from ap-southeast). A frontend on another continent therefore adds seconds to every step; the worst case we measured was 57 s per step from ap-southeast against 31 s from us-east.