Offscreen host: skip drawing held turns and pace submitted frames

README.md

@@ -366,7 +366,26 @@ host's batch on the next frame that runs. Pausing is the application's rather
 than the host's for the same reason: a held iteration keeps its events for the
 frame that runs and a stepped frame takes a fixed delta, so a paused game
 advances by exactly the frames granted, on any host, while the host keeps
-rendering and serving screenshots of the held world.
+serving screenshots of the held world.
+
+A held turn is reported to the host as its own `held` state rather than as
+`continue`. The offscreen host used to draw every turn, held or not, with no
+pacing, so a paused test host spun a core and redrew an unchanged frame as fast
+as the GPU would take it; several side by side then slowed each other down. It
+now draws a held turn only when a screenshot, an image command or a model upload
+is waiting, and otherwise sleeps a quarter of a millisecond, which keeps a
+tool call's round trip faster than the old loop's. Comparing the frame counter in
+`tecs.host` was chosen over a new application field because every frame that
+runs advances it and a held turn is exactly one that does not.
+
+Skipping held turns was not enough on its own. Nothing else held the offscreen
+loop back either, so the frames that do run were submitted as fast as they were
+extracted, and a GPU shared with other hosts accumulated them along with the
+staging memory each holds: hosts grew to about 1 GB and side by side they still
+slowed each other down. The loop now keeps at most two frames in flight, waiting
+for the frame before last after each submit. One frame in flight bounds memory
+as well but serializes extraction behind the GPU, which cost a lone host about
+40 percent; two keeps that overlap.
 
 ### Application lifecycle and presentation timing
 

native/rust/winit-host/src/bridge.rs

@@ -179,6 +179,9 @@ pub struct TouchEvent<'a> {
 pub enum FrameState {
     Parked,
     Continue,
+    /// The paused application held the turn: no frame ran, so the world is
+    /// what the last frame left apart from debug-tool edits between frames.
+    Held,
     Stopped,
 }
 
@@ -452,6 +455,7 @@ impl Bridge {
         match one_text(&values, "tecs.host.iterate")?.as_str() {
             "parked" => Ok(FrameState::Parked),
             "continue" => Ok(FrameState::Continue),
+            "held" => Ok(FrameState::Held),
             "stopped" => Ok(FrameState::Stopped),
             state => bail!("tecs.host.iterate returned unknown frame state {state:?}"),
         }

native/rust/winit-host/src/graphics.rs

@@ -482,6 +482,9 @@ pub struct Graphics {
     offscreen: Option<wgpu::Texture>,
     capture_requested: bool,
     captured: Option<Result<Capture>>,
+    /// The submission that ended the previous paced frame, which the next one
+    /// waits for. See `pace_frame`.
+    paced: Option<wgpu::SubmissionIndex>,
     device: Device,
     queue: Queue,
     config: SurfaceConfiguration,
@@ -943,6 +946,7 @@ impl Graphics {
             offscreen,
             capture_requested: false,
             captured: None,
+            paced: None,
             tile_chunks,
             frame_table,
             particle_pool: None,
@@ -1578,6 +1582,28 @@ impl Graphics {
         Ok(())
     }
 
+    /// Keeps at most two frames in flight: waits until the GPU has finished the
+    /// frame before the one just submitted.
+    ///
+    /// Nothing else holds an offscreen loop back, so without this it submits as
+    /// fast as the CPU extracts. A GPU that is slower, or shared with other
+    /// hosts, then accumulates submitted frames and the staging memory each one
+    /// holds, and the hosts sharing it slow each other down. Waiting for the
+    /// frame before last rather than the last keeps extraction overlapping the
+    /// GPU's work on the previous frame.
+    pub fn pace_frame(&mut self) {
+        // An empty submission marks everything submitted so far.
+        let latest = self.queue.submit(std::iter::empty());
+        if let Some(previous) = self.paced.replace(latest) {
+            // A lost device fails the wait; the next render reports and
+            // recovers from it, so the wait has nothing to add.
+            let _ = self.device.poll(wgpu::PollType::Wait {
+                submission_index: Some(previous),
+                timeout: None,
+            });
+        }
+    }
+
     /// The generation a new frame may reference; zero requests a complete upload.
     pub fn scene_revision(&self) -> u32 {
         if let Some(revision) = self.view_revision {

native/rust/winit-host/src/main.rs

@@ -19,7 +19,7 @@ mod shadowtests;
 use std::collections::HashMap;
 use std::path::PathBuf;
 use std::sync::Arc;
-use std::time::Instant;
+use std::time::{Duration, Instant};
 
 use anyhow::{anyhow, bail, Context, Result};
 use bridge::{
@@ -390,7 +390,8 @@ impl App {
                 self.frame_parked = true;
                 return Ok(());
             }
-            FrameState::Continue => self.frame_parked = false,
+            // The window keeps drawing a held world; its presentation paces it.
+            FrameState::Continue | FrameState::Held => self.frame_parked = false,
             FrameState::Stopped => {
                 self.frame_parked = false;
                 event_loop.exit();
@@ -989,7 +990,7 @@ fn run_render_benchmark(mut config: Config) -> Result<()> {
         match state {
             FrameState::Parked => continue,
             FrameState::Stopped => break,
-            FrameState::Continue => {}
+            FrameState::Continue | FrameState::Held => {}
         }
         if let Some(failure) = bridge.crashed()? {
             return Err(anyhow!("Nupp application crashed: {failure}"));
@@ -1050,18 +1051,26 @@ fn run_headless(config: Config) -> Result<()> {
     } else {
         None
     };
+    // Frames that ran and were drawn; `--frames` counts these. A held turn
+    // drawn for a screenshot is not one of them.
     let mut rendered = 0_u32;
+    let mut drawn = false;
+    // Whether a held turn went undrawn since the last draw. A debug tool can
+    // edit the world between frames, so the exit screenshot draws again then.
+    let mut stale = false;
     let mut synthetic = SyntheticPointer::from_environment()?;
     loop {
         if let Some(synthetic) = &mut synthetic {
             synthetic.push(&mut bridge)?;
         }
         let state = bridge.iterate(if config.offscreen { 1.0 / 60.0 } else { 0.0 })?;
+        let mut changed = false;
         while let Some(pending) = bridge.next_image_command()? {
             match &mut graphics {
                 Some(graphics) => {
                     let reason = apply_pending_image(graphics, &pending.command);
                     bridge.report_image_result(pending.request, reason.as_deref())?;
+                    changed = true;
                 }
                 // Nothing will ever apply it, so the image stays pending.
                 None => bridge.skip_image_command(pending.request)?,
@@ -1070,31 +1079,53 @@ fn run_headless(config: Config) -> Result<()> {
         match state {
             FrameState::Stopped => break,
             FrameState::Parked => continue,
-            FrameState::Continue => {}
+            FrameState::Continue | FrameState::Held => {}
         }
+        let held = state == FrameState::Held;
         if let Some(graphics) = &mut graphics {
             while let Some((id, bytes)) = bridge.next_model_upload()? {
                 graphics.upload_model(id, &bytes)?;
+                changed = true;
             }
             let mut captures = Vec::new();
             while let Some(id) = bridge.next_capture()? {
                 captures.push(id);
             }
-            if !captures.is_empty() {
-                graphics.request_capture();
-            }
-            let packet = bridge.render_packet(graphics.scene_revision())?;
-            graphics.render(&packet)?;
-            if !captures.is_empty() {
-                let capture = graphics.take_capture();
-                for id in captures {
-                    bridge.capture_result(id, &capture)?;
+            changed |= !captures.is_empty();
+            // A held turn left the frame the last draw showed, so drawing it
+            // again only redraws the same image. It is drawn when a screenshot
+            // asks for it or residency changed under it.
+            if held && !changed && drawn {
+                stale = true;
+            } else {
+                if !captures.is_empty() {
+                    graphics.request_capture();
                 }
+                let packet = bridge.render_packet(graphics.scene_revision())?;
+                graphics.render(&packet)?;
+                graphics.pace_frame();
+                if !captures.is_empty() {
+                    let capture = graphics.take_capture();
+                    for id in captures {
+                        bridge.capture_result(id, &capture)?;
+                    }
+                }
+                drawn = true;
+                stale = false;
             }
         }
         if let Some(failure) = bridge.crashed()? {
             return Err(anyhow!("Nupp application crashed: {failure}"));
         }
+        if held {
+            // Nothing runs until a debug tool grants a frame, so the turn
+            // waits rather than spinning. The wait bounds how late a tool
+            // call is answered.
+            if !changed {
+                std::thread::sleep(HELD_TURN_WAIT);
+            }
+            continue;
+        }
         if config.offscreen {
             rendered += 1;
             if rendered >= config.max_frames.unwrap_or(120) {
@@ -1105,14 +1136,27 @@ fn run_headless(config: Config) -> Result<()> {
     if let Some(failure) = bridge.crashed()? {
         return Err(anyhow!("Nupp application crashed: {failure}"));
     }
-    if let (Some(graphics), Some(path)) = (&graphics, &config.screenshot) {
+    if let (Some(graphics), Some(path)) = (&mut graphics, &config.screenshot) {
+        if stale {
+            let packet = bridge.render_packet(graphics.scene_revision())?;
+            graphics.render(&packet)?;
+        }
         let capture = graphics.capture_offscreen()?;
         std::fs::write(path, capture.png)
             .with_context(|| format!("write screenshot {}", path.display()))?;
     }
     bridge.shutdown()
 }
 
+/// How long the headless loop waits after a held turn that had nothing to do.
+///
+/// A tool call takes a few turns to be read, answered and flushed, so each one
+/// waits this out several times while the clock is paused. Measured on a
+/// paused colonize host: 1 ms took a `clock_info` round trip from 2.5 ms (the
+/// old loop, drawing every turn) to 3.1 ms at 4% of a core idle; 250 us brings
+/// it to 1.2 ms at 9%; 100 us gains little more for 16%.
+const HELD_TURN_WAIT: Duration = Duration::from_micros(250);
+
 /// Pointer moves pushed before every headless frame, for measuring the input
 /// crossing without a window. `TECS_SYNTH_POINTER` names how many per frame.
 struct SyntheticPointer {

src/tecs/application.nupp

@@ -29,8 +29,10 @@ host's batch for that turn exactly as a platform event is, so per-frame edges
 such as `keyPressed` survive the frame's `beginFrame`. `pause` holds the
 simulation: an iteration that has no granted frame runs no system, folds no
 input, leaves `frame` and `elapsed` unchanged, and keeps the host's events for
-the frame that eventually runs, honoring only a quit. The host keeps rendering
-the held world, so a screenshot requested while paused shows it. `step` grants
+the frame that eventually runs, honoring only a quit. A held turn has nothing
+new to draw, so the offscreen host stops rendering and waits between turns
+instead, but it still renders the held world for a screenshot requested while
+paused, which therefore shows it. `step` grants
 frames that run while paused, each with the fixed delta it names rather than
 the host's measured one, so a paused application advances by exactly the
 granted frames and simulated time.
@@ -144,9 +146,10 @@ export record Application is mcpbindings.Application
     --- A held iteration runs no system and folds no input, and leaves `frame`
     --- and `elapsed` unchanged. Host events wait for the next frame that runs,
     --- up to the latest 1024, except that a quit or close request still stops
-    --- the application. The
-    --- host keeps rendering the held world. Pausing again does nothing, and
-    --- frames already granted by `step` are kept.
+    --- the application. `tecs.host` reports a held iteration as `held`, and the
+    --- offscreen host then renders the held world only for a pending
+    --- screenshot. Pausing again does nothing, and frames already granted by
+    --- `step` are kept.
     --- @param self the application to hold
     pause: function(exclusive self: Application): nil
 

src/tecs/host.nupp

@@ -37,6 +37,12 @@ local images = require("tecs.gfx.images")
 local platformevents = require("tecs.platform.events")
 local window = require("tecs.platform.window")
 
+--- What one host turn did, as `iterate` reports it to the host.
+---
+--- `held` is a turn the paused application held, which ran no frame. The
+--- states are a compatibility surface between this module and the host.
+export type FrameState = hostpump.State | "held"
+
 --- The address every input route delivers at. The host drives one window.
 export const INPUT_ADDRESS: integer = 1
 
@@ -567,8 +573,13 @@ export function init(exclusive session: Session): boolean
 end
 
 --- Starts one frame or polls the frame already parked on asynchronous work.
---- @return `parked`, `continue`, or `stopped`
-export function iterate(exclusive session: Session, dt: number): hostpump.State
+---
+--- A turn the paused application held ran no frame and left the world as the
+--- last one did, apart from what a debug tool changed between frames, so a
+--- host has nothing new to draw and may wait before its next turn.
+--- @return `parked`, `continue` after a frame ran, `held` after a held turn, or
+---     `stopped`
+export function iterate(exclusive session: Session, dt: number): FrameState
     const pump = session.pump
     session.app:poll(pump:parked())
     if pump:parked() then
@@ -582,8 +593,10 @@ export function iterate(exclusive session: Session, dt: number): hostpump.State
     local batch = session.events
     session.events = {}
     local app = session.app
-
-    return pump:start(function(): boolean
+    -- Every iteration that runs a frame advances `frame`, so a turn that
+    -- finished with it unchanged is one `pause` held.
+    const frame = app.frame
+    const state = pump:start(function(): boolean
         if not app:iterate(dt, batch) then
             return false
         end
@@ -593,6 +606,11 @@ export function iterate(exclusive session: Session, dt: number): hostpump.State
 
         return true
     end)
+    if state == "continue" and app.frame == frame then
+        return "held"
+    end
+
+    return state
 end
 
 --- Runs application shutdown once.

src/tecs/mcp.nupp

@@ -64,9 +64,11 @@ create a security boundary.
 `send_event` queues its event for the next frame that runs, behind the host's
 own input for that frame, so a key it sends is pressed on exactly that frame.
 `clock_pause` holds the simulation: host turns run no system and fold no input,
-and `frame` and `elapsed` stand still, while the host keeps rendering the held
-world. `clock_step` grants frames to a paused clock, one per host turn, each
-with a fixed `dt` (1/60 s by default) instead of the measured one, and
+and `frame` and `elapsed` stand still. An `--offscreen` host renders a held
+turn only for a pending screenshot and otherwise waits a quarter of a
+millisecond before the next one, so a paused host stays nearly idle.
+`clock_step` grants frames to a paused clock, one per host turn, each with a
+fixed `dt` (1/60 s by default) instead of the measured one, and
 `clock_resume` returns to the host's delta. `clock_info` reports `settled` once
 a paused clock has run every granted frame and no frame is parked mid-flight,
 which is when the world is safe to read.

tests/hosttest.nupp

@@ -351,4 +351,27 @@ function suite.shutdownUnwindsAParkedUpdateWithoutReportingACrash(): nil
     test.equal(host.crashed(session), nil)
 end
 
+function suite.reportsAHeldTurnApartFromAFrameThatRan(): nil
+    local session = host.create(nil, nil, nil, false, nil)
+    host.init(session)
+    test.equal(host.iterate(session, 0.25), "continue")
+    session.app:pause()
+
+    -- A held turn runs no frame, which is what lets a host skip drawing it.
+    test.equal(host.iterate(session, 0.25), "held")
+    test.equal(host.iterate(session, 0.25), "held")
+    test.equal(session.app.frame, 1)
+    session.app:step(1, 0.5)
+    test.equal(host.iterate(session, 0.25), "continue")
+    test.equal(session.app.frame, 2)
+    test.equal(session.app.elapsed, 0.75)
+    test.equal(host.iterate(session, 0.25), "held")
+
+    -- A quit is still honored on a held turn, and stops rather than holds.
+    session.app:queueEvent(platformevents.quit())
+    test.equal(host.iterate(session, 0.25), "stopped")
+    test.equal(session.app.frame, 2)
+    test.assert(host.shutdown(session))
+end
+
 return suite