Repository navigation
Sixel fixes for VT DOOM: decoder crash, dropped frames, aspect ratio, shared registers, GC pressure - #177
Conversation
… colour registers VT DOOM showed a black screen. Three things, found by running it headless through Porta.Pty into XTerm.NET: - A wide picture growing downwards near the pixel budget crashed the decoder. Growing for a taller band doubled the width too, overshot the budget, and the retry at "exactly the size asked for" took the new band's narrower width, so copying the old rows ran off the end of the new buffer. The exception escaped Terminal.Write and ended the host's read loop, so ConPTY blocked and the program behind it stopped on its first frame. Only the short dimension doubles now, and the retry never shrinks the canvas. - The raster attribute's aspect ratio (Pan;Pad) was ignored. VT DOOM scales vertically with it, so the game drew at a fifth of its height. Each sixel row is now Pan/Pad pixels tall, capped at 16; P1 stays ignored, as in xterm. - Colour registers are shared between images by default, as on a VT340 and in Windows Terminal, instead of xterm's private default. VT DOOM sends its palette only when it changes (29 frames in 496); the rest drew in the sixteen default colours on black. CSI ? 1070 h still selects private registers, and RIS now resets the shared ones. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… strips in blocks VT DOOM ran, but froze for seconds at a time and stopped for good after a key press, and input arrived in bursts. - Frames were discarded. The canvas doubles while a picture of unknown size is drawn, so a 2,188-pixel-wide frame sat on a 4,096-wide canvas; growing it taller tested that capacity against MaxSixelPixels, failed, and the retry kept the whole capacity and failed again. At 307x75 cells frames that grew that way were dropped: data arriving, nothing drawn, and after the menu appeared, every frame. The retry now shrinks to what has been drawn, and the growth copies only that. - The decoder allocated a fresh canvas per picture, again per growth, and once more for the exactly sized copy -- several large-object-heap arrays per frame, two dozen full collections a second. A per-terminal SixelCanvas now holds two buffers reused from picture to picture (a Sixel picture is discarded once painted), the result is compacted in place, and the canvas starts at the last picture's size, so frames of one size never grow. SixelPlaced subscribers get a copy. - SixelStrip held a line's pixels in one array, past the large-object threshold at an ordinary width; a host capturing frames shares every line, and the next paint copied the whole row. Strips are now blocks of 16 cells, allocated when painted and copied on write one block at a time. SixelStrip.Pixels is replaced by CopyPixelsTo(destination, stride, columns); README updated. Full collections while VT DOOM runs: 373 in 15 s before, a handful after. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
BlockColumns is private, and the README describes CopyPixelsTo rather than how a strip stores its pixels: CopyPixelsTo hands out an ordinary contiguous image, and nothing outside depends on the block size. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Perf comparison — this change, against its base3 run(s) of each side, alternating on one machine. Allocation is a count and is gated exactly. Time is a measurement, so its gate is derived from the spread this job just observed in itself rather than fixed in advance.
Each corpus is gated at assemblies measured
Perf comparison — cumulative, everything since 2.0.73 run(s) of each side, alternating on one machine. Allocation is a count and is gated exactly. Time is a measurement, so its gate is derived from the spread this job just observed in itself rather than fixed in advance.
Each corpus is gated at assemblies measured
|
* Update XTerm.NET to 2.0.7 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Example: font size menu and per-frame timings A Font Size menu (and Ctrl+Plus/Minus) to tell cell count from pixel area, and a fixed-height status panel with Avalonia's own frame timings -- UI render, layout, compositor update and compositor paint -- plus the read loop's throughput and time in Terminal.Write. Render > Repaint test redraws the whole terminal for 120 frames and reports what each frame cost on each thread. Render also toggles the Skia renderer and Avalonia's timing graphs; EXAMPLE_SOFTWARE=1 and EXAMPLE_GPU_CACHE_MB change how Avalonia renders. The read-loop counters are internal to TerminalView, visible to Example. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Draw pictures in the Skia renderer, and Sixel as one image per line (4.1.0) The Skia renderer handed every row holding a picture back to the classic renderer, text and all. A SixelBuffer app has pictures on every row, so the whole screen went classic: thousands of recorded draws a frame. - Rows with pictures in front of the text (Sixel, Kitty z >= 0) stay on the Skia path. A Kitty placement's rows are drawn as one draw clipped to all of them, ordered by z then placement serial -- every row's own order. The plan is the classic path's TryPlanImageBlit, from its own runs. - Pictures are uploaded once as GPU textures, released on the render thread. Painting tracked total picture size (~5 ms/MB) before this. - Picture bitmaps on the classic path are immutable. - Sixel comes from XTerm.NET 3.0's per-line SixelStrip: one draw per row out to the last painted cell, over the text and under Kitty pictures in front, uploaded again only when its version moves. Double-width rows draw it inside their transform. ControlCatalog in Sixel mode at 307x77: 593 ms per frame on 2.0.7, 0.6 ms now. Skia and classic output are pixel-identical in the headless harness. Requires XTerm.NET 3.0.0. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Draw text on the classic path and pictures on a Skia picture layer Measured at full screen (307x75, 2540x1231) the two renderers each won half: the classic path drew text in 1 ms against the Skia text layer's 5, because it replays each line's shaped glyph runs, while the Skia layer drew pictures from GPU textures -- Sixel 0.7 ms against 3.7-5.5, 741 Kitty pictures 35 ms against 250. A larger GPU cache did not close that gap. So text always goes through the classic path, the one Consolonia sees, and the pictures in front of it go to a PictureLayer: one custom draw operation per frame, planned on the UI thread from the classic path's own runs and TryPlanImageBlit, drawing every row's Sixel strip and then each Kitty placement once from GPU textures. A placement's rows are clipped to one rectangle when they stack into one (nearly always) and row by row when not; the path clip they had cost more than the draws it saved. Where Avalonia has no Skia canvas the layer reports it and pictures go back to DrawImage. The Skia text renderer is removed (snapshot builder, text layer, font cache). UseSkiaRenderer stays on all three controls as an obsolete no-op. Now: text 0.46 ms, Sixel 0.6 ms, Kitty (741 pictures) 14.7 ms per frame. The picture layer and the DrawImage path are pixel-identical headless. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * README: pictures go through the picture layer, text through DrawingContext Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Reference XTerm.NET 3.0.0-rc001, the published prerelease Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Upload Sixel strips through SixelStrip.CopyPixelsTo XTerm.NET now holds a strip in blocks of 16 cells (tomlm/XTerm.NET#177), so there is no Pixels array to read. Both upload paths -- the Skia picture layer's texture and the DrawImage fallback's bitmap -- copy the strip out with CopyPixelsTo into a pooled buffer, so an upload per frame is not a large allocation per frame, and only up to the last painted cell, which is all a row draws. Requires the XTerm.NET release containing #177. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Reference XTerm.NET 3.0.0-rc002, which has SixelStrip.CopyPixelsTo Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
VT DOOM (https://github.com/j4james/vtdoom) showed a black screen in Avalonia.Terminal and works in Windows Terminal. Diagnosed by running it headless through Porta.Pty into XTerm.NET and into a headless
TerminalView, then at the user's real window size.What was wrong, in the order found
MaxSixelPixels, and the retry at "exactly the size asked for" took the new band's narrower width; copying the old rows ran off the end. The exception escapedTerminal.Writeand ended the host's read loop, so ConPTY blocked and VT DOOM stopped on its first frame. Only the short dimension doubles now, and the retry never cuts into drawn pixels.Pan;Pad("5;1). Each sixel row is now Pan/Pad pixels tall (capped at 16). P1 stays ignored, as in xterm.CSI ? 1070 hstill selects private registers, DECRQM reports it, and RIS now also resets the shared registers.SixelCanvaswith two reused buffers, in-place compaction, starting at the last picture's size;SixelStripheld a row in one array past the large-object threshold, and a host capturing frames made every repaint copy it → strips are now blocks of 16 cells, allocated on paint and copied on write per block.Measured: full collections 373 → a handful in 15 s; VT DOOM at 307×75 places every frame (150/151, the last one in flight) through Escape and Enter.
API change (3.0 is at rc001)
SixelStrip.Pixelsis replaced byCopyPixelsTo(Span<byte> destination, int destinationStride, int columns), which hands out an ordinary contiguous BGRA image. The block storage behind it is internal; nothing outside depends on it. README's 3.0 section and drawing guide updated. Avalonia.Terminal's matching change is in its own PR.Behaviour changes worth noting for 3.0: shared colour registers by default; the aspect ratio is honoured.
Tests
2,312 pass. New: the growth crash (fails without the fix), a picture inside the budget surviving the canvas doubling past it, aspect ratio (3:1, square == none, capped), shared/private/RIS colour registers, a reused canvas not showing the previous picture. Strip tests read pixels through
CopyPixelsTo; the budget test works from real strip sizes.🤖 Generated with Claude Code