diff --git a/README.md b/README.md index 7ed0d40..b982e3e 100644 --- a/README.md +++ b/README.md @@ -8,11 +8,11 @@ A .NET terminal emulator library inspired by [xterm.js](https://github.com/xterm XTerm.NET provides a headless terminal emulator that parses and processes VT100/ANSI escape sequences, making it easy to host console applications in your .NET applications. -**Sixel and Kitty graphics are supported.** A picture is held as a run on the line it appears on, so -it scrolls with that line, is cleared by `ED`/`EL`, and is freed when the line falls out of the -scrollback — and a resize does nothing to it at all. Sixel is *content*, so printing over it replaces -that part of the picture; Kitty is an *overlay* ordered against the text by its z-index. See -[Images](#images). +**Sixel and Kitty graphics are supported.** Sixel is *content*: each line keeps the pixels Sixel +painted on it, a later Sixel paints over them, and printing over a cell replaces its pixels. Kitty is +an *overlay*: a picture is held as a run on the line it appears on, ordered against the text by its +z-index. Either way a picture scrolls with its line, is cleared by `ED`/`EL`, and is freed when the +line falls out of the scrollback — and a resize does nothing to it at all. See [Images](#images). ## Features @@ -31,8 +31,10 @@ that part of the picture; Kitty is an *overlay* ordered against the text by its - **Left and Right Margins** — `DECSLRM` under `DECLRMM`, so text and scrolling stay inside a pane - **Shell Integration** — `OSC 133` prompt marks survive reflow, `OSC 8` hyperlinks, `OSC 52` clipboard, and desktop notifications (`OSC 9`/`99`/`777`) -- **Sixel Graphics** — Decodes Sixel images (`ESC P … q`) and holds them as runs on the lines they - appear on, so `img2sixel`, `chafa`, `lsix` and `timg` work against a host that renders them +- **Sixel Graphics** — Decodes Sixel images (`ESC P … q`) and paints them into the lines they land + on, so `img2sixel`, `chafa`, `lsix` and `timg` work against a host that renders them — and a + program repainting small regions over and over costs one strip of pixels per line, however many + Sixels it sends - **Kitty Graphics** — Decodes the Kitty protocol (`ESC _ G …`), including chunked transmission, PNG, transmit-once/place-many by image id, animation, and U+10EEEE Unicode placeholders, so `icat`, `chafa -f kitty`, `timg -pk`, `yazi` and `image.nvim` work the same way @@ -45,6 +47,54 @@ that part of the picture; Kitty is an *overlay* ordered against the text by its ## Compliance ![Compliance](https://raw.githubusercontent.com/tomlm/Iciclecreek.Avalonia.Terminal/main/compliance.png) +## Upgrading to 3.0 + +**3.0 changes how Sixel is stored.** If you only host a terminal and draw text, or draw only Kitty +pictures, nothing changes. If you draw Sixel, you draw it from a new place. + +In 2.x a Sixel was a picture: a `TerminalImage` and one `LinePlacement` per row, split wherever +printing or a later Sixel covered part of it. A program that repaints small regions — a cursor, a +hover highlight, a diffing framebuffer such as SixelBuffer — left a patchwork behind: thousands of +fragments per screen, each still pinning its whole picture and each its own draw. Sixel is content, +so in 3.0 a line keeps the *pixels* Sixel painted on it, in a `SixelStrip`, and the picture that +painted them is not kept at all. + +**What changed.** + +| 2.x | 3.0 | +| --- | --- | +| A Sixel appears in `line.Placements` and `line.Images` | It is painted into `line.Sixel`; `Placements` and `Images` hold Kitty pictures only | +| `PlacementKind.Sixel` runs | Never created. The value remains so code that compares against it still compiles | +| `TryGetPlacementAt` / `TryGetImageAt` find Sixel | They find Kitty pictures; ask `line.Sixel?.HasPixelsAt(column)` | +| A later Sixel splits the one under it | It paints over it: opaque pixels replace, unset pixels (background select 1) leave what was there | +| `ICH`/`DCH`, insert mode and margin scrolls left Sixel behind | Its pixels move with their cells | +| Kitty deletes by position or z-index also removed Sixel runs | They no longer touch Sixel; `a=d,d=a` still clears everything | +| `MaxImageBytes` counted pictures | It counts pictures and strips; scrollback strips go first, oldest line first, and what is on screen is never taken | +| iTerm2 inline images were Sixel placements | They are painted into strips, as Sixel is | + +`line.HasImages` is true for a line with a strip as well as one with Kitty runs. + +**What a renderer does now.** Per line: Kitty pictures behind the text (negative `ZIndex`), then the +text, then the line's Sixel strip, then Kitty pictures in front — see [Images](#images) for the whole +row. The strip is one blit: + +```csharp +if (line.Sixel is { } strip) +{ + // BGRA8888, straight alpha, top row first: PaintedColumns cells of CellWidth x CellHeight pixels. + // Cache the upload against the strip and upload again when strip.Version changes. + var cols = Math.Min(strip.PaintedColumns, line.Length); + DrawImage(bitmapFor(strip), + source: (0, 0, cols * strip.CellWidth, strip.CellHeight), + dest: (0, row * cellHeight, cols * cellWidth, cellHeight)); +} +``` + +A strip is painted at the cell size in force when its Sixels arrived, so after a font change it is +stretched onto the new grid, as a picture is. A strip can be painted by the thread writing to the +terminal while you read it: read `Version` before copying and again after, and treat a copy whose +version moved as stale. + ## Upgrading to 2.0 **2.0 targets .NET 10** and removes the per-cell image API. If you only host a terminal and draw @@ -434,15 +484,21 @@ are reported honestly either way. ### Images Two graphics protocols are decoded — Sixel (`ESC P … q … ESC \`) and Kitty -(`ESC _ G … ESC \`) — and both end up in the same place: a **run held by the line** the picture -appears on. Nothing about a picture is written into cells, which is what makes it survive a resize: -a run keeps its natural width, so narrowing a window shows less of a picture instead of destroying -it. The run scrolls with its line, is cleared by `ED`/`EL`, and is freed when the line falls out of -the scrollback. - -A **placement** is one appearance of a picture: which image it draws, which rectangle of that image -it takes, and how many cells it fills. Sixel makes one placement per image. Kitty can transmit a -picture once and place it many times, so several placements may share one `TerminalImage`. +(`ESC _ G … ESC \`) — and both end up held by the **line** the picture appears on. Nothing about a +picture is written into cells, which is what makes it survive a resize: what a line holds keeps its +natural width, so narrowing a window shows less of a picture instead of destroying it. It scrolls +with its line, is cleared by `ED`/`EL`, and is freed when the line falls out of the scrollback. + +The two protocols differ in what a picture *is*: + +- **Sixel is content.** Each line keeps the pixels Sixel painted on it, in a `SixelStrip` + (`line.Sixel`): one cell tall, as wide as anything painted. A later Sixel paints over it, printing + or erasing over a cell clears that cell's pixels, and an insert or delete moves them with their + cells. The picture that painted them is not kept. +- **Kitty is an overlay.** A **placement** is one appearance of a picture: which image it draws, + which rectangle of that image it takes, and how many cells it fills. It is held as one run per line + in `line.Placements`. Kitty can transmit a picture once and place it many times, so several + placements may share one `TerminalImage`. **Tell the terminal your cell size.** XTerm.NET is headless and cannot measure a font, so it cannot work out how many columns an image covers unless you say. Set these from your renderer's metrics, @@ -477,55 +533,116 @@ up to a whole row of the control's height belongs to no row at all. An applicati figure by the row count is told the terminal is taller than it is, sizes a picture to fill it, and the surplus runs off the bottom and scrolls the screen. -**Drawing the runs.** This is per *line*, not per cell — a run is one blit, not a row of tiles: +**Say when the cell size changes.** A font change, a zoom, or the window moving to a monitor with +another DPI changes how many pixels a cell is while the grid can stay the same — so `Resize` is not +called and nothing tells the application. Update the options and then call +`NotifyTextAreaPixelsChanged()`: + +```csharp +terminal.Options.CellWidthPixels = newCellWidth; +terminal.Options.CellHeightPixels = newCellHeight; +terminal.NotifyTextAreaPixelsChanged(); +``` + +That sends an in-band resize report (`CSI 48 ; rows ; cols ; height ; width t`) to any application +that asked for one with mode 2048, and is a no-op otherwise, so it is safe to call whenever the +metrics are recomputed. Without it an application that sizes its pictures from the cell size goes on +drawing at the old one: after a move to a higher-DPI monitor every picture lands smaller than its +cells, with gaps between them. + +**Drawing a row.** Per *line*, not per cell, in this order: + +1. Cell backgrounds — but not under a Kitty picture *behind* the text (negative `ZIndex`), which the + text sits on. +2. Kitty pictures behind the text, back to front. +3. The text. +4. The line's Sixel pixels, if it has any — one blit for the whole line. +5. Kitty pictures in front of the text (`ZIndex >= 0`), back to front. ```csharp var line = buffer.Lines[buffer.YDisp + row]; -// Back to front. A stable sort, so equal depths keep the order they were placed in, which is age. -foreach (var p in line.Placements.OrderBy(p => p.ZIndex)) +// Kitty runs, back to front. A stable sort, so equal depths keep the order they were placed in, +// which is age. +var kitty = line.Placements.OrderBy(p => p.ZIndex).ToList(); + +DrawBackgrounds(line, row); // skipping cells under a negative-z picture +foreach (var p in kitty.Where(p => p.ZIndex < 0)) + DrawKittyRun(line, p, row); +DrawText(line, row); + +if (line.Sixel is { } strip) { - // Pixels are BGRA8888 with straight (unpremultiplied) alpha, top row first. - // Cache your framework's bitmap against the TerminalImage — a ConditionalWeakTable keyed on it - // lets the bitmap die when the image does, with no eviction list to maintain, and two - // placements of one picture share the single upload. - // Matched by id rather than by column: two pictures may overlap the same columns, and a - // column lookup answers with whichever placement comes first, not necessarily this one. + // BGRA8888, straight alpha, top row first: cells of strip.CellWidth x strip.CellHeight pixels. + // PaintedColumns is how far anything is painted, so a narrow picture on a wide line does not + // draw the rest of the row; clip it to the line as well. + var cols = Math.Min(strip.PaintedColumns, line.Length); + DrawImage(StripBitmap(strip), + source: (0, 0, cols * strip.CellWidth, strip.CellHeight), + dest: (0, row * cellHeight, cols * cellWidth, cellHeight)); +} + +foreach (var p in kitty.Where(p => p.ZIndex >= 0)) + DrawKittyRun(line, p, row); + +void DrawKittyRun(BufferLine line, LinePlacement p, int row) +{ + // Matched by id rather than by column: two pictures may overlap the same columns, and a column + // lookup answers with whichever placement comes first, not necessarily this one. var image = line.Images.FirstOrDefault(i => i.Id == p.ImageId); - if (image is null) continue; - var bitmap = _bitmaps.GetOrCreate(image); + if (image is null) return; - DrawImage(bitmap, + DrawImage(PictureBitmap(image), source: (p.SrcX, p.SrcY, p.SrcWidth, p.SrcHeight), dest: ((p.Column * cellWidth) + p.OffsetX, (row * cellHeight) + p.OffsetY, p.Cols * cellWidth, cellHeight)); } ``` -Clip the destination to the line's width and narrow the source by the same proportion — that is what -makes a narrowed window show less of a picture rather than a squashed one. `OffsetX`/`OffsetY` are -the sub-cell shift a Kitty placement asks for with `X=`/`Y=`; they are zero for Sixel and for most -Kitty placements. - -Paint each cell's own background once, underneath the runs, and from the bottom-most run covering it -only — painting it again for a nearer run erases what is behind instead of letting it show through. - -Text goes down *between* the negative z-indices and the rest: see [Draw order](#kitty-graphics) -below for why a negative `ZIndex` means behind the text rather than merely further back. +**Upload once, and key the upload on what changes.** + +- A Kitty picture: cache your bitmap against the `TerminalImage` — a `ConditionalWeakTable` keyed on + it lets the bitmap die when the image does, with no eviction list to maintain, and two placements + of one picture share the single upload. For animation draw `image.CurrentPixels` and upload again + when `image.FrameSerial` changes. +- A Sixel strip: cache against the `SixelStrip` and upload again when `strip.Version` changes. A + strip is painted by the thread writing to the terminal, so read `Version` before copying the + pixels and again after: a copy whose version moved may be torn, so draw it but do not keep it. + A line's copies (`Clone`, `CopyFrom`) share its strip until one of them is written, so a frame + captured for a renderer costs a reference rather than the pixels. + +Draws, not pixels, are what a busy screen costs. A program repainting small regions over and over +leaves at most one strip per row however many Sixels it sends, and a Kitty client that tiles the +screen into blocks costs one draw per block per frame — keep pictures on the GPU between frames +rather than handing the framework a fresh bitmap each time. + +**Mapping onto the grid.** A strip is painted at the cell size in force when its Sixels arrived, and +a picture records the cell size it was laid out against (`image.CellWidth`/`CellHeight`). After a +font or DPI change, stretch them onto the current grid — map cells to cells, as above, rather than +pixels to pixels — so a picture stays locked to the text it was drawn among. + +For Kitty runs, clip the destination to the line's width and narrow the source by the same +proportion — that is what makes a narrowed window show less of a picture rather than a squashed one. +`OffsetX`/`OffsetY` are the sub-cell shift a placement asks for with `X=`/`Y=`; they are zero for most +placements. Paint each cell's own background once, underneath the runs, and from the bottom-most run +covering it only — painting it again for a nearer run erases what is behind instead of letting it +show through. See [Draw order](#kitty-graphics) below for why a negative `ZIndex` means behind the +text rather than merely further back. Cells under a picture hold `" "` as their content, so `TranslateToString` and selection copy yield blanks. -To ask what covers one column rather than walking the list: +To ask what covers one column rather than walking the line: ```csharp -if (line.TryGetPlacementAt(col, out LinePlacement placement)) { … } -if (line.TryGetImageAt(col, out TerminalImage image)) { … } +if (line.Sixel?.HasPixelsAt(col) == true) { … } // Sixel pixels +if (line.TryGetPlacementAt(col, out LinePlacement placement)) { … } // a Kitty run +if (line.TryGetImageAt(col, out TerminalImage image)) { … } // the Kitty picture it shows ``` -Both answer with the *first* placement covering that column, so they are the convenient form for the -ordinary case of one picture at a time — a renderer handling overlapping pictures should walk -`line.Placements` and resolve through `line.Images` as above. +The Kitty lookups answer with the *first* placement covering that column, so they are the convenient +form for the ordinary case of one picture at a time — a renderer handling overlapping pictures should +walk `line.Placements` and resolve through `line.Images` as above. **Options:** @@ -535,12 +652,13 @@ ordinary case of one picture at a time — a renderer handling overlapping pictu | `KittyGraphicsEnabled` | `true` | Decode Kitty graphics sequences and answer their queries | | `CellWidthPixels` / `CellHeightPixels` | `10` / `20` | Cell size images are laid out against | | `MaxSixelPixels` | `4_000_000` | Largest single image accepted | -| `MaxImageBytes` | `64 MB` | Budget for image data live in the buffer; oldest are dropped past it | +| `MaxImageBytes` | `64 MB` | Budget for image data live in the buffer — Kitty pictures and Sixel strips; past it, scrollback strips go first, then the oldest pictures | | `MaxImageRegistryBytes` | `32 MB` | Budget for transmitted-but-unplaced Kitty images, evicted oldest first | -Images are dropped when the terminal is resized to a different **column** count, because reflow -re-wraps lines by copying ranges of cells and the pieces would reassemble in the wrong places. A -change of row count alone keeps them. +A change of column count keeps pictures: narrowing shows less of them and widening shows more. Only +lines that are part of a wrap chain drop theirs, because reflow re-wraps those by copying ranges of +cells between lines and the pieces would reassemble in the wrong places. A change of row count alone +keeps everything. #### Kitty graphics diff --git a/src/XTerm.NET.Tests/Graphics/ImageAssertions.cs b/src/XTerm.NET.Tests/Graphics/ImageAssertions.cs index 8f1221c..3789bb1 100644 --- a/src/XTerm.NET.Tests/Graphics/ImageAssertions.cs +++ b/src/XTerm.NET.Tests/Graphics/ImageAssertions.cs @@ -94,6 +94,53 @@ public static (int Col, int Row)? TileAt(Terminal terminal, int col, int screenR return (source.X / image.CellWidth, source.Y / image.CellHeight); } + /// The Sixel pixels on a screen row, or null when no Sixel has painted it. + public static SixelStrip? StripOn(Terminal terminal, int screenRow) + => terminal.Buffer.Lines[terminal.Buffer.YBase + screenRow]?.Sixel; + + /// + /// Whether a screen position shows Sixel pixels -- the question asks of a + /// Kitty run, asked of the pixels Sixel paints instead. + /// + public static bool IsSixelAt(Terminal terminal, int col, int screenRow) + { + var line = terminal.Buffer.Lines[terminal.Buffer.YBase + screenRow]; + return line?.Sixel is { } strip && col < line.Length && strip.HasPixelsAt(col); + } + + /// + /// The Sixel pixel at (, ) inside a cell, as RGBA, or null + /// when the cell shows none. + /// + public static (byte R, byte G, byte B, byte A)? SixelPixel(Terminal terminal, int col, int screenRow, int x = 0, int y = 0) + { + if (!IsSixelAt(terminal, col, screenRow)) + return null; + + var strip = StripOn(terminal, screenRow)!; + var offset = y * strip.Stride + (col * strip.CellWidth + x) * TerminalImage.BytesPerPixel; + var p = strip.Pixels.Span; + return (p[offset + 2], p[offset + 1], p[offset], p[offset + 3]); + } + + /// How many cells of the buffer show Sixel pixels, clipped to each line's width. + public static int VisibleSixelCells(Terminal terminal) + { + var count = 0; + for (var i = 0; i < terminal.Buffer.Lines.Length; i++) + { + var line = terminal.Buffer.Lines[i]; + if (line?.Sixel is not { } strip) + continue; + + for (var c = 0; c < System.Math.Min(strip.Columns, line.Length); c++) + if (strip.HasPixelsAt(c)) + count++; + } + + return count; + } + /// Every run on a screen row, in the order they were placed. public static IReadOnlyList PlacementsOn(Terminal terminal, int screenRow) { diff --git a/src/XTerm.NET.Tests/Graphics/ImageCellLifetimeTests.cs b/src/XTerm.NET.Tests/Graphics/ImageCellLifetimeTests.cs index 9263357..dbdc37d 100644 --- a/src/XTerm.NET.Tests/Graphics/ImageCellLifetimeTests.cs +++ b/src/XTerm.NET.Tests/Graphics/ImageCellLifetimeTests.cs @@ -6,22 +6,35 @@ namespace XTerm.Tests.Graphics; /// -/// The behaviour that justifies keeping images on the cells rather than in an overlay: a picture -/// is terminal content, and everything that happens to text has to happen to it. +/// The behaviour that makes Sixel terminal content: everything that happens to text happens to the +/// pixels a Sixel painted. /// -/// Printing over a cell replaces it, erasing clears it, scrolling carries it, and falling -/// off the end of the scrollback disposes of it -- none of which needed code, because a cell is a -/// struct and every one of those paths already builds or copies whole cells. These tests exist to -/// keep it that way. +/// Each line keeps the pixels Sixel painted on it (), not the pictures +/// that painted them. Printing over a cell replaces its pixels, erasing clears them, a later Sixel +/// paints over them, scrolling carries them with their line, and falling off the end of the +/// scrollback disposes of them. These tests exist to keep it that way. /// public class ImageCellLifetimeTests { private const string Esc = "\u001b"; private const string St = Esc + "\\"; - /// Four pixels wide, twelve tall: two cells across, four down. + /// Four pixels wide, twelve tall, red: two cells across, four down. private const string TwoByFourCells = "#0;2;100;0;0!4~-!4~"; + /// The same size in blue. + private const string BlueTwoByFour = "#0;2;0;0;100!4~-!4~"; + + /// One cell across, four down, green. + private const string GreenOneByFour = "#0;2;0;100;0!2~-!2~"; + + /// Two by four, a different colour in each cell row: red, green, blue, yellow. + private const string Striped = "#0;2;100;0;0!4F$#1;2;0;100;0!4w-#2;2;0;0;100!4F$#3;2;100;100;0!4w"; + + private static readonly (byte R, byte G, byte B, byte A) Red = (255, 0, 0, 255); + private static readonly (byte R, byte G, byte B, byte A) Green = (0, 255, 0, 255); + private static readonly (byte R, byte G, byte B, byte A) Blue = (0, 0, 255, 255); + private static Terminal Fresh(Action? configure = null) { var options = new TerminalOptions @@ -41,41 +54,26 @@ private static void WriteSixel(Terminal terminal, string body = TwoByFourCells) private static BufferCell Cell(Terminal terminal, int col, int screenRow) => terminal.Buffer.Lines[terminal.Buffer.YBase + screenRow]![col]; - private static int ImageCellCount(Terminal terminal) - { - int count = 0; - for (int i = 0; i < terminal.Buffer.Lines.Length; i++) - { - var line = terminal.Buffer.Lines[i]; - if (line is null) - continue; - // Counted from the runs and clamped to the line's width, so this reports what is - // VISIBLE — anything wider is hidden by the window, not destroyed. - foreach (var placement in line.Placements) - { - var end = Math.Min(placement.EndColumn, line.Length); - count += Math.Max(0, end - placement.Column); - } - } - return count; - } + private static BufferLine Line(Terminal terminal, int screenRow) + => terminal.Buffer.Lines[terminal.Buffer.YBase + screenRow]!; + + private static int SixelCells(Terminal terminal) => ImageAssertions.VisibleSixelCells(terminal); [Fact] - public void Sixel_over_a_sixel_replaces_the_runs_it_covers() + public void Sixel_over_a_sixel_paints_over_it() { var terminal = Fresh(); WriteSixel(terminal); - var first = ImageAssertions.ImageAt(terminal, 0, 0); - // The same picture drawn again at the same place: one run per row, not two terminal.Write($"{Esc}[1;1H"); - WriteSixel(terminal); - - var line = terminal.Buffer.Lines[terminal.Buffer.YBase]!; - Assert.Single(line.Placements); - Assert.Single(line.Images); - Assert.NotSame(first, ImageAssertions.ImageAt(terminal, 0, 0)); - Assert.Equal(8, ImageCellCount(terminal)); + WriteSixel(terminal, BlueTwoByFour); + + // One strip per line holding the newer pixels; nothing of the older picture is kept, and + // nothing about either becomes a run. + Assert.Equal(Blue, ImageAssertions.SixelPixel(terminal, 0, 0)); + Assert.Equal(Blue, ImageAssertions.SixelPixel(terminal, 1, 3)); + Assert.Empty(Line(terminal, 0).Placements); + Assert.Equal(8, SixelCells(terminal)); } [Fact] @@ -83,37 +81,60 @@ public void Sixel_over_part_of_a_sixel_keeps_the_rest_of_it() { var terminal = Fresh(); WriteSixel(terminal); - var first = ImageAssertions.ImageAt(terminal, 0, 0); // A one-cell-wide picture on the second column, over the right half of the first one terminal.Write($"{Esc}[1;2H"); - WriteSixel(terminal, "#0;2;0;100;0!2~-!2~"); + WriteSixel(terminal, GreenOneByFour); - var line = terminal.Buffer.Lines[terminal.Buffer.YBase]!; - Assert.Equal(2, line.Placements.Count); - Assert.Same(first, ImageAssertions.ImageAt(terminal, 0, 0)); - Assert.NotSame(first, ImageAssertions.ImageAt(terminal, 1, 0)); - Assert.Equal(8, ImageCellCount(terminal)); + Assert.Equal(Red, ImageAssertions.SixelPixel(terminal, 0, 0)); + Assert.Equal(Green, ImageAssertions.SixelPixel(terminal, 1, 0)); + Assert.Equal(8, SixelCells(terminal)); } + /// + /// The case the strips exist for: a program repainting a small region over and over keeps one + /// strip per line, the same size it started, however many Sixels it sends. + /// [Fact] - public void Repainting_a_cell_many_times_does_not_accumulate_runs() + public void Repainting_a_cell_many_times_accumulates_nothing() { var terminal = Fresh(); WriteSixel(terminal); + var bytes = Enumerable.Range(0, 4).Sum(row => ImageAssertions.StripOn(terminal, row)!.ByteCount); for (int i = 0; i < 50; i++) { terminal.Write($"{Esc}[1;2H"); - WriteSixel(terminal, "#0;2;0;0;100!2~-!2~"); + WriteSixel(terminal, i % 2 == 0 ? GreenOneByFour : "#0;2;0;0;100!2~-!2~"); } for (int row = 0; row < 4; row++) { - var line = terminal.Buffer.Lines[terminal.Buffer.YBase + row]!; - Assert.Equal(2, line.Placements.Count); - Assert.Equal(2, line.Images.Count); + Assert.Empty(Line(terminal, row).Placements); + Assert.Equal(Red, ImageAssertions.SixelPixel(terminal, 0, row)); + Assert.Equal(Blue, ImageAssertions.SixelPixel(terminal, 1, row)); } + + Assert.Equal(bytes, Enumerable.Range(0, 4).Sum(row => ImageAssertions.StripOn(terminal, row)!.ByteCount)); + } + + /// + /// Background select 1: a Sixel's unset pixels are transparent, and whatever was painted under + /// them shows through -- which a program relies on to draw a sprite over a picture. + /// + [Fact] + public void A_transparent_sixel_leaves_what_is_under_its_unset_pixels() + { + var terminal = Fresh(); + WriteSixel(terminal); + + // Only the top pixel row of each sixel is set ('@'): rows 1 and 2 of the first cell stay red. + terminal.Write($"{Esc}[1;1H"); + WriteSixel(terminal, "#0;2;0;100;0!4@"); + + Assert.Equal(Green, ImageAssertions.SixelPixel(terminal, 0, 0, y: 0)); + Assert.Equal(Red, ImageAssertions.SixelPixel(terminal, 0, 0, y: 1)); + Assert.Equal(Red, ImageAssertions.SixelPixel(terminal, 0, 0, y: 2)); } [Fact] @@ -124,12 +145,12 @@ public void Printing_over_a_tile_replaces_it_with_the_character() terminal.Write($"{Esc}[1;1HX"); - Assert.Null(ImageAssertions.ImageAt(terminal, 0, 0)); + Assert.False(ImageAssertions.IsSixelAt(terminal, 0, 0)); Assert.Equal("X", Cell(terminal, 0, 0).Content); // and only that cell - Assert.NotNull(ImageAssertions.ImageAt(terminal, 1, 0)); - Assert.NotNull(ImageAssertions.ImageAt(terminal, 0, 1)); + Assert.True(ImageAssertions.IsSixelAt(terminal, 1, 0)); + Assert.True(ImageAssertions.IsSixelAt(terminal, 0, 1)); } [Fact] @@ -140,9 +161,12 @@ public void Erase_in_line_clears_the_tiles_on_that_row() terminal.Write($"{Esc}[1;1H{Esc}[K"); - Assert.Null(ImageAssertions.ImageAt(terminal, 0, 0)); - Assert.Null(ImageAssertions.ImageAt(terminal, 1, 0)); - Assert.NotNull(ImageAssertions.ImageAt(terminal, 0, 1)); + Assert.False(ImageAssertions.IsSixelAt(terminal, 0, 0)); + Assert.False(ImageAssertions.IsSixelAt(terminal, 1, 0)); + Assert.True(ImageAssertions.IsSixelAt(terminal, 0, 1)); + + // A line with nothing left drops its strip rather than keeping a blank one. + Assert.Null(ImageAssertions.StripOn(terminal, 0)); } [Fact] @@ -153,7 +177,7 @@ public void Erase_in_display_clears_every_tile_on_screen() terminal.Write($"{Esc}[2J"); - Assert.Equal(0, ImageCellCount(terminal)); + Assert.Equal(0, SixelCells(terminal)); } [Fact] @@ -164,8 +188,43 @@ public void Erase_characters_clears_the_tiles_it_covers() terminal.Write($"{Esc}[1;1H{Esc}[1X"); - Assert.Null(ImageAssertions.ImageAt(terminal, 0, 0)); - Assert.NotNull(ImageAssertions.ImageAt(terminal, 1, 0)); + Assert.False(ImageAssertions.IsSixelAt(terminal, 0, 0)); + Assert.True(ImageAssertions.IsSixelAt(terminal, 1, 0)); + } + + /// + /// An insert moves the text after the cursor right, and the picture it sits in moves with it. + /// Pictures held as runs stayed where they were while their cells moved out from under them. + /// + [Fact] + public void Inserting_characters_moves_the_pixels_with_their_cells() + { + var terminal = Fresh(); + WriteSixel(terminal, Striped); + + terminal.Write($"{Esc}[1;1H{Esc}[3@"); // ICH 3 on the top row + + Assert.False(ImageAssertions.IsSixelAt(terminal, 0, 0)); + Assert.False(ImageAssertions.IsSixelAt(terminal, 2, 0)); + Assert.Equal(Red, ImageAssertions.SixelPixel(terminal, 3, 0)); + Assert.Equal(Red, ImageAssertions.SixelPixel(terminal, 4, 0)); + + // Only that row: the one below still starts at the left margin. + Assert.Equal(Green, ImageAssertions.SixelPixel(terminal, 0, 1)); + } + + [Fact] + public void Deleting_characters_moves_the_pixels_with_their_cells() + { + var terminal = Fresh(); + terminal.Write($"{Esc}[1;4H"); + WriteSixel(terminal, Striped); + + terminal.Write($"{Esc}[1;1H{Esc}[2P"); // DCH 2 on the top row + + Assert.Equal(Red, ImageAssertions.SixelPixel(terminal, 1, 0)); + Assert.Equal(Red, ImageAssertions.SixelPixel(terminal, 2, 0)); + Assert.False(ImageAssertions.IsSixelAt(terminal, 3, 0)); } [Fact] @@ -176,99 +235,88 @@ public void A_full_reset_clears_every_tile() terminal.Write($"{Esc}c"); - Assert.Equal(0, ImageCellCount(terminal)); + Assert.Equal(0, SixelCells(terminal)); } [Fact] public void Scrolling_carries_the_tiles_with_their_lines() { var terminal = Fresh(); - WriteSixel(terminal); - var image = ImageAssertions.ImageAt(terminal, 0, 0); - Assert.NotNull(image); + WriteSixel(terminal, Striped); // The absolute line the top of the image went onto. Scrolling moves the viewport over the // buffer, so this index is what should stay put while the screen row changes. var topLine = terminal.Buffer.YBase; + var strip = terminal.Buffer.Lines[topLine]!.Sixel; + Assert.NotNull(strip); // Push the screen up by two rows from the bottom. terminal.Write($"{Esc}[10;1H\r\n\r\n"); Assert.Equal(2, terminal.Buffer.YBase - topLine); - // Nothing copied the tiles anywhere: the same line object still holds them, unchanged. - var moved = terminal.Buffer.Lines[topLine]!; - Assert.True(moved.TryGetImageAt(0, out var movedImage) && ReferenceEquals(movedImage, image), - "the run did not travel with its line"); - Assert.True(moved.TryGetPlacementAt(0, out var movedPlacement)); - Assert.Equal(0, movedPlacement.SrcY); + // Nothing copied the pixels anywhere: the same line object still holds the same strip. + Assert.Same(strip, terminal.Buffer.Lines[topLine]!.Sixel); // Which puts the top of the picture two rows higher on screen than it was. - Assert.True(ReferenceEquals(ImageAssertions.ImageAt(terminal, 0, -2), image)); - Assert.Equal(8, ImageCellCount(terminal)); + Assert.Equal(Red, ImageAssertions.SixelPixel(terminal, 0, -2)); + Assert.Equal(Green, ImageAssertions.SixelPixel(terminal, 0, -1)); + Assert.Equal(8, SixelCells(terminal)); } /// - /// The disposal story: an image dies with the last cell holding it, so a picture that scrolls - /// out of a short scrollback leaves nothing behind to evict. + /// The disposal story: pixels die with the line holding them, so a picture that scrolls out of a + /// short scrollback leaves nothing behind to evict. /// [Fact] public void An_image_scrolled_out_of_the_scrollback_leaves_no_references() { var terminal = Fresh(o => o.Scrollback = 4); WriteSixel(terminal); - Assert.Equal(8, ImageCellCount(terminal)); + Assert.Equal(8, SixelCells(terminal)); terminal.Write($"{Esc}[10;1H"); for (int i = 0; i < 40; i++) terminal.Write("\r\n"); - Assert.Equal(0, ImageCellCount(terminal)); + Assert.Equal(0, SixelCells(terminal)); } /// - /// A change of width keeps the pictures. + /// A change of width keeps the pictures: a strip keeps its natural width and the renderer draws + /// as much of it as the line allows. Only a wrap chain drops them, because reflow moves cells + /// between lines. /// - /// - /// This test used to assert the opposite, and the reasoning it carried — reflow re-wraps a - /// logical line by copying ranges of cells, so tiles carried through would reassemble as a - /// shuffled mosaic — was right about reflow and wrong about the blast radius. It applied to lines - /// that actually re-wrap, and the code dropped every picture in the buffer on any width change at - /// all, including widening a window, which is the most common resize there is. - /// With a picture held as a run on its line rather than tiles in cells, there is nothing to - /// shuffle: the renderer draws as much of the run as the width allows. Narrowing shows less, - /// widening shows more, and the wrap-chain case is still dropped, on its own, below. - /// [Fact] public void A_change_of_width_keeps_the_images() { var terminal = Fresh(); WriteSixel(terminal); - Assert.Equal(8, ImageCellCount(terminal)); + Assert.Equal(8, SixelCells(terminal)); terminal.Resize(15, 10); - Assert.Equal(8, ImageCellCount(terminal)); + Assert.Equal(8, SixelCells(terminal)); terminal.Resize(40, 10); - Assert.Equal(8, ImageCellCount(terminal)); + Assert.Equal(8, SixelCells(terminal)); } /// /// Narrowing past a picture hides the overhang rather than destroying it, and widening brings it - /// back — because a run keeps its natural width and only the drawing is clipped. + /// back. /// [Fact] public void Narrowing_past_a_picture_hides_it_and_widening_restores_it() { var terminal = Fresh(); WriteSixel(terminal); - Assert.Equal(8, ImageCellCount(terminal)); + Assert.Equal(8, SixelCells(terminal)); terminal.Resize(1, 10); // the picture is two columns wide - Assert.Equal(4, ImageCellCount(terminal)); + Assert.Equal(4, SixelCells(terminal)); terminal.Resize(20, 10); - Assert.Equal(8, ImageCellCount(terminal)); + Assert.Equal(8, SixelCells(terminal)); } /// A change of height moves whole lines, so there is nothing to be confused about. @@ -280,7 +328,7 @@ public void A_change_of_height_alone_keeps_the_images() terminal.Resize(20, 14); - Assert.Equal(8, ImageCellCount(terminal)); + Assert.Equal(8, SixelCells(terminal)); } [Fact] @@ -297,26 +345,31 @@ public void Clearing_a_dropped_image_leaves_a_blank_rather_than_a_hole() } /// - /// The backstop for the case reference counting cannot reach: a deep scrollback full of - /// pictures, every one still referenced and every one still in memory. + /// The backstop for a deep scrollback full of pictures: over budget, the Sixel pixels furthest + /// back in the history go first. /// [Fact] - public void The_oldest_images_are_dropped_when_the_budget_is_exceeded() + public void Scrollback_pixels_are_dropped_first_when_the_budget_is_exceeded() { - // Each image here is 4x12 BGRA -- 192 bytes -- so a 200 byte budget holds exactly one. - var terminal = Fresh(o => o.MaxImageBytes = 200); - + // Each painted line is a 20-cell strip of 2x3 BGRA cells: 480 bytes. Two pictures of four + // lines are 3840; 2500 holds one of them. + var terminal = Fresh(o => o.MaxImageBytes = 2500); WriteSixel(terminal); - var first = ImageAssertions.ImageAt(terminal, 0, 0); - Assert.NotNull(first); + var oldest = terminal.Buffer.Lines[terminal.Buffer.YBase]!; + // Scroll the first picture off the screen into the scrollback, then draw a second. + terminal.Write($"{Esc}[10;1H" + string.Concat(Enumerable.Repeat("\r\n", 10))); + terminal.Write($"{Esc}[1;1H"); WriteSixel(terminal); - var second = ImageAssertions.ImageAt(terminal, 0, 4); - Assert.NotNull(second); - Assert.Null(ImageAssertions.ImageAt(terminal, 0, 0)); - Assert.True(ReferenceEquals(ImageAssertions.ImageAt(terminal, 0, 4), second), - "the newest image should be the one that survives"); + terminal.EnforceImageBudget(); + + // Just enough goes, oldest line first: three of the old picture's four lines take the + // buffer under budget, so the fourth -- the newest of them -- stays. + Assert.Null(oldest.Sixel); + Assert.Equal(8 + 2, SixelCells(terminal)); + for (int row = 0; row < 4; row++) + Assert.True(ImageAssertions.IsSixelAt(terminal, 0, row), $"the picture on screen should survive (row {row})"); } [Fact] @@ -326,80 +379,53 @@ public void A_generous_budget_keeps_both_images() WriteSixel(terminal); WriteSixel(terminal); + terminal.EnforceImageBudget(); - Assert.NotNull(ImageAssertions.ImageAt(terminal, 0, 0)); - Assert.NotNull(ImageAssertions.ImageAt(terminal, 0, 4)); + Assert.True(ImageAssertions.IsSixelAt(terminal, 0, 0)); + Assert.True(ImageAssertions.IsSixelAt(terminal, 0, 4)); } /// - /// Evicting one picture leaves another that shares its line. + /// What is on screen is never taken: at most one screen of pixels, and it is what the user is + /// looking at. /// - /// - /// Caught in review. Dropping the whole line because one of its pictures was over budget took - /// the others with it — more destructive than the per-cell code this replaced, and it evicted - /// images the sweep had just decided to keep. - /// [Fact] - public void Evicting_one_image_leaves_another_on_the_same_line() + public void The_budget_never_takes_what_is_on_screen() { - // Two 192-byte images; a 400 byte budget holds both, then a third forces one out. - var terminal = Fresh(o => o.MaxImageBytes = 400); + var terminal = Fresh(o => o.MaxImageBytes = 1); WriteSixel(terminal); - terminal.Write($"{Esc}[1;5H"); - WriteSixel(terminal); - - var line = terminal.Buffer.Lines[terminal.Buffer.YBase]!; - Assert.Equal(2, line.Images.Count); - var newer = line.Images[1]; - - // A third image pushes past the budget, dooming the oldest — which shares this line. - terminal.Write($"{Esc}[10;1H"); WriteSixel(terminal); + terminal.EnforceImageBudget(); - Assert.True(ReferenceEquals(newer, line.Images.SingleOrDefault()), - "the line's other picture should survive its neighbour being evicted"); + Assert.Equal(16, SixelCells(terminal)); } /// - /// Printing over one picture releases it, while another on the same line stays. + /// Printing over one picture clears its pixels and leaves another on the same line alone. /// - /// - /// Caught in review. Ownership is derived from the runs, so anything that removes one has to - /// rebuild it — waiting for the run list to empty kept a picture alive that nothing displayed, - /// and hid it from the budget sweep, which decides what is live by walking runs. - /// [Fact] - public void Overwriting_one_picture_releases_only_that_one() + public void Overwriting_one_picture_leaves_another_on_the_same_line() { var terminal = Fresh(); WriteSixel(terminal); terminal.Write($"{Esc}[1;5H"); - WriteSixel(terminal); - - var line = terminal.Buffer.Lines[terminal.Buffer.YBase]!; - Assert.Equal(2, line.Images.Count); - var survivor = line.Images[1]; + WriteSixel(terminal, BlueTwoByFour); // Print across the whole span of the first picture, which covers columns 0..1. terminal.Write($"{Esc}[1;1HXX"); - Assert.True(ReferenceEquals(survivor, line.Images.SingleOrDefault()), - "the overwritten picture should be released and the other kept"); + Assert.False(ImageAssertions.IsSixelAt(terminal, 0, 0)); + Assert.False(ImageAssertions.IsSixelAt(terminal, 1, 0)); + Assert.Equal(Blue, ImageAssertions.SixelPixel(terminal, 4, 0)); + Assert.Equal(Blue, ImageAssertions.SixelPixel(terminal, 5, 0)); } /// - /// A cell under a picture is an ordinary space, and that is now correct rather than a bug. + /// A cell under a picture is an ordinary space: nothing about a picture is drawn from cells, so + /// cells beneath one coalesce exactly like the spaces they are. /// - /// - /// This inverts what these tests used to assert. When a cell carried an image reference and a - /// tile coordinate, two cells showing different pieces of a picture HAD to compare unequal, - /// because renderers coalesce adjacent cells into one run by comparing them and merging two - /// different tiles would have drawn the wrong thing. With pictures held as runs on the line, - /// nothing about a picture is drawn from cells — so cells beneath one should coalesce exactly - /// like the spaces they are, and the distinction has no work left to do. - /// [Fact] public void A_cell_under_a_picture_is_an_ordinary_space() { @@ -409,25 +435,20 @@ public void A_cell_under_a_picture_is_an_ordinary_space() Assert.Equal(BufferCell.Space, Cell(terminal, 0, 0)); Assert.Equal(Cell(terminal, 0, 0), Cell(terminal, 1, 0)); - // And the picture is still there — held by the line, which is the whole point. - Assert.NotNull(ImageAssertions.ImageAt(terminal, 0, 0)); + // And the picture is still there -- held by the line, which is the whole point. + Assert.True(ImageAssertions.IsSixelAt(terminal, 0, 0)); } - /// - /// What must stay distinguishable is the RUNS: each line shows its own slice of the picture. - /// + /// Each line shows its own band of the picture. [Fact] - public void Runs_on_different_lines_carry_different_slices() + public void Lines_carry_different_bands_of_the_picture() { var terminal = Fresh(); - WriteSixel(terminal); + WriteSixel(terminal, Striped); - var first = ImageAssertions.PlacementAt(terminal, 0, 0); - var second = ImageAssertions.PlacementAt(terminal, 0, 1); - - Assert.NotNull(first); - Assert.NotNull(second); - Assert.NotEqual(first!.Value.SrcY, second!.Value.SrcY); + Assert.Equal(Red, ImageAssertions.SixelPixel(terminal, 0, 0)); + Assert.Equal(Green, ImageAssertions.SixelPixel(terminal, 0, 1)); + Assert.Equal(Blue, ImageAssertions.SixelPixel(terminal, 0, 2)); } [Fact] @@ -437,10 +458,10 @@ public void The_alternate_buffer_keeps_its_own_images() WriteSixel(terminal); terminal.Write($"{Esc}[?1049h"); // to the alternate screen - Assert.Equal(0, ImageCellCount(terminal)); + Assert.Equal(0, SixelCells(terminal)); terminal.Write($"{Esc}[?1049l"); // and back - Assert.Equal(8, ImageCellCount(terminal)); + Assert.Equal(8, SixelCells(terminal)); } /// @@ -449,45 +470,38 @@ public void The_alternate_buffer_keeps_its_own_images() /// /// /// DECCARA sets attributes over an area and never touches a character, so a picture inside that - /// area is not being overwritten and must not be split. Going through the ordinary text-write - /// path made it look like one -- that path splits the placement at the column being written, - /// which is right for printing over a picture and wrong for recolouring the cell under it. + /// area is not being overwritten and must not be cleared. /// [Fact] public void An_attribute_change_over_a_picture_leaves_it_whole() { var terminal = Fresh(); WriteSixel(terminal); - var image = ImageAssertions.ImageAt(terminal, 0, 0); - Assert.NotNull(image); - Assert.Equal(8, ImageCellCount(terminal)); + Assert.Equal(8, SixelCells(terminal)); // Bold over the whole picture and then some. terminal.Write($"{Esc}[2*x{Esc}[1;1;4;4;1$r"); - Assert.Equal(8, ImageCellCount(terminal)); - Assert.True(ReferenceEquals(ImageAssertions.ImageAt(terminal, 0, 0), image)); - Assert.True(ReferenceEquals(ImageAssertions.ImageAt(terminal, 1, 3), image)); + Assert.Equal(8, SixelCells(terminal)); + Assert.Equal(Red, ImageAssertions.SixelPixel(terminal, 1, 3)); Assert.True(terminal.Buffer.Lines[terminal.Buffer.YBase]![0].Attributes.IsBold()); } /// /// And a request that changes nothing must not even write the cells back, because writing a - /// cell back unchanged is still a write as far as the placements are concerned. + /// cell back unchanged is still a write as far as the pixels are concerned. /// [Fact] public void An_attribute_change_naming_nothing_we_implement_touches_no_cell() { var terminal = Fresh(); WriteSixel(terminal); - var image = ImageAssertions.ImageAt(terminal, 0, 0); - Assert.Equal(8, ImageCellCount(terminal)); + Assert.Equal(8, SixelCells(terminal)); // 31 is a colour, which DECCARA does not carry; the request names nothing this implements. terminal.Write($"{Esc}[2*x{Esc}[1;1;4;4;31$r"); - Assert.Equal(8, ImageCellCount(terminal)); - Assert.True(ReferenceEquals(ImageAssertions.ImageAt(terminal, 0, 0), image)); + Assert.Equal(8, SixelCells(terminal)); } /// A DECRARA whose toggles cancel each other is the same nothing. @@ -496,11 +510,56 @@ public void Toggles_that_cancel_leave_the_picture_and_the_rendition_alone() { var terminal = Fresh(); WriteSixel(terminal); - Assert.Equal(8, ImageCellCount(terminal)); + Assert.Equal(8, SixelCells(terminal)); terminal.Write($"{Esc}[2*x{Esc}[1;1;4;4;1;1$t"); - Assert.Equal(8, ImageCellCount(terminal)); + Assert.Equal(8, SixelCells(terminal)); Assert.False(terminal.Buffer.Lines[terminal.Buffer.YBase]![0].Attributes.IsBold()); } + + /// + /// A copy of a line shares its pixels until one of them changes, and then each keeps its own: + /// a frame captured for a renderer must not see the live line being painted. + /// + [Fact] + public void A_copy_of_a_line_keeps_its_pixels_when_the_original_is_painted() + { + var terminal = Fresh(); + WriteSixel(terminal); + var line = Line(terminal, 0); + + var clone = line.Clone(); + Assert.Same(line.Sixel, clone.Sixel); + + terminal.Write($"{Esc}[1;1H"); + WriteSixel(terminal, BlueTwoByFour); + + Assert.NotSame(line.Sixel, clone.Sixel); + var pixels = clone.Sixel!.Pixels.Span; + Assert.Equal((byte)255, pixels[2]); // still red: B, G, R, A + Assert.Equal((byte)0, pixels[0]); + } + + /// + /// A Sixel laid out against a new cell size rescales what the line already shows, so every pixel + /// in a strip agrees on how big a cell is. + /// + [Fact] + public void A_sixel_at_a_new_cell_size_rescales_the_strip() + { + var terminal = Fresh(); + WriteSixel(terminal); + + terminal.Options.CellWidthPixels = 4; + terminal.Options.CellHeightPixels = 6; + terminal.Write($"{Esc}[1;5H"); + WriteSixel(terminal, "#0;2;0;0;100!4~"); + + var strip = ImageAssertions.StripOn(terminal, 0)!; + Assert.Equal(4, strip.CellWidth); + Assert.Equal(6, strip.CellHeight); + Assert.Equal(Red, ImageAssertions.SixelPixel(terminal, 0, 0, x: 3, y: 5)); + Assert.Equal(Blue, ImageAssertions.SixelPixel(terminal, 4, 0)); + } } diff --git a/src/XTerm.NET.Tests/Graphics/SixelDecoderTests.cs b/src/XTerm.NET.Tests/Graphics/SixelDecoderTests.cs index 4d16db1..aa9bd8f 100644 --- a/src/XTerm.NET.Tests/Graphics/SixelDecoderTests.cs +++ b/src/XTerm.NET.Tests/Graphics/SixelDecoderTests.cs @@ -23,13 +23,26 @@ public class SixelDecoderTests /// Background select 0: pixels left unset take the terminal background. private const int OpaqueBackground = 0; + private static readonly System.Runtime.CompilerServices.ConditionalWeakTable> Seen = new(); + private static Terminal Fresh(Action? configure = null) { var options = new TerminalOptions { Cols = 40, Rows = 10 }; configure?.Invoke(options); - return new Terminal(options); + var terminal = new Terminal(options); + + // The buffer keeps a Sixel's pixels, not the picture, so the decoded picture is caught as + // it is placed. + var seen = new List(); + terminal.SixelPlaced += seen.Add; + Seen.Add(terminal, seen); + return terminal; } + /// The last picture a Sixel placed, or null if none reached the buffer. + private static TerminalImage? Placed(Terminal terminal) + => Seen.TryGetValue(terminal, out var seen) && seen.Count > 0 ? seen[^1] : null; + private static void WriteSixel(Terminal terminal, string body, int backgroundSelect = Transparent) => terminal.Write($"{Esc}P0;{backgroundSelect};0q{body}{St}"); @@ -38,7 +51,7 @@ private static void WriteSixel(Terminal terminal, string body, int backgroundSel { var terminal = Fresh(configure); WriteSixel(terminal, body, backgroundSelect); - return (terminal.Buffer.Lines[0]!.TryGetImageAt(0, out var __i1) ? __i1 : null); + return Placed(terminal); } private static TerminalImage Decode(string body, int backgroundSelect = Transparent, @@ -232,7 +245,7 @@ public void An_abandoned_payload_produces_no_image() // CAN mid-payload: the sequence is dropped rather than terminated. terminal.Write($"{Esc}P0;1;0q#0;2;100;0;0!20~\u0018"); - Assert.Null((terminal.Buffer.Lines[0]!.TryGetImageAt(0, out var __i2) ? __i2 : null)); + Assert.Null(Placed(terminal)); } [Fact] @@ -241,7 +254,7 @@ public void An_empty_payload_produces_no_image() var terminal = Fresh(); terminal.Write($"{Esc}P0;1;0q{St}"); - Assert.Null((terminal.Buffer.Lines[0]!.TryGetImageAt(0, out var __i3) ? __i3 : null)); + Assert.Null(Placed(terminal)); } /// @@ -283,8 +296,8 @@ public void A_payload_split_across_writes_decodes_the_same() split.Write("0;0;0!4~-#1;2;0"); split.Write($";0;100!4~{St}"); - var a = whole.Buffer.Lines[0]!.TryGetImageAt(0, out var wholeImage) ? wholeImage : null; - var b = split.Buffer.Lines[0]!.TryGetImageAt(0, out var splitImage) ? splitImage : null; + var a = Placed(whole); + var b = Placed(split); Assert.NotNull(a); Assert.NotNull(b); @@ -300,7 +313,7 @@ public void Sixel_can_be_switched_off_entirely() var terminal = Fresh(o => o.SixelEnabled = false); WriteSixel(terminal, "#0;2;100;0;0~"); - Assert.Null((terminal.Buffer.Lines[0]!.TryGetImageAt(0, out var __i4) ? __i4 : null)); + Assert.Null(Placed(terminal)); } [Fact] diff --git a/src/XTerm.NET.Tests/Graphics/SixelPlacementTests.cs b/src/XTerm.NET.Tests/Graphics/SixelPlacementTests.cs index 18cd636..657c8ea 100644 --- a/src/XTerm.NET.Tests/Graphics/SixelPlacementTests.cs +++ b/src/XTerm.NET.Tests/Graphics/SixelPlacementTests.cs @@ -19,6 +19,22 @@ public class SixelPlacementTests /// Four pixels wide, twelve tall: two cells across, four down. private const string TwoByFourCells = "#0;2;100;0;0!4~-!4~"; + /// + /// The same size, with each cell row its own colour -- red, green, blue, yellow from the top -- + /// so the colour a screen row shows says which band of the picture landed there. 'F' sets a + /// sixel's top three pixels and 'w' its bottom three; a cell is three pixels tall. + /// + private const string Striped = "#0;2;100;0;0!4F$#1;2;0;100;0!4w-#2;2;0;0;100!4F$#3;2;100;100;0!4w"; + + private static readonly (byte R, byte G, byte B, byte A)[] Stripes = + { + (255, 0, 0, 255), (0, 255, 0, 255), (0, 0, 255, 255), (255, 255, 0, 255), + }; + + /// Which stripe of a screen position shows, or -1 for none. + private static int StripeOn(Terminal terminal, int col, int screenRow) + => ImageAssertions.SixelPixel(terminal, col, screenRow) is { } pixel ? Array.IndexOf(Stripes, pixel) : -1; + private static Terminal Fresh(int rows = 10, Action? configure = null) { var options = new TerminalOptions @@ -39,30 +55,21 @@ private static BufferCell Cell(Terminal terminal, int col, int screenRow) => terminal.Buffer.Lines[terminal.Buffer.YBase + screenRow]![col]; [Fact] - public void An_image_covers_one_cell_per_tile() + public void An_image_paints_each_cell_it_covers_with_its_own_band() { var terminal = Fresh(); - WriteSixel(terminal); - - var image = ImageAssertions.ImageAt(terminal, 0, 0); - Assert.NotNull(image); - Assert.Equal(2, image!.Cols); - Assert.Equal(4, image.Rows); + WriteSixel(terminal, Striped); - // One run per line, each covering the picture's full width and carrying its own slice of - // the source — which is what the per-cell tile grid said, expressed once per row instead of - // once per cell. + // Two columns by four rows, each row showing the band of the picture that belongs there, + // and every line's strip at the cell size the picture was laid out against. for (int row = 0; row < 4; row++) { - var placement = ImageAssertions.PlacementAt(terminal, 0, row); - Assert.NotNull(placement); - Assert.True(ReferenceEquals(ImageAssertions.ImageAt(terminal, 0, row), image), - $"row {row} should show part of the image"); - - Assert.Equal(0, placement!.Value.Column); - Assert.Equal(2, placement.Value.Cols); - Assert.Equal(0, placement.Value.SrcX); - Assert.Equal(row * terminal.Options.CellHeightPixels, placement.Value.SrcY); + for (int col = 0; col < 2; col++) + Assert.Equal(row, StripeOn(terminal, col, row)); + + var strip = ImageAssertions.StripOn(terminal, row)!; + Assert.Equal(terminal.Options.CellWidthPixels, strip.CellWidth); + Assert.Equal(terminal.Options.CellHeightPixels, strip.CellHeight); } } @@ -72,8 +79,9 @@ public void Cells_beyond_the_image_are_left_alone() var terminal = Fresh(); WriteSixel(terminal); - Assert.Null(ImageAssertions.ImageAt(terminal, 2, 0)); - Assert.Null(ImageAssertions.ImageAt(terminal, 0, 4)); + Assert.False(ImageAssertions.IsSixelAt(terminal, 2, 0)); + Assert.False(ImageAssertions.IsSixelAt(terminal, 0, 4)); + Assert.Null(ImageAssertions.StripOn(terminal, 4)); } [Fact] @@ -83,9 +91,9 @@ public void An_image_starts_at_the_cursor() terminal.Write($"{Esc}[3;6H"); // row 3, column 6, one-based WriteSixel(terminal); - Assert.True(ReferenceEquals(ImageAssertions.ImageAt(terminal, 5, 2), ImageAssertions.ImageAt(terminal, 6, 2))); - Assert.NotNull(ImageAssertions.ImageAt(terminal, 5, 2)); - Assert.Null(ImageAssertions.ImageAt(terminal, 4, 2)); + Assert.True(ImageAssertions.IsSixelAt(terminal, 5, 2)); + Assert.True(ImageAssertions.IsSixelAt(terminal, 6, 2)); + Assert.False(ImageAssertions.IsSixelAt(terminal, 4, 2)); } [Fact] @@ -133,8 +141,8 @@ public void Decsdm_pins_the_image_to_the_top_left_and_leaves_the_cursor_alone() terminal.Write($"{Esc}[3;6H"); WriteSixel(terminal); - Assert.NotNull(ImageAssertions.ImageAt(terminal, 0, 0)); - Assert.Null(ImageAssertions.ImageAt(terminal, 5, 2)); + Assert.True(ImageAssertions.IsSixelAt(terminal, 0, 0)); + Assert.False(ImageAssertions.IsSixelAt(terminal, 5, 2)); Assert.Equal(5, terminal.Buffer.X); Assert.Equal(2, terminal.Buffer.Y); } @@ -148,7 +156,7 @@ public void Decsdm_reset_restores_the_scrolling_behaviour() terminal.Write($"{Esc}[3;6H"); WriteSixel(terminal); - Assert.NotNull(ImageAssertions.ImageAt(terminal, 5, 2)); + Assert.True(ImageAssertions.IsSixelAt(terminal, 5, 2)); } /// @@ -159,15 +167,11 @@ public void An_image_that_runs_past_the_bottom_scrolls_the_screen() { var terminal = Fresh(rows: 5); terminal.Write($"{Esc}[4;1H"); // last-but-one row - WriteSixel(terminal); + WriteSixel(terminal, Striped); // Four image rows plus the cursor's own row need five: the screen scrolled until they fit. for (int row = 0; row < 4; row++) - { - var placement = ImageAssertions.PlacementAt(terminal, 0, row); - Assert.NotNull(placement); - Assert.Equal(row * terminal.Options.CellHeightPixels, placement!.Value.SrcY); - } + Assert.Equal(row, StripeOn(terminal, 0, row)); Assert.Equal(0, terminal.Buffer.X); Assert.Equal(4, terminal.Buffer.Y); @@ -177,12 +181,12 @@ public void An_image_that_runs_past_the_bottom_scrolls_the_screen() public void An_image_taller_than_the_screen_keeps_its_last_rows() { var terminal = Fresh(rows: 3); - WriteSixel(terminal); + WriteSixel(terminal, Striped); // Three rows of screen, four of image, and the cursor still needs a row of its own below // it -- so the picture scrolled up until its last two rows and the cursor fit. - Assert.Equal(2 * terminal.Options.CellHeightPixels, ImageAssertions.PlacementAt(terminal, 0, 0)!.Value.SrcY); - Assert.Equal(3 * terminal.Options.CellHeightPixels, ImageAssertions.PlacementAt(terminal, 0, 1)!.Value.SrcY); + Assert.Equal(2, StripeOn(terminal, 0, 0)); + Assert.Equal(3, StripeOn(terminal, 0, 1)); Assert.Equal(2, terminal.Buffer.Y); } @@ -191,11 +195,11 @@ public void Decsdm_clips_a_tall_image_instead_of_scrolling() { var terminal = Fresh(rows: 3); terminal.Write($"{Esc}[?80h"); - WriteSixel(terminal); + WriteSixel(terminal, Striped); // Pinned at the top, so the first rows are the ones that survive. - Assert.Equal(0 * terminal.Options.CellHeightPixels, ImageAssertions.PlacementAt(terminal, 0, 0)!.Value.SrcY); - Assert.Equal(2 * terminal.Options.CellHeightPixels, ImageAssertions.PlacementAt(terminal, 0, 2)!.Value.SrcY); + Assert.Equal(0, StripeOn(terminal, 0, 0)); + Assert.Equal(2, StripeOn(terminal, 0, 2)); } [Fact] @@ -203,10 +207,13 @@ public void An_image_is_clipped_at_the_right_edge() { var terminal = Fresh(configure: o => o.Cols = 6); terminal.Write($"{Esc}[1;6H"); // one column from the right edge - WriteSixel(terminal); + WriteSixel(terminal, Striped); - Assert.NotNull(ImageAssertions.ImageAt(terminal, 5, 0)); - Assert.Equal(0 * terminal.Options.CellWidthPixels, ImageAssertions.PlacementAt(terminal, 5, 0)!.Value.SrcX); + // The picture's first column lands in the last cell; the rest runs past the edge, kept at + // its natural width for a wider window but not shown on this one. + Assert.True(ImageAssertions.IsSixelAt(terminal, 5, 0)); + Assert.Equal(0, StripeOn(terminal, 5, 0)); + Assert.False(ImageAssertions.IsSixelAt(terminal, 6, 0)); } /// @@ -243,18 +250,18 @@ public void Placing_an_image_drops_the_lines_render_cache() } [Fact] - public void Two_images_do_not_share_tiles() + public void A_second_image_paints_below_the_first_and_leaves_it_alone() { var terminal = Fresh(); - WriteSixel(terminal); - WriteSixel(terminal); - - var first = ImageAssertions.ImageAt(terminal, 0, 0); - var second = ImageAssertions.ImageAt(terminal, 0, 4); + WriteSixel(terminal, Striped); + WriteSixel(terminal, Striped); - Assert.NotNull(first); - Assert.NotNull(second); - Assert.False(ReferenceEquals(first, second)); - Assert.NotEqual(first!.Id, second!.Id); + // The first picture's rows are untouched by the second, which starts where the cursor + // was left: the row below the first. + for (int row = 0; row < 4; row++) + { + Assert.Equal(row, StripeOn(terminal, 0, row)); + Assert.Equal(row, StripeOn(terminal, 0, row + 4)); + } } } diff --git a/src/XTerm.NET.Tests/ITerm2OscTests.cs b/src/XTerm.NET.Tests/ITerm2OscTests.cs index a92b564..533fc9f 100644 --- a/src/XTerm.NET.Tests/ITerm2OscTests.cs +++ b/src/XTerm.NET.Tests/ITerm2OscTests.cs @@ -21,12 +21,16 @@ public class ITerm2OscTests public void File_InlinePng_IsDecodedAndPlaced() { var terminal = CreateTerminal(); + XTerm.Graphics.TerminalImage? image = null; + terminal.SixelPlaced += placed => image = placed; + terminal.Write(Osc($"File=inline=1:{Png}")); - var image = ImageAssertions.ImageAt(terminal, 0, 0); + // An inline image is content, as a Sixel is: decoded, then painted into the line. Assert.NotNull(image); Assert.Equal(5, image!.PixelWidth); Assert.Equal(3, image.PixelHeight); + Assert.True(ImageAssertions.IsSixelAt(terminal, 0, 0)); } [Fact] diff --git a/src/XTerm.NET.Tests/XtGetTcapTests.cs b/src/XTerm.NET.Tests/XtGetTcapTests.cs index 22ab064..66cb82d 100644 --- a/src/XTerm.NET.Tests/XtGetTcapTests.cs +++ b/src/XTerm.NET.Tests/XtGetTcapTests.cs @@ -269,7 +269,7 @@ public void A_sixel_payload_is_not_mistaken_for_a_capability_request() terminal.Write($"{Esc}P0;1;0q{SixelImage}{St}"); Assert.Empty(replies); - Assert.NotNull(ImageAssertions.ImageAt(terminal, 0, 0)); + Assert.True(ImageAssertions.IsSixelAt(terminal, 0, 0)); } [Fact] @@ -292,6 +292,6 @@ public void An_image_after_a_capability_request_is_still_decoded() terminal.Write($"{Esc}P0;1;0q{SixelImage}{St}"); - Assert.NotNull(ImageAssertions.ImageAt(terminal, 0, 0)); + Assert.True(ImageAssertions.IsSixelAt(terminal, 0, 0)); } } diff --git a/src/XTerm.NET/Buffer/BufferLine.cs b/src/XTerm.NET/Buffer/BufferLine.cs index 26797b4..bc4a051 100644 --- a/src/XTerm.NET/Buffer/BufferLine.cs +++ b/src/XTerm.NET/Buffer/BufferLine.cs @@ -47,6 +47,22 @@ public class BufferLine : IEnumerable /// with no eviction pass and nothing to keep in step with a buffer that scrolls. /// private List? _images; + + /// + /// The Sixel pixels this line shows, or null -- which is every line no Sixel has landed on. + /// + /// + /// Sixel is kept as the pixels it painted rather than as pictures: see + /// . Kitty pictures are overlays with identities and z-order and + /// stay as runs in . + /// + private Graphics.SixelStrip? _sixel; + + /// + /// Whether is shared with a copy of this line, so the next write must take a + /// private copy first. Set on both sides by and . + /// + private bool _sixelShared; private int _length; private bool _isWrapped; private LineAttribute _lineAttribute; @@ -179,6 +195,10 @@ public void SetCell(int index, ref BufferCell cell) if (_placements is not null) SplitPlacementsAt(index); + // And the Sixel pixels in that cell, the same rule. + if (_sixel is not null) + ClearSixel(index, 1); + Cache = null; } } @@ -214,6 +234,8 @@ public void SetSingleWidthRun(int index, ReadOnlySpan text, AttributeData // One field test on a line with no pictures, which is nearly every line. if (_placements is not null) SplitPlacementsOver(index, text.Length); + if (_sixel is not null) + ClearSixel(index, text.Length); Cache = null; } @@ -240,6 +262,8 @@ public void SetSingleWidthRun(int index, ReadOnlySpan ascii, AttributeData // treatment SetCell gives them. if (_placements is not null) SplitPlacementsOver(index, ascii.Length); + if (_sixel is not null) + ClearSixel(index, ascii.Length); Cache = null; } @@ -324,6 +348,8 @@ public void Fill(BufferCell fillCell, int startCol = 0, int endCol = -1) if (_placements is not null) SplitPlacementsOver(startCol, Math.Max(0, Math.Min(endCol, _length) - startCol), includeOverlays: true); + if (_sixel is not null) + ClearSixel(startCol, Math.Max(0, Math.Min(endCol, _length) - startCol)); // And a LINK, unlike a mark. A mark records a position in the history and survives the // shell redrawing its prompt; a link is a property of its text, and text that has been @@ -381,9 +407,33 @@ public void CopyCellsFrom(BufferLine src, int srcCol, int destCol, int length, b } } } + + // The Sixel pixels follow their cells: an insert, a delete or a margin scroll that moved the + // text moved the picture it sits in. Kitty runs are anchored overlays and stay put. + if (src._sixel is not null || _sixel is not null) + CopySixelCells(src, srcCol, destCol, length); + Cache = null; } + private void CopySixelCells(BufferLine src, int srcCol, int destCol, int length) + { + var count = Math.Min(length, Math.Min(_length - destCol, src._length - srcCol)); + if (count <= 0 || destCol < 0 || srcCol < 0) + return; + + // Blanks onto a line with no pixels there: nothing to do, and nothing to allocate. + if (src._sixel is null && !SixelPaintedIn(destCol, count)) + return; + + var strip = WritableSixel(src._sixel); + var source = ReferenceEquals(src, this) ? strip : src._sixel; + strip.CopyCells(source, srcCol, destCol, count); + + if (strip.IsEmpty) + DropSixel(); + } + /// /// Translates the line to a string. /// @@ -454,21 +504,101 @@ public void ReplaceCells(int startCol, int endCol, BufferCell fillCell) /// True if the line held any, which is also the signal that it needs repainting. public bool ClearImages() { - if (_placements is null && _images is null) + if (_placements is null && _images is null && _sixel is null) return false; // Nothing to clean up in the cells — they never held anything. Releasing the strong // references is what actually frees the pixels. _placements = null; _images = null; + _sixel = null; + _sixelShared = false; Cache = null; return true; } /// - /// Whether this line shows any part of a picture. One field test — a renderer can ask per row. + /// Whether this line shows any part of a picture -- a Kitty run or Sixel pixels. One field test + /// each, so a renderer can ask per row. + /// + public bool HasImages => _placements is { Count: > 0 } || _sixel is not null; + + /// The Sixel pixels this line shows, or null. See . + public Graphics.SixelStrip? Sixel => _sixel; + + /// + /// Paints one cell-row of a Sixel (or an inline image) into this line, from + /// on. The band of the picture painted starts at pixel row + /// . + /// + internal void PaintSixel(Graphics.TerminalImage image, int sourceY, int column) + { + var strip = WritableSixel(null, image.CellWidth, image.CellHeight); + strip.Paint(image, sourceY, column); + Cache = null; + } + + /// Drops this line's Sixel pixels altogether. What the image budget reclaims through. + /// The bytes released, or zero when there were none. + internal long DropSixel() + { + if (_sixel is null) + return 0; + + var bytes = _sixel.ByteCount; + _sixel = null; + _sixelShared = false; + Cache = null; + return bytes; + } + + /// Clears the Sixel pixels in a span of cells: something printed or erased there replaced them. + private void ClearSixel(int column, int count) + { + if (_sixel is null || count <= 0 || !SixelPaintedIn(column, count)) + return; + + var strip = WritableSixel(null); + strip.Clear(column, count); + if (strip.IsEmpty) + DropSixel(); + } + + private bool SixelPaintedIn(int column, int count) + { + if (_sixel is null) + return false; + + var end = Math.Min(_sixel.Columns, column + count); + for (var c = Math.Max(0, column); c < end; c++) + { + if (_sixel.HasPixelsAt(c)) + return true; + } + + return false; + } + + /// + /// This line's strip, made private if it was shared and created if there was none -- at the cell + /// size of , or the one given. /// - public bool HasImages => _placements is { Count: > 0 }; + private Graphics.SixelStrip WritableSixel(Graphics.SixelStrip? like, int cellWidth = 0, int cellHeight = 0) + { + if (_sixel is null) + { + _sixel = Graphics.SixelStrip.Create( + like?.CellWidth ?? cellWidth, like?.CellHeight ?? cellHeight, _length); + _sixelShared = false; + } + else if (_sixelShared) + { + _sixel = _sixel.Copy(); + _sixelShared = false; + } + + return _sixel; + } /// Whether this line carries any shell-integration mark. public bool HasMarks => _marks is { Count: > 0 }; @@ -1139,6 +1269,14 @@ public BufferLine Clone() newLine._placements = new List(_placements); newLine._images = _images is null ? null : new List(_images); } + + // Shared, not copied: whichever of the two writes first takes its own copy. + if (_sixel is not null) + { + newLine._sixel = _sixel; + newLine._sixelShared = true; + _sixelShared = true; + } for (int i = 0; i < _length; i++) { newLine._cells[i] = _cells[i]; @@ -1178,6 +1316,12 @@ public void CopyFrom(BufferLine line) ? null : new List(line._images); + // Shared with the source until one of them writes, as in Clone. + _sixel = line._sixel; + _sixelShared = _sixel is not null; + if (_sixel is not null) + line._sixelShared = true; + // Assigned, not OR-ed: this REPLACES the line's contents rather than adding to them, so // the latch belongs to the incoming cells. A recycled scrollback line that once held a // wide character would otherwise carry that latch forever. diff --git a/src/XTerm.NET/Graphics/SixelStrip.cs b/src/XTerm.NET/Graphics/SixelStrip.cs new file mode 100644 index 0000000..41947dd --- /dev/null +++ b/src/XTerm.NET/Graphics/SixelStrip.cs @@ -0,0 +1,327 @@ +namespace XTerm.Graphics; + +/// +/// The Sixel pixels one line shows: a strip one cell tall, painted by every Sixel that lands on the +/// line and erased cell by cell by whatever is printed or erased over it. +/// +/// +/// Pixels, not pictures. A Sixel is content: it paints the cells it lands on, a later +/// Sixel paints over it, and printing over a cell replaces that part of it. So the line keeps the +/// RESULT -- one strip of pixels -- rather than every picture that ever arrived and a record of which +/// slivers of each are still showing. A program that repaints small regions over and over (a cursor, +/// a hover, a diffing framebuffer) costs one strip per line however many Sixels it sends, where +/// keeping the pictures grew thousands of fragments per screen, every one a separate draw. +/// Layout. BGRA, straight alpha, top row first, exactly like +/// : cells of pixels +/// across and pixels down. A pixel nothing has painted is fully transparent, +/// so the cell beneath shows through it. is the widest anything has painted and +/// is never clipped to the line, so a terminal widened later shows more rather than having lost it -- +/// the renderer draws min(Columns, line width), as it does for a picture. +/// One scale per strip. A Sixel is laid out against the cell size in force when it +/// arrives. A strip painted at another size is resampled to the new one first, so every pixel in it +/// agrees on how big a cell is and a renderer maps it to the grid with a single scale. +/// Sharing. A copy of a line (, +/// ) shares its strip, and whichever line writes first takes a +/// private copy, so a frame captured for a renderer costs a reference rather than the pixels. Nothing +/// outside this assembly can write to a strip. +/// Change. is taken from one counter for every strip and moves +/// after each change is complete, so a renderer holding an upload keyed on it knows the upload is +/// current, and one that read the pixels while they were being written sees the version move and +/// uploads again. +/// +public sealed class SixelStrip +{ + private static long _versionCounter; + + private byte[] _pixels; + private bool[] _painted; + private int _paintedCount; + + private SixelStrip(int cellWidth, int cellHeight, int columns) + { + CellWidth = cellWidth; + CellHeight = cellHeight; + Columns = columns; + _pixels = new byte[columns * cellWidth * cellHeight * TerminalImage.BytesPerPixel]; + _painted = new bool[columns]; + Version = NextVersion(); + } + + /// Width of one cell in this strip's pixels. + public int CellWidth { get; private set; } + + /// Height of one cell, which is the strip's height, in pixels. + public int CellHeight { get; private set; } + + /// How many cells across the strip holds pixels for. + public int Columns { get; private set; } + + /// The strip's width in pixels. + public int PixelWidth => Columns * CellWidth; + + /// The strip's height in pixels: one cell. + public int PixelHeight => CellHeight; + + /// Bytes per row of . + public int Stride => PixelWidth * TerminalImage.BytesPerPixel; + + /// The pixels, BGRA with straight alpha, top row first. See the remarks on the class. + public ReadOnlyMemory Pixels => _pixels; + + /// Bytes the pixels occupy. + public long ByteCount => _pixels.LongLength; + + /// Changes after every change to the pixels, and is never reused by another strip. + public long Version { get; private set; } + + /// Whether anything has painted since it was last cleared. + public bool HasPixelsAt(int column) => column >= 0 && column < Columns && _painted[column]; + + /// + /// One past the rightmost painted cell: how much of the strip a renderer needs to draw. Zero when + /// nothing is painted. + /// + public int PaintedColumns + { + get + { + for (var c = Columns - 1; c >= 0; c--) + { + if (_painted[c]) + return c + 1; + } + + return 0; + } + } + + /// Whether every column is clear, in which case the line drops the strip. + internal bool IsEmpty => _paintedCount == 0; + + private static long NextVersion() => Interlocked.Increment(ref _versionCounter); + + /// A strip ready to be painted at the given cell size. + internal static SixelStrip Create(int cellWidth, int cellHeight, int columns) + => new(Math.Max(1, cellWidth), Math.Max(1, cellHeight), Math.Max(1, columns)); + + /// A private copy, for a line that shared this strip and is about to write to it. + internal SixelStrip Copy() + { + var copy = new SixelStrip(CellWidth, CellHeight, Columns); + System.Buffer.BlockCopy(_pixels, 0, copy._pixels, 0, _pixels.Length); + Array.Copy(_painted, copy._painted, _painted.Length); + copy._paintedCount = _paintedCount; + return copy; + } + + /// + /// Paints one cell-row of -- the band starting at pixel row + /// -- into this strip from on. + /// + /// + /// A fully transparent source pixel leaves the strip alone, which is Sixel's own rule: a + /// picture drawn with background select 1 lets whatever was there show through its unset pixels. + /// A fully opaque one replaces the strip's pixel. Anything between -- an inline PNG's soft edge -- + /// is blended over it. + /// The cells the picture covers are marked painted, the last one included even where the + /// picture ends partway through it: that cell shows part of the picture. + /// + internal void Paint(TerminalImage image, int sourceY, int column) + { + if (column < 0 || sourceY < 0 || sourceY >= image.PixelHeight) + return; + + if (image.CellWidth != CellWidth || image.CellHeight != CellHeight) + Resample(image.CellWidth, image.CellHeight); + + var imageCols = (image.PixelWidth + CellWidth - 1) / CellWidth; + EnsureColumns(column + imageCols); + + var rows = Math.Min(CellHeight, image.PixelHeight - sourceY); + var width = image.PixelWidth; + var source = image.CurrentPixels.Span; + var sourceStride = image.Stride; + var stride = Stride; + var destX = column * CellWidth * TerminalImage.BytesPerPixel; + + for (var y = 0; y < rows; y++) + { + var src = source.Slice((sourceY + y) * sourceStride, width * TerminalImage.BytesPerPixel); + var dst = _pixels.AsSpan(y * stride + destX, width * TerminalImage.BytesPerPixel); + + for (var x = 0; x < src.Length; x += TerminalImage.BytesPerPixel) + { + var alpha = src[x + 3]; + if (alpha == 0) + continue; + + if (alpha == 255 || dst[x + 3] == 0) + { + src.Slice(x, TerminalImage.BytesPerPixel).CopyTo(dst.Slice(x)); + continue; + } + + Blend(src.Slice(x, TerminalImage.BytesPerPixel), dst.Slice(x, TerminalImage.BytesPerPixel)); + } + } + + for (var c = column; c < column + imageCols; c++) + MarkPainted(c); + + Version = NextVersion(); + } + + /// Clears cells from : printing or erasing there replaced the picture. + /// Whether anything was cleared. + internal bool Clear(int column, int count) + { + var start = Math.Max(0, column); + var end = Math.Min(Columns, column + count); + var cleared = false; + + for (var c = start; c < end; c++) + { + if (!_painted[c]) + continue; + + ClearCell(c); + _painted[c] = false; + _paintedCount--; + cleared = true; + } + + if (cleared) + Version = NextVersion(); + return cleared; + } + + /// + /// Copies cells of (or blanks, when it is null) + /// to -- the pixels following their cells through an insert, a + /// delete or a margin scroll. Overlapping ranges of one strip copy as if through a buffer. + /// + internal void CopyCells(SixelStrip? source, int sourceColumn, int destColumn, int count) + { + if (count <= 0) + return; + + if (source is not null && (source.CellWidth != CellWidth || source.CellHeight != CellHeight)) + { + source = source.Copy(); + source.Resample(CellWidth, CellHeight); + } + + EnsureColumns(destColumn + count); + + // Through a scratch copy when the ranges are one strip's: simpler than choosing a direction + // per row, and a shift within a line is a keystroke's worth of work. + var from = ReferenceEquals(source, this) ? Copy() : source; + var cellBytes = CellWidth * TerminalImage.BytesPerPixel; + + for (var i = 0; i < count; i++) + { + var dest = destColumn + i; + var src = sourceColumn + i; + var has = from is not null && from.HasPixelsAt(src); + + if (!has) + { + if (_painted[dest]) + { + ClearCell(dest); + _painted[dest] = false; + _paintedCount--; + } + continue; + } + + for (var y = 0; y < CellHeight; y++) + System.Buffer.BlockCopy(from!._pixels, y * from.Stride + src * cellBytes, _pixels, y * Stride + dest * cellBytes, cellBytes); + + MarkPainted(dest); + } + + Version = NextVersion(); + } + + private void MarkPainted(int column) + { + if (_painted[column]) + return; + _painted[column] = true; + _paintedCount++; + } + + private void ClearCell(int column) + { + var cellBytes = CellWidth * TerminalImage.BytesPerPixel; + for (var y = 0; y < CellHeight; y++) + Array.Clear(_pixels, y * Stride + column * cellBytes, cellBytes); + } + + /// Widens the strip to hold at least cells; new cells are clear. + private void EnsureColumns(int columns) + { + if (columns <= Columns) + return; + + var oldStride = Stride; + var grown = new byte[columns * CellWidth * CellHeight * TerminalImage.BytesPerPixel]; + var newStride = columns * CellWidth * TerminalImage.BytesPerPixel; + for (var y = 0; y < CellHeight; y++) + System.Buffer.BlockCopy(_pixels, y * oldStride, grown, y * newStride, oldStride); + + Array.Resize(ref _painted, columns); + _pixels = grown; + Columns = columns; + } + + /// + /// Rescales every pixel to a new cell size, nearest neighbour: what was painted when cells were + /// another size keeps its place in the grid. Only reached when the cell size has changed. + /// + private void Resample(int cellWidth, int cellHeight) + { + cellWidth = Math.Max(1, cellWidth); + cellHeight = Math.Max(1, cellHeight); + + var oldWidth = PixelWidth; + var oldHeight = CellHeight; + var oldStride = Stride; + var width = Columns * cellWidth; + var stride = width * TerminalImage.BytesPerPixel; + var resampled = new byte[stride * cellHeight]; + + for (var y = 0; y < cellHeight; y++) + { + var sy = Math.Min(oldHeight - 1, y * oldHeight / cellHeight); + for (var x = 0; x < width; x++) + { + var sx = Math.Min(oldWidth - 1, x * oldWidth / width); + System.Buffer.BlockCopy(_pixels, sy * oldStride + sx * TerminalImage.BytesPerPixel, + resampled, y * stride + x * TerminalImage.BytesPerPixel, TerminalImage.BytesPerPixel); + } + } + + _pixels = resampled; + CellWidth = cellWidth; + CellHeight = cellHeight; + } + + /// Source over destination, both straight alpha. + private static void Blend(ReadOnlySpan src, Span dst) + { + var sa = src[3] / 255f; + var da = dst[3] / 255f; + var outA = sa + da * (1 - sa); + if (outA <= 0) + { + dst.Clear(); + return; + } + + for (var i = 0; i < 3; i++) + dst[i] = (byte)Math.Clamp((int)Math.Round((src[i] * sa + dst[i] * da * (1 - sa)) / outA), 0, 255); + dst[3] = (byte)Math.Clamp((int)Math.Round(outA * 255), 0, 255); + } +} diff --git a/src/XTerm.NET/InputHandler.KittyGraphics.cs b/src/XTerm.NET/InputHandler.KittyGraphics.cs index 35a6f1e..fccb114 100644 --- a/src/XTerm.NET/InputHandler.KittyGraphics.cs +++ b/src/XTerm.NET/InputHandler.KittyGraphics.cs @@ -989,19 +989,19 @@ private void ReplyToKitty(Graphics.KittyCommand command, Graphics.KittyError err } /// - /// Writes an image into the buffer as one run per line. + /// Writes an image into the buffer, a cell-row at a time: Sixel as pixels painted into each + /// line's strip, Kitty as one run per line. /// /// - /// A picture spanning several rows becomes several s, - /// one per line, each carrying its own slice of the source. The line owns the run and the image, - /// so scrolling, scrollback eviction and ownership all keep working without anything being - /// written into cells — and a resize does nothing to a picture at all. - /// Cells are not touched. That is the whole difference between the two protocols here: - /// Sixel is CONTENT, so printing over it replaces that part of the picture and - /// does it explicitly; Kitty is an OVERLAY the - /// z-index orders against the text, so a picture placed over a character hides it while it is - /// there and reveals it again when it is deleted. Blanking the cell would make that - /// irreversible. + /// The two protocols differ in what a picture IS. Sixel is CONTENT: it paints the cells it + /// lands on, a later Sixel paints over it, and printing over a cell replaces that part -- so each + /// line keeps the pixels themselves, in a , and the picture that + /// delivered them is not kept at all. Kitty is an OVERLAY with an identity and a z-index: a + /// picture placed over a character hides it while it is there and reveals it again when it is + /// deleted, so it is kept as several s, one per line, each + /// carrying its own slice of the source. + /// Either way the line owns what it shows, so scrolling, scrollback eviction and ownership + /// keep working without anything written into cells, and a resize does nothing to a picture. /// private void PlaceImage(Graphics.ImagePlacement placement, Graphics.PlacementKind kind, @@ -1019,6 +1019,9 @@ private void PlaceImage(Graphics.ImagePlacement placement, var lastRowDrawn = row; + if (kind == Graphics.PlacementKind.Sixel) + _terminal.NoteSixelPlaced(placement.Image); + // One serial for the placement, shared by every row of it, so a delete aimed at any one // cell can find and remove the whole picture. var serial = Graphics.LinePlacement.NextSerial(); @@ -1040,14 +1043,21 @@ private void PlaceImage(Graphics.ImagePlacement placement, if (line is null) break; - // Sixel is content, so a picture landing on cells that already show one replaces that - // part of it, exactly as printing a character there does. Without this every Sixel a - // program draws over an earlier one is kept beneath it: a line in the alternate screen - // never dies, so an application repainting small regions (a cursor, a hover highlight) - // grew an unbounded stack of runs per line, each drawn every frame and each pinning its - // picture in memory. Kitty placements are overlays ordered by z-index and are left alone. + // Sixel is content: its pixels are painted into the line, over whatever Sixel was there, + // and nothing about the picture is kept. Keeping pictures and splitting the ones + // underneath grew thousands of fragments per screen under a program that repaints small + // regions, every one of them its own draw. if (kind == Graphics.PlacementKind.Sixel) - line.SplitPlacementsOver(startCol, placement.Cols); + { + if (placement.TryGetTileLayout(0, tileRow, out _, out var sixelY, out _, out _, + out _, out _, out _, out _)) + line.PaintSixel(placement.Image, sixelY, startCol); + + lastRowDrawn = row; + if (tileRow < placement.Rows - 1) + row++; + continue; + } // One run per line. Cols is the picture's NATURAL width — deliberately NOT clipped to // the terminal, so a window widened later reveals more of the picture rather than diff --git a/src/XTerm.NET/Terminal.cs b/src/XTerm.NET/Terminal.cs index feda20e..d88aca1 100644 --- a/src/XTerm.NET/Terminal.cs +++ b/src/XTerm.NET/Terminal.cs @@ -1367,6 +1367,18 @@ public void ScrollLines(int lines) /// picture. What that trades away is exactness: the buffer can sit up to one budget over before the /// sweep, which is a ceiling on overshoot rather than an unbounded one. /// + /// + /// Raised with each Sixel or inline image as it is painted into the buffer. + /// + /// + /// The buffer keeps what a Sixel painted, not the picture that painted it (see + /// ), so this is the one place the decoded picture can be seen + /// whole -- which is what the decoder's tests need to see. + /// + internal event Action? SixelPlaced; + + internal void NoteSixelPlaced(Graphics.TerminalImage image) => SixelPlaced?.Invoke(image); + internal void NoteImagePlaced(Graphics.TerminalImage image) { if (Options.MaxImageBytes <= 0) @@ -1584,13 +1596,20 @@ internal void EnforceImageBudget() return; var live = CollectLiveImages(); - long total = 0; + long total = SixelBytes(_normalBuffer) + SixelBytes(_altBuffer); foreach (var image in live) total += image.ByteCount; if (total <= budget) return; + // Sixel pixels in the scrollback go first, oldest line first: they are the history furthest + // from the screen, and a strip has no picture behind it to keep. What is on either screen is + // never taken here -- it is at most one screen of pixels. + total -= DropScrollbackSixels(_normalBuffer, total - budget); + if (total <= budget) + return; + var doomed = new HashSet(); foreach (var image in live.OrderBy(i => i.Id)) { @@ -1607,6 +1626,36 @@ internal void EnforceImageBudget() DropImages(_altBuffer, doomed); } + /// The bytes of Sixel pixels a buffer holds, scrollback and screen. + private static long SixelBytes(Buffer.TerminalBuffer? buffer) + { + if (buffer is null) + return 0; + + long bytes = 0; + for (int i = 0; i < buffer.Lines.Length; i++) + { + if (buffer.Lines[i]?.Sixel is { } strip) + bytes += strip.ByteCount; + } + + return bytes; + } + + /// Drops scrollback Sixel strips, oldest first, until bytes are freed. + /// The bytes freed. + private static long DropScrollbackSixels(Buffer.TerminalBuffer? buffer, long excess) + { + if (buffer is null) + return 0; + + long freed = 0; + for (int i = 0; i < buffer.YBase && i < buffer.Lines.Length && freed < excess; i++) + freed += buffer.Lines[i]?.DropSixel() ?? 0; + + return freed; + } + private HashSet CollectLiveImages() { var live = new HashSet(); diff --git a/src/XTerm.NET/XTerm.NET.csproj b/src/XTerm.NET/XTerm.NET.csproj index 95f19a0..8d9d6ea 100644 --- a/src/XTerm.NET/XTerm.NET.csproj +++ b/src/XTerm.NET/XTerm.NET.csproj @@ -7,8 +7,8 @@ XTerm - 2.0.8 - 2.0.8.0 + 3.0.0 + 3.0.0.0 Dotnet port of xterm.js, giving you ability to host console applications, with Sixel and Kitty graphics. Iciclecreek Tom Laird-McConnell