Skip to content

Terminal: follow the VT100 and ECMA-48 parser rules, and support the VT100 commands vttest checks - #3059

Open
insjang wants to merge 1 commit into
eclipse-platform:masterfrom
insjang:parser-robustness
Open

insjang wants to merge 1 commit into
eclipse-platform:masterfrom
insjang:parser-robustness

Conversation

@insjang

@insjang insjang commented Nov 5, 2026

Copy link
Copy Markdown
Contributor

The parser now handles what it does not know as DEC's terminals and xterm do (https://vt100.net/emu/dec_ansi_parser):

  • an escape with intermediate bytes is dropped up to its final byte; ESC # 6 used to show "6", ESC SP F showed "F"
  • DCS, SOS, PM and APC strings (ESC P, ESC X, ESC ^, ESC _) are dropped up to their ST instead of being shown, e.g. an XTGETTCAP query or a tmux passthrough
  • C0 controls without a meaning here (SO and SI aside) and DEL are not written as characters; VT and FF move down as LF does
  • inside a sequence, CAN and SUB cancel it, ESC starts a new one, and the other controls run in place where they used to become part of a parameter
  • a control sequence with an intermediate byte, or with a private marker other than '?' and the '>' of DA2, is dropped instead of being run as another command: CSI 1 SP @ is not ICH, CSI = 1049 h does not switch to the alternate screen

Commands and modes added:

  • IND, NEL, HTS (ESC D, ESC E, ESC H), TBC (CSI g), CHT and CBT (CSI I, CSI Z); tab stops start every 8 columns
  • HVP (CSI f), REP (CSI b), SCOSC and SCORC (CSI s, CSI u)
  • DA1 answers a VT100 with advanced video (CSI ? 1 ; 2 c), as xterm in VT100 mode; DA2 answers CSI > 0 ; 0 ; 0 c
  • DECALN (ESC # 8) fills the screen with E's in the default rendition, resets the margins and homes the cursor
  • DEC special graphics (ESC ( 0, ESC ) 0, SO, SI) draw the line drawing characters that TERM=xterm promises with smacs/acsc; the national sets are taken as ASCII
  • DECAWM off (CSI ? 7 l): a character at the right margin replaces the last one and the cursor stays
  • DECOM (CSI ? 6 h): cursor positions and CPR count from the top margin and stay inside the scroll region

Fixed on the way:

  • a count or position of 0 means 1 (ECMA-48), for the cursor moves, ICH, DCH, ECH, IL, DL, SU and SD; the backend still takes a count of 0 as nothing
  • DECSTBM homes the cursor, as on a VT100
  • CUU, CUD, CNL and CPL stop at the margins from inside the scroll region
  • IL, DL, SU and SD with a count past the scroll region empty it and leave the lines outside it alone; they used to clear lines below the region, and above it when IL and DL followed each other
  • ESC 7 saves the rendition, the origin mode and the character sets with the cursor, as DECSC; a full reset forgets what was saved

Not covered: 8-bit controls (S7C1T, S8C1T), VT52 mode, 132 columns (DECCOLM), double width and height lines, protected areas (DECSCA), DECSTR and DECRQM.

Tests: 23 new cases in VT100EmulatorTest, each failing without the change it covers.

vttest 2.7 (menus 1-4 and 6-11, keyboard and mouse left out), with the wrap deferred as the local, SSH and Telnet connectors set it, compared screen by screen with tmux running the same keys: 370 of the 390 screens both runs reach match, 307 on master. Of the screens that matched on master, the two that differ now show the line drawing characters, which tmux's capture prints as the letters it was sent.

How to test: run vttest (e.g. apt install vttest) in a Terminal, and go through menu 1 (cursor movements), 2 (screen features), 3 (character sets) and 8 (insert/delete). On master the E frame of menu 1 is missing, digits of ESC # show up, and the line drawing set shows letters.

This branch is on master. It touches the emulator as #2901, #2902, #2903 and #2904 do, so it will need a rebase as those are merged; I will do that.

@akurtakov

Copy link
Copy Markdown
Member

I would appreciate if you slow down a bit opening PRs as there are way too many notifications coming in from rebasing all terminal PRs when one lands in. In an ideal world that should lure more reviewers but I truly doubt this will happen.

@github-actions

github-actions Bot commented Nov 5, 2026 •

Copy link
Copy Markdown
Contributor

Test Results

    54 files  ±  0      54 suites  ±0   34m 1s ⏱️ - 4m 32s
 5 039 tests + 35   5 017 ✅ + 35   22 💤 ±0  0 ❌ ±0 
13 008 runs  +105  12 854 ✅ +105  154 💤 ±0  0 ❌ ±0 

Results for commit f7eb418. ± Comparison against base commit bc125e3.

♻️ This comment has been updated with latest results.

@insjang

insjang commented Nov 5, 2026

Copy link
Copy Markdown
Contributor Author

Sorry about the noise, and thanks for saying so. I'll stop opening new PRs until the open ones are through, and I won't rebase the others whenever one lands: I'll rebase a PR only when you are about to look at it or ask for it, and push only to answer a review, with the fixes for one review in one push. #3059 can wait until the rest are done. I'll run Copilot on separate branches in my fork first, so those runs don't notify anyone here.

@insjang
insjang force-pushed the parser-robustness branch 2 times, most recently from aae003e to fee3247 Compare November 6, 2026 02:35
@akurtakov
akurtakov requested a balanced review from Copilot November 6, 2026 05:26

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Unresolved terminal-emulation defects cause incorrect wrapping, stale character-set state, incomplete cursor-state restoration, unintended screen clearing, and truncated repeated output.

6 open issues
What changed in this PR

This PR improves Eclipse Terminal’s VT100 and ECMA-48 compatibility by tightening escape-sequence parsing and expanding support for terminal commands exercised by vttest.

Changes:

  • Handles unsupported sequences and embedded controls without displaying stray characters.
  • Adds tab stops, line drawing, device reports, cursor modes, and related commands; corrects scrolling and wrapping behavior.
  • Adds regression tests and captures terminal replies in the test mock.
File Description
terminal/​tests/​org.eclipse.terminal.test/​src/​org/​eclipse/​terminal/​internal/​emulator/​VT100EmulatorTest.java Adds parser, command, mode, and screen-behavior regression tests.
terminal/​tests/​org.eclipse.terminal.test/​src/​org/​eclipse/​terminal/​internal/​emulator/​MockTerminalControlForText.java Captures terminal replies for assertions.
terminal/​bundles/​org.eclipse.terminal.control/​src/​org/​eclipse/​terminal/​internal/​emulator/​VT100EmulatorBackend.java Implements auto-wrap control, screen filling, and bounded region scrolling.
terminal/​bundles/​org.eclipse.terminal.control/​src/​org/​eclipse/​terminal/​internal/​emulator/​VT100Emulator.java Expands parsing, command handling, character sets, and saved state.
terminal/​bundles/​org.eclipse.terminal.control/​src/​org/​eclipse/​terminal/​internal/​emulator/​VT100BackendTraceDecorator.java Delegates and traces the new backend operations.
terminal/​bundles/​org.eclipse.terminal.control/​src/​org/​eclipse/​terminal/​internal/​emulator/​IVT100EmulatorBackend.java Defines backend operations needed by the new terminal behavior.

Review effort: Balanced

…VT100 commands vttest checks

The parser now handles what it does not know as DEC's terminals and
xterm do (https://vt100.net/emu/dec_ansi_parser):
- an escape with intermediate bytes is dropped up to its final byte;
  ESC # 6 used to show "6", ESC SP F showed "F"
- DCS, SOS, PM and APC strings (ESC P, ESC X, ESC ^, ESC _) are dropped
  up to their ST instead of being shown, e.g. an XTGETTCAP query or a
  tmux passthrough
- C0 controls without a meaning here (SO and SI aside) and DEL are not
  written as characters; VT and FF move down as LF does
- inside a sequence, CAN and SUB cancel it, ESC starts a new one, and the
  other controls run in place where they used to become part of a
  parameter
- a control sequence with an intermediate byte, or with a private marker
  other than '?' and the '>' of DA2, is dropped instead of being run as
  another command: CSI 1 SP @ is not ICH, CSI = 1049 h does not switch
  to the alternate screen

Commands and modes added:
- IND, NEL, HTS (ESC D, ESC E, ESC H), TBC (CSI g), CHT and CBT (CSI I,
  CSI Z); tab stops start every 8 columns
- HVP (CSI f), REP (CSI b), SCOSC and SCORC (CSI s, CSI u)
- DA1 answers a VT100 with advanced video (CSI ? 1 ; 2 c), as xterm in
  VT100 mode; DA2 answers CSI > 0 ; 0 ; 0 c
- DECALN (ESC # 8) fills the screen with E's in the default rendition,
  resets the margins and homes the cursor
- DEC special graphics (ESC ( 0, ESC ) 0, SO, SI) draw the line drawing
  characters that TERM=xterm promises with smacs/acsc; the national sets
  are taken as ASCII
- DECAWM off (CSI ? 7 l): a character at the right margin replaces the
  last one and the cursor stays
- DECOM (CSI ? 6 h): cursor positions and CPR count from the top margin
  and stay inside the scroll region

Fixed on the way:
- a count or position of 0 means 1 (ECMA-48), for the cursor moves, ICH,
  DCH, ECH, IL, DL, SU and SD; the backend still takes a count of 0 as
  nothing
- DECSTBM homes the cursor, as on a VT100, takes either margin alone,
  and ignores a region of fewer than two lines
- IL and DL put the cursor on the left margin; a TAB with no stop ahead
  keeps a pending wrap
- CUU, CUD, CNL and CPL stop at the margins from inside the scroll region
- IL, DL, SU and SD with a count past the scroll region empty it and
  leave the lines outside it alone; they used to clear lines below the
  region, and above it when IL and DL followed each other
- ESC 7 saves the rendition, the origin mode, the character sets and a
  pending wrap with the cursor, as DECSC; ESC 8 does not act on the wrap
  with DECAWM off, and in origin mode brings the cursor back inside the
  region; a full reset forgets what was saved
- a new connection turns application cursor keys (DECCKM) off, as a full
  reset did already: a program cut off with them on left the arrow keys
  of the next shell sending SS3 sequences
- one test sets every DEC private mode from 1 to 9999, every ANSI mode
  and the other state a program can change, then checks that ESC c and
  the reset before a connection both leave the emulator as a new one, so
  a mode added later cannot be left out of either reset unnoticed
- mode 1049 saves and restores the same modes in a slot of its own, so
  what ESC 7 saved on the normal screen survives the switch
- a character set with a longer name (ESC ( % 5) or a 96-character set
  (ESC - A) replaces G0 or G1 and is not line drawing
- a TAB to a stop past a screen narrowed after the stops were set stops
  at the right margin
- REP with a count past a screenful writes only what changes the screen
  and the cursor, a line measured in characters of the repeated width

Not covered: 8-bit controls (S7C1T, S8C1T), VT52 mode, 132 columns
(DECCOLM), double width and height lines, protected areas (DECSCA),
DECSTR and DECRQM.

vttest 2.7 (menus 1-4 and 6-11, keyboard and mouse left out), with the
wrap deferred as the local, SSH and Telnet connectors set it, compared
screen by screen with tmux running the same keys: 370 of the 390 screens
both runs reach match, 307 on master. Of the screens that matched on
master, the two that differ now show the line drawing characters, which
tmux's capture prints as the letters it was sent.
insjang added a commit to insjang/eclipse-modern-terminal that referenced this pull request Nov 7, 2026
…T100 commands vttest checks (origin mode, auto wrap off, tab stops, REP, DA, line drawing, DECALN); line counts past the scroll region stay in it; IL/DL, TAB, DECSC/DECRC and DECSTBM as in xterm. As in eclipse-platform/eclipse.platform#3059.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants