Repository navigation
Terminal: DEC private mode lists, DECTCEM and bracketed paste - #2901
Conversation
insjang
changed the title
1767cc4 to
f214496
Compare
|
This is part of what eclipse-cdt/cdt#737 asks for: the terminal reports TERM=xterm, so shells and prompts (zsh with starship, syntax highlighting, autosuggestions) send xterm sequences that it did not handle. #2900, #2903 and #2904 cover more of the same list. |
|
Rebased on master (with #2894 in), one commit, e8629e9; all terminal tests pass. The flag that hides the cursor is now volatile, since the emulator thread sets it and the paint reads it. How to test, in a runtime Eclipse with this org.eclipse.terminal.control, Terminal view, local shell:
Unit tests: VT100EmulatorTest covers several modes in one sequence, DECTCEM, bracketed paste with an embedded end marker, and the reset. |
edb0dab to
85c191e
Compare
|
DEC private mode sequences may carry several modes at once, and ncurses does exactly that when it turns the mouse on: CSI ? 1006 ; 1000 h. The emulator looked at the first parameter only and dropped the rest, so a program could end up with half of what it asked for. Every parameter is now set or reset. DECTCEM (mode 25) lets a program hide the cursor while it draws, or for good; a full screen program that hides it, such as htop, showed a cursor wandering over its screen. The request is passed to the view, which keeps its own blinking and focus handling and simply does not draw the cursor while a program has hidden it. Bracketed paste (mode 2004) marks pasted text with CSI 200~ and CSI 201~ so that a program can take it as text rather than as keys typed - a shell does not run every line of a pasted snippet, an editor does not auto-indent it. Programs that ask for it and do not get it misread multi-line pastes. The text cannot close its own bracket early: every ESC and C1 CSI in it is dropped, as xterm does, so no end marker is left in it or put together from the pieces around one taken out (ESC[20ESC[201 A paste sends each line break as a CR, as the Enter key does and as Windows Terminal (microsoft/terminal#3449) and tmux paste. The CR LF of a Windows clipboard reached the shell as two breaks: readline drew two rows for every pasted line, a paste taller than the screen pushed the prompt off it, and the redraw on the next key landed where the prompt no longer was. Input goes to the process in writes of at most 1000 bytes with a pause between them. A write no longer ends inside a UTF-8 character or an escape sequence: ConPTY has passed the pieces of a sequence split across reads on as plain text (microsoft/terminal#16352), and an end marker cut as ESC[20 and 1~ left the shell taking everything typed after it as part of the paste, Enter included. How to test, in a Terminal on Windows with WSL bash (bracketed paste is on in bash 5.1 and later):
|
There was a problem hiding this comment.
🟡 Changes recommended
Unresolved paste-handling and terminal-state defects can discard line breaks, prevent command execution, and leave cursor visibility or bracketed-paste modes incorrect.
5 open issues
What changed in this PR
This PR extends the Eclipse terminal’s DEC private-mode handling, adding application-controlled cursor visibility and bracketed paste support.
Changes:
- Process every parameter in DEC private-mode set/reset sequences.
- Support cursor visibility and bracketed paste, including newline normalization and removal of embedded escape characters.
- Add boundary-aware input chunking and regression tests.
| File | Description |
|---|---|
| terminal/tests/org.eclipse.terminal.test/src/org/eclipse/terminal/view/ui/tests/InputChunksTest.java | Tests character and escape-sequence chunk boundaries. |
| terminal/tests/org.eclipse.terminal.test/src/org/eclipse/terminal/test/AutomatedTestSuite.java | Registers the input-chunk tests. |
| terminal/tests/org.eclipse.terminal.test/src/org/eclipse/terminal/internal/emulator/VT100EmulatorTest.java | Tests private modes, connection resets, and paste formatting. |
| terminal/tests/org.eclipse.terminal.test/src/org/eclipse/terminal/internal/emulator/MockTerminalControlForText.java | Records cursor visibility and bracketed-paste state. |
| terminal/bundles/org.eclipse.terminal.view.ui/src/org/eclipse/terminal/view/ui/streams/InputStreamMonitor.java | Uses boundary-aware input chunking. |
| terminal/bundles/org.eclipse.terminal.view.ui/src/org/eclipse/terminal/view/ui/internal/InputChunks.java | Adds chunk-boundary detection for UTF-8 and escape sequences. |
| terminal/bundles/org.eclipse.terminal.control/src/org/eclipse/terminal/internal/textcanvas/AbstractTextCanvasModel.java | Suppresses cursor rendering when requested by an application. |
| terminal/bundles/org.eclipse.terminal.control/src/org/eclipse/terminal/internal/emulator/VT100TerminalControl.java | Formats pasted text and forwards cursor visibility changes. |
| terminal/bundles/org.eclipse.terminal.control/src/org/eclipse/terminal/internal/emulator/VT100Emulator.java | Handles mode lists, modes 25 and 2004, and connection resets. |
| terminal/bundles/org.eclipse.terminal.control/src/org/eclipse/terminal/internal/control/impl/ITerminalControlForText.java | Adds callbacks for bracketed paste and cursor visibility. |
Review effort: Balanced
akurtakov
left a comment
akurtakov
left a comment
There was a problem hiding this comment.
Thanks, taking every mode of a DEC private set/reset, DECTCEM and bracketed paste are all welcome, and the input chunking that keeps escape sequences whole is a nice touch. One API concern before this goes in, details inline, and a couple of small things.
- Three of the Copilot fixes have no test yet: the hidden cursor surviving a tab move (this would fit
TerminalControlRecreateUITest), Enter from the input field staying outside the brackets, and local-echo pastes going through unchanged. - Nit: with the loop, an empty parameter now counts too, and
getAnsiParameter()makes it 1, soCSI ? 1000 ; halso switches application cursor keys on. Skipping empty parameters in the loop would avoid that.
| // a stream connector that echoes locally converts line breaks itself | ||
| // (InputStreamMonitor), from what it was always given | ||
| ITerminalConnector connector = getTerminalConnector(); | ||
| sendString(connector != null && connector.isLocalEcho() ? strText : pasteText(strText, fBracketedPaste)); |
There was a problem hiding this comment.
pasteString() is API (ITerminalViewControl, "write string to terminal"), and callers use it to run commands, not only to paste: CDT's GdbExtendedInfoThreadsAction does pasteString("info threads\n"). With bash 5.1+ and anything on readline 8.1+, gdb included, bracketed paste is on by default, so such a command now arrives inside the brackets and is only inserted, not run, which is the problem Copilot found for CommandInputFieldWithHistory, but for callers outside this repository that cannot be adjusted. Could pasteString() stay as it was, and the line break conversion and bracketing apply only where the user pastes, paste() and the middle-click paste in TabFolderManager? Then CommandInputFieldWithHistory can stay as it was too.
There was a problem hiding this comment.
Thanks, that is the better split, done in bd1190a. pasteString() is back to what it was and writes its text as it is, and CommandInputFieldWithHistory has no change left. What the user pastes goes through a new default method, ITerminalViewControl.paste(String) (@SInCE 1.2, the package version raised to 1.1.0): Paste and the middle-click paste in TabFolderManager use it, and only there are line breaks sent as Enter and the text bracketed. An empty clipboard sends nothing, and a connector with local echo gets the text unchanged.
Tests: TerminalControlRecreateUITest checks that a cursor the program hid stays hidden after the move, and the new TerminalPasteUITest checks, through a connector that records what it is sent, that paste() converts and brackets, that pasteString("info threads\n") arrives as it is with bracketed paste on, and that a local-echo connector gets the text unchanged. The empty parameter is skipped now, so CSI ? 2004 ; h no longer turns on application cursor keys (testEmptyDecPrivateModeNamesNoMode).
… on request, and support bracketed paste DEC private mode sequences may carry several modes at once, and ncurses does exactly that when it turns the mouse on: CSI ? 1006 ; 1000 h. The emulator looked at the first parameter only and dropped the rest, so a program could end up with half of what it asked for. Every parameter is now set or reset; an empty one names no mode, and the ones past the 16 that are stored are dropped instead of throwing. DECTCEM (mode 25) lets a program hide the cursor while it draws, or for good; a full screen program that hides it, such as htop, showed a cursor wandering over its screen. The request is passed to the view, which keeps its own blinking and focus handling and simply does not draw the cursor while a program has hidden it. Moving the terminal tab to another view keeps it hidden: the flag is kept on the control and copied to the new canvas model under one lock with the reader's writes. The moved canvas of a connected terminal also blinks its cursor again; it used to start, and stay, with the cursor off. Scroll Lock, set in the terminal's menu like Inverted colors, stays on after a move. Bracketed paste (mode 2004) marks pasted text with CSI 200~ and CSI 201~ so that a program can take it as text rather than as keys typed - a shell does not run every line of a pasted snippet, an editor does not auto-indent it. Programs that ask for it and do not get it misread multi-line pastes. The text cannot close its own bracket early: every ESC and C1 CSI in it is dropped, as xterm does, so no end marker is left in it or put together from the pieces around one taken out (ESC[20ESC[201~1~). What the user pastes (Paste, and the middle-click paste of the terminals view, through the new ITerminalViewControl.paste(String)) is bracketed when the program asked for it, and sends each line break as a CR, as the Enter key does and as Windows Terminal and tmux paste. pasteString() still writes its text as it is: callers run commands with it (CDT sends gdb commands, the command input field its line), and inside brackets a shell would only insert them. A connector with local echo gets pasted text unchanged too, since InputStreamMonitor converts its line breaks itself. The CR LF of a Windows clipboard reached the shell as two breaks: readline drew two rows for every pasted line, a paste taller than the screen pushed the prompt off it, and the redraw on the next key landed where the prompt no longer was. Input goes to the process in writes of at most 1000 bytes with a pause between them. A write no longer ends inside a UTF-8 character or an escape sequence: ConPTY has passed the pieces of a sequence split across reads on as plain text (microsoft/terminal#16352), and an end marker cut as ESC[20 and 1~ left the shell taking everything typed after it as part of the paste, Enter included. A full reset (ESC c), and the reset before every connection, shows the cursor again and turns bracketed paste off, so a program that hid the cursor or asked for bracketed paste and was cut off by a disconnect or a crash does not leave them set for whatever runs on the next connection.
akurtakov
left a comment
akurtakov
left a comment
There was a problem hiding this comment.
Tested locally:
- htop no longer shows a cursor wandering over its screen
- a two-line paste at a bash prompt lands on the command line and runs only on Enter
- indented block pasted into mcedit arrives unchanged, without the growing indentation.
…does; pasting to a stream connector with local echo leaves line breaks to it; a hidden cursor stays hidden when the tab moves; erasing into a wide character at the margin clears the wrap mark; drag scrolling follows where the pointer is now. As reviewed for eclipse-platform/eclipse.platform#2901 and #2905.
…mmands; what the user pastes (Paste, middle click, dropped files, Run Selection, the input field) goes through the new ITerminalViewControl.paste(String), package io.github.insjang.terminal.control 1.3.0; an empty DEC private mode parameter names no mode; every command but writing text ends the cluster written last; an empty write keeps the layout; copying a whole row keeps what a narrowed screen holds past its margin. As reviewed for eclipse-platform/eclipse.platform#2901 and #2904.
DEC private mode sequences may carry several modes at once, and ncurses does exactly that when it turns the mouse on: CSI ? 1006 ; 1000 h. The emulator looked at the first parameter only and dropped the rest, so a program could end up with half of what it asked for. Every parameter is now set or reset.
DECTCEM (mode 25) lets a program hide the cursor while it draws, or for good; a full screen program that hides it, such as htop, showed a cursor wandering over its screen. The request is passed to the view, which keeps its own blinking and focus handling and simply does not draw the cursor while a program has hidden it.
Bracketed paste (mode 2004) marks pasted text with CSI 200~ and CSI 201~ so that a program can take it as text rather than as keys typed - a shell does not run every line of a pasted snippet, an editor does not auto-indent it. Programs that ask for it and do not get it misread multi-line pastes. An end marker inside the text is removed so the paste cannot close its own bracket early.