Expression cards in the Logic Programmer, editable as text

[rubensworks]

One variable card can now hold a whole expression built in the existing Logic Programmer, for example

item -> size(nbt(item)) > 3
l -> reduce((+), map((stringLength), l), 0)
(a, b, c) -> a > b ? a * c : b - c

instead of 6 to 8 cards in a Variable Store glued together with operator, apply, pipe and flip. The expression can be built with slots, or typed and copied as text, in the same view.

This PR now also contains what were the stacked PRs #1767 (text) and #1768 (editing in place), which are closed.

Screenshots of the current Logic Programmer: https://claude.ai/artifact/Fwyz9rB9VqPMmZY6FYasuF

What was built

Data model (api/evaluate/expression/tree, core/evaluate/expression/tree)

• Expression(root, parameters) with sealed ExpressionNode: Operator(op, children), Ref(variableId), Const(value), Param(index), plus Empty for slots that are still being edited.
• Serialized as nested NBT (not nested strings). Deserialization rejects trees deeper than 64 levels, more than 8 parameters, unknown operators/types and out-of-range parameter indexes. A 60-level tree is about 3.5 KB of binary NBT (unit test).
• The card also stores the known output types of the cards it references (referenceTypes). The LP uses them when the card is loaded again, and the card's tooltip uses them for its type, so it matches what the LP showed. Networks validate against the type inferred without them, as they check the actual variables.

Evaluation

• Without parameters, ExpressionVariableFacade.getVariable builds nested LazyExpressions. The root is cached in the network under the card id (same as OperatorVariableFacade, so LazyExpression-based advancement predicates still see it). Inner nodes use a cache local to the tree instead of derived ids, so they can never collide with other variable ids. Invalidation propagates through the normal listener chain; lastNetworkHash handling mirrors OperatorVariableFacade.
• With parameters, the card's value is a ValueOperator wrapping the new ExpressionOperator (extends OperatorBase). Inputs are the parameter types, output is inferred. Referenced cards are bound as variables when the card is built, like CurriedOperator does, and a wrapper variable forwards their invalidation. Each call evaluates nodes lazily and at most once, without registering listeners, so choice only evaluates the branch it needs.
• materialize() replaces refs by their current materialized values and keeps parameters. Materializing a pipe that contains an expression operator also materializes it (reuses the Fix crash when displaying piped operators #1705 pattern in CombinedOperator). ExpressionOperator.Serializer writes refs as their current values, like the curry serializer.

Compatibility rule: a single operator whose inputs are all cards and that has no parameters is written as a plain OperatorVariableFacade, exactly as before. Only trees that need the new features produce the new integrateddynamics:expression facade. Old cards load and evaluate unchanged; loading an old card into the LP yields a one-level tree.

Type inference (ExpressionTypeChecker): walks the tree with getInputTypes, validateTypes and getConditionalOutputType, returning structured errors (node path, slot, expected, actual, message) and the number of empty slots.

Names (ExpressionPrinter, OperatorNames): operators are named the way Integrated Scripting names them. A call uses the operator's scoped interact name (length, nbt, getInteger), which IS uses for methods on values, when the type of its first argument tells it apart from other operators with that name; otherwise it uses the global interact name (stringLength), which IS uses for idContext.ops. 248 of the 267 scoped names belong to one operator, and the other 19 are always told apart by a concrete first argument type, so the fallback mostly shows up for empty slots, untyped parameters and operator values ((stringLength)). Only + - * / % == != < <= > >= && || are infix and ! prefix, each meaning exactly one operator (strings use concat), and complete choices print as c ? a : b, so names never contain LP symbols such as NBT{}.get_integer or =NBT=. OperatorNames.resolve(name, firstArgumentType) is the inverse, and a unit test checks for every registered operator that its printed name is an identifier that resolves back to it. Used in the card tooltip (Expression: and Type: lines) and in the LP, where the text, chips, breadcrumb tabs and status messages use the same names.

Logic Programmer (upgraded OperatorLPElement, no new element)

• A slot accepts a card (ref), a typed value, a real item (item constant, for slots that accept items), a parameter, a nested operator, or an operator as a value.
• Left-click a slot to select it. A value field appears for text-editable types (Boolean, Integer, Long, Double, String, NBT, Number, Any); Enter writes the constant. While a slot is selected, the operator list only shows operators whose output fits the slot. Picking one puts it in the slot and opens it, so its inputs can be filled in next. Tabs at the top of the config area lead back up; clicking a nested chip twice reopens it. Right-click clears a slot; Escape or a click on the empty config area cancels a selection.
• In an Operator slot (the first input of map, filter, reduce, apply, ...) every operator is listed. Picking one that cannot give an operator puts it in as a value, so map((stringLength), l) needs no extra card. Operators that may give an operator (flip, pipe, apply) are nested as calls, since that is what you usually want there; shift-click puts those in as a value too.
• Chips show the operator's short name (size, nbt, >) or the value at the largest scale that fits, with a badge while a nested operator has empty inputs. A chip is outlined red when it or anything inside it has a type error, and hovering it highlights its part of the expression.
• A side panel shows the expression:
  • Right of its title: Undo, Redo and Copy (see Text).
  • Below the title: the parameters as chips, followed by a + that reads + Parameter while there are none. Drag a chip onto a slot to use it there, or click it and then the slot; while selected, type to rename it and use the arrows to set its type; right-click to remove it. Chips show only the name, with the type in their tooltip and in the type of the expression, and wrap onto as many rows as needed at most half the panel wide each, so all 8 parameters always show.
  • Below that, a dark box that fills the rest of the panel with the expression as text: it wraps, shows the edited node in yellow, errors in red and empty slots as a gray ?, highlights the part of a hovered slot, and can be clicked and edited (see Text).
  • The footer of that box shows the inferred type in the colors of the value types (as in card tooltips), or the first type error instead: 2 type error(s), the first in input 2 of *, or the message of an error of an operator as a whole (such as choice branches of different types), since that has no slot to hover. Hovering an error shows its full message. Whether the expression can be written is shown by the existing check mark next to the operator name.
  • All text in the panel is drawn at full size, since text scaled to 75% gets uneven strokes at GUI scales 2 and 3.
• Parameter types default to Any and take the type of the first slot they're dropped in that is not Any (a concrete type or Number).
• Loading a card back into the LP shows the actual referenced cards when they are in the player's inventory. Otherwise the slot gets a placeholder card with the referenced id and its stored type ("Reference to the variable card with this ID"), instead of the fake value cards with default values that were used before. Cards placed in slots are remembered while navigating through nested operators.
• The panel can be hidden with an arrow button at the top right, like the vanilla recipe book. The Logic Programmer then has its old 256px width and expressions remain editable in the slots.
• Slot edits are applied identically on both sides through one LogicProgrammerExpressionActionPacket carrying small actions (const, op, param, clear, nav, parameter add/remove/rename/type), handled by the side-independent ExpressionEditor. The server only accepts slot edits for the node being edited and ignores malformed actions.
• Labeller and write slot are unchanged.

Text (ExpressionParser, ExpressionPrinter.printExact)

• Clicking the expression places the caret and opens the operator under it in the slots, so text and slots follow each other. Typing makes a draft. The text around what was typed keeps its colors, except for the red of errors, and the footer shows the type of the typed text, or the parse error with its line and column (Line 1, column 24: The text ends too early, a value is missing), which is also underlined in the text. Enter applies the draft, Escape reverts it, and Shift+Enter adds a line. While there is a draft, slots, parameters and tabs ignore clicks, so the draft and the slots cannot both change the expression. Hiding the panel reverts the draft.
• Undo, Redo and Copy are 12 px icon buttons right of the panel title. During a draft, a cross (revert, Esc) and a check (apply, Enter) take their place, and the check is disabled while the text does not parse. The font has no symbols for these, so the icons are drawn pixel by pixel, in the colors and with the shadow of the text of text buttons. Undo and Redo go back and forth through up to 64 changes to the expression, from slots, parameters and text alike, also across a change of root operator, and are reset when another element or card is opened. Ctrl+Z and Ctrl+Y (or Ctrl+Shift+Z) work too. Copy puts the text on the clipboard and shows a check for a moment.
• Typing never waits on the server. Parsing, name resolution and type checking all run on the client, against the ref types the editor knows plus the cards in the player's inventory. Nothing is sent while typing. On apply, the client applies the result locally and sends the tree once, as a set action in the same packet. The server deserializes it with the existing limits (depth 64, 8 parameters, known operators and types) and also checks ExpressionEditor.isEditable: the root is an operator, at most 1024 nodes, parameter names that the text form can read back, parameter types from the LP's list, and only operators that have a slot for each input. Anything else is ignored.
• Another root operator switches the element. Each LP element is one root operator, so if the text has another root (typing (a + b) * 2 while in >), the element of that operator becomes active, the same way loading a card does. The known ref types and the actual cards placed in slots are carried over first, since deactivating the old element forgets them. Otherwise, applying keeps the open operator in the slots if it still exists.
• Format. It uses the names above, so it reads like Integrated Scripting.
  • Calls use the scoped name when the type of the first argument tells it apart (length(s) for strings, length(l) for lists), and otherwise the global one (stringLength). Both are accepted when parsing.
  • Infix + - * / % == != < <= > >= && || with C precedence, prefix !, and c ? a : b.
  • ? is an empty slot, and (name) is an operator as a value, such as (+) or (stringLength).
  • Parameters are declared in front: x -> ..., or (x: integer, y) -> ... with optional types.
  • Literals are true/false, 1, 5L, 1.5/1e3/2d/NaN/Infinity, and quoted strings with \" \\ \n \t \r \uXXXX escapes.
  • Any other value is written as its type with its serialized form, @type{snbt}, so the printer is exact for every value, not only the common ones.
  • #42 refers to a card by ID. #limit or #"my label" refers to it by label, and only resolves when exactly one card has that label. Copy prints #label only in that case, and #id otherwise.
• ExpressionPrinter.printExact is the inverse of ExpressionParser.parse, and the tests check this both ways.
• Implementation:
  • ExpressionParser is a recursive descent parser with a small lexer, for at most 8192 characters, with 28 translatable error messages in ExpressionParseException.
  • ExpressionTextField (client only) uses vanilla MultilineTextField as the text model, and draws it with a color and background per character, a selection, a blinking caret, an underline for parse errors and a scrollbar, since vanilla's edit boxes can only draw a single color. It is kept in the container, so it survives the panel being recreated when navigating.
  • ExpressionPrinter.tokenizeExact gives the text as tokens with the path of their node, which maps each character to its node for colors and for navigating from the caret.
  • ExpressionHistory records changes by observing the expression while rendering, so no action needs to know about it.
  • ContainerScreenLogicProgrammerBase forwards scrolling, dragging and releasing to the panel, since CyclopsCore sub-GUIs only get clicks and keys.
  • Renaming a parameter now also rejects names the text could not read back, such as true, NaN or 1x.

Book: new "Expressions" page after "Logic Programming" in the manual, covering slots, nesting, operator values, the panel, parameters and editing as text. Tutorial advancements and their criteria are untouched.

Also fixed: pasting onto panels with the Wrench

Not specific to expressions, found while testing card copies: since #1721, right-clicking a Display Panel or Dynamic Light Panel with a Wrench in a config mode rotated the panel instead of pasting the configuration, because PartTypePanelVariableDriven.onPartActivated handled every wrench before the generic part code. A Wrench in a config mode now pastes; in the other modes it still rotates. Covered by GameTestsWrenchConfig.testWrenchConfigPasteOntoPanel, which fails without the fix. This is a separate commit, so it can be split out into its own PR if you prefer.

Found, not fixed here

OperatorBuilders.FLUIDSTACK_3 is built from ITEMSTACK, so the fluid with_data operator has the unique name itemstack_withdata of the item one, and both share a translation key. Everything that looks operators up by unique name (operator values, the card codec, LP elements, expression trees) gets the fluid operator, so an item with_data operator turns into the fluid one after a save and load. It is the only duplicate unique name. Fixing it changes which operator existing itemstack_withdata data loads as, so I left it for a separate change. OperatorNames uses global names to recognize registered operators, so it does not depend on this.

Deviations from the plan, and why

• A collapsible side panel instead of extra height. I first added 40px of height, and it was clipped: at an 854x480 window the GUI scale gives 240 rows, and 1080p at auto scale gives 270, both less than 280. A 150px side panel keeps the GUI at 402x240. Because that still costs width (at 1080p auto scale, JEI's list would drop from about 112 to about 39 GUI pixels per side), the panel can be hidden, restoring the old width. It is shown by default for discoverability; the choice is remembered for the game session only. No texture change was needed; logic_programmer.png is untouched.
• Nesting opens the nested operator right away, rather than requiring a double-click, since its inputs are almost always filled in next.
• Action packets instead of a full-tree packet for slot edits, so the server never has to trust client refs or deep trees for those. Applying text is the exception, with the checks described under Text.
• Labelled refs print as #limit, not "limit". Quotes are used for string constants, so a quoted label would be ambiguous.
• Parameters are dragged onto slots, or clicked and then the slot. While dragging, the chip follows the cursor and the slot it would go in is outlined; dropping it anywhere else cancels. Clicking both stays, since it mirrors how vanilla moves items.
• Parameter types are set with arrows on the selected parameter rather than a separate type picker. They cycle through the common types (Any, Boolean, Integer, Long, Double, Number, String, List, Operator, NBT, Item, Block, Fluid, Entity, Ingredients, Recipe).
• Inner-node cache identity uses a per-tree local cache instead of ids derived from card id + node path (see above).
• Item constants come from placing a real item in the slot (phantom, not consumed). JEI/EMI ghost dragging into those slots is not wired up, since the JEI integration lives outside this repo.
• A text parser, which was a non-goal at first. The read-only text in the panel became editable in place, instead of a separate text mode with Edit, Apply and Cancel buttons, which an earlier version of this work had.

Behavior changes players will notice

• Left-clicking an operator input slot with an empty hand now selects it instead of clearing it. Right-click clears.
• Non-card items can be placed in operator slots that accept items (Item or Any), where they become inline constants.
• Escape first reverts typed text or cancels a slot or parameter selection; a second Escape closes the gui.
• Ctrl+Z and Ctrl+Y undo and redo changes to an expression in the Logic Programmer.
• With the panel shown (the default), the LP is 146px wider. The arrow button restores the old width.
• Loading a classic operator card into the LP shows its referenced cards from the inventory, or reference placeholders, instead of value cards holding default values.

Things to decide on

• Typed references. With slots, refs only come from cards placed in slots, or from a card that was loaded. With text, you can type #123 for any ID. Evaluation still needs that variable in the same network, so I don't think this gives access to anything new, but you should decide on it explicitly. Requiring the card in the inventory for every #id on the server would be easy to add, but it would also reject loading and re-applying a card whose referenced cards are elsewhere.
• Shift+Enter line breaks do not survive Enter, since the text is printed again from the expression tree. Either drop Shift+Enter, or let the printer break long expressions over lines.
• The lock during a draft is silent. Clicks on slots, chips and tabs do nothing until the draft is applied or reverted, without saying why. Since a draft keeps its colors, a draft that is not focused looks almost the same as the expression itself, apart from the cross and check in the toolbar.
• Hiding the panel throws a draft away, as a hidden draft would otherwise keep the slots locked.

Known limitations

• No quick fixes (for example "Parse as Integer") and no "Try" slot.
• No inline lambdas: parameters belong to the whole card, so map(x -> x * 2, l) cannot be written as such. It needs a second card, or partial application inside the slot, as in map(apply((*), 2), l).
• No completion of operator names while typing. The names are those of the operator tooltips' interact names.
• Operators that may give an operator can only be put in as a value with shift-click, which only the book mentions.
• When a loaded classic card references cards that are not in the inventory, the types of refs in Any slots are unknown, so related type errors only show once the card is put back in. Network-side validation still checks them. Expression cards store these types, so this only affects classic cards.
• An expression card used as a reference in another expression has the type inferred without its own references, so a card such as #5 + 1 counts as Any there, and type errors involving it only show in the network.
• The panel's open/closed state is not saved between game sessions.
• Encoding/decoding (Allow materialized values to be encoded into strings and decoded again #1737): an expression operator value serializes through its own operator serializer, which the gametest round-trips. I did not separately test the Materializer encode/decode flow with an expression card.
• No new optional advancement for writing a card with a parameter.

API additions for add-on authors

• org.cyclops.integrateddynamics.api.evaluate.expression.tree: Expression, ExpressionNode (sealed records), ExpressionParameter.
• IExpressionVariableFacade (getExpression(), getReferenceTypes() with a default), handler ExpressionVariableFacadeHandler with type key integrateddynamics:expression.
• ReferencePlaceholderVariableFacade with handler integrateddynamics:reference_placeholder, only used in LP slots (phantom, cannot be taken out). In a network it resolves to the card with the same id.
• ExpressionOperator (new IOperator implementation) with operator serializer integrateddynamics:expression.
• ExpressionTrees (serialization, paths, queries), ExpressionTypeChecker (including the inferred type of every node), ExpressionPrinter (including printExact and tokenizeExact), ExpressionParser with ExpressionParseException, OperatorNames, ExpressionEvaluator.
• LP: ExpressionEditor (including the set action and isEditable), OperatorLPElement.getEditor() / applyExpressionAction(...) / getCurrentOperator() / findReferencedCards(Player) / setExpression(...), ContainerLogicProgrammerBase.applyExpressionAction, refreshElementInventory, setExpressionSlotFilter, layout constants EXPRESSION_PANEL_*, ContainerScreenLogicProgrammerBase.isExpressionPanelOpen(), and RenderPattern.getRenderSymbol().
• LabelsWorldStorage.getVariableIds(label).
• IOperatorVariableFacade is unchanged and expression facades do not implement it, so add-ons reading it see exactly the cards they saw before.

Testing

• ./gradlew build: passes, 983 unit tests. New:
  • TestExpressionTrees (55 tests: serialization round trip, 60-level tree and its binary size, too-deep and invalid-index rejection, paths, compatibility rule, evaluation equivalence with a LazyExpression chain, invalidation, parameter binding, apply/filter/map/reduce/pipe/flip/currying, lazy choice, operator serialization, materialization including inside pipe, type inference and error positions, the card's type with and without stored reference types, printer output including shared names told apart by type and former LP symbols written as calls; and more complex cases: three parameters with partial application, apply_2 and flip, higher-order parameters, expression operators as inline values and as refs, map(apply((*), 2), l) as an inline doubler, nested materialization with live and frozen refs, a 30-level parameter chain, a parameter used in several subtrees, and an evaluation error deep in the tree).
  • TestExpressionEditor (11 tests: actions, parameter type inference including Number, picking operators as values, cycling, value parsing, rejection of bad actions, the set action and what cannot be edited).
  • TestOperatorNames (4 tests, including that every registered operator gets an identifier name that resolves back to it, with and without a known first argument type).
  • TestExpressionParser (12 tests: every editable operator, more than 250, prints and parses back to itself; the display form also parses; values, including NBT and a curried operator, and refs and labels print and parse back; precedence, parameters, type-dependent names, literals and errors with positions; depth and node limits). Breaking the L suffix on purpose makes two of them fail.
  • TestExpressionHistory (4 tests) and TestExpressionTextField (5 tests, for the colors kept around typed text).
• ./gradlew runGameTestServer: all 1054 pass. New GameTestsExpressions: ref + inline constants on a Display Panel, equality with the equivalent classic card chain, item -> size(nbt(item)) > 1 used by a classic filter card over an Inventory Reader list, apply of an expression operator with a ref, an expression operator on a Display Panel (serialized and evaluated), card data save/load, a missing reference, the LP compatibility rule and loading old/new cards, a type error position in the LP, l -> reduce(#add, map(#double, l), 0) where #add and #double are expression cards in the same network, three parameters partially applied through apply, a Display Panel following a changed referenced card, two expression cards referencing each other and one referencing itself (validation error, no crash), reference types surviving an LP write and load, copying expression cards with and without parameters in a crafting grid, pasting one onto another Display Panel with the Wrench (the copies get a new id, keep the expression and reference types, and evaluate like the original), applying (#id + 3) * 2 as text to an LP on > (Arithmetic Multiplication becomes active, a const root is ignored, and the written card shows 36 on a Display Panel), and applying text keeping the open operator. I confirmed the new class actually runs by breaking one assertion once and seeing it fail.
• Building expressions with slots, with clientdevbridge-cli: built item -> size(nbt(item)) > 3 from scratch, changed a parameter type to cause a nested type error, cancelled with Escape, entered an invalid value, wrote the card, and hid and reopened the panel. Built (x, y, z) -> x > y ? x * z : y - z, l -> reduce((+), map((stringLength), l), 0) (checked the written card's tooltip: List -> Integer) and x -> x > 1 ? x * 2 : x (x inferred as Number), and caused a branch type mismatch in choice to check that its message is shown. Built #greeting == (#greeting + "!") with a String card, navigated in and out (the actual card stays), wrote it, and loaded it back with the card in the inventory (actual card shown) and with the card moved to a chest (placeholder with id and type shown). On an earlier UI revision (the network side is unchanged since): built filter(#bignbt, #items) (written as a classic card) and length(filter(#bignbt, #1)) in a real network with an Inventory Reader, Variable Store and two Display Panels, reloaded the world (both panels kept [Sword] and 1), used the Labeller, made an item constant in the Portable LP, and checked that the List and Recipe elements are unchanged. Built x -> getInteger(nbt(x), "key"), which shows itemstackNbt while the nested slot is empty and nbt once x is in it. Copied the (x, y, z) card in a crafting table (the card left in the grid gets a new id, both keep the expression), and copied a Display Panel holding it onto another one with the Wrench (pasted with a new id, after the panel fix above).
• The panel and text, on the current version (screenshots above):
  • the type in its colors (Integer, Double, String, Number, and signatures such as Integer -> String -> String), a wrapped type, a slot error, and the full message on hover
  • parameters: none, three, 8 with 16-character names, a long name that is cut off and wraps, dragging one onto a slot (the chip follows the cursor and the slot is outlined), dropping it onto nothing, and clicking a chip and then a slot
  • clicking the text placing the caret and opening that operator in the slots
  • typing a draft, applying it with Enter and with the check, the open operator staying open, and staying yellow around a typed character
  • a parse error underlined, with the check disabled, and reverting with Esc and with the cross, without closing the GUI, and typing e not closing it either
  • selecting by dragging, and replacing everything with (a: integer) -> a + #5, which switched to Arithmetic Addition with the real #5 card in its slot and rewrote the card
  • the tooltip of cards written from text matching the panel: a -> a + #5 as Integer -> Integer (it said Integer -> Any before the reference types were used), and #5 + 1 as Integer
  • undo and redo with the buttons, over text changes, slot changes and a change of root operator
  • slot clicks being ignored during a draft, hiding the panel reverting it, and Clear or loading a card resetting the history
  • Copy giving the exact text, and its check
  • scoped names (length, concat) and string literals
  • the margins, measured in a screenshot
• Not exercised in the client: Ctrl+Z, Ctrl+Y, Shift+Enter and paste. clientdevbridge sends modifier bits, but vanilla checks the physical key state with Screen.hasControlDown() and Screen.hasShiftDown(), so I used the buttons instead.
• Integrated Tunnels 1.13.0-801 (from the CyclopsMC Maven, in the dev client's mods folder): item -> length(name(item)) > #threshold in an Item Importer's Import Items Predicate aspect, with the threshold an Integer card in a Variable Store. With single items, Diamond, Cobblestone and Oak Planks were imported and Apple, Stick and Egg stayed. With the threshold card taken out of the network, the importer showed "Variable with id 5 could not be found within the current network" and an added Emerald stayed; after putting the card back, the Emerald was imported. With stacks, note that ID's name of an item stack includes the count ("Apple (10)"), which is existing behavior. Screenshots, taken before the panel changes: https://claude.ai/artifact/WZMMY6VcHoVUWMW4d27N42 (a private page owned by @rubensworks; GitHub's API does not accept image uploads, so share or copy them from there).

Issues

Closes #194, closes #546, closes #860, closes #1112, closes #1417, closes #1495, closes #367 (these are also in the commit message). Partially addresses #193, #216, #527, #804, #930, #1293 (type shown before writing, errors pointing at the slot) and #43, #906, #1326, #525, #1130 (fewer intermediate cards).

🤖 Generated with Claude Code

https://claude.ai/code/session_01MHtucHo4Qy2pTfBMgeE54y

[coveralls]

coverage: 48.401% (-0.3%) from 48.683% — feature/expression-cards into master-1.21-lts