Skip to content

Multilingual wiki: 13 languages, i18n generators + test suite, map and search overhaul - #6

Open
clouths wants to merge 55 commits into
mainfrom
feat-multilanguage
Open

clouths wants to merge 55 commits into
mainfrom
feat-multilanguage

Conversation

@clouths

@clouths clouths commented Oct 14, 2026

Copy link
Copy Markdown
Contributor

What this does

Turns the wiki into a full multilingual site: English + 13 languages (Čeština, Deutsch, Español, Français, Italiano, 日本語, 한국어, Nederlands, Polski, Português, Русский, 简体中文, 繁體中文).

  • All generated pages translated — robots, ores, plants, deployables, missions, extensions, shop, tech tree, zones, zone index and the world map, per language (content/<lang>/…). All translatable strings live in translations/<lang>.json (510+ reference keys, full parity enforced by a CI gate, make i18n-check); the Python generators consume them, and the committed output is byte-identical on rebuild (idempotency-tested).
  • i18n runtime — one generated static/i18n.js (UI chrome + game names for all languages), topbar language menu, /<lang>/ routes, and per-language menu labels in the search results.
  • Zone/world map overhaul — real-terrain zone maps with Base/Height/Color modes, Field Terminal + Outpost landmarks, zoom/pan with progressive disclosure; the world map shows client display names on zone labels (Shinjalar, New Virginia, Omega, …) with outlined text, and the home-page thumbnail is re-rendered from the base rendering.
  • Search — the sidenav menu tree is merged into the search index (menu labels find their pages in the current language), with full keyboard navigation (arrows/Home/End/Enter) and ARIA combobox semantics; page titles show their translated form where the menu label is a translation of the title (e.g. "World" → "Monde").
  • UI chrome — diagonal Home/Dev mode tabs (matrix theme + Dev group, persisted), a ? keyboard-shortcuts window, and a responsive topbar brand ("Open Perpetuum Wiki" / "OP Wiki").
  • Tests — 61-test Python suite for the generators (make test-py, with coverage) + expanded Playwright suite (171 passing: nav, search, zone maps, shortcuts, i18n).

Adding a new language

One file + one command — see the "Languages" section in the README: copy translations/en.json to translations/<lang>.json, translate the values (same key set; make i18n-check enforces it), run make build. The language menu, routes and search labels pick the language up automatically. File format, conventions (in-game terms stay English, placeholders) and the game-name extraction workflow are in TRANSLATIONS.md.

Notes

  • No database needed: the build uses the committed generated markdown; the .NET generator (also updated here for the zone-label change) only runs when the game DB changes.
  • CI runs make build (includes the i18n gate) and the Playwright suite.
  • In-game proper nouns (PvP/PvE, Alpha/Beta/Gamma, Training, Stronghold, currencies, item/ore/robot names, galaxy names) intentionally stay English in all languages.

clouths added 30 commits October 5, 2026 20:38
…1+3)

- translations/*.json: per-language UI strings + game names (EN + FR)
- tools/gen_i18n.py: generates static/i18n.js from the JSON files
- tools/tag_defs.py: post-build, adds data-def attrs to item/ore/zone links
- static/i18n.js: runtime language switcher (chrome + names, no reload)
- templates/base.html: data-i18n* attributes on all chrome elements
- tests/i18n.spec.js: 7 tests covering switch, persistence, aria, WIKI_UI
- Makefile: build now runs gen_i18n + tag_defs automatically
- TRANSLATIONS.md: documentation for adding/editing translations

Adding a language = drop a translations/<lang>.json + make build.
- config.toml: languages = {en, fr} (Zola multilingual mode)
- content/fr/: French content directory (served at /fr/)
  - _index.md: French home page stub
  - features/faq.md: sample translated page
- tools/gen_lang_pages.py: generates static/lang-pages.json manifest
  of available translated pages (used by the JS to decide navigation)
- static/lang-pages.json: build output (committed for CI)
- tools/gen_i18n.py: updated setLang() to navigate to the translated
  page if one exists (via lang-pages.json), otherwise chrome-only swap
- tests/i18n.spec.js: updated for navigation behavior (8 tests)

English stays in content/ (served at /), French in content/fr/ (served
at /fr/). Adding a translated page = create content/fr/<path>.md.
- robots.md: added hero card at top linking to /content/robots/
  - Wide/tall card with robot silhouettes (from OBJ convex hulls)
  - 'ROBOTS' label with border, 'Browse all robot classes' subtitle
- static/robots-silhouettes.svg: generated SVG (4 robot shapes)
- tools/gen_robot_silhouettes.py: extracts 2D silhouettes from OBJ files
  (scarab_v2, waspish, symbiont_v2, baphomet)
- style.css: .robots-hero card styles (hover glow, border, layout)
- plan.md: Phase 2 marked DONE, noted content pages are NOT auto-translated
  (by design — each page must be manually written per language)
- map.md: added .map-legend div inside .zonemap-wrap (top-right)
  - Zones: TM (cyan), ICS (green), ASI (orange), Gamma (purple)
  - Landmarks: Labels, Base, Field terminal, Teleport, Teleport path
  - Collapsible: click section header to toggle items
- style.css: .map-legend styles (positioned overlay, responsive hide)
- ZonesMapPage.cs: LegendHtml() method for regeneration
…eanup

Robots page (/content/robots/):
- Removed mermaid diagram and 'How a robot is made' section (moved to feature page)
- Robot classes table now uses icon-based ratings (⚔»■□$ × 1-5)
- Each robot card shows class ratings + size + variants
- Defense turrets removed (moved to PBS page)

Robot feature page (/features/robots/):
- Renamed from 'Robots & fitting' to 'Robot'
- Two hero cards: ROBOTS (links to data reference) + FITTING (links to fitting page)
- Added 'How a robot is made' section (moved from generated page)

Fitting page (/features/fitting/):
- New dedicated sub-page with modules, ammo, presets, slot categories

World map (/zones/map/):
- Removed intro paragraph (map + legend speak for themselves)
- Interactive legend overlay (zones by faction + landmarks)

Nav: 'Robots & fitting' → 'Robot'
Mobile: removed sidenav drawer transition animation
CSS: .class-ratings table, .robot-card-ratings, .map-legend, .robot-cards
- Replaced single 'Robots & fitting' card with two hero-style cards
- Same design as the robots feature page (silhouettes bg, border text)
- Each spans 2 columns × 2 rows in the grid
- Placed directly below the world map card
- Each rating (Combat/Speed/Armor/Storage/Cost) gets its own row
- Size moved to top-right of the card header
- Removed 'Variants' row (redundant info)
- Ratings (icons) on the left with reserved space for max 5
- Labels on the right, left-aligned, no icon prefix in text
…egend toggle

- Individual robot pages at /content/robots/<slug>/ with stats, recipe, vendor, variants
- Robot cards on /content/robots/ now link to individual pages
- Home page Robots hero card links to /content/robots/
- Robot detail page: card max 50% width, property table top-right, variants before production
- Tech tree: replaced mermaid with SVG tree diagram in group pages
- Tech tree: added category sub-items in nav (Common 1/2, Industrial, Nuimqol, PBS, Pelistal, Thelodica)
- All mermaid/diagrams: disabled interaction + hidden reset buttons
- World map: reset button hidden, legend closed by default with toggle button + backdrop
- Colorblind selector stays open after selection
- Nav test updated for expanded Research sub-list
Translate all feature, menu, and format pages to French:
- features/: 22 pages (combat, gathering, missions, production,
  research, character, items, market, robots, fitting, modules,
  transport, sparks, groups, npcs, pbs, exploration, probes,
  intrusion, outposts, social, architecture, server, ui)
- menu/: 3 pages (start, play, systems)
- formats/: 4 pages (plant_fields, ore_fields, zone_files,
  deployable_fields)
- _index.md (pre-existing)

Internal links use /fr/ prefix. Generated content (~5,537 pages)
is NOT hand-translated — game names are handled via the runtime
ClientNames translation system (Phase 3).

Also fix the i18n test: the 'no translation' test now uses
/content/items/titanium/ (a generated page) instead of
/features/getting-started/ (which now has a French translation).
Add full Spanish translation:
- translations/es.json: UI chrome translations
- content/es/: 36 hand-written content pages (features, menu, formats)
- config.toml: add [languages.es]

Internal links use /es/ prefix. Generated content (~5,537 pages)
is NOT hand-translated — game names are handled via the runtime
ClientNames translation system (Phase 3).
Add full German translation:
- translations/de.json: UI chrome translations
- content/de/: 36 hand-written content pages (features, menu, formats)
- config.toml: add [languages.de]

Internal links use /de/ prefix. Generated content (~5,537 pages)
is NOT hand-translated — game names are handled via the runtime
ClientNames translation system (Phase 3).
Add full Portuguese translation:
- translations/pt.json: UI chrome translations
- content/pt/: 36 hand-written content pages (features, menu, formats)
- config.toml: add [languages.pt]

Internal links use /pt/ prefix. Generated content (~5,537 pages)
is NOT hand-translated — game names are handled via the runtime
ClientNames translation system (Phase 3).
Add full Italian translation:
- translations/it.json: UI chrome translations
- content/it/: 36 hand-written content pages (features, menu, formats)
- config.toml: add [languages.it]

Internal links use /it/ prefix. Generated content (~5,537 pages)
is NOT hand-translated — game names are handled via the runtime
ClientNames translation system (Phase 3).
Add full Dutch translation:
- translations/nl.json: UI chrome translations
- content/nl/: 36 hand-written content pages (features, menu, formats)
- config.toml: add [languages.nl]

Internal links use /nl/ prefix. Generated content (~5,537 pages)
is NOT hand-translated — game names are handled via the runtime
ClientNames translation system (Phase 3).
Add full Russian translation:
- translations/ru.json: UI chrome translations
- content/ru/: 36 hand-written content pages (features, menu, formats)
- config.toml: add [languages.ru]

Internal links use /ru/ prefix. Generated content (~5,537 pages)
is NOT hand-translated — game names are handled via the runtime
ClientNames translation system (Phase 3).
Add full Polish translation:
- translations/pl.json: UI chrome translations
- content/pl/: 36 hand-written content pages (features, menu, formats)
- config.toml: add [languages.pl]

Internal links use /pl/ prefix. Generated content (~5,537 pages)
is NOT hand-translated — game names are handled via the runtime
ClientNames translation system (Phase 3).
Add full Czech translation:
- translations/cs.json: UI chrome translations
- content/cs/: 36 hand-written content pages (features, menu, formats)
- config.toml: add [languages.cs]

Internal links use /cs/ prefix. Generated content (~5,537 pages)
is NOT hand-translated — game names are handled via the runtime
ClientNames translation system (Phase 3).
… chrome

Add full Chinese (Simplified) translation:
- translations/zh-CN.json: UI chrome translations
- content/zh-CN/: 36 hand-written content pages (features, menu, formats)
- config.toml: add [languages.zh-CN]

Internal links use /zh-CN/ prefix. Generated content (~5,537 pages)
is NOT hand-translated — game names are handled via the runtime
ClientNames translation system (Phase 3).
…I chrome

Add full Chinese (Traditional) translation:
- translations/zh-TW.json: UI chrome translations
- content/zh-TW/: 36 hand-written content pages (features, menu, formats)
- config.toml: add [languages.zh-TW]

Internal links use /zh-TW/ prefix. Generated content (~5,537 pages)
is NOT hand-translated — game names are handled via the runtime
ClientNames translation system (Phase 3).
Add full Japanese translation:
- translations/ja.json: UI chrome translations
- content/ja/: 36 hand-written content pages (features, menu, formats)
- config.toml: add [languages.ja]

Internal links use /ja/ prefix. Generated content (~5,537 pages)
is NOT hand-translated — game names are handled via the runtime
ClientNames translation system (Phase 3).
Add full Korean translation:
- translations/ko.json: UI chrome translations
- content/ko/: 36 hand-written content pages (features, menu, formats)
- config.toml: add [languages.ko]

Internal links use /ko/ prefix. Generated content (~5,537 pages)
is NOT hand-translated — game names are handled via the runtime
ClientNames translation system (Phase 3).
Unquoted hyphenated identifiers (zh-CN, zh-TW) are invalid JavaScript
object keys, causing a SyntaxError that prevented the entire i18n.js
from parsing. This meant buildLangMenu() never ran and the language
selector showed no options.

Fix: quote the keys as "zh-CN" and "zh-TW".
- content/fr/_index.md: add proper French home page content
  (was empty — only had a title in front matter)
- content/ja/_index.md: add proper Japanese home page content
  (was truncated with </parameter> leftover)
- content/ko/_index.md: add proper Korean home page content
  (was truncated with </parameter> leftover)
- content/ja/*.md, content/ko/*.md: remove </parameter> and </invoke>
  garbage lines from 64 files that were truncated during creation
getLang() only read from localStorage, so navigating to /fr/ would
show English chrome. Now getLang() first checks the URL path for a
known language prefix (e.g. /fr/, /ja/, /zh-CN/) and uses that as
the active language, syncing it to localStorage.

This ensures the sidebar nav, topbar, footer, and all data-i18n
elements are translated when viewing a non-default language page.
- All language _index.md files: add template: "index.html" to force
  Zola to render section.content (without it, the default section.html
  template checks {% if section.content %} which was falsy for
  language root sections)
- content/fr/_index.md: full rich home page with world map card,
  robot/fitting hero cards, feature cards (translated)
- Fix corrupted front matter in 12 language files (sed artifact)
Zola does not populate section.content for language _index.md files in
multilingual mode (all fields are empty). The home cards HTML is now
rendered directly in the index.html template, which is used for ALL
language root pages. This ensures every language home page shows the
same rich card layout (world map, robot/fitting heroes, feature cards).

- templates/index.html: add home-cards div with data-i18n attributes
- content/_index.md: remove duplicate cards HTML (now in template)
- content/{lang}/_index.md: minimal front matter only (content comes
  from template, not section body)
- translations/*.json: add home_* keys (13 languages)
- tests/i18n.spec.js: new test 'translated home page has rich HTML
  cards' verifies /fr/ has map card, 2 hero cards, 10+ feature cards,
  data-i18n attributes, and world-map-thumb.png image
clouths added 22 commits October 6, 2026 13:27
Two issues fixed:
1. Nav links always pointed to English (/features/...) even when on a
   translated page. Now i18n.js rewrites internal links to include the
   language prefix (e.g. /fr/features/combat/) when the target page
   exists in the current language (checked against lang-pages.json).

2. Flash of English chrome: the HTML renders with English text from
   config.extra.ui, then JS swaps it. Now the head script detects the
   language from the URL and sets data-i18n-pending on <html>. CSS hides
   [data-i18n] elements (opacity:0) until i18n.js applies the translation
   and removes the attribute.

Changes:
- static/i18n.js: add rewriteLinks() + i18nReady(), make loadLangPages()
  return a Promise, new init() function
- static/style.css: html[data-i18n-pending] [data-i18n] { opacity: 0 }
- templates/base.html: head script detects lang from URL path, sets
  data-i18n-pending attribute
- tests/i18n.spec.js: 2 new tests (link rewriting + no-flash)
Hybrid builds and NPC units now use the same .robot-card ID-card
layout as player robots. Since the DB has no volume/mass/class bits
for these (they are placeholder definitions — hybrids are assembled
from parts by the player, NPC units are not acquirable), the cards
show '?' with a descriptive label instead of guessed stat ratings:
- Hybrids: '? Stats vary by parts'
- NPC: '? NPC unit — not player-acquirable'

Also fix: tools/gen_i18n.py now quotes hyphenated language codes
(zh-CN, zh-TW) as JS object keys to prevent SyntaxError.
Detect placeholder robots (no class bits + volume=0) in RobotPages.cs.
Their individual pages now show:
- Size: — (not 0)
- Stats: ? with 'Stats vary by parts' (not fake 3/3/3/2/3)
- Class/Size/Mass/Health in props table: — (not 0/100)

Affects: 27 hybrid pages + 1 NPC unit page.
The C# make generate step overwrote the zone SVGs with raw base output.
Re-ran the Python post-processor (make zonemaps) to restore terrain
images, render mode support, and proper exit lines for all 80 zones.
- Remove the connected-only filter: all zones are now plotted (including
  Daoden, training, PvP arena, strongholds, and other zones without TP
  connections) to mirror the in-game world map layout
- Add a dark blue background rect (#0a1628) matching the in-game ocean
- Exclude sentinel rows (x/y >= 49000) from the plotted set

Verified against in-game screenshot: zone positions match (Alpha cluster
bottom-center, Daoden far-left, SH zones bottom-right, Gamma top, Beta middle).
Second (outer) outline on each zone island, colored by protection level:
- PVE zones (protected, _pve, training): no extra outline
- PvP Beta zones + Gamma TC zones: yellow (#fbbf24)
- PvP Gamma zones: red (#ef4444)

The outline is a 2px-offset rect/circle with 1.5px stroke at 0.9 opacity,
rendered behind the main island shape.
Protection level outer rings (visible via CSS fix):
- Red ring: Gamma PvP zones (terraformable)
- Yellow ring: Beta PvP + Gamma TC zones
- No ring: PVE/Alpha zones
- CSS fix: .zonemap circle/rect stroke rule now excludes .prot-ring

Legend reworked:
- New top section: Protection (Alpha/Beta/Gamma with color swatches)
- Zones section renamed to Galaxies

Zoom improvements:
- Max zoom increased from 8x to 12x
- Vertical zoom slider (draggable, right side, 'ZOOM' title)
- Progressive disclosure support (data-zoom-level attribute)

Cleanup:
- Removed gate/stronghold exit links (dashed yellow lines)
- Removed ASI_/TM_ zones without TP connections (isolated duplicates)
Instead of a separate ring element, the zone shape is now slightly
larger than the terrain image (2.5px pad). Its fill is the protection
color, creating a visible border:
- Red fill: Gamma PvP zones
- Yellow fill: Beta PvP + Gamma TC zones
- Background fill (#0a1628): PVE/Alpha zones (invisible border)

Removed the .prot-ring separate element and its CSS exclusion.
Updated CSS to exclude .zonemap-node-* elements from the default
stroke rule, keeping all zone outlines consistent.
…ang fix

- TP hexagon icons rendered at each teleport column's tile coordinates
- Home page hero cards moved above content paragraph
- Hover: green drop-shadow ring replaces neon stroke; radius-based zone detection
- i18n: fix data-i18n-pending never removed, add URL-based getLang(), skip empty strings
- Zone fill: island interior painted with galaxy color (flood-fill inside test)
- Max zoom increased to 48x (doubled from 24)
- Zoom slider moved to bottom-left corner
- Legend toggle button (shows/hides the legend panel)
- i18n: fix gen_i18n.py template (i18nReady, getLangFromUrl, truthy checks)
- French: add all home card translations (25 keys)
- i18n.js regenerated from translations/*.json
…nique clip paths

- Removes pixel downsampling for crisp contour mapping

- Inlines height.svg and color.svg directly into the zone DOM to bypass browser <image> rendering restrictions

- Scopes clip-path IDs to prevent multi-map intersection collisions

- Applies protection color to zone borders dynamically
…tor test suite

- translations: full key parity (510 keys) across all 13 languages; every
  translatable prose string on generator-managed pages now translated
  (robots, ores, missions, extensions, shop, techtree, zones, deployables,
  plants, stat reference, recipes, content index, world-map legend/tooltips,
  plants growth-cycle mermaid diagram)
- generators: consume translations/*.json via t() lookup (new
  tools/i18n_common.py); tools/zones_i18n_data.py deleted; gen_lang_pages.py
  no longer depends on relative paths (bug: an absolute content dir produced
  a manifest under a bogus language key); world-map "Zone listings" line
  rebuilt with translated link labels (was English in all 13 languages)
- CI gate: tools/i18n_check.py (key parity, {placeholder} consistency, no
  empty values) runs as a make build prerequisite
- search: sidenav menu tree (tools/gen_menu_tree.py -> static/menu_tree.json)
  merged into results; ARIA combobox keyboard navigation (Up/Down/Home/End/
  Enter, hover sync); / focuses the search box (placeholder hint: / or
  Ctrl+F or Cmd+F)
- zone map: runtime landmarks panel + reset-zoom title now translated
  (9 new WIKI_UI keys wired by applyLang; EN pre-JS defaults in config.toml)
- tests: tests_py/ Python suite for the generators (unit + fixture +
  in-place idempotency; make test-py with coverage, 92-100% on the
  i18n tooling), tests/search.spec.js (10 new browser tests), landmarks
  panel i18n spec; .coveragerc + .gitignore entries
- docs: README test targets; TRANSLATIONS.md updated (142 ui keys,
  zonemap_*, test workflow, plants diagram)
…l, translated search titles

- Diagonal Home/Dev mode tabs (matrix theme + Dev group, persisted with
  no-flash restore); palette selector locked in dev mode with a
  'click Home' tooltip (new aria_palette_dev key, 14 languages, 152 ui keys)
- '?' keyboard-shortcuts window (centered in the content area, 7 rows,
  2-col grid when it fits, Esc/click/✕ to close)
- Responsive topbar brand: 'Open Perpetuum Wiki' / 'OP Wiki' at 801px
- World map zone labels use client display names where the client named
  the zone (Shinjalar, New Virginia, Omega, ...); font sizes recomputed
  with the generator formula; labels get a black outline for contrast
  (ZonesMapPage.cs changed to match for future DB regenerations)
- World map thumbnail re-rendered with the base rendering: islands, TP
  dash lines, local TPs and labels only (chrome hidden); fixed framing
  (strongholds/arena/training excluded from the alpha box, live CTM)
  and a stroke override for the Chromium non-scaling-stroke quirk
- Search results show the translated page title when the menu label is
  a translation of it (world map: 'World' -> 'Monde'/'Welt'/...)
- Tests: extended dev-mode test (palette lock), translated-title search
  test, shortcuts spec — 171 JS + 61 Python pass
@clouths clouths self-assigned this Oct 14, 2026
Sellafield
Sellafield previously approved these changes Oct 14, 2026
…esh codeql-action pins

- CI ran gen_zone_teleport_maps.py with plain python3, but the FBM
  world-thumb fallback (zones whose layers only ship in the game client)
  lazily imports numpy + matplotlib — the same deps the Makefile zonemaps
  target provides via uv. Install them in both jobs' terrain steps.
- Ratchet: the github/codeql-action pins had drifted from v3.
…svg for meta-only zones

The world map (map.md, all 14 language copies) references
/zonemaps/<slug>/{height,color}.png for 53 zones whose layer data only
ships in the game client, plus /zonemaps/zone-strghld-{71,72}/thumb.svg
for two strongholds that have no teleport map of their own. CI only
fetches the public layer archive, so none of those 110 references
resolved and the links test failed.

- Commit height.png + color.png fallback pairs for the 53 client-only
  zones in static/zonemaps-fallback/ (the existing mechanism — 20
  grid-zone pairs were already committed; these zones simply lacked
  theirs).
- The meta-only branch (zones without their own SVG) now also writes
  thumb.svg — the world map references it, but only world.png +
  thumb.png were generated.
…ps public/

The world map (map.md, all language copies) references
/zonemaps/<slug>/{height,color}.png, but the tool's REAL branch — taken
in CI for the 53 client-only zones, whose layers ARE in the public
archive — only wrote height.svg/color.svg (the PNG output was dropped
by the inline-SVG refactor; stale local PNGs from before the refactor
masked it). The FBM branch wrote no PNGs either.

- real branch: write height.png + color.png alongside the SVGs
- FBM branch: write the procedural heightmap as both PNGs, so the
  references resolve even for zones with no layer data at all
- drop the 53 committed fallback pairs from the previous commit: they
  are never used (those zones have public layers); the pre-existing 20
  pairs (grid zones, arena, strghld-70) stay as the real fallback

make serve: zola serve writes the site's ASSETS into its output dir
(default: public/), wiping a previous 'make build' output on start.
Point it at a scratch dir (/src/.serve-public) so public/ is left
untouched; 'make clean' removes the scratch dir too.
@clouths
clouths requested a review from Sellafield October 14, 2026 06:12

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants