Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 0 additions & 9 deletions docs/developer/api.md
Original file line number Diff line number Diff line change
Expand Up @@ -322,15 +322,6 @@ other HRP. `hrp` is a case-insensitive string stored in lowercase. The context f
- `immutable_prefix`, program-supplied text outside the correction domain; and
- `excluded_indices`, ordinary share indices already accepted in a recovery.

Applications restoring a share set can call
`CorrectionContext.from_shares(accepted, hrp=None, expected_length=None, immutable_prefix=None)`.
It derives the fixed length, complete header prefix, and excluded indices from
compatible, validated ordinary shares. An explicit length, prefix, or HRP may
only agree with those shares. With no accepted shares, `hrp` is required and
optional length and prefix hints have the same meaning as the direct constructor.
The CLI uses this factory for interactive recovery and standalone correction;
graphical and other clients can share that constraint logic.

With or without `expected_length`, `ms` correction searches only valid
master-seed lengths reachable by a supported structural family. Another HRP
considers character offsets -4 through +4 and group offsets -8/-4/0/+4/+8,
Expand Down
8 changes: 5 additions & 3 deletions docs/security/invariants.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,9 +3,11 @@
These rules are mandatory. The [security model](model.md) defines their limits
and evidence.

1. Constructing a `Codex32String` or `MasterSeed` validates its text. Generic container
and header checks precede the checksum; the `ms` length rule follows it.
Other HRPs remain generic and never gain invented application semantics.
1. Constructing a `Codex32String` or `MasterSeed` validates its text;
`MasterSeed.from_seed` and the private `_from_unchecksummed_string()`
worksheet helper build text only to construct one. Generic container and
header checks precede the checksum; the `ms` length rule follows it. Other
HRPs remain generic and never gain invented application semantics.
2. APIs accept only immutable validated artifacts. Outputs are reparsed;
recovery and derivation reject sets with mismatched normalized HRPs or shapes.
3. Shared creation uses a separate OS-CSPRNG call for each random initial share,
Expand Down
33 changes: 29 additions & 4 deletions docs/security/model.md
Original file line number Diff line number Diff line change
Expand Up @@ -114,6 +114,31 @@ complete explicit `ms1` string.

## Creation, sharing, and recovery controls

`ms32 checksum` generates a checksum only for new Codex32 Book worksheet
data. It is separate from `check` and `correct` and refuses input of any complete
`ms` string length, so a damaged complete string of a valid length is not taken
for a longer unchecksummed payload. This narrows the ambiguity without removing
it: a complete string that has lost or gained characters can still reach the
accepted length, and nothing can tell it from new data. The non-pink worksheet
data is entered twice, with the terminal cleared before the second entry, and
the entries must match before anything is displayed. Each entry must be ASCII
and entirely uppercase or lowercase. The full completed string is then displayed so the
operator can copy its checksum into the pink squares. After the operator presses Enter,
the terminal and scrollback are cleared where supported and only the pink
checksum squares must be re-entered, without editable prefill. These entries
catch typing errors; they cannot detect an error already written on the
worksheet. A mismatch, EOF, or interruption releases no completed string to
stdout, although the preview has already been displayed on stderr. Format errors
give no detail. `--plain` preserves the first entry's case; default terminal
output uses the normal card formatting. Plain or redirected output does not skip
checksum confirmation; noninteractive input is rejected before it is read.
Agreement cannot establish how the worksheet data was generated or detect an
error already present in its non-pink data. Terminal clearing remains best effort.
The private `_from_unchecksummed_string()` helper in `bip93.py` is the only
checksum construction from worksheet data. It applies the same parsing rules
for any HRP, plus the `ms` payload rules for `ms`, but has none of these
interactive safeguards, so every façade that uses it must provide them.

Fresh shared creation generates *k* random initial shares. Each uses a separate
full-payload OS-CSPRNG request. The current share string must be re-entered
exactly, ignoring case and whitespace, before the next request. Confirmation
Expand Down Expand Up @@ -174,10 +199,10 @@ cancelled operations are removed only with a coverage proof. Pruning does not
reallocate the ledger's mass to additional hypotheses.

`codex32` checks, corrects, recovers, and derives shares for any HRP. `ms32`
applies an `ms` input filter and alone owns Bitcoin creation, xprv, and wallet
commands. Both façades offer worksheet-residue correction and a known
`--prefix`; only `ms32 correct` offers `--bytes`, and only `codex32 correct`
offers `--length`.
applies an `ms` input filter and alone owns Bitcoin creation, checksum
generation, xprv, and wallet commands. Both façades offer worksheet-residue
correction and a known `--prefix`; only `ms32 correct` offers `--bytes`, and
only `codex32 correct` offers `--length`.

### Low-discrimination correction disclosure

Expand Down
21 changes: 19 additions & 2 deletions docs/user/guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -281,14 +281,31 @@ follow Bitcoin Core v32's maintained
confirmed.” `--plain` skips card confirmation. Redirected damaged input can
still require an interactive correction confirmation.

### Advanced: Codex32 Book checksum worksheet

`ms32 checksum` computes the checksum for new data from the
[Codex32 Book](https://secretcodex32.com) checksum worksheet, instead of
computing it by hand. To verify a hand-computed checksum, enter the complete
string in `ms32 check`. `ms32 checksum` is not a validation or repair tool: it refuses
complete strings, and you must never use it to replace the checksum of an
existing backup. A checksum cannot make weak input secure, so use only data
from the Book's dice procedure. The command needs an interactive terminal,
asks for the non-pink worksheet data twice, clearing the screen between
entries, and then shows the full completed string. Copy the checksum into the
pink squares and press Enter; the screen clears, then you type back only the
pink checksum squares. The repeated entries catch typing errors, not errors
already on the worksheet, so check the worksheet carefully first. Each entry
must be entirely uppercase or lowercase. `--plain` preserves the first entry's
case; the default terminal output uses the usual uppercase card layout.
The completed string reaches stdout only after the checksum is confirmed.

### Advanced: completing missing trailing characters

The correction command can fill trailing erasures when the final characters
are actually unknown. For a normal codex32 string, replace the final 13 missing
characters with `?`; a long-checksum string uses 15. Thirteen genuinely
unreadable characters at the end of an existing backup are therefore a valid
recovery case. The same mechanism can complete newly generated worksheet data
whose final 13 squares have not yet been filled.
recovery case. To checksum new Codex32 Book data instead, use `ms32 checksum`.

Never intentionally delete or replace the existing final characters from a
backup that merely fails validation. Doing so can hide a transcription or
Expand Down
8 changes: 2 additions & 6 deletions src/codex32/_cli_input.py
Original file line number Diff line number Diff line change
Expand Up @@ -341,13 +341,9 @@ def _corrections(
if isinstance(byte_length, int):
target = _text_length(byte_length)
immutable = prefix if prefix and value.lower().startswith(prefix.lower()) else value[: separator + 1]
excluded = tuple(artifact.share_idx for artifact in accepted)
context = CorrectionContext(hrp, target, _prefix_for_suffix(immutable, value), excluded)
try:
context = CorrectionContext.from_shares(
accepted,
hrp=hrp,
expected_length=target,
immutable_prefix=_prefix_for_suffix(immutable, value),
)
candidates, complete = _correct_complete(context, value, allowed=allowed, competitors=True)
except InvalidCorrectionInput:
return (), True
Expand Down
8 changes: 8 additions & 0 deletions src/codex32/_cli_parser.py
Original file line number Diff line number Diff line change
Expand Up @@ -195,6 +195,14 @@ def parser(prog: str = "codex32", *, master_seed: bool = False) -> argparse.Argu
help="use an existing Bitcoin codex32 secret or hexadecimal seed",
)

checksum = _command(commands, "checksum", "generate the checksum for a new Codex32 Book worksheet")
checksum.description = (
"Generate the checksum for new data on a Codex32 Book checksum worksheet. "
"Never use it on an existing backup: use 'ms32 check' or 'ms32 correct'. "
"Requires an interactive terminal and re-entry of the pink checksum squares."
)
_terminal_output(checksum)

wallet = _command(commands, "wallet", "restore a Bitcoin Core wallet")
_wallet_options(wallet)

Expand Down
16 changes: 16 additions & 0 deletions src/codex32/bip93.py
Original file line number Diff line number Diff line change
Expand Up @@ -206,6 +206,22 @@ def _from_parts(hrp: str, header: str, payload: Sequence[int], *, uppercase: boo
return parse_codex32(text.upper() if uppercase else text)


def _from_unchecksummed_string(unchecksummed_text: str) -> Codex32String:
"""Add the checksum to new worksheet data, such as a Codex32 Book checksum worksheet.

Never use this on an existing backup: it turns transcription errors into a
valid string. A valid checksum does not establish safe entropy.
"""
hrp, body = bech32_decode(unchecksummed_text)
header = _u5_to_chars(body[:6])
if len(header) != 6:
raise InvalidLength("codex32 header must contain six symbols")
if header[0] not in "023456789":
raise InvalidThreshold(f"The threshold must be 0 or a number from 2 through 9; found {header[0]!r}.")
header = _header(int(header[0]), header[1:5], header[5])
return _from_parts(hrp, header, tuple(body[6:]), uppercase=unchecksummed_text.isupper())
Comment thread
BenWestgate marked this conversation as resolved.


def _lagrange_weights(points: tuple[int, ...], target: int) -> tuple[int, ...]:
weights: list[int] = []
for position, point in enumerate(points):
Expand Down
64 changes: 64 additions & 0 deletions src/codex32/cli.py
Original file line number Diff line number Diff line change
Expand Up @@ -31,18 +31,23 @@
from codex32._cli_input import read_artifacts as _artifacts
from codex32._cli_input import read_text as _text
from codex32._cli_parser import parser as _parser
from codex32.bech32 import _validate_single_case_ascii
from codex32.bip93 import (
IDX_SORT,
MS,
Codex32String,
MasterSeed,
_from_unchecksummed_string,
_header,
_normalize_target,
_text_length,
derive_share,
parse_codex32,
recover_secret,
)
from codex32.bip93 import (
TEXT_LENGTHS as _MS_TEXT_LENGTHS,
)
from codex32.correction import (
CorrectionContext,
_best,
Expand Down Expand Up @@ -522,6 +527,63 @@ def _create(
)


def _worksheet_text(prompt: str) -> str:
value = _text(prompt, prompt_end=":\n\n> MS1")
return value if "1" in value else ("MS1" if value.isupper() else "ms1") + value


def _same_entry(entered: str, expected: str) -> bool:
try:
_validate_single_case_ascii(entered)
except CodexError:
return False
return entered.lower() == expected.lower()


def _checksum(plain: bool) -> int:
# Generation only: complete strings belong to check and correct (codex32 discussion #78).
if not sys.stdin.isatty():
raise _UsageError("Checksum generation requires an interactive terminal.")
warning = (
"DANGER: Incorrect input can cause permanent loss of funds.\n"
"Use this only for new data from the Codex32 Book and its Dice De-biasing Worksheet.\n"
"Never use it to replace the checksum of an existing backup. Use 'ms32 check' or 'ms32 correct'.\n"
"Check every non-pink square on the worksheet; errors written there cannot be detected here.\n"
"This command cannot verify how your data was generated.\n"
"Do not enter raw dice rolls, seed words, hexadecimal seeds, or passwords.\n"
)
_print(warning, err=True, danger=True)
text = _worksheet_text("Checksum worksheet non-pink bold squares")
if len(text) in _MS_TEXT_LENGTHS:
raise _UsageError("This command does not accept complete strings. Use 'ms32 check' to check one.")
try:
# The Book's checksum worksheet has 32 non-pink squares after the ms prefix.
if len(text) != 35:
raise ValueError
artifact = _from_unchecksummed_string(text)
if artifact.hrp != MS:
raise ValueError
except (CodexError, ValueError) as error:
raise _UsageError(
"The worksheet input is not in the expected format.\n"
"Consult the Codex32 Book and check the non-pink bold squares."
) from error
clear = "\x1b[3J\x1b[2J\x1b[H" if sys.stderr.isatty() else ""
if not _same_entry(_worksheet_text(clear + "Re-enter the worksheet non-pink bold squares"), text):
raise _CommandError("Entries do not match. Restart and re-enter the worksheet.")
_emit(artifact, plain, err=True)
_text(
"Copy the checksum into the worksheet's pink squares, then press Enter",
optional=True,
prompt_end=". ",
)
entered = _text(clear + "Re-enter only the pink checksum squares", prompt_end=":\n> ")
if not _same_entry(entered, artifact.checksum):
raise _CommandError("Checksum not confirmed. Check the pink squares before using this backup.")
_emit(artifact, plain)
return 0


def _correct(
residue: bool,
erasures: tuple[int, ...],
Expand Down Expand Up @@ -687,6 +749,8 @@ def _dispatch(arguments: argparse.Namespace, context: _CliContext) -> int:
cast(str | None, arguments.indices),
bool(arguments.existing),
)
if command == "checksum":
return _checksum(plain)
if command == "correct":
return _correct(
bool(arguments.residue),
Expand Down
43 changes: 0 additions & 43 deletions src/codex32/correction.py
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,6 @@
from collections.abc import Callable, Sequence
from dataclasses import dataclass, field, replace
from functools import cache, lru_cache
from itertools import islice
from math import comb
from time import monotonic
from typing import Literal
Expand All @@ -45,7 +44,6 @@
_has_generation_padding,
_header,
_validate_ms_length,
_validate_recovery_prefix,
parse_codex32,
)
from codex32.checksums import _CODEX32, _CODEX32_LONG, _Checksum
Expand Down Expand Up @@ -82,47 +80,6 @@ def __init__(
object.__setattr__(self, "immutable_prefix", immutable_prefix)
object.__setattr__(self, "excluded_indices", excluded_indices)

@classmethod
def from_shares(
cls,
accepted: Sequence[Codex32String],
*,
hrp: str | None = None,
expected_length: int | None = None,
immutable_prefix: str | None = None,
) -> "CorrectionContext":
"""Constrain a correction using compatible, previously accepted ordinary shares.

When no shares have been accepted, ``hrp`` identifies the application.
Supplied length and prefix hints may narrow, but never contradict, a set.
"""
shares = tuple(islice(accepted, 10))
if shares:
try:
_validate_recovery_prefix(shares)
except (CodexError, TypeError, ValueError) as error:
raise InvalidCorrectionInput("accepted shares must be compatible ordinary shares") from error
first = shares[0]
if hrp is not None and (not isinstance(hrp, str) or hrp.lower() != first.hrp):
raise InvalidCorrectionInput("hrp conflicts with accepted shares")
if expected_length is not None and expected_length != len(first.s):
raise InvalidCorrectionInput("expected_length conflicts with accepted shares")
header_prefix = f"{first.hrp}1{first.k}{first.ident}"
if immutable_prefix is not None and (
not isinstance(immutable_prefix, str)
or not header_prefix.startswith(immutable_prefix.lower())
):
raise InvalidCorrectionInput("immutable_prefix conflicts with accepted shares")
if immutable_prefix is not None and immutable_prefix.isupper():
header_prefix = header_prefix.upper()
context = cls(first.hrp, len(first.s), header_prefix, tuple(share.share_idx for share in shares))
else:
if hrp is None:
raise InvalidCorrectionInput("hrp is required without accepted shares")
context = cls(hrp, expected_length, immutable_prefix)
_validate_context(context)
return context


@dataclass(frozen=True, slots=True)
class CorrectionEdit:
Expand Down
2 changes: 1 addition & 1 deletion tests/data/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@
than `ms`.
- `p70_correction_vectors.json` is the frozen PR #70 correction corpus.
- `malformed_inputs.json` freezes independent rejection cases for parsing,
interpolation, correction, and CLI tokenization.
checksum completion, interpolation, correction, and CLI tokenization.
- `wallet_fingerprints.json` freezes BIP32 master fingerprints derived by real
Bitcoin Core. `tools/_wallet_test_vectors.py` owns lookup/stub behavior, while
`tools/bitcoin_core_regtest.py` independently verifies every frozen value.
Expand Down
7 changes: 7 additions & 0 deletions tests/data/malformed_inputs.json
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,13 @@
{"id": "non-ascii", "text": "ms10testsxxxxxxxxxxxxxxxxxxxxxxxxxx4nzvca9cmczlé"},
{"id": "embedded-newline", "text": "ms10testsxxxxxxxxxxxxxxxx\nxxxxxxxxxx4nzvca9cmczlw"}
],
"checksum_completion": [
{"id": "invalid-symbol", "text": "ms10testsxxxxxxxxxxxxxxxxxxxxxxxxx!"},
{"id": "mixed-case", "text": "Ms10testsxxxxxxxxxxxxxxxxxxxxxxxxxx"},
{"id": "short-ms", "text": "ms10testsqq"},
{"id": "invalid-threshold", "text": "zz1xtestsqqqqqqqqqqqqqq"},
{"id": "short-header", "text": "zz10tes"}
],
"interpolation": [
{
"id": "duplicate-index",
Expand Down
16 changes: 16 additions & 0 deletions tests/test_bip93.py
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,7 @@
recover_secret,
)
from codex32.bech32 import _chars_to_u5, convertbits
from codex32.bip93 import _from_unchecksummed_string
from codex32.errors import CodexError, InvalidChecksum


Expand Down Expand Up @@ -146,3 +147,18 @@ def test_generic_parser_accepts_checksum_valid_non_ms_hrps(value: str) -> None:
artifact = parse_codex32(value)
assert artifact.hrp in {"m", "s"}
assert not isinstance(artifact, MasterSeed)


@pytest.mark.parametrize(
("text", "checksum_length"),
[
(VECTOR_1["secret_s"], 13),
(VECTOR_2["share_A"], 13),
(VECTOR_3["share_a"], 13),
(VECTOR_4["secret_s"], 13),
(VECTOR_5["secret_s"], 15),
(VECTOR_6["codex32_luea"], 13),
],
)
def test_from_unchecksummed_string_restores_vector_checksums(text: str, checksum_length: int) -> None:
assert _from_unchecksummed_string(text[:-checksum_length]).s == text
Loading
Loading