Skip to content

fix(core): redact grouped credit cards before phone numbers - #504

Merged
loveRhythm1990 merged 2 commits into
matrixorigin:mainfrom
loveRhythm1990:fix/sensitivity-card-before-phone
Nov 17, 2026
Merged

loveRhythm1990 merged 2 commits into
matrixorigin:mainfrom
loveRhythm1990:fix/sensitivity-card-before-phone

Conversation

@loveRhythm1990

Copy link
Copy Markdown
Collaborator

Fixes #495

Problem

The MEDIUM tier applies email → phone → ssn → credit_card in order. The phone regex matches a 6-4 digit sub-group inside a hyphen- or dot-grouped card, so card 3056-930902-5904 on file was stored as card 3056-[phone] on file. The BIN stayed in the clear and the card was never labeled. Space grouping already worked because phone only accepts - and ..

Change (memoria-core/src/sensitivity.rs)

  1. Order: credit_card now runs before phone, as suggested in the issue.
  2. Dot grouping: the Luhn card scanner only accepted spaces and hyphens, so reordering alone would still have left 3056.930902.5904 as 3056.[phone]. It now accepts . as a separator, but only after a 4 to 6 digit group. Real grouped cards fit that, and fractional timestamps like 1730284800.123456 do not get scanned.
  3. Neighbouring phone or SSN: with cards first, 555-867-5309 4111 1111 1111 1111 matched a 15 digit Luhn-valid span that started at the phone number. That redacted the phone plus part of the card and left 12 digits of the card in clear text. I confirmed this with a probe before adding the guard. Card candidates now cannot start or end inside a phone or SSN match. A card that merely contains one, like the 4-6-4 grouping, is still accepted.

Tests

On the original source, card_redaction_is_not_preempted_by_phone_for_any_grouping and phone_still_redacted_next_to_a_card fail. The adjacency test passes there, because the old order happened to shield it. It guards the reorder.

Notes

  • matched_labels order follows pattern order, so a text with both now lists credit_card before phone. The existing contract test for label order only covers email, phone and SSN and is unchanged.
  • A dotted run with 4-6 digit groups that passes Luhn is now redacted as a card. That is the behavior the issue asks for, with a roughly 1 in 10 chance for any other 13 to 19 digit dotted number of that shape.

Verification

  • cargo test -p memoria-core: 59 unit, 12 contract and 1 doc tests pass.
  • cargo clippy -p memoria-core --all-targets: no new warnings. The one warning, manual_is_multiple_of, is in existing code.

…igin#495)

The phone pattern matched 6-4 digit sub-groups inside hyphen- or dot-grouped
card numbers, so 3056-930902-5904 was stored as 3056-[phone]: the BIN leaked
and the card was never detected.

- Run the credit_card rule before phone in the MEDIUM sequence.
- Accept '.' as a card group separator, only after a 4-6 digit group so
  fractional timestamps such as 1730284800.123456 are not scanned.
- Keep card candidates from starting or ending inside a phone or SSN match.
  Without this, moving cards first lets 555-867-5309 4111 1111 1111 1111
  absorb 15 digits as one Luhn-valid span and leave the rest of the card.
@mergify

mergify Bot commented Nov 16, 2026 •

Copy link
Copy Markdown
Contributor

This pull request is merged.

@loveRhythm1990 loveRhythm1990 left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Request changes — one reproducible redaction regression remains at c8959cf.

The issue in #495 is real, and this patch fixes its hyphen/dot examples. However, the new dot separator handling can join two separate cards across sentence punctuation and leave the second card partially exposed. See the inline finding.

Validation: independently reviewed both changed files and the service consumer (2/2 files, 100% coverage, none skipped). Rust 1.85.0: all 59 existing unit tests, 12 contract tests and the diagnostic probe pass. My additional regression assertion passes on base b155eb0 and fails on this PR. No database writes were needed.

Non-blocking: tests/zz_probe.rs only prints results and cannot detect incorrect redaction. Please turn its useful cases into assertions or remove the temporary probe.

GitHub does not allow the PR author to submit a formal REQUEST_CHANGES review, so this is posted as COMMENT. The finding above is blocking; this is not an LGTM.

// Dot-grouped cards (3056.930902.5904) use 4-6 digit groups. Requiring
// that keeps fractional timestamps such as 1730284800.123456 out.
if !(4..=6).contains(&group_len) {
break;

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P2] Do not join independent cards across a sentence-ending period

The new dot branch checks only the preceding group length, then allows a space before continuing into another number. With this synthetic input:

card 3056 930902 5904. 5553 0000 0000 0000 on file

both complete numbers pass Luhn. Base produces card [card]. [card] on file, but this head produces card [card] 0000 0000 0000 on file.

The first 14 digits plus the next card's first four digits form a Luhn-valid 18-digit candidate. Longest-at-the-same-start selection chooses that candidate; the actual second card overlaps it and is discarded, exposing its remaining 12 digits. The phone/SSN protected spans do not address this case. The service uses this redacted string for embedding and storage, so the residual digits can persist.

Preserve punctuation boundaries (in particular a period followed by whitespace), while retaining support for actual dot-grouped cards. Please add the exact full-output assertion above; I ran it against both base and PR with Rust 1.85.0.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 31f54c3.

A dot now has to be followed directly by a digit, and dots are never mixed with spaces or hyphens inside one candidate, so a sentence-ending period can no longer continue into the next number. Dot-grouped cards (3056.930902.5904) still work.

Added your exact case as a full-output assertion (card 3056 930902 5904. 5553 0000 0000 0000 on file -> card [card]. [card] on file), plus the dotted and no-space variants. On the previous logic that test fails with card [card] 0000 0000 0000 on file, matching your report.

I also removed tests/zz_probe.rs. It was a leftover probe that I committed by mistake. Its useful cases are now assertions in sensitivity.rs.

The dot separator only checked the preceding group length, then allowed a
space and a following number. With '3056 930902 5904. 5553 0000 0000 0000'
the first 14 digits plus four digits of the next card formed a Luhn-valid
18-digit run that won over the real second card and left 12 of its digits.

A dot must now be followed directly by a digit, and dots are never mixed
with spaces or hyphens inside one card. Dot-grouped cards keep working.

Also remove the temporary tests/zz_probe.rs that was committed by mistake,
and turn its useful cases into assertions.

@loveRhythm1990 loveRhythm1990 left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM at 31f54c3. The blocking punctuation-boundary regression reported above is fixed.

I independently reviewed the complete current diff and candidate-selection logic. The exact two-card reproduction now preserves the period and fully redacts both numbers. Requiring a digit immediately after a dot and separating dotted grouping from space/hyphen grouping addresses that regression while retaining the original #495 fixes. The print-only probe has also been removed.

Validation on Rust 1.85.0: 61 core unit tests and 12 contract tests pass, including the exact regression assertion. An additional independent matrix of 294 synthetic cases passes: 13–19 digit cards, ASCII/fullwidth digits, space/hyphen/dot grouping, adjacent phone/SSN in both orders, and period-plus-space/tab boundaries.

Coverage: 1/1 current changed files reviewed, 100%, none skipped; the removed probe was also checked. No further blocking findings. The head is conflict-free against current main. DB Tests are still running; this code LGTM does not imply that all CI has finished.

@loveRhythm1990
loveRhythm1990 merged commit 7ba6586 into matrixorigin:main Nov 17, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: phone redaction consumes digit groups inside hyphen/dot-grouped card numbers, leaving '3056-[phone]' (BIN leaked, card never detected)

2 participants