Repository navigation
fix(core): redact grouped credit cards before phone numbers - #504
loveRhythm1990 merged 2 commits into
Conversation
…igin#495) The phone pattern matched 6-4 digit sub-groups inside hyphen- or dot-grouped card numbers, so 3056-930902-5904 was stored as 3056-[phone]: the BIN leaked and the card was never detected. - Run the credit_card rule before phone in the MEDIUM sequence. - Accept '.' as a card group separator, only after a 4-6 digit group so fractional timestamps such as 1730284800.123456 are not scanned. - Keep card candidates from starting or ending inside a phone or SSN match. Without this, moving cards first lets 555-867-5309 4111 1111 1111 1111 absorb 15 digits as one Luhn-valid span and leave the rest of the card.
|
This pull request is merged. |
loveRhythm1990
left a comment
There was a problem hiding this comment.
Request changes — one reproducible redaction regression remains at c8959cf.
The issue in #495 is real, and this patch fixes its hyphen/dot examples. However, the new dot separator handling can join two separate cards across sentence punctuation and leave the second card partially exposed. See the inline finding.
Validation: independently reviewed both changed files and the service consumer (2/2 files, 100% coverage, none skipped). Rust 1.85.0: all 59 existing unit tests, 12 contract tests and the diagnostic probe pass. My additional regression assertion passes on base b155eb0 and fails on this PR. No database writes were needed.
Non-blocking: tests/zz_probe.rs only prints results and cannot detect incorrect redaction. Please turn its useful cases into assertions or remove the temporary probe.
GitHub does not allow the PR author to submit a formal REQUEST_CHANGES review, so this is posted as COMMENT. The finding above is blocking; this is not an LGTM.
| // Dot-grouped cards (3056.930902.5904) use 4-6 digit groups. Requiring | ||
| // that keeps fractional timestamps such as 1730284800.123456 out. | ||
| if !(4..=6).contains(&group_len) { | ||
| break; |
There was a problem hiding this comment.
[P2] Do not join independent cards across a sentence-ending period
The new dot branch checks only the preceding group length, then allows a space before continuing into another number. With this synthetic input:
card 3056 930902 5904. 5553 0000 0000 0000 on file
both complete numbers pass Luhn. Base produces card [card]. [card] on file, but this head produces card [card] 0000 0000 0000 on file.
The first 14 digits plus the next card's first four digits form a Luhn-valid 18-digit candidate. Longest-at-the-same-start selection chooses that candidate; the actual second card overlaps it and is discarded, exposing its remaining 12 digits. The phone/SSN protected spans do not address this case. The service uses this redacted string for embedding and storage, so the residual digits can persist.
Preserve punctuation boundaries (in particular a period followed by whitespace), while retaining support for actual dot-grouped cards. Please add the exact full-output assertion above; I ran it against both base and PR with Rust 1.85.0.
There was a problem hiding this comment.
Fixed in 31f54c3.
A dot now has to be followed directly by a digit, and dots are never mixed with spaces or hyphens inside one candidate, so a sentence-ending period can no longer continue into the next number. Dot-grouped cards (3056.930902.5904) still work.
Added your exact case as a full-output assertion (card 3056 930902 5904. 5553 0000 0000 0000 on file -> card [card]. [card] on file), plus the dotted and no-space variants. On the previous logic that test fails with card [card] 0000 0000 0000 on file, matching your report.
I also removed tests/zz_probe.rs. It was a leftover probe that I committed by mistake. Its useful cases are now assertions in sensitivity.rs.
The dot separator only checked the preceding group length, then allowed a space and a following number. With '3056 930902 5904. 5553 0000 0000 0000' the first 14 digits plus four digits of the next card formed a Luhn-valid 18-digit run that won over the real second card and left 12 of its digits. A dot must now be followed directly by a digit, and dots are never mixed with spaces or hyphens inside one card. Dot-grouped cards keep working. Also remove the temporary tests/zz_probe.rs that was committed by mistake, and turn its useful cases into assertions.
loveRhythm1990
left a comment
There was a problem hiding this comment.
LGTM at 31f54c3. The blocking punctuation-boundary regression reported above is fixed.
I independently reviewed the complete current diff and candidate-selection logic. The exact two-card reproduction now preserves the period and fully redacts both numbers. Requiring a digit immediately after a dot and separating dotted grouping from space/hyphen grouping addresses that regression while retaining the original #495 fixes. The print-only probe has also been removed.
Validation on Rust 1.85.0: 61 core unit tests and 12 contract tests pass, including the exact regression assertion. An additional independent matrix of 294 synthetic cases passes: 13–19 digit cards, ASCII/fullwidth digits, space/hyphen/dot grouping, adjacent phone/SSN in both orders, and period-plus-space/tab boundaries.
Coverage: 1/1 current changed files reviewed, 100%, none skipped; the removed probe was also checked. No further blocking findings. The head is conflict-free against current main. DB Tests are still running; this code LGTM does not imply that all CI has finished.
Fixes #495
Problem
The MEDIUM tier applies
email→phone→ssn→credit_cardin order. The phone regex matches a 6-4 digit sub-group inside a hyphen- or dot-grouped card, socard 3056-930902-5904 on filewas stored ascard 3056-[phone] on file. The BIN stayed in the clear and the card was never labeled. Space grouping already worked becausephoneonly accepts-and..Change (
memoria-core/src/sensitivity.rs)credit_cardnow runs beforephone, as suggested in the issue.3056.930902.5904as3056.[phone]. It now accepts.as a separator, but only after a 4 to 6 digit group. Real grouped cards fit that, and fractional timestamps like1730284800.123456do not get scanned.555-867-5309 4111 1111 1111 1111matched a 15 digit Luhn-valid span that started at the phone number. That redacted the phone plus part of the card and left 12 digits of the card in clear text. I confirmed this with a probe before adding the guard. Card candidates now cannot start or end inside a phone or SSN match. A card that merely contains one, like the 4-6-4 grouping, is still accepted.Tests
-,., space for 4-6-4, plus Amex 4-6-5 and 4-4-4-4) is fully redacted withmatched_labels == ["credit_card"].555-867-5309; card 3056-930902-5904redacts both. A phone or SSN next to a card cannot absorb its digits, in either order.sensitivity_contracttests still pass.On the original source,
card_redaction_is_not_preempted_by_phone_for_any_groupingandphone_still_redacted_next_to_a_cardfail. The adjacency test passes there, because the old order happened to shield it. It guards the reorder.Notes
matched_labelsorder follows pattern order, so a text with both now listscredit_cardbeforephone. The existing contract test for label order only covers email, phone and SSN and is unchanged.Verification
cargo test -p memoria-core: 59 unit, 12 contract and 1 doc tests pass.cargo clippy -p memoria-core --all-targets: no new warnings. The one warning,manual_is_multiple_of, is in existing code.