Skip to content

Fix small internal bugs, input validation and report without CompletenessScore - #125

Merged
alpae merged 5 commits into
devfrom
fix/small-bugs-input-validation
Nov 4, 2026
Merged

alpae merged 5 commits into
devfrom
fix/small-bugs-input-validation

Conversation

@alpae

@alpae alpae commented Nov 4, 2026 •

Copy link
Copy Markdown
Member

Step 1 of the bug list in #124: small fixes that don't change the inferred HOGs. One commit per issue.

#122: small internal bugs

  • UnionFind: takes over the fixed version from zoo.
    • remove() of a set's root, or of an element that others still point to, no longer breaks find().
    • union() counts a shared root's weight only once.
    • The identical copy of the class in _utils_roothog.py is removed; it shadowed the import.
  • Representative: Representative(rep, elements) copies rep's sets instead of extending them in place.
  • HOG class:
    • find_representative() uses get_subelements() instead of the non-existent get_members().
    • A merged HOG without representatives raises a clear ValueError; the default argument was a typing object.
  • batch_roothogs: rejects rootHOG files with the same name in different subfolders instead of letting them overwrite each other.
  • Pairwise relations: an empty orthologGroup no longer crashes set.union().
  • prepare_species_tree: returns the species leaf for a single-species rootHOG. Output is unchanged: such rootHOGs still produce no group.
  • copy_tree: the dead feature-collecting loop is removed (its set.union() result was discarded), and the docstring now says what the newick copy keeps. The output is identical.

#118: input validation

  • Input folders:
    • Hidden files are ignored, e.g. the ._<name> AppleDouble files of macOS archives or .DS_Store. check-input and parse_proteomes share list_proteome_files(), so both agree on what a proteome is.
    • Files without an extension no longer crash parse_proteomes.
    • Species names taken from hogmap files keep their dots.
    • An empty proteome is reported as invalid instead of raising ZeroDivisionError.
  • Species tree:
    • Duplicate internal node names are reported.
    • Trees without internal nodes no longer raise UnboundLocalError.
    • Internal node names are replaced by internal_<n> if they are empty or shorter than 3 characters after removing special characters, or if they equal a leaf name. Generated names never clash with existing ones. The checked tree of testdata is unchanged.
    • An unreadable species tree exits with status 1 instead of 0.
  • Splice files: single-line splice files are no longer ignored by check_splice.

#123: report without CompletenessScore

  • The four report cells that plot CompletenessScore now check that the column exists.
  • The options nf-test now disables all three scores and checks that report.html is still produced.

Tests

  • The 18 strict xfails of these issues pass and their markers are removed; some characterisation tests are updated to the new behaviour.
  • New tests:
    • UnionFind edge cases, including the union weight bug;
    • internal-node renaming edge cases;
    • exit status of an unreadable species tree;
    • a merged HOG without representatives.
  • Locally:
    • fast pytest suite green: 339 passed, 11 skipped (fungi-30 golden tests, tool versions differ locally), 26 xfailed;
    • nf-tests/options.nf.test green.
  • No golden snapshot changes.

Fixes #118
Fixes #122
Fixes #123

🤖 Generated with Claude Code

alpae and others added 3 commits November 4, 2026 16:21
- UnionFind: take over the fixed zoo version. remove() of a set's root or of
  an element others still point to no longer breaks find(), and union() counts
  the weight of a shared root only once. Drop the identical copy of the class
  in _utils_roothog.py, which shadowed the import.
- Representative(rep, elements) copies rep's sets instead of extending them
  in place; find_representative() uses get_subelements(); HOG() without
  representatives for a merged HOG raises a clear ValueError (the default was
  a typing object).
- batch_roothogs rejects rootHOG files with the same name in different
  subfolders instead of letting them overwrite each other.
- pairwise relations: an empty orthologGroup no longer crashes set.union().
- prepare_species_tree returns the species leaf for a single-species rootHOG
  (no output change: such rootHOGs still produce no group).
- copy_tree: drop the dead feature-collecting loop (its set.union() result was
  discarded) and document what the newick copy keeps; same output as before.

Fixes #122

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Input folders
- hidden files (the ._<name> AppleDouble files of macOS tar/zip archives,
  .DS_Store) are no longer taken as proteomes or hogmaps; check-input and
  parse_proteomes share list_proteome_files(), so both agree on what a
  proteome is
- files without an extension no longer crash parse_proteomes
- species names derived from hogmap files keep their dots
  (Homo.sapiens.fa.hogmap -> Homo.sapiens)
- an empty proteome is reported as invalid instead of raising
  ZeroDivisionError

Species tree
- duplicate internal node names are reported, and trees without internal
  nodes no longer raise UnboundLocalError
- internal node names that are empty or shorter than 3 characters after
  removing special characters, or that equal a leaf name, are replaced by
  internal_<n>; generated names never clash with existing ones. The checked
  tree of testdata is unchanged.
- an unreadable species tree exits with status 1 instead of 0
  (read_species_tree), so Nextflow no longer treats the task as successful

Splice files
- a species whose splice file has a single line is no longer ignored by
  check_splice

Fixes #118

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The four report cells that plot CompletenessScore now check that the column
exists and print a note otherwise, so --report together with
--disable_completeness_score no longer fails.

The options nf-test now disables all three scores (it kept CompletenessScore
before because the report failed without it) and checks that report.html is
still produced.

Fixes #123

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The node-name allocator can move an existing taxonomic label to a different clade, and hidden FASTA files remain eligible for downstream omamer processing.

2 open issues
What changed in this PR

This PR addresses internal helper bugs, input-validation failures, and reporting without CompletenessScore, as part of the pre-ete4 migration cleanup. It is intended to preserve inferred HOGs and existing golden snapshots.

Changes:

  • Fix UnionFind, representative handling, rootHOG batching, and tree/XML helper edge cases.
  • Improve input filtering, species-tree validation, and error handling.
  • Guard optional-score plots and expand regression coverage.
File Description
tests/​test_unionfind.py Cover removal, shared-root weights, and use of the shared implementation.
tests/​test_species_tree_prep.py Verify single-species trees and failure exit codes.
tests/​test_roothog_io.py Enable hidden-file and extension-handling regression tests.
tests/​test_report_contract.py Enable checks for guarded completeness plots.
tests/​test_pairwise_relations.py Enable empty-group regression coverage.
tests/​test_hog_class.py Verify representative copying, lookup, and required arguments.
tests/​test_check_input.py Expand input-validation and node-renaming coverage.
tests/​test_batch_roothogs.py Require rejection of duplicate rootHOG filenames.
nf-tests/​options.nf.test Check report generation with all scores disabled.
FastOMA/​zoo/​unionfind.py Repair removal and avoid double-counting shared roots.
FastOMA/​zoo/​hog/​transform.py Handle empty ortholog groups.
FastOMA/​fastoma_notebook_stat.ipynb Guard four optional completeness-score plots.
FastOMA/​check_input.py Improve file filtering, validation, and internal-node naming.
FastOMA/​batch_roothogs.py Reject filename collisions before batching.
FastOMA/​_utils_subhog.py Correct tree-read exit codes and single-species selection.
FastOMA/​_utils_roothog.py Share proteome discovery and remove duplicate UnionFind code.
FastOMA/​_infer_subhog.py Remove ineffective feature collection and clarify copy semantics.
FastOMA/​_hog_class.py Fix representative copying, lookup, and constructor defaults.

Review effort: Balanced

Comment thread FastOMA/check_input.py Resolved
Comment thread FastOMA/check_input.py Resolved
alpae and others added 2 commits November 4, 2026 20:49
check-input and parse_proteomes ignore hidden files, but FastOMA.nf selected
proteomes by their .fa/.fasta suffix only, so the ._<name>.fa AppleDouble files
of macOS archives still reached omamer. New nf-test runs the pipeline on the
original macOS-made test archive (with ._ files for every entry).

Refs #118 (review comment on #125)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…lades

Generated names (internal_<n>, <cleaned>_<n>) only avoided leaf names and names
of nodes visited before, so in (((A,B),C)internal_0,D)root; the unnamed (A,B)
clade took internal_0 and the clade that was called internal_0 was renamed.
Original names of all internal nodes are now reserved up front, also for names
cleaned of special characters. The tests assert the clade-to-name mapping.

Refs #118 (review comment on #125)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No active review issues are recorded for the input-validation, helper, and reporting fixes.

1 open issue
1 resolved since last review

Review effort: Balanced

@alpae
alpae merged commit 08d677c into dev Nov 4, 2026
27 checks passed
@alpae
alpae deleted the fix/small-bugs-input-validation branch November 4, 2026 20:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants