Repository navigation
Fix search settings, Sage run matching and rescoring tolerance units; add independent LFQ groups - #734
Fix search settings, Sage run matching and rescoring tolerance units; add independent LFQ groups#734Shen-YuFei wants to merge 17 commits into
Conversation
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. Please check the settings in the CodeRabbit UI or the ⚙️ Run configuration
You can disable this status message by setting the Use the checkbox below for a quick retry:
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
PR Summary by QodoAllow empty SDRF fixed modifications with safe MS-GF+ validation
AI Description
Diagram
High-Level Assessment
Files changed (6)
|
Code Review by Qodo
1.
|
There was a problem hiding this comment.
🟡 Changes recommended
Trim or normalize whitespace-only variable modifications and add regression coverage.
Get a fresh assessment by requesting another Copilot review.
Pull request overview
Fixes SDRF handling so empty fixed modifications remain valid while preventing unsafe empty MS-GF+ modification configurations.
Changes:
- Updates modification validation and fallback behavior.
- Adds regression tests and documentation/changelog updates.
- Restores the pipeline-output hook executable bit.
Review note: whitespace-only variable modifications can bypass the MS-GF+ validation and require normalization.
File summaries
| File | Summary |
|---|---|
subworkflows/local/create_input_channel/tests/main.nf.test |
Adds regression coverage for SDRF modifications. |
subworkflows/local/create_input_channel/main.nf |
Adjusts modification validation and metadata handling. |
nextflow_schema.json |
Documents updated input behavior. |
docs/usage.md |
Explains modification handling and MS-GF+ limitations. |
CHANGELOG.md |
Records the fix. |
.hooks/block_pipeline_outdir.sh |
Restores the executable bit without logic changes. |
Review details
- Files reviewed: 5/6 changed files
- Comments generated: 1
- Review effort level: Lite
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
| meta.variablemodifications = params.variable_mods | ||
| } | ||
|
|
||
| if (params.search_engines.contains('msgf') && !meta.fixedmodifications && !meta.variablemodifications) { |
|
I encountered the same issue, and this commit appears to fix it for me (at least while using comet search engine). I noticed the PR was closed without being merged, but I couldn’t find an explanation in the thread. Was the fix superseded or intentionally not merged? |
|
@nikita-a-v Hi Nikita, I ran into this issue (and a few others) while running quantms on a batch of PXD datasets, so I'm planning to fix all the problems I hit before reopening this PR. It should be done soon. |
|
ⓘ Qodo reviews are paused because the subscription is no longer active. Ask your workspace admin to reactivate the subscription to resume reviews. Manage billing |
|
There was a problem hiding this comment.
When using the quantms-utils container, it is best to move this Python script into the quantms-utils package.
|
Can you elaborate on why the auto settings for comet were a problem? Most people do not know what a bin width is for that or how to set it. |
Thanks @jpfeuffer. Comet: You are right, this is too strict. The old automatic settings were not a real problem: in all the ppm datasets I re-analysed, I used the same values. I will restore the automatic default and keep the new Comet options as optional overrides. LFQ groups: each group has its own QPX output ( |
|
Yeah that is better, thank you! |
|
@Shen-YuFei What is the split LFQS? Can you explain the problem here.? |
@ypriverol I had this problem with the following two datasets. Each dataset has parts that cannot be processed together in one ProteomicsLFQ run:
ProteomicsLFQ combines the identifications from all runs. When the search settings are different, it stops with an error. --lfq_group_by takes one or more SDRF columns, for example comment[instrument],comment[gradient duration]. Runs with the same values form one group. The search and rescoring still run once for all files. After that, each group has its own ProteomicsLFQ run, design, MSstats table and QPX output. Before this, the only way was to split the SDRF by hand and run quantms once for each group. |
|
I think it is an interesting feature that I was also thinking about for longer. I wasnt sure how much complexity this adds though. |
Thanks @jpfeuffer. The added complexity is small: only ProteomicsLFQ and the QPX conversion run once per group. Search, rescoring and the report still run once. Splitting the SDRF beforehand also works, but then each part needs its own pipeline run and report. You are right about pmultiqc. It still builds the report. I tried it locally: group support adds about 180 lines to pmultiqc, mostly in the QPX module, plus tests, and quantms only needs to pass the group table it already writes to the report step. The summary counts and the cross-run comparisons (PCA, heatmap, intensity SD) are then computed per group; the per-run sections do not change. |
Summary
Fixes found while re-analysing public spike-in datasets with quantms.
Search modifications from SDRF
Comet fragment binning
--comet_fragment_bin_tol(native full width in Da),--comet_fragment_bin_offsetand--comet_instrument. Per-run overrides are available throughext.comet_fragment_bin_tol,ext.comet_fragment_bin_offsetandext.comet_instrument. The width is divided by two for OpenMS CometAdapter.high_res; otherwise 1.0005 Da, offset 0.4,low_res) and log a warning..nextflow.logat DEBUG level.MS2 rescoring tolerance
MSRESCORE_FEATURESpasses the SDRF fragment tolerance value and unit (Da or ppm) unchanged. The configured fallback is used only when both fields are absent.Sage
fr1*_sage.idparquetalso matchedfr10(seen in PXD011691)..druns without the.dsuffix, as IDRipper names them.Independent LFQ groups (
--lfq_group_by)qpx/<group>.sdrf/).bigbio/qpx/openmsconsensus(tuple input[consensusXML, SDRF]andtask.ext.prefix) is recorded inmodules.json, regenerated against the qpx 1.1.5 module from chore(qpx): update qpx/openmsconsensus module to qpx 1.1.5 #738.PREPARE_LFQ_GROUPScallsquantmsutilsc splitlfqgroupsfrom quantms-utils 0.0.31 (Add splitlfqgroups: split an LFQ design into independent quantification groups by SDRF columns quantms-utils#84).bin/split_lfq_groups.pyand its pytest are removed from this repository.lfq_groups.tsvis also passed to the pmultiqc report step. feat(qpx): Report cross-run QC per quantms LFQ group pmultiqc#746 uses it to report cross-run QC per group. The current container, pmultiqc 0.0.48, ignores the file; after a pmultiqc release with #746, only the container version needs to change.--lfq_group_bythe output layout is unchanged.Other
.hooks/block_pipeline_outdir.shexecutable, as the pre-commit hook calls it directly.quantms-rescoring: 0.0.25 and the pending release
When
PSM_CLEANmerges several search engines, 0.0.24 could give identifications of different spectra the same index. 0.0.25 indexes each engine's identifications separately, which removes those collisions, but a spectrum found by two engines then becomes two identifications:test_lfq: on identical Comet and Sage results, 0.0.24 gives psm 51, feature 41, pg 25, and 0.0.25 gives psm 61, feature 46, pg 23. The 9 PSMs found only with 0.0.24 all had the same PEP (0.047619); of 16 checked spectra found only with 0.0.25, 11 match BSA.test_tmtandtest_tmt_corrfail in CI. IsobaricWorkflow quantifies each identification, so a spectrum with two identifications is quantified twice, and the QPX export rejects the repeated feature key (ConsensusMapArrowExport: refusing invalid QPX feature batch ... repeats the QPX feature primary key). Intest_tmt, 1851 of 5599 spectra were split, and 46 still had two identifications after Percolator.bigbio/quantms-rescoring#102 keeps one identification per spectrum. Until a release includes it,
test_tmtandtest_tmt_corrfail on this branch; after the release, the four rescoring modules move to that version. With a container built from quantms-rescoringmainplus #102:test_tmt: one identification per spectrum after Percolator. Against 0.0.24, target PSMs go from 3166 to 3282 and proteins in the MSstats input from 463 to 477.test_lfq: psm 62, feature 46, pg 24.Verification
Nextflow 26.04.6, Docker.
pre-commit run --files <changed files>: passed.nf-core pipelines lint: 0 failures.--profile docker): 37 tests passed (Comet binning 17, LFQ grouping 3,qpx/openmsconsensus1,create_input_channel16).pytest tests/test_sage_outputs.py: 1 passed.-profile test_lfq,docker: passed with released containers only. QPX rows: psm 61, feature 46, pg 23.-profile test_lfq,docker --lfq_group_by 'source name': passed with released containers (quantms-utils 0.0.31 biocontainer). Three groups, three ProteomicsLFQ tasks, and each QPX dataset contains only the two fractions of its sample. The report step receiveslfq_groups.tsv: with pmultiqc 0.0.48 the report data are the same as without it, and pmultiqc from feat(qpx): Report cross-run QC per quantms LFQ group pmultiqc#746, run on the same report inputs, shows one summary row per group.-profile test_dda_id_ms2pip,dockerwith the SDRF's 0.2 Da and with a 20 ppm copy: passed with quantms-rescoring 0.0.25 (MS2PIP 4.2.0). MS2PIP received the tolerance in Da and in ppm.high_res) gave 2766 PSMs at q < 0.01, compared with 2778 at the SDRF's 0.2 Da.test_lfq,test_tmt,test_tmt_corr,test_localize,test_dda_id_ms2pip,test_dda_id_alphapeptdeep,test_dda_id_fine_tuning) with-profile <test>,docker,devon Nextflow 25.10.4 and 26.10.2, using the quantms-rescoring container built with [DOC] how to debug #102: passed.-profile test_lfq_sage,dockerfails atSAMPLESHEET_CHECK(AttributeError: 'float' object has no attribute 'strip'in sdrf-pipelines) on both this branch and currentdev.test_lfqruns Sage as well (search_engines = "comet,sage").