[darksim33]

Summary

requirements.txt is now a fully pinned (==), hash-verified lock file compiled from the pyproject.toml. The version ranges (what the project supports) live in pyproject.toml; the exact versions and hashes (what was tested) live in requirements.txt. Added ranges where a break is plausible or already enforced by a dependency: numpy>=2,<3 (nlopt requires numpy 2.x), scipy>=1.13 (cvxpy), torch>=2.2,<3, torchio>=1.2,<2, dipy>=1.12.1,<2, pandas>=2.2,<4. Added a preproc extra (nipype, itk, pyyaml), which the preprocessing code imports, and included it in all.

requirements.txt is generated from pyproject.toml with:

uv pip compile pyproject.toml --extra all --universal --python-version 3.11 --generate-hashes -o requirements.txt

• --extra all covers the runtime dependencies plus the test, docs, plot and preproc extras.
• --generate-hashes pins every package (direct and transitive) with sha256 hashes. Installing with pip enables hash checking automatically.
• --universal resolves for all platforms in one pass. Platform-specific packages carry environment markers, e.g. colorama and win32-setctime are sys_platform == 'win32', and the nvidia-*/triton CUDA stack is sys_platform == 'linux'.
• --python-version 3.11 matches the requires-python = ">=3.11" floor in pyproject.toml.

Installing necessary packages like before:

pip install -r requirements.txt

Downsides

• requirements.txt only resolves Python 3.11
  • for all other python versions pip install . from pyproject.toml
• 'torch' package pulls full CUDA stack regardless of an available Nvidia GPU
• hash pinning is really strickt and can lead to failed installs when packages arent available or matching
• changing requirements.in forces a re-compilation of the requirements.txt

[IvanARashid]

Thanks for looking into this!

1. Remove all the preprocessing stuff in src/original/preprocessing that managed to come in here.

2. Regarding fixing requirements in requirements.txt and keeping it open in pyproject.toml, it sounds good. But currently, the github actions use requirements.txt to create the environments, so all our python versions will use the same package versions which makes the testing across python versions a bit redundant. The intention was to test functionality on newer version of python and their packages.

Could you adjust the unit tests/github actions so that it install packages via pyproject.toml? And perhaps it would be good to have that default 3.11 from requirements.txt as a test.

[darksim33]

Thanks for the feedback.
I thought I choose the preproc-main branch as target since I started from there... seems like I didnt...
Two options: Change Merge Target to preproc and leave main as is or move changes to fresh main derived branch.

Regarding the pyproject-as-source-of-truth for CI I can make the edits but the downside will be that on newer python version CI will break if a package is update (like dipy), still matches the range but has an altered API. The requirements.txt will still be a valid choice for py311, py312 and py313

[IvanARashid]

I think this is a PR relevant for main and the entire repo :) We will stop using preproc-main as a separate main branch, since it was just a temporary setup before we did some restructuring for preprocessing code. So you could move the requirements- and testing-related changes to a new branch derived from main without the preproc stuff which are handled in another PR.

Otherwise I think the new changes look good! The downsides you mention are true, but it's also the reason for testing different python versions. I think the changes you've made here are good since we now also have a properly pinned and functioning version as well in case anything happens.

[darksim33]

I agree and have moved the changes to a new branch derived from main instead of preproc-main.
The new PR is #188