From 47788eabf90cbd27256da792682674775568e95e Mon Sep 17 00:00:00 2001 From: Alexey Khrulenko Date: Thu, 8 Oct 2026 13:23:18 +0200 Subject: [PATCH] +conda recipe to install on windows --- CHANGELOG.md | 1 + README.md | 28 +++++++++--- docs/installation_windows.md | 83 ++++++++++++++++++++++++++++++++++++ docs/tutorials/enif.md | 3 ++ environment.yml | 41 ++++++++++++++++++ mkdocs.yml | 1 + 6 files changed, 151 insertions(+), 6 deletions(-) create mode 100644 docs/installation_windows.md create mode 100644 environment.yml diff --git a/CHANGELOG.md b/CHANGELOG.md index 7b4e9227..b4cc3f01 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -8,6 +8,7 @@ and versions follow [Semantic Versioning](https://semver.org/spec/v2.0.0.html). ## [Unreleased] ### Added +- **A conda recipe for Windows.** `pip install -e .` cannot work on Windows: `graphite-maps` has no Windows build on PyPI, and the `scikit-sparse` it imports needs the CHOLMOD C library. `environment.yml` installs `scikit-sparse` from conda-forge, `graphite-maps` 0.0.11 from GitHub and PET itself in editable mode with one `conda env create -f environment.yml`; `docs/installation_windows.md` describes it. With that environment 583 of 589 non-slow tests pass on Windows; the six failures are five bit-for-bit comparisons in `test_subspace2.py` that differ in the last floating-point digit, and one test that expects a `/` path separator. - **`GenOpt` is back**, on `OptimizerBase`, together with the `CMA` covariance-matrix adaptation it uses. Both were removed during the popt restructuring — `GenOpt` by `a6938529` with nothing put in its place, `cma.py` later as code with no remaining caller — while `src/popt/README.md` went on advertising the method. It draws from `GeneralizedEnsemble`'s marginals and advances the sampling distribution along with the controls: `theta` follows its own gradient at `alpha_theta`, and the correlation matrix follows `corr_adapt`, which is either a `CMA` instance or any callable. `GeneralizedEnsemble.mutation_gradient` and `.mutation_hessian` had survived the restructuring with no consumer; this is the consumer. `mutation_gradient` gains `return_ensembles=True`, returning `(nat_grad, {'gaussian': enZ, 'objective': enF})`, so the CMA path reuses the ensemble the gradient came from instead of drawing and simulating a second one. Two fixes over the version that was removed: `alpha_corr` is read from `alpha_corr` rather than from `alpha_theta`, so it is no longer silently ignored, and the optimizer no longer runs itself from its own constructor — call `run_optimization()` or `GenOpt.minimize(...)`, as with every other optimizer here. - **`analysis = "subspace2"`**, the ensemble-transform IES of Raanes, Stordal & Evensen (2019), on ES-MDA, LM-EnRML and GN-EnRML. It solves for the `ne × ne` transform `W` directly, starting from `W = I`, and uses the analytic data covariance through `scale_data` rather than the ensemble representation `E Eᵀ` that `subspace` uses — so it takes no SVD and reads neither `energy` nor `iteration.energy`. It is exactly `margis` with the marginalised error scale `Ratio` fixed at 1, i.e. with the data uncertainty taken as known; `tests/assimilation/test_subspace2.py` pins that identity bit-for-bit. Ported from upstream 6f313d7, with two departures: the transform is initialised at iteration 0 rather than 1 (schemes here count from 0, so the reference version never initialises and dies on the first call), and the whitened observations are recomputed every call instead of cached on the first. ES-MDA redraws its observations and their scale at every assimilation step, so the cache would drive later steps with the first step's observations whitened by the first step's factor — the run still completes and the misfit still falls, which is why it would have gone unnoticed. On the characterisation case it is the difference between a posterior misfit of 175.7 and 201.1; the cached, lower number is the wrong one. The 39 existing golden arrays are unchanged. diff --git a/README.md b/README.md index 591bd45a..cc736f35 100644 --- a/README.md +++ b/README.md @@ -16,6 +16,15 @@ at NORCE Norwegian Research Centre AS. PET requires Python 3.12 through 3.14. The standard installation includes EnIF and EnIF-MDA with their dependencies. +To install PET, first clone the repo (assuming you have added the SSH key) + +``` +git clone git@github.com:Python-Ensemble-Toolbox/PET.git PET +``` +Alternatively, [click here to download ZIP](https://github.com/Python-Ensemble-Toolbox/PET/archive/refs/heads/main.zip). + +### On Linux + Before installing ensure you have python3 pre-requisites. On a Debian system run: ``` @@ -25,12 +34,6 @@ sudo apt-get install python3-pip sudo apt-get install python3-venv ``` -To install PET, first clone the repo (assuming you have added the SSH key) - -```sh -git clone git@github.com:Python-Ensemble-Toolbox/PET.git PET -``` - Make sure you have the latest version of `pip` and `setuptools`: ```sh @@ -64,6 +67,19 @@ To also install the tools needed for running tests and linting locally: python3 -m pip install -e ".[dev]" ``` +### On Windows + +TL;DR: use `conda` to create the environment and install PET. Inside the PET folder, +run: + +```powershell +conda env create -f environment.yml -n pet +conda activate pet +``` +For more details refer to +[Installing PET on Windows](docs/installation_windows.md). +This setup may also work on Linux, though it has not been tested yet. + ## Documentation The [configuration reference](docs/configuration.md) lists every key of the diff --git a/docs/installation_windows.md b/docs/installation_windows.md new file mode 100644 index 00000000..61c201bd --- /dev/null +++ b/docs/installation_windows.md @@ -0,0 +1,83 @@ +# Installing PET on Windows + +On Windows, install PET with the `conda` package manager. A plain +`pip install -e .` (as well as `uv pip install -e .`) fails, because of +`graphite-maps` and its dependencies: PET's EnIF analysis depends on +`graphite-maps`, which in turn needs `scikit-sparse` and the CHOLMOD C library. +Neither package has a Windows build on PyPI; conda-forge provides +`scikit-sparse` and CHOLMOD ready-made. + +## Prerequisites + +- [Miniforge](https://conda-forge.org/download/), or another conda + installation that can reach the conda-forge channel. +- [Git for Windows](https://git-scm.com/download/win) on your `PATH`. + `graphite-maps` and PET's `geostat` dependency are installed from git. + +## Install + +From the root of your PET folder: + +```powershell +conda env create -f environment.yml -n pet +conda activate pet +``` + +This creates an environment named `pet` with Python 3.13, installs the compiled +dependencies from conda-forge, `graphite-maps` 0.0.11 from GitHub, and PET +itself in editable mode, so changes to the source take effect immediately. + +To run the Eclipse or OPM simulators, also install +[SimulatorWrap](https://github.com/Python-Ensemble-Toolbox/SimulatorWrap) from +its checkout: + +```powershell +pip install -e ..\SimulatorWrap +``` + +## Check the installation + +```powershell +python -c "import pipt; from sksparse.cholmod import cholesky; print('PET OK')" +pet version +``` + +To run the test suite, install `pytest` and run it from the PET folder: + +```powershell +pip install pytest +python -m pytest -m "not slow" +``` + +## Things to know + +- **Always activate the environment** (`conda activate pet`) before running + PET. Calling `envs\pet\python.exe` directly, without activation, crashes in + NumPy's linear algebra because its DLLs are not found. +- **Keep `scikit-sparse` below 0.5.** `graphite-maps` 0.0.11 requires it, and + conda-forge also offers 0.5. +- **After updating PET**, rerun `pip install -e .` in the activated + environment if `pyproject.toml` has changed. An editable install picks up + source changes, but not new dependencies. +- **Behind a corporate proxy**, `mamba` and conda-forge's own `git` package can + fail with `schannel: the revocation status is unknown`. Use `conda` rather + than `mamba`, and Git for Windows rather than a conda-installed git. This is + why `git` is not listed in `environment.yml`. +- **`conda list` may show `scipy` as coming from `pypi`.** It is still the + conda-forge build; pip does not reinstall it. + +## Known test failures on Windows + +With the environment above, 583 of 589 non-slow tests pass (October 2026). The +six failures are not installation problems: + +- Five tests in `tests/assimilation/test_subspace2.py` require two analyses to + agree bit for bit. On Windows they differ in the last floating-point digit + (about 1e-16). +- `tests/test_logging_and_paths.py::test_save_folder_is_not_created_by_reading_it` + expects a `/` path separator and gets `\`. + +## Conda on Linux + +The same `environment.yml` is expected to work on Linux for those who use +`conda` as a package manager, though it has not been tested. diff --git a/docs/tutorials/enif.md b/docs/tutorials/enif.md index eeacb150..7accfbd8 100644 --- a/docs/tutorials/enif.md +++ b/docs/tutorials/enif.md @@ -15,6 +15,9 @@ python -m pip install -e . PET requires Python 3.12 through 3.14, matching its `graphite-maps` dependency. +On Windows, `graphite-maps` cannot be installed with pip alone; see +[Installing PET on Windows](../installation_windows.md). + ## Select the analysis Keep your existing ensemble, observation and simulator settings. EnIF is an diff --git a/environment.yml b/environment.yml new file mode 100644 index 00000000..d10cf579 --- /dev/null +++ b/environment.yml @@ -0,0 +1,41 @@ +# Conda environment for PET. Needed on Windows, where pip alone cannot install +# PET; tested there only. See docs/installation_windows.md. +# +# conda env create -f environment.yml +# conda activate pet +# +# Requires git on PATH (Git for Windows on Windows): graphite-maps and PET's +# geostat dependency are installed from git. (git is deliberately not a conda +# dependency here: the conda-forge build fails its certificate revocation check +# behind some proxies.) +# +# Windows needs conda because scikit-sparse (required by graphite-maps, which +# the EnIF analysis uses) has no Windows wheel on PyPI. +name: pet +channels: + - conda-forge +dependencies: + - python=3.13 + - scikit-sparse<0.5 # brings suitesparse (CHOLMOD); graphite-maps needs <0.5 + - numpy + - scipy + - matplotlib + - h5py + - pandas + - scikit-learn # graphite-maps + - networkx # graphite-maps + - joblib # graphite-maps + - tqdm + - p-tqdm + - pywavelets + - pyyaml + - sympy + - tomli + - tomli-w + - pip + - pip: + # Not on conda-forge, and no Windows build on PyPI. Pinned to the release + # that PET's pyproject.toml requires. + - git+https://github.com/equinor/graphite-maps@0.0.11 + # PET itself, editable. The path is relative to this file. + - -e . diff --git a/mkdocs.yml b/mkdocs.yml index be67f434..8eaa12ab 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -9,6 +9,7 @@ watch: [.] nav: - Home: index.md + - Windows installation: installation_windows.md - Reference: reference/ - Tutorials: tutorials/ - Configuration: configuration.md