Skip to content

Reorganize JEDI meta-module enviornments to separate advanced python dependencies. - #2134

Open
eap wants to merge 5 commits into
JCSDA:developfrom
eap:feature/python-jedi-reorg
Open

eap wants to merge 5 commits into
JCSDA:developfrom
eap:feature/python-jedi-reorg

Conversation

@eap

@eap eap commented Sep 24, 2026 •

Copy link
Copy Markdown
Collaborator

This change moves scientific python dependencies out of the primary c++/fortran build path for some JEDI components.

Motivation: AI work often is picky about python requirements and can clash with our primary environment. Many jedi components can build with a minimal python environment.

Fixes: https://github.com/JCSDA-internal/jedi-bundle/issues/146
Fixes: #1888

@eap eap changed the title draft: reorganize jedi enviornments to separate out advanced python dependencies. Reorganize JEDI meta-module enviornments to separate advanced python dependencies. Sep 29, 2026
eap added 3 commits October 14, 2026 02:52
jedi-base-env pulled in 12 python packages, so every environment built on
it inherited python whether or not it needed it -- including the model
interface environments jedi-mpas-env and jedi-fv3-env.

Move 11 of them into a new jedi-python-env bundle, following the existing
neptune-env / neptune-python-env split. py-pybind11 stays in jedi-base-env
because core JEDI packages link its headers.

jedi-mpas-env and jedi-fv3-env are left with no python dependency other
than py-pybind11. The other eight jedi-base-env dependents (ai-env,
ewok-env, gmao-swell-env, jedi-geos-env, jedi-neptune-env, jedi-ufs-env,
jedi-um-env, soca-env) gain an explicit jedi-python-env dependency so
their content is unchanged.

jedi-python-env is added to the unified-dev and skylab-dev templates.
@eap
eap force-pushed the feature/python-jedi-reorg branch 2 times, most recently from b833ed6 to 5db8091 Compare October 14, 2026 02:54
# (shared with py-pyogrio) so only py-versioneer itself is duplicated.
- ^py-versioneer@0.28
prefer:
- '~performance'

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This got added in the last change and ended up setting py-versioneer@0.28 as preferred, I had to move it down to make sure py-versioneer@0.28 stayed in the require: block.

@eap
eap marked this pull request as ready for review October 14, 2026 14:39
@mathomp4

Copy link
Copy Markdown
Collaborator

@eap Be wary that now that you did this, I might ask/annoy/bother you for a similar geos-python-env or something.

(But that's 2.3+ timeframe)

@eap

eap commented Oct 14, 2026

Copy link
Copy Markdown
Collaborator Author

@mathomp4

@eap Be wary that now that you did this, I might ask/annoy/bother you for a similar geos-python-env or something.

So far my process has been.

  1. remove the jedi-python-env module dependency.
  2. try the build, read the configure steps carefully.
  3. fix the build upstream when things inevitably break.
  4. push

I'm probably sneaking soca-env into this change if I can do it faster than Dom or Alex can review.

@climbfuji climbfuji left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is fine with me. I may take jedi-python-env out of the jedi-neptune-env, because most if not all packages are already in neptune-python-env. Or I'll simplify it in different ways. But that's a later change after the 2.2.0 release.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: In Progress

Development

Successfully merging this pull request may close these issues.

[INSTALL/QUESTION]: Add support for zarr

3 participants