Repository navigation
Add water consumption from electricity mix - #1407
benoit-cty wants to merge 8 commits into
Conversation
Report in water_consumed the water used to generate the electricity consumed, on top of the direct cooling water given by the wue parameter. The water intensity (L/kWh) is computed as a weighted average of per-source water consumption factors over the energy mix of the country (or Canadian region) where the machine runs, with a world-average fallback when coverage of the mix is below 90%. The intensity is cached per location and a failure of the water estimation can never break the emissions tracking. Ported and reworked from feat/water_consumption_S, started by students: - fix the source keys of water_consumption_per_source.json so they match global_energy_mix.json (the original computation always fell back to the world average), and add oil/hydro/biofuel factors - fix a gal/kWh vs gal/MWh unit error in the cloud fallback path - make the world-average fallback consistent with the computed country values (weighted by the world energy mix) to avoid a 5x discontinuity at the coverage threshold - accumulate water incrementally in _update_emissions alongside emissions, instead of recomputing from the total at report time - drop code duplicated from the Emissions class and the dead USA regional branch (only Canada has regional energy mix data) Co-authored-by: Sifflet_blanc <flo.crahay@gmail.com> Co-authored-by: Simon Decostanzi <gbsny8zh@gmail.com> Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Add the water_consumed column to the emissions table (alembic migration included), accept it on emission creation, and sum it in the run, experiment, project and organization reports. Co-authored-by: Simon Decostanzi <gbsny8zh@gmail.com> Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Show a water radial chart next to the energy one on the organization, project and public project dashboards, and add water_consumed to the CSV exports. Co-authored-by: Simon Decostanzi <gbsny8zh@gmail.com> Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #1407 +/- ##
==========================================
+ Coverage 92.07% 92.18% +0.10%
==========================================
Files 49 50 +1
Lines 5202 5300 +98
==========================================
+ Hits 4790 4886 +96
- Misses 412 414 +2 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
|
Thanks for this integration Benoît! |
|
Merged master into this branch (two CLI conflicts, took master's side since the water feature does not touch them) and added |
Description
Adds an estimation of the water consumed by your compute, reported in the existing
water_consumedfield (litres), across the whole stack:water_consumedis now the sum ofwueparameter (unchanged, defaults to 0), andcodecarbon/data/private_infra/water_consumption_per_source.json(Meldrum et al. 2013, Macknick et al. 2012). When sources with known water intensity cover less than 90% of a country's mix (e.g. geothermal-heavy Kenya/Iceland), a world average of ~3.75 L/kWh is used (computed from the same per-source table weighted by the world mix, so the fallback is consistent with computed country values). Cloud providers publish no per-region water data, so cloud runs fall back to the water intensity of the host country.water_consumedcolumn onemissions(alembic migration included), accepted on emission creation and summed in run/experiment/project/organization reports.water_consumedadded to CSV exports.examples/water_consumption.pyadded.This is a rework of the student branches
feat/water_consumptionandfeat/water_consumption_S(the latter is a superset; both can be deleted once this merges). The branches were 481 commits behind master, so the feature was ported onto master rather than rebased, keeping the students as co-authors. Beyond the rebase, the main fixes over the original branch:global_energy_mix.json(natural_gasvsgas, missingoil/hydroelectricity/biofuel), so the energy-mix computation always fell back to the world average — and one fallback path called a method that did not exist (WaterPerKWh.from_g_per_kWh). Keys now match, hydro/oil/biofuel factors were added, and coverage is ≥90% for 207 of 213 countries.gal/kWhvsgal/MWh) in the cloud fallback path._update_emissions()alongside emissions instead of being recomputed from the total at report time;wuewater is added rather than replacing the estimate (direct + indirect, as in Li et al. 2023, "Making AI Less Thirsty").Emissionsclass, the noisy per-measurement warnings, an unrelatedalembic/env.pychange, and a migration that carried unrelated auto-generated schema changes with a wrongdown_revision.An adversarial multi-angle review was then run on the branch and its accepted findings applied: the water intensity is cached per location (it is constant for a run, and its fallback warnings now log once instead of every measurement cycle), the water estimation is wrapped so a failure can never break the carbon tracking, the new data file was added to
[tool.setuptools.package-data](verified present in a built wheel), the dead USA regional branch was dropped (only Canada ships a regional energy mix), anullwater_consumedis coalesced to 0 before insert (the column is NOT NULL), and the duplicated webappRadialChartDatatype/skeleton markup were consolidated.Behavior change to be aware of (not an API break):
water_consumedwas previouslywue × energyand therefore 0 unlesswuewas set. It is now non-zero for every run, since the indirect water of electricity generation is always estimated. Orders of magnitude: France ≈ 3.8 L/kWh (nuclear+hydro), Poland ≈ 1.8 L/kWh (coal), Norway ≈ 15 L/kWh (hydro reservoir evaporation — high but consistent with the literature; flagged as uncertain in the docs). If maintainers prefer the estimate to stay opt-in, gating it behind a parameter is a small follow-up.Related Issue
Extends #894 (water usage estimation), which was partially addressed by the
wueparameter in #941. Supersedes the branchesfeat/water_consumptionandfeat/water_consumption_S.Motivation and Context
Computing does not only emit CO₂: generating the electricity it uses also consumes fresh water (thermal/nuclear plant cooling, reservoir evaporation). Reporting only the datacenter WUE misses this — usually dominant — indirect part, and required the user to know their WUE. With this change every run gets a water estimate out of the box, computed from the same energy-mix data already used for carbon intensity.
How Has This Been Tested?
tests/test_water_consumption.py(14 tests): unit conversions, per-country values (FRA/POL), scaling with energy, world-average fallbacks (unknown country, low-coverage country, cloud without geo), CAN regional mix, USA regional fallback, and a data-file consistency test that fails if the JSON keys ever drift fromglobal_energy_mix.json.uv run task test-package— passes excepttest_task_energy_with_live_update_interference, which fails identically on master (pre-existing, unrelated: energy sum mismatch).uv run task test-api-unit— 119 passed (fixtures extended withwater_consumed; the existing wue default/custom tests now also assert the new field).tsc --noEmitclean,vitest run— same 3 pre-existing interaction-test failures as master, everything else passes; prettier clean.water_consumedtoemissions.csvwith the expected 3.83 L/kWh ratio for France;examples/water_consumption.pyruns.Screenshots (if appropriate):
Types of changes
AI Usage Disclosure
Checklist: