Add force_mode_constant parameter to bypass psutil overhead

[Copilot]

This PR adds a new force_mode_constant parameter to EmissionsTracker that allows users to bypass psutil completely and use constant CPU power estimation instead. This addresses significant performance overhead issues where psutil.cpu_percent(interval=0.5) calls were causing 10x+ slowdowns in ML training workflows.

Problem

Users reported severe performance overhead when tracking short ML training iterations:

• Without CodeCarbon: 11 seconds
• With CodeCarbon v3.0.2: 1 minute 55 seconds (10x slower!)

The overhead was caused by repeated psutil.cpu_percent(interval=0.5) calls that take 0.5 seconds each. Additionally, uninstalling psutil to force fallback mode resulted in ModuleNotFoundError due to unconditional imports.

Solution

1. Graceful psutil imports

Made psutil imports conditional in codecarbon/core/cpu.py and codecarbon/core/util.py:

try:
    import psutil
    PSUTIL_AVAILABLE = True
except ImportError:
    PSUTIL_AVAILABLE = False
    psutil = None

2. New force_mode_constant parameter

Added force_mode_constant parameter to bypass psutil entirely:

# Bypass psutil completely - eliminates 0.5s overhead per measurement
tracker = EmissionsTracker(force_mode_constant=True)

# Works with custom CPU power for better accuracy
tracker = EmissionsTracker(
    force_mode_constant=True, 
    force_cpu_power=100  # 100W TDP
)

3. Updated CPU tracking logic

Modified codecarbon/core/resource_tracker.py to prioritize force_mode_constant before all other tracking methods, ensuring it completely bypasses psutil when enabled.

Features

• ✅ Zero psutil overhead: Completely eliminates psutil.cpu_percent() calls
• ✅ Graceful fallback: Handles missing psutil without crashing
• ✅ Custom power support: Works with force_cpu_power parameter
• ✅ Precedence control: Takes priority over force_mode_cpu_load
• ✅ Backward compatibility: No breaking changes to existing code
• ✅ Comprehensive testing: Full test suite covering all scenarios

Performance Impact

Testing shows successful elimination of psutil overhead:

• Normal mode: Uses CPU load tracking (slower due to psutil calls)
• Force constant mode: Uses constant power estimation (eliminates overhead)
• Both produce valid emissions measurements

Documentation

Added parameter documentation to docs/edit/parameters.rst:

* - force_mode_constant
  - | Force the use of constant mode for CPU power consumption measurement,
    | defaults to ``False``. When enabled, bypasses psutil completely and
    | uses a constant power consumption based on CPU TDP. Useful when
    | psutil overhead is significant or psutil is unavailable.

Fixes #878.

Warning

Firewall rules blocked me from connecting to one or more addresses

I tried to connect to the following addresses, but was blocked by firewall rules:

• get.geojs.io
  • Triggering command: python /tmp/test_force_constant.py (dns block)
  • Triggering command: python -c from codecarbon import EmissionsTracker import tempfile with tempfile.TemporaryDirectory() as temp_dir: tracker = EmissionsTracker(output_dir=temp_dir) print('Available attributes:', [attr for attr in dir(tracker) if 'resource' in attr.lower() or 'tracker' in attr.lower()]) (dns block)
  • Triggering command: python /tmp/test_simple_constant.py (dns block)
• ip-api.com
  • Triggering command: python /tmp/test_force_constant.py (dns block)
  • Triggering command: python -c from codecarbon import EmissionsTracker import tempfile with tempfile.TemporaryDirectory() as temp_dir: tracker = EmissionsTracker(output_dir=temp_dir) print('Available attributes:', [attr for attr in dir(tracker) if 'resource' in attr.lower() or 'tracker' in attr.lower()]) (dns block)
  • Triggering command: python /tmp/test_simple_constant.py (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

• Configure Actions setup steps to set up my environment, which run before the firewall is enabled
• Add the appropriate URLs or hosts to my firewall allow list

---

💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.

[benoit-cty]

Tested working with:

CODECARBON_FORCE_MODE_CONSTANT=True uv run codecarbon monitor

@inimaz as it's me who run Copilot, I can merge by myself 😉

[inimaz]

Nice thanks! Minor comment

[inimaz]

is it possible to have force_mode_cpu_load=True and force_mode_constant=True at the same time? What does it mean in that case?

[benoit-cty]

Good catch, we need force_mode_constant to take precedence on force_mode_cpu_load.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 92.24%. Comparing base (8a8776a) to head (7fb59cb).

Additional details and impacted files

@@            Coverage Diff             @@
##           master     #883      +/-   ##
==========================================
+ Coverage   92.07%   92.24%   +0.16%     
==========================================
  Files          49       49              
  Lines        5202     5249      +47     
==========================================
+ Hits         4790     4842      +52     
+ Misses        412      407       -5     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[davidberenstein1957]

Rebased onto master by merging it in: kept master's platform-ordered backends and get_cached_tdp, dropped the PR's two refactor helpers that conflicted with them, and adapted one test to the new backend order. force_mode_constant still sits in set_CPU_tracking before the cpu_load branch. 685 tests pass. Could you re-approve once CI is green?