[prabhkodes]

Issue

CPU_Accelerator.total_memory() and available_memory() return host RAM from psutil.virtual_memory(). In a container or a Kubernetes pod, the process can use only its cgroup limit. Callers that plan memory from these values on the CPU accelerator include ZeRO 1/2 flattening (stage_1_and_2.py:451) and the autotuner. Fixes #8929.

Related: #1891 asks for cgroup-aware CPU memory limits. #3458 (Docker), #2632 and #2788 (Slurm) report processes killed by a cgroup limit without a message.

Fix

Only the CPU accelerator, its tests and its docs change.

• total_memory() returns the smallest memory limit from the process cgroup up to the root, capped at host RAM.
• available_memory() returns the smallest free headroom over every limited cgroup on that path. A parent limit also counts the usage of its other children, so each limited level is checked. Usage excludes the inactive page cache, which the kernel can reclaim.
• The cgroup mount point and mount root come from /proc/self/mountinfo. When several mounts match, the one whose root contains the process cgroup is used. This supports mounts that expose a subtree, cgroup v1 co-mounts such as cpu,memory, and delegated subtrees mounted before the main mount.
• Off by default. RESPECT_CGROUP_MEMORY_BY_DEFAULT = False is the one line that sets the default. DS_CPU_RESPECT_CGROUP_MEMORY=1 or =0 overrides it. export_envs() returns the variable, so the launcher forwards it to every node.
• The accelerator setup guide documents the variable. abstract_accelerator.py does not change.

How to test

Needs Docker with cgroup v2. Save the script from the collapsed section at the end as check_cgroup_memory.sh. One command installs master (before) and this branch (after) in the same container with 1 GiB, and prints total_memory() and available_memory() from three cgroups. The script is in the collapsed section below.

sh check_cgroup_memory.sh "git+https://github.com/deepspeedai/DeepSpeed@master" \
                          "git+https://github.com/prabhkodes/DeepSpeed@cpu-accelerator-cgroup-memory"

Output (Apple M4, Docker Desktop VM with 7.8 GiB):

        process cgroup (container limit 1 GiB)                     total   available
before  A. container root                                      7837 MiB    6342 MiB
before  B. child cgroup, limit 256 MiB                         7837 MiB    6438 MiB
before  C. child 600 MiB; parent 800 MiB, sibling uses 500     7837 MiB    6452 MiB
after   A. container root                                      1024 MiB     187 MiB
after   B. child cgroup, limit 256 MiB                          256 MiB      89 MiB
after   C. child 600 MiB; parent 800 MiB, sibling uses 500      600 MiB     129 MiB

Before, every case reports the host. After, total is the limit of each case. In C, available is the free headroom of the 800 MiB parent, which the 500 MiB sibling also uses. In A, the container's 1 GiB also holds that sibling.

Proof

ZeRO training on the CPU accelerator, three Adam steps, flag 0 and 1, two fresh containers each:

processes	precision	ZeRO 1	ZeRO 2	ZeRO 3
1	fp32	identical losses	identical losses	identical losses
1	bf16	identical losses	identical losses	identical losses
2 (deepspeed launcher, gloo)	fp32	identical losses	identical losses	identical losses

Performance

Throughput and peak RSS, flag 0 and 1, 30 timed steps on a 4×1024 MLP. Four runs per cell, two fresh containers, flag alternated. Median steps/s with range:

processes	precision	ZeRO	steps/s, flag 0	steps/s, flag 1	peak RSS, flag 0 / 1
1	fp32	1	23.8 (22.9–24.7)	24.2 (23.2–25.2)	575 / 575 MiB
1	fp32	2	24.3 (24.0–24.9)	23.3 (22.4–24.1)	572 / 571 MiB
1	fp32	3	22.1 (21.1–26.5)	22.1 (20.9–22.7)	606 / 606 MiB
1	bf16	1	41.5 (37.7–42.4)	41.2 (40.7–42.5)	530 / 530 MiB
1	bf16	2	42.0 (40.1–42.5)	41.6 (33.5–42.6)	530 / 530 MiB
1	bf16	3	54.1 (45.2–55.4)	54.2 (51.3–56.7)	556 / 556 MiB
2	fp32	1	11.1 (10.8–11.4)	11.4 (10.4–11.7)	524 / 524 MiB
2	fp32	2	11.2 (10.3–12.4)	11.9 (11.2–12.6)	524 / 522 MiB
2	fp32	3	11.0 (10.7–11.1)	10.6 (9.9–11.0)	564 / 564 MiB

Peak RSS does not change. The throughput differences go in both directions and stay inside the run-to-run range. A call counter shows why: ZeRO 1 and 2 call the memory queries once at setup, ZeRO 3 does not call them, and no configuration calls them during training steps.

Cost of one query, best of 5 × 5000 calls, process three levels below the cgroup root:

	total_memory()	available_memory()
master	12.4 µs	12.5 µs
this PR, flag unset	13.1 µs	12.9 µs
this PR, flag 1	81.5 µs	80.7 µs

With the flag set, a query reads /proc/self/cgroup, /proc/self/mountinfo and the cgroup files: about 70 µs more, once per run.

pytest tests/unit/v1/accelerator/test_accelerator.py   # 30 passed, 2 skipped, two runs
pre-commit run --files <changed files>                 # all hooks pass

Eleven new tests. They cover a limit on a parent cgroup, the smallest limit, the v1 sentinel, the headroom of every limited level, mountinfo resolution with several mounts, the launcher export, and default × override.

Hardware. Apple M4 host, Docker Desktop Linux 6.10 VM (aarch64, 10 CPUs, 7.8 GiB, cgroup v2). Python 3.12.16, torch 2.14.1+cpu, DS_BUILD_OPS=0. No GPU.

Compatibility matrix.

• Validated: CPU accelerator; ZeRO 1, 2 and 3; fp32 and bf16; 1 and 2 processes; no offload.
• Not reachable on the CPU accelerator: deepspeed.compile (is_deepcompile_supported() requires CUDA) and the inference v2 KV cache (CUDA kernels only).
• Not validated: CPU and NVMe offload, fp16, multi-node runs (the launcher export is unit-tested only).
• Not affected: CUDA, XPU, NPU, MLU, HPU, SDAA, SUPA and MPS read their own device memory. see_memory_usage() and the timer read host RAM only to log it.

No open PR changes the CPU accelerator memory functions (searched cgroup, cpu_accelerator available_memory, cpu accelerator memory, 1891 in:body on 2026-10-27).
AI assistance was used: an AI coding agent (Claude Code) wrote the change and the tests; I reviewed each changed line and ran the tests, the reproduction and the ZeRO runs.

check_cgroup_memory.sh

#!/bin/sh
# Before/after check of DeepSpeed's CPU accelerator memory values inside cgroups. Needs Docker with cgroup v2.
#   sh check_cgroup_memory.sh <before pip spec> <after pip spec>
# Example:
#   sh check_cgroup_memory.sh "git+https://github.com/deepspeedai/DeepSpeed@master" \
#                             "git+https://github.com/prabhkodes/DeepSpeed@cpu-accelerator-cgroup-memory"
set -eu
docker run --rm -i --privileged --cgroupns=private --memory=1g ${EXTRA_DOCKER_ARGS:-} \
  -e BEFORE="${1:?before spec}" -e AFTER="${2:?after spec}" python:3.12-slim sh -s <<'INNER'
set -e
export DEBIAN_FRONTEND=noninteractive
apt-get -qq update >/dev/null 2>&1 && apt-get -qq install -y git >/dev/null 2>&1
pip install -q torch --index-url https://download.pytorch.org/whl/cpu >/dev/null 2>&1
pip install -q psutil numpy pydantic packaging py-cpuinfo tqdm hjson msgpack ninja einops >/dev/null 2>&1
cat > /tmp/probe.py <<'PY'
from deepspeed.accelerator import get_accelerator
a = get_accelerator()
print(f"{a.total_memory() >> 20:>7} MiB {a.available_memory() >> 20:>7} MiB")
PY
cat > /tmp/hold.py <<'PY'
import time
b = b"\x01" * (500 << 20)
time.sleep(3600)
PY
C=/sys/fs/cgroup
mkdir -p $C/init; for p in $(cat $C/cgroup.procs); do echo $p > $C/init/cgroup.procs 2>/dev/null || true; done
echo "+memory" > $C/cgroup.subtree_control
mkdir -p $C/nested; echo $((256 << 20)) > $C/nested/memory.max
mkdir -p $C/shared; echo $((800 << 20)) > $C/shared/memory.max; echo 0 > $C/shared/memory.swap.max
echo "+memory" > $C/shared/cgroup.subtree_control
mkdir -p $C/shared/sibling $C/shared/me; echo $((600 << 20)) > $C/shared/me/memory.max
sh -c "echo \$\$ > $C/shared/sibling/cgroup.procs; echo -1000 > /proc/self/oom_score_adj; exec python /tmp/hold.py" &
sleep 3
run() {
  printf "%-7s %-52s" "$1" "$2"
  sh -c "echo \$\$ > $C/$3/cgroup.procs; DS_ACCELERATOR=cpu DS_CPU_RESPECT_CGROUP_MEMORY=1 exec python /tmp/probe.py 2>/dev/null"
}
printf "%-7s %-52s %11s %11s\n" "" "process cgroup (container limit 1 GiB)" "total" "available"
for v in before after; do
  if [ $v = before ]; then spec=$BEFORE; else spec=$AFTER; fi
  DS_BUILD_OPS=0 pip install -q --no-deps --force-reinstall "$spec" >/dev/null 2>&1
  run $v "A. container root" init
  run $v "B. child cgroup, limit 256 MiB" nested
  run $v "C. child 600 MiB; parent 800 MiB, sibling uses 500" shared/me
done
INNER

Precedent. The same walk is already in production elsewhere. Rust's std::thread::available_parallelism takes the smallest cpu.max from the process cgroup up to the mount (quota_v2). The JDK fixed limits on parent nested cgroups in JDK-8322420 (2024) and the hierarchical memory limit in JDK-8370572 (2025). Go 1.25 finds the process cgroup through /proc/self/cgroup and /proc/self/mountinfo (cgroup_linux.go), reading the leaf only. vLLM's CPU-quota reader walks the hierarchy (_cgroup_cpu_limit); vllm-project/vllm#64104 adds the same for memory. The same fix for OpenObserve merged on 2026-10-28 (openobserve/openobserve#15906).

[chatgpt-codex-connector]

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	✅ Completed 2026-10-27T19:18:55.859895Z	dc5b6c1	Manual request

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: dbc0fd95f5

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

[prabhkodes]

@codex review

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: dc5b6c1834

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".