Skip to content

Pin cloudpickle for the dask benchmark on Python 3.16 - #519

Open
sendaoYan wants to merge 1 commit into
python:mainfrom
sendaoYan:fix-bm-dask-cloudpickle
Open

sendaoYan wants to merge 1 commit into
python:mainfrom
sendaoYan:fix-bm-dask-cloudpickle

Conversation

@sendaoYan

Copy link
Copy Markdown
Contributor

Pin a Python 3.16-compatible cloudpickle for the dask benchmark

Fixes #518

Summary

bm_dask pins dask[distributed]==2024.10.0 but leaves cloudpickle to the
resolver (>=3.0.0). Python 3.16 removed DELETE_GLOBAL, and cloudpickle
< 3.1.3 raises KeyError: 'DELETE_GLOBAL' at import time, so dask fails.

cloudpickle 3.1.3 fixes this (cloudpipe/cloudpickle#599). Fresh venvs resolve
to it, but existing venvs keep 3.1.2 because ensure_reqs installs with
upgrade=False and 3.1.2 satisfies >=3.0.0.

Change

+cloudpickle==3.1.3
 dask[distributed]==2024.10.0

Testing

Downgraded the benchmark venv to cloudpickle 3.1.2, then ran:

python -m pyperformance run -b dask -f -o /tmp/dask.json

ensure_reqs installed cloudpickle==3.1.3 (uninstalling 3.1.2) and the
benchmark passed:

dask: Mean +- std dev: 3.33 sec +- 0.06 sec

No other benchmark depends on cloudpickle. cloudpickle 3.1.3 requires
Python >=3.8.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Pin a Python 3.16-compatible cloudpickle for the dask benchmark

1 participant