Skip to content

perf!: reduce import time and drop yarl - #1734

Merged
JP-Ellis merged 6 commits into
mainfrom
perf/import-time
Oct 16, 2026
Merged

JP-Ellis merged 6 commits into
mainfrom
perf/import-time

Conversation

@JP-Ellis

@JP-Ellis JP-Ellis commented Oct 16, 2026 •

Copy link
Copy Markdown
Member

✈️ Pre-flight checklist

  • I have read the Contributing Guidelines on pull requests.
  • If this is a code change: I have written unit tests and/or added dogfooding pages to fully verify the new behavior.
  • If this is a new API or substantial change: the PR has an accompanying issue (closes #0000) and the maintainers have approved on my working plan.

📝 Summary

This PR cuts the time import pact takes, and removes yarl from Pact Python's dependencies. With pytest already loaded, which is where Pact tests run, import pact drops from 35 ms to 14 ms.

Python 3.14, median main this PR
import pact, standalone 74 ms 44 ms
import pact, after import pytest 35 ms 14 ms

Dropping yarl is a breaking change, intended for 4.1.0. 4.0.0 is recent, and public code uses srv.url only in ways that keep working. See Breaking change.

The commits fall into two groups. The first three remove import costs:

  1. perf(ffi) and perf: write the version file at build time. Both packages called importlib.metadata.version() on import. Hatchling's version build hook now writes __version__.py from pyproject.toml, so scripts/release.py still edits a single source of truth. The generated files are gitignored. On its own, this saves only 2–4 ms, because pact imports most of importlib.metadata's dependencies anyway. It also gives frozen builds without package metadata a real __version__ instead of "unknown".
  2. perf: import the message and state servers on demand. pact.verifier imported pact._server, and with it http.server, email, html and ssl, into every process. Only the message handler and the function and dictionary state handlers use these servers.

The last three replace yarl with a small str-based URL type:

  1. feat(types): add pact.types.URL. It is a str subclass. / appends a path segment and returns a new URL, with the same percent-encoding as yarl. scheme, host, port and path are parsed on first access. pact.types gains an __all__, which mkdocstrings needs to render the re-exported URL.
  2. refactor(verifier)!: accept only str URLs. Verifier URL parameters are annotated str, and StateHandlerUrl is an alias of str. The verifier parses URLs with urllib.parse, and a new helper, strip_url_credentials, removes credentials from a URL. add_source reads file:// strings as local sources, using url2pathname so that Windows drive letters work.
  3. refactor!: PactServer.url returns pact.types.URL. yarl leaves pyproject.toml, so import pact no longer loads it. The tests, the compatibility suite and the protobuf example move to urllib.parse, aiohttp's params= and pact.types.URL.

🔥 Motivation

Most of import pact went into dependencies a given test run might never use: http.server for provider message and state handlers, and yarl with multidict, propcache and idna. Deferring the yarl import would have kept it as a dependency for one property, PactServer.url.

A search of public GitHub code found that people use str(srv.url) or f"{srv.url}/path", or pass srv.url straight to an HTTP client. The only public uses of /, .with_query and .with_path were Pact Python's own tests, and one repository that vendors them.

Alternatives considered:

  • Keep yarl, imported lazily. It is non-breaking, but every install still pulls in yarl, multidict, propcache and idna for one property.
  • Return a plain str. It is the simplest option, but breaks srv.url / "path" and srv.url.port, which Pact Python's own tests and docs use.
  • A str subclass with / and read-only attributes. This is the option taken. Every use found in public code keeps working. Query strings belong in the HTTP client's params= argument, so URL has no query builder.
  • PEP 810 lazy imports. __lazy_modules__ works on Python 3.11 as a plain variable, but the syntax only takes effect on 3.15+. A global lazy mode mostly postpones the cost: pact_ffi and pact's own modules load as soon as a test creates a Pact. Under pytest, the remaining 14 ms is close to that floor.
  • Splitting pact_ffi/__init__.py. The 8,000-line file costs about 1.5 ms with cached bytecode, so splitting it would save almost nothing.

🚨 Breaking change

PactServer.url and PactServer / "path" return pact.types.URL instead of yarl.URL. These keep working:

str(srv.url)
f"{srv.url}/users/123"
requests.get(srv.url)
srv.url / "users" / "123"
srv.url.scheme, srv.url.host, srv.url.port, srv.url.path

Every other yarl.URL method is gone, including with_query, with_path, with_user, query, user, password, update_query, joinpath, parent, name and %. A query string goes in the HTTP client instead:

# Before
requests.get(str(srv.url.with_query({"id": "1"})))
# After
requests.get(srv.url, params={"id": "1"})

Two / details differ from yarl. An empty segment leaves the URL unchanged, and .. segments are kept as written.

Verifier parameters typed str | yarl.URL are now typed str, and pact.types.StateHandlerUrl is an alias of str. Verifier.state_handler and Verifier.add_source raise TypeError for a yarl.URL; pass str(url) instead.

Verifier.add_source("file:///path/to/pact.json") now loads the local file. Before this PR, a file:// string took the remote path and failed.

__version_tuple__ now keeps non-numeric version parts as strings, as setuptools-scm does: 4.1.0.dev0 gives (4, 1, 0, 'dev0'), where main dropped them. Both packages also gain the version and version_tuple aliases.

The release PR will set 4.1.0 and call out this change, and the 4.0.0 release notes will point to it.

🔨 Test Plan

  • tests/test_url.py covers /: chained segments, trailing slashes, percent-encoding of space, % and ?, credentials, dropped query and fragment, and a leading /. It also covers the attributes (default ports, IPv6, unknown schemes) and str behaviour, including hashing.
  • tests/test_import.py checks in a fresh interpreter that import pact loads neither http.server, pact._server nor yarl.
  • tests/test_util.py covers strip_url_credentials with expected values written out.
  • tests/test_verifier.py covers file:// sources (including a Windows drive letter through nturl2path) and Windows paths. It checks that bytes, paths and objects defining __str__ raise TypeError from state_handler and add_source.
  • Locally on Python 3.14:
    • The full suite passes, including the compatibility suite against a Pact Broker container: 584 passed, 14 skipped.
    • The example suite passes (23), including the protobuf plugin example.
    • ruff and mypy pass on src/, tests/ and examples/.
    • hatch run docs-build resolves every pact.types.URL cross-reference.

🔗 Related issues/PRs

  • Verifier.broker_source rejects conflicts with credentials embedded in the URL, but never uses those credentials. It strips them, unlike _add_source_remote. This PR keeps that behaviour; the fix will be a separate change.
  • pactflow/pactflow-agent-skills describes srv.url as "a yarl.URL" in scripts/generate/dsl_python.py. A PR there will correct it.

@codecov

codecov Bot commented Oct 16, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 97.61905% with 2 lines in your changes missing coverage. Please review.
✅ Project coverage is 70%. Comparing base (a225d82) to head (0ecdecf).
⚠️ Report is 3 commits behind head on main.
✅ All tests successful. No failed tests found.

Files with missing lines Patch % Lines
src/pact/_util.py 86% 2 Missing ⚠️
Additional details and impacted files
@@         Coverage Diff          @@
##           main   #1734   +/-   ##
====================================
+ Coverage    68%     70%   +1%     
====================================
  Files        20      19    -1     
  Lines      3288    3331   +43     
====================================
+ Hits       2261    2337   +76     
+ Misses     1027     994   -33     
Flag Coverage Δ
examples 52% <44%> (-1%) ⬇️
tests 69% <96%> (+1%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

@JP-Ellis JP-Ellis mentioned this pull request Oct 16, 2026
2 of 3 tasks
@JP-Ellis JP-Ellis changed the title perf: reduce import time perf!: reduce import time and drop yarl Oct 16, 2026
`pact_ffi.__version__` looked up its own version with
`importlib.metadata.version()` on every import. That loads
`importlib.metadata` and scans the installed distributions.

Hatchling's `version` build hook now writes `src/pact_ffi/__version__.py`
from `project.version` during every build, editable installs included.
The template emits `__version__` and `__version_tuple__`, with the
`version` and `version_tuple` aliases, matching the layout hatch-vcs
produced. `pyproject.toml` remains the single source of truth, so
`scripts/release.py` is unaffected. The generated file is gitignored.

`__version_tuple__` keeps non-numeric parts as strings, so `4.0.0rc1`
becomes `(4, 0, '0rc1')`, as with setuptools-scm.

The import-time saving is small, about 2-4 ms. `pact` already imports
most of the modules `importlib.metadata` depends on.

Assisted-by: Claude Code:claude-opus-5-5
`pact.__version__` looked up its own version with
`importlib.metadata.version()` on every import, as `pact_ffi` did.
Hatchling's `version` build hook now writes `src/pact/__version__.py`
from `project.version`, with the same template as `pact_ffi`. The
generated file is gitignored.

`pact._util` imported `importlib.metadata` at module level for
`user_agent()`. The function now takes the pact version from
`pact.__version__`. It imports `importlib.metadata` itself, which it
still needs for the `requests` version. With both changes, `import pact`
no longer loads `importlib.metadata` at all.

Assisted-by: Claude Code:claude-opus-5-5
`pact.verifier` imported `pact._server` at module level, and
`pact._server` imports `http.server`. That chain loads `email`, `html`,
`http.client` and `ssl` into every process that imports `pact`. Only
`Verifier.message_handler` and the function and dictionary state handlers
use these servers, so they now import them when called.

Stubbing `pact._server` out cut about 8 ms from `import pact` on Python
3.14. A new test checks in a fresh interpreter that `import pact` leaves
`http.server` and `pact._server` unloaded.

Assisted-by: Claude Code:claude-opus-5-5
`pact.types.URL` is a `str` subclass. `/` appends a path segment and
returns a new `URL`, percent-encoding the segment as `yarl` does. The
`scheme`, `host`, `port` and `path` attributes are parsed on first
access. It is meant to replace `yarl.URL` as the type of
`PactServer.url`.

`pact.types` now lists its exports in `__all__`. mkdocstrings renders an
imported name only when `__all__` lists it, so `URL` would otherwise be
missing from the API reference.

Assisted-by: Claude Code:claude-opus-5-5
Verifier URL parameters are annotated `str`, and
`pact.types.StateHandlerUrl` is now an alias of `str`.
`Verifier.state_handler` and `Verifier.add_source` raise `TypeError` for
anything else.

The verifier parses URLs with `urllib.parse` and no longer imports
`yarl`. The new `pact._util.strip_url_credentials` replaces
`with_user(None).with_password(None)`. It percent-decodes the
credentials, lowercases the host and drops a default port, as `yarl`
did.

`add_source` reads a `file://` string as a local source, converting its
path with `url2pathname` so that `Path.as_uri()` on Windows gives
`C:\...`. A `file://` string used to take the remote path and fail
inside `yarl`. `urllib.request` is imported only for these sources,
since loading it adds about 18 ms to `import pact`.

The tests and the compatibility suite pass verifier URLs as strings.

Breaking: pass `str(url)` instead of a `yarl.URL` to `state_handler` and
`add_source`. `StateHandlerUrl` no longer includes `yarl.URL`.

Assisted-by: Claude Code:claude-opus-5-5
`PactServer.url` and `PactServer / str` return `pact.types.URL`, and
`yarl` is no longer a dependency. `import pact` therefore no longer
loads `yarl`, `multidict`, `propcache` or `idna`, and a new test checks
this in a fresh interpreter.

Public code and the documentation use `str(srv.url)` or
`f"{srv.url}/path"`, which keep working, as does `srv.url / "segment"`.
The tests, the compatibility suite and the protobuf example now build
query strings with `urllib.parse` or the HTTP client's `params`
argument.

Breaking: the `yarl.URL` methods are gone, including `with_query`,
`with_path`, `with_user`, `query`, `user`, `password`, `update_query`,
`joinpath`, `parent`, `name` and the `%` operator. `scheme`, `host`,
`port` and `path` remain as read-only attributes. Query strings belong in
the HTTP client's `params` argument.

Assisted-by: Claude Code:claude-opus-5-5
@JP-Ellis
JP-Ellis merged commit ddbf773 into main Oct 16, 2026
35 checks passed
@JP-Ellis
JP-Ellis deleted the perf/import-time branch October 16, 2026 06:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant