Repository navigation
perf!: reduce import time and drop yarl - #1734
Merged
Merged
Conversation
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #1734 +/- ##
====================================
+ Coverage 68% 70% +1%
====================================
Files 20 19 -1
Lines 3288 3331 +43
====================================
+ Hits 2261 2337 +76
+ Misses 1027 994 -33
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. |
JP-Ellis
force-pushed
the
perf/import-time
branch
from
October 16, 2026 01:31
aa069c5 to
e4c80d9
Compare
2 of 3 tasks
`pact_ffi.__version__` looked up its own version with `importlib.metadata.version()` on every import. That loads `importlib.metadata` and scans the installed distributions. Hatchling's `version` build hook now writes `src/pact_ffi/__version__.py` from `project.version` during every build, editable installs included. The template emits `__version__` and `__version_tuple__`, with the `version` and `version_tuple` aliases, matching the layout hatch-vcs produced. `pyproject.toml` remains the single source of truth, so `scripts/release.py` is unaffected. The generated file is gitignored. `__version_tuple__` keeps non-numeric parts as strings, so `4.0.0rc1` becomes `(4, 0, '0rc1')`, as with setuptools-scm. The import-time saving is small, about 2-4 ms. `pact` already imports most of the modules `importlib.metadata` depends on. Assisted-by: Claude Code:claude-opus-5-5
`pact.__version__` looked up its own version with `importlib.metadata.version()` on every import, as `pact_ffi` did. Hatchling's `version` build hook now writes `src/pact/__version__.py` from `project.version`, with the same template as `pact_ffi`. The generated file is gitignored. `pact._util` imported `importlib.metadata` at module level for `user_agent()`. The function now takes the pact version from `pact.__version__`. It imports `importlib.metadata` itself, which it still needs for the `requests` version. With both changes, `import pact` no longer loads `importlib.metadata` at all. Assisted-by: Claude Code:claude-opus-5-5
`pact.verifier` imported `pact._server` at module level, and `pact._server` imports `http.server`. That chain loads `email`, `html`, `http.client` and `ssl` into every process that imports `pact`. Only `Verifier.message_handler` and the function and dictionary state handlers use these servers, so they now import them when called. Stubbing `pact._server` out cut about 8 ms from `import pact` on Python 3.14. A new test checks in a fresh interpreter that `import pact` leaves `http.server` and `pact._server` unloaded. Assisted-by: Claude Code:claude-opus-5-5
`pact.types.URL` is a `str` subclass. `/` appends a path segment and returns a new `URL`, percent-encoding the segment as `yarl` does. The `scheme`, `host`, `port` and `path` attributes are parsed on first access. It is meant to replace `yarl.URL` as the type of `PactServer.url`. `pact.types` now lists its exports in `__all__`. mkdocstrings renders an imported name only when `__all__` lists it, so `URL` would otherwise be missing from the API reference. Assisted-by: Claude Code:claude-opus-5-5
Verifier URL parameters are annotated `str`, and `pact.types.StateHandlerUrl` is now an alias of `str`. `Verifier.state_handler` and `Verifier.add_source` raise `TypeError` for anything else. The verifier parses URLs with `urllib.parse` and no longer imports `yarl`. The new `pact._util.strip_url_credentials` replaces `with_user(None).with_password(None)`. It percent-decodes the credentials, lowercases the host and drops a default port, as `yarl` did. `add_source` reads a `file://` string as a local source, converting its path with `url2pathname` so that `Path.as_uri()` on Windows gives `C:\...`. A `file://` string used to take the remote path and fail inside `yarl`. `urllib.request` is imported only for these sources, since loading it adds about 18 ms to `import pact`. The tests and the compatibility suite pass verifier URLs as strings. Breaking: pass `str(url)` instead of a `yarl.URL` to `state_handler` and `add_source`. `StateHandlerUrl` no longer includes `yarl.URL`. Assisted-by: Claude Code:claude-opus-5-5
`PactServer.url` and `PactServer / str` return `pact.types.URL`, and
`yarl` is no longer a dependency. `import pact` therefore no longer
loads `yarl`, `multidict`, `propcache` or `idna`, and a new test checks
this in a fresh interpreter.
Public code and the documentation use `str(srv.url)` or
`f"{srv.url}/path"`, which keep working, as does `srv.url / "segment"`.
The tests, the compatibility suite and the protobuf example now build
query strings with `urllib.parse` or the HTTP client's `params`
argument.
Breaking: the `yarl.URL` methods are gone, including `with_query`,
`with_path`, `with_user`, `query`, `user`, `password`, `update_query`,
`joinpath`, `parent`, `name` and the `%` operator. `scheme`, `host`,
`port` and `path` remain as read-only attributes. Query strings belong in
the HTTP client's `params` argument.
Assisted-by: Claude Code:claude-opus-5-5
JP-Ellis
force-pushed
the
perf/import-time
branch
from
October 16, 2026 05:13
465430a to
0ecdecf
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
📝 Summary
This PR cuts the time
import pacttakes, and removesyarlfrom Pact Python's dependencies. With pytest already loaded, which is where Pact tests run,import pactdrops from 35 ms to 14 ms.mainimport pact, standaloneimport pact, afterimport pytestDropping
yarlis a breaking change, intended for 4.1.0. 4.0.0 is recent, and public code usessrv.urlonly in ways that keep working. See Breaking change.The commits fall into two groups. The first three remove import costs:
perf(ffi)andperf: write the version file at build time. Both packages calledimportlib.metadata.version()on import. Hatchling'sversionbuild hook now writes__version__.pyfrompyproject.toml, soscripts/release.pystill edits a single source of truth. The generated files are gitignored. On its own, this saves only 2–4 ms, becausepactimports most ofimportlib.metadata's dependencies anyway. It also gives frozen builds without package metadata a real__version__instead of"unknown".perf: import the message and state servers on demand.pact.verifierimportedpact._server, and with ithttp.server,email,htmlandssl, into every process. Only the message handler and the function and dictionary state handlers use these servers.The last three replace
yarlwith a smallstr-based URL type:feat(types): addpact.types.URL. It is astrsubclass./appends a path segment and returns a newURL, with the same percent-encoding asyarl.scheme,host,portandpathare parsed on first access.pact.typesgains an__all__, which mkdocstrings needs to render the re-exportedURL.refactor(verifier)!: accept onlystrURLs. Verifier URL parameters are annotatedstr, andStateHandlerUrlis an alias ofstr. The verifier parses URLs withurllib.parse, and a new helper,strip_url_credentials, removes credentials from a URL.add_sourcereadsfile://strings as local sources, usingurl2pathnameso that Windows drive letters work.refactor!:PactServer.urlreturnspact.types.URL.yarlleavespyproject.toml, soimport pactno longer loads it. The tests, the compatibility suite and the protobuf example move tourllib.parse, aiohttp'sparams=andpact.types.URL.🔥 Motivation
Most of
import pactwent into dependencies a given test run might never use:http.serverfor provider message and state handlers, andyarlwithmultidict,propcacheandidna. Deferring theyarlimport would have kept it as a dependency for one property,PactServer.url.A search of public GitHub code found that people use
str(srv.url)orf"{srv.url}/path", or passsrv.urlstraight to an HTTP client. The only public uses of/,.with_queryand.with_pathwere Pact Python's own tests, and one repository that vendors them.Alternatives considered:
yarl, imported lazily. It is non-breaking, but every install still pulls inyarl,multidict,propcacheandidnafor one property.str. It is the simplest option, but breakssrv.url / "path"andsrv.url.port, which Pact Python's own tests and docs use.strsubclass with/and read-only attributes. This is the option taken. Every use found in public code keeps working. Query strings belong in the HTTP client'sparams=argument, soURLhas no query builder.__lazy_modules__works on Python 3.11 as a plain variable, but the syntax only takes effect on 3.15+. A global lazy mode mostly postpones the cost:pact_ffiand pact's own modules load as soon as a test creates aPact. Under pytest, the remaining 14 ms is close to that floor.pact_ffi/__init__.py. The 8,000-line file costs about 1.5 ms with cached bytecode, so splitting it would save almost nothing.🚨 Breaking change
PactServer.urlandPactServer / "path"returnpact.types.URLinstead ofyarl.URL. These keep working:Every other
yarl.URLmethod is gone, includingwith_query,with_path,with_user,query,user,password,update_query,joinpath,parent,nameand%. A query string goes in the HTTP client instead:Two
/details differ fromyarl. An empty segment leaves the URL unchanged, and..segments are kept as written.Verifier parameters typed
str | yarl.URLare now typedstr, andpact.types.StateHandlerUrlis an alias ofstr.Verifier.state_handlerandVerifier.add_sourceraiseTypeErrorfor ayarl.URL; passstr(url)instead.Verifier.add_source("file:///path/to/pact.json")now loads the local file. Before this PR, afile://string took the remote path and failed.__version_tuple__now keeps non-numeric version parts as strings, as setuptools-scm does:4.1.0.dev0gives(4, 1, 0, 'dev0'), wheremaindropped them. Both packages also gain theversionandversion_tuplealiases.The release PR will set 4.1.0 and call out this change, and the 4.0.0 release notes will point to it.
🔨 Test Plan
tests/test_url.pycovers/: chained segments, trailing slashes, percent-encoding of space,%and?, credentials, dropped query and fragment, and a leading/. It also covers the attributes (default ports, IPv6, unknown schemes) andstrbehaviour, including hashing.tests/test_import.pychecks in a fresh interpreter thatimport pactloads neitherhttp.server,pact._servernoryarl.tests/test_util.pycoversstrip_url_credentialswith expected values written out.tests/test_verifier.pycoversfile://sources (including a Windows drive letter throughnturl2path) and Windows paths. It checks thatbytes, paths and objects defining__str__raiseTypeErrorfromstate_handlerandadd_source.src/,tests/andexamples/.hatch run docs-buildresolves everypact.types.URLcross-reference.🔗 Related issues/PRs
Verifier.broker_sourcerejects conflicts with credentials embedded in the URL, but never uses those credentials. It strips them, unlike_add_source_remote. This PR keeps that behaviour; the fix will be a separate change.srv.urlas "ayarl.URL" inscripts/generate/dsl_python.py. A PR there will correct it.