Skip to content

gh-157144: Let unittest discovery and regrtest find tests inside archives on sys.path - #157145

Draft
gpshead wants to merge 4 commits into
python:mainfrom
gpshead:regrtest-nonfs-test-packages
Draft

gpshead wants to merge 4 commits into
python:mainfrom
gpshead:regrtest-nonfs-test-packages

Conversation

@gpshead

@gpshead gpshead commented Sep 8, 2026 •

Copy link
Copy Markdown
Member

When the test package lives inside an archive on sys.path (a zipped standard library, a zipapp), the test suite cannot run:

  • unittest.TestLoader.discover() requires a real directory and raises "Start directory is not importable", which breaks every test package whose load_tests() uses test.support.load_package_tests().
  • libregrtest.findtests() calls os.listdir() on it, which raises NotADirectoryError (FileNotFoundError on Windows).

Fix this in unittest rather than in regrtest: when the start directory is not on the file system but a path hook claims it, discover() lists it through the archive's path entry finder with pkgutil.iter_modules(), which also distinguishes modules from packages. Module naming, importing and the load_tests protocol already work on such paths. findtests() gets the same pkgutil fallback for the top-level listing, keyed on os.path.isdir().

(thank you Claude Fable 5.1)

When the standard library is a zip archive on sys.path, the test package
is not a directory: findtests() failed with NotADirectoryError from
os.listdir() and load_package_tests() failed with "Start directory is
not importable" from unittest discovery. Fall back to listing the test
modules with pkgutil.iter_modules() through the import system.
Comment thread Lib/test/support/__init__.py Resolved
… regrtest

Make unittest.TestLoader.discover() work when the start directory is
inside an archive on sys.path, such as a zip file: the archive's path
entry finder lists the entries through pkgutil.iter_modules() and tells
modules and packages apart, and the rest of discovery (module naming,
importing, the load_tests protocol) already works on such paths.  With
that, test.support.load_package_tests() needs no special case and goes
back to plain discovery.

libregrtest's findtests() keeps a small fallback for listing the top
level test directory, now keyed on os.path.isdir() rather than on the
exception os.listdir() raises, which is NotADirectoryError on POSIX but
FileNotFoundError on Windows.
@gpshead gpshead changed the title gh-157144: Let regrtest find tests that are not on the file system gh-157144: Let unittest discovery and regrtest find tests inside archives on sys.path Sep 8, 2026
@read-the-docs-community

read-the-docs-community Bot commented Sep 8, 2026 •

Copy link
Copy Markdown

Documentation build overview

📚 cpython-previews | 🛠️ Build #34442024 | 📁 Comparing d3f5935 against main (c8da735)

  🔍 Preview build  

2 files changed
± library/unittest.html
± whatsnew/changelog.html

Comment thread Lib/unittest/loader.py Resolved
Address review: describe what _find_tests() hands _find_test_path()
with module-level constants (_FILESYSTEM_ENTRY, _FILESYSTEM_MODULE,
_FILESYSTEM_PACKAGE, _ARCHIVE_MODULE, _ARCHIVE_PACKAGE) instead of
inline strings and None, and split the archive helper into a predicate
and a listing function so that neither returns None to mean anything.

Drop the regrtest NEWS entry; the unittest one covers the change.
@serhiy-storchaka
serhiy-storchaka self-requested a review September 9, 2026 14:07
…in discovery

Address review of unittest discovery inside archives:

* A path hook such as zipimport claims every path below its archive, so
  a mistyped start directory there gave an empty suite.  discover() now
  raises ImportError unless a top level directory in an archive is the
  archive itself or lists a module, and regrtest's findtests() lets
  os.listdir() raise when the archive lists nothing.
* Leave out names which the archive's finder lists but cannot import,
  such as extension modules in a zip file, instead of reporting them as
  failed imports.
* Pass *kind* to _find_test_path() for archive entries only, so that
  subclasses overriding it with the previous signature keep working for
  file system discovery.
* Use pkgutil.get_importer() in _is_archive_dir(); it does not cache a
  negative result, contrary to what the docstring said.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
@bedevere-app bedevere-app Bot added the type-feature A feature request or enhancement label Nov 10, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

type-feature A feature request or enhancement

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant