Skip to content

Commit 4e1d94f

Browse files
vstinnergpsheadethanfurmanEclips4
committed
[3.14] gh-161705: Prevent shadowing stdlib in gzip/tarfile/zipfile CLI (#161706) (#161719)
gh-161705: Prevent shadowing stdlib in gzip/tarfile/zipfile CLI (#161706) Remove sys.path[0] in gzip, tarfile and zipfile command line interface to prevent shadowing standard library modules from extracted files. * Add sys._path0 variable. * pymain_sys_path_add_path0() sets sys._path0. * Add runpy._remove_sys_path0(). * gzip, tarfile and zipfile main functions now call _remove_sys_path0() to prevent shadowing stdlib modules from extracted files. (cherry picked from commit e891a91) Co-authored-by: Gregory P. Smith <68491+gpshead@users.noreply.github.com> Co-authored-by: Ethan Furman <ethan@stoneleaf.us> Co-authored-by: Kirill Podoprigora <kirill.bast9@mail.ru> (cherry picked from commit 8aa5d65)
1 parent 42155a0 commit 4e1d94f

14 files changed

Lines changed: 196 additions & 21 deletions

File tree

‎Doc/c-api/init_config.rst‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -581,8 +581,8 @@ PyConfig
581581
582582
.. c:member:: int safe_path
583583
584-
If equals to zero, ``Py_RunMain()`` prepends a potentially unsafe path to
585-
:data:`sys.path` at startup:
584+
If equals to zero, ``Py_RunMain()`` prepends a potentially unsafe
585+
:data:`sys._path0` to :data:`sys.path` at startup:
586586
587587
* If :c:member:`argv[0] <PyConfig.argv>` is equal to ``L"-m"``
588588
(``python -m module``), prepend the current working directory.

‎Doc/library/sys.rst‎

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1394,6 +1394,24 @@ always available. Unless explicitly noted otherwise, all variables are read-only
13941394
* Module :mod:`site` This describes how to use .pth files to
13951395
extend :data:`sys.path`.
13961396

1397+
.. data:: _path0
1398+
1399+
Path prepended to :data:`sys.path` at Python startup:
1400+
1401+
* Current directory if running a module
1402+
with ``python -m module`` command line.
1403+
* Script's directory if running a script
1404+
with ``python script.py``.
1405+
* Otherwise an empty string, which means the current working directory,
1406+
for ``python -c code`` or ``python`` (REPL) command line.
1407+
1408+
If :option:`-P` or :option:`-I` command line options are used or if
1409+
:envvar:`PYTHONSAFEPATH` environment variable is used, :data:`sys._path0` is
1410+
``None`` and :data:`sys.path` is left unchanged.
1411+
1412+
.. versionadded:: next
1413+
1414+
13971415
.. data:: path_hooks
13981416

13991417
A list of callables that take a path argument to try to create a

‎Doc/using/cmdline.rst‎

Lines changed: 11 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -310,10 +310,12 @@ Miscellaneous options
310310
Run Python in isolated mode. This also implies :option:`-E`, :option:`-P`
311311
and :option:`-s` options.
312312

313-
In isolated mode :data:`sys.path` contains neither the script's directory nor
314-
the user's site-packages directory. All ``PYTHON*`` environment
315-
variables are ignored, too. Further restrictions may be imposed to prevent
316-
the user from injecting malicious code.
313+
In isolated mode, Python does not prepend the potentially unsafe path
314+
:data:`sys._path0` to :data:`sys.path`, and ignores all ``PYTHON*``
315+
environment variables, e.g. :envvar:`PYTHONPATH` and :envvar:`PYTHONHOME`,
316+
that might be set.
317+
Further restrictions may be imposed to prevent the user from injecting
318+
malicious code.
317319

318320
.. versionadded:: 3.4
319321

@@ -341,14 +343,8 @@ Miscellaneous options
341343

342344
.. option:: -P
343345

344-
Don't prepend a potentially unsafe path to :data:`sys.path`:
345-
346-
* ``python -m module`` command line: Don't prepend the current working
347-
directory.
348-
* ``python script.py`` command line: Don't prepend the script's directory.
349-
If it's a symbolic link, resolve symbolic links.
350-
* ``python -c code`` and ``python`` (REPL) command lines: Don't prepend an
351-
empty string, which means the current working directory.
346+
Don't prepend the potentially unsafe path :data:`sys._path0` to
347+
:data:`sys.path`.
352348

353349
See also the :envvar:`PYTHONSAFEPATH` environment variable, and :option:`-E`
354350
and :option:`-I` (isolated) options.
@@ -746,7 +742,9 @@ conflict.
746742
.. envvar:: PYTHONSAFEPATH
747743

748744
If this is set to a non-empty string, don't prepend a potentially unsafe
749-
path to :data:`sys.path`: see the :option:`-P` option for details.
745+
path :data:`sys._path0` to :data:`sys.path`.
746+
747+
See also the :option:`-P` option.
750748

751749
.. versionadded:: 3.11
752750

‎Lib/gzip.py‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -643,6 +643,11 @@ def decompress(data):
643643

644644
def main():
645645
from argparse import ArgumentParser
646+
import runpy
647+
648+
# Prevent shadowing a stdlib module from the extracted file
649+
runpy._remove_sys_path0()
650+
646651
parser = ArgumentParser(description=
647652
"A simple command line interface for the gzip module: act like gzip, "
648653
"but do not delete the input file.")

‎Lib/runpy.py‎

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -317,6 +317,26 @@ def run_path(path_name, init_globals=None, run_name=None):
317317
pass
318318

319319

320+
def _remove_sys_path0():
321+
"""
322+
Remove sys._path0 from sys.path. Do nothing if sys._path0 is
323+
sys._stdlib_dir or if it has already been removed. Similar to using -P
324+
(safe path) command line option.
325+
"""
326+
path0 = sys._path0
327+
if path0 is None:
328+
return
329+
330+
if path0 == sys._stdlib_dir:
331+
return
332+
333+
# Remove all path0 copies, not only the first one.
334+
# We are intentionally not trying to normalize & match alternate
335+
# representations of path0, this is only intended to remove copies
336+
# of the exact value the runtime itself may have added.
337+
sys.path[:] = [path for path in sys.path if path != path0]
338+
339+
320340
if __name__ == "__main__":
321341
# Run the module specified as the next command line argument
322342
if len(sys.argv) < 2:

‎Lib/tarfile.py‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3058,6 +3058,10 @@ def is_tarfile(name):
30583058

30593059
def main():
30603060
import argparse
3061+
import runpy
3062+
3063+
# Prevent shadowing stdlib modules from extracted files
3064+
runpy._remove_sys_path0()
30613065

30623066
description = 'A simple command-line interface for tarfile module.'
30633067
parser = argparse.ArgumentParser(description=description)

‎Lib/test/test_runpy.py‎

Lines changed: 92 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -20,6 +20,8 @@
2020
requires_subprocess,
2121
verbose,
2222
)
23+
from test import support
24+
from test.support import os_helper
2325
from test.support.import_helper import forget, make_legacy_pyc, unload
2426
from test.support.os_helper import create_empty_file, temp_dir, FakePath
2527
from test.support.script_helper import make_script, make_zip_script
@@ -875,5 +877,95 @@ def test_pymain_run_module(self):
875877
self.assertSigInt(["-m", ham.stem], cwd=ham.parent)
876878

877879

880+
class MiscTests(unittest.TestCase):
881+
def test_remove_sys_path0(self):
882+
# Check that all path0 copies are removed
883+
PATH0 = '/path/zero'
884+
STDLIB_PATH = '/to/stdlib'
885+
OTHER_PATH = '/other/path'
886+
sys_path = [PATH0, STDLIB_PATH, PATH0, '', OTHER_PATH, PATH0]
887+
expected = [STDLIB_PATH, '', OTHER_PATH]
888+
with (support.swap_attr(sys, '_stdlib_dir', STDLIB_PATH),
889+
support.swap_attr(sys, '_path0', PATH0),
890+
support.swap_attr(sys, 'path', sys_path)):
891+
runpy._remove_sys_path0()
892+
self.assertEqual(sys.path, expected)
893+
894+
# Calling it twice does nothing
895+
runpy._remove_sys_path0()
896+
self.assertEqual(sys.path, expected)
897+
898+
# Do nothing if sys._path0 is sys._stdlib_dir
899+
sys_path = ['', STDLIB_PATH, OTHER_PATH]
900+
expected = sys_path.copy()
901+
with (support.swap_attr(sys, '_stdlib_dir', STDLIB_PATH),
902+
support.swap_attr(sys, '_path0', STDLIB_PATH),
903+
support.swap_attr(sys, 'path', sys_path)):
904+
runpy._remove_sys_path0()
905+
self.assertEqual(sys.path, expected)
906+
907+
# Do nothing if sys._path0 is None
908+
sys_path = ['', STDLIB_PATH, OTHER_PATH]
909+
expected = sys_path.copy()
910+
with (support.swap_attr(sys, '_stdlib_dir', STDLIB_PATH),
911+
support.swap_attr(sys, '_path0', None),
912+
support.swap_attr(sys, 'path', sys_path)):
913+
runpy._remove_sys_path0()
914+
self.assertEqual(sys.path, expected)
915+
916+
def run_python(self, *args, PYTHONPATH=None):
917+
cmd = [sys.executable, *args]
918+
env = support.make_clean_env()
919+
if PYTHONPATH:
920+
env['PYTHONPATH'] = PYTHONPATH
921+
proc = subprocess.run(cmd, text=True, stderr=subprocess.PIPE, env=env)
922+
self.assertEqual(proc.returncode, 0, proc.stderr)
923+
924+
@support.requires_subprocess()
925+
def test_remove_sys_path0_run_python(self):
926+
CODE = textwrap.dedent("""
927+
import runpy, sys
928+
929+
# Remove sys.path[0]
930+
orig_path = sys.path.copy()
931+
path0 = sys._path0
932+
runpy._remove_sys_path0()
933+
934+
if path0 is not None and path0 in sys.path:
935+
raise Exception(f"path0 {path0!r} found in sys.path {sys.path!r}")
936+
if not sys.flags.safe_path:
937+
if sys.path == orig_path:
938+
raise Exception("sys.path was not changed")
939+
else:
940+
if sys.path != orig_path:
941+
raise Exception("sys.path was changed")
942+
943+
# Calling it twice must do nothing
944+
clean_path = sys.path.copy()
945+
runpy._remove_sys_path0()
946+
if sys.path != clean_path:
947+
raise Exception("sys.path was changed")
948+
""")
949+
950+
# Test "python -c code"
951+
self.run_python('-c', CODE)
952+
self.run_python('-P', '-c', CODE)
953+
954+
with tempfile.TemporaryDirectory() as tmpdir:
955+
name = 'test_remove_sys_path0'
956+
filename = os.path.join(tmpdir, f'{name}.py')
957+
with open(filename, 'w') as fp:
958+
fp.write(CODE)
959+
960+
# Test "python script.py"
961+
self.run_python(filename)
962+
self.run_python('-P', filename)
963+
964+
with os_helper.change_cwd(tmpdir):
965+
# Test "python -m module"
966+
self.run_python('-m', name)
967+
self.run_python('-P', '-m', name, PYTHONPATH=tmpdir)
968+
969+
878970
if __name__ == "__main__":
879971
unittest.main()

‎Lib/test/test_sys.py‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1416,6 +1416,15 @@ def test_pystats(self):
14161416
def test_disable_gil_abi(self):
14171417
self.assertEqual('t' in sys.abiflags, support.Py_GIL_DISABLED)
14181418

1419+
def test_path(self):
1420+
self.assertIsInstance(sys.path, list)
1421+
for path in sys.path:
1422+
self.assertIsInstance(path, str)
1423+
1424+
def test_path0(self):
1425+
self.assertTrue(sys._path0 is None or isinstance(sys._path0, str),
1426+
sys._path0)
1427+
14191428

14201429
@test.support.cpython_only
14211430
class UnraisableHookTest(unittest.TestCase):

‎Lib/test/test_zipfile/test_core.py‎

Lines changed: 18 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -49,6 +49,16 @@ def get_files(test):
4949
yield f
5050
test.assertFalse(f.closed)
5151

52+
53+
@contextlib.contextmanager
54+
def save_restore_sys_path():
55+
old_path = sys.path.copy()
56+
try:
57+
yield
58+
finally:
59+
sys.path[:] = old_path
60+
61+
5262
class AbstractTestsWithSourceFile:
5363
@classmethod
5464
def setUpClass(cls):
@@ -3674,17 +3684,18 @@ def test_add_comment(self):
36743684
def test_cli_with_metadata_encoding(self):
36753685
errmsg = "Non-conforming encodings not supported with -c."
36763686
args = ["--metadata-encoding=shift_jis", "-c", "nonesuch", "nonesuch"]
3677-
with captured_stdout() as stdout:
3678-
with captured_stderr() as stderr:
3687+
with (captured_stdout() as stdout,
3688+
captured_stderr() as stderr,
3689+
save_restore_sys_path()):
36793690
self.assertRaises(SystemExit, zipfile.main, args)
36803691
self.assertEqual(stdout.getvalue(), "")
36813692
self.assertIn(errmsg, stderr.getvalue())
36823693

3683-
with captured_stdout() as stdout:
3694+
with captured_stdout() as stdout, save_restore_sys_path():
36843695
zipfile.main(["--metadata-encoding=shift_jis", "-t", TESTFN])
36853696
listing = stdout.getvalue()
36863697

3687-
with captured_stdout() as stdout:
3698+
with captured_stdout() as stdout, save_restore_sys_path():
36883699
zipfile.main(["--metadata-encoding=shift_jis", "-l", TESTFN])
36893700
listing = stdout.getvalue()
36903701
for name in self.file_names:
@@ -3703,7 +3714,9 @@ def test_cli_with_metadata_encoding_extract(self):
37033714
except UnicodeEncodeError:
37043715
self.skipTest(f'cannot encode file name {fn!a}')
37053716

3706-
zipfile.main(["--metadata-encoding=shift_jis", "-e", TESTFN, TESTFN2])
3717+
with save_restore_sys_path():
3718+
zipfile.main(["--metadata-encoding=shift_jis", "-e",
3719+
TESTFN, TESTFN2])
37073720
listing = os.listdir(TESTFN2)
37083721
for name in self.file_names:
37093722
self.assertIn(name, listing)

‎Lib/zipfile/__init__.py‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2340,6 +2340,10 @@ def _compile(file, optimize=-1):
23402340

23412341
def main(args=None):
23422342
import argparse
2343+
import runpy
2344+
2345+
# Prevent shadowing stdlib modules from extracted files
2346+
runpy._remove_sys_path0()
23432347

23442348
description = 'A simple command-line interface for zipfile module.'
23452349
parser = argparse.ArgumentParser(description=description)

0 commit comments

Comments
 (0)