Skip to content

TarFile.extractall() doesn't properly handle UNC paths #143663

Closed

Description

@0xd6cb6d73

Bug report

Bug description:

TarFile.extractall() does not perform path normalization before calling os.lstat(). This is an issue on Windows as the resulting path ends us using Windows-style (\\) and unix-style (/) path separators. This results in the following error: OSError: [WinError 123] The filename, directory name, or volume label syntax is incorrect: '\\\\?\\Z:\\extracted\\double-conversion-v3.1.4.tar.gz\\double-conversion-3.1.4/test/cctest'.

from Lib import tarfile


with tarfile.open(r'\\?\Z:\downloads\double-conversion-v3.1.4.tar.gz') as t:
    t.extractall(r'\\?\Z:\extracted\double-conversion-v3.1.4.tar.gz')

CPython versions tested on:

3.13, CPython main branch

Operating systems tested on:

Windows

Linked PRs

Activity

  1. picnixz commented on Jan 10, 2026

    @picnixz
    Member

    I don't think we should perform normalization by ourselves. For instance _getmember() has a normalize flag but we never use it by default, only when dealing with symlinks (in _find_link_target).

    As such, I think this behavior should rather be documented, though most of the APIs assume that path normalization doesn't happen unless stated otherwise.

    In ZipFile there is some sanitization that is done before and there seems to be some normalization:

            # build the destination pathname, replacing
            # forward slashes to platform specific separators.
            arcname = member.filename.replace('/', os.path.sep)
    
            if os.path.altsep:
                arcname = arcname.replace(os.path.altsep, os.path.sep)
            # interpret absolute pathname as relative, remove drive letter or
            # UNC path, redundant separators, "." and ".." components.
            arcname = os.path.splitdrive(arcname)[1]
            invalid_path_parts = ('', os.path.curdir, os.path.pardir)
            arcname = os.path.sep.join(x for x in arcname.split(os.path.sep)
                                       if x not in invalid_path_parts)
            if os.path.sep == '\\':
                # filter illegal characters on Windows
                arcname = self._sanitize_windows_name(arcname, os.path.sep)
    
            if not arcname and not member.is_dir():
                raise ValueError("Empty filename.")
    
            targetpath = os.path.join(targetpath, arcname)
            targetpath = os.path.normpath(targetpath)

    However, os.path.normpath does more than just changing separtors. It also collapses .. and changes the meaning of symlinks:

    This string manipulation may change the meaning of a path that contains symbolic links.

    In particular, this may not be desired. I think it's better if you already normalize the input path instead.

  2. added
    type-featureA feature request or enhancement
    stdlibStandard Library Python modules in the Lib/ directory
    and removed
    type-bugAn unexpected behavior, bug, or error
    on Jan 10, 2026
  3. picnixz commented on Jan 10, 2026

    @picnixz
    Member

    Related/prior discussion #73974 and #57911.

  4. picnixz commented on Jan 10, 2026

    @picnixz
    Member

    Also, I'm unsure whether the reason is because of UNC paths or not (does the issue also occur when you do not use r'\\?\Z:\downloads' but use paths of the form r'C:\Downloads' directly? This could also be a separate issue in tarfile itself.

  5. 0xd6cb6d73 commented on Jan 10, 2026

    @0xd6cb6d73
    ContributorAuthor

    I came upon this issue here: https://github.com/facebookincubator/fizz/blob/v2026.01.05.00/build/fbcode_builder/getdeps/fetcher.py#L1013. Where the following code does normalize the input path as you suggest:

    if is_windows():
                # Ensure that we don't fall over when dealing with long paths
                # on windows
                src = r"\\?\%s" % os.path.normpath(self.src_dir)
    else:
        src = self.src_dir
    
    with opener(self.file_name) as t:
        # The `str` here is necessary to ensure that we don't pass a unicode
        # object down to tarfile.extractall on python2.  When extracting
        # the boost tarball it makes some assumptions and tries to convert
        # a non-ascii path to ascii and throws.
        src = str(src)
        t.extractall(src)

    Where opener​ is tarfile.open​().

    Looking at the error message, the part of the path that contains the unix-style separators come from the tarball's internal structure. Therefore it looks like simply normalizing the input path may not be enough.

    To your second point, the issue does not manifest when the \\?\ prefix isn't used.

  6. picnixz commented on Jan 10, 2026

    @picnixz
    Member

    To your second point, the issue does not manifest when the \?\ prefix isn't used.

    Yeah so I think the problem is more the UNC rather than the missing normalization. I suspect that normalization wouldn't be the only way to solve this, but we could detect UNC paths like that and make this work. For instance, the culprit may be _get_filtered_attrs instead which seems to do:

        if name.startswith(('/', os.sep)):
            name = new_attrs['name'] = member.path.lstrip('/' + os.sep)

    I'm not familiar enough with this part of the code and Windows though to help you so I'll ask some experts who worked on paths and tarfile filters @barneygale @encukou @jaraco (and @zooba as our Windows expert)

  7. changed the title [-]TarFile.extractall() doesn't normalize path[/-] [+]TarFile.extractall() doesn't properly handle UNC paths[/+] on Jan 10, 2026
  8. added
    type-bugAn unexpected behavior, bug, or error
    and removed
    type-featureA feature request or enhancement
    on Jan 10, 2026
  9. 0xd6cb6d73 commented on Jan 10, 2026

    @0xd6cb6d73
    ContributorAuthor

    Maybe just semantics, but I think such a detection should be careful to distinguish between UNC paths and "raw" paths such as this. My understanding of the docs is that the \\?\ prefix simply tells Win32 "do not interpret this path". As such, I think that \\my_share\export\dir\..\myfile.tar should not be handled the same way as \\?\Z:\downloads\..\double-conversion-v3.1.4.tar.gz. Indeed, in the former case .. should be expanded, whereas it should be treated as a literal in the latter case.

    It looks like the above may be side-stepped by replacing the tarball separator by the OS-specific one in _get_extract_tarinfo which is called both by extract and extractall. This solves the issue for both raw and normal paths styles.

  10. encukou commented on Jan 12, 2026

    @encukou
    Member

    tarfile follows the behaviour of os.path.join(), which is:

    >>> import ntpath
    >>> ntpath.join(r'\\?\Z:\a\b\c', 'x/y/z')
    '\\\\?\\Z:\\a\\b\\c\\x/y/z'

    And that's an incorrect path.

    It looks like the above may be side-stepped by replacing the tarball separator by the OS-specific one in _get_extract_tarinfo which is called both by extract and extractall. This solves the issue for both raw and normal paths styles.

    That does look like the correct solution. If I was designing tarfile now, with the benefit of hindsight, it would strictly separate “archive” paths from “filesystem” ones.

  11. 0xd6cb6d73 commented on Jan 12, 2026

    @0xd6cb6d73
    ContributorAuthor

    I have updated the PR branch to implement this, unless you want me to open another one?

  12. picnixz commented on Jan 12, 2026

    @picnixz
    Member

    I will re-open the PR!

  13. victorvianna commented on Nov 19, 2026

    @victorvianna

    @picnixz Chromium just ran into this issue. Is #143665 still being considered?

  14. added a commit that references this issue on Nov 19, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    OS-windowsstdlibStandard Library Python modules in the Lib/ directorytype-bugAn unexpected behavior, bug, or error

    Projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions