You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
TarFile.extractall() does not perform path normalization before calling os.lstat(). This is an issue on Windows as the resulting path ends us using Windows-style (\\) and unix-style (/) path separators. This results in the following error: OSError: [WinError 123] The filename, directory name, or volume label syntax is incorrect: '\\\\?\\Z:\\extracted\\double-conversion-v3.1.4.tar.gz\\double-conversion-3.1.4/test/cctest'.
I don't think we should perform normalization by ourselves. For instance _getmember() has a normalize flag but we never use it by default, only when dealing with symlinks (in _find_link_target).
As such, I think this behavior should rather be documented, though most of the APIs assume that path normalization doesn't happen unless stated otherwise.
In ZipFile there is some sanitization that is done before and there seems to be some normalization:
# build the destination pathname, replacing# forward slashes to platform specific separators.arcname=member.filename.replace('/', os.path.sep)
ifos.path.altsep:
arcname=arcname.replace(os.path.altsep, os.path.sep)
# interpret absolute pathname as relative, remove drive letter or# UNC path, redundant separators, "." and ".." components.arcname=os.path.splitdrive(arcname)[1]
invalid_path_parts= ('', os.path.curdir, os.path.pardir)
arcname=os.path.sep.join(xforxinarcname.split(os.path.sep)
ifxnotininvalid_path_parts)
ifos.path.sep=='\\':
# filter illegal characters on Windowsarcname=self._sanitize_windows_name(arcname, os.path.sep)
ifnotarcnameandnotmember.is_dir():
raiseValueError("Empty filename.")
targetpath=os.path.join(targetpath, arcname)
targetpath=os.path.normpath(targetpath)
However, os.path.normpath does more than just changing separtors. It also collapses .. and changes the meaning of symlinks:
This string manipulation may change the meaning of a path that contains symbolic links.
In particular, this may not be desired. I think it's better if you already normalize the input path instead.
Also, I'm unsure whether the reason is because of UNC paths or not (does the issue also occur when you do not use r'\\?\Z:\downloads' but use paths of the form r'C:\Downloads' directly? This could also be a separate issue in tarfile itself.
ifis_windows():
# Ensure that we don't fall over when dealing with long paths# on windowssrc=r"\\?\%s"%os.path.normpath(self.src_dir)
else:
src=self.src_dirwithopener(self.file_name) ast:
# The `str` here is necessary to ensure that we don't pass a unicode# object down to tarfile.extractall on python2. When extracting# the boost tarball it makes some assumptions and tries to convert# a non-ascii path to ascii and throws.src=str(src)
t.extractall(src)
Where opener is tarfile.open().
Looking at the error message, the part of the path that contains the unix-style separators come from the tarball's internal structure. Therefore it looks like simply normalizing the input path may not be enough.
To your second point, the issue does not manifest when the \\?\ prefix isn't used.
To your second point, the issue does not manifest when the \?\ prefix isn't used.
Yeah so I think the problem is more the UNC rather than the missing normalization. I suspect that normalization wouldn't be the only way to solve this, but we could detect UNC paths like that and make this work. For instance, the culprit may be _get_filtered_attrs instead which seems to do:
I'm not familiar enough with this part of the code and Windows though to help you so I'll ask some experts who worked on paths and tarfile filters @barneygale@encukou@jaraco (and @zooba as our Windows expert)
Maybe just semantics, but I think such a detection should be careful to distinguish between UNC paths and "raw" paths such as this. My understanding of the docs is that the \\?\ prefix simply tells Win32 "do not interpret this path". As such, I think that \\my_share\export\dir\..\myfile.tar should not be handled the same way as \\?\Z:\downloads\..\double-conversion-v3.1.4.tar.gz. Indeed, in the former case .. should be expanded, whereas it should be treated as a literal in the latter case.
It looks like the above may be side-stepped by replacing the tarball separator by the OS-specific one in _get_extract_tarinfo which is called both by extract and extractall. This solves the issue for both raw and normal paths styles.
It looks like the above may be side-stepped by replacing the tarball separator by the OS-specific one in _get_extract_tarinfo which is called both by extract and extractall. This solves the issue for both raw and normal paths styles.
That does look like the correct solution. If I was designing tarfile now, with the benefit of hindsight, it would strictly separate “archive” paths from “filesystem” ones.
Activity
picnixz commentedon Jan 10, 2026
I don't think we should perform normalization by ourselves. For instance
_getmember()has anormalizeflag but we never use it by default, only when dealing with symlinks (in_find_link_target).As such, I think this behavior should rather be documented, though most of the APIs assume that path normalization doesn't happen unless stated otherwise.
In ZipFile there is some sanitization that is done before and there seems to be some normalization:
However,
os.path.normpathdoes more than just changing separtors. It also collapses..and changes the meaning of symlinks:In particular, this may not be desired. I think it's better if you already normalize the input path instead.
picnixz commentedon Jan 10, 2026
Related/prior discussion #73974 and #57911.
picnixz commentedon Jan 10, 2026
Also, I'm unsure whether the reason is because of UNC paths or not (does the issue also occur when you do not use
r'\\?\Z:\downloads'but use paths of the formr'C:\Downloads'directly? This could also be a separate issue in tarfile itself.0xd6cb6d73 commentedon Jan 10, 2026
I came upon this issue here: https://github.com/facebookincubator/fizz/blob/v2026.01.05.00/build/fbcode_builder/getdeps/fetcher.py#L1013. Where the following code does normalize the input path as you suggest:
Where
opener istarfile.open().Looking at the error message, the part of the path that contains the unix-style separators come from the tarball's internal structure. Therefore it looks like simply normalizing the input path may not be enough.
To your second point, the issue does not manifest when the
\\?\prefix isn't used.picnixz commentedon Jan 10, 2026
Yeah so I think the problem is more the UNC rather than the missing normalization. I suspect that normalization wouldn't be the only way to solve this, but we could detect UNC paths like that and make this work. For instance, the culprit may be
_get_filtered_attrsinstead which seems to do:I'm not familiar enough with this part of the code and Windows though to help you so I'll ask some experts who worked on paths and tarfile filters @barneygale @encukou @jaraco (and @zooba as our Windows expert)
[-]TarFile.extractall() doesn't normalize path[/-][+]TarFile.extractall() doesn't properly handle UNC paths[/+]0xd6cb6d73 commentedon Jan 10, 2026
Maybe just semantics, but I think such a detection should be careful to distinguish between UNC paths and "raw" paths such as this. My understanding of the docs is that the
\\?\prefix simply tells Win32 "do not interpret this path". As such, I think that\\my_share\export\dir\..\myfile.tarshould not be handled the same way as\\?\Z:\downloads\..\double-conversion-v3.1.4.tar.gz. Indeed, in the former case..should be expanded, whereas it should be treated as a literal in the latter case.It looks like the above may be side-stepped by replacing the tarball separator by the OS-specific one in
_get_extract_tarinfowhich is called both byextractandextractall. This solves the issue for both raw and normal paths styles.encukou commentedon Jan 12, 2026
tarfile follows the behaviour of
os.path.join(), which is:And that's an incorrect path.
That does look like the correct solution. If I was designing tarfile now, with the benefit of hindsight, it would strictly separate “archive” paths from “filesystem” ones.
0xd6cb6d73 commentedon Jan 12, 2026
I have updated the PR branch to implement this, unless you want me to open another one?
picnixz commentedon Jan 12, 2026
I will re-open the PR!
victorvianna commentedon Nov 19, 2026
@picnixz Chromium just ran into this issue. Is #143665 still being considered?
gh-143663: Perform path normalization in TarFile.extractall() (GH-143665