[aik099]

When executing tests in Google Chrome (or any Chromium-based Web Browser) on Mac you'll notice your temp folder starts to grow faster than lightning strikes.

This happens because upon launching, a Web Browser copies itself (the whole ~300MB app folder) into a temp folder. Normally a cleanup helper deletes that copy when the browser exits, but ChromeDriver kills the browser with SIGKILL, so the helper never runs (details in #53 (comment)). macOS clears that folder only on reboot.

Since numerous sessions are started/stopped during a test suite run, you'll get numerous ~300MB folders residing under a temp folder.

This affects regular Chrome, Edge, Brave, etc. on macOS. Only Chrome for Testing doesn't clone itself.

Solution: specify --disable-features=MacAppCodeSignClone in the default Chrome arguments. Cloning only matters when the browser is updated while running, which tests don't need. On other OSes the flag does nothing.

Why not other options:

• updating ChromeDriver: the fix in https://issues.chromium.org/issues/379125944 only disables cloning in Chrome for Testing builds;
• goog:chromeOptions.quitGracefully: true: the cleanup helper would run, but every session stop gets slower and a clone is still made on each start;
• documenting it: anyone running tests locally on macOS fills up their disk before finding that page.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 88.86%. Comparing base (efa6de0) to head (1dbfc29).

Additional details and impacted files

@@            Coverage Diff            @@
##               main      #53   +/-   ##
=========================================
  Coverage     88.86%   88.86%           
  Complexity      197      197           
=========================================
  Files             1        1           
  Lines           494      494           
=========================================
  Hits            439      439           
  Misses           55       55           

Flag	Coverage Δ
chrome	88.25% <ø> (ø)
chromium	87.85% <ø> (ø)
edge	88.25% <ø> (ø)
firefox	88.05% <ø> (ø)
selenium-2.53.1	87.04% <ø> (ø)
selenium-3	88.05% <ø> (ø)
selenium-4	88.46% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[aik099]

Tested Chromedriver 131.0.6778.139 and the issue is still there. Even though https://issues.chromium.org/issues/379125944 reported (if I'm reading that page correctly), that issue was fixed in the 131.0.6778.70. 😞

[uuf6429]

I'm using other arguments in the same way you did it (e.g. args: ["headless=new"] for --headless=new), so I can confidently say that part works. 🤔

Could you maybe run it and then either inspect selenium settings in http://localhost:4444/ui or the chrome settings with chrome://version/? (you should be able to see the CLI args)

[stof]

Tested Chromedriver 131.0.6778.139 and the issue is still there.

Are you using Chrome for Testing or a stable Chrome/Chromium release ?
The issue has been fixed for Chrome for Testing only.

[aik099]

Tested Chromedriver 131.0.6778.139 and the issue is still there.

Are you using Chrome for Testing or a stable Chrome/Chromium release ? The issue has been fixed for Chrome for Testing only.

I'm using Brave Browser (which is Chromium-based). I guess specifying --disable-features=MacAppCodeSignClone either in driver defaults or manually is the only way to avoid huge temp folder sizes.

[oleg-andreyev]

This is some sort of security/sandboxing , are we sure we want to disable it? I'd rather document it. MacOS itself should handle cleanup, plus how many people or running Chromedriver on MacOS for CI.

[uuf6429]

Off-topic warning

I have mixed feelings about the default capabilities (including the current one).

Cons:

• we end up shipping defaults that might not be what the end users wants/expects
• they are difficult to fine-tune - users need to redefine the whole config, or modify a copy of the config (which is why we made it public const). But it is annoying and cumbersome for end users and for us (to maintain backword compatibility)

Pros:

• capabilities are in general poorly documented, providing sensible defaults is a huge advantage to our users, so they can jump straight into using the driver
• the capabilities are better documented on our side - users can see the code and the commits introducing the changes, plus we can/should add links to official docs of the capabilities
• one of the objectives of mink and the web drivers is to provide a unified experience - if we require our users to hack up capabilities to make browsers behave in a consistent way, it feels like that goes against the core concept.

In general, I feel like we should have some sort of mechanism (or a sub-project / sub-driver?) to handle capabilities better. For example, it would be nice for our users to just pass a "enable_headless" config which we translate to the right set of capabilities for the specified browser.

I haven't spent too much time thinking about this, so it would be really nice if someone else took over or thinks it over.

[aik099]

Could you maybe run it and then either inspect selenium settings in http://localhost:4444/ui or the chrome settings with chrome://version/? (you should be able to see the CLI args)

@uuf6429 , I've found out why driver defaults weren't used. I'm passing this to allow chromedriver to operate on non-Chrome (Chromium-based) Web Browser:

'goog:chromeOptions' => array(
    'binary' => '/Applications/Brave Browser.app/Contents/MacOS/Brave Browser'
),

and that completely replaced driver-provided defaults:

'goog:chromeOptions' => [
        // disable "Chrome is being controlled.." notification bar
        'excludeSwitches' => ['enable-automation'],
        'args' => array(
            // disable browser folder cloning on Mac, when tests are executed
            'disable-features=MacAppCodeSignClone',
        ),
 ],

. The mix-and-match approach and on/off switch, as @oleg-andreyev proposed, for default desired capabilities is an interesting thing to discuss.

In general, I feel like we should have some sort of mechanism (or a sub-project / sub-driver?) to handle capabilities better. For example, it would be nice for our users to just pass a "enable_headless" config which we translate to the right set of capabilities for the specified browser.

@oleg-andreyev , we're discussing this a bit in the #51 . Probably we can have mink:browserOptions sub-array in desired capabilities, that is empty by default and user can enable/disable flags, that conceptually enable/disable feature sub-sets of desired capabilities, like headless=>true, honor_browser_defaults=>true, honor_driver_defaults=>true, etc.

This is some sort of security/sandboxing , are we sure we want to disable it? I'd rather document it. MacOS itself should handle cleanup, plus how many people or running Chromedriver on MacOS for CI.

@oleg-andreyev , I have no idea what this functionality is (probably Chromium source code at https://source.chromium.org/search?q=MacAppCodeSignClone&sq=&ss=chromium can reveal it), but it surely eats a lot of disk space pretty quickly and since on macOS you don't reboot quite often to auto-clear temp folder (as you do on Windows) you can run out of disk space soon enough.

Before creating a PR I prefer to execute the whole Selenium-based test suite locally to see if it passes and only then let GitHub Actions run it. This would save me time waiting for GitHub Actions to crash after some time when I can see the same crash locally and already start to fix things.

[aik099]

This is some sort of security/sandboxing , are we sure we want to disable it? I'd rather document it. MacOS itself should handle cleanup, plus how many people or running Chromedriver on MacOS for CI.

The comments in https://source.chromium.org/chromium/chromium/src/+/main:chrome/browser/mac/code_sign_clone_manager.h?q=MacAppCodeSignClone&ss=chromium explain a lot why this was done. If I've understood correctly, then it was done for updating to new version purposes and there is a background process to delete the cloned folder. My guess is that this background process doesn't work or is stopped as long as Selenium-started browser exists as part of driver.quit() Selenium command.

I have no idea really why the clones are not deleted, when they should.

More links:

• https://issues.chromium.org/issues/338582873 (when this was implemented)
• https://docs.google.com/document/u/1/d/e/2PACX-1vSWYGCD-hFxN64Fdha43dIqtiC06ckoOW0nNPnjv4m_hwnAfOS6vZuWp04k8hCCX9s1fZRnQuI2cwAv/pub (how it was planned to be implemented)

[aik099]

I've created #54 for discussing general problems with desired capability handling.

[aik099]

Found why the clones aren't deleted.

The clone is deleted by a cleanup helper process, which the browser launches on normal shutdown (code_sign_clone_manager.mm). But ChromeDriver doesn't shut the browser down normally: when it uses its own temporary profile (the default), it just sends SIGKILL (ChromeDesktopImpl::QuitImpl):

// If we're using a temporary user data directory, we're going to delete the temporary
// directory anyway, so just send SIGKILL immediately.
bool quit_gracefully = quit_gracefully_ || !user_data_dir_.IsValid();

So every session leaves a clone in /var/folders/.../X/<bundle id>.code_sign_clone/, and macOS cleans that folder only on reboot. With Brave 154 a few local test runs left 61 of them.

The goog:chromeOptions.quitGracefully: true capability makes ChromeDriver send Browser.close instead (and wait up to 10s for the browser to exit), so the cleanup would run. But I think that would slow down every session stop, and the clone would still be created on each start.

Chrome for Testing doesn't clone at all (cloning is disabled with #if !BUILDFLAG(CHROME_FOR_TESTING)), but regular Chrome, Brave, Edge, etc. do.

So I think the original proposal (--disable-features=MacAppCodeSignClone) is better: no clone is created, and tests don't need it anyway (it only matters when the browser updates while running).

[aik099]

Merge plan:

1. merge the Test Edge default capabilities #73 (adds capability test for edge)
2. rebase this PR on top of it
3. change this PR to also patch Edge capability test
4. merge this PR

P.S.
Adding a macOS-specific flag won't hurt browsers running on different OSes.

[aik099]

I've updated the PR description with the findings from the comments above: why the clones aren't deleted, which browsers are affected and why updating ChromeDriver, quitGracefully or documenting it isn't enough.