[zvonand]

Fixes defects 1–3 from the audit of native CREATE TABLE / DROP TABLE for DataLakeCatalog:

• CREATE TABLE ... AS from an S3 Tables catalog table without keys no longer throws LOGICAL_ERROR. A table shown without storage is accepted as a source only when the target is a DataLakeCatalog database.
• Engine SETTINGS of a table created in a DataLakeCatalog are now rejected when they would be lost when the table is reopened. This covers format settings, and storage settings that differ from the database value. iceberg_format_version and obsolete settings are still accepted. iceberg_* settings can now be set at the database level.
• Iceberg metadata-log now records the previous metadata file for TRUNCATE, OPTIMIZE ... MANIFEST, schema ALTERs and EXPORT PARTITION. Before, it recorded the new file itself or nothing.

Related: #2500
Follow-up for: #2305

Changelog category (leave one):

• Bug Fix (user-visible misbehavior in an official stable release)

Changelog entry (a user-readable short description of the changes that goes to CHANGELOG.md):

Fix CREATE TABLE ... AS from an S3 Tables table without keys, reject DataLakeCatalog table settings that are lost on reopen, allow iceberg_* settings at the DataLakeCatalog database level, and record the previous metadata file in Iceberg metadata-log for TRUNCATE, manifest compaction, ALTER and EXPORT PARTITION.

Documentation entry for user-facing changes

...

CI/CD Options

Exclude tests:

• Fast test
• Integration Tests
• Stateless tests
• Stateful tests
• Unit tests
• Performance tests
• Aarch64 tests
• All with ASAN
• All with TSAN
• All with MSAN
• All with UBSAN
• All with Coverage
• All Regression
• Disable CI Cache

Regression jobs to run:

• Fast suites (mostly <1h)
• Aggregate Functions (2h)
• Alter (1.5h)
• Benchmark (30m)
• CAS (content-addressed storage; Antalya only)
• ClickHouse Keeper (1h)
• Iceberg (2h)
• LDAP (1h)
• OAuth (5m)
• Parquet (1.5h)
• RBAC (1.5h)
• SSL Server (1h)
• S3 (2h)
• S3 Export (2h)
• Swarms (30m)
• Tiered Storage (2h)

[github-actions]

Workflow [PR], commit [69fa635]

[zvonand]

@blau-ai

[blau-ai]

CI triage for 69fa635

Verdict: none of the 7 failing checks look caused by this PR. 5 also fail on other antalya-26.8 PRs, including the already-merged #2773. 1 is infra (a test container could not start). 1 (s3_azure_1) has the same root cause as a failure that does repeat on other PRs, but only this PR ran that suite. No code changes are needed here.

Check	Failing tests	Verdict
Integration 5/8	21× test_storage_iceberg_with_spark/test_export_partition_iceberg.py	Pre-existing on base
Integration 5/8	37× test_cas_gcs/test.py	Infra (container didn't start)
Integration 1/8, 7/8	2× test_backward_compatibility/test_custom_s3_disk_remote_url_allow_hosts.py	Pre-existing on base
Stateless amd_asan_ubsan cas s3 1/2, 2/2	none (job stopped mid-run)	Pre-existing (job timeout)
Regression s3_azure_1	/s3/azure/part 1/disk/{remote host filter,syntax}	Not PR-related (same root cause as the backward-compat failure)
Regression swarms	3 scenarios	Pre-existing on base

Details

1. test_export_partition_iceberg.py (21 tests): pre-existing

Code: 497. DB::Exception: default: Not enough privileges. To execute this query, it's necessary to have the grant SHOW NAMED COLLECTIONS SECRETS ON s3.
  at helpers/iceberg_utils.py:480 create_iceberg_table

This touches Iceberg, so I checked it carefully. The same 21 tests fail with the same count on #2773 (merged), #2794 and #2800. None of those touch DataLake code. The PR's InterpreterCreateQuery change only affects the CREATE ... AS <datalake catalog table> path. These tests fail earlier, at a named-collection access check. → Base-branch problem: either the test user needs the grant or there's an access regression on antalya-26.8. Fix it separately.

2. test_cas_gcs/test.py (37 tests): infra / test environment

Timed out while waiting for instance `fakegcs' ... to start.
bash: line 1: /usr/bin/clickhouse: cannot execute: required file not found

The fake GCS container never started because the binary inside it couldn't be executed, so every test in the module failed at the fixture. The PR doesn't touch CAS/GCS. Safe to re-run. If it repeats, the fakegcs container setup needs fixing on base.

3. test_custom_s3_disk_remote_url_allow_hosts.py (2 tests, shards 1/8 and 7/8): pre-existing
The node container times out on startup (cluster.start() fixture). Same 2 failures on #2773, #2794 and #2800 (and in their targeted jobs). Probably related to the upstream backports ClickHouse#124453 / ClickHouse#124930 (remote_url_allow_hosts for S3 disks) on base.

4. Stateless amd_asan_ubsan, cas s3 storage, parallel 1/2 and 2/2: pre-existing job timeout
No individual test failed. Both shards were cut off at about 3h38m, at 3661/6890 and 2820/6931 tests. #2773 shows the same ERROR-in-the-middle-of-the-run pattern on both shards. The suite is too slow under ASan with CAS S3 and doesn't fit the job timeout.

5. Regression s3_azure_1: not PR-related

ConfigReloader: Error updating configuration ... URL "http://azurite:10000" is not allowed in configuration file, see <remote_url_allow_hosts>. (UNACCEPTABLE_URL)

The disk config is rejected by the new remote_url_allow_hosts enforcement on S3/Azure disks. That's the same base change as item 3, and the PR doesn't touch disk config. Caveat: none of the other PRs I compared ran this suite, so I couldn't check it directly against base. The regression config probably needs azurite added to remote_url_allow_hosts.

6. Regression swarms: pre-existing
Fails on every recent antalya-26.8 PR I checked (#2773, #2794, #2799, #2800).

Suggested next steps

• Nothing to fix in this PR. The rest of the Iceberg/DataLake integration shards that cover the changed code passed (e.g. Integration 2/8).
• Optionally re-run Integration 5/8 to get a clean test_cas_gcs result.
• Track separately on base: the SHOW NAMED COLLECTIONS SECRETS failure in the export-partition tests, the remote_url_allow_hosts breakage (backward-compat test + azure regression), and the CAS-S3 ASan stateless timeout.

_{Evidence: praktika result_pr.json for this PR and for #2773/#2794/#2800, plus the GitHub Actions logs of run 42001873005. Several jobs (stress, AST fuzzer, some stateless) were DROPPED/skipped, so they give no signal either way.}

[zvonand]

@Selfeer could you pls also verify this one?

[Selfeer]

On it