From b215d164eceae63dee07920a069063e4f1e44ac8 Mon Sep 17 00:00:00 2001 From: glasstiger Date: Thu, 5 Nov 2026 17:56:07 +0000 Subject: [PATCH 1/5] docs: update Enterprise SHOW CREATE DATABASE behavior --- documentation/changelog.mdx | 7 + documentation/concepts/live-views.md | 21 +- documentation/concepts/materialized-views.md | 4 + documentation/concepts/views.md | 35 +- .../copy-schema-between-instances.md | 49 +-- documentation/operations/upgrade.md | 21 ++ .../query/functions/access-control.md | 11 +- documentation/query/sql/create-live-view.md | 10 +- documentation/query/sql/create-mat-view.md | 13 +- documentation/query/sql/create-view.md | 12 + documentation/query/sql/pivot.md | 7 + documentation/query/sql/show.md | 299 +++++++++++------- documentation/security/governance.md | 20 +- documentation/security/rbac.md | 7 + 14 files changed, 355 insertions(+), 161 deletions(-) diff --git a/documentation/changelog.mdx b/documentation/changelog.mdx index 296809763..d84faf753 100644 --- a/documentation/changelog.mdx +++ b/documentation/changelog.mdx @@ -8,6 +8,13 @@ description: Recent updates and improvements to the QuestDB documentation. This page tracks significant updates to the QuestDB documentation. +## November 2026 + +### Updated + +- [SHOW CREATE DATABASE](/docs/query/sql/show/#show-create-database) - Documented Enterprise ACL output, principal/schema/grant ordering, the requirement for both `LIST USERS` and `USER DETAILS`, active storage policies, and replay limits. Documented the removal of `OWNED BY` from all `SHOW CREATE` output, and updated the schema-copy recipe, RBAC, and governance guidance +- [Views](/docs/concepts/views/#administrative-operations-in-view-metadata) - Documented restrictions on administrative operations in `PIVOT IN` subqueries, schema-only database dumps in materialized and live views, and [upgrade guidance](/docs/operations/upgrade/#schema-dumps-and-view-definitions) for existing scripts and views + ## October 2026 ### Releases diff --git a/documentation/concepts/live-views.md b/documentation/concepts/live-views.md index d0f27f4fc..d80748d3a 100644 --- a/documentation/concepts/live-views.md +++ b/documentation/concepts/live-views.md @@ -405,10 +405,12 @@ replaying it: `START FROM BEGINNING` to include all history still present in the base table. - If the view was created with a trailing `;`, the saved query keeps it inside `AS ( ... )` and replaying fails with `')' expected`. Remove that `;`. -- On QuestDB Enterprise, the saved `OWNED BY` names the user running - `SHOW CREATE`, and dropping the view removes its grants. Set `OWNED BY` to - the user or group that should receive the creator's permissions, and grant - the other permissions on the new view again. +- On QuestDB Enterprise, the saved DDL contains no `OWNED BY` or grants, and + dropping the view removes its grants. Save them first with + [`SHOW CREATE DATABASE INCLUDE (PERMISSIONS)`](/docs/query/sql/show/#enterprise-access-control) + or inspect them with `active_grants()`. On recreation, add `OWNED BY` if + another principal should receive the creator's permissions, and restore the + other grants on the new view. Dropping the invalid view removes its materialized rows, including rows whose source history is no longer retained by the base table. @@ -533,11 +535,12 @@ in [Base table lifecycle](#base-table-lifecycle). objects, such as `tables()`, follow the [materialized view rules](/docs/query/sql/create-mat-view/#catalogue-functions): in QuestDB Enterprise, only a creator with `SYSTEM ADMIN` can use them. -- **No administrative functions.** Enterprise functions that check the - caller's permissions, such as `permissions()` and `backup_instance_name()`, - are rejected, since a live view refreshes in the background. See - [query constraints](/docs/query/sql/create-live-view/#query-constraints) for - the full list. +- **No administrative functions or ACL dumps.** Enterprise functions that + check the caller's permissions, such as `permissions()` and + `backup_instance_name()`, and `SHOW CREATE DATABASE` with any ACL category + are rejected, since a live view refreshes in the background. Use + `INCLUDE (SCHEMA)` for schema metadata under the catalogue rules above. See + [query constraints](/docs/query/sql/create-live-view/#query-constraints). - **No TTL on the view.** Live-view disk growth is unbounded in this version. Base-table retention does not shrink the view, because `DROP PARTITION`, `TRUNCATE`, and TTL eviction on the base table leave the view's rows in place. diff --git a/documentation/concepts/materialized-views.md b/documentation/concepts/materialized-views.md index d7898c31d..a1207958a 100644 --- a/documentation/concepts/materialized-views.md +++ b/documentation/concepts/materialized-views.md @@ -548,6 +548,10 @@ Materialized view queries: refresh: `active_grants()`, `active_permissions()`, `backup_instance_name()`, `backup_status()`, `backups()`, `permissions()`, `reload_tick_calendars()`, and `reload_tls()` +- Must not use `SHOW CREATE DATABASE` with any ACL category when Enterprise + access control is enabled. The default includes ACL output; use + `INCLUDE (SCHEMA)` under the + [catalogue rules](/docs/query/sql/create-mat-view/#catalogue-functions) - Must use join conditions compatible with incremental refresh - When the base table uses [deduplication](/docs/concepts/deduplication/), non-aggregate columns must be a subset of the `DEDUP` keys diff --git a/documentation/concepts/views.md b/documentation/concepts/views.md index 0bc3d9dd0..e979f535a 100644 --- a/documentation/concepts/views.md +++ b/documentation/concepts/views.md @@ -418,9 +418,10 @@ they can read through `views()` and `SHOW CREATE VIEW`, still names the table. The view's definer permissions are preserved even if the creator's account is later disabled or deleted. -Access control functions such as `active_grants()` and `active_permissions()` -are an exception: in a view, they check the permissions of the user who queries -the view, not those of its definer. See +Access control functions such as `active_grants()` and `active_permissions()`, +and [`SHOW CREATE DATABASE`](/docs/query/sql/show/#use-in-views) with any ACL +category, are exceptions: in a view's query body, they check the permissions of +the user who queries the view, not those of its definer. See [access control functions](/docs/query/functions/access-control/#limitations). ### Metadata functions in views (Enterprise) {#metadata-functions-in-views} @@ -464,6 +465,34 @@ table-name functions name, so it resolves them with the reader's own permissions until you run [`ALTER VIEW`](/docs/query/sql/alter-view/) or `CREATE OR REPLACE VIEW` on it. +### Administrative operations in view metadata + +A view's `PIVOT IN` subquery runs while QuestDB derives the view's columns, +including during background recompilation and startup. It cannot use +administrative functions or administrative `SHOW` statements that need a +calling session or have side effects. This restriction also applies to nested +subqueries and operations reached through another view. + +With Enterprise access control enabled, `SHOW CREATE DATABASE` with any ACL +category counts as an administrative statement. Use +`SHOW CREATE DATABASE INCLUDE (SCHEMA)` if the subquery only needs schema +metadata. Catalogue queries that do not require a calling session remain +allowed. Administrative operations outside `PIVOT IN` in a regular view's +query body still run with the reader's permission checks. + +:::warning Existing views + +Creating, altering, or replacing a view with a prohibited operation fails. +Previously accepted views become invalid at their next compilation, including +at startup. Inspect `view_status` and `invalidation_reason` in +[`views()`](/docs/query/functions/meta/#views). Replace ACL dumps with +`INCLUDE (SCHEMA)` where appropriate, or remove the administrative operation +from the subquery, then update the definition with +[`ALTER VIEW`](/docs/query/sql/alter-view/) and run +[`COMPILE VIEW`](/docs/query/sql/compile-view/). + +::: + ### No column-level permissions on views Unlike tables, views do **not** support column-level permissions. You can only diff --git a/documentation/cookbook/operations/copy-schema-between-instances.md b/documentation/cookbook/operations/copy-schema-between-instances.md index bf2f63af5..dc92cb455 100644 --- a/documentation/cookbook/operations/copy-schema-between-instances.md +++ b/documentation/cookbook/operations/copy-schema-between-instances.md @@ -7,7 +7,8 @@ description: --- Recreate the structure of one QuestDB instance on another: tables, views, -materialized views, and, on QuestDB Enterprise, the access control layer. +materialized views, and live views, with the option to copy access control on +QuestDB Enterprise. ## Problem @@ -15,16 +16,16 @@ You want to copy the schema from one QuestDB instance to another. ## Solution -[`SHOW CREATE DATABASE`](/docs/query/sql/show/#show-create-database) returns a -data-free dump of the whole database, with one round-trippable DDL statement per -row: +[`SHOW CREATE DATABASE INCLUDE (SCHEMA)`](/docs/query/sql/show/#show-create-database) +returns a data-free schema dump, with one replayable DDL statement per row. The +explicit category keeps this recipe schema-only on both editions: ```questdb-sql title="Dump the database schema" demo -SHOW CREATE DATABASE; +SHOW CREATE DATABASE INCLUDE (SCHEMA); ``` -Statements come back in dependency order, so replaying them top to bottom on an -empty instance recreates the database. +Statements come back in dependency order. Replay them top to bottom on an empty +instance to recreate the schema. On QuestDB Enterprise, the dump contains only the objects the caller [can see](/docs/security/rbac/#object-visibility). To copy the whole schema, @@ -38,7 +39,7 @@ result set as JSON, with the statements under `.dataset[][0]`: ```shell curl -s -G "https://demo.questdb.io/api/v1/sql/execute" \ - --data-urlencode "query=SHOW CREATE DATABASE" + --data-urlencode "query=SHOW CREATE DATABASE INCLUDE (SCHEMA)" ``` From there you can either save the statements to a file or apply them straight @@ -48,7 +49,7 @@ to the target instance. ```shell curl -s -G "https://demo.questdb.io/api/v1/sql/execute" \ - --data-urlencode "query=SHOW CREATE DATABASE" \ + --data-urlencode "query=SHOW CREATE DATABASE INCLUDE (SCHEMA)" \ | jq -r '.dataset[][0] | sub(";?$";";") + "\n"' > schema.sql ``` @@ -63,7 +64,7 @@ Send each statement to the target's `/execute` endpoint as you read it: ```shell curl -s -G "https://demo.questdb.io/api/v1/sql/execute" \ - --data-urlencode "query=SHOW CREATE DATABASE" \ + --data-urlencode "query=SHOW CREATE DATABASE INCLUDE (SCHEMA)" \ | jq --raw-output0 '.dataset[][0]' \ | xargs -0 -n1 sh -c \ 'curl -sG "http://localhost:9000/api/v1/sql/execute" \ @@ -104,7 +105,7 @@ export QDB_TOKEN="your-rest-api-token" curl -s -G "https://source:9000/api/v1/sql/execute" \ -H "Authorization: Bearer $QDB_TOKEN" \ - --data-urlencode "query=SHOW CREATE DATABASE" + --data-urlencode "query=SHOW CREATE DATABASE INCLUDE (SCHEMA)" ``` ## Enterprise: schema without permissions @@ -121,19 +122,29 @@ curl -s -G "https://source:9000/api/v1/sql/execute" \ -H "Authorization: Bearer $QDB_TOKEN" \ --data-urlencode "query=SHOW CREATE DATABASE INCLUDE (SCHEMA)" -# Permissions only, no tables or views +# Identities, memberships, and grants; no schema curl -s -G "https://source:9000/api/v1/sql/execute" \ -H "Authorization: Bearer $QDB_TOKEN" \ --data-urlencode "query=SHOW CREATE DATABASE INCLUDE (ACL)" ``` -The ACL categories each require the matching `LIST` or `USER DETAILS` -permission, while the schema categories need none, so a user with only `SELECT` -can still dump the structure of the objects they can see. - -Passwords and tokens are never dumped, so `CREATE USER` and -`CREATE SERVICE ACCOUNT` statements replay without credentials. Set those on the -target after the replay. +Every selection that includes an ACL category, including plain +`SHOW CREATE DATABASE`, requires both `LIST USERS` and `USER DETAILS`. Missing +either permission aborts the dump. `INCLUDE (SCHEMA)` requires neither, so a +user with only `SELECT` can still dump the structure of the objects they can see. +Active table storage policies are included in the schema. + +To copy access control too, use plain `SHOW CREATE DATABASE` in the commands +above. It creates principals first, schema objects next, then memberships, +service account assignments, and grants. Replay on an empty target as its +built-in admin to avoid adding another principal's creator grants. Schema-only +and per-object dumps contain neither grants nor `OWNED BY`. + +Before opening the target to users, set credentials and reapply memory limits, +resource groups, and resource group mappings: none are dumped. Only the external +alias shown by `SHOW GROUPS` is restored for each group. Avoid concurrent schema +and ACL changes during the export; the dump is best-effort, not a consistent +snapshot. See [Replay limits](/docs/query/sql/show/#replay-limits). ## Manual alternative: copy from the Web Console diff --git a/documentation/operations/upgrade.md b/documentation/operations/upgrade.md index 40b2cba64..12397b0bb 100644 --- a/documentation/operations/upgrade.md +++ b/documentation/operations/upgrade.md @@ -28,6 +28,27 @@ If this is your first time setting up QuestDB, see the especially for production instances. See [Backup and restore](/docs/operations/backup/) for details. +### Schema dumps and view definitions + +Check schema-export scripts and stored views for these compatibility changes: + +- With Enterprise access control enabled, plain `SHOW CREATE DATABASE` now + requires both `LIST USERS` and `USER DETAILS`. Schema-only scripts should use + `INCLUDE (SCHEMA)` instead of acquiring more permissions. See + [database dumps](/docs/query/sql/show/#enterprise-access-control). +- Per-object `SHOW CREATE` and database dumps no longer append `OWNED BY`. + Schema-only output does not restore grants. Use the ACL dump for permissions, + and review [creator permissions on replay](/docs/query/sql/show/#enterprise-variant). +- With Enterprise access control enabled, materialized and live views reject + `SHOW CREATE DATABASE` with any ACL category, including the default. Change + schema-metadata queries to + `INCLUDE (SCHEMA)` under the + [catalogue rules](/docs/query/sql/create-mat-view/#catalogue-functions). +- A regular view's `PIVOT IN` subquery rejects administrative functions and + administrative `SHOW` statements. Existing definitions become invalid at + their next compilation, including startup. See + [view migration guidance](/docs/concepts/views/#administrative-operations-in-view-metadata). + ## Upgrade steps ### Docker diff --git a/documentation/query/functions/access-control.md b/documentation/query/functions/access-control.md index 7be2bf1d6..451a67d3a 100644 --- a/documentation/query/functions/access-control.md +++ b/documentation/query/functions/access-control.md @@ -91,7 +91,9 @@ These apply to both `active_grants()` and `active_permissions()`: the permissions a user gains by assuming a service account. Use [`SHOW GROUPS userName`](/docs/query/sql/show/#show-groups) and [`SHOW SERVICE ACCOUNTS userName`](/docs/query/sql/show/#show-service-accounts) - to see them. + to see them. To export memberships, assignments, and direct grants as + replayable SQL, use + [`SHOW CREATE DATABASE INCLUDE (PERMISSIONS)`](/docs/query/sql/show/#enterprise-access-control). - Access through a view is not shown on the tables the view reads. A view reads them with the permissions of its [definer](/docs/concepts/views/#definer-security-model-enterprise), so a user @@ -105,8 +107,11 @@ These apply to both `active_grants()` and `active_permissions()`: refresh in the background without a session whose permissions could be checked. Creating such a view fails with an error such as `administrative function cannot be used in materialized view: active_grants`. - A regular view can call them. Unlike the tables it reads, the functions check - the permissions of the user who queries the view, not those of its definer. + A regular view can call them in its query body, but not in a `PIVOT IN` + subquery, including through nested queries or another view. Unlike the tables + it reads, the functions check the permissions of the user who queries the + view, not those of its definer. See + [administrative operations in view metadata](/docs/concepts/views/#administrative-operations-in-view-metadata). ### Examples diff --git a/documentation/query/sql/create-live-view.md b/documentation/query/sql/create-live-view.md index f779c2f60..eef516638 100644 --- a/documentation/query/sql/create-live-view.md +++ b/documentation/query/sql/create-live-view.md @@ -267,13 +267,16 @@ The view query is validated at creation time and must: `systimestamp()`, or `rnd_*()`. - Not use [catalogue functions](/docs/query/sql/create-mat-view/#catalogue-functions) - or `SHOW` statements, such as `tables()` in a `WHERE` sub-query, unless the - creator holds `SYSTEM ADMIN` (Enterprise). The same rules as for materialized - views apply. + or catalogue `SHOW` statements, such as schema-only `SHOW CREATE DATABASE` + in a `WHERE` subquery, unless the creator holds `SYSTEM ADMIN` (Enterprise). + The same rules as for materialized views apply. - Not use Enterprise administrative functions, which check the permissions of the calling session: `active_grants()`, `active_permissions()`, `backup_instance_name()`, `backup_status()`, `backups()`, `permissions()`, `reload_tick_calendars()`, and `reload_tls()`. +- Not use `SHOW CREATE DATABASE` with any ACL category when access control is + enabled, even as an administrator. The default statement includes ACL output; + use `INCLUDE (SCHEMA)` under the catalogue rules above. - Not read another live view. ## Examples @@ -476,6 +479,7 @@ the query. | `live view select cannot filter on the designated timestamp yet` | The `WHERE` clause filters the designated timestamp | | `non-deterministic function cannot be used in live view` | The query uses `now()`, `rnd_*()`, or a similar non-deterministic function | | `administrative function cannot be used in live view: ` | The query calls an Enterprise administrative function, such as `permissions()` or `reload_tls()` | +| `administrative statement cannot be used in live view: SHOW CREATE DATABASE` | The query includes an ACL dump (Enterprise); use `INCLUDE (SCHEMA)` under the [catalogue rules](/docs/query/sql/create-mat-view/#catalogue-functions) | ### Window errors diff --git a/documentation/query/sql/create-mat-view.md b/documentation/query/sql/create-mat-view.md index eea4ebfb6..8519780b0 100644 --- a/documentation/query/sql/create-mat-view.md +++ b/documentation/query/sql/create-mat-view.md @@ -329,7 +329,8 @@ Materialized view queries must: - Not read [catalogue functions](#catalogue-functions) through a regular view, and in QuestDB Enterprise call them only with `SYSTEM ADMIN` - Not use Enterprise administrative functions, such as `permissions()` or - `backups()` + `backups()`, or `SHOW CREATE DATABASE` with any ACL category. Use + `INCLUDE (SCHEMA)` for schema metadata, subject to the catalogue rules below See [query constraints](/docs/concepts/materialized-views/#query-constraints) for the full list. @@ -350,7 +351,14 @@ metadata: `files()`, and `glob()` - `SHOW TABLES`, `SHOW COLUMNS`, `SHOW PARTITIONS`, `SHOW CREATE TABLE`, `SHOW CREATE VIEW`, `SHOW CREATE MATERIALIZED VIEW`, `SHOW CREATE LIVE VIEW`, - and `SHOW CREATE DATABASE` + and schema-only `SHOW CREATE DATABASE`, such as + `SHOW CREATE DATABASE INCLUDE (SCHEMA)` + +With Enterprise access control enabled, a database dump that includes any ACL +category is administrative, not just a catalogue query. Materialized and live +views reject it for every principal, even one holding `SYSTEM ADMIN` or +`DATABASE ADMIN`. This includes plain `SHOW CREATE DATABASE`, since its default +is `INCLUDE ALL`. The check applies at creation and refresh. A materialized view refreshes under a context that sees every object, and every reader of the view reads what the refresh stored. To keep a view from @@ -443,6 +451,7 @@ GRANT DROP MATERIALIZED VIEW ON trades_hourly TO user1; | `catalogue function from view cannot be used in materialized view: ` | A regular view the query reads calls a [catalogue function](#catalogue-functions). A `SHOW` statement reports `catalogue statement from view` | | `catalogue function cannot be used in materialized view without SYSTEM ADMIN: ` | The query calls a catalogue function and the creator lacks `SYSTEM ADMIN` (Enterprise). A `SHOW` statement reports `catalogue statement` | | `administrative function cannot be used in materialized view: ` | The query calls an Enterprise administrative function, such as `active_permissions()` or `backups()` | +| `administrative statement cannot be used in materialized view: SHOW CREATE DATABASE` | The query includes an ACL dump (Enterprise); use `INCLUDE (SCHEMA)` under the [catalogue rules](#catalogue-functions) | | `permission denied` | Missing required permission (Enterprise) | ## See also diff --git a/documentation/query/sql/create-view.md b/documentation/query/sql/create-view.md index 7d71619e6..5c3e9b70a 100644 --- a/documentation/query/sql/create-view.md +++ b/documentation/query/sql/create-view.md @@ -192,6 +192,16 @@ CREATE VIEW with_timestamp AS ( ) ``` +## Query constraints + +A `PIVOT IN` subquery in a view cannot use administrative functions or +administrative `SHOW` statements, including Enterprise `SHOW CREATE DATABASE` +with any ACL category. Use `INCLUDE (SCHEMA)` when only schema metadata is +needed. The restriction also applies through nested queries and other views, +and is checked when creating, replacing, altering, or recompiling a view. See +[administrative operations in view metadata](/docs/concepts/views/#administrative-operations-in-view-metadata) +for existing-view migration guidance. + ## Errors | Error | Cause | @@ -200,6 +210,8 @@ CREATE VIEW with_timestamp AS ( | `table does not exist` | Referenced table doesn't exist | | `Invalid column` | Column in query doesn't exist | | `circular dependency detected` | View would create circular reference | +| `administrative function cannot be used in view PIVOT IN subquery: ` | The `PIVOT IN` subquery uses an administrative function, directly or through another query or view | +| `administrative statement cannot be used in view PIVOT IN subquery: SHOW CREATE DATABASE` | The `PIVOT IN` subquery includes an ACL dump (Enterprise); use `INCLUDE (SCHEMA)` if only schema is needed | | `variable is not overridable` | Attempted to override a non-`OVERRIDABLE` DECLARE variable | | `Access denied [AUDIT VIEW]` | `WITH AUDIT`, or a replacement of an audited view, without the `AUDIT VIEW` permission (Enterprise) | | `cannot audit an existing view, drop and re-create it WITH AUDIT` | `CREATE OR REPLACE VIEW ... WITH AUDIT` over a view that is not audited (Enterprise) | diff --git a/documentation/query/sql/pivot.md b/documentation/query/sql/pivot.md index ebbfcde83..dd70f2a8f 100644 --- a/documentation/query/sql/pivot.md +++ b/documentation/query/sql/pivot.md @@ -266,6 +266,13 @@ For exploratory analysis, dynamic subqueries are convenient. For production quer use a constant list or store keys in a small dimension table for better performance. ::: +In a stored view, an `IN` subquery cannot use administrative functions or +administrative `SHOW` statements, including Enterprise ACL dumps. This also +applies through nested queries and other views. Schema-only +`SHOW CREATE DATABASE INCLUDE (SCHEMA)` remains allowed in a regular view. See +[administrative operations in view metadata](/docs/concepts/views/#administrative-operations-in-view-metadata) +for restrictions and migration guidance. + ### With CTEs ```questdb-sql demo title="Pivot with CTE" diff --git a/documentation/query/sql/show.md b/documentation/query/sql/show.md index ceda0aaa4..6f68a4ae7 100644 --- a/documentation/query/sql/show.md +++ b/documentation/query/sql/show.md @@ -39,8 +39,8 @@ SHOW { COLUMNS FROM tableName - `SHOW COLUMNS` returns all the columns and their metadata for the selected table. -- `SHOW CREATE DATABASE` returns DDL statements that recreate every object in - the database, one per row, ordered so dependencies come first. +- `SHOW CREATE DATABASE` returns statements to recreate the schema and, on + Enterprise, access control, one per row in replay order. - `SHOW CREATE LIVE VIEW` returns a DDL query that allows you to recreate a live view. - `SHOW CREATE MATERIALIZED VIEW` returns a DDL query that allows you to @@ -65,11 +65,13 @@ SHOW { COLUMNS FROM tableName - `SHOW USERS` lists all users with their enabled flag, memory limit and resource group (enterprise-only) -In QuestDB Enterprise, `SHOW TABLES` and `SHOW CREATE DATABASE` cover only the -objects you [can see](/docs/security/rbac/#object-visibility). The statements -that name an object, such as `SHOW COLUMNS` or `SHOW CREATE TABLE`, fail with -the missing-object error, for example `table does not exist [table=trades]`, -for an object hidden from you. +In QuestDB Enterprise, `SHOW TABLES` and the schema portion of +`SHOW CREATE DATABASE` cover only the objects you +[can see](/docs/security/rbac/#object-visibility). Dumping any access control +category also requires [administrative permissions](#enterprise-access-control). +Statements that name an object, such as `SHOW COLUMNS` or `SHOW CREATE TABLE`, +fail with the missing-object error, for example +`table does not exist [table=trades]`, for an object hidden from you. ## Examples @@ -96,10 +98,11 @@ a comma-separated string. ### SHOW CREATE DATABASE -`SHOW CREATE DATABASE` returns a logical, data-free dump of the whole database: -one round-trippable DDL statement per row for every user object, much like -`pg_dump --schema-only`. Replaying the statements from top to bottom on an empty -instance recreates the database. No table data and no credentials are included. +`SHOW CREATE DATABASE` returns a logical, data-free dump as replayable SQL: +one statement per row for tables, views, materialized views, and live views. +QuestDB Enterprise also includes users, groups, service accounts, memberships, +and grants by default. Use it to copy a database's structure and access control +to an empty instance. It is not a data backup; see [Replay limits](#replay-limits). ```questdb-sql title="SHOW CREATE DATABASE syntax" SHOW CREATE DATABASE @@ -109,36 +112,29 @@ SHOW CREATE DATABASE An optional `INCLUDE` or `EXCLUDE` clause selects which object categories to dump. Each `category` is one of: -- **Schema objects**: `TABLES`, `VIEWS`, `MATERIALIZED_VIEWS`, `LIVE_VIEWS`. -- **Access control** (Enterprise only): `USERS`, `GROUPS`, `SERVICE_ACCOUNTS`, - `PERMISSIONS`. -- **Umbrellas**: `SCHEMA` (all schema objects), `ACL` (all access control - objects), `ALL` (`SCHEMA` plus `ACL`). +- **Schema objects**: `LIVE_VIEWS`, `MATERIALIZED_VIEWS`, `TABLES`, `VIEWS`. +- **Access control** (Enterprise only): `GROUPS`, `PERMISSIONS`, + `SERVICE_ACCOUNTS`, `USERS`. +- **Umbrellas**: `ACL` (all access control categories), `ALL` (`SCHEMA` plus + `ACL`), `SCHEMA` (all schema objects). `INCLUDE`/`EXCLUDE` accept either `ALL` or a parenthesised list, so `INCLUDE ALL`, `EXCLUDE (MATERIALIZED_VIEWS)`, and `INCLUDE (TABLES, VIEWS)` are -all valid. Called without a clause, the statement dumps the whole database: +all valid. With no clause, the statement defaults to `INCLUDE ALL`: -```questdb-sql title="Dump the database schema" demo +```questdb-sql title="Dump schema and access control" SHOW CREATE DATABASE; ``` +In QuestDB open source, `ALL` and `SCHEMA` produce the same output. On +Enterprise, select `SCHEMA` explicitly to omit access control: + +```questdb-sql title="Dump only the database schema" demo +SHOW CREATE DATABASE INCLUDE (SCHEMA); +``` + The result set has a single `ddl` column with one self-contained statement per -row. Run against a database holding the [demo](https://demo.questdb.io) tables -and materialized views, it returns one row per object: - -| ddl | -| ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | -| CREATE TABLE 'market_data' ( timestamp TIMESTAMP, symbol SYMBOL, bids DOUBLE[][], asks DOUBLE[][], best_bid DOUBLE, best_ask DOUBLE ) timestamp(timestamp) PARTITION BY HOUR TTL 3 DAYS; | -| CREATE MATERIALIZED VIEW 'bbo_1s' WITH BASE 'market_data' REFRESH IMMEDIATE AS ( SELECT timestamp, symbol, last(bids[1][1]) AS bid, last(asks[1][1]) AS ask FROM market_data SAMPLE BY 1s ) PARTITION BY DAY; | -| CREATE MATERIALIZED VIEW 'bbo_1m' WITH BASE 'bbo_1s' REFRESH EVERY 1m DEFERRED START '2025-06-01T00:00:00.000000Z' AS ( SELECT timestamp, symbol, max(bid) AS bid, min(ask) AS ask FROM bbo_1s SAMPLE BY 1m ) PARTITION BY DAY; | -| CREATE MATERIALIZED VIEW 'bbo_1h' WITH BASE 'bbo_1m' REFRESH EVERY 10m DEFERRED START '2025-06-01T00:00:00.000000Z' AS ( SELECT timestamp, symbol, max(bid) AS bid, min(ask) AS ask FROM bbo_1m SAMPLE BY 1h ) PARTITION BY MONTH; | -| CREATE MATERIALIZED VIEW 'bbo_1d' WITH BASE 'bbo_1h' REFRESH EVERY 1h DEFERRED START '2025-06-01T00:00:00.000000Z' AS ( SELECT timestamp, symbol, max(bid) AS bid, min(ask) AS ask FROM bbo_1h SAMPLE BY 1d ) PARTITION BY YEAR; | -| ... | -| CREATE TABLE 'trips' ( cab_type SYMBOL, vendor_id SYMBOL, pickup_datetime TIMESTAMP, dropoff_datetime TIMESTAMP, rate_code_id SYMBOL, pickup_latitude DOUBLE, pickup_longitude DOUBLE, dropoff_latitude DOUBLE, dropoff_longitude DOUBLE, passenger_count INT, trip_distance DOUBLE, fare_amount DOUBLE, extra DOUBLE, mta_tax DOUBLE, tip_amount DOUBLE, tolls_amount DOUBLE, ehail_fee DOUBLE, improvement_surcharge DOUBLE, congestion_surcharge DOUBLE, total_amount DOUBLE, payment_type SYMBOL, trip_type SYMBOL, pickup_location_id INT, dropoff_location_id INT ) timestamp(pickup_datetime) PARTITION BY MONTH; | - -Each `ddl` value is stored with formatting characters, so pasting a row into a -text editor expands it to the indented form shown by +row. Each value preserves newlines and indentation, as in [`SHOW CREATE TABLE`](#show-create-table). For turning the result set into a `.sql` file or replaying it on another server, @@ -147,13 +143,21 @@ see #### Output order -Objects are emitted in dependency order: a materialized view or view is never -reported before the base table or base materialized view it reads from. Within -that constraint objects are ordered alphabetically. The demo chains several -materialized views, for example `market_data` then `bbo_1s`, `bbo_1m`, `bbo_1h`, -`bbo_1d`, and `fx_trades` then `fx_trades_ohlc_1m`, `fx_trades_ohlc_1d`. Each -view in a chain appears only after the object it depends on, so a top-to-bottom -replay always succeeds. +A full Enterprise dump has three parts: + +1. Users, groups, and service accounts, ordered by name across all three kinds. + A disabled user or service account has an `ALTER ... DISABLE` statement + immediately after its `CREATE` statement. +2. Schema objects in dependency order, with alphabetical ordering where + dependencies allow. A view follows the objects it reads, including other + views in a chain. +3. Group memberships, service account assignments, and direct grants, grouped + by principal name. Within each principal's grants, database scope precedes + table scope, which precedes column scope. + +Schema-only dumps contain just the second part. Preserve the returned order +when replaying: a broader grant can clear a narrower grant option, which a +later `WITH GRANT OPTION` statement restores. #### Filtering by category @@ -174,21 +178,18 @@ SHOW CREATE DATABASE INCLUDE (TABLES, MATERIALIZED_VIEWS); SHOW CREATE DATABASE EXCLUDE (MATERIALIZED_VIEWS); ``` -With no clause the statement defaults to `INCLUDE ALL`. In QuestDB open source -there is no access control layer, so `ALL` and `SCHEMA` produce the same output. -In [QuestDB Enterprise](/enterprise/) the default `ALL` also dumps the access -control block, so use `INCLUDE (SCHEMA)` when you want the structure only. - Filtering is applied per category, like `pg_dump -t`. Excluding a category that others depend on can leave dangling references, so a dump that omits a base -table does not replay cleanly for the views built on it. +table does not replay cleanly for the views built on it. Similarly, +`INCLUDE (PERMISSIONS)` omits principal creation: those users, groups, and +service accounts must already exist on the target. To select individual objects, such as the objects of a single tenant, wrap the -statement in parentheses and filter the `ddl` column with `WHERE`. Each row -starts with `CREATE`, so match the object name with a leading wildcard: +statement in parentheses and filter the `ddl` column with `WHERE`. Match names +wherever they appear in the statement: -```questdb-sql title="Objects whose DDL mentions fx_" demo -(SHOW CREATE DATABASE) WHERE ddl ILIKE '%fx_%'; +```questdb-sql title="Schema statements whose DDL mentions fx_" demo +(SHOW CREATE DATABASE INCLUDE (SCHEMA)) WHERE ddl ILIKE '%fx_%'; ``` The filtered rows keep dependency order. A pattern also matches views that read @@ -197,34 +198,112 @@ included before replaying it. #### Enterprise: access control -In [QuestDB Enterprise](/enterprise/), `SHOW CREATE DATABASE` also dumps the -access control layer after the schema objects, so a dump captures identities, -memberships, and permissions alongside the tables and views: - -- `CREATE USER`, `CREATE GROUP`, and `CREATE SERVICE ACCOUNT` for each entity. -- Memberships, as `ADD USER ... TO ...` and `ASSUME SERVICE ACCOUNT ... TO ...`. -- Grants, as `GRANT [ON ...] TO ... [WITH GRANT OPTION]`. - -QuestDB stores no owner for an object, so the `GRANT` statements are what -recreate its permissions. The `CREATE TABLE`, `CREATE VIEW`, and -`CREATE MATERIALIZED VIEW` statements in an Enterprise dump also carry an -`OWNED BY` clause, which names the principal that ran the dump, as in -[`SHOW CREATE TABLE`](#enterprise-variant). Replaying the dump gives that -principal all permissions on each of these objects, with the `GRANT` option, on -top of the dumped grants. Edit or remove the clause before replaying if that -principal should not hold them. - -Credentials are never dumped: the `CREATE USER` and `CREATE SERVICE ACCOUNT` -statements carry no password or token, so set these after replaying the dump. - -The Enterprise ACL categories are `USERS`, `GROUPS`, `SERVICE_ACCOUNTS`, and -`PERMISSIONS`, grouped by the `ACL` umbrella. Each requires the matching `LIST` -or `USER DETAILS` permission, while the schema categories need no access control -permission, so a user with only `SELECT` can still dump the structure of the -objects it can see. To dump the whole schema, run the statement as a principal -that sees every object, such as the built-in admin or a user holding -`DATABASE ADMIN`. When access control is disabled the command degrades to a -schema-only dump. +With access control enabled, selecting any ACL category requires both +`LIST USERS` and `USER DETAILS`, even `INCLUDE (USERS)` or +`INCLUDE (PERMISSIONS)` alone. Missing either permission aborts the whole dump; +it does not return a partial schema-only result. + +:::warning Breaking change + +Plain `SHOW CREATE DATABASE` now includes ACL output and requires both +permissions. For callers that only need the visible schema, change it to +`SHOW CREATE DATABASE INCLUDE (SCHEMA)` rather than granting administrative +permissions. + +::: + +Schema-only categories require neither ACL permission, but still follow +[object visibility](/docs/security/rbac/#object-visibility). To dump the whole +user schema, run as the built-in admin or a principal holding `DATABASE ADMIN`. +These visibility rules do not filter the ACL output: an authorized ACL dump +lists all stored principals and their direct grants, including grants naming +objects hidden from the caller. When access control is disabled, the statement +returns only schema output, without the ACL permission checks. + +The `ACL` umbrella selects all four categories: + +| Category | Statements included | +| -------- | ------------------- | +| `GROUPS` | `CREATE GROUP`, including the external alias reported by `SHOW GROUPS` | +| `PERMISSIONS` | `ADD USER ... TO ...`, `GRANT ASSUME SERVICE ACCOUNT ... TO ...`, and database-, table-, and column-level `GRANT` statements | +| `SERVICE_ACCOUNTS` | `CREATE SERVICE ACCOUNT`, followed by `ALTER SERVICE ACCOUNT ... DISABLE` when disabled | +| `USERS` | `CREATE USER`, followed by `ALTER USER ... DISABLE` when disabled | + +Grants preserve `WITH GRANT OPTION`, including grant options held at a narrower +scope than the permission itself. Only direct grants are emitted. Permissions +and service account assignments inherited from a group are restored through +group membership, not copied into direct user grants. + +For example, on an otherwise empty instance, run as the built-in admin: + +```questdb-sql title="Create a group-based entitlement to export" +CREATE GROUP fx_readers; +CREATE USER fx_analyst; +ADD USER fx_analyst TO fx_readers; +GRANT SELECT ON ALL TABLES TO fx_readers; + +SHOW CREATE DATABASE INCLUDE (ACL); +``` + +| ddl | +| --- | +| CREATE USER 'fx_analyst'; | +| CREATE GROUP 'fx_readers'; | +| ADD USER 'fx_analyst' TO 'fx_readers'; | +| GRANT SELECT ON ALL TABLES TO 'fx_readers'; | + +The built-in admin and external (SSO/OIDC) users are not dumped. External users +continue to get access through their identity provider and the dumped groups' +external aliases. + +Database dumps also include active table +[storage policies](#storage-policy-clause). No `SHOW CREATE` statement emits +`OWNED BY`: object permissions are restored through grants, not an owner field. +See [Enterprise variant](#enterprise-variant) for creator permissions during +replay. For an ACL restore, replay on an empty target as its built-in admin to +avoid adding another principal's creator grants. + +#### Replay limits + +An Enterprise dump omits the following settings. Reapply them on the target +before opening it to users: + +- Passwords and tokens. Recreated users and service accounts have no + credentials until you set them. +- Per-user, group, and service account + [memory limits](/docs/security/rbac/#memory-limits), + [resource groups](/docs/concepts/resource-groups/), and resource group + mappings. Recreated principals have no per-principal memory override and use + the default resource group until you configure them. +- Older external aliases that may still be active for a group. Only the alias + reported by `SHOW GROUPS` is restored. Review identity-provider mappings + separately if more than one alias has been assigned to a group. + +:::caution Best-effort dump + +The dump is not a transactionally consistent snapshot across schema objects, +principals, and memberships. Each principal's enabled state and direct grants +come from the same snapshot, but concurrent changes to other entities or group +memberships may still produce a mixed result. Avoid schema and ACL changes +while exporting for replay, and verify the target's grants afterwards with +[`active_grants()`](/docs/query/functions/access-control/#active_grants). + +::: + +#### Use in views + +When Enterprise access control is enabled: + +- A regular view may read an ACL dump in its query body. The dump checks the + reader's `LIST USERS` and `USER DETAILS`, not the view definer's permissions. +- A regular view's `PIVOT IN` subquery cannot read an ACL dump, directly or + through another view. Use `INCLUDE (SCHEMA)` if only schema is needed. See + [administrative operations in view metadata](/docs/concepts/views/#administrative-operations-in-view-metadata). +- Materialized and live views reject any ACL category, including the default + `SHOW CREATE DATABASE`, even for an administrator. A schema-only form such as + `INCLUDE (SCHEMA)` is allowed under the + [catalogue statement rules](/docs/query/sql/create-mat-view/#catalogue-functions): + it must be in the view's own query, and the creator must hold `SYSTEM ADMIN`. ### SHOW CREATE LIVE VIEW @@ -238,9 +317,9 @@ SHOW CREATE LIVE VIEW trades_ma; This returns the `CREATE LIVE VIEW` statement that would recreate the [live view](/docs/concepts/live-views/), including its `FLUSH EVERY`, -`IN MEMORY`, `PARTITION BY`, and `START FROM` clauses. On QuestDB Enterprise the -output also carries an `OWNED BY` clause naming the user running `SHOW CREATE`. -See the [Enterprise variant](#enterprise-variant) of `SHOW CREATE TABLE`. +`IN MEMORY`, `PARTITION BY`, and `START FROM` clauses. Like other `SHOW CREATE` +statements, it does not include `OWNED BY` or grants. See the +[Enterprise variant](#enterprise-variant) of `SHOW CREATE TABLE`. ### SHOW CREATE MATERIALIZED VIEW @@ -317,52 +396,44 @@ CREATE TABLE sensors ( When an active [storage policy](/docs/concepts/storage-policy/) is attached to a table (Enterprise only), the policy renders as a `STORAGE POLICY(...)` clause in -the `SHOW CREATE TABLE` output: +the `SHOW CREATE TABLE` and `SHOW CREATE DATABASE` output: -```questdb-sql -SHOW CREATE TABLE sensor_data; +```questdb-sql title="Retrieve a table's storage policy in its DDL" +SHOW CREATE TABLE fx_quotes; ``` -```text -CREATE TABLE 'sensor_data' ( - ts TIMESTAMP, - value DOUBLE -) timestamp(ts) PARTITION BY DAY +```questdb-sql title="Returned DDL" +CREATE TABLE 'fx_quotes' ( + timestamp TIMESTAMP, + bid DOUBLE +) timestamp(timestamp) PARTITION BY DAY STORAGE POLICY(TO PARQUET 3 DAYS, DROP LOCAL 1 MONTH); ``` Stages that are not configured on the policy are omitted from the clause. Only an active policy renders: after `ALTER TABLE ... DISABLE STORAGE POLICY`, the -policy is not shown in `SHOW CREATE TABLE`. See +policy is not shown in either dump. See [ALTER TABLE SET STORAGE POLICY](/docs/query/sql/alter-table-set-storage-policy/). #### Enterprise variant In [QuestDB Enterprise](/enterprise/), `SHOW CREATE TABLE`, `SHOW CREATE VIEW`, -`SHOW CREATE MATERIALIZED VIEW`, and `SHOW CREATE LIVE VIEW` add an `OWNED BY` -clause naming the user running `SHOW CREATE`. QuestDB stores no owner for an -object, so the clause does not tell you who created it or who holds permissions -on it. To see the permissions on an object, use -[`active_grants()`](/docs/query/functions/access-control/#active_grants). - -For example, run as `admin`: - -```questdb-sql -CREATE TABLE trades ( - symbol SYMBOL CAPACITY 256 CACHE, - side SYMBOL CAPACITY 256 CACHE, - price DOUBLE, - amount DOUBLE, - timestamp TIMESTAMP -) timestamp(timestamp) PARTITION BY DAY -WITH maxUncommittedRows=500000, o3MaxLag=600000000us -OWNED BY 'admin'; -``` - -When the statement is replayed, the clause gives that user all permissions on -the new table, with the `GRANT` option. To give them to a different user, group, -or service account, change the clause before replaying it. See -[OWNED BY](/docs/query/sql/create-table/#owned-by). +`SHOW CREATE MATERIALIZED VIEW`, and `SHOW CREATE LIVE VIEW` return structure, +not permissions. They no longer append an `OWNED BY` clause naming the caller. +QuestDB stores no owner for an object; `OWNED BY` at creation only decides who +receives the creator's permissions. + +On replay without `OWNED BY`, the principal executing the `CREATE` receives the +normal creator permissions, with the `GRANT` option. To give those permissions +to another principal, add an explicit +[`OWNED BY`](/docs/query/sql/create-table/#owned-by) clause. External users must +still supply that clause when creating objects. + +To inspect existing permissions, use +[`active_grants()`](/docs/query/functions/access-control/#active_grants). To +export them as replayable SQL, use +[`SHOW CREATE DATABASE INCLUDE (ACL)`](#enterprise-access-control). A schema-only +dump or a per-object `SHOW CREATE` does not preserve grants. ### SHOW CREATE VIEW diff --git a/documentation/security/governance.md b/documentation/security/governance.md index f73b5b269..1b542c071 100644 --- a/documentation/security/governance.md +++ b/documentation/security/governance.md @@ -128,17 +128,20 @@ functions have [limitations](/docs/query/functions/access-control/#limitations): the built-in admin and OIDC users are not listed, for example. -For a point-in-time snapshot of the whole access control layer, -[`SHOW CREATE DATABASE`](/docs/query/sql/show/#enterprise-access-control) -dumps every user, group, service account, membership, and grant as replayable -SQL, without credentials: +For a replayable export of stored identities, memberships, and direct grants, +use [`SHOW CREATE DATABASE`](/docs/query/sql/show/#enterprise-access-control). +Every ACL category requires both `LIST USERS` and `USER DETAILS`: -```questdb-sql title="Snapshot of identities, memberships, and grants" +```questdb-sql title="Export identities, memberships, and direct grants" SHOW CREATE DATABASE INCLUDE (ACL); ``` Storing that output on a schedule gives you a history of entitlements that you -can diff between two reviews. +can diff between two reviews. It is best-effort, not a transactionally +consistent snapshot across principals and memberships. The export omits the +built-in admin, external users, credentials, memory limits, and resource groups +and their mappings. For a group's external aliases, it preserves only the one +reported by `SHOW GROUPS`. See [Replay limits](/docs/query/sql/show/#replay-limits). ## Audit data access @@ -195,8 +198,9 @@ FROM materialized_views(); ``` [`SHOW CREATE`](/docs/query/sql/show/) returns the full DDL of a single table or -view, and `SHOW CREATE DATABASE` returns every object in dependency order, so -each view appears after the objects it reads from. Running it on a schedule and +view, and `SHOW CREATE DATABASE INCLUDE (SCHEMA)` returns visible schema +objects in dependency order, so each view appears after the objects it reads +from when those dependencies are included. Running it on a schedule and keeping the output records how the schema changed over time. ## Retain and protect data diff --git a/documentation/security/rbac.md b/documentation/security/rbac.md index 8025247e6..05bbb4074 100644 --- a/documentation/security/rbac.md +++ b/documentation/security/rbac.md @@ -930,6 +930,13 @@ even to read your own rows. ::: +To export identities, group memberships, service account assignments, and direct +grants as replayable SQL, use +[`SHOW CREATE DATABASE INCLUDE (ACL)`](/docs/query/sql/show/#enterprise-access-control). +It requires both `LIST USERS` and `USER DETAILS`, as does plain +`SHOW CREATE DATABASE`. For structure only, use `INCLUDE (SCHEMA)`, which +requires neither permission and includes only visible objects. + ## Memory limits {#memory-limits} QuestDB Enterprise can limit the native memory tracked for a single query, From b4308f26fd42649bee1341ee20a8481db6440343 Mon Sep 17 00:00:00 2001 From: glasstiger Date: Thu, 5 Nov 2026 18:45:59 +0000 Subject: [PATCH 2/5] docs: clarify 4.2.0 schema dump migration Scope the Enterprise ACL dump, OWNED BY, and view compatibility changes to release 4.2.0. Update the existing changelog entries with the release boundary. Make the schema-and-access-control recipe discoverable while preserving its existing anchor, and link the materialized/live view references to the canonical SHOW guidance. --- documentation/changelog.mdx | 4 +-- documentation/concepts/views.md | 7 ++--- .../copy-schema-between-instances.md | 9 ++++--- documentation/operations/upgrade.md | 3 ++- documentation/query/sql/create-live-view.md | 12 +++++---- documentation/query/sql/create-mat-view.md | 14 +++++----- documentation/query/sql/show.md | 27 ++++++++++--------- 7 files changed, 42 insertions(+), 34 deletions(-) diff --git a/documentation/changelog.mdx b/documentation/changelog.mdx index d84faf753..870e5e0d7 100644 --- a/documentation/changelog.mdx +++ b/documentation/changelog.mdx @@ -12,8 +12,8 @@ This page tracks significant updates to the QuestDB documentation. ### Updated -- [SHOW CREATE DATABASE](/docs/query/sql/show/#show-create-database) - Documented Enterprise ACL output, principal/schema/grant ordering, the requirement for both `LIST USERS` and `USER DETAILS`, active storage policies, and replay limits. Documented the removal of `OWNED BY` from all `SHOW CREATE` output, and updated the schema-copy recipe, RBAC, and governance guidance -- [Views](/docs/concepts/views/#administrative-operations-in-view-metadata) - Documented restrictions on administrative operations in `PIVOT IN` subqueries, schema-only database dumps in materialized and live views, and [upgrade guidance](/docs/operations/upgrade/#schema-dumps-and-view-definitions) for existing scripts and views +- [SHOW CREATE DATABASE](/docs/query/sql/show/#show-create-database) - Documented QuestDB Enterprise 4.2.0 ACL output, principal/schema/grant ordering, the requirement for both `LIST USERS` and `USER DETAILS`, active storage policies, and replay limits. Documented the removal of `OWNED BY` from all `SHOW CREATE` output in 4.2.0, and updated the schema-copy recipe, RBAC, and governance guidance +- [Views](/docs/concepts/views/#administrative-operations-in-view-metadata) - Documented QuestDB Enterprise 4.2.0 restrictions on administrative operations in `PIVOT IN` subqueries, schema-only database dumps in materialized and live views, and [upgrade guidance](/docs/operations/upgrade/#schema-dumps-and-view-definitions) for existing scripts and views ## October 2026 diff --git a/documentation/concepts/views.md b/documentation/concepts/views.md index e979f535a..759470711 100644 --- a/documentation/concepts/views.md +++ b/documentation/concepts/views.md @@ -482,9 +482,10 @@ query body still run with the reader's permission checks. :::warning Existing views -Creating, altering, or replacing a view with a prohibited operation fails. -Previously accepted views become invalid at their next compilation, including -at startup. Inspect `view_status` and `invalidation_reason` in +Starting with QuestDB Enterprise 4.2.0, creating, altering, or replacing a view +with a prohibited operation fails. Previously accepted views become invalid at +their next compilation after upgrading, including at startup. Inspect +`view_status` and `invalidation_reason` in [`views()`](/docs/query/functions/meta/#views). Replace ACL dumps with `INCLUDE (SCHEMA)` where appropriate, or remove the administrative operation from the subquery, then update the definition with diff --git a/documentation/cookbook/operations/copy-schema-between-instances.md b/documentation/cookbook/operations/copy-schema-between-instances.md index dc92cb455..a34b90895 100644 --- a/documentation/cookbook/operations/copy-schema-between-instances.md +++ b/documentation/cookbook/operations/copy-schema-between-instances.md @@ -108,11 +108,12 @@ curl -s -G "https://source:9000/api/v1/sql/execute" \ --data-urlencode "query=SHOW CREATE DATABASE INCLUDE (SCHEMA)" ``` -## Enterprise: schema without permissions +## Enterprise: schema and access control {#enterprise-schema-without-permissions} -In [QuestDB Enterprise](/enterprise/), the default dump also carries the access -control layer: users, groups, service accounts, memberships, and grants. That is -usually not what you want when seeding a development instance from production. +In [QuestDB Enterprise](/enterprise/) 4.2.0 and later, the default dump also +carries the access control layer: users, groups, service accounts, memberships, +and grants. That is usually not what you want when seeding a development +instance from production. Select what to copy with `INCLUDE`: diff --git a/documentation/operations/upgrade.md b/documentation/operations/upgrade.md index 12397b0bb..be9a5a9fc 100644 --- a/documentation/operations/upgrade.md +++ b/documentation/operations/upgrade.md @@ -30,7 +30,8 @@ If this is your first time setting up QuestDB, see the ### Schema dumps and view definitions -Check schema-export scripts and stored views for these compatibility changes: +Before upgrading from an earlier release to QuestDB Enterprise 4.2.0 or later, +check schema-export scripts and stored views for these compatibility changes: - With Enterprise access control enabled, plain `SHOW CREATE DATABASE` now requires both `LIST USERS` and `USER DETAILS`. Schema-only scripts should use diff --git a/documentation/query/sql/create-live-view.md b/documentation/query/sql/create-live-view.md index eef516638..6a35d2f7a 100644 --- a/documentation/query/sql/create-live-view.md +++ b/documentation/query/sql/create-live-view.md @@ -267,16 +267,18 @@ The view query is validated at creation time and must: `systimestamp()`, or `rnd_*()`. - Not use [catalogue functions](/docs/query/sql/create-mat-view/#catalogue-functions) - or catalogue `SHOW` statements, such as schema-only `SHOW CREATE DATABASE` - in a `WHERE` subquery, unless the creator holds `SYSTEM ADMIN` (Enterprise). - The same rules as for materialized views apply. + or catalogue `SHOW` statements, such as schema-only + [`SHOW CREATE DATABASE`](/docs/query/sql/show/#use-in-views) in a `WHERE` + subquery, unless the creator holds `SYSTEM ADMIN` (Enterprise). The same rules + as for materialized views apply. - Not use Enterprise administrative functions, which check the permissions of the calling session: `active_grants()`, `active_permissions()`, `backup_instance_name()`, `backup_status()`, `backups()`, `permissions()`, `reload_tick_calendars()`, and `reload_tls()`. - Not use `SHOW CREATE DATABASE` with any ACL category when access control is - enabled, even as an administrator. The default statement includes ACL output; - use `INCLUDE (SCHEMA)` under the catalogue rules above. + enabled (QuestDB Enterprise 4.2.0 and later), even as an administrator. The + default statement includes ACL output; use `INCLUDE (SCHEMA)` under the + catalogue rules above. - Not read another live view. ## Examples diff --git a/documentation/query/sql/create-mat-view.md b/documentation/query/sql/create-mat-view.md index 8519780b0..92f69cf96 100644 --- a/documentation/query/sql/create-mat-view.md +++ b/documentation/query/sql/create-mat-view.md @@ -351,14 +351,16 @@ metadata: `files()`, and `glob()` - `SHOW TABLES`, `SHOW COLUMNS`, `SHOW PARTITIONS`, `SHOW CREATE TABLE`, `SHOW CREATE VIEW`, `SHOW CREATE MATERIALIZED VIEW`, `SHOW CREATE LIVE VIEW`, - and schema-only `SHOW CREATE DATABASE`, such as + and schema-only + [`SHOW CREATE DATABASE`](/docs/query/sql/show/#use-in-views), such as `SHOW CREATE DATABASE INCLUDE (SCHEMA)` -With Enterprise access control enabled, a database dump that includes any ACL -category is administrative, not just a catalogue query. Materialized and live -views reject it for every principal, even one holding `SYSTEM ADMIN` or -`DATABASE ADMIN`. This includes plain `SHOW CREATE DATABASE`, since its default -is `INCLUDE ALL`. The check applies at creation and refresh. +In QuestDB Enterprise 4.2.0 and later, with access control enabled, a database +dump that includes any ACL category is administrative, not just a catalogue +query. Materialized and live views reject it for every principal, even one +holding `SYSTEM ADMIN` or `DATABASE ADMIN`. This includes plain +`SHOW CREATE DATABASE`, since its default is `INCLUDE ALL`. The check applies at +creation and refresh. A materialized view refreshes under a context that sees every object, and every reader of the view reads what the refresh stored. To keep a view from diff --git a/documentation/query/sql/show.md b/documentation/query/sql/show.md index 6f68a4ae7..50d958c3f 100644 --- a/documentation/query/sql/show.md +++ b/documentation/query/sql/show.md @@ -100,9 +100,10 @@ a comma-separated string. `SHOW CREATE DATABASE` returns a logical, data-free dump as replayable SQL: one statement per row for tables, views, materialized views, and live views. -QuestDB Enterprise also includes users, groups, service accounts, memberships, -and grants by default. Use it to copy a database's structure and access control -to an empty instance. It is not a data backup; see [Replay limits](#replay-limits). +In QuestDB Enterprise 4.2.0 and later, the dump also includes users, groups, +service accounts, memberships, and grants by default. Use it to copy a database's +structure and access control to an empty instance. It is not a data backup; see +[Replay limits](#replay-limits). ```questdb-sql title="SHOW CREATE DATABASE syntax" SHOW CREATE DATABASE @@ -205,10 +206,10 @@ it does not return a partial schema-only result. :::warning Breaking change -Plain `SHOW CREATE DATABASE` now includes ACL output and requires both -permissions. For callers that only need the visible schema, change it to -`SHOW CREATE DATABASE INCLUDE (SCHEMA)` rather than granting administrative -permissions. +Starting with QuestDB Enterprise 4.2.0, plain `SHOW CREATE DATABASE` includes +ACL output and requires both permissions. For callers that only need the visible +schema, change it to `SHOW CREATE DATABASE INCLUDE (SCHEMA)` rather than granting +administrative permissions. ::: @@ -292,7 +293,7 @@ while exporting for replay, and verify the target's grants afterwards with #### Use in views -When Enterprise access control is enabled: +In QuestDB Enterprise 4.2.0 and later, when access control is enabled: - A regular view may read an ACL dump in its query body. The dump checks the reader's `LIST USERS` and `USER DETAILS`, not the view definer's permissions. @@ -417,11 +418,11 @@ policy is not shown in either dump. See #### Enterprise variant -In [QuestDB Enterprise](/enterprise/), `SHOW CREATE TABLE`, `SHOW CREATE VIEW`, -`SHOW CREATE MATERIALIZED VIEW`, and `SHOW CREATE LIVE VIEW` return structure, -not permissions. They no longer append an `OWNED BY` clause naming the caller. -QuestDB stores no owner for an object; `OWNED BY` at creation only decides who -receives the creator's permissions. +In [QuestDB Enterprise](/enterprise/) 4.2.0 and later, `SHOW CREATE TABLE`, +`SHOW CREATE VIEW`, `SHOW CREATE MATERIALIZED VIEW`, and `SHOW CREATE LIVE VIEW` +return structure, not permissions. Unlike earlier releases, they do not append +an `OWNED BY` clause naming the caller. QuestDB stores no owner for an object; +`OWNED BY` at creation only decides who receives the creator's permissions. On replay without `OWNED BY`, the principal executing the `CREATE` receives the normal creator permissions, with the `GRANT` option. To give those permissions From 85b35498f5665fdc85c40e29ed055236d8c9e1d4 Mon Sep 17 00:00:00 2001 From: glasstiger Date: Fri, 6 Nov 2026 12:55:14 +0000 Subject: [PATCH 3/5] docs: move 4.2.0 upgrade checks to the Enterprise section Follow the Upgrading to 4.1.0 pattern: the schema dump and view definition checks move from Before you upgrade to a new Upgrading to 4.2.0 section under QuestDB Enterprise. Point the changelog entry at the new anchor. --- documentation/changelog.mdx | 2 +- documentation/operations/upgrade.md | 45 +++++++++++++++-------------- 2 files changed, 24 insertions(+), 23 deletions(-) diff --git a/documentation/changelog.mdx b/documentation/changelog.mdx index 66d6f69a1..fd6fe582a 100644 --- a/documentation/changelog.mdx +++ b/documentation/changelog.mdx @@ -48,7 +48,7 @@ This page tracks significant updates to the QuestDB documentation. ### Updated - [SHOW CREATE DATABASE](/docs/query/sql/show/#show-create-database) - Documented QuestDB Enterprise 4.2.0 ACL output, principal/schema/grant ordering, the requirement for both `LIST USERS` and `USER DETAILS`, active storage policies, and replay limits. Documented the removal of `OWNED BY` from all `SHOW CREATE` output in 4.2.0, and updated the schema-copy recipe, RBAC, and governance guidance -- [Views](/docs/concepts/views/#administrative-operations-in-view-metadata) - Documented QuestDB Enterprise 4.2.0 restrictions on administrative operations in `PIVOT IN` subqueries, schema-only database dumps in materialized and live views, and [upgrade guidance](/docs/operations/upgrade/#schema-dumps-and-view-definitions) for existing scripts and views +- [Views](/docs/concepts/views/#administrative-operations-in-view-metadata) - Documented QuestDB Enterprise 4.2.0 restrictions on administrative operations in `PIVOT IN` subqueries, schema-only database dumps in materialized and live views, and [upgrade guidance](/docs/operations/upgrade/#upgrading-to-420) for existing scripts and views ## October 2026 diff --git a/documentation/operations/upgrade.md b/documentation/operations/upgrade.md index be9a5a9fc..256eb1bd7 100644 --- a/documentation/operations/upgrade.md +++ b/documentation/operations/upgrade.md @@ -28,28 +28,6 @@ If this is your first time setting up QuestDB, see the especially for production instances. See [Backup and restore](/docs/operations/backup/) for details. -### Schema dumps and view definitions - -Before upgrading from an earlier release to QuestDB Enterprise 4.2.0 or later, -check schema-export scripts and stored views for these compatibility changes: - -- With Enterprise access control enabled, plain `SHOW CREATE DATABASE` now - requires both `LIST USERS` and `USER DETAILS`. Schema-only scripts should use - `INCLUDE (SCHEMA)` instead of acquiring more permissions. See - [database dumps](/docs/query/sql/show/#enterprise-access-control). -- Per-object `SHOW CREATE` and database dumps no longer append `OWNED BY`. - Schema-only output does not restore grants. Use the ACL dump for permissions, - and review [creator permissions on replay](/docs/query/sql/show/#enterprise-variant). -- With Enterprise access control enabled, materialized and live views reject - `SHOW CREATE DATABASE` with any ACL category, including the default. Change - schema-metadata queries to - `INCLUDE (SCHEMA)` under the - [catalogue rules](/docs/query/sql/create-mat-view/#catalogue-functions). -- A regular view's `PIVOT IN` subquery rejects administrative functions and - administrative `SHOW` statements. Existing definitions become invalid at - their next compilation, including startup. See - [view migration guidance](/docs/concepts/views/#administrative-operations-in-view-metadata). - ## Upgrade steps ### Docker @@ -215,6 +193,29 @@ check how these changes affect your users and clients: versions started with them, then rejected every OIDC login. See [Startup validation](/docs/security/oidc/#startup-validation). +### Upgrading to 4.2.0 + +QuestDB Enterprise 4.2.0 changes `SHOW CREATE` output and restricts +administrative operations in views. Check schema-export scripts and stored +views for the following before upgrading: + +- With access control enabled, plain `SHOW CREATE DATABASE` now requires both + `LIST USERS` and `USER DETAILS`. Schema-only scripts should use + `INCLUDE (SCHEMA)` instead of acquiring more permissions. See + [database dumps](/docs/query/sql/show/#enterprise-access-control). +- Per-object `SHOW CREATE` and database dumps no longer append `OWNED BY`. + Schema-only output does not restore grants. Use the ACL dump for permissions, + and review + [creator permissions on replay](/docs/query/sql/show/#enterprise-variant). +- With access control enabled, materialized and live views reject + `SHOW CREATE DATABASE` with any ACL category, including the default. Change + schema-metadata queries to `INCLUDE (SCHEMA)` under the + [catalogue rules](/docs/query/sql/create-mat-view/#catalogue-functions). +- A regular view's `PIVOT IN` subquery rejects administrative functions and + administrative `SHOW` statements. Existing definitions become invalid at + their next compilation, including startup. See + [view migration guidance](/docs/concepts/views/#administrative-operations-in-view-metadata). + ### QuestDB Enterprise BYOC For QuestDB Enterprise BYOC customers, QuestDB manages the upgrade process. The From 8be4f6304dc7e7955de0230bc8324e41c21efa01 Mon Sep 17 00:00:00 2001 From: glasstiger Date: Fri, 6 Nov 2026 13:00:29 +0000 Subject: [PATCH 4/5] docs(changelog): merge duplicate November 2026 Updated headings The merge from main added a second Updated heading. Move this PR's entries under the existing one. --- documentation/changelog.mdx | 2 -- 1 file changed, 2 deletions(-) diff --git a/documentation/changelog.mdx b/documentation/changelog.mdx index fd6fe582a..9c96f5810 100644 --- a/documentation/changelog.mdx +++ b/documentation/changelog.mdx @@ -45,8 +45,6 @@ This page tracks significant updates to the QuestDB documentation. capped set. Likewise, move a window function into the query you run against the view. -### Updated - - [SHOW CREATE DATABASE](/docs/query/sql/show/#show-create-database) - Documented QuestDB Enterprise 4.2.0 ACL output, principal/schema/grant ordering, the requirement for both `LIST USERS` and `USER DETAILS`, active storage policies, and replay limits. Documented the removal of `OWNED BY` from all `SHOW CREATE` output in 4.2.0, and updated the schema-copy recipe, RBAC, and governance guidance - [Views](/docs/concepts/views/#administrative-operations-in-view-metadata) - Documented QuestDB Enterprise 4.2.0 restrictions on administrative operations in `PIVOT IN` subqueries, schema-only database dumps in materialized and live views, and [upgrade guidance](/docs/operations/upgrade/#upgrading-to-420) for existing scripts and views From ca4951371dd251b8e8cd9bf95341684cb24a7c1e Mon Sep 17 00:00:00 2001 From: glasstiger Date: Fri, 6 Nov 2026 13:08:32 +0000 Subject: [PATCH 5/5] docs: clarify view PIVOT IN subquery restrictions --- documentation/concepts/views.md | 22 +++++++++++++++------- documentation/query/sql/create-view.md | 5 +++-- documentation/query/sql/pivot.md | 5 +++-- 3 files changed, 21 insertions(+), 11 deletions(-) diff --git a/documentation/concepts/views.md b/documentation/concepts/views.md index 759470711..c8da56f77 100644 --- a/documentation/concepts/views.md +++ b/documentation/concepts/views.md @@ -467,18 +467,26 @@ permissions until you run [`ALTER VIEW`](/docs/query/sql/alter-view/) or ### Administrative operations in view metadata -A view's `PIVOT IN` subquery runs while QuestDB derives the view's columns, -including during background recompilation and startup. It cannot use -administrative functions or administrative `SHOW` statements that need a -calling session or have side effects. This restriction also applies to nested -subqueries and operations reached through another view. +A regular view can use administrative functions and administrative `SHOW` +statements in its query body, including ordinary subqueries. These operations +check the permissions of the user who queries the view, not those of its definer. + +A [`PIVOT IN` subquery](/docs/query/sql/pivot/#dynamic-in-list-from-subquery) is a +special case: its result values determine the output column names. QuestDB must +execute it at compile time to derive the view's columns, not just when a user +queries the view. This also happens during background recompilation and startup, +when there is no calling session. The subquery therefore cannot use administrative +operations that need a calling session or have side effects. + +This restriction extends to nested subqueries and other views read by the +`PIVOT IN` subquery. It does not apply to ordinary subqueries elsewhere in the +view's query body. With Enterprise access control enabled, `SHOW CREATE DATABASE` with any ACL category counts as an administrative statement. Use `SHOW CREATE DATABASE INCLUDE (SCHEMA)` if the subquery only needs schema metadata. Catalogue queries that do not require a calling session remain -allowed. Administrative operations outside `PIVOT IN` in a regular view's -query body still run with the reader's permission checks. +allowed. :::warning Existing views diff --git a/documentation/query/sql/create-view.md b/documentation/query/sql/create-view.md index 5c3e9b70a..a65d78aac 100644 --- a/documentation/query/sql/create-view.md +++ b/documentation/query/sql/create-view.md @@ -197,8 +197,9 @@ CREATE VIEW with_timestamp AS ( A `PIVOT IN` subquery in a view cannot use administrative functions or administrative `SHOW` statements, including Enterprise `SHOW CREATE DATABASE` with any ACL category. Use `INCLUDE (SCHEMA)` when only schema metadata is -needed. The restriction also applies through nested queries and other views, -and is checked when creating, replacing, altering, or recompiling a view. See +needed. The restriction extends to nested queries and other views read by the +`PIVOT IN` subquery, not to ordinary subqueries elsewhere in the view's query +body. It is checked when creating, replacing, altering, or recompiling a view. See [administrative operations in view metadata](/docs/concepts/views/#administrative-operations-in-view-metadata) for existing-view migration guidance. diff --git a/documentation/query/sql/pivot.md b/documentation/query/sql/pivot.md index dd70f2a8f..b43d5ee09 100644 --- a/documentation/query/sql/pivot.md +++ b/documentation/query/sql/pivot.md @@ -266,9 +266,10 @@ For exploratory analysis, dynamic subqueries are convenient. For production quer use a constant list or store keys in a small dimension table for better performance. ::: -In a stored view, an `IN` subquery cannot use administrative functions or +In a stored view, a `PIVOT IN` subquery cannot use administrative functions or administrative `SHOW` statements, including Enterprise ACL dumps. This also -applies through nested queries and other views. Schema-only +applies to nested queries and other views read by that subquery, not to ordinary +subqueries elsewhere in a regular view's query body. Schema-only `SHOW CREATE DATABASE INCLUDE (SCHEMA)` remains allowed in a regular view. See [administrative operations in view metadata](/docs/concepts/views/#administrative-operations-in-view-metadata) for restrictions and migration guidance.