Skip to content

feat(view): add view metadata types and validation - #1069

Open
kamcheungting-db wants to merge 1 commit into
apache:mainfrom
kamcheungting-db:feat/view-metadata-types
Open

kamcheungting-db wants to merge 1 commit into
apache:mainfrom
kamcheungting-db:feat/view-metadata-types

Conversation

@kamcheungting-db

Copy link
Copy Markdown
Contributor

Adds the Iceberg view format v1 metadata model as the first step of view support. This provides types, validation, and lookup helpers for representing view definitions and their history.

Metadata types

  • ViewVersion records a definition's version ID, schema ID, creation timestamp, summary, representations, optional default catalog, and default namespace.
  • ViewRepresentation is a variant of SqlViewRepresentation, which pairs SQL text with its engine dialect, and UnknownViewRepresentation, which stores an unrecognized representation's type identifier. GetSql() and RepresentationType() expose the active representation.
  • ViewMetadata holds the view UUID, format version, location, schemas, current version ID, versions, version log, and properties. ViewHistoryEntry records a timestamp and the version made current.
  • Equality operators compare the metadata, versions, representations, and history entries by value.

Validation and lookups

ViewMetadata::Validate() returns ErrorKind::kInvalid for the first violated invariant and leaves the metadata unchanged. It checks that:

  • The view format version is 1, and the UUID and location are non-empty.
  • Schemas and versions are non-empty lists containing non-null entries with unique IDs.
  • The current version is listed, and every version references a listed schema.
  • Every version has a non-empty default namespace.
  • Each SQL representation has non-empty SQL text and a non-empty dialect, with at most one SQL representation per dialect within a version. Dialect comparison uses ASCII case folding.
  • Unknown representations have a non-empty type identifier.

View schemas are validated independently of table-format schema restrictions, allowing projections to retain the types and nullability of referenced columns.

CurrentVersion(), VersionById(), CurrentSchema(), and SchemaById() return the matching non-null entry or ErrorKind::kNotFound when the requested version or schema is absent.

The change also adds StringUtils::EqualsIgnoreCaseAscii for dialect matching and registers the new sources, tests, and public headers with the build and C++20 compatibility check.

JSON serialization, full preservation of unknown representation payloads, metadata updates, and requirements are follow-up work.

Validation performed

  • view_metadata_test: all 24 tests pass. Coverage includes valid metadata, multiple versions and dialects, unknown representation types, individual validation failures with asserted error kinds, missing-entry lookups, equality, and representation helpers.
  • cpp20_compatibility_test: passes with the new public headers registered.
  • pre-commit run --files <changed files>: all applicable hooks pass, including trailing whitespace, end-of-file fixes, clang-format, cmake-format, and public-header generation.
  • The existing build directory's full CTest run completed with 20 of 23 tests passing. parquet_test failed because its stale binary references a fixture absent from this branch; encryption_test and scheduled_executor_test were not run because their binaries were not built. These are separate from the passing view and compatibility tests.

Deliver the view format v1 metadata model beside its table equivalent:

- ViewVersion carries version-id, schema-id, timestamp-ms, summary,
  representations, optional default catalog, and required default namespace.
- Representations are SQL (sql, dialect) or opaque unknown types preserved
  for round trips, modeled as a variant with GetSql and RepresentationType
  helpers.
- ViewMetadata carries view-uuid, format-version, location, schemas,
  current-version-id, versions, version-log, and properties, with
  CurrentVersion/VersionById/CurrentSchema/SchemaById accessors returning
  NotFound for missing entries.
- Validate() checks view-v1 invariants without applying table-format schema
  restrictions: supported format version, non-empty UUID/location, listed
  schemas/versions with unique IDs, current-version and schema references,
  non-empty default namespaces, non-empty SQL representations with at most
  one per dialect under ASCII case folding, and non-empty unknown
  representation types.
- Add StringUtils::EqualsIgnoreCaseAscii for the ASCII-only dialect folding;
  existing Unicode-aware EqualsIgnoreCase is unchanged.
- view_metadata_test validates synthetic metadata: every invariant violated
  one at a time with asserted error kinds, accepted shapes including unknown
  representations, accessor NotFound behavior, and equality.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant