MySQL and MariaDB driver

.github/workflows/e2e.yaml

@@ -87,16 +87,19 @@ jobs:
         kubectl wait --for=condition=Ready node --all --timeout=120s
         kubectl get nodes
 
-    - name: Deploy backing services (Redpanda, VersityGW, MinIO, fake-gcs-server)
+    - name: Deploy backing services (Redpanda, VersityGW, MinIO, fake-gcs-server, MariaDB)
       run: |
         set -euxo pipefail
+        # Generates the MariaDB account password into Secrets; prints none.
+        test/e2e/backing-services/mariadb-secret.sh
         kubectl apply -k test/e2e/backing-services
         # rollout status (not wait --for=Ready) tolerates the pods not
         # existing yet at the moment we ask.
         kubectl -n redpanda rollout status statefulset/redpanda --timeout=300s
         kubectl -n blobs rollout status deploy/versitygw --timeout=180s
         kubectl -n blobs rollout status deploy/minio --timeout=180s
         kubectl -n blobs rollout status deploy/fake-gcs --timeout=180s
+        kubectl -n mysql rollout status statefulset/mariadb --timeout=300s
 
     - name: Set up contain
       uses: solsson/setup-contain@49cc4cce1498df8a59e88fff52c1a9b747f11f08 # v1
@@ -127,7 +130,7 @@ jobs:
 
     - name: Run e2e harness
       env:
-        IMPLEMENTATIONS: redpanda,versitygw,minio,fakegcs
+        IMPLEMENTATIONS: redpanda,versitygw,minio,fakegcs,mariadb
         OCI_DIR: ./oci
         # PUSH_AS reaches the k3d local registry via its host-published
         # port; CONTROLLER_IMAGE uses the same registry's in-cluster
@@ -148,6 +151,25 @@ jobs:
         kubectl -n blobs describe pod -l app=versitygw || true
         kubectl -n blobs describe pod -l app=minio || true
         kubectl -n blobs describe pod -l app=fake-gcs || true
+        kubectl -n mysql describe pod -l app=mariadb || true
+        kubectl -n mysql logs statefulset/mariadb --tail=100 || true
+
+  # The mysql driver against real servers (MariaDB 10.11 and 11.4,
+  # MySQL 8.0 and 8.4), one container each; the script is the same
+  # one contributors run, and it owns the image list.
+  mysql-integration:
+    runs-on: ubuntu-latest
+    timeout-minutes: 20
+    steps:
+    - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
+
+    - name: Set up Go
+      uses: actions/setup-go@4a3601121dd01d1626a1e23e37211e3254c1c06c # v6.4.0
+      with:
+        go-version-file: go.mod
+
+    - name: mysql driver integration test
+      run: test/integration/mysql.sh
 
   # The image push job runs only on main-branch builds in the canonical
   # Yolean org with a successful e2e. It rebuilds (proves reproducibility),
@@ -159,7 +181,7 @@ jobs:
       github.ref == 'refs/heads/main'
       && github.event_name == 'push'
       && github.repository_owner == 'Yolean'
-    needs: e2e
+    needs: [e2e, mysql-integration]
     runs-on: ubuntu-latest
     permissions:
       contents: read

README.md

@@ -1,11 +1,11 @@
 # buckety-controller
 
 A small in-cluster operator that provisions named resources on
-backing services (Kafka topics, S3 buckets) and mints
-`secretKeyRef`-friendly credentials Secrets for consumers.
+backing services (Kafka topics, S3 buckets, MySQL databases) and
+mints `secretKeyRef`-friendly credentials Secrets for consumers.
 
-> **Status: v1alpha1 shipped.** Three drivers (`kadm`, `s3`,
-> `gcs`) are implemented and covered by the e2e suite in CI. Every commit on
+> **Status: v1alpha1 shipped.** Four drivers (`kadm`, `s3`,
+> `gcs`, `mysql`) are implemented and covered by the e2e suite in CI. Every commit on
 > `main` pins a reproducible image build in
 > [`deploy/kustomize/release/`](./deploy/kustomize/release), and
 > the same digest is pushed to `ghcr.io/yolean/buckety-controller`
@@ -56,16 +56,17 @@ here. The deviations are operational, not architectural:
 
 ## Status
 
-v1alpha1. Three drivers shipped:
+v1alpha1. Four drivers shipped:
 
 | Driver | Backing services | Notes |
 | --- | --- | --- |
 | `kadm` | Kafka-protocol brokers (Redpanda, Apache Kafka, Confluent) | Topic create/alter/delete. v1alpha1: no per-consumer SASL/SCRAM scoping. |
 | `s3` | S3-compatible (VersityGW, MinIO, AWS S3, Cloudflare R2, Hetzner, GCS interop) | Bucket create/delete. v1alpha1: all consumers receive the backend's root keys. |
 | `gcs` | Google Cloud Storage via the native JSON API | Bucket create/update/delete with location, uniform bucket-level access, versioning and lifecycle parameters. Access Secrets carry a static HMAC pair (S3-protocol data path); all consumers receive the same pair. Driver 0.2 adds opt-in per-bucket service accounts (`parameters.serviceAccount`): a bucket-scoped GCP SA whose key JSON lands in each access Secret for OAuth2 bearer-token auth (`examples/gcs/service-account/`). |
+| `mysql` | MariaDB 10.11+ and MySQL 8 | Database create/drop with `characterSet` and `collation` parameters. Each access gets its own user, with grants for its role on that one database (`Reader`: `SELECT`; `ReadWrite`: data and schema changes). Every name is confined to the backend's `namePrefix`; see [`docs/mysql.md`](./docs/mysql.md) for the account it needs and why. |
 
 e2e coverage in CI runs against Redpanda (`kadm`), VersityGW +
-MinIO (`s3`) and fake-gcs-server (`gcs`). The other listed S3
+MinIO (`s3`), fake-gcs-server (`gcs`) and MariaDB (`mysql`). The other listed S3
 backends share the same client library and the same e2e shape; if
 you hit a compatibility issue with one of them, please file an
 issue. For `gcs`, behaviours the emulator cannot exercise (HMAC
@@ -202,8 +203,24 @@ backends:
     # examples/gcs/service-account/ for required grants.
     # serviceAccounts:
     #   project: my-buckety-identities
+
+- name: cluster-mysql
+  driver: mysql
+  config:
+    host: mysql.mysql.svc.cluster.local
+    adminUser: buckety
+    adminPassword: ${MYSQL_BUCKETY_PASSWORD}
+    # Every database and user the driver touches starts with this;
+    # the account's grants are scoped to it. See docs/mysql.md.
+    namePrefix: b_
 ```
 
+The mysql backend's account and its first-boot creation from a
+Secret are in [`examples/mysql/`](./examples/mysql). `CREATE USER`
+cannot be scoped by name on MariaDB or MySQL, so the driver refuses
+every database and user outside `namePrefix`; keep accounts that
+matter on an instance the controller has no backend for.
+
 Access Secrets carry the S3-interop `endpoint` (a bare host - the
 scheme is the consumer's choice) and `region`, derived per bucket
 from its location: `storage.<region>.rep.googleapis.com` for
@@ -356,16 +373,19 @@ explicit `BucketyAccess` in a second apply, if zero-gap matters.
 See [`SPEC.md`](./SPEC.md#implicit-access-defaultaccess) for the
 full lifecycle.
 
-> **`role` is advisory in v1alpha1.** `BucketyAccess.spec.role`
-> accepts `Reader`, `Writer`, or `ReadWrite`, but the v1alpha1
-> kadm and s3 drivers do not yet scope credentials per role.
+> **`role` is advisory in v1alpha1, except for `mysql`.**
+> `BucketyAccess.spec.role` accepts `Reader`, `Writer`, or
+> `ReadWrite`, but the v1alpha1 kadm, s3 and gcs drivers do not yet
+> scope credentials per role.
 > Every Secret minted for the same `Buckety` carries identical
 > root credentials regardless of `role`. The controller surfaces
 > a `ScopingNotImplemented=True` condition on each affected
 > `BucketyAccess` (visible via `kubectl describe bucketyaccess`)
 > so the gap is honest, not silent. Scoped credentials
 > (SASL/SCRAM, IAM users) are v1alpha2 work. Until then, treat
-> `role` as documentation of intent, not enforcement.
+> `role` as documentation of intent, not enforcement. The `mysql`
+> driver does enforce it: each access is its own database user with
+> grants for its role, and `Writer` is refused.
 
 S3 is the same shape; see [`examples/s3/`](./examples/s3).
 
@@ -386,13 +406,14 @@ are in [`SPEC.md`](./SPEC.md#naming-templates).
 
 ## Resources
 
-- `Buckety` — a topic / bucket / future-MySQL-database. Selects
+- `Buckety` — a topic / bucket / MySQL database. Selects
   a backend by name. Carries mutable parameters; the controller
   reconciles drift to the backing service.
 - `BucketyAccess` — a Secret request. Each one mints exactly one
   Secret in the same namespace as the `BucketyAccess`. Multiple
   accesses can target the same Buckety (in v1alpha1 they all
-  receive identical credentials).
+  receive identical credentials, except with `mysql`, where each
+  gets its own user).
 
 Both kinds are namespaced. Cross-namespace `bucketyRef` is not
 supported in v1alpha1.
@@ -491,9 +512,8 @@ Auto-migration is not in v1alpha1.
 
 The full list is in [`SPEC.md`](./SPEC.md). Highlights:
 
-- No per-consumer credential scoping. SASL/SCRAM and IAM-user
-  minting are v1alpha2 work.
-- No MySQL driver.
+- No per-consumer credential scoping, except in the `mysql`
+  driver. SASL/SCRAM and IAM-user minting are v1alpha2 work.
 - No cross-namespace `bucketyRef`.
 - No hot-reload of `buckety-controller.yaml`.
 

SPEC.md

@@ -14,8 +14,8 @@ the two disagree, this file wins until both are revised together.
 ## Context
 
 Buckety provisions named, individually-configurable resources on
-backing services (Kafka topics, S3 buckets, future: MySQL
-databases) and mints per-consumer credentials.
+backing services (Kafka topics, S3 buckets, MySQL databases) and
+mints per-consumer credentials.
 
 The design is informed by a Yolean exploration of the Container
 Object Storage Interface (COSI). COSI's two-resource split
@@ -73,13 +73,14 @@ classes — they pick the named thing their cluster offers.
 | `kadm` | Kafka-protocol topics (Redpanda, Apache Kafka, Confluent) | `github.com/twmb/franz-go/pkg/kadm` | Redpanda |
 | `s3` | S3 API (VersityGW, MinIO, AWS S3, R2, Hetzner, GCS interop) | `github.com/aws/aws-sdk-go-v2/service/s3` | VersityGW + MinIO |
 | `gcs` | Google Cloud Storage, provisioned via the native JSON API | `cloud.google.com/go/storage` | fake-gcs-server |
+| `mysql` | MariaDB and MySQL databases, one user per access | `github.com/go-sql-driver/mysql` | MariaDB |
 
 Driver names are short, semantic, and stable; they MUST NOT
-change after v1alpha1 ships. New drivers — `mysql` via
-`go-sql-driver/mysql`, `minio-admin` for scoped MinIO IAM,
-`versitygw-iam` for scoped VersityGW IAM — are follow-ups, not
-v1alpha1. The `gcs` driver shipped as the first such follow-up
-(same v1alpha1 CRDs, additive driver registration).
+change after v1alpha1 ships. New drivers — `minio-admin` for
+scoped MinIO IAM, `versitygw-iam` for scoped VersityGW IAM — are
+follow-ups, not v1alpha1. The `gcs` driver shipped as the first
+such follow-up and `mysql` as the second (same v1alpha1 CRDs,
+additive driver registration).
 
 The `gcs` / `s3` boundary: the s3 driver's client-library bet
 covers GCS on the DATA path (HMAC keys work with SigV4 against
@@ -278,7 +279,9 @@ credentials drawn from the backend's root config. The CRD shape
 already supports per-access scoping for v1alpha2 — drivers are
 expected to ignore role/parameter values they don't yet
 implement and surface a `ScopingNotImplemented` condition rather
-than silently treating Reader as ReadWrite.
+than silently treating Reader as ReadWrite. The mysql driver
+scopes per access and role already, and refuses a role it has no
+grant set for (`Writer`) instead of ignoring it.
 
 ## Implicit access (`defaultAccess`)
 
@@ -568,7 +571,8 @@ applied against minio, versitygw and the GCS emulator.
 
 Subfamily hierarchies (definitions shared by a subset of a
 family) are deferred until a third bucket driver forces the
-question; today the family is flat and small.
+question; today the family is flat and small. `mysql` is not in
+a family; a PostgreSQL driver would make a database family.
 
 ## Schemas
 
@@ -597,7 +601,7 @@ standalone whole-document schemas for editors (the
 kubernetes-json-schema pattern): `buckety.schema.json` plus one
 per family and per driver (`buckety-blobstore` for the
 object-store family, `buckety-gcs`, `buckety-s3`,
-`buckety-kadm`) and `bucketyaccess.schema.json`.
+`buckety-kadm`, `buckety-mysql`) and `bucketyaccess.schema.json`.
 The suffix forms a specialize/generalize ladder walked by
 switching the URL: the family rung accepts only family-common
 parameters, so a resource annotated with it provably stays
@@ -639,11 +643,13 @@ call, guided by the chosen operator SDK.
   (see *`Delete` is recursive*), idempotent on NotFound; called
   only when `retentionPolicy == Delete`.
 - `GrantAccess` — mints credentials, applies backend-side
-  permissions (no-op in v1alpha1 for both shipped drivers),
-  returns the Secret payload as a flat `map[string][]byte`.
-  Idempotent for the same `(Buckety, BucketyAccess)` pair.
+  permissions (no-op in v1alpha1 for kadm and s3; mysql creates
+  a user per access with the role's grants), returns the Secret
+  payload as a flat `map[string][]byte`. Idempotent for the same
+  `(Buckety, BucketyAccess)` pair, which the request identifies
+  by the access's namespace and name.
 - `RevokeAccess` — removes the backend-side principal (no-op in
-  v1alpha1), idempotent on NotFound.
+  v1alpha1 for kadm and s3), idempotent on NotFound.
 - `ValidateParameters` — used by the admission webhook and, with
   the webhook disabled, by the reconciler; returns an error
   naming the offending key.
@@ -720,8 +726,8 @@ identity:
   on the broker.
 - `s3` driver → key `bucket` carrying the resolved bucket name
   on the backend.
-- Future drivers follow the same pattern: `database`,
-  `namespace`, etc.
+- `mysql` driver → key `database` carrying the database name.
+- Future drivers follow the same pattern: `namespace`, etc.
 
 The resource-type key is the unambiguous handle a consumer uses
 to operate on the resource. The other keys (`bootstrap`,
@@ -879,6 +885,48 @@ ever wedges an SA at the key cap, surplus `USER_MANAGED` keys on
 a marker-verified SA are safe to delete server-side and the fleet
 re-mints within one reconcile.
 
+### `mysql` driver
+
+```yaml
+apiVersion: v1
+kind: Secret
+metadata:
+  name: orders-db
+  namespace: shop
+type: Opaque
+data:
+  host:     <base64>    # mysql.mysql.svc.cluster.local
+  port:     <base64>    # 3306
+  database: <base64>    # b_shop_orders (resource-type key)
+  username: <base64>    # b_shop_orders (this access's own user)
+  password: <base64>
+  jdbcUrl:  <base64>    # jdbc:mariadb://mysql.mysql.svc.cluster.local:3306/b_shop_orders
+  url:      <base64>    # mysql://<username>:<password>@mysql.mysql.svc.cluster.local:3306/b_shop_orders
+```
+
+Unlike the v1alpha1 kadm and s3 drivers, credentials are scoped:
+each `BucketyAccess` gets a user of its own, named from the
+access's namespace and name, with privileges on its one database
+by role (`Reader`: `SELECT`; `ReadWrite`: data changes plus the DDL
+schema migrations need; `Writer` is refused). `GrantResult.Scoped`
+is true, `status.principal` is `user@host`, and `RevokeAccess`
+drops the user under every retention policy. The password is
+generated at the first grant and read back from the Secret on every
+later one, so a password written into the Secret is applied to the
+server and deleting the Secret rotates it.
+
+**The prefix boundary.** The controller's account needs
+`CREATE USER` globally, which MariaDB and MySQL cannot scope by
+name and which also allows altering and dropping every account on
+the server. The backend's `namePrefix` is therefore the driver's
+security boundary: every database and user name it creates or
+touches must start with it, `spec.name` must resolve to it
+(`b_${namespace}_${name}`), and anything outside it is refused,
+including principals read back from status. Accounts that matter
+belong on instances the controller has no backend for.
+[`docs/mysql.md`](docs/mysql.md) has the account grants, naming
+rules, drift detection and the decisions taken.
+
 ## Adoption
 
 A resolved backend resource name can collide with a resource that
@@ -910,7 +958,8 @@ when the resource already exists:
 - `AdoptEmpty` (default): adopt only when the resource holds no
   live content (no current objects for buckets; noncurrent
   versions and delete markers are not consulted. No retained
-  records for topics; fully expired records count as none).
+  records for topics; fully expired records count as none. No
+  tables, views, routines or events for databases).
   A non-empty resource surfaces `Ready=False` with reason
   `BackendResourceExists`, mints no Secret, stamps nothing, and
   re-checks on the periodic cadence. The message names the
@@ -1149,6 +1198,7 @@ Required CI matrix for v1alpha1:
 | `kadm` | redpanda |
 | `s3`   | versitygw, minio |
 | `gcs`  | fakegcs (fake-gcs-server; covers the JSON-API control plane — real-GCS-only behaviours like HMAC auth enforcement, the 90-day UBLA disable window and the serviceAccounts IAM surface (no iam.googleapis.com or bucket-IAM emulation; unit-tested against an httptest fake instead) are documented, not e2e-gated) |
+| `mysql` | mariadb (MariaDB 10.11; the driver's integration test covers MariaDB 10.11 and 11.4 and MySQL 8.0 and 8.4 in a separate CI job, `test/integration/mysql.sh`) |
 
 Adding an implementation later (e.g. AWS S3 once the project has
 credentials and a budget) requires no example or harness changes,
@@ -1182,14 +1232,14 @@ and the corresponding GHA secret.
 
 ## Non-goals in v1alpha1
 
-- MySQL driver.
 - Per-consumer credential scoping (SASL/SCRAM for kafka, IAM
   users for S3). All `BucketyAccess` instances for the same
   `Buckety` receive identical credentials. Partial exception
   since gcs driver 0.2: opt-in per-BUCKET service accounts give
   each access its own key for a bucket-scoped identity (see
   Secret output > gcs driver), but role scoping remains
-  unimplemented.
+  unimplemented. The mysql driver is the exception: a user per
+  access, scoped by role (see Secret output > mysql driver).
 - Cross-namespace `bucketyRef`.
 - Quota enforcement.
 - Hot-reload of `buckety-controller.yaml` (envsubst is

cmd/buckety/main.go

@@ -46,6 +46,7 @@ import (
 	// this import list.
 	_ "github.com/Yolean/buckety-controller/pkg/drivers/gcs"
 	_ "github.com/Yolean/buckety-controller/pkg/drivers/kadm"
+	_ "github.com/Yolean/buckety-controller/pkg/drivers/mysql"
 	_ "github.com/Yolean/buckety-controller/pkg/drivers/s3"
 )
 

deploy/kustomize/release/kustomization.yaml

@@ -7,7 +7,7 @@ kind: Kustomization
 resources:
 - ../base
 images:
-- digest: sha256:19e3172abce9c8be53ea19fbcd60ffc3ec636b11c3457af56e829d063e877efe
+- digest: sha256:8c3c493780935b8084e2241722f60605464f2cf3d70f83a9c68c2edd43f222d6
   name: ghcr.io/yolean/buckety-controller
   newName: ghcr.io/yolean/buckety-controller
-  newTag: 20260904T064507Z
+  newTag: 20261002T051036Z