Lakehouse catalog and orchestration built on object storage.
Arco manages table metadata, tracks lineage, and schedules data pipelines. It publishes catalog and run metadata as Parquet files for your query engine. You supply the query engine and task workers.
Status: Incubating. Arco is not yet production ready, and APIs may change.
- Catalog: Table identity, schemas, and storage locations. New table registrations default to Delta Lake; Iceberg and plain Parquet have documented support levels.
- Orchestration: Planning, scheduling, and run history. External workers execute tasks and report results to Arco.
- Metadata: Catalog, lineage, and run history in open Parquet files. The legacy catalog API provides scoped signed URLs for published files.
You need Rust 1.88+ and the Protocol Buffers compiler (protoc).
Clone, build, and test:
git clone https://github.com/daxis-io/arco.git
cd arco
cargo build --workspace
cargo test --workspaceFollow the quick start to run a local pipeline that registers tables, commits Delta data, and executes a task.
- Installation
- Architecture
- Published metadata
- Integration guide
- Flow dispatcher example
- API Reference
- Documentation Index
To browse the guide locally:
cargo install mdbook --version 0.4.52 --locked
cd docs/guide && mdbook build --openFor protobuf changes, follow the
compatibility policy
and run cargo xtask proto-breaking-check.