Repository navigation
Feature/1074 lua attach - #1084
Merged
Merged
Conversation
…butes and trackables
…d and has the full freedom
Inside opensak.transaction() the columns OpenSAK maintains (distance, bearing, log/waypoint/trackable counts, log dates) are no longer recalculated after every write. The touched caches are collected and recalculated once, just before the commit. This covers sql_write() and the ORM writes (update, insert, insert_/update_<child>). opensak.resync() recalculates them on demand inside a transaction. Codes of caches added by SQL INSERT are still normalised immediately, so the cache can be found by its code inside the transaction. A rolled-back transaction or savepoint leaves nothing pending, and a failed recalculation undoes the whole transaction. Outside a transaction, every write is still recalculated before its own commit. 2000 insert_log calls on one cache in a transaction: 4.5 s -> 1.8 s.
- New feature flag lua-macro-write (beta builds only): update, insert, insert_*/update_*, sql_write, transaction, in_transaction and resync are nil without it; set_corrected/clear_corrected then change the database without asking, as before. The Settings list of approved databases is hidden too. - Docs steer authors to update/insert/insert_<x>/update_<x> first and present sql_write as the tool for bulk work; gated functions are marked beta. - opensak.macro_db() checks write access to the macro's folder through the folder-permission model (OpenSAK-Org#978), prints the path when it creates the file, and a read-only macro database gives a clear error. - Test fixture: reset_flags restores features.json before reloading, so patched flag values no longer leak into later tests.
opensak.macro_db() built the file name from the called macro's folder and the caller's chunk name, so a macro started with opensak.run_macro() opened <caller>.db3 in the callee's folder (OpenSAK-Org#1039). - Name the file after _macro_path (switched by run_macro()); fall back to base_dir / chunk_name for a top-level run without a macro path. - In a called macro, macro_db() detaches the caller's file and attaches the callee's own as `macro`; the caller's file is attached again when the called macro returns, also after an error (the callee's error wins over a failed re-attach). - Until the callee calls macro_db(), `macro` still means the caller's file (DETACH is impossible inside opensak.transaction()). - WritableDatabase.detach_macro_db(); macro_db() docs updated. - Test: caller and helper in lib/, normal and failed return.
…#1034) Before a macro's first write, OpenSAK now asks whether this macro may change the OpenSAK databases (Deny / Until OpenSAK closes / Always). "Always" is stored by macro file path (macros.write_always) and covers all databases; macros still only write the active one. Editing the macro keeps the approval. - Unsaved macros get Deny / Until OpenSAK closes only. - Macros started with run_macro() are asked about on their own. - set_corrected/clear_corrected use the same approval. - Settings → Folder permissions lists approved macros by name (path in tooltip), removable. - MacroHost.approve_database_write(name, path) is now approve_macro_write(macro). - New dialog texts in all languages; docs and tests updated.
…OpenSAK-Org#1035) New macro functions (behind the lua-macro-write flag, write permission as for opensak.update()): - opensak.delete(code | codes): deletes caches with their logs, waypoints, attributes, trackables and user note. One code returns a boolean; a list returns the number deleted. - opensak.delete_log/waypoint/attribute/trackable(id): deletes one row and recalculates the cache's counts and log dates. opensak.sql_write() now allows DELETE on the cache tables. The schema's foreign keys have no ON DELETE CASCADE, so a temporary BEFORE DELETE trigger on caches deletes a cache's rows first. AFTER DELETE triggers on the child tables record the cache for the derived-column refresh. Also covers INSERT OR REPLACE, which now replaces a cache and its rows instead of being refused. Deletes take part in opensak.transaction() and are undone with it. The GUI reloads through the existing caches_removed() host hook, which also clears the detail panel. At the end of a run the macro output reports the number of deleted caches per database, without deletes a transaction undid. The bulk delete of the main window and of move_caches now share db/cache_delete.delete_caches(). Docs (API registry, Changing caches section) updated. The generated api.md and Lua stub are refreshed by CI. Tests: test_macro_delete_1035.py, plus existing tests adjusted for DELETE no longer being refused.
…SAK-Org#1074) Macros can attach other OpenSAK databases read-only to the active database's SQL connection, so one opensak.sql() or sql_write() statement can join them with the active database and opensak.macro_db(). This is a controlled replacement for GSAK's SQL ATTACH, which stays refused. - opensak.attach(name [, schema]) attaches a database by its name in the database list and returns the schema name. Without a schema, the name comes from the database name ("Found 2020" -> found_2020). Each attach prints a line in the macro output. - opensak.detach(schema) and opensak.attached() list or remove them. - Read-only twice: the file is attached with mode=ro, and the authorizers allow only reading on any schema other than main, temp and macro. PRAGMA <schema>.user_version can be read. - Refused: reserved, invalid or used schema names (main, temp, macro, companion, opensak_*), the active database, a database that is already attached, the macro_db() file, missing and non-SQLite files, attach or detach inside opensak.transaction(), and more than SQLite's attach limit (macro_db() included). - Attaching the same database under the same schema again does nothing. - Lifetime: everything is detached when the run ends, also on error or cancel. A macro started with run_macro() sees its caller's attachments, and its own attaches and detaches are undone when it returns. Attachments stay after switch_database(), except a database that becomes the active one. - tables() and columns() list attached tables as <schema>.<name>. - The write connection is opened with uri=True so ATTACH can take a file:...?mode=ro URI. - Behind the lua-macro-write flag (new ApiFunction.beta: flag-gated, but changes nothing, so the docs say no approval is needed). - Docs: new section "Several databases in one query", with a FindStatGen-style example and the GSAK ATTACH mapping, and a new row in the GSAK comparison. The generated files are left to the docs bot. - Tests: test_macro_attach_1074.py. Phase 2 (attach_file() for any SQLite file) is not included.
Member
|
Thanks @nagisml — this is very carefully designed:
I also tested that a WAL-mode database attaches fine with Agree that no approval is needed, since it only reads. Phase 2 ( Approving. This PR is stacked on #1080 and #1083, so it'll be merged once those are in. |
Root bypasses POSIX file permissions, so chmod(S_IREAD) can't make the file read-only and the write-rejection assertions fail. Skip the test when euid == 0 (reported in PR review).
REPLACE resolves a conflict by deleting the existing row, and the cache delete trigger then removes the cache's logs, waypoints, attributes, trackables and user note. sql_write now refuses INSERT OR REPLACE, REPLACE INTO, UPDATE OR REPLACE and ON CONFLICT REPLACE before the statement runs, and points to INSERT ... ON CONFLICT (gc_code) DO UPDATE.
…OpenSAK into feature/1074-lua-attach
This was referenced Oct 19, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Lua macros: attach other databases read-only (
opensak.attach()), phase 1 of #1074This covers phase 1 of #1074: attaching other OpenSAK databases by name. Phase 2 (
attach_file()for any SQLite file) is still open, so the issue should stay open after this is merged.What it does
Macros can attach other OpenSAK databases read-only to the active database's SQL connection. One
opensak.sql()/sql_write()statement can then join the active database, the attached ones andopensak.macro_db(). This replaces GSAK's SQLATTACHin a controlled way; rawATTACH/DETACHstay refused.opensak.attach(name [, schema]),opensak.detach(schema),opensak.attached()mode=ro, and the authorizers allow only reading on attached schemas.main,temp,macro,companion,opensak_*), the active database, a database that is already attached, themacro_db()file, missing or non-SQLite files, attach/detach insideopensak.transaction(), and more than SQLite's limit of 10 attached databases (macroincluded).run_macro(), the called macro sees its caller's attachments, and its own changes are undone when it returns. Attachments stay afterswitch_database(), except one that becomes the active database.opensak.tables()/opensak.columns()show attached tables as<schema>.<name>. Attached databases are not migrated, so an older schema may lack columns.lua-macro-writefeature flag.Decisions (from the open questions in #1074)
attach_file()) is left out of this PR.database =option andattach()keep separate code:database =uses its own read-only connection, whileattach()uses the active database's connection.Docs
ATTACHmapping table.ATTACH/DETACHin the GSAK comparison.docs/macros/api.mdand the editor stubs are left to the docs bot.Tests
The new
tests/unit-tests/test_macro_attach_1074.pyhas 51 tests. They cover:macrosql_writewith subqueries on an attached databasemode=rochecked on its ownATTACH/DETACHstill refusedswitch_database()-wal/-shmfilesThe existing macro tests pass too.