Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 16 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,22 @@ it shipped into a `## [x.y.z] - date` section of their own; the tag's

## Unreleased

### Security

- `examples/18_slack_daily_report.py` escapes the Slack messages and the
model's summary before they go into the report, and renders the PDF from
the written file through a URL fetcher that serves only that file. Unescaped, a message could add markup that made
the PDF renderer read a local file.

### Changed

- Web viewer: sending a file and waiting for the host's offer no longer await
inside a loop. The order of chunks, the 4 MiB backpressure limit and
one-transfer-at-a-time are unchanged; a channel that closes mid-transfer now
ends that transfer with an error instead of leaving it pending.

## [1.0.1] - 2026-10-09

### Changed

- `is_package_name` (the gate behind `AC_add_package_to_executor` and
Expand Down
1 change: 0 additions & 1 deletion Progress.md
Original file line number Diff line number Diff line change
Expand Up @@ -329,7 +329,6 @@ U-20261009-18 的每一個裝置面呼叫都只對假的 ADB/uiautomator2/WD
這台機器沒有 free-threaded 直譯器:驗證用的是模擬繼承與真的 `-X thread_inherit_context=1`(3.14.7,GIL 開著);`free-threaded-scope` job(3.14t)是第一次真的跑。
- **能力探測**:Windows 的鎖定工作站、session 0、低完整性與擷取失敗分支只用假資料判斷;macOS 的三個 preflight 呼叫只對假物件跑過,
pyobjc 的 `Quartz` 有沒有 `CGPreflightListenEventAccess` 未知(沒有的話 `recording`/`stop_shortcut` 回 `unknown`)。
- **SonarCloud/Codacy 的抑制還沒被下一次分析確認**(U-20261009-42):`web_viewer/index.html` 裡內嵌 script 的四個 `// NOSONAR`(S9382,同檔先前的 NOSONAR 就沒被認)、`run_history/history_store.py` 分兩行寫的 `nosemgrep`、以及 `examples/18_slack_daily_report.py` 的 realpath 檢查是否被 Sonar 認成 sanitizer(S2083)。分析跑完後還開著的:內嵌 script 搬到 `.js` 檔,其餘在儀表板標 accepted。
- **Windows 的版面鍵名**(`slash` 等 8 個)只在美式配置上實際呼叫過。
- **狀態機 `on_enter` 的單一動作寫法**以前一直被拒絕、從沒執行過(已修,U-20261009-24):依賴「它不會跑」的既有狀態機現在會跑它。

Expand Down
14 changes: 7 additions & 7 deletions architecture_explore.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ iOS(WebDriverAgent)。核心能力是滑鼠/鍵盤控制、影像辨識、
| 指標 | 數值 |
| --- | ---: |
| Python 模組總數(含周邊子專案) | 1,177 |
| 程式碼總行數 | 188,836 |
| 程式碼總行數 | 188,839 |
| `je_auto_control/utils/` 子套件數 | 311 |
| `AC_*` 動作指令數(`known_commands()` 實測) | 852 |
| 套件門面 `__all__` 公開名稱數 | 1,244 |
Expand Down Expand Up @@ -495,7 +495,7 @@ socket server 有 8 MiB 讀取上限與 30 秒 handler timeout。

### 5.4.9 AI / Agent / LLM

> 13 個套件、約 25,909 行。
> 13 個套件、約 25,912 行。

| 模組 | 行數 | 職責 |
| --- | ---: | --- |
Expand All @@ -508,7 +508,7 @@ socket server 有 8 MiB 讀取上限與 30 秒 handler timeout。
| `utils/cua_action/` | 204 | 標準化 computer-use 動作結構(Anthropic/OpenAI → `AC_*`) |
| `utils/llm/` | 384 | 自然語言 → action list 規劃器 + Anthropic/null 後端 |
| `utils/mcp_registry/` | 97 | MCP registry `server.json` 資訊清單產生(可被發現) |
| `utils/mcp_server/` | 21,219 | **無頭 MCP 伺服器**(16K LOC,預設註冊 757 個工具=738 個 `ac_*` + 19 個別名):stdio + HTTP 傳輸、工具工廠與處理器、資源、prompt、稽核、限流、外掛熱重載 |
| `utils/mcp_server/` | 21,222 | **無頭 MCP 伺服器**(16K LOC,預設註冊 757 個工具=738 個 `ac_*` + 19 個別名):stdio + HTTP 傳輸、工具工廠與處理器、資源、prompt、稽核、限流、外掛熱重載 |
| `utils/tool_use_schema/` | 195 | 把 `AC_*` 指令匯出成 Claude/OpenAI 的 tool-use schema |
| `utils/trajectory_eval/` | 132 | agent 軌跡評估:依評分規準為一次執行打分 |
| `utils/vision/` | 582 | VLM 元素定位器(依描述找元素)+ Anthropic/OpenAI/null 後端 |
Expand Down Expand Up @@ -721,7 +721,7 @@ socket server 有 8 MiB 讀取上限與 30 秒 handler timeout。
| `mouse_aliases.py` | 39 | 單鍵點擊別名(`AC_click_left` 等),executor 與 callback executor 共用。 |
| `flags.py` | 19 | 旗標讀取 `as_bool`:依拼法(`"true"`/`"yes"`/`"on"`/`"1"`)而非 Python 真值判斷,所以 JSON、CLI、MCP 傳來的 `"false"` 是關;所有 adapter 與流程指令共用。 |

#### `utils/mcp_server/`(21,219 行,757 個工具)— 最大子系統
#### `utils/mcp_server/`(21,222 行,757 個工具)— 最大子系統

| 檔案 | 行數 | 職責 |
| --- | ---: | --- |
Expand All @@ -737,7 +737,7 @@ socket server 有 8 MiB 讀取上限與 30 秒 handler timeout。
| `tools/_handlers_executor_bridge.py` | 1,429 | 252 個純委派(中位數 3 行,最長的 16 行全是參數簽章):每個都是 `from action_executor import _x` 再 `return _x(...)`,沒有分支邏輯。超過 750 行,理由記在 `Progress.md` 的豁免表(再切只能照 MCP 工廠領域分,會把同一種委派散進十幾個沒有語意邊界的檔)。 |
| `tools/_handlers_locators.py` | 483 | 同一種 adapter,定位主題:無障礙樹、智慧等待、自我修復、螢幕觀察、座標空間、視覺與 OCR、影像去重、元件倉庫、A/B 定位。 |
| `tools/_handlers_operations.py` | 654 | 同一種 adapter,營運主題:agent 與其記憶/追蹤、治理與合規、成本與遙測、失敗掛鉤、看門狗、速率限制、檢查點、核可、產物與資產、測試選擇與分片、佇列與 saga。 |
| `server.py` | 647 | JSON-RPC 2.0 over stdio 的最小 MCP 伺服器:連線範圍狀態、行內/併發分派、工具與 resource/prompt 處理器;握手時代的方法表(`_run_method`),兩個協定時代的逐請求分派在 `_stateless.py`。 |
| `server.py` | 650 | JSON-RPC 2.0 over stdio 的最小 MCP 伺服器:連線範圍狀態、行內/併發分派、工具與 resource/prompt 處理器;握手時代的方法表(`_run_method`),兩個協定時代的逐請求分派在 `_stateless.py`。 |
| `http_transport.py` | 656 | MCP 的 HTTP 傳輸;宣告 2026-07-28 的請求走 `_http_stateless.py` 的標頭規則,不發 session。 |
| `_http_stateless.py` | 185 | MCP 2026-07-28 在 Streamable HTTP 上的規則:`MCP-Protocol-Version`/`Mcp-Method`/`Mcp-Name` 必須與 body 相符(`=?base64?…?=` 先解碼),不符是 400+`HeaderMismatch`;版本與中繼資料錯誤 400、未知方法 404。純函式,由 `http_transport.py` 回覆。 |
| `http_sessions.py` | 290 | MCP 的 HTTP 傳輸用的 session 身分:`Mcp-Session-Id` 註冊表,以及每個 session 那條常駐的 server→client SSE 串流。 |
Expand Down Expand Up @@ -1102,7 +1102,7 @@ socket 預設綁 `127.0.0.1`;資源一律用 `with`。
| 層/子系統 | 檔案數 | 行數 |
| --- | ---: | ---: |
| `gui/` | 126 | 35,161 |
| `utils/mcp_server/` | 43 | 21,219 |
| `utils/mcp_server/` | 43 | 21,222 |
| `utils/remote_desktop/` | 57 | 14,178 |
| `utils/executor/` | 10 | 10,658 |
| `utils/usb/` | 20 | 5,057 |
Expand All @@ -1122,5 +1122,5 @@ socket 預設綁 `127.0.0.1`;資源一律用 `with`。
| `autocontrol-lsp/` | 8 | 744 |
| `utils/hotkey/` | 7 | 861 |
| 其餘模組(約 286 個 `utils/` 子套件 + `android/`/`ios/`/周邊小工具) | 724 | 68,874 |
| **總計** | **1,165** | **187,997** |
| **總計** | **1,165** | **188,000** |

10 changes: 10 additions & 0 deletions docs/updates/2026-10.md
Original file line number Diff line number Diff line change
Expand Up @@ -477,3 +477,13 @@ Jeffrey_RPA kept its own `KEY_ALIASES` (40 friendly names) and `_EXTRA_KEY_CODES
- **Suppressed, not changed** (each with `# NOSONAR <rule> # reason:` on its line): `dataclasses.replace` return types (S5886 / S5655, Sonar's stub is wrong); NaN guards written as `not x >= 0` (S1940); snapshots taken with `list()` before a loop that mutates (S7504); D-Bus and Qt method names (S100); `json_store._replace` (S8707: the source is its own `mkstemp` sibling); `history_store.list_runs` (Semgrep: only `?` placeholders are interpolated); sequential awaits in `web_viewer/index.html` (S9382).
- **Files**: 280 files; `architecture_explore.md` line counts re-measured with `test_doc_line_counts.py --fix`.
- **Open items**: whether the suppressions inside `web_viewer/index.html` and the two `nosemgrep` lines are honoured is only known after the next analysis. `Progress.md` › 雜項.

## U-20261009-43 · 2026-10-09 · The seven SonarCloud issues left on main after U-20261009-42 · #done #quality #sonar #security

- **What**: after #516 `main` still had 7 open issues and a failing gate (new security rating). Closes the `Progress.md` › 雜項 item on unconfirmed suppressions.
- **S2083 (blocker), `examples/18_slack_daily_report.py`**: the taint ran from the HTTP response into the HTML given to WeasyPrint, not into the output path that U-20261009-42 bounded. Messages and summary are escaped with `html.escape`; the PDF is rendered from the written file through a fetcher that serves only that file; the file is written through `open()`, because SonarCloud reported `Path.write_text(content)` as a path sink on the content.
- **S9382 ×4, `web_viewer/index.html`**: `// NOSONAR` is not honoured for script embedded in HTML. `pollOffer` makes one attempt per call, `sendFileChunks` sends chunks in order under the same 4 MiB limit, dropped files are chained. A channel that closes mid-transfer rejects the transfer.
- **S5778 ×2, `test_state_machine.py`**: the machine is built before the `pytest.raises` block.
- **Result**: SonarCloud's analysis of the pull request reports 0 open issues and a passing gate; Codacy had one left on `main`, pyflakes F401 on the `PROTOCOL_VERSION` re-export in `mcp_server/server.py` (it honours neither `# noqa` nor `X as X`); the module now names it in `__all__`. The two `nosemgrep` lines of U-20261009-42 are honoured.
- **Not run**: the viewer script (no Node on the machine that wrote it) and the WeasyPrint branch of the example (not installed).
- **Open items**: none.
1 change: 1 addition & 0 deletions docs/updates/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -58,6 +58,7 @@ In the same commit: delete the item from `Progress.md`, add a `#done` entry here

| ID | Date | Title | Tags | Batch |
|---|---|---|---|---|
| U-20261009-43 | 2026-10-09 | The seven SonarCloud issues left on main after U-20261009-42 | #done #quality #sonar #security | [2026-10](2026-10.md) |
| U-20261009-42 | 2026-10-09 | SonarCloud and Codacy backlog on main: 2,344 and 8 findings worked through | #done #quality #sonar #codacy | [2026-10](2026-10.md) |
| U-20261009-41 | 2026-10-09 | Release 1.0.0; the publish job releases a hand-set version as written | #release #ci | [2026-10](2026-10.md) |
| U-20261009-40 | 2026-10-09 | A result hook reports nested command results; secret-named result fields are masked at any depth and offered once | #done #executor #gui #security | [2026-10](2026-10.md) |
Expand Down
27 changes: 21 additions & 6 deletions examples/18_slack_daily_report.py
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,7 @@
"""
from __future__ import annotations

import html
import json
import os
import smtplib
Expand Down Expand Up @@ -176,11 +177,14 @@ def render_report(summary: str, raw_messages: Sequence[SlackMessage],
"""
output_dir.mkdir(parents=True, exist_ok=True)
today = datetime.now(tz=timezone.utc).date().isoformat()
# Slack messages and the model's summary are text from outside the
# program: escaped, they cannot add markup (an <img> or <link> that
# would make the PDF renderer open a local file or a URL).
rows = "".join(
f"<li><strong>{m.user}</strong>: {m.text}</li>"
f"<li><strong>{html.escape(m.user)}</strong>: {html.escape(m.text)}</li>"
for m in raw_messages
)
html = f"""<!doctype html>
page = f"""<!doctype html>
<html><head><meta charset="utf-8"><title>Daily Slack digest — {today}</title>
<style>
body {{ font-family: -apple-system, sans-serif; margin: 2em; }}
Expand All @@ -191,20 +195,31 @@ def render_report(summary: str, raw_messages: Sequence[SlackMessage],
<body>
<h1>Daily Slack digest — {today}</h1>
<h2>Summary</h2>
<pre>{summary}</pre>
<pre>{html.escape(summary)}</pre>
<h2>Raw messages ({len(raw_messages)})</h2>
<ul>{rows}</ul>
</body></html>
"""
html_path = _path_inside(output_dir, f"digest-{today}.html")
html_path.write_text(html, encoding="utf-8")
with open(html_path, "w", encoding="utf-8") as report:
report.write(page)
try:
from weasyprint import HTML
from weasyprint import HTML, default_url_fetcher
except ImportError:
print(" (no weasyprint — sending the HTML instead)")
return html_path
page_url = html_path.as_uri()

def only_the_page(url: str, *args: object, **kwargs: object) -> dict:
"""Let WeasyPrint read the page it renders and nothing else."""
if url != page_url:
raise ValueError(f"the digest loads no other resource: {url!r}")
return default_url_fetcher(url, *args, **kwargs)

pdf_path = _path_inside(output_dir, f"digest-{today}.pdf")
HTML(string=html).write_pdf(str(pdf_path))
# Rendered from the file just written, through a fetcher that serves only
# that file: the page needs no image, stylesheet or font from anywhere.
HTML(filename=str(html_path), url_fetcher=only_the_page).write_pdf(str(pdf_path))
return pdf_path


Expand Down
5 changes: 4 additions & 1 deletion je_auto_control/utils/mcp_server/server.py
Original file line number Diff line number Diff line change
Expand Up @@ -53,7 +53,7 @@
from je_auto_control.utils.mcp_server._stateless import StatelessDispatchMixin
from je_auto_control.utils.mcp_server._subscriptions import NO_RESPONSE, SubscriptionMixin
from je_auto_control.utils.mcp_server._protocol import (
PROTOCOL_VERSION as PROTOCOL_VERSION, # explicit re-export: callers import it from server
PROTOCOL_VERSION,
_capture_error_screenshot, negotiate_protocol_version,
_coerce_params, _DISPATCH_ERRORS, _error_response, _invalid_envelope, _InvalidToolArguments,
_is_hashable,
Expand All @@ -65,6 +65,9 @@
#: and shuts down anyway.
WORKER_DRAIN_TIMEOUT = 10.0

# PROTOCOL_VERSION is re-exported: callers import it from this module.
__all__ = ["MCPServer", "PROTOCOL_VERSION", "WORKER_DRAIN_TIMEOUT", "start_mcp_stdio_server"]


class MCPServer(ConnectionStateMixin, StatelessDispatchMixin, SubscriptionMixin,
ClientRequestMixin):
Expand Down
96 changes: 62 additions & 34 deletions je_auto_control/utils/remote_desktop/web_viewer/index.html
Original file line number Diff line number Diff line change
Expand Up @@ -585,20 +585,22 @@
return response.json();
}

// One attempt per call, then the next call a second later: each attempt
// waits for the one before it, until the deadline.
async function pollOffer(hostId, deadlineMs) {
while (Date.now() < deadlineMs) {
try {
const data = await signalingFetch(
`/sessions/${encodeURIComponent(hostId)}/offer`,
);
if (data && data.sdp) return data.sdp;
} catch (error) {
if (!String(error).includes("404")) throw error;
}
// NOSONAR javascript:S9382 - reason: polling, each attempt waits for the last
await new Promise(r => setTimeout(r, 1000)); // NOSONAR
if (Date.now() >= deadlineMs) {
throw new Error("timed out waiting for host offer");
}
try {
const data = await signalingFetch(
`/sessions/${encodeURIComponent(hostId)}/offer`,
);
if (data?.sdp) return data.sdp;
} catch (error) {
if (!String(error).includes("404")) throw error;
}
throw new Error("timed out waiting for host offer");
await new Promise(r => setTimeout(r, 1000));
return pollOffer(hostId, deadlineMs);
}

async function connect() {
Expand Down Expand Up @@ -931,6 +933,46 @@
els.mic.classList.remove("primary");
}

// Sends the file's bytes in order, one chunk at a time. Each step reads a
// chunk, waits (20 ms at a time) while the channel holds more than 4 MiB
// unsent, sends, then schedules the next step; the promise settles after the
// last chunk or on the first failure.
function sendFileChunks(file) {
const chunkSize = 16 * 1024;
const highWater = 4 * 1024 * 1024;
return new Promise((resolve, reject) => {
let offset = 0;
const sendWhenDrained = (buf) => {
// A channel closed mid-transfer throws here; that ends the transfer.
try {
if (filesChannel.bufferedAmount > highWater) {
setTimeout(() => sendWhenDrained(buf), 20);
return;
}
filesChannel.send(buf);
} catch (error) {
reject(error);
return;
}
offset += buf.byteLength;
if (offset % (256 * 1024) < chunkSize) {
setStatus(t("msg_sending",
{ name: file.name, sent: offset, total: file.size }));
}
step();
};
const step = () => {
if (offset >= file.size) {
resolve();
return;
}
file.slice(offset, offset + chunkSize).arrayBuffer()
.then(sendWhenDrained, reject);
};
step();
});
}

async function sendLocalFile(file) {
if (!filesChannel || filesChannel.readyState !== "open") {
setStatus(t("msg_files_channel_not_ready"), "#ff6666");
Expand All @@ -942,24 +984,8 @@
type: "file_begin", name: file.name,
size: file.size, transfer_id: transferId,
}));
const chunkSize = 16 * 1024;
let offset = 0;
setStatus(t("msg_sending", { name: file.name, sent: 0, total: file.size }));
while (offset < file.size) {
const slice = file.slice(offset, offset + chunkSize);
// NOSONAR javascript:S9382 - reason: chunks go out in file order, and the
// inner wait is backpressure on the one data channel; neither can be parallel.
const buf = await slice.arrayBuffer(); // NOSONAR
while (filesChannel.bufferedAmount > 4 * 1024 * 1024) {
await new Promise((r) => setTimeout(r, 20)); // NOSONAR
}
filesChannel.send(buf);
offset += buf.byteLength;
if (offset % (256 * 1024) < chunkSize) {
setStatus(t("msg_sending",
{ name: file.name, sent: offset, total: file.size }));
}
}
await sendFileChunks(file);
filesChannel.send(JSON.stringify({
type: "file_end", transfer_id: transferId,
}));
Expand Down Expand Up @@ -1207,13 +1233,15 @@
filesPanel.classList.remove("dragover");
if (!filesChannel || filesChannel.readyState !== "open") return;
const droppedFiles = Array.from(e.dataTransfer.files || []);
for (const f of droppedFiles) {
// NOSONAR javascript:S9382 - reason: one transfer at a time on the files channel
try { await sendLocalFile(f); } catch (err) { // NOSONAR
// One transfer at a time on the files channel: each starts when the one
// before it has finished or failed.
await droppedFiles.reduce(
(earlier, f) => earlier.then(() => sendLocalFile(f)).catch((err) => {
setStatus(t("msg_upload_failed",
{ name: f.name, error: String(err) }), "#ff6666");
}
}
}),
Promise.resolve(),
);
setTimeout(refreshFiles, 300);
});

Expand Down
20 changes: 17 additions & 3 deletions test/unit_test/headless/test_rest_read_data_routes.py
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@
import ast
import inspect
import json
import time
import re
import textwrap
import urllib.error
Expand Down Expand Up @@ -142,6 +143,21 @@ def _get(server, path: str, token: str):
return error.code, json.loads(error.read().decode("utf-8"))


def _wait_for_refusal(server, timeout_s: float = 5.0) -> bool:
"""Whether the audit log gets a READ_DATA refusal within ``timeout_s``.

The handler thread writes the row after it has sent the reply, so the
client can be back here first; that lost race failed this test on CI.
"""
deadline = time.monotonic() + timeout_s
wanted = f"forbidden:{Capability.READ_DATA}"
while time.monotonic() < deadline:
if any(wanted in json.dumps(row) for row in list(server._audit_log.rows)):
return True
time.sleep(0.01)
return False


# --- what a role may read -----------------------------------------------------

@pytest.mark.parametrize("path", ["/history", "/usb/acl"])
Expand All @@ -152,9 +168,7 @@ def test_a_viewer_is_refused_a_stored_record_route(rest, users, path):
assert body == {"error": "forbidden", "role": Role.VIEWER,
"required_capability": Capability.READ_DATA}
assert server.served == [], "the handler must not run for a refused caller"
refusals = [row for row in server._audit_log.rows
if f"forbidden:{Capability.READ_DATA}" in json.dumps(row)]
assert refusals, "the refusal is in the audit log with the capability it lacked"
assert _wait_for_refusal(server), "the refusal is in the audit log with the capability it lacked"


@pytest.mark.parametrize("role", [Role.OPERATOR, Role.ADMIN])
Expand Down
Loading
Loading