Repository navigation
Conversation
#262 A second run of review-pr-comments-converge re-validated every comment on the PR and drafted a reply to each, including feedback an earlier run had already answered. The fetch output did not show the PR author, edit times or who resolved a thread, so a run could not tell. fetch_pr_comments.py now asks for the viewer, the PR author, who resolved each thread, and each post's state and times in the same request. Before printing, it gives every review body, inline thread and conversation comment one verdict: answered with its evidence, not answered, or not feedback because only the PR author or the gh login wrote it. It prints a "# Answerers:" line and one "answered" line per item, and exits 1 with a sentence when a time, a comment state or the viewer cannot be read. Times compare as datetimes, a same-second tie counts as new feedback, and a deleted account never answers and no longer crashes the REPLIES lines. SKILL.md puts every item not marked "no" on the decided list before validation, lists it under "Already answered" with its evidence, and reports "Nothing unanswered" when nothing is left. The check before posting compares fresh and saved answered lines, so a skip is read from the PR each time and never carried over.
#262 Mutation testing of fetch_pr_comments.py left 20 changes uncaught. The fake gh returned its fixture whatever the query asked for, so dropping a field from QUERY changed nothing in the tests. It now returns only the fields the query selects, so a field the script reads without requesting fails the suite. New examples pin the rest: a reply's posted time rather than its edit time, every other comment in a thread checked against the latest reply, reviews with no submit time, a bot sharing the viewer's login, a review answering from when it was submitted, pending reviews that carry a submit time, the earliest answer chosen by posted time, the isResolved flag, per-list request flags on later pages, and an empty viewer login or non-string and null times exiting 1.
#262 Phase 2 said the list of skipped items "is rebuilt from the saved fetch at the start of each pass". Fresh agents replaying Phase 4's check read that as the whole decided list, and one re-validated items the first pass had already fixed, which would turn an approved reply into a "stale" one. Only the Already answered entries are rebuilt now. An item an earlier pass decided stays decided, and its draft goes into the new report for fresh approval unless its answered line changed to yes. The report section also says it lists the author's own items as well as answered ones.
#262 A wrap-up read of the branch found claims the change had made untrue or left unclear. Phase 2 kept every item an earlier pass decided, so a reviewer's follow-up in a thread the run had already fixed would never be read, though Phase 4 restarts a pass for exactly that case. An item Phase 4's check lists, with its line still "no", now comes off the decided list and is validated again with the new feedback. Step A reads the saved fetch rather than Phase 1's, "Nothing unanswered" applies only when no earlier draft carries over, and the check before posting names the edit times the fetch actually prints. The answered wording in SKILL.md and the script's docstring now says a resolved thread counts whoever resolved it. review-code says its own reconciliation ignores the answered lines, which are judged from the PR author's side. The README mentions the skip, the test file's run command adds hypothesis, and the print path reads a review body the same null-safe way the judging step does.
#262 A review body or conversation comment counted as answered by any later top-level post from the PR author or the gh login. On a real PR one unrelated closing remark from the author marked nine earlier review bodies and comments as answered, so a second run would have skipped feedback nobody had answered. An answer now has to address the item's author too: it opens with the item's thank line, or names the author the way refer_as does, @login with a boundary for a person or the BOT_NAMES name as a whole word for a bot, in any case. The match is decided from the Reviewer variant, so a deleted or unrecognised author can never be addressed and no top-level post answers their items. A post that names nobody answers nothing, and the run drafts a reply the author can decline instead of skipping silently. Inline threads are unchanged, since GitHub links each reply to its thread.
#262 Mutation testing of the addressing rule left two regex changes uncaught: dropping the @ so a bare login addressed its author, and requiring a boundary before the @. New cases pin both: "Thanks dana" and a login inside another word do not address dana, while "cc foo@dana" does, since the rule says @login anywhere.
#262 addresses() accepted a post that opened with the item's thank line, or one that named the author. Every thank line for an author who can be addressed already holds their @login or bot name, so the mention check always matched first and the opening check could never change a verdict. Mutation testing showed four changes to it as equivalent. addresses() is now only the mention check, and the docstrings and SKILL.md say an answer names the author the way refer_as does, as a reply opening with the thank line does.
#262 A person counted as addressed by "@login" anywhere in a post, so "foo@dana" or an email address such as "x@dana.io" marked dana's feedback answered, though GitHub renders neither as a mention. A false match skips feedback silently, so the @ must now not follow a word character, a dot or a hyphen. A wrap-up read of the branch also found wording the addressing rule had made untrue. The docstrings and SKILL.md now say one post that names an author answers all of that author's earlier review bodies and conversation comments, the Answer docstring says answers are filtered to answerers after they are built, and the check before posting says an edit to a thread comment or review body is seen only when it turns an answered item back to "no". The README and review-code name both answerers, and the tests' oracle, comments and names describe the mention rule rather than the removed thank-line check.
#262 A bot counted as addressed wherever its BOT_NAMES name appeared as a \b word, which includes file paths. A closing remark such as "updated .github/copilot-instructions.md" would have marked every earlier Copilot review body answered, so a run would skip them silently. The name must now stand on its own: not after a word character, a dot, a slash or a hyphen, and not before a word character, a slash, a hyphen or a dot that starts a file extension. "Thanks, Copilot." still counts. The docstrings and SKILL.md say the same.
#262 Mutation testing of the mention rules left five boundary changes uncaught. New cases pin each: "@dana_" still mentions dana, and a bot name after a dot or a hyphen, before an underscore, or before a dot and a digit ("settings.copilot", "pre-copilot", "copilot_instructions", "Copilot.2") does not address the bot.
#262 One line of fetch_pr_comments.py's module docstring ran to 139 columns after the addressing wording was added. It is rewrapped with no change to its words.
#262 The review of the draft PR found four ways the fetch could still mark feedback answered when nobody had answered it. A thread comment written inside a pending review carries the time it was drafted, so a follow-up drafted before the author's reply but published after it looked older than the reply. Thread comments are now timed by publishedAt. A resolved thread hid any later follow-up, and the run resolves every thread it replies to, so a resolution now stands only when no reviewer posted or edited at or after the latest answerer reply. A mention now has to be one GitHub links: not before an underscore, and never inside inline code or a fenced block, for people and bots alike. Reply bodies are indented on every line, so a commenter cannot print a line that reads as the script's own answered field. Every match over a union now ends in assert_never, and new tests pin a review submitted after it was started, a deleted account's reply in an orphan thread, and a PR opened by a bot.
#262 The review of the draft PR found places where the skill left a run stuck or unclear. A failed fetch now stops the run, since an empty fetch would read as nothing unanswered. A run with nothing to validate or post says so and tells the author to close the tab, because an open tab keeps monitor-prs from preparing the PR again. The check before posting starts a new pass for new feedback or a line moving to or from "no", not for one "yes" giving way to another. A skip the author overrules stays overruled for the rest of the run, and a failed post lists every item whose own reply did not go out, since a sibling reply naming the same reviewer can make a later fetch mark it answered. The report section is now Skipped, since it also lists the author's own items, the rule text matches the script's docstring, proposing a BOT_NAMES entry says the name decides what counts as answered, and the description, README and review-code describe the rule as it is.
#262 The rule that a mention inside code addresses nobody covered only single-backtick spans and triple-backtick fences. A double-backtick span, a tilde fence, a four-backtick fence holding a three-backtick block, an indented code block and a span that wraps a line all still counted, though GitHub links none of them. outside_code now follows CommonMark's code forms with the standard library: fences of three or more backticks or tildes closed by a matching fence, indented blocks after a blank line, and spans closed by a backtick run of the same length within a paragraph. Blanked text becomes whitespace, so text on either side of a span never joins. New tests also pin that a resolution is measured from when the latest answerer reply was posted, which mutation testing found unguarded.
#262 The hand-written code detection kept missing CommonMark's rules: mutation testing left 21 of 30 changes to it uncaught, and it still counted a mention in an indented code block after a fence or a heading, while blanking a list item's continuation and text after an escaped backtick. outside_code now parses the body with markdown-it-py's CommonMark preset, the same library and version range the repo's tests already install, and keeps only plain text: fenced and indented code blocks drop out and inline code becomes a space, so text on either side of it never joins. The script declares the dependency inline, so uv installs it on the first run. New cases pin each code form against what CommonMark gives.
#262 A review body printed under OVERVIEW at a two-space indent, the same as the script's own "answered" field, so a reviewer could write a line into their review that reads as the item's verdict. Review bodies now print at four spaces like every other body, and a test checks that no printed body can pass for an answered or Answerers line.
Automated reviewAn independent review of this PR, run before any fixes, given only the PR. Review report (Request Changes, 19 findings)Code ReviewVerdict: Request ChangesFix the three ways a run can still skip feedback nobody answered (findings 1 to 3). Those are the failure this PR exists to prevent, and the PR body itself argues that a wrong skip is the one nobody notices. Every agent finished. Findings
1.
2.
3. For a bot, 4. The docstrings and SKILL.md:92 say the rule is "
I checked both. The head's 5. "When nothing is left" sends the run to Phase 3, which says to stop and wait for approval to post, though nothing will be posted. The only close-the-tab instruction (SKILL.md:582) runs "once every reply is posted", which never happens here. Under decision 010, an open tab is the record that a PR is prepared. So this tab blocks 6.
7. The build-time design change counts a review's submit time in 8. The README says feedback is skipped when "answered by a reply addressed to its reviewer". An inline thread is also skipped when anyone resolves it, or when an answerer's reply is its newest comment, and neither needs to name the reviewer ( 9. Phase 1 says to run the fetch and save its output, but not what to do when it exits 1. A run that saves empty stdout now gets a clean-looking 10. Open PR #293 conflicts with this one in both shared files. Agent 16 merged the two heads in a scratch worktree:
The merge results are Agent 16's. I did not rerun them. [introduced] 11. The check lists "every item whose line changed". That includes 12. "Take it off the Already answered list and validate it in a new round" conflicts with three other rules:
The report should say which rule wins in each case. I read SKILL.md:200-216 and :589. [introduced] 13.
14.
15. One post naming a reviewer answers all of that reviewer's earlier items. So if Phase 4 posts the reply to alice's review body and the post for her conversation comment then fails, the rerun prints 16.
17. The frontmatter still says the skill fetches comments and validates "each against the actual code", but it now skips answered items. The description is what skill listings show. I read line 3. [pre-existing] line, made inaccurate by this change. 18. All of these are [introduced]. I confirmed each of these lines exists. The rest of the evidence is Agent 18's:
19. The rule is written out almost word for word in the script docstring ( Agents
Looks GoodJudging every item before the first print is a good choice: an unreadable time or state exits 1 with every reason and no half output. The fake These areas came back clean:
Dropped
(Stopped before Step 5. This was a read-only, unattended run, so nothing was posted to GitHub, and the question about which findings to fix locally was not asked.) What fixed each finding
|
#262 Replies were split with str.splitlines, which also breaks at U+2028, U+2029, U+0085, \v and \f, so a reply holding one of them printed as two lines where main printed one. Replies now split only at \r\n, \n and \r, so a Windows line ending still prints no carriage return and every other character prints as written. Tests pin both.
#262 A reply printed U+2028, U+2029, U+0085, \v and \f raw, so a reader that splits on them, as str.splitlines does, could find a line the reply started, such as a forged answered field. Replies now print each as a visible escape, while \r\n, \n and \r still split.
…olution #262 A resolved inline thread counted as answered whoever resolved it. A thread Copilot resolved itself, with no reply from the author, was skipped, and nobody ever answered it: the silent skip the change exists to prevent. Counting an unanswered thread as answered drops feedback with nothing to show it, while the opposite mistake only drafts a reply the author reads and can decline. thread_verdict now counts a resolution only when the one person, other than the answerers, who commented in the thread resolved it, and nothing they wrote came after the latest answerer reply. A bot, a deleted account, someone else or an answerer resolving it leaves the thread judged as if unresolved, so an answerer's newest reply still answers it. Resolved.by narrows to Person. A property test checks that any other resolver gives the same verdict as no resolver.
💪 What
review-pr-comments-convergeon a PR now skips feedback the PR author already answered, and its report lists every skipped item with the post that counted as the answer.fetch_pr_comments.pyprints one newansweredline on every review body, inline thread and conversation comment (yeswith its evidence,no, orn/a (yours)), plus a# Answerers:line naming the PR author and the gh login running it. Every line it printed before is still printed, some indented deeper, soreview-code, which reads the same output, sees what it saw before.@loginwhere GitHub would link it as a mention, or a bot's name standing on its own (Thanks, Copilot.counts,.github/copilot-instructions.mddoes not). Names inside code don't count, so the fetch now parses posts withmarkdown-it-py, declared inline souvinstalls it on first run. A post that names nobody, such as "Rebased on main", answers nothing.answeredline with the one it saved. An item that got answered in the meantime has its draft dropped, and an item that got new feedback is validated again. Nothing about what was answered is stored between runs.answeredverdict. In a reply, the Unicode line separators U+2028, U+2029, U+0085,\vand\fprint as escapes such as\u2028, so no reader can split one into a line of its own.🤔 Why
monitor-prsstarts a fresh run whenever new feedback lands, and each run used to re-validate and draft a reply to every comment on the PR, so feedback that already had an answer got a second one.monitor-prsstill has its own, looser idea of answered feedback, which the ticket leaves for later.👩🔬 How to validate
🛠️ Setup
💻 A checkout of this branch, with
uvand a signed-ingh(every claim)Check out this branch from a clone of
ooloth/dotfilesand stay at the repo root for every command below:📄 A closing remark that names no reviewer answers nothing
🔍 Verification steps
Fetch michaeluloth.com#83, which has a closing remark from the author after several bot reviews:
Expect
# Answerers: ooloth (PR author and gh login)first, thenanswered : noon every review body and bot comment (the remark names no reviewer),answered : noon the one resolved inline thread too, because Copilot resolved it and nobody replied, andanswered : n/a (yours)on the remark itself👀 What I saw
Every review body, bot comment and inline thread says
no, including inline threadPRRT_kwDOQQJOc86D-GpE, which Copilot resolved, and the last line is the closing remark (run at c89d8ff, output trimmed where 24 morenolines repeat):Before the rule that an answer must name the reviewer, the same fetch printed this for the 9 items that came before the remark:
📄 An answer that names the reviewer counts as answered
🔍 Verification steps
Fetch a PR where the author answers by name:
Expect lines such as
answered : yes (conversation comment 4068918466 by EliteTK at 2026-03-16T16:19:47Z), where that comment opens with@Da1sypetals👀 What I saw
Two items are answered by EliteTK's comment 4068918466, and another by a later comment:
📄 Apart from indentation, a trailing carriage return and Unicode line separators shown as escapes, every line main printed is still printed
🔍 Verification steps
Compare against
main's output for the same PR:Expect no output: apart from indentation, a trailing carriage return and Unicode line separators shown as escapes, no line
mainprinted has been removed or changed👀 What I saw
The step printed nothing, and
grepexited 1. Withouttr -d '\r', the same diff prints 78 lines. Each is a line from a reply written with Windows line endings, whichmainprints with a trailing carriage return (^M) and this branch prints without one. The first of those lines, cut short:An earlier check ran the same fetch with
main's script and this branch's on 20 public PRs: both made the same GitHub requests, and no linemainprinted was lost or changed apart from review bodies and multi-line replies now being indented deeper.📄 The skill validates only unanswered items and lists the rest as skipped
🔍 Verification steps
main's skill.👀 What I saw
Fresh agents given the edited skill and saved fetch output sent only the
noitems to validation, listed the rest under ⏭️ Skipped, and handled the before-posting check as described, while agents givenmain's skill did not.🧹 Clean up
Remove the copy of
main's script:🔗 Related links
Fixes #262