Repository navigation
Conversation
Require the flag, hypothesis, control, randomization unit, and primary metric from the user, and wait for an explicit yes before creating a metric or experiment or starting an iteration. Co-authored-by: Cursor <cursoragent@cursor.com>
Starting an experiment must not silently turn the flag on and begin serving variations. Co-authored-by: Cursor <cursoragent@cursor.com>
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using default effort and found 2 potential issues.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 02afad3. Configure here.
Skill eval results
Only suites whose source actually changed since their last recorded score were re-run. Soft-failing while we stabilise the baseline. |
A blank measure stays blank until the user names it, and that reply only restates the completed design for a separate approval. Co-authored-by: Cursor <cursoragent@cursor.com>
…s known A missing metric name is not an event key. Wait until the user says the metric is custom and names the key before calling list-metric-events. Co-authored-by: Cursor <cursoragent@cursor.com>
Fold the measure and event-key fixes into the existing 0.3.0 changelog entry instead of bumping the version. Co-authored-by: Cursor <cursoragent@cursor.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Updates
launchdarkly-experiment-setupso the agent never creates, starts, orchanges anything without the user's approval. Previously the skill went straight
from creating an experiment to starting it, and didn't ask the user for the design.
primary metric from the user; asks for any that are missing in one message, and
never fills them in itself.
list-metric-eventsthat the event isreaching LaunchDarkly before creating it.
waits for an explicit yes before
create-metricorcreate-experiment.save-and-start-experiment-iteration) only afterthe user explicitly says to start.
turning it on serves its variations to users, and asks who should turn it on.
skills.jsonand the CHANGELOG.Testing
Automated:
scripts/validate_skills.py,python3 -m unittest discover -s tests,and
scripts/generate_catalog.py --checkall pass, run locally with Python 3.12(the system Python 3.9 can't run these scripts).
Live, against a demo project (test environment) with a local Node.js app, a
string flag, and a custom event:
created the metric and experiment without starting; stopped correctly with the
control as winner.
the flag on by itself when told to start. After the fix, it changes nothing, explains
the impact, asks who should turn it on, re-checks the flag version once it's on, and
confirms before starting.
metric in one message, offered examples without choosing, and created nothing.
reaching LaunchDarkly and created nothing.
iteration, asked which variation gets 70%, and waited for an explicit start.
Retested after review fixes: a complete design still goes straight to approval with no extra questions; a blank measure blocks creation until it’s filled in and the full design is re-approved; the event check waits until the user names the metric kind and event key; incomplete-design and missing-event tests behave as before.
Notes
docs/skills.mdanddocs/versioning.mdare referenced in CONTRIBUTING butmissing, so this follows the conventions of
launchdarkly-metric-create.skills, but no experiment skills. Should experiment skills be added to the plugin?
Note
Overview
Tightens
launchdarkly-experiment-setup(0.3.0) so the agent cannot create metrics/experiments, start iterations, or change flags without explicit user consent and a complete design.The skill now requires user input for flag/variations, hypothesis, control, randomization unit, and primary metric (never inferred). For new metrics, it defers
list-metric-eventsuntil the user confirms a custom event and names the event key (not the metric name); creation is blocked if the event is not flowing. Design review must show defaults and metric measurement; generic approvals like "yes" or "create it" do not fill a blank measurement type—only a named measure, then a second approval, unlockscreate-metric/create-experiment. Starting (start-experiment-iteration,save-and-start-experiment-iteration) waits for an explicit go; if the flag is off, the agent explains impact and asks who should enable it instead of toggling the flag itself.skills.jsonand CHANGELOG are updated for 0.3.0.Reviewed by Cursor Bugbot for commit 3eaf2e3. Bugbot is set up for automated code reviews on this repo. Configure here.