Skip to content

Add clip editing (split, delete, reorder, trim, per-clip speed) + fix recording stop race - #2

Merged
pranavms13 merged 3 commits into
Glyph-Software:mainfrom
AleksDoesCode:feat/clip-editing
Oct 7, 2026
Merged

pranavms13 merged 3 commits into
Glyph-Software:mainfrom
AleksDoesCode:feat/clip-editing

Conversation

@AleksDoesCode

@AleksDoesCode AleksDoesCode commented Sep 24, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Right now the editor can only trim the start and end of a recording. This PR adds real clip editing, so you can cut a recording the way you would in a video editor:

  • Split at the playhead (S or the scissors button)
  • Delete a clip (⌫). The gap closes automatically.
  • Reorder clips by dragging them. An insertion marker shows where the clip will land.
  • Trim a clip from either edge. The preview shows the exact frame at the cut, and you can drag the edge back out to recover what you trimmed.
  • Per-clip speed from 0.25× to 16×, with presets (0.5, 1, 1.5, 2, 4, 8, 10, 16) or a custom value. For example, you can cut out a long "waiting for the build" section and play it at 10× as a timelapse.
  • Duplicate and mute individual clips
  • Every operation can be undone with ⌘Z (it goes through the existing history).

It also fixes a bug where stopping a recording failed.

Screenshots

A clip selected and set to 10×. Sped-up clips are hatched on the timeline, and the inspector shows the clip panel.
10x clip

Dragging a clip to reorder it
Reorder

Trimming a clip edge. The preview shows the frame at the cut.
Trim

Light theme at 1100×720
Light theme

How it works

  • src/lib/clips.ts (new). This is a pure, UI-free model. EditorState.clips is an ordered list of { srcStart, srcEnd, speed, muted }. An empty list means the whole recording. Helpers lay the clips out on the output timeline and map between output time and source time (layoutClips, outToSrc, srcToOut, srcRangePieces).
  • Timeline, playhead and export use output time. Zoom and effect segments, camera keyframes, audio-track trims and the cursor sidecar stay in source time and are mapped per clip for display. So when you move or speed up a clip, its zooms and effects move and speed up with it, and none of the existing data formats change.
  • Preview. A rAF loop walks through the clips. At each clip boundary it seeks the <video> and sets playbackRate to the clip's speed. The WebAudio tracks and the camera video follow the same rate. Muted clips silence all audio.
  • Export. The exporter renders output frames and maps each one to its source frame. For audio it emits one AudioTrackSpec per clip and track, with a new optional tempo field. On the Rust side this becomes an ffmpeg atempo chain, split into ×0.5 steps for speeds below 0.5, so exported audio keeps its pitch.
  • Projects. PROJECT_VERSION goes from 3 to 4. Older projects still open: their trimStart/trimEnd is converted into a single clip on load.
  • AGENTS.md has a short section describing the clip model.

Bug fix: stopping a recording (capture.rs)

After stop_capture() the code waited a fixed 400 ms and then moved the segment file into place. SCRecordingOutput finishes writing the file asynchronously, so on slower machines or debug builds the file didn't exist yet. Stopping then failed with Failed to move segment to final output: os error 2, and the recording was left behind as a hidden .oss-*-seg0.mp4. The fix waits for the recording delegate's finish/fail result via a channel instead. A failed stop_capture() skips the wait; delegate failures and timeouts are returned as errors. Final stop/discard uses a 15 s cap, while pause uses 2 s. Pause already runs as a background command; cancel/restart now do too, and restart releases the state mutex before discarding the old recording. Pause still serializes access to recording state during its bounded wait. This is in its own commit, so it can be cherry-picked separately if you prefer.

Review fixes

  • Preview audio uses the original recording duration for source-time trim checks, so sped-up clips retain mic/system audio throughout their source range.
  • Moving a zoom/effect piece shifts the complete source segment by the speed-adjusted delta, preserving its length and the portion across a cut. Edge trims update only the dragged edge and map through the owning clip with outToSrc. Dragging also keeps the piece length captured at drag start.
  • Recording finalization uses a result channel, skips waiting after stop errors, and bounds pause waits to 2 s. Cancel/restart run in the background and discard without holding the recording-state lock.

Testing

  • bunx tsc --noEmit, bun run build and cargo check all pass.
  • bun test tests/clips.test.ts: 6 committed tests (19 assertions), covering split, speed layout, time mapping, move/remove/duplicate, and zoom/effect piece edits across cuts, reordered clips and speed changes.
  • The real editor UI was driven in a browser with the Tauri IPC mocked and public/sample.mp4 loaded. Checked there: split, 10×, drag-reorder, delete, undo, and edge trim. Playback correctly switches between 1× and 10× and back, continues across reordered clips (for example, source 12.0 s to 0.0 s), and stops at the end.
  • I ran the atempo filter graph against the bundled ffmpeg: 5 s at 1× plus 20 s at 10× produces 7.0 s of audio, and the ×0.25 chain also works.
  • The capture fix was verified with a real recording (screen, mic, system audio and camera). Stopping now saves every file.
  • Not yet tested: a full MP4 export of a cut and sped-up project inside the app. The October review fixes passed the automated checks above; the earlier manual playback/capture checks have not been repeated for these changes.

Known limitations

  • In the preview, audio at speeds other than 1× changes pitch, because it uses AudioBufferSourceNode.playbackRate. The export keeps the pitch.
  • Importing external video files is out of scope. You can only arrange material from the recording itself.

🤖 Generated with Claude Code

AleksDoesCode and others added 2 commits September 24, 2026 11:11
The fixed 400 ms sleep after stop_capture raced the asynchronous file
finalisation; on slower (debug) builds the segment did not exist yet and
stop failed with "Failed to move segment to final output: os error 2".
Wait on the recording delegate's finish/fail callback instead (15 s cap).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The editor now edits an ordered clip sequence (src/lib/clips.ts) instead
of a single head/tail trim. Clips can be split at the playhead (S),
deleted, dragged to reorder, trimmed from either edge, duplicated, muted
and sped up or slowed down (0.25x-16x) for timelapses.

Zoom/effect segments, camera keyframes, audio trims and the cursor
sidecar stay in source time and are mapped per clip. Preview walks the
clips with per-clip playbackRate; export renders output frames and mixes
one audio piece per clip and track with an ffmpeg atempo chain.
Pre-v4 projects convert their trim into a single clip on load.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@AleksDoesCode

Copy link
Copy Markdown
Contributor Author

@pranavms13 please have a look at this, thanks!

@pranavms13 pranavms13 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@AleksDoesCode thanks for this, it's a big one and the overall shape is good. The source-vs-output time split is applied consistently and the atempo chain looks right. I went through it fairly carefully and left three inline comments on things I think need fixing before this goes in. Also, the description mentions bun test coverage for clips.ts but I don't see a test file in the branch, could you push those too?

const rate = clip?.speed ?? 1;
for (const n of nodes) {
const ts = tracks[n.key];
const inWindow = t >= ts.trimStart - 1e-3 && t <= dur - ts.trimEnd + 1e-3;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this one's a real bug. dur here is audioSyncRef.current.duration, which is now the edited output duration, but t is v.currentTime, i.e. source time. As soon as any clip is faster than 1x the output is shorter than the source, so inWindow flips to false partway through and the mic/system audio just drops out in the preview.

Easy repro: 60s recording, set the single clip to 10x, play from the start. Audio dies after ~6s of source and stays dead for the remaining 54s, even though the export still has it.

Should just be srcDuration going into the ref (around line 4564) instead of duration.

Comment thread src-tauri/src/capture.rs Outdated
// give it a beat so the segment file is playable when concatenated.
std::thread::sleep(Duration::from_millis(400));
// SCRecordingOutput writes the file asynchronously after stop; wait for its finish callback.
let deadline = std::time::Instant::now() + Duration::from_secs(15);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice catch on the race, waiting on the delegate is definitely the right call. One concern with the loop though: it runs with the RecordingState mutex held, and stop_active_segment is also reached from the sync pause_capture / cancel_capture / restart_capture commands (via discard_recording), not only the async stop path.

If the finish/fail callback never comes, e.g. stop_capture() itself returned an error or the stream is already dead, pause/cancel will freeze the main thread for the full 15s, and anything else that takes the lock (status polls from the HUD) stalls with it. Before this change the worst case was 400ms.

Could we skip the wait entirely when stop_capture failed, and maybe use a channel/condvar with recv_timeout rather than sleeping in 50ms steps? A shorter cap on the sync paths would help too.

Comment thread src/components/Editor/index.tsx Outdated

// Zoom/effect segments live in source time; show them where their source
// range lands in the edited sequence and map chip edits back to source.
const srcMsFromOut = (c: PlacedClip, outMs: number) =>

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This bit worries me for zoom/effect segments that span a cut. The chips are rendered per piece with seg rebased to the piece's output bounds, and mapChipPatch pushes both startMs and endMs back through this one clip, so moving a piece rewrites the whole segment to the piece's length.

Repro: make a 3-8s zoom, split at 5s, nudge the left chip. The zoom becomes a 2s segment and the 5-8s part is gone.

Related: srcMsFromOut clamps to the whole recording rather than the clip, so dragging a piece's edge past the clip boundary after a reorder maps to source time on the wrong clip. I'd use the clip-clamped outToSrc from clips.ts here and only patch the edge that was actually dragged (for a move, shift start/end by the delta instead of writing absolute values).

@AleksDoesCode

Copy link
Copy Markdown
Contributor Author

@pranavms13 thanks for the careful review — all three points were valid. I've pushed the fixes in 7dd2860 and updated the PR description.

  • Preview audio: audioSyncRef.duration now receives srcDuration in both initialization and updates, so source-time trim checks no longer cut audio off at the shorter output duration.
  • Recording finalization: replaced the atomic flag/polling loop with a channel carrying the delegate's success/error result and recv_timeout. A failed stream stop skips the wait entirely, and delegate failures/timeouts now return an explicit error. Pause already had #[tauri::command(async)] in this branch; cancel/restart now use that too. Restart drops the state guard before discarding the old recording (cancel already did). Pause still holds the lock to serialize recording transitions, but its wait is capped at 2 seconds; the final stop/discard paths retain 15 seconds outside the state lock.
  • Zoom/effect pieces across cuts: moving a visible piece shifts the entire underlying source segment by the speed-adjusted delta, preserving its length and the part beyond the cut. Edge drags update only that edge, using clip-clamped outToSrc, including reordered clips. I also fixed the drag's length calculation to use the piece length captured at mouse-down; it previously subtracted the grab offset from the end time.

The missing tests are now committed in tests/clips.test.ts: 6 tests / 19 assertions covering splitting, speed/time mapping, move/remove/duplicate, and the piece-edit regressions across cuts, reordering, speed changes and recording boundaries. bun test tests/clips.test.ts, bunx tsc --noEmit, bun run build, and cargo check pass.

The earlier manual playback/capture checks in the description have not been repeated for this revision, and full in-app export remains unverified. Could you take another look when you have time?

@pranavms13 pranavms13 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@pranavms13
pranavms13 merged commit 41cd8cd into Glyph-Software:main Oct 7, 2026
1 check failed
pranavms13 added a commit to AleksDoesCode/OpenScreenStudio that referenced this pull request Oct 8, 2026
Resolve conflicts with the Glyph-Software#2 follow-up fixes on main:
- capture.rs: take main's cancel/restart handling (#[tauri::command(async)]
  plus skipping the finish wait when stop_capture fails), which covers this
  branch's spawn_blocking version.
- Editor: keep this branch's segment chip mapping (segmentChip.ts, grabOffset,
  boundary-locked resize handles) over main's mapSegmentPiecePatch/lengthMs.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants