Repository navigation
tecs.audio.music: Music (MIDI) and Music (FM) asset kinds and players - #13
Closed
albanmgicquel wants to merge 7 commits into
Closed
albanmgicquel wants to merge 7 commits into
albanmgicquel wants to merge 7 commits into
Conversation
rustysynth (MIT) renders a SoundFont in the output callback as the input of a voice started by a PLAY carrying the new SYNTH flag. Events cross from the game thread through a bounded lock-free SPSC ring (4096) into a preallocated schedule (4096); a send that finds no room is refused and counted. The render splits at rustysynth's 8-frame block boundaries and applies each event at the first boundary at or after its frame, so the output is a deterministic function of the schedule. The clock is frames rendered over the rate. FFI: tecs_audio_load_soundfont, tecs_audio_release_soundfont, tecs_audio_synth_open/send/notes_off/reset/now/close. Tests cover the grid, ordering, reset discarding, queue bounds, and with TECS_TEST_SOUNDFONT a real SoundFont: sample-exact landing, determinism, voice gain, cuts and pause.
loadSoundFont(audio, path) and new(audio, font, {group, gain, polyphony,
effects}) with send, noteOn, noteOff, controlChange, programChange,
pitchBend, allNotesOff, reset, now, setGain and destroy. Sending methods
return whether the event was queued. The synth takes its voice slot the way
play does, through an internal per-mixer lender, so the voice ceiling, group
gain, mute and pause apply unchanged. The backend contract gains SynthPort and
optional loadSoundFont, releaseSoundFont and openSynth; a backend without them
and a mixer without an output get a silent port.
tecs.audio moves to audio/init.nupp for its child. README records why
synthesis is in-process and where a hardware MIDI output would plug in.
src/fm drives a YM3812 or YMF262 from ymfm-oxide, a pure-Rust, no_std port of ymfm (BSD-3-Clause), taken as a git dependency pinned to commit 4cc9e10. Tecs keeps no emulator source: the crate carries the port and its check against ymfm. A register 0x000-0x1FF maps onto the chip's address and data ports, offsets 0/1 below 0x100 and 2/3 for the OPL3's second bank; the OPL3 plays outputs A and B as left and right, the OPL2 its one output on both. The chip renders at 49715 Hz, the rate the emulator gives a 3.579545 MHz OPL2 and a 14.31818 MHz OPL3, and the voice resamples it. The chip goes through the synth's ring, schedule and source. The block grid is now per Target: eight frames for rustysynth, one for the OPL, so a write lands on the first chip sample at or after its time, and writes due on one sample all apply before it is generated. Port gains an all-or-nothing batch push, and a reset carries a frame: events sent before it and due before it still apply, those due at or after it are discarded (the synth's reset, at the next block, behaves as before). FFI: tecs_audio_fm_open/write/writes/reset/now/close and a RegisterWrite layout. THIRD_PARTY_NOTICES records ymfm-oxide and carries its licence. Tests: the grid, ordering, the second bank and the NEW bit, a key-off and key-on on one sample, queue bounds, batch atomicity, reset, the rate against the emulator's, and the engine path.
new(audio, {chip = "opl2" | "opl3", group, gain}) with write(register, value,
at?), writes({reg, val, ...}, at?), reset(at?), now, setGain, destroy and a
read-only rate of 49715. chip defaults to "opl2". Writing methods return
whether the writes were queued; a register or value out of the chip's range
raises, writes checks every pair before queuing any, and a batch past 4096
pairs is refused. A time within a millionth of a sample of a whole sample
counts as it. The chip takes its voice slot through the same internal lender
the synth uses. The backend contract gains FmChip, FmPort and an optional
openFm; a backend without it and a mixer without an output get silentFm.
README records why the interface is registers only, the per-target grid and
the same-sample rule, what a timed reset discards, the chip rate, and why
the emulator is the ymfm-oxide crate rather than source in this tree.
fm.new(mixer, {chip, clock?, group?, gain?}) now makes any of the 24 chips
ymfm-oxide emulates: ym3526 y8950 ym3812 ymf262 ymf289b ymf278b, ym2413
ym2423 ymf281 ds1001, ym2151 ym2164, ym2203 ym2608 ymf288 ym2610 ym2610b
ym2612 ym3438 ymf276, ym2414, ym3806 ym3533 and ym2149. "opl2" and "opl3"
stay as aliases of the YM3812 and YMF262, so the OPL calls and their rate of
49715 are unchanged. clock defaults to the chip's usual one and must be
100 kHz to 100 MHz. rate is now per chip (the native port reports it);
clock, lastRegister and lastPort are new read-only fields, and chips(),
defaultClock(), rate() and memoryClasses() describe the chips.
New methods: writePort/writePorts write the chip's own ports (offset, data)
beside the register convenience, which maps a register's high byte to the
bank's address/data pair (0/1, 2/3, the OPL4 PCM pair at 4/5, the YM2149's
data port at 2; the OPQ is written directly up to 0x11F). setMemory(class,
image) hands the chip an ADPCM-A, ADPCM-B or PCM image, which replaces the
class from the next frame and survives a reset; images cross on their own
8-deep channel paired with a write-queue event, and displaced ones are freed
on the game thread. pause/resume/paused, setStereo and destroy(fadeOut?)
work on the chip's voice.
Rendering: the OPN chips with an SSG run at ymfm's lowest fidelity, and a
chip whose emulator renders past 64 kHz (YM2203, YM2608 x3, YMF288 x2) has
samples averaged into one frame of its clock. Outputs fold to stereo by a
table (OPL3 A/B, OPL4 DO2, OPLL melody+rhythm, SSG at 2/9 a channel, ymfm's
own OPNA mix) and chips with an SSG get a 5 Hz DC blocker. Timers, busy
time, IRQ and status reads are not offered (CSM and ADPCM end flags need
them). The C ABI opens a chip by name and clock and gains rate, write_port,
a ports flag on writes, and memory.
Tests: every chip sounds through its register layout, rates match ymfm's
sample_rate for every chip and clock (and the Nupp table matches the native
port), decimation matches a direct render, ADPCM-A plays from an image and
keeps it across a reset, the memory queue is bounded and returns images, and
a YM2612 plays resampled through the engine. README records why the chip is
a name and a clock, the fidelity and averaging choices, the SSG filter, and
why memory is owned by the render side without a lock.
Synth:pause(), resume() and paused() pause the synth's own voice, which stops its clock with it, so scheduled events wait and the synth carries on exactly where it was. destroy(fadeOut?) fades the voice over fadeOut seconds instead of cutting it; without one it stops at once as before. tecs.audio.fm gained the same on its chip in the previous commit; the music players use both.
Two asset kinds, named "music.midi" and "music.fm" (new tecs.assets MUSIC_MIDI and MUSIC_FM constants, pinned in the compatibility test): - tecs.audio.music.midi.kind loads a Standard MIDI File as a Song: format, ppq or SMPTE rate, tracks, every event merged in time order with its seconds through the tempo map, the tempo map, duration, and loop points from loopStart/loopEnd markers or controller 111. Format 2 patterns play one after another. SysEx and meta events are kept, not sent. - tecs.audio.music.fm.kind loads VGM (to 1.72), VGZ (gzip, inflated through nupp.compression) and DRO v1/v2 as a RegisterLog: chips with clocks, memory images, packed timed writes (entry(i)), loop, GD3 tags, and a skipped list. Every Yamaha chip VGM names plays, dual chips from bit 30 and alternates from bit 31 (VRC7, YM3438, YM2164, YM2610B), the AY family as a YM2149; ROM/RAM data blocks become memory images; YM2612 DAC data (0x8n, 0xE0, DAC streams 0x90-0x95) and compressed blocks are turned into register writes when read. Other chips are skipped with a reason; a file with nothing playable is refused. DRO dual OPL2 plays hard left/right. Parsing is Nupp on the readiness pump, 65 536 commands a pass; parse() does the same synchronously for bytes a game holds. Files are capped at 16 MiB (MIDI) and 64 MiB inflated (logs); refusals name the byte offset. Players: midi.play(mixer, font, handle, options) / playSong, and fm.play(mixer, handle, options) / playLog return a tecs.audio.music.Player that owns its synth or chips and retains the handle. Audio.update advances every player on the mixer (a new seam list in tecs.internal.audiosynth). It schedules lookahead (default 0.05 s) ahead on the voices' clock; a stall makes only the first late event play at once and keeps the rest at their distance; refused sends retry next update. Loops forever when the song has a loop point (options.loop, setLoop), releases held notes at the loop end, flushes the last tick at the song end, then rings for a tail and frees its voices. stop(fadeOut?), pause/resume, seek (MIDI: synth reset plus chase of programs/controllers/bends; FM: chip reset plus each register's last value), setTempo, setGain, setMuted (MIDI channel or chip), position, duration, loops, finished, status, and a reload of the handle rebuilds the driver at the same position (new chips when the log's chips changed). A Driver record is the seam a game can implement for a format of its own (newPlayer). The host stages Nupp's native-compression feature beside native-files and native-net, in build.rs, build-support and the dev tools, so a component that reaches nupp.compression loads. Tests build every fixture byte by byte (no third-party music): SMF formats 0/1/2, running status, tempo maps, SMPTE, loops, refusals; VGM chips, flags, loops, GD3, data blocks, DAC writes and streams, compressed blocks, VGZ, pre-1.10 clocks, AY flags; DRO v1 both header widths, v2 codemaps, dual OPL2; kinds through a loader; players on a logging backend with a test clock for lookahead, stalls, finish and tail, loops, tempo, seek, mute, pause, fade, reload, full queues, chip memory and stereo. README records the choices and the alternatives that lost.
Collaborator
Author
|
Folded into #8, which now carries the synth, FM and music layers on one branch (audio-synth). |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Depends on #8 (
audio-synth) and #11 (audio-fm-ymfm); review after them. Only the last two commits are this PR's.Two asset kinds for music, and players for them.
Kinds:
tecs.assets.MUSIC_MIDI = "music.midi"andtecs.assets.MUSIC_FM = "music.fm", pinned in the compatibility test. Parsing is Nupp, on the loader's readiness pump (65 536 commands per pass). Caps: 16 MiB for MIDI, 64 MiB after inflating for register logs.tecs.audio.music.midi—kind,parse(bytes, name?),play(mixer, font, handle, options?),playSong(...).loopStart/loopEndmarkers or controller 111.Songrecord holds the format, timing, tracks, merged timed events, tempo map, duration, loop points and title. SysEx is kept but not sent.tecs.audio.synthGeneral MIDI synth.tecs.audio.music.fm—kind,parse,play(mixer, handle, options?),playLog(...).nupp.compression), DRO v1 and v2.tecs.audio.fm. Dual chips from header bit 30; bit 31 selects VRC7, YM3438, YM2164 or YM2610B. A DRO dual OPL2 plays one chip hard left, one hard right.skipped); a file with nothing playable is refused.RegisterLogrecord holds format, version, rate, length, duration, loop start, chips (clock, stereo, memory images),entry(i), GD3tagsandskipped.tecs.audio.music.PlayerAudio.update.group,gain,loop(forever by default when the file has a loop point),tempo,lookahead(0.05 s),start.stop(fadeOut?),pause,resume,seek,setLoop,loops,setTempo,setGain,setMuted(MIDI channel or chip index),position,duration,status,finished,reloadFailure.newPlayer(mixer, Driver, handle?, options?)is the seam for a game's own format.Behaviour, for review:
Synth:
tecs.audio.synthgainspause,resumeanddestroy(fadeOut?).Host:
native-compressionis added to the host features (winit-host/build.rs,build-support/src/nupp.rs,tools/tecs/dev/commands.nupp) so VGZ can inflate.All fixtures are synthetic. The README records the parse-in-Nupp, players-own-voices and skip-don't-refuse decisions.
Checks:
cargo fmt --all --check, clippy -D warnings (tecs-build-support, tecs-winit-host),cargo test -p tecs-build-support,nupp task format-check,nupp check --strict(0 warnings),nupp task test-tools,nupp test(1081 passed, including the 38-caseaudiomusictest),nupp task docs-check. Not run here: a fullnupp task package.