Add reusable sequencer groups with quantized execution control

[linuxificator]

Summary

This PR supersedes and replaces #1148.

I reworked the proposal around the abstraction suggested in the discussion on
#1148. Instead of introducing a separate “nested pattern” API, this extends
AMY’s existing sequencer with reusable sequencer groups:

• the existing ticks command gains an optional fourth group_tag;
• ordinary sequencer events can be collected under that group tag;
• one sequence_control command publishes, starts, stops, or gates a group;
• each execution starts from its own local tick zero and can run once, N times,
  or continuously.

This removes the previous pattern-specific begin/add/commit/trigger vocabulary.
The public interface remains an extension of AMY’s existing H sequencer
syntax and a single zQ control family.

Motivation

A flat root sequence can represent any final collection of notes, but it becomes
difficult to edit safely during live interaction.

Two representative use cases are:

1. A rhythm engine with preloaded fills

   A controller may preload many short fills and select which one will be used
   at a future phrase boundary. Without groups, it must expand the selected fill
   into the root timeline, replace all affected future events, coordinate
   background layers, and avoid truncating a fill already in progress.

   With sequencer groups, the root sequencer only schedules a small start
   command. Changing the future fill replaces that root event without changing
   an execution which has already started.

2. Arpeggios with live rate or note changes

   An arpeggio phrase contains both note-ons and their matching note-offs. If a
   controller rewrites a flat sequence while a note is sounding, it can remove
   the pending note-off and leave a hanging note. An immediate all-off avoids
   the hang but cuts the note short.

   A group execution retains the immutable revision with which it started.
   Future executions can use a newly published revision while the old execution
   still delivers its original note-off at the intended tick.

AMY does not know about fills, drums, arpeggios, or controller policy. It only
stores and executes ordinary AMY wire events.

Wire interface

The existing ticks tuple accepts an optional fourth value:

tick,period,event_tag,group_tag

For example, this stages an arpeggio in group 1:

H0,48,0,1i2n60l1Z
H10,48,1,1i2n60l0Z
H12,48,2,1i2n64l1Z
H22,48,3,1i2n64l0Z
H24,48,4,1i2n67l1Z
H34,48,5,1i2n67l0Z
H36,48,6,1i2n71l1Z
H46,48,7,1i2n71l0Z

The staged definition is published atomically with an explicit length:

zQ1,3,48Z

It can then be started once at the next 48-tick boundary:

zQ1,1,1,48Z

The control layout is:

group,action,value,quantize[,execution_tag]

Action	Value	Meaning
0 stop	reserved	Stop matching executions
1 start	repeat count	1 once, N N times, 0 continuously
2 gate	duration	Suppress event dispatch for N ticks; 0 releases
3 publish	length	Atomically publish the staged definition
4 clear	reserved	Remove stored revisions

An optional execution tag can address one active or pending execution. Without
one, stop and gate affect all executions of that group.

Because sequence_control is an ordinary AMY wire command, the root sequencer
can schedule it using its unchanged H syntax:

H960,0,40zQ1,1,1,0Z

Semantics

The implementation separates:

• a persistent group definition;
• its private staging revision;
• its immutable published revision;
• bounded active or quantized-pending executions.

Important guarantees are:

• every execution begins at local tick zero;
• publication is atomic;
• an active execution retains the exact revision it started with;
• edits and publication affect future executions only;
• finite and continuous repetition use the same execution mechanism;
• quantized start and stop use AMY’s sequencer clock;
• gating suppresses event dispatch while local phase keeps advancing;
• already-sounding audio is not forcibly stopped by a gate;
• group definitions survive RESET_SEQUENCER and RESET_TIMEBASE;
• active and pending executions do not survive those resets.

There is one bounded phrase level below the root sequencer. A root event may
start a group, but a group cannot start, publish, or clear another group. Group
payloads may contain stop or gate controls as leaf operations. This prevents
recursive scheduling and keeps lifetime and memory bounded.

Resource bounds

The limits are independently configurable through:

• max_sequence_groups;
• max_sequence_group_tags;
• max_sequence_group_executions.

The defaults are 32 groups, 64 local event tags per group, and 32 active or
pending executions. Setting any of the three values to zero disables sequencer
groups.

Inactive definitions are not scanned on every tick. The tick path scans only
the fixed execution pool, and starting or processing an execution does not
allocate group-engine memory.

Backward compatibility

The new behavior is opt-in.

An absent or zero fourth ticks value follows the existing root-sequencer path.
Existing three-value H messages retain their wire representation and
semantics, including:

• absolute and modulo timing;
• tagged replacement and clearing;
• anonymous events;
• root tag ordering;
• reset and timebase behavior.

The C amy_event.ticks array gains the optional group field, and
amy_config_t gains the three resource limits. Existing source using
amy_default_event() and amy_default_config() receives compatible defaults;
applications linking AMY as a C library should be rebuilt because the public
structure layouts have grown.

Testing

The dedicated native regression suite covers:

• legacy three-value wire and C-event behavior;
• group-zero compatibility with the root sequencer;
• local group-tag namespaces;
• one-shot, N-shot, and continuous execution;
• quantized and immediate start, stop, and gate;
• execution-tag selection and replacement;
• overlapping untagged executions;
• phase preservation during finite gating;
• atomic publication and repair after rejected publication;
• immutable active revisions;
• same-tick root launch and local tick zero;
• rejected recursive lifecycle operations;
• allowed stop/gate leaf controls;
• reset and timebase behavior;
• 32-bit sequencer tick rollover;
• configured group, tag, and execution limits;
• disabled and exhausted configurations;
• invalid definitions, ranges, and actions.

Validation performed:

• make ctest
• existing Python/audio suite, with results matching Shorepine main
• Python 3.12, 3.13, and 3.14
• WebAssembly build
• generated C, JavaScript, Python, and Godot API checks
• Godot Linux GDExtension build
• ESP32-S3, RP2040, RP2350, and Teensy 4.1 builds
• AddressSanitizer and GCC static analysis

All of these checks pass.

Documentation

• Sequencer groups
• Abstractions and implementation
• Musical use cases
• Step-by-step wire and Python how-to

Scope

This branch is based on the current Shorepine main.

It intentionally contains only the sequencer-group proposal and does not include
the independent Unix-socket transport or Godot lifecycle-signal work from
#1147.

[dpwe]

Thanks, I appreciate all the improvements here.

I'm still unclear what the purpose of the "explicit length" in the "publish" command is for, or indeed why we need a "publish" command at all. To write a group, you first stop it (maybe all groups except 0 are stopped by default), you write out its components, then you start it when it's done. Then, its looping occurs as indicated by the loop counts in the individual component ticks statements.

The "play once/play N times" might perhaps be handled via a second level of scheduled events:

• Define sequencer group A with an N-tick sequence
• Define sequencer group B as "trigger sequencer group A at time 0" plus "stop sequencer group A at time R * N"
  Then triggering group B would lead to R repeats of the sequence.

I know you had a provision that did not allow recursive use of patterns, nominally as a way to guarantee there were no loops, but I don't see any great danger in relaxing that. Maybe you could end up with out-of-control loops, but as long as each group can only be executing once at a time, it should all be recoverable.

I hadn't realized the need for non-phase-aligned playback, which is somewhat counter to the AMY sequencer philosophy, but I see the need. Right now, the sequencer aligns events to their repetition period (sequencer_ticks % ticks_period == 0). We could add another parameter, alignment_period, which supersedes ticks_period as the alignment quantum when provided; setting alignment_period to 1 would allow the sequence to start at the nearest available sequencer tick to when the event is processed (in practice, this would update the effective tick of the sequencer event to incorporate the current phase within its period). I think it makes sense to define alignment_period only for sequencer groups.

I think, then, we have one new wire command, sequence_control, which has a start/stop field, and an optional alignment_period field. I think including the alignment_period subsumes the field I had before to distinguish "at end of current sequence" from "immediately".

The problem of stuck notes is .. a problem. I've seen this plenty of times -- wanting to schedule a note-off alongside a note-on, in a way that guarantees they are both executed. I wonder if there's a way to use hierarchical sequencer groups so that each note becomes a pair of (note on, note off) events, then stopping the parent sequence still allows the note off to play out. That ends up being a lot of groups, but .. that's why we have computers. I'd rather get the API syntax clean, then worry about efficient implementation later. This might need a new "one off, now-relative" syntax for ticks, e.g. ticks='24,-1' means "do it 24 ticks from now". I'd proposed this before, but it was rejected as too confusing/unpredictable.

I'm not sure we need group definitions to survive RESET_SEQUENCER; I'm not clear what you want RESET_SEQUENCER to do other than reset all the sequencer definitions. If it's to stop all currently playing sequences, maybe a new MUTE_ALL_SEQUENCER_GROUPS is really what you want?

Finally, I'd really prefer you provided examples using the Python syntax instead of wire commands; wire commands are not supposed to be something that human look at. Also, the z prefix means commands related to the host-to-AMY interface (file transfer, python commands, etc). I would put this as an extension of the H prefix, but again that's just an implementation detail: Instead, it's amy.send(sequence_control=','.join(str(x) for x in (group_tag, is_playing, alignment_period))).

I noticed your syntax to allow group control to apply to individual tags below the level of group. That's interesting, but it makes me nervous (so much state!) and could maybe be handled with nested group definitions - you can directly send a mute command to a subgroup while the parent group is active to drop out some of the notes.

Finally, you pay substantial attention to preserving currently-active sequences independent of updates to the sequence for the next repeat. I find that somewhat appealing, but also complicated. Why do you need that? Does your application have incremental, real-time changes to the patterns? Can't we just require the application to batch pending updates and issue them all at once on phrase boundaries? I'm not convinced that needs to be AMY's job.

[dpwe]

Brian points out that we could have the existing sequencer tags be the groups themselves, i.e. add multiple events behind a single tag. Having one event per tag, from the API's point of view, was to allow event-level edits of what the sequencer is doing, but actually this doesn't seem particularly friendly, so allowing people to write as much as they want cumulating onto a single tag, then providing a per-tag RESET_SEQUENCER, seems like a good way to go.

[linuxificator]

Thanks for the detailed suggestions. I removed my previous reply because, reading it back, I had expanded the proposal into a much larger execution-policy framework while trying to cover every possible use case. That would work against the API simplification you and Brian are proposing.

A brief explanation for my earlier focus on wire commands: in my multi-platform application, the AMY wire protocol is the boundary between the application and the synthesizer process. It is transported through sockets on Linux and Android and through a named pipe on Windows. I will change that. I will also move the low-level control representation under the sequencer-oriented H family rather than z.

I also need to watch my own tendency to turn requirements from my application into default behavior for every AMY user. I would like to start with the smallest model and only add another primitive when a concrete generic requirement cannot be expressed without it.

My current understanding of the simpler direction is:

1. Existing sequencer tags become the identities of stored sequences.

A tag may contain multiple ordinary schedulable AMY events. There is no separate group namespace and no fourth ticks field.

2. Legacy tagged-event behavior remains compatible.

Existing code using:

 amy.send(..., ticks=(tick, period, tag))

should continue to replace the event at that tag rather than silently accumulating additional events.

Multi-event definition therefore needs to be explicit. The Python API could accept a complete list of events for one tag and replace that tag’s contents as a batch. Internally, this could translate to a per-tag reset followed by cumulative writes.

The exact Python spelling could be different. The useful properties are that cumulative behavior is opt-in and that no separate public publish/revision vocabulary is required.

3. A per-tag RESET_SEQUENCER clears all events belonging to that tag.

The existing global reset can retain its existing meaning. An application that wants to stop playback while retaining preloaded definitions should use a separate stop-all or mute-all operation rather than reset.

4. One sequence_control operation starts or stops a tagged sequence and optionally supplies alignment_period.

Looping remains defined by the component ticks periods. Repeating a sequence a fixed number of times can be expressed using a controlling sequence that starts the child and stops it after the required number of periods, as you described.

5. Tagged sequences may trigger other tagged sequences.

I will prototype this model rather than assuming in advance that recursion or another execution layer is necessary. A bounded “one active execution per tag” rule may be sufficient to keep cyclic definitions recoverable.

Before proposing additional controls, I want to verify three concrete behaviors with that smaller model:

• A preloaded finite rhythm fill can be selected and launched without expanding its complete event list into the root schedule.
• Stopping an arpeggio parent prevents new note children from starting while note-on/note-off children that already started can complete.
• A repeating percussion subgroup can be temporarily suppressed and later resume at the intended phase without requiring the controller to mirror AMY’s clock.

The note-lifetime case needs particular care. One application may want a started note-pair child to deliver its scheduled note-off, while another may deliberately want to terminate a long note early. I do not want to prescribe a large stop-policy API before understanding what the hierarchical model already provides.

The question I would like to clarify during the prototype is therefore: what should explicitly stopping an already-active leaf sequence mean? It may be enough to distinguihs stopping a parent, which prevents new children, from stopping a note-pair leaf, which deliberately terminates that leaf. I will avoid adding caller-side active-note bookeeping merely to force one particular application behavior.

This seems capable of removing the separate group IDs, fourth ticks value, public publication action, local group tags, and most revision machinery, while preserving existing tagged sequencer behavior.

Does this match the smaller direction you and Brian have in mind?

[dpwe]

Quick comment: I am inclined NOT to maintain backwards compatibility with the existing "repeating a tag overwrites its previous content" behavior. For harmony with the way synth= works, I think the default of repeating a tag should be to cumulate.

We can ask Claude to see how much this would break, and I am prepared to have my mind changed if it's a lot.

[linuxificator]

Ok, making repeated tags cumulative by default appears to allow more simplification than just reversing the compatibility condition.

At the moment this PR has two parallel ways of doing essentially the same job: the existing root sequencer, where a tag holds one event, and the separate group machinery added here.

I'm checking the consequences for start/stop behavior and for the current dynamic root scheduling, but so far it looks cleaner. If those checks don't make me crave chocolate, cumulative tags as the default seems to make more sense than preserving the old overwrite behavior.

[dpwe]

But broadly, yes to your final question. I put a high value on supporting a wide range of functionality with the minimum of parameters.

I think sequence groups allow us to replace a current thing like amy.send(synth=1, note=60, vel=1) with amy.send(ticks='0,-1,100', synth=1, note=60, vel=1); amy.send(ticks='24,-1,100', synth=1, note=60, vel=0) then amy.send(ticks='0,48,1', sequence_control='100,1,1').

I don't exactly love that syntax, though - I would like triggering a sequence group to look more like sending a note-on. Maybe this is somewhere else where we can use wire command templates.

[linuxificator]

I've updated the current PR rather than starting another one.

The branch now follows the simpler model we discussed: existing sequencer tags are the reusable sequences, repeated use of a tag accumulates events, and sequences can start/stop other sequences. I removed the separate group namespace and most of the extra machinery that came with it. I also adjusted the tests and documentation around that model.

The relevant documentation is here:

• Reusable sequences
• Abstractions and implementation
• Step-by-step Python and wire examples
• Musical use cases
• Status, compatibility, and migration

There is one unrelated change on the branch worth pointing out. While testing the Windows Godot build I found that current main fails under MSVC because pcm.c uses M_PI, which MSVC does not define by default.

The actual fix is isolated in commit 397488b3, and the cause, history and validation are documented separately here:

• Windows M_PI portability note

The fallback is guarded with #ifndef M_PI, so platforms which already provide it are unchanged. I kept this separate from the sequencer work so it can be reviewed or dropped independently.

Happy reading.

[linuxificator]

While testing this proposal with a sustained stream of one-shot percussion events, I found a separate, pre-existing issue in synth voice bookkeeping.

A synth configured with the existing SYNTH_FLAGS_IGNORE_NOTE_OFFS flag still recorded every stolen or retriggered note in its bounded forgotten-note pool. That pool exists only to absorb a later matching note-off. Because callers using this flag may intentionally never send note-offs, the entries were never consumed and the pool eventually overflowed.

I reproduced this on unmodified shorepine/main at commit 0fb0a00, using an ordinary four-voice synth and direct note-on events only. No reusable-sequence functionality was involved, so the issue was not introduced by this PR. Reusable sequences merely made it easier to encounter during prolonged one-shot playback.

I have prepared a small correction with regression coverage. For synths that ignore note-offs, AMY no longer stores bookkeeping that cannot be consumed, accepts a late unmatched note-off quietly, and clears obsolete entries when an existing synth switches to ignored-note-off behavior. A control test confirms that ordinary synths still retain their existing stolen-note matching behavior.

The implementation and rationale are documented here:
Ignored note-offs and voice-stealing bookkeeping

I placed the fix on a branch based directly on this PR so the test build rules are already integrated cleanly:
rework/sequencer-ignore-noteoffs

This is not an urgent issue, so it seems unnecessary to create two related PRs whose test changes need to be coordinated during merging. Would you prefer that I include this correction in PR #1151, or submit it later as a separate PR based on whichever sequencer code has been merged by then?

[dpwe]

Thanks for all this; sorry for the silence from my end, I was away over the weekend.

I'm thinking I'd rather build this up more incrementally, starting with multiple events per sequencer tag. Sorry, I know you have the whole thing fully implemented, but I'm uncomfortable with incorporating such a lot of changes all at once.

I haven't looked at the code yet, just read the description. I'm not crazy about allowing multiple overlapping instances of each sequence to be launched (I believe that's part of your design?). I feel like it's a simpler system if we simply restart any sequence when it is relaunched.

I'm also a bit daunted by the whole discussion of managing copies during modification. I don't quite understand, but it feels like overkill for this application.

I'd like to incorporate the M_PI and IGNORE_NOTE_OFF mods as independent PRs.

Thanks.

[linuxificator]

Well, maybe I'm a bit overenthusiastic.

I had some time off work, and it started with: “Hey, this can run on everything.” I made builds and tests for a variety of platforms, using the AMY wire protocol as a hard separation to avoid the intertwinement that usually results in spaghetti. When that worked, I wanted to give something back to the AMY community and answered a Godot question with an example using my Android JNI wrapper, exposing AMY over a socket on Android.

Then I took a first step towards adding simple extra one-shots to the sequencer. This turned into something more complex as the discussion continued. My test results also showed some timing bottlenecks under intensive sequencer usage, so I introduced things like RCU handling of changes.

But these are common real-time techniques, nothing fancy.

I also ran into the vTaskDelay(1) bug, which you solved.

But actually, this PR is just a start for me... Sorry, I had the idea that it could really improve and expand AMY, better performance en live interaction etc. But I should have asked if you even want that direction.

While working with the new sequencer under heavy load and live user interaction, I found some timing bottlenecks that were not introduced by the sequencer code, but came to light under heavy usage. I created a measurement framework on the ESP32 that hardly interferes with the code itself, but shows me where the time is spent.

This revealed another vTaskDelay bug, which I fixed. I also added indexing to the sequencer events, so there is no longer a need to walk through every event on every tick. That was one of the causes of some frames suddenly having a high execution time and then triggering a vTaskDelay.

For periodic events, I created an even faster path based on the predictability of the events. For the ESP32, I also moved the sequencer state to the fastest area of memory.

I replaced the simple vTaskDelay`after an over-budget calculation with a cumulative-debt model: frames are allowed to go over budget, and the DMA buffer prevents dropouts. vTaskDelay is only called when a problematic build-up of debt is detected—currently at least one FreeRTOS tick. I could make it stricter and base the condition on the available DMA buffer... always something to improve!

The next bottleneck I encountered was the reverbs. I used reverb and looked at the implementation—it is beautiful—but they were a bit resource-intensive on the ESP32-P4. First I tried using SIMD for optimization, but that would only work by converting to 16 bit because the ESP32 does not have 32-bit fixed-point SIMD arithmetic. Switching to 16 bit is a bad idea. Trying to parallelize the LPFs only saved about 10 µs, so it was not worth it.

But I was able to move the 108 kB delay line from PSRAM to SRAM, reserving an entire block of 128 kB per reverb to avoid any interference with other operations, such as memory-to-memory DMA from other tasks.

That improved it a lot. But the other problem was that I had to use one reverb per bus. That seemed to be an AMY limitation. I use different buses to separate the instruments, drums and lots of synths, but using one reverb per synth conflicted with the maximum of two reverbs in SRAM. Also, from a musical point of view, that many reverbs do not make sense. A reverb simulates a room, and I do not have any mystical powers, so I cannot be in multiple rooms at the same time.

So I created a “bus mixer”, a simple mixer that can take a weighted sum of the outputs of the other buses and send it to the reverb. That is how a reverb is normally used—unless I am missing the point of the AMY design requiring every bus to have its own reverb, or I am mistaken about that.

The result? Running more than 80 oscillators—or about 50 heavy FM/filtered oscillators—with two reverbs and 40 active sequencer patterns, with 1,280 preloaded patterns, on a simple 11-euro ESP32-P4 Pico M board, under heavy live interaction, while still having an average of 35% processing time free and remaining responsive in real time, using 128-sample render frames and a 256-sample DMA buffer at 48 kHz.

I can run the two reverbs on separate cores on the P4. They only take 110 µs each and run simultaneously. Without the optimization, the reverbs took 1.6 ms.

I think this could be valuable for the AMY board. Maybe you will want to use the P4 in the future; it is a bit more powerful than the S3.

All the aditional tests under heavy load also revealed some other real-time issues unrelated to the new code. I will fix them, but I will keep the fixes in my own fork and stop offering PRs. I will just post them in Discussions and point to my fork, so you can see for yourselves whether they are worthwhile.

I understand that all these changes take time. I do my best to provide extensive tests and documentation, and my PRs do not conflict with amy main. They are also tested across multiple platforms through my Omnichord builds on GitHub.

I am also working on the next improvement: being able to use samples from an SD card for PCM playback on the ESP32. It is possible to do this in real time, with low latency and multiple concurrent voices, by skipping the filesystem, using an A2 card in read-only mode, and using the command-queuing capabilities of the SD card.

The required AMY changes are minimal. I only need PCM playback that does not expect one continuous RAM segment and that can be updated with new blocks to play while playing. This would turn the AMY board into a sample player capable of using gigabytes of samples.

But I will keep the code in my fork, so you can just have a look whenever you want to. I do not want to be pushy, and I understand that you do not have the time for it. It all happened somewhat accidentally because I suddenly have some free time after 35 years of working in IT...
I wonder what to do with the other list of small (but sometimes important for timing) improvements.

One other question before I stop overloading you: is it okay to modify AMY this extensively and use with the Omnichord? I had not looked at the licensing—I tend to stay away from anything even remotely associated with the word “legal”—and the license I created for the Omnichord is more of a joke.

But perhaps I should mention AMY in some way? I have no idea how that works, the omnichord and amy are two separated things and it's all just a hobby for me, I just want people to have a good time with it.

https://github.com/linuxificator/LB_Omnichord/blob/main/licence.txt

I've added a picture, it's really fun to hear so much music coming out of a small cheap board, amy is amazing!

[dpwe]

Thanks for the detailed narrative, some of those changes sound really interesting.

AMY is MIT licensed, meaning you are free to use, modify, and redistribute it. To remain in compliance with the license, you have to include the AMY copyright and permission notice in your own distribution.

My goal so far has been to maintain familiarity with the entire AMY codebase. I'm guessing you've been using Claude or similar to help you with this coding. The growth of AI coders has made my maintenance model much harder to sustain, and I'm not sure if I can keep it up, but for now I'm going to keep on being conservative and slow adding features.

I really appreciate you submitting the PR, and I would appreciate you keeping us apprised of other changes and fixes you find.

A couple of other points: (a) Yes, we are experimenting with the ESP32-P4 which is very exiciting, and (b) Maybe I misunderstood, but I thought AMY already supported playing PCM samples from SD card, that's the disk_sample mechanism. It has some limitations, though - perhaps that's what you're talking about.

[linuxificator]

Hi,

yes, indeed, the sd card PCM player I'm working on is about low-latency asynchronous pre-fetching from sdcarrd, without using any filesystem on the card, just raw blocks.
So not the synchronous read from the renderer. I also add "first blocks caching" in psram, so when a PCM sample has been played there is about 80ms time to get the following blocks from sdcard. When loop start and stop is needed I also cache those parts in psram. In between the rendering cycles I do pre-fetching of data from the sdcard using DMA and A2 sd card capabilities like command queues. (but that is outside the amy code, and it created a dependency on the card type)
And separate software to prepare the data on the card. So it's all about performance, not that much about the user experience, putting wav files on an sd card is much easier. But it does create latency and limitations.
I'm still working in it, but 64 simultaneous stereo 48kHz 16 bit PCM's from sdcard without using much psram seems possible while maintaining sub-10ms lead time.

And indeed, working with AI for coding changes a lot... I just started with using amy so I really don't have a good view in my head of the codebase. AI can go completely haywire if one would just let it code, it takes a lot of clear contracts, code quality tools en reviews, design choices to be checked, heavy focus on testing and build quality, it can easily drift into weird changes. I'm getting used to it, it does help, but sometimes it also works against keeping things plain and simple.