Musings from my desk

Loop Supreme, part 7: Latency, and adding track functionality

2022-11-16 16:21:20 +0000 UTC

This is part 7 in a series about building a browser-based live looper


Fix the latency issue that was causing recordings to play back significantly behind the beat. Improve app usability by adding volume controls to the metronome and individual tracks.


This was a really interesting and exciting step for the app. I was able to resolve (mostly, see “Learnings” below) the latency issue which was causing recorded audio to be substantially delayed on playback. Of course for a looper app this was untenable; the audio must align with the playback loop near-perfectly. The solution to this was surprising to me. I knew that the AudioContext reported a baseLatency and outputLatency value, but I didn’t realize that these correspond primarily to playback. From what I read, the primary use-case for these is synchronizing audio playback from external sources with video, for example when streaming a video with sound.

I found was that the latency I wanted to use to account for recording latency was on the MediaStreamTracks that come from the MediaStream (i.e. the input device). However, getting this value is far from straightforward! If you read the code, you’ll see that there are 3 possible options for getting track latency. What surprised me is that the TS types in lib.dom.d.ts disagreed with MDN on 3 separate properties / methods. I ended up writing a highly defensive method to get the track latency in whatever way I could. My favorite quote from MDN was:

Note: getCapabilities() hasn’t been implemented yet by all major browsers. For the time being, you’ll have to try to get what you need, and if you can’t, decide what to do at that point. See Firefox bug 1179084, for example.

Indeed, getCapabilities was the only method I found that worked, and it only works in Chrome. I guess this is going to be yet-another “Chrome-only” app, which kinda breaks my heart 💔

Once I had the track latency, I was able to adjust for it by shifting the samples in the recording buffer by the amount of latency the track reported. This basically fixed the issue on playback, and now loops are well-synchronized (in the best-case scenario), which is very exciting!

The other major work in this update was adding capabilities to the Tracks and Metronome. This included some general refactoring of the audio graph* which had the serendipitous side effect of fixing the issue I had with recording multiple tracks! Turns out, sound software design really is important 😏. In addition, I was able to add volume controls (including a mute option) to both Metronome and Tracks, and I added a monitor option to the Track if you want to monitor the input during recording.

*Note: I’m not sure “audio graph” is an official term, but since the Web Audio API consists of various AudioNodes that get connected together to produce sound, I felt like “audio graph” was as good a term as any to describe the technical underpinnings of the app.


Loop Supreme Track Controls

State of the app

Next steps

  1. Test with a real audio interface. So far my primary testing has been sitting at my computer snapping into a webcam mic. The app is semi-functional now and I need to see if it actually works in a real musical setup
  2. Prepare for an initial deployment (assuming no major bugs found in testing). The MVP I’m shooting for is
    1. Builds and deploys automatically (probably using GitHub Actions; not sure about the host yet)
    2. Has some basic OG tags so social sharing works

Everything else in the roadmap can probably be moved to post-deploy, although many of these are things I’d like to happen very soon (keyboard bindings, waveform visual display, general design cleanup).