Musings from my desk

Loop Supreme, part 9: Visualizing the waveform

2022-11-23 14:34:05 +0000 UTC

This is part 9 in a series about building a browser-based audio live looper

Goal

Create a waveform visualization, and redesign the app to make it feel more user friendly

Implementation

The task of visualizing the waveform was really fun for me - it was just the kind of semi-mathy-but-low-stakes work that I love.

I’ve seen many examples online and several SO answers that draw values to a canvas to visualize a waveform. The general pattern is one “sample” (which might be a grouping of actual audio samples) is drawn as a fixed-width rectangle, with the height being determined by the amplitude.

The rectangles-on-a-canvas pattern works totally fine, but I have an unjustifiable love for SVG and I felt like it was a superior medium for drawing scalable waveforms. The other benefit is that an SVG waveform could be stretched to fit variable aspect ratios, which works much better for a web app where the window dimensions are unknown and highly variable.

I knew the general requirements were:

  1. Ingest frames (blocks of samples) at about 60hz (the RecordingProcessor audio worklet is responsible for publishing this data back to the app)
  2. Normalize the frames in realtime. I don’t know the bounds of the gain on a web audio input. In my experience the absolute gain values are « 1, but I didn’t know if this was universally true. Normalizing on the fly would prevent the need to guess.
  3. Scale the points to the correct x-axis dimensions based on the expected number of frames in the loop
  4. Create a string to represent the d attribute of the SVG path

Each of these steps are relatively processor intensive, so I reached for… (drumroll 🥁)… a Worker! Delegating all this processing to a worker ended up being really nice because I didn’t have to think much at all about serious optimizations, since all the major number crunching would be happening on a separate thread. You can see the completed waveform worker here.

The most challenging part of this was mentally mapping the data flow. In a UTM-style diagram, it looks like

[ Recording ]    [ App ]   [ Waveform ]
[ Processor ]       |      [  Worker  ]
      |             |             |
  Publish frame --> |             |
      |             |             |
      |       Publish frame ----> |
      |             |             |
      |             |         Collect frame
      |             |             |
      |             |         Normalize and
      |             |         scale horizontally
      |             |             |
      |             |         Generate SVG
      |             |         path of smooth
      |             |         cubic bezier points
      |             |             |
      |             |  <----  Publish path
      |         Update `d`        |
      |         attribute in      |
      |         SVG path          |
  (repeat)

The end result was simple and effective, just like I wanted.

Redesign

In addition to visualizing the waveform , I redesigned the app. A fun thing I tried that I’ve never done before is actually drawing out a mockup of my design idea before I coded it up. This ended up being super helpful! Instead of trying to code and design at the same time, I was able to look at things from a higher level and make choices about layout that I probably wouldn’t have made if I had been coding it at the same time. It was easy to experiment with layout variations, which would be impossible with coding. I’ll definitely be doing this for future projects where I care about the layout. Of course, I’m not a designer, and the final product could still use work, but I’m happy with the overall effect.

Here was the impact of the redesign

Before

loop-supreme-9-redesign-before.png

After

loop-supreme-9-redesign-after.png

Mockup

loop-supreme-9-redesign-source.png

Learnings

State of the app

Next steps