Vocoder
A channel vocoder with the picture as the carrier, for Resolume
What it does
An FFGL effect for Resolume Arena and Avenue. A channel vocoder splits the carrier into frequency bands and sets each band's gain from the matching band of the modulator. That is the whole machine; everything else is a choice of what to put in it. So put a picture in it: the carrier is the frame, its frequency is spatial frequency, and the modulator is the audio.
The frame is split into a Laplacian pyramid — eight octave bands, from one-pixel detail up to 128-pixel shapes, plus a residual — each band is multiplied by a gain, and the bands are summed back. With every gain at 1× the input comes back exactly, and not approximately: the reconstruction is arranged so that the thing being expanded is a difference that is identically zero, so the output is the input to the bit. That null is the plugin's foundation, and it is what makes everything else honest.
Three things come out of the same machine. With nothing routed it is a graphic EQ for spatial frequency: a boost at the 4 px band makes the picture ring at 4 px, a cut removes detail at that scale and leaves every other scale alone, and a tilt across the bank is sharpen at one end and soften at the other. This is not a blur and not a sharpen filter — it is nine independent faders across scale. With audio routed it is a vocoder: bass pumps the large shapes and treble sparkles the fine detail, or the reverse by a switch, and the audio gains multiply the slider gains so a band you have cut stays cut whatever the music does. And turning the Residual down leaves a new kind of edge picture — the picture as its detail bands, with its DC and coarsest shapes removed.
The band sliders are 0 to 4×, linear, with 1× at a quarter. That is unusual and deliberate: a graphic EQ's fader is symmetric in dB, but a vocoder band's gain is a multiplier that spends most of its life between off and as loud as the carrier, so this gives the cut side a usable length and the boost side room to ring. A band's name is its level's sampling pitch, and its power peaks a little lower — an octave band of a Burt–Adelson pyramid is centred well below its level's Nyquist — so the 4 px band rings with a period of about 21 px. The harness measures that rather than assuming it.
Sidechain Mode picks one envelope per band, which is the vocoder, or one envelope for the lot driving the Tilt, which is a dynamics processor across scale. Floor decides what a silent band does: at 0 quiet bands go dark, at 1 the audio only ever adds.
- An eight-band Laplacian pyramid plus residual — 1 px detail to 128 px shapes
- Exact reconstruction at unity: the output is the input to the bit, which is the null every other claim rests on
- A graphic EQ for spatial frequency with nothing routed, and a vocoder with audio routed
- Band gains 0 to 4× with unity at a quarter, so cuts have length and boosts have room to ring
- ±6 dB tilt across the bank — sharpen at one end, soften at the other
- Direct or Reverse audio mapping: bass to coarse shapes, or bass to fine detail
- Vocoder or Dynamics sidechain — one envelope per band, or one envelope driving the tilt
- Residual at zero for a detail-band edge picture that is not an edge detector
- RGB or Luma carrier, so the input's colour can be left alone
- Universal macOS bundle and a Windows x64 DLL
Where it stands
v0.1.0, and honestly early. The central claim is numerical, so tools/verify.sh measures it against a fresh universal Release build: reconstruction is exact with a maximum difference of 0 at 720p, 1080p and 4K, and 5.96e-08 through the Luma carrier's Y/Cb/Cr round trip; the eight band-cuts partition the picture to 9.54e-07; the shipped shaders agree with an independent CPU implementation of Burt–Adelson to 1.2e-07 over all eight bands; the bands peak at 2.0, 10.1, 21.5, 43.5, 87.3, 174.5, 347.0 and 677.1 px, doubling as an octave bank must; all 16 audio-band-to-picture-band mapping cases are exact, 10 of them re-checked through the picture against the equivalent slider at a maximum difference of 0; and the envelope followers hold their attack and release to within 0.0% at three settings. All 19 swept controls measurably change the picture, with 2 skipped and the reasons given. Render cost on macOS is 0.77 ms at 720p, 0.66 at 1080p and 1.89 at 4K — 1080p being faster than 720p is not a typo: there are 33 passes for eight levels, most of them tiny, so at small sizes the per-pass overhead dominates. On 21 September 2026 the x64 DLL was registered, loaded and instantiated in Resolume Arena 7.27.1 on win-lab, listed as SW Vocoder under idstring VC01 among 112 video effects, shaders compiling and the inspector drawn, with a clean diag log. That VM has no GPU — OpenGL came from Mesa llvmpipe — so it has never run on a GPU in Resolume, has never been instantiated in Arena on macOS, and nothing was timed on Windows. No real audio reached it in Arena, so what Resolume's 64 FFT bins actually mean is still assumed rather than measured, and that is the biggest open question in the repo. No long session, composition save and reload or preset recall in the host; no OpenFX port, no user guide and no factory presets. The browser demo is live, but it is the spatial-frequency EQ side only — the whole audio group is absent from it by design, because a browser has no equivalent of Resolume's FFT parameter. Check it in your own rig before trusting it in a show.
Watch it running
Nothing loads from YouTube until you press play.Watch it there instead