Databend
A picture through audio effects, for Resolume
What it does
An FFGL effect for Resolume Arena and Avenue. Databending is opening a picture in an audio editor and running audio effects on it. The editor does not see a picture: it sees one long stream of samples in the order the file lays them out, row by row, one plane after another or R, G, B per pixel, with a row stride that may carry padding, and a delay-line effect on that stream does not know where the lines are. Databend models that stream exactly and runs real delay-line effects on it, in the order an audio editor would: echo, flanger, phaser, pitch shift, then the export back to pixels.
Everything falls out of the layout. An echo of k lines and m samples comes back k lines down and m samples across, and with feedback the echoes march down the picture on a slant, each m g to the k as bright as the last. A flanger is a delay swept by an LFO in stream time, and stream time is the scan, so the ghost's offset changes line by line: a comb across the scanlines. A phaser is a cascade of all-pass stages, which pass every frequency at unity gain and delay smooth content more than edges, so the picture smears rightwards with ringing and keeps its energy. A pitch shifter resamples the scan in grains, stretching the line inside a grain and repeating or skipping at the seam.
The file layout is part of the mechanism. In an interleaved RGB stream a delay that is not a multiple of three puts one channel's samples in another's slot, so the echo comes back in the wrong colour, and every effect rotates hue because hue lives at the stream's period-3 frequency. In a planar stream a delay past a plane's start reads the previous plane: the first lines of green carry echoes of the last lines of red. The sample format decides what happens past full scale: 8-bit and 16-bit files wrap (the harsh look) or clip, a float file clips; an unsigned file's silence is black, a signed one's is mid-grey.
Each frame is its own file: the delay memories do not cross frames, but the LFOs carry their phase across frames so the comb and the sweep move. The phaser runs on the GPU as a windowed restart whose window is derived from a proved bound on the cascade's forgetting; a swept phaser with feedback has no such bound, so that control is deliberately absent. Every coordinate is an integer, every coefficient is computed in double on the CPU, and the LFOs are exact triangles, which is what lets the harness hold each effect to its closed form.
- Planar or Interleaved layout, a row stride with padding, rows or columns as the line
- 8-bit unsigned, 16-bit signed or float format; Wrap or Clip past full scale
- Echo: lines and samples of delay, feedback to 7/8, its own mix
- Flanger: base delay, depth and rate in stream time; a crossfade
- Phaser: 2 to 12 all-pass stages, rate and depth, at unity gain
- Pitch shifter: half to double, grains of 64 to 4096 samples
- Universal macOS bundle and a Windows x64 DLL
Where it stands
v0.1.0, and honestly early. The effects are measured, not asserted, by an offline harness that drives the real plugin class headlessly at two rasters: an echo of one line and k samples lands one line down and k across and moves with the row stride when the file has padding, with the whole picture matching the closed form to zero error; the nth echo sits at n·(k, 1) with amplitude m g to the n; in an interleaved file a delay of 1 mod 3 puts red's echo in green, green's in blue and blue's in the next pixel's red; the flanger's ghost on every line sits where the LFO says at that line's stream time; the phaser keeps a noise picture's energy and matches a serial double run of the cascade within a bound it proves; an 8-bit file wraps past full scale, a 16-bit one wraps, a float one clips, exactly. Seven negative controls and a one-character mutation of the shipped shader are caught. All 25 controls change the picture. Render cost on macOS at the defaults is 5.8 ms a frame at 1080p and 18.7 ms at 4K, so 4K is not real time with the phaser on. There is no phaser feedback, by design. It has not yet been loaded into Resolume on macOS. There is a user guide; no presets and no OpenFX port. Check it in your own rig before trusting it in a show.
Watch it running
Nothing loads from YouTube until you press play.Watch it there instead