Video plugin APIs

Choosing a target

The decision is usually made for you by the host somebody has already bought. Where it is not, the row that matters most is render model: FFGL hands you frames in order and lets you keep state between them, OpenFX does neither, and a plugin written for the first will not survive the second without being rewritten as a pure function.

All three share a skeleton, and it is worth naming before the detail buries it. A plugin is a shared library that describes itself to a host that has just loaded it — a name, a type, a list of parameters — then gets instantiated once per use, gets told about parameter changes, gets asked to render, and is eventually torn down. The differences are entirely in who owns the pixels, what a parameter is allowed to be, and whether you are permitted to remember anything between frames.

FFGL

FFGL is FreeFrame with an OpenGL renderer bolted on. The FreeFrame lineage goes back to a CPU-buffer plugin format; everything that matters now is the 2.x API maintained by Resolume at github.com/resolume/ffgl, MIT licensed. The SDK's own CFFGLPluginInfo asserts a major version of at least 2, so 1.x is not a thing you can target with it.

The ABI is one function. Everything — describe, instantiate, set a parameter, render, destroy — arrives at plugMain as an integer opcode plus a union that is either a number or a pointer.

You do not write that dispatch. The SDK's FFGL.cpp implements plugMain, decodes the opcode and calls a virtual on your class, which derives from CFFGLPlugin. What you write is a subclass and a registration object.

Trap — a static archive can delete your plugin. That CFFGLPluginInfo is a file-scope constructor and is never referenced by name. In a static library the linker is entitled to drop the whole translation unit, and you get a bundle that loads, exports plugMain, and reports that it contains no plugins. Use a CMake OBJECT library. Verify with nm -gU … | grep _plugMain and an actual host load — the symbol check alone passes.

Plugin types and the lifecycle

Three types: FF_EFFECT (0), FF_SOURCE (1), FF_MIXER (2) — a filter, a generator, and a two-input blend. Input count is declared with SetMinInputs / SetMaxInputs.

The opcodes, in the order a host uses them:

• FF_GET_INFO (0) and FF_GET_EXTENDED_INFO (13) — name, ID, type, version, description.
• FF_INITIALISE_V2 (34) / FF_DEINITIALISE (2) — once per library load.
• FF_GET_NUM_PARAMETERS (4) and the parameter queries, at describe time.
• FF_INSTANTIATE_GL (18) → your InitGL, with an FFGLViewportStruct.
• FF_SET_PARAMETER (8), FF_SET_TIME (20), FF_SET_BEATINFO (38), FF_RESIZE (23) — per frame or on change.
• FF_PROCESS_OPENGL (17) → your ProcessOpenGL.
• FF_DEINSTANTIATE_GL (19) → your DeInitGL.

Rendering

You are handed the host's GL context, already current, and a list of input textures. You draw. There is no image to allocate and no buffer to return — whatever you leave in the currently bound framebuffer is the output.

Trap — the input texture is bigger than the picture. HardwareWidth can exceed Width, and the padding is undrawn garbage. A filter that samples where it was told never notices; a warp samples wherever it likes, so it does. Do geometry in picture space 0–1, apply MaxUV only at the fetch, and keep every fetch half a texel inside — at the picture edge GL_LINEAR takes half its weight from the padding.

Trap — ScopedFBOBinding restores the framebuffer and not the viewport. Every pass's ResizeViewPort() leaks into the next one, and a final pass drawing to the host's framebuffer has no buffer of its own to size itself from. Capture glGetIntegerv( GL_VIEWPORT ) at the top of ProcessOpenGL and restore it. The symptom does not look like a viewport bug: the effect renders correctly into a corner of the frame, and in any viewer that shows transparency as white it reads as the effect blowing out to solid white.

Trap — every ffglex::Scoped* binding clears to 0 on scope exit; it does not restore. FFGLFBO::Initialise sizes its new colour texture under one of those, so allocating a buffer silently unbinds your input texture from the active unit. Correct on every frame except the one that allocates — so one wrong frame after load, and one each time a size drag reallocates. Save and restore GL_TEXTURE_BINDING_2D around allocation. (Also: FFGLFBO::Release() leaks its colour texture — it tests depthBufferID twice where it plainly meant colorTextureID.)

Parameters

A parameter is identified by index, not by name, and that index is what a saved composition stores. Renaming a parameter is safe; reordering one is not. Add new parameters at the end of the enum, always.

FF_TYPE_STANDARD is a 0–1 float and is what most controls should be. RED/GREEN/BLUE/ALPHA and HUE/SATURATION/BRIGHTNESS are how a host knows to gather three adjacent parameters into one colour picker. FF_TYPE_OPTION is a dropdown, its entries supplied through FF_GET_PARAMETER_ELEMENT_NAME. FF_TYPE_EVENT is a button. FF_TYPE_BUFFER with FF_USAGE_FFT is how the host hands you an audio spectrum, which is the whole basis of an audio-reactive plugin.

Trap — a ranged FF_TYPE_STANDARD cannot have a ranged default. SetParamInfo clamps the default into 0–1 before returning, SetParamRange can only be called afterwards because it finds the parameter by ID, and there is no SetParamDefault. So a STANDARD parameter declared in degrees cannot declare a default in degrees — 90 silently becomes 1. Keep those 0–1 and map to physical units internally. The clamp is guarded by if( pType == FF_TYPE_STANDARD ) and nothing else, so an FF_TYPE_INTEGER default does pass through untouched and real typed integers with real ranges do work — which matters the moment a control means "5 columns" rather than "about sixty".

Trap — Resolume truncates every parameter name to 16 characters, and nothing plugin-side ever notices. FF_GET_PARAMETER_NAME gives the host a 16-character, non-null-terminated buffer, but the SDK's GetParamName returns a pointer to the full std::string and the host copies 16. Your offline harness passes, your --list output is right, the OpenFX build shows the full name, and only the host is wrong. Six shipped plugins here displayed Background Opaci before a release gate caught it. Worse: a 16-character name is the only signal, so the host cannot tell "truncated" from "fits exactly". Group names (FF_GET_PARAM_GROUP) and display names (FF_GET_PARAM_DISPLAY_NAME) write into a host-sized buffer and are not limited.

Trap — a display-only TEXT parameter can make the plugin refuse to instantiate. The SDK's instantiateGL sets every parameter's default on the fresh instance and deletes the instance if any set returns FF_FAIL — and the base CFFGLPlugin::SetTextParameter is a stub returning FF_FAIL. Declare a text line and override only GetTextParameter, and no real host can instantiate the plugin at all. It is invisible to any harness that calls the plugin class directly and bypasses plugMain. Always override SetTextParameter to return FF_SUCCESS.

Time, and the worst trap on this page

FF_SET_TIME hands you a double. The API does not say what unit it is in, and hosts disagree. Resolume sends milliseconds — measured live at 20.0 per frame at its 50fps, and the SDK's own Particles sample divides by 1000. Every offline harness, and any std::chrono::steady_clock fallback, sends seconds. A plugin that consumes it raw is a thousand times too fast in the only host that matters and correct everywhere you test it.

The user-visible signature is a rate control that reads as broken rather than fast: dead at the very bottom of its travel, pinned at maximum everywhere above. Read a rate-control bug report as a clock-unit bug until proven otherwise.

Do not infer the unit from the magnitude of the first frame delta — the obvious heuristic, and it has three holes: while undecided it has to guess and the wrong guess is the 1000× one; deltas between 0.5 and 2.0 never decide at all; and a sub-millisecond burst at load, which is exactly what a thumbnail render on a fast GPU looks like, locks it to "seconds" permanently. Measure instead. Ask steady_clock how much real time passed, ask the host how much host time passed, and let the ratio name the unit; take several agreeing frames before settling, and fail safe to the wall clock while undecided.

Trap — CFFGLPlugin::hostTime is uninitialised. The constructor initialises bpm and barPhase and nothing else; double hostTime; is a bare member. Reading it before the host's first SetTime reads garbage. Shadow it with your own double hostTime = -1.0; plus a SetTime override, or gate on a hostTimeSeen flag.

Trap — phase = clock * speed teleports when Speed changes. The phase moves by clock * delta the instant the control moves, and clock is however long the composition has been open. An hour in, a small nudge is worth hundreds of cycles and every instance jumps. Unusable live, and it reads as the animation "restarting from a specific position". Anchor the phase instead: keep phaseAnchor and anchorClock, and on a speed change do phaseAnchor += (clock - anchorClock) * oldSpeed; anchorClock = clock; — once per change, not per frame. Beat and Bar sync must keep jumping; their contract is that phase 0 lands on the bar line.

FF_SET_BEATINFO is separate and much better behaved: a SetBeatinfoStruct of float bpm and float barPhase, straight off the host's transport. That is the right input for anything that should land on a beat.

Shipping one

macOS gets a .bundle, ideally universal; Windows a .dll; Linux a .so. Resolume looks in its own folder — on macOS ~/Documents/Resolume Arena/Extra Effects — and there is no registry, no manifest and no signing requirement beyond the platform's own.

Trap — the GLSL will bite you silently. A uniform name that does not match the C++ makes glGetUniformLocation return −1, and glUniform(-1) is a documented no-op — so a control is stone dead while everything compiles, links, loads and renders. ffglex::FFGLShader::Set has no integer-vector overload, so Set( name, someInt, someInt ) silently resolves to (float,float) and issues a glUniform2f against your ivec2 — a GL_INVALID_OPERATION that leaves the uniform at zero. And layout, filter, input, output, sample and common are all GLSL keywords. Only a parameter sweep — render, move one control, assert the pixels changed — catches any of these.

OpenFX

OpenFX is an actual open standard rather than one vendor's SDK, BSD-3 licensed, and it is the reason one effect can appear in DaVinci Resolve, Fusion, Nuke, Natron, Vegas, Flame and Baselight from one binary. The API string is "OfxImageEffectPluginAPI" at version 1; the current specification is 1.4.

It is a C ABI with three exported symbols, and everything past that is property sets: string-keyed bags of values, fetched and set through suites you ask the host for by name and version. That indirection is what lets the standard grow without breaking an ABI, and it is also why raw OFX is verbose enough that almost nobody writes it directly.

pluginIdentifier is the plugin's identity forever. A host stores it in the project file. Change it and every existing timeline loses the effect; keep it and you can rename and re-version freely.

The one thing to internalise

Render is called with an arbitrary time, in an arbitrary order, on an arbitrary thread, possibly several at once. A user scrubs backwards. Resolve renders a cache in whatever order suits it. There is no "previous frame" unless you declare setTemporalClipAccess( true ) and go and fetch one, and even then you are fetching, not remembering.

The practical consequence is that an OFX plugin should be a pure function of position and time. Anything in an FFGL plugin that accumulates — a feedback buffer, an integrated phase, a particle sim — either becomes closed-form arithmetic, or gets reconstructed each frame from a bounded window of temporal clip access, or is honestly declared FFGL-only. All three are legitimate; picking one silently is not. Declare your thread safety truthfully with setRenderThreadSafety: eRenderFullySafe means the host may call you concurrently on the same instance.

Contexts

A plugin declares which contexts it supports, and is described again per context. eContextFilter is one input and one output. eContextGenerator has no input. eContextTransition has two inputs plus a mix. eContextGeneral is the catch-all a host uses when it does not care. eContextPaint and eContextRetimer exist and are rarer. A filter that can also generate declares both, and gets asked twice.

Images, clips and parameters

A clip is fetched by name — kOfxImageEffectSimpleSourceClipName is "Source", kOfxImageEffectOutputClipName is "Output" — and an image is fetched from a clip at a time. An image gives you a pixel pointer, a bounds rectangle, a row-bytes stride and a pixel depth. Handle all three depths you declared: eBitDepthUByte, eBitDepthUShort, eBitDepthFloat. Row bytes can be negative — some hosts hand you a bottom-up image — and bounds are not the region of definition, so never index from (0,0).

setSupportsTiles( false ) is the right answer for any effect whose output pixel depends on distant input pixels. Say true and the host is entitled to ask you to render a 64-pixel strip out of the middle, with only that strip of input.

Parameters are strongly typed — Double, Int, Boolean, Choice, RGB/RGBA, Double2D, String, PushButton, Group, Page — and each has a script name that identifies it forever and a label you can change. They are keyframable by default, so read them with getValueAtTime( args.time ) and never with getValue() from inside a render.

Unlike FFGL, ranges are real: setRange is the hard limit, setDisplayRange is what the slider spans, and defaults are in the parameter's own units. There is no 16-character limit on a label either — which is its own hazard, because renaming an FFGL parameter to fit and leaving the OFX label long gives users two different names for one control across two hosts.

The C++ Support library

The SDK ships a C++ layer — ofxsImageEffect.h, ofxsParam.h, ofxsProcessing.h — that turns the property-set API into classes and turns actions into virtuals: describe, describeInContext, createInstance, render, changedParam, isIdentity. Use it. Nothing is lost — the handles are still there when you need to reach past it — and OFX::ImageProcessor gets you host-managed multithreading for free.

Trap — do not leave an exit-time destructor in the module. Register your factory as a deliberately leaked heap allocation in getPluginIDs. At exit(), __cxa_finalize runs each dlopened plugin module's finalizers before the host's own globals, so a by-value static destroys the OfxPlugin struct the host's cache still points at, and the process dies at PC=0 with an unwindable single-frame backtrace. Exit code 139 after correct render output is the signature. Raw-C plugins survive it, which makes it look plugin-specific; it is not.

Trap — isIdentity is not an optimisation, it is a contract. Return true when your parameters mean "do nothing" and the host skips you entirely. Get it wrong in the other direction — return true when you would in fact have changed the picture — and the effect silently vanishes at some parameter values.

The bundle

An OFX plugin is a directory with a fixed layout. It is not a macOS bundle in the CMake sense, so you assemble it by hand.

Trap — Linux, and the four green builds. If any transitive dependency starts a thread you must link Threads::Threads explicitly: pthread_create only moved into libc in glibc 2.34, and without it the plugin compiles, links, passes a symbol check, passes a glibc-version check, and then fails at dlopen with undefined symbol: pthread_create. macOS and Windows never show it. Resolve on Linux runs on an EL8-family distribution, so build against that glibc floor rather than your desktop's.

Trap — Windows import-library races. If your FFGL and OFX targets share an output base name, MSVC's .exp/.lib for both land in the same default directory and a parallel build races on the file (LNK1104). Pin ARCHIVE_OUTPUT_DIRECTORY separately — and pin the *_OUTPUT_DIRECTORY_RELEASE variants too, or the Release configuration ignores the ones you set.

Pixera RenderBridge

Status: read, mocked, not run in Pixera. Everything in this section comes from the public SDK headers and from executing an extension against a host written here on macOS. Nothing below has been near a copy of Pixera, and the questions that matter most — whether engine pixels can reach an extension at all, and whether OpenGL can import a D3D11 shared texture on real hardware — are open. Treat this as a map of the mechanism, not a report from the field.

RenderBridge is Pixera's documented extension mechanism, MIT licensed and public at avstumpfl/render-bridge-sdk. An implementor publishes a layer parameter structure and receives API invocations at render time. It runs in an isolated process, so a crash cannot take the media server down — which, for a machine with a show on it, is the entire argument. Pixera's own Notch and NDI integrations are RenderBridge extensions.

The SDK ships RX.exe, a minimal host engine. Extensions are therefore developable and testable without Pixera, which is worth more than it sounds: the alternative on plugin work of this kind is a full application restart per diagnosis.

Note the shape: no inheritance, no COM, no opcode dispatch. Each interface is a plain struct whose first member is the object pointer's own vtable-by-hand. It is a C ABI wearing C++ types (std::string, std::vector, std::function cross the boundary), which means host and extension must be built with a compatible standard library — a real constraint, not a detail.

Textures cross by share handle

This is the finding that makes the whole thing viable for an existing OpenGL renderer. The HandleType values are not invented; they are the GL_HANDLE_TYPE_*_EXT constants from GL_EXT_memory_object_win32, verbatim. Format values are VK_FORMAT numbers. Synchronisation is a shared binary or timeline semaphore.

So an OpenGL renderer can import what the engine exports, and there is no encode, no network hop and no second application to launch. Compare the alternative that already works today — route the source out over NDI, run the effect chain in another application, bring it back — which is a network round trip and a set of NDI names to wire, with the parameters living somewhere other than the layer they affect.

Parameters, and the one that is time

Parameters are typed and carry metadata as named string properties. The important one is purpose: a value the engine binds, rather than something the user sets.

purpose: TimelineTime is what makes an animated plugin possible here at all. Pixera's plain .glsl custom effect has no iTime; the vendor's own Shadertoy tutorial says to add a slider and keyframe it. A RenderBridge extension gets the transport.

The shader ABI is shared with that standalone .glsl effect: an extension can hand the engine GLSL through a shader_source property, in the same dialect a hand-written Pixera effect uses. A .glsl generator is therefore a subset of this work rather than a separate project.

Trap — TextureRef's pointer constructor does not acquire. explicit RefBase(T* p) : p(p), with no acquire — so the host is handing over a reference it already owns. Release it as well and you free a texture the stream still holds. The segfault lands several frames later inside unrelated code and looks nothing like a refcount bug; everything downstream stays correct right up to the crash, which makes it harder, not easier.

Trap — a guest process that dies produces an empty log. Redirected stdout is fully buffered, so an extension that crashes in a callback prints nothing at all despite having got most of the way through. Put std::cout << std::unitbuf in anything that might die inside a host callback. On Windows, note also that <GL/gl.h> is OpenGL 1.1 — no GLuint64, so the EXT_memory_object entry-point typedefs will not compile without either a loader or one local typedef.

One model, three hosts

Nineteen plugins here ship to both FFGL and OpenFX from one repository, and the pattern that made that survivable is worth stating on its own.

• The effect is a pure function of position and time, by design rather than by discovery. Write it that way from the start and the OFX build is a port; write it stateful and the OFX build is a rewrite. State the constraint in the header of the file that would otherwise accumulate.
• The model lives on the CPU, once. Shared cores link straight into both targets from the same source. Only the GPU-side machinery is mirrored, with an edit-both comment at each mirrored site. What must not happen is two implementations of the same arithmetic drifting.
• Where maths is mirrored, change the contract rather than the expression. One effect here had its rain model in C++, in GLSL and in a browser demo. Rather than rewrite one expression three times, the shared state stopped carrying time and started carrying travel — the position reached — so all three copies just read it. Deleting the old field rather than leaving it unused is what makes that safe: every call site surfaces at compile time.
• A stateful control that OFX cannot honour should say so, not be faked. Reconstructing a feedback buffer from a decay-bounded window of temporal clip access works and is documented in the guides that use it — but a plugin description that says "this control is FFGL-only" is better than a control that quietly means something different in Resolve.
• Test the number, never the rendered frame. A wrapped value hides a jump: a roll through fract, a sprite sheet that loops, a glitch that is random either way — all render identically before and after a bug. Read the value on both sides of the change.

There is also a bridge worth knowing exists before you write an adapter: resolume-ofx-bridge wraps an FFGL bundle as a self-contained OFX plugin and the reverse, so "does this need porting at all?" is a question with a third answer.

Where this came from

The FFGL constants, structs and opcode numbers were read out of FFGL.h at the SDK revision this fleet pins (b1afaf9). The OpenFX entry points and the OfxPlugin struct came from ofxCore.h in the SDK subset vendored per repo. The RenderBridge types, enums and property names came from rxext.h and rxext_util.h. None of it is transcribed from memory or from a tutorial.

Every trap on this page cost time in a plugin that shipped. The 16-character truncation was found by a release gate after six plugins had already shipped the wrong label. The clock unit was found by an external bug report reading "0.01 stationary, 0.02 super fast". The teardown crash was found at exit code 139 after a correct render. They are here because none of them are in any SDK's documentation.

The plugins themselves are on the video plugins page, and they are all open source — an existing FFGL and OpenFX build of the same effect is often the fastest way to see how a thing is done twice.

Corrections are welcome and wanted, particularly on the RenderBridge section: get in touch.