The Image Effect API

Introduction

In general, image effects plug-ins take zero or more input clips and produce an output clip. So far so simple, however there are many devils hiding in the details. Several supporting suites are required from the host and the plug-in needs to respond to a range of actions to work correctly. How an effect is intended to be used also complicates the issue, forcing sets of behaviours depending on the context of an effect.

Plug-ins that implement the image effect API set the pluginApi member of the OfxPlugin struct returned by the global OfxGetPlugin to be:

kOfxImageEffectPluginApi

String used to label OFX Image Effect Plug-ins.

Set the pluginApi member of the OfxPluginHeader inside any OfxImageEffectPluginStruct to be this so that the host knows the plugin is an image effect.

The current version of the API is 1. This is enough to label the plug-in as an image effect plug-in.

Image Effect API Header Files

The header files used to define the OFX Image Effect API are…

  • ofxCore.h Provides the core definitions of the general OFX architecture that allow the bootstrapping of specific APIs, as well as several core actions,

  • ofxProperty.h

    Provides generic property fetching suite used to get and set values about objects in the API,

  • ofxParam.h Provides the suite for defining user visible parameters to an effect

  • ofxMultiThread.h Provides the suite for basic multi-threading capabilities

  • ofxInteract.h Provides the suite that allows a plug-in to use OpenGL to draw their own interactive GUI tools

  • ofxKeySyms.h Provides key symbols used by ‘Interacts’ to represent keyboard events

  • ofxMemory.h Provides a simple memory allocation suite,

  • ofxMessage.h Provides a simple messaging suite to communicate with an end user

  • ofxImageEffect.h Defines a suite and set of actions that draws all the above together to create an visual effect plug-in.

  • ofxDrawSuite.h Provides an optional suite that allows a plug-in to draw their own interactive GUI tools without using OpenGL

These contain the suite definitions, property definitions and action definitions that are used by the API.

Actions Used by the API

All image effect plug-ins have a main entry point. This is used to trap all the standard actions used to drive the plug-in. They can also have other optional entry points that allow the plug-in to create custom user interface widgets. These interact entry points are specified during the two description actions.

The following actions can be passed to a plug-in’s main entry point…

  • The Generic Load Action called just after a plug-in is first loaded,

  • The Generic Unload Action called just before a plug-in is unloaded,

  • The Generic Describe Action called to describe a plug-in’s behaviour to a host,

  • The Generic Create Instance Action called just after an instance is created,

  • The Generic Destroy Instance Action called just before an instance is destroyed,

  • The Generic Begin/End Instance Changed Actions , a pair of actions used to bracket a set of Instance Changed actions,

  • The Generic Instance Changed Action an action used to indicate that a value has changed in a plug-in instance,

  • The Generic Purge Caches Action called to have the plug-in delete any temporary private data caches it may have,

  • The Sync Private Data Action called to have the plug-in sync any private state data back to its data set,

  • The Generic Begin/End Instance Edit Actions a pair of calls that are used to bracket the fact that a user interface has been opened on an instance and it is being edited,

  • The Begin Sequence Render Action where a plug-in is told that it is about to render a sequence of images,

  • The Render Action where a plug-in is told that it is to render an output image,

  • The End Sequence Render Action where a plug-in is told it has finished rendering a sequence of images,

  • The Describe In Context Action used to have a plug-in describe itself in a specific context,

  • The Get Region of Definition Action where an instance gets to state how big an image it can create,

  • The Get Regions Of Interest Action where an instance gets to state how much of its input images it needs to create a give output image,

  • The Get Frames Needed Action where an instance gets to state how many frames of input it needs on a given clip to generate a single frame of output,

  • The Is Identity Action where an instance gets to state that its current state does not affect its inputs, so that the output can be directly copied from an input clip,

  • The Get Clip Preferences Action where an instance gets to state what data and pixel types it wants on its inputs and will generate on its outputs,

  • The Get Time Domain Action where a plug-in gets to state how many frames of data it can generate.

Main Objects Used by the API

The image effect API uses a variety of different objects. Some are defined via blind data handles, others via property sets, and some by a combination of the two. These objects are…

  • Host Descriptor - a descriptor object used by a host to describe its behaviour to a plug-in,

  • Image Effect Descriptor - a descriptor object used by a plug-in to describe its behaviour to a host,

  • Image Effect Instance - an instance object maintaining state about an image effect,

  • Clip Descriptor - a descriptor object for a sequence of images used as input or output a plug-in may use,

  • Clip Instance - a instance object maintaining state about a sequence of images used as input or output to an effect instance,

  • Parameter Descriptor - a descriptor object used to specify a user visible parameter in an effect descriptor,

  • Parameter Instance - an instance object that maintains state about a user visible parameter in an effect instance,

  • Parameter Set Descriptor - a descriptor object used to specify a set of user visible parameters in an effect descriptor,

  • Parameter Set Instance - an instance object that maintains state about a set of user visible parameters in an effect instance,

  • Image Instance - a instance object that maintains state about a 2D image being passed to an effect instance.

  • Interact Descriptor - which describes a custom openGL user interface, for example an overlay over the inputs to an image effect. These have a separate entry point to an image effect.

  • Interact Instance - which holds the state on a custom openGL user interface. These have a separate entry point to an image effect.

Host Descriptors

The host descriptor is represented by the properties found on the host property set handle in the OfxHost struct. The complete set of read only properties are found in the section Properties on the Image Effect Host.

These sets of properties are there to describe the capabilities of the host to a plug-in, thus giving a plug-in the ability to modify its behaviour depending on the capabilities of the host.

A host descriptor is valid while a plug-in is loaded.

Effects

An effect is an object in the OFX Image Effect API that represents an image processing plug-in. It has associated with it a set of properties, a set of image clips and a set of parameters. These component objects of an effect are defined and used by an effect to do whatever processing it needs to. A handle to an image effect (instance or descriptor) is passed into a plug-in’s main entry point handle argument:

typedef struct OfxImageEffectStruct *OfxImageEffectHandle

Blind declaration of an OFX image effect.

The functions that directly manipulate an image effect handle are specified in the OfxImageEffectSuiteV1 found in the header file ofxImageEffect.h.

Effect Descriptors

An effect descriptor is an object of type OfxImageEffectHandle passed into an effect’s main entry point handle argument. The two actions it is passed to are:

A effect descriptor does not refer to a ‘live’ effect, it is a handle which the effect uses to describe itself back to the host. It does this by setting a variety of properties on an associated property handle, and specifying a variety of objects (such as clips and parameters) using functions in the available suites.

Once described, a host should cache away the description in some manner so that when an instance is made, it simply looks at the description and creates the necessary objects needed by that instance. This stops the overhead of having every instance be forced to describe itself over the API.

Effect descriptors are only valid in a effect for the duration of the instance they were passed into.

The properties on an effect descriptor can be found in the section Properties on an Effect Descriptor.

Effect Instances

A effect instance is an object of type OfxImageEffectHandle passed into an effect’s main entry point handle argument. The handle argument should be statically cast to this type. It is passed into all actions of an image effect that a descriptor is not passed into.

The effect instance represents a ‘live’ instance of an effect. Because an effect has previously been described, via a effect descriptor, an instance does not have to respecify the parameters, clips and properties that it needs. These means, that when an instance is passed to an effect, all the objects previously described will have been created.

Generally multiple instances of an effect can be in existence at the same time, each with a different set of parameters, clips and properties.

Effect instances are valid between the calls to kOfxActionCreateInstance and kOfxActionDestroyInstance, for which it is passed as the handle argument.

The properties on an effect instance can be found in the section Properties on an Effect Instance.

Clips

A clip is a sequential set of images attached to an effect. They are used to fetch images from a host and to specify how a plug-in wishes to manage the sequence.

Clip Descriptors

Clip descriptors are returned by the OfxImageEffectSuiteV1::clipDefine() function. They are used during the kOfxActionDescribe action by an effect to indicate the presence of an input or output clip and how that clip behaves.

A clip descriptor is only valid for the duration of the action it was created in.

The properties on a clip descriptor can be found in the section Properties on a Clip Descriptor.

Clip Instances

typedef struct OfxImageClipStruct *OfxImageClipHandle

Blind declaration of an OFX image effect.

Clip instances are returned by the OfxImageEffectSuiteV1::clipGetHandle() function. They are are used to access images and and manipulate properties on an effect instance’s input and output clips. A variety of functions in the OfxImageEffectSuiteV1 are used to manipulate them.

A clip instance is valid while the related effect instance is valid.

The properties on a clip instance can be found in the section Properties on a Clip Instance.

Parameters

Parameters are user visible objects that an effect uses to specify its state, for example a floating point value used to control the blur size in a blur effect. Parameters (both descriptors and instances) are represented as blind data handles of type:

typedef struct OfxParamStruct *OfxParamHandle

Blind declaration of an OFX param.

Parameter sets are the collection of parameters that an effect has associated with it. They are represented by the type OfxParamSetHandle. The contents of an effect’s parameter set are defined during the kOfxImageEffectActionDescribeInContext action. Parameters cannot be dynamically added to, or deleted from an effect instance.

Parameters can be of a wide range of types, each of which have their own unique capabilities and property sets. For example a colour parameter differs from a boolean parameter.

Parameters and parameter sets are manipulated via the calls and properties in the OfxParameterSuiteV1 specified in ofxParam.h. The properties on parameter instances and descriptors can be found in the section Properties on Parameter Descriptors and Instances.

Parameter Set Descriptors

Parameter set descriptors are returned by the :cpp:func`OfxImageEffectSuiteV1::getParamSet` function. This returns a handle associated with an image effect descriptor which can be used by the parameter suite routines to create and describe parameters to a host.

A parameter set descriptor is valid for the duration of the kOfxImageEffectActionDescribeInContext action in which it is fetched.

Parameter Descriptors

Parameter descriptors are returned by the OfxParameterSuiteV1::paramDefine() function. They are used to define the existence of a parameter to the host, and to set the various attributes of that parameter. Later, when an effect instance is created, an instance of the described parameter will also be created.

A parameter descriptor is valid for the duration of the kOfxImageEffectActionDescribeInContext action in which it is created.

Parameter Set Instances

Parameter set instances are returned by the OfxImageEffectSuiteV1::getParamSet() function. This returns a handle associated with an image effect instance which can be used by the parameter suite routines to fetch and describe parameters to a host.

A parameter set handle instance is valid while the associated effect instance remains valid.

Parameter Instances

Parameter instances are returned by the OfxParameterSuiteV1::paramGetHandle() function. This function fetches a previously described parameter back from the parameter set. The handle can then be passed back to the various functions in the OfxParameterSuite1V to manipulate it.

A parameter instance handle remains valid while the associated effect instance remains valid.

Image Instances

An image instance is an object returned by the OfxImageEffectSuiteV1::clipGetImage() function. This fetches an image out of a clip and returns it as a property set to the plugin. The image can be accessed by looking up the property values in that set, which includes the data pointer to the image.

An image instance is valid until the effect calls OfxImageEffectSuiteV1::clipReleaseImage() on the property handle. The effect must release all fetched images before it returns from the action.

The set of properties that make up an image can be found in the section Properties on an Image.

Interacts

An interact is an OFX object that is used to draw custom user interface elements, for example overlays on top of a host’s image viewer or custom parameter widgets. Interacts have their own main entry point, which is separate to the effect’s main entry point. Typically an interact’s main entry point is specified as a pointer property on an OFX object, for example the kOfxImageEffectPluginPropOverlayInteractV1 property on an effect descriptor.

The functions that directly manipulate interacts are in the OfxInteractSuiteV1 found in the header file ofxInteract.h , as well as the properties and specific actions that apply to interacts.

Interact Descriptors

Interact descriptors are blind handles passed to the kOfxActionDescribeInteract sent to an interact’s separate main entry point. They should be cast to the type OfxInteractHandle.

The properties found on a descriptor are found in section Properties on Interact Descriptors.

Interact Instances

Interact instances are blind handles passed to all actions but the kOfxActionDescribe sent to an interact’s separate main entry point. They should be cast to the type

typedef struct OfxInteract *OfxInteractHandle

Blind declaration of an OFX interactive gui.

The properties found on an instance are found in section Properties on Interact Instance.