File ofxGPURender.h

This file contains an optional suite for performing GPU-accelerated
rendering of OpenFX Image Effect Plug-ins.  For details see
\ref ofxGPURender.

It allows hosts and plug-ins to support OpenGL, OpenCL, CUDA, and Metal.
Additional GPU APIs, such a Vulkan, could use similar techniques.

StatusReturnValues

OfxStatus returns indicating that a OpenGL render error has occurred:

  • If a plug-in returns kOfxStatGLRenderFailed, the host should retry the render with OpenGL rendering disabled.

  • If a plug-in returns kOfxStatGLOutOfMemory, the host may choose to free resources on the GPU and retry the OpenGL render, rather than immediately falling back to CPU rendering.

kOfxStatGPUOutOfMemory

GPU render ran out of memory.

kOfxStatGLOutOfMemory

OpenGL render ran out of memory (same as kOfxStatGPUOutOfMemory)

kOfxStatGPURenderFailed

GPU render failed in a non-memory-related way.

kOfxStatGLRenderFailed

OpenGL render failed in a non-memory-related way (same as kOfxStatGPURenderFailed)

Defines

__OFXGPURENDER_H__
kOfxOpenGLRenderSuite

The name of the OpenGL render suite, used to fetch from a host via OfxHost::fetchSuite.

kOfxImageEffectPropOpenGLRenderSupported

Indicates whether a host or plug-in can support OpenGL accelerated rendering.

  • Type - C string X 1

  • Property Set - plug-in descriptor (read/write), host descriptor (read only) - plug-in instance change (read/write)

  • Default - “false” for a plug-in

  • Valid Values - This must be one of

    • ”false” - in which case the host or plug-in does not support OpenGL accelerated rendering

    • ”true” - which means a host or plug-in can support OpenGL accelerated rendering, in the case of plug-ins this also means that it is capable of CPU based rendering in the absence of a GPU

    • ”needed” - only for plug-ins, this means that an plug-in has to have OpenGL support, without which it cannot work.

V1.4: It is now expected from host reporting v1.4 that the plug-in can during instance change switch from true to false and false to true.

kOfxOpenGLPropPixelDepth

Indicates the bit depths supported by a plug-in during OpenGL renders.

This is analogous to kOfxImageEffectPropSupportedPixelDepths. When a plug-in sets this property, the host will try to provide buffers/textures in one of the supported formats. Additionally, the target buffers where the plug-in renders to will be set to one of the supported formats.

Unlike kOfxImageEffectPropSupportedPixelDepths, this property is optional. Shader-based effects might not really care about any format specifics when using OpenGL textures, so they can leave this unset and allow the host the decide the format.

kOfxImageEffectPropOpenGLEnabled

Indicates that a plug-in SHOULD use OpenGL acceleration in the current action.

When a plug-in and host have established they can both use OpenGL renders then when this property has been set the host expects the plug-in to render its result into the buffer it has setup before calling the render. The plug-in can then also safely use the ‘OfxImageEffectOpenGLRenderSuite’

v1.4: kOfxImageEffectPropOpenGLEnabled should probably be checked in Instance Changed prior to try to read image via clipLoadTexture

Note

Once this property is set, the host and plug-in have agreed to use OpenGL, so the effect SHOULD access all its images through the OpenGL suite.

kOfxImageEffectPropOpenGLTextureIndex

Indicates the texture index of an image turned into an OpenGL texture by the host.

kOfxImageEffectPropOpenGLTextureTarget

Indicates the texture target enumerator of an image turned into an OpenGL texture by the host.

The property set of the following actions should contain this property:

kOfxActionOpenGLContextAttached

Action called when an effect has just been attached to an OpenGL context.

The purpose of this action is to allow a plug-in to set up any data it may need to do OpenGL rendering in an instance. For example…

  • allocate a lookup table on a GPU,

  • create an OpenCL or CUDA context that is bound to the host’s OpenGL context so it can share buffers.

The plug-in will be responsible for deallocating any such shared resource in the kOfxActionOpenGLContextDetached action.

A host cannot call kOfxActionOpenGLContextAttached on the same instance without an intervening kOfxActionOpenGLContextDetached. A host can have a plug-in swap OpenGL contexts by issuing a attach/detach for the first context then another attach for the next context.

The arguments to the action are…

  • handle handle to the plug-in instance, cast to an OfxImageEffectHandle

  • inArgs is redundant and set to NULL

  • outArgs is redundant and set to NULL

A plug-in can return…

  • kOfxStatOK, the action was trapped and all was well

  • kOfxStatReplyDefault, the action was ignored, but all was well anyway

  • kOfxStatErrMemory, in which case this may be called again after a memory purge

  • kOfxStatFailed, something went wrong, but no error code appropriate, the plug-in should to post a message if possible and the host should not attempt to run the plug-in in OpenGL render mode.

kOfxActionOpenGLContextDetached

Action called when an effect is about to be detached from an OpenGL context.

The purpose of this action is to allow a plug-in to deallocate any resource allocated in kOfxActionOpenGLContextAttached just before the host decouples a plug-in from an OpenGL context. The host must call this with the same OpenGL context active as it called with the corresponding kOfxActionOpenGLContextAttached.

The arguments to the action are…

  • handle handle to the plug-in instance, cast to an OfxImageEffectHandle

  • inArgs is redundant and set to NULL

  • outArgs is redundant and set to NULL

A plug-in can return…

  • kOfxStatOK, the action was trapped and all was well

  • kOfxStatReplyDefault, the action was ignored, but all was well anyway

  • kOfxStatErrMemory, in which case this may be called again after a memory purge

  • kOfxStatFailed, something went wrong, but no error code appropriate, the plug-in should to post a message if possible and the host should not attempt to run the plug-in in OpenGL render mode.

kOfxImageEffectPropCudaRenderSupported

Indicates whether a host or plug-in can support CUDA render.

  • Type - string X 1

  • Property Set - plug-in descriptor (read/write), host descriptor (read only)

  • Default - “false” for a plug-in

  • Valid Values - This must be one of

    • ”false” - the host or plug-in does not support CUDA render

    • ”true” - the host or plug-in can support CUDA render

kOfxImageEffectPropCudaEnabled

Indicates that a plug-in SHOULD use CUDA render in the current action.

If a plug-in and host have both set kOfxImageEffectPropCudaRenderSupported=”true” then the host MAY set this property to indicate that it is passing images as CUDA memory pointers.

kOfxImageEffectPropCudaStreamSupported

Indicates whether a host or plug-in can support CUDA streams.

  • Type - string X 1

  • Property Set - plug-in descriptor (read/write), host descriptor (read only)

  • Default - “false” for a plug-in

  • Valid Values - This must be one of

    • ”false” - in which case the host or plug-in does not support CUDA streams

    • ”true” - which means a host or plug-in can support CUDA streams

kOfxImageEffectPropCudaStream

The CUDA stream to be used for rendering.

This property will only be set if the host and plug-in both support CUDA streams.

If set:

  • this property contains a pointer to the stream of CUDA render (cudaStream_t). In order to use it, reinterpret_cast<cudaStream_t>(pointer) is needed.

  • the plug-in SHOULD ensure that its render action enqueues any asynchronous CUDA operations onto the supplied queue.

  • the plug-in SHOULD NOT wait for final asynchronous operations to complete before returning from the render action, and SHOULD NOT call cudaDeviceSynchronize() at any time.

If not set:

  • the plug-in SHOULD ensure that any asynchronous operations it enqueues have completed before returning from the render action.

kOfxImageEffectPropMetalRenderSupported

Indicates whether a host or plug-in can support Metal render.

  • Type - string X 1

  • Property Set - plug-in descriptor (read/write), host descriptor (read only)

  • Default - “false” for a plug-in

  • Valid Values - This must be one of

    • ”false” - the host or plug-in does not support Metal render

    • ”true” - the host or plug-in can support Metal render

kOfxImageEffectPropMetalEnabled

Indicates that a plug-in SHOULD use Metal render in the current action.

If a plug-in and host have both set kOfxImageEffectPropMetalRenderSupported=”true” then the host MAY set this property to indicate that it is passing images as Metal buffers.

kOfxImageEffectPropMetalCommandQueue

The command queue of Metal render.

This property contains a pointer to the command queue to be used for Metal rendering (id<MTLCommandQueue>). In order to use it, reinterpret_cast<id<MTLCommandQueue>>(pointer) is needed.

The plug-in SHOULD ensure that its render action enqueues any asynchronous Metal operations onto the supplied queue.

The plug-in SHOULD NOT wait for final asynchronous operations to complete before returning from the render action.

kOfxImageEffectPropOpenCLRenderSupported

Indicates whether a host or plug-in can support OpenCL Buffers render.

  • Type - string X 1

  • Property Set - plug-in descriptor (read/write), host descriptor (read only)

  • Default - “false” for a plug-in

  • Valid Values - This must be one of

    • ”false” - the host or plug-in does not support OpenCL Buffers render

    • ”true” - the host or plug-in can support OpenCL Buffers render

kOfxImageEffectPropOpenCLSupported

Indicates whether a host or plug-in can support OpenCL Images render.

  • Type - string X 1

  • Property Set - plug-in descriptor (read/write), host descriptor (read only)

  • Default - “false” for a plug-in

  • Valid Values - This must be one of

    • ”false” - in which case the host or plug-in does not support OpenCL Images render

    • ”true” - which means a host or plug-in can support OpenCL Images render

kOfxImageEffectPropOpenCLEnabled

Indicates that a plug-in SHOULD use OpenCL render in the current action.

If a plug-in and host have both set kOfxImageEffectPropOpenCLRenderSupported=”true” or have both set kOfxImageEffectPropOpenCLSupported=”true” then the host MAY set this property to indicate that it is passing images as OpenCL Buffers or Images.

When rendering using OpenCL Buffers, the cl_mem of the buffers are retrieved using kOfxImagePropData. When rendering using OpenCL Images, the cl_mem of the images are retrieved using kOfxImageEffectPropOpenCLImage. If both kOfxImageEffectPropOpenCLSupported (Buffers) and kOfxImageEffectPropOpenCLRenderSupported (Images) are enabled by the plug-in, it should use kOfxImageEffectPropOpenCLImage to determine which is being used by the host.

kOfxImageEffectPropOpenCLCommandQueue

Indicates the OpenCL command queue that should be used for rendering.

This property contains a pointer to the command queue to be used for OpenCL rendering (cl_command_queue). In order to use it, reinterpret_cast<cl_command_queue>(pointer) is needed.

The plug-in SHOULD ensure that its render action enqueues any asynchronous OpenCL operations onto the supplied queue.

The plug-in SHOULD NOT wait for final asynchronous operations to complete before returning from the render action.

kOfxImageEffectPropOpenCLImage

Indicates the image handle of an image supplied as an OpenCL Image by the host.

  • Type - pointer X 1

  • Property Set - image handle returned by clipGetImage

This value should be cast to a cl_mem and used as the image handle when performing OpenCL Images operations. The property should be used (not kOfxImagePropData) when rendering with OpenCL Images (kOfxImageEffectPropOpenCLSupported), and should be used to determine whether Images or Buffers should be used if a plug-in supports both kOfxImageEffectPropOpenCLSupported and kOfxImageEffectPropOpenCLRenderSupported. Note: the kOfxImagePropRowBytes property is not required to be set by the host, since OpenCL Images do not have the concept of row bytes.

kOfxOpenCLProgramSuite

Typedefs

typedef struct OfxImageEffectOpenGLRenderSuiteV1 OfxImageEffectOpenGLRenderSuiteV1

OFX suite that provides image to texture conversion for OpenGL processing.

typedef struct OfxOpenCLProgramSuiteV1 OfxOpenCLProgramSuiteV1

OFX suite that allows a plug-in to get OpenCL programs compiled.

This is an optional suite the host can provide for building OpenCL programs for the plug-in, as an alternative to calling clCreateProgramWithSource / clBuildProgram. There are two advantages to doing this: The host can add flags (such as -cl-denorms-are-zero) to the build call, and may also cache program binaries for performance (however, if the source of the program or the OpenCL environment changes, the host must recompile so some mechanism such as hashing must be used).

struct OfxImageEffectOpenGLRenderSuiteV1
#include <ofxGPURender.h>

OFX suite that provides image to texture conversion for OpenGL processing.

Public Members

OfxStatus (*clipLoadTexture)(OfxImageClipHandle clip, OfxTime time, const char *format, const OfxRectD *region, OfxPropertySetHandle *textureHandle)

loads an image from an OFX clip as a texture into OpenGL

  • clip clip to load the image from

  • time effect time to load the image from

  • format requested texture format (As in none,byte,word,half,float, etc..) When set to NULL, the host decides the format based on the plug-in’s kOfxOpenGLPropPixelDepth setting.

  • region region of the image to load (optional, set to NULL to get a ‘default’ region) this is in the CanonicalCoordinates.

  • textureHandle property set containing information about the texture

An image is fetched from a clip at the indicated time for the given region and loaded into an OpenGL texture. When a specific format is requested, the host ensures it gives the requested format. When the clip specified is the “Output” clip, the format is ignored and the host must bind the resulting texture as the current color buffer (render target). This may also be done prior to calling the kOfxImageEffectActionRender action. If the region parameter is set to non-NULL, then it will be clipped to the clip’s Region of Definition for the given time. The returned image will be at least as big as this region. If the region parameter is not set or is NULL, then the region fetched will be at least the Region of Interest the effect has previously specified, clipped to the clip’s Region of Definition. Information about the texture, including the texture index, is returned in the textureHandle argument. The properties on this handle will be…

With the exception of the OpenGL specifics, these properties are the same as the properties in an image handle returned by clipGetImage in the image effect suite.

Note

Pre:

  • clip was returned by clipGetHandle

  • Format property in the texture handle

Post:

  • texture handle to be disposed of by clipFreeTexture before the action returns

  • when the clip specified is the “Output” clip, the format is ignored and the host must bind the resulting texture as the current color buffer (render target). This may also be done prior to calling the render action.

Return:

  • kOfxStatOK - the image was successfully fetched and returned in the handle,

  • kOfxStatFailed - the image could not be fetched because it does not exist in the clip at the indicated time and/or region, the plug-in should continue operation, but assume the image was black and transparent.

  • kOfxStatErrBadHandle - the clip handle was invalid,

  • kOfxStatErrMemory - not enough OpenGL memory was available for the effect to load the texture. The plug-in should abort the GL render and return kOfxStatErrMemory, after which the host can decide to retry the operation with CPU based processing.

OfxStatus (*clipFreeTexture)(OfxPropertySetHandle textureHandle)

Releases the texture handle previously returned by clipLoadTexture.

For input clips, this also deletes the texture from OpenGL. This should also be called on the output clip; for the Output clip, it just releases the handle but does not delete the texture (since the host will need to read it).

Pre:

  • textureHandle was returned by clipGetImage

Post:

  • all operations on textureHandle will be invalid, and the OpenGL texture it referred to has been deleted (for source clips)

Return:

OfxStatus (*flushResources)()

Request the host to minimize its GPU resource load.

When a plug-in fails to allocate GPU resources, it can call this function to request the host to flush its GPU resources if it holds any. After the function the plug-in can try again to allocate resources which then might succeed if the host actually has released anything.

Pre:

Post:

  • No changes to the plug-in GL state should have been made.

Return:

struct OfxOpenCLProgramSuiteV1
#include <ofxGPURender.h>

OFX suite that allows a plug-in to get OpenCL programs compiled.

This is an optional suite the host can provide for building OpenCL programs for the plug-in, as an alternative to calling clCreateProgramWithSource / clBuildProgram. There are two advantages to doing this: The host can add flags (such as -cl-denorms-are-zero) to the build call, and may also cache program binaries for performance (however, if the source of the program or the OpenCL environment changes, the host must recompile so some mechanism such as hashing must be used).

Public Members

OfxStatus (*compileProgram)(const char *pszProgramSource, int fOptional, void *pResult)

Compiles the OpenCL program.