Recently I had to integrate video playback on my job’s Java OpenGL engine, which uses JOGL.

Java has a support to media playback through it’s Java Media Framework, which unfortunately, on it’s current version (2.1.1e) does not support many formats for video playback.

So I quickly looked for alternatives, including IBM Toolkit for mpeg4, that hadn’t a sufficient production performance I was looking for, and didn’t offer an easy option for frame grabbing or plugin extensions as JMF does.

Next was Fobs4JMF, which is JMF + ffmpeg. This solution was much more interesting, since it offers a wide variety of codecs (ogg, mp3, m4a, divx, xvid, h264, mov, etc) and is based on the solid ffmpeg solution to decode audio and video.

My implementation, uses the plug-in capabilities of JMF to extend a custom renderer that does a pixel type conversion and rendering to a texture:

This custom renderer works with RGB textures, a type I seemed to make work on my two test machines:

• MacBook with a Integ GMA x3100 - Leopard;
• PC with a Radeon x600 - Debian.

You might wanna try different pixel types to increase the performance for different target machines.

First, lets describe how the Renderer works:

It got to be an implementation of a javax.media.renderer.VideoRenderer since it will be installed as a plugin on JMF.

For the different methods we need to implement, there are a few we need to take proper care of:

• process: this is the method JMF calls passing the movie current frame buffer. Here we treat this buffer and leave it in a way we can latter render in OpenGL;
• getSupportedInputFormats: we return RGBFormat, our target texture format;
• setInputFormat: here we simply tell JMF that the format it chooses is the one we want. Since RGB was the only one we returned as supported there is not much to do here as well.
• getName: returns the renderer neat name!

Next we need a way to access this render from the outside of the JMF world, we need the texture so that we can do GL calls with it (teapot). For this purpose our class must also be a javax.media.Control. Then we can easily get it through an getControl call as:

player.getControl(”javax.media.renderer.VideoRenderer”);

So we implement:

• getControl: returns it’s instance;
• getControls: returns an array containing only it’s instance as a valid control.

The renderer implementation is org.pirelenito.multimedia.jmf.plugin.RGBGLTextureRenderer that you can check on the source code attachment bellow.

And also, to make further development easy, there is an IGLTextureRenderer interface with the public methods called by the Canvas:

• render: that plots the buffer on the texture surface;
• getTexture: to retrieve the texture instance.

Last but not least important, you need to register the render on JMF using the JMFRegistry application. The easiest way to do start it is through Eclipse, since we will need our custom render on the class path:

• Create a new run configuration;
• Set the main class as: JMFRegistry;
• Start and go to the Plugins tab then Renderer;
• Add the org.pirelenito.multimedia.jmf.plugin.RGBGLTextureRenderer;
• Move it to the top of the list;
• Push Commit and you are good to go!

To test it out there is also a helper class to instantiate the movie player, and the Main class which is an OpenGL canvas, to render the teapot with the texture on its surface.

I am not very experienced with OpenGL, so there might be more effitient ways to do this using, for instance PBO (Pixel Buffer Object), if you have any question or suggestions on how to improve this, don’t hesitate on drooping a comment!

Here is the Eclipse project. You will need to download a few dependencies, check the Readme for more information.

Update: I wasn’t properly initializing the Texture. and since I want to add support for more pixel formats, it is easy for you to checkout the updated Eclipse project at: http://labs.pirelenito.org/experiments/svn/java/MovieGL/ latter this afternoon now.

Update 2: Added the repository link on the image above.

Cheers!

This entry was posted on Wednesday, August 20th, 2008 at 4:37 pm and is filed under Java. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.