Java has a support to media playback through it’s Java Media Framework, which unfortunately, on it’s current version (2.1.1e) does not support many formats for video playback.

So I quickly looked for alternatives, including IBM Toolkit for mpeg4, that hadn’t a sufficient production performance I was looking for, and didn’t offer an easy option for frame grabbing or plugin extensions as JMF does.

Next was Fobs4JMF, which is JMF + ffmpeg. This solution was much more interesting, since it offers a wide variety of codecs (ogg, mp3, m4a, divx, xvid, h264, mov, etc) and is based on the solid ffmpeg solution to decode audio and video.

My implementation, uses the plug-in capabilities of JMF to extend a custom renderer that does a pixel type conversion and rendering to a texture:

This custom renderer works with RGB textures, a type I seemed to made work on my two test machines:

• MacBook with a Integ GMA x3100 – Leopard;
• PC with a Radeon x600 – Debian.

You might wanna try different pixel types to increase the performance on different target machines.

First, lets describe how the Renderer works:

It got to be an implementation of a javax.media.renderer.VideoRenderer since it will be installed as a plugin on JMF.

For the different methods we need to implement, there are a few we need to take proper care of:

• process: this is the method JMF calls passing the movie’s current frame buffer, here we process the buffer so that we can latter render it in OpenGL;
• getSupportedInputFormats: we return RGBFormat, our target texture format;
• setInputFormat: here we simply tell JMF that the format it chooses is the one we want. Since RGB was the only one we returned as supported, there is not much to do here as well.
• getName: returns the renderer neat name!

Next we need a way to access this renderer outside of the JMF world, so that we can get the texture to render it on the teapot. For this purpose our class must also be a javax.media.Control, then we can easily get it through an getControl call, such as:

player.getControl(“javax.media.renderer.VideoRenderer”);

So we implement:

• getControl: returns it’s instance;
• getControls: returns an array containing only it’s instance as a valid control.

The renderer implementation is org.pirelenito.multimedia.jmf.plugin.RGBGLTextureRenderer.

And also, to make further development easy, there is an IGLTextureRenderer interface with the public methods called by the Canvas:

• render: that plots the buffer on the texture surface;
• getTexture: to retrieve the texture instance.

Last but not least important, you will need to register the renderer on JMF, this is done through the JMFRegistry application. The easiest way to start it is inside Eclipse, where our custom renderer is already on the class path:

• Create a new run configuration;
• Set the main class as: JMFRegistry;
• Start it, and go to the Plugins tab, then Renderer;
• Add the org.pirelenito.multimedia.jmf.plugin.RGBGLTextureRenderer;
• Move it to the top of the list;
• Push Commit, and you are good to go!

To test it out there is also a helper class to instantiate the movie player, and a Main class which is an OpenGL canvas used to render the teapot with the texture on its surface.

I am not very experienced with OpenGL, so there might be more effitient ways to do this using, for instance, PBO (Pixel Buffer Object). If you have any question or suggestions on how to improve this solution, don’t hesitate on drooping a comment!

Here is the project’s repository. You will need to download a few dependencies, so check the Readme file for more information.

Update: I wasn’t properly initializing the Texture. and since I want to add support for more pixel formats, it is easy for you to checkout the updated Eclipse project at: http://labs.pirelenito.org/experiments/svn/java/MovieGL/ latter this afternoon now.

Update 2: Added the repository link on the image above.

Update 3: I’ve migrated my server to a new location, and the Subversion wasn’t up until today. Sorry.

Update 4: Small review on the post.

Update 5: Moved the repository to github.

Cheers!