Tuesday, 21 June 2016

qmlglsink - GStreamer and Qt's QML

Windows Support - NEW!!

Just recently, I added the missing piece for adding Windows support for qmlglsink for QtQuick 2 applications (tracked by this bug) with this commit which means that qmlglsink is now usable on all major platforms1. With the list of supported platforms (Android, iOS, OS X, Windows, Linux) for qmlglsink growing, I thought I'd write a little bit about it and why you should use it.

qmlglsink is a GStreamer element that will render a video stream into a QtQuick QML scene. As you can see from this example, qmlglsink does this by providing a GstGLVideoItem QML item that can be placed anywhere in the scene. The GstGLVideoItem is given a unique name to be able to link the element and the QML item. Without this link, the element cannot render the stream anywhere and will fail changing state. In the C++ code, we then retrieve the GstGLVideoItem by name like so. Then we set the pipeline to playing, start Qt's mainloop and watch the show.

Why Use qmlglsink?

The main reason that you should have a look at qmlglsink is if you need the full flexibility of GStreamer across multiple platforms. QtMultimedia only provides a small subset of the features possible with GStreamer and only uses GStreamer on unix systems. Only file-based playback or Buffer-based playback (depending on the backend) is supported by QtMultimedia and it does not come by default with zerocopy support for any platform.

GStreamer on the other hand supports a multitude of media sources from network streaming like rtp and rtsp, file and memory sources as well as recent GStreamer versions containing zerocopy support for Android, iOS2, the RPi, and some support for zerocopy on desktop Linux with VA-API. GStreamer is also extremely flexible and extensible as a multitude of different multimedia tasks are just a plugin away.

1 - Not all platforms are easy to build against Qt. Notably the recent Windows support requires manual intervention with qmake utilizing the added qtplugin.pro file. iOS has been tested against specially modified Qt pkg-config and libtool files.
2 - Unfortunately the iOS zerocopy support is not usable with Qt due to slight incompatibilities between OpenGL|ES 2 and OpenGL|ES 3 on the iOS platform. Qt currently defaults to OpenGL|ES 3 while GStreamer is still choosing OpenGL|ES 2 for zerocopy support. On all other platforms OpenGL ES 2 is compatible with OpenGL ES 3. Edit: This is no longer an issue with this and this commit

1 comment: