This post serves as my first main post relating to the GSoC project I’ve been working on. Earlier I discussed how to set up the development environment.
Recently I got accepted as a student for Google Summer of Code 2017 to work on Mesa to add a new OpenMax IL state tracker that uses tizonia instead of bellagio that the current state tracker uses.
Project link: https://summerofcode.withgoogle.com/projects/#4737166321123328
Progress so far
Most of the effort till now has been to make tizonia compatible with gst-omx so that we can test out the new component.
Changes in gst-omx
The commits are are on my github fork https://github.com/gpalsingh/gst-omx/commits/tizonia
The first thing needed to use gst-omx to test for OMX IL 1.2 was to add tizonia as a target. https://github.com/gpalsingh/gst-omx/commit/0aff53c9f01f04a09b41b01a313afa7255ad20a1 is the last of the three commits to do the same. They simply involve adding tizonia target in addition to bellagio, generic and rpi. Right now it needs external IL headers unlike other targets that can fall back to internal headers in case no external IL headers are available.
Next we worked on making it work with the mp3 decoder component that comes with tizonia. Luckily that needed only small change to ignore an error https://github.com/gpalsingh/gst-omx/commit/2f8be06754077847c3f0a0862e6336fe37907e5e
Right now work on getting the tizonia vp8 decoder to work with gst-omx is in progress. It is possible to play the video but there are still some improvements to be made.
Changes to tizonia
Since tizonia is more aligned to audio cases it is was still missing and AVC port. As Juan suggested in our discussion here https://github.com/tizonia/tizonia-openmax-il/issues/319#issuecomment-304410741, I added the new AVC port to tizonia which was merged after Julien fixed some minor errors in it https://github.com/tizonia/tizonia-openmax-il/commit/9c9e6d0f74b6c80984b48d71c46ae52d46d83b7d.
This mostly involved changing the existing code from the VP8 port according to the AVC port as given in the OMX IL specifications. It would need still more work later on for initialising the structures that depend on different profile levels which is currently marked as TODO.
This work is the main project work involving adding the new component to mesa/gallium.
On this branch I keep the final work that is (almost) ready to be commited. It is located at https://github.com/gpalsingh/mesa/tree/gsoc
Currently it is complete up to adding an empty component to the omx-tiz state tracker, the openmax state tracker that uses tizonia. It can be used by passing the –use-omx-tiz argument to autogen.sh script.
This is the branch that I do all the experimental work on. It is located at https://github.com/gpalsingh/mesa/tree/gsoc-dev
To avoid confusion this branch uses omx_tizonia instead of omx_tiz in all places.
I’ve added an H.264 decoder which works with tizonia. It uses most of the code from the existing bellagio H.264 state tracker. Bellagio uses a private class (priv) to hold the decoder data while tizonia uses a processor (p_prc) for similar purpose. So it just needed to replace priv with p_prc.
Currently I’m working on improving the decode_stream function that decodes the input stream and shows the ouput https://github.com/gpalsingh/mesa/blob/gsoc-dev/src/gallium/state_trackers/omx_tizonia/h264dprc.c#L1242
In bellagio the decoding and buffer management take place in two different functions (vid_dec_DecodeBuffer and vid_dec_FrameDecoded) whereas in tizonia only one function does the work of both (*_buffers_ready).
I’ve used h264d_frame_decoded as replacement for bellagio’s base_port_SendBufferFunction since while checking with gdb I noticed it invokes vid_dec_FrameDecoded which performs the output and is the buffer management function for bellagio. https://github.com/gpalsingh/mesa/blob/gsoc-dev/src/gallium/state_trackers/omx_tizonia/h264dprc.c#L1296
For replacing port->ReturnBufferFunction (https://cgit.freedesktop.org/mesa/mesa/tree/src/gallium/state_trackers/omx/vid_dec.c#n519) I’m trying out h264d_release_input_buffer.
Trying with that gst-omx compiled for tizonia shows a blank window and the execution stops after that (without any error thrown). My focus will be on fixing this in the upcoming week and hopefully have a working decoder soon.