GSoC 2017: H.264 encoder in the works

This week the project moved into the harder part. I’ll explain the progress briefly below.

EGLImage support in H.264 decoder delayed

We are still stuck on the issue of how to output to the EGLImage. The task was a bit more difficult than we expected. My question in the mesa-dev mailing list is still answered as of now. According to Julien, we might need to add more to add more plumbing/callback in vl_screen or pipe_screen. Since we decided to spend only a week on EGLImage support research we’ve moved on to the next goal.

New component being added: H.264 encoder

The component being developed on the gsoc-dev branch on my clone of the mesa repo.  So far the work has been mainly on the core elements of the encoder i.e. the processor is ready. Still the encoder is unable to encode the data at the moment because the stock tizvideoport that it uses doesn’t provide the same capabilities that are required from the ports. In bellagio, these custom behaviours can be set up by replacing the port or component methods with user made functions. For example, This code from vid_enc.c changes how the port works

port->Port_SendBufferFunction = vid_enc_EncodeFrame;
port->Port_AllocateBuffer = vid_enc_AllocateInBuffer;
port->Port_UseBuffer = vid_enc_UseInBuffer;
port->Port_FreeBuffer = vid_enc_FreeInBuffer;</pre >

With Tizonia, this process is much more organised and decoupled. With Bellagio, ports and components are instantiated in the same constructor but with Tizonia they should have different processors and the component can simply use the new port as a type. Instead of replacing / assigning the functions, Tizonia has predefined values which tell it what function performs what function while registering it. For example, in this code

void *
vp8d_prc_init (void * ap_tos, void * ap_hdl)
{
  void * tizprc = tiz_get_type (ap_hdl, "tizprc");
  void * vp8dprc_class = tiz_get_type (ap_hdl, "vp8dprc_class");
  TIZ_LOG_CLASS (vp8dprc_class);
  void * vp8dprc = factory_new
    /* TIZ_CLASS_COMMENT: class type, class name, parent, size */
    (vp8dprc_class, "vp8dprc", tizprc, sizeof (vp8d_prc_t),
     /* TIZ_CLASS_COMMENT: */
     ap_tos, ap_hdl,
     /* TIZ_CLASS_COMMENT: class constructor */
     ctor, vp8d_prc_ctor,
     /* TIZ_CLASS_COMMENT: class destructor */
     dtor, vp8d_prc_dtor,
     /* TIZ_CLASS_COMMENT: */
     tiz_srv_allocate_resources, vp8d_prc_allocate_resources,
     /* TIZ_CLASS_COMMENT: */
     tiz_srv_deallocate_resources, vp8d_prc_deallocate_resources,
     /* TIZ_CLASS_COMMENT: */
     tiz_srv_prepare_to_transfer, vp8d_prc_prepare_to_transfer,
     /* TIZ_CLASS_COMMENT: */
     tiz_srv_transfer_and_process, vp8d_prc_transfer_and_process,
     /* TIZ_CLASS_COMMENT: */
     tiz_srv_stop_and_return, vp8d_prc_stop_and_return,
     /* TIZ_CLASS_COMMENT: */
     tiz_prc_buffers_ready, vp8d_prc_buffers_ready,
     /* TIZ_CLASS_COMMENT: */
     tiz_prc_port_flush, vp8d_prc_port_flush,
     /* TIZ_CLASS_COMMENT: */
     tiz_prc_port_disable, vp8d_prc_port_disable,
     /* TIZ_CLASS_COMMENT: */
     tiz_prc_port_enable, vp8d_prc_port_enable,
     /* TIZ_CLASS_COMMENT: stop value*/
     0);

  return vp8dprc;
}

The values in factory_new are like key->value. So vp8d_prc_ctor will be set as the constructor for the new component. With bellagio you’ll have to set it as comp->Construcor=vp8_prc_ctor while loading the component.

The same process works for ports but different functions. Here is an example from the tizavcport.c on which I worked

void *
tiz_avcport_init (void * ap_tos, void * ap_hdl)
{
  void * tizvideoport = tiz_get_type (ap_hdl, "tizvideoport");
  void * tizavcport_class = tiz_get_type (ap_hdl, "tizavcport_class");
  TIZ_LOG_CLASS (tizavcport_class);
  void * tizavcport = factory_new
    /* TIZ_CLASS_COMMENT: class type, class name, parent, size */
    (tizavcport_class, "tizavcport", tizvideoport, sizeof (tiz_avcport_t),
     /* TIZ_CLASS_COMMENT: */
     ap_tos, ap_hdl,
     /* TIZ_CLASS_COMMENT: class constructor */
     ctor, avcport_ctor,
     /* TIZ_CLASS_COMMENT: class destructor */
     dtor, avcport_dtor,
     /* TIZ_CLASS_COMMENT: */
     tiz_api_GetParameter, avcport_GetParameter,
     /* TIZ_CLASS_COMMENT: */
     tiz_api_SetParameter, avcport_SetParameter,
     /* TIZ_CLASS_COMMENT: */
     tiz_api_GetConfig, avcport_GetConfig,
     /* TIZ_CLASS_COMMENT: */
     tiz_api_SetConfig, avcport_SetConfig,
     /* TIZ_CLASS_COMMENT: */
     tiz_port_check_tunnel_compat, avcport_check_tunnel_compat,
     /* TIZ_CLASS_COMMENT: stop value*/
     0);

  return tizavcport;
}

The big advantage of this is that ports become reusable! This means you can use instances of a port and also modify a port and create new port out of it (similar to subclasses). In the above example, tizavcport inherits from tizvideoport which in turn inherits from tizport which acts in turn acts as base class for ports. Just for the curious ones the hierarchy is: tizavcport -> tizvideoport -> tizport -> tizapi -> tizobject .

For the H.264 encoder a new custom port needs to be made which adds a little extra functionality to the existing tizavcport for the out port and tizvideoport for the in port. The ports will have custom behaviour for tiz_api_AllocateBuffer, tiz_api_FreeBuffer, and tiz_api_UseBuffer (in port only). The code will remain the same as their vid_enc* counterparts in vid_enc.c After this obstacle comes the major blocking obstacle.

Registering multiple components

So far, looking at the Tizonia’s plugins it’s approach has been to provide a single entry point (OMX_ComponentInit) through which it can register it’s roles and types. Each component can have different roles. Each component has it’s own shared object library form which the component can be used. But in gallium, a single libomx_mesa.so provides all the components.

Bellagio provides

int omx_component_library_Setup(stLoaderComponentType **stComponents)

for setting up multitple components. It is invoked two times while setting up the components. First time the argument is NULL, which means the function must tell how many components it intends to register. The return value determines the dimensions of stComponents. For two functions it’s 2×1, for three it’s 3×1 and so on. Each stLoaderComponentType* is a specialised structure to initialise a separate component.

As of now Tizonia only has

OMX_ERRORTYPE OMX_ComponentInit (OMX_HANDLETYPE ap_hdl)

as entry point which supports only one component.

This means as of now it’s only possible to add only one component in libomx_mesa.so. For testing I’ve enabled only the video encoder. I’ve asked Juan about ways to add multiple components. One possible way might be to add a single component and add other components as it’s roles and check for roles in the hooks which I haven’t tried properly yet. Even if that worked it will be a bit ugly and will require extra work inside the hooks so a different and cleaner solution is more desirable.

Next week’s goals

Making the encoder work takes priority over the multiple component issue even when it is a blocker since it can be solved any time before the project ends. We will also be hoping to make some progress on the EGLImage research.

Advertisements

One comment

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s