notes

[webkit_codecamp] / notes

00:00:04 00:12:53
Hi, I am Eleni Maria Stea and I am going to talk to you about some
ideas I explored and the experiments I've ran concerning the adoption of
ANGLE in WebKit/WPE.

=======================

Before I begin, let's se what ANGLE is and what we want to do with it
in WebKit.

So,ANGLE is an EGL/GLES implementation.

GLES is the API we use to render graphics with the GPU
and EGL is the API we use to create the GLES context

Both are implemented on top of other APIs.
In case of EGL these APIs can be the native system EGL, GLX, gbm or others.
In case of GLES they can be the native system driver GLES, OpenGL, Vulkan
or others.

This is because the primary purpose of ANGLE is to provide an EGL/GLES
implementation to systems lacking it.

The users can set the backend they prefer using some extra EGL
extensions provided by ANGLE in eglext_angle.h header file.

==============

On WebKit we'd like to use ANGLE in WebGL two that is implemented in
GLES two. The main reason is the performance.

But there is a problem with our current design:
As ANGLE renders on a GLES two texture created by an ANGLE context

and the WebKit graphics pipeline renders on OpenGL textures that are
later assembled by the WebKit compositor, we currently need to
copy the WebGL texture data from an ANGLE texture to an OpenGL one.

So all the experiments I am going to present are to replace this copy
with something better.

===================

Let's see a list of these experiments.

First I've written a program that uses ANGLE to render an image on an X11
window and the native system drivers to render another image on another
X11 window and display them.

That was to create a similar situation to the one we have on WebKit.

Then, I experimented with shared context and textures. I've created a
shared texture between the native context and the native context of the
ANGLE backend.

Then I filled it using the ANGLE context and displayed the result using
the native driver context. I had to abandon the shared context idea
for reasons that I'll explain later.

My next experiment was to use the Linux kerne dma buffers framework to
associate two textures with the same kernel memory. Then filling one
texture was automatically filling the other. This method seemed to do
exactly what we want and so I am going to use it in WebKit as well.

Last experiment was to try something similar but with multiple processes.
We might need it in the future.

======================

In the follow up slides I am going to explain them in detail.

For all my experiments I've written small standalone programs because
WebKit is complex and building it takes ages.

So, in order to use ANGLE in them I needed to build ANGLe with debugging
symbols, and to find some
way to use two implementations of the same API in the same program.

So the first step of the first experiment was to set up ANGLE for debugging
with GDB. ANGLE by default redirects the
debugging symbols and uses non-standard paths so I had to modify the
build system arguments and the GDB settings to be able to step into ANGLE
functions.

You can read about these modifications in my blog post.

==========================

So in my first test, I've rendered two images. one from a context 
created with ANGLE and
one with EGL like in this screenshot where the green image is the ANGLE one
and the XOR pattern comes from the native driver.

In order to have both implementations working together I linked with the
system driver libEGL and libGLES and I dynamically opened ANGLE's libEGL
and libGLES and loaded each EGL and gl function prefixed by angle_.

One thing I needed for this experiment to work was to invalidate the
ANGLE context before every draw call. This is an interesting ANGLE problem.

======

ANGLE is using some sort of internal context caching. This is to avoid
calling MakeCurrent when it's not absolutely necessary. So every time we
call angle_MakeCurrent ANGLE checks if the context has been changed and if
not it ignores the call.

When we use two contexts from two drivers in the same process ANGLE
considers it's the only EGL implementation available in the program.
So, in a rendering loop like the one of the slide, ANGLE would not be
aware of any EGL changes to the context (for example that EGL changed
a texture) it will only see angle_MakeCurrent being called with the
same context and so, it might not update it. 

In that case we'd need to invalidate the cached context by calling
angle_MakeCurrent using a null context at the beginning of display. 
This problem wouldn't happen with threads or multiple processes but
it's interesting to know it.

You can read more about these in my blog in the blog post sections
mentioned in the bullets.

=======

Let's jump to the next experiment. In this one I tried to exchange texture data
using shared textures and shared context.
========

So, what's shared context?

When we create a new context with EGL we can set an existing context
as parameter and use it as "shared". After that all textures created by
the shared context can be directly accessed from the new context while
in GPU.

For the shared context to work both contexts must be created by the same
process and both contexts must be created by the same API.

This means that we couldn't use shared context directly between EGL and ANGLE,
but we could use it between the EGL/OpenGL native system driver and the
EGL/OpenGL ANGLE backend that uses the native driver as well.

==========

To do so in WebKit, we'd need to force the EGL/OpenGL backend in ANGLE
and to write an ANGLE extension that could tell ANGLE when the shared
context is native and not ANGLE so that it's used in the backend call.

=========

This is how I had designed this extension. In the first picture of the
slide we see how we'd create shared context with native EGL and with
ANGLE.

In the second image we see how we'd pass a native context to ANGLE. One
of the EGL attributes passed to angle_eglCreateContext should indicate
that the context is native
==========

And this is what happens internally in ANGLE:
When the extension attribute is set we create a shared context in the
backend. When not, we handle the shared context as ANGLE shared context.

These are a few programs I've written to test the extension, and the ANGLE
shared context implementation
===========

Why we've rejected this method?
Main reason is that we plan to split the graphics operations in
multiple processes on WebKit.

But fortunately, there was a better approach to follow.

=============================

In that approach, I've used a Linux kernel framework for content sharing:
DMA buffers.

========================

This framework can be used to import dma buffers that are used 
as backing storage for textures from other drivers. It is handy
because if two textures from two drivers use the same dma buffer
storage, filling one will fill the other.

It has some advantages like that it can easily used by EGL as there are
EGL extensions for it.
it's driver independent
it works with multiple processes and

as long as ANGLE can implement and expose the extensions to import a
dma buffer it's also angle backend independent.

The only problem is that DMA buffers are used from Linux only while
shared context is universal. But in this particular WebKit backend,
we don't care.

===================================

I have a list of the EGL extensions that can be used to share a buffer
among drivers and links to the two example programs I've written.

====================================

Let's see how we could access a DMA buffer from context A in context B.
In this example texture texA corresponds to contextA and texture texB is
from context B.

In the exporter that is context A, we create an EGL image from tex A
and we export some information about it as well as the file descriptor
that corresponds to its underlying kernel dma buffer.

struct tex_storage_info stores the extra information.

=======================================

In the importer that is context B we can then
create another image that accesses the dma buffer using
the file descriptor and the information we extracted before

and we can use it to create another texture tex B using
where the underline dma buffer is the imported one

Then filling tex B will fill tex A.

=========================================

I've followed these steps with context A being native and context B
being ANGLE and created a similar case to the one we had in WebKit.

DMA buffers worked well, filling a texture would fill the other without
copy which is what we want in WebKit!
So, I've tested also the multiple processes case.

==============================

DMA buffers seem to work with multiple processes but require some sort
of Interprocess communication to exchange the file descriptor and the
information about the buffer we've seen before.

I've found a client server example that uses unix sockets to exchange the
file descriptor and works well. Code is on gitlab.

================================

So, finally let's see if this could be integrated to WebKit. 

The extensions to import dma buffers are fully implemented and exposed in
ANGLE and so in WebGL we could import a dma buffer from a graphics pipeline
texture when we create the WebGL render target.

The extension to export dma buffers is implemented on Mesa and so it can be
used in main pipeline. In case it's not found we could fallback to some
alternative that is copying or maybe libgbm.

So this week I am investigating what needs to be done on WebKit.

- I've build ANGLE and fixed the compile errors of that backend
- I am currently fixing some link errors (unfortunately there's one I 
haven't found yet)
- and I've taken a look at the code where I need to make the change.

==========================

References... I intend to upload these slides, and so I've appended a list
of links that could be useful in this task at the end. Feel free to
take a look at it

============================

And Closing
I'd like to thank you very much for watching my talk and for
giving me the opportunity to work on this task!

Goodbye and have a nice day or evening!

==========================