Platform/GFX/InternProjects: Difference between revisions

From MozillaWiki
< Platform‎ | GFX
Jump to navigation Jump to search
 
(6 intermediate revisions by 4 users not shown)
Line 27: Line 27:
| joe/bholley
| joe/bholley
|-
|-
| Enable proper decode-on-draw on desktop
| GPU SVG filters
| We have the ability to throw away an image when it hasn't been drawn for a while, but we currently don't to eliminate flicker on the currently-displayed tab. We can probably get better heuristics for this, but doing so will also probably involve making asynchronous decoding more responsive, either by tweaking the way we asynchronously decode or the amount of data we asynchronously decode. This will greatly reduce the amount of memory we use on image-heavy pages.
| SVG filters are currently implemented through software rendering on the CPU, even when Firefox is using hardware for its general 2D rendering (i.e., when using Direct2D). These should be reimplemented using HLSL shaders to avoid readback. The biggest challenges here will be allowing arbitrary chaining of filters, as well as making it possible to avoid reimplementation once we need to support OpenGL shaders too.
| ''TBD''
| ''TBD''
| joe/bholley
| roc
|-
|-
| Do YCbCr/YUV->RGB conversion of JPEGs on the GPU
| GPU Clipping
| When we're using hardware-accelerated layers, we can reduce the amount of work we need to do on the CPU by decoding JPEGs to YUV only, uploading those channels to the GPU, and then converting the YUV to RGB in a shader.<br><br>'''Note''': This is likely not a good project to start before we have a fully colour-managed pipeline or Emerald is complete.
| Our current GPU layer clipping code requires the clip to be an axis-aligned rectangle, and in other cases we are allocating an intermediate surface and rendering to that (transformations), or handling the render in software (SVG). We should be able to handle this using the stencil buffer or similar.
| ''TBD''
| ''TBD''
| joe/jrmuizel/mattwoodrow
| mattwoodrow/roc
|-
|-
| Use ARB_robustness/ARB_robustness2 in our WebGL implementation where it's available
| Store Images as textures
| Currently, even if a graphics driver implements and exports the ARB_robustness extension to make it less likely for shaders to DoS a computer, we don't use it. We should.
| When we separate an image into it's own dedicated layer we have two copies of the data, one in system memory (owned by imglib) and one as a texture. We can teach imglib about this second copy and free the system memory one in some cases to save memory usage. This is particularly useful for mobile where image scaling is much cheaper on the GPU, but the memory cost of two copies will hurt.<br><br>'''Note''': This would also be a good stepping stone towards decoding JPEGs on the GPU.
| {{bug|656824}}
| {{bug|650988}} for mobile's usage of this.
| bjacob
| mattwoodrow/joe/roc
|-
|-
| Implement anti-aliasing in WebGL
| Underline breaking
| We don't currently honour the anti-aliasing hint in WebGL contexts, so our rendering looks inferior to Chrome's on the same demo on the same hardware. We should implement anti-aliasing.
| Implement the css text-decoration-skip property
| {{bug|615976}}
| ''TBD''
| bjacob
| jrmuizel
|-
|-
| Use OpenGL for WebGL, instead of ANGLE, on "good enough" drivers
| ANGLE D3D10
| ANGLE lets us support drivers with poor OpenGL implementations by rendering to Direct3D 9, but using it incurs a performance penalty. We should define what drivers are good enough, which will necessarily include Direct3D/OpenGL texture interop, and measure how good performance is on these drivers with OpenGL vs using ANGLE.
| Implement Angle on top of D3D10 instead of D3D9
| ''TBD''
| ''TBD''
| bjacob/joe
| bjacob
|-
|-
| Subpixel positioning of glyphs in the Cairo image surface
| WebGL software rendering
| Currently we snap glyphs to pixel boundaries in the image surface, which is good for performance because we can rasterize glyphs only once (in glyph space) and then re-use those rasterized glyphs as a glyph cache. We should change this to rasterize glyphs at (for example) third-pixel boundaries, and then snap our glyph positions to these third-pixel boundaries.
| This would probably be based on Mesa's software renderer
| ''TBD''
| ''TBD''
| jrmuizel
| jrmuizel
|-
|-
| Write a new scanline rasterizer for Cairo
| WebGL extensions
| The current approach is fast and high quality, but complicated, and has problems with clipping invariance. We can probably serve our needs better with something simpler, lower quality, and hopefully very fast, like skia.
| There are a number of possible extensions to WebGL that we're considering. This likely includes anything from OpenGL ES 3.0
| ''TBD''
| ''TBD''
| jrmuizel
| bjacob
|-
|-
| Speed up gradient rendering in Cairo
| Improve texture upload performance
| Rendering gradients is slower than it should be, especially on ARM. The current performance needs to be measured and the current rendering methods worked out, then it needs to be sped up significantly. This can be through better rendering methods, parallelization (e.g. SIMD instruction use), or other methods.
| Ali did a bunch of work here that he hasn't been able to finish. This would also match well with the off main thread composting video. i.e. make video work really well
| ''TBD''
| ''TBD''
| jrmuizel
| ajuma
|-
| Off main thread compositing work
| There's a bunch of work that will need to be done here.
| {{bug|omtc}}
| ajuma/bgirard
|-
|-
| GPU SVG filters
| Off main thread image decoding
| SVG filters are currently implemented through software rendering on the CPU, even when Firefox is using hardware for its general 2D rendering (i.e., when using Direct2D). These should be reimplemented using HLSL shaders to avoid readback. The biggest challenges here will be allowing arbitrary chaining of filters, as well as making it possible to avoid reimplementation once we need to support OpenGL shaders too.
| This will need some significant rearchitecture to work well
| ''TBD''
| ''TBD''
| roc
| jrmuizel/joedrew
|-
|-
| GPU Clipping
| Image lib improvements
| Our current GPU layer clipping code requires the clip to be an axis-aligned rectangle, and in other cases we are allocating an intermediate surface and rendering to that (transformations), or handling the render in software (SVG). We should be able to handle this using the stencil buffer or similar.
| This includes using/building a new more generic cache and fixing up some of the interfaces exposed by imagelib (e.g. imgIRequest should not be an nsIRequest)
| ''TBD''
| ''TBD''
| mattwoodrow/roc
| jrmuizel/joedrew
|}
|}

Latest revision as of 20:43, 6 January 2012

Intern Projects

Ideally, we'd like to have a set of intern projects from which prospective or new interns can pick and choose. These tasks should be relatively self-contained, challenging but not impossible, and well-defined.

Please link to bugs where appropriate.

Name Description Bugs Mentor
Implement the rest of ICCv4 in qcms Benoit Girard has a patch to significantly improve qcms's ICCv4 support, but even after his patches we don't fully support ICCv4. (CMYK support, for example, is entirely absent.) TBD BenWa/jrmuizel
Split imgIContainer into two interfaces imgIContainer should be split into two separate interfaces, one for decoders and libpr0n itself to use, and one for external users who just want to draw or get information about the current image. bug 503973 joe/bholley
Don't allocate memory for unused areas in animated GIF files when compositing Huge GIF files can be mostly empty space, but we currently allocate the full size of these images when we're compositing the frames together and drawing them to screen. (Other browsers are smarter.) We should emulate other browsers, and only allocate the parts of the frame that need to change. bug 289763 joe/bholley
GPU SVG filters SVG filters are currently implemented through software rendering on the CPU, even when Firefox is using hardware for its general 2D rendering (i.e., when using Direct2D). These should be reimplemented using HLSL shaders to avoid readback. The biggest challenges here will be allowing arbitrary chaining of filters, as well as making it possible to avoid reimplementation once we need to support OpenGL shaders too. TBD roc
GPU Clipping Our current GPU layer clipping code requires the clip to be an axis-aligned rectangle, and in other cases we are allocating an intermediate surface and rendering to that (transformations), or handling the render in software (SVG). We should be able to handle this using the stencil buffer or similar. TBD mattwoodrow/roc
Store Images as textures When we separate an image into it's own dedicated layer we have two copies of the data, one in system memory (owned by imglib) and one as a texture. We can teach imglib about this second copy and free the system memory one in some cases to save memory usage. This is particularly useful for mobile where image scaling is much cheaper on the GPU, but the memory cost of two copies will hurt.

Note: This would also be a good stepping stone towards decoding JPEGs on the GPU. 		bug 650988 for mobile's usage of this. mattwoodrow/joe/roc
Underline breaking Implement the css text-decoration-skip property TBD jrmuizel
ANGLE D3D10 Implement Angle on top of D3D10 instead of D3D9 TBD bjacob
WebGL software rendering This would probably be based on Mesa's software renderer TBD jrmuizel
WebGL extensions There are a number of possible extensions to WebGL that we're considering. This likely includes anything from OpenGL ES 3.0 TBD bjacob
Improve texture upload performance Ali did a bunch of work here that he hasn't been able to finish. This would also match well with the off main thread composting video. i.e. make video work really well TBD ajuma
Off main thread compositing work There's a bunch of work that will need to be done here. bug omtc ajuma/bgirard
Off main thread image decoding This will need some significant rearchitecture to work well TBD jrmuizel/joedrew
Image lib improvements This includes using/building a new more generic cache and fixing up some of the interfaces exposed by imagelib (e.g. imgIRequest should not be an nsIRequest) TBD jrmuizel/joedrew
Retrieved from "https://wiki.allizom.org/index.php?title=Platform/GFX/InternProjects&oldid=384590"