bugThe FreeType Project - Bugs: bug #44689, Support for vector layered color...

 
 

bug #44689: Support for vector layered color fonts

Submitted by:  None
Submitted on:  Wed 01 Apr 2015 03:01:03 PM UTC  
 
Severity:  3 - Normal Item Group:  Wishlist
Status:  In Progress Privacy:  Public
Assigned to:  Werner LEMBERG <wl> Open/Closed:  Open
Planned Release:  None

Add a New Comment (Rich Markup)
   

You are not logged in

Please log in, so followups can be emailed to you.

 

( Jump to the original submission)

Sun 20 May 2018 08:43:52 PM UTC, comment #30: 

No, no, FreeType already supports BGRA!  This means that the restriction to colorless bitmaps is not necessary.

FT_Load_Render, as a convenience, can produce a BGRA bitmap from layered outline glyphs – why not?  It's just that a glyph slot can't hold more than a single layer of outlines.  COLR was exactly defined for such a setup.

To summarize: For the simplest use, if COLR and CPAL is available, FT_Load_Render with the FT_LOAD_COLOR glyph flag set should blend the colored glyph layers to return a BGRA bitmap.  We define the used blending algorithm, which we again make publicly available for convenience.

If more control is wanted (i.e., a different blending algorithm, further processing of the glyph layer outlines, etc.), the client should use `FT_Get_GlyphLayers' and do the blending by itself.

And we don't need FT_LOAD_LAYERED.  FT_LOAD_COLOR is fully sufficient.

Werner LEMBERG <wl>
Project AdministratorIn charge of this item.
Sun 20 May 2018 04:16:29 PM UTC, comment #29: 

Exactly, we must respect FreeType fundamentals: to render means to produce colorless coverage map to be used by clients in blending. We must never FT_LOAD_RENDER colored glyphs in this sense.

FT_LOAD_LAYERED (similar to FT_LOAD_NO_RECURSE) must simply produce FT_GLYPH_FORMAT_LAYERED (similar to FT_GLYPH_FORMAT_COMPOSITE) which only contains subglyphs. Then FT_Get_Layer_Info (similar to FT_Get_SubGlyph_info) obtain color information.

Finaly, instead of FT_Render_Glyph, we need to implement FT_Blend_Glyph, which would be able to process FT_GLYPH_FORMAT_LAYERED glyphs rendering each layer separately and blending them successively.

FT_Blend_Glyph would be able to blend simple glyphs too, of course. It would implement gamma corrected alpha blending. I am optimistic about gamma correction now because we can produce very accurate lookup tables without power function.

Alexei Podtelezhnikov <podtelez>
Project Member
Sun 20 May 2018 10:18:29 AM UTC, comment #28: 

I have thought of that, too, but finally rejected the idea.  In FreeType, after expanding all sub-glyphs of a composite glyph, there is exactly one final layer in a glyph slot that can be directly rendered.  Changing that would mean a complete redesign of FreeType – I don't say that this would be a bad idea, but I'm not enthused to work on such a project...

Werner LEMBERG <wl>
Project AdministratorIn charge of this item.
Sun 20 May 2018 10:04:33 AM UTC, comment #27: 

I must say that the concept of layered glyphs is very similar to composite glyphs. So I wonder if we can or should just load them as FT_GLYPH_FORMAT_LAYERED, similar to FT_GLYPH_FORMAT_COMPOSITE, and reuse num_subglyphs and subglyphs of FT_GlyphSlotRec.

We can simply render them just like composite glyphs, possibly omitting the base layer so that we can see details. This would give a basic look.

For colored look we need successive blending of layers into target surface. 

Alexei Podtelezhnikov <podtelez>
Project Member
Sun 13 May 2018 01:30:16 AM UTC, comment #26: 

The patch is now in master, thanks!

Please check – I've only slightly adapted it without any testing yet.  Dominik, please send me a COLOR/CPAL font for testing privately!  And please open new bug reports for any issues you still encounter.

Behdad, Dominik, can you suggest API prototypes for the still missing functionality?

Werner LEMBERG <wl>
Project AdministratorIn charge of this item.
Wed 18 Apr 2018 12:30:23 PM UTC, comment #25: 

OK, will work on it after the next FreeType release.

Regarding the API: I'm bad at designing such beasts.  Please make suggestions how it should look like.

Werner LEMBERG <wl>
Project AdministratorIn charge of this item.
Wed 18 Apr 2018 10:09:49 AM UTC, comment #24: 

PS: In terms of API granularity: In a way to retrieve the layers in one call per glyph, but retrieve the palette index to color table in one call to retrieve the whole palette for one palette index.

Dominik Röttsches <drott>
Wed 18 Apr 2018 09:42:33 AM UTC, comment #23: 

Ack, public API for retrieving color layers, and translate from palette index to RGBA values would indeed be useful and needed. 

Dominik Röttsches <drott>
Tue 17 Apr 2018 12:59:03 PM UTC, comment #22: 

The error we found was that if a glyph did not have color layers, glyph load failed.

We definitely want API to fetch color layers.

We definitely also want FreeType to do blending for glyphs with no foreground-color layers at least.

Would be easiest at this point if Werner finishes this.

Behdad Esfahbod <behdad>
Tue 17 Apr 2018 07:32:11 AM UTC, comment #21: 

OK, thanks.  Note that I am willing to add COLR/CPAL support!  Are you going to fix the issue you've found recently, also taking account of Behdad's remarks in comment #15?  I haven't yet looked at the patch in more detail...

Werner LEMBERG <wl>
Project AdministratorIn charge of this item.
Tue 17 Apr 2018 06:31:45 AM UTC, comment #20: 

It looks we will implement COLR/CPAL at a higher level where we can do masking and blitting, not necessarily relying on FreeType support for it. From our point of view it's not urgent.

Dominik Röttsches <drott>
Mon 16 Apr 2018 06:56:58 PM UTC, comment #19: 

Question: I want to release a new FreeType version within a few days.  Shall I delay it?  Is it urgent to merge COLR/CPAL font support?

Werner LEMBERG <wl>
Project AdministratorIn charge of this item.
Mon 16 Apr 2018 04:40:11 PM UTC, comment #18: 

Let's not merge it yet, it seems to break metrics retrieval for COLR/CPAL fonts.

Dominik Röttsches <drott>
Mon 16 Apr 2018 09:57:55 AM UTC, comment #17: 

Werner, can we get this in please? Possibly without the blending if you don't feel comfortable about that. Thanks.

Behdad Esfahbod <behdad>
Mon 16 Apr 2018 09:51:40 AM UTC, comment #16: 
Dominik Röttsches <drott>
Wed 09 Aug 2017 11:10:54 PM UTC, comment #15: 

I did a more thorough review:

- I don't think FT_GlyphSlotRec can change.  You should put your FT_Colr_Internal inside FT_Slot_Internal,

- ttcolr.h header still says ttsbit.h; maybe other places as well,

- I don't see how clients can get the layer info, as FT_Colr_Internal is private, and I don't see any getter API.  Am I missing something?

- I'm also unsure about blending, even though I was the one who proposed it initially.  That said, I think there's value in it, given that practically all emoji fonts will be using color-only glyphs...

Behdad Esfahbod <behdad>
Wed 20 Apr 2016 09:06:18 PM UTC, comment #14: 

Shao, any update on this?

Thanks,
behdad
(Can't seem to log in right now)

Anonymous
Fri 08 Jan 2016 12:38:09 AM UTC, comment #13: 

Thanks Ben for the review.  I see the blended version as a fallback implementation, for smaller clients who don't want to load the layers and do their own blending.  The sRGB vs linear is a bug and should be fixed.
behdad

Anonymous
Thu 07 Jan 2016 11:14:51 PM UTC, comment #12: 

Note that the proposed tt_face_colr_blend_layer is problematic. FreeType has avoided so far any color blending, and for good reason. For example, the blend being proposed is incorrect, since the colors are specified in sRGB (the proposed blend is linear). Also, by providing the blend here, the issue of the 0xFFFF color needs to be addressed.

bungeman <bungeman>
Mon 04 Jan 2016 05:44:57 PM UTC, comment #11: 

Thanks for the review.

I'll make the changes.

Shao Yu Zhang <shao>
Sat 02 Jan 2016 12:39:46 PM UTC, comment #10: 

Hi Shao,

I checked the patch  I like your approach of keeping regular outline next to the color layers.  A couple of things:

  - Might have been better to not load the entire colr/cpal tables and just peek through the as needed, but that can be done as an optimization later,

  - Make sure it's easy enough for clients to access the layers and color directly if they choose to,

  - The foreground color issue can be fixed at a later time,

  - I don't think we need to worry about your example of transform changing between load and render time.  I don't think that's supported.

Cheers,
behdad

Behdad Esfahbod <behdad>
Mon 30 Nov 2015 09:53:01 PM UTC, comment #9: 

I initially thought that the transform step in FT_Load_Glyph (about ftobjs.c:855) would morph the image. But it's a false alarm.
if ( ( load_flags & FT_LOAD_IGNORE_TRANSFORM ) == 0 )

The thought process led me to suspect that the glyph's size and shape can be modified due to this transform. So it might be possible to do the following:

  FT_Load_Glyph(face, glyph_index, load_flags);
  FT_Get_Glyph(face->glyph, glyph_slot);
  FT_Set_Transform(face, new_transform, new_delta);
  FT_Render_Glyph(glyph_slot, FT_RENDER_MODE_NORMAL)
// Now render glyph for color vector will use the the new transform
// and rasterize into an image different than the previous transform?

Hopefully there's no other mutable state that I'm not aware of.

Shao Yu Zhang <shao>
Thu 26 Nov 2015 06:03:43 PM UTC, comment #8: 

Very nice, thanks!  I'll check this in after the next release.

Can you elaborate on your reservations, probably giving an example?  I don't really understand what you are worried about.

Werner LEMBERG <wl>
Project AdministratorIn charge of this item.
Wed 25 Nov 2015 10:40:48 PM UTC, comment #7: 

Hello,

Here is my attempt at this. I hope you guys like it.
https://github.com/ShaoYuZhang/freetype/commit/27d9e6f6012b55d4001c33e3d46f6814c94ebc0d

I started hacking on this a little independently from Behdad, and followed roughly the same line of thought. I borrowed the FT_Load_Glyph technique from Behdad.

My biggest reservation to the current diff is loading subglyph outlines during FT_Render_Glyph. It works for me so far but I suspect there may be state change with other APIs that cause final raster to differ from loading subglyph outline during FT_Load_Glyph?

Shao Yu Zhang <shao>
Mon 26 Oct 2015 11:05:48 PM UTC, comment #6: 

I did a proof-of-concept two years ago.  That's here:

https://github.com/behdad/freetype/commit/d23a43dac26f34aa7cdaec9cfd81211298b0b5de

Note that a proper implementation needs to augment glyph formats.  Here's a start:

https://github.com/behdad/freetype/commit/5e9023eabf916198a912244d5fef89b54096fda1

Behdad Esfahbod <behdad>
Sat 24 Oct 2015 08:22:11 AM UTC, comment #5: 

Great, I think Gecko’s implementation can be ported into FreeType.

Hsing-yen He <hammerandsickle>
Wed 07 Oct 2015 07:11:08 AM UTC, comment #4: 

For reference, Gecko’s implementation: https://bugzilla.mozilla.org/show_bug.cgi?id=889401

Anonymous
Tue 07 Apr 2015 05:02:24 AM UTC, comment #3: 

Great!

Werner LEMBERG <wl>
Project AdministratorIn charge of this item.
Mon 06 Apr 2015 11:20:16 PM UTC, comment #2: 

Will get to it, eventually. :)

Behdad Esfahbod <behdad>
Thu 02 Apr 2015 01:36:12 PM UTC, comment #1: 

I'm CCing Behdad :-)

Werner LEMBERG <wl>
Project AdministratorIn charge of this item.
Wed 01 Apr 2015 03:01:03 PM UTC, original submission:  

After supporting Google’s way to do emoji (embedded PNGs in font files), it would be interesting to support Microsoft’s way (layered color fonts).

References for this:

A layered color font is a standard TrueType/OpenType, with standard monochrome glyphs as a fallback. Additional glyphs are used for each layer of the color version of every character.

There are two additional tables: CPAL for defining one or more color palettes. COLR to link the glyphs together as layers and pick their color.

Implementations:

Anonymous

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by podtelez (Posted a comment)
  • -email is unavailable- added by drott (Posted a comment)
  • -email is unavailable- added by hadess
  • -email is unavailable- added by bungeman (Posted a comment)
  • -email is unavailable- added by shao (Posted a comment)
  • -email is unavailable- added by hammerandsickle (Posted a comment)
  • -email is unavailable- added by behdad (Posted a comment)
  • -email is unavailable- added by wl (Posted a comment)
  • -email is unavailable- added by wl
  •  

    Do you think this task is very important?
    If so, you can add your encouragement to it.
    This task has 0 encouragements so far.

    Only logged-in users can vote.

     

     

     

    Follow 4 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2018-01-23 hadess Carbon-Copy- => Added hadess
    2015-11-26 wl StatusNone => In Progress
        Assigned toNone => wl
    2015-04-02 wl Carbon-Copy- => Added behdad

    Back to the top


    Powered by Savane 3.5