bugThe FreeType Project - Bugs: bug #52165, blank line between characters...

 
 

You are not allowed to post comments on this tracker with your current authentication level.

bug #52165: blank line between characters (regression in 2.8)

Submitter:  Vincent Lefèvre <vinc17>
Submitted:  Wed 04 Oct 2017 12:02:54 PM UTC
   
 
Severity:  3 - Normal Item Group:  Incorrect behaviour
Status:  Wont Fix Privacy:  Public
Assigned to:  wl Open/Closed:  Closed
Planned Release:  None

Jump to the original submission

Thu 12 Oct 2017 04:01:24 PM UTC, comment #28: 

The new tracker 52215.

Alexei Podtelezhnikov <podtelez>
Group Member
Wed 11 Oct 2017 09:40:01 PM UTC, comment #27: 

This is actually a nice idea, thanks!

Please create a new tracker issue and tag it as `wishlist'.

However, I don't understand your last sentence.  Please elaborate.

Werner LEMBERG <wl>
Group administrator
Wed 11 Oct 2017 04:46:13 PM UTC, comment #26: 

Why? You made autohinter work for all kinds of scripts. The box drawing is just another script with a very defined unicode range. We can do whatever autohinting to fit our rounding convention. We can make a special blue zone for them too. Adjusting spacing for these glyphs is actually wrong on the other hand.

And the bytecode for these special glyph can do the work too according to whatever spacing TrueType sets.

Why not?


Alexei Podtelezhnikov <podtelez>
Group Member
Wed 11 Oct 2017 04:27:35 PM UTC, comment #25: 

No no no!  If in auto-hinter mode, it's neither a bug in FreeType nor a bug in xterm (or rather, a bug in Xft) – it's simply not possible to know that those characters should be stacked perfectly![*]  As mentioned earlier, the auto-hinter sometimes increases the vertical glyph dimension at small sizes to increase readability.  [A possible solution would be to check the (hinted) dimensions of block characters and adjust the vertical spacing accordingly.]

Only in native TrueType bytecode mode the user should round differently, because this is what the bytecode expects.  And due to the selection of hinting at `FT_Load_Glyph' time FreeType can't switch to the correct rounding mode in the FT_Size object.

[*] At least not currently.  There were some ideas to make the auto-hinter more aware of characters properties; for example, that the character `i' has a dot that must not be accidentally merged with the stem.  Similarly, there could be a property that ensures that the block characters' vertical dimensions exactly fit the global ascender and descender.

Werner LEMBERG <wl>
Group administrator
Wed 11 Oct 2017 01:52:08 PM UTC, comment #24: 

Fully agreed. So the lack of rounding instructions for this characters is the DejaVu font bug. The autohinter should also take care of this in accordance with FreeType rounding convention. This is a FreeType autohinter bug.

This is not an xterm bug.

Alexei Podtelezhnikov <podtelez>
Group Member
Wed 11 Oct 2017 12:55:02 PM UTC, comment #23: 

Box characters are special.  To make them work two assumptions must hold: (1) The rendering environment must use the line spacing as given in the font – systems like TeX ignore this.  (2) To avoid visible overlaps, the y values of the box elements must be aligned to the grid, exactly fitting the global ascender and descender values.

That ascender and descender values are rounded to integers (instead of using `ceil' and `floor', respectively) is an arbitrary choice of TrueType.  It would work equally well the other way.  In hindsight I believe that flooring and ceiling would even have been a better choice due to the infamous glyph cut-offs in Windows: It would allow for (slightly) increasing the glyphs vertically while hinting, providing more flexibility for small ppem values.

Werner LEMBERG <wl>
Group administrator
Wed 11 Oct 2017 12:37:01 PM UTC, comment #22: 

On the other hand, all fonts are monospaced in y-direction, so it make sense to limit the user choice to the rounded values. But the box drawing characters must be hinted to the rounded values. I do not know if it can be done through blue zones, or phantom points, or if TrueType has some special facility for that. There is no way around it. The top and bottom of box drawing must be aligned to the line edges.

This reminds me of Arabic or Devanagari script, ccing Behdad.

Alexei Podtelezhnikov <podtelez>
Group Member
Wed 11 Oct 2017 11:19:32 AM UTC, comment #21: 

We round for historical reasons – I fully agree that it is a bad idea in general – the user should do it to fit the desired purpose –, but changing that would be backward incompatible.

If you don't do hinting, it would be appropriate to not round the advance width and not round the ascender and descender either.

Feel free to improve the documentation.

Werner LEMBERG <wl>
Group administrator
Wed 11 Oct 2017 10:57:58 AM UTC, comment #20: 

The glyph is question is neither hinted nor autohinted. The problem occurs regardless of hinting. Your suggested code would not work.

To me, this demonstrates that rounded (all-inclusive) ascender and descender are useless and confusing.

Alexei Podtelezhnikov <podtelez>
Group Member
Wed 11 Oct 2017 06:13:38 AM UTC, comment #19: 

FreeType's values for `ascender' and `descender' in FT_Size are `all-inclusive'.  If you need something better – in particular for TTFs –, follow the advice given in commit 08e2e311.

Werner LEMBERG <wl>
Group administrator
Wed 11 Oct 2017 03:27:42 AM UTC, comment #18: 

Werner, rounding the ascender up and the descender down can clearly leave a huge gapping hole, see attached. The table drawing glyphs do not seem to be hinted or autohinted in this font. What fix do you suggest in this case?

I have just added the ascender and descender lines to ftgrid.


Alexei Podtelezhnikov <podtelez>
Group Member
Sun 08 Oct 2017 10:20:48 AM UTC, comment #17: 

I've already contacted the author of xterm (with CC to you), pointing to this bug report – hopefully, xterm can be fixed soon.  Please do the same for other programs where you notice bad behaviour.

Werner LEMBERG <wl>
Group administrator
Sun 08 Oct 2017 10:08:55 AM UTC, comment #16: 

Unfortunately, it was not announced in Debian. And the fact is that several applications are currently based on this "bug" (perhaps because the TrueType fonts are the most common and the default in practice, at least for these applications and most users use the default hinting method). Otherwise the problem would have already been reported.

Vincent Lefèvre <vinc17>
Sun 08 Oct 2017 09:38:55 AM UTC, comment #15: 

It was properly announced in the CHANGES file!

And no, it's neither an ABI nor API change, because the new code corrects the metrics for TTFs in all other hinting modes (i.e., unhinted and auto-hinted).  The old FreeType behaviour was essentially a bug, which (more or less accidentally) did the right thing for TTFs with native bytecode hinting.

You can call it a design problem of FreeType's API that hinting can be freely selected at glyph loading time.  [In hindsight, it would have been better to select the hinting mode while creating the size object.]  However, to make this late hinting mode selection universally work for all font formats, the global size metrics must be derived independently of the hinting mode – I thus had to sacrifice the special handling of natively hinted TTFs.

Werner LEMBERG <wl>
Group administrator
Sun 08 Oct 2017 09:18:14 AM UTC, comment #14: 

Note that's an ABI change (because of visible change of the behavior, affecting some applications), and an API change too, because some applications are now required to use "values from the `FT_Face' structure and scale them manually". This should have properly been announced.

Vincent Lefèvre <vinc17>
Sun 08 Oct 2017 08:39:10 AM UTC, comment #13: 

I've just improved the documentation in git.

Werner LEMBERG <wl>
Group administrator
Sat 07 Oct 2017 05:45:04 PM UTC, comment #12: 

Regarding comment #9: Thanks for the data, I now know the cause.  It's a change introduced in version 2.8 to make the handling of all fonts the same.  From the CHANGES file:



Global size metrics values in the `FT_Size_Metrics' structure can be different for TrueType fonts.  Reason is that in older FreeType versions the metrics were rounded differently to integer pixels compared to all other font formats, yielding an inconsistent behaviour if you used non-native hinting.  Starting with this version, global size metrics for TrueType fonts are handled the same as other font formats: `ascender' gets rounded up, `descender' getsrounded down, `height' gets normally rounded, and
`max_advance' gets normally rounded, too.

If you need more precise values of (global) ascender, descender, height, or `max_advance', please take the corresponding values from the `FT_Face' structure and scale them manually.



As you can see, the rounding for ascender and descender has changed.

Werner LEMBERG <wl>
Group administrator
Sat 07 Oct 2017 04:44:57 PM UTC, comment #11: 

Regarding comment #8: In your GNOME snapshots, you are definitely using native v35 TrueType hinting in your 2.6.1 image!  And you are definitely using native v40 TrueType hinting in your 2.8 image.  No slight hinting at all...

Use a magnifier and compare with the attached image created with the `ftdiff' demo program (from current git).


Werner LEMBERG <wl>
Group administrator
Fri 06 Oct 2017 02:44:04 PM UTC, comment #10: 

Should FreeType enforce that ascend + descend = height? Or, should Xft prioritize the height?

What about actual glyph metrics? Do we need blue zones to keep the box drawing glyphs aligned?

Alexei Podtelezhnikov <podtelez>
Group Member
Fri 06 Oct 2017 11:02:37 AM UTC, comment #9: 

On a different machine (my desktop machine, which shows the same issues), I've rebuilt xterm with --enable-trace to get debugging information and compare the traces obtained with FreeType 2.6.3 and 2.8.0. Concerning the differences for the fonts, I get:


 xtermComputeFontInfo font 0: norm(face DejaVu Sans Mono:10, size 14.0)
 ...matched normal font
 ...matched bold font
 ...matched italic font
-Xft metrics screen->renderFontNorm[0] = 20 (16,4) advance 10, actual 10
-Xft metrics screen->renderFontBold[0] = 20 (16,4) advance 10, actual 10
-Xft metrics screen->renderFontItal[0] = 20 (16,4) advance 10, actual 10
+Xft metrics screen->renderFontNorm[0] = 20 (16,5)* advance 10, actual 10
+Xft metrics screen->renderFontBold[0] = 20 (16,5)* advance 10, actual 10
+Xft metrics screen->renderFontItal[0] = 20 (16,5)* advance 10, actual 10
 xtermComputeFontInfo wide(face <null>, char_width 20)
-setRenderFontsize result 10x20 (16+4)
+...increase height from 20
+setRenderFontsize result 10x21 (16+5)
+...increase height from 20
 setRenderFontsize bold unchanged
+...increase height from 20
 setRenderFontsize ital unchanged
-...packed TrueType font 20x10 vs 10
-xtermComputeFontInfo window 20x10 (full 24x23), fontsize 20x10 (asc 16, dsc 4)
+...packed TrueType font 21x10 vs 10
+xtermComputeFontInfo window 21x10 (full 25x23), fontsize 21x10 (asc 16, dsc 5)
[...]


The later differences seem to be a consequence of that. Concerning the "Xft metrics" lines, they are output by:


#define CACHE_XFT(dst,src) if (src != 0) {\
            checkXft(xw, &(dst[fontnum]), src);\
            TRACE(("Xft metrics %s[%d] = %d (%d,%d)%s advance %d, actual %d%s\n",\
                #dst,\
                fontnum,\
                src->height,\
                src->ascent,\
                src->descent,\
                ((src->ascent + src->descent) > src->height ? "*" : ""),\
                src->max_advance_width,\
                dst[fontnum].map.min_width,\
                dst[fontnum].map.mixed ? " mixed" : ""));\
        }


So, that's the "descent" that is modified.

And the fact that (src->ascent + src->descent) > src->height with FreeType 2.8.0 possibly yields the issues.

Vincent Lefèvre <vinc17>
Fri 06 Oct 2017 08:07:39 AM UTC, comment #8: 

When doing my tests comparing 2.6.3 and 2.8.0, I just upgrade/downgrade the libfreetype6 Debian package with apt/dpkg. So the fonts and the fontconfig versions remain the same.

Concerning the fontconfig hinting style, I use Debian's default, which has been "Slight" (hintslight) since July. If I switch to full hinting, the window height is still incorrect and there is still a blank line (and the font rendering is of lower quality).

Note that not just xterm is affected, but also Emacs and GNOME Terminal, and for all of them, the font appears to be higher. Firefox 52.4 ESR does not seem to be affected, but perhaps it doesn't use all information from FreeType.

What would be interesting to know is which change in FreeType leads to the issue. Also, is there some simple FreeType-based program that would give information about a font, such as height information?

Vincent Lefèvre <vinc17>
Fri 06 Oct 2017 04:35:57 AM UTC, comment #7: 

If the FREETYPE_PROPERTIES environment variable doesn't make a difference, then the machine with FreeType 2.8 (assuming a normal build of the library) uses the auto-hinter, at least for the `DejaVu Sans Mono' font.  You should check your fontconfig settings and switch to `full' hinting.  And just to be sure you should also check whether your DejaVu fonts are identical.

I'm at the end of my wit.  Please contact the xterm people so that they can help you debug the issue.  I suspect an incorrect rounding somewhere – most likely in xterm, but it can also be a bug in FreeType.  However, since I have no idea how xterm works, I can't further help.

Werner LEMBERG <wl>
Group administrator
Thu 05 Oct 2017 08:06:27 AM UTC, comment #6: 

fc-match says that it's "DejaVu Sans Mono" "Book", and if I use this font directly in the GNOME Terminal preferences, this looks the same and I get the same issue.

The dpi: Xft.dpi is 132 on this machine with a HiDPI screen. On another machine showing the same issue, Xft.dpi is 88.

The FREETYPE_PROPERTIES=truetype:interpreter-version=35 environment variable doesn't have any effect (neither on GNOME Terminal, nor on xterm).

For xterm, "*scaleHeight: 0.9" allows me to get the same window size as with FreeType 2.6.3, but I still get the blank line, which means that the rendering is very ugly. Moreover, the characters are not correctly erased (e.g. with zsh, when navigating through the command history).

Vincent Lefèvre <vinc17>
Thu 05 Oct 2017 05:44:58 AM UTC, comment #5: 

`Monospace Regular 10' is an alias.  Please try to find out what font this really is (for example, using the `fc-list' program) so that I can check the font's built-in line height values.  What dpi value are you using?

Additionally, your 2.6.3 Gnome snapshot obviously uses native TrueType hinting (perhaps v35), while the 2.8.0 snapshot uses either native v40 hinting or auto-hinting – if you magnify the image you can see that it is fuzzier along the x axis.  If the latter is true (which I believe), chances are very high that vertical scaling is different.

You can activate v35 hinting with setting an environment variable (provided your system is set up to use native TrueType hinting).

  FREETYPE_PROPERTIES=truetype:interpreter-version=35

In general, there is no reliable way for non-bitmap fonts to get `correct' vertical spacing.  There are too many ways to define `correct', unfortunately.  For example, TeX uses 1.2*em size as the line height by default, ignoring the fonts' line height values altogether.

I think that you have to adjust the vertical spacing manually to get block characters line up correctly.  KDE's `konsole' application allows this, but gnome doesn't, it seems.  BTW, xterm allows this also, cf.

  https://wiki.archlinux.org/index.php/Xterm#Adjust_line_spacing

Werner LEMBERG <wl>
Group administrator
Wed 04 Oct 2017 10:28:10 PM UTC, comment #4: 

I can reproduce the problem with GNOME Terminal. See the attached screenshots. This is with Monospace Regular 10.

Vincent Lefèvre <vinc17>
Wed 04 Oct 2017 09:16:16 PM UTC, comment #3: 

AFAIK, there are no such problems with applications that are based on Qt or Pango, for example KDE or Gnome – can you verify that?

It seems that xft is causing the differences you are noticing; cf. https://savannah.nongnu.org/bugs/?52124 for a similar report.

Alas, xft is largely unmaintained, AFAIK, but maybe you get an answer or advice if you write to the xorg people.

You might also try to compile FreeType with AF_CONFIG_OPTION_TT_SIZE_METRICS; maybe it fixes your issues.

Werner LEMBERG <wl>
Group administrator
Wed 04 Oct 2017 09:01:17 PM UTC, comment #2: 

I normally use 10. If I set 9.9, the window height is slightly smaller (but not as small as with 2.6) but I still get the blank line. Same result with 9.8, 9.7 and 9.6. Ditto with 9.5 except that the window is narrower, which is not OK.

Vincent Lefèvre <vinc17>
Wed 04 Oct 2017 08:32:06 PM UTC, comment #1: 

Try using a slightly smaller, fractional point size for your terminal font, e.g., 12.8pt instead of 13pt.

Werner LEMBERG <wl>
Group administrator
Wed 04 Oct 2017 12:02:54 PM UTC, original submission:  

Under Debian/unstable, after the upgrade of the libfreetype6 Debian package from 2.6.3-3.2 to 2.8-0.2, there is a blank line between characters. This has several consequences:

1. Tables look a bit ugly.

2. Windows (with the same number of text lines) are higher than before.

3. As a consequence of (2), this breaks existing configurations. For instance, the xterms started at the beginning of my X session appear partly off-screen.

This issue appears with both xterm and emacs (at least), and is more visible with xterm (but this might depend on the config).

I've attached two snapshots of xterm:
  ft263.png with libfreetype6 2.6.3-3.2 (correct)
  ft280.png with libfreetype6 2.8-0.2 (bad)

This is with the Monospace font (e.g. "xterm -fa Monospace"), which is, according to fc-match:
DejaVuSansMono.ttf: "DejaVu Sans Mono" "Book"

I initially reported this bug in the Debian BTS three months ago:

  https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=866960

but it didn't get any comment.

Vincent Lefèvre <vinc17>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attached Files
file #42118:  bar_gap.png added by podtelez (11KiB - image/png)
file #42079:  v35-v40-light.png added by wl (5KiB - image/png)
file #42066:  gnome-term-263.png added by vinc17 (41KiB - image/png - screenshots with GNOME Terminal)
file #42067:  gnome-term-280.png added by vinc17 (44KiB - image/png - screenshots with GNOME Terminal)
file #42059:  ft263.png added by vinc17 (1KiB - image/png)
file #42060:  ft280.png added by vinc17 (1KiB - image/png)

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by podtelez
  • -email is unavailable- added by podtelez (Posted a comment)
  • -email is unavailable- added by wl (Posted a comment)
  • -email is unavailable- added by vinc17 (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

     

    Follow 11 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2017-10-11 podtelez Carbon-Copy- Added -email is unavailable-
    2017-10-11 podtelez Attached File- Added bar_gap.png, #42118
    2017-10-08 wl StatusNeed Info Wont Fix
        Open/ClosedOpen Closed
    2017-10-07 wl Attached File- Added v35-v40-light.png, #42079
    2017-10-06 wl StatusNone Need Info
        Assigned toNone wl
    2017-10-04 vinc17 Attached File- Added gnome-term-263.png, #42066
        Attached File- Added gnome-term-280.png, #42067
    2017-10-04 vinc17 Attached File- Added ft263.png, #42059
        Attached File- Added ft280.png, #42060

    Back to the top

    Powered by Savane 3.14-d582.
    Corresponding source code