Tue 06 Jul 2010 05:51:24 PM UTC, comment #15:
Thank you! I close this issue.
|
Tue 06 Jul 2010 01:31:46 PM UTC, comment #14:
FWIW, I don't see any crashes/valgrind warnings now.
|
Sun 04 Jul 2010 07:50:23 PM UTC, comment #13:
This looks good! Two minor formatting issues:
. Please use two spaces after a period if a new sentence starts.
. Please replace
FT_ERROR(( "foo " ));
FT_ERROR(( "bar" ));
with
FT_ERROR (( "foo"
" bar" ));
|
Sun 04 Jul 2010 11:09:40 AM UTC, comment #12:
oops, sorry for my mistake in file name.
the bug number is 30059, not 30509.
I replaced.
(file #20893)
|
Sun 04 Jul 2010 11:03:20 AM UTC, comment #11:
Attached is a patch to load all charmap but search only
#0 - #FT_MAX_CACHEABLE_CHARMAP (=15) subtables
in charmap lookup by FT2 public API.
I will try to write additional patch to insert/shrink the array
of face->charmap[].
(file #20892)
|
Sun 04 Jul 2010 04:35:54 AM UTC, comment #10:
Werner, thank you for comment.
Checking the interfaces of cache subsystem, I think, very wide change
is required to insert the hooks to mark exceptional fonts including many
charmaps and passing the marks to cache subsystem. Also it will insert
many dirty flags into unrelated objects.
I have to conclude that your proposal is the most appropriate solution
of this issue.
I'm writing a patch which separates the maximum number of
the charmaps to include/freetype/config/ftoption.h
(because this restriction is only for the rogue-client compatibility,
enabled by FT_CONFIG_OPTION_OLD_INTERNALS).
Then, src/cache/ftccmap.c and the charmap selector reflect it.
|
Sat 03 Jul 2010 12:54:47 PM UTC, comment #9:
What about the following:
. While loading a font, if there are more than 16 cmaps,
take the first 16 and ignore all others.
. If there is no Unicode cmap within those 16 cmaps,
search one and replace, say, cmap #15 (the last we
actually use) with it.
|
Fri 02 Jul 2010 05:27:54 PM UTC, comment #8:
I think this issue is related with garbled cmap table
of sigsegv3.ttf and back-compatibility hook in cache
system. My analysis is following.
1) FT2 recognizes garbled cmap table as if it includes 106 subtables.
If I execute ftdump for sigsegv3.ttf, it reports 106 subtables.
charmaps
0: platform 0, encoding 468
1: platform 0, encoding 596
...
100: platform 0, encoding 14749
101: platform 0, encoding 14157
102: platform 0, encoding 11944
103: platform 0, encoding 11953
104: platform 0, encoding 11964
105: platform 0, encoding 11966
106: platform 0, encoding 11972 (active)
FT2 does not count heavily broken subtable. If I count the declared
number of subtables (including heavily broken subtables), it is 464.
(oh my god)
All encoding is invalid, thus FT2 selects the final (106th) subtable
for default/Unicode cmap subtable.
2) Too large indice of cmap subtable makes FT2 cache system
misunderstand as if the request comes from older/rogue client.
There is a part in ftccmap.c::FTC_CMapCache_Lookup()
310 #ifdef FT_CONFIG_OPTION_OLD_INTERNALS */
311
312 /*
313 * Detect a call from a rogue client that thinks it is linking
314 * to FreeType 2.1.7. This is possible because the third parameter
315 * is then a character code, and we have never seen any font with
316 * more than a few charmaps, so if the index is very large...
317 *
318 * It is also very unlikely that a rogue client is interested
319 * in Unicode values 0 to 15.
320 *
321 * NOTE: The original threshold was 4, but we found a font from the
322 * Adobe Acrobat Reader Pack, named `KozMinProVI-Regular.otf',
323 * which contains more than 5 charmaps.
324 */
325 if ( cmap_index >= 16 && !no_cmap_change )
326 {
327 FTC_OldCMapDesc desc = (FTC_OldCMapDesc) face_id;
328
329
330 char_code = (FT_UInt32)cmap_index;
331 query.face_id = desc->face_id;
332
333
334 switch ( desc->type )
335 {
336 case FTC_OLD_CMAP_BY_INDEX:
337 query.cmap_index = desc->u.index;
338 query.char_code = (FT_UInt32)cmap_index;
339 break;
340
341 case FTC_OLD_CMAP_BY_ENCODING:
342 {
343 FT_Face face;
344
345
346 error = FTC_Manager_LookupFace( cache->manager, desc->face_id,
347 &face );
348 if ( error )
349 return 0;
350
351 FT_Select_Charmap( face, desc->u.encoding );
352
353 return FT_Get_Char_Index( face, char_code );
354 }
355
356 default:
357 return 0;
358 }
359 }
360 else
361
362 #endif /* FT_CONFIG_OPTION_OLD_INTERNALS */
363
364 {
365 query.face_id = face_id;
366 query.cmap_index = (FT_UInt)cmap_index;
367 query.char_code = char_code;
368 }
369
370 hash = FTC_CMAP_HASH( face_id, cmap_index, char_code );
371
372 #if 1
373 FTC_CACHE_LOOKUP_CMP( cache, ftc_cmap_node_compare, hash, &query,
374 node, error );
The query to be passed to FTC_CACHE_LOOKUP_CMP() is different
in normal case (latter) and back-compatible case (former). The huge
cmap indice misguides the query for normal case to back-compatible
case. The node is set to inappropriate value. Afterwards, it makes
FTC_Manager_LookupFace() crashed, by invalid value in node->face_id.
3) FT_CONFIG_OPTION_OLD_INTERNALS is defined in
include/freetype/config/ftoption.h by default.
If I disable it, the crash disappears. Maybe, disabling the part in above
(ftccmap.c:310-362) is sufficient to prevent SEGV (at least, I checked
it on my machine).
So, jagger, could you try FT2 built with disabling
FT_CONFIG_OPTION_OLD_INTERNALS?
The next task of me is thinking new method to distinguish old
rogue client and broken cmap.
|
Fri 02 Jul 2010 10:04:07 AM UTC, comment #7:
Also I could reproduce the issue on GNU/Linux i386 machine.
When I invoke ftstring directly, it crashes by SEGV, but
if I invoke via valgrind, ftstring shows a window of 3 CID=0 glyph.
Here is my valgrind log on i386 machine.
==405== Memcheck, a memory error detector
==405== Copyright (C) 2002-2009, and GNU GPL'd, by Julian Seward et al.
==405== Using Valgrind-3.5.0-Debian and LibVEX; rerun with -h for copyright info
==405== Command: ./bin/ftstring -m ABC 20 sigsegv3.ttf
==405==
==405== Syscall param write(buf) points to uninitialised byte(s)
==405== at 0x42313F3: __write_nocancel (in /lib/i686/cmov/libc-2.10.2.so)
==405== by 0x407829E: _X11TransWrite (in /usr/lib/libX11.so.6.2.0)
==405== by 0x407DBD5: ??? (in /usr/lib/libX11.so.6.2.0)
==405== by 0x407DCAA: _XReply (in /usr/lib/libX11.so.6.2.0)
==405== by 0x4063F70: XInternAtom (in /usr/lib/libX11.so.6.2.0)
==405== by 0x4077DCA: XSetWMProperties (in /usr/lib/libX11.so.6.2.0)
==405== by 0x80BEF8E: gr_x11_surface_init (grx11.c:1409)
==405== by 0x80B7D46: grNewSurface (grdevice.c:149)
==405== by 0x804AF89: FTDemo_Display_New (ftcommon.c:102)
==405== by 0x804A70B: main (ftstring.c:556)
==405== Address 0x43299b8 is 272 bytes inside a block of size 16,384 alloc'd
==405== at 0x4023F8B: calloc (vg_replace_malloc.c:418)
==405== by 0x4068CBD: XOpenDisplay (in /usr/lib/libX11.so.6.2.0)
==405== by 0x80BE3D5: gr_x11_device_init (grx11.c:903)
==405== by 0x80B7F86: grInitDevices (grinit.c:76)
==405== by 0x804AF5B: FTDemo_Display_New (ftcommon.c:95)
==405== by 0x804A70B: main (ftstring.c:556)
==405==
==405== Invalid read of size 4
==405== at 0x804B10B: my_face_requester (ftcommon.c:212)
==405== by 0x80AA128: ftc_face_node_init (ftcmanag.c:245)
==405== by 0x80A9BB1: FTC_MruList_New (ftcmru.c:269)
==405== by 0x80AA296: FTC_Manager_LookupFace (ftcmanag.c:327)
==405== by 0x80AB5BA: FTC_CMapCache_Lookup (ftccmap.c:398)
==405== by 0x804BCA2: FTDemo_Get_Index (ftcommon.c:645)
==405== by 0x804C5CF: FTDemo_String_Set (ftcommon.c:1045)
==405== by 0x8049A12: event_font_change (ftstring.c:138)
==405== by 0x804A76A: main (ftstring.c:567)
==405== Address 0x43260a0 is 3 bytes after a block of size 13 alloc'd
==405== at 0x4024C4C: malloc (vg_replace_malloc.c:195)
==405== by 0x804B7DA: FTDemo_Install_Font (ftcommon.c:434)
==405== by 0x804A654: main (ftstring.c:541)
==405==
==405== Syscall param open(filename) points to unaddressable byte(s)
==405== at 0x4230E83: __open_nocancel (in /lib/i686/cmov/libc-2.10.2.so)
==405== by 0x804EA76: FT_Stream_New (ftobjs.c:162)
==405== by 0x8050E15: FT_Open_Face (ftobjs.c:1957)
==405== by 0x804FDDA: FT_New_Face (ftobjs.c:1172)
==405== by 0x804B16F: my_face_requester (ftcommon.c:219)
==405== by 0x80AA128: ftc_face_node_init (ftcmanag.c:245)
==405== by 0x80A9BB1: FTC_MruList_New (ftcmru.c:269)
==405== by 0x80AA296: FTC_Manager_LookupFace (ftcmanag.c:327)
==405== by 0x80AB5BA: FTC_CMapCache_Lookup (ftccmap.c:398)
==405== by 0x804BCA2: FTDemo_Get_Index (ftcommon.c:645)
==405== by 0x804C5CF: FTDemo_String_Set (ftcommon.c:1045)
==405== by 0x8049A12: event_font_change (ftstring.c:138)
==405== Address 0x73676973 is not stack'd, malloc'd or (recently) free'd
==405==
Execution completed successfully.
==405==
==405== HEAP SUMMARY:
==405== in use at exit: 23,891 bytes in 390 blocks
==405== total heap usage: 1,443 allocs, 1,053 frees, 1,846,045 bytes allocated
==405==
==405== LEAK SUMMARY:
==405== definitely lost: 0 bytes in 0 blocks
==405== indirectly lost: 0 bytes in 0 blocks
==405== possibly lost: 0 bytes in 0 blocks
==405== still reachable: 23,891 bytes in 390 blocks
==405== suppressed: 0 bytes in 0 blocks
==405== Rerun with --leak-check=full to see details of leaked memory
==405==
==405== For counts of detected and suppressed errors, rerun with: -v
==405== Use --track-origins=yes to see where uninitialised values come from
==405== ERROR SUMMARY: 7 errors from 3 contexts (suppressed: 29 from 8)
|
Fri 02 Jul 2010 09:47:50 AM UTC, comment #6:
I could reproduce SEGV on my GNU/Linux amd64 machine.
I will try to find the root of the problem.
|
Sun 20 Jun 2010 01:39:59 PM UTC, comment #5:
I still can't find the cause of the problem. `font->file_address' gets initialized in `FTDemo_Install_Font' which is called before the cache is created.
How do you compile the demo programs and the library? Maybe it is sensitive to compilation flags...
|
Thu 10 Jun 2010 07:15:42 PM UTC, comment #4:
Hm... so my guess is that problem lies in
freetype2-demos/src/ftcommon.c:212
if ( font->file_address != NULL )
font->file_address is uninitialized (it means it may not be NULL).
The font->file_addres, aka file_base should be initialized to NULL duriong creation (I presume in freetype2/src/cache/ftccmap.c:393), or there should be no
if ( font->file_address != NULL )
check in the freetype2-demos/src/ftcommon.c:212 assuming that font->file_address might be uninitialized
(gdb) hbreak my_face_requester
Hardware assisted breakpoint 1 at 0x804b022: file /usr/local/google/src/freetype2-demos/src/ftcommon.c, line 208.
(gdb) r -m A 20 sigsegv3.ttf
Starting program: /usr/local/google/fuzz/ttf/ftstring -m A 20 sigsegv3.ttf
Breakpoint 1, my_face_requester (face_id=0x8105c50, lib=0x80e48b0, request_data=0x0, aface=0x80fdfbc) at /usr/local/google/src/freetype2-demos/src/ftcommon.c:208
208 PFont font = (PFont)face_id;
(gdb) nexti
0x0804b025 208 PFont font = (PFont)face_id;
(gdb)
212 if ( font->file_address != NULL )
(gdb) p font->file_address
$1 = (void *) 0x6105a20
(gdb) x/x 0x6105a20
0x6105a20: Cannot access memory at address 0x6105a20
|
Thu 10 Jun 2010 07:02:49 PM UTC, comment #3:
I left out everything not related to the problem (mostly Xorg libs)..
VALGRIND_MEMORY_LIMIT_IN_MB=4095; VALGRIND_TIMEOUT=3600
==4161== Memcheck, a memory error detector
==4161== Copyright (C) 2002-2009, and GNU GPL'd, by Julian Seward et al.
==4161== Using Valgrind-3.6.0.SVN and LibVEX; rerun with -h for copyright info
==4161== Command: ./ftstring -m ABC 20 sigsegv3.ttf
==4161==
...
...
==4654== Invalid read of size 4
==4654== at 0x804B02B: my_face_requester /usr/local/google/src/freetype2-demos/src/ftcommon.c:212
==4654== by 0x80A9304: ftc_face_node_init /usr/local/google/src/freetype2/src/cache/ftcmanag.c:245
==4654== by 0x80A8D8D: FTC_MruList_New /usr/local/google/src/freetype2/src/cache/ftcmru.c:269
==4654== by 0x80A9472: FTC_Manager_LookupFace /usr/local/google/src/freetype2/src/cache/ftcmanag.c:327
==4654== by 0x80AA796: FTC_CMapCache_Lookup /usr/local/google/src/freetype2/src/cache/ftccmap.c:398
==4654== by 0x804BBC2: FTDemo_Get_Index /usr/local/google/src/freetype2-demos/src/ftcommon.c:645
==4654== by 0x804C4EF: FTDemo_String_Set /usr/local/google/src/freetype2-demos/src/ftcommon.c:1045
==4654== by 0x8049968: event_font_change /usr/local/google/src/freetype2-demos/src/ftstring.c:138
==4654== by 0x804A6C0: main /usr/local/google/src/freetype2-demos/src/ftstring.c:567
==4654== Address 0x6f190a0 is 3 bytes after a block of size 13 alloc'd
==4654== at 0x47E20D8: malloc /tmp/vg/coregrind/m_replacemalloc/vg_replace_malloc.c:236
==4654== by 0x804B6FA: FTDemo_Install_Font /usr/local/google/src/freetype2-demos/src/ftcommon.c:434
==4654== by 0x804A5AA: main /usr/local/google/src/freetype2-demos/src/ftstring.c:541
==4654==
==4654== Syscall param open(filename) points to unaddressable byte(s)
==4654== at 0x6E1E883: __open_nocancel /build/buildd/eglibc-2.11.1/io/../sysdeps/unix/syscall-template.S:82
==4654== by 0x804E936: FT_Stream_New /usr/local/google/src/freetype2/src/base/ftobjs.c:162
==4654== by 0x8050CA7: FT_Open_Face /usr/local/google/src/freetype2/src/base/ftobjs.c:1935
==4654== by 0x804FC9A: FT_New_Face /usr/local/google/src/freetype2/src/base/ftobjs.c:1172
==4654== by 0x804B08F: my_face_requester /usr/local/google/src/freetype2-demos/src/ftcommon.c:219
==4654== by 0x80A9304: ftc_face_node_init /usr/local/google/src/freetype2/src/cache/ftcmanag.c:245
==4654== by 0x80A8D8D: FTC_MruList_New /usr/local/google/src/freetype2/src/cache/ftcmru.c:269
==4654== by 0x80A9472: FTC_Manager_LookupFace /usr/local/google/src/freetype2/src/cache/ftcmanag.c:327
==4654== by 0x80AA796: FTC_CMapCache_Lookup /usr/local/google/src/freetype2/src/cache/ftccmap.c:398
==4654== by 0x804BBC2: FTDemo_Get_Index /usr/local/google/src/freetype2-demos/src/ftcommon.c:645
==4654== by 0x804C4EF: FTDemo_String_Set /usr/local/google/src/freetype2-demos/src/ftcommon.c:1045
==4654== by 0x8049968: event_font_change /usr/local/google/src/freetype2-demos/src/ftstring.c:138
==4654== Address 0x73676973 is not stack'd, malloc'd or (recently) free'd
==4654==
|
Wed 09 Jun 2010 04:42:11 AM UTC, comment #2:
Again a problem which works just fine on my box. valgrind shows nothing :( Without being able to reproduce the bug it is quite hard for me to find out the real culprit.
May I ask you for further investigation?
BTW, the current SVN version of valgrind has experimental support for the ARM architecture, according to an announcement on http://www.valgrind.org.
|
Tue 08 Jun 2010 02:17:30 PM UTC, comment #1:
This link should work better: http://alt.swiecki.net/j/f/sigsegv3.ttf
|
Mon 07 Jun 2010 02:02:08 AM UTC, original submission:
Not sure if this is problem with freetype or freetype-demos. Cannot reporduce it with any freetype based app by simply displaying fonts (like firefox). Anyway ;)..
Font file: http://alt.swiecki.net/j/f/SIGSEGV.PC.0x8056ad7.CODE.1.ADDR.0x30333037.INSTR.movsx_eax,_[eax.TIME.2010-06-07.02.25.04.PID.10097.ttf]
Linux amd64, with 32bit app (compiled with -m32 -O0), git sources as of 2010-06-05
$ ftview -m ABC 20 SIGSEGV.PC.0x8056ad7.CODE.1.ADDR.0x30333037.INSTR.movsx_eax\,_\[eax\].TIME.2010-06-07.02.25.04.PID.10097.ttf
Segmentation fault (core dumped)
Program received signal SIGSEGV, Segmentation fault.
0x08056ad7 in FT_Stream_ReadLong (stream=0x8134a60, error=0xffffd688) at /usr/local//src/freetype2/src/base/ftstream.c:643
643 result = FT_NEXT_LONG( p );
(gdb) bt
#0 0x08056ad7 in FT_Stream_ReadLong (stream=0x8134a60, error=0xffffd688) at /usr/local//src/freetype2/src/base/ftstream.c:643
#1 0x080912f3 in sfnt_open_font (stream=0x8134a60, face=0x8134a90) at /usr/local//src/freetype2/src/sfnt/sfobjs.c:370
#2 0x0809156c in sfnt_init_face (stream=0x8134a60, face=0x8134a90, face_index=777406277, num_params=0, params=0x0) at /usr/local//src/freetype2/src/sfnt/sfobjs.c:452
#3 0x0805dac5 in tt_face_init (stream=0x8134a60, ttface=0x8134a90, face_index=777406277, num_params=0, params=0x0) at /usr/local//src/freetype2/src/truetype/ttobjs.c:225
#4 0x0804fa75 in open_face (driver=0x80e6978, stream=0x8134a60, face_index=777406277, num_params=0, params=0x0, aface=0xffffd794) at /usr/local//src/freetype2/src/base/ftobjs.c:1110
#5 0x08050d0d in FT_Open_Face (library=0x80e28a8, args=0xffffd7d0, face_index=777406277, aface=0x80fc154) at /usr/local//src/freetype2/src/base/ftobjs.c:1995
#6 0x0804fbb9 in FT_New_Memory_Face (library=0x80e28a8, file_base=0x37646136 <Address 0x37646136 out of bounds>, file_size=1146045230, face_index=777406277, aface=0x80fc154) at /usr/local//src/freetype2/src/base/ftobjs.c:1199
#7 0x0804ae52 in my_face_requester (face_id=0x8135630, lib=0x80e28a8, request_data=0x0, aface=0x80fc154) at /usr/local//src/freetype-demos/src/ftcommon.c:213
#8 0x080a778e in ftc_face_node_init (ftcnode=0x80fc148, ftcface_id=0x8135630, ftcmanager=0x80e7658) at /usr/local//src/freetype2/src/cache/ftcmanag.c:245
#9 0x080a7226 in FTC_MruList_New (list=0x80e76b4, key=0x8135630, anode=0xffffd8b0) at /usr/local//src/freetype2/src/cache/ftcmru.c:269
#10 0x080a78ff in FTC_Manager_LookupFace (manager=0x80e7658, face_id=0x8135630, aface=0xffffd8d8) at /usr/local//src/freetype2/src/cache/ftcmanag.c:327
#11 0x080a8b79 in FTC_CMapCache_Lookup (cmap_cache=0x80e7880, face_id=0x8103bd0, cmap_index=106, char_code=106) at /usr/local//src/freetype2/src/cache/ftccmap.c:398
#12 0x0804b9ca in FTDemo_Get_Index (handle=0x80df008, charcode=65) at /usr/local//src/freetype-demos/src/ftcommon.c:645
#13 0x0804c2c9 in FTDemo_String_Set (handle=0x80df008, string=0xffffdc73 "ABC") at /usr/local//src/freetype-demos/src/ftcommon.c:1045
#14 0x080497d9 in event_font_change (delta=0) at /usr/local//src/freetype-demos/src/ftstring.c:138
#15 0x0804a50c in main (argc=0, argv=0xffffdaf8) at /usr/local//src/freetype-demos/src/ftstring.c:566
My analysis:
in FTC_CMapCache_Lookup (freetype2/src/cache/ftccmap.c:393) a stack variable is created
FT_Face face;
at the offset 0x10, it holds random value (in my case 0x37646136)
Breakpoint 23, FTC_Manager_LookupFace (manager=0x80e7658, face_id=0x8135630, aface=0xffffd8d8) at /usr/local/src/freetype2/src/cache/ftcmanag.c:316
316 if ( aface == NULL )
(gdb) x/x face_id+0x10
0x8135640: 0x37646136
later in the code, that value (via face_id structure, and later on as a file_base variable as seen in the argument to FT_New_Memory_Face in the stacktrace above) will be used to initialize new stream's stream->base field
In the function crashing:
(gdb) p stream
$74 = (FT_Stream) 0x8134a60
(gdb) p p
$72 = (FT_Byte *) 0x3764613a <Address 0x3764613a out of bounds>
(gdb) p stream->base
$73 = (unsigned char *) 0x37646136 <Address 0x37646136 out of bounds>
In my opinion something forgets to initialize a field in the face_id structure. I didn't investigate deeper. Not sure if it's some component of freetype-demo which interacts badly with the library (and maybe overwrites some parts of the cache), or maybe there's really something in the library that is broken. Hope that you guys with better understanding of the library internals will find it much faster than me.
|