bugGNU roff - Bugs: bug #65601, [troff] bogus 'bogus composite'...

 
 

bug #65601: [troff] bogus 'bogus composite' errors introduced by commit 6008b6b7aa

Submitter:  Deri James <deri>
Submitted:  Tue 16 Apr 2024 11:21:28 AM UTC
   
 
Category:  Core Severity:  3 - Normal
Item Group:  Incorrect behaviour Status:  Fixed
Privacy:  Public Assigned to:  gbranden
Open/Closed:  Closed Planned Release:  1.24.0
* Mandatory Fields

Add a New Comment Rich Markup
   

Jump to the original submission

Thu 02 May 2024 11:17:10 AM UTC, comment #7: 

Hi Deri,

Long reply.  Feel free to take your time with it.  More on that very issue below.

comment #5:

> > It would help if you didn't assume autocratic motives behind my code changes.
>
> But I'm unsure how to achieve that.


I find this admission startling.  (More fool me, perhaps.)  One does this by first having an internal dialogue wherein one weighs alternative explanations for the observations one has made before committing a hypothesis to paper.  (Or bits.)

> By autocratic I am simply meaning that you seem to assume that your ideas are automatically better than your inferiors.


What evidence do you have for this claim?  Considering you in particular, I have expressed respect for your superior knowledge of and skill in graphic design and PDF language and composition in multiple forums, including a published interview.  While I'm here, I'd add Perl programming to that list, and believe I have already said as much on the mailing list.

And what's this about "inferiors"?  Where and how do I assert my superiority over anyone?  Please cite examples.

On the other side of the coin, should anyone be deliberately committing anything less than the best code they can come up with under the circumstances?  And cannot peer review processes aid them to do so?  I'll return to that point.

> In this bug for example, I initially gave you a one liner which illustrated the problem:-


> xzcat /usr/share/man/man7/iso_8859-6.7.xz |preconv|nroff -Tutf8 -man -t -z


I wasn't sure what to make of it given my misapprehension of how special character escape sequences (which, in this case, get internally transformed, even after preconv works on them) interact with groff's composite character concept.

Your bug report resembled a classic case of problem delivery by a manager, wherein one walks up to a worker's desk, drops a stack of papers on it, says "fix this", and walks away.

This approach to what we might call management is pretty common in my professional experience, so I wouldn't say you violated any norms with that, but by (1) not offering any resources in aid; nor (2) articulating any other constraints on the problem's solution, it's fair for the worker to assume they have broad latitude to resolve the issue as they see fit.  (This is largely by design in corporate environments, as the manager then can attempt to claim ignorance to escape liability when the worker's remedy ends up running afoul of good taste, company policy, or the law.)  The "manager" has delegated management, leaving the worker to manage themselves.

Which, I would add, ain't so bad.  The institutional risk is then that the workers might figure out what value the manager isn't bringing to the organization, so to speak.

But, to return from my socialist critique of the U.S. tech sector workplace...

> Then, over 2 weeks later, you post more than 1200 words, essentially defending your change.


You have surprised me again!  Not so long ago you complained of me producing too many words in too short a time.  "Yesterday, you managed over 2900 words in approx 80 minutes whilst also doing a code review, and probably 3 other things as well!"

(Probably not even one other thing, if I'm honest.  I have many limitations and a large cost when mentally switching contexts is one of them.)

Perhaps half of those 1200 words are quotations of text with which I assume you're familiar as an active groff developer.  (There is the complication of your--interrupted--sabbatical, to which I return below.)

Is there a word count limit you'd like me to adhere to in my communications with you?  Do quoted words count toward the limit?  Does it matter if the quotations are of you, of myself, of third parties, or of groff documentation, change logs, etc.?  Does the fact that I'm the author of some, but certainly not all material in the last category influence this calculus?

A person could reasonably regard these questions as persnickety.  I think a person could also wonder why you're assuming that I require you to do anything but take all the time you need to respond.  Or don't reply at all if my statements don't merit response, in your view.  This is a volunteer project.

But, if you are unavailable to collaborate on a solution, for whatever reason, in such a volunteer project you must expect development to proceed in your absence.  I know that you grasp this principle--to it, as well, I return below.

> What am I to think? You know that if you "snow" me under with words I will either give up or make myself ill trying to answer all your points.


Again you are assuming an extremely negative attitude on my part.  Why?  If you require me to rate-limit my communications with you, please give me at least some parameters to work with.

But to critique your analogy, 2 weeks sounds like ample time to prepare for a snowstorm.  Especially comparing the ratios of 1200 words in two weeks to 2900 words in 80 minutes.  A quick bc(1) computation reveals that three orders of magnitude separate these extremes.

Nevertheless, even between those generously spaced guard rails, if I'm candid I don't think there is a Goldilocks solution here.  I surmise that your biggest grievance with me lies elsewhere.

> I know you are not stupid so you must have run the example and seen the errors it produced, but your reply was "no I am right to make this change".


Not exactly.  If I thought your report was invalid, I'd have closed it as such.  Instead, it made me come to regard my change as incomplete.  Hence all the stuff about font-specific composite characters.  That was me trying to think through the issue.  I do that a lot.  It gives others the chance to catch me out in error, so that my comprehension of the system can improve.  I feel that expertise should be shared, not hoarded.  This attitude should not shock you given the volume of my contributions to documentation.

And, regarding stupidity, I manage it all the time.  Like a lot of people, I think.  My opinion is that engineering is, in part, the practice of refashioning our tools and surroundings such that our innate stupidity has less horrible consequences for us than it otherwise might.

For a distraction from our dispute, I recommend the story of the Los Alamos "demon core", an assembly of plutonium that managed to kill two Ph.D. researchers on separate occasions thanks to ad hoc processes of materials handling rather than engineered ones.  Slotin and Daghlian were avowedly not stupid men.  Yet they each managed to do stupid things.  Because they're human.

> I hope you see my point that this behaviour seems autocratic.


No, not at all.  I think you have expectations of me that I am failing to meet, but which you have not clearly expressed.

> I had to simplify the one liner for you to see the issue, but you then insist that what you did was correct in some way, but you would undo it anyway.


How do you know that?  Especially given that I gave up, reverted the commit, and pushed.  I guess I'll have to ask you to take my word for it that the (first) one-liner in comment #3 was a necessary and sufficient condition for me to do that.  Another interpretation is I don't care about correct behavior in groff one way or the other, and you heroically prevailed over the autocrat in this instance.

I don't think St. George and the dragon is a good model for teamwork.

But if I'm that easily defeated, I must underwhelm as a nemesis.

> Unfortunately, there are other examples of autocratic behaviour. How about taking advantage of my sabbatical to introduce a change to pdf.tmac which not only had bugs in it,


Introducing changes introduces bugs.  That is a fact of software development everywhere.  If you question my commitment to quality, then I wonder how you account for the approximately 200 automated tests I've written for groff.

> but caused one job to go from a few seconds to 13 minutes.


I have not been able to reproduce a degradation this severe, nor has Alex Colomar.  Maybe you have another succinct comment #3-esque one-liner with which to penetrate my scaly green hide again.

> A reasonable person would consult the author of pdf.tmac after his sabbatical,


You haven't said you've returned from it, and indeed you have not replied to the groff list even in threads (mostly from Alex, I think) that looped it in.  I therefore assume that are, technically, still on that sabbatical.  But a FLOSS project does not suspend operations because one contributor goes on a sabbatical.  When I had a death in the family and announced my reduced availability, I offered no clear timeline for my return to full status, and expressed no wish for development to halt in my absence.  Nor would I suppose that people should have understood my message to imply such.  I still don't "feel" 100%, but I'm slogging through.

If I wanted to repay you in the same coin with which you're trading with me, I could observe that it took you only 3 days to interrupt your sabbatical and swoop in with changes that were under-documented ("[V]arious problems have been resolved") and themselves flawed.

So--is that really the road you want us walking down together?  I would prefer not to.  I think we have complementary strengths when it comes to groff development, strengths that can redound to the quality of the software and the experiences of its users.

I have expressed respect for your abilities on multiple occasions, and expect to continue to do so.  This respect is, however, not something you seem to be absorbing, and so a tone of resentment creeps into your communications with me.

Again, I feel that there are expectations you have of me (and possibly of other groff contributors) that you are not expressing.

To put my finger on one that I've raised before, more than once I think, and to which you have not responded: if you feel that gropdf should be your personal playground, and other developers should not be touching it without your prior agreement (much as Peter has with mom [excepting, as I recall, its Automake script and man page, which I do not understand to express his authorial voice]), then you need to say so, and it needs to be moved to the "contrib" directory of the repository so that the state of its maintenance is clear.

The impact of such a change on users would be practically nil.  But it might make development processes go more smoothly.

And, I must note, there are going to be limits even to the authorial playground principle.  Here's a sobering recent case study.  Ideally, *all* code gets reviewed.  Even that in playgrounds.

If a contribution does not merit peer review, it is not worth making in the first place.

> an autocrat would just go and do it.


An autocrat feels no need to justify or motivate their actions.  I explain mine at such length that it exasperates you (see supra).  Perhaps you are in need of a better metaphor.
 

> Even something as innocuous as spacewidth in afmtodit yields examples. Rather than something like "I agree with Deri" or "Deri is correct", we see "Looks like Deri is right", which could be construed as adding an element of surprise!


That's...one way to interpret it--again, one that is toward the negative pole of plausible readings.  A more positive one is that you had considered a possibility I hadn't--or thought I hadn't.  As I said in that very ticket, apparently I had made the change you recommended (or something I thought was similar), already, years previously, unprompted by you (as far as I recall), hadn't even remembered it, and was surprised at my own lack of recollection.

I guess how far-fetched that is is an inescapably subjective matter.

(I see now that you've further replied to bug #65619.  Fortunately, my joints do not ache with the threat of an impending blizzard.)

> And, I agree, I am skating on thin ice here, because we are considering autocratic behaviour, and mere words add nought, to the scales, it just may be indicative of the mind set which accompanies autocratic behaviour.


This may explain why I've been in such a hurry to seize the title of official GNU maintainer of groff.  Seven years and I still ain't got it...

A ridiculous hairpiece and trophy wife also continue to elude me.  And my bankruptcy counter sits at a pathetic zero.  At this rate I'll never get around to running for U.S. President!

To return to a serious tone, I think you have misconceived my function and activities in this project.  I recommend that you ask other people whom you respect how often they interpret my actions in the same light you have done.

G. Branden Robinson <gbranden>
Group administrator
Thu 02 May 2024 09:29:19 AM UTC, comment #6: 


commit bedd4ca476bcb9da7fff1b7fad9e0aeb9ad3cca5 (HEAD -> master, origin/master, origin/HEAD)
Author: G. Branden Robinson <g.branden.robinson@gmail.com>
Date:   Wed May 1 22:07:15 2024 -0500

    Revert "[troff]: Diagnose bogus composite char escapes."

    This reverts commit 6008b6b7aa2920035e09d1dea44d262d30391195.

    Fixes <https://savannah.gnu.org/bugs/?65601>.  Thanks to Deri James for
    the report.


G. Branden Robinson <gbranden>
Group administrator
Thu 02 May 2024 08:09:00 AM UTC, comment #5: 


> It would help if you didn't assume autocratic motives behind my code changes.


But I'm unsure how to achieve that. By autocratic I am simply meaning that you seem to assume that your ideas are automatically better than your inferiors. In this bug for example, I initially gave you a one liner which illustrated the problem:-

xzcat /usr/share/man/man7/iso_8859-6.7.xz |preconv|nroff -Tutf8 -man -t -z

Then, over 2 weeks later, you post more than 1200 words, essentially defending your change. What am I to think? You know that if you "snow" me under with words I will either give up or make myself ill trying to answer all your points. I know you are not stupid so you must have run the example and seen the errors it produced, but your reply was "no I am right to make this change". I hope you see my point that this behaviour seems autocratic. I had to simplify the one liner for you to see the issue, but you then insist that what you did was correct in some way, but you would undo it anyway.

Unfortunately, there are other examples of autocratic behaviour. How about taking advantage of my sabbatical to introduce a change to pdf.tmac which not only had bugs in it, but caused one job to go from a few seconds to 13 minutes. A reasonable person would consult the author of pdf.tmac after his sabbatical, an autocrat would just go and do it.

Even something as innocuous as spacewidth in afmtodit yields examples. Rather than something like "I agree with Deri" or "Deri is correct", we see "Looks like Deri is right", which could be construed as adding an element of surprise! And, I agree, I am skating on thin ice here, because we are considering autocratic behaviour, and mere words add nought, to the scales, it just may be indicative of the mind set which accompanies autocratic behaviour.


Deri James <deri>
Group Member
Thu 02 May 2024 03:00:20 AM UTC, comment #4: 

comment #3:

> It is very simple!


> [derij@pip build (master)]$ echo "This is the arabic alef with a madda above -> آ"|groff -Tutf8 -Kutf8 -z
> troff:<standard input>:1: error: cannot format glyph: 'u0627_0653' is not a valid composite character


Yes, thank you--that's helpful.

> It is no longer possible to output this particular utf-8 character (along with hundreds of others), all perfectly valid utf-8 to the utf-8 device.


Apparently.  Bummer.

> I can't believe that is your intention, to police which particular parts of unicode are acceptable to you.


But you can contemplate it seriously enough to put it into words.

It would help if you didn't assume autocratic motives behind my code changes.

This is the complete repertoire of composite components known to groff for quite a while now.


87909b1715 (Werner LEMBERG      2003-03-01 07:34:52 +0000  1) .\" composite.tmac
87909b1715 (Werner LEMBERG      2003-03-01 07:34:52 +0000  2) .
77d9af6df8 (Werner LEMBERG      2003-03-12 23:00:25 +0000  3) .do composite ga u0300
77d9af6df8 (Werner LEMBERG      2003-03-12 23:00:25 +0000  4) .do composite `  u0300
77d9af6df8 (Werner LEMBERG      2003-03-12 23:00:25 +0000  5) .do composite aa u0301
77d9af6df8 (Werner LEMBERG      2003-03-12 23:00:25 +0000  6) .do composite '  u0301
77d9af6df8 (Werner LEMBERG      2003-03-12 23:00:25 +0000  7) .do composite a^ u0302
77d9af6df8 (Werner LEMBERG      2003-03-12 23:00:25 +0000  8) .do composite ^  u0302
77d9af6df8 (Werner LEMBERG      2003-03-12 23:00:25 +0000  9) .do composite a~ u0303
77d9af6df8 (Werner LEMBERG      2003-03-12 23:00:25 +0000 10) .do composite ~  u0303
77d9af6df8 (Werner LEMBERG      2003-03-12 23:00:25 +0000 11) .do composite a- u0304
77d9af6df8 (Werner LEMBERG      2003-03-12 23:00:25 +0000 12) .do composite -  u0304
77d9af6df8 (Werner LEMBERG      2003-03-12 23:00:25 +0000 13) .do composite ab u0306
77d9af6df8 (Werner LEMBERG      2003-03-12 23:00:25 +0000 14) .do composite a. u0307
77d9af6df8 (Werner LEMBERG      2003-03-12 23:00:25 +0000 15) .do composite .  u0307
77d9af6df8 (Werner LEMBERG      2003-03-12 23:00:25 +0000 16) .do composite ad u0308
94c91fca8c (Werner LEMBERG      2006-02-28 13:04:27 +0000 17) .do composite :  u0308
77d9af6df8 (Werner LEMBERG      2003-03-12 23:00:25 +0000 18) .do composite ao u030A
77d9af6df8 (Werner LEMBERG      2003-03-12 23:00:25 +0000 19) .do composite a" u030B
77d9af6df8 (Werner LEMBERG      2003-03-12 23:00:25 +0000 20) .do composite "  u030B
77d9af6df8 (Werner LEMBERG      2003-03-12 23:00:25 +0000 21) .do composite ah u030C
77d9af6df8 (Werner LEMBERG      2003-03-12 23:00:25 +0000 22) .do composite ac u0327
be90ad7557 (Werner LEMBERG      2003-12-19 23:30:02 +0000 23) .do composite ,  u0327
77d9af6df8 (Werner LEMBERG      2003-03-12 23:00:25 +0000 24) .do composite ho u0328
87909b1715 (Werner LEMBERG      2003-03-01 07:34:52 +0000 25) .
47fc0a18b8 (G. Branden Robinson 2017-11-18 17:49:36 -0500 26) .\" Local Variables:
47fc0a18b8 (G. Branden Robinson 2017-11-18 17:49:36 -0500 27) .\" mode: nroff
47fc0a18b8 (G. Branden Robinson 2017-11-18 17:49:36 -0500 28) .\" fill-column: 72
47fc0a18b8 (G. Branden Robinson 2017-11-18 17:49:36 -0500 29) .\" End:
47fc0a18b8 (G. Branden Robinson 2017-11-18 17:49:36 -0500 30) .\" vim: set filetype=groff textwidth=72:


If a policeman's baton is being swung here, it's not mine.  (Though Keith Marshall might disagree as regards editor settings.)
 

> Just in case the character in the above example gets mangled by savannah, here's the equivalent:


It came through fine for me, fortunately.

> I hope this gives you enough information to fix this bug.


Yup, I'll revert the change.  At some point I would like to validate the segregated/spacey/whatever form of escape sequence differently.

Or, if it's too hard, not at all.  That also would be a bummer.

Some notes, probably mainly for my own benefit.  Not particularly PDF-relevant.

This problem has clarified my thinking around what the formatter needs to know to delegate grapheme cluster composition to the device,[1][2] and strengthens my suspicion that HTML output should take place in nroff mode.  This is because font families and type sizes are less HTML's job than CSS's, and we might as well communicate such things to the document via different means ("tags", but not exactly as the Mulley/Lemberg solution has applied them).  Moreover, stylesheet selection should probably be an option in the output driver, with a stock one generated or embedded in the absence of a user's choice.  (This isn't too different from how grops uses a PostScript prologue, for example.)

[1] We won't have problems if grapheme cluster composition can't turn a half-width character into a full-width one or vice versa (mostly an issue for terminals), and moreover the formatter doesn't need to know how wide a grapheme cluster is if it's not responsible for line breaking decisions.  And for HTML, it isn't.

[2] Typesetting devices will still have to get such information back to the formatter.  If a composite character has different metrics from the base character, the formatter *must* know this; hence the warnings already produced.

G. Branden Robinson <gbranden>
Group administrator
Thu 02 May 2024 12:13:41 AM UTC, comment #3: 

It is very simple!

[derij@pip build (master)]$ echo "This is the arabic alef with a madda above -> آ"|groff -Tutf8 -Kutf8 -z
troff:<standard input>:1: error: cannot format glyph: 'u0627_0653' is not a valid composite character

It is no longer possible to output this particular utf-8 character (along with hundreds of others), all perfectly valid utf-8 to the utf-8 device. I can't believe that is your intention, to police which particular parts of unicode are acceptable to you.

Just in case the character in the above example gets mangled by savannah, here's the equivalent:-

[derij@pip build (master)]$ echo "This is the arabic alef with a madda above -> \[u0622]"|groff -Tutf8 -z
troff:<standard input>:1: error: cannot format glyph: 'u0627_0653' is not a valid composite character

I hope this gives you enough information to fix this bug.

Deri James <deri>
Group Member
Wed 01 May 2024 11:22:47 PM UTC, comment #2: 

original submission:

> If preconv produces a valid composite character groff should not reject it. Of course if the composite is not available in any available font


Unfortunately that's not the way GNU troff works.  (Or I'm not understanding the bug report.)

The list of composite characters is global.

Here's what our Texinfo manual says in Git HEAD.


 -- Escape sequence: \[base-glyph combining-component ...]
...
     GNU 'troff' resolves '\[...]' with more than a single component as
     follows:

        * Any component that is found in the GGL [groff glyph list --GBR]
          is converted to the 'uXXXX' form.

        * Any component 'uXXXX' that is found in the list of
          decomposable glyphs is decomposed.

        * The resulting elements are then concatenated with '_' in
          between, dropping the leading 'u' in all elements but the
          first.

     No check for the existence of any component (similar to 'tr'
     request) is done.

     Examples:

     '\[A ho]'
          'A' maps to 'u0041', 'ho' maps to 'u02DB', thus the final
          glyph name would be 'u0041_02DB'.  This is not the expected
          result: the ogonek glyph 'ho' is a spacing ogonek, but for a
          proper composite a non-spacing ogonek (U+0328) is necessary.
          Looking into the file 'composite.tmac', one can find
          '.composite ho u0328', which changes the mapping of 'ho' while
          a composite glyph name is constructed, causing the final glyph
          name to be 'u0041_0328'.

     '\[^E u0301]'
     '\[^E aa]'
     '\[E a^ aa]'
     '\[E ^ ']'
          '^E' maps to 'u0045_0302', thus the final glyph name is
          'u0045_0302_0301' in all forms (assuming proper calls of the
          'composite' request).

     It is not possible to define glyphs with names like 'A ho' within a
     'groff' font file.  This is not really a limitation; instead, you
     have to define 'u0041_0328'.
...
 -- Request: .composite c1 c2
     Map ordinary or special character name C1 to C2 when C1 is a
     combining component in a composite character.  See above for
     examples.  This is a strict rewriting of the special character
     name; no check is performed for the existence of a glyph for
     either.  Typically, 'composite' is used to map a spacing character
     to a combining one.  A set of default mappings for many accents can
     be found in the file 'composite.tmac', loaded by the default
     'troffrc' at startup.

     You can obtain a report of mappings defined by 'composite' on the
     standard error stream with the 'pcomposite' request.  *Note
     Debugging::.



> Personally I see little value in this error,


I do find value in it; in the ChangeLog entry, I provided my rationale.  In the commit message I even provided exhibits of cases that should have produced a diagnostic but did not.


    [troff]: Diagnose bogus composite character escape sequences.  That is,
    when a composite character escape sequence like \[a ~] has a bogus
    modifier (as opposed to base) character, meaning one that has not been
    defined as the source _or_ destination of a `composite` request, warn
    about it.  For instance, \[a $] is nonsense, barring a request like
    `.composite $ \[uFF00]`, which would map `$`, when used as a modifier
    character in a composite special character escape sequence, to U+FF00,
    which would be a modifier form of the dollar sign in an alternate
    universe.
...
    Input:
    .nf
    \[A a~]
    \[A ~]
    \[u0041_0301]
    \[u0041_007E] \" should fail because 007E is explicitly spacing
    \[u0041_0041] \" same reason, more obviously
    \[u0041_0301_0301] \" should fail, would have a different meaning
    \[u0041_007E_0301] \" both problems above

    groff 1.23.0 and earlier:
    $ groff -T ps -z EXPERIMENTS/composite_character_construction.groff
    troff:...:5: warning: special character 'u0041_007E' not defined
    troff:...:6: warning: special character 'u0041_0041' not defined
    troff:...:7: warning: special character 'u0041_0301_0301' not defined
    troff:...:8: warning: special character 'u0041_007E_0301' not defined
    $ groff -Tutf8 -z EXPERIMENTS/composite_character_construction.groff
    [no output due to Savannah #65109]

    Now:
    $ ./build/test-groff -T ps -z EXPERIMENTS/composite_character_construction.groff
    troff:...:5: warning: special character 'u0041_007E' not defined
    troff:...:6: error: cannot format glyph: 'u0041_0041' is not a valid composite character
    troff:...:7: warning: special character 'u0041_0301_0301' not defined
    troff:...:8: warning: special character 'u0041_007E_0301' not defined
    $ ./build/test-groff -T utf8 -z EXPERIMENTS/composite_character_construction.groff
    troff:...:6: error: cannot format glyph: 'u0041_0041' is not a valid composite character


> the existing error reporting of a special character not defined is more helpful since if you find a font which contains the correct glyph, the error will be gone.


Is this true in full generality?  Does it also apply to output devices that don't even have a "charset" section in their fonts because they're "unicode" [sic] devices?

groff_font(5):


     unicode
             The output device supports the complete Unicode repertoire.
             This directive is useful only for devices which produce
             character entities instead of glyphs.

             If unicode is present, no charset section is required in
             the font description files since the Unicode handling built
             into groff is used.  However, if there are entries in a
             font description file’s charset section, they either
             override the default mappings for those particular
             characters or add new mappings (normally for composite
             characters).

             The utf8, html, and xhtml output devices use this
             directive.


(I feel that that's a badly named directive.  As I understand it, it, it more precisely means that a different glyph resolution mechanism is used--or none at all, instead assuming that the device is happy to attempt to combine any sequence of Unicode code points as a grapheme cluster.)

> I'm sure there are users capable of creating a font with all sorts of weird composite glyphs, why should we police what they can do?


Because we have no mechanism for defining font-specific composite character components.  (Meaning: "foo" in `\[a foo]`; contrast with the composed composite characters contemplated by the second paragraph of the "unicode" directive description quoted above.)  Maybe we should, but that in turn would mean having font-specific macro files that users' documents would need to load.

And we'd probably need a tool to generate them.

Might be better/more scalable to ask authors of such documents issue the `composite` requests itself.  We can add commonly used ones that we are presently missing to "composite.tmac".

My anticipation of this problem is why I added a (rather, stopped discouraging use of an existing) mechanism to delete composite character mappings and a new request for reporting the ones the formatter knows about.

Or people can bypass this escape sequence syntax entirely and spell their grapheme clusters in Unicode directly as is already supported.

Our Texinfo manual again:


   * A glyph representing more than a single input character is named

          'u' COMPONENT1 '_' COMPONENT2 '_' COMPONENT3 ...

     Example: 'u0045_0302_0301'.


There may be an opportunity for some terminological revision here.  This section of the manual is one of those I haven't finished my first revision pass on yet.  I still have things to learn.  Maybe you can shed some light where things are dark for me.


commit 2c76a931b81b1e22dd419c7027d3517325c23193
Author: G. Branden Robinson <g.branden.robinson@gmail.com>
Date:   Wed Jan 17 14:02:28 2024 -0600

    [troff]: Fix Savannah #64937 (del composite char).

    * src/roff/troff/input.cpp (map_composite_character): Stop throwing
      diagnostic message when `composite` request invoked with only one
      argument.  This has long worked just fine to delete a composite
      character mapping.  That is something a (rare) user might conceivably
      want to do.

    Fixes <https://savannah.gnu.org/bugs/?64937>.

commit e958bb4fc65326dd9cd0d775e96aff15e944795e
Author: G. Branden Robinson <g.branden.robinson@gmail.com>
Date:   Wed Jan 17 13:49:40 2024 -0600

    [troff]: Implement new `pcomposite` request.

    * src/roff/troff/input.cpp (report_composite_characters): Add.
      (init_input_requests): Wire up `pcomposite` request name to
      `report_composite_characters()`.

    * doc/groff.texi (Colors, Debugging):
    * man/groff.7.man (Request short reference, Debugging):
    * man/groff_diff.7.man (New requests, Debugging):
    * NEWS: Document it.


G. Branden Robinson <gbranden>
Group administrator
Mon 22 Apr 2024 03:14:18 PM UTC, comment #1: 

Assigning to Branden as the expert on this, since it seems to be his new code causing the issue.  See bug #65109

Deri James <deri>
Group Member
Tue 16 Apr 2024 11:21:28 AM UTC, original submission:  

If preconv produces a valid composite character groff should not reject it. Of course if the composite is not available in any available font it will elicit the normal "not defined" error, but, if it is, this bug prevents it being used. I have many examples including these using groff -man on some man pages:-

troff:charsets.7:76: error: cannot format glyph: 'u0073_0326' is not a valid composite character
troff:charsets.7:76: error: cannot format glyph: 'u0074_0326' is not a valid composite character
troff:iso_8859-6.7:54: error: cannot format glyph: 'u0627_0653' is not a valid composite character
troff:iso_8859-6.7:55: error: cannot format glyph: 'u0627_0654' is not a valid composite character
troff:iso_8859-6.7:56: error: cannot format glyph: 'u0648_0654' is not a valid composite character
troff:iso_8859-6.7:57: error: cannot format glyph: 'u0627_0655' is not a valid composite character
troff:iso_8859-6.7:58: error: cannot format glyph: 'u064A_0654' is not a valid composite character
troff:iso_8859-6.7:54: error: cannot format glyph: 'u0627_0653' is not a valid composite character
troff:iso_8859-6.7:55: error: cannot format glyph: 'u0627_0654' is not a valid composite character
troff:iso_8859-6.7:56: error: cannot format glyph: 'u0648_0654' is not a valid composite character
troff:iso_8859-6.7:57: error: cannot format glyph: 'u0627_0655' is not a valid composite character
troff:iso_8859-6.7:58: error: cannot format glyph: 'u064A_0654' is not a valid composite character
troff:iso_8859-16.7:56: error: cannot format glyph: 'u0053_0326' is not a valid composite character
troff:iso_8859-16.7:72: error: cannot format glyph: 'u0073_0326' is not a valid composite character
troff:iso_8859-16.7:108: error: cannot format glyph: 'u0054_0326' is not a valid composite character
troff:iso_8859-16.7:140: error: cannot format glyph: 'u0074_0326' is not a valid composite character
troff:iso_8859-16.7:56: error: cannot format glyph: 'u0053_0326' is not a valid composite character
troff:iso_8859-16.7:72: error: cannot format glyph: 'u0073_0326' is not a valid composite character
troff:iso_8859-16.7:108: error: cannot format glyph: 'u0054_0326' is not a valid composite character
troff:iso_8859-16.7:140: error: cannot format glyph: 'u0074_0326' is not a valid composite character

Even nroff -T utf8 can't reproduce the man page properly!

xzcat /usr/share/man/man7/iso_8859-6.7.xz |preconv|nroff -Tutf8 -man -t -z

Personally I see little value in this error, the existing error reporting of a special character not defined is more helpful since if you find a font which contains the correct glyph, the error will be gone. I'm sure there are users capable of creating a font with all sorts of weird composite glyphs, why should we police what they can do?

Deri James <deri>
Group Member

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by gbranden (Posted a comment)
  • -email is unavailable- added by deri (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only logged-in users can vote.

     

    Follow 7 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2024-05-02 gbranden StatusConfirmed Fixed
        Open/ClosedOpen Closed
        Planned ReleaseNone 1.24.0
    2024-05-02 deri StatusNeed Info Confirmed
    2024-05-01 gbranden StatusNone Need Info
        SummaryBogus 'bogus composite' errors introduced by commit 6008b6b7aa [troff] bogus 'bogus composite' errors introduced by commit 6008b6b7aa
    2024-04-22 deri Assigned toNone gbranden

    Back to the top

    Powered by Savane 3.13-bb6a.
    Corresponding source code