bugGNU roff - Bugs: bug #57538, .ne is off by one in some macro...

 
 

bug #57538: .ne is off by one in some macro packages even when using no other facilities of that package

Submitter:  Dave <barx>
Submitted:  Sat 04 Jan 2020 08:12:45 PM UTC
   
 
Category:  Macro - others/general Severity:  3 - Normal
Item Group:  Incorrect behaviour Status:  Need Info
Privacy:  Public Assigned to:  gbranden
Open/Closed:  Open Planned Release:  None
* Mandatory Fields

Add a New Comment Rich Markup
   

Jump to the original submission

Fri 19 May 2023 02:53:30 AM UTC, comment #22: 

Yes, this bug does suffer from comment overload.  Comment #13 is the basic problem statement as I understand it now.  Previous discussion was about specific cases of what comment #13 illustrates, and yes, some of those examples are overblown and not particularly illuminating.

Dave <barx>
Group Member
Thu 18 May 2023 11:44:03 PM UTC, comment #21: 

Comment to comment #20:

  Your examples show the right number of lines on the pages,
so there is no bug in groff's software about this.

  Your counting procedure is either unknown or I have missed that in the
flood of comments,
so you should start describing it well enough, that others can reproduce
it or find it defective.

N.B.  The (Some) examples are too complicated, long, and data is
missing to understand the problem.

Bjarni Ingi Gislason <bjarniig>
Thu 18 May 2023 07:27:21 PM UTC, comment #20: 

comment #18:

> I ported your reproducer to ms,


Your port doesn't actually reproduce the situation the other examples are demonstrating, at least not with a US-letter paper setting.  I get this output:

$ groff -a need1.ms 2> /dev/null
<beginning of page>
1
2
3
4
5
6

<beginning of page>
 -2-

But having all lines on the first page doesn't set up the situation where we can test the bug; for that, there needs to be a page break before line 6.

I was able to bring this about by changing the 9i to 9.3i.  And once that's done, the two files do in fact show the bug present in -ms:

$ sed s/9i/9.3i/ need1.ms | groff -a 2> /dev/null
<beginning of page>
1
2
3
4
5

<beginning of page>
 -2-
6

$ sed s/9i/9.3i/ need2.ms | groff -a 2> /dev/null
<beginning of page>
1
2

<beginning of page>
 -2-
3
4
5
6

This output is the same for me with 1.22.4 and 1.23.0.rc4.

Dave <barx>
Group Member
Thu 18 May 2023 10:56:45 AM UTC, comment #19: 

Sorry, the environment differences were a TOTAL red herring.

I manually sorted the environments in both output files and the only difference that remains is this.


$ diff -u need[12]s.out
--- need1s.out  2023-05-18 05:53:01.718588778 -0500
+++ need2s.out  2023-05-18 05:53:55.990392111 -0500
@@ -255,7 +255,7 @@
 pg@top 0
 pg@bottom      -648000
 pg*footer      -324000
-.t=60000
+.t=48000
 .ne=0, .trunc=3600, .trap=pg@bottom
 <beginning of page>
 1


Which is exactly what we'd expect from issuing the `ne` request one vee later.  (At a resolution of 72000u/inch, 1v = 1/6inch = 12000u.)

Maybe ms is managing its traps better than me and mm do.

G. Branden Robinson <gbranden>
Group administrator
Thu 18 May 2023 10:51:37 AM UTC, comment #18: 

I ported your reproducer to ms, and it doesn't reproduce.


$ cat EXPERIMENTS/need1.ms
.mso s.tmac
.nr FM 9i
.nf
1
.pev
.ptr
.tm .t=\n[.t]
.ne 3
.tm .ne=\n[.ne], .trunc=\n[.trunc], .trap=\n[.trap]
2
3
4
5
6
$ cat EXPERIMENTS/need2.ms
.mso s.tmac
.nr FM 9i
.nf
1
2
.pev
.ptr
.tm .t=\n[.t]
.ne 3
.tm .ne=\n[.ne], .trunc=\n[.trunc], .trap=\n[.trap]
3
4
5
6
$ ./build/test-groff -a EXPERIMENTS/need1.ms >| need1s.out 2>&1
$ ./build/test-groff -a EXPERIMENTS/need2.ms >| need2s.out 2>&1
$ $ diff -u need[12]s.out
--- need1s.out  2023-05-18 05:43:03.592759442 -0500
+++ need2s.out  2023-05-18 05:43:11.324731344 -0500
@@ -40,7 +40,7 @@
   maximum number of consecutive hyphenated lines: -1
   hyphenation space: 0u
   hyphenation margin: 0u
-Environment h:
+Environment ne:
   previous line length: 468000u
   line length: 468000u
   previous title length: 468000u
@@ -55,7 +55,7 @@
   family: 'T'
   space size: 12/36 em
   sentence space size: 12/36 em
-  previous line interrupted: no
+  previous line interrupted: yes
   fill mode: on
   adjust mode: both
   previous vertical spacing: 12000u
@@ -69,7 +69,7 @@
   temporary indentation: 0u
   have temporary indentation: no
   currently used indentation: 0u
-  target text length: 0u
+  target text length: 468000u
   previous text length: 0u
   total width: 0u
   total number of spaces: 0
@@ -77,12 +77,12 @@
   line tabs: no
   discarding: no
   spread flag set: no
-  hyphenation_flags: on, not last line, not last two chars
+  hyphenation_flags: on
   number of consecutive hyphenated lines: 0
   maximum number of consecutive hyphenated lines: -1
   hyphenation space: 0u
   hyphenation margin: 0u
-Environment fn:
+Environment nf:
   previous line length: 468000u
   line length: 468000u
   previous title length: 468000u
@@ -98,10 +98,10 @@
   space size: 12/36 em
   sentence space size: 12/36 em
   previous line interrupted: no
-  fill mode: on
+  fill mode: off
   adjust mode: both
   previous vertical spacing: 12000u
-  vertical spacing: 8000u
+  vertical spacing: 12000u
   previous post-vertical spacing: 0u
   post-vertical spacing: 0u
   previous line spacing: 1
@@ -124,7 +124,9 @@
   maximum number of consecutive hyphenated lines: -1
   hyphenation space: 0u
   hyphenation margin: 0u
-Environment ne:
+Environment 0:
+  current
+Environment k:
   previous line length: 468000u
   line length: 468000u
   previous title length: 468000u
@@ -139,7 +141,7 @@
   family: 'T'
   space size: 12/36 em
   sentence space size: 12/36 em
-  previous line interrupted: yes
+  previous line interrupted: no
   fill mode: on
   adjust mode: both
   previous vertical spacing: 12000u
@@ -153,7 +155,7 @@
   temporary indentation: 0u
   have temporary indentation: no
   currently used indentation: 0u
-  target text length: 468000u
+  target text length: 0u
   previous text length: 0u
   total width: 0u
   total number of spaces: 0
@@ -166,7 +168,7 @@
   maximum number of consecutive hyphenated lines: -1
   hyphenation space: 0u
   hyphenation margin: 0u
-Environment nf:
+Environment h:
   previous line length: 468000u
   line length: 468000u
   previous title length: 468000u
@@ -182,7 +184,7 @@
   space size: 12/36 em
   sentence space size: 12/36 em
   previous line interrupted: no
-  fill mode: off
+  fill mode: on
   adjust mode: both
   previous vertical spacing: 12000u
   vertical spacing: 12000u
@@ -203,14 +205,12 @@
   line tabs: no
   discarding: no
   spread flag set: no
-  hyphenation_flags: on
+  hyphenation_flags: on, not last line, not last two chars
   number of consecutive hyphenated lines: 0
   maximum number of consecutive hyphenated lines: -1
   hyphenation space: 0u
   hyphenation margin: 0u
-Environment 0:
-  current
-Environment k:
+Environment fn:
   previous line length: 468000u
   line length: 468000u
   previous title length: 468000u
@@ -229,7 +229,7 @@
   fill mode: on
   adjust mode: both
   previous vertical spacing: 12000u
-  vertical spacing: 12000u
+  vertical spacing: 8000u
   previous post-vertical spacing: 0u
   post-vertical spacing: 0u
   previous line spacing: 1
@@ -255,7 +255,7 @@
 pg@top 0
 pg@bottom      -648000
 pg*footer      -324000
-.t=60000
+.t=48000
 .ne=0, .trunc=0, .trap=pg@bottom
 <beginning of page>
 1


I don't have any idea which of those environmental differences are red herrings.  The environments are getting spat out in a different order, which may be introducing significant noise.

G. Branden Robinson <gbranden>
Group administrator
Thu 18 May 2023 07:53:31 AM UTC, comment #17: 

As comment #13 indicates, this bug is not confined to -me's .(b blocks, or to the -me macros at all.  Updating fields accordingly.

Preceding the .ne with a .br (a practice Branden recommends in bug #64207) in each of the comment #13 examples has no effect.

Dave <barx>
Group Member
Sat 08 Jan 2022 01:55:29 AM UTC, comment #16: 

comment #15:

> All of the me(7) keep macros emit a break request before closing
> the diversion.  That means there's always an extra space down
> to the next baseline at the ends of the diversions.


I'm sure that's true as far as it goes, but the comment #13 examples were intended to show that the off-by-one behavior happens even without using any -me diversions, keeps, or other macro-defined structures -- even, for that matter, in the absence of using any -me features at all, save for the changing of the bottom margin.  (And that's only for example-shortening sake.  The same behavior is observable without that; you just need a lot more lines to get there.)  The mere loading of the macro package makes .ne behave slightly off.

As you note, this does not happen in plain roff, but as comment #13 notes, it does also happen with at least -mm, even without using any -mm features.  (I don't know any, save the setting of the bottom margin, which I looked up specifically for this example.)  I didn't try any other packages; two seemed sufficient to demonstrate that this is a problem beyond a single macro package.

Dave <barx>
Group Member
Fri 07 Jan 2022 11:18:47 PM UTC, comment #15: 

comment #14:

> The upshot (which I fear may have gotten lost in the minutiae)
> is that the proposed fix to .)b in comment #8 amounts to "lie
> to .ne about how many lines are needed."  But there are
> numerous other cases, as comment #13 demonstrates, where lying
> to .ne by telling it one fewer line is also necessary to
> produce the correct result.


Well, yes and no.  I think the problem isn't that we need to lie
to .ne about how many vees of space are needed, but that we
(me(7)) has been lying to the `ne` request for a long time.

All of the me(7) keep macros emit a break request before closing
the diversion.  That means there's always an extra space down
to the next baseline at the ends of the diversions.

It's really reliable about this.  Let me show you.


1133 .de @)                  \" --- end keep
1134 .br
1135 .if !"\\n(.z"|k" .do @err attempted close of never-opened keep
1136 .nr ?k 0
1137 .di
1138 .in 0
1139 .ce 0
1140 ..

1154 .de )c                  \" *** end block centered text
1155 .if !"\\n(.z"|c" .do @err unmatched block centering call
1156 .br                             \" force out final line
1157 .di
1158 .if \n@>4 .tm >> .)c .l=\\n(.l .i=\\n(.i $i=\\n($i dl=\\n(dl
1159 .ev 1
1160 .ls 1
1161 .nr __ (\\n(.lu-\\n(.iu-\\n(dlu)/2u
1162 .if \\n(__u<0 .nr __ 0
1163 .in \\n(__u
1164 .rr __
1165 .if \n@>4 .tm -- .)c << .in .l=\\n(.l .i=\\n(.i dl=\\n(dl
1166 .if \\n(?n \
1167 .       nm \\n(ln
1168 .nf
1169 .|c
1170 .in
1171 .ls
1172 .if \\n(?n \
1173 .       nm
1174 .ev
1175 .rm |c
1176 .if \\n(?n \
1177 .       nm \\n(ln
1178 ..

1253 .de EN                  \" *** equation end
1254 .br
1255 .do HTML-IMAGE-END
1256 .if "\*(.T"html" \
1257 .       do ll \\n[e-EQ-ll]u
1258 .ie "\\$1"C" \
1259 \{\
1260 .       nr ?e 1
1261 .       sp \\n(esu
1262 .\}
1263 .el \
1264 \{\
1265 .       nr ?e 0
1266 .       di
1267 .       if \\n(dn \
1268 .               @q\"            \" actual equation output
1269 .       rm |e
1270 .       ev
1271 .\}
1272 ..


(In fact, we can see that, if I'm right, the `if` at line 1267
is unnecessary; its condition is always true.)

Since this is groff, we could just `.chop` that line break away.
Alternatively, and for portability to less featureful troffs,
we could do a subtraction after closing the diversion,
decrementing 'dn'.  I'm leaning toward the latter.

> So the first fix should be figuring out how to tell .ne the
> truth in general.  Only then, if the -1v is still needed in
> .)b due to diversion semantics, should that change be
> considered.  (But I bet it won't.)


I think you're right, given the foregoing.

I'll give a subtracting 1v from 'dn' after closing these
diversions a try.

G. Branden Robinson <gbranden>
Group administrator
Thu 23 Dec 2021 02:09:50 AM UTC, comment #14: 

The upshot (which I fear may have gotten lost in the minutiae) is that the proposed fix to .)b in comment #8 amounts to "lie to .ne about how many lines are needed."  But there are numerous other cases, as comment #13 demonstrates, where lying to .ne by telling it one fewer line is also necessary to produce the correct result.

So the first fix should be figuring out how to tell .ne the truth in general.  Only then, if the -1v is still needed in .)b due to diversion semantics, should that change be considered.  (But I bet it won't.)

Dave <barx>
Group Member
Thu 23 Dec 2021 01:41:00 AM UTC, comment #13: 

Here are much smaller examples showing the problem in both -me and -mm.

.mso e.tmac
.nr bm 9i
.nf
1
.ne 3
2
3
4
5
6


.mso m.tmac
.VM 0 8.25i
.nf
1
.ne 3
2
3
4
5
6

These both put lines 1 to 5 on the first page and line 6 on the second.  "groff -a" is an easy way to see where the page break occurs without having to generate and examine PostScript.

But move the .ne down one line in either file -- which should tell it to place lines 3, 4, and 5 on page 1 if they fit (and we know they do; we just saw it) -- and only lines 1 and 2 appear on the first page; 3, 4, and 5 are bumped to the second.

The same behavior occurring in two macro packages tells us it's not specific to -me or -mm.  The behavior not occurring in the absence of a macro package (at least that either of us has been able to demonstrate) tells us the basic use of .ne works as expected.  But something odd is going on when they try to work together, and the oddity is not confined to just one macro package.

Dave <barx>
Group Member
Thu 23 Dec 2021 01:04:11 AM UTC, comment #12: 

I'll play with it some more, but I'm not making the general claim that .ne is broken in pure roff, only that it exhibits off-by-one behavior under -me (and under -mm, if Ulrich is to be believed, though I haven't tested this).  That is, it's perhaps not purely a -me bug or an .ne one, but something subtle in the interactions of these things.

Dave <barx>
Group Member
Thu 23 Dec 2021 12:37:02 AM UTC, comment #11: 

comment #9:

> Thanks for tracking down this mewling kitten, Branden!
>
> The original longstanding bug (as the opening comment here observes, it was first reported to bug-groff in 2012)


...and the line of code I changed was present at the Big Bang, authored by the demiurge James Clark himself, apparently.  I can't lay this one at George Helffrich's doorstep.  :)

> made me long ago excise .(b/.)b from my own -me documents.


That should not be necessary--you should be able to pick it back up now.
 

> But why should .ne's parameter need to be adjusted so?  The diversion-adds-an-extra-line-feed explanation only takes you so far, since using .ne in place of the .(b/.)b pair in the original example -- which, absent those two -me macros, does not use a diversion -- still results in a premature page break when the seven-line block would fit on the last seven lines of the page.


Hmm, I guess I did not share my experiments with a raw roff document.  I thought I had.

As far as my mental model of the formatter goes, the .ne request works as I expect.

Try this rofflet.


$ cat EXPERIMENTS/count-lines.roff
.de XX
.  br
.  nop \\$1\c
.  nr N \\$1-1
.  if \\nN .XX \\nN
..
.nf
.ne \nNv
.XX \nN


(U.S. letter paper is assumed here.)

Call it like this (groff 1.22.4 is fine, and, I suspect, any groff before that).


$ nroff -rN66 EXPERIMENTS/count-lines.roff


You will notice that no extraneous blank lines are output.  We're exactly filling the 66-line virtual page height.

Change 66 to 67 and you'll see something else happen.

This example can of course be made simpler, deparameterized, more straightforward, and longer, with the help of the `seq` command (or laborious typing).  Here is one equivalent.


.nf
1
2
3
.\" yadda
64
65
66\c


The above also does not produce extraneous lines, but take off that last `\c` and it will.

That is because the newline at the end of the file is read, interpreted as a word break, and causes a downward vertical motion (guaranteed with filling disabled, conditional otherwise).  But we're already at the bottom-of-page double-secret-probation page location trap, so the page is ejected and a new one begun.

As I understand it, .ne takes us only as far as we need to go--literally!

I will admit that diversions make this issue more difficult to deal with.  When adding them to the mix, things get more complicated because of the "partially collected line" business.  When diverting, I couldn't use `\c` anymore and I had to use groff's `chop` request to get the final newline/break out of the diversion.  BSD me(7), given its original date of authorship, could not avail itself of that facility.


$ cat EXPERIMENTS/divert-lines-param.roff
.de XX
.  nop \\$1
.  nr N \\$1-1
.  if \\nN .XX \\nN
..
.di YY
.nf
.XX \nN
.di
.chop YY
.YY


(You can add a `ne .\nNv` request before outputting the diversion `YY`--it makes no difference because the diversion fits a page--_if_ its last break is removed.)

> That makes this fix feel more like a workaround to the underlying problem, which is that .ne "grabs" more space than necessary.  And the thread linked in comment #1 suggests this problem isn't limited to the -me package.


I'll look at that thread but I think my examples above establish that it doesn't, and it works the same in troff and nroff modes.
 

> That is, while I originally reported this problem in -me's .)b macro, I'm beginning to think the underlying problem is with .ne itself.


Please play with the foregoing and see if you still feel that way.  If you don't, please supply some example pure roff input that seems to misbehave, and we can start a new ticket against the core.  :)

G. Branden Robinson <gbranden>
Group administrator
Wed 22 Dec 2021 11:22:00 PM UTC, comment #10: 

comment #9:

> And the thread linked in comment #1


For the sake of completeness: the archive bot mangled Ulrich's example in this thread because it referred to internal mm macro pg@footer, which the bot mistook for an email address.  The code snippet as posted read:

.ch pg@footer
.BL
[any stuff here]
.LE
.wh \n[pg*foot-trap]u pg@footer
.BP

(If you're reading this note in the bug-groff archive, it'll have been remangled, so follow the link to the savannah bug report to see it unredacted.)

Dave <barx>
Group Member
Wed 22 Dec 2021 10:23:32 PM UTC, comment #9: 

Thanks for tracking down this mewling kitten, Branden!

The original longstanding bug (as the opening comment here observes, it was first reported to bug-groff in 2012) made me long ago excise .(b/.)b from my own -me documents.  Thus my only test cases now are the ones posted here, which you've already verified.

But why should .ne's parameter need to be adjusted so?  The diversion-adds-an-extra-line-feed explanation only takes you so far, since using .ne in place of the .(b/.)b pair in the original example -- which, absent those two -me macros, does not use a diversion -- still results in a premature page break when the seven-line block would fit on the last seven lines of the page.

That makes this fix feel more like a workaround to the underlying problem, which is that .ne "grabs" more space than necessary.  And the thread linked in comment #1 suggests this problem isn't limited to the -me package.

That is, while I originally reported this problem in -me's .)b macro, I'm beginning to think the underlying problem is with .ne itself.

Dave <barx>
Group Member
Wed 22 Dec 2021 01:42:04 AM UTC, comment #8: 

Aha.

All that excavation (and venting my spleen over George Helffrich) seems to have enabled me to unearth a mewling kitten of insight buried amid the macrology rubble.

This line has been bothering me.


.      ne \\n(dnu              \" make it all on one page


I thought it was because my fastidious mind objected to the casual, imprecise language in the comment.  But maybe it was something else.

Try this patch.


diff --git a/tmac/e.tmac b/tmac/e.tmac
index cbe4ae6a7..a20fa2d09 100644
--- a/tmac/e.tmac
+++ b/tmac/e.tmac
@@ -1072,7 +1072,7 @@
 .de )b                 \" *** end block
 .@)
 .if (\\n(bt=0):(\\n(.t<\\n(bt) \
-.      ne \\n(dnu              \" make it all on one page
+.      ne \\n(dnu-1v           \" reserve room for diversion
 .@k |k ns
 .ev                             \" return from display environment
 .nr ?V 0


The idea here is that we don't need to reserve room for the trailing line break--more precisely, for a vertical motion down to the next baseline--in the diversion, because we don't need to avoid a page break there.  It's perfectly fine with us if the page breaks right after the block.  Including that trailing break in our calculation reserves one more line than we need.

The obvious question is, does this f*** up single-line blocks (which normally have no reason to exist), exposing our math as fallacious?

No, it does not.  See attached keep3.me.  And, for the more practical 2-line case, keep4.me.

I think this constitutes an RCA and a fix.  Please let me know if you concur, or don't.

(file #52549, file #52550)

G. Branden Robinson <gbranden>
Group administrator
Wed 22 Dec 2021 01:21:06 AM UTC, comment #7: 

comment #2:

> Because as it turns out, while the bug was in groff 1.21, and is in current groff, it has not been in every version of groff released in between.  I'm not sure which all official releases it appears in -- it is absent in at least 1.22.2 and 1.22.3 -- but I've iterated through the various versions of tmac/e.tmac / tmac/e.tmac-u (the file was renamed at some point) to find out exactly where it came and went.
>
> The bug disappeared in commit b6906269 of this file and reappeared in commit 98b028d6.  But the interesting thing about this is that these commits also respectively introduced and then removed another bug.
>
> As documented, all "displays" in -me (of which ".(b" is one) should have space corresponding to the \n(bs register above and below; the test case at hand explicitly sets this register to 0, specifying no extra space above or below the block.  But starting with commit b6906269, this test case began emitting an extra line above the .(b block; commit 98b028d6 fixed this.
>
> So somehow this extra-space bug removed, or at least masked, the page-break bug as a side effect.


To make this history hyperlinked and concrete, here are Savannah Git repo links for commit b6906269[1] and commit 98b028d6.

Here are the changes to e.tmac in each.


diff --git a/tmac/e.tmac b/tmac/e.tmac
index a5bfaabb5..af0a28e89 100644
--- a/tmac/e.tmac
+++ b/tmac/e.tmac
@@ -253,7 +253,7 @@
 .      sp |\\n(tmu             \" move to top of text
 .ev
 .mk _k                         \" for columned output
-.if \\n(?n .nm 1               \" restore line numbering if n1 mode
+.if \\n(?n=1 .nm 1             \" restore line numbering if n1 mode
 .nr $c 1                       \" set first column
 .if \n@>4 .tm -- @h >> .ns nl=\\n(nl %=\\n% _k=\\n(_k tm=\\n(tm
 .ie \\n(?s \
@@ -297,9 +297,13 @@
 .if \\n(?T \
 \{\
 .      nr _i \\n(.i
+.      if \\n(?n \
+.              nm \\n(ln
 .      in 0
 .      |h\"                    \" output the table header
 .      in \\n(_iu
+.      if \\n(?n \
+.              nm
 .      rr _i
 .      mk #T                   \" for tbl commands
 .      ns
@@ -351,7 +355,11 @@
 .in 0
 .if \n@>2 .tm @o last printed text = \\n(nl placing @r trap at -\\n(_B
 .wh -\\n(_Bu @r
+.if \\n(?n \
+.      nm \\n(ln
 .|f
+.if \\n(?n \
+.      nm
 .fi
 .if \n@>2 .tm @o triggered @r (?o) = \\n(?o
 .if \\n(?o \
@@ -394,9 +402,13 @@
 .      sz \\n(tp\"             \" set header/footer type fonts, etc.
 .      @F \\n(tf
 .      lt \\n(_Lu              \" make title span entire page
+.      if \\n(?n \
+.              nm \\n(ln
 .      nf
 .      \\$1
 .      br
+.      if \\n(?n \
+.              nm
 .\}
 ..
 .
@@ -506,16 +518,46 @@
 .
 .de n1                 \" *** line numbering 1
 .nm 1
-.xl -\w'0000'u
+.ll -\w'0000'u
 .nr ?n 1
 ..
 .
 .de n2                 \" *** line numbering 2
-.nm \\$1
 .ie \\n(.$ \
-.      xl -\w'0000'u
+\{\
+.      ie !`\\$2`c` \
+.              po -\w'0000'u
+.      el \
+.              ll -\w'0000'u
+.      ie \\B`\\$1` \
+\{\
+.              ds |i \\$1
+.              ds |j \\$1
+.              do substring |i 0 0  \" pick off leading char and rest to check
+.              do substring |j 1    \" for +N / -N
+.              ie `\\*(|i`+` \
+.                      nr ln (\\n[_ln])+\\*(|j
+.              el \
+.                      ie `\\*(|i`-` \
+.                              nr ln (\\n[_ln])-\\*(|j
+.                      el \
+.                              nr ln \\$1  \" unsigned N
+.              nm \\n(ln
+.              nr ?n 2
+.              rm |i |j
+.\}
+.      el \
+.              tm Line \\n(.c -- Bad .n2 value
+.\}
 .el \
-.      xl \\n($lu
+\{\
+.      nm
+.      nr _ln \\n(ln
+.      po +\w'0000'u
+.      ll \\n($lu
+.      nr ?n 0
+.      nr ln 0
+.\}
 ..
 .
 .de pa                 \" *** new page
@@ -914,7 +956,11 @@
 .ls 1
 .in 0
 .sp \\n(zsu
+.if \\n(?n \
+.      nm \\n(ln
 .\\$1
+.if \\n(?n \
+.      nm
 .ec
 .br
 .rm \\$1
@@ -942,15 +988,8 @@
 .@)
 .if (\\n(bt=0):(\\n(.t<\\n(bt) \
 .      ne \\n(dnu              \" make it all on one page
-.ls 1
-.nf
-.|k
-.ec
-.fi
-.in 0
-.xl \\n($lu
-.ev
-.rm |k
+.@k |k
+.ev                             \" return from display environment
 .sp \\n(bsu+\\n(.Lv-1v
 ..
 .
@@ -992,6 +1031,7 @@
 .
 .de (c                 \" *** begin block centered text
 .if "\\n(.z"|c" .tm Line \\n(c. -- Nested .(c requests
+.if \\n(?n .nm
 .di |c
 ..
 .
@@ -1004,11 +1044,15 @@
 .ls 1
 .in (\\n(.lu-\\n(.iu-\\n(dlu)/2u
 .if \n@>4 .tm -- .)c << .in .l=\\n(.l .i=\\n(.i dl=\\n(dl
+.if \\n(?n \
+.      nm \\n(ln
 .nf
 .|c
 .ec
 .in
 .ls
+.if \\n(?n \
+.      nm
 .ev
 .rm |c
 ..
@@ -1025,10 +1069,14 @@
 .in +\\n(qiu
 .xl -\\n(qiu
 .sz \\n(qp
+.if \\n(?n \
+.      nm \\n(ln
 ..
 .
 .de )q                 \" *** end block quote
 .br
+.if \\n(?n \
+.      nm
 .ev
 .sp \\n(qsu+\\n(.Lv-1v
 .nr ?k 0
@@ -1039,11 +1087,15 @@
 .sp \\n(bsu
 .@D 3 \\$1 \\$2
 .@M
+.if \\n(?n \
+.      nm \\n(ln
 .if '\*(.T'html' .sp \\n(bsu
 ..
 .
 .de )l                 \" *** end list
 .br
+.if \\n(?n \
+.      nm
 .ev
 .sp \\n(bsu+\\n(.Lv-1v
 .nr ?k 0
@@ -1123,7 +1175,11 @@
 .      tl """\\*(|p"
 .      rt \\n(_qu
 .\}
+.if \\n(?n \
+.      nm \\n(ln
 .|e
+.if \\n(?n \
+.      nm
 .sp |\\n(_qu+\\n(_Qu
 .sp \\n(esu+\\n(.Lv-1v
 .rr _q
@@ -1157,12 +1213,14 @@
 .de TH                 \" *** end header part of table
 .nr T. 0
 .T# 0
+.if \\n(?n \!.nm
 .di
 .nr _T \\n(?T
 .nr ?T 0
 .ne \\n(dnu+1v
 .nr ?T \\n(_T
 .nr _i \\n(.i
+.if \\n(?n .nm \\n(ln
 .in 0
 .|h\"                          \" put in the initial header
 .in \\n(_iu
@@ -1220,9 +1278,41 @@
 .ne \\$1u
 .nr g7 \\n(.u
 .ls 1
+.if \\n(?n \
+\{\
+.      de PS_nm_check &&        \" define macro to emit .nm at top diversion
+.      if \n@>4 .tm -- PS: \\\\$1 \\\\$2 .z=\\\\n(.z PS_nm_cnt=\\\\n[PS_nm_cnt]
+.      \"                       \" multiple .PS/.PE in a keep.
+.      if `\\\\$2`init` \
+.              do nr PS_nm_cnt \\\\n[PS_nm_cnt]+1
+.      ie `\\\\n(.z`` \
+\{\
+.              if `\\\\$1`suspend` \
+.                      nm
+.              if `\\\\$1`resume` \
+\{\
+.                      nm \\\\n(ln
+.                      do nr PS_nm_cnt \\\\n[PS_nm_cnt]-1
+.                      do if !\\\\n[PS_nm_cnt] \
+\{\
+.                              do rm PS_nm_check
+.                              do rr PS_nm_cnt
+.\}
+.\}
+.\}
+.el \!.PS_nm_check \\\\$1
+.&&
+.      mk _q                    \" emit a single numbered line for PS picture
+.      rs
+\&\
+.      br
+.      rt \\n(_qu
+.      PS_nm_check suspend init
+.\}
 ..
 .
 .de PE                 \" *** end picture
+.if \\n(?n .PS_nm_check resume
 .ls
 .in
 .if \\n(g7 .fi
@@ -1380,6 +1470,7 @@
 .      nr _D \\n(dn
 .      nr _0 1v+\\n(nlu
 .      ev 2
+.      nm
 .      da |f
 .      in 0
 .      xl \\n($lu-\\n(fuu
@@ -1420,6 +1511,7 @@
 .el \
 \{\
 .      ev 2
+.      nm
 .      in 0
 .      xl \\n($lu-\\n(fuu
 .      @F \\n(ff


(A bit less atomic than one might hope.)


diff --git a/tmac/e.tmac-u b/tmac/e.tmac-u
index 778e3adb1..3c380c87f 100644
--- a/tmac/e.tmac-u
+++ b/tmac/e.tmac-u
@@ -986,7 +986,7 @@
 .nf
 .ls 1
 .in 0
-.sp \\n(zsu
+.if !`\\$2`ns` .sp \\n(zsu     \" no pre-spacing if (b .. )b
 .if \\n(?n \
 .      nm \\n(ln
 .\\$1
@@ -1019,7 +1019,7 @@
 .@)
 .if (\\n(bt=0):(\\n(.t<\\n(bt) \
 .      ne \\n(dnu              \" make it all on one page
-.@k |k
+.@k |k ns
 .ev                             \" return from display environment
 .sp \\n(bsu+\\n(.Lv-1v
 ..


[1] This is the notorious George Helffrich "bane of my existence" commit.

G. Branden Robinson <gbranden>
Group administrator
Wed 22 Dec 2021 01:09:24 AM UTC, comment #6: 

comment #5:

> comment #4:
> > I therefore do not think there is a bug here; if only half a
> > line of space is available, a full-height line cannot be typeset.
>
> Yes, but we know that a full-height line can be typeset, because when that line appears below the .(b block, a full-height line is typeset.  Moving it from below to above the block shouldn't change how many full-height lines fit on the page.


That certainly makes intuitive sense.

I went back and re-instrumented e.tmac and keep.me a bit differently, and parameterized the latter so that it was easy to control the output of lines before and after the block.  I added Boolean flags for them, so that gives us four combinations.  All I can find is internal consistency.


$ tg -me -rA1=0 -rA2=0 EXPERIMENTS/keep.me >| keep-00.ps
GBR: height of 7 lines: 84000u
GBR: before (b: nl=624000u
GBR: )b: dn=84000u, .t=90000u
GBR: height of 7-line diversion: 84000u
$ tg -me -rA1=0 -rA2=1 EXPERIMENTS/keep.me >| keep-01.ps
GBR: height of 7 lines: 84000u
GBR: before (b: nl=624000u
GBR: )b: dn=84000u, .t=90000u
GBR: height of 7-line diversion: 84000u
$ tg -me -rA1=1 -rA2=0 EXPERIMENTS/keep.me >| keep-10.ps
GBR: height of 7 lines: 84000u
GBR: before (b: nl=636000u
GBR: )b: dn=84000u, .t=78000u
GBR: height of 7-line diversion: 84000u
$ tg -me -rA1=1 -rA2=1 EXPERIMENTS/keep.me >| keep-11.ps
GBR: height of 7 lines: 84000u
GBR: before (b: nl=636000u
GBR: )b: dn=84000u, .t=78000u
GBR: height of 7-line diversion: 84000u


I observe that (1) the diversion is always the height we expect (I think this rules out confounding effects of vertical spacing or the presence/absence of an additional break); (2) adding a line before it increases the page position by 1v; and (3) that increase makes me(7) decide there is not enough room for the diversion on the page, so the break happens when and where you expect based on these numbers.

I'll attach my new keep.me (I might have to name it keep2.me).  Here's the diff to e.tmac.


diff --git a/tmac/e.tmac b/tmac/e.tmac
index cbe4ae6a7..ba8ac212f 100644
--- a/tmac/e.tmac
+++ b/tmac/e.tmac
@@ -1071,6 +1071,7 @@
 .
 .de )b                 \" *** end block
 .@)
+.tm GBR: )b: dn=\\n(dnu, .t=\\n(.tu
 .if (\\n(bt=0):(\\n(.t<\\n(bt) \
 .      ne \\n(dnu              \" make it all on one page
 .@k |k ns


> However, this may be less a -me bug and more a groff one: replacing the ".(b" and ".)b" bracketing the block with a ".ne 7" before the block exhibits the same behavior.


I'm not sure precisely what behavior we're observing.  Motivated by your remark above, I presumed that I could get groff to produce two pages from the following input.


$ cat EXPERIMENTS/ne.roff
.nf
.ne 66v
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66\c


...but I can't.  (Note the absence of a break on the very last line of the page--it makes all the difference.)

Starting from there, can you elucidate where the surprise creeps in?

(I tried to reason directly from that example, particularly including the `\c` factor, but I can't--me(7) always inserts `br` requests around diversions anyway (similarly to groff's "box" diversions), and anyway if there were a missing line break at the end, the diversion's height would be shorter.  Trying it out, adding `\c` after "fifty-two" made no difference.)

G. Branden Robinson <gbranden>
Group administrator
Mon 20 Dec 2021 05:59:16 PM UTC, comment #5: 

comment #4:

> I therefore do not think there is a bug here; if only half a
> line of space is available, a full-height line cannot be typeset.


Yes, but we know that a full-height line can be typeset, because when that line appears below the .(b block, a full-height line is typeset.  Moving it from below to above the block shouldn't change how many full-height lines fit on the page.

However, this may be less a -me bug and more a groff one: replacing the ".(b" and ".)b" bracketing the block with a ".ne 7" before the block exhibits the same behavior.

Dave <barx>
Group Member
Sun 19 Dec 2021 09:57:02 AM UTC, comment #4: 

Hi Dave,

comment #3:

>   The symptom is caused by a higher footer in troff-mode than in the
> nroff-mode.
>
>   The footer in troff-mode is half a line bigger (6.5 v).
>
>   The documentation ("doc/meref.ps") mentions "6v ...[nominal]" for
> the register "bm".
>
>   It is common to have a different page layout in troff from that in
> nroff as the resolution is very different.
>
>   troff is a typesetting "machine"; nroff a typewriter, line printer
> "machine".


I believe Bjarni [quoted] is correct here.

I instrumented e.tmac's `)b` macro to report the .t (distance to next trap), dn (diversion height), and bt (me's "break threshold"), and to dump all active traps.

I also instrumented your reproducer (which I called "keep.me") to also report the distance to the next trap and the vertical line height (.v) in interesting places.

I noticed that the distance to the next trap was exactly half a line height in your original reproducer (before the slight modification that makes nroff and troff diverge).

I then went ahead with your procedure of adding one line, but modified it by typesetting the new line in 5-point type on 6-point vertical spacing.

Lo and behold, the example started working as you expected again.

I therefore do not think there is a bug here; if only half a line of space is available, a full-height line cannot be typeset.

Do you concur?

I am including my "keep.me" and the patch I applied to e.tmac to instrument it.  (It might apply with an offset--I have other experiments in the pipeline.)


$ cat EXPERIMENTS/keep.me
.nr bs 0         \" no extra space before or after blocks
.nf
.\".ig
.ps 5z
.vs 6p
Hello, world!
.vs
.ps
.\"..
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
.tm GBR: keep.me: before (b: .t=\n(.t, .v=\n(.v
.(b
forty-six
forty-seven
forty-eight
forty-nine
fifty
fifty-one
fifty-two
.tm GBR: keep.me: before )b: .t=\n(.t, .v=\n(.v
.)b
.tm GBR: keep.me: after )b: .t=\n(.t, .v=\n(.v
53
54
55
56
57
58
59
60



$ ./build/test-groff -b -ww -a -me EXPERIMENTS/keep.me | grep -FC8 page
GBR: keep.me: before (b: .t=84000, .v=12000
GBR: keep.me: before )b: .t=2147483646, .v=12000
GBR: )b: .t=84000, dn=84000, bt=0
@h      0
@f      -78000
GBR: keep.me: after )b: .t=630000, .v=12000
<beginning of page>
Hello, world!
1
2
3
4
5
6
7
--
45
 forty-six
 forty-seven
 forty-eight
 forty-nine
 fifty
 fifty-one
 fifty-two
<beginning of page>
53
54
55
56
57
58
59
60



diff --git a/tmac/e.tmac b/tmac/e.tmac
index 4f74a1966..912d43401 100644
--- a/tmac/e.tmac
+++ b/tmac/e.tmac
@@ -1071,6 +1071,8 @@
 .de )b                 \" *** end block
 .br
 .@)
+.tm GBR: )b: .t=\\n(.t, dn=\\n(dn, bt=\\n(bt
+.ptr
 .if (\\n(bt=0):(\\n(.t<\\n(bt) \
 .      ne \\n(dnu              \" make it all on one page
 .@k |k ns


G. Branden Robinson <gbranden>
Group administrator
Thu 06 Feb 2020 11:25:37 PM UTC, comment #3: 

  The symptom is caused by a higher footer in troff-mode than in the
nroff-mode.

  The footer in troff-mode is half a line bigger (6.5 v).

  The documentation ("doc/meref.ps") mentions "6v ...[nominal]" for
the register "bm".

  It is common to have a different page layout in troff from that in
nroff as the resolution is very different.

  troff is a typesetting "machine"; nroff a typewriter, line printer
"machine".

Bjarni Ingi Gislason <bjarniig>
Sun 05 Jan 2020 02:01:38 AM UTC, comment #2: 

Additionally, the thread mentioned in comment #1 continued for a bit off-list when Deri emailed me some PDF attachments he couldn't easily send to the list.  And thanks to his work, I've been able to home in a little more on this bug.

Because as it turns out, while the bug was in groff 1.21, and is in current groff, it has not been in every version of groff released in between.  I'm not sure which all official releases it appears in -- it is absent in at least 1.22.2 and 1.22.3 -- but I've iterated through the various versions of tmac/e.tmac / tmac/e.tmac-u (the file was renamed at some point) to find out exactly where it came and went.

The bug disappeared in commit b6906269 of this file and reappeared in commit 98b028d6.  But the interesting thing about this is that these commits also respectively introduced and then removed another bug.

As documented, all "displays" in -me (of which ".(b" is one) should have space corresponding to the \n(bs register above and below; the test case at hand explicitly sets this register to 0, specifying no extra space above or below the block.  But starting with commit b6906269, this test case began emitting an extra line above the .(b block; commit 98b028d6 fixed this.

So somehow this extra-space bug removed, or at least masked, the page-break bug as a side effect.

Dave <barx>
Group Member
Sat 04 Jan 2020 09:20:23 PM UTC, comment #1: 

Further discussion about this bug, though nothing conclusive, took place on the groff email list, in the thread starting at http://lists.gnu.org/archive/html/groff/2013-03/msg00012.html

Dave <barx>
Group Member
Sat 04 Jan 2020 08:12:45 PM UTC, original submission:  

This bug has existed at least as far back as groff version 1.21, and continues to exist in a groff built from recent git code (GNU groff version 1.22.4.74-b400-dirty).  It was originally reported in http://lists.gnu.org/archive/html/bug-groff/2012-12/msg00011.html.  The code below assumes a paper size of "letter" (groff's default in the file /usr/share/groff/current/font/devps/DESC).

In the -me reference manual, the description for .(b says:

   If the block will not fit on the current page a new page
   is begun, unless that would leave more than \n(bt [0]
   white space at the bottom of the text.  If \n(bt is
   zero, the threshold feature is turned off.

This is not what happens in the edge case, however.  Consider:

------------------------------
.mso me.tmac     \" -me macros
.nr bs 0         \" no extra space before or after blocks
.nf
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
.(b
forty-six
forty-seven
forty-eight
forty-nine
fifty
fifty-one
fifty-two
.)b
53
54
55
56
57
58
59
60
------------------------------

With either -Tps or -Tascii, on the first page this outputs lines 1 to 45, then the seven-line block, then one more line, 53, before the page break.  This tells us that the block would fit exactly at the bottom of the page, if there were one more line above it (which would take the place of line 53 below).

So add one line somewhere above the .(b.

Now, -Tascii output does what's expected, putting line "fifty-two" at the bottom of the first page, and 53 at the top of the second.  But -Tps output puts the entire block on the second page, even though there is room for it on the first page.

An easy way to tell where the page break falls in -Tps output without having to generate and display a PostScript file is (using GNU's grep):

groff -Tps -a [test-file] | fgrep -C3 page

Dave <barx>
Group Member

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

Attached Files
file #52549:  keep3.me added by gbranden (304B - text/x-troff-me)
file #52550:  keep4.me added by gbranden (311B - text/x-troff-me)
file #52548:  keep2.me added by gbranden (484B - text/x-troff-me)

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by gbranden (Updated the item)
  • -email is unavailable- added by bjarniig (Posted a comment)
  • -email is unavailable- added by barx (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only logged-in users can vote.

     

    Follow 10 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2023-05-18 barx CategoryMacro me Macro - others/general
        Summary[me] incorrect computation of space `ne`eded by `(b` blocks .ne is off by one in some macro packages even when using no other facilities of that package
    2021-12-22 gbranden Summary[me] incorrect PostScript output of macro .(b [me] incorrect computation of space `ne`eded by `(b` blocks
    2021-12-22 gbranden Attached File- Added keep3.me, #52549
        Attached File- Added keep4.me, #52550
    2021-12-22 gbranden Attached File- Added keep2.me, #52548
    2021-12-19 gbranden StatusNone Need Info
        Assigned toNone gbranden
    2021-06-04 barx Summary-me macros: incorrect postscript output of macro .(b [me] incorrect PostScript output of macro .(b
    2020-07-25 gbranden CategoryMacro - others/general Macro me

    Back to the top

    Powered by Savane 3.13-758e.
    Corresponding source code