## bug #31243: A space after renewcommand in preamble causes crash

 Submitted by: Jose Ramon Alvarez-Sanchez Submitted on: Mon 04 Oct 2010 04:58:04 PM UTC Category: None Severity: 3 - Normal Item Group: None Status: Fixed Privacy: Public Assigned to: Alex Fernandez Open/Closed: Closed

Add a New Comment (Rich Markup):

## You are not logged in

Thu 14 Oct 2010 11:00:56 PM UTC, comment #16:

Yes, this time you really got it. At least I cannot find anything more related to this bug, so, please, close it and go ahead with 1.0.3 release. Nice work.
Thanks a lot.

Jose Ramon Alvarez-Sanchez <jras>
Thu 14 Oct 2010 09:49:57 PM UTC, comment #15:

Hi again,

It has cost me a bit of a headache, but I think I got it right now.

• The first problem was child documents in a comment should be ignored. Since in 1.0.3 child documents are extracted from the containing layout and processed separately, the empty output in a comment was ignored. It has been solved with a custom filter which does not process any child documents in a container with empty output (such as a comment).
• The second problem was child documents with numbering are numbered twice. The problem here was double post-processing (which is where numbering is done) for child document contents. A simple control (but which needs a lot of fine-tuning to get right) has been added -- or rather re-added, since it was there not long ago. Sigh.

Once these issues are confirmed as solved (and if nothing else comes up) I will release 1.0.3; there are a lot of fixes in the release queue. Thanks again!

Alex Fernandez <alexfernandez>
Sun 10 Oct 2010 08:32:09 AM UTC, comment #14:

Indeed, some changes were done in the way child documents are parsed: now they are extracted from the containing layout and rendered independently. This strategy solves some issues when child documents are inserted in different layouts, but I see now that it creates others. In particular, a layout of type comment should be ignored -- but the child document inside isn't.

The other problem has to be added to my test suite, your test files make the problem clear.

I will see how to solve both. Thanks!

Alex Fernandez <alexfernandez>
Sat 09 Oct 2010 06:56:12 PM UTC, comment #13:

Spaces and comments inside commands and equations are OK now, but while testing with a large document I found some differences respect to version 1.0.2. I'm not sure if they are related or not with spaces or with the changes done to correct this bug.

• The contents of a child document that is inside a comment (menu: Insert->Note->Comment in LyX) are shown with last version, but with LaTeX or with eLyXer 1.0.2 they are not shown. This only occurs with the contents of the child document, but the text directly in a Note-Comment is not shown.

Although with version 1.0.2 the contents of the child document inside comment were erroneously included in the table of contents and the subsequent chapter, section, subsection, etc. numbers were affected.

• The chapters, sections, subsections, etc. numbering is shown twice for the sections inside all the child documents but shown normally for sections in the master document. Also in Table of Contents the title of the sections inside child document are shown with an extra space at the begining.

Attached 3 test documents (master.lyx, child1.lyx, child2.lyx) to show both problems and compare results of rendering.

Jose Ramon Alvarez-Sanchez <jras>
Sat 09 Oct 2010 07:44:46 AM UTC, comment #12:

True, the current code was insisting in finding a regular formula bit, but comments are ignored and not returned. Now we can deal with a comment at the end, ignore it and go on.

Ready to test again. Keep torturing eLyXer, it is getting quite robust by now!

Alex Fernandez <alexfernandez>
Sat 09 Oct 2010 01:02:40 AM UTC, comment #11:

Empty equation and lone comment are OK.

Comment inside \mbox (and similar) almost correct, except when there is nothing after the comment. In this case there is a message error and the closing "}" is shown in conversion. The LyX file in the attach (eqmboxcom.lyx) have 3 test cases, first and second are OK, the third one gives the following messages:

Line 90: * Unrecognized formula at *}
Line 90: * Pending endings [Ending }] left open
Line 90: * No ending out of bounds
Line 90: * Expected ending }, got
Line 90: * Pending endings [Ending }] left open

Best regards,
Jose R.

Jose Ramon Alvarez-Sanchez <jras>
Fri 08 Oct 2010 10:28:34 PM UTC, comment #10:

The solution to the empty equation (and the lone comment) was more or less easy: just check for an empty equation after removing spaces and comments. Note that LyX (at least with pdflatex and pdf2ps) is not able to convert a lone comment to PDF form, so I had to remove that test from my test case.

For the comment inside a text command (such as \mbox or \textrm) I just added it to the list of special characters interpreted inside text, along with \, ^ and _.

The attached version should (one more time) solve both issues. You are earning your place among eLyXer testers!

Alex Fernandez <alexfernandez>
Thu 07 Oct 2010 11:56:10 PM UTC, comment #9:

Yes, this one works well with all the previous cases.

I was not able to imagine a new case to torture eLyXer, but by chance I got a message in a simple file without any apparent equation. It turned out to be just an empty equation left from other tries.
So, just create an equation (inline or displayed) in LyX, and move cursor out of it without typing anything inside. The edition box disapears, but equation remains hidden, and eLyXer convert it to a red '*' (unknown) and it shows the message:
Line 60: * Unknown formula at: *

Also the same message occurs if there is just only a comment inside an equation.

Another different case was found also by chance. When editing an equation and then pressing Ctrl-m inside it, LyX inserts an "\mbox". The problem is that comments inside that mbox are shown in conversion done by eLyXer (including the % symbol), but LaTeX correctly does not show them. The same occurs with comments inside other functions like "\textrm", etc.

Thanks,
Jose R.

Jose Ramon Alvarez-Sanchez <jras>
Thu 07 Oct 2010 09:52:27 PM UTC, comment #8:

Once again, you have unveiled subtle bugs -- not necessarily related to the symptoms.

• The first situation was caused by a bug in the multiliner $...$ formula parsing code: line returns were ignored, which had no effect in regular parsing but caused errors when parsing a comment, since the ending \n was not found.
• The second sample causes my poor parsing code to crash for comments before the end of an element; it is more robust now.

Once more, test version attached. Thanks, and keep them coming!

Alex Fernandez <alexfernandez>
Thu 07 Oct 2010 09:51:43 AM UTC, comment #7:

With comments inside commands in the LaTeX preamble it works OK, even with more than a comment line and with a blank line.

I tried also to insert a comment inside an equation through LyX equation editor (monitoring at the same time with view source active) and, surprisingly, by typing "\%" inside an equation LyX automaticaly inserted the comment and a new line in the source (to insert spaces inside the comment you need to type "\ "). The file is correctly rendered with LaTeX.

If the comment in equation is inserted inside a function element, like "\frac" (see attached example "cmeq1.lyx"), then the conversion by eLyXer is done (incorrect in the equation) but there is a message:

Line 53: * Pending endings [Ending }] left open
Line 53: * No ending out of bounds
Line 53: * Expected ending }, got
Line 53: * Pending endings [Ending }] left open
Line 53: * No parameter found at: *
Line 53: * Pending endings [Ending }] left open
-----

If the comment in the equation is inserted outside function elements (see attach example "cmeq2.lyx") then the conversion fails and eLyXer crashes with error messages copied in the attach "cmeq2-error.txt".

Thanks,
Jose R.

Jose Ramon Alvarez-Sanchez <jras>
Thu 07 Oct 2010 06:24:06 AM UTC, comment #6:

Hi again,

Now it should be solved. Can you try the attached version and let me know? Keep in mind that extensive changes to the math parsing code have been done, so something else (not uncovered in my automatic tests) might fail. Thanks!

Alex.

Alex Fernandez <alexfernandez>
Tue 05 Oct 2010 11:12:20 PM UTC, comment #5:

Well, that way to insert comments inside commands is not very usual. Also, with your last version it was able to do the conversion without crash, it just give a message about the command being ignored, so this is only a minor problem.
Thanks a lot for your work.

Jose Ramon Alvarez-Sanchez <jras>
Tue 05 Oct 2010 10:40:43 PM UTC, comment #4:

This nut is proving harder to crack: I am changing the way LaTeX commands are parsed so that spaces and comments can be recognized anywhere. It requires quite a bit of code changes, but I am slowly getting there. I will let you know ASAP to test it!

Alex Fernandez <alexfernandez>
Tue 05 Oct 2010 12:04:58 AM UTC, comment #3:

Yes, the version attached in your comment #2 solves the initial problem with spaces after renewcommand in preamble in my system.

I just tested it with an example file that caused the error with elyxer 1.0.2, but with your patched copy of elyxer.py it work correctly. It even works inserting a new line after \renewcommand.

But also inserting a comment after \renewcommand like the following (in the preamble):
\renewcommand % comment
{\labelenumi} {\Alph{enumi}.}

does the conversion but also gives a message in stderr:
Line 46: * Unknown formula bit in defining function at *% comment
{\labelenumi} {\Alp

The comment at the end of line is accepted viewing the DVI without any problem.

Jose Ramon Alvarez-Sanchez <jras>
Mon 04 Oct 2010 11:12:32 PM UTC, comment #2:

Hi again,

I think the problem has been solved. If you want to try it out there is a test version of elyxer.py attached to the bug; just replace your copy with this one and let me know how it goes!

Thanks, Alex.

Alex Fernandez <alexfernandez>
Mon 04 Oct 2010 09:56:03 PM UTC, comment #1:

Hi José Ramón,

The problem was easy enough to reproduce with your detailed description. For starters I avoided the crash; now eLyXer just ignores the definition if it does not understand any piece of it.

I found after a quick search that spaces in math mode are supposed to be ignored, so I am going to implement just that. Are you interested in testing the solution? If so I can send you a test version (and add you to the acknowledgements as a tester!).

Alex Fernandez <alexfernandez>
Mon 04 Oct 2010 04:58:04 PM UTC, original submission:

With eLyXer 1.0.2, inside the LaTeX preamble of the LyX document, putting a space just after a "\renewcommand" causes eLyXer to exit with an error window in Lyx.

For example, in a simple LyX document, insert the following line into the LaTeX preamble (note the space after "\renewcommand"):
\renewcommand {\labelenumi}{\Alph{enumi}.}

There is no problem in LaTeX (view DVI), but trying to convert with eLyXer gives the error messages attached and an error window in LyX.

If all the spaces after "\renewcommand" are removed, then all works OK in eLyXer.

Also, the same command (with spaces) in the document body (as ERT) does not cause any error in eLyXer.

Jose Ramon Alvarez-Sanchez <jras>

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach File(s):

Comment:

Attached Files
file #21678:  elyxer.py added by alexfernandez (264kB - text/x-python)
file #21649:  master.lyx added by jras (2kB - application/octet-stream)
file #21650:  child1.lyx added by jras (1kB - application/octet-stream)
file #21651:  child2.lyx added by jras (1kB - application/octet-stream)
file #21645:  elyxer.py added by alexfernandez (261kB - text/x-python)
file #21643:  eqmboxcom.lyx added by jras (1kB - application/octet-stream)
file #21642:  elyxer.py added by alexfernandez (261kB - text/x-python)
file #21635:  elyxer.py added by alexfernandez (261kB - text/x-python)
file #21629:  cmeq1.lyx added by jras (985B - application/octet-stream)
file #21630:  cmeq2-error.txt added by jras (1kB - text/plain)
file #21631:  cmeq2.lyx added by jras (984B - application/octet-stream)
file #21628:  elyxer.py added by alexfernandez (261kB - text/x-python)
file #21615:  elyxer.py added by alexfernandez (259kB - text/x-python)
file #21610:  space-error-msgs.txt added by jras (2kB - text/plain)

Depends on the following items: None found

Items that depend on this one: None found

Carbon-Copy List
• -unavailable- added by alexfernandez (Posted a comment)
• -unavailable- added by jras (Submitted the item)
•

Do you think this task is very important?
This task has 0 encouragements so far.

Only logged-in users can vote.

Please enter the title of George Orwell's famous dystopian book (it's a date):

Date Changed By Updated Field Previous Value => Replaced By
Fri 15 Oct 2010 07:51:22 PM UTCalexfernandezStatusReady For TestFixed
Open/ClosedOpenClosed
Thu 14 Oct 2010 09:49:57 PM UTCalexfernandezAttached File-Added elyxer.py, #21678
Sun 10 Oct 2010 08:32:09 AM UTCalexfernandezStatusReady For TestConfirmed
Sat 09 Oct 2010 06:56:12 PM UTCjrasAttached File-Added master.lyx, #21649
Sat 09 Oct 2010 07:44:46 AM UTCalexfernandezAttached File-Added elyxer.py, #21645
Sat 09 Oct 2010 01:02:40 AM UTCjrasAttached File-Added eqmboxcom.lyx, #21643
Fri 08 Oct 2010 10:28:34 PM UTCalexfernandezAttached File-Added elyxer.py, #21642
Thu 07 Oct 2010 09:52:27 PM UTCalexfernandezAttached File-Added elyxer.py, #21635
Thu 07 Oct 2010 09:51:43 AM UTCjrasAttached File-Added cmeq1.lyx, #21629