bugManeage - Bugs: bug #61811, jinja2 and extension-helpers...

 
 

You are not allowed to post comments on this tracker with your current authentication level.

bug #61811: jinja2 and extension-helpers packages not found in Astropy installation

Submitted by:  Raul Infante-Sainz <infantesainz>
Submitted on:  Thu 13 Jan 2022 11:17:24 AM UTC
 
Category:  Software Severity:  3 - Normal
Item Group:  Crash Status:  None
Privacy:  Public Assigned to:  None
Open/Closed:  Open
* Mandatory Fields

Add a New Comment (Rich Markup)
   

Comment Type & Canned Response:
       No canned response available

 

Wed 19 Jan 2022 10:33:06 AM UTC, comment #5: 

Indeed, a non-reproducible bug is the most annoying (showing that we still haven't understood it!).

I tried repeating the 'pybuild' commands of 'packaging', 'jinja2' and 'extension-helpers' within their recipes to see if this problem can be fixed with a repeat. But I found that a simple repeated build doesn't fix the problem!

So its a dependency issue: in order for their presence to be recognized by the problematic packages (for example 'pyerfa' depends on 'packaging'), some other Python package needs to be present (which is present when the build finishes once!).

That software X has to become a dependency of the three problematic packages whose build is not recognized (be present before we build them).

Mohammad Akhlaghi <makhlaghi>
Project Administrator
Fri 14 Jan 2022 03:27:42 PM UTC, comment #4: 

In each case, I re-ran './project configure --existing-conf' without modifying either TARGETS.conf or python.mk.

In both separate installs I had (still have):

# Programs and libraries (for Python or R modules, use respective variable).
top-level-programs  = gnuastro

# Python libraries/modules.
top-level-python    = astropy

I didn't try this with '--all-highlevel'.

If we could make the bug exactly reproducible ;), then it would be easier to solve...

Boud Roukema <boud>
Project Member
Fri 14 Jan 2022 08:50:00 AM UTC, comment #3: 

Thanks a lot for the help Boud.

I need to remove the final target of the problematic package (jinja2, etc.) to be able to finish the configure step and install all programs. Boud, do you need to remove the final targets? or it is only necessary to re-run the configure step again?

Raul Infante-Sainz <infantesainz>
Project Member
Thu 13 Jan 2022 06:30:57 PM UTC, comment #2: 

I tried this in two separate builds (not completely from scratch, but partly from scratch, and both python and the python libraries were rebuilt from scratch). Both builds were on the same many-cpu machine.

The second build, which updated many programs including gcc, compiled commit f9c8b81 (with 'astropy' enabled in TARGETS.conf) completely, without any errors. => irreproducible error.

The first build had a similar problem to the jinja2 and extension-helpers bugs. => reproducible error. Compiling 'astropy' failed with fatal compile errors of 'pyyaml':

/BUILD/software/installed/include/yaml.h:636:55: note: expected 'const yaml_char_t *' {aka 'const unsigned char *'} but argument is of type 'char *'
  636 |         const yaml_char_t *anchor, const yaml_char_t *tag, int implicit,
      |                                    ~~~~~~~~~~~~~~~~~~~^~~
ext/_yaml.c: In function '__pyx_tp_dealloc_5_yaml_CParser':
ext/_yaml.c:23870:5: error: lvalue required as increment operand
23870 |     ++Py_REFCNT(o);
      |     ^~
ext/_yaml.c:23872:5: error: lvalue required as decrement operand
23872 |     --Py_REFCNT(o);
      |     ^~
ext/_yaml.c: In function '__pyx_tp_dealloc_5_yaml_CEmitter':
ext/_yaml.c:24040:5: error: lvalue required as increment operand
24040 |     ++Py_REFCNT(o);
      |     ^~
ext/_yaml.c:24042:5: error: lvalue required as decrement operand
24042 |     --Py_REFCNT(o);
      |     ^~
...

This seems to imply a missing cython dependency. But pyyaml does depend on cython in python.mk.

A second 'configure' gave an immediate error for compiling pyerfa:

  File "/dev/shm/maneage-maneage_project_bis-boud/pyerfa-2.0.0.1/setup.py", line 12, in <module>
    import packaging.version
ModuleNotFoundError: No module named 'packaging'

But 'packaging' was already installed, and pyyerfa depends on 'packaging' in python.mk.

A third 'configure' successfully installed 'astropy'.

Possible clue, though probably not the explanation: Before and after the third configure step, I did 'find .local/ -name "*packaging*"'. Here's the difference:

$ diff -ub find.local.name.packaging.*
--- find.local.name.packaging.1 2022-01-13 17:50:27.652452401 +0100
+++ find.local.name.packaging.2 2022-01-13 17:51:48.575848367 +0100
@@ -1,4 +1,5 @@
 .local/version-info/python/packaging-21.3
+.local/version-info/.for-check-config/packaging-21.3
 .local/lib/python3.10/site-packages/packaging-21.3-py3.10.egg
 .local/lib/python3.10/site-packages/setuptools-58.3.0-py3.10.egg/pkg_resources/_vendor/packaging
 .local/lib/python3.10/site-packages/setuptools-58.3.0-py3.10.egg/setuptools/_vendor/packaging

The directory '.local/version-info/.for-check-config' is created by './project'. While that's useful for --check-config (I assume :)), I don't see how it can affect the rules in 'python.mk'.

This seems to me to be a semi-reproducible bug.

Boud Roukema <boud>
Project Member
Thu 13 Jan 2022 04:03:38 PM UTC, comment #1: 

I wonder if this is due to the same bug as https://savannah.nongnu.org/bugs/?61271 - Python build bug with curses.h in commit 775fc03 .

Boud Roukema <boud>
Project Member
Thu 13 Jan 2022 11:17:24 AM UTC, original submission:  

I am trying to run the installation of all software of Maneage from scratch. All basic software runs smoothly without any error.

For the high-level software there is something strange with Astropy. It crashes about not finding 'jinja2'. But it is a prerequisite of Astropy! If I remove the target of this package (.local/version-info/python/jinja2-3.0.3) and run the configure step again, it finishes without any problem.

The problem described above with 'jinja2' also happen with the package 'extension-helpers'.

I have been investigating a bit the origin of the problem but I found nothing. If you have any idea let me know.

Issue detected in task #16103 Software update 2022-1.

Raul Infante-Sainz <infantesainz>
Project Member

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by makhlaghi (Posted a comment)
  • -email is unavailable- added by boud (Posted a comment)
  • -email is unavailable- added by infantesainz (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only logged-in users can vote.

     

     

     

     

    No changes have been made to this item

    Back to the top


    Powered by Savane 3.9