bugManeage - Bugs: bug #61271, Python build bug with curses.h in...

 
 

You are not allowed to post comments on this tracker with your current authentication level.

bug #61271: Python build bug with curses.h in commit 775fc03

Submitter:  Boud Roukema <boud>
Submitted:  Sun 03 Oct 2021 08:22:43 PM UTC
   
 
Category:  None Severity:  3 - Normal
Item Group:  None Status:  Works For Me
Privacy:  Public Assigned to:  None
Open/Closed:  Open

Wed 21 Sep 2022 05:05:38 PM UTC, comment #4: 

DESCRIPTION: ncurses header files are typically provided in several major systems (Debian) directly in /usr/include . The 'configure' of python-3.10.6, for commit 4318670 of core maneage, still had fatal errors because it couldn't find either ncurses.h or curses.h in either the Maneage include/ directory or the host OS /usr/include .

EXAMPLE LOG:

 18136  /BUILD/software/installed/bin/gcc -pthread -fPIC -Wno-unused-result -Wsign-compare
 -DNDEBUG -g -fwrapv -O3 -Wall -std=c99 -Wextra -Wno-unused-result -Wno-unused-parameter -Wno-missing-field-initializers -Werror=implic
it-function-declaration -fvisibility=hidden -I./Include/internal -I./Include -I. -I/BUILD/
software/installed/include -I/usr/include/x86_64-linux-gnu -I/usr/local/include -I/dev/shm/BUILD/python-3.10.6/I
nclude -I/dev/shm/BUILD/python-3.10.6 -c /dev/shm/BUILD/python-3.10.6/Modules/termios.c -
o build/temp.linux-x86_64-3.10/dev/shm/BUILD/python-3.10.6/Modules/termios.o
 18137  /dev/shm/BUILD/python-3.10.6/Modules/readline.c: In function 'setup_readline':
 18138  /dev/shm/BUILD/python-3.10.6/Modules/readline.c:1258:37: warning: assignment discards 'const' qualifier
from pointer target type [-Wdiscarded-qualifiers]
 18139   1258 |     completer_word_break_characters =
 18140        |                                     ^
 18141  building 'resource' extension
 18142  /BUILD/software/installed/bin/gcc -pthread -fPIC -Wno-unused-result -Wsign-compare
 -DNDEBUG -g -fwrapv -O3 -Wall -std=c99 -Wextra -Wno-unused-result -Wno-unused-parameter -Wno-missing-field-initializers -Werror=implic
it-function-declaration -fvisibility=hidden -I./Include/internal -I./Include -I. -I/BUILD/
software/installed/include -I/usr/include/x86_64-linux-gnu -I/usr/local/include -I/dev/shm/BUILD/python-3.10.6/I
nclude -I/dev/shm/BUILD/python-3.10.6 -c /dev/shm/BUILD/python-3.10.6/Modules/resource.c
-o build/temp.linux-x86_64-3.10/dev/shm/BUILD/python-3.10.6/Modules/resource.o
 18143  In file included from /dev/shm/BUILD/python-3.10.6/Modules/_curses_panel.c:15:
 18144  ./Include/py_curses.h:36:10: fatal error: curses.h: No such file or directory
 18145     36 | #include <curses.h>
 18146        |          ^~~~~~~~~~
 18147  compilation terminated.
...
 18171  In file included from /dev/shm/BUILD/python-3.10.6/Modules/_cursesmodule.c:114:
 18172  ./Include/py_curses.h:36:10: fatal error: curses.h: No such file or directory
 18173     36 | #include <curses.h>
 18174        |          ^~~~~~~~~~
 18175  compilation terminated.
...
 19279  checking for getnameinfo... yes
 19280  checking for strlcpy... In file included from /dev/shm/BUILD/python-3.10.6/Modules/_curses_panel.c:15:
 19281  ./Include/py_curses.h:36:10: fatal error: curses.h: No such file or directory
 19282     36 | #include <curses.h>
 19283        |          ^~~~~~~~~~
 19284  compilation terminated.
 19285  In file included from /dev/shm/BUILD/python-3.10.6/Modules/_cursesmodule.c:114:
 19286  ./Include/py_curses.h:36:10: fatal error: curses.h: No such file or directory
 19287     36 | #include <curses.h>
 19288        |          ^~~~~~~~~~
 19289  compilation terminated.
...
 19302  Failed to build these modules:
 19303  _curses               _curses_panel



SUGGESTED FIX:
Commit 7286ad6 on branch 'fix_ncurses_for_python' [1] works for me. It adds the two lines

         cd "$(iidir)"
         ln -fsv $(idir)/include/ncursesw/*.h .

to the 'ncurses' build rules in 'basic.mk'.

[1] https://codeberg.org/boud/maneage_dev/commit/7286ad62014acaa8c46504493040c5acba7bd535

Boud Roukema <boud>
Group Member
Sun 09 Jan 2022 01:38:58 PM UTC, comment #3: 

comment #2:

> Thanks for the good bug report Boud!
>
> During your test builds of Python 3.10 (as part of
> task #16103; Software update 2022-1), did you still encounter this problem?


Yes. The bug still occurs for python-3.10.0, with the same error messages. Running the
configure script a second time is an effective workaround, but it is not a real solution.

Boud Roukema <boud>
Group Member
Sat 08 Jan 2022 11:40:04 PM UTC, comment #2: 

Thanks for the good bug report Boud!

During your test builds of Python 3.10 (as part of
task #16103; Software update 2022-1), did you still encounter this problem?

Mohammad Akhlaghi <makhlaghi>
Group administrator
Sun 03 Oct 2021 08:22:43 PM UTC, comment #1: 

This item has been reassigned from the project Maneage task tracker to your tracker.

The original report is still available at task #16051

Following are the information included in the original report:

[field #0]                  Item ID: 16051

[field #1]                 Group ID: 11920

[field #2]              Open/Closed: Open

[field #3]                 Severity: 3 - Normal

[field #4]                  Privacy: Public

[field #8]                         : Unknown task Field Display Type

[field #9]                 Category: Software

[field #10]             Submitted by: boud

[field #11]              Assigned to: None

[field #12]             Submitted on: Sun 03 Oct 2021 08:21:46 PM UTC

[field #13]                  Summary: Python build bug with curses.h in commit 775fc03

[field #14]      Original Submission: BUG: Maneage commit 775fc03 has fatal errors compiling some python modules. A second round of './project configure' leads to the Maneage system considering all packages to be compiled correctly, despite the errors declared as fatal. It is not known if the fatal errors cause problems when running python.


SYSTEM DETAILS:

  • Debian GNU/Linux bullseye 11.0, i.e stable, x86_64 architecture
  • Maneage commit 775fc03



LOG:
https://cosmo.torun.pl/~boud/log.20211003.1.775fc03.python.bug.bz2


EXTRACTS from the log:

This is a parallel compile, so the specific commands might be interleaved. [Using the '--debug' option doesn't seem to work fully in parallel, giving some lines printed in an interleaved way - probably some output is buffered and not rapidly synced.]

There are a few sections such as the following:


/BUILD/software/installed/bin/gcc -pthread -fPIC -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -std=c99 -Wextra -Wno-unused-result -Wno-unused-parameter -Wno-missing-field-initializers -Werror=implicit-function-declaration -I./Include/internal -I./Include -I. -I/BUILD/software/installed/include -I/usr/local/include -I/BUILD/software/build-tmp/Python-3.8.5/Include -I/BUILD/software/build-tmp/Python-3.8.5 -c /BUILD/software/build-tmp/Python-3.8.5/Modules/_blake2/blake2module.c -o build/temp.linux-x86_64-3.8/BUILD/software/build-tmp/Python-3.8.5/Modules/_blake2/blake2module.o
/BUILD/software/installed/bin/gcc -pthread -fPIC -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -std=c99 -Wextra -Wno-unused-result -Wno-unused-parameter -Wno-missing-field-initializers -Werror=implicit-function-declaration -I./Include/internal -I./Include -I. -I/BUILD/software/installed/include -I/usr/local/include -I/BUILD/software/build-tmp/Python-3.8.5/Include -I/BUILD/software/build-tmp/Python-3.8.5 -c /BUILD/software/build-tmp/Python-3.8.5/Modules/grpmodule.c -o build/temp.linux-x86_64-3.8/BUILD/software/build-tmp/Python-3.8.5/Modules/grpmodule.o
In file included from /BUILD/software/build-tmp/Python-3.8.5/Modules/_curses_panel.c:15:
./Include/py_curses.h:36:10: fatal error: curses.h: No such file or directory
   36 | #include
      |          ^~~~~~~~~~
compilation terminated.
In file included from /BUILD/software/build-tmp/Python-3.8.5/Modules/_cursesmodule.c:113:
./Include/py_curses.h:36:10: fatal error: curses.h: No such file or directory
   36 | #include
      |          ^~~~~~~~~~
compilation terminated.
building '_elementtree' extension


The python management system itself seems to consider the build successful:


Python build finished successfully!
The necessary bits to build these optional modules were not found:
_dbm                  _gdbm                 _sqlite3
_tkinter
To find the necessary bits, look in setup.py in detect_modules() for the module's name.


The following modules found by detect_modules() in setup.py, have been
built by the Makefile instead, as configured by the Setup files:
_abc                  atexit                pwd
time


Failed to build these modules:
_curses               _curses_panel


Following modules built successfully but were removed because they could not be imported:
_uuid                 nis

running build_scripts
creating build/scripts-3.8


Nevertheless, the first '.project configure' run fails. A second iteration of './project configure' runs without further interruption.

PRESENCE of curses.h:

All these are present:


.local/include/ncurses/cursesapp.h
.local/include/ncurses/cursesf.h
.local/include/ncurses/curses.h
.local/include/ncurses/cursesm.h
.local/include/ncurses/cursesp.h
.local/include/ncurses/cursesw.h
.local/include/ncurses/ncurses_dll.h
.local/include/ncurses/ncurses.h -> curses.h
.local/include/ncursesw/cursesapp.h
.local/include/ncursesw/cursesf.h
.local/include/ncursesw/curses.h
.local/include/ncursesw/cursesm.h
.local/include/ncursesw/cursesp.h
.local/include/ncursesw/cursesw.h
.local/include/ncursesw/ncurses_dll.h
.local/include/ncursesw/ncurses.h -> curses.h
.local/include/python3.8/py_curses.h


and the compilation path appears to be valid for finding 'curses.h', so the bug is puzzling.


STATUS as a bug:

So far I'm not aware of this having consequences in running python scripts. However, since we clearly have 'curses' installed as part of the basic.mk system, in principle it should be easy to solve the bug of not finding 'curses.h'.

Moreover, the build should compile in a single run, without needing a second configure round.

However, I've rated this as "3 = low" on the priority scale, since practical problems following from the bug are not (yet) known.

I see that the commented URL for python in 'reproduce/software/conf/urls.conf' is http://akhlaghi.org/src . This leads to a related issue: for URLs that have akhlaghi.org URLs alone, it would be better to include an example of the upstream URL as a comment just before/after, and any differences should be described. Even if the upstream URL risks being volatile and disappearing, in many cases these will survive in the long term so making the user's task easier would be better.


[field #16]               Item Group: None

[field #17]                   Status: None

[field #18]        Component Version: None

[field #19]         Operating System: None

[field #20]          Reproducibility: None

[field #21]               Size (loc): None

[field #22]            Fixed Release: None

[field #23]          Planned Release: None

[field #24]                   Effort: 0.00

[field #28]                 Priority: 3 - Low

[field #29]          Should Start On: Sun 03 Oct 2021 12:00:00 AM UTC

[field #30]    Should be Finished on: Sun 03 Oct 2021 12:00:00 AM UTC

[field #31]         Percent Complete: 0%

[field #33]                  Release: None

[field #58]     Custom Select Box #1: None

[field #59]     Custom Select Box #2: None

[field #60]     Custom Select Box #3: None

[field #61]     Custom Select Box #4: None

[field #62]     Custom Select Box #5: None

[field #63]     Custom Select Box #6: None

[field #64]     Custom Select Box #7: None

[field #65]     Custom Select Box #8: None

[field #66]     Custom Select Box #9: None

[field #67]    Custom Select Box #10: None


Boud Roukema <boud>
Group Member
Sun 03 Oct 2021 08:22:43 PM UTC, original submission:  

BUG: Maneage commit 775fc03 has fatal errors compiling some python modules. A second round of './project configure' leads to the Maneage system considering all packages to be compiled correctly, despite the errors declared as fatal. It is not known if the fatal errors cause problems when running python.


SYSTEM DETAILS:

  • Debian GNU/Linux bullseye 11.0, i.e stable, x86_64 architecture
  • Maneage commit 775fc03



LOG:
https://cosmo.torun.pl/~boud/log.20211003.1.775fc03.python.bug.bz2


EXTRACTS from the log:

This is a parallel compile, so the specific commands might be interleaved. [Using the '--debug' option doesn't seem to work fully in parallel, giving some lines printed in an interleaved way - probably some output is buffered and not rapidly synced.]

There are a few sections such as the following:


/BUILD/software/installed/bin/gcc -pthread -fPIC -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -std=c99 -Wextra -Wno-unused-result -Wno-unused-parameter -Wno-missing-field-initializers -Werror=implicit-function-declaration -I./Include/internal -I./Include -I. -I/BUILD/software/installed/include -I/usr/local/include -I/BUILD/software/build-tmp/Python-3.8.5/Include -I/BUILD/software/build-tmp/Python-3.8.5 -c /BUILD/software/build-tmp/Python-3.8.5/Modules/_blake2/blake2module.c -o build/temp.linux-x86_64-3.8/BUILD/software/build-tmp/Python-3.8.5/Modules/_blake2/blake2module.o
/BUILD/software/installed/bin/gcc -pthread -fPIC -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -std=c99 -Wextra -Wno-unused-result -Wno-unused-parameter -Wno-missing-field-initializers -Werror=implicit-function-declaration -I./Include/internal -I./Include -I. -I/BUILD/software/installed/include -I/usr/local/include -I/BUILD/software/build-tmp/Python-3.8.5/Include -I/BUILD/software/build-tmp/Python-3.8.5 -c /BUILD/software/build-tmp/Python-3.8.5/Modules/grpmodule.c -o build/temp.linux-x86_64-3.8/BUILD/software/build-tmp/Python-3.8.5/Modules/grpmodule.o
In file included from /BUILD/software/build-tmp/Python-3.8.5/Modules/_curses_panel.c:15:
./Include/py_curses.h:36:10: fatal error: curses.h: No such file or directory
   36 | #include <curses.h>
      |          ^~~~~~~~~~
compilation terminated.
In file included from /BUILD/software/build-tmp/Python-3.8.5/Modules/_cursesmodule.c:113:
./Include/py_curses.h:36:10: fatal error: curses.h: No such file or directory
   36 | #include <curses.h>
      |          ^~~~~~~~~~
compilation terminated.
building '_elementtree' extension


The python management system itself seems to consider the build successful:


Python build finished successfully!
The necessary bits to build these optional modules were not found:
_dbm                  _gdbm                 _sqlite3
_tkinter
To find the necessary bits, look in setup.py in detect_modules() for the module's name.


The following modules found by detect_modules() in setup.py, have been
built by the Makefile instead, as configured by the Setup files:
_abc                  atexit                pwd
time


Failed to build these modules:
_curses               _curses_panel


Following modules built successfully but were removed because they could not be imported:
_uuid                 nis

running build_scripts
creating build/scripts-3.8


Nevertheless, the first '.project configure' run fails. A second iteration of './project configure' runs without further interruption.

PRESENCE of curses.h:

All these are present:


.local/include/ncurses/cursesapp.h
.local/include/ncurses/cursesf.h
.local/include/ncurses/curses.h
.local/include/ncurses/cursesm.h
.local/include/ncurses/cursesp.h
.local/include/ncurses/cursesw.h
.local/include/ncurses/ncurses_dll.h
.local/include/ncurses/ncurses.h -> curses.h
.local/include/ncursesw/cursesapp.h
.local/include/ncursesw/cursesf.h
.local/include/ncursesw/curses.h
.local/include/ncursesw/cursesm.h
.local/include/ncursesw/cursesp.h
.local/include/ncursesw/cursesw.h
.local/include/ncursesw/ncurses_dll.h
.local/include/ncursesw/ncurses.h -> curses.h
.local/include/python3.8/py_curses.h


and the compilation path appears to be valid for finding 'curses.h', so the bug is puzzling.


STATUS as a bug:

So far I'm not aware of this having consequences in running python scripts. However, since we clearly have 'curses' installed as part of the basic.mk system, in principle it should be easy to solve the bug of not finding 'curses.h'.

Moreover, the build should compile in a single run, without needing a second configure round.

However, I've rated this as "3 = low" on the priority scale, since practical problems following from the bug are not (yet) known.

I see that the commented URL for python in 'reproduce/software/conf/urls.conf' is http://akhlaghi.org/src . This leads to a related issue: for URLs that have akhlaghi.org URLs alone, it would be better to include an example of the upstream URL as a comment just before/after, and any differences should be described. Even if the upstream URL risks being volatile and disappearing, in many cases these will survive in the long term so making the user's task easier would be better.

Boud Roukema <boud>
Group Member

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by makhlaghi (Posted a comment)
  • -email is unavailable- added by boud (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

     

    Follow 2 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2022-09-21 boud StatusNone Works For Me
    2021-10-03 boud Reassign itemManeage, task #16051 Maneage, bug #61271

    Back to the top

    Powered by Savane 3.13-4448.
    Corresponding source code