Commit 67daeee [0], which is one commit beyond core maneage 4318670, works for me to solve the 'pyyaml' compilation bug listed below here in bug #61811. The solution is to update from pyyaml-5.1 to pyyaml-6.0. (This seems to be independent from the other set of python bugs listed here.)

The cause of the problem is stated in the python documentation: Py_REFCNT changed in implementation to a function [1], so increasing its value with '++Py_REFCNT' is invalid, which is why the error message states 'lvalue required as increment operand'. An 'lvalue' is a 'left-hand value', i.e. that can be put in the left-hand part of an assignment operation, written '=' in C; the "value" of a function is (I guess) the address of the function in RAM, so augmenting the value of that address would be nonsense.

To see the problem for yourself, try compiling the standalone program

int Py_REFCNT(const double *o){
};

int main(void){
  double x;
  ++Py_REFCNT(&x);
}

with a C compiler. You should obtain the same error as when installing pyyaml-5.1 with python-3.10.6. Grep your log file if you didn't notice the bug before.

Commit 67daeee also adds 'setuptools_scm' dependencies for the python build rules for 'soupsieve' and 'webencodings', since otherwise these builds may fail (depending on the speed and parallel nature of compiles; these did fail for me); see also bug #63054 [2].

Astropy (for example) has to be enabled in TARGETS.conf for 67daeee to be tested.

[0] https://codeberg.org/boud/maneage_dev/commit/67daeee967fafd44e6f8199748f5cae4dd7ad5d6
 
[1] https://docs.python.org/3/c-api/structures.html#c.Py_REFCNT

[2] https://savannah.nongnu.org/bugs/?63054

DESCRIPTION: Running commit 4318670 of core maneage, with the addition of pyyaml as a target:

--- a/reproduce/software/config/TARGETS.conf
+++ b/reproduce/software/config/TARGETS.conf
@@ -39,7 +39,7 @@
 top-level-programs = gnuastro

 # Python libraries/modules.
-top-level-python =
+top-level-python = pyyaml

leads to compilation errors of the C source code of pyyaml. Independently, in a research project package, the compilation error does not appear to block a full configure of python packages, with matplotlib selected in TARGETS.conf. Thus, this is a bug in principle, though not (so far) an obvious problem in practice. Nevertheless, it should be fixed, and seems to be included in bug #61811 [0], which is why I'm adding it here.

LOG EXCERPT:

187806  /BUILD/software/installed/include/yaml.h:636:55: note: expected 'const yaml_char_t *' {aka '
const unsigned char *'} but argument is of type 'char *'
187807    636 |         const yaml_char_t *anchor, const yaml_char_t *tag, int implicit,
187808        |                                    ~~~~~~~~~~~~~~~~~~~^~~
187809  config.status: executing libtool commands
187810  ext/_yaml.c: In function '__pyx_tp_dealloc_5_yaml_CParser':
187811  ext/_yaml.c:23870:5: error: lvalue required as increment operand
187812  23870 |     ++Py_REFCNT(o);
187813        |     ^~
187814  ext/_yaml.c:23872:5: error: lvalue required as decrement operand
187815  23872 |     --Py_REFCNT(o);
187816        |     ^~
187817  ext/_yaml.c: In function '__pyx_tp_dealloc_5_yaml_CEmitter':
187818  ext/_yaml.c:24040:5: error: lvalue required as increment operand
187819  24040 |     ++Py_REFCNT(o);
187820        |     ^~
187821  ext/_yaml.c:24042:5: error: lvalue required as decrement operand
187822  24042 |     --Py_REFCNT(o);
187823        |     ^~
187824  ext/_yaml.c: In function '__Pyx_modinit_type_init_code':
187825  ext/_yaml.c:24844:25: error: 'PyTypeObject' {aka 'struct _typeobject'} has no member named 'tp_print'
187826  24844 |   __pyx_type_5_yaml_Mark.tp_print = 0;
187827        |                         ^
187828  ext/_yaml.c:24864:28: error: 'PyTypeObject' {aka 'struct _typeobject'} has no member named 'tp_print'
187829  24864 |   __pyx_type_5_yaml_CParser.tp_print = 0;
187830        |                            ^

HINTS:
It might be necessary to shift from our default python build rules to rules that use 'wheel', per the discussion at [1]. Bug [2] appears to be related, but it's not immediately clear what the bug is and whether there is a generic, rather than OS-specific, fix.

Some descriptions of 'wheel' are at [3][4].

A related guess is in [5][6] - it could be that the 'wheel' files with the 'sdist' option are not correctly built in the upstream 'pyyaml' (or 'jinja2' and so on) distributions.

Debugging this might require understanding the relation of cython to python, pyyaml and the other packages, and understanding the various updates and bugs in cython, and errors that people make when preparing python packages in relation to cython. Disclaimer: I'm just guessing here.


WHAT DID NOT WORK:

I tried

--- a/reproduce/software/make/python.mk
+++ b/reproduce/software/make/python.mk
@@ -581,7 +581,8 @@ $(ipydir)/pythran-$(pythran-version): \

 $(ipydir)/pyyaml-$(pyyaml-version): \
                  $(ibidir)/yaml-$(yaml-version) \
-                 $(ipydir)/cython-$(cython-version)
+                 $(ipydir)/cython-$(cython-version) \
+                 $(ipydir)/wheel-$(wheel-version)
        tarball=pyyaml-$(pyyaml-version).tar.gz
        $(call import-source, $(pyyaml-url), $(pyyaml-checksum))
        $(call pybuild, tar -xf, PyYAML-$(pyyaml-version), , \

but the same error occurred.

URLs:

[0] https://savannah.nongnu.org/bugs/?61811
[1] https://github.com/PyWavelets/pywt/issues/602
[2] https://github.com/yaml/pyyaml/issues/416
[3] https://realpython.com/python-wheels
[4] https://pythonwheels.com
[5] https://github.com/yaml/pyyaml/issues/179
[6] https://github.com/yaml/pyyaml/pull/188

Indeed, a non-reproducible bug is the most annoying (showing that we still haven't understood it!).

I tried repeating the 'pybuild' commands of 'packaging', 'jinja2' and 'extension-helpers' within their recipes to see if this problem can be fixed with a repeat. But I found that a simple repeated build doesn't fix the problem!

So its a dependency issue: in order for their presence to be recognized by the problematic packages (for example 'pyerfa' depends on 'packaging'), some other Python package needs to be present (which is present when the build finishes once!).

That software X has to become a dependency of the three problematic packages whose build is not recognized (be present before we build them).

Thanks a lot for the help Boud.

I need to remove the final target of the problematic package (jinja2, etc.) to be able to finish the configure step and install all programs. Boud, do you need to remove the final targets? or it is only necessary to re-run the configure step again?

I wonder if this is due to the same bug as https://savannah.nongnu.org/bugs/?61271 - Python build bug with curses.h in commit 775fc03 .