Further discussion at #python on Libera Chat led to the general feeling that python developers have multiple ideas on how to improve python packaging. After several discussions and some reading, my impression was that E Schwartz (ztrawhcse)'s recommendations for 'bdist_wheel' and 'installer' made the best sense for Maneage. The strategy based on bdist_wheel is summarised in 'python.mk' in commit d5775fa [2] so that people who go directly to python.mk understand the (current) strategy.

Commit d5775fa [2] solves bug #63054 (so that the crude hack of manually adding a specific .egg file to the python path is no longer needed) [3].

Here is the proposed new strategy:

# 2022-10-02 Build strategy for python modules as of October 2022, for
# python 3.10.6. Python build strategies are continuing to evolve. The
# dependency strategy used for Maneage, based on recommendations by
# E Schwartz (ztrawhcse) on #python (Libera Chat) to use
# python's 'installer' module [1], is as follows:
#
# (1) python-installer rule:  depends only on python
# (2) setuptools:  depends only on python
# (3) wheel: depends only on setuptools
# (4) 'medium' level modules:
#     any module that previously only depended on setuptools:
#     now depends on python-installer and wheel (indirectly on
#     setuptools via wheel);
# (5) 'high' level modules: depend either
#       (5a) on at least one 'medium' level module that indirectly
#            depends on (1)+(3), and thus also on (2)
#       (5b) directly depends on (1)+(2)+(3), so listed dependencies
#            are (1) python-installer + (3) wheel, and indirect is
#            (2) setuptools
#
# Related changes:
# * The module setuptools_scm is a 'medium' level module.
#
# * (1) + (2) + (3) are installed with the old method,
#   without python-installer, so
#   their build rule has the fourth (opt-in) parameter set to
#   NOINSTALLER.
#
# * (4) + (5) medium and high level modules are installed
#   using 'python setup.py bdist_wheel;
#          python -m installer dist/*.whl'
#
# [1] https://pypi.org/project/installer

If people think that this strategy is valid, or can improve it, and if it looks OK in further testing (it worked fine for me) then this task should be closable as 'done'.

The python ecosystem is going to continue developing - apparently meson is starting to be used for python builds. So in 2023 or 2024 or 2025 or so it's quite possible that this bdist_wheel + installer strategy will become obsolete. I'm just guessing - I'm not a python developer!

There are some Python Enhancement Proposals (PEPs) related to the issues of building and installing, though I didn't keep track of them. These seem to be something like RfCs for python. Most of the discussion doesn't seem to show if our reproducibility needs - minimal complexity, separate downloading and compiling/installing - are satisfied by the various 'recommended' methods.

[2] https://codeberg.org/boud/maneage_dev/commit/d5775faef8fb1d44c989c6d97be82e2536724d13

[3] https://savannah.nongnu.org/bugs/index.php?63054

Do you mean that with the option '--single-version-externally-managed' bug #63054 would be fixed? I could check it.