bugManeage - Bugs: bug #66223, old binutils/new glibc...

 
 

You are not allowed to post comments on this tracker with your current authentication level.

bug #66223: old binutils/new glibc incompatibility makes Maneage fail after two years

Submitter:  Boud Roukema <boud>
Submitted:  Mon 16 Sep 2024 04:55:11 PM UTC
   
 
Category:  Software Severity:  3 - Normal
Item Group:  Crash Status:  None
Privacy:  Public Assigned to:  None
Open/Closed:  Open

Tue 17 Sep 2024 01:42:01 PM UTC, comment #3: 

comment #2:

> comment #1:

 

> I only tried updating binutils, i.e. "Updating to commit 79ee52e bypasses this specific error, since binutils 2.39 is modern enough for the current Debian bookworm." Commit 79ee52e is exactly one (big) commit in official Maneage more recent than af723d9 .

 
Actually, 79ee52e makes many other updates, not just binutils. It updates gcc from 11.2.1 to 12.1.0 , increasing the amount of RAM required in slurm from 10Gb to about 120Gb (keep in mind that this especially for the ramdisk) - see https://savannah.nongnu.org/bugs/?62879 - i.e. changing from

sbatch <OPTIONS> --mem=10000M <scriptpath>

to

sbatch <OPTIONS> --mem=120000M <scriptpath>




Boud Roukema <boud>
Group Member
Tue 17 Sep 2024 01:15:04 PM UTC, comment #2: 

comment #1:

> For this particular case, did you try updating the Maneage version of Binutils and/or GCC to their latest ones?


I only tried updating binutils, i.e. "Updating to commit 79ee52e bypasses this specific error, since binutils 2.39 is modern enough for the current Debian bookworm." Commit 79ee52e is exactly one (big) commit in official Maneage more recent than af723d9 .

> But the good news is that it paid off, and we recently hired a research software engineer here at CEFCA dedicated to Gnuastro and Maneage. With his help, we will hopefully get the GNU C library built automatically within Maneage (from December this year his current Gnuastro work will finish and he'll be able to focus on Maneage).


That's good news - looking forward to it. :)

Boud Roukema <boud>
Group Member
Tue 17 Sep 2024 10:46:02 AM UTC, comment #1: 

Thanks for reporting this bug and the nice analysis Baud :-).

As Maneage ages/matures we are confronting these very nice bugs to help improve it. The spirit of this bug (the Maneage version being older than than the host) is very similar to the cause of bug #65677.

For this particular case, did you try updating the Maneage version of Binutils and/or GCC to their latest ones?

Indeed, in the last two years, Maneage's development has suffered because of work load. But the good news is that it paid off, and we recently hired a research software engineer here at CEFCA dedicated to Gnuastro and Maneage. With his help, we will hopefully get the GNU C library built automatically within Maneage (from December this year his current Gnuastro work will finish and he'll be able to focus on Maneage). After the C library, we also need to build LaTeX (in task #15267: which also causes a lot of headaches because currently just let TeXLive manager download its own binaries).

Mohammad Akhlaghi <makhlaghi>
Group administrator
Mon 16 Sep 2024 04:55:11 PM UTC, original submission:  

BUG: On a slightly old (July 2022) Maneage version, in particular maneage commit
af723d9 [1], the gcc-11.2.0 source code compilation step on a host OS running the
current (Sep 2024) Debian/stable (Debian/bookworm) gives a fatal error due to ld in the old
Maneage binutils being incompatible with the host GNU libc (which is too new).

CONTEXT:
host: Debian GNU/Linux 12.7 (bookworm), Linux 6.1.0-23-amd64

gcc                                    12.2.0-3
libc-dev-bin                           2.36-9+deb12u8
binutils                               2.40-2


Maneage af723d9:

$ grep -E "binutils|gcc" reproduce/software/config/versions.conf
binutils-version = 2.37
gcc-version = 11.2.0


REPRODUCE THE BUG:

./project configure

gives a fatal error at the gcc step because 'relr.dyn' in /lib/x86_64-linux-gnu/libc.so.6 is not understood
by ld in the version of binutils chosen by Maneage:

LOG:

configure:4465: checking whether the C compiler works
configure:4487: gcc  -I/BUILD/software/installed/include  -Wno-nullability-completeness -Wl,-rpath-link=/BUILD/software/installed/lib -L/BUILD/software/installed/lib  conftest.c  >&5
/BUILD/software/installed/bin/ld: /lib/x86_64-linux-gnu/libc.so.6: unknown type [0x13] section `.relr.dyn'
/BUILD/software/installed/bin/ld: skipping incompatible /lib/x86_64-linux-gnu/libc.so.6 when searching for /lib/x86_64-linux-gnu/libc.so.6
/BUILD/software/installed/bin/ld: cannot find /lib/x86_64-linux-gnu/libc.so.6
/BUILD/software/installed/bin/ld: /lib/x86_64-linux-gnu/libc.so.6: unknown type [0x13] section `.relr.dyn'
/BUILD/software/installed/bin/ld: skipping incompatible /lib/x86_64-linux-gnu/libc.so.6 when searching for /lib/x86_64-linux-gnu/libc.so.6
/BUILD/software/installed/bin/ld: /lib64/ld-linux-x86-64.so.2: unknown type [0x13] section `.relr.dyn'
/BUILD/software/installed/bin/ld: skipping incompatible /lib64/ld-linux-x86-64.so.2 when searching for /lib64/ld-linux-x86-64.so.2
/BUILD/software/installed/bin/ld: cannot find /lib64/ld-linux-x86-64.so.2
/BUILD/software/installed/bin/ld: /lib64/ld-linux-x86-64.so.2: unknown type [0x13] section `.relr.dyn'
/BUILD/software/installed/bin/ld: skipping incompatible /lib64/ld-linux-x86-64.so.2 when searching for /lib64/ld-linux-x86-64.so.2
collect2: error: ld returned 1 exit status
configure:4491: $? = 1
configure:4529: result: no
configure: failed program was:
| /* confdefs.h */
| #define PACKAGE_NAME ""
| #define PACKAGE_TARNAME ""
| #define PACKAGE_VERSION ""
| #define PACKAGE_STRING ""
| #define PACKAGE_BUGREPORT ""
| #define PACKAGE_URL ""
| /* end confdefs.h.  */
|
| int
| main ()
| {
|
|   ;
|   return 0;
| }
configure:4534: error: in `/BUILD/software/build-tmp/gcc-11.2.0/build':
configure:4536: error: C compiler cannot create executables


DISCUSSION:
The official peer-reviewed version-of-record commits of
Maneage papers risk having gcc being uncompilable on systems where the
binutils provided by Maneage is too old to be compatible with the OS GNU libc.
To reproduce an "old" peer-reviewed paper, unless the authors have provided
a live git (or other) update to the package, the version of record will
require shifting to the host gcc once the version-of-record binutils
is too old.

The cleanest solution to this would be (I guess) the long-standing task 15390 of
building the GNU C library within Maneage [2]. That would significantly
improve long-term reproducibility. It seems that this was last updated
at four years ago.

Updating to commit 79ee52e bypasses this specific error, since binutils 2.39
is modern enough for the current Debian bookworm. However, that doesn't solve the
general future problem for Maneage binutils on version 2.N being run on
Debian stable with binutils 2.(N+m), where m is the number of minor
versions needed for incompatibility. If we want 10 or 20 years of
research paper reproducible longevity, then having tools from binutils version 2.N e.g. from 2024,
working correctly on Debian in 2044, with e.g. binutils.2.(N+m) or 4.P,
seems unlikely.

POSSIBLE HACK:
While waiting for task 15390 to be done, a possible hack would be to do a 'configure' type
test within the binutils build script in basic.mk, after it has been built.
Try to compile a C program; if it fails, then assume (or check) that this failure is
because of the 'new' binutils . If the likely interpretation is that this
is a binutils problem, then do something to remove or disable the
binutils binaries that have just been created and installed, so that
the host binutils binaries (presumably more modern) are used instead.

Tricky bit: the question remains what to do with $(ibidir)/binutils-$(binutils-version) .
If it is created despite the binaries having been removed, then gcc will
likely compile OK using the host binutils , but the information claiming
that $(ibidir)/binutils-$(binutils-version) has been installed will be false,
which will mislead the user and is a high cost to pay for the hack.

If the false $(ibidir)/binutils-$(binutils-version) is not created, then
the gcc build rule would need to be hacked.

Possible solution to the tricky bit: make the gcc build rule depend on a
special target such as ${ibidir}/binutils-maneage-or-host , which will be
created no matter if the Maneage or host binutils is to be used. If the
Maneage binutils is acceptable, then
$(ibidir)/binutils-$(binutils-version) is also created. Problems:
${ibidir}/binutils-maneage-or-host may mess up the script for listing
which packages are installed, and require that script to be hacked too.


[1] http://git.maneage.org/project.git/commit/?id=af723d9c5b36b7b5580d1a00ca9fbe8778237e7c
[2] http://savannah.nongnu.org/task/index.php?15390
[3] https://gitlab.com/maneage/project-dev/-/tree/glibc?ref_type=heads

Boud Roukema <boud>
Group Member

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

No files currently attached

 

Digest:
   task dependencies.

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by makhlaghi (Posted a comment)
  • -email is unavailable- added by boud (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

     

    Follows 1 latest change.

    Date Changed by Updated Field Previous Value => Replaced by
    2024-09-16 boud Dependencies- Depends on task #15390

    Back to the top

    Powered by Savane 3.14-f13d.
    Corresponding source code