GCC 12.1.0 was fully built and attached I am sending the output of the command below (file #53560), with a slight modification (I removed the '-h' from 'free' so we have better resolution):

rm -f freecheck.txt && while true; do free | awk '/^Mem:/' >> freecheck.txt; sleep 1; done

The resulting plot is shown in the P.S. Note that GCC itself was uncompressed in '/dev/shm' (the ram), and that this is the total RAM usage. The slight increase at the base of the end of the plot is because it had started building other programs.

So probably keeping the previous 7GB (or maybe 8GB) size for available RAM should be enough.

P.S.

Thanks a lot Raul and Boud,

Just now I finished a build on a system that still has a Debian from 2010! And only three basic software needed to be degraded (because they need more recent C++ standards): PatchELF, Binutils and GCC! Everything else built sucessfully :-).

After Maneage was fully installed, I was able to jump 12 years ahead (in terms of the software environment) by simply running './project shell' :-).

Based on this experience, I have added a small section in README.md about building on older systems

I also found some bugs that we hadn't noticed until, and they are now in Commit e6535484  of the 'update-software' branch.

Raul, as you see in the commit above, I added those software to 'TARGETS.conf' for now, so they are tested automatically in our final tests before the merge, thanks for the good suggestion ;-). I should say that they all built smoothly on the 12 year old system also ;-).

Boud, I agree with your point about having a unified script to check the RAM! It is indeed a great idea! Can you write a first version?

Thanks a lot Mohammad.

In addition to the high-level programs that are ready to be tested as part of that commit (bc49bfc), I also tested: 'emacs', 'imfit', 'missfits' and 'galsim'.

The 'TARGETS.conf' file is then:

top-level-programs = astrometrynet emacs gnuastro imfit missfits scamp sextractor swarp vim

top-level-python = astropy galsim matplotlib

It would be great if you can also test them before the final merge.

I just re-built Maneage and this time, I commented the deletion of the 'gcc-12.1.0' directory (that GCC was built in). So that directory remained.

Afterwards, when GCC was fully built and installed, I checked the disk usage of that directory and got 7.4G:

$ du -sh gcc-12.1.0/
7.4G        gcc-12.1.0/

So I am a little confused how it has taken ~12G on your system?

Gcc 12.1.0 needs about 12 GiB (approx 12.9 GB) of RAM, so I've proposed commit 692c1aa on branch RAM12GiB [1] to increase the size of the RAM in the test in basic.mk and to clarify the discussion there.

Commit 692c1aa [1] works for me from scratch through to the verify.tex macros and the pdf, on debian and centOS7, provided that I manually set --mem=12000M or greater in sbatch on the centOS7 system. Bug #62879 [2] explains why being unable to check for consistency in RAM disk constraints automatically is risky, and suggests how this could be fixed.

[1] https://codeberg.org/boud/maneage_dev/commit/692c1aafceda84809cc76ae4de976c874422d535

[2] https://savannah.nongnu.org/bugs/index.php?62879

Thanks a lot for the reminder!

That commit has been rebased over the 'update-software' branch (as Commit fe4188c64) followed by Commit 6e5f33e760 with some cleanup on the implementation and some other fixes that I found after building in a minimal CentOS 7 container.

I also tested on a Debian-Slim container, and it also builds on that.

It is almost one year since the last general Maneage software update (that was done in the autumn of 2021; see task #16103).

To start this round of updates, I just updated almost all of the basic software with a version increase, and some of the default ones. The job is being done on the 'update-software' branch of the development repository:

https://gitlab.com/maneage/project-dev/-/tree/update-software

Currently there is only one commit (Commit 504e79d3cb), the commit message details all the version updates and is in the P.S. of this post. I will hopefully merge this branch into Maneage in one week (Saturday April 13th 2022). It would be great if you can test this branch on your systems and help fix any bugs that you may find. Also, in case there are software you would like to update, please feel free to add them as commits over this branch until the deadline ;-).

As mentioned in the first comment of task #16103, updating non-default Maneage software needs to be done by those who use/need them (in particular Python or R packages, which have a completely different, and evolving build system). I will be happy to test and ultimately merge them once they build on a sufficient number of systems. But due to time constraints, I can't update them myself.

P.S. Commit message of first set of software updates in this round.

    IMPORTANT: Updates to many default software

    New versions of many of the Maneage software had come since the last major
    update in Maneage (almost one year ago)! With this commit, the software in
    the P.S. have been updated, and a new software (PLplot) has been added.

    Besides the updates in the core software, the followimg improvements have
    also been implemented in this commit:

     - When you run './project shell':

       - A welcome message is printed that will remind the caller that they
         have entered a new environment, it will print the location of 'HOME'
         and the location of the shell startup file.

       - The 'reproduce/software/shell/bashrc.sh' is loaded as a startup
         file. This allows you to customize your interactive Maneage shell. A
         default step has already been placed there that will put the git
         branch name (in green) within the shell prompt (which was
         purple). This greatly helps when dealing with directories under Git
         version control. These settings won't bother with Maneage's default
         operations: through environment variables we make sure that these
         './project shell' features will not slow-down the calls to the shell
         within the non-interactive Make calls.

       - The host's 'COLORTERM' is passed to the Maneage environment. It is
         used by some programs that can have color outputs on the terminal.

     - The host's 'ldd' is linked inside the custom environment. This belongs
       to the GNU C Library (which is not yet installed in Maneage). But helps
       in checking the linking status of the binaries.

     - The Zip and Unzip programs have been moved as high-level software that
       have to be manually requested when necessary. This is because they are
       not used by any of the basic software anymore. They were just installed
       as dependencies of GNU Tar to be close the other compression
       programs. Also, in the past we would use the original tarballs, and some
       (for example Numpy) were distributed in Zip format. However, by default,
       we now use a custom Lzip tarball and don't need Zip or Unzip. This was
       suggested by Zahra Sharbaf and Raul Infante-Sainz.

     - Some minor edits in 'reproduce/software/shell/tarball-prepare.sh'. In
       particular the 'awk' command was effectively just replacing a '_' with
       '-', so it just uses a simple SED expression instead.

    P.S. List of software:

    Basic software:
       cURL                                7.79.1    -->  7.84.0
       Dash                                0.5.11.5  -->  0.5.11-057cd65
       File                                5.41      -->  5.42
       Git                                 2.36.0    -->  2.37.1
       GNU AWK                             5.1.0     -->  5.1.1
       GNU Bash                            5.1.8     -->  5.2-rc2
       GNU Gzip                            1.11      -->  1.12
       GNU Nano                            6.0       -->  6.4
       GNU Libtool                         2.4.6     -->  2.4.7
       GNU libiconv                        0.16      -->  0.17
       GNU Readline                        8.1.1     -->  8.2-rc2
       GNU Binutils                        2.37      -->  2.39
       GNU Findutils                       4.8.0     -->  4.9.0
       GNU Integer Set Library (ISL)       0.18      -->  0.24
       GNU Compiler Collection (GCC)       11.2.1    -->  12.1.0
       PatchELF                            0.13      -->  0.15.0
       Perl                                5.34.0    -->  5.36.0

    High-level software:
       Libjpeg                             9d        -->  9e
       Libtiff                             4.3.0     -->  4.4.0
       CFITSIO                             4.0.0     -->  4.1.0
       Ghostscript                         9.55.0    -->  9.56.1
       GNU Astronomy Utilities (Gnuastro)  0.16.1    -->  0.18
       PLplot                              n/a       -->  5.15.0
       SCAMP                               2.6.7     -->  2.10.0
       SWarp                               2.38.0    -->  2.41.5
       WCSLIB                              7.7       -->  7.11