taskManeage - Tasks: task #16052, Only Lzip in basic, other...


You are not allowed to post comments on this tracker with your current authentication level.

task #16052: Only Lzip in basic, other compression software in high-level

Submitted by:  Mohammad Akhlaghi <makhlaghi>
Submitted on:  Mon 04 Oct 2021 12:51:37 PM UTC  
Should Start On:  Sun 03 Oct 2021 11:00:00 PM UTC Should be Finished on:  Sun 03 Oct 2021 11:00:00 PM UTC
Category:  Software Priority:  5 - Normal
Status:  Cancelled Privacy:  Public
Percent Complete:  100% Assigned to:  Mohammad Akhlaghi <makhlaghi>
Open/Closed:  Closed Effort:  0.00

Sat 01 Jan 2022 05:04:01 PM UTC, comment #4: 

Thanks to all the points raised, we should keep the the other compression programs in the same way there are now in 'basic.mk', so I am closing this task as "Cancelled".

Mohammad Akhlaghi <makhlaghi>
Project AdministratorIn charge of this item.
Sun 10 Oct 2021 10:04:23 PM UTC, comment #3: 

It seems that the gcc upstream source is by default xzipped, so that would argue for keeping 'xz' in 'basic.mk' too. However, I haven't checked if gnu.org also provides gzipped tar packages. If all basic packages are available with gzip, then gzip alone would be enough.

> (2) You can simply un-comment the software's '*-url" variable in 'reproduce/software/config/urls.conf'.

I agree that that is the case currently . However, if we remove too many compression programs from basic, then this won't work. For example, currently doing this gets the xzipped source of gcc. This would fail (or require system-level xzip) if xz were absent from basic.mk.

> (3) We don't want to imitate Debian

Probably 'imitate' is the wrong verb. :) What I meant is that with a much tinier team and infrastructure, we risk de facto having to repeat much of the work of a much bigger team, due to using upstream source that is fixed for many old bugs, but also has new features that very likely contain new bugs. Debian stable provides source that aims at maximum stability - features are normally not added, and only security or other significant bugs are fixed. Anyway, it seems that you're happy to keep gzip. :)

Boud Roukema <boud>
Project Member
Tue 05 Oct 2021 11:08:46 AM UTC, comment #2: 

Another important point before thinking of doing this: we should also check which basic (and high-level) programs link with the libraries of these tools (except for zip/unzip and gzip, the rest also install libraries, and I have seen them getting used, although I can't remember the software exactly).

Thanks for raising the good Boud, here are my comments:

(1) I agree with keeping Gzip in 'basic.mk', those are valid points!

(2) You don't have to "wait until it's integrated officially into zenodo and akhlaghi.org"! You can simply un-comment the software's '*-url" variable in 'reproduce/software/config/urls.conf'. When a software's '*-url' variable is not empty, that URL will have a higher precedence than Zenodo, or all the other backup locations of Maneage ;-).

(3) We don't want to imitate Debian (although we learn a lot from it). The goals/methods are different between Maneage and Debian. Since we will keep Gzip anyway, it won't conflict with the proposed task #15739 (I'll try to follow that discussion in due time).

Mohammad Akhlaghi <makhlaghi>
Project AdministratorIn charge of this item.
Mon 04 Oct 2021 06:44:33 PM UTC, comment #1: 

(1) One worry about removing gzip from the basic.mk system is that I have to had to hack into the basic.mk system quite a few times to analyse bugs and propose fixes. Being able to switch to a different upstream source package efficiently is useful. But having to gunzip the upstream archive with a host system gzip package and then lzip it to make maneage happy would add extra work to an already heavy task. This is just for debugging for my own install.

(2) If I make a proposed fix to basic.mk , I don't want to have to wait until it's integrated officially into zenodo and akhlaghi.org with an lzipped archive before I can invite others to test a draft project.

(3) Long-term stability and security of Maneage basic software: it seems pointless to me to try to imitate the large-scale teamwork, communication structure, software pipelines, and social infrastructure of Debian. While it's true we only have a few dozen basic packages, it's still a big long-term maintenance task to update these with the aim of long-term reproducibility, while also keeping an eye on security. Unless we convince Debian to provide the pristine-tar packages in .lz format so that they can be patched, we'll need to have gzip available if we want to keep task 15739 [1] as a serious long-term option.

So I'd rather keep gzip in basic.mk . I don't think that there are many upstream source repositories that do not provide a gzipped tar archive, so bzip2, zip+unzip, and xz seem lower priority to me - I don't see a strong argument for shifting these to high-level.mk .

[1] https://savannah.nongnu.org/task/?15739

Boud Roukema <boud>
Project Member
Mon 04 Oct 2021 12:51:37 PM UTC, original submission:  

As discussed in task #15699, in the spirit of proper long-term archival, we are re-creating creating the software tarballs that we archive ourselves (that are hosted on Zenodo, and our own Git repository.

By the end of task #15699, all the software tarballs used by the default Maneage branch will have a '.tar.lz' suffix.

Therefore, we don't need the other compression software in 'basic.mk' any more and we can move them to high-level.mk in case someone has some use for them in their high-level project.

This is just a brain-storm for now, but if there is no major reason against it until task #15699 is complete, we will do it ;-).

Mohammad Akhlaghi <makhlaghi>
Project AdministratorIn charge of this item.


No files currently attached


Depends on the following items: None found

Items that depend on this one: None found


Carbon-Copy List
  • -email is unavailable- added by boud (Posted a comment)
  • -email is unavailable- added by makhlaghi (Submitted the item)

    Do you think this task is very important?
    If so, you can add your encouragement to it.
    This task has 0 encouragements so far.

    Only logged-in users can vote.




    Follow 4 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2022-01-01 makhlaghi StatusNeed Info => Cancelled
        Percent Complete0% => 100%
        Assigned toNone => makhlaghi
        Open/ClosedOpen => Closed

    Back to the top

    Powered by Savane 3.7