taskManeage - Tasks: task #16052, Only Lzip in basic, other...


You are not allowed to post comments on this tracker with your current authentication level.

task #16052: Only Lzip in basic, other compression software in high-level

Submitter:  Mohammad Akhlaghi <makhlaghi>
Submitted:  Mon 04 Oct 2021 12:51:37 PM UTC
Should Start On:  Sun 03 Oct 2021 11:00:00 PM UTC Should be Finished on:  Sun 03 Oct 2021 11:00:00 PM UTC
Category:  Software Priority:  * 5 - Normal
Status:  Cancelled Privacy:  Public
Percent Complete:  100% Assigned to:  makhlaghi
Open/Closed:  Closed Effort:  0.00
* Mandatory Fields

Add a New Comment (Rich Markup)

Comment Type & Canned Response:
       No canned response available


Sat 01 Jan 2022 05:04:01 PM UTC, comment #4: 

Thanks to all the points raised, we should keep the the other compression programs in the same way there are now in 'basic.mk', so I am closing this task as "Cancelled".

Mohammad Akhlaghi <makhlaghi>
Project Administrator
Sun 10 Oct 2021 10:04:23 PM UTC, comment #3: 

It seems that the gcc upstream source is by default xzipped, so that would argue for keeping 'xz' in 'basic.mk' too. However, I haven't checked if gnu.org also provides gzipped tar packages. If all basic packages are available with gzip, then gzip alone would be enough.

> (2) You can simply un-comment the software's '*-url" variable in 'reproduce/software/config/urls.conf'.

I agree that that is the case currently . However, if we remove too many compression programs from basic, then this won't work. For example, currently doing this gets the xzipped source of gcc. This would fail (or require system-level xzip) if xz were absent from basic.mk.

> (3) We don't want to imitate Debian

Probably 'imitate' is the wrong verb. :) What I meant is that with a much tinier team and infrastructure, we risk de facto having to repeat much of the work of a much bigger team, due to using upstream source that is fixed for many old bugs, but also has new features that very likely contain new bugs. Debian stable provides source that aims at maximum stability - features are normally not added, and only security or other significant bugs are fixed. Anyway, it seems that you're happy to keep gzip. :)

Boud Roukema <boud>
Project Member
Tue 05 Oct 2021 11:08:46 AM UTC, comment #2: 

Another important point before thinking of doing this: we should also check which basic (and high-level) programs link with the libraries of these tools (except for zip/unzip and gzip, the rest also install libraries, and I have seen them getting used, although I can't remember the software exactly).

Thanks for raising the good Boud, here are my comments:

(1) I agree with keeping Gzip in 'basic.mk', those are valid points!

(2) You don't have to "wait until it's integrated officially into zenodo and akhlaghi.org"! You can simply un-comment the software's '*-url" variable in 'reproduce/software/config/urls.conf'. When a software's '*-url' variable is not empty, that URL will have a higher precedence than Zenodo, or all the other backup locations of Maneage ;-).

(3) We don't want to imitate Debian (although we learn a lot from it). The goals/methods are different between Maneage and Debian. Since we will keep Gzip anyway, it won't conflict with the proposed task #15739 (I'll try to follow that discussion in due time).

Mohammad Akhlaghi <makhlaghi>
Project Administrator
Mon 04 Oct 2021 06:44:33 PM UTC, comment #1: 

(1) One worry about removing gzip from the basic.mk system is that I have to had to hack into the basic.mk system quite a few times to analyse bugs and propose fixes. Being able to switch to a different upstream source package efficiently is useful. But having to gunzip the upstream archive with a host system gzip package and then lzip it to make maneage happy would add extra work to an already heavy task. This is just for debugging for my own install.

(2) If I make a proposed fix to basic.mk , I don't want to have to wait until it's integrated officially into zenodo and akhlaghi.org with an lzipped archive before I can invite others to test a draft project.

(3) Long-term stability and security of Maneage basic software: it seems pointless to me to try to imitate the large-scale teamwork, communication structure, software pipelines, and social infrastructure of Debian. While it's true we only have a few dozen basic packages, it's still a big long-term maintenance task to update these with the aim of long-term reproducibility, while also keeping an eye on security. Unless we convince Debian to provide the pristine-tar packages in .lz format so that they can be patched, we'll need to have gzip available if we want to keep task 15739 [1] as a serious long-term option.

So I'd rather keep gzip in basic.mk . I don't think that there are many upstream source repositories that do not provide a gzipped tar archive, so bzip2, zip+unzip, and xz seem lower priority to me - I don't see a strong argument for shifting these to high-level.mk .

[1] https://savannah.nongnu.org/task/?15739

Boud Roukema <boud>
Project Member
Mon 04 Oct 2021 12:51:37 PM UTC, original submission:  

As discussed in task #15699, in the spirit of proper long-term archival, we are re-creating creating the software tarballs that we archive ourselves (that are hosted on Zenodo, and our own Git repository.

By the end of task #15699, all the software tarballs used by the default Maneage branch will have a '.tar.lz' suffix.

Therefore, we don't need the other compression software in 'basic.mk' any more and we can move them to high-level.mk in case someone has some use for them in their high-level project.

This is just a brain-storm for now, but if there is no major reason against it until task #15699 is complete, we will do it ;-).

Mohammad Akhlaghi <makhlaghi>
Project Administrator


(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:

No files currently attached


Depends on the following items: None found

Items that depend on this one: None found


Carbon-Copy List
  • -email is unavailable- added by boud (Posted a comment)
  • -email is unavailable- added by makhlaghi (Submitted the item)

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only logged-in users can vote.





    Follow 4 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2022-01-01 makhlaghi StatusNeed Info Cancelled
        Percent Complete0% 100%
        Assigned toNone makhlaghi
        Open/ClosedOpen Closed

    Back to the top

    Powered by Savane 3.9