taskReproducible paper template - Tasks: task #15739, Wishlist - Debian patched stable...

 
 

You are not allowed to post comments on this tracker with your current authentication level.

task #15739: Wishlist - Debian patched stable or testing or unstable set of sources

Submitted by:  Boud Roukema <boud>
Submitted on:  Sun 02 Aug 2020 12:34:23 PM UTC  
 
Should Start On:  Sun 02 Aug 2020 12:00:00 AM UTC Should be Finished on:  Sun 02 Aug 2020 12:00:00 AM UTC
Category:  None Priority:  5 - Normal
Status:  None Privacy:  Public
Percent Complete:  0% Assigned to:  None
Open/Closed:  Open Effort:  0.00

Sun 02 Aug 2020 06:45:57 PM UTC, comment #3: 

comment #1:

> By "download" software from Debian, you mean the source tarballs, right? In particular, the "pristine-tar" branch of each package (like this one for Gnuastro. You don't mean downloading the binary packaged files, right?


I mean the source tarballs (pristine-tar) plus (at least) the debian patches, e.g. https://salsa.debian.org/science-team/gnuplot/-/tree/master/debian/patches or https://salsa.debian.org/debian-astro-team/inhomog/-/tree/master/debian/patches or https://salsa.debian.org/debian-astro-team/cfitsio/-/tree/master/debian/patches .

It's the patches that represent a major part of the Debian quality assurance pipeline in terms of fixing source code.

I definitely don't mean downloading the binaries - the Debian build system is only run on 22 different systems currently (9 official, 13 unofficial).

There may be other steps in the build process - checking parameters in debian/control , for example, that we might also want to implement. The debian/rules file often gives security hardening steps for compilation. The question for each step would be: is the advantage in reliability/security worth the cost in needing more tools?

Applying the patches - in the recommended order!!! - would already, it seems to me, implement a significant part of the benefit of Debian.

> If so, then your suggestion would translate to an interesting suggestion: only updating versions of software that are in certain stages of the Debian release (stable, testing or unstable). We can even write a small script to parse the unified Debian interface and get the list of packages that need updating.


Yes, that's definitely a good idea!

> Running this script can even be provided at configure time, so the person running Maneage can say that for example, I want all my software versions (with build instructions in Maneage!) to match the current Debian stable. That script would then update the project's 'versions.conf' and 'checksums.conf' source files and build those versions. Later, the user can commit the changes made in these files to be able to reproduce their project's software environment independent of future updates to the Debian stable, testing or unstable branches.


Would we want to add "our" archival copies of e.g. the debian stable packages in Zenodo/maneage? Or how long-term are the debian archives of orig tarballs and patch lists?

I guess by "commit the changes", you mean to archive them in Zenodo/maneage? My guess is that's probably reasonable. We could either choose to archive the patched files as a "patched archive" - for simplicity - or in the original debian format - for easier checking against the official debian versions.

> This would also greatly simplify the task of us Maneage maintainers (every month, we would just have to check with Debian testing and update the packages to versions that are now used there! Instead of manually going and changing all the versions like what I did the other day!


In terms of software security and reliability, this would seem a big improvement to me. I know you have good software skills :), but it's hard to believe that you really thoroughly checked all the updates to 20-30 or however many packages you updated.

Boud Roukema <boud>
Project Member
Sun 02 Aug 2020 03:20:34 PM UTC, comment #2: 

However, for the users, we can't guarantee that build instructions within Maneage work with the different software versions. It will be their own responsibility to fix any build failure (for example adding/removing a configuration option).

Mohammad Akhlaghi <makhlaghi>
Project Administrator
Sun 02 Aug 2020 03:15:55 PM UTC, comment #1: 

Thanks for the interesting suggestion Boud.

By "download" software from Debian, you mean the source tarballs, right? In particular, the "pristine-tar" branch of each package (like this one for Gnuastro. You don't mean downloading the binary packaged files, right?

If so, then your suggestion would translate to an interesting suggestion: only updating versions of software that are in certain stages of the Debian release (stable, testing or unstable). We can even write a small script to parse the unified Debian interface and get the list of packages that need updating.

Running this script can even be provided at configure time, so the person running Maneage can say that for example, I want all my software versions (with build instructions in Maneage!) to match the current Debian stable. That script would then update the project's 'versions.conf' and 'checksums.conf' source files and build those versions. Later, the user can commit the changes made in these files to be able to reproduce their project's software environment independent of future updates to the Debian stable, testing or unstable branches.

This would also greatly simplify the task of us Maneage maintaiers (every month, we would just have to check with Debian testing and update the packages to versions that are now used there! Instead of manually going and changing all the versions like what I did the other day!

Mohammad Akhlaghi <makhlaghi>
Project Administrator
Sun 02 Aug 2020 12:34:23 PM UTC, original submission:  

Following up from https://savannah.nongnu.org/task/index.php?15698, a reasonably straightforward, and reasonably automatable, way of significantly increasing the reliability and quality assurance of software in Maneage would be to provide a Debian option. This option would download as much as possible of the software from Debian, with an option to choose either stable (the best checked, including security checks), testing (moderately checked), or unstable (checked by some people with Debian upload rights).

This is likely apply to all of the packages in the basic part of the system: if a basic package is not yet available in Debian, then it's probably not basic. A lot of python tool packages, like astropy, are in Debian. Science research level packages will mostly not be in Debian, partly because of the "direct action" atmosphere in astronomy software in which "you can use it, don't bother me with wasting time about licences!" replaces free-software licensing.

A pre-configure (host system!) optional step could be provided that converts the Debian URLs/configure/compile instructions (the install will be different to Debian, because Maneage does not install in /usr or other root directories) into an html page with clickable urls for the user to use a web browser for a sanity check of the https://tracker.debian.org/<package> pages, such as https://tracker.debian.org/gnuplot or https://tracker.debian.org/inhomog .

This host system pre-configure step could also list which packages are not from Debian.

The Debian option would be able to operate reasonably early in the tool chain, but it would need (at least) patch .

Remember that Debian is one of the major root GNU/(Linux|Kfreebsd|Hurd) distributions - what goes into Debian generally feeds through to a huge number of derivative distributions, including Ubuntu.

Boud Roukema <boud>
Project Member

 

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by makhlaghi (Posted a comment)
  • -email is unavailable- added by boud (Submitted the item)
  •  

    Do you think this task is very important?
    If so, you can add your encouragement to it.
    This task has 0 encouragements so far.

    Only logged-in users can vote.

     

     

     

    No changes have been made to this item

    Back to the top


    Powered by Savane 3.5