taskReproducible paper template - Tasks: task #15286, Template's package manger

 
 

You are not allowed to post comments on this tracker with your current authentication level.

task #15286: Template's package manger

Submitted by:  Mohammad Akhlaghi <makhlaghi>
Submitted on:  Mon 27 May 2019 12:13:20 PM UTC  
 
Should Start On:  Sun 26 May 2019 11:00:00 PM UTC Should be Finished on:  Sun 26 May 2019 11:00:00 PM UTC
Category:  Software Priority:  5 - Normal
Status:  Need Info Privacy:  Public
Percent Complete:  0% Assigned to:  None
Open/Closed:  Open Effort:  0.00

Mon 29 Jul 2019 02:36:44 AM UTC, comment #1: 

I recently learned about GNU Stow which can be one way to implement a Nix/Guix-like feature within our existing software building process.

GNU Staw has a very interesting suggestion: Assume you have a "store" directory. Build every software (with its unique configuration) in a unique top-level installation directory (for example `store/b6gvzjyb2pg0kjfwrjmg1vfhh54ad73z-firefox-33.1/'). This installation directory will then have all the basic directories like `bin', `lib', `man' and etc. But only filled with the installed files of that particular software.

Afterwards the job of GNU Staw will be to fill each project's `.build/software/installed' directory with symbolic links to the necessary software for each project from `store'.

Of course, this is not as comprehensive a solution as Nix, or GNU Guix, but if we can't find a solution for the problems mentioned before, it may be one step in the right direction, until we can finally adapt them.

Mohammad Akhlaghi <makhlaghi>
Project Administrator
Mon 27 May 2019 12:13:20 PM UTC, original submission:  

So far, we have written a custom pipeline to install all the necessary software for a research project. But it is growing fast and our current solution will not easily scale up to address all the complexities that will arise (and to maintain all those complexities).

But fortunately there is a very good system that is designed precisely for this job: GNU Guix. As a relatively long-term project, I am defining this task to start migrating to GNU Guix as the pipeline's package manager.

Guix has many advantages, the most important is that it keeps track of the dependencies (with MD5 sums in the filenames!), producing a specialized environment that can allow the coexistance of multiple versions of a software without any conflict. In particular, this will allow us to share similar software between different projects (for example not have to compile the same GCC for different projects as we do now!).

But it does have some limitations compared to our current solution. Two of the most prominent ones are these:

  • It needs root access for installing Guix itself. Guix does have a proposed solution for this.
  • It won't work on Mac (because of macOS's poor/changing infra-structure that hamper reproducibility). This can be solved with containers like Docker (which we'll need to include anyway in the near future).

However, in my view, the benefits of using GNU Guix, far out-weight these limitations.

Please share your thoughts and ideas about such a migration.

Of course, for the time being, we'll keep maintaining and working on the current package manager. But work on migrating to GNU Guix will go ahead in parallel until Guix is fully working within the pipeline and and robust solutions to the draw-backs above have been found/implemented.

Mohammad Akhlaghi <makhlaghi>
Project Administrator

 

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by makhlaghi (Submitted the item)
  • -email is unavailable- added by makhlaghi
  •  

    Do you think this task is very important?
    If so, you can add your encouragement to it.
    This task has 0 encouragements so far.

    Only logged-in users can vote.

     

     

     

    Follow 2 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2019-07-29 makhlaghi SummaryUsing GNU Guix as template's package manger => Template's package manger
    2019-05-27 makhlaghi Carbon-Copy- => Added ludovic.courtes@inria.fr

    Back to the top


    Powered by Savane 3.5