taskReproducible paper template - Tasks: task #15698, Queue for adding new software


You are not allowed to post comments on this tracker with your current authentication level.

task #15698: Queue for adding new software

Submitted by:  Mohammad Akhlaghi <makhlaghi>
Submitted on:  Thu 18 Jun 2020 05:25:20 PM UTC  
Should Start On:  Wed 17 Jun 2020 11:00:00 PM UTC Should be Finished on:  Wed 17 Jun 2020 11:00:00 PM UTC
Category:  Software Priority:  5 - Normal
Status:  In Progress Privacy:  Public
Percent Complete:  30% Assigned to:  None
Open/Closed:  Open Effort:  0.00

( Jump to the original submission)

Tue 04 Aug 2020 02:41:42 PM UTC, comment #10: 

Thanks a lot for taking the initiative on this Mohammadreza. I just defined task #15741 (Continuous integration) for followup discussions on Continuous Integration. It would be great if we followup CI discussions in that task and keep this task only for the design and implementation of updating software ;-).

By the way, all posts on Savannah go to the reproduce-devel mailing list, so anyone who is a member there will receive any comment on any issue ;-). Raul is also on that list (along with several others).

Mohammad Akhlaghi <makhlaghi>
Project Administrator
Tue 04 Aug 2020 01:59:36 PM UTC, comment #9: 


Mohammadreza Khellat <mkhellat>
Project Member
Tue 04 Aug 2020 01:24:57 PM UTC, comment #8: 

Oops! it seems that here we only have Boud, Mohammad, and I.

Consequences of being disoriented for being on the computer and phone nonstop!

Mohammadreza Khellat <mkhellat>
Project Member
Tue 04 Aug 2020 01:20:49 PM UTC, comment #7: 

Hi Boud, Mohammad, Raul, and all!

Last night, Mohammad and I played around a little bit with Travis. very simple system, most probably taking advantage of Ansible underneath.

We tested it under (check the ci-build branch)

Just to have few things on the record,

  • First, since I have always had respect for Dutch theoretical physicists ;), Jos Vermaseren FORM code:
  • And finally, replacements for Travis CI - If we can find one which does not necessarily integrate with GitHub, it would be great!
Mohammadreza Khellat <mkhellat>
Project Member
Sun 02 Aug 2020 12:45:26 PM UTC, comment #6: 

hi Mohammadreza, excellent links! :)

comment #5:

> Of course, in this crazy world, we really do not have reliable software.
> https://users.ece.cmu.edu/~koopman/des_s99/sw_reliability/
> https://en.wikipedia.org/wiki/Software_reliability_testing
> Loss of reliability is the price that is easily paid to ship software to wide-range of consumers. Yet, I think with opensource, a fair share of the responsibility of making the software work for YOU is with YOU.

I would agree with some of the responsibility being on the user for checking the software directly, but s/he also has responsibility in checking the quality of the community checking of the software, and similar checking at higher meta-levels, and checking the sources of software. Are any of us working on Maneage seriously going to read through all the code lines of Linux? or gcc? And not just read the code, but check that it's safe and reliable? :)

I've posted a Debian option wishlist here:


I think that that would be one way to build on what the wider free-software community has already done in terms of quality assurance for software reliability.

Of course, reproducibility and reliability are not exactly the same topic. As I understand it, the primary idea of reproducibility is that it should be possible to recreate an identical software environment so that a particular published research project can be checked on different systems immediately or a decade or more later on whatever systems exist. Reliability only becomes a question when people wish to modify the project - assumptions, method, input data or analysis - in which case a project may be fully reproducible but unreliable in the software sense when modified.

Boud Roukema <boud>
Project Member
Sun 02 Aug 2020 04:12:19 AM UTC, comment #5: 

Placing/testing package updates through a separate fork and then merging them when the number of updates reaches a specific number through single commit with a popping-out title starting with CAPS is very very good idea and practice which would make things nicely organized.

Of course, in this crazy world, we really do not have reliable software.


Loss of reliability is the price that is easily paid to ship software to wide-range of consumers. Yet, I think with opensource, a fair share of the responsibility of making the software work for YOU is with YOU.

In my opinion, our share of reliability responsibility is to first refrain from adding unnecessary software to the main branch and secondly as you mentioned have a separate fork for adding new necessary software and testing upcoming updates. Then when we have 10 rather reliable software updates, the fork will be rebased.

Mohammadreza Khellat <mkhellat>
Project Member
Sun 02 Aug 2020 12:05:59 AM UTC, comment #4: 

It was a long time that we hadn't updated the software in Maneage and some were already becoming outdated! So I went through all the basic software (as well as some of the high-level and python software), and updated their versions (+35 packages) in a commit on project-dev.

I tried them in a clean build (with '--all-highlevel'), and Maneage built nicely on ArchLinux and a slim Debian (on Docker). Raul, if you get the chance to check them on your macOS it would be great (but no problem if you don't get a chance ;-)).

I will merge this branch into Maneage in one week, so if anyone gets a chance to try it out, please do so and post any problem you may find as a bug ;-).

Also, if anyone is familiar with Travis CI, it would be great if we could setup an automatic build for such version updates. Apparently it has no cost for free software, and it also tests on macOS, as well as several GNU/Linux distros.

Mohammad Akhlaghi <makhlaghi>
Project Administrator
Sun 28 Jun 2020 01:42:29 PM UTC, comment #3: 

Allowing bug fixes to move higher in the queue seems reasonable.

But keeping a time delay before integrating them is also reasonable.

Bug fixes often introduce new bugs, or force users to update the way they use the software.

Having different branches for bug fixes and adding new software also makes sense: these are two different types of things.

Boud Roukema <boud>
Project Member
Sun 28 Jun 2020 03:21:32 AM UTC, comment #2: 

Comment 11 of Task 15686 and its Comment 19, brought up the need to setting a good policy for updating packages out of line for bug fixes.

Should we really push them up and let them skip the queue that is discussed here? Or should we put them in this branch, and tell anyone needs it urgently, to pull from that branch until its merged?

Please share your thoughts on this here ;-)...

Mohammad Akhlaghi <makhlaghi>
Project Administrator
Fri 19 Jun 2020 04:01:45 PM UTC, comment #1: 

Having a time delay would be good too, e.g. a minimum of 7 days.

This would not guarantee that the new software package is checked on all (or any) so-far tested systems, but it would increase the likelihood of testing.

So the criterion for merging could be, e.g. at least 10 new packages that are at least 7 days old as defined by commit dates.

BTW, software in English is an uncountable noun. It's like "water" - another uncountable noun.  We cannot have "a software" or "the number of software" or "the software that are" . Just a warning for when this goes into #comments. :)
We can have a "software package", or "a package", when the context makes it clear what sort of package it is.

Boud Roukema <boud>
Project Member
Thu 18 Jun 2020 05:25:20 PM UTC, original submission:  

The versions of software used in Maneage will continue to evolve and we will be adding new software continuously.

For example, in the last few months that we were mainly focused on the paper, already almost all the basic software now have new versions, not to mention many of the high-level ones!

But having single commits devoted to updating the version of a single software, that is randomly distributed in the middle of other commits in the core Maneage branch can be annoying for the readers who want to follow the recent commits.

The updated software versions can also cause "semantic conflicts" due to some updated/changed feature. So users have to be careful when updating software versions. It would be much more easier for the users to only worry about one commit rather than random commits scattered in the middle of other updates.

So I propose this solution: we keep a special branch on the development fork project-dev, dedicated to updating software versions. When the number of new software reach 10 (just an arbitrary number!) or more, we rebase that branch into the core Maneage branch as one commit. The commit title can also be very clear for example starting with "SOFTWARE VERSION UPDATE: ".

If a user wants to use the "bleeding edge" software that are not yet in the 'maneage' branch, they can simply merge the 'project-dev' branch into their project and use that new version.

I have already started this proposed scenario with Raul's commit to update the version of Gnuastro in the new-software branch.

In fact, if it is a high-level software, we can add it to this branch's 'TARGETS.conf' so it is tested by other people working on this branch automatically. This will allow us to detect fast compilation bugs early (before merging into the Maneage branch).

Note that this can include fundamentally new software also, not just updated versions.

What do you think about this? Please share your thoughts here...

Mohammad Akhlaghi <makhlaghi>
Project Administrator


No files currently attached


Depends on the following items: None found

Items that depend on this one: None found


Carbon-Copy List
  • -email is unavailable- added by mkhellat (Posted a comment)
  • -email is unavailable- added by boud (Posted a comment)
  • -email is unavailable- added by makhlaghi (Submitted the item)

    Do you think this task is very important?
    If so, you can add your encouragement to it.
    This task has 0 encouragements so far.

    Only logged-in users can vote.




    Follows 1 latest change.

    Date Changed by Updated Field Previous Value => Replaced by
    2020-06-28 makhlaghi SummaryNew software (versions) => Queue for adding new software

    Back to the top

    Powered by Savane 3.5