taskReproducible paper template - Tasks: task #15708, Only connecting to Zenodo when...

 
 

You are not allowed to post comments on this tracker with your current authentication level.

task #15708: Only connecting to Zenodo when necesssary

Submitted by:  Mohammad Akhlaghi <makhlaghi>
Submitted on:  Sun 28 Jun 2020 12:09:54 AM UTC  
 
Should Start On:  Sat 27 Jun 2020 11:00:00 PM UTC Should be Finished on:  Sat 27 Jun 2020 11:00:00 PM UTC
Category:  Software Priority:  5 - Normal
Status:  Postponed Privacy:  Public
Percent Complete:  0% Assigned to:  None
Open/Closed:  Open Effort:  0.00

Tue 30 Jun 2020 08:17:05 PM UTC, comment #2: 

Good point! Yes! The DOI is the more trustable link (in case Zenodo changes its internal file structure later!).

But its not as straightforward as seeing if a software directory is given because the software directory may not be complete (and the project's software versions may have been updated since the last time that directory was updated.

Mohammad Akhlaghi <makhlaghi>
Project Administrator
Tue 30 Jun 2020 06:20:55 PM UTC, comment #1: 

There's not only a connection to Zenodo, there's first a connection to doi.org :

Resolving doi.org (doi.org)... 172.67.72.147, 104.26.8.237, 104.26.9.237, ...
Connecting to doi.org (doi.org)|172.67.72.147|:443... connected.
HTTP request sent, awaiting response... 302
Location: https://zenodo.org/record/3883409 [following]

This also has to be tidied up and checked for users who start out with their own tarballs. One of the officially claimed [1] features of maneage is that a maneaged project can be reproduced completely offline. This should be straightforward to handle: if the user has set a local tarball directory, then checking at Zenodo:3883409 is disabled, together with all other downloading.

[1] https://arxiv.org/abs/2006.03018

Boud Roukema <boud>
Project Member
Sun 28 Jun 2020 12:09:54 AM UTC, original submission:  

Currently, every time we run './project configure', it will attempt to connect to Zenodo and find the most recent upload to use in possible downloads.

But especially in the development phase, this is really annoying and slows down the testing. It can also cause confusion for a user who has all their project's software already in a directory and doesn't want to connect to the internet.

Fortunately the software downloading phases have become highly modular and all downloads (except for those in 'pre-make-build.sh') are organized in a singular place now. They are also 'flock'ed (to avoid parallel downloads). So we can move this check within that part of the script and only connect to Zenodo for a simple check when its necessary.

The resulting Zenodo URL can be written to a file which is later checked by other download attempts. Also, in the 'configure.sh' script (before the first download attempt) we can delete this file (if it exists) to make sure that the same Zenodo URL is not used in two rounds of configuration (possibly separated by months!).

Mohammad Akhlaghi <makhlaghi>
Project Administrator

 

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by boud (Posted a comment)
  • -email is unavailable- added by makhlaghi (Submitted the item)
  •  

    Do you think this task is very important?
    If so, you can add your encouragement to it.
    This task has 0 encouragements so far.

    Only logged-in users can vote.

     

     

     

    No changes have been made to this item

    Back to the top


    Powered by Savane 3.5