taskReproducible paper template - Tasks: task #15347, Tarball integrity checks with...

 
 

You are not allowed to post comments on this tracker with your current authentication level.

task #15347: Tarball integrity checks with SHA512 checksums

Submitted by:  Mohammad Akhlaghi <makhlaghi>
Submitted on:  Mon 29 Jul 2019 01:20:12 PM UTC  
 
Should Start On:  Sun 28 Jul 2019 11:00:00 PM UTC Should be Finished on:  Sun 28 Jul 2019 11:00:00 PM UTC
Category:  Software Priority:  5 - Normal
Status:  Done Privacy:  Public
Percent Complete:  100% Assigned to:  Mohammad Akhlaghi <makhlaghi>
Open/Closed:  Closed Effort:  0.00

Mon 29 Jul 2019 07:55:29 PM UTC, comment #4: 

This issue was bugging me for a long time. So I just implemented it (with SHA512 checksums). See the commit message for more.

Mohammad Akhlaghi <makhlaghi>
Project AdministratorIn charge of this item.
Mon 29 Jul 2019 03:22:47 PM UTC, comment #3: 

The calculation time for checksumming is tiny compared to downloading
and compiling. I think we should prefer sha512sum over md5sum:

https://unix.stackexchange.com/questions/260513/diffference-between-sha1sum-sha256sum-and-md5sum

https://stackoverflow.com/questions/2117732/reasons-why-sha512-is-superior-to-md5

Boud Roukema <boud>
Project Member
Mon 29 Jul 2019 02:25:05 PM UTC, comment #2: 

After having a look at how we check MD5 sums of input data sets in download.mk, I see that it is indeed best to do the checksum checks separate from downloading.

This is because the file may exist on the host computer, in that case the download script shouldn't be called. But the MD5 sum should be checked in any case.

Mohammad Akhlaghi <makhlaghi>
Project AdministratorIn charge of this item.
Mon 29 Jul 2019 01:26:20 PM UTC, comment #1: 

Mohammad-reza Khellat has recently worked on the download script file as part of task #15258), so he may be able to do this easily on his branch (which is not yet merged into the master branch).

If anyone else would want to do this, please get in touch with Mohammad-reza to coordinate the work and avoid conflicts.

Mohammad Akhlaghi <makhlaghi>
Project AdministratorIn charge of this item.
Mon 29 Jul 2019 01:20:12 PM UTC, original submission:  

Currently the pipeline just downloads the tarballs and starts using them, without any integrity checks on their content.

We are currently doing/encouraging integrity checks on input datasets (for example in INPUTS.mk). But not on software tarballs (which are equally important).

It is not too hard to to do this: we can add an option to the download script to also accept hash values and check them before returning control back to the Makefiles. Once that is done, we just have to define the hash for every tarball in the same place we define its URL.

This follows a discussion on task #15345.

This also slightly relates with task #15286 (Template's package manger). Once that task is done, for every package, we'll have a unique file with the build instructions and that file will also contain the has of the tarball.

Mohammad Akhlaghi <makhlaghi>
Project AdministratorIn charge of this item.

 

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by boud (Posted a comment)
  • -email is unavailable- added by makhlaghi (Submitted the item)
  • -email is unavailable- added by makhlaghi
  •  

    Do you think this task is very important?
    If so, you can add your encouragement to it.
    This task has 0 encouragements so far.

    Only logged-in users can vote.

     

     

     

    Follow 7 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2019-07-29 makhlaghi StatusPostponed => Done
        Percent Complete0% => 100%
        Assigned toNone => makhlaghi
        Open/ClosedOpen => Closed
        SummaryTarball integrity checks with md5 checksums => Tarball integrity checks with SHA512 checksums
    2019-07-29 makhlaghi SummaryStore and check tarball hash before using it => Tarball integrity checks with md5 checksums
    2019-07-29 makhlaghi Carbon-Copy- => Added boud@astro.uni.torun.pl

    Back to the top


    Powered by Savane 3.5