taskSavannah Administration - Tasks: task #15221, Submission of Reproducible paper...

 
 

task #15221: Submission of Reproducible paper template

Submitter:  Mohammad Akhlaghi <makhlaghi>
Submitted:  Tue 26 Mar 2019 11:35:04 AM UTC
   
 
Should Start On:  Tue 26 Mar 2019 12:00:00 AM UTC Should be Finished on:  Fri 05 Apr 2019 12:00:00 AM UTC
Category:  Project Approval Priority:  * 5 - Normal
Status:  Done Privacy:  Public
Assigned to:  ineiev Open/Closed:  Closed
* Mandatory Fields

Add a New Comment Rich Markup
   

Jump to the original submission

Fri 19 Apr 2019 02:05:36 AM UTC, comment #30: 

Thank you very much, I just posted it to the administration support tracker.

Mohammad Akhlaghi <makhlaghi>
Thu 18 Apr 2019 09:18:57 AM UTC, comment #29: 

Bob Proulx said he fixed Cgit; further requests please file to savannah-help-public@ or to administration support tracker.

Ineiev <ineiev>
Site Administrator
Mon 15 Apr 2019 12:33:26 PM UTC, comment #28: 

You are right! I am updating the customization checklist (in `README-hacking.md') to start with cloning the `master' branch.

I also pushed this change to Savannah (so there is only one `master' branch). But I still can't browse the Git history (here: http://git.savannah.nongnu.org/cgit/reproduce.git ), even though the refreshing time is later than the time I pushed this change.

Is there anything special I have to do to activate the Savannah Git web-browsing? I can't remember if I did anything special back when I started Gnuastro.


Mohammad Akhlaghi <makhlaghi>
Mon 15 Apr 2019 10:06:01 AM UTC, comment #27: 


> Its users must actively customize it from the start. They do this customization in the source code, but on the "master" branch they create them selves.


This doesn't seem intuitive to me.  First, it would be natural to name their branch after their specific "project", then, they may have multiple "projects".  Which suggests me that having templates in the "master" branch is justified.

> Is there a problem with not having a master branch on Savannah?


You reported that there are.

Ineiev <ineiev>
Site Administrator
Mon 15 Apr 2019 09:27:57 AM UTC, comment #26: 

Yes. But as I mentioned before, "This is intentional: users clone this branch, define their own project's "master" branch, and keep their project's infra-structure updated by pulling from the "template" branch when they want."

This is a unique project. It does run with no modifications. But its raw output is useless.

its users are not just passive runners of the code. Its users must actively customize it from the start. They do this customization in the source code, but on the "master" branch they create them selves.

In other words we want the users to be able to follow common conventions.

Is there a problem with not having a master branch on Savannah?


Mohammad Akhlaghi <makhlaghi>
Mon 15 Apr 2019 09:05:23 AM UTC, comment #25: 

Why not use common conventions?  If you don't, you are likely to have other issues, aren't you?

Ineiev <ineiev>
Site Administrator
Sun 14 Apr 2019 01:36:31 PM UTC, comment #24: 

Thanks a lot!

I corrected the "copyright information" text in `README.md' so it clearly points to the copyright notice (at the top of the file) and the license notice (rest of that section).

Afterwards, I activated Git in the project's page on Savannah and made the first push of the existing history. Things were pushed cleanly (without any error). However, the cgit web interface didn't show anything.

Just to see if everything is fine, I cloned the pushed project and got this warning after it had fully downloaded it:


$ git clone https://git.savannah.nongnu.org/git/reproduce.git
Cloning into 'reproduce'...
remote: Counting objects: 2179, done.
remote: Compressing objects: 100% (773/773), done.
remote: Total 2179 (delta 1292), reused 2179 (delta 1292)
Receiving objects: 100% (2179/2179), 3.33 MiB | 1.31 MiB/s, done.
Resolving deltas: 100% (1292/1292), done.
warning: remote HEAD refers to nonexistent ref, unable to checkout.


After some investigation, I found out that this is due to the fact that the main branch of this project is not "master", but "template".

This is intentional: users clone this branch, define their own project's "master" branch, and keep their project's infra-structure updated by pulling from the "template" branch when they want.

To fix this problem, apparently we have to run the following command to manually point `HEAD' to the `template' branch (instead of the default `master' branch).


git symbolic-ref HEAD refs/heads/template


I hope this also fixes the problem with cgit.

Could you please run this command for this project?

Thank you very much for all the great help.

Mohammad Akhlaghi <makhlaghi>
Sun 14 Apr 2019 07:55:24 AM UTC, comment #23: 

Thank you, approving.

.file-metadata still lacks a copyright notice, you should fix that.

By the way, README.md wrongly lists the license notice under "Copyright information", while that section in fact contains no copyright notices.

Ineiev <ineiev>
Site Administrator
Sat 13 Apr 2019 11:31:22 PM UTC, comment #22: 

They are corrected, here is the tarball link:
https://gitlab.com/makhlaghi/reproducible-paper/-/archive/pipeline/reproducible-paper-pipeline.tar.gz

Indeed, I really needed to refresh myself on the Legal matters section of the maintainers guide. For Gnuastro, I didn't have this problem (people with contributions more than 10 lines assign their copyright to the FSF and then I would merge it). Thanks a lot for the reminder.

Based on this part of it ("you should always include a copyright notice in the name of main copyright holder of the file. You can also include copyright notices for other copyright holders as well"), my second guess was the solution:


Copyright (C) YYYY-YYYY, ORIGINAL AUTHOR.
Copyright (C) YYYY-YYYY, MODIFYING AUTHOR.


This is now also done for the two git hook scripts (originally from Metastore, with an MIT license). The years in the copyright notices (2016 and 2018-2019) clearly show which one is the original copyright holder. Since its just these two files, and the MIT license is short, I also just put the whole MIT license in the two files (and decided to release any modifications to these files as MIT license also to keep things clean and simple).

Mohammad Akhlaghi <makhlaghi>
Sat 13 Apr 2019 05:45:00 PM UTC, comment #21: 


> I am not a lawyer, but based on this part of the GPL (under Section 10), "Each time you convey a covered work, the recipient automatically receives a license from the original licensors, to run, modify and propagate that work, subject to this License."


When you get a license, you by no means become a copyright holder; you would need a copyright assignment rather than a license.

> is it correct for the person modifying the code to replace their name with name of the original author(s)?


I think I don't quite understand this.

If someone modifies code non-trivially, they (or those who they assigned their copyright to, like in case of gnuastro) becomes an additional copyright holder; check https://www.gnu.org/prep/maintain/html_node/Copyright-Notices.html for more details on what non-trivial changes are and how to write copyright notices for code with multiple copyright holders.

(Probably you should re-read whole https://www.gnu.org/prep/maintain/maintain.html#Legal-Matters section.)

> This question will also fix the last problem you raised on the two git scripts. In that case, I have modified a previously written code (under MIT license). In fact this case is a little more complicated: as far as I know, unlike the GPL, the MIT license allows usage of any license. So how should I say that the original author distributed it as MIT, but I distribute my modifications to it under GPL?


I think you can just put the MIT license with the license notice for the GPL.

Ineiev <ineiev>
Site Administrator
Thu 11 Apr 2019 04:33:09 PM UTC, comment #20: 

Thanks for the very thorough checks.

1) GNU FDL is now also available as COPYING.FDL in the top Gnuastro source directory. As described in the commit-message of the link before, it was imported from Gnulib during bootstrapping and added to the manual while building. So it wasn't present in the raw Gnuastro version-controlled source.

2) I agree! The main problem is that this "Reproducible paper template" project is just template: designed such that users must modify the source to customize it for their projects. I used this wrongly written copyright statement to encourage people to add a copyright to their added files.

But as you correctly point out, before their correction it makes the copyright notice meaning-less. I went through the GPL, but couldn't find any example on how to add a license to an already licensed code.

I am not a lawyer, but based on this part of the GPL (under Section 10), "Each time you convey a covered work, the recipient automatically receives a license from the original licensors, to run, modify and propagate that work, subject to this License." is it correct for the person modifying the code to replace their name with name of the original author(s)?

Or, should it be something like this:


Copyright (C) YYYY-YYYY, ORIGINAL AUTHOR.
Copyright (C) YYYY-YYYY, MODIFYING AUTHOR.


This question will also fix the last problem you raised on the two git scripts. In that case, I have modified a previously written code (under MIT license). In fact this case is a little more complicated: as far as I know, unlike the GPL, the MIT license allows usage of any license. So how should I say that the original author distributed it as MIT, but I distribute my modifications to it under GPL?

After your recommendation, I'll be able to fix these two issues also.

Thanks a lot for this wonderful review.

Mohammad Akhlaghi <makhlaghi>
Wed 10 Apr 2019 04:59:56 PM UTC, comment #19: 

Thank you!

One more notice on gnuastro: I couldn't find a copy of the GFDL in its sources.

Now, back to this package.

Files like
reproduce/src/make/paper.mk
reproduce/src/make/dependencies-python.mk
have no valid copyright notice.

.file-metadata lacks copyright and license notices.

reproduce/src/bash/git-pre-commit lacks valid license notice;
even worse, it looks like it has a wrong copyright notice:
the primary author of that file is Przemyslaw Pawelczyk <przemoc@gmail.com>
rather than Mohammad Akhlaghi.  Could you comment on this?

Ineiev <ineiev>
Site Administrator
Sat 06 Apr 2019 11:14:44 PM UTC, comment #18: 

Thanks a lot for the clarification it has been fixed in this template and Gnuastro. I now understood the problem: I was confusing a "copyright notice" with a "license notice".

Mohammad Akhlaghi <makhlaghi>
Fri 05 Apr 2019 05:20:26 AM UTC, comment #17: 


> I tried it, but noticed that it misses short files, or configuration files that have the short all-permissive copyright statement:


I don't think it does, e.g. in gnuastro, it shows about 90 files longer than 15 lines, the first 10 are:

.dir-locals.el
.autom4te.cfg
bin/arithmetic/astarithmetic.conf
bin/buildprog/astbuildprog.conf.in
bin/fits/astfits.conf
bin/segment/astsegment.conf
bin/convolve/astconvolve.conf
bin/warp/astwarp.conf
bin/cosmiccal/astcosmiccal.conf
bin/convertt/astconvertt.conf

Ineiev <ineiev>
Site Administrator
Thu 04 Apr 2019 06:21:27 PM UTC, comment #16: 

Thanks a lot. I tried it, but noticed that it misses short files, or configuration files that have the short all-permissive copyright statement:


Copying and distribution of this file, with or without modification,
are permitted in any medium without royalty provided the copyright
notice and this notice are preserved.  This file is offered as-is,
without any warranty.


All the relevant files thus now have a copyright statement in the source codes of both Gnuastro and this new project.

I hope there is no other issue and we can start continuing this new project on Savannah.

Mohammad Akhlaghi <makhlaghi>
Thu 04 Apr 2019 04:46:05 PM UTC, comment #15: 

I check tarballs manually; a command like

find . -type f ! -path ./.git/\* |while read i;do grep -qi 'copyright \((c)\|[&]copy;\|@copyright\)' $i || echo $i;done

must detect most files that miss copyright lines.

Ineiev <ineiev>
Site Administrator
Wed 03 Apr 2019 03:00:26 PM UTC, comment #14: 

They have both been implemented.

Thanks a lot for pointing them out.

If you have any automatic script to do this, it would be great if you could point me to it so every once and a while, I could run and see which new files lack a copyright.

Mohammad Akhlaghi <makhlaghi>
Wed 03 Apr 2019 07:33:01 AM UTC, comment #13: 

Thank you; there are some minor issues left:


> I have been really busy in the last few days and couldn't do a proper systematic search for such cases, but hopefully this should do it. Thanks a lot for all the patience, I really appreciate it.


There is no haste.

Ineiev <ineiev>
Site Administrator
Tue 02 Apr 2019 12:03:52 PM UTC, comment #12: 

Thanks a lot for pointing this out, all relevant files are now copyrighted.

To finally fix this problem (in Gnuastro and this project) I ran the following script to print all the files that don't have a "copyright" term in them and added the notice to them.


for f in $(find ./ -type f); do \
  if [[ $f != ./.git/* ]]; then \
    n=$(grep -i copyright $f | wc -l); \
    echo "$n $f"; \
  fi; \
done | awk '$1==0'


Of course, it does have the caveat that if "copyright" is used within the file (not as an actual copyright notice), it will not report it. But at least in Gnuastro, I don't think we have such cases.

As before the updated tarball is available here:
https://gitlab.com/makhlaghi/reproducible-paper/-/archive/pipeline/reproducible-paper-pipeline.tar.gz

I have been really busy in the last few days and couldn't do a proper systematic search for such cases, but hopefully this should do it. Thanks a lot for all the patience, I really appreciate it.

Mohammad Akhlaghi <makhlaghi>
Tue 02 Apr 2019 04:53:33 AM UTC, comment #11: 

Great!  What about files like .dir-locals.el?

Ineiev <ineiev>
Site Administrator
Mon 01 Apr 2019 01:37:26 PM UTC, comment #10: 

That was a good point! I had completely missed these files for a copyright notice. It is done.

Thanks...

Mohammad Akhlaghi <makhlaghi>
Mon 01 Apr 2019 06:01:52 AM UTC, comment #9: 

It looks like you repeatedly miss some files.

Wouldn't you mind to train on your approved GNU package, gnuastro? In its git, files like NEWS lack copyright and license notices.

Could you check all files (as in `find . -type f`), and make sure that all copyrightable ones have notices?

Ineiev <ineiev>
Site Administrator
Sun 31 Mar 2019 11:16:21 PM UTC, comment #8: 

Indeed, some configuration files (standard name-value pairs) were missing a copyright. I added the short all permissive copyright notice, suggested by the GNU maintainer guide.

The same original link now contains the fixes:
https://gitlab.com/makhlaghi/reproducible-paper/-/archive/pipeline/reproducible-paper-pipeline.tar.gz

Thanks a lot,

Mohammad Akhlaghi <makhlaghi>
Sun 31 Mar 2019 03:19:05 PM UTC, comment #7: 

Some files in your tarball still lack copyright and license notices; did you check them all?

Ineiev <ineiev>
Site Administrator
Sat 30 Mar 2019 10:04:04 PM UTC, comment #6: 

The updated tarball is attached (the link of the original submission also points to this tarball).

Thanks for the great points ;-).

(file #46675)

Mohammad Akhlaghi <makhlaghi>
Sat 30 Mar 2019 08:45:42 AM UTC, comment #5: 

Please attach your new tarball to a comment.

Ineiev <ineiev>
Site Administrator
Fri 29 Mar 2019 08:06:57 PM UTC, comment #4: 

Thanks a lot for the note.

A copy right notice is now added to the README files and also the LaTeX source files. The long configuration files have also been given a copyright notice.

Mohammad Akhlaghi <makhlaghi>
Thu 28 Mar 2019 06:35:45 AM UTC, comment #3: 


> could you please point me to some of the copyrightable files that lack a copyright?


README.md, for example.

> The `*.tex' files don't indeed have a copyright yet (which
> I wanted to consult with you about since they aren't computer code).


Copyrightable files are files more than ten lines long; it doesn't matter if they are "code" or anything else.  The point is to make them easily re-usable. https://www.gnu.org/licenses/gpl-faq.html#NoticeInSourceFile discusses "code," but it applies to any kind of files.

Ineiev <ineiev>
Site Administrator
Wed 27 Mar 2019 05:54:32 PM UTC, comment #2: 

Hi, Thanks for the check.

You are right about the first point! I didn't see GPLv3 in the list, so I clicked on GPLv2+. But generally, its GPLv3.

About the second point, could you please point me to some of the copyrightable files that lack a copyright? The `*.tex' files don't indeed have a copyright yet (which I wanted to consult with you about since they aren't computer code), but any other script (for example `configure', or `reproduce/src/make/*.mk' and `reproduce/src/bash/*.sh' which are the heart of this project all have copyright notices).

Adding a copyright statement on every file is indeed very important and it is even mentioned in the checklist section of "README-hacking.mk" file (that describes what users should do to customize it for their projects). So if any file is missing a copyright statement, its a mistake and will be fixed.

Mohammad Akhlaghi <makhlaghi>
Wed 27 Mar 2019 05:08:58 PM UTC, comment #1: 

Hello, Mohammad!

First, you say in your submission that it's GPLv2+, but COPYING in your tarball is GPLv3; what is the intended license?

Then, many copyrightable files in your tarball lack copyright and license notices.  Do you maintain your GNU package the same way?

Ineiev <ineiev>
Site Administrator
Tue 26 Mar 2019 11:35:04 AM UTC, original submission:  

A new project has been registered at Savannah
This project account will remain inactive until a site admin approves
or discards the registration.


Registration Administration


While this item will be useful to track the registration process,
approving or discarding the registration must be done using the specific Group Administration page, accessible only to site administrators,
effectively logged as site administrators (superuser):



Registration Details


  • Name: Reproducible paper template
  • System Name:  reproduce
  • Type: non-GNU software and documentation
  • License: GNU General Public License v2 or later





Description:

This is a template that enables easy adoption of best-practices for doing a project in a fully reproducible way, while also building the final paper. It includes instructions to download the software tarballs and input data, how to build the software and use them to analyze the input data as well as how to finally build the paper. This automation allows the project to be exactly reproducible at a later time (for example after publication). It is written in Make and heavily relies on Git. It has been founded by Mohammad Akhlaghi (maintainer of GNU Astronomy Utilities) and we will be applying for it to be accepted into GNU later.


Other Software Required:

Low-level operating system components (Make, shell, AWK, SED and a C compiler)


Tarball URL:

https://gitlab.com/makhlaghi/reproducible-paper/-/archive/pipeline/reproducible-paper-pipeline.tar.gz


Mohammad Akhlaghi <makhlaghi>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

Attached Files

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by ineiev (Posted a comment)
  • -email is unavailable- added by makhlaghi (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only logged-in users can vote.

     

    Follow 5 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2019-04-14 ineiev StatusIn Progress Done
        Open/ClosedOpen Closed
    2019-03-30 makhlaghi Attached File- Added reproducible-paper-pipeline.tar.gz, #46675
    2019-03-27 ineiev StatusNone In Progress
        Assigned toNone ineiev

    Back to the top

    Powered by Savane 3.13-cf05.
    Corresponding source code