comment #16:

> While doing some other tests, I also noticed that without python, ICU will crash, so in the commit where I slightly edited your commit, I also added Python as a prerequisite of ICU in 6fafc7453.

Good work - I hadn't noticed that. So R depends on icu which depends on the python core!

I added the reference for the base R system:

https://gitlab.com/maneage/project-dev/-/commit/769ed43480b24e8d0920f9321015ff6b12ffd25a

I'll leave the individual packages as a task to do later by any volunteer (me or other).

comment #13:

> By the way, can you please also make sure if any of the R packages come with a paper to cite?

Starting at the R Journal very first issue, https://journal.r-project.org/archive/2009-1 , and 'Facets of R' by 'John M. Chambers', and https://www.r-project.org/contributors.html which says that "R was initially written by Robert Gentleman and Ross Ihaka", I think that

author = {{Ihaka}, Ross and {Gentleman}, Robert},
title = "{R: A language for data analysis and graphics}",
journal = {J.Comput.Graph.Stat.},
year = {1996},
volume = {5},
pages = {299-314},
doi = {10.2307/1390807},
issn = {10618600},

would be the best justified for giving proper credit to R as a whole. For individual R packages, that's the sort of thing that will need more automation using metadata...

By the way, can you please also make sure if any of the R packages come with a paper to cite?

Thanks for the check Boud!

Which hardware/OS/compiler were you using?

I tried it last night (before merging), and it built successfully, both on Arch GNU/Linux and Debain (within a Docker container), on an x86_64 system.

Today, I also tried both of them again (from scratch) and again there was no problem.

Thanks a lot Boud!

After checking out to the branch, I was able to build it successfully and all the packages were properly installed!

Since I had the tarballs, I generalized the simple command in task #15699 (as a script that is now in comment 15), and was able to unify their formatting within a few seconds :-).

I then pushed the tarballs into our project on archiving software tarballs (as Commit 81a8cdfe29 (I mistakenly wrote 20, while it should have been 30!) and made the following two commits after your commit in that branch (since they will ultimately be squashed into all other version-update commits, I am putting their text in the P.S. Please have a look at them ;-).

With these changes, I was able to build and install the updated ICU, R and all the 30 R packages within Maneage. I have therefore already merged it with the 'version-update' branch of task #16103 and am therefore closing this task :-).

If there is any remaining issues on the installation, we can discuss it in task #16103.

It was a great effort Boud, thanks a lot!

On the 'one-letter evil' topic, I personally feel having too long variable names dramatically decreases the readability of the code). So I would have preferred a simple 'r-' prefix (after all, these variables will almost exclusively be in 'r-cran.mk'; where they are repeated a lot!). But you are the maintainer of the R packages, so I respect your choice and won't complain ;-).

P.S.

commit 2ccbc6f150b7f4ae81801ae3aae65bcc324cf644
Author: Mohammad Akhlaghi <mohammad@akhlaghi.org>
Date:   Sun Jan 2 01:56:20 2022 +0100

    Minor formatting edits in r-cran.mk

    In order to fit the formatting and style of the rest of Maneage's source,
    the formattings below are implemented in this commit:

     - When using pipes or '&&' and the line needs breaking (with a '\'), the
       '\' is placed before the '|' or '&&', so the next line starts with these
       tokens. This greatly helps in readability (because they can be missed at
       the end of a long line).

     - Emacs's 'M-q' (ALT + q key) was pressed on the multi-line comment
       paragraphs to justify the lines with a standard width to all comments.

     - The 'version-info/r-cran' directory is now made in
       'reproduce/software/shell/configure.sh', like all other similar files
       and removed from the recipe for 'R'.

     - Generally, where possible we should break lines that are longer than 75
       characters, so where necessary, the prerequisites been moved to a new
       line.

     - For the text that goes into the LaTeX macro to be printed in the final
       PDF (not as an internal variable), I have removed the common 'r-cran',
       because it will not generally be recognized and may cause
       confusion. Ideally, it should be the official name of the package, on
       its webpage in basic English, for example also respecting the large-caps
       letters in its name.

     - When the number of independent lines that are immediately following each
       other (for example the list of prerequisites when there are many of
       them), it is much more human-friendly for them to be sorted by
       length. The order of the prerequisites doesn't affect our recipes, so
       its best to do this and help the human eye when inspecting the code ;-).

commit 64deae4ce04b4c1a1f9567348bcc9b573e8f5649
Author: Mohammad Akhlaghi <mohammad@akhlaghi.org>
Date:   Sun Jan 2 04:53:06 2022 +0100

    Included R package tarballs are now in Maneage format

    The included R packages have been formatted in the Maneage '.tar.lz'
    format, and those tarballs are now being used. In the process, the
    following generalizations also have been done to the 'r_cran_build'
    function:

     - The indentation has been fixed.

     - Generally, all the variables defined in this function open inside the
       recipe of rules here, so there is no chance of conflicting with other
       variables. As a result (to have a significant help in readability!), the
       'r_cran_' prefix has been removed from the variable names.

     - By definition, none of these variables will also have a space or white
       character! So again, to help in readability, the '$${}' was just changed
       to '$$'. Where necessary (one instance, since '_' is used) double-quotes
       have been put around them.

     - As explained in the comments, since R's 'install.packages' only accepts
       '.tar.gz' files, we now unpack the tarball before calling R, and give it
       the directory (this works for tarballs from R's server and Maneage's
       tarballs).

     - The extra 'versioncheck' has been removed.

    To follow a similar convention to other software's build rules, the
    'tarball' variable is now defined at the start of the recipe of each R
    package to let the users customize it as necessary and per-package
    (especially when not using Maneage's archived tarball). A comment has been
    added, explaining what to do.

    Furthermore, the version of ICU has also been updated and its tarball is
    now also in Maneage.

Thanks Boud, I just committed it over the rebased branch (as Commit 0b6e88dc).

It would be great if you do further updates on the inclusion of R packages over this branch (there were several minor conflict during the rebase).

Thanks for all the great work on this Boud!

I rebased the two commits over the 'version-update' branch, but couldn't find any rules for all the R packages added! Reading through the message of Commit 3ee3e321f6 (hash after rebasing), I noticed that there is suppose to be a 'r-cran.mk' file that was apparently forgotten in the commits.

Can you please add the file to this fork of the R branch?

Also, could you please format the tarballs in the Maneage convention for longevity (as described in task #15699)? You can use the latest box of code in that task as the recipe (it is designed to be easily placed into a 'for' loop and done over many tarballs in cases like this).

Once the new tarballs are made, you can update the checksums and push them to a development branch of tarballs-software project, so I later merge them in (I have already added you as a "developer", so you can push branches there and avoid making a large fork!). Keeping a clone of this repository on your computer is also good to provide as the "software" directory in all your Maneage'd projects and avoiding to have many copies of the tarballs (symbolic links are placed in the build directory of each project). Alternatively, you can put the tarballs in a certain URL that I can pick-up and put in that repository myself.

As described in task #15699, it is very important for longevity that we have a unified format for all the newly added software tarballs ;-)! We can't do anything for the old ones, but all new ones should have this format.

In the process of re-building the tarballs, I also recommend to standardize the unpacked directory name to avoid complicating the un-packing+building code (and further unifying the format of the tarball).

Please ignore https://gitlab.com/maneage/project-dev/-/merge_requests/5 and instead use this updated commit:

https://codeberg.org/boud/maneage_dev/commit/1ea9d6db9fab5fd2d88fb51d149777a8090b038f

that branches directly off commit 3a1b96766c. Commit 1ea9d6d includes a dependence on the 'icu' libraries, to avoid becoming system-dependent or failing when the icu libaries are absent.

As before, offline testing and Darwin/xnu testing would be good.

When you added the 'R CMD INSTALL ...' command, did it download all the dependencies as binary from the internet? The "make line" is a little ambiguous, did you add a full new Make rule (with a separate target for this package), or just added in the same rule that R is built.

I have never needed R, but I know a lot of people/projects use it. So have always been looking for someone who needs it (and is thus motivated!), to help in adding R packages in Maneage similar to Python. Many projects use R and if we have the infra-structure to add R packages easily (like in Python or X.org), it would be great.

Following the Python/X.org model, we can define a 'reproduce/software/make/r.mk' Makefile file which will be only in charge of building R and its packages. I also have a similar model in mind for TeXLive packages in the development branch (with a 'reproduce/software/make/tex.mk'). We can then "include" these language-specific packages into 'high-level.mk' the same way that we add Python and X.org's packages.

Having a look at the ggplot2 page you sent, it seems the dependencies are under the "Imports" title (in this case "digest, glue, grDevices, grid, gtable, isoband, MASS, mgcv, rlang, scales, stats, tibble, withr"). The page also has the link to the "package source" tarball. So effectively it has all the components we need!

If you can find the relevant command to disable internet connection when building from the tarball, here is an "easy" way to climb down the dependency tree! This is what we did in the first days of adding Python packages:

You simply add the rule to install your high-level package, 'ggplot2' in this case, but without any internet. Once it doesn't find its dependencies, it's build should fail, saying that you need to have XXXX (if it lists many depenendencies, just take the first one).

You then add another Make rule for XXXX, and set it as a prerequisite of 'ggplot2'. In the build of XXXX, it will probably complain about not having YYYY (as a dependency of XXXX), so you add a rule for YYYY and set it as a dependency for XXXX. You follow this loop until the first package (lowest-level package!) successfully builds!

Make will then automatically come up the dependency tree you have already setup, and will complain on the first package that needs another dependency (a different branch in the dependency tree!). So again you continue and soon you'll have all the low-level infrastructure built :-).

Like Python (and any high-level language), the core dependencies should be the same for most packages, so once we build them for one package, it is much more easier to do it for other packages.