taskManeage - Tasks: task #15818, Debian-slim container and specific...

 
 

You are not allowed to post comments on this tracker with your current authentication level.

task #15818: Debian-slim container and specific GCC .deb file on Zenodo

Submitter:  Mohammad Akhlaghi <makhlaghi>
Submitted:  Sun 22 Nov 2020 09:02:25 PM UTC
   
 
Should Start On:  Sun 22 Nov 2020 12:00:00 AM UTC Should be Finished on:  Sun 22 Nov 2020 12:00:00 AM UTC
Category:  None Priority:  5 - Normal
Status:  None Privacy:  Public
Assigned to:  None Percent Complete:  0%
Open/Closed:  Open Effort:  0.00

Fri 09 Apr 2021 09:27:59 AM UTC, comment #4: 

Oh. I didn't consider the economic side of it :)

We can also make the base docker image ourselves. Here is a simple tutorial. So, instead of giving a link that is already available, one can use the code below:

FROM scratch

Is this going to help?

Pedram Ashofteh-Ardakani <pedram>
Group Member
Fri 09 Apr 2021 09:22:10 AM UTC, comment #3: 

Thanks Pedram, I didn't know we can host a Duckerhub ourselves!

The main issue here is who to make the base/first docker image ourselves in a reproducible way. Hosting/distributing that docker image is another issue ;-).

Based on the idea of longevity (and that hosting a large and evolving docker image for the long term will not be cheap), I think the best solution is to use Zenodo for the distribution ;-).

Mohammad Akhlaghi <makhlaghi>
Group administrator
Fri 09 Apr 2021 08:59:54 AM UTC, comment #2: 

You can host a personal docker repository like dockerhub. Quote from https://docs.docker.com/registry/

= What Is It =

The Registry is a stateless, highly scalable server side application that stores and lets you distribute Docker images. The Registry is open-source, under the permissive Apache license.

= Why Use It =

You should use the Registry if you want to:

  • tightly control where your images are being stored
  • fully own your images distribution pipeline
  • integrate image storage and distribution tightly into your in-house development workflow


I found this solution in this stackoverflow question: how to share my docker image without using the dockerhub

Pedram Ashofteh-Ardakani <pedram>
Group Member
Tue 01 Dec 2020 07:14:24 PM UTC, comment #1: 

About the second point below, I think the multi-staged build is not what I had in mind! After reading a little more about it, I think we want something like Docker's save feature.

For the debian-slim base, with GCC, G++ and WGET, the "saved" image (after being compressed) was 109Mb.

However, a "saved" image has to be loaded later, preserving all the sub-image (meta)data.

The thing I mainly had in mind about point 2 was being able to make something that can be put into Docker's 'FROM' directive in a Dockerfile, but with a URL (to zenodo).

Mohammad Akhlaghi <makhlaghi>
Group administrator
Sun 22 Nov 2020 09:02:25 PM UTC, original submission:  

Building Maneage within a Docker image can simplify moving a built project from one computer to another immediately without having to built it from scratch. Therefore README.md has a special section to build the project in a Docker image.

However, currently README.md recommends to use the 'debian:stable-slim' image as its base. As mentioned throughout the Maneage paper, this changes all the frequently. Ideally that change should not cause any problem for Maneage, because Maneage builds everything itself (except for the C library, which will be done in task #15390).

The actual file that gets downloaded for the slim Debian is very small (27.11 Mb right now). So instead of relying on Dockerhub and its links to Debian's servers, we can just keep that file ourselves on Zenodo. That file can then be the recommended base image for building Maneage in a container. We can update it every few years (when Debian stable gets updated).

But to be able to build Maneage, we also need the Debian 'gcc', 'g++' and 'wget' packages. So the very top of our recommended Dockerfile looks something like this:


FROM debian:stable-slim
RUN apt-get update && apt-get install -y gcc g++ wget


I see two ways to do this:

1. In our archival, we can also store all the '.deb' files of these packages and their dependencies and use them instead of Debian's own server.

2. We build a new custom Docker image that blends in the slim image and the image with the compilers and wget into one core image and archives that on Zenodo (it shouldn't be larger than 300 Mb, and can be updated every 2 years!). I just had a fast search and apparently Docker supports this with its multistaged builds.

I currently prefer the second one because it is more simpler/cleaner for ourselves and the users (we will store the Dockerfile that is used to create it with the image). But please let me know what you think.

Mohammad Akhlaghi <makhlaghi>
Group administrator

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by pedram (Posted a comment)
  • -email is unavailable- added by makhlaghi (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

     

    No changes have been made to this item

    Back to the top

    Powered by Savane 3.13-758e.
    Corresponding source code