taskReproducible paper template - Tasks: task #15741, Continuous integration

 
 

You are not allowed to post comments on this tracker with your current authentication level.

task #15741: Continuous integration

Submitted by:  Mohammad Akhlaghi <makhlaghi>
Submitted on:  Tue 04 Aug 2020 02:35:07 PM UTC  
 
Should Start On:  Mon 03 Aug 2020 11:00:00 PM UTC Should be Finished on:  Mon 03 Aug 2020 11:00:00 PM UTC
Category:  Software Priority:  5 - Normal
Status:  In Progress Privacy:  Public
Percent Complete:  30% Assigned to:  Mohammadreza Khellat <mkhellat>
Open/Closed:  Open Effort:  0.00

( Jump to the original submission)

Fri 28 Aug 2020 11:35:58 AM UTC, comment #10: 

Update:
maneage-ci has been re-based https://gitlab.com/mkhellat/maneage-ci.

  • did major changes in the options, and
  • tested deploying a new worker on a HighSierra VM.
Mohammadreza Khellat <mkhellat>
Project MemberIn charge of this item.
Mon 17 Aug 2020 04:10:16 AM UTC, comment #9: 

I have been editting the script fir the past 3 days.

This is what is the nice about shell, you can easily glue tools together, in most cases where the creator is loyal to the philosophy of unix shell.

Will commit the changes by today or tomorrow and start working on qemu.

== NOTE ==

To refresh your memory:

If you remember, comparison of the created instruction sets or the assembly codes was something that I brought up during the first skype call with Iran. And this is definitely a very interesting core question when pursuing the objective of reproducible research workflow; however, it is something that should be pursued when OS independence is achieved.

This is the approach that has been followed in maneage to organize your workflow by first having some sort of LFS structure in place with its own shell and everything, from there you would have the opportunity to move in lots of interestingndirections!! And I am happynabout it :)

Going back to the question, it is clear that the difference between diffrerent CPU archs, related to our problem, is in the instruction sets. This is the compilor that who preprocess a code and then weave together a symbolic assembly code. In other words, the difference in the execution of an application or the behavior of libraries comes from first how it was assembled and secind the design of the CPU itself and the instruction sets released for it.

In theory, a part of a response to the reviewer comment could be something like this:

  • creating proper and self-consistent instruction sets is the responsibility of the compilor after the preprocessing, when it is generating the assembly symbolic code and when it is doing the linking. In order to refrain from dealing with instruction set related investigations, in the first phase, in the research workflow, the researchet should, as the referee suggesting, explicitly mention the CPU architecture that s/he has used to build the project.
  • in the second phase, the workflow could take advantage of tools like qemu to test project build and execution on different CPU archs.
  • in a lower-level, there is no other way than directly studying the instruction sets.

( and definitely, there are existing research and nice discussions around this, whuch i personally prefer not to look into in the begining and to think about them myself first ;) )

Mohammadreza Khellat <mkhellat>
Project MemberIn charge of this item.
Mon 17 Aug 2020 12:18:52 AM UTC, comment #8: 

The first review of the Maneage paper was recently publicly released: https://doi.org/10.22541/au.159724632.29528907

In it, the reviewer recommends testing on different CPU architectures which is directly related to this task. If we can get BuildBot to build on different QEMU instances (https://www.qemu.org), we can test new releases on different CPU architectures from a single machine.

Mohammad Akhlaghi <makhlaghi>
Project Administrator
Wed 12 Aug 2020 06:18:16 AM UTC, comment #7: 

Thanks Mohammad ;)
Yeah, I understand. Creating an automation system might take a lot of effort in the begining. But of course, if done with proper mindset, it could save a lot of energy later on.

> Maybe later, you can also add a small README.md to describe the general purpose, the necessary tools and environment that is necessary to test the script. I am just curious: is Python the only interface to run BuildBot?
>
> When its ready for testing, please add the necessary steps to try it out and we'll hopefully try it for the next time we want to update Maneage software (so there is no immediate rush).


Yeah, BuildBot is built upon python3 but as I have mentioned in the body of the script.
It would be run in a SandBoxed VertualEnv(python venv module): It would have all the python libraries and binaries inside the BuildBot Master and Worker directories and would not need any permission elevation.

Since the primary user of this would have been ourselves and I had only added a very first preview of a sample BuildBot deployment, I didn't add the README.md. In fact, I was hoping you guys would try this out first so then I would add the README; however, I am adding one now which I guess is gonna change very much later on.

I have tried to write the script in a very organized manner. It has help and everything and is informing you what it is doing at each and every step. You can run it without any flags.

Furthermore, thanks to the BuildBot design, I could write the script in a way that you can run it cleanly how many times you want.

So you can test it now and let me know if you encountered any issues.

>
> For macOS indeed, we need a virtual machine, but for things like FreeBSD, or a few standard/older GNU/Linux distributions, it may be better to use Docker (it is so much more light-weight than a virtual machine) and designed precisely for such temporary OS creation and deletions ;-).
>
> I hope we can get a macOS in a virtual machine! It will facilitate testing SO MUCH! Hopefully on various versions of macOS!


After the README, I will work on the MacOS VM.

Mohammadreza Khellat <mkhellat>
Project MemberIn charge of this item.
Tue 11 Aug 2020 06:57:31 PM UTC, comment #6: 

This looks very interesting! Thanks Mohammad-reza ;-).

Maybe later, you can also add a small README.md to describe the general purpose, the necessary tools and environment that is necessary to test the script. I am just curious: is Python the only interface to run BuildBot?

When its ready for testing, please add the necessary steps to try it out and we'll hopefully try it for the next time we want to update Maneage software (so there is no immediate rush).

For macOS indeed, we need a virtual machine, but for things like FreeBSD, or a few standard/older GNU/Linux distributions, it may be better to use Docker (it is so much more light-weight than a virtual machine) and designed precisely for such temporary OS creation and deletions ;-).

I hope we can get a macOS in a virtual machine! It will facilitate testing SO MUCH! Hopefully on various versions of macOS!

Mohammad Akhlaghi <makhlaghi>
Project Administrator
Tue 11 Aug 2020 06:37:45 PM UTC, comment #5: 

At last, I finished and uploaded the a sample buildbot deployment script for build tetst:

https://gitlab.com/mkhellat/maneage-ci

Next, I will define a second worker in the buildmaster configuration file (master.cfg) for an OSX VirtualBox virtual Machine.

Also asked around about Jenkins and GoCD. It seems that Jenkins has become an industry standard in the cloud. If we really had to investigate other CI/CDs, I guess in the next steps, it is best to check the GitLab's own CI/CD first and then Jenkins and GoCD.

Mohammadreza Khellat <mkhellat>
Project MemberIn charge of this item.
Wed 05 Aug 2020 12:49:57 PM UTC, comment #4: 

I have started by looking at Wiki's CI comparison list

BuildBot is the only one in the list licensed under GPL and seems very promising; that is why I am testing it on my laptop and will soon update you regarding the results.

Next ones to check from the list would be GoCD, GitLab built-in CI, and Jenkins CI.

Mohammadreza Khellat <mkhellat>
Project MemberIn charge of this item.
Tue 04 Aug 2020 03:05:32 PM UTC, comment #3: 

It seems that there is a 50 mins limit on running jobs.

One option is to run parallel jobs which maneage is already doing.

I just did a quick search on alternative CIs. It seems that Jenkins CI is very popular!

Lets check them out...

Mohammadreza Khellat <mkhellat>
Project MemberIn charge of this item.
Tue 04 Aug 2020 02:50:43 PM UTC, comment #2: 

I just checked job #1.1, and this time, it crashed with this error message:

The job exceeded the maximum time limit for jobs, and has been terminated.

Mohammadreza, is there a way we can request more time (without paying money!)?

Mohammad Akhlaghi <makhlaghi>
Project Administrator
Tue 04 Aug 2020 02:47:46 PM UTC, comment #1: 

One issue that we encountered in our initial tests yesterday was that apparently TravisCI only allows roughly 36000 lines in the output! So Maneage's test wasn't completed. It should also happen for job #1.1 (the build log that Mohammadreza posted before).

This is the reason for Mohammadreza's second point on "transferring files from within Travis CI Worker's filesystem". We have to redirect the output of './project configure' to a file instead of printing on standard output. But then, the problem becomes this: how we can get that file out of the virtual machine created by TravisCI before the whole VM is deleted! I am happy to see that there are pointers on the links Mohammadreza mentioned.

Mohammad Akhlaghi <makhlaghi>
Project Administrator
Tue 04 Aug 2020 02:35:07 PM UTC, original submission:  

Continuous integration (CI) is a system that is widely used in software development projects. In summary, with this system, a virtual machine is created to build a project using a script upon every commit on many different systems automatically.

It thus removes the need for us to manually try the build on our different OSs. Mohammadreza has already started setting up a TravisCI system to test Maneage on Ubuntu and macOS in task #15698 (Queue for adding new software). In particular Comment 7 of that task.

But the issue of CI is mostly independent of that task (which is more about the design and strategy of a queue system to update software). So I am defining this task here and let us focus the CI-related discussions here.

Mohammad Akhlaghi <makhlaghi>
Project Administrator

 

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by mkhellat (Posted a comment)
  • -email is unavailable- added by makhlaghi (Submitted the item)
  •  

    Do you think this task is very important?
    If so, you can add your encouragement to it.
    This task has 0 encouragements so far.

    Only logged-in users can vote.

     

     

     

    No changes have been made to this item

    Back to the top


    Powered by Savane 3.5