Maneage - Tasks: task #15262, Source code downloads and Job...
You are not allowed to post comments on this tracker with your current authentication level.
task #15262: Source code downloads and Job manager
Submitter: | Mohammadreza Khellat <mkhellat> | ||
Submitted: | Wed 01 May 2019 05:21:10 AM UTC | ||
Should Start On: | Wed 01 May 2019 12:00:00 AM UTC | Should be Finished on: | Wed 01 May 2019 12:00:00 AM UTC |
Category: | Software | Priority: | 7 - High |
Status: | In Progress | Privacy: | Public |
Assigned to: | mkhellat | Percent Complete: | 80% |
Open/Closed: | Open | Effort: | 0.00 |
Jump to the original submission
Fri 02 Aug 2019 01:17:53 AM UTC, comment #28: |
Mohammad Akhlaghi <makhlaghi> |
Mon 08 Jul 2019 04:18:14 AM UTC, comment #27: Mohammad,yeah i remember our lengthy discussions around this.
|
Mohammadreza Khellat <mkhellat> |
Sun 07 Jul 2019 10:08:38 PM UTC, comment #26: Great! Thanks a lot Mohammad-reza! This commit message is perfectly consistent with the other commit messages in this project (and many other projects) and nicely describes the context (purpose) of the solutions.
|
Mohammad Akhlaghi <makhlaghi> |
Sun 07 Jul 2019 08:40:01 PM UTC, comment #25: All the nice suggested formatting is done, my old friend.
|
Mohammadreza Khellat <mkhellat> |
Sun 07 Jul 2019 08:14:20 PM UTC, comment #24: Not relevant to my comment on 75 character!! Any terminal would wrap the line!! And there is a reason for that!! So you could have lines with 80+ character length.
|
Mohammadreza Khellat <mkhellat> |
Sun 07 Jul 2019 08:02:26 PM UTC, comment #23: These have been the conventions we have been using until now (they just weren't actually written down, but if you run `git log' you'll see them implemented consistently).
|
Mohammad Akhlaghi <makhlaghi> |
Sun 07 Jul 2019 07:28:40 PM UTC, comment #22: Now getting back to the main point:
|
Mohammadreza Khellat <mkhellat> |
Sun 07 Jul 2019 07:18:19 PM UTC, comment #21: As for the suggestion of discussing my disagreement with the laid down convention, there are two points:
|
Mohammadreza Khellat <mkhellat> |
Sun 07 Jul 2019 07:14:04 PM UTC, comment #20: Haha, yes, no doubt about low-level conventions required for a collaboration.
|
Mohammadreza Khellat <mkhellat> |
Sun 07 Jul 2019 06:45:06 PM UTC, comment #19: In short: In a collaboration, mere understandability is not enough, we need a certain level of unity also (as well as understandability and other things). |
Mohammad Akhlaghi <makhlaghi> |
Sun 07 Jul 2019 06:41:09 PM UTC, comment #18: Given your disdain for standards, I am happy that you have at agreed to use English as the "standard" language ;-)!
|
Mohammad Akhlaghi <makhlaghi> |
Sun 07 Jul 2019 06:22:38 PM UTC, comment #17: Well, the comments are written in english and as you said well explained which means that they are understandable.
Only your number 2 should stand in its existing state. In my opinion, the rest should be gone. |
Mohammadreza Khellat <mkhellat> |
Sun 07 Jul 2019 05:52:06 PM UTC, comment #16: Yes, it is very well explained. The explanation isn't the problem ;-).
|
Mohammad Akhlaghi <makhlaghi> |
Sun 07 Jul 2019 05:43:21 PM UTC, comment #15: Hi Mohammad,
|
Mohammadreza Khellat <mkhellat> |
Sun 07 Jul 2019 05:02:32 PM UTC, comment #14: Great! Thanks. I am now able to see the commit.
|
Mohammad Akhlaghi <makhlaghi> |
Sun 07 Jul 2019 04:17:04 PM UTC, comment #13: Hi Mohammad and Raul,
|
Mohammadreza Khellat <mkhellat> |
Sun 07 Jul 2019 03:09:41 PM UTC, comment #12: Hi Mohammad-reza, I couldn't open that page. Has it been built as a "private" project? If so, you should add me (atleast) as a member so I can access and pull from it. |
Mohammad Akhlaghi <makhlaghi> |
Tue 25 Jun 2019 03:31:49 AM UTC, comment #11: hi guys,
|
Mohammadreza Khellat <mkhellat> |
Sun 23 Jun 2019 11:16:18 AM UTC, comment #10: Great! Thanks a lot Mohammad-reza.
|
Mohammad Akhlaghi <makhlaghi> |
Sun 23 Jun 2019 11:04:31 AM UTC, comment #9: The relevant changes as well as a new logging and messaging is available in the branch: task-tarball_downloader-15262.
|
Mohammadreza Khellat <mkhellat> |
Mon 27 May 2019 12:24:44 PM UTC, comment #8: You are right! I didn't notice that you had used `--user-agent' already, thanks for the correction.
|
Mohammad Akhlaghi <makhlaghi> |
Mon 27 May 2019 08:15:52 AM UTC, comment #7: Mohammad, please check my first post, you could see the --user-agent flag, there.
|
Mohammadreza Khellat <mkhellat> |
Sun 26 May 2019 09:50:36 AM UTC, comment #6: Thanks for the check. Did you also try the `--user-agent' option to Wget (from my previous message on this task)?
|
Mohammad Akhlaghi <makhlaghi> |
Sun 26 May 2019 09:32:33 AM UTC, comment #5: Okay, checked it. It is related to geoprotection. so if we wanna download with curl or wget we need an anonymizor.
|
Mohammadreza Khellat <mkhellat> |
Sun 26 May 2019 08:19:32 AM UTC, comment #4: Hey guy, sorry for the late reply.
|
Mohammadreza Khellat <mkhellat> |
Wed 01 May 2019 09:07:59 PM UTC, comment #3: Just as a confirmation:
|
Mohammad Akhlaghi <makhlaghi> |
Wed 01 May 2019 11:46:30 AM UTC, comment #2: You see the complication comes from the hosting web-server.
And here is the main msg, so you would be happy ;) and it would look nicer: The source code download issues are really annoying.
|
Mohammadreza Khellat <mkhellat> |
Wed 01 May 2019 10:48:11 AM UTC, comment #1: These are good points. Thanks ;-).
|
Mohammad Akhlaghi <makhlaghi> |
Wed 01 May 2019 05:21:10 AM UTC, original submission:
The source code download issues are really annoying.
|
Mohammadreza Khellat <mkhellat> |
No files currently attached
Depends on the following items: None found
Items that depend on this one: None found
There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.
Follow 9 latest changes.
Date | Changed by | Updated Field | Previous Value | => | Replaced by |
---|---|---|---|---|---|
2019-08-02 | makhlaghi | Status | Postponed | In Progress | |
Percent Complete | 0% | 80% | |||
Assigned to | None | mkhellat | |||
2019-08-01 | mkhellat | Status | In Progress | Postponed | |
2019-07-08 | mkhellat | Percent Complete | 80% | 0% | |
2019-07-07 | makhlaghi | Percent Complete | 0% | 80% | |
2019-05-26 | mkhellat | Status | Postponed | In Progress | |
2019-05-01 | mkhellat | Carbon-Copy | - | Added makhlaghi | |
Carbon-Copy | - | Added infantesainz |
Powered by Savane 3.13-758e.
Corresponding source code
Fortunately today I had some time to inspect this work and try merging it. Having logs for the downloads and builds is GREAT! Thanks a lot for doing the job Mohammad-reza.
Some minor corrections were done on the branch (as fully described in the commit message). Then I imported the `master' branch into this branch to have all the new features since this branch diverged.
All this work is now pushed onto my own fork's `makelog' branch, but its not yet merged with the master branch as described below: https://gitlab.com/makhlaghi/reproducible-paper/tree/makelog
However, while doing some tests, I noticed a big problem: if the building of a program fails, the processing continues!
Until now, we solved this problem like this: we used `&&' between the main building commands until the final target is built. Therefore, as soon as one command failed, it wouldn't go onto the rest and it would stop. But here, because we need to pipe the outputs to `tee' the final status of the command is the returned value of the last command in the pipe (the `tee', which always succeeds).
For example if the `./configure' fails, it will go onto `make', `make install', and finally build the program's main target, while it shouldn't.
One solution to this problem is when we know the system has GNU Bash: with the `-o pipefail' setting (for example by running `set -o pipefail', we can tell Bash to return a failed result if any of the pipe'e commands fail. However, for the first set of software in the template (until Bash is built), we can't assume the existence of Bash.
I searched a lot for a solution, but I don't have much time and had to stop after a few hours. In particular the `script' command seemed useful, but it messed with the terminal and parallel building, causing a crash.
So the updated branch is now available on my fork until we can find a solution to this problem.
Having download and build logs is a wonderful feature, and we have done most of the job, so I have marked it as 80% complete. I hope we can find a solution to this problem soon and merge it into the `master' branch.