bugGNU Parallel - Bugs: bug #65660, ETA is incorrect when running with...

 
 

bug #65660: ETA is incorrect when running with -xargs

Submitter:  None
Submitted:  Mon 29 Apr 2024 04:06:30 PM UTC
   
 
Category:  None Severity:  3 - Normal
Item Group:  None Status:  Confirmed
Privacy:  Public Assigned to:  tange
Open/Closed:  Open
* Mandatory Fields

Add a New Comment Rich Markup
   

Mon 29 Apr 2024 04:18:04 PM UTC, comment #1: 

This is my bug (sorry should not have filed as anonymous, new to savannah).  Happy to provide more information.

Luke Sandberg <lukesandberg>
Mon 29 Apr 2024 04:06:30 PM UTC, original submission:  

I am running a command like

```
</tmp/large-file parallel --bar --noswap --nice 17 -j100  --xargs <CMD>

```

The command is happy to take as many args as will fit on the command line (and there is some benefit to batching), however it appears that the ETA suggested by `--bar` is inaccurate.

for one run with about 56K inputs, the penultimate ETA in the progress output was

```
0% 82:56701=2d11h23m56s
```

quickly followed by

```
0% 111:56672=12m27s ....
```

and then it exited successfully.  So the `12m27s` is correct, but the `0%` is not and on the previous output the `2d11h23m56s` was wildly off.... so the `--bar` output wasn't helpful.


I'm guessing that the ETA logic isn't taking int account the fact that multiple arguments are getting consumed for each command (in this case about 500 args per job).  So the ETA and percentage were off by a factor of 500.


```
$ parallel --version
GNU parallel 20231122
Copyright (C) 2007-2023 Ole Tange, http://ole.tange.dk and Free Software
Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <https://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
GNU parallel comes with no warranty.

Web site: https://www.gnu.org/software/parallel

When using programs that use GNU Parallel to process data for publication
please cite as described in the manpage.
```



Anonymous

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by tange (Updated the item)
  • -email is unavailable- added by lukesandberg (Posted a comment)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only logged-in users can vote.

     

    Follow 2 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2024-05-02 tange StatusNone Confirmed
        Assigned toNone tange

    Back to the top

    Powered by Savane 3.13-4b48.
    Corresponding source code