bugrdiff-backup - Bugs: bug #7592, Feature Request - line buffered...

 
 

bug #7592: Feature Request - line buffered stdio/logs

Submitter:  Anthony Wood <s2156945>
Submitted:  Thu 05 Feb 2004 05:36:55 AM UTC
   
 
Category:  None Severity:  2 - Minor
Item Group:  None Status:  Postponed
Privacy:  Public Assigned to:  None
Open/Closed:  Closed
* Mandatory Fields

Add a New Comment Rich Markup
   

Jump to the original submission

Wed 05 Feb 2020 07:54:39 PM UTC, comment #7: 

This issue has been closed because it doesn't seem critical. If it is important to you, please re-open it under https://github.com/rdiff-backup/rdiff-backup/issues

Eric L. <ericzolf>
Group administrator
Mon 08 Aug 2005 02:33:23 AM UTC, comment #6: 

Oh about webalizer, that's an interesting idea, but I wonder if a web server-like format is right for rdiff-backup's native log files.

There are two goals which may need separate treatments:  1) we may want to know what rdiff-backup is doing at the moment or 2) we may want statistics on what rdiff-backup did in previous sessions.  The answer to #2 is supposed to be the file_statistics file.  Perhaps a webalizerable file could be produced by reading that file.

Unfortunately the file_statistics file is gzipped, so doesn't readily indicate rdiff-backup's current activities.

Benedict Escoto <bescoto>
Group administrator
Mon 08 Aug 2005 02:24:02 AM UTC, comment #5: 

I just changed log.py to flush after each log entry is added (as Anthony Wood suggested a long time ago).  Not sure why I didn't do that earlier.

This doesn't help him determine when a file is being processed though.

Benedict Escoto <bescoto>
Group administrator
Sun 19 Jun 2005 08:56:01 PM UTC, comment #4: 

one option you have is to invoke rdiff-backup with "python -u /path/to/rdiff-backup" ... which makes stdin/out/err totally unbuffered.

i can't figure out how to select line buffering in python... that's what i would do in perl or C rather than completely unbuffered.  if someone knows how to do this let me know...

-dean

dean gaudet <dgaudet>
Group administrator
Mon 08 Mar 2004 05:04:40 AM UTC, comment #3: 

What about the common log file format?
http://www.bacuslabs.com/WsvlCLF.html
you could bugerise around with it a bit so it did what you wanted - ie map the fields it defines to useful things in rdiff-backup and then we can graph stuff with webalizer or something. ie anything that interprets CLF can parse the logs and make something pretty.
A graph of most requested file, size of backups etc would be great and mapping rdiff-backup logs to this format would be awesome.

dave

david kempe <davekempe>
Group administrator
Mon 09 Feb 2004 04:25:47 AM UTC, comment #2: 

I was just going to do really rudimentary parsing, like ripping out the filename.

I don't have any particular needs apart from some sort of indication that process is being made.

The first part of this is flushing stdout.

Machine readable output is a second step, I guess a start of a simple spec would be:

Fraction below defaults to 20, settable with flag --machine-readable-progress-markers or similar

(1) Output the filename when it is started to be processed.
(2) Output 1 progress character for each fraction of the comparison operation (= for same, X for difference).
(3) Output 1 progress character for each fraction of the data transferred.
(4) Output bytes transferred
(5) Output filesize in bytes
(6) Newline

E.g. for one file which has been editted in the middle, the output might be

/home/woody/test.txt ========XXXX========.................... 927 4627

This is obviously human readable also.

(2),(3),(4) might be difficult depending on how librsync is used, (1) + (6) only is sufficient for me.
An output at the beginning of the total size of the data being backed up would enable a total progress bar to work too if (5) was included or implemented by the GUI.

The documentation says --parsable-output only affects the --list-increments

Anthony Wood <s2156945>
Thu 05 Feb 2004 07:35:46 AM UTC, comment #1: 

Are you planning on just setting the verbosity fairly high?  Looking at the source, there actually is a --parsable-output option, but I don't think it does much.  In the past a number of people have started (or said they started) GUIs, but no one has ever finished one to my knowledge.

Anyway, I think it may be a better idea to have output specifically for some program.  It could even be on a separate channel (and not stdout/stdin).  It would help here  if someone wrote up a document detailing exactly what output they are looking for and/or suggested an easy-to-parse yet flexible standard for this output.

Benedict Escoto <bescoto>
Group administrator
Thu 05 Feb 2004 05:36:55 AM UTC, original submission:  

Most other programs which log stuff do it in a line buffered fashion - tailing the backup.log is wierd :-)
I'm writing a GUI for rdiff-backup and it would be nice to be able to do some sort of rudimentary progress by interpretting the log, but it's a bit jerky at the moment :-)

Anthony Wood <s2156945>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by ericzolf (Posted a comment)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only logged-in users can vote.

     

    Follow 4 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2020-02-05 ericzolf StatusNeed Info Postponed
        Open/ClosedOpen Closed
    2005-06-19 dgaudet Severity3 - Normal 2 - Minor
        StatusNone Need Info

    Back to the top

    Powered by Savane 3.13-bb6a.
    Corresponding source code