bugrdiff-backup - Bugs: bug #7592, Feature Request - line buffered...

 
 

bug #7592: Feature Request - line buffered stdio/logs

Submitted by:  Anthony Wood <s2156945>
Submitted on:  Thu 05 Feb 2004 05:36:55 AM UTC  
 
Category: NoneSeverity: 2 - Minor
Item Group: NoneStatus: Need Info
Privacy: PublicAssigned to: None
Open/Closed: Open

Add a New Comment(Rich Markup)
   

You are not logged in

Please log in, so followups can be emailed to you.

 

(Jump to the original submission Jump to the original submission)

Mon 08 Aug 2005 02:33:23 AM UTC, comment #6:

Oh about webalizer, that's an interesting idea, but I wonder if a web server-like format is right for rdiff-backup's native log files.

There are two goals which may need separate treatments: 1) we may want to know what rdiff-backup is doing at the moment or 2) we may want statistics on what rdiff-backup did in previous sessions. The answer to #2 is supposed to be the file_statistics file. Perhaps a webalizerable file could be produced by reading that file.

Unfortunately the file_statistics file is gzipped, so doesn't readily indicate rdiff-backup's current activities.

Benedict Escoto <bescoto>
Project Administrator
Mon 08 Aug 2005 02:24:02 AM UTC, comment #5:

I just changed log.py to flush after each log entry is added (as Anthony Wood suggested a long time ago). Not sure why I didn't do that earlier.

This doesn't help him determine when a file is being processed though.

Benedict Escoto <bescoto>
Project Administrator
Sun 19 Jun 2005 08:56:01 PM UTC, comment #4:

one option you have is to invoke rdiff-backup with "python -u /path/to/rdiff-backup" ... which makes stdin/out/err totally unbuffered.

i can't figure out how to select line buffering in python... that's what i would do in perl or C rather than completely unbuffered. if someone knows how to do this let me know...

-dean

dean gaudet <dgaudet>
Project Administrator
Mon 08 Mar 2004 05:04:40 AM UTC, comment #3:

What about the common log file format?
http://www.bacuslabs.com/WsvlCLF.html
you could bugerise around with it a bit so it did what you wanted - ie map the fields it defines to useful things in rdiff-backup and then we can graph stuff with webalizer or something. ie anything that interprets CLF can parse the logs and make something pretty.
A graph of most requested file, size of backups etc would be great and mapping rdiff-backup logs to this format would be awesome.

dave

david kempe <davekempe>
Project Administrator
Mon 09 Feb 2004 04:25:47 AM UTC, comment #2:

I was just going to do really rudimentary parsing, like ripping out the filename.

I don't have any particular needs apart from some sort of indication that process is being made.

The first part of this is flushing stdout.

Machine readable output is a second step, I guess a start of a simple spec would be:

Fraction below defaults to 20, settable with flag --machine-readable-progress-markers or similar

(1) Output the filename when it is started to be processed.
(2) Output 1 progress character for each fraction of the comparison operation (= for same, X for difference).
(3) Output 1 progress character for each fraction of the data transferred.
(4) Output bytes transferred
(5) Output filesize in bytes
(6) Newline

E.g. for one file which has been editted in the middle, the output might be

/home/woody/test.txt ========XXXX========.................... 927 4627

This is obviously human readable also.

(2),(3),(4) might be difficult depending on how librsync is used, (1) + (6) only is sufficient for me.
An output at the beginning of the total size of the data being backed up would enable a total progress bar to work too if (5) was included or implemented by the GUI.

The documentation says --parsable-output only affects the --list-increments

Anthony Wood <s2156945>
Thu 05 Feb 2004 07:35:46 AM UTC, comment #1:

Are you planning on just setting the verbosity fairly high? Looking at the source, there actually is a --parsable-output option, but I don't think it does much. In the past a number of people have started (or said they started) GUIs, but no one has ever finished one to my knowledge.

Anyway, I think it may be a better idea to have output specifically for some program. It could even be on a separate channel (and not stdout/stdin). It would help here if someone wrote up a document detailing exactly what output they are looking for and/or suggested an easy-to-parse yet flexible standard for this output.

Benedict Escoto <bescoto>
Project Administrator
Thu 05 Feb 2004 05:36:55 AM UTC, original submission:

Most other programs which log stuff do it in a line buffered fashion - tailing the backup.log is wierd :-)
I'm writing a GUI for rdiff-backup and it would be nice to be able to do some sort of rudimentary progress by interpretting the log, but it's a bit jerky at the moment :-)

Anthony Wood <s2156945>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach File(s):
   
   
Comment:
   

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

CC list is empty

 

Do you think this task is very important?
If so, you can click here to add your encouragement to it.
This task has 0 encouragements so far.

Only logged-in users can vote.

 

Please enter the title of George Orwell's famous dystopian book (it's a date):

 

 

Follow 2 latest changes.

Date Changed By Updated Field Previous Value => Replaced By
Sun 19 Jun 2005 08:56:01 PM UTCdgaudetSeverity3 - Normal=>2 - Minor
  StatusNone=>Need Info

Back to the top


Powered by Savane 3.1-cleanup1