bugConcurrent Versions System - Bugs: bug #29302, Optimize checkout/update by not...

 
 

bug #29302: Optimize checkout/update by not creating empty directories

Submitted by:  Julio Merino <jmmv>
Submitted on:  Mon 22 Mar 2010 07:15:05 PM UTC  
Votes:  1  
 
Category: NoneSeverity: 3 - Normal
Item Group: NoneStatus: None
Privacy: PublicAssigned to: None
Open/Closed: OpenRelease: 
Fixed Release: NoneFixed Feature Release: None

Add a New Comment (Rich MarkupRich Markup):
   

You are not logged in

Please log in, so followups can be emailed to you.

 

Fri 09 Apr 2010 09:39:35 PM UTC, comment #1:

That is real design issue and cause performance problems with many folder repository. I think that no need to create empty folder if 0 files belong to the branch you checkout.

Olexiy Buyanskyy <olexiyb>
Mon 22 Mar 2010 07:15:05 PM UTC, original submission:

When running "checkout -P" or "update -P" on a large tree, it's easy to spot that the cleanup of empty directories at the end of the processing takes a really long time. It is important to realize that these supposedly-empty directories are not really empty: they also contain the CVS control directory and their associated admin files.

The creation of these empty directories and their admin files, coupled with the delayed removal at the end, causes the files that have to stay to end up spread across the disk. The system cannot know what files/directories will be deleted, and as there is a lot of disk churn due to the touching of many files, the file system code must flush those to-be-deleted files to disk.

Seeing this behavior is really easy: e.g. one just needs to check out a fresh copy of the pkgsrc CVS tree (see http://www.pkgsrc.org/ for the address) and keep updating it a few times. Timing these operations across the different runs will result in increasing numbers. Add to the equation a slow laptop-sized hard disk and it becomes an "annoying" issue.

Some projects like the one mentioned above cannot move away from CVS anytime soon, so they force all of their developers to be tied to the slowness of these operations.

I would like to request an optimization to the checkout/update code: instead of creating directories right when they are seen from the server, delay their creation until the first non-admin file, non-directory in them gets created. E.g. instead of doing the pruning at the end, avoid the pruning overall by not creating the directories in the first place. This would avoid unnecessary accesses to the file system and speed up the operation of cvs checkout/update by a great deal.

Julio Merino <jmmv>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach File(s):
   
   
Comment:
   

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -unavailable- added by olexiyb (Posted a comment)
  • -unavailable- added by olexiyb (Voted in favor of this item)
  • -unavailable- added by jmmv (Submitted the item)
  •  

    Do you think this task is very important?
    If so, you can click here to add your encouragement to it.
    This task has 1 encouragement so far.

    Only logged-in users can vote.

     

    Please enter the title of George Orwell's famous dystopian book (it's a date):

     

     

    Follows 1 latest change.

    Date Changed By Updated Field Previous Value => Replaced By
    Fri 09 Apr 2010 09:37:25 PM UTColexiybCarbon-Copy-=>Added olexiyb

    Back to the top


    Powered by Savane 3.1-cleanup