bugConcurrent Versions System - Bugs: bug #29302, Optimize checkout/update by not...

 
 

bug #29302: Optimize checkout/update by not creating empty directories

Submitter:  Julio Merino <jmmv>
Submitted:  Mon 22 Mar 2010 07:15:05 PM UTC
Votes: 1
 
Category:  None Severity:  3 - Normal
Item Group:  None Status:  None
Privacy:  Public Assigned to:  None
Open/Closed:  Open Release: 
Fixed Release:  None Fixed Feature Release:  None
* Mandatory Fields

Add a New Comment Rich Markup
   

Fri 09 Apr 2010 09:39:35 PM UTC, comment #1: 

That is real design issue and cause performance problems with many folder repository. I think that no need to create empty folder if 0 files belong to the branch you checkout.

Olexiy Buyanskyy <olexiyb>
Mon 22 Mar 2010 07:15:05 PM UTC, original submission:  

When running "checkout -P" or "update -P" on a large tree, it's easy to spot that the cleanup of empty directories at the end of the processing takes a really long time. It is important to realize that these supposedly-empty directories are not really empty: they also contain the CVS control directory and their associated admin files.

The creation of these empty directories and their admin files, coupled with the delayed removal at the end, causes the files that have to stay to end up spread across the disk. The system cannot know what files/directories will be deleted, and as there is a lot of disk churn due to the touching of many files, the file system code must flush those to-be-deleted files to disk.

Seeing this behavior is really easy: e.g. one just needs to check out a fresh copy of the pkgsrc CVS tree (see http://www.pkgsrc.org/ for the address) and keep updating it a few times. Timing these operations across the different runs will result in increasing numbers. Add to the equation a slow laptop-sized hard disk and it becomes an "annoying" issue.

Some projects like the one mentioned above cannot move away from CVS anytime soon, so they force all of their developers to be tied to the slowness of these operations.

I would like to request an optimization to the checkout/update code: instead of creating directories right when they are seen from the server, delay their creation until the first non-admin file, non-directory in them gets created. E.g. instead of doing the pruning at the end, avoid the pruning overall by not creating the directories in the first place. This would avoid unnecessary accesses to the file system and speed up the operation of cvs checkout/update by a great deal.

Julio Merino <jmmv>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by olexiyb (Posted a comment)
  • -email is unavailable- added by olexiyb (Voted in favor of this item)
  • -email is unavailable- added by jmmv (Submitted the item)
  •  

    There is 1 vote so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only logged-in users can vote.

     

    Follows 1 latest change.

    Date Changed by Updated Field Previous Value => Replaced by
    2010-04-09 olexiyb Carbon-Copy- Added olexiyb

    Back to the top

    Powered by Savane 3.13-f8d8.
    Corresponding source code