bugfastcgi++ - Bugs: bug #34894, poor performance when handling big...

 
 

bug #34894: poor performance when handling big uploads

Submitted by:  Smeesters Olivier <osmeest>
Submitted on:  Fri 25 Nov 2011 01:25:11 PM UTC  
 
Category: NoneSeverity: 3 - Normal
Item Group: NoneStatus: Ready For Test
Privacy: PublicAssigned to: Eddie Carle <eddic>
Open/Closed: Open

Add a New Comment (Rich MarkupRich Markup):
   

You are not logged in

Please log in, so followups can be emailed to you.

 

Sat 26 Nov 2011 01:57:31 AM UTC, comment #1:

Ugh, yeah, that code I wrote is really a piece of junk. I'm really quite ashamed of that. Normally I would just apply the patch but I'd kind of like to try and fix this one myself just for my own "education". I'm going to try and tackle this one and would certainly appreciate any feedback you can offer as I go. This library is in many ways a learning experience for me. I've actually noticed very poor performance on large file uploads but never thought to check the library. Thanks for the report and patch.

Eddie Carle <eddic>
Project AdministratorIn charge of this item.
Fri 25 Nov 2011 01:25:11 PM UTC, original submission:

When uploading a multi-megabyte file through a POST, the library is handling it in a way that's subefficient.

- a new buffer is reallocated for each 64KB chunk received (+ copy of previous content)
- boundary is searched in the whole buffer (not only in the newly received chunk)
- remaining buffer content is copied into another smaller buffer after each part

This results in a very poor performance.

I've patched http.hpp/.cpp to ensure a higher performance:
- buffer allocated as contentLength bytes from start (allows extension if content length is bogus)
- boundary is searched in the end of remainder buffer + new content only)
- all parts are processed before the remaining buffer content is move inside the same buffer

Patch against an older version of fastcgi++ is included.

Smeesters Olivier <osmeest>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach File(s):
   
   
Comment:
   

Attached Files

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -unavailable- added by eddic (Posted a comment)
  • -unavailable- added by osmeest (Submitted the item)
  •  

    Do you think this task is very important?
    If so, you can click here to add your encouragement to it.
    This task has 0 encouragements so far.

    Only logged-in users can vote.

     

    Please enter the title of George Orwell's famous dystopian book (it's a date):

     

     

    Follow 4 latest changes.

    Date Changed By Updated Field Previous Value => Replaced By
    Thu 29 Dec 2011 07:48:55 PM UTCeddicStatusIn Progress=>Ready For Test
    Sat 26 Nov 2011 01:57:31 AM UTCeddicStatusNone=>In Progress
      Assigned toNone=>eddic
    Fri 25 Nov 2011 01:25:11 PM UTCosmeestAttached File-=>Added fastcgipp-post-performance.patch, #24450

    Back to the top


    Powered by Savane 3.1-cleanup