{"thread":{"id":"7538","subject":"[PATCH 09/13] drop objects larger than --blob-limit if specified","startedAt":"2007-04-05T22:36:52Z","lastAt":"2007-04-08T01:53:59Z","messageCount":11,"participants":["Dana How","Nicolas Pitre","Linus Torvalds","Junio C Hamano","David Lang"],"isPatch":true,"patchVersion":1,"patchTotal":13},"messages":[{"id":"38750","messageId":"56b7f5510704051536s7de9638fs8cd811d580f6a7dc@mail.gmail.com","threadId":"7538","inReplyTo":null,"subject":"[PATCH 09/13] drop objects larger than --blob-limit if specified","fromName":"Dana How","fromEmail":"danahow@gmail.com","sentAt":"2007-04-05T22:36:52Z","receivedAt":"2007-04-05T22:36:52Z","isPatch":true,"sender":{"key":"danahow@gmail.com","avatar":null},"body":"---\n builtin-pack-objects.c |    1 +\n 1 files changed, 1 insertions(+), 0 deletions(-)\n\n-- \nDana L. How  danahow@gmail.com  +1 650 804 5991 cell\n\n\nFrom 62532de2a8c19ca8d8bfb2e4bc02bff39bcc7ca8 Mon Sep 17 00:00:00 2001\nFrom: Dana How <how@deathvalley.cswitch.com>\nDate: Thu, 5 Apr 2007 14:06:05 -0700\nSubject: [PATCH 09/13] drop objects larger than --blob-limit if specified\n\n---\n builtin-pack-objects.c |    1 +\n 1 files changed, 1 insertions(+), 0 deletions(-)\n\ndiff --git a/builtin-pack-objects.c b/builtin-pack-objects.c\nindex 37b0150..ccc2d15 100644\n--- a/builtin-pack-objects.c\n+++ b/builtin-pack-objects.c\n@@ -1116,6 +1116,7 @@ static void check_object(struct object_entry *entry)\n \t}\n \n \tentry->type = sha1_object_info(entry->sha1, &entry->size);\n+\tnr_skipped += entry->no_write = blob_limit && (unsigned long)entry->size >= blob_limit;\n \tif (entry->type < 0)\n \t\tdie(\"unable to get type of object %s\",\n \t\t    sha1_to_hex(entry->sha1));\n-- \n1.5.1.rc2.18.g9c88-dirty\n\n"},{"id":"38770","messageId":"alpine.LFD.0.98.0704052103410.28181@xanadu.home","threadId":"7538","inReplyTo":"56b7f5510704051536s7de9638fs8cd811d580f6a7dc@mail.gmail.com","subject":"Re: [PATCH 09/13] drop objects larger than --blob-limit if specified","fromName":"Nicolas Pitre","fromEmail":"nico@cam.org","sentAt":"2007-04-06T01:04:11Z","receivedAt":"2007-04-06T01:04:11Z","isPatch":true,"sender":{"key":"nico@fluxnic.net","avatar":"https://avatars.githubusercontent.com/u/702790?v=4"},"body":"\nI still consider this feature to make no sense.\n\n\nNicolas\n"},{"id":"38772","messageId":"56b7f5510704051919v7daac590m6ac52c4fcabd5321@mail.gmail.com","threadId":"7538","inReplyTo":"alpine.LFD.0.98.0704052103410.28181@xanadu.home","subject":"Re: [PATCH 09/13] drop objects larger than --blob-limit if specified","fromName":"Dana How","fromEmail":"danahow@gmail.com","sentAt":"2007-04-06T02:19:39Z","receivedAt":"2007-04-06T02:19:39Z","isPatch":true,"sender":{"key":"danahow@gmail.com","avatar":null},"body":"On 4/5/07, Nicolas Pitre <nico@cam.org> wrote:\n> I still consider this feature to make no sense.\n\nWell, suppose I'm packing my 55GB of data into 2GB\npackfiles.  There seemed to be some agreement that\nlimiting packfile size was useful.  700MB is another example.\n\nNow,  suppose there is an object whose packing would\nresult in a packfile larger than the limit.  What should we do?\n\n(1) Refuse to run.  This solution means I can't pack my repository.\n(2) Pack the object any way and let the packfile size exceed\n     my specification.  Ignoring a clear preference from the user\n     doesn't seem good.\n(3) Pack the object by itself in its own pack. This is better than the\n     previous since I haven't wrapped up any small object in a pack\n     whose size I dont't want to deal with.  The resulting pack is too big,\n     but the original object was also too big so at least I haven't made\n     the problem worse.  But why bother wrapping the object so?\n     I just made the list of packs to look through longer for every access,\n     instead of leaving the big object in the objects/xx directories which\n     are already used to handle exceptions (usually meaning more recent).\n     In my 55GB example, I have 9 jumbo objects, and this solution\n     would more than double the number of packs to step through.\n     Having them randomly placed in 256 subdirectories seems better.\n(4) Just leave the jumbo object by itself, unpacked.\n\nWhat do you think?\n\nThanks,\n-- \nDana L. How  danahow@gmail.com  +1 650 804 5991 cell\n"},{"id":"38775","messageId":"alpine.LFD.0.98.0704052257200.28181@xanadu.home","threadId":"7538","inReplyTo":"56b7f5510704051919v7daac590m6ac52c4fcabd5321@mail.gmail.com","subject":"Re: [PATCH 09/13] drop objects larger than --blob-limit if specified","fromName":"Nicolas Pitre","fromEmail":"nico@cam.org","sentAt":"2007-04-06T03:20:44Z","receivedAt":"2007-04-06T03:20:44Z","isPatch":true,"sender":{"key":"nico@fluxnic.net","avatar":"https://avatars.githubusercontent.com/u/702790?v=4"},"body":"On Thu, 5 Apr 2007, Dana How wrote:\n\n> On 4/5/07, Nicolas Pitre <nico@cam.org> wrote:\n> > I still consider this feature to make no sense.\n> \n> Well, suppose I'm packing my 55GB of data into 2GB\n> packfiles.  There seemed to be some agreement that\n> limiting packfile size was useful.  700MB is another example.\n> \n> Now,  suppose there is an object whose packing would\n> result in a packfile larger than the limit.  What should we do?\n\nYou error out.\n\n> (1) Refuse to run.  This solution means I can't pack my repository.\n\nExactly.  If you want packs not to be larger than 10MB and you have a \n100MB blob then you are screwed.  Just lift your pack size limit in such \ncase.\n\n> (2) Pack the object any way and let the packfile size exceed\n>     my specification.  Ignoring a clear preference from the user\n>     doesn't seem good.\n\nIt is not indeed.\n\n> (3) Pack the object by itself in its own pack. This is better than the\n>     previous since I haven't wrapped up any small object in a pack\n>     whose size I dont't want to deal with.  The resulting pack is too big,\n>     but the original object was also too big so at least I haven't made\n>     the problem worse.  But why bother wrapping the object so?\n>     I just made the list of packs to look through longer for every access,\n>     instead of leaving the big object in the objects/xx directories which\n>     are already used to handle exceptions (usually meaning more recent).\n>     In my 55GB example, I have 9 jumbo objects, and this solution\n>     would more than double the number of packs to step through.\n>     Having them randomly placed in 256 subdirectories seems better.\n\nYou forget about the case where those jumbo blobs could delta well \nagainst each other.  That means that one pack could possibly contain \nthose 9 objects because 8 of them are tiny deltas against the first big \none.\n\n> (4) Just leave the jumbo object by itself, unpacked.\n\nHmmmmm.\n\n> What do you think?\n\nLet's say I wouldn't mind much if it was implemented differently.  The \nobjects array is probably the biggest cost in terms of memory usage for \npack-objects.  When you have 4 milions objects like in the kde repo that \nmeans each new field you add will cost between 4 to 16 MB of memory.  I \nthink this is too big a cost for filtering out a couple big objects once \nin a while.\n\nInstead, I think you should apply the filtering in add_object_entry() \ndirectly and simply skip adding the unwanted object to the list \naltogether.\n\n\nNicolas\n"},{"id":"38792","messageId":"Pine.LNX.4.64.0704060845120.6730@woody.linux-foundation.org","threadId":"7538","inReplyTo":"alpine.LFD.0.98.0704052257200.28181@xanadu.home","subject":"Re: [PATCH 09/13] drop objects larger than --blob-limit if specified","fromName":"Linus Torvalds","fromEmail":"torvalds@linux-foundation.org","sentAt":"2007-04-06T15:49:22Z","receivedAt":"2007-04-06T15:49:22Z","isPatch":true,"sender":{"key":"torvalds@linux-foundation.org","avatar":"https://avatars.githubusercontent.com/u/1024025?v=4"},"body":"\n\nOn Thu, 5 Apr 2007, Nicolas Pitre wrote:\n> \n> > (2) Pack the object any way and let the packfile size exceed\n> >     my specification.  Ignoring a clear preference from the user\n> >     doesn't seem good.\n> \n> It is not indeed.\n\nWell, I think there is an easy solution.\n\nJust go back and say that when the user limits the size, it limits the \noffset at which objects can *start*.\n\nNot only is that the only thing that the index file itself cares about, it \nalso means that\n\n - you don't ever need to undo anything at all (because you know the \n   starting offset) when you begin packing a new object.\n\n   This should simplify your patch a lot.\n\n - the object size issue just goes away. Sure, the pack-file limit looks a \n   bit strange (it's no longer a hard limit on the *size* of the \n   pack-file, just on the object offsets), but let's face it, we really \n   really don't care.\n\nAnd in practice, by setting the pack-file limit to 2GB (or even 1GB), you \nnever ever have to worry about the 32-bit filesystem limits any more, \nunless your repository is fundamentally so screwed that you simply \n*cannot* reporesent it well on something like FATFS (ie any object that is \n2GB in size will probably have a blob that is even bigger, and FATFS \nalready has problems with it).\n\nSo in practice, just limiting the index offsets is what you really want \nanyway.\n\n\t\tLinus\n"},{"id":"38795","messageId":"56b7f5510704061109n2878a221p391b7c3edba89c63@mail.gmail.com","threadId":"7538","inReplyTo":"Pine.LNX.4.64.0704060845120.6730@woody.linux-foundation.org","subject":"Re: [PATCH 09/13] drop objects larger than --blob-limit if specified","fromName":"Dana How","fromEmail":"danahow@gmail.com","sentAt":"2007-04-06T18:09:16Z","receivedAt":"2007-04-06T18:09:16Z","isPatch":true,"sender":{"key":"danahow@gmail.com","avatar":null},"body":"On 4/6/07, Linus Torvalds <torvalds@linux-foundation.org> wrote:\n> On Thu, 5 Apr 2007, Nicolas Pitre wrote:\n> > > (2) Pack the object any way and let the packfile size exceed\n> > >     my specification.  Ignoring a clear preference from the user\n> > >     doesn't seem good.\n> > It is not indeed.\n>\n> Well, I think there is an easy solution.\n>\n> Just go back and say that when the user limits the size, it limits the\n> offset at which objects can *start*.\nYes,  this is what my original, unsplit patch did in order to support\nthe --pack-limit && --stdout combination, since lseek doesn't\nalways work on stdout.\n\n> Not only is that the only thing that the index file itself cares about, it\n> also means that\n>\n>  - you don't ever need to undo anything at all (because you know the\n>    starting offset) when you begin packing a new object.\nYes, that was true.\n\n>    This should simplify your patch a lot.\nIt removes the calls to, and the additions of, sha1mark() and sha1undo()\nin csum-file.[ch].  The changes to builtin-pack-objects.c are almost\nthe same -- an \"if\" moves from after the write_object() to before.\n\n>  - the object size issue just goes away. Sure, the pack-file limit looks a\n>    bit strange (it's no longer a hard limit on the *size* of the\n>    pack-file, just on the object offsets), but let's face it, we really\n>    really don't care.\n>\n> And in practice, by setting the pack-file limit to 2GB (or even 1GB), you\n> never ever have to worry about the 32-bit filesystem limits any more,\n> unless your repository is fundamentally so screwed that you simply\n> *cannot* reporesent it well on something like FATFS (ie any object that is\n> 2GB in size will probably have a blob that is even bigger, and FATFS\n> already has problems with it).\n>\n> So in practice, just limiting the index offsets is what you really want\n> anyway.\nI agree with your arguments, but packs have different uses\nand to me there are several differing (non-)reasons for --pack-limit.\n* To split packs into smaller chunks when the .pack format is used as a\n  communications protocol. But the discussion has converged to\n  \"we're not going to split packs at the transmitter\". I agree with this\n  conclusion,  since it would make the total transmission larger (loss\nof deltas).\n  Since no .idx file is sent there is no requirement for --pack-limit here.\n* To avoid offsets larger than 31 bits in .idx files.  Your proposal,\n  and what I was doing for --pack-limit && --stdout, is sufficient to\naddress this.\n* Avoiding (e.g.) 2GB+ files when none already exist in the repository --\n  either the filesystem doesn't support anything beyond the limit,\n  or we don't want to use a >31b off_t with mmap.  (Perhaps\n  the latter case is completely avoided by some glibc 64b trickery,\n  but is that always true?)  Only the write rollback approach can address this.\n* Creating .pack files that fit on e.g. CDROM etc.\nThe 2nd and 3rd cases are what I'm thinking about,\nwhich is why the first version of my patch did both.\n\nAnyway, now that I understand Nicolas's issues with the patchset,\nand realize I agree with his concerns, I'm going to let this percolate\na little bit until I've got the least number of added behaviors and\ncommand line options.  At the moment I'm leaning towards killing my\n--blob-limit idea, keeping --pack-limit based on rollback but with the\nadditional twist that the first object written to a packfile is never\n\"rolled back\" [meaning (a) everything is packed to make Nicolas happy,\n(b) any illegally-sized pack has only one object and could be removed\nto make me happy, and (c) my patchset has one less bug], and\nadding --index-limit which is what normal people might use on a large\nrepository [and could be made the default later].  So I think what you\ndiscuss is the most common use for limiting.\n\nThanks,\n-- \nDana L. How  danahow@gmail.com  +1 650 804 5991 cell\n"},{"id":"38794","messageId":"alpine.LFD.0.98.0704061511390.28181@xanadu.home","threadId":"7538","inReplyTo":"56b7f5510704061109n2878a221p391b7c3edba89c63@mail.gmail.com","subject":"Re: [PATCH 09/13] drop objects larger than --blob-limit if specified","fromName":"Nicolas Pitre","fromEmail":"nico@cam.org","sentAt":"2007-04-06T19:21:24Z","receivedAt":"2007-04-06T19:21:24Z","isPatch":true,"sender":{"key":"nico@fluxnic.net","avatar":"https://avatars.githubusercontent.com/u/702790?v=4"},"body":"On Fri, 6 Apr 2007, Dana How wrote:\n\n> * To avoid offsets larger than 31 bits in .idx files.  Your proposal,\n>  and what I was doing for --pack-limit && --stdout, is sufficient to\n> address this.\n\nBTW I have code for an index version 2 that allows for 64-bit offsets.  \nIt also adds a CRC32 for each object data as stored in the pack to make \ndata reuse safer when repacking.  I only have to fix a couple loose ends \nbefore sending patches out.\n\nThe new index only uses 64-bit offsets when it has to.  Meaning that \nobjects in front of a pack which usually are the most recent ones won't \npay the overhead of a larger offset table at run time.\n\n\nNicolas\n"},{"id":"38796","messageId":"Pine.LNX.4.64.0704061214230.6730@woody.linux-foundation.org","threadId":"7538","inReplyTo":"56b7f5510704061109n2878a221p391b7c3edba89c63@mail.gmail.com","subject":"Re: [PATCH 09/13] drop objects larger than --blob-limit if specified","fromName":"Linus Torvalds","fromEmail":"torvalds@linux-foundation.org","sentAt":"2007-04-06T19:24:53Z","receivedAt":"2007-04-06T19:24:53Z","isPatch":true,"sender":{"key":"torvalds@linux-foundation.org","avatar":"https://avatars.githubusercontent.com/u/1024025?v=4"},"body":"\n\nOn Fri, 6 Apr 2007, Dana How wrote:\n>\n> I agree with your arguments, but packs have different uses\n> and to me there are several differing (non-)reasons for --pack-limit.\n> * To split packs into smaller chunks when the .pack format is used as a\n>  communications protocol. But the discussion has converged to\n>  \"we're not going to split packs at the transmitter\". I agree with this\n>  conclusion,  since it would make the total transmission larger (loss\n>  of deltas).\n\nI don't agree with it, since I don't think the delta advantage is \nnoticeable once packs are \"big enough\". \n\nSplitting on the sending side means that it needs to split just in one \nplace, rather than have the same logic in two different places.\n\nI don't think it really \"convered\" on anything, I think the discussion \njust didn't continue along that thing.\n\n>  Since no .idx file is sent there is no requirement for --pack-limit here.\n\nI really don't think the idx file is the only - or even primary - reason. \nWe need *some* size limiter for a lot of reasons. If the idx file was the \nonly reason, we should just switch to a new index format and forget about \nit.\n\nBut since there are *other* reasons that cannot go away, that doesn't \nobviate the need for size limits, so while I think the idx one is the \n*first* reason to do this, I don't think it is really the most important \none, exactly because the idx file we *could* solve other ways.\n\n> * To avoid offsets larger than 31 bits in .idx files.  Your proposal,\n>  and what I was doing for --pack-limit && --stdout, is sufficient to\n>  address this.\n\nRight. \n\n> * Avoiding (e.g.) 2GB+ files when none already exist in the repository --\n>  either the filesystem doesn't support anything beyond the limit,\n>  or we don't want to use a >31b off_t with mmap.  (Perhaps\n>  the latter case is completely avoided by some glibc 64b trickery,\n>  but is that always true?)  Only the write rollback approach can address this.\n\nI disagree violently. \n\nIN THEORY only write rollback can address that. But \"theory\" is not \npractice, and anybody who thinks that theory is even *relevant* here is \nmissing the big picture.\n\nIf you use FATFS as the place to store your objects, it's really easy to \njust say: you can't have objects bigger than 2GB. That's the real life \nsolution. The \"theory\" that you could have pack-files larger than 32 bits \nis just about as relevant as quantum mechanics is to designing the tensile \nstrength of a sky-scraper. Sure, *in*theory* quantum mechanics matters, \nbut in practice, you'd be crazy if you tried to convince anybody that it \nshould be done using QM rather than traditional means.\n\nQuite frankly, if you have objects that compress to >2GB, you're going to \nbe in so much pain even on *other* filesystems, that I seriously doubt \nyou'd want to track them using git anyway. And even if you want to, just \ntell people: don't use FATFS, because this repository is insane.\n\n> * Creating .pack files that fit on e.g. CDROM etc.\n> The 2nd and 3rd cases are what I'm thinking about,\n> which is why the first version of my patch did both.\n\nAgain, *in*practice*, for any sane situation, if you want to fit things on \na CD-ROM, just give a limit of 600MB, and I can pretty much guarantee that \nyou'll see a slop of just a percent or two for any realistic setup. And if \nit goes up to 660MB, you'll still fit on any CD.\n\nSo please ignore theory, when theory isn't relevant. Designing for \nsomething that practically cannot happen sounds pretty wrong.\n\n(Of course, since you have a 55GB archive, you probably have an insane \nthing in the first place. My condoleances if so ;)\n\n\t\tLinus\n"},{"id":"38797","messageId":"alpine.LFD.0.98.0704061604160.28181@xanadu.home","threadId":"7538","inReplyTo":"56b7f5510704061109n2878a221p391b7c3edba89c63@mail.gmail.com","subject":"Re: [PATCH 09/13] drop objects larger than --blob-limit if specified","fromName":"Nicolas Pitre","fromEmail":"nico@cam.org","sentAt":"2007-04-06T20:12:57Z","receivedAt":"2007-04-06T20:12:57Z","isPatch":true,"sender":{"key":"nico@fluxnic.net","avatar":"https://avatars.githubusercontent.com/u/702790?v=4"},"body":"On Fri, 6 Apr 2007, Dana How wrote:\n\n> * Avoiding (e.g.) 2GB+ files when none already exist in the repository --\n>  either the filesystem doesn't support anything beyond the limit,\n>  or we don't want to use a >31b off_t with mmap.  (Perhaps\n>  the latter case is completely avoided by some glibc 64b trickery,\n>  but is that always true?)  Only the write rollback approach can address this.\n\nIf the filesystem doesn't support anything beyond the limit, you cannot \nperform any rollback either.  Because if you roll back that's because \nyou crossed the limit already.  But you're not supposed to even be able \nto cross that limit in the first place if the filesystem is limited.\n\n\nNicolas\n"},{"id":"38814","messageId":"7v1wixjfpo.fsf@assigned-by-dhcp.cox.net","threadId":"7538","inReplyTo":"Pine.LNX.4.64.0704061214230.6730@woody.linux-foundation.org","subject":"Re: [PATCH 09/13] drop objects larger than --blob-limit if specified","fromName":"Junio C Hamano","fromEmail":"junkio@cox.net","sentAt":"2007-04-06T22:33:23Z","receivedAt":"2007-04-06T22:33:23Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Linus Torvalds <torvalds@linux-foundation.org> writes:\n\n> On Fri, 6 Apr 2007, Dana How wrote:\n>> ...\n>> * Avoiding (e.g.) 2GB+ files when none already exist in the repository --\n>>  either the filesystem doesn't support anything beyond the limit,\n>>  or we don't want to use a >31b off_t with mmap.  (Perhaps\n>>  the latter case is completely avoided by some glibc 64b trickery,\n>>  but is that always true?)  Only the write rollback approach can address this.\n>\n> I disagree violently. \n>\n> IN THEORY only write rollback can address that. But \"theory\" is not \n> practice, and anybody who thinks that theory is even *relevant* here is \n> missing the big picture.\n\nAlthough I agree the \"starting offset\" is a very good\ncompromise, if we talk about THEORY, you should be able to\nnotice you are going to go beyond the limit inside write_one().\nAt that point, we know everything we need to know about all\nobjects that might be involved in writing that one object: if\nthey have been written out, what their offsets are, and how big\ntheir in-pack representation will be.\n\nAnd I agree with Nico that rollback after a failed write beyond\nthe boundary may not work correctly, so if we really want to do\nthis safely and sanely while satisfying Dana's desire to have a\nhard limit, I think something like this is needed:\n\n - use \"starting offset\" to decide when to split;\n\n - introduce a \"hard limit\", perhaps optionally, to make sure\n   that the result of writing out a packfile does not overstep\n   that limit (i.e. the last object written below the \"starting\n   offset limit\" might make the pack go over 700MB).\n\nwhich means you would specify 600 as starting offset limit and\n680 (or something like that) as the hard tail offset limit\n\n> Again, *in*practice*, for any sane situation, if you want to fit things on \n> a CD-ROM, just give a limit of 600MB, and I can pretty much guarantee that \n> you'll see a slop of just a percent or two for any realistic setup. And if \n> it goes up to 660MB, you'll still fit on any CD.\n\nif you really care the result fits on a CD.\n"},{"id":"38872","messageId":"Pine.LNX.4.63.0704071846340.25121@qynat.qvtvafvgr.pbz","threadId":"7538","inReplyTo":"7v1wixjfpo.fsf@assigned-by-dhcp.cox.net","subject":"Re: [PATCH 09/13] drop objects larger than --blob-limit if specified","fromName":"David Lang","fromEmail":"david.lang@digitalinsight.com","sentAt":"2007-04-08T01:53:59Z","receivedAt":"2007-04-08T01:53:59Z","isPatch":true,"sender":{"key":"david.lang@digitalinsight.com","avatar":null},"body":"On Fri, 6 Apr 2007, Junio C Hamano wrote:\n\n> Linus Torvalds <torvalds@linux-foundation.org> writes:\n>\n>> On Fri, 6 Apr 2007, Dana How wrote:\n>\n> And I agree with Nico that rollback after a failed write beyond\n> the boundary may not work correctly, so if we really want to do\n> this safely and sanely while satisfying Dana's desire to have a\n> hard limit, I think something like this is needed:\n>\n> - use \"starting offset\" to decide when to split;\n>\n> - introduce a \"hard limit\", perhaps optionally, to make sure\n>   that the result of writing out a packfile does not overstep\n>   that limit (i.e. the last object written below the \"starting\n>   offset limit\" might make the pack go over 700MB).\n>\n> which means you would specify 600 as starting offset limit and\n> 680 (or something like that) as the hard tail offset limit\n>\n>> Again, *in*practice*, for any sane situation, if you want to fit things on\n>> a CD-ROM, just give a limit of 600MB, and I can pretty much guarantee that\n>> you'll see a slop of just a percent or two for any realistic setup. And if\n>> it goes up to 660MB, you'll still fit on any CD.\n>\n> if you really care the result fits on a CD.\n\nthere are going to be cases whereeyou have a fixed size, but would really like \nto stream things rather then write a temp file and then send that (one example \nof wanting to stream things, but without the size cap is the network pull where \nwe atart sending things before we've finished figuring out the details of what \nwe are going to send). with the appropriately sized buffer you could stream to a \nCD burner for example, and while this may not make a huge difference in the time \ntaken to make the CD, it will save you a significant amount of disk I/O and \nbuffer space that is now still populated by info that's more useful to the user.\n\nDavid Lang\n"}]}