Re: [PATCH 0/4] Honor core.deltaBaseCacheLimit during index-pack

> Nicolas Pitre <nico@cam.org> wrote:
> > 
> > Those delta chains aren't simple chained lists.  They are trees of 
> > deltas where one object might be the base for an unlimited number of 
> > deltas of depth 1, and in turn each of those deltas might constitute the 
> > base for an unlimited number of deltas of depth 2, and so on.
> > 
> > So what the code does is to find out which objects are not deltas but 
> > are the base for a delta.  Then, for each of them, all deltas having 
> > given object for base are found and they are recursively resolved so 
> > each resolved delta is then considered a possible base for more deltas, 
> > etc.  In other words, those deltas are resolved by walking the delta 
> > tree in a "depth first" fashion.
> > 
> > If we discard previous delta bases, we will have to recreate them each 
> > time a delta sibling is processed.  And if those delta bases are 
> > themselves deltas then you have an explosion of delta results to 
> > re-compute.
> 
> Yes, it would be horrible if we had to recompute 10 deltas in order
> to recover a previously discarded delta base in order to visit new
> siblings.
> 
> But its even more horrible that we use 512M of memory in our working
> set size on a 256M machine to process a pack that is only 300M in
> size, due to long delta chains on large objects.  In such a case
> the system will swap and perform fairly poorly due to the huge disk
> IO necessary to keep moving the working set around.
> 
> We're better off keeping our memory usage low and recomputing
> the delta base when we need to return to it to process a sibling.