git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH] pack-objects: only throw away data during memory pressure

From
Nicolas Pitre <nico@cam.org>
Date
Feb 12, 2008, 14:26 UTC
Message-ID
<alpine.LFD.1.00.0802120910440.2732@xanadu.home>
In-Reply-To
<20080212082211.GE27535@lavos.net>
On Tue, 12 Feb 2008, Brian Downing wrote:
Show 32 quoted lines
> On Mon, Feb 11, 2008 at 11:00:33AM -0500, Nicolas Pitre wrote:
> > On Mon, 11 Feb 2008, Martin Koegler wrote:
> > > If pack-objects hit the memory limit, it deletes objects from the
> > > delta window.
> > > 
> > > This patch make it only delete the data, which is recomputed, if
> > > needed again.
> > > 
> > > Signed-off-by: Martin Koegler <mkoegler@auto.tuwien.ac.at>
> > 
> > Looks fine.
> > 
> > Acked-by: Nicolas Pitre <nico@cam.org>
> 
> Unfortunately this patch (if I understand what it's doing correctly)
> basically defeats my intended use-case for which I wrote the memory
> limiter.  I have a repository with files of very mixed size.  I want the
> window to be very large for small files, for good archival repacking,
> but I don't want it to be very large for my 20+MB files with hundreds of
> revisions, because I want it to finish someday.
> 
> Also, I've gotten into the habit of just doing:
>     git repack --window=100000 --window-memory=256m
> for archival repacks and just letting the memory limit automatically
> size the window.  Basically, I don't really want to specify a window
> size, I just want it to use 512mb of RAM (and go at the speed that size
> of a window would entail.)  While this is slow, it tends to be a
> relatively constant speed, and it tends to find some very interesting
> deltas in my trees that I wouldn't have otherwise expected.
> 
> If this patch is accepted, I'd really like a way to maintain the old
> behavior as an option.

I think your use case has merits, but the previous behavior had semantics problems. We always had constant window size with dynamic memory usage, and now we have constant window size with bounded memory usage.

If what you want is really to have a dynamic window size using a constant memory usage then it needs a different and coherent way to be specified.

Nicolas
Previous: Brian DowningNext: Brian Downing
Message 7 of 12 in “pack-objects: only throw away data during memory pressure”
  1. pack-objects: only throw away data during memory pressureMartin Koegler, Feb 11, 2008
  2. Johannes SchindelinFeb 11, 2008
  3. Nicolas PitreFeb 11, 2008
  4. Johannes SchindelinFeb 11, 2008
  5. Nicolas PitreFeb 11, 2008
  6. Brian DowningFeb 12, 2008
  7. Nicolas PitreFeb 12, 2008
  8. Brian DowningFeb 12, 2008
  9. Brian DowningFeb 12, 2008
  10. Brian DowningFeb 12, 2008
  11. Nicolas PitreFeb 12, 2008
  12. Nicolas PitreFeb 13, 2008

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.