git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: Public repro case! Re: [PATCH/RFC] Allow writing loose objects that are corrupted in a pack file

From
RBR. Tyler Ballance <tyler@slide.com>
Date
Jan 8, 2009, 00:57 UTC
Message-ID
<1231376259.8870.633.camel@starfruit>
In-Reply-To
<alpine.LFD.2.00.0901071644330.3283@localhost.localdomain>
On Wed, 2009-01-07 at 16:48 -0800, Linus Torvalds wrote:
Show 29 quoted lines
> 
> On Wed, 7 Jan 2009, R. Tyler Ballance wrote:
> >
> > My most esteemed colleague (Ken aka kb) who pointed out the memory issue
> > was on the right path (I think), and I have a reproduction case you can
> > try with your very own Linux kernel tree!
> > 
> > WOO!
> > 
> > I set ulimit -v really low (150M), and the operations I made got an
> > mmap(2) fatal error, but there is a sweet spot that I found, see the
> > transcript below.
> 
> This is indeed the packfile mapping. The sweet spot you found depends on 
> how big the biggest two pack-files are, I do believe.
> 
> And if you do that
> 
> 	[core]
> 		packedgitwindowsize = 64M
> 
> I think you'll find that it works. Of course, with a _really_ low ulimit, 
> you'd need to make it even smaller, but at some point you start hitting 
> other problems than the pack-file limits, ie just the simple fact that git 
> wants and expects you to have a certain amount of memory available ;)
> 
> Can you cnfirm that your "reproducible" case starts working with that 
> addition to your ~/.gitconfig? If so, the solution is pretty simple: we 
> should just lower the default pack windowsize.

This certainly corrected the issue, is there some magic packedgitwindowsize that i should be looking at my own repository (our internal one) in order to prevent the issue from occurring?

Looking into .git/objects/pack, I think the two biggest pack files are 3.5G and 177MBG respectively :-!

Cheers
-- 
-R. Tyler Ballance
Slide, Inc.
Previous: Linus TorvaldsNext: Linus Torvalds
Message 23 of 49 in “Allow writing loose objects that are corrupted in a pack file”
  1. Allow writing loose objects that are corrupted in a pack fileJan Krüger, Dec 9, 2008
  2. R. Tyler BallanceDec 9, 2008
  3. Shawn O. PearceDec 9, 2008
  4. R. Tyler BallanceJan 6, 2009
  5. Nicolas PitreJan 7, 2009
  6. R. Tyler BallanceJan 7, 2009
  7. Nicolas PitreJan 7, 2009
  8. R. Tyler BallanceJan 7, 2009
  9. Nicolas PitreJan 7, 2009
  10. Linus TorvaldsJan 7, 2009
  11. R. Tyler BallanceJan 7, 2009
  12. Junio C HamanoJan 7, 2009
  13. R. Tyler BallanceJan 7, 2009
  14. Junio C HamanoJan 7, 2009
  15. R. Tyler BallanceJan 7, 2009
  16. Nicolas PitreJan 7, 2009
  17. Linus TorvaldsJan 7, 2009
  18. Linus TorvaldsJan 7, 2009
  19. R. Tyler BallanceJan 7, 2009
  20. Linus TorvaldsJan 7, 2009
  21. Public repro case! Re: [PATCH/RFC] Allow writing loose objects that are corrupted in a pack fileR. Tyler Ballance, Jan 8, 2009
  22. Linus TorvaldsJan 8, 2009
  23. R. Tyler BallanceJan 8, 2009
  24. Linus TorvaldsJan 8, 2009
  25. Linus TorvaldsJan 8, 2009
  26. Shawn O. PearceJan 8, 2009
  27. James PickensJan 8, 2009
  28. Shawn O. PearceJan 8, 2009
  29. Junio C HamanoJan 8, 2009
  30. Shawn O. PearceJan 8, 2009
  31. Boyd Stephen Smith Jr.Jan 8, 2009
  32. Linus TorvaldsJan 8, 2009
  33. Shawn O. PearceJan 8, 2009
  34. Linus TorvaldsJan 8, 2009
  35. Shawn O. PearceJan 8, 2009
  36. Wrap inflateInit to retry allocation after releasing pack memoryShawn O. Pearce, Jan 8, 2009
  37. Linus TorvaldsJan 8, 2009
  38. Junio C HamanoJan 8, 2009
  39. Linus TorvaldsJan 8, 2009
  40. Shawn O. PearceJan 8, 2009
  41. Linus TorvaldsJan 8, 2009
  42. R. Tyler BallanceJan 8, 2009
  43. Linus TorvaldsJan 8, 2009
  44. R. Tyler BallanceJan 8, 2009
  45. Junio C HamanoJan 9, 2009
  46. Linus TorvaldsJan 8, 2009
  47. R. Tyler BallanceJan 8, 2009
  48. Linus TorvaldsJan 8, 2009
  49. R. Tyler BallanceJan 8, 2009

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.