git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: win2k/cygwin cannot handle even moderately sized packs

From
NGNoel Grandin <noel@peralex.com>
Date
Nov 7, 2006, 12:17 UTC
Message-ID
<45507965.3010806@peralex.com>
In-Reply-To
<81b0412b0611070302h50541cd5mf0758afe0d6befda@mail.gmail.com>

Looking at http://msdn2.microsoft.com/en-us/library/ms810627.aspx it looks like (a) windows provides 2G of address space to play with (b) VirtualAlloc is constrained to allocating contiguous ranges of memory within that 2G address space

So the problem is probably memory fragmentation.

You might have more joy if you allocated one HUGE chunk immediately on startup to use for the pack, and then kept re-using that chunk.

Alex Riesen wrote:
Show 23 quoted lines
> For me, it fails even on 332Mb pack:
>
> $ git reset --hard 61bb7fcb
> fatal: packfile .git/objects/pack/pack-ad37...pack cannot be mapped.
>
> Instrumenting the code reveals that it fails on 348876870 bytes.
> Strangely enough, a cygwin program which just reads that pack
> many times without freeing the mem goes up to 1395507480 (1330Mb).
>
> If I replace the malloc (cygwin) with native VirtualAlloc(MEM_COMMIT)
> it reports that "Not enough storage is available to process this
> command",
> which is just ENOMEM, I think.
>
> This is a 2Gb machine, with almost 1.3Gb free, so I'm a bit confused,
> what could here go wrong (besides what already is wrong).
>
> Any ideas?
> -
> To unsubscribe from this list: send the line "unsubscribe git" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>
NOTICE: This email, and the contents thereof, 
are subject to the standard Peralex email disclaimer, which may 
be found at: http://www.peralex.com/disclaimer.html
If you cannot access the disclaimer through the URL attached 
 and you wish to receive a copy thereof please send 
 an email to email@peralex.com
Previous: Alex RiesenNext: Alex Riesen
Message 2 of 21 in “win2k/cygwin cannot handle even moderately sized packs”
  1. Alex RiesenNov 7, 2006
  2. Noel GrandinNov 7, 2006
  3. Alex RiesenNov 7, 2006
  4. Jakub NarebskiNov 7, 2006
  5. Alex RiesenNov 7, 2006
  6. Shawn PearceNov 7, 2006
  7. Alex RiesenNov 7, 2006
  8. Shawn PearceNov 7, 2006
  9. Shawn PearceNov 7, 2006
  10. Shawn PearceNov 7, 2006
  11. Alex RiesenNov 7, 2006
  12. Shawn PearceNov 8, 2006
  13. Alex RiesenNov 8, 2006
  14. Shawn PearceNov 8, 2006
  15. Alex RiesenNov 8, 2006
  16. Shawn PearceNov 8, 2006
  17. Alex RiesenNov 7, 2006
  18. Christopher FaylorNov 8, 2006
  19. Johannes SchindelinNov 13, 2006
  20. Alex RiesenNov 13, 2006
  21. Alex RiesenNov 13, 2006

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.