git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH] git-{repack,pack-objects} accept --{pack,blob}-limit to control pack size

From
Shawn O. Pearce <spearce@spearce.org>
Date
Apr 5, 2007, 06:54 UTC
Message-ID
<20070405065433.GD5436@spearce.org>
In-Reply-To
<56b7f5510704041555q4e735961ra9ee8008be0d33db@mail.gmail.com>
Dana How <danahow@gmail.com> wrote:
> it's not clear to me that
> the mmap window code works on 32b systems
> with >2GB-sized objects in packs.
Hmmph.  Depends on the system.

For glibc we do try to set _FILE_OFFSET_BITS to 64, so that even on 32 bit glibcs we pick up a 64 bit off_t. But really old glibcs won't have any 64 bit off_t support no matter what we do. And some systems don't know what _FILE_OFFSET_BITS is, so they give us whatever size off_t they want.

Forget mmap. In a 32 bit off_t case there is absolutely no way that open_packed_git_1 can verify the packfile signature (last 20 bytes) against the index if the packfile is larger than 2 GiB. In this case we will have an off_t that is negative, subtract 20 from it, and its still probably negative. I doubt SEEK_SET will like a negative offset. This of course assumes that the earlier fstat call actually succeeded on such a file a large file with such a small off_t.

If we get through that open_packed_git_1 and actually verify the signature, we either have some random sequence in the middle of the packfile that matches the signature in the index (sort of unlikely), or our off_t is actually large enough to handle the window position tests that the use_pack and in_window functions perform. In this latter case we shouldn't have any problems with the mmap code.

-- 
Shawn.
Previous: Nicolas PitreNext: Linus Torvalds
Message 7 of 12 in “git-{repack,pack-objects} accept --{pack,blob}-limit to control pack size”
  1. git-{repack,pack-objects} accept --{pack,blob}-limit to control pack sizeDana How, Apr 4, 2007
  2. Nicolas PitreApr 4, 2007
  3. Dana HowApr 4, 2007
  4. Nicolas PitreApr 5, 2007
  5. Shawn O. PearceApr 5, 2007
  6. Nicolas PitreApr 5, 2007
  7. Shawn O. PearceApr 5, 2007
  8. Linus TorvaldsApr 5, 2007
  9. Shawn O. PearceApr 5, 2007
  10. Linus TorvaldsApr 5, 2007
  11. Shawn O. PearceApr 5, 2007
  12. Florian WeimerApr 5, 2007

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.