git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: git fsck failure on OS X with files >= 4 GiB

From
Filipe Cabecinhas <filcab@gmail.com>
Date
Oct 29, 2015, 15:15 UTC
Message-ID
<CAEDE853n2HR-SK9_sGn8n3j0xoTb3eQa86UvdW0DBrc+Z1sDrg@mail.gmail.com>
In-Reply-To
<CAG3jRe+23sy1k9QNdpdn3GF3nbzPMmYO=TM=SufEq83OtwNxbA@mail.gmail.com>

Defining BLK_SHA1 = YesPlease (when calling make) should just change the SHA functions, instead of completely removing OpenSSL or CommonCrypto.

Regards,
  Filipe

On Thu, Oct 29, 2015 at 3:46 AM, Rafael Espíndola <rafael.espindola@gmail.com> wrote:

Show 67 quoted lines
> Awesome, building with
>
> NO_OPENSSL = 1
> NO_GETTEXT = 1
>
> produces a working git :-)
>
> Cheers,
> Rafael
>
>
> On 28 October 2015 at 23:37, Filipe Cabecinhas <filcab@gmail.com> wrote:
>> I did some debugging, and it seems CC_SHA1_Update (used by
>> write_sha1_file_prepare if APPLE_COMMON_CRYPTO is defined in the Makefile)
>> takes a uint32_t as a "length" parameter, which explains why it stops
>> working at 4GiB (UINT_MAX+1).
>>
>> In the OS X 10.11 SDK header CommonCrypto/CommonDigest.h, we have:
>>
>> typedef uint32_t CC_LONG;       /* 32 bit unsigned integer */
>> //...
>> extern int CC_SHA1_Update(CC_SHA1_CTX *c, const void *data, CC_LONG len)
>>
>> A possible fix would be to either call SHA1_Update with a maximum of
>> UINT_MAX, looping if necessary. Or have a compatibility SHA1_Update for OS X
>> which can handle data longer than UINT_MAX.
>>
>> I'm not sure what the git maintainers would prefer.
>>
>> Regards,
>>
>>   Filipe
>>
>> On Wed, Oct 28, 2015 at 4:10 PM, Rafael Espíndola
>> <rafael.espindola@gmail.com> wrote:
>>>
>>> I first noticed this with "2.4.9 (Apple Git-60)", but it reproduces
>>> with git built from 37023ba381b6d251d7140a997b39b566dbc63c42.
>>>
>>> Create two files with just 0s:
>>>
>>> -rw-r--r--  1 espindola  staff  4294967296 28 Oct 11:09 exactly-4gib
>>> -rw-r--r--  1 espindola  staff  4294967295 28 Oct 11:09 one-less-than-4gib
>>>
>>>
>>> and run
>>>
>>> git init
>>> git add one-less-than-4gib
>>> git commit -m bar
>>> git fsck
>>> git add exactly-4gib
>>> git commit -m bar
>>> git fsck
>>>
>>> The first fsck will run with no problems, but the second one fails:
>>>
>>> error: packed cfdaf54c9ccfd8f5e4cee562f7d5f92df13d3106 from
>>> .git/objects/pack/pack-ff08480fd7f767b6bd0aeb559f0f5dea2245b0b3.pack
>>> is corrupt
>>>
>>> Using the very same revision on freebsd doesn't cause any errors.
>>>
>>> Cheers,
>>> Rafael
>>
>>
Previous: Rafael EspíndolaNext: Atousa Duprat
Message 4 of 27 in “git fsck failure on OS X with files >= 4 GiB”
  1. Rafael EspíndolaOct 28, 2015
  2. Filipe CabecinhasOct 29, 2015
  3. Rafael EspíndolaOct 29, 2015
  4. Filipe CabecinhasOct 29, 2015
  5. Atousa DupratOct 29, 2015
  6. Junio C HamanoOct 29, 2015
  7. Atousa DupratOct 30, 2015
  8. Limit the size of the data block passed to SHA1_Update()Atousa Pahlevan Duprat, Oct 30, 2015
  9. Junio C HamanoOct 30, 2015
  10. Atousa DupratNov 1, 2015
  11. Junio C HamanoNov 1, 2015
  12. Eric SunshineNov 1, 2015
  13. Limit the size of the data block passed to SHA1_Update()atousa.p@gmail.com, Nov 1, 2015
  14. Eric SunshineNov 1, 2015
  15. Junio C HamanoNov 1, 2015
  16. Atousa DupratNov 2, 2015
  17. Junio C HamanoNov 2, 2015
  18. 1/2 Limit the size of the data block passed to SHA1_Update()atousa.p@gmail.com, Nov 3, 2015
  19. Torsten BögershausenNov 3, 2015
  20. Limit the size of the data block passed to SHA1_Update()atousa.p@gmail.com, Nov 4, 2015
  21. Eric SunshineNov 4, 2015
  22. Atousa DupratNov 4, 2015
  23. Junio C HamanoNov 4, 2015
  24. Limit the size of the data block passed to SHA1_Update()Atousa Pahlevan Duprat, Oct 30, 2015
  25. Randall S. BeckerOct 30, 2015
  26. Junio C HamanoOct 31, 2015
  27. Atousa DupratNov 1, 2015

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.