# Bug with corruption on clone/fsck/... with large packs + 64-bit Windows, problem with usage of "long" datatype for sizes/offsets? 4 messages from 2017-08-11 to 2017-08-14. Participants: Dr.-Ing. Christoph Cullmann, Johannes Schindelin, Stefan Beller, Junio C Hamano. Thread: https://gitlist.dev/t/46566 ## Dr.-Ing. Christoph Cullmann, 2017-08-11 07:04 Subject: Bug with corruption on clone/fsck/... with large packs + 64-bit Windows, problem with usage of "long" datatype for sizes/offsets? Message-ID: <1232852793.287524.1502435085656.JavaMail.zimbra@absint.com> URL: https://gitlist.dev/e/1232852793.287524.1502435085656.JavaMail.zimbra%40absint.com ``` Hi, on Windows 64-bit, for a repository having a .pack file > 4GB I get during cloning: $ git clone file:///repositories/test.git test Cloning into 'test'... remote: Counting objects: 210294, done. remote: error: bad object header remote: error: bad object header remote: fatal: packed object cae50116ebe36de8bded4811bd262d90670bde2f (stored in ./objects/pack/pack-30ba8b19b029cc6554853ae07729aa807995ebb8.pack) is corrupt error: git upload-pack: git-pack-objects died with error. fatal: git upload-pack: aborting due to possible repository remote: aborting due to possible repository corruption on the remote side. corruption on the remote side. fatal: early EOF fatal: index-pack failed We use git 2.14.1, the 64-bit version. On systems which have (long int) == 64-bit datatype, all works well, e.g. Linux 64-bit. (same goes for fscks/...., they work on this git on Linux 64-bit, not on Windows 64-bit) It seems the Windows port did take care of using the right 64-bit offsets and all in the OS specific parts, but things like pack-objects.c in the generic parts use "unsigned long" like it behaves like size_t (at least on the first glance). As I think the issue is in the generic part, I report this here, not on the Windows list, thought it seems to be an IL32P64 related issue. Greetings Christoph -- ----------------------------- Dr.-Ing. Christoph Cullmann --------- AbsInt Angewandte Informatik GmbH Email: cullmann@AbsInt.com Science Park 1 Tel: +49-681-38360-22 66123 Saarbrücken Fax: +49-681-38360-20 GERMANY WWW: http://www.AbsInt.com -------------------------------------------------------------------- Geschäftsführung: Dr.-Ing. Christian Ferdinand Eingetragen im Handelsregister des Amtsgerichts Saarbrücken, HRB 11234 ``` ## Johannes Schindelin, 2017-08-14 16:05 Subject: Re: Bug with corruption on clone/fsck/... with large packs + 64-bit Windows, problem with usage of "long" datatype for sizes/offsets? Message-ID: URL: https://gitlist.dev/e/alpine.DEB.2.21.1.1708141802510.19382%40virtualbox In-Reply-To: <1232852793.287524.1502435085656.JavaMail.zimbra@absint.com> ``` Hi Christoph, On Fri, 11 Aug 2017, Dr.-Ing. Christoph Cullmann wrote: > on Windows 64-bit, for a repository having a .pack file > 4GB I get > during cloning: The reason is Git's source code that over-uses the `unsigned long` datatype. In a nearby-thread, an underappreciated effort by Martin Koegler is underway to get the ball rolling in getting it fixed. Maybe you can help making Martin a lot more welcome on the Git mailing list, and maybe even help getting his patches reviewed and integrated? Ciao, Johannes ``` ## Stefan Beller, 2017-08-14 17:07 Subject: Re: Bug with corruption on clone/fsck/... with large packs + 64-bit Windows, problem with usage of "long" datatype for sizes/offsets? Message-ID: URL: https://gitlist.dev/e/CAGZ79kaYi%3Dmqw615ZHiNAbDdnnKLawzZDy3ecLYDVV%3De7e5qbQ%40mail.gmail.com In-Reply-To: ``` On Mon, Aug 14, 2017 at 9:05 AM, Johannes Schindelin wrote: > Hi Christoph, > > On Fri, 11 Aug 2017, Dr.-Ing. Christoph Cullmann wrote: > >> on Windows 64-bit, for a repository having a .pack file > 4GB I get >> during cloning: > > The reason is Git's source code that over-uses the `unsigned long` > datatype. > > In a nearby-thread, an underappreciated effort by Martin Koegler is > underway to get the ball rolling in getting it fixed. Maybe you can help > making Martin a lot more welcome on the Git mailing list, and maybe even > help getting his patches reviewed and integrated? 'nearby' as in [1] ;-) [1] https://public-inbox.org/git/1502527643-21944-1-git-send-email-martin@mail.zuhause/ I had the impression the review is going well there? > > Ciao, > Johannes ``` ## Junio C Hamano, 2017-08-14 18:57 Subject: Re: Bug with corruption on clone/fsck/... with large packs + 64-bit Windows, problem with usage of "long" datatype for sizes/offsets? Message-ID: URL: https://gitlist.dev/e/xmqqk226ugcw.fsf%40gitster.mtv.corp.google.com In-Reply-To: ``` Stefan Beller writes: > On Mon, Aug 14, 2017 at 9:05 AM, Johannes Schindelin > wrote: >> Hi Christoph, >> >> On Fri, 11 Aug 2017, Dr.-Ing. Christoph Cullmann wrote: >> >>> on Windows 64-bit, for a repository having a .pack file > 4GB I get >>> during cloning: >> >> The reason is Git's source code that over-uses the `unsigned long` >> datatype. >> >> In a nearby-thread, an underappreciated effort by Martin Koegler is >> underway to get the ball rolling in getting it fixed. Maybe you can help >> making Martin a lot more welcome on the Git mailing list, and maybe even >> help getting his patches reviewed and integrated? > > 'nearby' as in [1] ;-) > > [1] https://public-inbox.org/git/1502527643-21944-1-git-send-email-martin@mail.zuhause/ > > I had the impression the review is going well there? I do not know if it is "going well", but I do not agree with the "underappreciated" bit at all. I find such a blanket statement toxic and detrimental to the community. It is true that many topics in flight are broken with a tree-wide change that is presented as a single ball of wax. Unlike the ongoing "struct object_id" effort, which also is tree-wide but tries to find a step-wise refinement to reduce its impact on other topics, it is harder to integrate such a change. But that does not have any relation to how much the effort is appreciated. ```