{"thread":{"id":"46566","subject":"Bug with corruption on clone/fsck/... with large packs + 64-bit Windows, problem with usage of \"long\" datatype for sizes/offsets?","startedAt":"2017-08-11T07:10:03Z","lastAt":"2017-08-14T18:57:58Z","messageCount":4,"participants":["Dr.-Ing. Christoph Cullmann","Johannes Schindelin","Stefan Beller","Junio C Hamano"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"326131","messageId":"1232852793.287524.1502435085656.JavaMail.zimbra@absint.com","threadId":"46566","inReplyTo":null,"subject":"Bug with corruption on clone/fsck/... with large packs + 64-bit Windows, problem with usage of \"long\" datatype for sizes/offsets?","fromName":"Dr.-Ing. Christoph Cullmann","fromEmail":"cullmann@absint.com","sentAt":"2017-08-11T07:04:45Z","receivedAt":"2017-08-11T07:10:03Z","isPatch":false,"sender":{"key":"cullmann@absint.com","avatar":null},"body":"Hi,\n\non Windows 64-bit, for a repository having a .pack file > 4GB I get during cloning:\n\n$ git clone file:///repositories/test.git test\nCloning into 'test'...\nremote: Counting objects: 210294, done.\nremote: error: bad object header\nremote: error: bad object header\nremote: fatal: packed object cae50116ebe36de8bded4811bd262d90670bde2f (stored in ./objects/pack/pack-30ba8b19b029cc6554853ae07729aa807995ebb8.pack) is corrupt\nerror: git upload-pack: git-pack-objects died with error.\nfatal: git upload-pack: aborting due to possible repository remote: aborting due to possible repository corruption on the remote side.\ncorruption on the remote side.\nfatal: early EOF\nfatal: index-pack failed\n\nWe use git 2.14.1, the 64-bit version.\n\nOn systems which have (long int) == 64-bit datatype, all works well, e.g. Linux 64-bit.\n(same goes for fscks/...., they work on this git on Linux 64-bit, not on Windows 64-bit)\n\nIt seems the Windows port did take care of using the right 64-bit offsets and all\nin the OS specific parts, but things like pack-objects.c in the generic parts use \"unsigned long\"\nlike it behaves like size_t (at least on the first glance).\n\nAs I think the issue is in the generic part, I report this here, not on the Windows list,\nthought it seems to be an IL32P64 related issue.\n\nGreetings\nChristoph\n\n-- \n----------------------------- Dr.-Ing. Christoph Cullmann ---------\nAbsInt Angewandte Informatik GmbH Email: cullmann@AbsInt.com\nScience Park 1 Tel: +49-681-38360-22\n66123 Saarbrücken Fax: +49-681-38360-20\nGERMANY WWW: http://www.AbsInt.com\n--------------------------------------------------------------------\nGeschäftsführung: Dr.-Ing. Christian Ferdinand\nEingetragen im Handelsregister des Amtsgerichts Saarbrücken, HRB 11234\n"},{"id":"326273","messageId":"alpine.DEB.2.21.1.1708141802510.19382@virtualbox","threadId":"46566","inReplyTo":"1232852793.287524.1502435085656.JavaMail.zimbra@absint.com","subject":"Re: Bug with corruption on clone/fsck/... with large packs + 64-bit Windows, problem with usage of \"long\" datatype for sizes/offsets?","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2017-08-14T16:05:10Z","receivedAt":"2017-08-14T16:05:25Z","isPatch":false,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi Christoph,\n\nOn Fri, 11 Aug 2017, Dr.-Ing. Christoph Cullmann wrote:\n\n> on Windows 64-bit, for a repository having a .pack file > 4GB I get\n> during cloning:\n\nThe reason is Git's source code that over-uses the `unsigned long`\ndatatype.\n\nIn a nearby-thread, an underappreciated effort by Martin Koegler is\nunderway to get the ball rolling in getting it fixed. Maybe you can help\nmaking Martin a lot more welcome on the Git mailing list, and maybe even\nhelp getting his patches reviewed and integrated?\n\nCiao,\nJohannes\n"},{"id":"326280","messageId":"CAGZ79kaYi=mqw615ZHiNAbDdnnKLawzZDy3ecLYDVV=e7e5qbQ@mail.gmail.com","threadId":"46566","inReplyTo":"alpine.DEB.2.21.1.1708141802510.19382@virtualbox","subject":"Re: Bug with corruption on clone/fsck/... with large packs + 64-bit Windows, problem with usage of \"long\" datatype for sizes/offsets?","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-08-14T17:07:04Z","receivedAt":"2017-08-14T17:07:12Z","isPatch":false,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"On Mon, Aug 14, 2017 at 9:05 AM, Johannes Schindelin\n wrote:\n> Hi Christoph,\n>\n> On Fri, 11 Aug 2017, Dr.-Ing. Christoph Cullmann wrote:\n>\n>> on Windows 64-bit, for a repository having a .pack file > 4GB I get\n>> during cloning:\n>\n> The reason is Git's source code that over-uses the `unsigned long`\n> datatype.\n>\n> In a nearby-thread, an underappreciated effort by Martin Koegler is\n> underway to get the ball rolling in getting it fixed. Maybe you can help\n> making Martin a lot more welcome on the Git mailing list, and maybe even\n> help getting his patches reviewed and integrated?\n\n'nearby' as in [1] ;-)\n\n[1] https://public-inbox.org/git/1502527643-21944-1-git-send-email-martin@mail.zuhause/\n\nI had the impression the review is going well there?\n\n>\n> Ciao,\n> Johannes\n"},{"id":"326296","messageId":"xmqqk226ugcw.fsf@gitster.mtv.corp.google.com","threadId":"46566","inReplyTo":"CAGZ79kaYi=mqw615ZHiNAbDdnnKLawzZDy3ecLYDVV=e7e5qbQ@mail.gmail.com","subject":"Re: Bug with corruption on clone/fsck/... with large packs + 64-bit Windows, problem with usage of \"long\" datatype for sizes/offsets?","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2017-08-14T18:57:51Z","receivedAt":"2017-08-14T18:57:58Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Stefan Beller writes:\n\n> On Mon, Aug 14, 2017 at 9:05 AM, Johannes Schindelin\n> wrote:\n>> Hi Christoph,\n>>\n>> On Fri, 11 Aug 2017, Dr.-Ing. Christoph Cullmann wrote:\n>>\n>>> on Windows 64-bit, for a repository having a .pack file > 4GB I get\n>>> during cloning:\n>>\n>> The reason is Git's source code that over-uses the `unsigned long`\n>> datatype.\n>>\n>> In a nearby-thread, an underappreciated effort by Martin Koegler is\n>> underway to get the ball rolling in getting it fixed. Maybe you can help\n>> making Martin a lot more welcome on the Git mailing list, and maybe even\n>> help getting his patches reviewed and integrated?\n>\n> 'nearby' as in [1] ;-)\n>\n> [1] https://public-inbox.org/git/1502527643-21944-1-git-send-email-martin@mail.zuhause/\n>\n> I had the impression the review is going well there?\n\nI do not know if it is \"going well\", but I do not agree with the\n\"underappreciated\" bit at all. I find such a blanket statement\ntoxic and detrimental to the community.\n\nIt is true that many topics in flight are broken with a tree-wide\nchange that is presented as a single ball of wax. Unlike the\nongoing \"struct object_id\" effort, which also is tree-wide but tries\nto find a step-wise refinement to reduce its impact on other topics,\nit is harder to integrate such a change. But that does not have any\nrelation to how much the effort is appreciated.\n"}]}