{"thread":{"id":"41764","subject":"Creates unreadable pack files on platforms with sizeof(unsigned long) != sizeof(uintmax_t)","startedAt":"2016-03-20T21:20:02Z","lastAt":"2016-03-20T21:20:02Z","messageCount":1,"participants":["Thomas Braun"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"281310","messageId":"56EF1402.4050708@virtuell-zuhause.de","threadId":"41764","inReplyTo":null,"subject":"Creates unreadable pack files on platforms with sizeof(unsigned long) != sizeof(uintmax_t)","fromName":"Thomas Braun","fromEmail":"thomas.braun@virtuell-zuhause.de","sentAt":"2016-03-20T21:20:02Z","receivedAt":"2016-03-20T21:20:02Z","isPatch":false,"sender":{"key":"thomas.braun@virtuell-zuhause.de","avatar":"https://avatars.githubusercontent.com/u/1185677?v=4"},"body":"Hi,\n\nwhile playing around with some git settings I encountered some problems on Windows x64\nusing the 64bit build of git.\nAnd it is not restricted to that platform.\n\nRecipe to break:\nmkdir test &&\ncd test &&\ntruncate -s 5g largefile.bin &&\ngit init &&\ngit add . &&\ngit commit -m \"changes\" &&\ngit fsck\n\nResult:\nInitialized empty Git repository in E:/ttest/.git/\n[master (root-commit) d19adaf] changes\n 1 file changed, 0 insertions(+), 0 deletions(-)\n create mode 100644 largefile.bin\nChecking object directories: 100% (256/256), done.\nerror: bad object header\nerror: unknown object type -1 at offset 12 in\n.git/objects/pack/pack-25250ce5c176078ba51a42fee177c2f03f8845ca.pack\nerror: cannot unpack 0be2be10a4c8764f32c4bf372a98edc731a4b204 from\n.git/objects/pack/pack-25250ce5c176078ba51a42fee177c2f03f8845ca.pack at offset\n12\nChecking objects: 100% (1/1), done.\n\nSo I've created a repository which I can now not use.\n\nThe die() call is from unpack_object_header_buffer() in sha1_file.c. On windows x64\nbitsizeof(long) returns 32 and equals shift at some point.\nunpack_object_header_buffer() returns the size in an unsigned long (32bit). [1, 2, 3]\n\nThe questions why this has not been detected on creating the pack leads to\nencode_in_pack_object_header()\nwhich uses uintmax_t (64 bit wide) for storing the size.\n\nunsigned long is used in more places for file sizes e.g. in struct object_entry\nin pack-objects.h.\n\nThe proper solution now would be, I guess, to convert file sizes from unsigned long to something\nwhich is wider on windows x64 and in the best case the same size on linux. My git code base foo is rather\nlow and it looks much more involved than a simple s//.\n\nA first intermediate solution could be to die on pack creation e.g. in \n\ndiff --git a/pack-write.c b/pack-write.c\nindex 33293ce..ebb8b0a 100644\n--- a/pack-write.c\n+++ b/pack-write.c\n@@ -313,6 +313,9 @@ int encode_in_pack_object_header(enum object_type type, uintmax_t size, unsigned\n        if (type < OBJ_COMMIT || type > OBJ_REF_DELTA)\n                die(\"bad type %d\", type);\n\n+       if (bitsizeof(unsigned long) != bitsizeof(uintmax_t) && size > (unsigned long) size)\n+               die(\"Cannot handle files this big\");\n+\n        c = (type << 4) | (size & 15);\n        size >>= 4;\n        while (size) {\n\nWith that patch I get\n\n$ ../git add .\nfatal: Cannot handle files this big\n\nI know that usually people don't add big binary files to git. But I do, so I care ;)\nIf this direction sounds reasonable I can provide a proper patch.\n\nThanks,\nThomas\n\n[1]: https://msdn.microsoft.com/en-us/library/323b6b3k.aspx\n[2]: https://msdn.microsoft.com/en-us/library/s3f49ktz.aspx\n[3]: http://stackoverflow.com/questions/7607502/sizeoflong-in-64-bit-c\n"}]}