{"thread":{"id":"6431","subject":"git push problem with v1.5.0-rc1","startedAt":"2007-01-18T21:26:15Z","lastAt":"2007-01-20T18:00:54Z","messageCount":6,"participants":["Bart Trojanowski","Simon 'corecode' Schubert","Sergey Vlasov"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"32023","messageId":"20070118212615.GO8624@jukie.net","threadId":"6431","inReplyTo":null,"subject":"git push problem with v1.5.0-rc1","fromName":"Bart Trojanowski","fromEmail":"bart@jukie.net","sentAt":"2007-01-18T21:26:15Z","receivedAt":"2007-01-18T21:26:15Z","isPatch":false,"sender":{"key":"bart@jukie.net","avatar":"https://avatars.githubusercontent.com/u/6721?v=4"},"body":"I just stared using 1.5.0-rc yesterday, and ran into a problem with\ngit-push.\n\nI am running vanilla 2.6.19 on amd64 SMP box. For this project I am\nworking in a 32bit chroot -- the 64/32bit setup is part of the problem.\n\nThere are 3 repositories involved here:\n\nlinux-2.6.git \n - clone of a few branches from kernel.org\n\nklips-vault.git\n - linux tree from project upstream\n - alternates = /.../linux-2.6.git/.git/objects\n\nworking dir:\n - clone of klips-vault.git\n - alternates = /.../klips-vault.git/.git/objects\n\n - I made a new branch\n - applied ~100 patches\n\n - went to push and...\n\n $ git version \n git version 1.5.0.rc1.gdf1b \n \n $ cat .git/config \n [core] \n repositoryformatversion = 0 \n filemode = true \n logallrefupdates = true \n [remote \"origin\"] \n url = /home/jukie/bart/work/xelerance/klips-vault.git/.git \n fetch = +refs/heads/*:refs/remotes/origin/* \n push = refs/heads/my-ocf+fsm_v2.6.18:refs/heads/my-ocf+fsm_v2.6.18 \n [branch \"master\"] \n remote = origin \n merge = refs/heads/master \n \n $ git branch \n master \n * my-ocf+fsm_v2.6.18 \n \n $ git push origin \n updating 'refs/heads/my-ocf+fsm_v2.6.18' \n from 0000000000000000000000000000000000000000 \n to 380541e91358d7a5e2fe37c81c520c92a3094951 \n Generating pack... \n Done counting 727 objects. \n Result has 708 objects. \n Deltifying 708 objects. \n 100% (708/708) done \n Writing 708 objects. \n 100% (708/708) done \n Total 708 (delta 535), reused 275 (delta 218) \n fatal: cannot fstat packfile: Value too large for defined data type \n unpack index-pack exited with error code \n ng refs/heads/my-ocf+fsm_v2.6.18 n/a (unpacker error) \n\nThere was some question as to why fstat fails in the chroot... I don't\nhave any 2TB pack files. The above repos are relatively small. All\n.git directories sum up to about 500M.\n\nI got some help from Shawn Pearce on #git. He told me to set the\n[receive] unpackLimit=5000 in .git/config on the remote. That did\nwork, and I could push.\n\nThe conclusion was that my \"chroot is broken\". It's Ubuntu, so it\nshould Just Work(TM) :D\n\nI have tested a few configurations:\n\ngit 1.4.4.4 - push OK\ngit 1.4.4.4 w/unpackLimit=100 - push fails\n\ngit 1.5.0-rc0 - push fails\ngit 1.5.0-rc1 - push fails\ngit 1.5.0-rc1 w/unpackLimit=5000 - push OK\n\nI am not sure why 1.4.4.4 works w/o limit changes.\n\nI will try not to touch anything for a while, and would be glad to help\nfurther if I can.\n\n-Bart\n\n-- \n\t\t\t\tWebSig: http://www.jukie.net/~bart/sig/\n"},{"id":"32030","messageId":"20070119024436.GQ8624@jukie.net","threadId":"6431","inReplyTo":"20070118212615.GO8624@jukie.net","subject":"[PATCH] Don't call fstat() on stdin in index-pack.","fromName":"Bart Trojanowski","fromEmail":"bart@jukie.net","sentAt":"2007-01-19T02:44:36Z","receivedAt":"2007-01-19T02:44:36Z","isPatch":true,"sender":{"key":"bart@jukie.net","avatar":"https://avatars.githubusercontent.com/u/6721?v=4"},"body":"* Bart Trojanowski [070118 16:26]:\n> $ git push origin \n> updating 'refs/heads/my-ocf+fsm_v2.6.18' \n> from 0000000000000000000000000000000000000000 \n> to 380541e91358d7a5e2fe37c81c520c92a3094951 \n> Generating pack... \n> Done counting 727 objects. \n> Result has 708 objects. \n> Deltifying 708 objects. \n> 100% (708/708) done \n> Writing 708 objects. \n> 100% (708/708) done \n> Total 708 (delta 535), reused 275 (delta 218) \n> fatal: cannot fstat packfile: Value too large for defined data type \n\nI had a look at the code in index-pack.c and noticed that this error is\nprinted input_fd is set to 0 in open_pack_file().\n\n| if (fstat(input_fd, &st))\n| die(\"cannot fstat packfile: %s\", strerror(errno));\n| if (S_ISREG(st.st_mode) && st.st_size != consumed_bytes)\n| die(\"pack has junk at the end\");\n\nIt seems strange to me to call fstat on fd 0 to get st_size info.\n\nGranted, st_mode should tell us that it's not a regular file, but\nalready know it's not a regular file.\n\nSo I removed it. And now it works (I verified that the git-diff output\non both sides matches).\n\n-Bart\n\n----\n\n>From fcd359655e12a4b6424f989b7c01cbbb8a551287 Mon Sep 17 00:00:00 2001\nFrom: Bart Trojanowski \nDate: Fri, 19 Jan 2007 02:39:27 +0000\nSubject: [PATCH] Don't call fstat() on stdin in index-pack.\n\nThis fixes a issues with a large git-push with a\n32bit git on a 64bit kernel.\n\nSigned-off-by: Bart Trojanowski \n---\n index-pack.c | 10 ++++++----\n 1 files changed, 6 insertions(+), 4 deletions(-)\n\ndiff --git a/index-pack.c b/index-pack.c\nindex 72e0962..e7870a9 100644\n--- a/index-pack.c\n+++ b/index-pack.c\n@@ -455,10 +455,12 @@ static void parse_pack_objects(unsigned char *sha1)\n \tuse(20);\n \n \t/* If input_fd is a file, we should have reached its end now. */\n-\tif (fstat(input_fd, &st))\n-\t\tdie(\"cannot fstat packfile: %s\", strerror(errno));\n-\tif (S_ISREG(st.st_mode) && st.st_size != consumed_bytes)\n-\t\tdie(\"pack has junk at the end\");\n+ if (input_fd) {\n+ if (fstat(input_fd, &st))\n+ die(\"cannot fstat packfile: %s\", strerror(errno));\n+ if (S_ISREG(st.st_mode) && st.st_size != consumed_bytes)\n+ die(\"pack has junk at the end\");\n+ }\n \n \tif (!nr_deltas)\n \t\treturn;\n-- \n1.5.0.rc1.gdf1b-dirty\n"},{"id":"32031","messageId":"45B034D2.6010007@fs.ei.tum.de","threadId":"6431","inReplyTo":"20070119024436.GQ8624@jukie.net","subject":"Re: [PATCH] Don't call fstat() on stdin in index-pack.","fromName":"Simon 'corecode' Schubert","fromEmail":"corecode@fs.ei.tum.de","sentAt":"2007-01-19T03:02:42Z","receivedAt":"2007-01-19T03:02:42Z","isPatch":true,"sender":{"key":"corecode@fs.ei.tum.de","avatar":"https://gravatar.com/avatar/eff9dbf0cdac0d1e6a6cd7ed0e50763edcb376b493b5253a35ff167918ad79e1?d=mp&s=160"},"body":"Bart Trojanowski wrote:\n> \t/* If input_fd is a file, we should have reached its end now. */\n> -\tif (fstat(input_fd, &st))\n> -\t\tdie(\"cannot fstat packfile: %s\", strerror(errno));\n> -\tif (S_ISREG(st.st_mode) && st.st_size != consumed_bytes)\n> -\t\tdie(\"pack has junk at the end\");\n> + if (input_fd) {\n> + if (fstat(input_fd, &st))\n> + die(\"cannot fstat packfile: %s\", strerror(errno));\n> + if (S_ISREG(st.st_mode) && st.st_size != consumed_bytes)\n> + die(\"pack has junk at the end\");\n> + }\n\nThis is clearly the wrong fix. input_fd being 0 doesn't mean that it is *not* a regular file. Only doing a fstat can tell. You are simply hiding your real issue there, which is that you can't fstat on a pipe or whatever input_fd is.\n\nThe problem here is that your 64bit kernel can't fit the data into your struct stat provided by your 32bit libc. Not a problem of git. However, it would be interesting to know what exactly produces the EOVERFLOW.\n\ncheers\n simon\n\n-- \nServe - BSD +++ RENT this banner advert +++ ASCII Ribbon /\"\\\nWork - Mac +++ space for low €€€ NOW!1 +++ Campaign \\ /\nParty Enjoy Relax | http://dragonflybsd.org Against HTML \\\nDude 2c 2 the max ! http://golden-apple.biz Mail + News / \\\n\n"},{"id":"32032","messageId":"20070119031637.GR8624@jukie.net","threadId":"6431","inReplyTo":"20070118212615.GO8624@jukie.net","subject":"Re: git push problem with v1.5.0-rc1","fromName":"Bart Trojanowski","fromEmail":"bart@jukie.net","sentAt":"2007-01-19T03:16:37Z","receivedAt":"2007-01-19T03:16:37Z","isPatch":false,"sender":{"key":"bart@jukie.net","avatar":"https://avatars.githubusercontent.com/u/6721?v=4"},"body":"Thanks, Simon.\n\n* Bart Trojanowski [070118 16:26]:\n> fatal: cannot fstat packfile: Value too large for defined data type \n> unpack index-pack exited with error code \n\nInterestingly enough upgrading to 2.6.20-rc (from 2.6.19) fixed the real\nproblem.\n\n-Bart\n\n-- \n\t\t\t\tWebSig: http://www.jukie.net/~bart/sig/\n"},{"id":"32157","messageId":"20070120183519.3a182064.vsu@altlinux.ru","threadId":"6431","inReplyTo":"45B034D2.6010007@fs.ei.tum.de","subject":"Re: [PATCH] Don't call fstat() on stdin in index-pack.","fromName":"Sergey Vlasov","fromEmail":"vsu@altlinux.ru","sentAt":"2007-01-20T15:35:19Z","receivedAt":"2007-01-20T15:35:19Z","isPatch":true,"sender":{"key":"vsu@altlinux.ru","avatar":"https://avatars.githubusercontent.com/u/616082?v=4"},"body":"On Fri, 19 Jan 2007 04:02:42 +0100 Simon 'corecode' Schubert wrote:\n\n> Bart Trojanowski wrote:\n> > \t/* If input_fd is a file, we should have reached its end now. */\n> > -\tif (fstat(input_fd, &st))\n> > -\t\tdie(\"cannot fstat packfile: %s\", strerror(errno));\n> > -\tif (S_ISREG(st.st_mode) && st.st_size != consumed_bytes)\n> > -\t\tdie(\"pack has junk at the end\");\n> > + if (input_fd) {\n> > + if (fstat(input_fd, &st))\n> > + die(\"cannot fstat packfile: %s\", strerror(errno));\n> > + if (S_ISREG(st.st_mode) && st.st_size != consumed_bytes)\n> > + die(\"pack has junk at the end\");\n> > + }\n>\n> This is clearly the wrong fix. input_fd being 0 doesn't mean that\n> it is *not* a regular file. Only doing a fstat can tell. You are\n> simply hiding your real issue there, which is that you can't fstat\n> on a pipe or whatever input_fd is.\n>\n> The problem here is that your 64bit kernel can't fit the data into\n> your struct stat provided by your 32bit libc. Not a problem of git.\n> However, it would be interesting to know what exactly produces the\n> EOVERFLOW.\n\nMost likely it is the st_ino field - the kernel assigns unique inode\nnumbers for pipes from an \"unsigned long\" counter, which is 64-bit in\nthis case, and *stat() calls must fail with EOVERFLOW if the inode\nnumber does not fit into ino_t, which is 32-bit here. This problem is\nknown for some time, and there is even a kernel patch proposed as a\nworkaround (which makes the counter 32-bit):\n\nhttp://permalink.gmane.org/gmane.linux.file-systems/12526\n\nAFAIK, that patch is not upstream yet - so upgrading the kernel did\nnot really fix the issue, it will appear again once the system will\nuse more than 4G of pipe inodes.\n\nCompiling git with -D_FILE_OFFSET_BITS=64 will make ino_t 64-bit and\ntherefore will fix the problem (however, I'm not sure whether the git\ncode is ready for this).\n"},{"id":"32159","messageId":"45B258D6.8090602@fs.ei.tum.de","threadId":"6431","inReplyTo":"20070120183519.3a182064.vsu@altlinux.ru","subject":"Re: [PATCH] Don't call fstat() on stdin in index-pack.","fromName":"Simon 'corecode' Schubert","fromEmail":"corecode@fs.ei.tum.de","sentAt":"2007-01-20T18:00:54Z","receivedAt":"2007-01-20T18:00:54Z","isPatch":true,"sender":{"key":"corecode@fs.ei.tum.de","avatar":"https://gravatar.com/avatar/eff9dbf0cdac0d1e6a6cd7ed0e50763edcb376b493b5253a35ff167918ad79e1?d=mp&s=160"},"body":"Sergey Vlasov wrote:\n> Compiling git with -D_FILE_OFFSET_BITS=64 will make ino_t 64-bit and\n> therefore will fix the problem (however, I'm not sure whether the git\n> code is ready for this).\n\nIt better be :) My OS of choice has 64bit off_t and ino_t since a long time now...\n\ncheers\n simon\n\n-- \nServe - BSD +++ RENT this banner advert +++ ASCII Ribbon /\"\\\nWork - Mac +++ space for low €€€ NOW!1 +++ Campaign \\ /\nParty Enjoy Relax | http://dragonflybsd.org Against HTML \\\nDude 2c 2 the max ! http://golden-apple.biz Mail + News / \\\n\n"}]}