{"thread":{"id":"23788","subject":"[remote rejected] master -> master (n/a (unpacker error))","startedAt":"2010-05-12T19:45:52Z","lastAt":"2015-11-27T21:37:33Z","messageCount":8,"participants":["Robert Buck","Chris Packham","Jonathan Nieder","Greg Troxel","Andreas Schwab","DavidLeeCrites"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"141544","messageId":"AANLkTinV2U6Lbbl0N7jVAESEi0mZQ_D3slMEYa68vRT4@mail.gmail.com","threadId":"23788","inReplyTo":null,"subject":"[remote rejected] master -> master (n/a (unpacker error))","fromName":"Robert Buck","fromEmail":"buck.robert.j@gmail.com","sentAt":"2010-05-12T19:45:52Z","receivedAt":"2010-05-12T19:45:52Z","isPatch":false,"sender":{"key":"buck.robert.j@gmail.com","avatar":"https://gravatar.com/avatar/1686742e8ac2595378aac67f26fd638ddaf494194f9af0cb6082c4a7eca1a366?d=mp&s=160"},"body":"Today, just after someone else committed to my public repository I\nstarted getting errors. Until then Git worked great.\n\nDoes anyone know what is going on here? Are there particular versions\nof Git with known issues around this?\n\n\nuname@hostname:~/dev/workspaces/scm-evaluations/welcome.git/install/git-config$\ngit push\nCounting objects: 7, done.\nDelta compression using up to 2 threads.\nCompressing objects: 100% (4/4), done.\nWriting objects: 100% (4/4), 922 bytes, done.\nTotal 4 (delta 0), reused 4 (delta 0)\nerror: unable to create temporary sha1 filename ./objects/e6: File\nexists\n\nfatal: failed to write object\nerror: unpack failed: unpacker exited with error code\nTo ssh://git.projectbedrock.com/var/cache/git/welcome.git\n ! [remote rejected] master -> master (n/a (unpacker error))\nerror: failed to push some refs to\n'ssh://git.projectbedrock.com/var/cache/git/welcome.git'\n\n\nAs an aside, where the heck is the git bug tracker? I've searched, and\nsearched, and ... All I found is a Debian tracking system, which\nappears to have no full text search capabilities.\n"},{"id":"141553","messageId":"AANLkTilfXLdYCIZAu_I5vGTrbI08fbqUpIsjx5yP1q47@mail.gmail.com","threadId":"23788","inReplyTo":"AANLkTinV2U6Lbbl0N7jVAESEi0mZQ_D3slMEYa68vRT4@mail.gmail.com","subject":"Re: [remote rejected] master -> master (n/a (unpacker error))","fromName":"Chris Packham","fromEmail":"judge.packham@gmail.com","sentAt":"2010-05-13T00:06:52Z","receivedAt":"2010-05-13T00:06:52Z","isPatch":false,"sender":{"key":"judge.packham@gmail.com","avatar":"https://avatars.githubusercontent.com/u/155667?v=4"},"body":"On Wed, May 12, 2010 at 12:45 PM, Robert Buck wrote:\n> Today, just after someone else committed to my public repository I\n> started getting errors. Until then Git worked great.\n>\n> Does anyone know what is going on here? Are there particular versions\n> of Git with known issues around this?\n>\n>\n> uname@hostname:~/dev/workspaces/scm-evaluations/welcome.git/install/git-config$\n> git push\n> Counting objects: 7, done.\n> Delta compression using up to 2 threads.\n> Compressing objects: 100% (4/4), done.\n> Writing objects: 100% (4/4), 922 bytes, done.\n> Total 4 (delta 0), reused 4 (delta 0)\n> error: unable to create temporary sha1 filename ./objects/e6: File\n> exists\n>\n> fatal: failed to write object\n> error: unpack failed: unpacker exited with error code\n> To ssh://git.projectbedrock.com/var/cache/git/welcome.git\n>  ! [remote rejected] master -> master (n/a (unpacker error))\n> error: failed to push some refs to\n> 'ssh://git.projectbedrock.com/var/cache/git/welcome.git'\n\nThis is probably a permissions problem on the server. We use git over\nssh at $dayjob and we need to make sure everyone who pushes to a\nrepository on the server is a member of the same group and that the\nrepositories are created with \"git init --shared\" otherwise we run\ninto problems like this. Its not too much of an issue for us because\nwe have a maintainer model and the maintainers generally have the\nright permissions and don't change frequently.\n\nI think the \"shared\" part is probably the problem in this case because\nyou can both obviously create files on the server. Rhe problem appears\nto be when one of you needs to update a file (or directory) the other\ncreated.\n\nTo fix your current problem you'll just need to ssh into that server\nand find the welcome.git/objects directory and check the permissions\non the \"e6\" directory and its contents. You will keep running into\nthis problem until the permissions/sharing is sorted. Theres probably\na config variable which dictates the permissions to use when creating\nobjects on the server which is changed when you pass the \"--shared\"\noption to \"git init\", but I'm not sure what its is (I see some man\npages in your future).\n\n> As an aside, where the heck is the git bug tracker? I've searched, and\n> searched, and ... All I found is a Debian tracking system, which\n> appears to have no full text search capabilities.\n\nYou're looking at it bugs, patches, questions all go to this mailing\nlist. The archive on gmane[1] is conveniently search-able.\n\n[1] http://news.gmane.org/gmane.comp.version-control.git\n"},{"id":"141556","messageId":"20100513005218.GA20655@progeny.tock","threadId":"23788","inReplyTo":"AANLkTinV2U6Lbbl0N7jVAESEi0mZQ_D3slMEYa68vRT4@mail.gmail.com","subject":"Re: [remote rejected] master -> master (n/a (unpacker error))","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2010-05-13T00:52:19Z","receivedAt":"2010-05-13T00:52:19Z","isPatch":false,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"Hi Robert,\n\nRobert Buck wrote:\n\n> error: unable to create temporary sha1 filename ./objects/e6: File exists\n\nYeah, this error message is not so great.\n\nThe relevant code is in sha1_file.c.\n\n\tfd = create_tmpfile(tmpfile, sizeof(tmpfile), filename);\n\twhile (fd < 0 && errno == EMFILE && unuse_one_window(packed_git, -1))\n\t\tfd = create_tmpfile(tmpfile, sizeof(tmpfile), filename);\n\tif (fd < 0) {\n\t\tif (errno == EACCES)\n\t\t\treturn error(\"insufficient permission for adding an object to repository database %s\\n\", get_object_directory());\n\t\telse\n\t\t\treturn error(\"unable to create temporary sha1 filename %s: %s\\n\", tmpfile, strerror(errno));\n\t}\n\ncreate_tmpfile() creates a filename of the form\n./objects/e6/tmp_obj_ and tries to open that file.\nThe random value is based on the current time and the process ID of\nthe current process. If the file exists, it tries again with another\ncollection of random letters, up to 16384 times.\n\nIn your case, all 16384 trials yielded the same result: file already\nexisted. As a workaround, I’d suggest\n\n rm -f .git/objects/??/tmp_obj_*\n\nbut it might be nice to get a listing with \"ls -lR .git/objects\" first\nfor post-mortem analysis.\n\nAnd presumably the directory filled with temporary files that could\nnot be renamed to a proper name for some reason. Probably a permissions\nproblem, as Chris suggested.\n\n-- 8< --\nSubject: write_loose_object(): improve error message for some mkstemp failures\n\nIf the .git/objects/ab/ directory fills up with tmp_obj_ files, the\nresult is a cryptic error:\n\n error: unable to create temporary sha1 filename ./objects/e6: File exists\n\nReplace it with the slightly less cryptic\n\n error: cannot write temporary file under ./objects/e6: all the good filenames are taken\n\nReported-by: Robert Buck \nSigned-off-by: Jonathan Nieder \n---\n> As an aside, where the heck is the git bug tracker?\n\nHere is an answer from the last time it came up[1]:\n\n See http://thread.gmane.org/gmane.comp.version-control.git/136500\n\n Short answer: the usual method is to report bugs to the list,\n preferably with a patch for t/ or even better, a fix. \n\n> I've searched, and\n> searched, and ... All I found is a Debian tracking system, which\n> appears to have no full text search capabilities.\n\nhttp://merkel.debian.org/~don/cgi/search.cgi\nhttp://www.google.com/search?q=site:bugs.debian.org+\"Package:+git\"+\"file+exists\"\n\nThoughts? Improvements?\nJonathan\n\n[1] http://thread.gmane.org/gmane.linux.debian.devel.bugs.general/680778/focus=141598\n\n sha1_file.c | 4 ++++\n 1 files changed, 4 insertions(+), 0 deletions(-)\n\ndiff --git a/sha1_file.c b/sha1_file.c\nindex 28c056e..a2aa301 100644\n--- a/sha1_file.c\n+++ b/sha1_file.c\n@@ -2288,6 +2288,10 @@ static int write_loose_object(const unsigned char *sha1, char *hdr, int hdrlen,\n \tif (fd < 0) {\n \t\tif (errno == EACCES)\n \t\t\treturn error(\"insufficient permission for adding an object to repository database %s\\n\", get_object_directory());\n+\t\telse if (errno == EEXIST)\n+\t\t\treturn error(\"cannot write temporary file under %s: \"\n+\t\t\t \"all the good filenames are taken\\n\",\n+\t\t\t tmpfile);\n \t\telse\n \t\t\treturn error(\"unable to create temporary sha1 filename %s: %s\\n\", tmpfile, strerror(errno));\n \t}\n-- \n1.7.1\n"},{"id":"141567","messageId":"AANLkTilz_gbHl_RLyOuvEIdjPoUDIfZTUCpnswdHTiej@mail.gmail.com","threadId":"23788","inReplyTo":"20100513005218.GA20655@progeny.tock","subject":"Re: [remote rejected] master -> master (n/a (unpacker error))","fromName":"Robert Buck","fromEmail":"buck.robert.j@gmail.com","sentAt":"2010-05-13T09:30:43Z","receivedAt":"2010-05-13T09:30:43Z","isPatch":false,"sender":{"key":"buck.robert.j@gmail.com","avatar":"https://gravatar.com/avatar/1686742e8ac2595378aac67f26fd638ddaf494194f9af0cb6082c4a7eca1a366?d=mp&s=160"},"body":"Thanks.\n\nYes, the repository is shared by several people, and in geographically\ndifferent locations, ssh-ing to the same host, under different groups.\nSo your recommendation would be to use --shared. But this won't work\nso well out in the wild will it? Meaning, what if people's accounts\nare NOT under the same group that is?\n\nIt would sound to me like in general, when one goes to production with\na git environment that some sort of chrooted account is preferable if\nnot highly recommended?\n\nBob\n"},{"id":"141579","messageId":"rmiy6fo3t3n.fsf@fnord.ir.bbn.com","threadId":"23788","inReplyTo":"AANLkTilz_gbHl_RLyOuvEIdjPoUDIfZTUCpnswdHTiej@mail.gmail.com","subject":"Re: [remote rejected] master -> master (n/a (unpacker error))","fromName":"Greg Troxel","fromEmail":"gdt@ir.bbn.com","sentAt":"2010-05-13T12:05:00Z","receivedAt":"2010-05-13T12:05:00Z","isPatch":false,"sender":{"key":"gdt@ir.bbn.com","avatar":null},"body":"\nRobert Buck writes:\n\n> Yes, the repository is shared by several people, and in geographically\n> different locations, ssh-ing to the same host, under different groups.\n> So your recommendation would be to use --shared. But this won't work\n> so well out in the wild will it? Meaning, what if people's accounts\n> are NOT under the same group that is?\n\nGit simply rides on filesystem permissions. So you choose a group to\ncontrol access to the repository, chgrp -R the repo to that group, and\nconfig shared=0660. Then you put people in the group to give them\naccess; it doesn't have to be their primary gid. I don't follow your\nobjection; you seem to want to use groups to control access yet not set\nup a group for the repo.\n\nOn some systems (e.g. BSD), directories automatically inherit the parent\ndir's group. On others, you need setgid bit. I have the impression\nthat git will deal with this all correctly if you simply have\n\"sharedrepository = 0660\" under [core] in config; I would expect it to\nchgrp new files/dirs as needed to match the repo dir's group.\n\nI don't see how chroot would change the issues above.\n\n"},{"id":"141587","messageId":"m2r5lgaqdb.fsf@igel.home","threadId":"23788","inReplyTo":"20100513005218.GA20655@progeny.tock","subject":"Re: [remote rejected] master -> master (n/a (unpacker error))","fromName":"Andreas Schwab","fromEmail":"schwab@linux-m68k.org","sentAt":"2010-05-13T13:22:08Z","receivedAt":"2010-05-13T13:22:08Z","isPatch":false,"sender":{"key":"schwab@linux-m68k.org","avatar":"https://avatars.githubusercontent.com/u/2175493?v=4"},"body":"Jonathan Nieder writes:\n\n> In your case, all 16384 trials yielded the same result: file already\n> existed.\n\nIMHO it is much more likely that a race happened between two git\nprocesses each wanting to create the .git/objects/e6 directory.\n\nAndreas.\n\n-- \nAndreas Schwab, schwab@linux-m68k.org\nGPG Key fingerprint = 58CA 54C7 6D53 942B 1756 01D3 44D5 214B 8276 4ED5\n\"And now for something completely different.\"\n"},{"id":"141589","messageId":"20100513135619.GA16848@progeny.tock","threadId":"23788","inReplyTo":"m2r5lgaqdb.fsf@igel.home","subject":"Re: [remote rejected] master -> master (n/a (unpacker error))","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2010-05-13T13:56:19Z","receivedAt":"2010-05-13T13:56:19Z","isPatch":false,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"Andreas Schwab wrote:\n> Jonathan Nieder writes:\n\n>> In your case, all 16384 trials yielded the same result: file already\n>> existed.\n>\n> IMHO it is much more likely that a race happened between two git\n> processes each wanting to create the .git/objects/e6 directory.\n\nGood catch. But wasn’t the problem reproducible?\n\nIn any event, that such a race is possible is not so nice. Here’s\na naïve fix; it does not address other races, such as hash-object\nversus prune. Maybe git ought to acquire some sort of lock before\nwriting to the object dir in a shared clone.\n\ndiff --git a/sha1_file.c b/sha1_file.c\nindex bbb819f..d305e53 100644\n--- a/sha1_file.c\n+++ b/sha1_file.c\n@@ -2244,6 +2244,13 @@ static inline int directory_size(const char *filename)\n \treturn s - filename + 1;\n }\n \n+static int ensure_directory_exists(const char *dir)\n+{\n+\tif (mkdir(dir, 0777) && errno != EEXIST)\n+\t\treturn -1;\n+\treturn adjust_shared_perm(dir);\n+}\n+\n /*\n * This creates a temporary file in the same directory as the final\n * 'filename'\n@@ -2266,7 +2273,7 @@ static int create_tmpfile(char *buffer, size_t bufsiz, const char *filename)\n \t\t/* Make sure the directory exists */\n \t\tmemcpy(buffer, filename, dirlen);\n \t\tbuffer[dirlen-1] = 0;\n-\t\tif (mkdir(buffer, 0777) || adjust_shared_perm(buffer))\n+\t\tif (ensure_directory_exists(buffer))\n \t\t\treturn -1;\n \n \t\t/* Try again */\n-- \n"},{"id":"273802","messageId":"1448660253143-7643470.post@n2.nabble.com","threadId":"23788","inReplyTo":"20100513005218.GA20655@progeny.tock","subject":"Re: [remote rejected] master -> master (n/a (unpacker error))","fromName":"DavidLeeCrites","fromEmail":"lee@critesclan.com","sentAt":"2015-11-27T21:37:33Z","receivedAt":"2015-11-27T21:37:33Z","isPatch":false,"sender":{"key":"lee@critesclan.com","avatar":null},"body":"I am getting the same kinds of errors, but the resolutions offered here did\nnot work. After using the ideas (that there was a tmp_* file I did not have\nperms to write to, I started doing some global searches. \n\nOne such was this (from inside .git/objects):\n# ls -alR | grep tmp\nls: reading directory ./97: Input/output error\n\nSo I tried:\n# cd 97\n# ls -l\nls: reading directory .: Input/output error\ntotal 0K\n\nTo fix it, I did this:\n# cd ..\n# rm -fr ./97\n\nThe git push then worked fine.\n\nI'll add a few more pieces to the puzzle. I have some of my git repositories\non a USB drive (the ones I get this issue with). I move it from system to\nsystem. When git works, it works okay. But this irritant hits me about once\na week. My previous solution was to blow away the repo and rebuild it\n(something suggested several times here). This is the first time I have\nfound a workaround.\n\nThese are my private repositories that hold my private files. I have a\ngithub account I use for my public ones, plus my company has both a public\ngithub and their own privately hosted github. So the same exact computer\nsystems (laptops and VMs) use all four with impunity. Almost all of them are\nlinux based -- a mix of CentOS 7.x and Linux Mint 14.x; all using git\nv1.9.1. The one exception is osx. Thus the (brand new Toshiba 4T) USB drive\nis built with the exFAT filesystem. When it works, it works okay; but as I\nsaid, one of my dozen git repos will fail like this on a weekly basis. \n\nNone of the items in my dockerhub or artifactory fail, nor do my rsnapshot\nprocesses, or VLC/Banshee, etc. \n\nI've pretty much isolated it down to git. It is the ONLY app that fails. I\nhave noted in the past few months that the frequency of errors tells me I\ncannot be using the USB drive for anything else while git is accessing the\ndrive. [mac specific: it is better when I use a USB 2.0 hub to plug the\ndrive in; we all are probably aware that the mac seems to have more issues\nwith USB 3.0...]\n\nAnyway, I figured I'd toss this into the mix. Since it only happens once a\nweek or so, I cannot guarantee I'll have an update soon, but if someone is\ncurious, ping me, and I'll let you know when it happens again.\n\nDL\n\n\n\n\n\n--\nView this message in context: http://git.661346.n2.nabble.com/remote-rejected-master-master-n-a-unpacker-error-tp5043046p7643470.html\nSent from the git mailing list archive at Nabble.com.\n"}]}