{"thread":{"id":"32710","subject":"auto packing with simultaneous pushes: \"error: Could not stat 'objects/[…]/[…]'\"","startedAt":"2013-01-23T23:50:38Z","lastAt":"2013-01-24T01:39:50Z","messageCount":2,"participants":["Ivan D Vasin"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"207651","messageId":"CADcu=0jCkNr6J_-iiKp00aSXHPf=+QV6oFvFHUZ_uc02JTaCbQ@mail.gmail.com","threadId":"32710","inReplyTo":null,"subject":"auto packing with simultaneous pushes: \"error: Could not stat 'objects/[…]/[…]'\"","fromName":"Ivan D Vasin","fromEmail":"nisavid@gmail.com","sentAt":"2013-01-23T23:50:38Z","receivedAt":"2013-01-23T23:50:38Z","isPatch":false,"sender":{"key":"nisavid@gmail.com","avatar":null},"body":"my coworker and i just pushed some commits seconds apart from each\nother.  both of our pushes triggered a server-side auto pack\n(presumably as part of ``receive-pack``, as a result of the setting\n``receive.autogc = true``).  each auto pack produced multiple errors\nof the form ``error: Could not stat 'objects/[…]/[…]'``.  there\nappears to be no overlap between the two auto packs' lists of failed\nobject hashes.\n\nmy suspicion is that the two auto pack processes each generated a list\nof objects to pack, then proceeded to pack them, removing the\ncorresponding files as they went along.  as each one encountered an\nentry in its list that was previously handled by the other process, it\nfound that it was unable to stat the corresponding file because it had\nbeen removed by the other process.\n\nmy suggestion is that an auto pack should lock the repository,\npreventing at least other auto packs (and perhaps other operations)\nfrom running until it is finished.  alternatively, the auto pack can\nskip over objects that were already packed by another process;\nhowever, i imagine this would be much less efficient if done\ncorrectly.\n\nhere is the console output from my push:\n\n$ git push\nX11 forwarding request failed on channel 0\nCounting objects: 32, done.\nDelta compression using up to 8 threads.\nCompressing objects: 100% (18/18), done.\nWriting objects: 100% (18/18), 2.70 KiB, done.\nTotal 18 (delta 12), reused 0 (delta 0)\nAuto packing the repository for optimum performance.\nerror: Could not stat 'objects/00/\nc706ccf0ef06092134ccb1f9fdf0e7d39a9d5f'\nerror: Could not stat 'objects/02/7c2932047c9c9c7bacd6e1abdc3810866161bc'\nerror: Could not stat 'objects/13/b2c3f0a55e8de099553cad6901a150f19b83c2'\nerror: Could not stat 'objects/2a/3992fa9c096f7514a62781a3c3923a996a7073'\nerror: Could not stat 'objects/3d/cb6e9055b9b51911aae52f4bb0ca8d8719d645'\nerror: Could not stat 'objects/50/c2f6d8e0de1eba988256597cc1d6e1a387902c'\nerror: Could not stat 'objects/50/9f494f6b60741957a45cf51f8d097f0578c89a'\nerror: Could not stat 'objects/52/6597c12d730c5903d46f598c6ec438cfabfe0f'\nerror: Could not stat 'objects/52/a6b4ee4bebb043d21858455fd0cd33b4812a42'\nerror: Could not stat 'objects/53/71c6f99023efecb3a4c16daeeee63f534d63b2'\nerror: Could not stat 'objects/5f/bd49df3c4abdc93c7118f762ac770a0fcd6ce4'\nerror: Could not stat 'objects/67/900e6e598e8e1c7bde5776fd220f7daf147cde'\nerror: Could not stat 'objects/72/42ecd310d0286da6c3967985c6446cf105b0bd'\nerror: Could not stat 'objects/79/4afecb2d348dde5ad4d24cf62c8fcd4db3b63a'\nerror: Could not stat 'objects/7e/4c612c51c5d76f03f12c04241d3b3c5aa212e4'\nerror: Could not stat 'objects/84/fd1e038d4345bac1fa2b33edb73fd73813ac4f'\nerror: Could not stat 'objects/89/c32fdbf3400c797d01659f9ee644aa8a713574'\nerror: Could not stat 'objects/8b/cbe10b0d09325b49876416341d6cf4bccc9e35'\nerror: Could not stat 'objects/8d/6dcc95d6bea57cfafe8a08fed7947561682b68'\nerror: Could not stat 'objects/b9/67af0f23deb87f1708689d159ee6444503462a'\nerror: Could not stat 'objects/bb/9d6fafe3ba2189582ac48088ee6f732755dddb'\nerror: Could not stat 'objects/e0/549c20e56d6ec270ec3ffee6c93a6d6cb8f6b5'\nerror: Could not stat 'objects/f2/737f0d8635265a56d57c27a3ddf155da48b07a'\nerror: Could not stat 'objects/fc/8b2f0837aefa21d86de65f6230c86f64addb8d'\nerror: Could not stat 'objects/fc/ac21d190770fdee23e5e69cdba36eefd6e001a'\nerror: Could not stat 'objects/fd/ae521d6a3404ceabff223385d7b5b5c783f0fe'\nTo git@code.adverplex.com:syseng\n   fb49a86..e161cf5  HEAD -> uber_release_alpha\n\nhere is the console output from my coworker's push:\n\n$ git push\nCounting objects: 181, done.\nDelta compression using up to 4 threads.\nCompressing objects: 100% (154/154), done.\nWriting objects: 100% (159/159), 1.81 MiB, done.\nTotal 159 (delta 74), reused 0 (delta 0)\nAuto packing the repository for optimum performance.\nerror: Could not stat 'objects/06/b2aee0b4adca015d072be4ef6a655c118b6af3'\nerror: Could not stat 'objects/0d/66f41003c9112b8093dfee5fd56e8db25a142b'\nerror: Could not stat 'objects/0d/6c3e296fb7b83998677d207f9df6a69e144168'\nerror: Could not stat 'objects/14/64e2bd863a15e08be7aa2f33b8df5b5f695608'\nerror: Could not stat 'objects/14/19cad4e2a38720186eb36b3c756d05e1963877'\nerror: Could not stat 'objects/16/1b9e65004a9120ea250ef8ef8292d5416ed984'\nerror: Could not stat 'objects/23/b6bc50c16fe3740393c8a08cf57cc07a56c778'\nerror: Could not stat 'objects/60/1d7c562cafaf6c71c57dbbd0bab55c3698d0c7'\nerror: Could not stat 'objects/6c/c5c7ededba86ce291b531def840e757ce3eccc'\nerror: Could not stat 'objects/74/8d6bece2b4ccec41584d22152d0fd903e01908'\nerror: Could not stat 'objects/9a/6c21779e7c063b87bbf055e4c6b6baecc8a3e1'\nerror: Could not stat 'objects/9a/6519e7bb625f681ead373e9a9226f862ee1550'\nerror: Could not stat 'objects/9b/7027422cbce1f5f2c5929ae1313ecee2cd3537'\nerror: Could not stat 'objects/a4/7e0b5906c2b552158a1e5943063e983532988f'\nerror: Could not stat 'objects/a4/6db00ea7b7d2e618731d7b4f1e59b5f0276aff'\nerror: Could not stat 'objects/ab/b3a8fd9585af5a7f6dbd609e6b4ed8c4359ca6'\nerror: Could not stat 'objects/ad/0c98f2674807beee093cead1179d95756c1788'\nerror: Could not stat 'objects/bf/4fd1b9cc617d3fc832350783064418e0d7fdfd'\nerror: Could not stat 'objects/bf/c65d6bc705dfc5018c3d25228a4469d069f3b8'\nerror: Could not stat 'objects/c0/adb5b39efd31698789ee8aefcedf91913fc7f1'\nerror: Could not stat 'objects/ca/f4ee60f223039d2ec6185adbb4c28ab7df461e'\nerror: Could not stat 'objects/ca/e3b7c60b4b3a34bba4b4a5738edf0f7786a784'\nerror: Could not stat 'objects/cb/4a470752088b708290713b0a513e22bb0dcb15'\nerror: Could not stat 'objects/d3/d44ba75332e3e7e21a89eafe724e682cf124dd'\nerror: Could not stat 'objects/d3/8a635a420af938f680ab30c6252fdddd448a5d'\nerror: Could not stat 'objects/d8/f102045fc608b65cb1d75ad8de11a673ebd656'\nerror: Could not stat 'objects/da/97ac29d9024f589b4aec46bc0ea1f597b00802'\nerror: Could not stat 'objects/e5/3e9d05c4e9a3e336453af3b43fe06dfa5c6d57'\nerror: Could not stat 'objects/ea/b586bcc6925bcb34cdeede1562a851d5b58d90'\nerror: Could not stat 'objects/ec/f2cf1dd8d6045242676cd22320902c085ecbb3'\nerror: Could not stat 'objects/f3/819de13ed535bdd92580e2f2b01580714188e1'\nTo git@code.adverplex.com:syseng\n   fb49a86..a39c897  master -> master\n\n``git fsck`` is successful on both of our repos and on the bare repo\nto which we pushed.\n"},{"id":"207659","messageId":"CADcu=0jiEEOkWZXshLdMmOBYJt=SzHfJ39r_qo8GzjdY6BTRaA@mail.gmail.com","threadId":"32710","inReplyTo":"7vbocffmpn.fsf@alter.siamese.dyndns.org","subject":"Re: auto packing with simultaneous pushes: \"error: Could not stat 'objects/[…]/[…]'\"","fromName":"Ivan D Vasin","fromEmail":"nisavid@gmail.com","sentAt":"2013-01-24T01:39:50Z","receivedAt":"2013-01-24T01:39:50Z","isPatch":false,"sender":{"key":"nisavid@gmail.com","avatar":null},"body":"On Wed, Jan 23, 2013 at 7:28 PM, Junio C Hamano <gitster@pobox.com> wrote:\n> Ivan D Vasin <nisavid@gmail.com> writes:\n>\n>> my suggestion is that an auto pack should lock the repository,\n>> preventing at least other auto packs (and perhaps other operations)\n>> ...\n>>\n>> ``git fsck`` is successful on both of our repos and on the bare repo\n>> to which we pushed.\n>\n> Successful after you pushed, or before you pushed, or both?\n\nboth\n\n>\n> I suspect both.\n>\n> I do not think such a lock is necessary for correctness of the\n> operation, but running two auto packing sumultaneously is wasteful,\n> so it would help performance.  But that would produce a larger\n> problem.  What if your modified auto-packer takes a lock and then\n> dies without relinquishing the lock?  The repository will never be\n> repacked after such an event forever?\n\nperhaps the lock could contain the PID of the auto pack process.  if\nthat PID has gone away, the lock is ignored and replaced with a new\none.\n\nthat's what comes to my mind.  of course, there could be other ways to\nhandle this that i'm not thinking of.\n\nin any case, the error messages, though spurious, are alarming to the\nuninformed user.  it looks like Git is saying that there is actual\ndata loss, where in fact there is none.  if Git doesn't prevent these\nmessages from appearing (via locking behavior or otherwise), then it\nshould at least annotate them with a message that describes their\npossibly spurious nature and perhaps instructs the user to verify\neverything with ``git fsck``.\n"}]}