{"thread":{"id":"40723","subject":"Re: drop connectivity check for local clones","startedAt":"2015-11-04T17:34:58Z","lastAt":"2015-11-04T17:34:58Z","messageCount":1,"participants":["Jeff King"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"272894","messageId":"20151104173458.GA11917@sigill.intra.peff.net","threadId":"40723","inReplyTo":"CAJR51WY-HSoTHpBm1jek2+fZyHQL3-n2J6aS-MBCDO5WtYTKRg@mail.gmail.com","subject":"Re: drop connectivity check for local clones","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2015-11-04T17:34:58Z","receivedAt":"2015-11-04T17:34:58Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"[I'm cc-ing the list, since I think this answer is of general interest.]\n\nOn Wed, Nov 04, 2015 at 11:00:34AM +0100, Matej Buday wrote:\n\n> I have a question somewhat regarding this old commit of yours:\n> https://github.com/git/git/commit/125a05fd0b45416558923b753f6418c24208d443\n> \n> Let me preface this by saying that I don't completely understand what the\n> connectivity check does...\n\nOne of the invariants git tries to remain in the repository is that for\nany object reachable from a ref (i.e., a branch or tag), we have all of\nthe ancestor objects. So if you have commit 125a05, you also have the\nparent, and its parent, and so on, down to the root.\n\nWhen we fetch or clone from a remote repository, it sends us some\nobjects, and we plan to point one of our refs at it. But rather than\ntrust that the remote sent us everything we need to maintain that\ninvariant, we actually walk the graph to make sure that is the case.\n\nThis can catch bugs or transfer errors early. So the operation is safer,\nat the expense of spending some CPU time.\n\nWe skip it for local disk-to-disk clones. We trust the source clone\nmore, and since the point of a local clone is to be very fast, the\nsafety/CPU tradeoff doesn't make as much sense.\n\n> Well, the question is -- is this check necessary\n> for local clones that use the --reference option?\n\nSort of. If you say:\n\n  git clone --reference /some/local/repo git://some-remote-repo\n\nThen we do check the incoming objects from some-remote-repo. However,\nthere is an optimization we don't do: we could assume that everything in\n/some/local/repo is fine, and stop traversing there. So if you fetch\nonly a few objects from the remote, that is all you would check.\n\nThe optimization would look something like this:\n\n  https://github.com/peff/git/commit/1254ff54b49eff19ec8a09c36e3edd24d490cae1\n\nI wrote that last year, but haven't actually submitted the patch yet.\nThere are two reasons:\n\n  1. It needs minor cleanup due to the sha1/oid transition that is\n     ongoing (see the \"ugh\" comment). I think this could be fixed by\n     refactoring some of the callback interfaces, but I haven't gotten\n     around to it.\n\n  2. Using alternates to optimize can backfire at a certain scale. If\n     you have a very large number of refs in the alternate repository,\n     just accessing and processing those refs can be more expensive than\n     walking the history graph in the first place.\n\n     This is the case for us at GitHub, where our alternates have the\n     refs for _all_ of the forks of a given project. So I would want\n     some flag to turn this behavior off.\n\n     Of course, we are in an exceptional circumstance at GitHub, and\n     that is no reason the topic cannot go upstream (we already carry\n     custom patches to disable alternates for things like receive-pack,\n     and could do the same here).\n\n     So that is not a good reason not to submit, only an explanation why\n     I have not yet bothered to spend the time on it. :)\n\n-Peff\n"}]}