{"thread":{"id":"41143","subject":"Some issues when trying to set up a shallow git mirror server","startedAt":"2016-01-07T16:54:17Z","lastAt":"2016-01-13T17:43:32Z","messageCount":11,"participants":["Richard Maw","Junio C Hamano","Duy Nguyen"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"275491","messageId":"20160107165417.GB3397@logi.codethink.co.uk","threadId":"41143","inReplyTo":null,"subject":"Some issues when trying to set up a shallow git mirror server","fromName":"Richard Maw","fromEmail":"richard.maw@codethink.co.uk","sentAt":"2016-01-07T16:54:17Z","receivedAt":"2016-01-07T16:54:17Z","isPatch":false,"sender":{"key":"richard.maw@codethink.co.uk","avatar":"https://gravatar.com/avatar/5f76a769a68607d8ebd829d59eef4d9ce85f711b2e9b3269697ec0ea6bfe1b7a?d=mp&s=160"},"body":"Hi all.\n\nI've been working on a service that mirrors open source code into git,\n(http://git.baserock.org/cgi-bin/cgit.cgi/?q=delta if interested).\n\nSome repositories are too unwieldy to mirror the whole history,\nso we're looking at shallow support,\nby fetching specified branches with `--depth=1`.\n\nSince we'll be fetching in only a very shallow history,\nwe found that most of the time git classifies this as non-fast-forward.\n\nI can see why this happens,\nsince it ends up with non-overlapping ranges of history\nit hasn't fetched enough commits to join the dots.\n\nThis is inconvenient for us,\nas we were explicitly using refspecs which didn't force the fetch,\nsince we were using the \"non fast-forward update\" errors\nto detect whether upstream force pushed important refs\nwhich could be a sign of tampering.\n\nWhile the client doesn't have enough information\nthe server has those commits.\nWould it make sense for the server to be able to tell the client\n\"trust me, that commit is a descendant of the previous one\"?\n\nHere's a patch to the test suite that demonstrates the failure,\nsince I felt I had to put some commands together to demonstrate,\nand I may as well put it in a useful format.\n\ndiff --git a/t/t5537-fetch-shallow.sh b/t/t5537-fetch-shallow.sh\nindex a980574..0996d12 100755\n--- a/t/t5537-fetch-shallow.sh\n+++ b/t/t5537-fetch-shallow.sh\n@@ -186,4 +186,24 @@ EOF\n \ttest_cmp expect actual\n '\n \n+test_expect_success 'fetch --depth with discontinuous history' '\n+\tgit init --quiet --bare discontinuous.git &&\n+\t(\n+\tcd discontinuous.git &&\n+\tgit fetch --quiet ../.git --depth=1 \"refs/*:refs/*\"\n+\t) &&\n+\tcommit 5 &&\n+\tcommit 6 &&\n+\t(\n+\tcd discontinuous.git &&\n+\tgit fetch ../.git --depth=1 \"refs/*:refs/*\" &&\n+\tgit fsck &&\n+\tgit log --format=%s master >actual &&\n+\tcat <expect &&\n+6\n+EOF\n+\ttest_cmp expect actual\n+\t)\n+'\n+\n test_done\n\nI also encountered weird behaviour when trying to push the history,\nthe general symptom being that the server would only update one ref,\nso the client would fail because the server didn't mention all the refs.\n\ndiff --git a/t/t5538-push-shallow.sh b/t/t5538-push-shallow.sh\nindex ceee95b..d0bd538 100755\n--- a/t/t5538-push-shallow.sh\n+++ b/t/t5538-push-shallow.sh\n@@ -120,4 +120,27 @@ EOF\n \tgit cat-file blob `echo 1|git hash-object --stdin` >/dev/null\n \t)\n '\n+\n+test_expect_success 'push --mirror from shallow clone' '\n+\tgit --git-dir=full/.git tag r0 &&\n+\tgit init --quiet --bare mirror-pusher.git &&\n+\tgit init --quiet --bare mirror.git &&\n+\tgit --git-dir=mirror.git config receive.shallowUpdate true &&\n+\t(\n+\tcd mirror-pusher.git &&\n+\tgit remote add --mirror=fetch origin ../full/.git &&\n+\tgit remote add mirror ../mirror.git &&\n+\tgit fetch --quiet origin --depth=1 &&\n+\tgit push --mirror mirror\n# The push should work, but does not.\n# The following command works for creations, but not deletions.\n#git for-each-ref -s --format \"git push mirror +%(refname)\" | sh\n+\t) &&\n+\tgit --git-dir=mirror.git for-each-ref --format \"%(refname)\" --sort refname >actual &&\n+\tcat <expect &&\n+refs/heads/master\n+refs/remotes/origin/HEAD\n+refs/remotes/origin/master\n+refs/tags/r0\n+EOF\n+\ttest_cmp expect actual\n+'\n+\n test_done\n"},{"id":"275492","messageId":"xmqq4mep5kyg.fsf@gitster.mtv.corp.google.com","threadId":"41143","inReplyTo":"20160107165417.GB3397@logi.codethink.co.uk","subject":"Re: Some issues when trying to set up a shallow git mirror server","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2016-01-07T18:00:07Z","receivedAt":"2016-01-07T18:00:07Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Richard Maw writes:\n\n> This is inconvenient for us,\n> as we were explicitly using refspecs which didn't force the fetch,\n> since we were using the \"non fast-forward update\" errors\n> to detect whether upstream force pushed important refs\n> which could be a sign of tampering.\n>\n> While the client doesn't have enough information\n> the server has those commits.\n> Would it make sense for the server to be able to tell the client\n> \"trust me, that commit is a descendant of the previous one\"?\n\nIt does not in our security model, as you do not blindly trust the\nother side, whether you are a \"client\" or a \"server\".\n"},{"id":"275526","messageId":"20160108101903.GC3397@logi.codethink.co.uk","threadId":"41143","inReplyTo":"xmqq4mep5kyg.fsf@gitster.mtv.corp.google.com","subject":"Re: Some issues when trying to set up a shallow git mirror server","fromName":"Richard Maw","fromEmail":"richard.maw@codethink.co.uk","sentAt":"2016-01-08T10:19:03Z","receivedAt":"2016-01-08T10:19:03Z","isPatch":false,"sender":{"key":"richard.maw@codethink.co.uk","avatar":"https://gravatar.com/avatar/5f76a769a68607d8ebd829d59eef4d9ce85f711b2e9b3269697ec0ea6bfe1b7a?d=mp&s=160"},"body":"On Thu, Jan 07, 2016 at 10:00:07AM -0800, Junio C Hamano wrote:\n> Richard Maw writes:\n> > This is inconvenient for us,\n> > as we were explicitly using refspecs which didn't force the fetch,\n> > since we were using the \"non fast-forward update\" errors\n> > to detect whether upstream force pushed important refs\n> > which could be a sign of tampering.\n> >\n> > While the client doesn't have enough information\n> > the server has those commits.\n> > Would it make sense for the server to be able to tell the client\n> > \"trust me, that commit is a descendant of the previous one\"?\n> \n> It does not in our security model, as you do not blindly trust the\n> other side, whether you are a \"client\" or a \"server\".\n\nFair enough.\nI didn't know whether Git passed responsibility for that to the transport layer.\n\nWould a mode for fetch to also include the commit chain without the trees fit\nthe security model?\n"},{"id":"275528","messageId":"CACsJy8CBj-oMTLrn8gVudhLPEixZfkwFv9BaN8LSNFvOHkqKkg@mail.gmail.com","threadId":"41143","inReplyTo":"20160108101903.GC3397@logi.codethink.co.uk","subject":"Re: Some issues when trying to set up a shallow git mirror server","fromName":"Duy Nguyen","fromEmail":"pclouds@gmail.com","sentAt":"2016-01-08T10:44:30Z","receivedAt":"2016-01-08T10:44:30Z","isPatch":false,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"On Fri, Jan 8, 2016 at 5:19 PM, Richard Maw wrote:\n> On Thu, Jan 07, 2016 at 10:00:07AM -0800, Junio C Hamano wrote:\n>> Richard Maw writes:\n>> > This is inconvenient for us,\n>> > as we were explicitly using refspecs which didn't force the fetch,\n>> > since we were using the \"non fast-forward update\" errors\n>> > to detect whether upstream force pushed important refs\n>> > which could be a sign of tampering.\n>> >\n>> > While the client doesn't have enough information\n>> > the server has those commits.\n>> > Would it make sense for the server to be able to tell the client\n>> > \"trust me, that commit is a descendant of the previous one\"?\n>>\n>> It does not in our security model, as you do not blindly trust the\n>> other side, whether you are a \"client\" or a \"server\".\n>\n> Fair enough.\n> I didn't know whether Git passed responsibility for that to the transport layer.\n>\n> Would a mode for fetch to also include the commit chain without the trees fit\n> the security model?\n\nIt sounds a lot like what I did with narrow clone [2] prototype. A\nnarrow clone only contains enough objects for certain paths so there's\na chance that we just don't have enough to do a proper merge. A\nserver-side command was added [1] to retrieve enough objects for the\ntask. If you fetch commit chain without all necessary trees and\nobjects, your repo is \"broken\" from Git point of view and you'll need\nto do some extra work to make sure your repo is not actually broken.\n\n[1] http://article.gmane.org/gmane.comp.version-control.git/154371\n[2] http://thread.gmane.org/gmane.comp.version-control.git/154343\n-- \nDuy\n"},{"id":"275530","messageId":"20160108105204.GD3397@logi.codethink.co.uk","threadId":"41143","inReplyTo":"CACsJy8CBj-oMTLrn8gVudhLPEixZfkwFv9BaN8LSNFvOHkqKkg@mail.gmail.com","subject":"Re: Some issues when trying to set up a shallow git mirror server","fromName":"Richard Maw","fromEmail":"richard.maw@codethink.co.uk","sentAt":"2016-01-08T10:52:04Z","receivedAt":"2016-01-08T10:52:04Z","isPatch":false,"sender":{"key":"richard.maw@codethink.co.uk","avatar":"https://gravatar.com/avatar/5f76a769a68607d8ebd829d59eef4d9ce85f711b2e9b3269697ec0ea6bfe1b7a?d=mp&s=160"},"body":"On Fri, Jan 08, 2016 at 05:44:30PM +0700, Duy Nguyen wrote:\n> It sounds a lot like what I did with narrow clone [2] prototype. A\n> narrow clone only contains enough objects for certain paths so there's\n> a chance that we just don't have enough to do a proper merge. A\n> server-side command was added [1] to retrieve enough objects for the\n> task. If you fetch commit chain without all necessary trees and\n> objects, your repo is \"broken\" from Git point of view and you'll need\n> to do some extra work to make sure your repo is not actually broken.\n> \n> [1] http://article.gmane.org/gmane.comp.version-control.git/154371\n> [2] http://thread.gmane.org/gmane.comp.version-control.git/154343\n\nThanks for the links. I'll have a read to see how it fits together.\n"},{"id":"275576","messageId":"xmqqd1tb21oh.fsf@gitster.mtv.corp.google.com","threadId":"41143","inReplyTo":"xmqq4mep5kyg.fsf@gitster.mtv.corp.google.com","subject":"Re: Some issues when trying to set up a shallow git mirror server","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2016-01-08T21:37:02Z","receivedAt":"2016-01-08T21:37:02Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Junio C Hamano writes:\n\n> Richard Maw writes:\n>\n>> This is inconvenient for us,\n>> as we were explicitly using refspecs which didn't force the fetch,\n>> since we were using the \"non fast-forward update\" errors\n>> to detect whether upstream force pushed important refs\n>> which could be a sign of tampering.\n>>\n>> While the client doesn't have enough information\n>> the server has those commits.\n>> Would it make sense for the server to be able to tell the client\n>> \"trust me, that commit is a descendant of the previous one\"?\n>\n> It does not in our security model, as you do not blindly trust the\n> other side, whether you are a \"client\" or a \"server\".\n\nThinking about it more, I think my answer was flawed. \n\nA client may ask the server to give a history, and before it accepts\nthe result, it must do its own consistency check. A client may send\nits history, and before the server accepts it, the server must do\nits own consistency check. This is the general principle around our\nconsistency model (it is not limited to \"security\", but it is more\nabout \"correctness\" in general).\n\nWhile that consistency principle must hold everywhere in our system,\nit does not mean a client cannot ask a server to do something whose\nresult it has to trust, at least to some degree, because there is\nfundamentally no way to independenty verify the result.\n\nI think what you were hinting falls into that category. The client\ncannot verify that the new tip is a descendant of the old one\nwithout having the full history between these two points, but\ntransfering that history would defeat the whole point of using a\nshallowed history.\n\nSo in that sense I do think that such a protocol extension does make\nsense. But it would involve some extra pieces of information that\nneed to be sent between the client and the server, not just \"I'll\ntrust you\".\n\nIn the current \"git fetch\" protocol, the sender gives the receiver a\nlist of refs, and for each of these refs, the object name it points\nat. Then the receiver asks the sender to give it the objects that\nappear in the history leading to a set of objects (aka \"want\"), and\ntells the sender what objects it completely has already (aka\n\"have\"). The idea is that the sender can exclude objects that are\nreachable from \"have\"s when it enumerates the objects that need to\nbe sent. When the receiver is shallow, it also tells the sender\nthat its current history is truncated at such and such commits,\nlacking things before them. After that exchange, the sender just\ngives the receiver a packfile that contains the objects requested.\nThe receiver verifies that the packfile makes sense, e.g. it has all\nthe \"want\" objects it asked for, and the objects that they refer to\n(recursively) are available, before updating its refs with (some of)\nthe \"want\" objects.\n\nIn the simplest case, for example, where you might have a single\nrefs/heads/master that currently points at O and the other side has\nN at its refs/heads/master, the sender would say \"I have N at\nrefs/heads/master\" and the receiver would say \"I want N, and I have\nO\". A shallow receiver may also say \"I do not have history behind\nO\" and \"I only want a history 1 commit deep\". And the sender would\nsend objects that is necessary to satisfy the request, e.g. commit N\nitself and the trees and blobs in commit N that are not common with\ncommit O.\n\nNotice that nowhere in this exchange the receiver tells the sender\nwhat it intends to do with the \"want\" objects?\n\nBecause the receiver does not say \"I want N and I intend to replace\nO I currently have refs/heads/master\", there is no way for the\nsender to say \"trust me, N is a descendant of O\". It simply does\nnot know if the receiver _cares_ how N and O are related with each\nother.\n\nSo if you want to do this, a new protocol extension needs to allow\nyour updated sender (upload-pack) and receiver (fetch-pack) to work\nmore like this:\n\n * The sender would advertise \"I support that extension\", while\n giving the usual \"here are my refs and its current values\".\n\n * The receiver would say \"I want to use that extension\", and to\n each of its \"want\" (which usually consists of \"want\" followed by\n an object name and nothing else), it optionally adds names of the\n objects it wants to verify ancestry relationship with.\n\n E.g. if you have O at the tip of the master branch and P at the\n tip of the maint branch, the sender has N at both of these two\n branches, and if you are updating your master and maint with\n their master and maint, you would say something like \"want N O P\"\n to tell the sender that you want history leading to N, and you\n want to see if N is a descendant of O and if N is a descendant of\n P.\n\n * The receiver and the sender then does the usual \"have\"/\"ack\"\n exchange, which does not have to change any behaviour with this\n extension.\n\n * Finally, when the sender sends out the resulting packfile, it\n also has to tell the receiver which of the object pairs the\n receiver asked it to check the ancestry relationship violate the\n fast-forward rule. In the earlier example of fast-forwarding O\n and P with N, where the receiver asked \"want N O P\", the receiver\n asked to check object pairs and . If P fast-forwards\n to N but O does not, then the sender would tell the receiver the\n fact that \"O does not fast forward to N\".\n\nWith such an extension, your updated receiver can receive the\nnecessary objects to update your history to \"N\", but notice that it\nwould result in non-ff update to update master (that used to be O)\nwith the new commit N.\n"},{"id":"275649","messageId":"20160111155153.GE3397@logi.codethink.co.uk","threadId":"41143","inReplyTo":"xmqqd1tb21oh.fsf@gitster.mtv.corp.google.com","subject":"Re: Some issues when trying to set up a shallow git mirror server","fromName":"Richard Maw","fromEmail":"richard.maw@codethink.co.uk","sentAt":"2016-01-11T15:51:53Z","receivedAt":"2016-01-11T15:51:53Z","isPatch":false,"sender":{"key":"richard.maw@codethink.co.uk","avatar":"https://gravatar.com/avatar/5f76a769a68607d8ebd829d59eef4d9ce85f711b2e9b3269697ec0ea6bfe1b7a?d=mp&s=160"},"body":"On Fri, Jan 08, 2016 at 01:37:02PM -0800, Junio C Hamano wrote:\n\nThanks for the write-up.\nI knew most of how the protocol worked, but not all,\nand I appreciate the write-up of how the extension could work.\n\nIf I have time I can see if I can implement this protocol extension myself.\n\n\n> While that consistency principle must hold everywhere in our system,\n> it does not mean a client cannot ask a server to do something whose\n> result it has to trust, at least to some degree, because there is\n> fundamentally no way to independenty verify the result.\n\nSo there'll be a greater level of trust required in the transport protocol,\nso you wouldn't necessarily want to allow this protocol over git:// or http://.\n\n\n> So if you want to do this, a new protocol extension needs to allow\n> your updated sender (upload-pack) and receiver (fetch-pack) to work\n> more like this:\n> \n> * The sender would advertise \"I support that extension\", while\n> giving the usual \"here are my refs and its current values\".\n> \n> * The receiver would say \"I want to use that extension\", and to\n> each of its \"want\" (which usually consists of \"want\" followed by\n> an object name and nothing else), it optionally adds names of the\n> objects it wants to verify ancestry relationship with.\n> \n> E.g. if you have O at the tip of the master branch and P at the\n> tip of the maint branch, the sender has N at both of these two\n> branches, and if you are updating your master and maint with\n> their master and maint, you would say something like \"want N O P\"\n> to tell the sender that you want history leading to N, and you\n> want to see if N is a descendant of O and if N is a descendant of\n> P.\n> \n> * The receiver and the sender then does the usual \"have\"/\"ack\"\n> exchange, which does not have to change any behaviour with this\n> extension.\n> \n> * Finally, when the sender sends out the resulting packfile, it\n> also has to tell the receiver which of the object pairs the\n> receiver asked it to check the ancestry relationship violate the\n> fast-forward rule. In the earlier example of fast-forwarding O\n> and P with N, where the receiver asked \"want N O P\", the receiver\n> asked to check object pairs and . If P fast-forwards\n> to N but O does not, then the sender would tell the receiver the\n> fact that \"O does not fast forward to N\".\n\nSo this would be another step after the receiver communicates \"done\",\nbut before the pack file itself gets sent,\nso the sender can determine that it doesn't need to tell the reciever\nthe relationship between two commits,\nsince it knows you can work it out yourself?\n\nOtherwise if it can probably communicate the relationships before the have list\nis sent, and extend the shallow-update part of the protocol instead, which\nmight be simpler.\n\n> With such an extension, your updated receiver can receive the\n> necessary objects to update your history to \"N\", but notice that it\n> would result in non-ff update to update master (that used to be O)\n> with the new commit N.\n\nCode-wise, does the following make sense?\n\n1. Add a field to `struct ref` to flag a \"trusted fast-forward\".\n2. Change `find_common()` in `fetch-pack.c`\n and `receive_needs()` in `upload-pack.c`\n to communicate the relationships we're interested in as above,\n and set the \"trusted fast-forward\" flag.\n3. Change `update_local_ref()` in `builtin/fetch.c`\n to check `ref->trusted_fast_forward || in_merge_bases(current, updated)`.\n"},{"id":"275824","messageId":"xmqqy4butzw8.fsf@gitster.mtv.corp.google.com","threadId":"41143","inReplyTo":"20160111155153.GE3397@logi.codethink.co.uk","subject":"Re: Some issues when trying to set up a shallow git mirror server","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2016-01-12T18:29:27Z","receivedAt":"2016-01-12T18:29:27Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Richard Maw writes:\n\n> On Fri, Jan 08, 2016 at 01:37:02PM -0800, Junio C Hamano wrote:\n> ...\n>> So if you want to do this, a new protocol extension needs to allow\n>> your updated sender (upload-pack) and receiver (fetch-pack) to work\n>> more like this:\n>> ...\n>> * Finally, when the sender sends out the resulting packfile, it\n>> also has to tell the receiver which of the object pairs the\n>> receiver asked it to check the ancestry relationship violate the\n>> fast-forward rule. In the earlier example of fast-forwarding O\n>> and P with N, where the receiver asked \"want N O P\", the receiver\n>> asked to check object pairs and . If P fast-forwards\n>> to N but O does not, then the sender would tell the receiver the\n>> fact that \"O does not fast forward to N\".\n>\n> So this would be another step after the receiver communicates \"done\",\n> but before the pack file itself gets sent,\n> so the sender can determine that it doesn't need to tell the reciever\n> the relationship between two commits,\n> since it knows you can work it out yourself?\n>\n> Otherwise if it can probably communicate the relationships before the have list\n> is sent, and extend the shallow-update part of the protocol instead, which\n> might be simpler.\n\nI left the detail as vague ;-).\n\nThe new request does not have to piggyback on existing \"want\"\nmessage. And thinking about it again, it probably is cleaner if it\ndidn't. After the use of the protocol extension \"ancestry-check\" is\nnegotiated the usual way between the sender and the receiver, the\nreceiver would send \"check-ff N O\" and \"check-ff N P\" after it sends\nall of its \"want\" messages but before it sends the \"flush\" to go\ninto the \"have\"/\"ack\" common ancestry discovery.\n\nI do not have a strong opinion on where the sender should reply with\n\"not-ff N O\" in the protocol. Immediately after the receiver says\n\"I've done with my 'want's (and now 'check-ff's)\" by flushing may be\na good place to do so.\n\n>> With such an extension, your updated receiver can receive the\n>> necessary objects to update your history to \"N\", but notice that it\n>> would result in non-ff update to update master (that used to be O)\n>> with the new commit N.\n>\n> Code-wise, does the following make sense?\n>\n> 1. Add a field to `struct ref` to flag a \"trusted fast-forward\".\n> 2. Change `find_common()` in `fetch-pack.c`\n> and `receive_needs()` in `upload-pack.c`\n> to communicate the relationships we're interested in as above,\n> and set the \"trusted fast-forward\" flag.\n> 3. Change `update_local_ref()` in `builtin/fetch.c`\n> to check `ref->trusted_fast_forward || in_merge_bases(current, updated)`.\n"},{"id":"275888","messageId":"20160113113743.GF3397@logi.codethink.co.uk","threadId":"41143","inReplyTo":"xmqqy4butzw8.fsf@gitster.mtv.corp.google.com","subject":"Re: Some issues when trying to set up a shallow git mirror server","fromName":"Richard Maw","fromEmail":"richard.maw@codethink.co.uk","sentAt":"2016-01-13T11:37:43Z","receivedAt":"2016-01-13T11:37:43Z","isPatch":false,"sender":{"key":"richard.maw@codethink.co.uk","avatar":"https://gravatar.com/avatar/5f76a769a68607d8ebd829d59eef4d9ce85f711b2e9b3269697ec0ea6bfe1b7a?d=mp&s=160"},"body":"On Tue, Jan 12, 2016 at 10:29:27AM -0800, Junio C Hamano wrote:\n> I left the detail as vague ;-).\n> \n> The new request does not have to piggyback on existing \"want\"\n> message. And thinking about it again, it probably is cleaner if it\n> didn't. After the use of the protocol extension \"ancestry-check\" is\n> negotiated the usual way between the sender and the receiver, the\n> receiver would send \"check-ff N O\" and \"check-ff N P\" after it sends\n> all of its \"want\" messages but before it sends the \"flush\" to go\n> into the \"have\"/\"ack\" common ancestry discovery.\n> \n> I do not have a strong opinion on where the sender should reply with\n> \"not-ff N O\" in the protocol. Immediately after the receiver says\n> \"I've done with my 'want's (and now 'check-ff's)\" by flushing may be\n> a good place to do so.\n\nIn that case, I can think of two other useful times to do it:\n\n1. Before any \"want\" requests.\n\n This would also let you extend ls-remote to let it display ancestry.\n\n This is complicated by the fact that normally the client responds\n with which features it supports in the first \"want\",\n so the sender would have to support \"check-ff N O \"\n if it advertised \"ancestry-check\".\n\n2. After the pack is transferred.\n\n Then the receiver can check ancestry with the objects,\n and only request ancestry if it's missing history.\n\n I'm not sure whether there's any value in not requesting it,\n since while it would reduce the amount of work the sender needs to do,\n it still means the sender has to wait for the client to do the check,\n rather than hanging up and allowing it to process another connection.\n"},{"id":"275932","messageId":"xmqqlh7tpfkt.fsf@gitster.mtv.corp.google.com","threadId":"41143","inReplyTo":"20160113113743.GF3397@logi.codethink.co.uk","subject":"Re: Some issues when trying to set up a shallow git mirror server","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2016-01-13T17:14:10Z","receivedAt":"2016-01-13T17:14:10Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Richard Maw writes:\n\n> 1. Before any \"want\" requests.\n>\n> This would also let you extend ls-remote to let it display ancestry.\n>\n> This is complicated by the fact that normally the client responds\n> with which features it supports in the first \"want\",\n> so the sender would have to support \"check-ff N O \"\n> if it advertised \"ancestry-check\".\n\nYes, that sounds like a good thing. Actually, ls-remote is a\ndegenerated case of fetch that sends 0 \"want\" requests, so the above\nmay be identical to what I suggested in the message you are\nresponding to.\n\n> 2. After the pack is transferred.\n\nI do not think the protocol is structured to allow you to do this\neasily. The reading side is expected to read thru til the end of\nthe stream, not stopping at the end of the packdata, IIRC.\n"},{"id":"275943","messageId":"20160113174331.GI3397@logi.codethink.co.uk","threadId":"41143","inReplyTo":"xmqqlh7tpfkt.fsf@gitster.mtv.corp.google.com","subject":"Re: Some issues when trying to set up a shallow git mirror server","fromName":"Richard Maw","fromEmail":"richard.maw@codethink.co.uk","sentAt":"2016-01-13T17:43:32Z","receivedAt":"2016-01-13T17:43:32Z","isPatch":false,"sender":{"key":"richard.maw@codethink.co.uk","avatar":"https://gravatar.com/avatar/5f76a769a68607d8ebd829d59eef4d9ce85f711b2e9b3269697ec0ea6bfe1b7a?d=mp&s=160"},"body":"On Wed, Jan 13, 2016 at 09:14:10AM -0800, Junio C Hamano wrote:\n> Richard Maw writes:\n> \n> > 1. Before any \"want\" requests.\n> >\n> > This would also let you extend ls-remote to let it display ancestry.\n> >\n> > This is complicated by the fact that normally the client responds\n> > with which features it supports in the first \"want\",\n> > so the sender would have to support \"check-ff N O \"\n> > if it advertised \"ancestry-check\".\n> \n> Yes, that sounds like a good thing. Actually, ls-remote is a\n> degenerated case of fetch that sends 0 \"want\" requests, so the above\n> may be identical to what I suggested in the message you are\n> responding to.\n\nThanks. I've got a good idea of what I'd need to do now.\n"}]}