{"thread":{"id":"31744","subject":"Is anyone working on a next-gen Git protocol?","startedAt":"2012-10-07T19:57:56Z","lastAt":"2012-10-22T04:59:36Z","messageCount":12,"participants":["Ævar Arnfjörð Bjarmason","Ilari Liusvaara","Jeff King","Junio C Hamano","Andreas Ericsson","Steffen Prohaska","Philip Oakley","Nguyen Thai Ngoc Duy","Shawn Pearce"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"200707","messageId":"CACBZZX6b+3P8M+z+X13k9Pq3tvVUfs_k1=foQVreX8K801=efQ@mail.gmail.com","threadId":"31744","inReplyTo":null,"subject":"Is anyone working on a next-gen Git protocol?","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2012-10-07T19:57:56Z","receivedAt":"2012-10-07T19:57:56Z","isPatch":false,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"On Wed, Oct 3, 2012 at 9:13 PM, Junio C Hamano <gitster@pobox.com> wrote:\n> Ævar Arnfjörð Bjarmason <avarab@gmail.com> writes:\n>\n>> I'm creating a system where a lot of remotes constantly fetch from a\n>> central repository for deployment purposes, but I've noticed that even\n>> with a remote.$name.fetch configuration to only get certain refs a\n>> \"git fetch\" will still call git-upload pack which will provide a list\n>> of all references.\n>\n> It has been observed that the sender has to advertise megabytes of\n> refs because it has to speak first before knowing what the receiver\n> wants, even when the receiver is interested in getting updates from\n> only one of them, or worse yet, when the receiver is only trying to\n> peek the ref it is interested has been updated.\n\nHas anyone started working on a next-gen Git protocol as a result of\nthis discussion? If not I thought I'd give it a shot if/when I have\ntime.\n\nThe current protocol is basically (S = Server, C = Client)\n\n S: Spew out first ref\n S: Advertisement of capabilities\n S: Dump of all our refs\n C/S: Declare wanted refs, negotiate with server\n S: Send pack to client, if needed\n\nAnd I thought I'd basically turn it into:\n\n C: Connect to server, declare what protocol we understand\n C: Advertisement of capabilities\n S: Advertisement of capabilities\n C/S: Negotiate what we want\n C/S: Same as v1, without the advertisement of capabilities, and maybe\ndon't dump refs at all\n\nBasically future-proofing it by having the client say what it supports\nto begin with along with what it can handle (like in HTTP).\n\nThen in the negotiation phase the client & server would go back &\nforth about what they want & how they want it. I'd planned to\nimplement something like:\n\n    C: want_refs refs/heads/*\n    S: OK to that\n    C: want_refs refs/tags/*\n    S: OK to that\n\nOr:\n\n    C: want_refs refs/heads/master\n    S: OK to that\n    C: want_refs refs/tags/v*\n    S: OK to that\n\nAs a proof of concept (and also something that'll solve the issue I\nhad), but by adding an initial negotiation phase the protocol should\nbe open to any future extensions without making assumptions about the\nclient wanting to know about all of the server's refs, unlike the\ncurrent protocol.\n"},{"id":"200709","messageId":"20121007202220.GA16115@LK-Perkele-VI.localdomain","threadId":"31744","inReplyTo":"CACBZZX6b+3P8M+z+X13k9Pq3tvVUfs_k1=foQVreX8K801=efQ@mail.gmail.com","subject":"Re: Is anyone working on a next-gen Git protocol?","fromName":"Ilari Liusvaara","fromEmail":"ilari.liusvaara@elisanet.fi","sentAt":"2012-10-07T20:22:20Z","receivedAt":"2012-10-07T20:22:20Z","isPatch":false,"sender":{"key":"ilari.liusvaara@elisanet.fi","avatar":null},"body":"On Sun, Oct 07, 2012 at 09:57:56PM +0200, Ævar Arnfjörð Bjarmason wrote:\n> \n> Has anyone started working on a next-gen Git protocol as a result of\n> this discussion? If not I thought I'd give it a shot if/when I have\n> time.\n\nUnfortunately, client signaling the version is nasty to do in ways that\nwouldn't cause current servers to hang up or do other undesirable things.\n\ngit://: Git-daemon will hang up[1] if it receives command it won't\nunderstand (and one can't add arguments either).\n\nssh://: Commands are NAKed in non-standard ways (e.g. Gitolite vs. shell)\nand one can't add arguments.\n\nfile://: That's easy.\n\nCONNECT: The helper needs to be told that v2 is supported (helper doing\nthe rest).\n\nMaybe with git://, one could hack the stuff in similar way as virtual\nhosting was added. But that won't work with SSH (nor one can use environment\nwith SSH).\n\n:-/\n\n[1] And there is no guarantee that the server end of git:// is git-daemon.\nThere's at least one git:// server implemetation that responds to unknown\ncommands by ERR packet followed by hangup. \n\n-Ilari\n"},{"id":"200720","messageId":"20121007220833.GD1743@sigill.intra.peff.net","threadId":"31744","inReplyTo":"CACBZZX6b+3P8M+z+X13k9Pq3tvVUfs_k1=foQVreX8K801=efQ@mail.gmail.com","subject":"Re: Is anyone working on a next-gen Git protocol?","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2012-10-07T22:08:33Z","receivedAt":"2012-10-07T22:08:33Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Sun, Oct 07, 2012 at 09:57:56PM +0200, Ævar Arnfjörð Bjarmason wrote:\n\n> Has anyone started working on a next-gen Git protocol as a result of\n> this discussion? If not I thought I'd give it a shot if/when I have\n> time.\n\nI haven't, and don't really plan on it soon (I have a few smaller things\nI'm working on, then I'd like to look into the EWAH bitmap stuff from\nShawn next).\n\n> The current protocol is basically (S = Server, C = Client)\n> \n>  S: Spew out first ref\n>  S: Advertisement of capabilities\n>  S: Dump of all our refs\n>  C/S: Declare wanted refs, negotiate with server\n>  S: Send pack to client, if needed\n\nIn the \"C\" portion there, there is also \"client acknowledges a\nsubset of capabilities shown by server\" while it is declaring wanted\nrefs.\n\n> And I thought I'd basically turn it into:\n> \n>  C: Connect to server, declare what protocol we understand\n>  C: Advertisement of capabilities\n>  S: Advertisement of capabilities\n\nThe capability negotiation right now is that the server offers and the\nclient accepts. Are you swapping that so that the client offers and the\nserver accepts? Or are you thinking that they would be sent\nsimultaneously here? That could drop one round-trip (it's probably not\nthat important for git-over-tcp, but smart-http cares a lot about round\ntrips). But it also introduces a complexity with future additions (one\nside may not know how to present its capabilities until understanding\nwhat the other side can do).\n\n>  C/S: Negotiate what we want\n\nRefs we want, or capabilities we want?\n\n>  C/S: Same as v1, without the advertisement of capabilities, and maybe\n> don't dump refs at all\n> \n> Basically future-proofing it by having the client say what it supports\n> to begin with along with what it can handle (like in HTTP).\n\nI feel like this \"maybe...\" bit needs more fleshed out before designing\nthe first part. I like the idea of future-proofing first and then adding\nnew features second, but what does the \"don't advertise all refs\"\nprotocol look like? Presumably the client is going to say \"I'm\ninterested in refs/heads/* and refs/tags/*\" or something. Does that come\nwith the capabilities? Or is it a new protocol phase?\n\nI think we need to know what the second half of the two-step process\nwill look like to be sure the first half will accommodate it (and the\nanswer may be as simple as saying \"they're not sending capabilities,\nthey're sending arbitrary key/value items, with the knowledge that the\nother side may not understand particular keys, and we have to be\nprepared to handle both cases).\n\n> Then in the negotiation phase the client & server would go back &\n> forth about what they want & how they want it. I'd planned to\n> implement something like:\n> \n>     C: want_refs refs/heads/*\n>     S: OK to that\n>     C: want_refs refs/tags/*\n>     S: OK to that\n> \n> Or:\n> \n>     C: want_refs refs/heads/master\n>     S: OK to that\n>     C: want_refs refs/tags/v*\n>     S: OK to that\n\nThat seems simple. But how will it work over smart-http? Are we adding a\nround-trip to do want_refs negotiation?\n\n-Peff\n"},{"id":"200723","messageId":"7va9vxq5gp.fsf@alter.siamese.dyndns.org","threadId":"31744","inReplyTo":"CACBZZX6b+3P8M+z+X13k9Pq3tvVUfs_k1=foQVreX8K801=efQ@mail.gmail.com","subject":"Re: Is anyone working on a next-gen Git protocol?","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2012-10-07T22:31:50Z","receivedAt":"2012-10-07T22:31:50Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Ævar Arnfjörð Bjarmason <avarab@gmail.com> writes:\n\n> On Wed, Oct 3, 2012 at 9:13 PM, Junio C Hamano <gitster@pobox.com> wrote:\n>> Ævar Arnfjörð Bjarmason <avarab@gmail.com> writes:\n>>\n>>> I'm creating a system where a lot of remotes constantly fetch from a\n>>> central repository for deployment purposes, but I've noticed that even\n>>> with a remote.$name.fetch configuration to only get certain refs a\n>>> \"git fetch\" will still call git-upload pack which will provide a list\n>>> of all references.\n>>\n>> It has been observed that the sender has to advertise megabytes of\n>> refs because it has to speak first before knowing what the receiver\n>> wants, even when the receiver is interested in getting updates from\n>> only one of them, or worse yet, when the receiver is only trying to\n>> peek the ref it is interested has been updated.\n>\n> Has anyone started working on a next-gen Git protocol as a result of\n> this discussion?\n\nI and Shawn helped privately somebody from Gerrit circle, where the\ninitial ref advertisement is a huge problem (primarily because they\nadd tons of refs to one commit that eventually goes to their\nintegration branch), to coming up with a problem description and\nproposal document to kick-start a discussion some time ago, but not\nmuch has happened since.  Unless I hear from them soonish, I'll send\na cleaned-up version of the draft before I leave for my vacation.\n\nThe gist of it is that the current protocol cannot be upgraded in\nplace because \"who speaks first\" is not something you can update\nwith capability, so we would need upload-pack-v2 that lets the\nfetching side speak first.\n\n\"What is spoken in the first message\" is a separate issue, and one\nof the things it can address is to allow the ends to reduce the\namount of ref advertisement that ends up not getting used in the\nend, but once we allow the fetcher to speak first, we have much\nwider possibilities.\n"},{"id":"200744","messageId":"5072973D.4080703@op5.se","threadId":"31744","inReplyTo":"CACBZZX6b+3P8M+z+X13k9Pq3tvVUfs_k1=foQVreX8K801=efQ@mail.gmail.com","subject":"Re: Is anyone working on a next-gen Git protocol?","fromName":"Andreas Ericsson","fromEmail":"ae@op5.se","sentAt":"2012-10-08T09:05:01Z","receivedAt":"2012-10-08T09:05:01Z","isPatch":false,"sender":{"key":"ae@op5.se","avatar":"https://gravatar.com/avatar/426e89595c75a8f5252dd0c989e5fabe5bcac616e68557427ad9aef6b0ca342a?d=mp&s=160"},"body":"On 10/07/2012 09:57 PM, Ævar Arnfjörð Bjarmason wrote:\n> On Wed, Oct 3, 2012 at 9:13 PM, Junio C Hamano <gitster@pobox.com> wrote:\n>> Ævar Arnfjörð Bjarmason <avarab@gmail.com> writes:\n>>\n>>> I'm creating a system where a lot of remotes constantly fetch from a\n>>> central repository for deployment purposes, but I've noticed that even\n>>> with a remote.$name.fetch configuration to only get certain refs a\n>>> \"git fetch\" will still call git-upload pack which will provide a list\n>>> of all references.\n>>\n>> It has been observed that the sender has to advertise megabytes of\n>> refs because it has to speak first before knowing what the receiver\n>> wants, even when the receiver is interested in getting updates from\n>> only one of them, or worse yet, when the receiver is only trying to\n>> peek the ref it is interested has been updated.\n> \n> Has anyone started working on a next-gen Git protocol as a result of\n> this discussion? If not I thought I'd give it a shot if/when I have\n> time.\n> \n> The current protocol is basically (S = Server, C = Client)\n> \n>   S: Spew out first ref\n>   S: Advertisement of capabilities\n>   S: Dump of all our refs\n>   C/S: Declare wanted refs, negotiate with server\n>   S: Send pack to client, if needed\n> \n> And I thought I'd basically turn it into:\n> \n>   C: Connect to server, declare what protocol we understand\n>   C: Advertisement of capabilities\n>   S: Advertisement of capabilities\n>   C/S: Negotiate what we want\n>   C/S: Same as v1, without the advertisement of capabilities, and maybe\n> don't dump refs at all\n> \n> Basically future-proofing it by having the client say what it supports\n> to begin with along with what it can handle (like in HTTP).\n> \n> Then in the negotiation phase the client & server would go back &\n> forth about what they want & how they want it. I'd planned to\n> implement something like:\n> \n>      C: want_refs refs/heads/*\n>      S: OK to that\n>      C: want_refs refs/tags/*\n>      S: OK to that\n> \n> Or:\n> \n>      C: want_refs refs/heads/master\n>      S: OK to that\n>      C: want_refs refs/tags/v*\n>      S: OK to that\n> \n\nYou'll want that to be a single \"wants\" message to avoid incurring\ninsane amounts of roundtrip latency with lots of refs. github and\nother hosted services are quite popular, but with my 120ms ping\nrtt I'd be spending half a minute just telling the other side what\nI want when I fetch from a repo with 250 refs.\n\nIt's a flagday and a half to change the protocol though, so I expect\nit'll have to wait for 2.0, unless the current client-side part of\nit is dumb and ignores existing refs when requesting its \"wants\", in\nwhich case the server can just stop advertising existing refs and\nmost of the speedup is already done.\n\n-- \nAndreas Ericsson                   andreas.ericsson@op5.se\nOP5 AB                             www.op5.se\nTel: +46 8-230225                  Fax: +46 8-230231\n\nConsidering the successes of the wars on alcohol, poverty, drugs and\nterror, I think we should give some serious thought to declaring war\non peace.\n"},{"id":"200753","messageId":"7vtxu5lyjr.fsf@alter.siamese.dyndns.org","threadId":"31744","inReplyTo":"5072973D.4080703@op5.se","subject":"Re: Is anyone working on a next-gen Git protocol?","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2012-10-08T16:27:04Z","receivedAt":"2012-10-08T16:27:04Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Andreas Ericsson <ae@op5.se> writes:\n\n> You'll want that to be a single \"wants\" message to avoid incurring\n> insane amounts of roundtrip latency with lots of refs. github and\n> other hosted services are quite popular, but with my 120ms ping\n> rtt I'd be spending half a minute just telling the other side what\n> I want when I fetch from a repo with 250 refs.\n\nPeff's recent patch when applied on the server side would help\nalleviate the load to produce these refs, but it obviously would not\ncut the network cost.  In order to change this, we need to swap \"who\nspeaks first\".\n\nOnce we go into \"want/have\" phase, I do not think there is a need\nfor fundamental change in the protocol (by this, I am not counting a\nchange to send \"have\"s sparsely and possibly backtracking to bisect\nhistory, etc. as \"fundamental\").\n"},{"id":"200938","messageId":"035A66D9-FAF0-48EE-B161-7D0CAD92F2FB@zib.de","threadId":"31744","inReplyTo":"7vtxu5lyjr.fsf@alter.siamese.dyndns.org","subject":"Re: Is anyone working on a next-gen Git protocol?","fromName":"Steffen Prohaska","fromEmail":"prohaska@zib.de","sentAt":"2012-10-10T19:13:37Z","receivedAt":"2012-10-10T19:13:37Z","isPatch":false,"sender":{"key":"prohaska@zib.de","avatar":"https://avatars.githubusercontent.com/u/217580?v=4"},"body":"On Oct 8, 2012, at 6:27 PM, Junio C Hamano wrote:\n\n> Once we go into \"want/have\" phase, I do not think there is a need\n> for fundamental change in the protocol (by this, I am not counting a\n> change to send \"have\"s sparsely and possibly backtracking to bisect\n> history, etc. as \"fundamental\").\n\nI've recently discovered that the current protocol can be amazingly\ninefficient when it comes to transferring binary objects.  Assuming two\nrepositories that are in sync.  After a 'git checkout --orphan && git\ncommit', a subsequent transfers sends all the blobs attached to the new\ncommit, although the other side already has all the blobs.\n\nThis behavior is especially annoying when (mis)using git to store binary\nfiles.  I was thinking for a while that it might be a reasonable idea to\nstore binary files in a submodule and frequently cut the history in\norder to save space.  The history would have little value anyway, since\ndiff and merge don't make much sense with binary files.\n\nEventually, I abandoned the idea due to the current behavior of the\nprotocol.  I had expected that git would be smarter and behave more like\nrsync, for example, by skipping big blobs as soon as it recognizes that\nthey are already available at both sides.\n\nMaybe the new protocol could include an optimization for the described\ncase.  I don't know whether this would be a fundamental change.\n\n    Steffen\n"},{"id":"200951","messageId":"7vlifeawd5.fsf@alter.siamese.dyndns.org","threadId":"31744","inReplyTo":"035A66D9-FAF0-48EE-B161-7D0CAD92F2FB@zib.de","subject":"Re: Is anyone working on a next-gen Git protocol?","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2012-10-10T20:46:30Z","receivedAt":"2012-10-10T20:46:30Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Steffen Prohaska <prohaska@zib.de> writes:\n\n> I've recently discovered that the current protocol can be amazingly\n> inefficient when it comes to transferring binary objects.  Assuming two\n> repositories that are in sync.  After a 'git checkout --orphan && git\n> commit', a subsequent transfers sends all the blobs attached to the new\n> commit, although the other side already has all the blobs.\n\nI do not think it has anything to do with binary, but what you\ndeserve from using orphan, where you declared that the history does\nnot have anything to do with the original.\n\nIf both of your repositories had the two paralle lines of these\nhistories as branches, the transfer would have went well with or\nwithout binary objects.\n"},{"id":"200961","messageId":"4CD4F3E1194047F5B91D4D4E1CE146AC@PhilipOakley","threadId":"31744","inReplyTo":"7vlifeawd5.fsf@alter.siamese.dyndns.org","subject":"Re: Is anyone working on a next-gen Git protocol?","fromName":"Philip Oakley","fromEmail":"philipoakley@iee.org","sentAt":"2012-10-10T22:32:00Z","receivedAt":"2012-10-10T22:32:00Z","isPatch":false,"sender":{"key":"philipoakley@iee.email","avatar":"https://avatars.githubusercontent.com/u/914343?v=4"},"body":"From: \"Junio C Hamano\" <gitster@pobox.com>\n> Steffen Prohaska <prohaska@zib.de> writes:\n>\n>> I've recently discovered that the current protocol can be amazingly\n>> inefficient when it comes to transferring binary objects.  Assuming \n>> two\n>> repositories that are in sync.  After a 'git checkout --orphan && git\n>> commit', a subsequent transfers sends all the blobs attached to the \n>> new\n>> commit, although the other side already has all the blobs.\n>\n> I do not think it has anything to do with binary, but what you\n> deserve from using orphan, where you declared that the history does\n> not have anything to do with the original.\n>\n> If both of your repositories had the two paralle lines of these\n> histories as branches, the transfer would have went well with or\n> without binary objects.\n> --\nSteffen,\nAn alternative could be a shallow clone for just those branches with the \nbinary objects, so that the git objects are still identical. Or use a \nreplace/graft to trim the line of development. It's still a fudge, but \nsomething you could look at. \n"},{"id":"200966","messageId":"CACsJy8DMStBNjucU2eitNdkYgk-1K04dxhqV2gpKOZkpLzR_iA@mail.gmail.com","threadId":"31744","inReplyTo":"7vlifeawd5.fsf@alter.siamese.dyndns.org","subject":"Re: Is anyone working on a next-gen Git protocol?","fromName":"Nguyen Thai Ngoc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2012-10-11T01:44:38Z","receivedAt":"2012-10-11T01:44:38Z","isPatch":false,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"On Thu, Oct 11, 2012 at 3:46 AM, Junio C Hamano <gitster@pobox.com> wrote:\n> Steffen Prohaska <prohaska@zib.de> writes:\n>\n>> I've recently discovered that the current protocol can be amazingly\n>> inefficient when it comes to transferring binary objects.  Assuming two\n>> repositories that are in sync.  After a 'git checkout --orphan && git\n>> commit', a subsequent transfers sends all the blobs attached to the new\n>> commit, although the other side already has all the blobs.\n>\n> I do not think it has anything to do with binary, but what you\n> deserve from using orphan, where you declared that the history does\n> not have anything to do with the original.\n>\n> If both of your repositories had the two paralle lines of these\n> histories as branches, the transfer would have went well with or\n> without binary objects.\n\nOn the same inefficient subject, git does not try to share common\nobjects for non-commit refs, for example tags pointing to trees. I\nhave such a peculiar repo and if a new tag shares 90% the tree with\nexisting tags, git-fetch to sends the whole tree of the new tag over\nthe wire. It does not seem easy to fix though and is probably rare\nenough that does not justify proper support. As a work around, I\ngenerate commits that link all these tags/trees together in a\npredetermined order. Not nice but works ok.\n-- \nDuy\n"},{"id":"200968","messageId":"CAJo=hJsFLqV7ye0LZzQOrt6EpUXNVjqYfPp9ixO52=CBWcQtdw@mail.gmail.com","threadId":"31744","inReplyTo":"CACsJy8DMStBNjucU2eitNdkYgk-1K04dxhqV2gpKOZkpLzR_iA@mail.gmail.com","subject":"Re: Is anyone working on a next-gen Git protocol?","fromName":"Shawn Pearce","fromEmail":"spearce@spearce.org","sentAt":"2012-10-11T03:08:41Z","receivedAt":"2012-10-11T03:08:41Z","isPatch":false,"sender":{"key":"spearce@spearce.org","avatar":"https://avatars.githubusercontent.com/u/34844?v=4"},"body":"On Wed, Oct 10, 2012 at 6:44 PM, Nguyen Thai Ngoc Duy <pclouds@gmail.com> wrote:\n> On Thu, Oct 11, 2012 at 3:46 AM, Junio C Hamano <gitster@pobox.com> wrote:\n>> Steffen Prohaska <prohaska@zib.de> writes:\n>>\n>>> I've recently discovered that the current protocol can be amazingly\n>>> inefficient when it comes to transferring binary objects.  Assuming two\n>>> repositories that are in sync.  After a 'git checkout --orphan && git\n>>> commit', a subsequent transfers sends all the blobs attached to the new\n>>> commit, although the other side already has all the blobs.\n>>\n>> I do not think it has anything to do with binary, but what you\n>> deserve from using orphan, where you declared that the history does\n>> not have anything to do with the original.\n>>\n>> If both of your repositories had the two paralle lines of these\n>> histories as branches, the transfer would have went well with or\n>> without binary objects.\n>\n> On the same inefficient subject, git does not try to share common\n> objects for non-commit refs, for example tags pointing to trees. I\n> have such a peculiar repo and if a new tag shares 90% the tree with\n> existing tags, git-fetch to sends the whole tree of the new tag over\n> the wire. It does not seem easy to fix though and is probably rare\n> enough that does not justify proper support. As a work around, I\n> generate commits that link all these tags/trees together in a\n> predetermined order. Not nice but works ok.\n\nAside from saving a huge amount of CPU during the \"Counting objects\"\nphase, the compressed bitmap work we presented in JGit solves this by\nworking off the complete reachability graph, and not just some subset\nrelated to a cut made across the commit graph. Unfortunately we took a\nshortcut and didn't create bitmaps for non-commits, but this is a\ntrivial modification to the algorithm and the storage.\n"},{"id":"201681","messageId":"CAPc5daUJ9zZXci=N+v819FsMKvEaY7ar6xwG9TRt0CP=HBzbZQ@mail.gmail.com","threadId":"31744","inReplyTo":"7va9vxq5gp.fsf@alter.siamese.dyndns.org","subject":"Re: Is anyone working on a next-gen Git protocol?","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2012-10-22T04:59:36Z","receivedAt":"2012-10-22T04:59:36Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"On Sun, Oct 7, 2012 at 3:31 PM, Junio C Hamano <gitster@pobox.com> wrote:\n>\n> I and Shawn helped privately somebody from Gerrit circle, where the\n> initial ref advertisement is a huge problem (primarily because they\n> add tons of refs to one commit that eventually goes to their\n> integration branch), to coming up with a problem description and\n> proposal document to kick-start a discussion some time ago, but not\n> much has happened since.  Unless I hear from them soonish, I'll send\n> a cleaned-up version of the draft before I leave for my vacation.\n\nWhich I forgot and never happened. Here is a link (not cleaned-up)\n\nhttp://tinyurl.com/WhoSpeaksFirstInGit\n"}]}