{"thread":{"id":"54320","subject":"Question About Git V2 Protocol & SHA256","startedAt":"2020-09-29T01:18:03Z","lastAt":"2020-10-01T23:53:25Z","messageCount":9,"participants":["Wu, Zhichen","Jeff King","brian m. carlson"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"406608","messageId":"9FC3DDB4-DE6F-45B3-95F9-1048991713A4@amazon.com","threadId":"54320","inReplyTo":null,"subject":"Question About Git V2 Protocol & SHA256","fromName":"Wu, Zhichen","fromEmail":"zhwu@amazon.com","sentAt":"2020-09-29T01:17:59Z","receivedAt":"2020-09-29T01:18:03Z","isPatch":false,"sender":{"key":"zhwu@amazon.com","avatar":null},"body":"Hi Git Community,\n \nHope this email finds you well.\n \nI’m Zhichen from AWS CodeCommit. I’m reaching out regarding 2 git features that we are planning for CodeCommit and your input would be critical for our next step.\n\n1. Since v2 protocol has been launched for 2 years, is there any plan in the near future about v1 deprecation?\n2. I see v2 has a capability called “object-format” that provides SHA1 option. I’m wondering if that capability will be the only way for client and server to start using SHA256? Or put it as another word, will v2 protocol be the prerequisite of SHA256?\n \nLooking forward to hearing back from you.\n \nThanks in advance!\nZhichen Wu\n\n\n"},{"id":"406650","messageId":"20200929221311.GA15129@coredump.intra.peff.net","threadId":"54320","inReplyTo":"9FC3DDB4-DE6F-45B3-95F9-1048991713A4@amazon.com","subject":"Re: Question About Git V2 Protocol & SHA256","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2020-09-29T22:13:11Z","receivedAt":"2020-09-29T22:13:14Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Tue, Sep 29, 2020 at 01:17:59AM +0000, Wu, Zhichen wrote:\n\n> 1. Since v2 protocol has been launched for 2 years, is there any plan\n>    in the near future about v1 deprecation?\n\nI suppose it depends what you mean by deprecation.\n\nv2 is not even the default yet (but hopefully will be soon, perhaps in\nthe next release).\n\nOnce that happens, I doubt that we will add new capabilities or features\nto v1 in general. Old versions of Git would not be able to take\nadvantage of them anyway, and new ones would be using v2.\n\nBut I don't think v1 will go away for a long time. There will still be\nv1 clients and v1 servers hanging around for many years, and modern\nversions of Git will need to speak both.\n\n> 2. I see v2 has a capability called “object-format” that provides SHA1\n>    option. I’m wondering if that capability will be the only way for\n>    client and server to start using SHA256? Or put it as another word,\n>    will v2 protocol be the prerequisite of SHA256?\n\nI think it would be impossible to handle object-format via v1, because\nthe v1 protocol writes the ref advertisement before any capabilities are\nnegotiated. So I think v1 must implicitly remain sha1-only (and a sha256\nrepository on the server side would need to either reject a v1 client,\nor back-translate as it would for a v2 client which asks for sha1).\n\n-Peff\n"},{"id":"406651","messageId":"60F39754-A524-475C-A0B8-CA107767B14B@amazon.com","threadId":"54320","inReplyTo":"20200929221311.GA15129@coredump.intra.peff.net","subject":"Re: Question About Git V2 Protocol & SHA256","fromName":"Wu, Zhichen","fromEmail":"zhwu@amazon.com","sentAt":"2020-09-29T22:41:49Z","receivedAt":"2020-09-29T22:41:56Z","isPatch":false,"sender":{"key":"zhwu@amazon.com","avatar":null},"body":"Hi Peff,\n\nThank you for the quick response! That is very helpful and I think I have all the information I need.\n\nAppreciate it a lot!\nZhichen\n\nOn 9/29/20, 3:13 PM, \"Jeff King\" <peff@peff.net> wrote:\n\n    CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe.\n\n\n\n    On Tue, Sep 29, 2020 at 01:17:59AM +0000, Wu, Zhichen wrote:\n\n    > 1. Since v2 protocol has been launched for 2 years, is there any plan\n    >    in the near future about v1 deprecation?\n\n    I suppose it depends what you mean by deprecation.\n\n    v2 is not even the default yet (but hopefully will be soon, perhaps in\n    the next release).\n\n    Once that happens, I doubt that we will add new capabilities or features\n    to v1 in general. Old versions of Git would not be able to take\n    advantage of them anyway, and new ones would be using v2.\n\n    But I don't think v1 will go away for a long time. There will still be\n    v1 clients and v1 servers hanging around for many years, and modern\n    versions of Git will need to speak both.\n\n    > 2. I see v2 has a capability called “object-format” that provides SHA1\n    >    option. I’m wondering if that capability will be the only way for\n    >    client and server to start using SHA256? Or put it as another word,\n    >    will v2 protocol be the prerequisite of SHA256?\n\n    I think it would be impossible to handle object-format via v1, because\n    the v1 protocol writes the ref advertisement before any capabilities are\n    negotiated. So I think v1 must implicitly remain sha1-only (and a sha256\n    repository on the server side would need to either reject a v1 client,\n    or back-translate as it would for a v2 client which asks for sha1).\n\n    -Peff\n\n"},{"id":"406652","messageId":"20200929224356.GH1392312@camp.crustytoothpaste.net","threadId":"54320","inReplyTo":"20200929221311.GA15129@coredump.intra.peff.net","subject":"Re: Question About Git V2 Protocol & SHA256","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2020-09-29T22:43:56Z","receivedAt":"2020-09-29T22:44:04Z","isPatch":false,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"On 2020-09-29 at 22:13:11, Jeff King wrote:\n> On Tue, Sep 29, 2020 at 01:17:59AM +0000, Wu, Zhichen wrote:\n> > 2. I see v2 has a capability called “object-format” that provides SHA1\n> >    option. I’m wondering if that capability will be the only way for\n> >    client and server to start using SHA256? Or put it as another word,\n> >    will v2 protocol be the prerequisite of SHA256?\n> \n> I think it would be impossible to handle object-format via v1, because\n> the v1 protocol writes the ref advertisement before any capabilities are\n> negotiated. So I think v1 must implicitly remain sha1-only (and a sha256\n> repository on the server side would need to either reject a v1 client,\n> or back-translate as it would for a v2 client which asks for sha1).\n\nI don't think that's the case.  You can indeed use v1 with SHA-256, but\nif you have a SHA-1-only Git, it will choke because the object ID is\nlonger than it expects.  If you want to negotiate the algorithm when we\nsupport both and the client can't deal with translating the initial ref\nadvertisement, then yes, you'll need v2.\n\nWe even support SHA-256 via bundles and the DAV-based HTTP protocol, but\nthe latter will never support negotiation of hash algorithms because\nit's based on static files.\n\nIt is required that you understand the object-format capability on the\nclient side to support SHA-256, since if you fail to announce it, the\ndefault is SHA-1, and right now, the server side will produce an error\nif the client doesn't announce it (or sends SHA-1 data).\n-- \nbrian m. carlson: Houston, Texas, US\n"},{"id":"406653","messageId":"D510F2C4-1BCB-4DF7-9F36-17EA93FB7F99@amazon.com","threadId":"54320","inReplyTo":"20200929224356.GH1392312@camp.crustytoothpaste.net","subject":"Re: Question About Git V2 Protocol & SHA256","fromName":"Wu, Zhichen","fromEmail":"zhwu@amazon.com","sentAt":"2020-09-29T23:07:06Z","receivedAt":"2020-09-29T23:07:17Z","isPatch":false,"sender":{"key":"zhwu@amazon.com","avatar":null},"body":"Hi Brian,\n\nThanks for the details!\n\nNow it sounds like although SHA1/SHA256 and v1/v2 are separate features, v2 capability is the only way for the client and server to negotiate the object format so that they won't send out something that is not understandable by the other party.\n\nIn that case, I think it's still valid that v2 needs to be supported first so that SHA256 can later be supported with enough flexibility.\n\nBests,\nZhichen\n\n\nOn 9/29/20, 3:45 PM, \"brian m. carlson\" <sandals@crustytoothpaste.net> wrote:\n\n    On 2020-09-29 at 22:13:11, Jeff King wrote:\n    > On Tue, Sep 29, 2020 at 01:17:59AM +0000, Wu, Zhichen wrote:\n    > > 2. I see v2 has a capability called “object-format” that provides SHA1\n    > >    option. I’m wondering if that capability will be the only way for\n    > >    client and server to start using SHA256? Or put it as another word,\n    > >    will v2 protocol be the prerequisite of SHA256?\n    > \n    > I think it would be impossible to handle object-format via v1, because\n    > the v1 protocol writes the ref advertisement before any capabilities are\n    > negotiated. So I think v1 must implicitly remain sha1-only (and a sha256\n    > repository on the server side would need to either reject a v1 client,\n    > or back-translate as it would for a v2 client which asks for sha1).\n\n    I don't think that's the case.  You can indeed use v1 with SHA-256, but\n    if you have a SHA-1-only Git, it will choke because the object ID is\n    longer than it expects.  If you want to negotiate the algorithm when we\n    support both and the client can't deal with translating the initial ref\n    advertisement, then yes, you'll need v2.\n\n    We even support SHA-256 via bundles and the DAV-based HTTP protocol, but\n    the latter will never support negotiation of hash algorithms because\n    it's based on static files.\n\n    It is required that you understand the object-format capability on the\n    client side to support SHA-256, since if you fail to announce it, the\n    default is SHA-1, and right now, the server side will produce an error\n    if the client doesn't announce it (or sends SHA-1 data).\n    -- \n    brian m. carlson: Houston, Texas, US\n\n"},{"id":"406655","messageId":"20200930004630.GA623061@coredump.intra.peff.net","threadId":"54320","inReplyTo":"20200929224356.GH1392312@camp.crustytoothpaste.net","subject":"Re: Question About Git V2 Protocol & SHA256","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2020-09-30T00:46:30Z","receivedAt":"2020-09-30T00:46:35Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Tue, Sep 29, 2020 at 10:43:56PM +0000, brian m. carlson wrote:\n\n> On 2020-09-29 at 22:13:11, Jeff King wrote:\n> > On Tue, Sep 29, 2020 at 01:17:59AM +0000, Wu, Zhichen wrote:\n> > > 2. I see v2 has a capability called “object-format” that provides SHA1\n> > >    option. I’m wondering if that capability will be the only way for\n> > >    client and server to start using SHA256? Or put it as another word,\n> > >    will v2 protocol be the prerequisite of SHA256?\n> > \n> > I think it would be impossible to handle object-format via v1, because\n> > the v1 protocol writes the ref advertisement before any capabilities are\n> > negotiated. So I think v1 must implicitly remain sha1-only (and a sha256\n> > repository on the server side would need to either reject a v1 client,\n> > or back-translate as it would for a v2 client which asks for sha1).\n> \n> I don't think that's the case.  You can indeed use v1 with SHA-256, but\n> if you have a SHA-1-only Git, it will choke because the object ID is\n> longer than it expects.  If you want to negotiate the algorithm when we\n> support both and the client can't deal with translating the initial ref\n> advertisement, then yes, you'll need v2.\n\nI agree that we _could_ just dump sha256 within a v1 protocol and wait\nfor the client to choke. But that seems like an awfully lousy user\nexperience. By contrast, assuming that the client wants sha1 means that\neither:\n\n  - we can reject it with a sensible ERR message that tells the user\n    what is going on\n\n  - we can serve them by talking in terms of sha1 if we're willing to\n    do the extra conversion work server-side (and/or have a cache of\n    sha1-format objects)\n\nThe only thing we lose is that a recent client who understands sha256\nwouldn't be able to contact us and do a sha256-over-v1 transaction. But\nwhy would they want to do so?\n\nThis is all hypothetical at this point, though, right? I tried finding\ndetails in the hash transition document, but protocol changes are listed\nas out of scope there. It does say that sha256 servers may just reject\nsha1 clients; but even so I'd prefer if we could do it with a nice\nmessage (i.e., my bullet one above).\n\nMy suggestion does also require that we have a v2 receive-pack protocol,\nwhich does not yet exist (but following the blueprint for fetch, I don't\nexpect it to be a big deal).\n\n-Peff\n"},{"id":"406656","messageId":"20200930021925.GI1392312@camp.crustytoothpaste.net","threadId":"54320","inReplyTo":"20200930004630.GA623061@coredump.intra.peff.net","subject":"Re: Question About Git V2 Protocol & SHA256","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2020-09-30T02:19:25Z","receivedAt":"2020-09-30T02:19:35Z","isPatch":false,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"On 2020-09-30 at 00:46:30, Jeff King wrote:\n> I agree that we _could_ just dump sha256 within a v1 protocol and wait\n> for the client to choke. But that seems like an awfully lousy user\n> experience. By contrast, assuming that the client wants sha1 means that\n> either:\n> \n>   - we can reject it with a sensible ERR message that tells the user\n>     what is going on\n> \n>   - we can serve them by talking in terms of sha1 if we're willing to\n>     do the extra conversion work server-side (and/or have a cache of\n>     sha1-format objects)\n\nIf you have a suitably new Git, it will fail with an appropriate\nmessage on both sides.  The client will complain that you are trying to\nuse a remote that uses the wrong hash algorithm and the server will\ncomplain that you have failed to pass a suitable object-format\nextension.  That will look like this:\n\n  fatal: mismatched object format: server sha256; client sha1\n  fatal: expected flush after ref listing\n\nThat's true if you're using protocol v2 at all, or a suitably new Git\nwith protocol v1, although the messages may differ slightly.  However,\nif you're using an older Git with v1, then you'll get this:\n\n  fatal: protocol error: unexpected '304c98b4860fa40b3e04f3e2e24db1a13ab959922a63209685813908c4fabc83 HEAD'\n  fatal: the remote end hung up unexpectedly\n\nThat's because, as you pointed out, we don't get to the point where the\nclient fails to send the value.\n\n> The only thing we lose is that a recent client who understands sha256\n> wouldn't be able to contact us and do a sha256-over-v1 transaction. But\n> why would they want to do so?\n\nBecause v2 isn't implemented for pushes yet.  The testsuite does not\neven come close to passing unless you have a fully functional remote\nsystem.  That's why I implemented protocol support, even though it\nwasn't originally planned.\n\nThat was mentioned in the cover letter of the series that introduced\nprotocol support.\n\nI should also point out that v2 SSH support doesn't work at all with\nOpenSSH unless the server is specifically configured to allow the\nGIT_PROTOCOL environment variable.  Since there are many use cases for\npeople to push Git data over SSH to servers they do not control, there\nis a valid and viable use case for v1 support.  I personally know people\nwith shell accounts who could not use v2 over SSH but could viably use\nv1.  Heck, that's even true for our shell host at work.\n\nAbsent a significant outreach campaign to all major operating system\nvendors to get this environment variable added to all supported versions\nof their OpenSSH packages, I don't see how restricting SHA-256 to\nprotocol v2 is acceptable.  Even then, many systems would still fail to\nwork correctly due to local config file modifications.\n\nIn retrospect, we probably should have sent \"GIT_PROTOCOL=version=2\" as\npart of the shell command and let implementers deal with the fact that\ntheir SSH servers would need to adequately parse and interpret shell\ncommands to function properly.  But it's probably too late to do that\nnow without bumping the protocol to v3.\n\n> This is all hypothetical at this point, though, right? I tried finding\n> details in the hash transition document, but protocol changes are listed\n> as out of scope there. It does say that sha256 servers may just reject\n> sha1 clients; but even so I'd prefer if we could do it with a nice\n> message (i.e., my bullet one above).\n\nIt is not hypothetical.  Git in master has a fully functional SHA-256\nimplementation that fully interoperates with other SHA-256 repositories.\nAs I mentioned above and elsewhere, that is a prerequisite for a fully\nfunctional testsuite.  I would not have shipped user-visible SHA-256\nsupport with a testsuite as broken as it would be without protocol\nsupport.\n\n> My suggestion does also require that we have a v2 receive-pack protocol,\n> which does not yet exist (but following the blueprint for fetch, I don't\n> expect it to be a big deal).\n\nIf someone would like to implement protocol v2 for push in time for\n2.29, then that could be a viable approach if we can address the serious\nproblem of the SSH situation in that time frame as well.\n\nI don't plan to implement v2 push support because that is a substantial\namount of work and I have even less time to work on Git than normal due\nto taking classes multiple nights a week.  As I know you are aware,\nworking on Git is not my day job.\n\nI will admit to being a bit annoyed that we're discussing changes like\nthis at this point.  I've tried to be open about the design and\nimplementation, answering questions and providing complete branches and\nreasonably thorough cover letters that people can peruse to know what's\ngoing on even if they aren't interested in the full series themselves.\nLWN has even summarized some of the later work.  It's fine if folks\ndon't want to participate in that process, but it's hard to incorporate\nfeedback that comes in well after the fact.  I certainly hope the nature\nand content of my SHA-256 work isn't a surprise to regulars on the list,\nbecause I've clearly failed to communicate effectively if that's the\ncase.\n\nI am at this point not planning any other major changes to SHA-256\nsupport before the interop work starts to land.  I am of course happy to\naddress any bug reports that come up, and if others would like to send\nin patches related to SHA-256, I will review them and provide feedback\nas my time allows.\n-- \nbrian m. carlson: Houston, Texas, US\n"},{"id":"406665","messageId":"20200930122006.GA1900495@coredump.intra.peff.net","threadId":"54320","inReplyTo":"20200930021925.GI1392312@camp.crustytoothpaste.net","subject":"Re: Question About Git V2 Protocol & SHA256","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2020-09-30T12:20:06Z","receivedAt":"2020-09-30T12:20:11Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Wed, Sep 30, 2020 at 02:19:25AM +0000, brian m. carlson wrote:\n\n> If you have a suitably new Git, it will fail with an appropriate\n> message on both sides.  The client will complain that you are trying to\n> use a remote that uses the wrong hash algorithm and the server will\n> complain that you have failed to pass a suitable object-format\n> extension.  That will look like this:\n> \n>   fatal: mismatched object format: server sha256; client sha1\n>   fatal: expected flush after ref listing\n\nThat makes sense, and the message isn't too bad.\n\n> That's true if you're using protocol v2 at all, or a suitably new Git\n> with protocol v1, although the messages may differ slightly.  However,\n> if you're using an older Git with v1, then you'll get this:\n> \n>   fatal: protocol error: unexpected '304c98b4860fa40b3e04f3e2e24db1a13ab959922a63209685813908c4fabc83 HEAD'\n>   fatal: the remote end hung up unexpectedly\n> \n> That's because, as you pointed out, we don't get to the point where the\n> client fails to send the value.\n\nThat makes switching to sha256 on the server a bit of a flag day, then,\ndoesn't it? Everybody with older Git will not be able to clone such a\nrepo. That may be OK for some cases, but I suspect it is going to cause\na delay in servers moving to sha256 themselves. Even if a project is\nready to migrate, it may be hesitant to do so because of third parties\noutside of its control (whether external tools like CI, or just wanting\nto be open to cloning by random users).\n\nOn the other hand, any kind of hash negotiation would require on-the-fly\nconversion for that case (i.e., a sha256 server serving an older sha1\nclient needs to produce the alternate-universe sha1 objects). That's\nexpensive either in CPU or in disk space, and it may well be that\nthird-party hosting sites would refuse to enable such an option anyway.\n\n> > The only thing we lose is that a recent client who understands sha256\n> > wouldn't be able to contact us and do a sha256-over-v1 transaction. But\n> > why would they want to do so?\n> \n> Because v2 isn't implemented for pushes yet.  The testsuite does not\n> even come close to passing unless you have a fully functional remote\n> system.  That's why I implemented protocol support, even though it\n> wasn't originally planned.\n> \n> That was mentioned in the cover letter of the series that introduced\n> protocol support.\n\nYeah, I think I misunderstood the direction there. I took the\nobject-format capability to mean that the client and server would\nnegotiate. But it really is the server saying \"I'm going to speak this;\nyou'd better understand it and be able to translate objects on your\nside\".\n\nI did realize that getting the test suite to pass meant that we'd be\npassing sha256 hashes over the wire. But I had assumed that was part of\nthe strategy to make everything work in a sha256 world, and that more\ninteroperability stuff was coming on top. I.e., I thought our current\nstate was less mature than you ended up implementing.\n\n> I should also point out that v2 SSH support doesn't work at all with\n> OpenSSH unless the server is specifically configured to allow the\n> GIT_PROTOCOL environment variable.  Since there are many use cases for\n> people to push Git data over SSH to servers they do not control, there\n> is a valid and viable use case for v1 support.  I personally know people\n> with shell accounts who could not use v2 over SSH but could viably use\n> v1.  Heck, that's even true for our shell host at work.\n\nThanks, that's an interesting counter-example that I didn't know about.\n\n> In retrospect, we probably should have sent \"GIT_PROTOCOL=version=2\" as\n> part of the shell command and let implementers deal with the fact that\n> their SSH servers would need to adequately parse and interpret shell\n> commands to function properly.  But it's probably too late to do that\n> now without bumping the protocol to v3.\n\nThat was definitely discussed, but rejected because of the\ninteroperability problems. Not just for servers who actually want v2,\nbut it would mean that the v2 probe would cause existing v1 servers to\nstop working. I.e., it would stop being safe to send v2 probes, which\nwas the whole point of the scheme.\n\n> > My suggestion does also require that we have a v2 receive-pack protocol,\n> > which does not yet exist (but following the blueprint for fetch, I don't\n> > expect it to be a big deal).\n> \n> If someone would like to implement protocol v2 for push in time for\n> 2.29, then that could be a viable approach if we can address the serious\n> problem of the SSH situation in that time frame as well.\n\nHmm. I hadn't really thought of 2.29 as the time-frame. In my mind,\nsha256 support is still considered experimental, and we are free to\nadjust things as necessary. But maybe that's naive. Anything in the\nhands of users may be a source of annoying cross-version problems.\n\n> I will admit to being a bit annoyed that we're discussing changes like\n> this at this point.  I've tried to be open about the design and\n> implementation, answering questions and providing complete branches and\n> reasonably thorough cover letters that people can peruse to know what's\n> going on even if they aren't interested in the full series themselves.\n> LWN has even summarized some of the later work.  It's fine if folks\n> don't want to participate in that process, but it's hard to incorporate\n> feedback that comes in well after the fact.  I certainly hope the nature\n> and content of my SHA-256 work isn't a surprise to regulars on the list,\n> because I've clearly failed to communicate effectively if that's the\n> case.\n\nThat's a fair criticism. I definitely have not followed your work as\nclosely as I could have (or perhaps should have). I'm glad we're having\nthis conversation now at least, though. I had some misunderstandings\nabout how we planned to proceed. And now I have a much better sense of\nwhere you're at with it (and hopefully Zhichen sees that half of what I\nwrote in my original reply is BS).\n\nI can live with the scheme you described. The \"flag day\" thing is really\nmy biggest reservation, but it may just be something we have to accept.\n\n-Peff\n"},{"id":"406794","messageId":"20201001235247.GK1392312@camp.crustytoothpaste.net","threadId":"54320","inReplyTo":"20200930122006.GA1900495@coredump.intra.peff.net","subject":"Re: Question About Git V2 Protocol & SHA256","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2020-10-01T23:52:47Z","receivedAt":"2020-10-01T23:53:25Z","isPatch":false,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"On 2020-09-30 at 12:20:06, Jeff King wrote:\n> On Wed, Sep 30, 2020 at 02:19:25AM +0000, brian m. carlson wrote:\n> \n> > If you have a suitably new Git, it will fail with an appropriate\n> > message on both sides.  The client will complain that you are trying to\n> > use a remote that uses the wrong hash algorithm and the server will\n> > complain that you have failed to pass a suitable object-format\n> > extension.  That will look like this:\n> > \n> >   fatal: mismatched object format: server sha256; client sha1\n> >   fatal: expected flush after ref listing\n> \n> That makes sense, and the message isn't too bad.\n\nYeah, it seems reasonable.  If people like something else better, it can\nbe adjusted.\n\n> > That's true if you're using protocol v2 at all, or a suitably new Git\n> > with protocol v1, although the messages may differ slightly.  However,\n> > if you're using an older Git with v1, then you'll get this:\n> > \n> >   fatal: protocol error: unexpected '304c98b4860fa40b3e04f3e2e24db1a13ab959922a63209685813908c4fabc83 HEAD'\n> >   fatal: the remote end hung up unexpectedly\n> > \n> > That's because, as you pointed out, we don't get to the point where the\n> > client fails to send the value.\n> \n> That makes switching to sha256 on the server a bit of a flag day, then,\n> doesn't it? Everybody with older Git will not be able to clone such a\n> repo. That may be OK for some cases, but I suspect it is going to cause\n> a delay in servers moving to sha256 themselves. Even if a project is\n> ready to migrate, it may be hesitant to do so because of third parties\n> outside of its control (whether external tools like CI, or just wanting\n> to be open to cloning by random users).\n\nRight, but as the code stands now, those people couldn't make use of a\nSHA-256 repository anyway because there's no interoperability.\n\n> On the other hand, any kind of hash negotiation would require on-the-fly\n> conversion for that case (i.e., a sha256 server serving an older sha1\n> client needs to produce the alternate-universe sha1 objects). That's\n> expensive either in CPU or in disk space, and it may well be that\n> third-party hosting sites would refuse to enable such an option anyway.\n\nWe will eventually have some sort of compatibility, and it's\ntheoretically possible that hosting could offer SHA-1 compatibility for\nthose repositories, even if that's only for protocol v1.  So the\nrepository could use SHA-1 for v1 and negotiate for v2.\n\nThe desired behavior for a large hoster like GitHub or GitLab may be to\nforce v2 only for SHA-256; that's a justifiable decision there, because\nwe know all clients will support it and we don't have to deal with\nremote systems that are poorly configured.  So I wouldn't be opposed to\nseeing an option that did that.  As mentioned elsewhere, though, it\nisn't a 100% viable option everywhere.\n\n> > Because v2 isn't implemented for pushes yet.  The testsuite does not\n> > even come close to passing unless you have a fully functional remote\n> > system.  That's why I implemented protocol support, even though it\n> > wasn't originally planned.\n> > \n> > That was mentioned in the cover letter of the series that introduced\n> > protocol support.\n> \n> Yeah, I think I misunderstood the direction there. I took the\n> object-format capability to mean that the client and server would\n> negotiate. But it really is the server saying \"I'm going to speak this;\n> you'd better understand it and be able to translate objects on your\n> side\".\n\nYes, at this point it is, and it will remain that way for v1.\n\n> I did realize that getting the test suite to pass meant that we'd be\n> passing sha256 hashes over the wire. But I had assumed that was part of\n> the strategy to make everything work in a sha256 world, and that more\n> interoperability stuff was coming on top. I.e., I thought our current\n> state was less mature than you ended up implementing.\n\nMore interoperability is coming, but the stage 4 (SHA-256 only)\nimplementation was easier to implement, so we got that first.\n\nThe good news is that the compat code is less involved than I expected\nbecause we providently made the decision to write every new loose object\nin the repository using a single function.  The biggest problem is now\nthe testsuite because we try to stuff all manner of invalid and broken\nobject IDs in places and that doesn't work anymore because they can't be\nmapped.  I'm chipping away at this problem while watching movies on the\nweekend, so it's coming.\n\n> > In retrospect, we probably should have sent \"GIT_PROTOCOL=version=2\" as\n> > part of the shell command and let implementers deal with the fact that\n> > their SSH servers would need to adequately parse and interpret shell\n> > commands to function properly.  But it's probably too late to do that\n> > now without bumping the protocol to v3.\n> \n> That was definitely discussed, but rejected because of the\n> interoperability problems. Not just for servers who actually want v2,\n> but it would mean that the v2 probe would cause existing v1 servers to\n> stop working. I.e., it would stop being safe to send v2 probes, which\n> was the whole point of the scheme.\n\nYeah.  I guess we can't have everything we want in this case.\n-- \nbrian m. carlson: Houston, Texas, US\n"}]}