{"thread":{"id":"64139","subject":"What kind of help is needed for SHA-256 work in the next ~year?","startedAt":"2025-09-12T17:59:19Z","lastAt":"2025-09-12T21:04:22Z","messageCount":2,"participants":["Emily Shaffer","brian m. carlson"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"526215","messageId":"CAJoAoZm+yeF7KUbVBqUh0zc58b1jXVPEtWimrUutX_5ifixxgw@mail.gmail.com","threadId":"64139","inReplyTo":null,"subject":"What kind of help is needed for SHA-256 work in the next ~year?","fromName":"Emily Shaffer","fromEmail":"nasamuffin@google.com","sentAt":"2025-09-12T17:59:06Z","receivedAt":"2025-09-12T17:59:19Z","isPatch":false,"sender":{"key":"nasamuffin@google.com","avatar":"https://avatars.githubusercontent.com/u/1606826?v=4"},"body":"brian has been working on the SHA-256 implementation and now on the\ninterop, pretty much solo, for quite some time. I realize that it's a\nbit late in the party to ask, but as we're talking about switching the\ndefault for new repositories in Git 3.0, I think it is past time for\nthe rest of the project to pitch in if we can.\n\nWhat kind of help would be useful to you at this point, brian? How\nmuch of the work is planned and ready for you to delegate to someone\nelse (and what's the timeline like, if you have one)? Do you need help\nwith testing any parts of the existing code in scaled scenarios? My\nunderstanding is that you have a roadmap to guide your own work, but\nif it's not shareable, is that something you could use some program\nmanagement help with? Anything like that?\n\nI can't guarantee that Google will be able to jump on and help right\naway, but at least understanding what needs doing is a good start for\nme to be able to ask around - especially if we're looking ahead to\n2026, that gives me more room to try and get help. I thought to ask on\nthe list instead of mailing brian directly because I assume that's the\ncase for the other corporate contributors to the project, too ;)\n\n - Emily\n"},{"id":"526230","messageId":"aMSKztysdyCKK50X@fruit.crustytoothpaste.net","threadId":"64139","inReplyTo":"CAJoAoZm+yeF7KUbVBqUh0zc58b1jXVPEtWimrUutX_5ifixxgw@mail.gmail.com","subject":"Re: What kind of help is needed for SHA-256 work in the next ~year?","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2025-09-12T21:04:14Z","receivedAt":"2025-09-12T21:04:22Z","isPatch":false,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"On 2025-09-12 at 17:59:06, Emily Shaffer wrote:\n> brian has been working on the SHA-256 implementation and now on the\n> interop, pretty much solo, for quite some time. I realize that it's a\n> bit late in the party to ask, but as we're talking about switching the\n> default for new repositories in Git 3.0, I think it is past time for\n> the rest of the project to pitch in if we can.\n\nI do very much appreciate you asking this.\n\n> What kind of help would be useful to you at this point, brian? How\n> much of the work is planned and ready for you to delegate to someone\n> else (and what's the timeline like, if you have one)? Do you need help\n> with testing any parts of the existing code in scaled scenarios? My\n> understanding is that you have a roadmap to guide your own work, but\n> if it's not shareable, is that something you could use some program\n> management help with? Anything like that?\n\nI have about 93 patches in my `sha256-interop` branch right now, which\nis based off v4 of Patrick's Rust series.  Much of the functionality\nworks: the legacy loose object maps (which I'm replacing with the new\nbinary format), pack index v3, full clones and pushes, and some shallow\nfunctionality.\n\nA lot of what I need help with is getting these patches production ready\nand sent to the list.  (Some of them are clearly marked as WIP with a\ncomment why.)  For instance, pack index v3 works just fine, but we need\nmore tests for it.  I haven't done any sort of scale testing yet,\neither, so if that's something we want, then help with that would be\ngreat.\n\nSimilarly, even when the binary loose object maps work, we'll still need\nto prune old objects from them and compact the maps as part of `git gc`,\nwhich would be something I'd appreciate help with.\n\nI do have permission to work on this as part of my job (starting in\nabout October), but we want to release in a year and I'm expecting at\nleast 200 (if not 300 or more) total patches for this project.  What I\ndon't want to do is try to shovel several 50-patch series in at the last\nminute, which would be unkind to reviewers and not produce the best\nquality code, so trying to get the existing patches cleaned up and in\nrelatively soon would help us make more progress at a more leisurely\npace.  I also still do have other duties at work as well (after all, my\nteam is responsible for serving your Git traffic, which I think we'd all\nlike to continue), so assistance would be super helpful.\n\nThere are also a giant heap of broken tests when run in compatibility\nmode.  Some of those tests are broken because, say, we lack support for\npartial clone, and we'll fix those by implementing partial clone. But\nthere are lots of tests that are broken for boring reasons, such as the\nfact that in compatibility mode we can't accept broken objects (because\nthey can't be mapped into the other algorithm), and those need to be\nmarked or fixed accordingly.  Getting those marked or fixed would be a\nsuper helpful contribution (I even have a test prerequisite for this\npurpose), as would fixing other routine test failures.\n\nI have some tests for fetching and pushing in interoperability mode\nwhich will run even when the entire testsuite is run in single-hash\nmode, but I think we're also going to want more tests: HTTP, the Git\nprotocol, protocols v0 and v2, single-hash servers and dual-hash clients\nand the reverse, unsupported cases[0], and so on.  That would also be very\nhelpful since it will help us make sure our changes are very robust.\n\nWe'll also need to implement partial clone and submodule support.\nSubmodules are especially tricky because to look up the object mapping,\nwe also need the submodule to be in interoperability mode.  And, because\npeople are absolutely going to want this kind of thing, we ideally need\nsome script or command to convert repositories from a single-hash (say,\nSHA-1) to dual-hash (interoperability) mode taking into account\nsubmodules (which must be done _before_ the main repo) and all the other\nedge cases, whether that's in place[1] or to a separate (bare or non-bare)\nrepository[2].  Someone picking that work up would be greatly appreciated.\n\nAnd there's still more beyond that as well.  Some of this I can pick up,\nbut assistance would of course be appreciated.\n\nI'm going to spend the next week kind of tying up some loose ends in my\ncurrent work and getting things in my branch in a state where someone\ncould pick some of this work up.  I'll also write up a complete list of\nwhat still needs to be done in case folks would like to help out and\nsend it to the list in reply to this thread.\n\nAs for project management, I would be fine with simply using\nGitHub/GitLab/Forgejo issues/projects in an otherwise empty repository\nfor tracking who's working on what if that's acceptable to others.  I do\nfeel some sort of tracking like this would be useful if we have multiple\ncontributors, since it will help avoid accidentally working on the same\nthing as someone else, but I'm not super picky as to what it is.\n\n> I can't guarantee that Google will be able to jump on and help right\n> away, but at least understanding what needs doing is a good start for\n> me to be able to ask around - especially if we're looking ahead to\n> 2026, that gives me more room to try and get help. I thought to ask on\n> the list instead of mailing brian directly because I assume that's the\n> case for the other corporate contributors to the project, too ;)\n\nI appreciate the offer.  I think with our desired timeframe, there's\ndefinitely enough work for two or three, and possibly more, people.  I\nwould be very grateful for any assistance that can be provided here.\n\n[0] For instance, we cannot do a shallow or partial clone to a dual-hash\nclient unless the server supports mapping using both algorithms, since\nthose types of clone have incomplete history and therefore the client\ncannot perform all of the conversion themselves.  We will want to\nprovide a nice error message to the user and some documentation for this\ncase.\n[1] In place is ideal, since that could also be useful for forges who\nwant to do this conversion, but any command is better than no command.\n[2] I think a shell command would be fine for this, although Dscho and\nthe other Windows folks may not love the performance.  This might also\nbe an exciting opportunity to write some Rust if the authors prefer that\napproach.\n-- \nbrian m. carlson (they/them)\nToronto, Ontario, CA\n"}]}