From: apaterson@pm.me Date: Thu, 26 Mar 2026 14:21:07 GMT Subject: Re: [PATCH v2] odb: add write_packfile, for_each_unique_abbrev, convert_object_id Message-ID: In-Reply-To: Of course, and my apologies, gitgadget is not formatting these messages as clearly as I would like them to be. Both this series and the last were adapted from my fork that supports [1] with a feature similar to gitremote-helpers. My hope is that the fork can converge with master so that sqlite-git can become redistributable. The local backends vtable was already a step in this direction, so the question is if letting users bring their own local backends, the way they currently can with helpers for remote backends, is in scope for git core. Either way, it sounds like series 1 will be covered by upstream, so next I would like to contribute support for git-local-* helpers. This allows users to create .git repositories with storage formats other than packs and builtin alternatives like reftables, which seems appropriate as direct sqlite support would probably be out of scope for core. Local helpers are already implemented in [2] but if it makes sense to hold off and rebuild it after e.g. ps/odb-generic-object-name-handling is merged, I am not in such a rush. [1] https://github.com/mayCXC/sqlite-git [2] https://github.com/gitgitgadget/git/compare/master...MayCXC:git:ps/series-2-helpers-v3.patch - Aaron On Thursday, March 26th, 2026 at 7:58 AM, Patrick Steinhardt wrote: > On Thu, Mar 26, 2026 at 01:39:43PM +0000, Aaron Paterson via GitGitGadget wrote: > > From: Aaron Paterson > > > > Add three vtable methods to odb_source that were not part of the > > recent ps/odb-sources and ps/object-counting series: > > > > - write_packfile: ingest a pack from a file descriptor. The files > > backend chooses between index-pack (large packs) and > > unpack-objects (small packs below fetch.unpackLimit). Options > > cover thin-pack fixing, promisor marking, fsck, lockfile > > capture, and shallow file passing. > > > > - for_each_unique_abbrev: iterate objects matching a hex prefix > > for disambiguation. Searches loose objects via oidtree, then > > multi-pack indices, then non-MIDX packs. > > > > - convert_object_id: translate between hash algorithms using the > > loose object map. Used during SHA-1 to SHA-256 migration. > > This will conflict with ps/odb-generic-object-name-handling, which > already introduces generic callbacks for `for_each_unique_abbrev()`. > There's also ongoing work by Justin to handle writing packfiles via the > ODB transaction interface. > > > Also add ODB_SOURCE_HELPER to the source type enum, preparing for > > the helper backend in the next commit. > > Huh. > > > The write_packfile vtable method replaces the pattern where callers > > spawn index-pack/unpack-objects directly. fast-import already uses > > odb_write_packfile() and this allows non-files backends to handle > > pack ingestion through their own mechanism. > > I'm again a bit puzzled, same as with your previous patch series. It > would be nice to collaborate on this topic, but that will require a bit > more coordination than just sending in a patch series as things are > quite in flux here. > > Patrick >