[Summit topic] The state of getting a reftable backend working in git.git

This session was led by Ævar Arnfjörð Bjarmason (on behalf of Han-Wen Nienhuys, the driving force behind the reftable patches, who did not attend the Summit). Supporting cast: Jonathan "jrnieder" Nieder, Johannes "Dscho" Schindelin, Philip Oakley, Jeff "Peff" King, and Junio Hamano.

Notes:

 1.  Ævar: helping Han-Wen with reviewing

 2.  Was split into multiple patch series

 3.  Han-Wen implemented the reftable library, has been kicking on the mailing
     list for over a year

 4.  Before reftable, need to merge some preliminaries

 5.  Odd cases:

     1. slightly different semantics of reflog

     2. Probably more things that haven’t cropped up yet

     3. Some tests are still broken, question is still: are the tests wrong, or
        the code

 6.  Plan is to get reftable library and underlying fixes in place, and then do
     the process of actual reftable-as-ref-backend afterward

     1. Jrnieder: sounds like you’re alluding to a mailing list thread. Do you
        have a link?

     2. Ævar: there are multiple, as the initial patch series has been split
        multiple times

     3. “Reftable plan”:
        https://lore.kernel.org/git/87h7jqz7k5.fsf@evledraar.gmail.com/

     4. Also alluded to in Han-Wen’s later rerolls

 7.  What is reftable?

     1. It’s a custom way to store refs

     2. Instead of writing individual files per ref, it’s a single file (or
        multiple files when updating the refs)

 8.  Dscho: three issues that were outstanding when I reviewed it

     1. That said, I clashed with Han-Wen

     2. 1. licensing / contribution model

        2. Ævar: through the Software Freedom Conservancy we have good access
           to legal advice. I got advice there about how to document this well,
           will be getting us into an end state that will hopefully satisfy
           everybody

        3. To be clear, we already have some code in-tree that is under
           different licenses. xdiff is LGPL code used by libgit2, there’s
           contrib/ + compat/ code under various licenses

        4. For legal purposes want to make sure this is clear and unambiguous
           to everyone

        5. jrnieder: about contribution model, there is on-list discussion
           about this, taking patches in the normal way to this directory in
           git@vger.kernel.org is where I thought that ended up

        6. Ævar: yes, git.git as source-of-truth. Not like gitk where there’s a
           separate upstream repo

     3. 2. coding style consistency, + not using git core data structures
           enough

        3. Ævar: still substantially true. Integrating into git.git means any
           stylistic or structural changes to fit well into git are fair game.
           Carlo has been helping with that

     4. 3. I forgot the third :)

 9.  Philip Oakley: debugging when things go wrong

     1.  When reftable arrives, will people be unable to look behind the scenes
         at what’s going on when issues happen?

     2.  Especially for people who don’t understand refs as well

     3.  Jonathan: format =
         www.kernel.org/pub/software/scm/git/docs/technical/reftable.html
         [http://www.kernel.org/pub/software/scm/git/docs/technical/reftable.html]

     4.  Ævar: That’s a fair summary. It’s as though we didn’t have packfiles
         and only had loose files and then switched to using packfiles. Can’t
         just “cat” any more. Switching to a binary format

     5.  That said, you get advantages out of that. Situations where people end
         up needing to examine the low-level details are

     6.  Not a fully fair comparison, but we have this problem already with
         packed-refs, having to look in two places

     7.  Philip: An inspection tool to export as a directory tree might be
         handy, as an inspection tool

     8.  Peff: We have pretty good inspection tools that look at the whole ref
         database

     9.  Reftable has a set of files that go together. May want debugging tool
         to dump the content of a binary reftable file. But we can
         incrementally add those

     10. As we discover bugs, I expect to have to build tooling

     11. Dscho: We also have a .git/index file and don’t have tooling to
         interact with it other than the standard Git tools

     12. Ævar: To be clear, once these patches land it would still be optional,
         would not be the default ref backend

     13. Even if it’s 100% bug free, we still have concern for users in the
         wild that make it not so easy to just flip the switch

     14. Not going to be the default backend any time soon

     15. jrnieder: makes sense to wait for a while to make it the default, even
         once it is robust, since we have to pay attention to what Git versions
         + implementations are out there in the wild

     16. Ævar: When you run “git init”, it currently still creates a branches/
         directory. Dscho tried to get rid of it before

     17. jrnieder: I think that previous attempt was getting rid of read
         capability, too

     18. Dscho: don’t remember the details, has been a couple years

     19. Junio: I do not think it is a bad idea to drop branches from template.

 10. jrnieder: Question about how to handle this kind of large contribution

 11. At some point does it make sense to take it, mark as experimental, and
     improve in place?

 12. Hoping the previous discussion will help me think about that

 13. Ævar: I agree about importing the bulk of the code as-is and iterating
     from there

 14. At that point it’s still not accessible to users but we get portability,
     testing, etc

 15. Dscho: Agreed, that makes sense to me