git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: RFC/Pull Request: Refs db backend

From
Junio C Hamano <gitster@pobox.com>
Date
Jun 23, 2015, 05:36 UTC
Message-ID
<xmqqvbefdm0d.fsf@gitster.dls.corp.google.com>
In-Reply-To
<1435020656.28466.8.camel@twopensource.com>
David Turner <dturner@twopensource.com> writes:
Show 20 quoted lines
> I've revived and modified Ronnie Sahlberg's work on the refs db
> backend.  
>
> The work is on top of be3c13e5564, Junio's "First batch for 2.5 cycle".
> I recognize that there have been changes to the refs code since then,
> and that there are some further changes in-flight from e.g. Michael
> Haggerty.  If there is interest in this, I can rebase once Michael's
> changes land.
> ...
> The db backend runs git for-each-ref about 30% faster than the files
> backend with fully-packed refs on a repo with ~120k refs.  It's also
> about 4x faster than using fully-unpacked refs.  In addition, and
> perhaps more importantly, it avoids case-conflict issues on OS X.
>
> I chose to use LMDB for the database...
> ...
> Ronnie Sahlberg's original version of this patchset used tdb.  The
> advantage of tdb is that it's smaller (~125k).  The disadvantages are
> that tdb is hard to build on OS X.  It's also not in homebrew.  So lmdb
> seemed simpler.
"If there is interest"?  Shut up and take my money ;-)

More seriously, that's great that you stepped up to resurrect this topic. In a sense, the choice of sample database backend does not matter. I do not care if it is tdb, lmdb, or even Berkeley DB as long as it functions. ;-)

As long as the interface between ref-transaction system on the Git side and the database backend is designed right, your lmdb thing can serve as a reference implementation for other people to plug other database backends to the same interface, right? As one step to validate the interface to the database backends, it would be nice to eventually have at least two backends that talk to meaningfully different systems, but we have to start somewhere, and "for now we have lmdb" is as good a place to start as any other db backend.

I wonder if we can do a "filesystem" backend on top of the same backend interface---is that too much impedance mismatch to make it unpractical?

Thanks.
Previous: David TurnerNext: Duy Nguyen
Message 2 of 26 in “RFC/Pull Request: Refs db backend”
  1. David TurnerJun 23, 2015
  2. Junio C HamanoJun 23, 2015
  3. Duy NguyenJun 23, 2015
  4. David TurnerJun 23, 2015
  5. David TurnerJun 23, 2015
  6. Jeff KingJun 23, 2015
  7. Duy NguyenJun 23, 2015
  8. Jeff KingJun 24, 2015
  9. David TurnerJun 23, 2015
  10. Jeff KingJun 24, 2015
  11. David TurnerJun 24, 2015
  12. Shawn PearceJun 24, 2015
  13. Jeff KingJun 24, 2015
  14. brian m. carlsonJun 25, 2015
  15. Duy NguyenJun 24, 2015
  16. Michael HaggertyJun 23, 2015
  17. David TurnerJun 23, 2015
  18. Michael HaggertyJun 23, 2015
  19. David TurnerJun 24, 2015
  20. David TurnerJun 23, 2015
  21. Junio C HamanoJun 23, 2015
  22. Stefan BellerJun 23, 2015
  23. David TurnerJun 23, 2015
  24. Randall S. BeckerJun 23, 2015
  25. David TurnerJun 23, 2015
  26. Randall S. BeckerJun 23, 2015

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.