git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: Git commit generation numbers

From
GSGeorge Spelvin <linux@horizon.com>
Date
Jul 18, 2011, 05:13 UTC
Message-ID
<20110718051347.28952.qmail@science.horizon.com>
In-Reply-To
<CA+55aFwt+RDRK_r=9CXbdzsLuGDqswvGTtJDKi9Q3DQwB_Ha5Q@mail.gmail.com>
> Nobody has *ever* given a reason why the cache would be better than
> just making it explicit.
I thought I listed a few.  Let me be clearer.
1) It involves changing the commit format.  Since the change is
   backward-compatible, it's not too bad, but this is still fundamentally
   A Bad Thing, to be avoided if possible.
2) It can't be retrofitted to help historical browsing.
3) You have to support commits without generation numbers forever.
   This is a support burden.  If you can generate generation numbers for
   an entire repository, including pre-existing commits, you can *throw
   out* the commit date heuristic code entirely.
4) It can't be made to work with grafts or replace objects.
5) It includes information which is redundant, but hard to verify,
   in git objects.  Leading to potentially bizarre and version-dependent
   behaviour if it's wrong.  (Checking that the numbers are consistent
   is the same work as regenerating a cache.)
6) It makes git commits slightly larger.  (Okay, that's reaching.)
> Why is that so hard for people to understand? The cache is just EXTRA WORK.

That's why it *might* have been a good idea to include the number in the original design. But now that the design is widely deployed, it's better to avoid changing the design if not necessary.

With a bit of extra work, it's not necessary.
> To take your TLB example: it's like having a TLB for a page table that
> would be as easy to just create in a way that it's *faster* to look up
> in the actual data structure than it would be to look up in the cache.

You've subtly jumped points. The original point was that it's worth precomputing and storing the generation numbers. I was trying to say that this is fundamentally a caching operation.

Now we're talking about *where* to store the cached generation numbers.

Your point, which is a very valid one, is that they are to be stored on disk, exactly one per commit, can be computed when the commit is generated, and are accessed at the same time as the commit, so it makes all kinds of sense to store them *with* the commits. As part of them, even.

This has the huge benefit that it does away with the need for a *separate* data structure. (Kinda sorts like the way AMD stores instruction boundaries in the L1 I-cache, avoiding the need for a separate data structure.)

I'm arguing that, despite this annoying overhead, there are valid reasons to want to store it separately. There are some practical ones, but the basic one is an esthetic/maintainability judgement of "less cruft in the commit objects is worth more cruft in the code".

Git has done very well partly *because* of the minimality of its basic persistent object database format. I think we should be very reluctant to add to that without a demonstrated need that *cannot* be met in another way.

In this particular case, a TLB is not a transport format. It's okay to add redundant cruft to make it faster, because it only lasts until the next reboot. (A more apropos, software-oriented analogy might be "struct page".)

A git commit object *is* a transport format, one specifically designed for transporting data a very long way forward in time, so it should be designed with considerable care, and cruft ruthlessly eradicated.

Whatever you add to it has to be supported by every git implementation, forever. As does every implementation bug ever produced.

A cache, on the other hand, is purely a local implementation detail. It can be changed between versions with much less effort.

I agree it's more implementation work. But the upside is a cleaner struct commit. Which is a very good thing.

Previous: Linus TorvaldsNext: Anthony Van de Gejuchte
Message 6 of 35 in “Re: Git commit generation numbers”
  1. George SpelvinJul 17, 2011
  2. Long, MartinJul 17, 2011
  3. Linus TorvaldsJul 17, 2011
  4. George SpelvinJul 17, 2011
  5. Linus TorvaldsJul 17, 2011
  6. George SpelvinJul 18, 2011
  7. Anthony Van de GejuchteJul 18, 2011
  8. George SpelvinJul 18, 2011
  9. Nicolas PitreJul 20, 2011
  10. George SpelvinJul 20, 2011
  11. david@lang.hmJul 20, 2011
  12. Nicolas PitreJul 20, 2011
  13. Phil HordJul 21, 2011
  14. david@lang.hmJul 21, 2011
  15. Shawn PearceJul 21, 2011
  16. Phil HordJul 21, 2011
  17. david@lang.hmJul 21, 2011
  18. George SpelvinJul 21, 2011
  19. Jakub NarebskiJul 21, 2011
  20. George SpelvinJul 21, 2011
  21. Shawn PearceJul 21, 2011
  22. Jakub NarebskiJul 22, 2011
  23. Nicolas PitreJul 22, 2011
  24. david@lang.hmJul 22, 2011
  25. Jakub NarebskiJul 22, 2011
  26. Linus TorvaldsJul 22, 2011
  27. Jeff KingJul 22, 2011
  28. Felipe ContrerasJul 28, 2011
  29. Ramkumar RamachandraSep 6, 2011
  30. david@lang.hmJul 22, 2011
  31. Nicolas PitreJul 22, 2011
  32. david@lang.hmJul 22, 2011
  33. Phil HordJul 21, 2011
  34. Nicolas PitreJul 21, 2011
  35. Phil HordJul 21, 2011

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.