git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH 0/5] Suggested for PU: revision caching system to significantly speed up packing/walking

From
Johannes Schindelin <johannes.schindelin@gmx.de>
Date
Aug 7, 2009, 06:12 UTC
Message-ID
<alpine.DEB.1.00.0908070808340.8306@pacific.mpi-cbg.de>
In-Reply-To
<4A7B95A8.2010000@vilain.net>
Hi,
On Fri, 7 Aug 2009, Sam Vilain wrote:
Show 14 quoted lines
> Johannes Schindelin wrote:
> >> the short answer is that cache slices are totally independant of pack 
> >> files.
> >>     
> >
> > My idea with that was that you already have a SHA-1 map in the pack 
> > index, and if all you want to be able to accelerate the revision 
> > walker, you'd probably need something that adds yet another mapping, 
> > from commit to parents and tree, and from tree to sub-tree and blob 
> > (so you can avoid unpacking commit and tree objects).
> >   
> 
> Tying indexes together like that is not a good idea in the database
> world.

This is not the same index as in the database world. It is more comparable with a cached view. And there, it is generally a good idea to keep related things in the same cached view (with an outer join), rather than having two primary keys for almost every record.

> Especially as in this case as Nick mentions, the domain is subtly 
> different (ie pack vs dag). Unfortunately you just can't try to pretend 
> that they will always be the same; you can't force a full repack on 
> every ref change!

No, but you do not need that, either. In the setting that is most likely the most thankful one, i.e. a git:// server, you _want_ to keep the repository "as packed as possible", otherwise the rev cache improvements will be lost in the bad packing performance anyway.

Show 7 quoted lines
> > Still, there is some redundancy between the pack index and your cache, 
> > as you have to write out the whole list of SHA-1s all over again.  I 
> > guess it is time to look at the code instead of asking stupid 
> > questions.
> >   
> 
> "Disk is cheap" :-)
Disk I/O ain't.
(Size of the I/O caches, yaddayadda, I'm sure you get my point).

Ciao, Dscho

Previous: Nick EdelenNext: Nicolas Pitre
Message 24 of 34 in “Suggested for PU: revision caching system to significantly speed up packing/walking”
  1. 0/5 Suggested for PU: revision caching system to significantly speed up packing/walkingNick Edelen, Aug 6, 2009
  2. Johannes SchindelinAug 6, 2009
  3. Michael J GruberAug 6, 2009
  4. Nick EdelenAug 6, 2009
  5. Johannes SchindelinAug 6, 2009
  6. Nick EdelenAug 6, 2009
  7. Nick EdelenAug 6, 2009
  8. Shawn O. PearceAug 6, 2009
  9. A Large Angry SCMAug 6, 2009
  10. Shawn O. PearceAug 6, 2009
  11. A Large Angry SCMAug 6, 2009
  12. Nick EdelenAug 7, 2009
  13. Johannes SchindelinAug 7, 2009
  14. Nicolas PitreAug 7, 2009
  15. Sam VilainAug 7, 2009
  16. Nicolas PitreAug 7, 2009
  17. Johannes SchindelinAug 7, 2009
  18. Nicolas PitreAug 7, 2009
  19. Johannes SchindelinAug 8, 2009
  20. Junio C HamanoAug 8, 2009
  21. Sam VilainAug 8, 2009
  22. Nicolas PitreAug 9, 2009
  23. Nick EdelenAug 9, 2009
  24. Johannes SchindelinAug 7, 2009
  25. Nicolas PitreAug 7, 2009
  26. Nick EdelenAug 7, 2009
  27. Junio C HamanoAug 7, 2009
  28. Nick EdelenAug 7, 2009
  29. Junio C HamanoAug 8, 2009
  30. Nick EdelenAug 8, 2009
  31. Jeff KingAug 8, 2009
  32. Nick EdelenAug 8, 2009
  33. Jeff KingAug 8, 2009
  34. Junio C HamanoAug 8, 2009

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.