git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [WISH] Store also tag dereferences in packed-refs

From
Junio C Hamano <junkio@cox.net>
Date
Nov 19, 2006, 21:01 UTC
Message-ID
<7vbqn38831.fsf@assigned-by-dhcp.cox.net>
In-Reply-To
<Pine.LNX.4.64.0611191219350.3692@woody.osdl.org>
Linus Torvalds <torvalds@osdl.org> writes:
Show 30 quoted lines
> So the optimizations that _can_ be done are:
>
>  - add dereference info to .git/packed-refs
>
>    This would allow us to simply not do the expensive object lookup for 
>    every single tag. We'd still have to do it for non-packed objects, of 
>    course, but the cost here tends to be that over time you might have 
>    hundreds of tags, and even if each tag only takes 0.02s to look up, 
>    you're going to be slow.
>
>  - avoid the references for "heads/" (which we know are supposed to be 
>    commits, and cannot be tags) and when not specifying "-d". This won't 
>    help your case very much, though. If you want "-d", you want it, and 
>    the _big_ number of refs tends to be in tags, not branches, anyway.
>
>  - using a filesystem wih nicer locality behaviour for directory entries 
>    and inodes. This can cut down costs of cold-cache case by a factor of 
>    two, but right now there are no good filesystems that do this (but see 
>    for example "spadfs" that Mikulas Patocka announced a few weeks ago on 
>    linux-kernel - it would seem to have the possibility of being better in 
>    this area. I looked at the code and it looked like it could become 
>    very reasonable, but I've not actually _tested_ it, soo...)
>
> Anyway, I think that if we really want to make "git show-refs" go fast 
> when things are cold in the cache, and with lots ot tags and "-d" (which 
> is a reasonable case to optimize for: it's probably exactly what we end up 
> doing both for gitweb _and_ for "git-send-pack"), we'd need to expand the 
> packed-refs file with the deref cache.
>
> Junio?

Yes; the "for discussion only" WIP patch from yesterday was about the first point and half of the second point.

One downside about storing the peeled refs in .git/packed-refs is that the code in the wild does not have safety guard against them, so the version of show-ref from v1.4.4 recently released will list tags/v2.6.19^{} as if it is a true tag and worse yet it might even show tags/v2.6.19^{}^{} under -d option after the repository's refs are packed in the new format.

But I think we can avoid that problem by using a format slightly different from the WIP patch. We can for example use two spaces between SHA-1 and the name for them. refs.c::parse_ref_line() in existing code will say "oh, this line is nonsense" and skip them, while the updated code can say "ok, this is a peeled ref", since it has a seemingly incomplete "check for valid refname" there that only checks "isspace(*line)".

Did you do this "if (isspace(*line)) return NULL" to have extensibility later, I wonder...

Previous: Linus TorvaldsNext: Linus Torvalds
Message 17 of 29 in “[WISH] Store also tag dereferences in packed-refs”
  1. Marco CostalbaNov 18, 2006
  2. Junio C HamanoNov 18, 2006
  3. Petr BaudisNov 18, 2006
  4. Marco CostalbaNov 18, 2006
  5. Junio C HamanoNov 18, 2006
  6. Marco CostalbaNov 19, 2006
  7. Linus TorvaldsNov 19, 2006
  8. Junio C HamanoNov 19, 2006
  9. Junio C HamanoNov 19, 2006
  10. Linus TorvaldsNov 19, 2006
  11. Marco CostalbaNov 19, 2006
  12. Linus TorvaldsNov 19, 2006
  13. Marco CostalbaNov 19, 2006
  14. Marco CostalbaNov 19, 2006
  15. Linus TorvaldsNov 19, 2006
  16. Linus TorvaldsNov 19, 2006
  17. Junio C HamanoNov 19, 2006
  18. Linus TorvaldsNov 19, 2006
  19. Jakub NarebskiNov 19, 2006
  20. Linus TorvaldsNov 19, 2006
  21. Junio C HamanoNov 20, 2006
  22. Jakub NarebskiNov 20, 2006
  23. Marco CostalbaNov 20, 2006
  24. Linus TorvaldsNov 20, 2006
  25. Junio C HamanoNov 20, 2006
  26. Marco CostalbaNov 19, 2006
  27. Linus TorvaldsNov 19, 2006
  28. Linus TorvaldsNov 19, 2006
  29. Jakub NarebskiNov 20, 2006

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.