git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH 4/6] introduce a commit metapack

From
Duy Nguyen <pclouds@gmail.com>
Date
Feb 2, 2013, 09:49 UTC
Message-ID
<CACsJy8COq-A2KJ8rNCxB23s4SMYA4Gqpr53NzqMv=i=sswV_0Q@mail.gmail.com>
In-Reply-To
<20130201101503.GG30644@sigill.intra.peff.net>
On Fri, Feb 1, 2013 at 5:15 PM, Jeff King <peff@peff.net> wrote:
> The short-sha1 is a clever idea. Looks like it saves us on the order of
> 4MB for linux-2.6 (versus the full 20-byte sha1). Not as big as the
> savings we get from dropping the other 3 sha1's to uint32_t, but still
> not bad.

We could save another 4 bytes per commit by using 3 bytes for storing .idx offsets. linux-2.6 only has 3M objects. It'll take many years for big projects to reach 16M objects and need the fourth byte in uint32_t.

Show 11 quoted lines
> I guess the next steps in iterating on this would be:
>
>   1. splitting out the refactoring here into separate patches
>
>   2. squashing the cleaned-up bits into my patch 4/6
>
>   3. deciding whether this should go into a separate file or as part of
>      index v3. Your offsets depend on the .idx file having a sorted sha1
>      list. That is not likely to change, but it would still be nice to
>      make sure they cannot get out of sync. I'm still curious what the
>      performance impact is for mmap-ing N versus N+8MB.
4. Print some cache statistics in "count-objects -v"
Show 7 quoted lines
>> The length of SHA-1 is chosen to be able to unambiguously identify any
>> cached commits. Full SHA-1 check is done after to catch false
>> positives.
>
> Just to be clear, these false positives come because the abbreviation is
> unambiguous within the packfile, but we might be looking for a commit
> that is not even in our pack, right?

It may even be ambiguous within the pack, say an octopus (i.e. not cached) commit that shares the same sha-1 prefix with one of the cached commits.

-- 
Duy
Previous: Jeff KingNext: Jeff King
Message 31 of 43 in “commit caching”
  1. 0/6 commit cachingJeff King, Jan 29, 2013
  2. 1/6 csum-file: make sha1write const-correctJeff King, Jan 29, 2013
  3. 2/6 strbuf: add string-chomping functionsJeff King, Jan 29, 2013
  4. Michael HaggertyJan 29, 2013
  5. Jeff KingJan 29, 2013
  6. Michael HaggertyJan 30, 2013
  7. 3/6 introduce pack metadata cache filesJeff King, Jan 29, 2013
  8. Junio C HamanoJan 29, 2013
  9. Jeff KingJan 30, 2013
  10. Duy NguyenJan 30, 2013
  11. Jeff KingJan 30, 2013
  12. 4/6 introduce a commit metapackJeff King, Jan 29, 2013
  13. Michael HaggertyJan 29, 2013
  14. Jeff KingJan 29, 2013
  15. Junio C HamanoJan 29, 2013
  16. Junio C HamanoJan 29, 2013
  17. Jeff KingJan 30, 2013
  18. Junio C HamanoJan 30, 2013
  19. Jeff KingFeb 1, 2013
  20. Junio C HamanoJan 30, 2013
  21. Shawn PearceJan 31, 2013
  22. Jeff KingFeb 1, 2013
  23. Junio C HamanoFeb 2, 2013
  24. Jeff KingJan 30, 2013
  25. Duy NguyenJan 30, 2013
  26. Jeff KingJan 30, 2013
  27. Duy NguyenJan 30, 2013
  28. Duy NguyenJan 30, 2013
  29. Duy NguyenJan 31, 2013
  30. Jeff KingFeb 1, 2013
  31. Duy NguyenFeb 2, 2013
  32. Jeff KingFeb 1, 2013
  33. Duy NguyenMar 17, 2013
  34. Jeff KingMar 18, 2013
  35. Jeff KingFeb 1, 2013
  36. 5/6 add git-metapack commandJeff King, Jan 29, 2013
  37. 6/6 commit: look up commit info in metapackJeff King, Jan 29, 2013
  38. Duy NguyenJan 30, 2013
  39. Jeff KingJan 30, 2013
  40. Duy NguyenJan 30, 2013
  41. Shawn PearceJan 31, 2013
  42. Jeff KingFeb 1, 2013
  43. Shawn PearceFeb 2, 2013

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.