git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: git status: small difference between stating whole repository and small subdirectory

From
Thomas Rast <trast@inf.ethz.ch>
Date
Feb 22, 2012, 13:17 UTC
Message-ID
<878vjvxb1i.fsf@thomas.inf.ethz.ch>
In-Reply-To
<CACsJy8BZyokLSpbA=C6YEO1JajyemT9vLG=C9W4E4coA5OfszQ@mail.gmail.com>
Nguyen Thai Ngoc Duy <pclouds@gmail.com> writes:
Show 19 quoted lines
> On Sat, Feb 18, 2012 at 5:25 AM, Junio C Hamano <gitster@pobox.com> wrote:
>> Jeff King <peff@peff.net> writes:
>>
>>> That being said, we do have an index extension to store the tree sha1 of
>>> whole directories (i.e., we populate it when we write a whole tree or
>>> subtree into the index from the object db, and it becomes invalidated
>>> when a file becomes modified). This optimization is used by things like
>>> "git commit" to avoid having to recreate the same sub-trees over and
>>> over when creating tree objects from the index. But we could also use it
>>> here to avoid having to even read the sub-tree objects from the object
>>> db.
>>
>> Like b65982b (Optimize "diff-index --cached" using cache-tree, 2009-05-20)
>> perhaps?
>
> This optimizes the case when a cached tree matches entirely.I wonder
> whether it's faster if we switch to tree-tree diff whenever we find
> valid cached trees. If cache-tree is fully valid, "git diff --cached
> foo" would be equivalent to "git diff HEAD foo".

Not necessarily; the cache-tree is valid if it faithfully represents what is in the index. It does not have any direct relation to HEAD.

Show 7 quoted lines
> I tried "git diff --raw HEAD HEAD~100" (where HEAD was
> v3.1-rc1-272-g73e0881 on linux-2.6) and "git diff --cached --raw
> HEAD~100" with no cache-tree. The former is a little bit faster than
> the latter (177ms vs 275ms). On gentoo-x86, 70k worktree files, it's
> 4.33s vs 4.45s. But in tree-tree diff we pay high in cold cache case
> for loading trees from "HEAD". So no, probably not worth more code
> changes. Your optimization is good enough.

I'm still wondering about using mincore() to good effect. I tried it for git-grep, but it ended up slowing things down. However, it irks me that in some cases a clueful use of one form over the other can really make a huge performance difference, e.g.,

  git grep stuff
  git grep HEAD stuff

If I am in a big repository that I haven't used in a while, the HEAD form will be much faster as the worktree search would fault many files. OTOH if I am in a heavily-used repository (and perhaps just said 'make' minutes ago) the worktree version will avoid the pack decompression effort.

Sadly this also has the problem that we must first determine whether substituting HEAD for the worktree (or vice versa) is valid at all. For grep perhaps there could be a "just do a fast search somewhere" option since usually you are looking for something that hasn't changed in ages.

Ok, that was almost completely beside the point of this thread.
-- 
Thomas Rast
trast@{inf,student}.ethz.ch
Previous: Nguyen Thai Ngoc DuyNext: Nguyen Thai Ngoc Duy
Message 41 of 52 in “git status: small difference between stating whole repository and small subdirectory”
  1. Piotr KrukowieckiFeb 10, 2012
  2. Nguyen Thai Ngoc DuyFeb 10, 2012
  3. Piotr KrukowieckiFeb 10, 2012
  4. Nguyen Thai Ngoc DuyFeb 10, 2012
  5. Piotr KrukowieckiFeb 13, 2012
  6. Piotr KrukowieckiFeb 10, 2012
  7. Thomas RastFeb 14, 2012
  8. Piotr KrukowieckiFeb 15, 2012
  9. Nguyen Thai Ngoc DuyFeb 15, 2012
  10. Piotr KrukowieckiFeb 15, 2012
  11. Piotr KrukowieckiFeb 16, 2012
  12. Jeff KingFeb 15, 2012
  13. Piotr KrukowieckiFeb 16, 2012
  14. Thomas RastFeb 16, 2012
  15. Junio C HamanoFeb 16, 2012
  16. Piotr KrukowieckiFeb 17, 2012
  17. Jeff KingFeb 16, 2012
  18. Piotr KrukowieckiFeb 17, 2012
  19. Jeff KingFeb 17, 2012
  20. Junio C HamanoFeb 17, 2012
  21. Jeff KingFeb 17, 2012
  22. Piotr KrukowieckiFeb 20, 2012
  23. Jeff KingFeb 20, 2012
  24. Thomas RastFeb 20, 2012
  25. Nguyen Thai Ngoc DuyFeb 20, 2012
  26. Jeff KingFeb 20, 2012
  27. Jeff KingFeb 20, 2012
  28. Thomas RastFeb 20, 2012
  29. Jeff KingFeb 20, 2012
  30. Junio C HamanoFeb 20, 2012
  31. Jeff KingFeb 20, 2012
  32. Junio C HamanoFeb 20, 2012
  33. Piotr KrukowieckiFeb 21, 2012
  34. Junio C HamanoFeb 20, 2012
  35. Jeff KingFeb 20, 2012
  36. Nguyen Thai Ngoc DuyFeb 21, 2012
  37. Junio C HamanoFeb 21, 2012
  38. Nguyen Thai Ngoc DuyFeb 22, 2012
  39. Junio C HamanoFeb 22, 2012
  40. Nguyen Thai Ngoc DuyFeb 22, 2012
  41. Thomas RastFeb 22, 2012
  42. Nguyen Thai Ngoc DuyFeb 22, 2012
  43. Junio C HamanoFeb 22, 2012
  44. Piotr KrukowieckiApr 10, 2012
  45. Junio C HamanoApr 10, 2012
  46. Jeff KingApr 10, 2012
  47. Junio C HamanoFeb 20, 2012
  48. Thomas RastFeb 20, 2012
  49. Nguyen Thai Ngoc DuyFeb 20, 2012
  50. Jeff KingFeb 20, 2012
  51. Junio C HamanoFeb 20, 2012
  52. Jeff KingFeb 20, 2012

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.