git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH 0/3] Lazy-load trees when reading commit-graph

From
Derrick Stolee <stolee@gmail.com>
Date
Apr 3, 2018, 12:15 UTC
Message-ID
<00d5cd22-3492-73cf-3493-1f6f1ec15922@gmail.com>
In-Reply-To
<20180403120057.173849-1-dstolee@microsoft.com>
On 4/3/2018 8:00 AM, Derrick Stolee wrote:
Show 44 quoted lines
> There are several commit-graph walks that require loading many commits
> but never walk the trees reachable from those commits. However, the
> current logic in parse_commit() requires the root tree to be loaded.
> This only uses lookup_tree(), but when reading commits from the commit-
> graph file, the hashcpy() to load the root tree hash and the time spent
> checking the object cache take more time than parsing the rest of the
> commit.
>
> In this patch series, all direct references to accessing the 'tree'
> member of struct commit are replaced instead by one of the following
> methods:
>
> 	struct tree *get_commit_tree(struct commit *)
> 	struct object_id *get_commit_tree_oid(struct commit *)
>
> This replacement was assisted by a Coccinelle script, but the 'tree'
> member is overloaded in other types, so the script gave false-positives
> that were removed from the diff.
>
> After all access is restricted to use these methods, we can then
> change the postcondition of parse_commit_in_graph() to allow 'tree'
> to be NULL. If the tree is accessed later, we can load the tree's
> OID from the commit-graph in constant time and perform the lookup_tree().
>
> On the Linux repository, performance tests were run for the following
> command:
>
>      git log --graph --oneline -1000
>
> Before: 0.83s
> After:  0.65s
> Rel %: -21.6%
>
> Adding '-- kernel/' to the command requires loading the root tree
> for every commit that is walked. There was no measureable performance
> change as a result of this patch.
>
> This patch series depends on v7 of ds/commit-graph.
>
> Derrick Stolee (3):
>    commit: create get_commit_tree() method
>    treewide: use get_commit_tree() for tree access
>    commit-graph: lazy-load trees
>
This patch series is also available as a GitHub pull request [1]
[1] https://github.com/derrickstolee/git/pull/4
Previous: Derrick StoleeNext: Jeff King
Message 8 of 26 in “Lazy-load trees when reading commit-graph”
  1. 0/3 Lazy-load trees when reading commit-graphDerrick Stolee, Apr 3, 2018
  2. 1/3 commit: create get_commit_tree() methodDerrick Stolee, Apr 3, 2018
  3. 3/3 commit-graph: lazy-load treesDerrick Stolee, Apr 3, 2018
  4. Stefan BellerApr 3, 2018
  5. Derrick StoleeApr 3, 2018
  6. Stefan BellerApr 3, 2018
  7. 2/3 treewide: use get_commit_tree() for tree accessDerrick Stolee, Apr 3, 2018
  8. Derrick StoleeApr 3, 2018
  9. Jeff KingApr 3, 2018
  10. Derrick StoleeApr 3, 2018
  11. Jeff KingApr 3, 2018
  12. Derrick StoleeApr 4, 2018
  13. 0/4 Lazy-load trees when reading commit-graphDerrick Stolee, Apr 6, 2018
  14. 2/4 commit: create get_commit_tree() methodDerrick Stolee, Apr 6, 2018
  15. 1/4 treewide: rename tree to maybe_treeDerrick Stolee, Apr 6, 2018
  16. 3/4 treewide: replace maybe_tree with accessor methodsDerrick Stolee, Apr 6, 2018
  17. 4/4 commit-graph: lazy-load trees for commitsDerrick Stolee, Apr 6, 2018
  18. Jeff KingApr 6, 2018
  19. Derrick StoleeApr 6, 2018
  20. Stefan BellerApr 6, 2018
  21. Junio C HamanoApr 8, 2018
  22. Derrick StoleeApr 9, 2018
  23. Stefan BellerApr 9, 2018
  24. Jakub NarebskiApr 7, 2018
  25. Derrick StoleeApr 8, 2018
  26. Jakub NarebskiApr 11, 2018

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.