git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH 0/6] Sparse Index: Integrate with merge, cherry-pick, rebase, and revert

From
Derrick Stolee via GitGitGadget <gitgitgadget@gmail.com>
Date
Aug 17, 2021, 17:08 UTC
Message-ID
<pull.1019.git.1629220124.gitgitgadget@gmail.com>

This series integrates the sparse index with commands that perform merges such as 'git merge', 'git cherry-pick', 'git revert' (free with cherry-pick), and 'git rebase'.

When the ORT merge strategy is enabled, this allows most merges to succeed without expanding the sparse index, leading to significant performance gains. I tested these changes against an internal monorepo with over 2 million paths at HEAD but with a sparse-checkout that only has ~60,000 files within the sparse-checkout cone. 'git merge' commands went from 5-6 seconds to 0.750-1.250s.

In the case of the recursive merge strategy, the sparse index is expanded before the recursive algorithm proceeds. We expect that this is as good as we can get with that strategy. When the strategy shifts to ORT as the default, then this will not be a problem except for users who decide to change the behavior.

Most of the hard work was done by previous series, such as ds/sparse-index-ignored-files (which this series is based on).

Thanks, -Stolee
Derrick Stolee (6):
  t1092: use ORT merge strategy
  diff: ignore sparse paths in diffstat
  merge: make sparse-aware with ORT
  merge-ort: expand only for out-of-cone conflicts
  t1092: add cherry-pick, rebase tests
  sparse-index: integrate with cherry-pick and rebase
 builtin/merge.c                          |  3 +
 builtin/rebase.c                         |  6 ++
 builtin/revert.c                         |  3 +
 diff.c                                   |  8 ++
 merge-ort.c                              | 16 ++++
 merge-recursive.c                        |  3 +
 t/t1092-sparse-checkout-compatibility.sh | 97 +++++++++++++++++++++---
 7 files changed, 124 insertions(+), 12 deletions(-)
base-commit: febef675f051eb08896751bb5661b6deb5579ead
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-1019%2Fderrickstolee%2Fsparse-index%2Fmerge-v1
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-1019/derrickstolee/sparse-index/merge-v1
Pull-Request: https://github.com/gitgitgadget/git/pull/1019
-- 
gitgitgadget
Next: Derrick Stolee via GitGitGadget
Message 1 of 45 in “Sparse Index: Integrate with merge, cherry-pick, rebase, and revert”
  1. 0/6 Sparse Index: Integrate with merge, cherry-pick, rebase, and revertDerrick Stolee via GitGitGadget, Aug 17, 2021
  2. 1/6 t1092: use ORT merge strategyDerrick Stolee via GitGitGadget, Aug 17, 2021
  3. Taylor BlauAug 18, 2021
  4. Junio C HamanoAug 18, 2021
  5. Derrick StoleeAug 18, 2021
  6. Junio C HamanoAug 18, 2021
  7. Elijah NewrenAug 20, 2021
  8. Junio C HamanoAug 20, 2021
  9. Elijah NewrenAug 21, 2021
  10. Junio C HamanoAug 21, 2021
  11. Elijah NewrenAug 21, 2021
  12. 2/6 diff: ignore sparse paths in diffstatDerrick Stolee via GitGitGadget, Aug 17, 2021
  13. Elijah NewrenAug 20, 2021
  14. Derrick StoleeAug 24, 2021
  15. Elijah NewrenAug 27, 2021
  16. 3/6 merge: make sparse-aware with ORTDerrick Stolee via GitGitGadget, Aug 17, 2021
  17. Elijah NewrenAug 20, 2021
  18. 4/6 merge-ort: expand only for out-of-cone conflictsDerrick Stolee via GitGitGadget, Aug 17, 2021
  19. Elijah NewrenAug 20, 2021
  20. 5/6 t1092: add cherry-pick, rebase testsDerrick Stolee via GitGitGadget, Aug 17, 2021
  21. Elijah NewrenAug 20, 2021
  22. 6/6 sparse-index: integrate with cherry-pick and rebaseDerrick Stolee via GitGitGadget, Aug 17, 2021
  23. Elijah NewrenAug 21, 2021
  24. 0/6 Sparse Index: Integrate with merge, cherry-pick, rebase, and revertDerrick Stolee via GitGitGadget, Aug 24, 2021
  25. 2/6 merge: make sparse-aware with ORTDerrick Stolee via GitGitGadget, Aug 24, 2021
  26. Elijah NewrenAug 27, 2021
  27. Derrick StoleeAug 30, 2021
  28. Derrick StoleeSep 8, 2021
  29. 1/6 diff: ignore sparse paths in diffstatDerrick Stolee via GitGitGadget, Aug 24, 2021
  30. 3/6 merge-ort: expand only for out-of-cone conflictsDerrick Stolee via GitGitGadget, Aug 24, 2021
  31. Elijah NewrenAug 27, 2021
  32. Derrick StoleeAug 30, 2021
  33. 4/6 t1092: add cherry-pick, rebase testsDerrick Stolee via GitGitGadget, Aug 24, 2021
  34. 5/6 sequencer: ensure full index if not ORT strategyDerrick Stolee via GitGitGadget, Aug 24, 2021
  35. 6/6 sparse-index: integrate with cherry-pick and rebaseDerrick Stolee via GitGitGadget, Aug 24, 2021
  36. Elijah NewrenAug 27, 2021
  37. Derrick StoleeAug 30, 2021
  38. 0/6 Sparse Index: Integrate with merge, cherry-pick, rebase, and revertDerrick Stolee via GitGitGadget, Sep 8, 2021
  39. 1/6 diff: ignore sparse paths in diffstatDerrick Stolee via GitGitGadget, Sep 8, 2021
  40. 2/6 merge: make sparse-aware with ORTDerrick Stolee via GitGitGadget, Sep 8, 2021
  41. 4/6 t1092: add cherry-pick, rebase testsDerrick Stolee via GitGitGadget, Sep 8, 2021
  42. 3/6 merge-ort: expand only for out-of-cone conflictsDerrick Stolee via GitGitGadget, Sep 8, 2021
  43. 5/6 sequencer: ensure full index if not ORT strategyDerrick Stolee via GitGitGadget, Sep 8, 2021
  44. 6/6 sparse-index: integrate with cherry-pick and rebaseDerrick Stolee via GitGitGadget, Sep 8, 2021
  45. Elijah NewrenSep 9, 2021

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.