git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH v3 0/4] connected: add incremental connectivity check

From
Kristofer Karlsson <krka@spotify.com>
Date
Oct 13, 2026, 07:54 UTC
Message-ID
<CAL71e4OUW4rYqxaGgcrsy0aqjRSdWsz8ghtNRGQVYd9Gkz4NgQ@mail.gmail.com>
In-Reply-To
<xmqqzewi695w.fsf@gitster.g>
On Mon, 12 Oct 2026 at 20:55, Junio C Hamano <gitster@pobox.com> wrote:
Show 12 quoted lines
>
> "Kristofer Karlsson via GitGitGadget" <gitgitgadget@gmail.com>
> writes:
>
> > I'll just mention a short summary here, to avoid repeating what's already in
> > the commit message. The incremental mode is faster than the full mode when
> > there are few commits to verify and when the active object tree is large. In
> > the happy case, the work tracks the changed paths and their comparison trees
> > rather than the full reachable object closure, which substantially reduces
> > the dependence on total repository size.
>
> This is exciting.

I think so too, especially since I think it simplifies the follow-up where the scaling with number of refs can also be addressed and I hope to write up some more concrete ideas for that in the not too distant future.

Show 9 quoted lines
> > There are also regression cases in the synthetic fixtures. With long
> > incoming histories, the extra parent-tree scans accumulate; in the synthetic
> > fixture incremental is about 1.5x slower at 10000 commits. Per-commit
> > ...
> > A safety net for this regression could be to dynamically disable the
> > incremental mode if the number of incoming commits is too large, but this is
> > left out of the initial version to avoid overly speculative code.
>
> Hmm, ... OK.

This is where I would need feedback from you and the mailing list. I have been reasoning about this internally and decided to simply submit the smallest approach as a starting point.

Now with the config flag in place to gate usage as well as having the worst-case regression bounded in how bad it can be (2x more time, not worse in memory) I was thinking it may not need further protection. I was also not sure how common it would be to get pushes with so many commits that it becomes an issue, and that part is fundamentally hard to know for git as a general tool.

That said, I'd be happy to introduce some extra safety net here, perhaps add a fallback if there are more than N commits to verify. (Hardcoded based on synthetic benchmarks? Configurable?)

One partly annoying thing here may be that if we add fallback for the bad case, it forces us to keep both code paths and thus lose the potential for removing code. My dream scenario would be if there was a single implementation of the connectivity check that performs well enough in all scenarios, but maybe that will come anyway eventually and I should not speculate too much.

What would be your preferred approach here?

Thanks, Kristofer

Previous: Junio C HamanoNext: Kristofer Karlsson via GitGitGadget
Message 39 of 60 in “connected: add incremental connectivity check”
  1. 0/2 connected: add incremental connectivity checkKristofer Karlsson via GitGitGadget, Sep 14, 2026
  2. 1/2 Documentation: describe connectivity checkingKristofer Karlsson via GitGitGadget, Sep 14, 2026
  3. 2/2 connected: add incremental connectivity check via rev-listKristofer Karlsson via GitGitGadget, Sep 14, 2026
  4. Junio C HamanoSep 14, 2026
  5. Kristofer KarlssonSep 14, 2026
  6. Junio C HamanoSep 14, 2026
  7. 0/2 connected: add incremental connectivity checkKristofer Karlsson via GitGitGadget, Sep 28, 2026
  8. 1/2 Documentation: describe connectivity checkingKristofer Karlsson via GitGitGadget, Sep 28, 2026
  9. Patrick SteinhardtOct 5, 2026
  10. Junio C HamanoOct 5, 2026
  11. Patrick SteinhardtOct 6, 2026
  12. Kristofer KarlssonOct 6, 2026
  13. Kristofer KarlssonOct 6, 2026
  14. 2/2 connected: add incremental connectivity check via rev-listKristofer Karlsson via GitGitGadget, Sep 28, 2026
  15. Patrick SteinhardtOct 5, 2026
  16. Kristofer KarlssonOct 6, 2026
  17. Patrick SteinhardtOct 6, 2026
  18. Kristofer KarlssonOct 6, 2026
  19. 0/4 connected: add incremental connectivity checkKristofer Karlsson via GitGitGadget, Oct 12, 2026
  20. 1/4 Documentation: describe connectivity checkingKristofer Karlsson via GitGitGadget, Oct 12, 2026
  21. Patrick SteinhardtOct 15, 2026
  22. Kristofer KarlssonOct 15, 2026
  23. 2/4 p5413: add perf tests for connectivity checkKristofer Karlsson via GitGitGadget, Oct 12, 2026
  24. Patrick SteinhardtOct 15, 2026
  25. Kristofer KarlssonOct 15, 2026
  26. 3/4 connected: add transfer.connectivityCheck config and rev-list flagKristofer Karlsson via GitGitGadget, Oct 12, 2026
  27. Junio C HamanoOct 12, 2026
  28. Kristofer KarlssonOct 13, 2026
  29. Junio C HamanoOct 13, 2026
  30. Patrick SteinhardtOct 15, 2026
  31. Kristofer KarlssonOct 15, 2026
  32. 4/4 connected: add incremental connectivity check via rev-listKristofer Karlsson via GitGitGadget, Oct 12, 2026
  33. Patrick SteinhardtOct 15, 2026
  34. Kristofer KarlssonOct 15, 2026
  35. Patrick SteinhardtOct 15, 2026
  36. Kristofer KarlssonOct 16, 2026
  37. Patrick SteinhardtOct 19, 2026
  38. Junio C HamanoOct 12, 2026
  39. Kristofer KarlssonOct 13, 2026
  40. 0/6 connected: add incremental connectivity checkKristofer Karlsson via GitGitGadget, Oct 21, 2026
  41. 1/6 Documentation: describe connectivity checkingKristofer Karlsson via GitGitGadget, Oct 21, 2026
  42. 2/6 p5413: add perf tests for connectivity checkKristofer Karlsson via GitGitGadget, Oct 21, 2026
  43. 3/6 connected: drop dependency on `the_repository`Kristofer Karlsson via GitGitGadget, Oct 21, 2026
  44. 4/6 connected: add transfer.connectivityCheck configKristofer Karlsson via GitGitGadget, Oct 21, 2026
  45. 5/6 connected: add incremental connectivity check via rev-listKristofer Karlsson via GitGitGadget, Oct 21, 2026
  46. Junio C HamanoOct 21, 2026
  47. Kristofer KarlssonOct 22, 2026
  48. Kristofer KarlssonOct 22, 2026
  49. Junio C HamanoOct 22, 2026
  50. Kristofer KarlssonOct 22, 2026
  51. Junio C HamanoOct 23, 2026
  52. 6/6 connected: add "auto" connectivity check modeKristofer Karlsson via GitGitGadget, Oct 21, 2026
  53. 0/6 connected: add incremental connectivity checkKristofer Karlsson via GitGitGadget, Oct 23, 2026
  54. 1/6 Documentation: describe connectivity checkingKristofer Karlsson via GitGitGadget, Oct 23, 2026
  55. 2/6 p5413: add perf tests for connectivity checkKristofer Karlsson via GitGitGadget, Oct 23, 2026
  56. 3/6 connected: drop dependency on `the_repository`Kristofer Karlsson via GitGitGadget, Oct 23, 2026
  57. 4/6 connected: add transfer.connectivityCheck configKristofer Karlsson via GitGitGadget, Oct 23, 2026
  58. 5/6 connected: add incremental connectivity check via rev-listKristofer Karlsson via GitGitGadget, Oct 23, 2026
  59. 6/6 connected: add "auto" connectivity check modeKristofer Karlsson via GitGitGadget, Oct 23, 2026
  60. Junio C HamanoOct 23, 2026

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.