git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [External] Re: [PATCH v4 1/1] commit-graph.c: no lazy fetch in lookup_commit_in_graph()

From
Han Xin <hanxin.hx@bytedance.com>
Date
Jul 12, 2022, 01:52 UTC
Message-ID
<CAKgqsWVD2108f0PyJGp6mVKp2cGd_V_MiiQO3SAPm+LEHcb2mA@mail.gmail.com>
In-Reply-To
<xmqqk08jo147.fsf@gitster.g>
On Tue, Jul 12, 2022 at 4:17 AM Junio C Hamano <gitster@pobox.com> wrote:
Show 33 quoted lines
>
> Jeff King <peff@peff.net> writes:
>
> >> 512 is probably OK in CI in an isolated environment but is too low on a
> >> typical "What you mean I'm not working? I'm waiting for the test run!"
> >> developper workstation.
> >>
> >> Conversely, which number would be too high to catch what the test is
> >> supposed to catch? Does it incur a big performance penalty to go as high
> >> as possible?
> >
> > This bit me, too. It works if I run it standalone:
> >
> >   $ ./t5330-no-lazy-fetch-with-commit-graph.sh
> >   ok 1 - setup: prepare a repository with a commit
> >   ok 2 - setup: prepare a repository with commit-graph contains the commit
> >   ok 3 - setup: change the alternates to what without the commit
> >   ok 4 - fetch any commit from promisor with the usage of the commit graph
> >   # passed all 4 test(s)
> >
> > but it fails when I run the whole test suite with "prove -j32". Or even
> > easier, just run it under "--stress":
>
> Understandable.  I am usually on a datacentre VM without graphical
> UI so the process count there is much lower than on a typical
> developer workstation.
>
> I wonder if we can just run the test without any limit?  If in an
> unattended CI situation, hopefully they will kick the job out due to
> quota, and on a developer workstation, there may be processes killed
> left and right, but that is only when the "infinite respawning" bug
> reappears.
>

The tricky thing about using ulimit is that it's tied to the entire development station. I have tried to run the test without any limit [1], it did finally be canceled after 6 hours.

Remove the "ulimit", once the infinite loop reappears, this test seems like a bomb to developers that quickly consumes all resources? While "1024" works fine with "--stress" , there are reasons to wonder if it's enough, or maybe we can take a value like 10240 that we wouldn't normally reach?

1. https://github.com/chiyutianyi/git/runs/6962635320

Thanks -Han Xin

Previous: Junio C HamanoNext: Junio C Hamano
Message 40 of 50 in “Re: An endless loop fetching issue with partial clone, alternates and commit graph”
  1. Haiyng TanJun 14, 2022
  2. Taylor BlauJun 15, 2022
  3. 0/2 Re: An endless loop fetching issue with partial clone, alternates and commit graphHan Xin, Jun 16, 2022
  4. 1/2 commit-graph.c: add "flags" to lookup_commit_in_graph()Han Xin, Jun 16, 2022
  5. 2/2 fetch-pack.c: pass "oi_flags" to lookup_commit_in_graph()Han Xin, Jun 16, 2022
  6. Jonathan TanJun 17, 2022
  7. commit-graph.c: no lazy fetch in lookup_commit_in_graph()Han Xin, Jun 18, 2022
  8. Patrick SteinhardtJun 20, 2022
  9. 欣韩Jun 20, 2022
  10. Patrick SteinhardtJun 20, 2022
  11. Jonathan TanJun 21, 2022
  12. Han XinJun 22, 2022
  13. 0/2 commit-graph.c: no lazy fetch in lookup_commit_in_graph()Han Xin, Jun 24, 2022
  14. 1/2 test-lib.sh: add limited processes to test-libHan Xin, Jun 24, 2022
  15. Junio C HamanoJun 24, 2022
  16. Han XinJun 25, 2022
  17. Junio C HamanoJun 27, 2022
  18. 2/2 commit-graph.c: no lazy fetch in lookup_commit_in_graph()Han Xin, Jun 24, 2022
  19. Junio C HamanoJun 24, 2022
  20. Han XinJun 25, 2022
  21. Han XinJun 25, 2022
  22. 0/2 no lazy fetch in lookup_commit_in_graph()Han Xin, Jun 28, 2022
  23. 1/2 test-lib.sh: add limited processes to test-libHan Xin, Jun 28, 2022
  24. 2/2 commit-graph.c: no lazy fetch in lookup_commit_in_graph()Han Xin, Jun 28, 2022
  25. Ævar Arnfjörð BjarmasonJun 28, 2022
  26. Junio C HamanoJun 28, 2022
  27. Johannes SchindelinJun 30, 2022
  28. Ævar Arnfjörð BjarmasonJun 30, 2022
  29. Junio C HamanoJun 30, 2022
  30. Ævar Arnfjörð BjarmasonJun 30, 2022
  31. Johannes SchindelinJul 1, 2022
  32. Junio C HamanoJul 1, 2022
  33. Han XinJun 29, 2022
  34. test name conflict + js/ci-github-workflow-markup regression (was: [PATCH v3 0/2] no lazy fetch in lookup_commit_in_graph())Ævar Arnfjörð Bjarmason, Jun 30, 2022
  35. 0/1 no lazy fetch in lookup_commit_in_graph()Han Xin, Jul 1, 2022
  36. 1/1 commit-graph.c: no lazy fetch in lookup_commit_in_graph()Han Xin, Jul 1, 2022
  37. Michael J GruberJul 9, 2022
  38. Jeff KingJul 11, 2022
  39. Junio C HamanoJul 11, 2022
  40. Han XinJul 12, 2022
  41. Junio C HamanoJul 12, 2022
  42. Han XinJul 12, 2022
  43. Jeff KingJul 12, 2022
  44. Junio C HamanoJul 12, 2022
  45. 0/1 no lazy fetch in lookup_commit_in_graph()Han Xin, Jul 12, 2022
  46. 1/1 commit-graph.c: no lazy fetch in lookup_commit_in_graph()Han Xin, Jul 12, 2022
  47. Ævar Arnfjörð BjarmasonJul 12, 2022
  48. Han XinJul 13, 2022
  49. Jeff KingJul 12, 2022
  50. t5330: remove run_with_limited_processses()Han Xin, Jul 12, 2022

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.