git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH v5 0/1] no lazy fetch in lookup_commit_in_graph()

From
Han Xin <hanxin.hx@bytedance.com>
Date
Jul 12, 2022, 06:50 UTC
Message-ID
<cover.1657604799.git.hanxin.hx@bytedance.com>
In-Reply-To
<cover.1656593279.git.hanxin.hx@bytedance.com>

This patch fixes the following issue: When we found the commit in the graph in lookup_commit_in_graph(), but the commit is missing from the repository, we will try promisor_remote_get_direct() and then enter another loop.

Then we will go into an endless loop:
  git fetch -> deref_without_lazy_fetch() ->
    lookup_commit_in_graph() -> repo_has_object_file() ->
      promisor_remote_get_direct() -> fetch_objects() ->
        git fetch (a new loop round)
Changes since v4:
* Remove run_with_limited_processses() as it can be catched by CI
  settings and developer workstation. Keeping it will make a trouble
  when there are too many prcesses or stress is used.
Han Xin (1):
  commit-graph.c: no lazy fetch in lookup_commit_in_graph()
 commit-graph.c                             |  2 +-
 t/t5330-no-lazy-fetch-with-commit-graph.sh | 47 ++++++++++++++++++++++
 2 files changed, 48 insertions(+), 1 deletion(-)
 create mode 100755 t/t5330-no-lazy-fetch-with-commit-graph.sh
Range-diff against v4:
1:  96d4bb7150 ! 1:  3ffeed50de commit-graph.c: no lazy fetch in lookup_commit_in_graph()
    @@ t/t5330-no-lazy-fetch-with-commit-graph.sh (new)
     +
     +. ./test-lib.sh
     +
    -+run_with_limited_processses () {
    -+	# bash and ksh use "ulimit -u", dash uses "ulimit -p"
    -+	if test -n "$BASH_VERSION"
    -+	then
    -+		ulimit_max_process="-u"
    -+	elif test -n "$KSH_VERSION"
    -+	then
    -+		ulimit_max_process="-u"
    -+	fi
    -+	(ulimit ${ulimit_max_process-"-p"} 512 && "$@")
    -+}
    -+
    -+test_lazy_prereq ULIMIT_PROCESSES '
    -+	run_with_limited_processses true
    -+'
    -+
    -+if ! test_have_prereq ULIMIT_PROCESSES
    -+then
    -+	skip_all='skipping tests for no lazy fetch with the commit-graph, ulimit processes not available'
    -+	test_done
    -+fi
    -+
     +test_expect_success 'setup: prepare a repository with a commit' '
     +	git init with-commit &&
     +	test_commit -C with-commit the-commit &&
    @@ t/t5330-no-lazy-fetch-with-commit-graph.sh (new)
     +	git -C with-commit-graph config remote.origin.partialclonefilter blob:none &&
     +	test_commit -C with-commit any-commit &&
     +	anycommit=$(git -C with-commit rev-parse HEAD) &&
    -+
    -+	run_with_limited_processses env GIT_TRACE="$(pwd)/trace.txt" \
    ++	GIT_TRACE="$(pwd)/trace.txt" \
     +		git -C with-commit-graph fetch origin $anycommit 2>err &&
     +	! grep "fatal: promisor-remote: unable to fork off fetch subprocess" err &&
     +	grep "git fetch origin" trace.txt >actual &&
-- 
2.36.1
Previous: Junio C HamanoNext: Han Xin
Message 45 of 50 in “Re: An endless loop fetching issue with partial clone, alternates and commit graph”
  1. Haiyng TanJun 14, 2022
  2. Taylor BlauJun 15, 2022
  3. 0/2 Re: An endless loop fetching issue with partial clone, alternates and commit graphHan Xin, Jun 16, 2022
  4. 1/2 commit-graph.c: add "flags" to lookup_commit_in_graph()Han Xin, Jun 16, 2022
  5. 2/2 fetch-pack.c: pass "oi_flags" to lookup_commit_in_graph()Han Xin, Jun 16, 2022
  6. Jonathan TanJun 17, 2022
  7. commit-graph.c: no lazy fetch in lookup_commit_in_graph()Han Xin, Jun 18, 2022
  8. Patrick SteinhardtJun 20, 2022
  9. 欣韩Jun 20, 2022
  10. Patrick SteinhardtJun 20, 2022
  11. Jonathan TanJun 21, 2022
  12. Han XinJun 22, 2022
  13. 0/2 commit-graph.c: no lazy fetch in lookup_commit_in_graph()Han Xin, Jun 24, 2022
  14. 1/2 test-lib.sh: add limited processes to test-libHan Xin, Jun 24, 2022
  15. Junio C HamanoJun 24, 2022
  16. Han XinJun 25, 2022
  17. Junio C HamanoJun 27, 2022
  18. 2/2 commit-graph.c: no lazy fetch in lookup_commit_in_graph()Han Xin, Jun 24, 2022
  19. Junio C HamanoJun 24, 2022
  20. Han XinJun 25, 2022
  21. Han XinJun 25, 2022
  22. 0/2 no lazy fetch in lookup_commit_in_graph()Han Xin, Jun 28, 2022
  23. 1/2 test-lib.sh: add limited processes to test-libHan Xin, Jun 28, 2022
  24. 2/2 commit-graph.c: no lazy fetch in lookup_commit_in_graph()Han Xin, Jun 28, 2022
  25. Ævar Arnfjörð BjarmasonJun 28, 2022
  26. Junio C HamanoJun 28, 2022
  27. Johannes SchindelinJun 30, 2022
  28. Ævar Arnfjörð BjarmasonJun 30, 2022
  29. Junio C HamanoJun 30, 2022
  30. Ævar Arnfjörð BjarmasonJun 30, 2022
  31. Johannes SchindelinJul 1, 2022
  32. Junio C HamanoJul 1, 2022
  33. Han XinJun 29, 2022
  34. test name conflict + js/ci-github-workflow-markup regression (was: [PATCH v3 0/2] no lazy fetch in lookup_commit_in_graph())Ævar Arnfjörð Bjarmason, Jun 30, 2022
  35. 0/1 no lazy fetch in lookup_commit_in_graph()Han Xin, Jul 1, 2022
  36. 1/1 commit-graph.c: no lazy fetch in lookup_commit_in_graph()Han Xin, Jul 1, 2022
  37. Michael J GruberJul 9, 2022
  38. Jeff KingJul 11, 2022
  39. Junio C HamanoJul 11, 2022
  40. Han XinJul 12, 2022
  41. Junio C HamanoJul 12, 2022
  42. Han XinJul 12, 2022
  43. Jeff KingJul 12, 2022
  44. Junio C HamanoJul 12, 2022
  45. 0/1 no lazy fetch in lookup_commit_in_graph()Han Xin, Jul 12, 2022
  46. 1/1 commit-graph.c: no lazy fetch in lookup_commit_in_graph()Han Xin, Jul 12, 2022
  47. Ævar Arnfjörð BjarmasonJul 12, 2022
  48. Han XinJul 13, 2022
  49. Jeff KingJul 12, 2022
  50. t5330: remove run_with_limited_processses()Han Xin, Jul 12, 2022

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.