git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH 1/3] fetch: use batched reference updates

From
Patrick Steinhardt <ps@pks.im>
Date
May 15, 2025, 11:30 UTC
Message-ID
<aCXQcSsc4p6u9ljZ@pks.im>
In-Reply-To
<CAOLa=ZRbEqs6X1KJj-CikCANX-BC8r4RqbwoT06qmtF=x+7hQA@mail.gmail.com>
On Thu, May 15, 2025 at 11:13:32AM +0000, Karthik Nayak wrote:
Show 31 quoted lines
> Patrick Steinhardt <ps@pks.im> writes:
> >> +	 * since pruning must be an independent step, to avoid F/D conflicts.
> >> +	 */
> >> +	if (!transaction) {
> >> +		transaction = ref_store_transaction_begin(get_main_ref_store(the_repository),
> >> +							  REF_TRANSACTION_ALLOW_FAILURE, &err);
> >> +		if (!transaction) {
> >> +			retcode = -1;
> >> +			goto cleanup;
> >> +		}
> >> +	}
> >> +
> >>  	if (fetch_and_consume_refs(&display_state, transport, transaction, ref_map,
> >>  				   &fetch_head, config)) {
> >>  		retcode = 1;
> >
> > Don't transactions handle D/F conflicts for us? Isn't that the sole
> > reason why for example `refs_verify_refname_available()` accepts an
> > "extras" parameter that is supposed to contain refs that are about to be
> > deleted?
> >
> 
> My understanding was a little different, from the documentation for the
> function:
> 
>   If extras is non-NULL, it is a list of additional refnames with which
>   refname is not allowed to conflict.
> 
> This is to capture additional conflicts. We want a way to avoid said
> conflicts. That said, there is a 'skip' parameter which does exactly
> what you're saying.
Oh, right, my mistake -- that's what I actually meant.
Show 21 quoted lines
> But the transaction logic doesn't incorporate this
> entirely. Specifically in the files backend, where we create a lock in
> the filesystem, this would cause a conflict, consider the following:
> 
>   ❯ eza --tree .git/refs/remotes/
>   .git/refs/remotes
>   └── origin
>       ├── dir
>       │   └── file.lock
>       ├── dir.lock
>       └── HEAD
> 
> This is from the test 'branchname D/F conflict resolved by --prune', the
> test prunes the existing reference 'refs/remotes/origin/dir/file' while
> adding 'refs/remotes/origin/dir'. In 'lock_raw_ref()' we lock both and
> read the reference, but this causes an issue since
> 'refs/remotes/origin/dir' exists as a directory already.
> 
> I would say this is logically solvable if we start treating conflict
> resolution within updates as a first class problem. But perhaps that's
> something of a patch series in itself and better solved outside of this?

Fair enough, makes sense. Might be worth it to add a TODO comment there then?

Patrick
Previous: Karthik NayakNext: Karthik Nayak
Message 5 of 30 in “fetch/receive: use batched reference updates”
  1. 0/3 fetch/receive: use batched reference updatesKarthik Nayak, May 14, 2025
  2. 1/3 fetch: use batched reference updatesKarthik Nayak, May 14, 2025
  3. Patrick SteinhardtMay 14, 2025
  4. Karthik NayakMay 15, 2025
  5. Patrick SteinhardtMay 15, 2025
  6. Karthik NayakMay 15, 2025
  7. Junio C HamanoMay 14, 2025
  8. 3/3 receive-pack: use batched reference updatesKarthik Nayak, May 14, 2025
  9. Patrick SteinhardtMay 14, 2025
  10. Junio C HamanoMay 14, 2025
  11. Karthik NayakMay 15, 2025
  12. 2/3 send-pack: fix memory leak around duplicate refsKarthik Nayak, May 14, 2025
  13. Junio C HamanoMay 14, 2025
  14. Karthik NayakMay 15, 2025
  15. 0/4 fetch/receive: use batched reference updatesKarthik Nayak, May 15, 2025
  16. 1/4 refs: add function to translate errors to stringsKarthik Nayak, May 15, 2025
  17. Jeff KingMay 15, 2025
  18. Karthik NayakMay 16, 2025
  19. Junio C HamanoMay 15, 2025
  20. Karthik NayakMay 16, 2025
  21. 2/4 fetch: use batched reference updatesKarthik Nayak, May 15, 2025
  22. Patrick SteinhardtMay 16, 2025
  23. Karthik NayakMay 16, 2025
  24. Patrick SteinhardtMay 16, 2025
  25. Karthik NayakMay 18, 2025
  26. 3/4 send-pack: fix memory leak around duplicate refsKarthik Nayak, May 15, 2025
  27. 4/4 receive-pack: use batched reference updatesKarthik Nayak, May 15, 2025
  28. Jeff KingMay 15, 2025
  29. Jeff KingMay 15, 2025
  30. Karthik NayakMay 16, 2025

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.