git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH 6/6] fetch: make `--atomic` flag cover pruning of refs

From
Christian Couder <christian.couder@gmail.com>
Date
Feb 15, 2022, 09:12 UTC
Message-ID
<CAP8UFD3Fc9315jsbTFNzGunLyrm0P=zDLda2F=7O_5+B-ZtBOA@mail.gmail.com>
In-Reply-To
<682f16117b743bec59c533e15ae5a88d39250222.1644565025.git.ps@pks.im>
On Fri, Feb 11, 2022 at 9:25 AM Patrick Steinhardt <ps@pks.im> wrote:
Show 6 quoted lines
>
> When fetching with the `--prune` flag we will delete any local
> references matching the fetch refspec which have disappeared on the
> remote. This step is not currently covered by the `--atomic` flag: we
> delete branches even though updating of local references has failed,
> which means that the fetch is not an all-or-nothing operation.

It could perhaps be seen as a regression by some users though, if updating of local references doesn't work anymore when deleting a local reference matching the fetch refspec fails.

Show 8 quoted lines
> Fix this bug by passing in the global transaction into `prune_refs()`:
> if one is given, then we'll only queue up deletions and not commit them
> right away.
>
> This change also improves performance when pruning many branches in a
> repository with a big packed-refs file: every references is pruned in
> its own transaction, which means that we potentially have to rewrite
> the packed-refs files for every single reference we're about to prune.

Yeah, I wonder if there could be a performance improvement in the previous patch too as it looks like tag backfilling wasn't atomic too.

Show 17 quoted lines
> The following benchmark demonstrates this: it performs a pruning fetch
> from a repository with a single reference into a repository with 100k
> references, which causes us to prune all but one reference. This is of
> course a very artificial setup, but serves to demonstrate the impact of
> only having to write the packed-refs file once:
>
>     Benchmark 1: git fetch --prune --atomic +refs/*:refs/* (HEAD~)
>       Time (mean ± σ):      2.366 s ±  0.021 s    [User: 0.858 s, System: 1.508 s]
>       Range (min … max):    2.328 s …  2.407 s    10 runs
>
>     Benchmark 2: git fetch --prune --atomic +refs/*:refs/* (HEAD)
>       Time (mean ± σ):      1.369 s ±  0.017 s    [User: 0.715 s, System: 0.641 s]
>       Range (min … max):    1.346 s …  1.400 s    10 runs
>
>     Summary
>       'git fetch --prune --atomic +refs/*:refs/* (HEAD)' ran
>         1.73 ± 0.03 times faster than 'git fetch --prune --atomic +refs/*:refs/* (HEAD~)'
Nice!
Previous: Patrick SteinhardtNext: Patrick Steinhardt
Message 23 of 27 in “fetch: improve atomicity of `--atomic` flag”
  1. 0/6 fetch: improve atomicity of `--atomic` flagPatrick Steinhardt, Feb 11, 2022
  2. 1/6 fetch: increase test coverage of fetchesPatrick Steinhardt, Feb 11, 2022
  3. Christian CouderFeb 15, 2022
  4. Patrick SteinhardtFeb 17, 2022
  5. 2/6 fetch: backfill tags before setting upstreamPatrick Steinhardt, Feb 11, 2022
  6. Christian CouderFeb 15, 2022
  7. 3/6 fetch: control lifecycle of FETCH_HEAD in a single placePatrick Steinhardt, Feb 11, 2022
  8. Christian CouderFeb 15, 2022
  9. Patrick SteinhardtFeb 17, 2022
  10. 4/6 fetch: report errors when backfilling tags failsPatrick Steinhardt, Feb 11, 2022
  11. Christian CouderFeb 15, 2022
  12. Patrick SteinhardtFeb 17, 2022
  13. Patrick SteinhardtFeb 17, 2022
  14. Jonathan TanFeb 16, 2022
  15. Elijah NewrenFeb 17, 2022
  16. 5/6 fetch: make `--atomic` flag cover backfilling of tagsPatrick Steinhardt, Feb 11, 2022
  17. Christian CouderFeb 15, 2022
  18. Jonathan TanFeb 16, 2022
  19. Patrick SteinhardtFeb 17, 2022
  20. Elijah NewrenFeb 17, 2022
  21. Patrick SteinhardtFeb 17, 2022
  22. 6/6 fetch: make `--atomic` flag cover pruning of refsPatrick Steinhardt, Feb 11, 2022
  23. Christian CouderFeb 15, 2022
  24. Patrick SteinhardtFeb 17, 2022
  25. Jonathan TanFeb 16, 2022
  26. Elijah NewrenFeb 17, 2022
  27. Patrick SteinhardtFeb 17, 2022

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.