git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH 2/2] setup: don't fail if commondir reference is deleted.

From
Michal Suchánek <msuchanek@suse.de>
Date
Feb 21, 2019, 17:33 UTC
Message-ID
<20190221183348.30f759c2@kitsune.suse.cz>
In-Reply-To
<CAPig+cTQMZFF-oX-SOzB5JR=V1WThBihC+kNm-2wjbpAWf-OHA@mail.gmail.com>

On Thu, 21 Feb 2019 12:12:28 -0500 Eric Sunshine <sunshine@sunshineco.com> wrote:

Show 24 quoted lines
> On Thu, Feb 21, 2019 at 12:07 PM Phillip Wood <phillip.wood@talktalk.net> wrote:
> > On 21/02/2019 13:50, Michal Suchánek wrote:  
> > >> On Tue, Feb 19, 2019 at 12:05 AM Michal Suchanek <msuchanek@suse.de> wrote:  
> > > The problem is we don't forbid worktree names ending with ".lock".
> > > Which means that if we start to forbid them now existing worktrees
> > > might become inaccessible.  
> >
> > I think it is also racy as the renaming breaks the use of mkdir erroring
> > out if the directory already exists. One solution is to have a lock
> > entry in $GIT_COMMON_DIR/worktree-locks and make sure the code that
> > iterates over the entries in $GIT_COMMON_DIR/worktrees skips any that
> > have a corresponding ignores in $GIT_COMMON_DIR/worktree-locks. If the
> > worktree-locks/<dir> is created before worktree/<dir> then it should be
> > race free (you will have to remove the lock if the real entry cannot be
> > created and then increment the counter and try again). Entries could
> > also be locked on removal to prevent a race there.  
> 
> I wonder, though, how much this helps or hinders the use-case which
> prompted this patch series in the first place; to wit, creating
> hundreds or thousands of worktrees. Doing so serially was too slow, so
> the many "git worktree add" invocations were instead run in parallel
> (which led to "discovery" of race conditions). Using a global worktree
> lock would serialize worktree creation, thus slowing it down once
> again.

I created thousands of worktrees only for stress-testing. The real workload needs only a dozen of them. That still leads to hitting a race condition occasionally and automation failure.

Creating a separate lock directory will probably work. The question is when do you need to take the lock. Before adding a worktree, sure. Before deleting it as well. The problem is that deleting a worktree successfully without creating some broken state needs to exclude processes that might add stuff in the worktree directory. How many operations then do *not* need to take the lock?

Thanks
Michal
Previous: Michal SuchánekNext: Duy Nguyen
Message 11 of 27 in “worktree add race fix”
  1. 0/2 worktree add race fixMichal Suchanek, Feb 18, 2019
  2. 1/2 worktree: fix worktree add race.Michal Suchanek, Feb 18, 2019
  3. 2/2 setup: don't fail if commondir reference is deleted.Michal Suchanek, Feb 18, 2019
  4. Eric SunshineFeb 18, 2019
  5. Duy NguyenFeb 21, 2019
  6. Michal SuchánekFeb 21, 2019
  7. Phillip WoodFeb 21, 2019
  8. Eric SunshineFeb 21, 2019
  9. Phillip WoodFeb 21, 2019
  10. Michal SuchánekMar 4, 2019
  11. Michal SuchánekFeb 21, 2019
  12. Duy NguyenFeb 22, 2019
  13. Phillip WoodFeb 22, 2019
  14. Duy NguyenFeb 22, 2019
  15. 1/2 worktree: fix worktree add race.Michal Suchanek, Feb 20, 2019
  16. Eric SunshineFeb 20, 2019
  17. Michal SuchánekFeb 20, 2019
  18. Duy NguyenMar 8, 2019
  19. Eric SunshineMar 8, 2019
  20. Junio C HamanoMar 11, 2019
  21. 2/2 setup: don't fail if commondir reference is deleted.Michal Suchanek, Feb 20, 2019
  22. Eric SunshineFeb 20, 2019
  23. Michal SuchánekFeb 20, 2019
  24. Eric SunshineFeb 20, 2019
  25. Eric SunshineFeb 21, 2019
  26. Michal SuchánekFeb 21, 2019
  27. Michal SuchánekFeb 21, 2019

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.