Re: Parallel worktree checkouts result in index.lock exists

From: Raul Rangel <rrangel@chromium.org>
Date: Mar 7, 2023, 17:25 UTC
Message-ID: <CAHQZ30BrwiW6ia3v31gto9D8oejkj2UrqK93MjPFNwBJuOBK0Q@mail.gmail.com>
In-Reply-To: <Y/VcbWogtHrpXU8Z@tapette.crustytoothpaste.net>

On Tue, Feb 21, 2023 at 5:06 PM brian m. carlson <sandals@crustytoothpaste.net> wrote:

Show 39 quoted lines

>
> On 2023-02-21 at 17:33:28, Raul E Rangel wrote:
> > Hello,
> > I'm trying to extract multiple trees in parallel so I can create a
> > tarball of the trees. I can't use `git archive` since it doesn't
> > currently produce hermetic output, and I need to support older git
> > versions.
> >
> > In essence what I'm trying to do is:
> >
> >     git --work-tree ~/tmp/bob1 checkout ff27f5415797ead8bc775518a08f3a4ab24abd53 -- . &
> >     git --work-tree ~/tmp/bob2 checkout e70ebd7c76b9f9ad44b59e3002a5c57be5b9dc12 -- . &
> >
> > When I do this though, I get the following error:
> >     [1] 4027482
> >     [2] 4027483
> >     fatal: Unable to create '/home/rrangel/cros-bazel/.repo/project-objects/chromiumos/platform/vboot_reference.git/./index.lock': File exists.
> >
> >     Another git process seems to be running in this repository, e.g.
> >     an editor opened by 'git commit'. Please make sure all processes
> >     are terminated then try again. If it still fails, a git process
> >     may have crashed in this repository earlier:
> >     remove the file manually to continue.
> >
> > Is this expected? I'm not sure why the index is coming into play here.
> > Is there another method I should be using to extract a tree into a
> > directory?
>
> This is expected because when you do a checkout, the timestamps and
> other metadata of the files are written into the index.  This is what
> makes `git status` work quickly: if the metadata of the files hasn't
> changed, Git doesn't have to re-read them to verify their contents.  You
> definitely don't want to delete that because you'll likely end up with
> corrupt data.
>
> If you want to create multiple worktrees, use `git worktree add`, which
> can create multiple worktrees that each have their own index, but share
> the object store.  When you're done with it, run `git worktree remove`,
> and everything will be cleaned up.

Great thanks for confirming. I ended up doing the following:

    git clone --shared --bare vboot_reference.git ~/tmp/repo-1 && git
-C ~/tmp/repo-1 --work-tree ~/tmp/bob1 checkout
ff27f5415797ead8bc775518a08f3a4ab24abd53 -- . &
    git clone --shared --bare vboot_reference.git ~/tmp/repo-2 && git
-C ~/tmp/repo-2 --work-tree ~/tmp/bob1 checkout
e70ebd7c76b9f9ad44b59e3002a5c57be5b9dc12 -- . &

It's a lot cleaner than my hacky `find` command. I didn't want to use `git worktree` since that would modify the original repo. I just wanted to extract the files, so the shared + bare repo works great.

Thanks!

Show 6 quoted lines

>
> Note that with worktrees, you can have at most one worktree with a given
> branch (or detached head) checked out at a time.
> --
> brian m. carlson (he/him or they/them)
> Toronto, Ontario, CA