threads / patch / 38275

patch, 3 partsnd/multiple-work-trees updates

Subject: [PATCH 0/3] nd/multiple-work-trees updates

## tl;dr

14 messages between Jan 3, 2015 and Mar 18, 2015. Diffs are folded; open one to read it.

replies: 13people: 7as markdown or json

Nguyễn Thái Ngọc Duy· Jan 3, 2015, 09:41 UTC · lore

These patches are on top of what's in 'pu'. They add --ignore-other-worktrees and make a note about current submodule support status. I don't think submodule support is ready yet even with Max Kirillov's series [1]. His 03/03 is already fixed in 'pu' though, so only 01/03 and 02/03 are new.

[1] http://thread.gmane.org/gmane.comp.version-control.git/261107
Nguyễn Thái Ngọc Duy (3):
  checkout: pass whole struct to parse_branchname_arg instead of individual flags
  checkout: add --ignore-other-wortrees
  git-checkout.txt: a note about multiple checkout support for submodules
 Documentation/git-checkout.txt |  9 +++++++++
 builtin/checkout.c             | 19 +++++++++++--------
 t/t2025-checkout-to.sh         |  7 +++++++
 3 files changed, 27 insertions(+), 8 deletions(-)
-- 
2.2.0.84.ge9c7a8a
Nguyễn Thái Ngọc Duy· Jan 3, 2015, 09:41 UTC · re: Nguyễn Thái Ngọc Duy · lore

[PATCH 1/3] checkout: pass whole struct to parse_branchname_arg instead of individual flags

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
 builtin/checkout.c | 13 ++++++-------
 1 file changed, 6 insertions(+), 7 deletions(-)
Show changes to builtin/checkout.c +6 −7
diff --git a/builtin/checkout.c b/builtin/checkout.c
index 953b763..d8717ef 100644
--- a/builtin/checkout.c
+++ b/builtin/checkout.c
@@ -1079,11 +1079,12 @@ static void check_linked_checkouts(struct branch_info *new)
 static int parse_branchname_arg(int argc, const char **argv,
 				int dwim_new_local_branch_ok,
 				struct branch_info *new,
-				struct tree **source_tree,
-				unsigned char rev[20],
-				const char **new_branch,
-				int force_detach)
+				struct checkout_opts *opts,
+				unsigned char rev[20])
 {
+	struct tree **source_tree = &opts->source_tree;
+	const char **new_branch = &opts->new_branch;
+	int force_detach = opts->force_detach;
 	int argcount = 0;
 	unsigned char branch_rev[20];
 	const char *arg;
@@ -1420,9 +1421,7 @@ int cmd_checkout(int argc, const char **argv, const char *prefix)
 			opts.track == BRANCH_TRACK_UNSPECIFIED &&
 			!opts.new_branch;
 		int n = parse_branchname_arg(argc, argv, dwim_ok,
-					     &new, &opts.source_tree,
-					     rev, &opts.new_branch,
-					     opts.force_detach);
+					     &new, &opts, rev);
 		argv += n;
 		argc -= n;
 	}

-- 
2.2.0.84.ge9c7a8a
Nguyễn Thái Ngọc Duy· Jan 3, 2015, 09:41 UTC · re: Nguyễn Thái Ngọc Duy · lore

[PATCH 2/3] checkout: add --ignore-other-wortrees

Noticed-by: Mark Levedahl <mlevedahl@gmail.com>
Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
 Documentation/git-checkout.txt | 6 ++++++
 builtin/checkout.c             | 6 +++++-
 t/t2025-checkout-to.sh         | 7 +++++++
 3 files changed, 18 insertions(+), 1 deletion(-)
Show changes to 3 files +18 −1

Documentation/git-checkout.txt, builtin/checkout.c, t/t2025-checkout-to.sh

diff --git a/Documentation/git-checkout.txt b/Documentation/git-checkout.txt
index 0c13825..52eaa48 100644
--- a/Documentation/git-checkout.txt
+++ b/Documentation/git-checkout.txt
@@ -232,6 +232,12 @@ section of linkgit:git-add[1] to learn how to operate the `--patch` mode.
 	specific files such as HEAD, index... See "MULTIPLE WORKING
 	TREES" section for more information.
 
+--ignore-other-worktrees::
+	`git checkout` refuses when the wanted ref is already checked
+	out by another worktree. This option makes it check the ref
+	out anyway. In other words, the ref can be held by more than one
+	worktree.
+
 <branch>::
 	Branch to checkout; if it refers to a branch (i.e., a name that,
 	when prepended with "refs/heads/", is a valid ref), then that
diff --git a/builtin/checkout.c b/builtin/checkout.c
index d8717ef..8b2bf20 100644
--- a/builtin/checkout.c
+++ b/builtin/checkout.c
@@ -37,6 +37,7 @@ struct checkout_opts {
 	int writeout_stage;
 	int overwrite_ignore;
 	int ignore_skipworktree;
+	int ignore_other_worktrees;
 
 	const char *new_branch;
 	const char *new_branch_force;
@@ -1210,7 +1211,8 @@ static int parse_branchname_arg(int argc, const char **argv,
 		int flag;
 		char *head_ref = resolve_refdup("HEAD", 0, sha1, &flag);
 		if (head_ref &&
-		    (!(flag & REF_ISSYMREF) || strcmp(head_ref, new->path)))
+		    (!(flag & REF_ISSYMREF) || strcmp(head_ref, new->path)) &&
+		    !opts->ignore_other_worktrees)
 			check_linked_checkouts(new);
 		free(head_ref);
 	}
@@ -1341,6 +1343,8 @@ int cmd_checkout(int argc, const char **argv, const char *prefix)
 				N_("second guess 'git checkout no-such-branch'")),
 		OPT_FILENAME(0, "to", &opts.new_worktree,
 			   N_("check a branch out in a separate working directory")),
+		OPT_BOOL(0, "ignore-other-worktrees", &opts.ignore_other_worktrees,
+			 N_("do not check if another worktree is holding the given ref")),
 		OPT_END(),
 	};
 
diff --git a/t/t2025-checkout-to.sh b/t/t2025-checkout-to.sh
index 915b506..f8e4df4 100755
--- a/t/t2025-checkout-to.sh
+++ b/t/t2025-checkout-to.sh
@@ -79,6 +79,13 @@ test_expect_success 'die the same branch is already checked out' '
 	)
 '
 
+test_expect_success 'not die the same branch is already checked out' '
+	(
+		cd here &&
+		git checkout --ignore-other-worktrees --to anothernewmaster newmaster
+	)
+'
+
 test_expect_success 'not die on re-checking out current branch' '
 	(
 		cd there &&

-- 
2.2.0.84.ge9c7a8a
Mark Levedahl· Jan 5, 2015, 03:50 UTC · re: Nguyễn Thái Ngọc Duy · lore

Re: [PATCH 2/3] checkout: add --ignore-other-wortrees

On 01/03/2015 04:41 AM, Nguyễn Thái Ngọc Duy wrote:
Show 23 quoted lines
> Noticed-by: Mark Levedahl <mlevedahl@gmail.com>
> Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
> ---
>   Documentation/git-checkout.txt | 6 ++++++
>   builtin/checkout.c             | 6 +++++-
>   t/t2025-checkout-to.sh         | 7 +++++++
>   3 files changed, 18 insertions(+), 1 deletion(-)
>
> diff --git a/Documentation/git-checkout.txt b/Documentation/git-checkout.txt
> index 0c13825..52eaa48 100644
> --- a/Documentation/git-checkout.txt
> +++ b/Documentation/git-checkout.txt
> @@ -232,6 +232,12 @@ section of linkgit:git-add[1] to learn how to operate the `--patch` mode.
>   	specific files such as HEAD, index... See "MULTIPLE WORKING
>   	TREES" section for more information.
>   
> +--ignore-other-worktrees::
> +	`git checkout` refuses when the wanted ref is already checked
> +	out by another worktree. This option makes it check the ref
> +	out anyway. In other words, the ref can be held by more than one
> +	worktree.
> +
>

Thanks for adding this, I haven't had a chance to test but by reading this solves the problem I raised.

Mark
Nguyễn Thái Ngọc Duy· Jan 3, 2015, 09:41 UTC · re: Nguyễn Thái Ngọc Duy · lore

[PATCH 3/3] git-checkout.txt: a note about multiple checkout support for submodules

The goal seems to be using multiple checkouts to reduce disk space. But we have not reached an agreement how things should be. There are a couple options.

 - You may want to keep $SUB repos elsewhere (perhaps in a central
   place) outside $SUPER. This is also true for nested submodules
   where a superproject may be a submodule of another superproject.
 - You may want to keep all $SUB repos in $SUPER/modules (or some
   other place in $SUPER)
 - We could even push it further and merge all $SUB repos into $SUPER
   instead of storing them separately. But that would at least require
   ref namespace enabled.

On top of that, git-submodule.sh expects $GIT_DIR/config to be per-worktree, at least for the submodule.* part. Here I think we have two options, either update config.c to also read $GIT_DIR/config.worktree (which is per worktree) in addition to $GIT_DIR/config (shared) and store worktree-specific vars in the new place, or update git-submodule.sh to read/write submodule.* directly from $GIT_DIR/config.submodule (per worktree).

These take time to address properly. Meanwhile, make a note to the user that they should not use multiple worktrees in submodule context.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
 Documentation/git-checkout.txt | 3 +++
 1 file changed, 3 insertions(+)
Show changes to Documentation/git-checkout.txt +3 −0
diff --git a/Documentation/git-checkout.txt b/Documentation/git-checkout.txt
index 52eaa48..72def5b 100644
--- a/Documentation/git-checkout.txt
+++ b/Documentation/git-checkout.txt
@@ -463,6 +463,9 @@ to `/path/main/.git/worktrees/test-next` then a file named
 `test-next` entry from being pruned.  See
 linkgit:gitrepository-layout[5] for details.
 
+Multiple checkout support for submodules is incomplete. It is NOT
+recommended to make multiple checkouts of a superproject.
+
 EXAMPLES
 --------
 

-- 
2.2.0.84.ge9c7a8a
Mark Levedahl· Jan 5, 2015, 03:43 UTC · re: Nguyễn Thái Ngọc Duy · lore

Re: [PATCH 3/3] git-checkout.txt: a note about multiple checkout support for submodules

On 01/03/2015 04:41 AM, Nguyễn Thái Ngọc Duy wrote:
Show 9 quoted lines
> The goal seems to be using multiple checkouts to reduce disk space.
> But we have not reached an agreement how things should be. There are a
> couple options.
>
>   - You may want to keep $SUB repos elsewhere (perhaps in a central
>     place) outside $SUPER. This is also true for nested submodules
>     where a superproject may be a submodule of another superproject.
>
>

This is my preference: I keep a tree of bare git repos outside of all work areas, and use new-workdir to create trees of workdirs as needed. I explored trying to keep $SUB repos in others (including mods to submodule / new-workdir to manage this), found this really leads to too much complication compared to just having a set of bare repos elsewhere. This bare repo approach also has the advantage that no particular workdir is special, all workdirs that point to the same gitdir are equal.

Mark
Max Kirillov· Jan 6, 2015, 21:30 UTC · re: Nguyễn Thái Ngọc Duy · lore

Re: [PATCH 3/3] git-checkout.txt: a note about multiple checkout support for submodules

On Sat, Jan 03, 2015 at 04:41:27PM +0700, Nguyễn Thái Ngọc Duy wrote:
Show 25 quoted lines
> The goal seems to be using multiple checkouts to reduce disk space.
> But we have not reached an agreement how things should be. There are a
> couple options.
> 
>  - You may want to keep $SUB repos elsewhere (perhaps in a central
>    place) outside $SUPER. This is also true for nested submodules
>    where a superproject may be a submodule of another superproject.
> 
>  - You may want to keep all $SUB repos in $SUPER/modules (or some
>    other place in $SUPER)
> 
>  - We could even push it further and merge all $SUB repos into $SUPER
>    instead of storing them separately. But that would at least require
>    ref namespace enabled.
> 
> On top of that, git-submodule.sh expects $GIT_DIR/config to be
> per-worktree, at least for the submodule.* part. Here I think we have
> two options, either update config.c to also read
> $GIT_DIR/config.worktree (which is per worktree) in addition to
> $GIT_DIR/config (shared) and store worktree-specific vars in the new
> place, or update git-submodule.sh to read/write submodule.* directly
> from $GIT_DIR/config.submodule (per worktree).
> 
> These take time to address properly. Meanwhile, make a note to the
> user that they should not use multiple worktrees in submodule context.
I'd like to describe though how much the glass is full:

If all submodules exist in same paths in all checked-out commits, and there is no special settings, only `submodule.<name>.url` is set, then user can inspect, commit and update the submodule content. Submodules can be either independedntly initalized with `submodule update --init` (being a fully independet clone), or checked-out with `checkout --to` to the submodule worktree.

It is even verified by tests, so does it worth mentioning in some manpage or at least commit message?

-- 
Max
Junio C Hamano· Feb 12, 2015, 22:57 UTC · re: Nguyễn Thái Ngọc Duy · lore

Re: [PATCH 0/3] nd/multiple-work-trees updates

Nguyễn Thái Ngọc Duy  <pclouds@gmail.com> writes:
Show 7 quoted lines
> These patches are on top of what's in 'pu'. They add
> --ignore-other-worktrees and make a note about current submodule
> support status. I don't think submodule support is ready yet even
> with Max Kirillov's series [1]. His 03/03 is already fixed in 'pu'
> though, so only 01/03 and 02/03 are new.
>
> [1] http://thread.gmane.org/gmane.comp.version-control.git/261107

With the understanding (perhaps a strongly-worded paragraph in the release notes) that this is not suitable for submodule users yet, is this in a good enough shape to go to 'next'?

Jens Lehmann· Feb 12, 2015, 23:51 UTC · re: Junio C Hamano · lore

Re: [PATCH 0/3] nd/multiple-work-trees updates

Am 13.02.2015 um 11:57 schrieb Junio C Hamano:
Show 13 quoted lines
> Nguyễn Thái Ngọc Duy  <pclouds@gmail.com> writes:
>
>> These patches are on top of what's in 'pu'. They add
>> --ignore-other-worktrees and make a note about current submodule
>> support status. I don't think submodule support is ready yet even
>> with Max Kirillov's series [1]. His 03/03 is already fixed in 'pu'
>> though, so only 01/03 and 02/03 are new.
>>
>> [1] http://thread.gmane.org/gmane.comp.version-control.git/261107
>
> With the understanding (perhaps a strongly-worded paragraph in the
> release notes) that this is not suitable for submodule users yet,
> is this in a good enough shape to go to 'next'?

No objections from my side (and maybe we should also add a warning that *all* worktree-related configuration - e.g. EOL options - are currently always shared between all worktrees).

Adding submodule support can then be done in another series (and renaming core.worktree to something else is definitely *not* the way to do that! ;-).

Mark Levedahl· Feb 13, 2015, 03:17 UTC · re: Jens Lehmann · lore

Re: [PATCH 0/3] nd/multiple-work-trees updates

On 02/12/2015 06:51 PM, Jens Lehmann wrote:
Show 23 quoted lines
> Am 13.02.2015 um 11:57 schrieb Junio C Hamano:
>> Nguyễn Thái Ngọc Duy <pclouds@gmail.com> writes:
>>
>>> These patches are on top of what's in 'pu'. They add
>>> --ignore-other-worktrees and make a note about current submodule
>>> support status. I don't think submodule support is ready yet even
>>> with Max Kirillov's series [1]. His 03/03 is already fixed in 'pu'
>>> though, so only 01/03 and 02/03 are new.
>>>
>>> [1] http://thread.gmane.org/gmane.comp.version-control.git/261107
>>
>> With the understanding (perhaps a strongly-worded paragraph in the
>> release notes) that this is not suitable for submodule users yet,
>> is this in a good enough shape to go to 'next'?
>
> No objections from my side (and maybe we should also add a warning
> that *all* worktree-related configuration - e.g. EOL options - are
> currently always shared between all worktrees).
>
> Adding submodule support can then be done in another series (and
> renaming core.worktree to something else is definitely *not* the
> way to do that! ;-).
>

I concur the patch series is good enough for next. Better multiple worktree support for submodules is, I think, a sizeable topic that will take a while to settle, so should be worked after this base is integrated.

Mark
Dennis Kaarsemaker· Feb 13, 2015, 09:10 UTC · re: Junio C Hamano · lore

Re: [PATCH 0/3] nd/multiple-work-trees updates

On do, 2015-02-12 at 14:57 -0800, Junio C Hamano wrote:
Show 13 quoted lines
> Nguyễn Thái Ngọc Duy  <pclouds@gmail.com> writes:
> 
> > These patches are on top of what's in 'pu'. They add
> > --ignore-other-worktrees and make a note about current submodule
> > support status. I don't think submodule support is ready yet even
> > with Max Kirillov's series [1]. His 03/03 is already fixed in 'pu'
> > though, so only 01/03 and 02/03 are new.
> >
> > [1] http://thread.gmane.org/gmane.comp.version-control.git/261107
> 
> With the understanding (perhaps a strongly-worded paragraph in the
> release notes) that this is not suitable for submodule users yet,
> is this in a good enough shape to go to 'next'?

I've been using this for a while and really like it. However, it needs a few fixups to merge with next as there are a few merge conflicts.

(A version of this branch that I stuck on top of next last week can be found at https://github.com/seveas/git/tree/nd/multiple-work-trees )

-- 
Dennis Kaarsemaker
http://www.kaarsemaker.net
Duy Nguyen· Feb 13, 2015, 10:14 UTC · re: Junio C Hamano · lore

Re: [PATCH 0/3] nd/multiple-work-trees updates

On Fri, Feb 13, 2015 at 5:57 AM, Junio C Hamano <gitster@pobox.com> wrote:
Show 13 quoted lines
> Nguyễn Thái Ngọc Duy  <pclouds@gmail.com> writes:
>
>> These patches are on top of what's in 'pu'. They add
>> --ignore-other-worktrees and make a note about current submodule
>> support status. I don't think submodule support is ready yet even
>> with Max Kirillov's series [1]. His 03/03 is already fixed in 'pu'
>> though, so only 01/03 and 02/03 are new.
>>
>> [1] http://thread.gmane.org/gmane.comp.version-control.git/261107
>
> With the understanding (perhaps a strongly-worded paragraph in the
> release notes) that this is not suitable for submodule users yet,
> is this in a good enough shape to go to 'next'?
I'm not aware of any problems on this series (except submodules). So, yes.
-- 
Duy
Ephrim Khong· Mar 18, 2015, 08:04 UTC · re: Nguyễn Thái Ngọc Duy · lore

Re: [PATCH 0/3] nd/multiple-work-trees updates

Without having looked into this and nd/multiple-work-trees, but with "make multiple checkouts aware of each other" in mind: Could this mechanism be re-used to make alternates aware of each other, to mitigate the dangers of having git gc on an alternate remove objects that are used by a referencing repository?

Thanks
- Eph
On 03.01.2015 10:41, Nguyễn Thái Ngọc Duy wrote:
Show 18 quoted lines
> These patches are on top of what's in 'pu'. They add
> --ignore-other-worktrees and make a note about current submodule
> support status. I don't think submodule support is ready yet even
> with Max Kirillov's series [1]. His 03/03 is already fixed in 'pu'
> though, so only 01/03 and 02/03 are new.
>
> [1] http://thread.gmane.org/gmane.comp.version-control.git/261107
>
> Nguyễn Thái Ngọc Duy (3):
>    checkout: pass whole struct to parse_branchname_arg instead of individual flags
>    checkout: add --ignore-other-wortrees
>    git-checkout.txt: a note about multiple checkout support for submodules
>
>   Documentation/git-checkout.txt |  9 +++++++++
>   builtin/checkout.c             | 19 +++++++++++--------
>   t/t2025-checkout-to.sh         |  7 +++++++
>   3 files changed, 27 insertions(+), 8 deletions(-)
>
Duy Nguyen· Mar 18, 2015, 08:23 UTC · re: Ephrim Khong · lore

Re: [PATCH 0/3] nd/multiple-work-trees updates

On Wed, Mar 18, 2015 at 3:04 PM, Ephrim Khong <dr.khong@gmail.com> wrote:
Show 5 quoted lines
> Without having looked into this and nd/multiple-work-trees, but with "make
> multiple checkouts aware of each other" in mind: Could this mechanism be
> re-used to make alternates aware of each other, to mitigate the dangers of
> having  git gc  on an alternate remove objects that are used by a
> referencing repository?

If we can turn on ref namespace and make $GIT_DIR/config and hooks per worktree, I think it may have a chance of replacing alternate object mechanism entirely: one object database, one ref database (but refs of each worktree is namespaced so no conflicts), multiple worktrees, multiple config files, multiple hooks.

Because some config keys affect object database, having multiple/conflicting config keys imply that this worktree may change object database in a way trhat impacts performance (not correctness) of another worktree. Later on when we have multiple ref backends, if config keys can change ref backend behavior (or even choose the backend), we may run into other problems. This problem might go away if we define that those "global" config keys can't be per-worktree..

In short, I am good at confusing people.
-- 
Duy

← back to recent threads