[long] worktree setup cases

Hi,

I want to know how I expect setup code to work and whether it matches my expectation. So I list every possible cases and try to sum up what git does and what I expect. Hopefully all corner cases are covered (and fixed later). It's not an easy read, but any comments are appreciated. If all go well, this document should be turned into test cases.

As a side note, Debian bug #575917 is case #17. I did not take into account what setup_git_directory() does (which causes #575917) though. Once the cleanup is done, setup_git_directory() should become a one-liner around setup_git_directory_gently().

Assumptions:

1. When .git is a file and contains a relative path, I assume it is
   relative to .git file's parent directory.  read_gitfile_gently()
   function will make the path absolute, but it depends on (maybe
   wrong) cwd.

2. Relative path is preferred over absolute path.

3. core.worktree is overidden by GIT_WORK_TREE

There are a few factors that affect gitdir/worktree/cwd/prefix setup. Those are:

 - GIT_DIR (or --git-dir)
 - GIT_WORK_TREE (or --work-tree)
 - core.worktree
 - .git is a file pointing to real .git directory
 - Bare repo

So there are 2^5 = 32 cases in total. Let's look at them one by one. What describes here usually matches current code, except for "should" phrases/sentences. Note that I did not look at the code carefully so I might misunderstand things.

The following questions must be answered for each case:

0. What setup function handles that case?
1. Is worktree available?
2. Where is new cwd? Is it at worktree root?
3. Prefix?
4. Is (relative) $GIT_DIR accessible from current cwd?
5. What if original cwd is outside worktree, or new cwd if it's not at
   worktree root?

Table of Contents
=================
1 Cases
    1.1 Case 0, 8
    1.2 Case 1, 4, 5, 9, 12, 13
        1.2.1 cwd outside worktree
    1.3 Case 2, 10
    1.4 Case 3, 6, 7, 11, 14, 15
        1.4.1 cwd outside worktree
    1.5 Case 16, 24
    1.6 Case 17, 20, 21, 25, 28, 29
        1.6.1 cwd outside worktree
    1.7 Case 18, 26
    1.8 Case 19, 22, 23, 27, 30, 31

1 Cases ~~~~~~~~

Case# Bare .git-file core.worktree GIT_DIR GIT_WORK_TREE 0 0 0 0 0 0 1 0 0 0 0 1 2 0 0 0 1 0 3 0 0 0 1 1 4 0 0 1 0 0 5 0 0 1 0 1 6 0 0 1 1 0 7 0 0 1 1 1 8 0 1 0 0 0 9 0 1 0 0 1 10 0 1 0 1 0 11 0 1 0 1 1 12 0 1 1 0 0 13 0 1 1 0 1 14 0 1 1 1 0 15 0 1 1 1 1 16 1 0 0 0 0 17 1 0 0 0 1 18 1 0 0 1 0 19 1 0 0 1 1 20 1 0 1 0 0 21 1 0 1 0 1 22 1 0 1 1 0 23 1 0 1 1 1 24 1 1 0 0 0 25 1 1 0 0 1 26 1 1 0 1 0 27 1 1 0 1 1 28 1 1 1 0 0 29 1 1 1 0 1 30 1 1 1 1 0 31 1 1 1 1 1

1.1 Case 0, 8 ==============

The most used case, nothing special is set.

0. setup_discovered_git_dir()
1. work tree is .git dir's parent directory
2. cwd is at worktree root, i.e. .git dir's parent dir
3. prefix is calculated from original cwd
4. git_dir is set to ".git" (#0) or according to .git file, made
   absolute (#8)
5. N/A

TODO: turn git_dir to relative in #8

1.2 Case 1, 4, 5, 9, 12, 13 ============================

0. setup_discovered_git_dir()
1. work tree is set according to GIT_WORK_TREE (#1, #5, #9, #13) or
   core.worktree (#4, #12)
2. cwd is at worktree root, i.e. .git dir's parent dir
3. prefix is calculated from original cwd
4. git_dir is set to ".git" (#1, #4, #5) or according to .git file,
   made absolute (#9, #12, #13)

TODO: turn git_dir to relative in #9, #12, #13

1.2.1 cwd outside worktree ---------------------------

FIXME

1.3 Case 2, 10 ===============

0. setup_explicit_git_dir()
1. worktree is set at cwd for legacy reason
2. cwd is unchanged
3. prefix is NULL
4. git_dir is set according to GIT_DIR (#2) or according to .git file,
   made absolute (#10)

TODO: turn git_dir to relative path

Note on #10: setup_git_env() is used to read .git file

1.4 Case 3, 6, 7, 11, 14, 15 =============================

Another normal case where worktree is at an unsual case.

0. setup_explicit_git_dir()
1. worktree is set according to GIT_WORK_TREE (#3, #7, #11, #15) or
   core.worktree (#6, #14)
2. cwd is moved to worktree root dir if original cwd is inside
   worktree
3. prefix is calculated if original cwd is inside worktree
4. git_dir is set to GIT_DIR (#3, #6, #7) or according to .git file,
   made absolute (#11, #14, #15)

TODO: if GIT_DIR is relative, it is assumed relative to original cwd,
so it breaks because cwd now is changed. Right now this does not
happen because git_dir is turned absolute. Although it's better to be
relative.

TODO: turn git_dir to relative in #11, #14, #15

FIXME on #11, #14, #15: setup_git_env() is used to read .git file. cwd by that time is worktree root, not .git's parent dir anymore.

1.4.1 cwd outside worktree ---------------------------

cwd is left unchanged, prefix is NULL, which is sensible for full tree operations. is_inside_work_tree() returns 0, operations that require worktree should check and error out.

TODO: File path output however may not be what user expected because
it will be relative to worktree root, not original cwd.

1.5 Case 16, 24 ================

The most used case, nothing special is set.

0. setup_bare_git_dir()
1. no worktree
2. cwd is unchanged (actually it is moved back to original cwd)
3. prefix is NULL
4. git_dir is set in form "../../../" (or ".") (#16) or according to
   .git file, made absolute (#24)
5. N/A

TODO on #24: turn git_dir to relative

FIXME on #24: cwd is not at .git's parent dir, so relative git_dir fails assumption #1.

1.6 Case 17, 20, 21, 25, 28, 29 ================================

0. setup_bare_git_dir()
1. work tree is set according to GIT_WORK_TREE (#17, #21, #25, #29) or
   core.worktree (#20, #28)
2. cwd is unchanged (actually it is moved back to original
   cwd).
3. prefix is calculated from original cwd
4. git_dir is set in form "../../../" (or ".") (#17, #20, #21) or according to
   .git file, made absolute (#25 , #28, #29).

TODO: if core.bare = true, die()

FIXME: cwd should be moved to worktree's root instead. Similarly,
git_dir should be recalculated relative to worktree's root

TODO: turn git_dir to relative in #25, #28, #29 (or all if the above
FIXME turns everything to absolute)

Note on #25, #28, #9: pay attention to assumption #1 when the above FIXME is done.

1.6.1 cwd outside worktree ---------------------------

FIXME

1.7 Case 18, 26 ================

This is bare repo because of core.bare = true.

0. setup_explicit_git_dir()
1. no worktree
2. cwd is unchanged
3. prefix is NULL
4. git_dir is set according to GIT_DIR (#18) or according to .git file,
   made absolute (#126)

FIXME: in current code, I believe this is actually #2/#10 (worktree is
set). Should that be fixed??? "Legacy reason" hanging around so I'm
not sure.

TODO: turn git_dir to relative path

Note on #26: setup_git_env() is used to read .git file

1.8 Case 19, 22, 23, 27, 30, 31 ================================

This is bare repo because of core.bare = true.

0. setup_explicit_git_dir()
1. worktree is set according to GIT_WORK_TREE (#19, #23, #27, #31) or
   core.worktree (#22, #30)
2. cwd is moved to worktree root dir if original cwd is inside
   worktree
3. prefix is calculated if original cwd is inside worktree
4. git_dir is set to GIT_DIR (#19, #22, #23) or according to .git file,
   made absolute (#27, #30, #31)
5. N/A

FIXME: just die() in check_repository_format_gently().

Nguyen Thai Ngoc Duy wrote:

Yep.

I'm tempted to say: let's use absolute paths where it's more convenient. They can be changed to relative paths as an afterthought after the bugs are gone.

Relative paths have two big advantages over absolute paths, which are avoiding path traversal overhead and allowing paths longer than PATH_MAX. Supporting the latter consistently would presumably require using openat() and co, though.

Yeah.

Thanks! To summarize (and make sure I understand correctly): anything not following the below rules is a bug, yes?

A. Weird cases.

 - using a .git file is just like setting GIT_DIR;
 - setting core.worktree is just like setting GIT_WORK_TREE.

B. Repository search.

 - if GIT_DIR was set explicitly, GIT_WORK_TREE defaults to
   "." (for legacy reasons).

 - otherwise, if original cwd was under repository, it will not
   prompt a search for work tree, even if the repo happens
   to be named ".git" or core.bare is false.  That is, in
   this case, GIT_WORK_TREE defaults to unset.

 - otherwise, if original cwd was under a directory containing
   repository as ".git", GIT_WORK_TREE defaults to that
   directory (i.e., parent to .git dir).

 - otherwise, there is no repository.  GIT_DIR is unset,
   GIT_WORK_TREE defaults to unset.

C. Working directory and prefix

 - if GIT_WORK_TREE is still unset after repository search,
   stay in the original cwd, prefix = NULL.

 - if original cwd is inside worktree, chdir to toplevel,
   prefix = path to original cwd.

 - otherwise, stay in the original cwd, prefix = NULL.

D. User-supplied relative paths.

 - path in .git file is relative to containing directory
 - path in GIT_DIR is relative to original cwd
 - paths in GIT_WORK_TREE and core.worktree are relative to
   $GIT_DIR
 - paths passed to git commands are generally relative to
   original cwd

E. Internally used relative paths.

 - all paths are relative to current cwd.

Hi again,

One correction.

Jonathan Nieder wrote:

In this case GIT_WORK_TREE defaults to the original cwd. GIT_WORK_TREE=. would have meant work tree == git dir, which is wrong.

Sorry for the noise.

On Thu, Oct 21, 2010 at 2:07 AM, Jonathan Nieder <jrnieder@gmail.com> wrote:

Let's see.

Yes, except that GIT_DIR can be detected early when cwd has not been moved. When .git is found a file, cwd could have been changed.

Correct.

What do you mean by "under repository"? If the repo is /tmp/git/.git, then cwd is at /tmp/git/.git?

Yes.

- Otherwise, move up one dir and repeat?

if GIT_WORK_TREE and core.worktree are still unset, we get a bare repo here (or force it to be a bare repo), so yes, cwd should stay in original cwd and prefix = NULL.

Yes.

I'm not really happy with this, which is why I wrote the --cwd-to-worktree and --worktree-to-cwd patch. But this should be enough for full-tree operations to work, so yes.

I think $GIT_WORK_TREE is relative to original cwd. Yes, core.worktree should be relative to $GIT_DIR.

And filename output should also be relative to original cwd (except a few special, like diff output).

Yes.

Nguyen Thai Ngoc Duy wrote:

[...]

I meant if the original cwd lies within the repository.

Example:

 repo		/tmp/git/.git
 starting cwd	/tmp/git/.git/objects/pack

git.txt is confusing, then.  Actually it has some other insights:

 --work-tree=<path>
	Set the path to the working tree. The value
	will not be used in combination with
	repositories found automatically in a .git
	directory (i.e. $GIT_DIR is not set).

So GIT_WORK_TREE should be discarded or warned about when GIT_DIR is not set. (?)

	This can also be controlled by setting the
	GIT_WORK_TREE environment variable and the
	core.worktree configuration variable. It can be
	an absolute path or relative path to the
	directory specified by --git-dir or GIT_DIR.

This is where I got the impression about relative paths.

	Note: If --git-dir or GIT_DIR are specified but
	none of --work-tree, GIT_WORK_TREE and
	core.worktree is specified, the current working
	directory is regarded as the top directory of
	your working tree.

Nice to see this case is documented.

Speaking of which, it is not clear to me that core.worktree should fall under the forbidden case discussed above. If it does, what is the point of making it configurable?

Thanks for checking.

Jonathan

On Thu, Oct 21, 2010 at 10:30 AM, Jonathan Nieder <jrnieder@gmail.com> wrote:

OK it's considered a bare repo, so no worktree.

Yeah. Junio reminded me in another mail. It should warn (at least, me).

Hmm.. OK then. So worktree can be relative to gitdir, which in turn can be relative to original cwd. Fun.

I was not the one who introduced core.worktree, so I can't really tell. Maybe less keystrokes? You know, "GIT_DIR=foo.git git foo" is shorter than "GIT_DIR=foo.git GIT_WORK_TREE=/path/to/somewhere git foo". I tend to think that core.worktree is always effective regardless $GIT_DIR env setting. But I was wrong.

Also the "relative to $GIT_DIR" may be confusing. If $GIT_DIR points to a file that points to a true repo, then to which one it is relative?

Nguyen Thai Ngoc Duy wrote:

Yeah, it seems you're totally right. :(

So in clarifying the semantics, core.worktree without GIT_DIR should be forbidden. But consider this a feature request. By the time core.worktree has been read, it is obvious what the GIT_DIR was supposed to be:

References:

 - v0 (GIT_TOP_DIR):
   http://thread.gmane.org/gmane.comp.version-control.git/38382
 - v1 (GIT_WORK_DIR):
   http://thread.gmane.org/gmane.comp.version-control.git/41902
 - v2 (GIT_WORK_TREE):
   http://thread.gmane.org/gmane.comp.version-control.git/42416

On Thu, Oct 21, 2010 at 11:01 PM, Jonathan Nieder <jrnieder@gmail.com> wrote:

Actually I have my part in this mess too: f5e025a (Documentation: always respect core.worktree if set - 2009-12-29). I probably just updated the document according to the code. It matches what Enrico expects (i.e after .git discovery, worktree defaults is "../", if core.worktree exists, follow it instead). Going back to the first commit that introduced separate worktree, 892c41b (introduce GIT_WORK_TREE to specify the work tree - 2007-06-06), core.worktree is no different than --work-tree.

So what should it behave? Either revert f5e025a and update the code to disregard/warn core.worktree if GIT_DIR/--work-tree is unset, or keep the current behavior, i.e. core.worktree is different from --work-tree/GIT_WORK_TREE.

The cleanup commit, e90fdc3 (Clean up work-tree handling - 2007-08-01), also gives something interesting regarding bare repo: "--work-tree=bla overrides GIT_WORK_TREE, _which overrides core.bare = true_, which overrides core.worktree, which overrides GIT_DIR/.. when GIT_DIR ends in /.git, which overrides the directory in which .git/ was found.". I don't like this overriding chain, which is why I proposed to die() if core.bare is set and worktree is explicitly specified.

* Nguyen Thai Ngoc Duy <pclouds@gmail.com> wrote:

Not necessarily: if config tells it's a non-bare repo, we could assume, the worktree is where core.worktree tells (which per default is "../" - relative to gitdir).

But currently it seems to be implemented as you said: running git-status from within gitdir tells:

    "fatal: This operation must be run in a work tree"

That's not really bad, but sometimes a bit inconvenient.

Despite the fact that I doubt the usefulness of an .git file at all, it IMHO should be interpreted as an kind of userland symlink.

cu

On Fri, Oct 22, 2010 at 1:51 AM, Enrico Weigelt <weigelt@metux.de> wrote:

I do find .git file useful. I put my worktree in a server where it can be wiped out completely. So I don't want to put my .git there. .git file can be used this case so I don't have to set GIT_DIR/GIT_WORK_TREE every time and my .git is safe elsewhere.

* Nguyen Thai Ngoc Duy <pclouds@gmail.com> wrote:

Why not using a symlink here ?

cu

Enrico Weigelt wrote:

Suppose I wonder just that. grepping through setup.c for ".git file", I find the function read_gitfile_gently() --- why was that added?

$ git log -Sread_gitfile_gently setup.c
commit b44ebb19e3234c5dffe9869ceac5408bb44c2e20
Author: Lars Hjemli <hjemli@gmail.com>
Date:   Wed Feb 20 23:13:13 2008 +0100

    Add platform-independent .git "symlink"
    
    This patch allows .git to be a regular textfile containing the path of
    the real git directory (prefixed with "gitdir: "), which can be useful on
    platforms lacking support for real symlinks.
    
    Signed-off-by: Lars Hjemli <hjemli@gmail.com>
    Signed-off-by: Junio C Hamano <gitster@pobox.com>

Hope that helps.

On Fri, Oct 22, 2010 at 12:00 PM, Enrico Weigelt <weigelt@metux.de> wrote:

Paranoia. A fault in Makefile's clean rule may make deletion follow symlinks. Also symlinks are not available everywhere.

Nguyen Thai Ngoc Duy <pclouds@gmail.com> writes:

Ok. I agree that a regular file .git that has "gitdir: ../there" in it should be naming "there" directory next to the directory .git lives in. It is insane for it to take $(cwd) into account.

Hmm, why is core.worktree separate from the second item? In any case, the three mechanisms to specify the working tree should only be in effect when GIT_DIR/--git-dir is used, so the above are not orthogonal, and the total count should be smaller than 32 cases.

All good questions to ask, except for the last one which I cannot quite parse.

Ok.

How does it make sense to have GIT_WORK_TREE without GIT_DIR? Without GIT_DIR, we will run repository discovery, which means we will always go up to find a .git dir, and the reason for doing that is because we want to also work in a subdirectory of a working tree (the very original git never worked from anywhere other than the root level of the working tree). By definition, the root of the working tree is the same as in cases 0/8.

If you set GIT_DIR, we do no discovery, so git will work only from the root level of the working tree (or bare repository operation) if you do not tell us where the working tree is.

Without discovery, we know where the root level of the working tree is, and we know where the repository is, because you told us. The answers to questions 1-5, i.e. semantics observable by the end user, should be the same as case 0/8 even though the internal codepath to achieve that, i.e. question 0, may be different.

Shouldn't all of these 16 be the same, if the repository is bare? What is your definition of bareness? core.bare? In any case we should say "you are using a bare repository, there is no working tree" and cwd shouldn't change in these cases. They are all bare and there is no working tree.

Alternatively, you could give precedence to core.worktree and friends, in which case these can go to the other categories in your description, ignoring core.bare settings. For example, 31 explicitly specifies where the .git directory and the working tree are, which would fall into the same category as 3,6,7,11,14,15.

Either way is fine.

Ok.

As I said already, isn't this a nonsense combination you should error out?

Ok.

Ok.

This is operating out of bounds of the original design criteria; we probably do not error out now, which is an original bug, but I understand that you are trying to enhance this to reach into a submodule repository and its working tree from its superproject. As long as we can make such an enhancement in a way that produces sensible and consistent output, that kind of change would be fine.

I suspect that you would need to pass in "make the path relative to this" using some means that do not exist in the current structure of the code.

Also I suspect that for the purpose of your enhancement, the first three lines of this paragraph would not be true. Think of a case where you are at the superproject level and running a recursive grep into its submodule at "nitfol", that has contents recorded at the path "xyzzy/frotz.txt".

If you keep cwd unchanged, the path appears at nitfol/xyzzy/frotz.txt in the working tree from the end user's point of view, so you need to pass "nitfol/" as a different kind of "prefix" that needs to be used to modify the path recorded in the contents tracked by the submodule. The grep running in the submodule will read xyzzy/frotz.txt from the index (which is how it notices which paths are of interest), use the "nitfol/" prefix to turn it into a path in the working tree, read from that file and report hits. This is an example of an operation that clearly require a working tree, and does not have to error out.

Alternatively, if you move cwd to the top of the working tree of the submodule, you would still need to pass "nitfol/" prefix, but it needs to be used only in the output phase. The grep running in the submodule will read xyzzy/frotz.txt from the index (which is how it notices which paths are of interest), read from the file in the working tree (relative to cwd which now is at the root level of the submodule), and use "nitfol/" as output prefix when reporting.

On Thu, Oct 21, 2010 at 4:00 AM, Junio C Hamano <gitster@pobox.com> wrote:

It's how the code does it. GIT_WORK_TREE can be detected early, when cwd has not been moved. When core.worktree is found, cwd could be somewhere. I need to separate those cases to make sure cwd is not misused after it's been moved.

Good to hear. I forgot that GIT_DIR must be set for the three mechanisms in effect.

Skip "or new cwd... worktree root" part.

OK.

OK

OK too.

Better this way. Although some sanity checks can be used. Like setting core.bare and explicitly requesting worktree is insane.

I don't want to ignore core.bare. If there are bare/worktree conflicts, notify users. There are cases that a normal repo tends to become a bare repo for just one operation. It's when "." is found a repo, handled by setup_bare_git_dir(). In these cases, we might want to consider core.worktree and friends again.

* Junio C Hamano <gitster@pobox.com> wrote:

Well, we could look at config whether it's an non-bare repo and then lookup worktree via core.worktree (which would default to "../").

BTW: the whole discovery process IMHO should start w/ looking for 
the gitdir. Could be done this way:

a) explicitly given (via GIT_DIR or --git-dir), take this one. b) else: recursively scan for a valid .git dir from cwd upstairs to /

Once we've found gitdir, we can look for the worktree. Again:

a) explicitly given, then take it
b) bare repo: dont have any, bail out
c) non-bare repo: look up via core.worktree (w/ default to $GIT_DIR/../)
 

ACK. In a bare repo, the worktree lookup will fail per definition, so worktree operations can only run with having it explicitly given.

cu

Enrico Weigelt <weigelt@metux.de> writes:

core.worktree is "you tell us where the working tree is" in the above.

That should be more or less what we have been doing since the first day we started supporting git operations from subdirectories around d288a70 ([PATCH] Make "git diff" work inside relative subdirectories, 2005-08-16).

It is possible that we might have broken some over the course while adding core.worktree and GIT_WORK_TREE support, but I doubt it.