git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH 11/34] $GIT_COMMON_DIR: a new environment variable

From
Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Date
Nov 30, 2014, 08:24 UTC
Message-ID
<1417335899-27307-12-git-send-email-pclouds@gmail.com>
In-Reply-To
<1417335899-27307-1-git-send-email-pclouds@gmail.com>

This variable is intended to support multiple working directories attached to a repository. Such a repository may have a main working directory, created by either "git init" or "git clone" and one or more linked working directories. These working directories and the main repository share the same repository directory.

In linked working directories, $GIT_COMMON_DIR must be defined to point to the real repository directory and $GIT_DIR points to an unused subdirectory inside $GIT_COMMON_DIR. File locations inside the repository are reorganized from the linked worktree view point:

 - worktree-specific such as HEAD, logs/HEAD, index, other top-level
   refs and unrecognized files are from $GIT_DIR.
 - the rest like objects, refs, info, hooks, packed-refs, shallow...
   are from $GIT_COMMON_DIR (except info/sparse-checkout, but that's
   a separate patch)

Scripts are supposed to retrieve paths in $GIT_DIR with "git rev-parse --git-path", which will take care of "$GIT_DIR vs $GIT_COMMON_DIR" business.

The redirection is done by git_path(), git_pathdup() and strbuf_git_path(). The selected list of paths goes to $GIT_COMMON_DIR, not the other way around in case a developer adds a new worktree-specific file and it's accidentally promoted to be shared across repositories (this includes unknown files added by third party commands)

The list of known files that belong to $GIT_DIR are:

ADD_EDIT.patch BISECT_ANCESTORS_OK BISECT_EXPECTED_REV BISECT_LOG BISECT_NAMES CHERRY_PICK_HEAD COMMIT_MSG FETCH_HEAD HEAD MERGE_HEAD MERGE_MODE MERGE_RR NOTES_EDITMSG NOTES_MERGE_WORKTREE ORIG_HEAD REVERT_HEAD SQUASH_MSG TAG_EDITMSG fast_import_crash_* logs/HEAD next-index-* rebase-apply rebase-merge rsync-refs-* sequencer/* shallow_*

Path mapping is NOT done for git_path_submodule(). Multi-checkouts are not supported as submodules.

Helped-by: Jens Lehmann <Jens.Lehmann@web.de>
Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
 Documentation/git.txt                  |  8 ++++++
 Documentation/gitrepository-layout.txt | 45 +++++++++++++++++++++++++---------
 cache.h                                |  4 ++-
 environment.c                          | 29 ++++++++++++++++------
 path.c                                 | 34 +++++++++++++++++++++++++
 t/t0060-path-utils.sh                  | 16 ++++++++++++
 6 files changed, 117 insertions(+), 19 deletions(-)
diff --git a/Documentation/git.txt b/Documentation/git.txt
index afb48d3..96354af 100644
--- a/Documentation/git.txt
+++ b/Documentation/git.txt
@@ -809,6 +809,14 @@ Git so take care if using Cogito etc.
 	an explicit repository directory set via 'GIT_DIR' or on the
 	command line.
 
+'GIT_COMMON_DIR'::
+	If this variable is set to a path, non-worktree files that are
+	normally in $GIT_DIR will be taken from this path
+	instead. Worktree-specific files such as HEAD or index are
+	taken from $GIT_DIR. See linkgit:gitrepository-layout[5] for
+	details. This variable has lower precedence than other path
+	variables such as GIT_INDEX_FILE, GIT_OBJECT_DIRECTORY...
+
 Git Commits
 ~~~~~~~~~~~
 'GIT_AUTHOR_NAME'::
diff --git a/Documentation/gitrepository-layout.txt b/Documentation/gitrepository-layout.txt
index 79653f3..2b5966a 100644
--- a/Documentation/gitrepository-layout.txt
+++ b/Documentation/gitrepository-layout.txt
@@ -46,6 +46,9 @@ of incomplete object store is not suitable to be published for
 use with dumb transports but otherwise is OK as long as
 `objects/info/alternates` points at the object stores it
 borrows from.
++
+This directory is ignored if $GIT_COMMON_DIR is set and
+"$GIT_COMMON_DIR/objects" will be used instead.
 
 objects/[0-9a-f][0-9a-f]::
 	A newly created object is stored in its own file.
@@ -92,7 +95,8 @@ refs::
 	References are stored in subdirectories of this
 	directory.  The 'git prune' command knows to preserve
 	objects reachable from refs found in this directory and
-	its subdirectories.
+	its subdirectories. This directory is ignored if $GIT_COMMON_DIR
+	is set and "$GIT_COMMON_DIR/refs" will be used instead.
 
 refs/heads/`name`::
 	records tip-of-the-tree commit objects of branch `name`
@@ -114,7 +118,8 @@ refs/replace/`<obj-sha1>`::
 packed-refs::
 	records the same information as refs/heads/, refs/tags/,
 	and friends record in a more efficient way.  See
-	linkgit:git-pack-refs[1].
+	linkgit:git-pack-refs[1]. This file is ignored if $GIT_COMMON_DIR
+	is set and "$GIT_COMMON_DIR/packed-refs" will be used instead.
 
 HEAD::
 	A symref (see glossary) to the `refs/heads/` namespace
@@ -133,6 +138,11 @@ being a symref to point at the current branch.  Such a state
 is often called 'detached HEAD.'  See linkgit:git-checkout[1]
 for details.
 
+config::
+	Repository specific configuration file. This file is ignored
+	if $GIT_COMMON_DIR is set and "$GIT_COMMON_DIR/config" will be
+	used instead.
+
 branches::
 	A slightly deprecated way to store shorthands to be used
 	to specify a URL to 'git fetch', 'git pull' and 'git push'.
@@ -140,7 +150,10 @@ branches::
 	'name' can be given to these commands in place of
 	'repository' argument.  See the REMOTES section in
 	linkgit:git-fetch[1] for details.  This mechanism is legacy
-	and not likely to be found in modern repositories.
+	and not likely to be found in modern repositories. This
+	directory is ignored if $GIT_COMMON_DIR is set and
+	"$GIT_COMMON_DIR/branches" will be used instead.
+
 
 hooks::
 	Hooks are customization scripts used by various Git
@@ -149,7 +162,9 @@ hooks::
 	default.  To enable, the `.sample` suffix has to be
 	removed from the filename by renaming.
 	Read linkgit:githooks[5] for more details about
-	each hook.
+	each hook. This directory is ignored if $GIT_COMMON_DIR is set
+	and "$GIT_COMMON_DIR/hooks" will be used instead.
+
 
 index::
 	The current index file for the repository.  It is
@@ -161,7 +176,8 @@ sharedindex.<SHA-1>::
 
 info::
 	Additional information about the repository is recorded
-	in this directory.
+	in this directory. This directory is ignored if $GIT_COMMON_DIR
+	is set and "$GIT_COMMON_DIR/index" will be used instead.
 
 info/refs::
 	This file helps dumb transports discover what refs are
@@ -201,12 +217,15 @@ remotes::
 	when interacting with remote repositories via 'git fetch',
 	'git pull' and 'git push' commands.  See the REMOTES section
 	in linkgit:git-fetch[1] for details.  This mechanism is legacy
-	and not likely to be found in modern repositories.
+	and not likely to be found in modern repositories. This
+	directory is ignored if $GIT_COMMON_DIR is set and
+	"$GIT_COMMON_DIR/remotes" will be used instead.
 
 logs::
-	Records of changes made to refs are stored in this
-	directory.  See linkgit:git-update-ref[1]
-	for more information.
+	Records of changes made to refs are stored in this directory.
+	See linkgit:git-update-ref[1] for more information. This
+	directory is ignored if $GIT_COMMON_DIR is set and
+	"$GIT_COMMON_DIR/logs" will be used instead.
 
 logs/refs/heads/`name`::
 	Records all changes made to the branch tip named `name`.
@@ -217,10 +236,14 @@ logs/refs/tags/`name`::
 shallow::
 	This is similar to `info/grafts` but is internally used
 	and maintained by shallow clone mechanism.  See `--depth`
-	option to linkgit:git-clone[1] and linkgit:git-fetch[1].
+	option to linkgit:git-clone[1] and linkgit:git-fetch[1]. This
+	file is ignored if $GIT_COMMON_DIR is set and
+	"$GIT_COMMON_DIR/shallow" will be used instead.
 
 modules::
-	Contains the git-repositories of the submodules.
+	Contains the git-repositories of the submodules. This
+	directory is ignored if $GIT_COMMON_DIR is set and
+	"$GIT_COMMON_DIR/modules" will be used instead.
 
 SEE ALSO
 --------
diff --git a/cache.h b/cache.h
index 7da2ffb..bce1a7d 100644
--- a/cache.h
+++ b/cache.h
@@ -377,6 +377,7 @@ static inline enum object_type object_type(unsigned int mode)
 
 /* Double-check local_repo_env below if you add to this list. */
 #define GIT_DIR_ENVIRONMENT "GIT_DIR"
+#define GIT_COMMON_DIR_ENVIRONMENT "GIT_COMMON_DIR"
 #define GIT_NAMESPACE_ENVIRONMENT "GIT_NAMESPACE"
 #define GIT_WORK_TREE_ENVIRONMENT "GIT_WORK_TREE"
 #define GIT_PREFIX_ENVIRONMENT "GIT_PREFIX"
@@ -430,6 +431,7 @@ extern int is_inside_git_dir(void);
 extern char *git_work_tree_cfg;
 extern int is_inside_work_tree(void);
 extern const char *get_git_dir(void);
+extern const char *get_git_common_dir(void);
 extern int is_git_directory(const char *path);
 extern char *get_object_directory(void);
 extern char *get_index_file(void);
@@ -617,7 +619,7 @@ extern int fsync_object_files;
 extern int core_preload_index;
 extern int core_apply_sparse_checkout;
 extern int precomposed_unicode;
-extern int git_db_env, git_index_env, git_graft_env;
+extern int git_db_env, git_index_env, git_graft_env, git_common_dir_env;
 
 /*
  * The character that begins a commented line in user-editable file
diff --git a/environment.c b/environment.c
index fee12a6..050f160 100644
--- a/environment.c
+++ b/environment.c
@@ -81,9 +81,9 @@ static char *work_tree;
 static const char *namespace;
 static size_t namespace_len;
 
-static const char *git_dir;
+static const char *git_dir, *git_common_dir;
 static char *git_object_dir, *git_index_file, *git_graft_file;
-int git_db_env, git_index_env, git_graft_env;
+int git_db_env, git_index_env, git_graft_env, git_common_dir_env;
 
 /*
  * Repository-local GIT_* environment variables; see cache.h for details.
@@ -101,6 +101,7 @@ const char * const local_repo_env[] = {
 	NO_REPLACE_OBJECTS_ENVIRONMENT,
 	GIT_PREFIX_ENVIRONMENT,
 	GIT_SHALLOW_FILE_ENVIRONMENT,
+	GIT_COMMON_DIR_ENVIRONMENT,
 	NULL
 };
 
@@ -125,8 +126,8 @@ static char *expand_namespace(const char *raw_namespace)
 	return strbuf_detach(&buf, NULL);
 }
 
-static char *git_path_from_env(const char *envvar, const char *path,
-			       int *fromenv)
+static char *git_path_from_env(const char *envvar, const char *git_dir,
+			       const char *path, int *fromenv)
 {
 	const char *value = getenv(envvar);
 	if (!value) {
@@ -149,9 +150,18 @@ static void setup_git_env(void)
 		git_dir = DEFAULT_GIT_DIR_ENVIRONMENT;
 	gitfile = read_gitfile(git_dir);
 	git_dir = xstrdup(gitfile ? gitfile : git_dir);
-	git_object_dir = git_path_from_env(DB_ENVIRONMENT, "objects", &git_db_env);
-	git_index_file = git_path_from_env(INDEX_ENVIRONMENT, "index", &git_index_env);
-	git_graft_file = git_path_from_env(GRAFT_ENVIRONMENT, "info/grafts", &git_graft_env);
+	git_common_dir = getenv(GIT_COMMON_DIR_ENVIRONMENT);
+	if (git_common_dir) {
+		git_common_dir_env = 1;
+		git_common_dir = xstrdup(git_common_dir);
+	} else
+		git_common_dir = git_dir;
+	git_object_dir = git_path_from_env(DB_ENVIRONMENT, git_common_dir,
+					   "objects", &git_db_env);
+	git_index_file = git_path_from_env(INDEX_ENVIRONMENT, git_dir,
+					   "index", &git_index_env);
+	git_graft_file = git_path_from_env(GRAFT_ENVIRONMENT, git_common_dir,
+					   "info/grafts", &git_graft_env);
 	if (getenv(NO_REPLACE_OBJECTS_ENVIRONMENT))
 		check_replace_refs = 0;
 	namespace = expand_namespace(getenv(GIT_NAMESPACE_ENVIRONMENT));
@@ -174,6 +184,11 @@ const char *get_git_dir(void)
 	return git_dir;
 }
 
+const char *get_git_common_dir(void)
+{
+	return git_common_dir;
+}
+
 const char *get_git_namespace(void)
 {
 	if (!namespace)
diff --git a/path.c b/path.c
index 4910783..94db501 100644
--- a/path.c
+++ b/path.c
@@ -90,6 +90,38 @@ static void replace_dir(struct strbuf *buf, int len, const char *newdir)
 		buf->buf[newlen] = '/';
 }
 
+static const char *common_list[] = {
+	"/branches", "/hooks", "/info", "/logs", "/lost-found", "/modules",
+	"/objects", "/refs", "/remotes", "/rr-cache", "/svn",
+	"config", "gc.pid", "packed-refs", "shallow",
+	NULL
+};
+
+static void update_common_dir(struct strbuf *buf, int git_dir_len)
+{
+	char *base = buf->buf + git_dir_len;
+	const char **p;
+
+	if (is_dir_file(base, "logs", "HEAD"))
+		return;	/* keep this in $GIT_DIR */
+	for (p = common_list; *p; p++) {
+		const char *path = *p;
+		int is_dir = 0;
+		if (*path == '/') {
+			path++;
+			is_dir = 1;
+		}
+		if (is_dir && dir_prefix(base, path)) {
+			replace_dir(buf, git_dir_len, get_git_common_dir());
+			return;
+		}
+		if (!is_dir && !strcmp(base, path)) {
+			replace_dir(buf, git_dir_len, get_git_common_dir());
+			return;
+		}
+	}
+}
+
 static void adjust_git_path(struct strbuf *buf, int git_dir_len)
 {
 	const char *base = buf->buf + git_dir_len;
@@ -101,6 +133,8 @@ static void adjust_git_path(struct strbuf *buf, int git_dir_len)
 			      get_index_file(), strlen(get_index_file()));
 	else if (git_db_env && dir_prefix(base, "objects"))
 		replace_dir(buf, git_dir_len + 7, get_object_directory());
+	else if (git_common_dir_env)
+		update_common_dir(buf, git_dir_len);
 }
 
 static void do_git_path(struct strbuf *buf, const char *fmt, va_list args)
diff --git a/t/t0060-path-utils.sh b/t/t0060-path-utils.sh
index 33d2818..f5d6f80 100755
--- a/t/t0060-path-utils.sh
+++ b/t/t0060-path-utils.sh
@@ -262,5 +262,21 @@ test_expect_success 'setup fake objects directory foo' 'mkdir foo'
 test_git_path GIT_OBJECT_DIRECTORY=foo objects foo
 test_git_path GIT_OBJECT_DIRECTORY=foo objects/foo foo/foo
 test_git_path GIT_OBJECT_DIRECTORY=foo objects2 .git/objects2
+test_expect_success 'setup common repository' 'git --git-dir=bar init'
+test_git_path GIT_COMMON_DIR=bar index                    .git/index
+test_git_path GIT_COMMON_DIR=bar HEAD                     .git/HEAD
+test_git_path GIT_COMMON_DIR=bar logs/HEAD                .git/logs/HEAD
+test_git_path GIT_COMMON_DIR=bar objects                  bar/objects
+test_git_path GIT_COMMON_DIR=bar objects/bar              bar/objects/bar
+test_git_path GIT_COMMON_DIR=bar info/exclude             bar/info/exclude
+test_git_path GIT_COMMON_DIR=bar info/grafts              bar/info/grafts
+test_git_path GIT_COMMON_DIR=bar remotes/bar              bar/remotes/bar
+test_git_path GIT_COMMON_DIR=bar branches/bar             bar/branches/bar
+test_git_path GIT_COMMON_DIR=bar logs/refs/heads/master   bar/logs/refs/heads/master
+test_git_path GIT_COMMON_DIR=bar refs/heads/master        bar/refs/heads/master
+test_git_path GIT_COMMON_DIR=bar hooks/me                 bar/hooks/me
+test_git_path GIT_COMMON_DIR=bar config                   bar/config
+test_git_path GIT_COMMON_DIR=bar packed-refs              bar/packed-refs
+test_git_path GIT_COMMON_DIR=bar shallow                  bar/shallow
 
 test_done
-- 
2.1.0.rc0.78.gc0d8480
Previous: Nguyễn Thái Ngọc DuyNext: Nguyễn Thái Ngọc Duy
Message 12 of 45 in “nd/multiple-work-trees reroll”
  1. 00/34 nd/multiple-work-trees rerollNguyễn Thái Ngọc Duy, Nov 30, 2014
  2. 01/34 path.c: make get_pathname() return strbuf instead of static bufferNguyễn Thái Ngọc Duy, Nov 30, 2014
  3. 02/34 path.c: make get_pathname() call sites return const char *Nguyễn Thái Ngọc Duy, Nov 30, 2014
  4. 03/34 git_snpath(): retire and replace with strbuf_git_path()Nguyễn Thái Ngọc Duy, Nov 30, 2014
  5. 04/34 path.c: rename vsnpath() to do_git_path()Nguyễn Thái Ngọc Duy, Nov 30, 2014
  6. 05/34 path.c: group git_path(), git_pathdup() and strbuf_git_path() togetherNguyễn Thái Ngọc Duy, Nov 30, 2014
  7. 06/34 git_path(): be aware of file relocation in $GIT_DIRNguyễn Thái Ngọc Duy, Nov 30, 2014
  8. 07/34 *.sh: respect $GIT_INDEX_FILENguyễn Thái Ngọc Duy, Nov 30, 2014
  9. 08/34 reflog: avoid constructing .lock path with git_pathNguyễn Thái Ngọc Duy, Nov 30, 2014
  10. 09/34 fast-import: use git_path() for accessing .git dir instead of get_git_dir()Nguyễn Thái Ngọc Duy, Nov 30, 2014
  11. 10/34 commit: use SEQ_DIR instead of hardcoding "sequencer"Nguyễn Thái Ngọc Duy, Nov 30, 2014
  12. 11/34 $GIT_COMMON_DIR: a new environment variableNguyễn Thái Ngọc Duy, Nov 30, 2014
  13. 12/34 git-sh-setup.sh: use rev-parse --git-path to get $GIT_DIR/objectsNguyễn Thái Ngọc Duy, Nov 30, 2014
  14. 13/34 *.sh: avoid hardcoding $GIT_DIR/hooks/...Nguyễn Thái Ngọc Duy, Nov 30, 2014
  15. 14/34 git-stash: avoid hardcoding $GIT_DIR/logs/....Nguyễn Thái Ngọc Duy, Nov 30, 2014
  16. 15/34 setup.c: convert is_git_directory() to use strbufNguyễn Thái Ngọc Duy, Nov 30, 2014
  17. 16/34 setup.c: detect $GIT_COMMON_DIR in is_git_directory()Nguyễn Thái Ngọc Duy, Nov 30, 2014
  18. 17/34 setup.c: convert check_repository_format_gently to use strbufNguyễn Thái Ngọc Duy, Nov 30, 2014
  19. 18/34 setup.c: detect $GIT_COMMON_DIR check_repository_format_gently()Nguyễn Thái Ngọc Duy, Nov 30, 2014
  20. 19/34 setup.c: support multi-checkout repo setupNguyễn Thái Ngọc Duy, Nov 30, 2014
  21. 20/34 wrapper.c: wrapper to open a file, fprintf then closeNguyễn Thái Ngọc Duy, Nov 30, 2014
  22. 21/34 use new wrapper write_file() for simple file writingNguyễn Thái Ngọc Duy, Nov 30, 2014
  23. 22/34 checkout: support checking out into a new working directoryNguyễn Thái Ngọc Duy, Nov 30, 2014
  24. 23/34 prune: strategies for linked checkoutsNguyễn Thái Ngọc Duy, Nov 30, 2014
  25. 24/34 checkout: reject if the branch is already checked out elsewhereNguyễn Thái Ngọc Duy, Nov 30, 2014
  26. Mark LevedahlNov 30, 2014
  27. Duy NguyenDec 1, 2014
  28. Junio C HamanoDec 1, 2014
  29. Mark LevedahlDec 2, 2014
  30. Duy NguyenDec 2, 2014
  31. Junio C HamanoDec 2, 2014
  32. Mark LevedahlDec 3, 2014
  33. Duy NguyenDec 3, 2014
  34. Junio C HamanoDec 3, 2014
  35. Duy NguyenDec 2, 2014
  36. 25/34 checkout: clean up half-prepared directories in --to modeNguyễn Thái Ngọc Duy, Nov 30, 2014
  37. 26/34 gc: style change -- no SP before closing parenthesisNguyễn Thái Ngọc Duy, Nov 30, 2014
  38. 27/34 gc: factor out gc.pruneexpire parsing codeNguyễn Thái Ngọc Duy, Nov 30, 2014
  39. 28/34 gc: support prune --worktreesNguyễn Thái Ngọc Duy, Nov 30, 2014
  40. 29/34 count-objects: report unused files in $GIT_DIR/worktrees/...Nguyễn Thái Ngọc Duy, Nov 30, 2014
  41. 30/34 git_path(): keep "info/sparse-checkout" per work-treeNguyễn Thái Ngọc Duy, Nov 30, 2014
  42. 31/34 checkout: don't require a work tree when checking out into a new oneNguyễn Thái Ngọc Duy, Nov 30, 2014
  43. 32/34 t2025: add a test to make sure grafts is working from a linked checkoutNguyễn Thái Ngọc Duy, Nov 30, 2014
  44. 33/34 checkout: do not fail if target is an empty directoryNguyễn Thái Ngọc Duy, Nov 30, 2014
  45. 34/34 git-common-dir: make "modules/" per-working-directory directoryNguyễn Thái Ngọc Duy, Nov 30, 2014

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.