git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH v6 27/32] prune: strategies for linked checkouts

From
Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Date
Jul 9, 2014, 07:33 UTC
Message-ID
<1404891197-18067-28-git-send-email-pclouds@gmail.com>
In-Reply-To
<1404891197-18067-1-git-send-email-pclouds@gmail.com>
(alias R=$GIT_COMMON_DIR/repos/<id>)
 - linked checkouts are supposed to keep its location in $R/gitdir up
   to date. The use case is auto fixup after a manual checkout move.
 - linked checkouts are supposed to update mtime of $R/gitdir. If
   $R/gitdir's mtime is older than a limit, and it points to nowhere,
   repos/<id> is to be pruned.
 - If $R/locked exists, repos/<id> is not supposed to be pruned. If
   $R/locked exists and $R/gitdir's mtime is older than a really long
   limit, warn about old unused repo.
 - "git checkout --to" is supposed to make a hard link named $R/link
   pointing to the .git file on supported file systems to help detect
   the user manually deleting the checkout. If $R/link exists and its
   link count is greated than 1, the repo is kept.
Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
 Documentation/git-prune.txt            |  3 ++
 Documentation/gitrepository-layout.txt | 19 +++++++++
 builtin/checkout.c                     | 14 +++++++
 builtin/prune.c                        | 74 ++++++++++++++++++++++++++++++++++
 setup.c                                | 13 ++++++
 5 files changed, 123 insertions(+)
diff --git a/Documentation/git-prune.txt b/Documentation/git-prune.txt
index 7a493c8..50e39ec 100644
--- a/Documentation/git-prune.txt
+++ b/Documentation/git-prune.txt
@@ -48,6 +48,9 @@ OPTIONS
 --expire <time>::
 	Only expire loose objects older than <time>.
 
+--repos::
+	Prune directories in $GIT_DIR/repos.
+
 <head>...::
 	In addition to objects
 	reachable from any of our references, keep objects
diff --git a/Documentation/gitrepository-layout.txt b/Documentation/gitrepository-layout.txt
index 543d874..bed4f1a 100644
--- a/Documentation/gitrepository-layout.txt
+++ b/Documentation/gitrepository-layout.txt
@@ -256,6 +256,25 @@ repos::
 	$GIT_COMMON_DIR is set and "$GIT_COMMON_DIR/repos" will be
 	used instead.
 
+repos/<id>/gitdir::
+	A text file containing the absolute path back to the .git file
+	that points to here. This is used to check if the linked
+	repository has been manually removed and there is no need to
+	keep this directory any more. mtime of this file should be
+	updated every time the linked repository is accessed.
+
+repos/<id>/locked::
+	If this file exists, the linked repository may be on a
+	portable device and not available. It does not mean that the
+	linked repository is gone and `repos/<id>` could be
+	removed. The file's content contains a reason string on why
+	the repository is locked.
+
+repos/<id>/link::
+	If this file exists, it is a hard link to the linked .git
+	file. It is used to detect if the linked repository is
+	manually removed.
+
 SEE ALSO
 --------
 linkgit:git-init[1],
diff --git a/builtin/checkout.c b/builtin/checkout.c
index 069e803..98a2f5f 100644
--- a/builtin/checkout.c
+++ b/builtin/checkout.c
@@ -898,12 +898,22 @@ static int prepare_linked_checkout(const struct checkout_opts *opts,
 	junk_git_dir = sb_repo.buf;
 	is_junk = 1;
 
+	/*
+	 * lock the incomplete repo so prune won't delete it, unlock
+	 * after the preparation is over.
+	 */
+	strbuf_addf(&sb, "%s/locked", sb_repo.buf);
+	write_file(sb.buf, 1, "initializing\n");
+
 	strbuf_addf(&sb_git, "%s/.git", path);
 	if (safe_create_leading_directories_const(sb_git.buf))
 		die_errno(_("could not create leading directories of '%s'"),
 			  sb_git.buf);
 	junk_work_tree = path;
 
+	strbuf_reset(&sb);
+	strbuf_addf(&sb, "%s/gitdir", sb_repo.buf);
+	write_file(sb.buf, 1, "%s\n", real_path(sb_git.buf));
 	write_file(sb_git.buf, 1, "gitdir: %s/repos/%s\n",
 		   real_path(get_git_common_dir()), name);
 	/*
@@ -912,6 +922,7 @@ static int prepare_linked_checkout(const struct checkout_opts *opts,
 	 * value would do because this value will be ignored and
 	 * replaced at the next (real) checkout.
 	 */
+	strbuf_reset(&sb);
 	strbuf_addf(&sb, "%s/HEAD", sb_repo.buf);
 	write_file(sb.buf, 1, "%s\n", sha1_to_hex(new->commit->object.sha1));
 	strbuf_reset(&sb);
@@ -930,6 +941,9 @@ static int prepare_linked_checkout(const struct checkout_opts *opts,
 	ret = run_command(&cp);
 	if (!ret)
 		is_junk = 0;
+	strbuf_reset(&sb);
+	strbuf_addf(&sb, "%s/locked", sb_repo.buf);
+	unlink_or_warn(sb.buf);
 	strbuf_release(&sb);
 	strbuf_release(&sb_repo);
 	strbuf_release(&sb_git);
diff --git a/builtin/prune.c b/builtin/prune.c
index 144a3bd..6db6bcc 100644
--- a/builtin/prune.c
+++ b/builtin/prune.c
@@ -112,6 +112,70 @@ static void prune_object_dir(const char *path)
 	}
 }
 
+static const char *prune_repo_dir(const char *id, struct stat *st)
+{
+	char *path;
+	int fd, len;
+	if (file_exists(git_path("repos/%s/locked", id)))
+		return NULL;
+	if (stat(git_path("repos/%s/gitdir", id), st)) {
+		st->st_mtime = expire;
+		return _("gitdir does not exist");
+	}
+	fd = open(git_path("repos/%s/gitdir", id), O_RDONLY);
+	len = st->st_size;
+	path = xmalloc(len + 1);
+	read_in_full(fd, path, len);
+	close(fd);
+	while (path[len - 1] == '\n' || path[len - 1] == '\r')
+		len--;
+	path[len] = '\0';
+	if (!file_exists(path)) {
+		struct stat st_link;
+		free(path);
+		/*
+		 * the repo is moved manually and has not been
+		 * accessed since?
+		 */
+		if (!stat(git_path("repos/%s/link", id), &st_link) &&
+		    st_link.st_nlink > 1)
+			return NULL;
+		return _("gitdir points to non-existing file");
+	}
+	free(path);
+	return NULL;
+}
+
+static void prune_repos_dir(void)
+{
+	const char *reason;
+	DIR *dir = opendir(git_path("repos"));
+	struct dirent *d;
+	int removed = 0;
+	struct stat st;
+	if (!dir)
+		return;
+	while ((d = readdir(dir)) != NULL) {
+		if (!strcmp(d->d_name, ".") || !strcmp(d->d_name, ".."))
+			continue;
+		if ((reason = prune_repo_dir(d->d_name, &st)) != NULL &&
+		    st.st_mtime <= expire) {
+			struct strbuf sb = STRBUF_INIT;
+			if (show_only || verbose)
+				printf(_("Removing repos/%s: %s\n"), d->d_name, reason);
+			if (show_only)
+				continue;
+			strbuf_addstr(&sb, git_path("repos/%s", d->d_name));
+			remove_dir_recursively(&sb, 0);
+			strbuf_release(&sb);
+			removed = 1;
+		}
+	}
+	closedir(dir);
+	if (removed)
+		rmdir(git_path("repos"));
+}
+
 /*
  * Write errors (particularly out of space) can result in
  * failed temporary packs (and more rarely indexes and other
@@ -138,10 +202,12 @@ int cmd_prune(int argc, const char **argv, const char *prefix)
 {
 	struct rev_info revs;
 	struct progress *progress = NULL;
+	int prune_repos = 0;
 	const struct option options[] = {
 		OPT__DRY_RUN(&show_only, N_("do not remove, show only")),
 		OPT__VERBOSE(&verbose, N_("report pruned objects")),
 		OPT_BOOL(0, "progress", &show_progress, N_("show progress")),
+		OPT_BOOL(0, "repos", &prune_repos, N_("prune .git/repos/")),
 		OPT_EXPIRY_DATE(0, "expire", &expire,
 				N_("expire objects older than <time>")),
 		OPT_END()
@@ -154,6 +220,14 @@ int cmd_prune(int argc, const char **argv, const char *prefix)
 	init_revisions(&revs, prefix);
 
 	argc = parse_options(argc, argv, prefix, options, prune_usage, 0);
+
+	if (prune_repos) {
+		if (argc)
+			die(_("--repos does not take extra arguments"));
+		prune_repos_dir();
+		return 0;
+	}
+
 	while (argc--) {
 		unsigned char sha1[20];
 		const char *name = *argv++;
diff --git a/setup.c b/setup.c
index 8f90bc3..da2d669 100644
--- a/setup.c
+++ b/setup.c
@@ -390,6 +390,17 @@ static int check_repository_format_gently(const char *gitdir, int *nongit_ok)
 	return ret;
 }
 
+static void update_linked_gitdir(const char *gitfile, const char *gitdir)
+{
+	struct strbuf path = STRBUF_INIT;
+	struct stat st;
+
+	strbuf_addf(&path, "%s/gitfile", gitdir);
+	if (stat(path.buf, &st) || st.st_mtime + 24 * 3600 < time(NULL))
+		write_file(path.buf, 0, "%s\n", gitfile);
+	strbuf_release(&path);
+}
+
 /*
  * Try to read the location of the git directory from the .git file,
  * return path to git directory if found.
@@ -438,6 +449,8 @@ const char *read_gitfile(const char *path)
 
 	if (!is_git_directory(dir))
 		die("Not a git repository: %s", dir);
+
+	update_linked_gitdir(path, dir);
 	path = real_path(dir);
 
 	free(buf);
-- 
1.9.1.346.ga2b5940
Previous: Max KirillovNext: Eric Sunshine
Message 29 of 83 in “Support multiple checkouts”
  1. 00/32 Support multiple checkoutsNguyễn Thái Ngọc Duy, Jul 9, 2014
  2. 01/32 path.c: make get_pathname() return strbuf instead of static bufferNguyễn Thái Ngọc Duy, Jul 9, 2014
  3. 02/32 path.c: make get_pathname() call sites return const char *Nguyễn Thái Ngọc Duy, Jul 9, 2014
  4. 03/32 git_snpath(): retire and replace with strbuf_git_path()Nguyễn Thái Ngọc Duy, Jul 9, 2014
  5. 04/32 path.c: rename vsnpath() to do_git_path()Nguyễn Thái Ngọc Duy, Jul 9, 2014
  6. 05/32 path.c: group git_path(), git_pathdup() and strbuf_git_path() togetherNguyễn Thái Ngọc Duy, Jul 9, 2014
  7. 06/32 setup_git_env: use git_pathdup instead of xmalloc + sprintfNguyễn Thái Ngọc Duy, Jul 9, 2014
  8. 07/32 setup_git_env(): introduce git_path_from_env() helperNguyễn Thái Ngọc Duy, Jul 9, 2014
  9. 08/32 git_path(): be aware of file relocation in $GIT_DIRNguyễn Thái Ngọc Duy, Jul 9, 2014
  10. 09/32 *.sh: respect $GIT_INDEX_FILENguyễn Thái Ngọc Duy, Jul 9, 2014
  11. 10/32 reflog: avoid constructing .lock path with git_pathNguyễn Thái Ngọc Duy, Jul 9, 2014
  12. 11/32 fast-import: use git_path() for accessing .git dir instead of get_git_dir()Nguyễn Thái Ngọc Duy, Jul 9, 2014
  13. 12/32 commit: use SEQ_DIR instead of hardcoding "sequencer"Nguyễn Thái Ngọc Duy, Jul 9, 2014
  14. 13/32 $GIT_COMMON_DIR: a new environment variableNguyễn Thái Ngọc Duy, Jul 9, 2014
  15. 14/32 git-sh-setup.sh: use rev-parse --git-path to get $GIT_DIR/objectsNguyễn Thái Ngọc Duy, Jul 9, 2014
  16. 15/32 *.sh: avoid hardcoding $GIT_DIR/hooks/...Nguyễn Thái Ngọc Duy, Jul 9, 2014
  17. 16/32 git-stash: avoid hardcoding $GIT_DIR/logs/....Nguyễn Thái Ngọc Duy, Jul 9, 2014
  18. 17/32 setup.c: convert is_git_directory() to use strbufNguyễn Thái Ngọc Duy, Jul 9, 2014
  19. 18/32 setup.c: detect $GIT_COMMON_DIR in is_git_directory()Nguyễn Thái Ngọc Duy, Jul 9, 2014
  20. 19/32 setup.c: convert check_repository_format_gently to use strbufNguyễn Thái Ngọc Duy, Jul 9, 2014
  21. 20/32 setup.c: detect $GIT_COMMON_DIR check_repository_format_gently()Nguyễn Thái Ngọc Duy, Jul 9, 2014
  22. 21/32 setup.c: support multi-checkout repo setupNguyễn Thái Ngọc Duy, Jul 9, 2014
  23. 22/32 wrapper.c: wrapper to open a file, fprintf then closeNguyễn Thái Ngọc Duy, Jul 9, 2014
  24. 23/32 use new wrapper write_file() for simple file writingNguyễn Thái Ngọc Duy, Jul 9, 2014
  25. 24/32 checkout: support checking out into a new working directoryNguyễn Thái Ngọc Duy, Jul 9, 2014
  26. 25/32 checkout: clean up half-prepared directories in --to modeNguyễn Thái Ngọc Duy, Jul 9, 2014
  27. 26/32 checkout: detach if the branch is already checked out elsewhereNguyễn Thái Ngọc Duy, Jul 9, 2014
  28. Max KirillovJul 12, 2014
  29. 27/32 prune: strategies for linked checkoutsNguyễn Thái Ngọc Duy, Jul 9, 2014
  30. Eric SunshineJul 9, 2014
  31. 28/32 gc: style change -- no SP before closing bracketNguyễn Thái Ngọc Duy, Jul 9, 2014
  32. Eric SunshineJul 9, 2014
  33. Junio C HamanoJul 14, 2014
  34. 29/32 gc: support prune --reposNguyễn Thái Ngọc Duy, Jul 9, 2014
  35. Eric SunshineJul 9, 2014
  36. 30/32 count-objects: report unused files in $GIT_DIR/repos/...Nguyễn Thái Ngọc Duy, Jul 9, 2014
  37. 31/32 git_path(): keep "info/sparse-checkout" per work-treeNguyễn Thái Ngọc Duy, Jul 9, 2014
  38. 32/32 checkout: don't require a work tree when checking out into a new oneNguyễn Thái Ngọc Duy, Jul 9, 2014
  39. Dennis KaarsemakerJul 11, 2014
  40. 00/31 Support multiple checkoutsNguyễn Thái Ngọc Duy, Jul 13, 2014
  41. 01/31 path.c: make get_pathname() return strbuf instead of static bufferNguyễn Thái Ngọc Duy, Jul 13, 2014
  42. 02/31 path.c: make get_pathname() call sites return const char *Nguyễn Thái Ngọc Duy, Jul 13, 2014
  43. 03/31 git_snpath(): retire and replace with strbuf_git_path()Nguyễn Thái Ngọc Duy, Jul 13, 2014
  44. 04/31 path.c: rename vsnpath() to do_git_path()Nguyễn Thái Ngọc Duy, Jul 13, 2014
  45. 05/31 path.c: group git_path(), git_pathdup() and strbuf_git_path() togetherNguyễn Thái Ngọc Duy, Jul 13, 2014
  46. 06/31 git_path(): be aware of file relocation in $GIT_DIRNguyễn Thái Ngọc Duy, Jul 13, 2014
  47. 07/31 *.sh: respect $GIT_INDEX_FILENguyễn Thái Ngọc Duy, Jul 13, 2014
  48. 08/31 reflog: avoid constructing .lock path with git_pathNguyễn Thái Ngọc Duy, Jul 13, 2014
  49. 09/31 fast-import: use git_path() for accessing .git dir instead of get_git_dir()Nguyễn Thái Ngọc Duy, Jul 13, 2014
  50. 10/31 commit: use SEQ_DIR instead of hardcoding "sequencer"Nguyễn Thái Ngọc Duy, Jul 13, 2014
  51. 11/31 $GIT_COMMON_DIR: a new environment variableNguyễn Thái Ngọc Duy, Jul 13, 2014
  52. Eric SunshineJul 23, 2014
  53. 12/31 git-sh-setup.sh: use rev-parse --git-path to get $GIT_DIR/objectsNguyễn Thái Ngọc Duy, Jul 13, 2014
  54. 13/31 *.sh: avoid hardcoding $GIT_DIR/hooks/...Nguyễn Thái Ngọc Duy, Jul 13, 2014
  55. 14/31 git-stash: avoid hardcoding $GIT_DIR/logs/....Nguyễn Thái Ngọc Duy, Jul 13, 2014
  56. 15/31 setup.c: convert is_git_directory() to use strbufNguyễn Thái Ngọc Duy, Jul 13, 2014
  57. 16/31 setup.c: detect $GIT_COMMON_DIR in is_git_directory()Nguyễn Thái Ngọc Duy, Jul 13, 2014
  58. 17/31 setup.c: convert check_repository_format_gently to use strbufNguyễn Thái Ngọc Duy, Jul 13, 2014
  59. 18/31 setup.c: detect $GIT_COMMON_DIR check_repository_format_gently()Nguyễn Thái Ngọc Duy, Jul 13, 2014
  60. 19/31 setup.c: support multi-checkout repo setupNguyễn Thái Ngọc Duy, Jul 13, 2014
  61. 20/31 wrapper.c: wrapper to open a file, fprintf then closeNguyễn Thái Ngọc Duy, Jul 13, 2014
  62. 21/31 use new wrapper write_file() for simple file writingNguyễn Thái Ngọc Duy, Jul 13, 2014
  63. 22/31 checkout: support checking out into a new working directoryNguyễn Thái Ngọc Duy, Jul 13, 2014
  64. Max KirillovJul 17, 2014
  65. Junio C HamanoJul 17, 2014
  66. Eric SunshineJul 18, 2014
  67. 23/31 checkout: clean up half-prepared directories in --to modeNguyễn Thái Ngọc Duy, Jul 13, 2014
  68. Eric SunshineJul 20, 2014
  69. Eric SunshineJul 21, 2014
  70. Duy NguyenJul 23, 2014
  71. 24/31 checkout: detach if the branch is already checked out elsewhereNguyễn Thái Ngọc Duy, Jul 13, 2014
  72. 25/31 prune: strategies for linked checkoutsNguyễn Thái Ngọc Duy, Jul 13, 2014
  73. Thomas RastJul 18, 2014
  74. Duy NguyenJul 19, 2014
  75. 26/31 gc: style change -- no SP before closing bracketNguyễn Thái Ngọc Duy, Jul 13, 2014
  76. 27/31 gc: factor out gc.pruneexpire parsing codeNguyễn Thái Ngọc Duy, Jul 13, 2014
  77. 28/31 gc: support prune --reposNguyễn Thái Ngọc Duy, Jul 13, 2014
  78. 29/31 count-objects: report unused files in $GIT_DIR/repos/...Nguyễn Thái Ngọc Duy, Jul 13, 2014
  79. 30/31 git_path(): keep "info/sparse-checkout" per work-treeNguyễn Thái Ngọc Duy, Jul 13, 2014
  80. 31/31 checkout: don't require a work tree when checking out into a new oneNguyễn Thái Ngọc Duy, Jul 13, 2014
  81. Junio C HamanoJul 14, 2014
  82. Duy NguyenJul 14, 2014
  83. Junio C HamanoJul 14, 2014

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.