git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH 4/4] archive: queue directories for all types of pathspecs

From
René Scharfe <l.s.r@web.de>
Date
Sep 12, 2017, 22:43 UTC
Message-ID
<837ff6fe-a645-97fe-4b21-ec36f662ce19@web.de>
In-Reply-To
<20170820090629.tumvqwzkromcykjf@sigill.intra.peff.net>
Am 20.08.2017 um 11:06 schrieb Jeff King:
Show 10 quoted lines
> I actually think it would be reasonable to omit the empty directory in
> t5004. The main thing we care about there is that we produce an archive
> with no files rather than barfing. Checking that the empty directory is
> actually there was mostly "this is what it happens to produce" rather
> than any conscious decision, I think.
> 
> If the new rule is "we omit empty directories", then it would make sense
> for us to follow that, regardless of whether it happened by pathspec
> limiting or if the tree was empty in the first place (and such an
> empty tree is insane anyway; Git tries hard not to create them).
Right.
-- >8 --
Subject: [PATCH] archive: don't add empty directories to archives

While git doesn't track empty directories, git archive can be tricked into putting some into archives. One way is to construct an empty tree object, as t5004 does. While that is supported by the object database, it can't be represented in the index and thus it's unlikely to occur in the wild.

Another way is using the literal name of a directory in an exclude pathspec -- its contents are are excluded, but the directory stub is included. That's inconsistent: exclude pathspecs containing wildcards don't leave empty directories in the archive.

Yet another way is have a few levels of nested subdirectories (e.g. d1/d2/d3/file1) and ignoring the entries at the leaved (e.g. file1). The directories with the ignored content are ignored as well (e.g. d3), but their empty parents are included (e.g. d2).

As empty directories are not supported by git, they should also not be written into archives. If an empty directory is really needed then it can be tracked and archived by placing an empty .gitignore file in it.

There already is a mechanism in place for suppressing empty directories. When read_tree_recursive() encounters a directory excluded by a pathspec then it enters it anyway because it might contain included entries. It calls the callback function before it is able to decide if the directory is actually needed. For that reason git archive adds directories to a queue and writes entries for them only when it encounters the first child item -- but currently only if pathspecs with wildcards are used.

Queue *all* directories, no matter if there even are pathspecs present. This prevents git archive from writing entries for empty directories in all cases.

Suggested-by: Jeff King <peff@peff.net>
Signed-off-by: Rene Scharfe <l.s.r@web.de>
---
 archive.c                       | 19 ++-----------------
 t/t5001-archive-attr.sh         |  2 +-
 t/t5002-archive-attr-pattern.sh |  2 +-
 t/t5004-archive-corner-cases.sh |  4 ++--
 4 files changed, 6 insertions(+), 21 deletions(-)
diff --git a/archive.c b/archive.c
index 1ab8d3a1d7..1e41f4bbeb 100644
--- a/archive.c
+++ b/archive.c
@@ -121,11 +121,6 @@ static int check_attr_export_subst(const struct attr_check *check)
 	return check && ATTR_TRUE(check->items[1].value);
 }
 
-static int should_queue_directories(const struct archiver_args *args)
-{
-	return args->pathspec.has_wildcard;
-}
-
 static int write_archive_entry(const unsigned char *sha1, const char *base,
 		int baselen, const char *filename, unsigned mode, int stage,
 		void *context)
@@ -147,7 +142,7 @@ static int write_archive_entry(const unsigned char *sha1, const char *base,
 		strbuf_addch(&path, '/');
 	path_without_prefix = path.buf + args->baselen;
 
-	if (!S_ISDIR(mode) || !should_queue_directories(args)) {
+	if (!S_ISDIR(mode)) {
 		const struct attr_check *check;
 		check = get_archive_attrs(path_without_prefix);
 		if (check_attr_export_ignore(check))
@@ -169,14 +164,6 @@ static int write_archive_entry(const unsigned char *sha1, const char *base,
 	return write_entry(args, sha1, path.buf, path.len, mode);
 }
 
-static int write_archive_entry_buf(const unsigned char *sha1, struct strbuf *base,
-		const char *filename, unsigned mode, int stage,
-		void *context)
-{
-	return write_archive_entry(sha1, base->buf, base->len,
-				     filename, mode, stage, context);
-}
-
 static void queue_directory(const unsigned char *sha1,
 		struct strbuf *base, const char *filename,
 		unsigned mode, int stage, struct archiver_context *c)
@@ -290,9 +277,7 @@ int write_archive_entries(struct archiver_args *args,
 	}
 
 	err = read_tree_recursive(args->tree, "", 0, 0, &args->pathspec,
-				  should_queue_directories(args) ?
-				  queue_or_write_archive_entry :
-				  write_archive_entry_buf,
+				  queue_or_write_archive_entry,
 				  &context);
 	if (err == READ_TREE_RECURSIVE)
 		err = 0;
diff --git a/t/t5001-archive-attr.sh b/t/t5001-archive-attr.sh
index 897f6f06d5..e9aa97117a 100755
--- a/t/t5001-archive-attr.sh
+++ b/t/t5001-archive-attr.sh
@@ -73,7 +73,7 @@ test_expect_missing	archive-pathspec/ignored-by-tree
 test_expect_missing	archive-pathspec/ignored-by-tree.d
 test_expect_missing	archive-pathspec/ignored-by-tree.d/file
 test_expect_exists	archive-pathspec/ignored-by-worktree
-test_expect_missing	archive-pathspec/excluded-by-pathspec.d failure
+test_expect_missing	archive-pathspec/excluded-by-pathspec.d
 test_expect_missing	archive-pathspec/excluded-by-pathspec.d/file
 
 test_expect_success 'git archive with wildcard pathspec' '
diff --git a/t/t5002-archive-attr-pattern.sh b/t/t5002-archive-attr-pattern.sh
index 6667d159ab..bda6d7d7e9 100755
--- a/t/t5002-archive-attr-pattern.sh
+++ b/t/t5002-archive-attr-pattern.sh
@@ -76,7 +76,7 @@ test_expect_missing	archive/deep/and/slashless/ &&
 test_expect_missing	archive/deep/and/slashless/foo &&
 test_expect_missing	archive/deep/with/wildcard/ &&
 test_expect_missing	archive/deep/with/wildcard/foo &&
-test_expect_exists	archive/one-level-lower/
+test_expect_missing	archive/one-level-lower/
 test_expect_missing	archive/one-level-lower/two-levels-lower/ignored-only-if-dir/
 test_expect_missing	archive/one-level-lower/two-levels-lower/ignored-ony-if-dir/ignored-by-ignored-dir
 
diff --git a/t/t5004-archive-corner-cases.sh b/t/t5004-archive-corner-cases.sh
index f6207f42b5..ced44355ca 100755
--- a/t/t5004-archive-corner-cases.sh
+++ b/t/t5004-archive-corner-cases.sh
@@ -108,14 +108,14 @@ test_expect_success 'archive empty subtree with no pathspec' '
 	git archive --format=tar $root_tree >subtree-all.tar &&
 	make_dir extract &&
 	"$TAR" xf subtree-all.tar -C extract &&
-	check_dir extract sub
+	check_dir extract
 '
 
 test_expect_success 'archive empty subtree by direct pathspec' '
 	git archive --format=tar $root_tree -- sub >subtree-path.tar &&
 	make_dir extract &&
 	"$TAR" xf subtree-path.tar -C extract &&
-	check_dir extract sub
+	check_dir extract
 '
 
 ZIPINFO=zipinfo

-- 
2.14.1
Previous: Jeff KingNext: Jeff King
Message 12 of 17 in “Bug?: git archive exclude pathspec and gitattributes export-ignore”
  1. David AdamAug 13, 2017
  2. René ScharfeAug 14, 2017
  3. David AdamAug 15, 2017
  4. René ScharfeAug 19, 2017
  5. 1/4 t5001: add tests for export-ignore attributes and exclude pathspecsRené Scharfe, Aug 19, 2017
  6. 2/4 archive: factor out helper functions for handling attributesRené Scharfe, Aug 19, 2017
  7. 4/4 archive: queue directories for all types of pathspecsRené Scharfe, Aug 19, 2017
  8. René ScharfeAug 19, 2017
  9. René ScharfeAug 19, 2017
  10. Junio C HamanoAug 19, 2017
  11. Jeff KingAug 20, 2017
  12. René ScharfeSep 12, 2017
  13. Jeff KingSep 13, 2017
  14. René ScharfeSep 13, 2017
  15. Stefan BellerAug 21, 2017
  16. Junio C HamanoAug 19, 2017
  17. 3/4 archive: don't queue excluded directoriesRené Scharfe, Aug 19, 2017

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.