git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH v3] ls-files.c: add --dedup option

From
阿G阿德烈 via GitGitGadget <gitgitgadget@gmail.com>
Date
Jan 14, 2021, 12:22 UTC
Message-ID
<pull.832.v3.git.1610626942677.gitgitgadget@gmail.com>
In-Reply-To
<pull.832.v2.git.1610116600.gitgitgadget@gmail.com>
From: ZheNing Hu <adlternative@gmail.com>

In order to provide users a better experience when viewing information about files in the index and the working tree, the `--dedup` option will suppress some duplicate options under some conditions.

In a merge conflict, one item of "git ls-files" output may appear multiple times. For example,now the file `a.c` has a conflict,`a.c` will appear three times in the output of "git ls-files".We can use "git ls-files --dedup" to output `a.c` only one time.(unless `--stage` or `--unmerged` is used to view all the detailed information in the index)

In addition, if you use both `--delete` and `--modify` in the same time, The `--dedup` option can also suppress modified entries output.

`--dedup` option relevant descriptions in `Documentation/git-ls-files.txt`, the test script in `t/t3012-ls-files-dedup.sh` prove the correctness of the `--dedup` option.

this patch fixed: https://github.com/gitgitgadget/git/issues/198 Thanks.

Signed-off-by: ZheNing Hu <adlternative@gmail.com>
---
    builtin/ls-files.c:add git ls-file --dedup option
    
    I am reading the source code of git ls-files and learned that git ls
    -files may have duplicate entries when conflict occurs in a branch merge
    or when different options are used at the same time. Users may fell
    confuse when they see these duplicate entries.
    
    As Junio C Hamano said ,it have odd behaviour.
    
    Therefore, we can provide an additional option to git ls-files to delete
    those repeated information.
    
    This fixes https://github.com/gitgitgadget/git/issues/198
    
    Thanks!
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-832%2Fadlternative%2Fls-files-dedup-v3
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-832/adlternative/ls-files-dedup-v3
Pull-Request: https://github.com/gitgitgadget/git/pull/832
Range-diff vs v2:
 1:  0261e5d245e < -:  ----------- builtin/ls-files.c:add git ls-file --dedup option
 2:  a09a5098aa6 ! 1:  5ce52c8b7a4 builtin:ls-files.c:add git ls-file --dedup option
     @@ Metadata
      Author: ZheNing Hu <adlternative@gmail.com>
      
       ## Commit message ##
     -    builtin:ls-files.c:add git ls-file --dedup option
     +    ls-files.c: add --dedup option
      
     -    This commit standardizes the code format.
     -    For git ls-file --dedup option added
     -    relevant descriptions in Documentation/git-ls-files.txt
     -    and wrote t/t3012-ls-files-dedup.sh test script
     -    to prove the correctness of--dedup option.
     +    In order to provide users a better experience
     +    when viewing information about files in the index
     +    and the working tree, the `--dedup` option will suppress
     +    some duplicate options under some conditions.
      
     -    this patch fixed: https://github.com/gitgitgadget/git/issues/198
     +    In a merge conflict, one item of "git ls-files" output may
     +    appear multiple times. For example,now the file `a.c` has
     +    a conflict,`a.c` will appear three times in the output of
     +    "git ls-files".We can use "git ls-files --dedup" to output
     +    `a.c` only one time.(unless `--stage` or `--unmerged` is
     +    used to view all the detailed information in the index)
     +
     +    In addition, if you use both `--delete` and `--modify` in
     +    the same time, The `--dedup` option can also suppress modified
     +    entries output.
     +
     +    `--dedup` option relevant descriptions in
     +    `Documentation/git-ls-files.txt`,
     +    the test script in `t/t3012-ls-files-dedup.sh`
     +    prove the correctness of the `--dedup` option.
     +
     +    this patch fixed:
     +    https://github.com/gitgitgadget/git/issues/198
          Thanks.
      
          Signed-off-by: ZheNing Hu <adlternative@gmail.com>
     @@ Documentation/git-ls-files.txt: OPTIONS
       	See OUTPUT below for more information.
       
      +--dedup::
     -+	Suppress duplicates entries when conflicts happen or
     -+	specify -d -m at the same time.
     ++	Suppress duplicate entries when conflict happen or `--deleted`
     ++	and `--modified` are combined.
     ++
       -x <pattern>::
       --exclude=<pattern>::
       	Skip untracked files matching pattern.
      
       ## builtin/ls-files.c ##
     +@@ builtin/ls-files.c: static int line_terminator = '\n';
     + static int debug_mode;
     + static int show_eol;
     + static int recurse_submodules;
     ++static int delete_dup;
     + 
     + static const char *prefix;
     + static int max_prefix_len;
      @@ builtin/ls-files.c: static void show_files(struct repository *repo, struct dir_struct *dir)
       {
       	int i;
       	struct strbuf fullname = STRBUF_INIT;
     --	const struct cache_entry *last_stage=NULL;
      +	const struct cache_entry *last_stage = NULL;
       
       	/* For cached/deleted files we don't need to even do the readdir */
       	if (show_others || show_killed) {
      @@ builtin/ls-files.c: static void show_files(struct repository *repo, struct dir_struct *dir)
     - 	if (show_cached || show_stage) {
       		for (i = 0; i < repo->index->cache_nr; i++) {
       			const struct cache_entry *ce = repo->index->cache[i];
     --			if(show_cached && delete_dup){
     -+
     + 
      +			if (show_cached && delete_dup) {
     - 				switch (ce_stage(ce)) {
     - 				case 0:
     - 				default:
     -@@ builtin/ls-files.c: static void show_files(struct repository *repo, struct dir_struct *dir)
     - 					if (last_stage &&
     - 					!strcmp(last_stage->name, ce->name))
     - 						continue;
     --					last_stage=ce;
     ++				switch (ce_stage(ce)) {
     ++				case 0:
     ++				default:
     ++					break;
     ++				case 1:
     ++				case 2:
     ++				case 3:
     ++					if (last_stage &&
     ++					!strcmp(last_stage->name, ce->name))
     ++						continue;
      +					last_stage = ce;
     - 				}
     - 			}
     ++				}
     ++			}
       			construct_fullname(&fullname, repo, ce);
     + 
     + 			if ((dir->flags & DIR_SHOW_IGNORED) &&
      @@ builtin/ls-files.c: static void show_files(struct repository *repo, struct dir_struct *dir)
     - 			const struct cache_entry *ce = repo->index->cache[i];
       			struct stat st;
       			int err;
     --			if(delete_dup){
     -+
     + 
      +			if (delete_dup) {
     - 				switch (ce_stage(ce)) {
     - 				case 0:
     - 				default:
     -@@ builtin/ls-files.c: static void show_files(struct repository *repo, struct dir_struct *dir)
     - 					if (last_stage &&
     - 					!strcmp(last_stage->name, ce->name))
     - 						continue;
     --					last_stage=ce;
     ++				switch (ce_stage(ce)) {
     ++				case 0:
     ++				default:
     ++					break;
     ++				case 1:
     ++				case 2:
     ++				case 3:
     ++					if (last_stage &&
     ++					!strcmp(last_stage->name, ce->name))
     ++						continue;
      +					last_stage = ce;
     - 				}
     - 			}
     ++				}
     ++			}
       			construct_fullname(&fullname, repo, ce);
     + 
     + 			if ((dir->flags & DIR_SHOW_IGNORED) &&
      @@ builtin/ls-files.c: static void show_files(struct repository *repo, struct dir_struct *dir)
       			if (ce_skip_worktree(ce))
       				continue;
       			err = lstat(fullname.buf, &st);
     --			if(delete_dup && show_deleted && show_modified && err)
     +-			if (show_deleted && err)
      +			if (delete_dup && show_deleted && show_modified && err)
       				show_ce(repo, dir, ce, fullname.buf, tag_removed);
     --			else{
     --				if (show_deleted && err)/* you can't find it,so it's actually removed at all! */
     +-			if (show_modified && ie_modified(repo->index, ce, &st, 0))
     +-				show_ce(repo, dir, ce, fullname.buf, tag_modified);
      +			else {
      +				if (show_deleted && err)
     - 					show_ce(repo, dir, ce, fullname.buf, tag_removed);
     - 				if (show_modified && ie_modified(repo->index, ce, &st, 0))
     - 					show_ce(repo, dir, ce, fullname.buf, tag_modified);
     ++					show_ce(repo, dir, ce, fullname.buf, tag_removed);
     ++				if (show_modified && ie_modified(repo->index, ce, &st, 0))
     ++					show_ce(repo, dir, ce, fullname.buf, tag_modified);
     ++			}
     + 		}
     + 	}
     + 
      @@ builtin/ls-files.c: int cmd_ls_files(int argc, const char **argv, const char *cmd_prefix)
       			N_("pretend that paths removed since <tree-ish> are still present")),
       		OPT__ABBREV(&abbrev),
       		OPT_BOOL(0, "debug", &debug_mode, N_("show debugging data")),
     --		OPT_BOOL(0, "dedup", &delete_dup, N_("delete duplicate entry in index")),
      +		OPT_BOOL(0, "dedup", &delete_dup, N_("suppress duplicate entries")),
       		OPT_END()
       	};
     @@ t/t3012-ls-files-dedup.sh (new)
      +
      +. ./test-lib.sh
      +
     -+test_expect_success 'master branch setup and write expect1 expect2 and commit' '
     -+	touch a.txt &&
     -+	touch b.txt &&
     -+	touch delete.txt &&
     -+	cat <<-EOF >expect1 &&
     ++test_expect_success 'setup' '
     ++	> a.txt &&
     ++	> b.txt &&
     ++	> delete.txt &&
     ++	cat >expect1<<-\EOF &&
      +	M a.txt
      +	H b.txt
      +	H delete.txt
      +	H expect1
      +	H expect2
      +	EOF
     -+	cat <<-EOF >expect2 &&
     ++	cat >expect2<<-EOF &&
      +	C a.txt
      +	R delete.txt
      +	EOF
      +	git add a.txt b.txt delete.txt expect1 expect2 &&
     -+	git commit -m master:1
     -+'
     -+
     -+test_expect_success 'main commit again' '
     ++	git commit -m master:1 &&
      +	echo a>a.txt &&
      +	echo b>b.txt &&
     -+	echo delete>delete.txt &&
     ++	echo delete >delete.txt &&
      +	git add a.txt b.txt delete.txt &&
     -+	git commit -m master:2
     -+'
     -+
     -+test_expect_success 'dev commit' '
     ++	git commit -m master:2 &&
      +	git checkout HEAD~ &&
      +	git switch -c dev &&
     -+	echo change>a.txt &&
     ++	echo change >a.txt &&
      +	git add a.txt &&
     -+	git commit -m dev:1
     -+'
     -+
     -+test_expect_success 'dev merge master' '
     ++	git commit -m dev:1 &&
      +	test_must_fail git merge master &&
      +	git ls-files -t --dedup >actual1 &&
      +	test_cmp expect1 actual1 &&
 Documentation/git-ls-files.txt |  5 ++++
 builtin/ls-files.c             | 41 ++++++++++++++++++++++++--
 t/t3012-ls-files-dedup.sh      | 54 ++++++++++++++++++++++++++++++++++
 3 files changed, 97 insertions(+), 3 deletions(-)
 create mode 100755 t/t3012-ls-files-dedup.sh
diff --git a/Documentation/git-ls-files.txt b/Documentation/git-ls-files.txt
index cbcf5263dd0..0f8dbeeea20 100644
--- a/Documentation/git-ls-files.txt
+++ b/Documentation/git-ls-files.txt
@@ -13,6 +13,7 @@ SYNOPSIS
 		(--[cached|deleted|others|ignored|stage|unmerged|killed|modified])*
 		(-[c|d|o|i|s|u|k|m])*
 		[--eol]
+		[--dedup]
 		[-x <pattern>|--exclude=<pattern>]
 		[-X <file>|--exclude-from=<file>]
 		[--exclude-per-directory=<file>]
@@ -81,6 +82,10 @@ OPTIONS
 	\0 line termination on output and do not quote filenames.
 	See OUTPUT below for more information.
 
+--dedup::
+	Suppress duplicate entries when conflict happen or `--deleted`
+	and `--modified` are combined.
+
 -x <pattern>::
 --exclude=<pattern>::
 	Skip untracked files matching pattern.
diff --git a/builtin/ls-files.c b/builtin/ls-files.c
index c8eae899b82..bc4eded19ab 100644
--- a/builtin/ls-files.c
+++ b/builtin/ls-files.c
@@ -35,6 +35,7 @@ static int line_terminator = '\n';
 static int debug_mode;
 static int show_eol;
 static int recurse_submodules;
+static int delete_dup;
 
 static const char *prefix;
 static int max_prefix_len;
@@ -301,6 +302,7 @@ static void show_files(struct repository *repo, struct dir_struct *dir)
 {
 	int i;
 	struct strbuf fullname = STRBUF_INIT;
+	const struct cache_entry *last_stage = NULL;
 
 	/* For cached/deleted files we don't need to even do the readdir */
 	if (show_others || show_killed) {
@@ -316,6 +318,20 @@ static void show_files(struct repository *repo, struct dir_struct *dir)
 		for (i = 0; i < repo->index->cache_nr; i++) {
 			const struct cache_entry *ce = repo->index->cache[i];
 
+			if (show_cached && delete_dup) {
+				switch (ce_stage(ce)) {
+				case 0:
+				default:
+					break;
+				case 1:
+				case 2:
+				case 3:
+					if (last_stage &&
+					!strcmp(last_stage->name, ce->name))
+						continue;
+					last_stage = ce;
+				}
+			}
 			construct_fullname(&fullname, repo, ce);
 
 			if ((dir->flags & DIR_SHOW_IGNORED) &&
@@ -337,6 +353,20 @@ static void show_files(struct repository *repo, struct dir_struct *dir)
 			struct stat st;
 			int err;
 
+			if (delete_dup) {
+				switch (ce_stage(ce)) {
+				case 0:
+				default:
+					break;
+				case 1:
+				case 2:
+				case 3:
+					if (last_stage &&
+					!strcmp(last_stage->name, ce->name))
+						continue;
+					last_stage = ce;
+				}
+			}
 			construct_fullname(&fullname, repo, ce);
 
 			if ((dir->flags & DIR_SHOW_IGNORED) &&
@@ -347,10 +377,14 @@ static void show_files(struct repository *repo, struct dir_struct *dir)
 			if (ce_skip_worktree(ce))
 				continue;
 			err = lstat(fullname.buf, &st);
-			if (show_deleted && err)
+			if (delete_dup && show_deleted && show_modified && err)
 				show_ce(repo, dir, ce, fullname.buf, tag_removed);
-			if (show_modified && ie_modified(repo->index, ce, &st, 0))
-				show_ce(repo, dir, ce, fullname.buf, tag_modified);
+			else {
+				if (show_deleted && err)
+					show_ce(repo, dir, ce, fullname.buf, tag_removed);
+				if (show_modified && ie_modified(repo->index, ce, &st, 0))
+					show_ce(repo, dir, ce, fullname.buf, tag_modified);
+			}
 		}
 	}
 
@@ -578,6 +612,7 @@ int cmd_ls_files(int argc, const char **argv, const char *cmd_prefix)
 			N_("pretend that paths removed since <tree-ish> are still present")),
 		OPT__ABBREV(&abbrev),
 		OPT_BOOL(0, "debug", &debug_mode, N_("show debugging data")),
+		OPT_BOOL(0, "dedup", &delete_dup, N_("suppress duplicate entries")),
 		OPT_END()
 	};
 
diff --git a/t/t3012-ls-files-dedup.sh b/t/t3012-ls-files-dedup.sh
new file mode 100755
index 00000000000..aec7d364235
--- /dev/null
+++ b/t/t3012-ls-files-dedup.sh
@@ -0,0 +1,54 @@
+#!/bin/sh
+
+test_description='git ls-files --dedup test.
+
+This test prepares the following in the cache:
+
+    a.txt       - a file(base)
+    a.txt	- a file(master)
+    a.txt       - a file(dev)
+    b.txt       - a file
+    delete.txt  - a file
+    expect1	- a file
+    expect2	- a file
+
+'
+
+. ./test-lib.sh
+
+test_expect_success 'setup' '
+	> a.txt &&
+	> b.txt &&
+	> delete.txt &&
+	cat >expect1<<-\EOF &&
+	M a.txt
+	H b.txt
+	H delete.txt
+	H expect1
+	H expect2
+	EOF
+	cat >expect2<<-EOF &&
+	C a.txt
+	R delete.txt
+	EOF
+	git add a.txt b.txt delete.txt expect1 expect2 &&
+	git commit -m master:1 &&
+	echo a>a.txt &&
+	echo b>b.txt &&
+	echo delete >delete.txt &&
+	git add a.txt b.txt delete.txt &&
+	git commit -m master:2 &&
+	git checkout HEAD~ &&
+	git switch -c dev &&
+	echo change >a.txt &&
+	git add a.txt &&
+	git commit -m dev:1 &&
+	test_must_fail git merge master &&
+	git ls-files -t --dedup >actual1 &&
+	test_cmp expect1 actual1 &&
+	rm delete.txt &&
+	git ls-files -d -m -t --dedup >actual2 &&
+	test_cmp expect2 actual2
+'
+
+test_done

base-commit: 6d3ef5b467eccd2769f1aa1c555d317d3c8dc707

-- 
gitgitgadget
Previous: 胡哲宁Next: Junio C Hamano
Message 9 of 65 in “builtin/ls-files.c:add git ls-file --dedup option”
  1. builtin/ls-files.c:add git ls-file --dedup option阿德烈 via GitGitGadget, Jan 6, 2021
  2. Eric SunshineJan 7, 2021
  3. Junio C HamanoJan 7, 2021
  4. 0/2 builtin/ls-files.c:add git ls-file --dedup option阿德烈 via GitGitGadget, Jan 8, 2021
  5. 1/2 builtin/ls-files.c:add git ls-file --dedup optionZheNing Hu via GitGitGadget, Jan 8, 2021
  6. 2/2 builtin:ls-files.c:add git ls-file --dedup optionZheNing Hu via GitGitGadget, Jan 8, 2021
  7. Eric SunshineJan 14, 2021
  8. 胡哲宁Jan 14, 2021
  9. ls-files.c: add --dedup option阿德烈 via GitGitGadget, Jan 14, 2021
  10. Junio C HamanoJan 15, 2021
  11. 胡哲宁Jan 17, 2021
  12. Junio C HamanoJan 17, 2021
  13. Eric SunshineJan 16, 2021
  14. 胡哲宁Jan 17, 2021
  15. Eric SunshineJan 17, 2021
  16. Junio C HamanoJan 17, 2021
  17. Eric SunshineJan 18, 2021
  18. 0/3 builtin/ls-files.c:add git ls-file --dedup option阿德烈 via GitGitGadget, Jan 17, 2021
  19. 1/3 ls_files.c: bugfix for --deleted and --modifiedZheNing Hu via GitGitGadget, Jan 17, 2021
  20. Junio C HamanoJan 17, 2021
  21. 2/3 ls_files.c: consolidate two for loops into oneZheNing Hu via GitGitGadget, Jan 17, 2021
  22. 3/3 ls-files: add --deduplicate optionZheNing Hu via GitGitGadget, Jan 17, 2021
  23. Junio C HamanoJan 17, 2021
  24. Junio C HamanoJan 17, 2021
  25. 胡哲宁Jan 18, 2021
  26. 胡哲宁Jan 18, 2021
  27. Junio C HamanoJan 18, 2021
  28. 胡哲宁Jan 19, 2021
  29. 0/3 builtin/ls-files.c:add git ls-file --dedup option阿德烈 via GitGitGadget, Jan 19, 2021
  30. 3/3 ls-files.c: add --deduplicate optionZheNing Hu via GitGitGadget, Jan 19, 2021
  31. Junio C HamanoJan 20, 2021
  32. 胡哲宁Jan 21, 2021
  33. Junio C HamanoJan 21, 2021
  34. 胡哲宁Jan 22, 2021
  35. Johannes SchindelinJan 22, 2021
  36. Junio C HamanoJan 22, 2021
  37. GitGitGadget and `next`, was Re: [PATCH v5 3/3] ls-files.c: add --deduplicate optionJohannes Schindelin, Mar 19, 2021
  38. Junio C HamanoMar 19, 2021
  39. 胡哲宁Jan 23, 2021
  40. ls-files.c: add --deduplicate optionZheNing Hu, Jan 22, 2021
  41. Junio C HamanoJan 22, 2021
  42. 胡哲宁Jan 23, 2021
  43. 2/3 ls_files.c: consolidate two for loops into oneZheNing Hu via GitGitGadget, Jan 19, 2021
  44. Junio C HamanoJan 20, 2021
  45. 胡哲宁Jan 21, 2021
  46. 1/3 ls_files.c: bugfix for --deleted and --modifiedZheNing Hu via GitGitGadget, Jan 19, 2021
  47. Junio C HamanoJan 20, 2021
  48. 胡哲宁Jan 21, 2021
  49. 0/3 builtin/ls-files.c:add git ls-file --dedup option阿德烈 via GitGitGadget, Jan 23, 2021
  50. 1/3 ls_files.c: bugfix for --deleted and --modifiedZheNing Hu via GitGitGadget, Jan 23, 2021
  51. Junio C HamanoJan 23, 2021
  52. 2/3 ls_files.c: consolidate two for loops into oneZheNing Hu via GitGitGadget, Jan 23, 2021
  53. Junio C HamanoJan 23, 2021
  54. 3/3 ls-files.c: add --deduplicate optionZheNing Hu via GitGitGadget, Jan 23, 2021
  55. Junio C HamanoJan 23, 2021
  56. 1/3 ls_files.c: bugfix for --deleted and --modifiedJunio C Hamano, Jan 23, 2021
  57. 2/3 ls_files.c: consolidate two for loops into oneJunio C Hamano, Jan 23, 2021
  58. 3/3 ls-files.c: add --deduplicate optionJunio C Hamano, Jan 23, 2021
  59. 0/3 builtin/ls-files.c:add git ls-file --dedup option阿德烈 via GitGitGadget, Jan 24, 2021
  60. 1/3 ls_files.c: bugfix for --deleted and --modifiedZheNing Hu via GitGitGadget, Jan 24, 2021
  61. Junio C HamanoJan 24, 2021
  62. 胡哲宁Jan 25, 2021
  63. Junio C HamanoJan 25, 2021
  64. 2/3 ls_files.c: consolidate two for loops into oneZheNing Hu via GitGitGadget, Jan 24, 2021
  65. 3/3 ls-files.c: add --deduplicate optionZheNing Hu via GitGitGadget, Jan 24, 2021

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.