git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH] completion: improve ls-files filter performance

From
Clemens Buchacher <drizzd@gmx.net>
Date
Apr 4, 2018, 07:46 UTC
Message-ID
<20180404074658.GA5833@Sonnenschein.localdomain>
In-Reply-To
<20180318012618.32691-1-szeder.dev@gmail.com>

From the output of ls-files, we remove all but the leftmost path component and then we eliminate duplicates. We do this in a while loop, which is a performance bottleneck when the number of iterations is large (e.g. for 60000 files in linux.git).

$ COMP_WORDS=(git status -- ar) COMP_CWORD=3; time _git

real 0m11.876s user 0m4.685s sys 0m6.808s

Replacing the loop with the cut command improves performance significantly:

$ COMP_WORDS=(git status -- ar) COMP_CWORD=3; time _git

real 0m1.372s user 0m0.263s sys 0m0.167s

The measurements were done with Msys2 bash, which is used by Git for Windows.

When filtering the ls-files output we take care not to touch absolute paths. This is redundant, because ls-files will never output absolute paths. Remove the unnecessary operations.

The issue was reported here: https://github.com/git-for-windows/git/issues/1533

Signed-off-by: Clemens Buchacher <drizzd@gmx.net>
---
On Sun, Mar 18, 2018 at 02:26:18AM +0100, SZEDER Gábor wrote:
> 
> You didn't run the test suite, did you? ;)

My bad. I put the sort back in. Test t9902 is now pass. I did not run the other tests. I think the completion script is not used there.

I also considered Junio's and Johannes' comments.
> I have a short patch series collecting dust somewhere for a long
> while, [...]
> Will try to dig up those patches.
Cool. Bash completion can certainly use more performance improvements.
 contrib/completion/git-completion.bash | 7 +------
 1 file changed, 1 insertion(+), 6 deletions(-)
diff --git a/contrib/completion/git-completion.bash b/contrib/completion/git-completion.bash
index 6da95b8..69a2d41 100644
--- a/contrib/completion/git-completion.bash
+++ b/contrib/completion/git-completion.bash
@@ -384,12 +384,7 @@ __git_index_files ()
 	local root="${2-.}" file
 
 	__git_ls_files_helper "$root" "$1" |
-	while read -r file; do
-		case "$file" in
-		?*/*) echo "${file%%/*}" ;;
-		*) echo "$file" ;;
-		esac
-	done | sort | uniq
+	cut -f1 -d/ | sort | uniq
 }
 
 # Lists branches from the local repository.
-- 
2.7.4
Previous: Junio C HamanoNext: Johannes Schindelin
Message 6 of 36 in “completion: improve ls-files filter performance”
  1. 1/2 completion: improve ls-files filter performanceClemens Buchacher, Mar 17, 2018
  2. 2/2 completion: simplify ls-files filterClemens Buchacher, Mar 17, 2018
  3. Junio C HamanoMar 18, 2018
  4. SZEDER GáborMar 18, 2018
  5. Junio C HamanoMar 18, 2018
  6. completion: improve ls-files filter performanceClemens Buchacher, Apr 4, 2018
  7. Johannes SchindelinApr 4, 2018
  8. 00/11 completion: path completion improvements: speedup and quoted pathsSZEDER Gábor, Apr 16, 2018
  9. 01/11 t9902-completion: add tests demonstrating issues with quoted pathnamesSZEDER Gábor, Apr 16, 2018
  10. Junio C HamanoApr 17, 2018
  11. SZEDER GáborApr 17, 2018
  12. SZEDER GáborApr 17, 2018
  13. Junio C HamanoApr 18, 2018
  14. SZEDER GáborApr 26, 2018
  15. Junio C HamanoApr 26, 2018
  16. 0/2 Test improvements for 'sg/complete-paths'SZEDER Gábor, May 18, 2018
  17. 1/2 completion: don't return with error from __gitcomp_file_direct()SZEDER Gábor, May 18, 2018
  18. 2/2 t9902-completion: exercise __git_complete_index_file() directlySZEDER Gábor, May 18, 2018
  19. Eric SunshineMay 18, 2018
  20. Johannes SchindelinMay 21, 2018
  21. Johannes SchindelinMay 21, 2018
  22. Johannes SchindelinMay 21, 2018
  23. Johannes SchindelinApr 18, 2018
  24. SZEDER GáborApr 19, 2018
  25. 02/11 completion: move __git_complete_index_file() next to its helpersSZEDER Gábor, Apr 16, 2018
  26. 04/11 completion: support completing non-ASCII pathnamesSZEDER Gábor, Apr 16, 2018
  27. 08/11 t9902-completion: ignore COMPREPLY element order in some testsSZEDER Gábor, Apr 16, 2018
  28. 09/11 completion: remove repeated dirnames with 'awk' during path completionSZEDER Gábor, Apr 16, 2018
  29. 06/11 completion: let 'ls-files' and 'diff-index' filter matching pathsSZEDER Gábor, Apr 16, 2018
  30. 07/11 completion: use 'awk' to strip trailing path componentsSZEDER Gábor, Apr 16, 2018
  31. 05/11 completion: improve handling quoted paths on the command lineSZEDER Gábor, Apr 16, 2018
  32. 03/11 completion: simplify prefix path component handling during path completionSZEDER Gábor, Apr 16, 2018
  33. 10/11 completion: improve handling quoted paths in 'git ls-files's outputSZEDER Gábor, Apr 16, 2018
  34. 11/11 completion: fill COMPREPLY directly when completing pathsSZEDER Gábor, Apr 16, 2018
  35. Junio C HamanoMar 18, 2018
  36. Johannes SchindelinMar 19, 2018

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.