git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH 0/3] Improving performance of git clean

From
Erik Elfström <erik.elfstrom@gmail.com>
Date
Apr 6, 2015, 11:48 UTC
Message-ID
<1428320904-12366-1-git-send-email-erik.elfstrom@gmail.com>

This series addresses a performance issue of git clean previously discussed here: http://thread.gmane.org/gmane.comp.version-control.git/265560/focus=265560 and here: http://thread.gmane.org/gmane.comp.version-control.git/266777/focus=266777

The issue manifests when trying to clean a large number of untracked directories. In my case this scenario triggered by a test suite running in the repository creating a directory for each test resulting in a build directory with ~100000 sub directories that needs to be cleaned. For some extreme cases, clean times of more than 1h have been observed.

With this series, the time to clean an untracked directory containing 100000 sub directories goes from 61s to 1.7s.

The main change is to switch the repository check in clean.c:remove_dirs from using refs.c:resolve_gitlink_ref to setup.c:is_git_directory.

One potential issue that is_git_directory contains the following check:
	if (getenv(DB_ENVIRONMENT)) {
		if (access(getenv(DB_ENVIRONMENT), X_OK))
			return 0;
	}

I'm not sure how this will affect this usecase (checking for some other nested git repo). Please give some thought to this when reviewing.

Jeff King also expressed concerns that we may have similar performance issues in other commands and that it could be good to unify these "is this a repo?"-checks. This series only attempts to solve the git-clean case.

Erik Elfström (3):
  t7300: add tests to document behavior of clean and nested git
  p7300: added performance tests for clean
  clean: improve performance when removing lots of directories
 builtin/clean.c       | 23 ++++++++++++---
 t/perf/p7300-clean.sh | 37 +++++++++++++++++++++++
 t/t7300-clean.sh      | 82 +++++++++++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 138 insertions(+), 4 deletions(-)
 create mode 100755 t/perf/p7300-clean.sh
-- 
2.4.0.rc0.37.ga3b75b3
Next: Erik Elfström
Message 1 of 14 in “Improving performance of git clean”
  1. 0/3 Improving performance of git cleanErik Elfström, Apr 6, 2015
  2. 1/3 t7300: add tests to document behavior of clean and nested gitErik Elfström, Apr 6, 2015
  3. Eric SunshineApr 6, 2015
  4. erik elfströmApr 7, 2015
  5. Eric SunshineApr 7, 2015
  6. Torsten BögershausenApr 7, 2015
  7. 2/3 p7300: added performance tests for cleanErik Elfström, Apr 6, 2015
  8. Torsten BögershausenApr 6, 2015
  9. Eric SunshineApr 6, 2015
  10. erik elfströmApr 7, 2015
  11. 3/3 clean: improve performance when removing lots of directoriesErik Elfström, Apr 6, 2015
  12. Eric SunshineApr 6, 2015
  13. erik elfströmApr 7, 2015
  14. Eric SunshineApr 8, 2015

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.