git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH v3] Add an option not to use link(src, dest) && unlink(src) when that is unreliable

From
Johannes Schindelin <johannes.schindelin@gmx.de>
Date
Apr 27, 2009, 12:00 UTC
Message-ID
<alpine.DEB.1.00.0904271400180.10279@pacific.mpi-cbg.de>
In-Reply-To
<alpine.DEB.1.00.0904261940170.10279@pacific.mpi-cbg.de>

It seems that accessing NTFS partitions with ufsd (at least on my EeePC) has an unnerving bug: if you link() a file and unlink() it right away, the target of the link() will have the correct size, but consist of NULs.

It seems as if the calls are simply not serialized correctly, as single-stepping through the function move_temp_to_file() works flawlessly.

On Linux, this issue can be fixed by setting core.fsyncobjects to true (thanks Linus), but the same is not true on Windows.

So, force the use of rename() instead of the link() && unlink() incantation on Windows, and for good measure, add a core.unreliableHardlinks option to optionally force it on other platforms, too.

This fixes msysGit issues 222 and 229.
It was substantially improved by the help of Junio.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
[TENTATIVE] Acked-by: Johannes Sixt <j6t@kdbg.org>
---
	Hannes, is it okay to remove the [TENTATIVE]?
	Junio, do you want me to remove the config variable?
 Documentation/config.txt |    5 +++++
 Makefile                 |    8 ++++++++
 cache.h                  |    2 ++
 config.c                 |    5 +++++
 environment.c            |    4 ++++
 sha1_file.c              |    3 +++
 6 files changed, 27 insertions(+), 0 deletions(-)
diff --git a/Documentation/config.txt b/Documentation/config.txt
index 3188569..d31adb6 100644
--- a/Documentation/config.txt
+++ b/Documentation/config.txt
@@ -429,6 +429,11 @@ relatively high IO latencies.  With this set to 'true', git will do the
 index comparison to the filesystem data in parallel, allowing
 overlapping IO's.
 
+core.unreliableHardlinks::
+	Some filesystem drivers cannot properly handle hardlinking a file
+	and deleting the source right away.  In such a case, you need to
+	set this config variable to 'true'.
+
 alias.*::
 	Command aliases for the linkgit:git[1] command wrapper - e.g.
 	after defining "alias.last = cat-file commit HEAD", the invocation
diff --git a/Makefile b/Makefile
index 6f602c7..5c8e83a 100644
--- a/Makefile
+++ b/Makefile
@@ -171,6 +171,10 @@ all::
 # Define UNRELIABLE_FSTAT if your system's fstat does not return the same
 # information on a not yet closed file that lstat would return for the same
 # file after it was closed.
+#
+# Define UNRELIABLE_HARDLINKS if your operating systems has problems when
+# hardlinking a file to another name and unlinking the original file right
+# away (some NTFS drivers seem to zero the contents in that scenario).
 
 GIT-VERSION-FILE: .FORCE-GIT-VERSION-FILE
 	@$(SHELL_PATH) ./GIT-VERSION-GEN
@@ -835,6 +839,7 @@ ifneq (,$(findstring MINGW,$(uname_S)))
 	NO_NSEC = YesPlease
 	USE_WIN32_MMAP = YesPlease
 	UNRELIABLE_FSTAT = UnfortunatelyYes
+	UNRELIABLE_HARDLINKS = UnfortunatelySometimes
 	COMPAT_CFLAGS += -D__USE_MINGW_ACCESS -DNOGDI -Icompat -Icompat/regex -Icompat/fnmatch
 	COMPAT_CFLAGS += -DSNPRINTF_SIZE_CORR=1
 	COMPAT_CFLAGS += -DSTRIP_EXTENSION=\".exe\"
@@ -1018,6 +1023,9 @@ else
 		COMPAT_OBJS += compat/win32mmap.o
 	endif
 endif
+ifdef UNRELIABLE_HARDLINKS
+	COMPAT_CFLAGS += -DUNRELIABLE_HARDLINKS=1
+endif
 ifdef NO_PREAD
 	COMPAT_CFLAGS += -DNO_PREAD
 	COMPAT_OBJS += compat/pread.o
diff --git a/cache.h b/cache.h
index ab1294d..ff9e145 100644
--- a/cache.h
+++ b/cache.h
@@ -554,6 +554,8 @@ extern enum branch_track git_branch_track;
 extern enum rebase_setup_type autorebase;
 extern enum push_default_type push_default;
 
+extern int unreliable_hardlinks;
+
 #define GIT_REPO_VERSION 0
 extern int repository_format_version;
 extern int check_repository_format(void);
diff --git a/config.c b/config.c
index 8c1ae59..1750cfb 100644
--- a/config.c
+++ b/config.c
@@ -495,6 +495,11 @@ static int git_default_core_config(const char *var, const char *value)
 		return 0;
 	}
 
+	if (!strcmp(var, "core.unreliablehardlinks")) {
+		unreliable_hardlinks = git_config_bool(var, value);
+		return 0;
+	}
+
 	/* Add other config variables here and to Documentation/config.txt. */
 	return 0;
 }
diff --git a/environment.c b/environment.c
index 4696885..10578d2 100644
--- a/environment.c
+++ b/environment.c
@@ -43,6 +43,10 @@ unsigned whitespace_rule_cfg = WS_DEFAULT_RULE;
 enum branch_track git_branch_track = BRANCH_TRACK_REMOTE;
 enum rebase_setup_type autorebase = AUTOREBASE_NEVER;
 enum push_default_type push_default = PUSH_DEFAULT_UNSPECIFIED;
+#ifndef UNRELIABLE_HARDLINKS
+#define UNRELIABLE_HARDLINKS 0
+#endif
+int unreliable_hardlinks = UNRELIABLE_HARDLINKS;
 
 /* Parallel index stat data preload? */
 int core_preload_index = 0;
diff --git a/sha1_file.c b/sha1_file.c
index 8fe135d..0d289f4 100644
--- a/sha1_file.c
+++ b/sha1_file.c
@@ -2225,6 +2225,8 @@ int move_temp_to_file(const char *tmpfile, const char *filename)
 {
 	int ret = 0;
 
+	if (unreliable_hardlinks)
+		goto try_rename;
 	if (link(tmpfile, filename))
 		ret = errno;
 
@@ -2240,6 +2242,7 @@ int move_temp_to_file(const char *tmpfile, const char *filename)
 	 * left to unlink.
 	 */
 	if (ret && ret != EEXIST) {
+try_rename:
 		if (!rename(tmpfile, filename))
 			goto out;
 		ret = errno;

-- 
1.6.2.1.613.g25746
Previous: Johannes SchindelinNext: Linus Torvalds
Message 21 of 38 in “Add an option not to use link(src, dest) && unlink(src) when that is unreliable”
  1. Add an option not to use link(src, dest) && unlink(src) when that is unreliableJohannes Schindelin, Apr 23, 2009
  2. Johannes SixtApr 23, 2009
  3. Johannes SchindelinApr 23, 2009
  4. Add an option not to use link(src, dest) && unlink(src) when that is unreliableJohannes Schindelin, Apr 25, 2009
  5. Junio C HamanoApr 25, 2009
  6. Linus TorvaldsApr 25, 2009
  7. Michael GaberApr 25, 2009
  8. Linus TorvaldsApr 25, 2009
  9. Jay SoffianApr 27, 2009
  10. Johannes SixtApr 25, 2009
  11. Junio C HamanoApr 25, 2009
  12. Johannes SchindelinApr 26, 2009
  13. Linus TorvaldsApr 25, 2009
  14. Alex RiesenApr 23, 2009
  15. Johannes SchindelinApr 23, 2009
  16. Alex RiesenApr 24, 2009
  17. Linus TorvaldsApr 25, 2009
  18. Johannes SixtApr 25, 2009
  19. Junio C HamanoApr 26, 2009
  20. Johannes SchindelinApr 26, 2009
  21. Add an option not to use link(src, dest) && unlink(src) when that is unreliableJohannes Schindelin, Apr 27, 2009
  22. Linus TorvaldsApr 27, 2009
  23. Johannes SchindelinApr 27, 2009
  24. Linus TorvaldsApr 27, 2009
  25. Junio C HamanoApr 27, 2009
  26. Linus TorvaldsApr 27, 2009
  27. Linus TorvaldsApr 27, 2009
  28. Junio C HamanoApr 27, 2009
  29. Johannes SchindelinApr 27, 2009
  30. Linus TorvaldsApr 27, 2009
  31. Rename core.unreliableHardlinks to core.createObjectJohannes Schindelin, Apr 27, 2009
  32. Junio C HamanoApr 27, 2009
  33. Johannes SchindelinApr 28, 2009
  34. Junio C HamanoApr 28, 2009
  35. Johannes SchindelinApr 28, 2009
  36. Junio C HamanoApr 28, 2009
  37. Johannes SchindelinApr 28, 2009
  38. Johannes SchindelinApr 26, 2009

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.