{"thread":{"id":"28657","subject":"[PATCH] Makefile: add a knob to turn off hardlinks within same directory","startedAt":"2011-10-12T08:38:42Z","lastAt":"2011-10-12T19:57:05Z","messageCount":3,"participants":["Jonathan Nieder","Junio C Hamano"],"isPatch":true,"patchVersion":1,"patchTotal":null},"messages":[{"id":"177432","messageId":"20111012083842.GA21969@elie.hsd1.il.comcast.net","threadId":"28657","inReplyTo":null,"subject":"[PATCH] Makefile: add a knob to turn off hardlinks within same directory","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2011-10-12T08:38:42Z","receivedAt":"2011-10-12T08:38:42Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"The git builtins in $(gitexecdir) are implemented as hard links to a\nsingle git binary by default, so even the overhead of symlink\nresolution is not needed to run them. However, the trick can be\nharmful, in two cases:\n\n - on Windows, some tools to estimate directory size hugely\n overestimate the size of git (each hardlink counts as taking up the\n same amount of space as git.exe)\n\n - various filesystems have limits on the number of hardlinks that\n can be made to a particular file --- Linux's LINK_MAX is 127,\n _POSIX_LINK_MAX is 8, and btrfs has a limit of 4096 /\n ($len_filename + 8) or so links to a given inode in a single\n directory.\n\nNormally that second case is not a problem (when ln fails, \"make\ninstall\" falls back to \"ln -s\"), but if git is tar'ed up on one\nfilesystem and then extracted on a more limited one, it can result in\n\"Too many links\" errors.\n\nNowadays people are encouraged to (and typically do) run builtins\nusing the \"git\" command name directly rather than those dashed forms\nin scripts, making the use of hardlinks for the dashed forms a\nsomewhat pointless optimization. Introduce a new knob to allow people\nto turn it off with a simple \"make install NO_HARDLINKS=YesPlease\".\n\nTypically someone using this setting would want to set\nNO_CROSS_DIRECTORY_HARDLINKS, too, but that is not enforced, so you\ncan make $(bindir)/git and $(gitexecdir)/git into hardlinks to the\nsame inode and still make sure your tarball avoids the btrfs limits if\nyou want.\n\nRequested-by: Bastian Blank \nSigned-off-by: Jonathan Nieder \n---\nHi,\n\nSee for context. Sane?\n\n Makefile | 7 +++++++\n 1 files changed, 7 insertions(+), 0 deletions(-)\n\ndiff --git a/Makefile b/Makefile\nindex 9afdcf2a..ab64ff4c 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -226,6 +226,10 @@ all::\n # Define NO_CROSS_DIRECTORY_HARDLINKS if you plan to distribute the installed\n # programs as a tar, where bin/ and libexec/ might be on different file systems.\n #\n+# Define NO_HARDLINKS if you plan to distribute the installed programs as a tar\n+# that might be extracted on a filesystem like btrfs that does not cope well\n+# with many links to one inode in one directory.\n+#\n # Define USE_NED_ALLOCATOR if you want to replace the platforms default\n # memory allocators with the nedmalloc allocator written by Niall Douglas.\n #\n@@ -2326,12 +2330,14 @@ endif\n \t} && \\\n \tfor p in $(filter $(install_bindir_programs),$(BUILT_INS)); do \\\n \t\t$(RM) \"$$bindir/$$p\" && \\\n+\t\ttest -z \"$(NO_HARDLINKS)\" && \\\n \t\tln \"$$bindir/git$X\" \"$$bindir/$$p\" 2>/dev/null || \\\n \t\tln -s \"git$X\" \"$$bindir/$$p\" 2>/dev/null || \\\n \t\tcp \"$$bindir/git$X\" \"$$bindir/$$p\" || exit; \\\n \tdone && \\\n \tfor p in $(BUILT_INS); do \\\n \t\t$(RM) \"$$execdir/$$p\" && \\\n+\t\ttest -z \"$(NO_HARDLINKS)\" && \\\n \t\tln \"$$execdir/git$X\" \"$$execdir/$$p\" 2>/dev/null || \\\n \t\tln -s \"git$X\" \"$$execdir/$$p\" 2>/dev/null || \\\n \t\tcp \"$$execdir/git$X\" \"$$execdir/$$p\" || exit; \\\n@@ -2339,6 +2345,7 @@ endif\n \tremote_curl_aliases=\"$(REMOTE_CURL_ALIASES)\" && \\\n \tfor p in $$remote_curl_aliases; do \\\n \t\t$(RM) \"$$execdir/$$p\" && \\\n+\t\ttest -z \"$(NO_HARDLINKS)\" && \\\n \t\tln \"$$execdir/git-remote-http$X\" \"$$execdir/$$p\" 2>/dev/null || \\\n \t\tln -s \"git-remote-http$X\" \"$$execdir/$$p\" 2>/dev/null || \\\n \t\tcp \"$$execdir/git-remote-http$X\" \"$$execdir/$$p\" || exit; \\\n-- \n1.7.7\n"},{"id":"177470","messageId":"7vaa969go4.fsf@alter.siamese.dyndns.org","threadId":"28657","inReplyTo":"20111012083842.GA21969@elie.hsd1.il.comcast.net","subject":"Re: [PATCH] Makefile: add a knob to turn off hardlinks within same directory","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2011-10-12T19:02:51Z","receivedAt":"2011-10-12T19:02:51Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jonathan Nieder writes:\n\n> Typically someone using this setting would want to set\n> NO_CROSS_DIRECTORY_HARDLINKS, too, but that is not enforced, so you\n> can make $(bindir)/git and $(gitexecdir)/git into hardlinks to the\n> same inode and still make sure your tarball avoids the btrfs limits if\n> you want.\n>\n> Requested-by: Bastian Blank \n> Signed-off-by: Jonathan Nieder \n> ---\n> Hi,\n>\n> See for context. Sane?\n>\n> Makefile | 7 +++++++\n> 1 files changed, 7 insertions(+), 0 deletions(-)\n>\n> diff --git a/Makefile b/Makefile\n> index 9afdcf2a..ab64ff4c 100644\n> --- a/Makefile\n> +++ b/Makefile\n> @@ -226,6 +226,10 @@ all::\n> # Define NO_CROSS_DIRECTORY_HARDLINKS if you plan to distribute the installed\n> # programs as a tar, where bin/ and libexec/ might be on different file systems.\n> #\n> +# Define NO_HARDLINKS if you plan to distribute the installed programs as a tar\n> +# that might be extracted on a filesystem like btrfs that does not cope well\n> +# with many links to one inode in one directory.\n> +#\n\nHmm.... I would understand if the change were to eventually remove these\n\"git-foo\" anywhere from the filesystem perhaps after a large version bump\nin Git 2.0, but that is not what you are trying to do.\n\nWouldn't your use case be better served with\n\n $ tar zcf dist.tar.gz --hard-dereference $list_of_files_to_tar_up\n\ninstead?\n"},{"id":"177479","messageId":"20111012195705.GA13768@elie.hsd1.il.comcast.net","threadId":"28657","inReplyTo":"7vaa969go4.fsf@alter.siamese.dyndns.org","subject":"Re: [PATCH] Makefile: add a knob to turn off hardlinks within same directory","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2011-10-12T19:57:05Z","receivedAt":"2011-10-12T19:57:05Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"Junio C Hamano wrote:\n\n> Wouldn't your use case be better served with\n>\n> $ tar zcf dist.tar.gz --hard-dereference $list_of_files_to_tar_up\n>\n> instead?\n\nNo, because that duplicates the files instead of making symlinks.\n"}]}