Re: [PATCH] copy: prefer reflinks for file copies
- From
- Vlad Petric <vlad@drpetric.com>
- Date
- Aug 11, 2026, 18:31 UTC
- Message-ID
- <CAJuH0wC1gSo-8nGnPGh2PqzbJuDBuY1pQS7RH0svwTfJWat4JA@mail.gmail.com>
- In-Reply-To
- <pull.2380.git.git.1786460606222.gitgitgadget@gmail.com>
You’re right that a successful hardlink is preferable for immutable object files.
My motivation came from an actual local clone in my home dir (first repo cloned normally, second repo cloned with --reference to the first one).
Both repositories were on the same reflink-capable ZFS filesystem, but the normal git clone produced neither hardlinked nor reflinked object files, so the objects were copied byte-for-byte. With the FICLONE path enabled, the same setup successfully reflinked the object files.
That is a useful fallback when Git’s hardlink path does not produce links, but it does not justify trying reflink before a successful hardlink. I revised the order to:
hardlink
reflink
byte-for-byte copy.
With --no-hardlinks, the order would remain reflink followed by byte-for-byte copy.
(apologies if you received this multiple times already)
On Tue, Aug 11, 2026 at 11:03 AM Vlad Petric via GitGitGadget <gitgitgadget@gmail.com> wrote:
Show 493 quoted lines
>
> From: Vlad Petric <vlad@drpetric.com>
>
> Git currently copies files byte-for-byte through copy_file(). Local
> clones separately try to hardlink object files before falling back to
> copying them.
>
> On filesystems that support copy-on-write cloning, a reflink can share
> the underlying storage without making the source and destination names
> refer to the same inode. This provides most of the space and I/O
> benefits of hardlinks while allowing either file to be replaced or
> modified independently.
>
> On Linux, try FICLONE before performing a byte-for-byte copy. Treat
> reflinking as an optimization: if the ioctl is unavailable or fails,
> remove the partial destination and use the existing copy path.
>
> For local clones, try a reflink before the existing hardlink path. The
> resulting order is therefore:
>
> - reflink;
> - hardlink, unless --no-hardlinks was requested;
> - byte-for-byte copy.
>
> Preserve source timestamps when reflinking local object files. This
> matches the previous hardlink and copy behavior and is important for
> the expiry decisions made by prune and gc.
>
> Add an LD_PRELOAD-based test helper that can force FICLONE to succeed,
> report EOPNOTSUPP, or report another error. This exercises the reflink
> and fallback paths even when the test filesystem does not support
> reflinks. Cover generic file copying, local clones, --no-hardlinks,
> hardlink fallback, byte-copy fallback, object integrity, and timestamp
> preservation.
>
> The focused tests pass on three independent filesystems:
>
> - ZFS at /home;
> - ext4 at /tmp;
> - tmpfs at /dev/shm.
>
> The complete Git test suite also passes on all three filesystems with
> no unexpected failures.
>
> Signed-off-by: Vlad Petric <vlad@drpetric.com>
> ---
> copy: prefer reflinks for file copies
>
> Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-git-2380%2Fvladpetric%2Fvp%2Freflink-copy-v1
> Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-git-2380/vladpetric/vp/reflink-copy-v1
> Pull-Request: https://github.com/git/git/pull/2380
>
> Makefile | 1 +
> builtin/clone.c | 3 ++
> copy.c | 96 +++++++++++++++++++++++++++++++-----
> copy.h | 4 ++
> t/helper/meson.build | 1 +
> t/helper/test-copy-file.c | 13 +++++
> t/helper/test-fake-reflink.c | 72 +++++++++++++++++++++++++++
> t/helper/test-tool.c | 1 +
> t/helper/test-tool.h | 1 +
> t/meson.build | 1 +
> t/t0094-reflink.sh | 89 +++++++++++++++++++++++++++++++++
> t/t5605-clone-local.sh | 4 +-
> 12 files changed, 272 insertions(+), 14 deletions(-)
> create mode 100644 t/helper/test-copy-file.c
> create mode 100644 t/helper/test-fake-reflink.c
> create mode 100755 t/t0094-reflink.sh
>
> diff --git a/Makefile b/Makefile
> index fac3e8879c..802c0e9a37 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -814,6 +814,7 @@ TEST_BUILTINS_OBJS += test-bundle-uri.o
> TEST_BUILTINS_OBJS += test-cache-tree.o
> TEST_BUILTINS_OBJS += test-chmtime.o
> TEST_BUILTINS_OBJS += test-config.o
> +TEST_BUILTINS_OBJS += test-copy-file.o
> TEST_BUILTINS_OBJS += test-crontab.o
> TEST_BUILTINS_OBJS += test-csprng.o
> TEST_BUILTINS_OBJS += test-date.o
> diff --git a/builtin/clone.c b/builtin/clone.c
> index 5b25cca510..cd83093ec9 100644
> --- a/builtin/clone.c
> +++ b/builtin/clone.c
> @@ -309,6 +309,9 @@ static void copy_or_link_directory(struct strbuf *src, struct strbuf *dest,
>
> if (unlink(dest->buf) && errno != ENOENT)
> die_errno(_("failed to unlink '%s'"), dest->buf);
> + if (!copy_file_reflink_with_time(the_repository, dest->buf,
> + src->buf, 0666))
> + continue;
> if (!option_no_hardlinks) {
> if (!link(src->buf, dest->buf)) {
> struct stat st;
> diff --git a/copy.c b/copy.c
> index 6074132050..da8a285c1e 100644
> --- a/copy.c
> +++ b/copy.c
> @@ -5,6 +5,12 @@
> #include "strbuf.h"
> #include "abspath.h"
>
> +#ifdef __linux__
> +#include <sys/ioctl.h>
> +
> +#define FICLONE _IOW(0x94, 9, int)
> +#endif
> +
> int copy_fd(int ifd, int ofd)
> {
> while (1) {
> @@ -33,19 +39,9 @@ static int copy_times(const char *dst, const char *src)
> return 0;
> }
>
> -int copy_file(struct repository *repo,
> - const char *dst, const char *src, int mode)
> +static int finish_copy(struct repository *repo, const char *dst,
> + int fdi, int fdo, int status)
> {
> - int fdi, fdo, status;
> -
> - mode = (mode & 0111) ? 0777 : 0666;
> - if ((fdi = open(src, O_RDONLY)) < 0)
> - return fdi;
> - if ((fdo = open(dst, O_WRONLY | O_CREAT | O_EXCL, mode)) < 0) {
> - close(fdi);
> - return fdo;
> - }
> - status = copy_fd(fdi, fdo);
> switch (status) {
> case COPY_READ_ERROR:
> error_errno("copy-fd: read returned");
> @@ -64,6 +60,82 @@ int copy_file(struct repository *repo,
> return status;
> }
>
> +int copy_file_reflink(struct repository *repo,
> + const char *dst, const char *src, int mode)
> +{
> +#ifndef FICLONE
> + (void)repo;
> + (void)dst;
> + (void)src;
> + (void)mode;
> + errno = ENOTSUP;
> + return -1;
> +#else
> + int fdi, fdo, status;
> +
> + mode = (mode & 0111) ? 0777 : 0666;
> + if ((fdi = open(src, O_RDONLY)) < 0)
> + return fdi;
> + if ((fdo = open(dst, O_WRONLY | O_CREAT | O_EXCL, mode)) < 0) {
> + close(fdi);
> + return fdo;
> + }
> + status = ioctl(fdo, FICLONE, fdi);
> + if (status) {
> + int saved_errno = errno;
> +
> + close(fdi);
> + close(fdo);
> + unlink(dst);
> + errno = saved_errno;
> + return -1;
> + }
> +
> + return finish_copy(repo, dst, fdi, fdo, 0);
> +#endif
> +}
> +
> +int copy_file_reflink_with_time(struct repository *repo,
> + const char *dst, const char *src, int mode)
> +{
> + int saved_errno;
> +
> + if (copy_file_reflink(repo, dst, src, mode))
> + return -1;
> + if (!copy_times(dst, src))
> + return 0;
> +
> + saved_errno = errno;
> + unlink(dst);
> + errno = saved_errno;
> + return -1;
> +}
> +
> +static int copy_file_contents(struct repository *repo,
> + const char *dst, const char *src, int mode)
> +{
> + int fdi, fdo;
> +
> + mode = (mode & 0111) ? 0777 : 0666;
> + if ((fdi = open(src, O_RDONLY)) < 0)
> + return fdi;
> + if ((fdo = open(dst, O_WRONLY | O_CREAT | O_EXCL, mode)) < 0) {
> + close(fdi);
> + return fdo;
> + }
> +
> + return finish_copy(repo, dst, fdi, fdo, copy_fd(fdi, fdo));
> +}
> +
> +int copy_file(struct repository *repo,
> + const char *dst, const char *src, int mode)
> +{
> + if (!copy_file_reflink(repo, dst, src, mode))
> + return 0;
> +
> + return copy_file_contents(repo, dst, src, mode);
> +}
> +
> int copy_file_with_time(struct repository *repo,
> const char *dst, const char *src, int mode)
> {
> diff --git a/copy.h b/copy.h
> index 1059b118d6..4c603756a7 100644
> --- a/copy.h
> +++ b/copy.h
> @@ -6,6 +6,10 @@ struct repository;
> #define COPY_READ_ERROR (-2)
> #define COPY_WRITE_ERROR (-3)
> int copy_fd(int ifd, int ofd);
> +int copy_file_reflink(struct repository *repo,
> + const char *dst, const char *src, int mode);
> +int copy_file_reflink_with_time(struct repository *repo,
> + const char *dst, const char *src, int mode);
> int copy_file(struct repository *repo,
> const char *dst, const char *src, int mode);
> int copy_file_with_time(struct repository *repo,
> diff --git a/t/helper/meson.build b/t/helper/meson.build
> index 3235f10ab8..90b57fb86a 100644
> --- a/t/helper/meson.build
> +++ b/t/helper/meson.build
> @@ -7,6 +7,7 @@ test_tool_sources = [
> 'test-cache-tree.c',
> 'test-chmtime.c',
> 'test-config.c',
> + 'test-copy-file.c',
> 'test-crontab.c',
> 'test-csprng.c',
> 'test-date.c',
> diff --git a/t/helper/test-copy-file.c b/t/helper/test-copy-file.c
> new file mode 100644
> index 0000000000..fee14b60b6
> --- /dev/null
> +++ b/t/helper/test-copy-file.c
> @@ -0,0 +1,13 @@
> +#define USE_THE_REPOSITORY_VARIABLE
> +
> +#include "test-tool.h"
> +#include "copy.h"
> +#include "environment.h"
> +#include "repository.h"
> +
> +int cmd__copy_file(int argc, const char **argv)
> +{
> + if (argc != 3)
> + return 129;
> + return copy_file(the_repository, argv[2], argv[1], 0666) ? 1 : 0;
> +}
> diff --git a/t/helper/test-fake-reflink.c b/t/helper/test-fake-reflink.c
> new file mode 100644
> index 0000000000..9afc4c14d4
> --- /dev/null
> +++ b/t/helper/test-fake-reflink.c
> @@ -0,0 +1,72 @@
> +#define _GNU_SOURCE
> +#include <dlfcn.h>
> +#include <errno.h>
> +#include <fcntl.h>
> +#include <stdarg.h>
> +#include <stdlib.h>
> +#include <string.h>
> +#include <sys/ioctl.h>
> +#include <unistd.h>
> +
> +#define FICLONE _IOW(0x94, 9, int)
> +
> +static int emulate_clone(int dst, int src)
> +{
> + char buf[8192];
> + off_t pos = 0;
> +
> + for (;;) {
> + ssize_t nr = pread(src, buf, sizeof(buf), pos);
> + if (nr < 0)
> + return -1;
> + if (!nr)
> + return ftruncate(dst, pos);
> + if (pwrite(dst, buf, nr, pos) != nr)
> + return -1;
> + pos += nr;
> + }
> +}
> +
> +static void log_clone_attempt(void)
> +{
> + const char *path = getenv("GIT_TEST_FICLONE_LOG");
> + int fd;
> +
> + if (!path)
> + return;
> + fd = open(path, O_WRONLY | O_CREAT | O_APPEND, 0666);
> + if (fd < 0)
> + return;
> + write(fd, "FICLONE\n", 8);
> + close(fd);
> +}
> +
> +int ioctl(int fd, unsigned long request, ...)
> +{
> + static int (*real_ioctl)(int, unsigned long, ...);
> + va_list ap;
> + unsigned long arg;
> + const char *mode;
> +
> + va_start(ap, request);
> + arg = va_arg(ap, unsigned long);
> + va_end(ap);
> +
> + if (request != FICLONE) {
> + if (!real_ioctl)
> + real_ioctl = dlsym(RTLD_NEXT, "ioctl");
> + return real_ioctl(fd, request, arg);
> + }
> +
> + log_clone_attempt();
> + mode = getenv("GIT_TEST_FICLONE");
> + if (!mode || !strcmp(mode, "real")) {
> + if (!real_ioctl)
> + real_ioctl = dlsym(RTLD_NEXT, "ioctl");
> + return real_ioctl(fd, request, arg);
> + }
> + if (!strcmp(mode, "success"))
> + return emulate_clone(fd, (int)arg);
> + errno = !strcmp(mode, "unsupported") ? EOPNOTSUPP : EIO;
> + return -1;
> +}
> diff --git a/t/helper/test-tool.c b/t/helper/test-tool.c
> index b71a22b43b..51012fa46c 100644
> --- a/t/helper/test-tool.c
> +++ b/t/helper/test-tool.c
> @@ -17,6 +17,7 @@ static struct test_cmd cmds[] = {
> { "cache-tree", cmd__cache_tree },
> { "chmtime", cmd__chmtime },
> { "config", cmd__config },
> + { "copy-file", cmd__copy_file },
> { "crontab", cmd__crontab },
> { "csprng", cmd__csprng },
> { "date", cmd__date },
> diff --git a/t/helper/test-tool.h b/t/helper/test-tool.h
> index f2885b33d5..7565cceb86 100644
> --- a/t/helper/test-tool.h
> +++ b/t/helper/test-tool.h
> @@ -10,6 +10,7 @@ int cmd__bundle_uri(int argc, const char **argv);
> int cmd__cache_tree(int argc, const char **argv);
> int cmd__chmtime(int argc, const char **argv);
> int cmd__config(int argc, const char **argv);
> +int cmd__copy_file(int argc, const char **argv);
> int cmd__crontab(int argc, const char **argv);
> int cmd__csprng(int argc, const char **argv);
> int cmd__date(int argc, const char **argv);
> diff --git a/t/meson.build b/t/meson.build
> index a25f37d2f5..a6575b8b9a 100644
> --- a/t/meson.build
> +++ b/t/meson.build
> @@ -126,6 +126,7 @@ integration_tests = [
> 't0091-bugreport.sh',
> 't0092-diagnose.sh',
> 't0093-verify-cache-df-gap.sh',
> + 't0094-reflink.sh',
> 't0095-bloom.sh',
> 't0100-previous.sh',
> 't0101-at-syntax.sh',
> diff --git a/t/t0094-reflink.sh b/t/t0094-reflink.sh
> new file mode 100755
> index 0000000000..25e989d272
> --- /dev/null
> +++ b/t/t0094-reflink.sh
> @@ -0,0 +1,89 @@
> +#!/bin/sh
> +
> +test_description='reflink-first file copying'
> +
> +. ./test-lib.sh
> +
> +FAKE_REFLINK=/tmp/git-test-fake-reflink-$$.so
> +test_atexit 'rm -f "$FAKE_REFLINK"'
> +
> +test_lazy_prereq FICLONE_PRELOAD '
> + test_have_prereq !MINGW &&
> + test "$(uname -s)" = Linux &&
> + ${CC:-cc} -shared -fPIC -o "$FAKE_REFLINK" \
> + "$TEST_DIRECTORY/helper/test-fake-reflink.c" -ldl
> +'
> +
> +test_expect_success FICLONE_PRELOAD 'generic copy accepts reflink success' '
> + printf content >source &&
> + GIT_TEST_FICLONE=success \
> + GIT_TEST_FICLONE_LOG="$TRASH_DIRECTORY/generic-success.log" \
> + LD_PRELOAD="$FAKE_REFLINK" \
> + test-tool copy-file source destination &&
> + test_file_not_empty generic-success.log &&
> + test_cmp source destination &&
> + test "$(stat -c %i source)" != "$(stat -c %i destination)"
> +'
> +
> +test_expect_success FICLONE_PRELOAD 'generic copy falls back when unsupported' '
> + printf fallback >source-fallback &&
> + GIT_TEST_FICLONE=unsupported \
> + GIT_TEST_FICLONE_LOG="$TRASH_DIRECTORY/generic-unsupported.log" \
> + LD_PRELOAD="$FAKE_REFLINK" \
> + test-tool copy-file source-fallback destination-fallback &&
> + test_file_not_empty generic-unsupported.log &&
> + test_cmp source-fallback destination-fallback
> +'
> +
> +test_expect_success FICLONE_PRELOAD 'generic copy falls back after reflink error' '
> + printf error-fallback >source-error &&
> + GIT_TEST_FICLONE=error \
> + GIT_TEST_FICLONE_LOG="$TRASH_DIRECTORY/generic-error.log" \
> + LD_PRELOAD="$FAKE_REFLINK" \
> + test-tool copy-file source-error destination-error &&
> + test_file_not_empty generic-error.log &&
> + test_cmp source-error destination-error
> +'
> +
> +test_expect_success FICLONE_PRELOAD 'local clone prefers successful reflinks' '
> + git init source-repo &&
> + git -C source-repo commit --allow-empty -m base &&
> + GIT_TEST_FICLONE=success \
> + GIT_TEST_FICLONE_LOG="$TRASH_DIRECTORY/clone-success.log" \
> + LD_PRELOAD="$FAKE_REFLINK" \
> + git clone --bare source-repo reflink-clone &&
> + test_file_not_empty clone-success.log &&
> + find reflink-clone/objects -type f -links +1 >hardlinks &&
> + test_must_be_empty hardlinks &&
> + git -C reflink-clone fsck --no-dangling
> +'
> +
> +test_expect_success FICLONE_PRELOAD '--no-hardlinks also prefers successful reflinks' '
> + GIT_TEST_FICLONE=success \
> + GIT_TEST_FICLONE_LOG="$TRASH_DIRECTORY/no-hardlinks-success.log" \
> + LD_PRELOAD="$FAKE_REFLINK" \
> + git clone --bare --no-hardlinks source-repo no-hardlinks-reflink-clone &&
> + test_file_not_empty no-hardlinks-success.log &&
> + find no-hardlinks-reflink-clone/objects -type f -links +1 >hardlinks &&
> + test_must_be_empty hardlinks &&
> + git -C no-hardlinks-reflink-clone fsck --no-dangling
> +'
> +
> +test_expect_success FICLONE_PRELOAD 'local clone preserves hardlink fallback' '
> + GIT_TEST_FICLONE=unsupported \
> + LD_PRELOAD="$FAKE_REFLINK" \
> + git clone --bare source-repo hardlink-clone &&
> + find hardlink-clone/objects -type f -links +1 >hardlinks &&
> + test_file_not_empty hardlinks
> +'
> +
> +test_expect_success FICLONE_PRELOAD '--no-hardlinks preserves byte-copy fallback' '
> + GIT_TEST_FICLONE=unsupported \
> + LD_PRELOAD="$FAKE_REFLINK" \
> + git clone --bare --no-hardlinks source-repo copied-clone &&
> + find copied-clone/objects -type f -links +1 >hardlinks &&
> + test_must_be_empty hardlinks &&
> + git -C copied-clone fsck --no-dangling
> +'
> +
> +test_done
> diff --git a/t/t5605-clone-local.sh b/t/t5605-clone-local.sh
> index 156362f145..b3ab4d6faf 100755
> --- a/t/t5605-clone-local.sh
> +++ b/t/t5605-clone-local.sh
> @@ -58,10 +58,10 @@ test_expect_success 'With -no-hardlinks, local will make a copy' '
> ! repo_is_hardlinked w
> '
>
> -test_expect_success 'Even without -l, local will make a hardlink' '
> +test_expect_success 'local clone copies the complete object database' '
> rm -fr w &&
> git clone -l --bare x w &&
> - repo_is_hardlinked w
> + git -C w fsck --no-dangling
> '
>
> test_expect_success 'local clone of repo with nonexistent ref in HEAD' '
>
> base-commit: 010afd3166ddc64c9863b1506f12cbcdda0d4ea1
> --
> gitgitgadget