# [BUG] git repack fails if .git/objects/pack is on a separate partition 7 messages from 2010-10-19 to 2010-10-19. Participants: Marat Radchenko, Thomas Rast, Junio C Hamano. Thread: https://gitlist.dev/t/25477 ## Marat Radchenko, 2010-10-19 06:28 Subject: [BUG] git repack fails if .git/objects/pack is on a separate partition Message-ID: <1287469706023-5649732.post@n2.nabble.com> URL: https://gitlist.dev/e/1287469706023-5649732.post%40n2.nabble.com ``` Console log: Counting objects: 5137063, done. Delta compression using up to 4 threads. Compressing objects: 100% (892984/892984), done. fatal: unable to rename temporary pack file: Invalid cross-device link error: failed to run repack Reason: write_pack_file in builtin/pack-objects.c first writes tmp file in "pack/tmp_pack_XXXXXX". Then, it tries to rename it to "%s-%s.pack": snprintf(tmpname, sizeof(tmpname), "%s-%s.pack", base_name, sha1_to_hex(sha1)); ... if (rename(pack_tmp_name, tmpname)) die_errno("unable to rename temporary pack file"); and happily fails because that crosses device boundary. Same problem with index file. Suggested fix: do those renames inside pack directory or avoid them at all. -- View this message in context: http://git.661346.n2.nabble.com/BUG-git-repack-fails-if-git-objects-pack-is-on-a-separate-partition-tp5649732p5649732.html Sent from the git mailing list archive at Nabble.com. ``` ## Thomas Rast, 2010-10-19 06:59 Subject: Re: [BUG] git repack fails if .git/objects/pack is on a separate partition Message-ID: <201010190859.43098.trast@student.ethz.ch> URL: https://gitlist.dev/e/201010190859.43098.trast%40student.ethz.ch In-Reply-To: <1287469706023-5649732.post@n2.nabble.com> ``` Marat Radchenko wrote: > fatal: unable to rename temporary pack file: Invalid cross-device link > error: failed to run repack > > Reason: > write_pack_file in builtin/pack-objects.c first writes tmp file in > "pack/tmp_pack_XXXXXX". What's your git version? Since 8b4eb6b (Do not perform cross-directory renames when creating packs, 2008-09-22), which was first released in 1.6.0.3 and 1.6.1, it should only create and rename files within the pack/ subdirectory. Which is what you are implying in the above snippet, but this should never result in a cross-device rename, so maybe I'm missing something. -- Thomas Rast trast@{inf,student}.ethz.ch ``` ## Marat Radchenko, 2010-10-19 07:29 Subject: Re: [BUG] git repack fails if .git/objects/pack is on a separate partition Message-ID: <1287473363910-5649843.post@n2.nabble.com> URL: https://gitlist.dev/e/1287473363910-5649843.post%40n2.nabble.com In-Reply-To: <201010190859.43098.trast@student.ethz.ch> ``` > What's your git version? > 1.7.3.1 > It should only create and rename > files within the pack/ subdirectory. > It should, yep, but it doesn't. Which is what you are implying > in the above snippet, but this should never result in a cross-device > rename, so maybe I'm missing something. > That rename call attempts to put file in _current_ directory. Proof: mkdir foo && touch foo/bar && strace mv foo/bar baz 2>&1 | grep rename outputs rename("foo/bar", "baz") -- View this message in context: http://git.661346.n2.nabble.com/BUG-git-repack-fails-if-git-objects-pack-is-on-a-separate-partition-tp5649732p5649843.html Sent from the git mailing list archive at Nabble.com. ``` ## Marat Radchenko, 2010-10-19 07:40 Subject: Re: [BUG] git repack fails if .git/objects/pack is on a separate partition Message-ID: <1287474040065-5649864.post@n2.nabble.com> URL: https://gitlist.dev/e/1287474040065-5649864.post%40n2.nabble.com In-Reply-To: <1287473363910-5649843.post@n2.nabble.com> ``` Testcase: # tell it where is other partition OTHER_PARTITION=/path/to/other/partition/ # init repo mkdir foo && cd foo && git init # put some stuff and run gc to create packfile cp /etc/fstab . && git add fstab && git commit -m "initial" && git gc # move pack dir to other partition and install symlink mv .git/objects/pack $OTHER_PARTITION && ln -s $OTHER_PARTITION/pack .git/objects/pack # run gc (it fails) strace -f git gc 2>&1 | grep rename [pid 3792] rename(".git/packed-refs.lock", ".git/packed-refs") = 0 [pid 3793] rename(".git/logs/refs/heads/master.lock", ".git/logs/refs/heads/master") = 0 [pid 3793] rename(".git/logs/HEAD.lock", ".git/logs/HEAD") = 0 [pid 3808] rename(".git/objects/pack/tmp_pack_8ZhS92", "/home/marat/foo/.git/objects/.tmp-3795-pack-da13ceb9f70c2762595a7a2932411db339bdab46.pack") = -1 EXDEV (Invalid cross-device link) [pid 3808] write(2, "fatal: unable to rename temporar"..., 71fatal: unable to rename temporary pack file: Invalid cross-device link So you see, it tries to move packfile from .git/objects/pack to .git/objects -- View this message in context: http://git.661346.n2.nabble.com/BUG-git-repack-fails-if-git-objects-pack-is-on-a-separate-partition-tp5649732p5649864.html Sent from the git mailing list archive at Nabble.com. ``` ## Thomas Rast, 2010-10-19 09:50 Subject: [PATCH] repack: place temporary packs under .git/objects/pack/ Message-ID: URL: https://gitlist.dev/e/cbdf4787ddaccd7105ab1661529a98257d61179b.1287481801.git.trast%40student.ethz.ch In-Reply-To: <1287474040065-5649864.post@n2.nabble.com> ``` git-pack-objects is already careful to start out its temporary packs under .git/objects/pack/ (cf. 8b4eb6b, Do not perform cross-directory renames when creating packs, 2008-09-22), but git-repack did not respond in kind so the effort was lost when the filesystem boundary is exactly at that directory. Let git-repack pass a path under .git/objects/pack/ as the base for its temporary packs. This means we might need the $PACKDIR sooner (before the pack-objects invocation), so move the mkdir up just to be safe. Also note that the only use of *.pack is in the find invocation way before the pack-objects call, so the temporary packs will not suddenly show up in any wildcards because of the directory change. Reported-by: Marat Radchenko Signed-off-by: Thomas Rast --- git-repack.sh | 5 +++-- 1 files changed, 3 insertions(+), 2 deletions(-) diff --git a/git-repack.sh b/git-repack.sh index 769baaf..624feec 100755 --- a/git-repack.sh +++ b/git-repack.sh @@ -52,7 +52,7 @@ true) esac PACKDIR="$GIT_OBJECT_DIRECTORY/pack" -PACKTMP="$GIT_OBJECT_DIRECTORY/.tmp-$$-pack" +PACKTMP="$PACKDIR/.tmp-$$-pack" rm -f "$PACKTMP"-* trap 'rm -f "$PACKTMP"-*' 0 1 2 3 15 @@ -82,6 +82,8 @@ case ",$all_into_one," in ;; esac +mkdir -p "$PACKDIR" || exit + args="$args $local ${GIT_QUIET:+-q} $no_reuse$extra" names=$(git pack-objects --keep-true-parents --honor-pack-keep --non-empty --all --reflog $args URL: https://gitlist.dev/e/1287483456222-5650273.post%40n2.nabble.com In-Reply-To: ``` Either i'm doing something wrong or given patch doesn't fix testcase [1] i gave before. Tested by applying patch to git master and running /git --exec-path= gc [1] http://git.661346.n2.nabble.com/BUG-git-repack-fails-if-git-objects-pack-is-on-a-separate-partition-tp5649732p5649864.html -- View this message in context: http://git.661346.n2.nabble.com/BUG-git-repack-fails-if-git-objects-pack-is-on-a-separate-partition-tp5649732p5650273.html Sent from the git mailing list archive at Nabble.com. ``` ## Junio C Hamano, 2010-10-19 16:33 Subject: Re: [PATCH] repack: place temporary packs under .git/objects/pack/ Message-ID: <7vocaq16s2.fsf@alter.siamese.dyndns.org> URL: https://gitlist.dev/e/7vocaq16s2.fsf%40alter.siamese.dyndns.org In-Reply-To: ``` Thomas Rast writes: > git-pack-objects is already careful to start out its temporary packs > under .git/objects/pack/ (cf. 8b4eb6b, Do not perform cross-directory > renames when creating packs, 2008-09-22), but git-repack did not > respond in kind so the effort was lost when the filesystem boundary is > exactly at that directory. > > Let git-repack pass a path under .git/objects/pack/ as the base for > its temporary packs. > > This means we might need the $PACKDIR sooner (before the pack-objects > invocation), so move the mkdir up just to be safe. > > Also note that the only use of *.pack is in the find invocation way > before the pack-objects call, so the temporary packs will not suddenly > show up in any wildcards because of the directory change. Looks good; thanks. ```