{"thread":{"id":"885","subject":"RFE: git relink","startedAt":"2005-06-09T18:35:51Z","lastAt":"2005-06-11T03:44:01Z","messageCount":4,"participants":["Jeff Garzik","Ryan Anderson","Junio C Hamano"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"4797","messageId":"42A88C07.5050907@pobox.com","threadId":"885","inReplyTo":null,"subject":"RFE: git relink","fromName":"Jeff Garzik","fromEmail":"jgarzik@pobox.com","sentAt":"2005-06-09T18:35:51Z","receivedAt":"2005-06-09T18:35:51Z","isPatch":false,"sender":{"key":"jgarzik@pobox.com","avatar":null},"body":"\nIt would be nice if somebody were motivated enough to create a command \nthat functions like:\n\n\tgit relink repoA repoB repoC repoD... repoX\n\nwhich would examine\n\n\trepoA/.git\n\trepoB/.git\n\trepoC/.git\n\trepoD/.git\n\nand verify (updating, if necessary) that each of the A/B/C/D repos are \nhardlinked to repoX.\n\n\tJeff\n\n\n\n"},{"id":"4798","messageId":"20050609191548.GG21076@mythryan2.michonline.com","threadId":"885","inReplyTo":"42A88C07.5050907@pobox.com","subject":"Re: RFE: git relink","fromName":"Ryan Anderson","fromEmail":"ryan@michonline.com","sentAt":"2005-06-09T19:15:48Z","receivedAt":"2005-06-09T19:15:48Z","isPatch":false,"sender":{"key":"ryan@michonline.com","avatar":null},"body":"On Thu, Jun 09, 2005 at 02:35:51PM -0400, Jeff Garzik wrote:\n> \n> It would be nice if somebody were motivated enough to create a command \n> that functions like:\n> \n> \tgit relink repoA repoB repoC repoD... repoX\n> \n> which would examine\n> \n> \trepoA/.git\n> \trepoB/.git\n> \trepoC/.git\n> \trepoD/.git\n> \n> and verify (updating, if necessary) that each of the A/B/C/D repos are \n> hardlinked to repoX.\n\nSubmitted a while ago, dunno what happened with it. This only does 2\nrepositories, but it's trivial to do\n\tfor i in repoA repoB repoC repoD ; do git-relink-script \"$i\" repoX ; done\n\nProvide a tool to relink two git repositories.\n\nSigned-Off-By: Ryan Anderson \n\n---\ncommit a3bcc763d71bdb91a3b48e9105fbaa5e79abb807\ntree 2553e2d8befbe0cda3e413616fd4cc7bf04157ad\nparent a31c6d022e2435a514fcc8ca57f9995c4376a986\nauthor Ryan Anderson 1115185675 -0400\ncommitter Ryan Anderson 1115185675 -0400\n\nIndex: Makefile\n===================================================================\n--- 51a882a2dc62e0d3cdc79e0badc61559fb723481/Makefile (mode:100644 sha1:99b4753d34879842b972da9b68694c9d0485f216)\n+++ 2553e2d8befbe0cda3e413616fd4cc7bf04157ad/Makefile (mode:100644 sha1:a99665e252a2342caa84238e886a80a5f27ac3c8)\n@@ -13,7 +13,7 @@\n AR=ar\n \n SCRIPTS=git-apply-patch-script git-merge-one-file-script git-prune-script \\\n-\tgit-pull-script git-tag-script\n+\tgit-pull-script git-tag-script git-relink-script\n \n PROG= git-update-cache git-diff-files git-init-db git-write-tree \\\n \tgit-read-tree git-commit-tree git-cat-file git-fsck-cache \\\nIndex: git-relink-script\n===================================================================\n--- /dev/null (tree:51a882a2dc62e0d3cdc79e0badc61559fb723481)\n+++ 2553e2d8befbe0cda3e413616fd4cc7bf04157ad/git-relink-script (mode:100644 sha1:78c954edcc370d8be951c856bfbfd38975d08348)\n@@ -0,0 +1,115 @@\n+#!/usr/bin/env perl\n+# Copyright 2005, Ryan Anderson \n+# Distribution permitted under the GPL v2, as distributed\n+# by the Free Software Foundation.\n+# Later versions of the GPL at the discretion of Linus Torvalds\n+#\n+# Scan two git object-trees, and hardlink any common objects between them.\n+\n+use 5.006;\n+use strict;\n+use warnings;\n+\n+sub get_canonical_form($);\n+sub do_scan_directory($$$);\n+sub compare_two_files($$);\n+\n+# stats\n+my $linked = 0;\n+my $already = 0;\n+\n+my ($dir1, $dir2) = @ARGV;\n+\n+if (!defined $dir1 || !defined $dir2) {\n+\tprint(\"Usage: $0 \\nBoth dir1 and dir2 should contain a .git/objects/ subdirectory.\\n\");\n+\texit(1);\n+}\n+\n+$dir1 = get_canonical_form($dir1);\n+$dir2 = get_canonical_form($dir2);\n+\n+printf(\"Searching '%s' and '%s' for common objects and hardlinking them...\\n\",$dir1,$dir2);\n+\n+opendir(D,$dir1 . \"objects/\")\n+\tor die \"Failed to open $dir1/objects/ : $!\";\n+\n+my @hashdirs = grep !/^\\.{1,2}$/, readdir(D);\n+foreach my $hashdir (@hashdirs) {\n+\tdo_scan_directory($dir1, $hashdir, $dir2);\n+}\n+\n+printf(\"Linked %d files, %d were already linked.\\n\",$linked, $already);\n+\n+\n+sub do_scan_directory($$$) {\n+\tmy ($srcdir, $subdir, $dstdir) = @_;\n+\n+\tmy $sfulldir = sprintf(\"%sobjects/%s/\",$srcdir,$subdir);\n+\tmy $dfulldir = sprintf(\"%sobjects/%s/\",$dstdir,$subdir);\n+\n+\topendir(S,$sfulldir)\n+\t\tor die \"Failed to opendir $sfulldir: $!\";\n+\n+\tforeach my $file (grep(!/\\.{1,2}$/, readdir(S))) {\n+\t\tmy $sfilename = $sfulldir . $file;\n+\t\tmy $dfilename = $dfulldir . $file;\n+\n+\t\tcompare_two_files($sfilename,$dfilename);\n+\n+\t}\n+\tclosedir(S);\n+}\n+\n+sub compare_two_files($$) {\n+\tmy ($sfilename, $dfilename) = @_;\n+\n+\t# Perl's stat returns relevant information as follows:\n+\t# 0 = dev number\n+\t# 1 = inode number\n+\t# 7 = size\n+\tmy @sstatinfo = stat($sfilename);\n+\tmy @dstatinfo = stat($dfilename);\n+\n+\tif (@sstatinfo == 0 && @dstatinfo == 0) {\n+\t\tdie sprintf(\"Stat of both %s and %s failed: %s\\n\",$sfilename, $dfilename, $!);\n+\n+\t} elsif (@dstatinfo == 0) {\n+\t\treturn;\n+\t}\n+\n+\tif ( ($sstatinfo[0] == $dstatinfo[0]) &&\n+\t ($sstatinfo[1] != $dstatinfo[1])) {\n+\t\tif ($sstatinfo[7] == $dstatinfo[7]) {\n+\t\t\tunlink($dfilename)\n+\t\t\t\tor die \"Unlink of $dfilename failed: $!\\n\";\n+\n+\t\t\tlink($sfilename,$dfilename)\n+\t\t\t\tor die \"Failed to link $sfilename to $dfilename: $!\\n\" .\n+\t\t\t\t\t\"Git Repository containing $dfilename is probably corrupted, please copy '$sfilename' to '$dfilename' to fix.\\n\";\n+\n+\t\t\t$linked++;\n+\n+\t\t} else {\n+\t\t\tdie sprintf(\"ERROR: File sizes are not the same, cannot relink %s to %s.\\n\",\n+\t\t\t\t$sfilename, $dfilename);\n+\t\t}\n+\n+\t} elsif ( ($sstatinfo[0] == $dstatinfo[0]) &&\n+\t ($sstatinfo[1] == $dstatinfo[1])) {\n+\t\t$already++;\n+\t}\n+}\n+\n+sub get_canonical_form($) {\n+\tmy $dir = shift;\n+\tmy $original = $dir;\n+\n+\tdie \"$dir is not a directory.\" unless -d $dir;\n+\n+\t$dir .= \"/\" unless $dir =~ m#/$#;\n+\t$dir .= \".git/\" unless $dir =~ m#\\.git/$#;\n+\n+\tdie \"$original does not have a .git/ subdirectory.\\n\" unless -d $dir;\n+\n+\treturn $dir;\n+}\n\n-- \n\nRyan Anderson\n sometimes Pug Majere\n"},{"id":"4800","messageId":"20050609192921.GH21076@mythryan2.michonline.com","threadId":"885","inReplyTo":"20050609191548.GG21076@mythryan2.michonline.com","subject":"Re: RFE: git relink","fromName":"Ryan Anderson","fromEmail":"ryan@michonline.com","sentAt":"2005-06-09T19:29:21Z","receivedAt":"2005-06-09T19:29:21Z","isPatch":false,"sender":{"key":"ryan@michonline.com","avatar":null},"body":"On Thu, Jun 09, 2005 at 03:15:48PM -0400, Ryan Anderson wrote:\n> On Thu, Jun 09, 2005 at 02:35:51PM -0400, Jeff Garzik wrote:\n> > \n> > It would be nice if somebody were motivated enough to create a command \n> > that functions like:\n> > \n> > \tgit relink repoA repoB repoC repoD... repoX\n> > \n> > which would examine\n> > \n> > \trepoA/.git\n> > \trepoB/.git\n> > \trepoC/.git\n> > \trepoD/.git\n> > \n> > and verify (updating, if necessary) that each of the A/B/C/D repos are \n> > hardlinked to repoX.\n> \n> Submitted a while ago, dunno what happened with it. This only does 2\n> repositories, but it's trivial to do\n> \tfor i in repoA repoB repoC repoD ; do git-relink-script \"$i\" repoX ; done\n\nAnd of course, I didn't actually look at my code, what my code really\nwants is:\n\n \tfor i in repoA repoB repoC repoD ; do git-relink-script repoX \"$i\" ; done\n\nIt should be relatively trivial to convert it over to the other behavior\nif it matters.\n\n-- \n\nRyan Anderson\n sometimes Pug Majere\n"},{"id":"4849","messageId":"7vy89h36da.fsf@assigned-by-dhcp.cox.net","threadId":"885","inReplyTo":"42A88C07.5050907@pobox.com","subject":"Re: RFE: git relink","fromName":"Junio C Hamano","fromEmail":"junkio@cox.net","sentAt":"2005-06-11T03:44:01Z","receivedAt":"2005-06-11T03:44:01Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":">>>>> \"JG\" == Jeff Garzik writes:\n\nJG> It would be nice if somebody were motivated enough to create a command\nJG> that functions like:\n\nJG> \tgit relink repoA repoB repoC repoD... repoX\n\nJG> which would examine\n\nJG> \trepoA/.git\nJG> \trepoB/.git\nJG> \trepoC/.git\nJG> \trepoD/.git\n\nJG> and verify (updating, if necessary) that each of the A/B/C/D repos are\nJG> hardlinked to repoX.\n\nWhoever is doing this script needs to be a bit careful.\n\nIf you end up unlinking a full object and hard-linking a\ndeltified object representation (delta) in its place, the\nrepository can get corrupted, because it might not have the\nnecessary base object for the delta.\n\nThere are two strategies to solve this. Either (1) the relinker\nrefuses to replace a full object with a delta, or (2) the\nrelinker notices a delta, and makes an additional hard link to\nthe base object when replacing a full object with a delta (this\nneeds to be done recursively until you hit a full base object).\n\n(1) is simpler and cleaner, but does not get full advantage of\nthe delta compression. (2) gives you delta compression but it\nwill add possibly \"unwanted\" objects to a repository that\nhappens to slurp in a delta (fsck would not complain, though).\n\nMy knee-jerk vote goes to (1), but in either case the relinker\nneeds to check if it is dealing with a delta; this cannot be\ndone with git-cat-file -t AFAIK.\n\nRyan Anderson code will notice delta vs full object case most of\nthe time because it checks and makes sure the sizes of\ncorresponding files from two repositories match. The problem\nwith the code is that it dies, instead of just ignoring, when\nsize differs. Dying is good for ordinary case (two full object\nrepresentations of the same file should not have different\nsizes), but it is not the right thing to do when it sees one\nside using the full object representation and the other side\nusing a delta.\n\n"}]}