{"thread":{"id":"271","subject":"git pull issues...","startedAt":"2005-04-23T21:25:32Z","lastAt":"2005-04-25T23:43:26Z","messageCount":6,"participants":["Morten Welinder","Petr Baudis","Dan Holmsand"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"1416","messageId":"118833cc050423142573729ce2@mail.gmail.com","threadId":"271","inReplyTo":null,"subject":"git pull issues...","fromName":"Morten Welinder","fromEmail":"mwelinder@gmail.com","sentAt":"2005-04-23T21:25:32Z","receivedAt":"2005-04-23T21:25:32Z","isPatch":false,"sender":{"key":"mwelinder@gmail.com","avatar":null},"body":"Minor issues with \"git pull\":\n\n1. Multiple rsync call might connect to different servers (with\nround-robin DNS).  The effect\n   will be interesting.  One call, if possible, would be better.\n\n2. If any rsync fails, the process should be aborted.\n\nAs long as we do have multiple calls:\n\n3. The \"grep -v '^MOTD:'\" should be changed to kill blank lines after\nMOTDs also.\n\nMorten\n"},{"id":"1420","messageId":"20050423220049.GC13222@pasky.ji.cz","threadId":"271","inReplyTo":"118833cc050423142573729ce2@mail.gmail.com","subject":"Re: git pull issues...","fromName":"Petr Baudis","fromEmail":"pasky@ucw.cz","sentAt":"2005-04-23T22:00:50Z","receivedAt":"2005-04-23T22:00:50Z","isPatch":false,"sender":{"key":"pasky@ucw.cz","avatar":"https://avatars.githubusercontent.com/u/18439?v=4"},"body":"Dear diary, on Sat, Apr 23, 2005 at 11:25:32PM CEST, I got a letter\nwhere Morten Welinder <mwelinder@gmail.com> told me that...\n> Minor issues with \"git pull\":\n> \n> 1. Multiple rsync call might connect to different servers (with\n> round-robin DNS).  The effect\n>    will be interesting.  One call, if possible, would be better.\n\nIf you can do it without overwriting HEAD, please go ahead and send me\nthe patch. :-)\n\n> 2. If any rsync fails, the process should be aborted.\n\nFixed.\n\n> As long as we do have multiple calls:\n> \n> 3. The \"grep -v '^MOTD:'\" should be changed to kill blank lines after\n> MOTDs also.\n\nActually, I currently use this as a poor man's progress indicator. ;-)\n*hide*\n\n-- \n\t\t\t\tPetr \"Pasky\" Baudis\nStuff: http://pasky.or.cz/\nC++: an octopus made by nailing extra legs onto a dog. -- Steve Taylor\n"},{"id":"1450","messageId":"118833cc05042317391a441448@mail.gmail.com","threadId":"271","inReplyTo":"20050423220049.GC13222@pasky.ji.cz","subject":"Re: git pull issues...","fromName":"Morten Welinder","fromEmail":"mwelinder@gmail.com","sentAt":"2005-04-24T00:39:17Z","receivedAt":"2005-04-24T00:39:17Z","isPatch":false,"sender":{"key":"mwelinder@gmail.com","avatar":null},"body":"> > 1. Multiple rsync call might connect to different servers (with\n> > round-robin DNS).  The effect\n> >    will be interesting.  One call, if possible, would be better.\n> \n> If you can do it without overwriting HEAD, please go ahead and send me\n> the patch. :-)\n\nI'll have a go at it later, but something like this ought to work:\n\n-d .rsync-git && die \"previous pull failed -- cleanup\"\nmkdir .rsync-git || die \"cannot create .rsync-git\"\nln -s ../.git/objects .rsync-git/objects || die \"cannot create symlink\"\nrsync ... --keep-dirlinks ...\n...\n\nMorten\n"},{"id":"1652","messageId":"118833cc05042514076221624d@mail.gmail.com","threadId":"271","inReplyTo":"20050423220049.GC13222@pasky.ji.cz","subject":"Re: git pull issues...","fromName":"Morten Welinder","fromEmail":"mwelinder@gmail.com","sentAt":"2005-04-25T21:07:12Z","receivedAt":"2005-04-25T21:07:12Z","isPatch":false,"sender":{"key":"mwelinder@gmail.com","avatar":null},"body":"On 4/23/05, Petr Baudis <pasky@ucw.cz> wrote:\n> > 1. Multiple rsync call might connect to different servers (with\n> > round-robin DNS).  The effect\n> >    will be interesting.  One call, if possible, would be better.\n> \n> If you can do it without overwriting HEAD, please go ahead and send me\n> the patch. :-)\n\nThis ought to work.  It basically saves to a different directory while\nstill ignoring\nall the files we have.\n\nMorten\n\n\n\n\nIndex: gitpull.sh\n===================================================================\n--- 7de71a831508e51e0985cea173f3f7a7012c82b7/gitpull.sh  (mode:100755\nsha1:6abc7f5c00fd3e082d0a34a238a53b67c38b8a7f)\n+++ uncommitted/gitpull.sh  (mode:100775)\n@@ -37,28 +37,32 @@\n \t[ -s \".git/heads/$name\" ] && orig_head=$(cat \".git/heads/$name\")\n fi\n \n-\n-mkdir -p .git/heads\n-rsyncerr=\n-rsync $RSYNC_FLAGS -Lr \"$uri/heads/$rembranch\" \".git/heads/$name\"\n2>/dev/null || rsyncerr=1\n-if [ \"$rsyncerr\" ] && [ \"$rembranch\" = \"master\" ]; then\n-\trsyncerr=\n-\trsync $RSYNC_FLAGS -Lr \"$uri/HEAD\" \".git/heads/$name\" | grep -v\n'^MOTD:' || rsyncerr=1\n+dotgit=dot-git\n+test -d \"$dotgit\" && die \"$dotgit already exists -- please clean up.\"\n+mkdir -p \"$dotgit\" \"$dotgit/objects\" \"$dotgit/tags\" \\\n+\t|| die \"failed to create $dotgit.\"\n+\n+(cd .git && find objects -type f -print0 && find tags -type f -print0) | \\\n+\trsync $RSYNC_FLAGS --from0 --exclude-from=- --whole-file -Lrv\n\"$uri/\" \"$dotgit\" || \\\n+\t{ rm -rf \"$dotgit\"; die \"rsync from $uri/ failed.\"; }\n+\n+# Move new files into place\n+find \"$dotgit/objects\" -type f -print0 | \\\n+\tperl -n0e 'chomp; $src=$_; s{^[^/]+/}{.git/}; rename $src, $_ or die\n\"failed to move $src into place: $!\\n\";'\n+find \"$dotgit/tags\" -type f -print0 | \\\n+\tperl -n0e 'chomp; $src=$_; s{^[^/]+/}{.git/}; rename $src, $_ or die\n\"failed to move $src into place: $!\\n\";'\n+\n+mkdir -p .git/heads || die \"failed to create .git/heads\"\n+if [ -r \"$dotgit/heads/$rembranch\" ]; then\n+\tmv \"$dotgit/heads/$rembranch\" \".git/heads/$name\"\n+elif [ \"$rembranch\" = \"master\" -a -r \"$dotgit/HEAD\" ]; then\n+\tmv \"$dotgit/HEAD\" \".git/heads/$name\"\n+else\n+\trm -rf \"$dotgit\"\n+\tdie \"unable to get the head pointer of branch $rembranch\"\n fi\n-[ \"$rsyncerr\" ] && die \"unable to get the head pointer of branch $rembranch\"\n-\n-[ -d .git/objects ] || mkdir -p .git/objects\n-# We already saw the MOTD, thank you very much.\n-rsync $RSYNC_FLAGS --ignore-existing --whole-file \\\n-\t-v -Lr \"$uri/objects/.\" \".git/objects/.\" | grep -v '^MOTD:' || die\n\"rsync error\"\n-\n-# FIXME: Warn about conflicting tag names?\n-# XXX: We now throw stderr to /dev/null since not all repositories\n-# may have tags/ and users were confused by the harmless errors.\n-[ -d .git/tags ] || mkdir -p .git/tags\n-rsync $RSYNC_FLAGS --ignore-existing \\\n-\t-v -Lr \"$uri/tags/.\" \".git/tags/.\" 2>/dev/null | grep -v '^MOTD:' ||\ndie \"rsync error\"\n \n+rm -rf \"$dotgit\"\n \n new_head=$(cat \".git/heads/$name\")\n"},{"id":"1657","messageId":"d4jn91$n4f$1@sea.gmane.org","threadId":"271","inReplyTo":"118833cc05042514076221624d@mail.gmail.com","subject":"Re: git pull issues...","fromName":"Dan Holmsand","fromEmail":"holmsand@gmail.com","sentAt":"2005-04-25T21:31:02Z","receivedAt":"2005-04-25T21:31:02Z","isPatch":false,"sender":{"key":"holmsand@gmail.com","avatar":"https://gravatar.com/avatar/5c722084bafd85e754a02efad01fe69107eb6f393253c49232c5c9f7faa974df?d=mp&s=160"},"body":"Morten Welinder wrote:\n> On 4/23/05, Petr Baudis <pasky@ucw.cz> wrote:\n> \n>>>1. Multiple rsync call might connect to different servers (with\n>>>round-robin DNS).  The effect\n>>>   will be interesting.  One call, if possible, would be better.\n>>\n>>If you can do it without overwriting HEAD, please go ahead and send me\n>>the patch. :-)\n> \n> \n> This ought to work.  It basically saves to a different directory while\n> still ignoring\n> all the files we have.\n\n\nI was just looking at the same thing. Not saying this is better, or \nanything, bit it has, perhaps, the slight advantage that downloaded \nobjects aren't thrown away, so that a pull can be safely resumed. Also, \nit doesn't download anything from an url that doesn't contain ./HEAD, \n./heads or ./objects. That might be a good thing if you accidentally try \nto pull from, say, kernel.org/pub/...\n\n/dan\n\n\nHere goes:\n\nIndex: gitpull.sh\n===================================================================\n--- 7de71a831508e51e0985cea173f3f7a7012c82b7/gitpull.sh  (mode:100755 \nsha1:6abc7f5c00fd3e082d0a34a238a53b67c38b8a7f)\n+++ uncommitted/gitpull.sh  (mode:100755)\n@@ -18,7 +18,7 @@\n  [ \"$name\" ] || name=$(cat .git/tracking 2>/dev/null)\n  [ \"$name\" ] || die \"where to pull from?\"\n  uri=$(grep $(echo -e \"^$name\\t\" | sed 's/\\./\\\\./g') .git/remotes | cut \n-f 2)\n-[ \"$uri\" ] || die \"unknown remote\"\n+[ \"$uri\" ] || die \"unknown remote '$name'\"\n\n  rembranch=master\n  if echo \"$uri\" | grep -q '#'; then\n@@ -38,28 +38,39 @@\n  fi\n\n\n-mkdir -p .git/heads\n-rsyncerr=\n-rsync $RSYNC_FLAGS -Lr \"$uri/heads/$rembranch\" \".git/heads/$name\" \n2>/dev/null || rsyncerr=1\n-if [ \"$rsyncerr\" ] && [ \"$rembranch\" = \"master\" ]; then\n-\trsyncerr=\n-\trsync $RSYNC_FLAGS -Lr \"$uri/HEAD\" \".git/heads/$name\" | grep -v \n'^MOTD:' || rsyncerr=1\n+mkdir -p .git/heads .git/objects .git/tags || die\n+pulldir=$(mktemp -td gitpull.XXXXXX)\n+[ -d \"$pulldir\" ] || die \"failed to create temp dir\"\n+ln -s $(pwd)/.git/objects $(pwd)/.git/tags $pulldir || die\n+mkdir $pulldir/heads || die\n+\n+echo pulling $uri\n+\n+rout=$pulldir/rsync_errors\n+rsync $RSYNC_FLAGS -LKrv --whole-file --ignore-existing \\\n+    --include '/HEAD' --include '/heads/' --include '/tags/' \\\n+    --include '/objects/' --exclude='/*' \\\n+    \"$uri/.\" $pulldir 2>$rout || {\n+\tcat $rout; die \"failed to pull from $uri\"\n+    }\n+\n+if [ -f \"$pulldir/heads/$rembranch\" ]; then\n+    cp \"$pulldir/heads/$rembranch\" \".git/heads/$name\" || die\n+elif [ master = \"$rembranch\" -a -f $pulldir/HEAD ]; then\n+    cp $pulldir/HEAD \".git/heads/$name\" || die\n+else\n+    echo \"unable to get the head pointer of branch $rembranch\" >&2\n+    echo \"is $uri a valid git repository?\" >&2\n+    die \"pull failed\"\n  fi\n-[ \"$rsyncerr\" ] && die \"unable to get the head pointer of branch \n$rembranch\"\n\n-[ -d .git/objects ] || mkdir -p .git/objects\n-# We already saw the MOTD, thank you very much.\n-rsync $RSYNC_FLAGS --ignore-existing --whole-file \\\n-\t-v -Lr \"$uri/objects/.\" \".git/objects/.\" | grep -v '^MOTD:' || die \n\"rsync error\"\n+# TODO: Check that at least the head commit exists.\n\n-# FIXME: Warn about conflicting tag names?\n-# XXX: We now throw stderr to /dev/null since not all repositories\n-# may have tags/ and users were confused by the harmless errors.\n-[ -d .git/tags ] || mkdir -p .git/tags\n-rsync $RSYNC_FLAGS --ignore-existing \\\n-\t-v -Lr \"$uri/tags/.\" \".git/tags/.\" 2>/dev/null | grep -v '^MOTD:' || \ndie \"rsync error\"\n+rm -rf $pulldir\n\n\n+# FIXME: Warn about conflicting tag names?\n+\n  new_head=$(cat \".git/heads/$name\")\n\n  if [ ! \"$orig_head\" ]; then\n\n"},{"id":"1676","messageId":"118833cc050425164311f0bbbc@mail.gmail.com","threadId":"271","inReplyTo":"d4jn91$n4f$1@sea.gmane.org","subject":"Re: git pull issues...","fromName":"Morten Welinder","fromEmail":"mwelinder@gmail.com","sentAt":"2005-04-25T23:43:26Z","receivedAt":"2005-04-25T23:43:26Z","isPatch":false,"sender":{"key":"mwelinder@gmail.com","avatar":null},"body":"The two approaches are very much alike -- there are only so many ways to\nskin a cat, after all.\n\nI actually first considered the -K approach, but then realized that only very\nnew rsyncs support that.  My version works with rsync 2.6.2 (and maybe\nearlier).  It could be made to go further back by not using -from0 which isn't\nreally needed here.\n\nBut I like your check-for-HEAD detail.\n\nMorten\n"}]}