{"thread":{"id":"5612","subject":"[PATCH] Added --mirror-all to git-fetch.","startedAt":"2006-09-19T23:28:51Z","lastAt":"2006-09-20T21:42:03Z","messageCount":14,"participants":["Shawn Pearce","Petr Baudis","Junio C Hamano","A Large Angry SCM"],"isPatch":true,"patchVersion":1,"patchTotal":null},"messages":[{"id":"27198","messageId":"20060919232851.GA12195@spearce.org","threadId":"5612","inReplyTo":null,"subject":"[PATCH] Added --mirror-all to git-fetch.","fromName":"Shawn Pearce","fromEmail":"spearce@spearce.org","sentAt":"2006-09-19T23:28:51Z","receivedAt":"2006-09-19T23:28:51Z","isPatch":true,"sender":{"key":"spearce@spearce.org","avatar":"https://avatars.githubusercontent.com/u/34844?v=4"},"body":"The --mirror-all option to git-fetch can be used to obtain a copy of\nevery available remote ref into the current repository. This can be\na rather destructive update as the local repository will have its\nHEAD ref overwritten, as well as any ref which it shares in common\nwith the remote repository. On the other hand it can be useful if\nall you want to do is have this repository track another repository,\nsuch as if you are providing Git repository hosting and mirroring\nsource repositories on other systems.\n\nCurrently local refs are not deleted even if they do not exist in the\nremote repository. This may be taken as either a feature or a bug.\n\nSigned-off-by: Shawn O. Pearce \n---\n pasky was asking about this on #git. So here it is.\n\n Documentation/fetch-options.txt | 7 ++++++\n git-fetch.sh | 43 ++++++++++++++++++++++++++++++++++++---\n 2 files changed, 47 insertions(+), 3 deletions(-)\n\ndiff --git a/Documentation/fetch-options.txt b/Documentation/fetch-options.txt\nindex 13f34d3..5ed9b4f 100644\n--- a/Documentation/fetch-options.txt\n+++ b/Documentation/fetch-options.txt\n@@ -30,6 +30,13 @@\n \tflag lets all tags and their associated objects be\n \tdownloaded.\n \n+\\--mirror-all::\n+\tAll refs and tags on the remote side are downloaded to\n+\tthe local side. This option is not intendend for user\n+\trepositories as it will overwrite every local ref,\n+\tincluding HEAD. You probably don't mean to use this\n+\toption.\n+\n -k, \\--keep::\n \tKeep downloaded pack.\n \ndiff --git a/git-fetch.sh b/git-fetch.sh\nindex 09a5d6c..d22c560 100755\n--- a/git-fetch.sh\n+++ b/git-fetch.sh\n@@ -14,6 +14,7 @@ IFS=\"$LF\"\n rloga=fetch\n no_tags=\n tags=\n+mirror_all=\n append=\n force=\n verbose=\n@@ -36,6 +37,9 @@ do\n \t-f|--f|--fo|--for|--forc|--force)\n \t\tforce=t\n \t\t;;\n+\t--mirror-all)\n+\t\tmirror_all=t\n+\t\t;;\n \t-t|--t|--ta|--tag|--tags)\n \t\ttags=t\n \t\t;;\n@@ -216,12 +220,45 @@ case \"$update_head_ok\" in\n \t;;\n esac\n \n-# If --tags (and later --heads or --all) is specified, then we are\n-# not talking about defaults stored in Pull: line of remotes or\n+# If --tags or -mirror_all (and later --heads) is specified, then we\n+# are not talking about defaults stored in Pull: line of remotes or\n # branches file, and just fetch those and refspecs explicitly given.\n # Otherwise we do what we always did.\n \n-reflist=$(get_remote_refs_for_fetch \"$@\")\n+if test \"$mirror_all\"\n+then\n+\ttags=0; # tags are implied in all \n+\treflist=`IFS=\"\t\" &&\n+\t\t (\n+\t\t\tgit-ls-remote $upload_pack \"$remote\" ||\n+\t\t\techo fail ouch\n+\t\t ) |\n+\t while read sha1 name\n+\t\t do\n+\t\t\tcase \"$sha1\" in\n+\t\t\tfail)\n+\t\t\t\texit 1\n+\t\t\tesac\n+\t\t\tcase \"$name\" in\n+\t\t\t*^*) continue ;;\n+\t\t\tesac\n+\t\t \tif git-check-ref-format \"$name\"\n+\t\t\tthen\n+\t\t\t echo \".+${name}:${name}\"\n+\t\t\telse\n+\t\t\t\tif test \"x$name\" = xHEAD\n+\t\t\t\tthen\n+\t\t\t\t\techo \".+${name}:${name}\"\n+\t\t\t\telse\n+\t\t\t\t\techo >&2 \"warning: malformed ref ${name} ignored\"\n+\t\t\t\tfi\n+\t\t\tfi\n+\t\t done` || exit\n+else\n+\t# Not using all; do what we always have done.\n+\treflist=$(get_remote_refs_for_fetch \"$@\")\n+fi\n+\n if test \"$tags\"\n then\n \ttaglist=`IFS=\"\t\" &&\n-- \n1.4.2.1.g832e\n"},{"id":"27199","messageId":"20060919234108.GH8259@pasky.or.cz","threadId":"5612","inReplyTo":"20060919232851.GA12195@spearce.org","subject":"Re: [PATCH] Added --mirror-all to git-fetch.","fromName":"Petr Baudis","fromEmail":"pasky@suse.cz","sentAt":"2006-09-19T23:41:08Z","receivedAt":"2006-09-19T23:41:08Z","isPatch":true,"sender":{"key":"pasky@ucw.cz","avatar":"https://avatars.githubusercontent.com/u/18439?v=4"},"body":"Dear diary, on Wed, Sep 20, 2006 at 01:28:51AM CEST, I got a letter\nwhere Shawn Pearce said that...\n> Currently local refs are not deleted even if they do not exist in the\n> remote repository. This may be taken as either a feature or a bug.\n\n It's a bug for my purposes, but a minor one.\n\n> pasky was asking about this on #git. So here it is.\n\n Thanks!\n\n> @@ -216,12 +220,45 @@ case \"$update_head_ok\" in\n> \t;;\n> esac\n> \n> -# If --tags (and later --heads or --all) is specified, then we are\n> -# not talking about defaults stored in Pull: line of remotes or\n> +# If --tags or -mirror_all (and later --heads) is specified, then we\n ^^ missing dash ;-)\n> +# are not talking about defaults stored in Pull: line of remotes or\n> # branches file, and just fetch those and refspecs explicitly given.\n> # Otherwise we do what we always did.\n> \n> -reflist=$(get_remote_refs_for_fetch \"$@\")\n> +if test \"$mirror_all\"\n> +then\n> +\ttags=0; # tags are implied in all \n\n This should be tags=;\n\n-- \n\t\t\t\tPetr \"Pasky\" Baudis\nStuff: http://pasky.or.cz/\nSnow falling on Perl. White noise covering line noise.\nHides all the bugs too. -- J. Putnam\n"},{"id":"27241","messageId":"7vac4ujzf0.fsf@assigned-by-dhcp.cox.net","threadId":"5612","inReplyTo":"20060919232851.GA12195@spearce.org","subject":"Re: [PATCH] Added --mirror-all to git-fetch.","fromName":"Junio C Hamano","fromEmail":"junkio@cox.net","sentAt":"2006-09-20T16:06:11Z","receivedAt":"2006-09-20T16:06:11Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Shawn Pearce writes:\n\n> The --mirror-all option to git-fetch can be used to obtain a copy of\n> every available remote ref into the current repository. This can be\n> a rather destructive update as the local repository will have its\n> HEAD ref overwritten, as well as any ref which it shares in common\n> with the remote repository.\n\nI can sort of see where something like this is very much useful,\nbut it sounds like a tool quite different from git-fetch.\n\n (1) I really do not like rolling this kind of speciale purpose\n command into git-fetch, which is frequently used by end\n users, to avoid mistakes.\n\n If there is reluctance against adding yet another new\n command (and there certainly is), this feels more like a\n cousin of \"git-clone --bare\".\n\n (2) Although there is no inherent reason not allowing a working\n tree associated with the repository that is kept updated\n this way, the user will be utterly confused in a working\n tree whose current branch head is updated like this, until\n the working tree and the index is matched to the updated\n HEAD. It might be reasonable to run checkout -f HEAD when\n a working tree is associated with the repository (the\n command is screwing over even the current branch HEAD, so\n losing what happened to be in the working tree is really\n not an issue), but as a much easier safety measure we might\n want to allow this mode of updating only on a bare\n repository (that is, .git/index should not exist).\n\n (3) This feels primarily meant for something like Pasky is\n trying to run --- mirrored distribution point of git\n repositories perhaps displayed with gitweb. When updating\n such a repository, you would want to do things like running\n update-server-info and automatically repacking the object\n store. Especially the latter would be an interesting topic\n (the archive vs active repacking strategy we talked about,\n combined with set of packs with staggered spans to help\n commit walkers Pasky talked about quite a while ago).\n\n\n> ... \n> such as if you are providing Git repository hosting and mirroring\n> source repositories on other systems.\n>\n> Currently local refs are not deleted even if they do not exist in the\n> remote repository. This may be taken as either a feature or a bug.\n\nFor that purpose I would say that is definitely a bug.\n"},{"id":"27246","messageId":"20060920161407.GQ8259@pasky.or.cz","threadId":"5612","inReplyTo":"7vac4ujzf0.fsf@assigned-by-dhcp.cox.net","subject":"Re: [PATCH] Added --mirror-all to git-fetch.","fromName":"Petr Baudis","fromEmail":"pasky@suse.cz","sentAt":"2006-09-20T16:14:07Z","receivedAt":"2006-09-20T16:14:07Z","isPatch":true,"sender":{"key":"pasky@ucw.cz","avatar":"https://avatars.githubusercontent.com/u/18439?v=4"},"body":"Dear diary, on Wed, Sep 20, 2006 at 06:06:11PM CEST, I got a letter\nwhere Junio C Hamano said that...\n..snip..\n> If there is reluctance against adding yet another new\n> command (and there certainly is), this feels more like a\n> cousin of \"git-clone --bare\".\n\nCertainly. But, what exactly are you proposing here? :-) (Besides\npossible change of the switch name.) Making this a git-clone option\nsounds even much worse.\n\n..snip..\n> (2) Although there is no inherent reason not allowing a working\n> tree associated with the repository that is kept updated\n> this way, the user will be utterly confused in a working\n> tree whose current branch head is updated like this, until\n> the working tree and the index is matched to the updated\n> HEAD.\n\ngit-fetch --mirror-all can still be very useful even with a working copy\nif you are keeping all the remote heads in .git/refs/remotes/. I'd\nventure to say that in that case, this is frequently what you actually\nwant when fetching from the repository (given that you have already let\ngit clone do that once).\n\n..snip..\n> (the archive vs active repacking strategy we talked about,\n\nHmm, I think I've missed this, I must look that in the archive.\n\n> combined with set of packs with staggered spans to help\n> commit walkers Pasky talked about quite a while ago).\n\nYes, this is certainly one of things I would like to implement at\nrepo.or.cz.\n\n-- \n\t\t\t\tPetr \"Pasky\" Baudis\nStuff: http://pasky.or.cz/\nSnow falling on Perl. White noise covering line noise.\nHides all the bugs too. -- J. Putnam\n"},{"id":"27249","messageId":"20060920162145.GA23260@spearce.org","threadId":"5612","inReplyTo":"20060920161407.GQ8259@pasky.or.cz","subject":"Re: [PATCH] Added --mirror-all to git-fetch.","fromName":"Shawn Pearce","fromEmail":"spearce@spearce.org","sentAt":"2006-09-20T16:21:45Z","receivedAt":"2006-09-20T16:21:45Z","isPatch":true,"sender":{"key":"spearce@spearce.org","avatar":"https://avatars.githubusercontent.com/u/34844?v=4"},"body":"Petr Baudis wrote:\n> Dear diary, on Wed, Sep 20, 2006 at 06:06:11PM CEST, I got a letter\n> where Junio C Hamano said that...\n[snip]\n> > (2) Although there is no inherent reason not allowing a working\n> > tree associated with the repository that is kept updated\n> > this way, the user will be utterly confused in a working\n> > tree whose current branch head is updated like this, until\n> > the working tree and the index is matched to the updated\n> > HEAD.\n> \n> git-fetch --mirror-all can still be very useful even with a working copy\n> if you are keeping all the remote heads in .git/refs/remotes/. I'd\n> venture to say that in that case, this is frequently what you actually\n> want when fetching from the repository (given that you have already let\n> git clone do that once).\n\nI think that's more `git fetch --all`. You are pulling every remote\nhead available at a given remote into a subdirectory of your own ref\nspace. That's rather different than replacing your entire ref space\nwith the one available on the remote, which is what --mirror-all\nis doing and what you wanted for the hosting you are offering.\n\n> ..snip..\n> > (the archive vs active repacking strategy we talked about,\n> \n> Hmm, I think I've missed this, I must look that in the archive.\n\nJunio pushed the core code out but nobody has done the Porecelain\nfor it. The basic idea is to prevent repacking every pack all of\nthe time; there's probably no reason to repack a 100 MiB pack file\nevery time you repack your loose objects so you might want to keep\nsay a <5 MiB \"active pack\" holding your recent created objects\nand repack that frequently and a larger 100+ MiB \"history pack\"\nholding everything else. Maybe you repack everything on a longer\ntime scale, such as once a year.\n\n> > combined with set of packs with staggered spans to help\n> > commit walkers Pasky talked about quite a while ago).\n> \n> Yes, this is certainly one of things I would like to implement at\n> repo.or.cz.\n\nBorrowing your line:\nHmm, I think I've missed this, I must look that in the archive.\n\n:-)\n\n-- \nShawn.\n"},{"id":"27253","messageId":"7v1wq6jy3q.fsf@assigned-by-dhcp.cox.net","threadId":"5612","inReplyTo":"20060920162145.GA23260@spearce.org","subject":"Re: [PATCH] Added --mirror-all to git-fetch.","fromName":"Junio C Hamano","fromEmail":"junkio@cox.net","sentAt":"2006-09-20T16:34:33Z","receivedAt":"2006-09-20T16:34:33Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Shawn Pearce writes:\n\n> Petr Baudis wrote:\n>> Dear diary, on Wed, Sep 20, 2006 at 06:06:11PM CEST, I got a letter\n>> where Junio C Hamano said that...\n> [snip]\n>> > (2) Although there is no inherent reason not allowing a working\n>> > tree associated with the repository that is kept updated\n>> > this way, the user will be utterly confused in a working\n>> > tree whose current branch head is updated like this, until\n>> > the working tree and the index is matched to the updated\n>> > HEAD.\n>> \n>> git-fetch --mirror-all can still be very useful even with a working copy\n>> if you are keeping all the remote heads in .git/refs/remotes/. I'd\n>> venture to say that in that case, this is frequently what you actually\n>> want when fetching from the repository (given that you have already let\n>> git clone do that once).\n>\n> I think that's more `git fetch --all`. You are pulling every remote\n> head available at a given remote into a subdirectory of your own ref\n> space. That's rather different than replacing your entire ref space\n> with the one available on the remote, which is what --mirror-all\n> is doing and what you wanted for the hosting you are offering.\n\nI realize I am going around in circles, but Pasky's \"remotes/\"\nargument made me realize that this mirroring is not much more\nthan \"fetch --force --all\". I initially had an impression that\nthis was for only strict mirroring where you do not even want\nyour own \"origin\", but if you arrange the .git/remotes/origin\nfile the right way, \"fetch --force --all\" (if you remembered to\nput '+' in front of the refspecs, even without --force) would\nwhat --mirror-all would do wouldn't it?\n\nHaving said that, I am not sure if \"fetch --all\" should delete\nbranches that do not exist on the remote end. For mirroring, I\ndefinitely would want the tool to delete them, and that is one\nof the reason I feel this is much more similar to git-clone than\ngit-fetch, unless it is a separate command (perhaps git-mirror).\n"},{"id":"27260","messageId":"20060920164912.GD23260@spearce.org","threadId":"5612","inReplyTo":"7v1wq6jy3q.fsf@assigned-by-dhcp.cox.net","subject":"Re: [PATCH] Added --mirror-all to git-fetch.","fromName":"Shawn Pearce","fromEmail":"spearce@spearce.org","sentAt":"2006-09-20T16:49:12Z","receivedAt":"2006-09-20T16:49:12Z","isPatch":true,"sender":{"key":"spearce@spearce.org","avatar":"https://avatars.githubusercontent.com/u/34844?v=4"},"body":"Junio C Hamano wrote:\n> I realize I am going around in circles, but Pasky's \"remotes/\"\n> argument made me realize that this mirroring is not much more\n> than \"fetch --force --all\". I initially had an impression that\n> this was for only strict mirroring where you do not even want\n> your own \"origin\", but if you arrange the .git/remotes/origin\n> file the right way, \"fetch --force --all\" (if you remembered to\n> put '+' in front of the refspecs, even without --force) would\n> what --mirror-all would do wouldn't it?\n\nI started this change with '--all' and realized that ideally you\nwant '--all' to copy all available refs/heads/* from the remote to\nrefs/remotes//* here. You want to create any new branches\nwhich the remote has introduced since your last fetch.\n\nYou probably don't want to force a non-fast forward unless there's a\n'+' in the corresponding Pull line of remotes/ or if --force is\nused. However you probably also want to delete any removed branches.\n\nWhich I think is quite different from a mirror. A mirror wants to\nreplace the entire ref namespace with what's on the remote as it\nhas no need for a local namespace of its own.\n\n\nOriginally I gave Pasky a one-liner on #git:\n\n git fetch --force origin $(git ls-remote origin \\\n | awk '{if(!/\\^{}$/){print $2\":\"$2}}')\n\nbut he expressed interest in it being a native feature of the\ncore-Git fetch Porcelain. To be honest I disagreed with him but\nsubmitted the patch anyway.\n\nI think --all copying into .git/refs/remotes//* makes perfect\nsense.\n\nAnd I think this mirror thing may make more sense as a small wrapper\naround git-fetch. A wrapper that checks for:\n\n\t- its running in a bare repository;\n\t- it has a single remote named origin;\n\t- HEAD isn't a symlink or a symref (its a normal ref in its\n\t own right);\n\t- git-mirror.permitted is true in the config file.\n \n-- \nShawn.\n"},{"id":"27266","messageId":"7vac4uihr3.fsf@assigned-by-dhcp.cox.net","threadId":"5612","inReplyTo":"20060920164912.GD23260@spearce.org","subject":"Re: [PATCH] Added --mirror-all to git-fetch.","fromName":"Junio C Hamano","fromEmail":"junkio@cox.net","sentAt":"2006-09-20T17:13:04Z","receivedAt":"2006-09-20T17:13:04Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Shawn Pearce writes:\n\n> I started this change with '--all' and realized that ideally you\n> want '--all' to copy all available refs/heads/* from the remote to\n> refs/remotes//* here. You want to create any new branches\n> which the remote has introduced since your last fetch.\n>\n> You probably don't want to force a non-fast forward unless there's a\n> '+' in the corresponding Pull line of remotes/ or if --force is\n> used. However you probably also want to delete any removed branches.\n\nI think we are in agreement here.\n\n> Which I think is quite different from a mirror. A mirror wants to\n> replace the entire ref namespace with what's on the remote as it\n> has no need for a local namespace of its own.\n\nIf you have remotes/origin that does not use the refs/remotes\nhierarchy, does not copy master->origin (iow, the assumption\nthat \"origin\" branch is to be merged into local \"master\") but\ncopies remote master to local master, and forces all refs by\ndefault, then your local ref namespace is effectively a mirror\nof the remote, so it is very tempting to think --mirror is a\nlogical extension of \"fetch --all\".\n\nExcept that new refs are not created and old refs are not\ndeleted. And these two are rather big exceptions.\n\nIf we are to have another command (git-mirror which is a thin\nwrapper around git-fetch as you propose later), these two\nexceptions would be cleanly handled there. Otherwise, we would\nneed to give the command a hint (git-mirror.permitted) to allow\nthem to be done in addition to the usual \"fetch --all\", but then\nwe would end up doing the equivalent of your one-liner anyway,\nso there is not much point having it in git-fetch. So I am in\nfavor of your thin-wrapper idea.\n\n> Originally I gave Pasky a one-liner on #git:\n>\n> git fetch --force origin $(git ls-remote origin \\\n> | awk '{if(!/\\^{}$/){print $2\":\"$2}}')\n\nWhich sounds a very sane thing to do, especially given git is\ndesigned to be scripted this way.\n\n> And I think this mirror thing may make more sense as a small wrapper\n> around git-fetch. A wrapper that checks for:\n>\n> \t- its running in a bare repository;\n\nOk.\n\n> \t- it has a single remote named origin;\n\nDo you mean \".git/remotes must have only one file 'origin' and\nnothing else\", or do you mean \"There must be .git/remotes/foobla\nfor 'git-mirror foobla' to work\"?\n\n> \t- HEAD isn't a symlink or a symref (its a normal ref in its\n> \t own right);\n\nWhy? Usually HEAD is a pointer to the active branch. I\nconsider it is a bug in the upload-pack protocol not to convey\nthat information (but that is a separate topic).\n\n> \t- git-mirror.permitted is true in the config file.\n\nOk.\n"},{"id":"27269","messageId":"20060920173126.GG23260@spearce.org","threadId":"5612","inReplyTo":"7vac4uihr3.fsf@assigned-by-dhcp.cox.net","subject":"Re: [PATCH] Added --mirror-all to git-fetch.","fromName":"Shawn Pearce","fromEmail":"spearce@spearce.org","sentAt":"2006-09-20T17:31:26Z","receivedAt":"2006-09-20T17:31:26Z","isPatch":true,"sender":{"key":"spearce@spearce.org","avatar":"https://avatars.githubusercontent.com/u/34844?v=4"},"body":"Junio C Hamano wrote:\n> Shawn Pearce writes:\n> > Originally I gave Pasky a one-liner on #git:\n> >\n> > git fetch --force origin $(git ls-remote origin \\\n> > | awk '{if(!/\\^{}$/){print $2\":\"$2}}')\n> \n> Which sounds a very sane thing to do, especially given git is\n> designed to be scripted this way.\n\nExactly. So I'm not sure why he was against it. After all he must\nhave some script that's looping over all mirrored repositories and\ninvoking git fetch in them. :-)\n \n> > \t- it has a single remote named origin;\n> \n> Do you mean \".git/remotes must have only one file 'origin' and\n> nothing else\", or do you mean \"There must be .git/remotes/foobla\n> for 'git-mirror foobla' to work\"?\n\nI meant \".git/remotes must have only one file 'origin' and\nnothing else\". Because what does it mean to mirror two different\nrepositories at the same time into the same ref namespace? The most\nrecently fetched would always win.\n\nBut now that I'm writing this I can see someone wanting multiple\nremotes so they could store different URLs for the same repository\nand call fetch on another if the first isn't responding. So I\nretract this entirely. So there doesn't even need to be remotes\nfile; the URL could just be passed on the command line. A remotes\nfile (like the default \"origin\" one) is just a handy place to store\nthe URL.\n \n> > \t- HEAD isn't a symlink or a symref (its a normal ref in its\n> > \t own right);\n> \n> Why? Usually HEAD is a pointer to the active branch. I\n> consider it is a bug in the upload-pack protocol not to convey\n> that information (but that is a separate topic).\n\nBecause we don't want to update HEAD with the SHA1 that's on the\nremote if that's actually going to update some other ref which\ndoesn't have the same value. :-)\n\nI think its simple enough to verify HEAD isn't a symlink or symref\n(or just doesn't exist) early in the mirror script. If the user\nreally wants to do a mirror and they just did a `git init-db --bare`\nthey could also do a `rm HEAD` before starting `git-mirror`.\n\nBut I agree with you entirely about HEAD being a SHA1 and not its\nactual link target in the network protocol. An annoying bug.\n\n-- \nShawn.\n"},{"id":"27271","messageId":"7vu032h1g1.fsf@assigned-by-dhcp.cox.net","threadId":"5612","inReplyTo":"20060920173126.GG23260@spearce.org","subject":"Re: [PATCH] Added --mirror-all to git-fetch.","fromName":"Junio C Hamano","fromEmail":"junkio@cox.net","sentAt":"2006-09-20T17:50:38Z","receivedAt":"2006-09-20T17:50:38Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Shawn Pearce writes:\n\n> I meant \".git/remotes must have only one file 'origin' and\n> nothing else\". Because what does it mean to mirror two different\n> repositories at the same time into the same ref namespace? The most\n> recently fetched would always win.\n>\n> But now that I'm writing this I can see someone wanting multiple\n> remotes so they could store different URLs for the same repository\n> and call fetch on another if the first isn't responding. So I\n> retract this entirely. So there doesn't even need to be remotes\n> file; the URL could just be passed on the command line. A remotes\n> file (like the default \"origin\" one) is just a handy place to store\n> the URL.\n\nAnother possibility. The mirror might want to do something\nlike:\n\n .git/refs/remotes/davem/sparc-2.6.git/...\n\t.git/refs/remotes/torvalds/linux-2.6.git/...\n\t.git/refs/remotes/jgarzik/netdev-2.6.git/...\n\nto be a super-mirror of related projects. This would give a\nconvenient way for the visitors to browse them all at one place,\nand would save object store. There are minor details such as\ngitweb may not allow browsing refs/remotes and git-fetch may not\nlet you fetch from refs/remotes (I haven't checked), but they\nare fixable as needed.\n\n> Because we don't want to update HEAD with the SHA1 that's on the\n> remote if that's actually going to update some other ref which\n> doesn't have the same value. :-)\n\nWhat I thought you were going to do with HEAD would be to update\nit not by the object name but modifying where it points at.\nIOW, if HEAD is a symlink pointing at refs/heads/master and the\nremote HEAD now points at refs/heads/next, you do not:\n\n\techo $the_commit_object_name_of_remote_next >.git/HEAD \n\nwhich obviously is insane, but you would do\n\n\trm -f .git/HEAD && ln -s refs/heads/next .git/HEAD\n\n> But I agree with you entirely about HEAD being a SHA1 and not its\n> actual link target in the network protocol. An annoying bug.\n\nWhich fortunately is fixable ;-).\n"},{"id":"27275","messageId":"20060920182953.GT8259@pasky.or.cz","threadId":"5612","inReplyTo":"20060920173126.GG23260@spearce.org","subject":"Re: [PATCH] Added --mirror-all to git-fetch.","fromName":"Petr Baudis","fromEmail":"pasky@suse.cz","sentAt":"2006-09-20T18:29:54Z","receivedAt":"2006-09-20T18:29:54Z","isPatch":true,"sender":{"key":"pasky@ucw.cz","avatar":"https://avatars.githubusercontent.com/u/18439?v=4"},"body":"Dear diary, on Wed, Sep 20, 2006 at 06:21:45PM CEST, I got a letter\nwhere Shawn Pearce said that...\n> > > combined with set of packs with staggered spans to help\n> > > commit walkers Pasky talked about quite a while ago).\n> > \n> > Yes, this is certainly one of things I would like to implement at\n> > repo.or.cz.\n> \n> Borrowing your line:\n> Hmm, I think I've missed this, I must look that in the archive.\n> :-)\n\nI've proposed something long time ago in\n<20051112135947.GC30496@pasky.or.cz>.\n\nDear diary, on Wed, Sep 20, 2006 at 07:31:26PM CEST, I got a letter\nwhere Shawn Pearce said that...\n> Junio C Hamano wrote:\n> > Shawn Pearce writes:\n> > > Originally I gave Pasky a one-liner on #git:\n> > >\n> > > git fetch --force origin $(git ls-remote origin \\\n> > > | awk '{if(!/\\^{}$/){print $2\":\"$2}}')\n> > \n> > Which sounds a very sane thing to do, especially given git is\n> > designed to be scripted this way.\n> \n> Exactly. So I'm not sure why he was against it. After all he must\n> have some script that's looping over all mirrored repositories and\n> invoking git fetch in them. :-)\n\nBasically, I imagined that what I wanted would be very close or\nidentical to git fetch --all (or whatever you call it, just doing it\nwith .git/refs/remotes) which I think should certainly be supported\nwithout funny awk pipes. ;-)\n\n-- \n\t\t\t\tPetr \"Pasky\" Baudis\nStuff: http://pasky.or.cz/\n#!/bin/perl -sp0777i \n> Another possibility. The mirror might want to do something\n> like:\n> \n> .git/refs/remotes/davem/sparc-2.6.git/...\n> \t.git/refs/remotes/torvalds/linux-2.6.git/...\n> \t.git/refs/remotes/jgarzik/netdev-2.6.git/...\n> \n> to be a super-mirror of related projects. This would give a\n> convenient way for the visitors to browse them all at one place,\n> and would save object store. There are minor details such as\n> gitweb may not allow browsing refs/remotes and git-fetch may not\n> let you fetch from refs/remotes (I haven't checked), but they\n> are fixable as needed.\n> \n\nThis is what I do with the 2 attached, poorly named, scripts. Very \nconvenient for watching some of my favorite projects. ;-)\n\n\nexport GIT_COMMITTER_NAME=\"@\"\nexport GIT_AUTHOR_NAME=\"@\"\n\n./fred.sh git://www.kernel.org/pub/scm/git/git.git git/git\n\n./fred.sh git://www.kernel.org/pub/scm/gitk/gitk.git gitk/gitk\n\n./fred.sh git://www.kernel.org/pub/scm/linux/kernel/git/stable/linux-2.6.11.y.git stable/linux-2.6.11.y\n./fred.sh git://www.kernel.org/pub/scm/linux/kernel/git/stable/linux-2.6.12.y.git stable/linux-2.6.12.y\n./fred.sh git://www.kernel.org/pub/scm/linux/kernel/git/stable/linux-2.6.13.y.git stable/linux-2.6.13.y\n./fred.sh git://www.kernel.org/pub/scm/linux/kernel/git/stable/linux-2.6.14.y.git stable/linux-2.6.14.y\n./fred.sh git://www.kernel.org/pub/scm/linux/kernel/git/stable/linux-2.6.15.y.git stable/linux-2.6.15.y\n./fred.sh git://www.kernel.org/pub/scm/linux/kernel/git/stable/linux-2.6.16.y.git stable/linux-2.6.16.y\n./fred.sh git://www.kernel.org/pub/scm/linux/kernel/git/stable/linux-2.6.17.y.git stable/linux-2.6.17.y\n\n./fred.sh git://www.kernel.org/pub/scm/linux/kernel/git/tglx/history.git tglx/history\n\n./fred.sh git://www.kernel.org/pub/scm/linux/kernel/git/torvalds/linux-2.6.git torvalds/linux-2.6\n./fred.sh git://www.kernel.org/pub/scm/linux/kernel/git/torvalds/sparse.git torvalds/sparse\n"},{"id":"27294","messageId":"7vslimfcf9.fsf@assigned-by-dhcp.cox.net","threadId":"5612","inReplyTo":"20060920162145.GA23260@spearce.org","subject":"Re: [PATCH] Added --mirror-all to git-fetch.","fromName":"Junio C Hamano","fromEmail":"junkio@cox.net","sentAt":"2006-09-20T21:36:26Z","receivedAt":"2006-09-20T21:36:26Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Shawn Pearce writes:\n\n>> > (the archive vs active repacking strategy we talked about,\n>> \n>> Hmm, I think I've missed this, I must look that in the archive.\n>\n> Junio pushed the core code out but nobody has done the Porecelain\n> for it. The basic idea is to prevent repacking every pack all of\n> the time; there's probably no reason to repack a 100 MiB pack file\n> every time you repack your loose objects so you might want to keep\n> say a <5 MiB \"active pack\" holding your recent created objects\n> and repack that frequently and a larger 100+ MiB \"history pack\"\n> holding everything else. Maybe you repack everything on a longer\n> time scale, such as once a year.\n\nAnd IIRC we were stuck on the convention to tell which ones are\narchives and which ones are actives (i.e. fair game for\nrepacking). We were trying to figure out how to mark active\nones (whose name changes every time you repack).\n\nI realized that there is a very simple and obvious solution for\nit. Introduce $GIT_OBJECT_DIRECTORY/info/archived-packs, a flat\ntext file that lists the names of the archive packs. Any pack\nnot listed there are active one and are subject to repacking.\n"},{"id":"27297","messageId":"20060920214203.GE24415@spearce.org","threadId":"5612","inReplyTo":"7vslimfcf9.fsf@assigned-by-dhcp.cox.net","subject":"Re: [PATCH] Added --mirror-all to git-fetch.","fromName":"Shawn Pearce","fromEmail":"spearce@spearce.org","sentAt":"2006-09-20T21:42:03Z","receivedAt":"2006-09-20T21:42:03Z","isPatch":true,"sender":{"key":"spearce@spearce.org","avatar":"https://avatars.githubusercontent.com/u/34844?v=4"},"body":"Junio C Hamano wrote:\n> Shawn Pearce writes:\n> \n> >> > (the archive vs active repacking strategy we talked about,\n> >> \n> >> Hmm, I think I've missed this, I must look that in the archive.\n> >\n> > Junio pushed the core code out but nobody has done the Porecelain\n> > for it. The basic idea is to prevent repacking every pack all of\n> > the time; there's probably no reason to repack a 100 MiB pack file\n> > every time you repack your loose objects so you might want to keep\n> > say a <5 MiB \"active pack\" holding your recent created objects\n> > and repack that frequently and a larger 100+ MiB \"history pack\"\n> > holding everything else. Maybe you repack everything on a longer\n> > time scale, such as once a year.\n> \n> And IIRC we were stuck on the convention to tell which ones are\n> archives and which ones are actives (i.e. fair game for\n> repacking). We were trying to figure out how to mark active\n> ones (whose name changes every time you repack).\n\nYes. I had pretty much given up on a size threshold concept and\nwas thus in agreement with you that we just needed a way to flag\neach pack file...\n \n> I realized that there is a very simple and obvious solution for\n> it. Introduce $GIT_OBJECT_DIRECTORY/info/archived-packs, a flat\n> text file that lists the names of the archive packs. Any pack\n> not listed there are active one and are subject to repacking.\n\nLike that. :-)\n\n-- \nShawn.\n"}]}