{"thread":{"id":"8429","subject":"clarify git clone --local --shared --reference","startedAt":"2007-06-04T23:53:34Z","lastAt":"2007-06-13T23:07:29Z","messageCount":10,"participants":["Brandon Casey","Shawn O. Pearce","Loeliger Jon-LOELIGER"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"44018","messageId":"4664A5FE.30208@nrlssc.navy.mil","threadId":"8429","inReplyTo":null,"subject":"clarify git clone --local --shared --reference","fromName":"Brandon Casey","fromEmail":"casey@nrlssc.navy.mil","sentAt":"2007-06-04T23:53:34Z","receivedAt":"2007-06-04T23:53:34Z","isPatch":false,"sender":{"key":"drafnel@gmail.com","avatar":"https://avatars.githubusercontent.com/u/921167?v=4"},"body":"\nI think the goal of these three objects is space savings (and speed),\nbut I don't understand when I should prefer one option over another, or\nwhen/whether to use a combination of them. And I am unsure (SCARED)\nabout any side effects they may have.\n\nThis is all based on the information in git-clone.txt. If there is more\ndetail someplace else please let me know.\n\n1) What does local mean?\n --local says repository must be on the \"local\" machine and claims it\n attempts to make hardlinks when possible. Of course hard links cannot\n be created across filesystems, so are there other speedups/space\n savings when repository is on local machine but not on the same\n filesystem? Is this option still valid then?\n\n2) Does --shared imply shared write access? Does --local?\n I'll point out that git-init has an option with the same name.\n\n3) --shared seems like a special case of --reference? Are there\n differences?\n\n4) what happens if the source repository dissappears? Is --local ok\n but --shared screwed?\n\n4) is space savings obtained only at initial clone? or is it on going?\n does a future git pull from the source repository create new hard\n links where possible?\n\nCan --shared be used with --reference. Can --reference be used multiple \ntimes (and would I want to). Does -l with -s get you anything? (the\nexamples use this)\n\nthanks,\n-brandon\n"},{"id":"44034","messageId":"20070605045008.GC9513@spearce.org","threadId":"8429","inReplyTo":"4664A5FE.30208@nrlssc.navy.mil","subject":"Re: clarify git clone --local --shared --reference","fromName":"Shawn O. Pearce","fromEmail":"spearce@spearce.org","sentAt":"2007-06-05T04:50:08Z","receivedAt":"2007-06-05T04:50:08Z","isPatch":false,"sender":{"key":"spearce@spearce.org","avatar":"https://avatars.githubusercontent.com/u/34844?v=4"},"body":"Brandon Casey wrote:\n> \n> I think the goal of these three objects is space savings (and speed),\n> but I don't understand when I should prefer one option over another, or\n> when/whether to use a combination of them. And I am unsure (SCARED)\n> about any side effects they may have.\n\nYes, they are mainly about saving time setting up the new clone,\nand about disk space required by the new clone.\n \n> 1) What does local mean?\n> --local says repository must be on the \"local\" machine and claims it\n> attempts to make hardlinks when possible. Of course hard links cannot\n> be created across filesystems, so are there other speedups/space\n> savings when repository is on local machine but not on the same\n> filesystem? Is this option still valid then?\n\nBasically --local means instead of using the native Git transport to\ncopy object data from one repository to another we shortcut and use\n`find . | cpio -lpumd` or somesuch, so that cpio can use hardlinks if\npossible (same filesystem) but fallback to whole copy if it cannot.\nThis is usually faster than the native Git transport as we copy\nevery file, without first trying to compute if the file would be\nneeded by the new clone or not.\n\nSo --local may copy garbage that git-prune would have removed,\nor that git-repack/git-gc might have eliminated from a packfile.\nBut generally that's such a small amount of data that the faster\ncpio path (and even better, the hardlinks) saves disk.\n\nNote we only hardlink the immutable data under .git/objects; the\nmutable data and the working directory files that are checked out\nare *not* hardlinked.\n \n> 2) Does --shared imply shared write access? Does --local?\n> I'll point out that git-init has an option with the same name.\n\nNo. --shared means something entirely different in git-clone\nthan it does in git-init.\n\nThe --shared here implies adds the source repository to the new\nrepository's .git/objects/info/alternates. This means that the\nnew clone doesn't copy the object database; instead it just accesses\nthe source repository when it needs data.\n\nThis exposes two risks:\n\n a) Don't delete the source repository. If you delete the source\n repository then the clone repository is \"corrupt\" as it won't be\n able to access object data.\n\n b) Don't repack the source repository without accounting for the\n refs and reflogs of all --shared repositories that came from it.\n Otherwise you may delete objects that the source repository no\n longer needs, but that one or more of the --shared repositories\n still needs.\n\nObjects that are newly created in a --shared repository are written\nin the --shared area, not in the source repository. Hence the\nsource repository can be read-only to the current user.\n \n> 3) --shared seems like a special case of --reference? Are there\n> differences?\n\n--reference is actually a special case of --shared. --reference is\nmeant for cloning a remote repository over the network, where you\nalready have an existing local repository that has most of the\nobjects you need to successfully clone the remote repository.\n\nWith --reference we setup a temporary copy of refs from the\n--reference repository in the new repository, so that during the\nnetwork transfer from the remote system we don't download things\nthe --reference repository already has.\n\nBut --reference implies --shared, and has the same issues as above.\n \n> 4) what happens if the source repository dissappears? Is --local ok\n> but --shared screwed?\n\nCorrect.\n\n> 4) is space savings obtained only at initial clone? or is it on going?\n> does a future git pull from the source repository create new hard\n> links where possible?\n\nOnly on initial clone. Later pulls will copy. You can try using\ngit-relink to redo the hardlinks after the pull.\n\n> Can --shared be used with --reference. Can --reference be used multiple \n> times (and would I want to). Does -l with -s get you anything? (the\n> examples use this)\n\n--reference can only be given once in a git-clone; we only setup\none set of temporary references during the network transfer.\nAnd as I said above, --reference implies --shared.\n\n-- \nShawn.\n"},{"id":"44079","messageId":"46658F98.6020001@nrlssc.navy.mil","threadId":"8429","inReplyTo":"20070605045008.GC9513@spearce.org","subject":"Re: clarify git clone --local --shared --reference","fromName":"Brandon Casey","fromEmail":"casey@nrlssc.navy.mil","sentAt":"2007-06-05T16:30:16Z","receivedAt":"2007-06-05T16:30:16Z","isPatch":false,"sender":{"key":"drafnel@gmail.com","avatar":"https://avatars.githubusercontent.com/u/921167?v=4"},"body":"Shawn O. Pearce wrote:\n> Brandon Casey wrote:\n\n[snip]\n\n>> 2) Does --shared imply shared write access? Does --local?\n>> I'll point out that git-init has an option with the same name.\n> \n> No. --shared means something entirely different in git-clone\n> than it does in git-init.\n\nThis did cause the thought that git-init --shared and git-clone --shared\nmay be pairs to be used together in some special way.\n\nok. Rather selfish \"sharing\" in my opinion :)\n\n--reference did not cause any confusion and implied to me exactly\nwhat it does: Use supplied repository as a reference for objects\nwhich cannot be resolved locally.\n\n> The --shared here implies adds the source repository to the new\n> repository's .git/objects/info/alternates. This means that the\n> new clone doesn't copy the object database; instead it just accesses\n> the source repository when it needs data.\n> \n> This exposes two risks:\n> \n> a) Don't delete the source repository. If you delete the source\n> repository then the clone repository is \"corrupt\" as it won't be\n> able to access object data.\n> \n> b) Don't repack the source repository without accounting for the\n> refs and reflogs of all --shared repositories that came from it.\n> Otherwise you may delete objects that the source repository no\n> longer needs, but that one or more of the --shared repositories\n> still needs.\n\nHow should this be accomplished? Does this mean never run \ngit-gc/git-repack on the source repository? Or is there a way to\ncause the sharing repositories to copy over objects no longer\nrequired by the source repository?\n\n[snip]\n\n>> 4) is space savings obtained only at initial clone? or is it on going?\n>> does a future git pull from the source repository create new hard\n>> links where possible?\n> \n> Only on initial clone. Later pulls will copy. You can try using\n> git-relink to redo the hardlinks after the pull.\n\nHow about with --shared? Particularly with a fast-forward not much\nwould need to be copied over. Do later pulls into a repository with\nconfigured objects/info/alternates take advantage of space savings\nwhen possible?\n\nIf the answer above is \"yes\", then this brings up an interesting use \ncase. I assume that clone, fetch, etc follow the alternates of the \nsource repository? Otherwise a --shared repository would be unclone-able \nright? And only pull-able from the source repository? So if that is the \ncase (that remote alternates are followed), then a group of developers \ncould add all of the other developers to their alternates list (if \nmultiple alternates are supported) and reference their objects when \npossible. To the extent that it is possible, each developer would end up \nonly storing their commit objects. This would then create a distributed \nrepository.\n\nOf course, this new distributed repository may be somewhat fragile since \nthe entire thing could become unusable if any portion was corrupted. \nJust because you can do a thing, doesn't mean you should.\n\nthanks for your excellent reply,\n-brandon\n"},{"id":"44121","messageId":"20070606051111.GF9513@spearce.org","threadId":"8429","inReplyTo":"46658F98.6020001@nrlssc.navy.mil","subject":"Re: clarify git clone --local --shared --reference","fromName":"Shawn O. Pearce","fromEmail":"spearce@spearce.org","sentAt":"2007-06-06T05:11:11Z","receivedAt":"2007-06-06T05:11:11Z","isPatch":false,"sender":{"key":"spearce@spearce.org","avatar":"https://avatars.githubusercontent.com/u/34844?v=4"},"body":"Brandon Casey wrote:\n> Shawn O. Pearce wrote:\n> >\n> > b) Don't repack the source repository without accounting for the\n> > refs and reflogs of all --shared repositories that came from it.\n> > Otherwise you may delete objects that the source repository no\n> > longer needs, but that one or more of the --shared repositories\n> > still needs.\n> \n> How should this be accomplished? Does this mean never run \n> git-gc/git-repack on the source repository? Or is there a way to\n> cause the sharing repositories to copy over objects no longer\n> required by the source repository?\n\nWell, you can repack, but only if if you account for everything.\nThe easiest way to do this is push every branch from the --shared\nrepos to the source repository, repack the source repository, then\nyou can run `git prune-packed` in the --shared repos to remove\nloose objects that the source repository now has.\n\nYou can account for the refs by hand when you run pack-objects\nby hand, but its horribly difficult compared to the push and then\nrepack I just described. I think that long-lived --shared isn't that\ncommon of a workflow; most people use --shared for shortterm things.\nFor example contrib/continuous uses --shared when it clones the\nrepository to create a temporary build area.\n \n> >>4) is space savings obtained only at initial clone? or is it on going?\n> >> does a future git pull from the source repository create new hard\n> >> links where possible?\n> >\n> >Only on initial clone. Later pulls will copy. You can try using\n> >git-relink to redo the hardlinks after the pull.\n> \n> How about with --shared? Particularly with a fast-forward not much\n> would need to be copied over. Do later pulls into a repository with\n> configured objects/info/alternates take advantage of space savings\n> when possible?\n\nYes. Recently a --shared avoids copying the objects if at all\npossible. This makes fetches from the source repository into the\n--shared repository very, very fast, and uses no additional disk.\n \n> If the answer above is \"yes\", then this brings up an interesting use \n> case. I assume that clone, fetch, etc follow the alternates of the \n> source repository? Otherwise a --shared repository would be unclone-able \n> right? And only pull-able from the source repository? So if that is the \n> case (that remote alternates are followed),\n\nAlternates are followed as many as 5 deep. So you can do something like\nthis:\n\n\tgit clone --shared source share1\n\tgit clone --shared share1 share2\n\tgit clone --shared share2 share3\n\tgit clone --shared share3 share4\n\tgit clone --shared share4 share5\n\tgit clone --shared share5 corrupt\n\nI think corrupt is corrupt; it doesn't have access to the source anymore\nand therefore is missing 90%+ of the object database. To help make this\ncase work the objects/info/alternates should always contain absolute paths;\nwe store them absolute in git-clone by default but you could set them up\nby hand. The other repositories should however be intact and usable, but\nyou cannot clone from share5.\n\nNormal fetch/push/pull will work fine against any of those working\nrepos, as they are all using the normal Git object transport methods,\nwhich means we copy objects unless they are available to us already\n(see above).\n\n> then a group of developers \n> could add all of the other developers to their alternates list (if \n> multiple alternates are supported)\n\nYes, they are. I don't think we have a limit on the number of\nalternates you are allowed to have. However each additional\nalternate adds some cost to starting up any given Git process.\nThe more alternates you have (or the more deeply nested they are)\nthe slower Git will initialize itself. For 1 or 2 alternates its\nwithin the fork+exec noise of any good UNIX system; for 50 alternates\nI think you would notice it.\n\n> and reference their objects when \n> possible. To the extent that it is possible, each developer would end up \n> only storing their commit objects. This would then create a distributed \n> repository.\n\nYes, but that has very high risk. If developer Joe Smith quits and\nthen the administrator `rm -rf /home/jsmith` everyone is hosed as\nthey can no longer access the objects that were originally created\nby Joe. Then the administrator is off looking for backup tapes,\nassuming he has them and they are valid. One nice property of Git\n(really any DVCS) is that the data is automatically backed up by\nevery developer participating in the project. Its unlikely you\nwill lose the project that way.\n\nAlso this scheme doesn't really work well for packing. I don't\nthink we'll pack the loose objects that we borrow from the other\ndevelopers, and Git packfiles are a major performance improvement\nfor all Git operations. Plus they are very small, so they save a\nlot of disk.\n\nYou might find that it takes up less total disk to have everyone\nkeep a complete (non --shared) copy of the project, but repack\nregularly, then to have everyone using alternates against each\nother and nobody repacks.\n \n> Of course, this new distributed repository may be somewhat fragile since \n> the entire thing could become unusable if any portion was corrupted. \n> Just because you can do a thing, doesn't mean you should.\n\nYes, exactly. ;-)\n\nIn my day-job repositories I have about 150 MiB of blobs that\nare very common across a number of Git repositories. I've made a\nsingle repo that has all of those packed, and then setup that as an\nalternate for everything else. It saves a huge chunk of disk for us.\nBut that common-blob.git thing that I created never gets changed,\nand never gets repacked. Its sort of a \"historical archive\" for us.\nWorks very nicely. Alternates have their uses...\n\n-- \nShawn.\n"},{"id":"44180","messageId":"466701E0.4000108@nrlssc.navy.mil","threadId":"8429","inReplyTo":"20070606051111.GF9513@spearce.org","subject":"Re: clarify git clone --local --shared --reference","fromName":"Brandon Casey","fromEmail":"casey@nrlssc.navy.mil","sentAt":"2007-06-06T18:50:08Z","receivedAt":"2007-06-06T18:50:08Z","isPatch":false,"sender":{"key":"drafnel@gmail.com","avatar":"https://avatars.githubusercontent.com/u/921167?v=4"},"body":"Shawn O. Pearce wrote:\n> Brandon Casey wrote:\n>> Shawn O. Pearce wrote:\n>>> b) Don't repack the source repository without accounting for the\n>>> refs and reflogs of all --shared repositories that came from it.\n>>> Otherwise you may delete objects that the source repository no\n>>> longer needs, but that one or more of the --shared repositories\n>>> still needs.\n>> How should this be accomplished? Does this mean never run \n>> git-gc/git-repack on the source repository? Or is there a way to\n>> cause the sharing repositories to copy over objects no longer\n>> required by the source repository?\n> \n> Well, you can repack, but only if if you account for everything.\n> The easiest way to do this is push every branch from the --shared\n> repos to the source repository, repack the source repository, then\n> you can run `git prune-packed` in the --shared repos to remove\n> loose objects that the source repository now has.\n> \n> You can account for the refs by hand when you run pack-objects\n> by hand, but its horribly difficult compared to the push and then\n> repack I just described. I think that long-lived --shared isn't that\n> common of a workflow; most people use --shared for shortterm things.\n> For example contrib/continuous uses --shared when it clones the\n> repository to create a temporary build area.\n> \n\nok. I just want to make sure this is not really about prune'ing.\n\nIn the following, source and --shared repos are identical except...\n1) Source repo contains loose objects which are new commits.\n --shared repo does git-pull.\n we fast-forward, copying very little.\n success.\n2) Source repo contains loose objects which are new commits.\n Source repo does git-gc, which repacks but doesn't prune.\n --shared repo does git-pull.\n success?\n3) Source repo deletes a branch that --shared repo also has.\n This deletion creates dangling unreferenced objects.\n** --shared repo still ok here, right?\n Source repo does git-gc, which repacks but doesn't prune.\n** Is --shared screwed at this point? (This is what I understand\n you to say above) Or do the dangling objects still exist,\n so --shared is still ok?\n git gc --prune\n --shared is fubar, at least on the deleted branch.\n\n\nThe docs (git-repack.txt) seem to suggest that git-repack (without -d) \ndoes not delete any objects. And if -d is used, then at most objects \nalready referenced in other packs will be deleted. This makes me think \nthat repack is safe on the source repository.\n\nIf the above is wrong, then I'm missing a clue about git-rebase. And \ndon't feel like you have to explain it to me. Do feel free to give a \nshort *NO!! REPACK IS DANGEROUS ON SHARED REPOS YOU IDIOT!!* or other \nsuch beating. But if you see what I'm misunderstanding please let me know.\n\nIf the above is right, then it seems like the source repo developer \nshould be able to go about his developing, and git-gc'ing without\nregard for other developers who may be --share'ing. And only when the\nsource developer wants to do a prune of dangling objects must something \nspecial be done. git-prune.txt suggests:\n\n git prune $(cd ../another && $(git-rev-parse --all))\n\n> In my day-job repositories I have about 150 MiB of blobs that\n> are very common across a number of Git repositories. I've made a\n> single repo that has all of those packed, and then setup that as an\n> alternate for everything else. It saves a huge chunk of disk for us.\n> But that common-blob.git thing that I created never gets changed,\n> and never gets repacked. Its sort of a \"historical archive\" for us.\n> Works very nicely. Alternates have their uses...\n\nAhh, now that's interesting.\n\nCould I create something like that by doing something like this:\n\n(in a cloned repository with only a master branch)\ngit reset --hard HEAD^ # I know, HEAD is still in the history\ngit gc --prune # log so it doesn't get pruned, but\n # it's just an example\n\n(now back in my devel repo)\n\ngit prune-packed\n\nNow most everything is likely referenced in the archive repo except the \nlast commit. Well, maybe even the HEAD commit due to the history log.\n\nDepending on how you reply above, a periodic pull into the archive repo \n(and a repack?), then a 'git prune-packed' from the sharers could allow \ngood sharing? If not, then I guess the \narchive creation steps could be repeated periodically.\n\n-brandon\n"},{"id":"44181","messageId":"46670307.2090202@nrlssc.navy.mil","threadId":"8429","inReplyTo":"466701E0.4000108@nrlssc.navy.mil","subject":"Re: clarify git clone --local --shared --reference","fromName":"Brandon Casey","fromEmail":"casey@nrlssc.navy.mil","sentAt":"2007-06-06T18:55:03Z","receivedAt":"2007-06-06T18:55:03Z","isPatch":false,"sender":{"key":"drafnel@gmail.com","avatar":"https://avatars.githubusercontent.com/u/921167?v=4"},"body":"Brandon Casey wrote:\n\n> If the above is wrong, then I'm missing a clue about git-rebase.\n ^^^^^^\nI mean repack.\n\n-brandon\n"},{"id":"44311","messageId":"20070608053750.GB18521@spearce.org","threadId":"8429","inReplyTo":"466701E0.4000108@nrlssc.navy.mil","subject":"Re: clarify git clone --local --shared --reference","fromName":"Shawn O. Pearce","fromEmail":"spearce@spearce.org","sentAt":"2007-06-08T05:37:50Z","receivedAt":"2007-06-08T05:37:50Z","isPatch":false,"sender":{"key":"spearce@spearce.org","avatar":"https://avatars.githubusercontent.com/u/34844?v=4"},"body":"Brandon Casey wrote:\n> ok. I just want to make sure this is not really about prune'ing.\n> \n> In the following, source and --shared repos are identical except...\n> 1) Source repo contains loose objects which are new commits.\n> --shared repo does git-pull.\n> we fast-forward, copying very little.\n> success.\n\nCopying nothing actually. All of the objects required are in the\nsource repository, so --shared needs nothing additional.\n\n> 2) Source repo contains loose objects which are new commits.\n> Source repo does git-gc, which repacks but doesn't prune.\n> --shared repo does git-pull.\n> success?\n\nYes, same as above. Except: If --shared has a branch that points\nat a commit that used to be in source (hence its data isn't in\n--shared) and that data used to be packed, but source no longer\nhas a reference to it. When you repack source we won't include\nthat commit in the new pack, but we will delete the old pack.\nThat means the commit goes away.\n\n> 3) Source repo deletes a branch that --shared repo also has.\n> This deletion creates dangling unreferenced objects.\n\nYes... see above about the repack problem.\n\n> ** --shared repo still ok here, right?\n> Source repo does git-gc, which repacks but doesn't prune.\n\nBroken. The repack didn't include the dangling objects.\n\n> ** Is --shared screwed at this point? (This is what I understand\n> you to say above) Or do the dangling objects still exist,\n> so --shared is still ok?\n\n--shared is fubar, on the deleted branch.\n\n> git gc --prune\n> --shared is fubar, at least on the deleted branch.\n\nalso fubar, on the deleted branch.\n\n> The docs (git-repack.txt) seem to suggest that git-repack (without -d) \n> does not delete any objects. And if -d is used, then at most objects \n> already referenced in other packs will be deleted. This makes me think \n> that repack is safe on the source repository.\n\nYou are correct that leaving off the '-d' won't delete objects.\nBut a pack is created by listing the objects we need, and if we don't\nneed the object in source, we don't include it into the new pack.\n\n-a -d implies delete all packs that existed when we started the\nrepack. So if an object was in the old packfile, and we didn't\ncopy it to the new packfile, it gets deleted. ;-)\n\nPackfiles are immutable. We cannot delete something from it without\ndeleting the entire packfile.\n \n> If the above is wrong, then I'm missing a clue about git-rebase. And \n> don't feel like you have to explain it to me. Do feel free to give a \n> short *NO!! REPACK IS DANGEROUS ON SHARED REPOS YOU IDIOT!!* or other \n> such beating. But if you see what I'm misunderstanding please let me know.\n\n;-)\n\nNO!! REPACK IS DANGEROUS ON SHARED REPOS YOU IDIOT!!\n\nExactly because of what I'm saying above, and what you mention about\nsource deleting a branch and repacking or pruning, and --shared\nthen being corrupt because its now missing at least one object it\nwants to have.\n \n> If the above is right, then it seems like the source repo developer \n> should be able to go about his developing, and git-gc'ing without\n> regard for other developers who may be --share'ing. And only when the\n> source developer wants to do a prune of dangling objects must something \n> special be done. git-prune.txt suggests:\n> \n> git prune $(cd ../another && $(git-rev-parse --all))\n\nRight; but the thing that is shocking to some people (me in my\nearly days with Git) is that a repack is *also* doing a prune if\nyou add -d. Its not quite the same type of prune as it prunes only\npreviously packed objects, but its still a prune.\n \n> >In my day-job repositories I have about 150 MiB of blobs that\n> >are very common across a number of Git repositories. I've made a\n> >single repo that has all of those packed, and then setup that as an\n> >alternate for everything else. It saves a huge chunk of disk for us.\n> >But that common-blob.git thing that I created never gets changed,\n> >and never gets repacked. Its sort of a \"historical archive\" for us.\n> >Works very nicely. Alternates have their uses...\n> \n> Ahh, now that's interesting.\n> \n> Could I create something like that by doing something like this:\n> \n> (in a cloned repository with only a master branch)\n> git reset --hard HEAD^ # I know, HEAD is still in the history\n> git gc --prune # log so it doesn't get pruned, but\n> # it's just an example\n> \n> (now back in my devel repo)\n> \n> git prune-packed\n\nActually you want `git gc` in your --shared repo now. That way it\nminimizes the current packfiles. prune-packed only applies to loose\nobjects; not already packed stuff. Since you added a repo with a\nlarge packfile to your alternates list you probably want to shrink\nyour own packfile down as much as possible. That works because\n`git gc` is actually running `git repack -a -d -l`. The -l means\n\"only things that are local to this --shared repository.\n \n> Now most everything is likely referenced in the archive repo except the \n> last commit. Well, maybe even the HEAD commit due to the history log.\n\nRight.\n \n> Depending on how you reply above, a periodic pull into the archive repo \n> (and a repack?), then a 'git prune-packed' from the sharers could allow \n> good sharing? If not, then I guess the \n> archive creation steps could be repeated periodically.\n\nYes, exactly. The trick here is that once something enters the\nshared repo IT MUST STAY THERE. You cannot allow a branch to be\ndeleted from the shared repo, unless it is fully merged into another\nbranch (or tag) that is staying. You also cannot rewind a branch\n(e.g. push with --force).\n\nBut I do exactly what you are suggesting. I pull every so often\ninto a the shared common repo and repack it. And then everyone\nelse can repack and their repack deletes things that are in the\nshared common repo.\n\nJust make sure you have a good backup of the shared common repo. :)\n\n-- \nShawn.\n"},{"id":"44356","messageId":"AF0367BA48C55940A43CCF08DF35553C013C76BF@az33exm24.fsl.freescale.net","threadId":"8429","inReplyTo":"20070608053750.GB18521@spearce.org","subject":"RE: clarify git clone --local --shared --reference","fromName":"Loeliger Jon-LOELIGER","fromEmail":"jdl@freescale.com","sentAt":"2007-06-08T15:57:31Z","receivedAt":"2007-06-08T15:57:31Z","isPatch":false,"sender":{"key":"jdl@jdl.com","avatar":"https://gravatar.com/avatar/75ce9a10b151acd2c28ec4ab2136dba7b2ff1634530bd04b155981a749d08a64?d=mp&s=160"},"body":"Shawn O. Pearce wrote:\n> \n> Brandon Casey wrote:\n> > ok. I just want to make sure this is not really about prune'ing.\n> > \n> > In the following, source and --shared repos are identical except...\n> > 1) Source repo contains loose objects which are new commits.\n> > --shared repo does git-pull.\n> > we fast-forward, copying very little.\n> > success.\n> \n> Copying nothing actually. All of the objects required are in the\n> source repository, so --shared needs nothing additional.\n\nSo the thing I find myself wanting to do is\nA \"crib from local copy\". That is, the network\nCost is large, so when cloning point to a local\n(ie, already on same Filesystem) clone that is\nSimilar, use it as a reference, but, in the end,\nCreate a complete copy into the new repository.\n\nI don't want it hard linked with --local.\nI don't want it shared with --shared.\nI don't want it as an altrnate source with --reference.\n\nWhat I want is a new, clean, complete, unshared repository.\n\nBut for efficiency reasons, I want to grab objects\nFrom a different, filesystem-local clone if possible.\n\nDoes that work?\n\njdl\n"},{"id":"44366","messageId":"4669A18E.4080702@nrlssc.navy.mil","threadId":"8429","inReplyTo":"AF0367BA48C55940A43CCF08DF35553C013C76BF@az33exm24.fsl.freescale.net","subject":"Re: clarify git clone --local --shared --reference","fromName":"Brandon Casey","fromEmail":"casey@nrlssc.navy.mil","sentAt":"2007-06-08T18:35:58Z","receivedAt":"2007-06-08T18:35:58Z","isPatch":false,"sender":{"key":"drafnel@gmail.com","avatar":"https://avatars.githubusercontent.com/u/921167?v=4"},"body":"Loeliger Jon-LOELIGER wrote:\n> Shawn O. Pearce wrote:\n>> Brandon Casey wrote:\n>>> ok. I just want to make sure this is not really about prune'ing.\n>>>\n>>> In the following, source and --shared repos are identical except...\n>>> 1) Source repo contains loose objects which are new commits.\n>>> --shared repo does git-pull.\n>>> we fast-forward, copying very little.\n>>> success.\n>> Copying nothing actually. All of the objects required are in the\n>> source repository, so --shared needs nothing additional.\n> \n> So the thing I find myself wanting to do is\n> A \"crib from local copy\". That is, the network\n> Cost is large, so when cloning point to a local\n> (ie, already on same Filesystem) clone that is\n> Similar, use it as a reference, but, in the end,\n> Create a complete copy into the new repository.\n> \n> I don't want it hard linked with --local.\n> I don't want it shared with --shared.\n> I don't want it as an altrnate source with --reference.\n> \n> What I want is a new, clean, complete, unshared repository.\n> \n> But for efficiency reasons, I want to grab objects\n> From a different, filesystem-local clone if possible.\n> \n> Does that work?\n\nI don't think that exact behavior is implemented yet, but...\n\nIf the filesystem-local repo is pure subset version of the source repo \nyou could do this:\n\n(assuming the filesystem-local repo is on branch master, and that is \nwhat you want)\ngit clone -l \ncd \ngit pull \n\nNo reason not to use -l on clone in this case IMO.\n\nOtherwise...\nIf the filesystem-local repo has changes past the master HEAD on source \nrepo that you are not necessarily interested in...\n\n1) git clone -l -n \n2) cd \n3) git fetch master:tmp\n4) git branch -M tmp master\n5) git checkout master\n\n1) Here we use -l to encourage hard linking (no reason not to IMO),\n and tell clone not (-n) to checkout the active branch.\n3) Now fetch the master branch from the source_repo and store into\n a new branch named tmp.\n4) Rename tmp to master.\n5) Checkout the files.\n- Now the HEAD of master branch is at the same commit as the\n source_repo.\n\nOne drawback is that origin is now tracking the filesystem-local repo, \nso a git pull without supplying a repo will pull from filesystem-local repo.\n\n-brandon\n"},{"id":"45016","messageId":"467078B1.8080104@nrlssc.navy.mil","threadId":"8429","inReplyTo":"20070608053750.GB18521@spearce.org","subject":"Re: clarify git clone --local --shared --reference","fromName":"Brandon Casey","fromEmail":"casey@nrlssc.navy.mil","sentAt":"2007-06-13T23:07:29Z","receivedAt":"2007-06-13T23:07:29Z","isPatch":false,"sender":{"key":"drafnel@gmail.com","avatar":"https://avatars.githubusercontent.com/u/921167?v=4"},"body":"Shawn O. Pearce wrote:\n> Brandon Casey wrote:\n\n>> The docs (git-repack.txt) seem to suggest that git-repack (without -d) \n>> does not delete any objects. And if -d is used, then at most objects \n>> already referenced in other packs will be deleted. This makes me think \n>> that repack is safe on the source repository.\n> \n> You are correct that leaving off the '-d' won't delete objects.\n> But a pack is created by listing the objects we need, and if we don't\n> need the object in source, we don't include it into the new pack.\n> \n> -a -d implies delete all packs that existed when we started the\n> repack. So if an object was in the old packfile, and we didn't\n> copy it to the new packfile, it gets deleted. ;-)\n\nOk. There is the connection I did not make. repack -d is NOT harmless, \nsince a pack that contains only objects referenced in other packs and \ndangling unreferenced objects, will be deleted. Which will be all of the \npreexisting packs in the case of git-gc since as you mentioned it \nrepacks using -a -d -l.\n\nThanks for taking the time.\n\n-brandon\n"}]}