{"thread":{"id":"30324","subject":"git-fetch fetches blobs that are already in the local repository if no history is shared?","startedAt":"2012-04-24T14:19:06Z","lastAt":"2012-04-25T01:26:38Z","messageCount":3,"participants":["Adam Roben","Shawn Pearce","Sitaram Chamarty"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"189967","messageId":"D7CFF7BD2A9545148AC2553870AA3A15@gmail.com","threadId":"30324","inReplyTo":null,"subject":"git-fetch fetches blobs that are already in the local repository if no history is shared?","fromName":"Adam Roben","fromEmail":"adam@roben.org","sentAt":"2012-04-24T14:19:06Z","receivedAt":"2012-04-24T14:19:06Z","isPatch":false,"sender":{"key":"adam@roben.org","avatar":null},"body":"Hi all- \n\nThere are two main git mirrors of the WebKit Subversion repository: and . These repositories have the exact same trees/blobs, but have entirely different commits due to the GitHub mirror using a custom --authors-prog with git-svn.\n\nTor Arne (CCed) noticed something interesting today:\n\nIf you clone one of these repositories, then add the other as a remote and fetch it, all the trees/blobs seem to get pulled down again, even though they're already in the local repository. It seems like only the commit objects should be fetched, since they're the only difference between the two remotes.\n\nIs this a bug in git? \n\n-Adam\n"},{"id":"189968","messageId":"CAJo=hJsgk-5ByMoOJ-RSce+TUFLm43_SYHf1eHWsqE+xs6QNEA@mail.gmail.com","threadId":"30324","inReplyTo":"D7CFF7BD2A9545148AC2553870AA3A15@gmail.com","subject":"Re: git-fetch fetches blobs that are already in the local repository if no history is shared?","fromName":"Shawn Pearce","fromEmail":"spearce@spearce.org","sentAt":"2012-04-24T14:23:47Z","receivedAt":"2012-04-24T14:23:47Z","isPatch":false,"sender":{"key":"spearce@spearce.org","avatar":"https://avatars.githubusercontent.com/u/34844?v=4"},"body":"On Tue, Apr 24, 2012 at 07:19, Adam Roben wrote:\n> There are two main git mirrors of the WebKit Subversion repository: and . These repositories have the exact same trees/blobs, but have entirely different commits due to the GitHub mirror using a custom --authors-prog with git-svn.\n>\n> Tor Arne (CCed) noticed something interesting today:\n>\n> If you clone one of these repositories, then add the other as a remote and fetch it, all the trees/blobs seem to get pulled down again, even though they're already in the local repository. It seems like only the commit objects should be fetched, since they're the only difference between the two remotes.\n>\n> Is this a bug in git?\n\nNo. Its the way the Git protocol was designed to function. Git only\nnegotiates over the commit history, as trying to include the blob and\ntree information into the negotiation protocol would make the payloads\nunreasonable in size. Granted in this case sending the 100M or\nwhatever it takes to enumerate all SHA-1s is smaller than the 4G or\nwhatever that WebKit actually is, but the protocol assumes nobody\nwould be this crazy to establish a huge project with two different\ncompeting commit histories and then think they could fetch them\ntogether into one repository with a small network delta.\n\nBasically... Don't do this, and don't expect Git to save you.\n\nThere should be only one version of the WebKit history imported into\nGit that everyone agrees on as being the canonical version of that\nimport. And everyone else who mirrors or works with WebKit in Git\nshould base off that version.\n\nWebKit is a big enough project with enough users that you would think\nyou could trust the git.webkit.org conversion. Which suggests the\ngithub.com one should be done over.\n"},{"id":"190039","messageId":"CAMK1S_jxC=7TK=2ODtJ6+5CqhUnLpiD_HC-hW1voQvTO1213LA@mail.gmail.com","threadId":"30324","inReplyTo":"CAJo=hJsgk-5ByMoOJ-RSce+TUFLm43_SYHf1eHWsqE+xs6QNEA@mail.gmail.com","subject":"Re: git-fetch fetches blobs that are already in the local repository if no history is shared?","fromName":"Sitaram Chamarty","fromEmail":"sitaramc@gmail.com","sentAt":"2012-04-25T01:26:38Z","receivedAt":"2012-04-25T01:26:38Z","isPatch":false,"sender":{"key":"sitaramc@gmail.com","avatar":"https://avatars.githubusercontent.com/u/43316?v=4"},"body":"On Tue, Apr 24, 2012 at 7:53 PM, Shawn Pearce wrote:\n> On Tue, Apr 24, 2012 at 07:19, Adam Roben wrote:\n>> There are two main git mirrors of the WebKit Subversion repository: and . These repositories have the exact same trees/blobs, but have entirely different commits due to the GitHub mirror using a custom --authors-prog with git-svn.\n>>\n>> Tor Arne (CCed) noticed something interesting today:\n>>\n>> If you clone one of these repositories, then add the other as a remote and fetch it, all the trees/blobs seem to get pulled down again, even though they're already in the local repository. It seems like only the commit objects should be fetched, since they're the only difference between the two remotes.\n>>\n>> Is this a bug in git?\n>\n> No. Its the way the Git protocol was designed to function. Git only\n> negotiates over the commit history, as trying to include the blob and\n> tree information into the negotiation protocol would make the payloads\n> unreasonable in size. Granted in this case sending the 100M or\n> whatever it takes to enumerate all SHA-1s is smaller than the 4G or\n> whatever that WebKit actually is, but the protocol assumes nobody\n> would be this crazy to establish a huge project with two different\n> competing commit histories and then think they could fetch them\n> together into one repository with a small network delta.\n>\n> Basically... Don't do this, and don't expect Git to save you.\n\nThat said, if you have shell access to the remote server you *can* do\nthis. We needed to do something like this as a one-time thing once,\nand I think I just made a note of the SHA for the new branch, then ran\n'git rev-list new-branch | git pack-objects pack'. Copy the two files\ncreated to the other machine's \".git/objects/pack\" directory, then\ngive the SHA a name.\n\nIt's a hack but if you need it, you need it... :)\n\n> There should be only one version of the WebKit history imported into\n> Git that everyone agrees on as being the canonical version of that\n> import. And everyone else who mirrors or works with WebKit in Git\n> should base off that version.\n>\n> WebKit is a big enough project with enough users that you would think\n> you could trust the git.webkit.org conversion. Which suggests the\n> github.com one should be done over.\n\n\n-- \nSitaram\n"}]}