{"thread":{"id":"43326","subject":"Re: Tracking a repository for content instead of history","startedAt":"2006-12-12T12:35:08Z","lastAt":"2006-12-12T22:25:09Z","messageCount":10,"participants":["Andy Parkins","Jakub Narebski","Johannes Schindelin","Nguyen Thai Ngoc Duy"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"297682","messageId":"200612121235.09984.andyparkins@gmail.com","threadId":"43326","inReplyTo":null,"subject":"Tracking a repository for content instead of history","fromName":"Andy Parkins","fromEmail":"andyparkins@gmail.com","sentAt":"2006-12-12T12:35:08Z","receivedAt":"2006-12-12T12:35:08Z","isPatch":false,"sender":{"key":"andyparkins@gmail.com","avatar":null},"body":"Hello,\n\nFor interests sake I'd like to track the kernel.org linux repository.  \nHowever, I'm not that bothered about tracking the history - it's more that I \nlike to have the latest kernel release lying around.\n\nIs there a way that I could just pull individual commits from a git \nrepository?  In particular - could I make a repository (obviously not a \nclone, because it wouldn't have all the history) that contained only the \ntagged commits from an upstream repository?\n\nIs it even sensible to want that?  It strikes me that it's possible that there \nisn't that much space/bandwidth saving to be made.  Should I just clone the \nrepository and shut up?  :-)\n\n\nAndy\n-- \nDr Andy Parkins, M Eng (hons), MIEE\n"},{"id":"294950","messageId":"elm993$rf1$1@sea.gmane.org","threadId":"43326","inReplyTo":"200612121235.09984.andyparkins@gmail.com","subject":"Re: Tracking a repository for content instead of history","fromName":"Jakub Narebski","fromEmail":"jnareb@gmail.com","sentAt":"2006-12-12T13:04:51Z","receivedAt":"2006-12-12T13:04:51Z","isPatch":false,"sender":{"key":"jnareb@gmail.com","avatar":"https://avatars.githubusercontent.com/u/2706?v=4"},"body":"Andy Parkins wrote:\n\n> For interests sake I'd like to track the kernel.org linux repository.  \n> However, I'm not that bothered about tracking the history - it's more that I \n> like to have the latest kernel release lying around.\n> \n> Is there a way that I could just pull individual commits from a git \n> repository?  In particular - could I make a repository (obviously not a \n> clone, because it wouldn't have all the history) that contained only the \n> tagged commits from an upstream repository?\n\nAs of beta (in 'next') you can do 'shallow clone'm i.e. clone/fetch\nonly N commits depth history.\n \n> Is it even sensible to want that?  It strikes me that it's possible that there \n> isn't that much space/bandwidth saving to be made.  Should I just clone the \n> repository and shut up?  :-)\n\nI've had similar idea: search for \"sparse clone\" keyword. But no code.\n\n-- \nJakub Narebski\nWarsaw, Poland\nShadeHawk on #git\n\n"},{"id":"296922","messageId":"200612121326.24508.andyparkins@gmail.com","threadId":"43326","inReplyTo":"elm993$rf1$1@sea.gmane.org","subject":"Re: Tracking a repository for content instead of history","fromName":"Andy Parkins","fromEmail":"andyparkins@gmail.com","sentAt":"2006-12-12T13:26:22Z","receivedAt":"2006-12-12T13:26:22Z","isPatch":false,"sender":{"key":"andyparkins@gmail.com","avatar":null},"body":"On Tuesday 2006 December 12 13:04, Jakub Narebski wrote:\n\n> > Is it even sensible to want that?  It strikes me that it's possible that\n> > there isn't that much space/bandwidth saving to be made.  Should I just\n> > clone the repository and shut up?  :-)\n>\n> I've had similar idea: search for \"sparse clone\" keyword. But no code.\n\nWhile the functionality might not be built into git in terms of clone, would \nthere be a way to pull a particular commit from another repository? \n\nThe way I would do it given nothing else is to simply extract snapshots into a \nworking directory; and create a repository from scratch.  I was just \nwondering if a method existed that could reduce the size of the download.\n\nI think the best way is going to be to use the patches published at kernel.org \nand apply them one at a time with git-apply.\n\n\nAndy\n-- \nDr Andy Parkins, M Eng (hons), MIEE\n"},{"id":"295658","messageId":"Pine.LNX.4.63.0612121527070.2807@wbgn013.biozentrum.uni-wuerzburg.de","threadId":"43326","inReplyTo":"200612121326.24508.andyparkins@gmail.com","subject":"Re: Tracking a repository for content instead of history","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2006-12-12T14:28:21Z","receivedAt":"2006-12-12T14:28:21Z","isPatch":false,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi,\n\nOn Tue, 12 Dec 2006, Andy Parkins wrote:\n\n> The way I would do it given nothing else is to simply extract snapshots \n> into a working directory; and create a repository from scratch.  I was \n> just wondering if a method existed that could reduce the size of the \n> download.\n\nYou are not by any chance talking about the --remote option to \ngit-archive?\n\nIf you want to reduce the number of objects to be downloaded, by telling \nthe other side what you have, you literally end up with something like \nshallow clone: the other side _has_ to support it.\n\nCiao,\nDscho\n"},{"id":"294369","messageId":"200612121538.41197.andyparkins@gmail.com","threadId":"43326","inReplyTo":"Pine.LNX.4.63.0612121527070.2807@wbgn013.biozentrum.uni-wuerzburg.de","subject":"Re: Tracking a repository for content instead of history","fromName":"Andy Parkins","fromEmail":"andyparkins@gmail.com","sentAt":"2006-12-12T15:38:39Z","receivedAt":"2006-12-12T15:38:39Z","isPatch":false,"sender":{"key":"andyparkins@gmail.com","avatar":null},"body":"On Tuesday 2006 December 12 14:28, Johannes Schindelin wrote:\n\n> You are not by any chance talking about the --remote option to\n> git-archive?\n\nI wasn't; but that's certainly a helpful switch.  It's certainly a huge help.\n\n> If you want to reduce the number of objects to be downloaded, by telling\n> the other side what you have, you literally end up with something like\n> shallow clone: the other side _has_ to support it.\n\nI suppose so; but I was thinking more an automated way of getting the data \nthat is supplied for the kernel anyway.  So:\n\nbase-v1.0.0.tar.gz\npatch-v1.0.1.gz\npatch-v1.0.2.gz\netc\n\nEach patch is obviously smaller than \"base\".  Git could easily make the \npatches, and each of those patches could be fed by hand into a repository \nwith git-apply.  It doesn't seem like something that would require support on \nthe other side, because it isn't so much a shallow clone (which /would/ \npreserve history, making it available if wanted); it is pulling just, say, \ntagged commits out of an existing repository.\n\nGiven a list of tags it is almost:\n\ngit-archive <get me base>\nssh remote git-diff v1.0.0..v1.0.1 | git-apply; git commit\nssh remote git-diff v1.0.1..v1.0.2 | git-apply; git commit\n\nIf that makes sense?  Obviously though it would be possible to use git rather \nthan ssh to do this.\n\nHowever... please don't waste any more time thinking about this; it's not a \nproblem I have that needs a solution - it was more a \"because I'm curious\" \nsort of question.\n\n\n\nAndy\n-- \nDr Andy Parkins, M Eng (hons), MIEE\n"},{"id":"295379","messageId":"Pine.LNX.4.63.0612121715230.2807@wbgn013.biozentrum.uni-wuerzburg.de","threadId":"43326","inReplyTo":"200612121538.41197.andyparkins@gmail.com","subject":"Re: Tracking a repository for content instead of history","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2006-12-12T16:24:00Z","receivedAt":"2006-12-12T16:24:00Z","isPatch":false,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi,\n\nOn Tue, 12 Dec 2006, Andy Parkins wrote:\n\n> On Tuesday 2006 December 12 14:28, Johannes Schindelin wrote:\n> \n> > You are not by any chance talking about the --remote option to\n> > git-archive?\n> \n> I wasn't; but that's certainly a helpful switch.  It's certainly a huge \n> help.\n> \n> > If you want to reduce the number of objects to be downloaded, by telling\n> > the other side what you have, you literally end up with something like\n> > shallow clone: the other side _has_ to support it.\n> \n> I suppose so; but I was thinking more an automated way of getting the data \n> that is supplied for the kernel anyway.  So:\n> \n> base-v1.0.0.tar.gz\n> patch-v1.0.1.gz\n> patch-v1.0.2.gz\n> etc\n> \n> Each patch is obviously smaller than \"base\".  Git could easily make the \n> patches, and each of those patches could be fed by hand into a repository \n> with git-apply.\n\nIf it weren't for the recent discussion of kernel.org being overloaded \nwith gitweb processes, I'd just write down a hint like \nhttp://repo.or.cz/w/git/jnareb-git.git?a=commitdiff_plain;h=next;hp=master\n\nBut since kernel.org is overloaded, I will not do that.\n\nCiao,\n"},{"id":"298337","messageId":"Pine.LNX.4.63.0612121733580.2807@wbgn013.biozentrum.uni-wuerzburg.de","threadId":"43326","inReplyTo":"Pine.LNX.4.63.0612121715230.2807@wbgn013.biozentrum.uni-wuerzburg.de","subject":"Re: Tracking a repository for content instead of history","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2006-12-12T16:35:42Z","receivedAt":"2006-12-12T16:35:42Z","isPatch":false,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi,\n\nOn Tue, 12 Dec 2006, Johannes Schindelin wrote:\n\n> If it weren't for the recent discussion of kernel.org being overloaded \n> with gitweb processes, I'd just write down a hint like \n> [URL edited out]\n> \n> But since kernel.org is overloaded, I will not do that.\n\nSide note: it would probably not help you. The diff is uncompressed, and \nthus likely _substantially larger_ than getting the snapshot via gitweb, \nwhich _is_ compressed.\n\nCiao,\nDscho\n"},{"id":"297630","messageId":"fcaeb9bf0612121346s3c82bcdbh522c17f4ae9f279b@mail.gmail.com","threadId":"43326","inReplyTo":"200612121538.41197.andyparkins@gmail.com","subject":"Re: Tracking a repository for content instead of history","fromName":"Nguyen Thai Ngoc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2006-12-12T21:46:10Z","receivedAt":"2006-12-12T21:46:10Z","isPatch":false,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"On 12/12/06, Andy Parkins <andyparkins@gmail.com> wrote:\n> I suppose so; but I was thinking more an automated way of getting the data\n> that is supplied for the kernel anyway.  So:\n>\n> base-v1.0.0.tar.gz\n> patch-v1.0.1.gz\n> patch-v1.0.2.gz\n> etc\n>\n> Each patch is obviously smaller than \"base\".  Git could easily make the\n> patches, and each of those patches could be fed by hand into a repository\n> with git-apply.  It doesn't seem like something that would require support on\n> the other side, because it isn't so much a shallow clone (which /would/\n> preserve history, making it available if wanted); it is pulling just, say,\n> tagged commits out of an existing repository.\n>\n> Given a list of tags it is almost:\n>\n> git-archive <get me base>\n> ssh remote git-diff v1.0.0..v1.0.1 | git-apply; git commit\n> ssh remote git-diff v1.0.1..v1.0.2 | git-apply; git commit\n>\n> If that makes sense?  Obviously though it would be possible to use git rather\n> than ssh to do this.\n\nHm.. I'm no git:// expert. But is it possible doing as follow?\n1. git-archive <base>\n2. reconstruct commit, blobs and trees from the archive\n3. tell git server that you have one commit, you need another commit\n(maybe heads only, i'm not sure here)\n4. get the pack from git server, create new commit and a diff\n-- \n"},{"id":"296249","messageId":"fcaeb9bf0612121348q35549a4cof0adb85e0391b92c@mail.gmail.com","threadId":"43326","inReplyTo":"fcaeb9bf0612121346s3c82bcdbh522c17f4ae9f279b@mail.gmail.com","subject":"Re: Tracking a repository for content instead of history","fromName":"Nguyen Thai Ngoc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2006-12-12T21:48:12Z","receivedAt":"2006-12-12T21:48:12Z","isPatch":false,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"On 12/13/06, Nguyen Thai Ngoc Duy <pclouds@gmail.com> wrote:\n> Hm.. I'm no git:// expert. But is it possible doing as follow?\n> 1. git-archive <base>\n> 2. reconstruct commit, blobs and trees from the archive\n> 3. tell git server that you have one commit, you need another commit\n> (maybe heads only, i'm not sure here)\n> 4. get the pack from git server, create new commit and a diff\n\nOk. Stupid idea. The pack may base on objects that I don't have.\n\n-- \n"},{"id":"296451","messageId":"Pine.LNX.4.63.0612122316290.2807@wbgn013.biozentrum.uni-wuerzburg.de","threadId":"43326","inReplyTo":"fcaeb9bf0612121348q35549a4cof0adb85e0391b92c@mail.gmail.com","subject":"Re: Tracking a repository for content instead of history","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2006-12-12T22:25:09Z","receivedAt":"2006-12-12T22:25:09Z","isPatch":false,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi,\n\nOn Wed, 13 Dec 2006, Nguyen Thai Ngoc Duy wrote:\n\n> On 12/13/06, Nguyen Thai Ngoc Duy <pclouds@gmail.com> wrote:\n> > Hm.. I'm no git:// expert. But is it possible doing as follow?\n> > 1. git-archive <base>\n> > 2. reconstruct commit, blobs and trees from the archive\n> > 3. tell git server that you have one commit, you need another commit\n> > (maybe heads only, i'm not sure here)\n> > 4. get the pack from git server, create new commit and a diff\n> \n> Ok. Stupid idea. The pack may base on objects that I don't have.\n\nThe only not-so-brilliant idea is to reconstruct the commit from the \narchive. This is not possible, as not only some author and \ncommitter metadata is not reconstructable, but worse: the parents' hash is \nnot either. And since all these are hashed to get the commit hash, you \nlost.\n\nHowever, it could work like this:\n\n- reconstruct tree commit\n- ask for a diff between a certain commit, with respect to your tree\n\nIt might even be easy to convince git-upload-pack to construct a thin pack \ncontaining deltas _only_ against objects which are reachable from your \ntree.\n\nNote: this is feasible, but not necessarily sensible:\n\n- it puts more strain on the server, which otherwise could probably reuse \na lot of deltas, and\n- it contradicts the idea of _distributed_ development (for example, you \ncould not tell which HEAD commit is newer when you fetched from two \nrepos).\n\nProbably, you could add a third argument: merges are not necessarily \n_possible_ with that setup. Note that this argument applies to shallow \nclones, too!\n\nCiao,\nDscho\n"}]}