{"thread":{"id":"14628","subject":"[PATCH 08/12] git-clone: support --path to do sparse clone","startedAt":"2008-07-23T14:57:18Z","lastAt":"2008-07-25T08:54:53Z","messageCount":16,"participants":["Nguyễn Thái Ngọc Duy","Jeff King","Johannes Schindelin","Nguyen Thai Ngoc Duy","Petr Baudis","Sverre Rabbelier","James Pickens","Junio C Hamano"],"isPatch":true,"patchVersion":1,"patchTotal":12},"messages":[{"id":"84517","messageId":"20080723145718.GA29134@laptop","threadId":"14628","inReplyTo":null,"subject":"[PATCH 08/12] git-clone: support --path to do sparse clone","fromName":"Nguyễn Thái Ngọc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2008-07-23T14:57:18Z","receivedAt":"2008-07-23T14:57:18Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"\nSigned-off-by: Nguyễn Thái Ngọc Duy \n---\n builtin-clone.c | 13 +++++++++++++\n 1 files changed, 13 insertions(+), 0 deletions(-)\n\ndiff --git a/builtin-clone.c b/builtin-clone.c\nindex 3522245..229f2e2 100644\n--- a/builtin-clone.c\n+++ b/builtin-clone.c\n@@ -36,6 +36,7 @@ static const char * const builtin_clone_usage[] = {\n static int option_quiet, option_no_checkout, option_bare;\n static int option_local, option_no_hardlinks, option_shared;\n static char *option_template, *option_reference, *option_depth;\n+static char *option_sparse_prefix;\n static char *option_origin = NULL;\n static char *option_upload_pack = \"git-upload-pack\";\n \n@@ -43,6 +44,8 @@ static struct option builtin_clone_options[] = {\n \tOPT__QUIET(&option_quiet),\n \tOPT_BOOLEAN('n', \"no-checkout\", &option_no_checkout,\n \t\t \"don't create a checkout\"),\n+\tOPT_STRING(0, \"path\", &option_sparse_prefix, \"prefixes\",\n+\t\t \"limit checkout to specified paths (sparse checkout)\"),\n \tOPT_BOOLEAN(0, \"bare\", &option_bare, \"create a bare repository\"),\n \tOPT_BOOLEAN(0, \"naked\", &option_bare, \"create a bare repository\"),\n \tOPT_BOOLEAN('l', \"local\", &option_local,\n@@ -364,9 +367,14 @@ int cmd_clone(int argc, const char **argv, const char *prefix)\n \t\t\tdie(\"--bare and --origin %s options are incompatible.\",\n \t\t\t option_origin);\n \t\toption_no_checkout = 1;\n+\t\tif (option_sparse_prefix)\n+\t\t\tdie(\"--bare and --path options are incompatible.\");\n \t\tuse_separate_remote = 0;\n \t}\n \n+\tif (option_no_checkout && option_sparse_prefix)\n+\t\tdie(\"--no-checkout and --path options are incompatible.\");\n+\n \tif (!option_origin)\n \t\toption_origin = \"origin\";\n \n@@ -549,6 +557,11 @@ int cmd_clone(int argc, const char **argv, const char *prefix)\n \t\t/* We need to be in the new work tree for the checkout */\n \t\tsetup_work_tree();\n \n+\t\tif (option_sparse_prefix) {\n+\t\t\tgit_config_set(\"core.sparsecheckout\", option_sparse_prefix);\n+\t\t\tset_sparse_prefix(option_sparse_prefix);\n+\t\t}\n+\n \t\tfd = hold_locked_index(lock_file, 1);\n \n \t\tmemset(&opts, 0, sizeof opts);\n-- \n1.5.5.GIT\n"},{"id":"84752","messageId":"20080724171952.GB21043@sigill.intra.peff.net","threadId":"14628","inReplyTo":"20080723145718.GA29134@laptop","subject":"Re: [PATCH 08/12] git-clone: support --path to do sparse clone","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2008-07-24T17:19:52Z","receivedAt":"2008-07-24T17:19:52Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Wed, Jul 23, 2008 at 09:57:18PM +0700, Nguyễn Thái Ngọc Duy wrote:\n\n> +\t\tif (option_sparse_prefix) {\n> +\t\t\tgit_config_set(\"core.sparsecheckout\", option_sparse_prefix);\n> +\t\t\tset_sparse_prefix(option_sparse_prefix);\n> +\t\t}\n> +\n\nAs a user, I would expect \"sparse clone\" to also be sparse on the\nfetching. That is, to not even bother fetching tree objects that we are\nnot going to check out. But that is a whole other can of worms from\nlocal sparseness, so I think it is worth saving for a different series.\n\nSo instead I would suggest that this be mentioned in the documentation\nfor --path, but there doesn't seem to be any.\n\n-Peff\n"},{"id":"84755","messageId":"alpine.DEB.1.00.0807241837441.8986@racer","threadId":"14628","inReplyTo":"20080724171952.GB21043@sigill.intra.peff.net","subject":"sparse fetch, was Re: [PATCH 08/12] git-clone: support --path to do sparse clone","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2008-07-24T17:41:03Z","receivedAt":"2008-07-24T17:41:03Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi,\n\nOn Thu, 24 Jul 2008, Jeff King wrote:\n\n> As a user, I would expect \"sparse clone\" to also be sparse on the \n> fetching. That is, to not even bother fetching tree objects that we are \n> not going to check out. But that is a whole other can of worms from \n> local sparseness, so I think it is worth saving for a different series.\n\nI think this is not even worth of a series. Sure, it would have benefits \nfor those who want sparse checkouts. But it comes for a high price on \neveryone else:\n\n- security issues (you'd need to open the git protocol to give you \n something else than a ref, _including_ refs that were deleted)\n\n- performance issues (the server would have to do a lot more, faking \n commits, or in the alternative serving a gazillion more sessions if the \n client does the reconstruction)\n\n... and I am sure there are tons more issues.\n\nCiao,\nDscho\n"},{"id":"84765","messageId":"20080724182813.GA21186@sigill.intra.peff.net","threadId":"14628","inReplyTo":"alpine.DEB.1.00.0807241837441.8986@racer","subject":"Re: sparse fetch, was Re: [PATCH 08/12] git-clone: support --path to do sparse clone","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2008-07-24T18:28:13Z","receivedAt":"2008-07-24T18:28:13Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Thu, Jul 24, 2008 at 06:41:03PM +0100, Johannes Schindelin wrote:\n\n> > As a user, I would expect \"sparse clone\" to also be sparse on the \n> > fetching. That is, to not even bother fetching tree objects that we are \n> > not going to check out. But that is a whole other can of worms from \n> > local sparseness, so I think it is worth saving for a different series.\n> \n> I think this is not even worth of a series. Sure, it would have benefits \n> for those who want sparse checkouts. But it comes for a high price on \n> everyone else:\n\nI agree there are a lot of issues. I am just thinking of the person who\nsaid they had a >100G repository. But I am also not volunteering to do\nit, so I will let somebody who really cares about it try to defend the\nidea.\n\n-Peff\n"},{"id":"84768","messageId":"fcaeb9bf0807241144i6decac10q78c5962f3daa7a48@mail.gmail.com","threadId":"14628","inReplyTo":"alpine.DEB.1.00.0807241837441.8986@racer","subject":"Re: sparse fetch, was Re: [PATCH 08/12] git-clone: support --path to do sparse clone","fromName":"Nguyen Thai Ngoc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2008-07-24T18:44:01Z","receivedAt":"2008-07-24T18:44:01Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"On 7/25/08, Johannes Schindelin wrote:\n> Hi,\n>\n> On Thu, 24 Jul 2008, Jeff King wrote:\n>\n> > As a user, I would expect \"sparse clone\" to also be sparse on the\n> > fetching. That is, to not even bother fetching tree objects that we are\n> > not going to check out. But that is a whole other can of worms from\n> > local sparseness, so I think it is worth saving for a different series.\n>\n> I think this is not even worth of a series. Sure, it would have benefits\n> for those who want sparse checkouts. But it comes for a high price on\n> everyone else:\n>\n> - security issues (you'd need to open the git protocol to give you\n> something else than a ref, _including_ refs that were deleted)\n>\n> - performance issues (the server would have to do a lot more, faking\n> commits, or in the alternative serving a gazillion more sessions if the\n> client does the reconstruction)\n>\n> ... and I am sure there are tons more issues.\n\nWiden checkout won't work and probably more.\n-- \nDuy\n"},{"id":"84770","messageId":"fcaeb9bf0807241147k1175c8a3mffc12e3b3cc1aa3f@mail.gmail.com","threadId":"14628","inReplyTo":"20080724171952.GB21043@sigill.intra.peff.net","subject":"Re: [PATCH 08/12] git-clone: support --path to do sparse clone","fromName":"Nguyen Thai Ngoc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2008-07-24T18:47:54Z","receivedAt":"2008-07-24T18:47:54Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"On 7/25/08, Jeff King wrote:\n> On Wed, Jul 23, 2008 at 09:57:18PM +0700, Nguyen Thai Ngoc Duy wrote:\n>\n> > + if (option_sparse_prefix) {\n> > + git_config_set(\"core.sparsecheckout\", option_sparse_prefix);\n> > + set_sparse_prefix(option_sparse_prefix);\n> > + }\n> > +\n>\n>\n> As a user, I would expect \"sparse clone\" to also be sparse on the\n> fetching. That is, to not even bother fetching tree objects that we are\n> not going to check out. But that is a whole other can of worms from\n> local sparseness, so I think it is worth saving for a different series.\n>\n> So instead I would suggest that this be mentioned in the documentation\n> for --path, but there doesn't seem to be any.\n\nThanks. Will mention it when I write documentation for this.\n-- \nDuy\n"},{"id":"84773","messageId":"20080724185332.GQ32184@machine.or.cz","threadId":"14628","inReplyTo":"alpine.DEB.1.00.0807241837441.8986@racer","subject":"Re: sparse fetch, was Re: [PATCH 08/12] git-clone: support --path to do sparse clone","fromName":"Petr Baudis","fromEmail":"pasky@suse.cz","sentAt":"2008-07-24T18:53:32Z","receivedAt":"2008-07-24T18:53:32Z","isPatch":true,"sender":{"key":"pasky@ucw.cz","avatar":"https://avatars.githubusercontent.com/u/18439?v=4"},"body":" Hi,\n\nOn Thu, Jul 24, 2008 at 06:41:03PM +0100, Johannes Schindelin wrote:\n> On Thu, 24 Jul 2008, Jeff King wrote:\n> \n> > As a user, I would expect \"sparse clone\" to also be sparse on the \n> > fetching. That is, to not even bother fetching tree objects that we are \n> > not going to check out. But that is a whole other can of worms from \n> > local sparseness, so I think it is worth saving for a different series.\n> \n> I think this is not even worth of a series. Sure, it would have benefits \n> for those who want sparse checkouts. But it comes for a high price on \n> everyone else:\n> \n> - security issues (you'd need to open the git protocol to give you \n> something else than a ref, _including_ refs that were deleted)\n> \n> - performance issues (the server would have to do a lot more, faking \n> commits, or in the alternative serving a gazillion more sessions if the \n> client does the reconstruction)\n\n I don't follow how these two issues arise, if the server will do the\npruning for you. It will just skip entering some tree objects when doing\nobject traversal; why opening the git protocol or faking commits? This\nwould be a simple extra capability in the protocol.\n\n One question is what to do with delta chains including unwanted\nobjects, but I think that given the objects' associativity for delta\nchains, this shouldn't be huge practical issues and it could be\naffordable in principle to include even unwanted objects.\n\n> ... and I am sure there are tons more issues.\n\n I do agree on this. :-)\n\n-- \n\t\t\t\tPetr \"Pasky\" Baudis\nAs in certain cults it is possible to kill a process if you know\nits true name. -- Ken Thompson and Dennis M. Ritchie\n"},{"id":"84777","messageId":"bd6139dc0807241201v50cd5ef2m58ee7efc05119e20@mail.gmail.com","threadId":"14628","inReplyTo":"20080724185332.GQ32184@machine.or.cz","subject":"Re: sparse fetch, was Re: [PATCH 08/12] git-clone: support --path to do sparse clone","fromName":"Sverre Rabbelier","fromEmail":"alturin@gmail.com","sentAt":"2008-07-24T19:01:15Z","receivedAt":"2008-07-24T19:01:15Z","isPatch":true,"sender":{"key":"alturin@gmail.com","avatar":null},"body":"On Thu, Jul 24, 2008 at 8:53 PM, Petr Baudis wrote:\n> I don't follow how these two issues arise, if the server will do the\n> pruning for you. It will just skip entering some tree objects when doing\n> object traversal; why opening the git protocol or faking commits? This\n> would be a simple extra capability in the protocol.\n\nWouldn't that be as simple as passing a pathspec to git-rev-list? Not\na lot of overhead there I reckon.\n\n> One question is what to do with delta chains including unwanted\n> objects, but I think that given the objects' associativity for delta\n> chains, this shouldn't be huge practical issues and it could be\n> affordable in principle to include even unwanted objects.\n\nJust keep them? What we're doing here is trying to optimize in the\ncase that someone has a sparse checkout, nothing bad will happen if\nthey get too many info surely? (Save for them not getting as much\nimprovement as would have been possible would the pack have been\ncreated differently.)\n\n-- \nCheers,\n\nSverre Rabbelier\n"},{"id":"84814","messageId":"alpine.DEB.1.00.0807250208460.4140@eeepc-johanness","threadId":"14628","inReplyTo":"20080724182813.GA21186@sigill.intra.peff.net","subject":"Re: sparse fetch, was Re: [PATCH 08/12] git-clone: support --path to do sparse clone","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2008-07-25T00:09:25Z","receivedAt":"2008-07-25T00:09:25Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi,\n\nOn Thu, 24 Jul 2008, Jeff King wrote:\n\n> On Thu, Jul 24, 2008 at 06:41:03PM +0100, Johannes Schindelin wrote:\n> \n> > > As a user, I would expect \"sparse clone\" to also be sparse on the \n> > > fetching. That is, to not even bother fetching tree objects that we \n> > > are not going to check out. But that is a whole other can of worms \n> > > from local sparseness, so I think it is worth saving for a different \n> > > series.\n> > \n> > I think this is not even worth of a series. Sure, it would have \n> > benefits for those who want sparse checkouts. But it comes for a high \n> > price on everyone else:\n> \n> I agree there are a lot of issues. I am just thinking of the person who \n> said they had a >100G repository. But I am also not volunteering to do \n> it, so I will let somebody who really cares about it try to defend the \n> idea.\n\nI never said that there were no benefits. I argued that there are too \nmany _downsides_ to those who _don't_ benefit.\n\nCiao,\nDscho\n"},{"id":"84815","messageId":"alpine.DEB.1.00.0807250210090.4140@eeepc-johanness","threadId":"14628","inReplyTo":"bd6139dc0807241201v50cd5ef2m58ee7efc05119e20@mail.gmail.com","subject":"Re: sparse fetch, was Re: [PATCH 08/12] git-clone: support --path to do sparse clone","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2008-07-25T00:12:31Z","receivedAt":"2008-07-25T00:12:31Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi,\n\nOn Thu, 24 Jul 2008, Sverre Rabbelier wrote:\n\n> On Thu, Jul 24, 2008 at 8:53 PM, Petr Baudis wrote:\n> > I don't follow how these two issues arise, if the server will do the \n> > pruning for you. It will just skip entering some tree objects when \n> > doing object traversal; why opening the git protocol or faking \n> > commits? This would be a simple extra capability in the protocol.\n> \n> Wouldn't that be as simple as passing a pathspec to git-rev-list? Not a \n> lot of overhead there I reckon.\n\nSo the server would _not_ have to deflate the objects to inspect them? I \nthought you knew more about Git's object database.\n\n> > One question is what to do with delta chains including unwanted \n> > objects, but I think that given the objects' associativity for delta \n> > chains, this shouldn't be huge practical issues and it could be \n> > affordable in principle to include even unwanted objects.\n> \n> Just keep them?\n\nYou'd still have to inspect the objects, which is way more work than the \ncurrent code has to do. Remember: in the optimal case, upload-pack does \nnot more than just serve the existing deltas/base objects.\n\nCiao,\nDscho\n"},{"id":"84817","messageId":"20080725004210.GO10151@machine.or.cz","threadId":"14628","inReplyTo":"alpine.DEB.1.00.0807250210090.4140@eeepc-johanness","subject":"Re: sparse fetch, was Re: [PATCH 08/12] git-clone: support --path to do sparse clone","fromName":"Petr Baudis","fromEmail":"pasky@suse.cz","sentAt":"2008-07-25T00:42:10Z","receivedAt":"2008-07-25T00:42:10Z","isPatch":true,"sender":{"key":"pasky@ucw.cz","avatar":"https://avatars.githubusercontent.com/u/18439?v=4"},"body":" Hi,\n\nOn Fri, Jul 25, 2008 at 02:12:31AM +0200, Johannes Schindelin wrote:\n> On Thu, 24 Jul 2008, Sverre Rabbelier wrote:\n> \n> > On Thu, Jul 24, 2008 at 8:53 PM, Petr Baudis wrote:\n> > > I don't follow how these two issues arise, if the server will do the \n> > > pruning for you. It will just skip entering some tree objects when \n> > > doing object traversal; why opening the git protocol or faking \n> > > commits? This would be a simple extra capability in the protocol.\n> > \n> > Wouldn't that be as simple as passing a pathspec to git-rev-list? Not a \n> > lot of overhead there I reckon.\n> \n> So the server would _not_ have to deflate the objects to inspect them? I \n> thought you knew more about Git's object database.\n..snip..\n> You'd still have to inspect the objects, which is way more work than the \n> current code has to do. Remember: in the optimal case, upload-pack does \n> not more than just serve the existing deltas/base objects.\n\n then right now, exactly how does the server decide that the blob\n7a7ff130 should be served along git.git HEAD? I still see upload-pack.c\ncalling traverse_commit_list() that does process_tree() on every tree,\netc. But the code is not straightforward, maybe I'm missing some\nshortcut?\n\n-- \n\t\t\t\tPetr \"Pasky\" Baudis\nAs in certain cults it is possible to kill a process if you know\nits true name. -- Ken Thompson and Dennis M. Ritchie\n"},{"id":"84818","messageId":"loom.20080725T004541-298@post.gmane.org","threadId":"14628","inReplyTo":"20080724182813.GA21186@sigill.intra.peff.net","subject":"Re: sparse fetch, was Re: [PATCH 08/12] git-clone: support --path to do sparse clone","fromName":"James Pickens","fromEmail":"jepicken@gmail.com","sentAt":"2008-07-25T00:46:52Z","receivedAt":"2008-07-25T00:46:52Z","isPatch":true,"sender":{"key":"jepicken@gmail.com","avatar":null},"body":"Jeff King peff.net> writes:\n\n> I agree there are a lot of issues. I am just thinking of the person who\n> said they had a >100G repository. But I am also not volunteering to do\n> it, so I will let somebody who really cares about it try to defend the\n> idea.\n\nIf you're referring to me (I mentioned a 144G CVS repo), then let me\nclarify a couple of things:\n\n1. Probably more than 50% of the 144G is crud that should never have\nbeen checked in, but I have some undisciplined coworkers who like to\nblindly check in everything in their work trees. If/when we moved to\ngit, I would get rid of all that crud. I'm also thinking about throwing\nout a lot of the history, since those same undisciplined coworkers like\nto use empty and/or useless log messages, so a lot of the history isn't\nvery valuable anyways.\n\n2. Git of course will store the remaining ~70G much more efficiently\nthan CVS. I think git will be especially better than CVS for this repo,\nbecause it contains many instances of the same file(s) being checked in\nin multiple directories.\n\nI expect the git repo size to be less than 7G. In addition, all our\nwork is done on site on nfs, so we can use clone -s to avoid copying the\nwhole 7G.\n\nTo sum it up, sparse cloning would not be important to me.\n\nJames\n"},{"id":"84820","messageId":"20080725004954.GA24553@sigill.intra.peff.net","threadId":"14628","inReplyTo":"loom.20080725T004541-298@post.gmane.org","subject":"Re: sparse fetch, was Re: [PATCH 08/12] git-clone: support --path?to do sparse clone","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2008-07-25T00:49:54Z","receivedAt":"2008-07-25T00:49:54Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Fri, Jul 25, 2008 at 12:46:52AM +0000, James Pickens wrote:\n\n> If you're referring to me (I mentioned a 144G CVS repo), then let me\n> clarify a couple of things:\n> [...]\n> To sum it up, sparse cloning would not be important to me.\n\nI was referring to you, so thanks for the clarification.\n\n-Peff\n"},{"id":"84843","messageId":"bd6139dc0807250114qc1c6cf5ufa7df0862507010d@mail.gmail.com","threadId":"14628","inReplyTo":"alpine.DEB.1.00.0807250210090.4140@eeepc-johanness","subject":"Re: sparse fetch, was Re: [PATCH 08/12] git-clone: support --path to do sparse clone","fromName":"Sverre Rabbelier","fromEmail":"alturin@gmail.com","sentAt":"2008-07-25T08:14:16Z","receivedAt":"2008-07-25T08:14:16Z","isPatch":true,"sender":{"key":"alturin@gmail.com","avatar":null},"body":"On Fri, Jul 25, 2008 at 02:12, Johannes Schindelin\n wrote:\n> On Thu, 24 Jul 2008, Sverre Rabbelier wrote:\n>> Wouldn't that be as simple as passing a pathspec to git-rev-list? Not a\n>> lot of overhead there I reckon.\n>\n> So the server would _not_ have to deflate the objects to inspect them? I\n> thought you knew more about Git's object database.\n\nNope, I did not know this. I thought that the server already had to do\nall that to decide what to send, since not every request asks for the\nsame pack (someone might not have updated in 2 month, or someone might\nmight be up to date, or anything in between).\n\n>> Just keep them?\n>\n> You'd still have to inspect the objects, which is way more work than the\n> current code has to do. Remember: in the optimal case, upload-pack does\n> not more than just serve the existing deltas/base objects.\n\nAh, I can see how that would produce overhead then.\n\n-- \nCheers,\n\nSverre Rabbelier\n"},{"id":"84849","messageId":"7v7ibauz94.fsf@gitster.siamese.dyndns.org","threadId":"14628","inReplyTo":"20080724182813.GA21186@sigill.intra.peff.net","subject":"Re: sparse fetch, was Re: [PATCH 08/12] git-clone: support --path to do sparse clone","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2008-07-25T08:47:03Z","receivedAt":"2008-07-25T08:47:03Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jeff King writes:\n\n> On Thu, Jul 24, 2008 at 06:41:03PM +0100, Johannes Schindelin wrote:\n>\n>> > As a user, I would expect \"sparse clone\" to also be sparse on the \n>> > fetching. That is, to not even bother fetching tree objects that we are \n>> > not going to check out. But that is a whole other can of worms from \n>> > local sparseness, so I think it is worth saving for a different series.\n>> \n>> I think this is not even worth of a series. Sure, it would have benefits \n>> for those who want sparse checkouts. But it comes for a high price on \n>> everyone else:\n>\n> I agree there are a lot of issues. I am just thinking of the person who\n> said they had a >100G repository. But I am also not volunteering to do\n> it, so I will let somebody who really cares about it try to defend the\n> idea.\n\nI think sparse fetch is a lot worse than grafts and shallow clones which\nare already bad. These are all ways to introduce local inconsistency at\nthe object level and pretend everything is Ok, but the latter two do so\nonly at commit boundary and it is somewhat more manageable (but we still\ndo not handle it very well). With sparse fetch, you cannot even guarantee\nthe integrity of individual commits with subtrees here and there missing.\n\nI do think shallow checkout that says \"I'll have the whole tree in the\nindex but the work tree will have only these paths checked out\" makes\nsense. You do not need a fully populated work tree to create commits or\nmerges -- the only absolute minimum you need is a fully populated index.\n\nIn that sense, I think \"protect index entries outside of these paths\" (I\nremember that the first round of this series was done around that notion)\nis a wrong mentality to handle this. We should think of this as more like\n\"you still populate the index with the whole tree, and you are free to\nupdate them in any way you want, but we do not touch work tree outside\nthese areas\".\n\nThis has a few ramifications:\n\n - If the user can somehow check out a path outside the \"sparse\" area, it\n is perfectly fine for the user to edit and \"git add\" it. Such a method\n to check out a path outside the \"sparse\" area is a way to widen the\n \"sparse\" area the user originally set up;\n\n - When the user runs \"merge\", and it needs to present the user a working\n tree file because of conflicts at the file level, the user has to agree\n to widen the \"sparse\" area before being able to do so. One way to do\n this is to refuse and fail the merge (and then the user needs to do\n that \"unspecified way\" of widening the \"sparse\" area first). Another\n way would be to automatically widen the \"sparse\" area to include such\n conflicting paths.\n\n - And you would want to narrow it down after you do such a widening.\n\nFor many projects that has src/ and doc/ (git.git being one of them), it\nis perfectly valid for a code person and a doc person to work in tandem.\nIn such a project, after the code person makes changes in her sparsely\nchecked out repository and making changes only to the src/ area and pushes\nthe results out, the doc person would run \"git pull && git log -p\nORIG_HEAD\" and updates the documentation in his sparsely checked out\nrepository that has only doc/ area. The two parts are tied together and\nthey advance more or less in sync. I think sparse checkout would be a\nuseful feature to help such a configuration.\n\nHaving said that, I however think that this can easily be misused as a CVS\nstyle \"one CVSROOT houses millions of totally unrelated projects\" layout.\nIn CVS, the layout is perfectly fine because the system does not track\nchanges at anything higher than the level of individual files, but when\nyou naïvely map the layout to a system with tree-wide atomic commits, such\nas git, it will defeat the whole point of using such a system. The pace\nthese millions of unrelated projects advance do not have any relationship\nwith each other, but by tying them together in the same top-level tree,\nthe layout is introducing an unnecessary ordering between their commits.\n"},{"id":"84851","messageId":"bd6139dc0807250154r141a5dabn9585f3e23e7736ca@mail.gmail.com","threadId":"14628","inReplyTo":"7v7ibauz94.fsf@gitster.siamese.dyndns.org","subject":"Re: sparse fetch, was Re: [PATCH 08/12] git-clone: support --path to do sparse clone","fromName":"Sverre Rabbelier","fromEmail":"alturin@gmail.com","sentAt":"2008-07-25T08:54:53Z","receivedAt":"2008-07-25T08:54:53Z","isPatch":true,"sender":{"key":"alturin@gmail.com","avatar":null},"body":"On Fri, Jul 25, 2008 at 10:47, Junio C Hamano wrote:\n> For many projects that has src/ and doc/ (git.git being one of them), it\n\nWe are? That's great, that'd mean I can actually do a 'ls' in the\ngit.git root! Oh wait...\n$ ls | wc -l\n693\n\n-- \nCheers,\n\nSverre Rabbelier\n"}]}