{"thread":{"id":"12622","subject":"[QUESTION] Selective fetch possible?","startedAt":"2008-03-10T22:02:27Z","lastAt":"2008-03-11T12:29:10Z","messageCount":7,"participants":["Filippo Zangheri","Shawn O. Pearce","Jakub Narebski","Rogan Dawes"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"71621","messageId":"47D5AFF3.90000@yahoo.it","threadId":"12622","inReplyTo":null,"subject":"[QUESTION] Selective fetch possible?","fromName":"Filippo Zangheri","fromEmail":"filippo.zangheri@yahoo.it","sentAt":"2008-03-10T22:02:27Z","receivedAt":"2008-03-10T22:02:27Z","isPatch":false,"sender":{"key":"filippo.zangheri@yahoo.it","avatar":null},"body":"Hi everybody,\n\nIs it possible to git-fetch only a portion of the tree\nof the specified repository, say, fetch only one directory or a\nsubset of files matching some regular expression? This is currently\n- to my knowledge - only possible via wget iff the GIT repository\nhas gitweb enabled. But that's just a workaround.\n\nThanks for the great instrument that GIT is.\nBest regards.\n\nPS: please CC me in replies, because I'm not subscribed to git ML.\n\n-- \nFilippo Zangheri\n\nGPG key ID: 0xE1D879FA\nKey fingerprint: 816B CE57 D43C 0A47 EF35 3378 EA5F A72A E1D8 79FA\nKey server: pgp.mit.edu\n\n-----BEGIN GEEK CODE BLOCK-----\nVersion: 3.12\nGE d- s+:- a-- C++ UL+++ P+ L+++ E-- W+ N* o-- K- w--- O-- M--\nV- PS++ PE+ Y+ PGP++ t 5-- X++ R* tv b+ DI-- D---- G-- e++ h--\nr++ z*\n------END GEEK CODE BLOCK------\n"},{"id":"71622","messageId":"20080310225302.GE8410@spearce.org","threadId":"12622","inReplyTo":"47D5AFF3.90000@yahoo.it","subject":"Re: [QUESTION] Selective fetch possible?","fromName":"Shawn O. Pearce","fromEmail":"spearce@spearce.org","sentAt":"2008-03-10T22:53:02Z","receivedAt":"2008-03-10T22:53:02Z","isPatch":false,"sender":{"key":"spearce@spearce.org","avatar":"https://avatars.githubusercontent.com/u/34844?v=4"},"body":"Filippo Zangheri wrote:\n> Is it possible to git-fetch only a portion of the tree\n> of the specified repository, say, fetch only one directory or a\n> subset of files matching some regular expression? This is currently\n> - to my knowledge - only possible via wget iff the GIT repository\n> has gitweb enabled. But that's just a workaround.\n\nNo.\n\nYou can use a shallow clone to fetch only X commits back into\nhistory on any branch, and you can also manually configure the\nfetch specification in .git/config to only fetch specific branches,\nbut you must fetch the entire tree to get any of the files in it.\n\nIf the repository is available by git:// protocol you may be able\nto use git-archive to obtain a tarfile for just the directory you\nwant (service has to be enabled on the remote side) but that is\njust a raw UNIX tar; there is no Git repository and no ability to\ncommit/fetch/push/diff/apply/log/etc.\n \n-- \nShawn.\n"},{"id":"71626","messageId":"m3d4q2ywyx.fsf@localhost.localdomain","threadId":"12622","inReplyTo":"20080310225302.GE8410@spearce.org","subject":"Re: [QUESTION] Selective fetch possible?","fromName":"Jakub Narebski","fromEmail":"jnareb@gmail.com","sentAt":"2008-03-10T23:34:59Z","receivedAt":"2008-03-10T23:34:59Z","isPatch":false,"sender":{"key":"jnareb@gmail.com","avatar":"https://avatars.githubusercontent.com/u/2706?v=4"},"body":"\"Shawn O. Pearce\" writes:\n\n> Filippo Zangheri wrote:\n> >\n> > Is it possible to git-fetch only a portion of the tree\n> > of the specified repository, say, fetch only one directory or a\n> > subset of files matching some regular expression? This is currently\n> > - to my knowledge - only possible via wget iff the GIT repository\n> > has gitweb enabled. But that's just a workaround.\n> \n> No.\n> \n> You can use a shallow clone to fetch only X commits back into\n> history on any branch, and you can also manually configure the\n> fetch specification in .git/config to only fetch specific branches,\n> but you must fetch the entire tree to get any of the files in it.\n> \n> If the repository is available by git:// protocol you may be able\n> to use git-archive to obtain a tarfile for just the directory you\n> want (service has to be enabled on the remote side) but that is\n> just a raw UNIX tar; there is no Git repository and no ability to\n> commit/fetch/push/diff/apply/log/etc.\n\nNote that what you wanted is, I guess, something called partial\ncheckout or subtree checkout. This feature appears now and then in\nfeature requests; lately Nguyen Thai Ngoc Duy (pclouds) offered to do\nthis in \"on subtree checkout\" thread:\n http://thread.gmane.org/gmane.comp.version-control.git/74915\n\nThe problem is twofold, as far as I understand it. First, what to do\nif there is merge conflicts outside checked out (selected) directory?\nSecond, how to make repository contain only relevant objects: git in\nmany places assumes full connectivity, and that if it has an object it\nhass all objects depending on it.\n\n-- \nJakub Narebski\nPoland\nShadeHawk on #git\n"},{"id":"71662","messageId":"47D6341C.7030505@dawes.za.net","threadId":"12622","inReplyTo":"m3d4q2ywyx.fsf@localhost.localdomain","subject":"Re: [QUESTION] Selective fetch possible?","fromName":"Rogan Dawes","fromEmail":"lists@dawes.za.net","sentAt":"2008-03-11T07:26:20Z","receivedAt":"2008-03-11T07:26:20Z","isPatch":false,"sender":{"key":"lists@dawes.za.net","avatar":null},"body":"Jakub Narebski wrote:\n> \"Shawn O. Pearce\" writes:\n> \n>> Filippo Zangheri wrote:\n>>> Is it possible to git-fetch only a portion of the tree\n>>> of the specified repository, say, fetch only one directory or a\n>>> subset of files matching some regular expression? This is currently\n>>> - to my knowledge - only possible via wget iff the GIT repository\n>>> has gitweb enabled. But that's just a workaround.\n>> No.\n>>\n>> You can use a shallow clone to fetch only X commits back into\n>> history on any branch, and you can also manually configure the\n>> fetch specification in .git/config to only fetch specific branches,\n>> but you must fetch the entire tree to get any of the files in it.\n>>\n>> If the repository is available by git:// protocol you may be able\n>> to use git-archive to obtain a tarfile for just the directory you\n>> want (service has to be enabled on the remote side) but that is\n>> just a raw UNIX tar; there is no Git repository and no ability to\n>> commit/fetch/push/diff/apply/log/etc.\n> \n> Note that what you wanted is, I guess, something called partial\n> checkout or subtree checkout. This feature appears now and then in\n> feature requests; lately Nguyen Thai Ngoc Duy (pclouds) offered to do\n> this in \"on subtree checkout\" thread:\n> http://thread.gmane.org/gmane.comp.version-control.git/74915\n\nI still believe that it could be done fairly easily by simply recording \nthe SHA1's of the files and directories that are *not* checked out \nsomewhere in the .git directory, and just reusing those when checking \nthe working tree. i.e. rather than stat-ing a tree that was never \nchecked out, get the known SHA1 for that tree from where it was recorded \non checkout.\n\n> The problem is twofold, as far as I understand it. First, what to do\n> if there is merge conflicts outside checked out (selected) directory?\n\nThis is something that has been repeated many times, and I fail to see \nhow it can be an issue. How can there be a conflict in a directory that \nis not, and never has been, checked out, and therefore cannot have been \nmodified?\n\nThe only possibility that I can see is if the directory has been renamed \nelsewhere, but in that case, it *is* effectively checked out (just with \na different directory name).\n\n> Second, how to make repository contain only relevant objects: git in\n> many places assumes full connectivity, and that if it has an object it\n> hass all objects depending on it.\n> \n\nYes, this is the big problem as I see it.\n\nRogan\n"},{"id":"71665","messageId":"20080311075053.GQ8410@spearce.org","threadId":"12622","inReplyTo":"47D6341C.7030505@dawes.za.net","subject":"Re: [QUESTION] Selective fetch possible?","fromName":"Shawn O. Pearce","fromEmail":"spearce@spearce.org","sentAt":"2008-03-11T07:50:53Z","receivedAt":"2008-03-11T07:50:53Z","isPatch":false,"sender":{"key":"spearce@spearce.org","avatar":"https://avatars.githubusercontent.com/u/34844?v=4"},"body":"Rogan Dawes wrote:\n> Jakub Narebski wrote:\n> >\"Shawn O. Pearce\" writes:\n> >\n> >>Filippo Zangheri wrote:\n> >>>Is it possible to git-fetch only a portion of the tree\n> >>>of the specified repository, say, fetch only one directory or a\n> >>>subset of files matching some regular expression?\n> \n> >The problem is twofold, as far as I understand it. First, what to do\n> >if there is merge conflicts outside checked out (selected) directory?\n> \n> This is something that has been repeated many times, and I fail to see \n> how it can be an issue. How can there be a conflict in a directory that \n> is not, and never has been, checked out, and therefore cannot have been \n> modified?\n\nGiven two branches:\n\n\tcode\n\tdocs\n\nand the code people checkout the \"src/\" subdirectory and the docs\npeople checkout the \"Documentation/\" subdirectory, and they *only*\nevery work in that subdirectory, things are fine.\n\nUntil one day some developer also checks out \"Documentation/\" and\nfixes something in the documentation as part of the same commit\nthat makes a code change. The push this to the code branch.\n\nSomeday in the future a documentation writer merges the code branch\nover to the docs branch, \"just keeping it current\".\n\nNow there arises a possiblity of a merge conflict in a part of the\ntree that you do not have checked out.\n\n\nIf you want to say \"don't ever modify stuff outside of your branch's\npurpose\" then why aren't you just using submodules (one for docs and\none for code) and using a supermodule to tie everything together into\na \"release package\"?\n\n> >Second, how to make repository contain only relevant objects: git in\n> >many places assumes full connectivity, and that if it has an object it\n> >hass all objects depending on it.\n> >\n> \n> Yes, this is the big problem as I see it.\n\nThis is easy enough that if the above problem could be resolved\nsufficiently to the git gurus' satisfaction you would be able\nto get some advice on how to solve it. Its not difficult, just\ndamn annoying. We already do it (to some extent) with grafts and\nshallow clones.\n\n-- \nShawn.\n"},{"id":"71671","messageId":"47D64BE4.9010009@dawes.za.net","threadId":"12622","inReplyTo":"20080311075053.GQ8410@spearce.org","subject":"Re: [QUESTION] Selective fetch possible?","fromName":"Rogan Dawes","fromEmail":"lists@dawes.za.net","sentAt":"2008-03-11T09:07:48Z","receivedAt":"2008-03-11T09:07:48Z","isPatch":false,"sender":{"key":"lists@dawes.za.net","avatar":null},"body":"Shawn O. Pearce wrote:\n> Rogan Dawes wrote:\n>> Jakub Narebski wrote:\n>>> \"Shawn O. Pearce\" writes:\n>>>\n>>>> Filippo Zangheri wrote:\n>>>>> Is it possible to git-fetch only a portion of the tree\n>>>>> of the specified repository, say, fetch only one directory or a\n>>>>> subset of files matching some regular expression?\n>>> The problem is twofold, as far as I understand it. First, what to do\n>>> if there is merge conflicts outside checked out (selected) directory?\n>> This is something that has been repeated many times, and I fail to see \n>> how it can be an issue. How can there be a conflict in a directory that \n>> is not, and never has been, checked out, and therefore cannot have been \n>> modified?\n> \n> Given two branches:\n> \n> \tcode\n> \tdocs\n> \n> and the code people checkout the \"src/\" subdirectory and the docs\n> people checkout the \"Documentation/\" subdirectory, and they *only*\n> every work in that subdirectory, things are fine.\n> \n> Until one day some developer also checks out \"Documentation/\" and\n> fixes something in the documentation as part of the same commit\n> that makes a code change. The push this to the code branch.\n> \n> Someday in the future a documentation writer merges the code branch\n> over to the docs branch, \"just keeping it current\".\n> \n> Now there arises a possiblity of a merge conflict in a part of the\n> tree that you do not have checked out.\n> \n> \n> If you want to say \"don't ever modify stuff outside of your branch's\n> purpose\" then why aren't you just using submodules (one for docs and\n> one for code) and using a supermodule to tie everything together into\n> a \"release package\"?\n\nOk, fair enough. Thanks for the example.\n\nI think that one should not *expect* to be able to complete merges with \nonly a partial checkout, though. It *may* work in cases where there are \nno conflicts, but I think it would be a perfectly valid error path to \nfail if there is a conflicting merge in a part of the tree that has not \nbeen checked out.\n\nSo, for a user working on partial trees, they would be able to modify \ntheir partial tree, and check in their changes, but merges would have to \nbe done by someone with a complete checkout. For the given examples \nwhere partial trees make sense (documentation workers), this seems like \na reasonable compromise.\n\n>>> Second, how to make repository contain only relevant objects: git in\n>>> many places assumes full connectivity, and that if it has an object it\n>>> hass all objects depending on it.\n>>>\n>> Yes, this is the big problem as I see it.\n> \n> This is easy enough that if the above problem could be resolved\n> sufficiently to the git gurus' satisfaction you would be able\n> to get some advice on how to solve it. Its not difficult, just\n> damn annoying. We already do it (to some extent) with grafts and\n> shallow clones.\n\nHow's my suggestion above?\n\nRogan\n"},{"id":"71687","messageId":"47D67B16.8040709@yahoo.it","threadId":"12622","inReplyTo":"47D64BE4.9010009@dawes.za.net","subject":"Re: [QUESTION] Selective fetch possible?","fromName":"Filippo Zangheri","fromEmail":"filippo.zangheri@yahoo.it","sentAt":"2008-03-11T12:29:10Z","receivedAt":"2008-03-11T12:29:10Z","isPatch":false,"sender":{"key":"filippo.zangheri@yahoo.it","avatar":null},"body":"Rogan Dawes ha scritto:\n(...)\n> I think that one should not *expect* to be able to complete merges with \n> only a partial checkout, though. It *may* work in cases where there are \n> no conflicts, but I think it would be a perfectly valid error path to \n> fail if there is a conflicting merge in a part of the tree that has not \n> been checked out.\n> \n> So, for a user working on partial trees, they would be able to modify \n> their partial tree, and check in their changes, but merges would have to \n> be done by someone with a complete checkout. For the given examples \n> where partial trees make sense (documentation workers), this seems like \n> a reasonable compromise.\n\nI think this is what every reasonable developer should have in mind\nwhen he's working on just a project subset :). And I also think,\nthis is not a valid reason for forbidding/not implementing such a\npartial (or subtree) checkout.\n\n\n-- \nFilippo Zangheri\n\nGPG key ID: 0xE1D879FA\nKey fingerprint: 816B CE57 D43C 0A47 EF35 3378 EA5F A72A E1D8 79FA\nKey server: pgp.mit.edu\n\n-----BEGIN GEEK CODE BLOCK-----\nVersion: 3.12\nGE d- s+:- a-- C++ UL+++ P+ L+++ E-- W+ N* o-- K- w--- O-- M--\nV- PS++ PE+ Y+ PGP++ t 5-- X++ R* tv b+ DI-- D---- G-- e++ h--\nr++ z*\n------END GEEK CODE BLOCK------\n"}]}