{"thread":{"id":"43106","subject":"Re: [RFC] Submodules in GIT","startedAt":"2006-11-20T22:16:45Z","lastAt":"2006-12-02T15:16:30Z","messageCount":82,"participants":["Sven Verdoolaege","Daniel Barkalow","Andy Parkins","Shawn Pearce","Jakub Narebski","Martin Waitz","Steven Grimm","Yann Dirson","Linus Torvalds","sf","Andreas Ericsson","Stephan Feder","Junio C Hamano","J. Bruce Fields","Sam Vilain"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"296533","messageId":"ejt9dh$kfm$1@sea.gmane.org","threadId":"43106","inReplyTo":"20061120215116.GA20736@admingilde.org","subject":"Re: [RFC] Submodules in GIT","fromName":"Jakub Narebski","fromEmail":"jnareb@gmail.com","sentAt":"2006-11-20T22:16:45Z","receivedAt":"2006-11-20T22:16:45Z","isPatch":false,"sender":{"key":"jnareb@gmail.com","avatar":"https://avatars.githubusercontent.com/u/2706?v=4"},"body":"Martin Waitz wrote:\n\n> A submodule really is part of the parent tree, so it is very natural to\n> add the link to the submodule commit into the GIT tree data structure.\n> In addition to links to blobs and other trees, they can now also hold\n> a link to a commit, which in turn has the pointers to the submodule tree\n> and its history.  In order to differenciate a submodule entry with\n> normal file or directory entries, they get a special file mode.\n\nErm... isn't a _type_ of tree entry saved somewhere? Currently it can\nbe only 'tree' or 'blob', what you do is adding 'commit' (then permissions\nare permissions of top tree of module, of course).\n\nBy the way, in todo branch, in Subpro.txt, there is talk about adding\nlink to submodule trees in _commit object_... well link to submodule tree\nor commit, with the \"mount point\".\n-- \nJakub Narebski\nWarsaw, Poland\nShadeHawk on #git\n\n"},{"id":"297205","messageId":"20061120222853.GB20736@admingilde.org","threadId":"43106","inReplyTo":"ejt9dh$kfm$1@sea.gmane.org","subject":"Re: [RFC] Submodules in GIT","fromName":"Martin Waitz","fromEmail":"tali@admingilde.org","sentAt":"2006-11-20T22:28:53Z","receivedAt":"2006-11-20T22:28:53Z","isPatch":false,"sender":{"key":"tali@admingilde.org","avatar":"https://gravatar.com/avatar/3f89b03eee362187effabe257898735b475673a12265c398ea9161259ae91553?d=mp&s=160"},"body":"hoi :)\n\nOn Mon, Nov 20, 2006 at 11:16:45PM +0100, Jakub Narebski wrote:\n> Martin Waitz wrote:\n> > A submodule really is part of the parent tree, so it is very natural to\n> > add the link to the submodule commit into the GIT tree data structure.\n> > In addition to links to blobs and other trees, they can now also hold\n> > a link to a commit, which in turn has the pointers to the submodule tree\n> > and its history.  In order to differenciate a submodule entry with\n> > normal file or directory entries, they get a special file mode.\n>\n> Erm... isn't a _type_ of tree entry saved somewhere? Currently it can\n> be only 'tree' or 'blob', what you do is adding 'commit' (then permissions\n> are permissions of top tree of module, of course).\n\nIt is saved inside the object which is being refered to.\nRight now tree objects are also identified by their file mode and not\nby the type of object which is referenced.\n\n> By the way, in todo branch, in Subpro.txt, there is talk about adding\n> link to submodule trees in _commit object_... well link to submodule tree\n> or commit, with the \"mount point\".\n\nBut isn't the submodule really part of the tree?\nRight now the commit is used to construct the history of one project.\nAnd a submodule is not part of the history of the parent, it is part\nof the parent's tree.\n\n-- \nMartin Waitz\n"},{"id":"296797","messageId":"7v7ixp20za.fsf@assigned-by-dhcp.cox.net","threadId":"43106","inReplyTo":"ejt9dh$kfm$1@sea.gmane.org","subject":"Re: [RFC] Submodules in GIT","fromName":"Junio C Hamano","fromEmail":"junkio@cox.net","sentAt":"2006-11-20T22:43:21Z","receivedAt":"2006-11-20T22:43:21Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jakub Narebski writes:\n\n> By the way, in todo branch, in Subpro.txt, there is talk about adding\n> link to submodule trees in _commit object_... well link to submodule tree\n> or commit, with the \"mount point\".\n\nThat was shot down by Linus and I agree with him. \"bind\" was a\nbad idea because binding of a particular subproject commit into\na tree is a property of the tree, not one of the commits that\nhappen to have that tree.\n"},{"id":"298526","messageId":"ejtc3e$tod$2@sea.gmane.org","threadId":"43106","inReplyTo":"7v7ixp20za.fsf@assigned-by-dhcp.cox.net","subject":"Re: [RFC] Submodules in GIT","fromName":"Jakub Narebski","fromEmail":"jnareb@gmail.com","sentAt":"2006-11-20T23:02:34Z","receivedAt":"2006-11-20T23:02:34Z","isPatch":false,"sender":{"key":"jnareb@gmail.com","avatar":"https://avatars.githubusercontent.com/u/2706?v=4"},"body":"Junio C Hamano wrote:\n\n> Jakub Narebski writes:\n> \n>> By the way, in todo branch, in Subpro.txt, there is talk about adding\n>> link to submodule trees in _commit object_... well link to submodule tree\n>> or commit, with the \"mount point\".\n> \n> That was shot down by Linus and I agree with him. \"bind\" was a\n> bad idea because binding of a particular subproject commit into\n> a tree is a property of the tree, not one of the commits that\n> happen to have that tree.\n \n\"bind\" was kind of \"mount tree\" idea; I agree that adding subproject\ncommits to trees is better idea than adding commits or trees to\nsuperproject commit object.\n\nBy the way, what permissions get the subproject tree?\n\nI wonder if it makes sense to be able to add tag objects instead\nof commit objects to trees (depeel to tree or blob)...\n-- \nJakub Narebski\nWarsaw, Poland\nShadeHawk on #git\n\n"},{"id":"294999","messageId":"Pine.LNX.4.64.0611201501230.3338@woody.osdl.org","threadId":"43106","inReplyTo":"7v7ixp20za.fsf@assigned-by-dhcp.cox.net","subject":"Re: [RFC] Submodules in GIT","fromName":"Linus Torvalds","fromEmail":"torvalds@osdl.org","sentAt":"2006-11-20T23:05:47Z","receivedAt":"2006-11-20T23:05:47Z","isPatch":false,"sender":{"key":"torvalds@linux-foundation.org","avatar":"https://avatars.githubusercontent.com/u/1024025?v=4"},"body":"\n\nOn Mon, 20 Nov 2006, Junio C Hamano wrote:\n> \n> That was shot down by Linus and I agree with him. \"bind\" was a\n> bad idea because binding of a particular subproject commit into\n> a tree is a property of the tree, not one of the commits that\n> happen to have that tree.\n\nYes. I think it would be a _fine_ idea to have a new tree-entry type that \npoints to a sub-commit, but it really does need to be on a \"tree level\", \nnot a commit level.\n\nIf it's on a tree level, getting things like \"git diff\" etc to work is not \nimpossible, and it will also fit very well into the whole git \ninfrastructure.\n\nSo right now a tree entry can be another tree or a blob - and the only \nextension would be to add a \"commit\" type (which would largely _act_ as a \ntree entry, at least for sorting, ie it would use the same \"sorts as if it \nhad a '/' at the end\" logic).\n\nNow, to get everything to work seamlessly within such a commit thing \nmight be a fair amount of work, but I'm not sure you even _need_ to. It \nmight be ok to just say \"subproject 'xyzzy' differs\" in the diff, for \nexample, and have some rudimentary support for \"git status\" etc talking \nabout subprojects that need to be committed.\n\n"},{"id":"296823","messageId":"20061120232507.GH12285@fieldses.org","threadId":"43106","inReplyTo":"Pine.LNX.4.64.0611201501230.3338@woody.osdl.org","subject":"Re: [RFC] Submodules in GIT","fromName":"J. Bruce Fields","fromEmail":"bfields@fieldses.org","sentAt":"2006-11-20T23:25:07Z","receivedAt":"2006-11-20T23:25:07Z","isPatch":false,"sender":{"key":"bfields@citi.umich.edu","avatar":null},"body":"On Mon, Nov 20, 2006 at 03:05:47PM -0800, Linus Torvalds wrote:\n> \n> \n> On Mon, 20 Nov 2006, Junio C Hamano wrote:\n> > \n> > That was shot down by Linus and I agree with him. \"bind\" was a\n> > bad idea because binding of a particular subproject commit into\n> > a tree is a property of the tree, not one of the commits that\n> > happen to have that tree.\n> \n> Yes. I think it would be a _fine_ idea to have a new tree-entry type that \n> points to a sub-commit, but it really does need to be on a \"tree level\", \n> not a commit level.\n\nWould it also be possible to allow the \"Tree:\" line in the commit object\nto refer to a commit, or does the root of the project need to be a\nspecial case?\n\n"},{"id":"296799","messageId":"20061120232904.GC20736@admingilde.org","threadId":"43106","inReplyTo":"Pine.LNX.4.64.0611201501230.3338@woody.osdl.org","subject":"Re: [RFC] Submodules in GIT","fromName":"Martin Waitz","fromEmail":"tali@admingilde.org","sentAt":"2006-11-20T23:29:07Z","receivedAt":"2006-11-20T23:29:07Z","isPatch":false,"sender":{"key":"tali@admingilde.org","avatar":"https://gravatar.com/avatar/3f89b03eee362187effabe257898735b475673a12265c398ea9161259ae91553?d=mp&s=160"},"body":"hoi :)\n\nOn Mon, Nov 20, 2006 at 03:05:47PM -0800, Linus Torvalds wrote:\n> Now, to get everything to work seamlessly within such a commit thing \n> might be a fair amount of work, but I'm not sure you even _need_ to. It \n> might be ok to just say \"subproject 'xyzzy' differs\" in the diff, for \n> example, and have some rudimentary support for \"git status\" etc talking \n> about subprojects that need to be committed.\n\nthis is exactly the status of my implementation at the moment ;-)\n\nWell, it does not yet explicitly tell that a subproject diffs,\nbut it just creates a diff of the two commit objects.\n\nI guess we need some command line option to say if we only want\nto know about that the submodule changes or if the diff should\nrecurse into it.\n\n-- \nMartin Waitz\n"},{"id":"296426","messageId":"20061120233333.GD20736@admingilde.org","threadId":"43106","inReplyTo":"20061120232507.GH12285@fieldses.org","subject":"Re: [RFC] Submodules in GIT","fromName":"Martin Waitz","fromEmail":"tali@admingilde.org","sentAt":"2006-11-20T23:33:34Z","receivedAt":"2006-11-20T23:33:34Z","isPatch":false,"sender":{"key":"tali@admingilde.org","avatar":"https://gravatar.com/avatar/3f89b03eee362187effabe257898735b475673a12265c398ea9161259ae91553?d=mp&s=160"},"body":"hoi :)\n\nOn Mon, Nov 20, 2006 at 06:25:07PM -0500, J. Bruce Fields wrote:\n> On Mon, Nov 20, 2006 at 03:05:47PM -0800, Linus Torvalds wrote:\n> > Yes. I think it would be a _fine_ idea to have a new tree-entry type that \n> > points to a sub-commit, but it really does need to be on a \"tree level\", \n> > not a commit level.\n> \n> Would it also be possible to allow the \"Tree:\" line in the commit object\n> to refer to a commit, or does the root of the project need to be a\n> special case?\n\nthis would then be something like the branch-archival proposal.\nThe user interface for such a beast would be difficult, as you have\nto somehow specify if you mean the inner or outer repository.\n\n-- \nMartin Waitz\n"},{"id":"297805","messageId":"20061120235256.GE20736@admingilde.org","threadId":"43106","inReplyTo":"ejtc3e$tod$2@sea.gmane.org","subject":"Re: [RFC] Submodules in GIT","fromName":"Martin Waitz","fromEmail":"tali@admingilde.org","sentAt":"2006-11-20T23:52:56Z","receivedAt":"2006-11-20T23:52:56Z","isPatch":false,"sender":{"key":"tali@admingilde.org","avatar":"https://gravatar.com/avatar/3f89b03eee362187effabe257898735b475673a12265c398ea9161259ae91553?d=mp&s=160"},"body":"hoi :)\n\nOn Tue, Nov 21, 2006 at 12:02:34AM +0100, Jakub Narebski wrote:\n> By the way, what permissions get the subproject tree?\n\nIn my approach no permissions are saved in the object database,\nonly the special bit to mark the submodule.\nWhen checking out, the directory is created 0777 modulo umask,\njust as other directories. Then the submodule contents\nare checked out with their normal permissions.\n\n-- \nMartin Waitz\n"},{"id":"298130","messageId":"7v4pstzmk5.fsf@assigned-by-dhcp.cox.net","threadId":"43106","inReplyTo":"Pine.LNX.4.64.0611201501230.3338@woody.osdl.org","subject":"Re: [RFC] Submodules in GIT","fromName":"Junio C Hamano","fromEmail":"junkio@cox.net","sentAt":"2006-11-21T00:10:50Z","receivedAt":"2006-11-21T00:10:50Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Linus Torvalds writes:\n\n> Now, to get everything to work seamlessly within such a commit thing \n> might be a fair amount of work, but I'm not sure you even _need_ to. It \n> might be ok to just say \"subproject 'xyzzy' differs\" in the diff, for \n> example, and have some rudimentary support for \"git status\" etc talking \n> about subprojects that need to be committed.\n\nI agree with the static \"diff\" part, and probably \"checkout\" and\n\"merge\" are not all that difficult.\n\nHowever, if I recall correctly, it was rather nightmarish to\nmake this also work for reachability traversal necessary for\npack generation. It was painful enough even when the bind was\nat the commit level (which was way simpler to handle), but to do\nthis the right way, the bind needs to be done at the tree level,\nand \"rev-list --objects foo..bar\" would need some way to limit\nthe commit ancestry chain of subproject at the same time, by\ncomputing the commit ancestry of the embedded commits in the\ntrees.\n"},{"id":"296432","messageId":"ejthuh$fn8$1@sea.gmane.org","threadId":"43106","inReplyTo":"7v4pstzmk5.fsf@assigned-by-dhcp.cox.net","subject":"Re: [RFC] Submodules in GIT","fromName":"Jakub Narebski","fromEmail":"jnareb@gmail.com","sentAt":"2006-11-21T00:42:22Z","receivedAt":"2006-11-21T00:42:22Z","isPatch":false,"sender":{"key":"jnareb@gmail.com","avatar":"https://avatars.githubusercontent.com/u/2706?v=4"},"body":"Junio C Hamano wrote:\n\n> Linus Torvalds writes:\n> \n>> Now, to get everything to work seamlessly within such a commit thing \n>> might be a fair amount of work, but I'm not sure you even _need_ to. It \n>> might be ok to just say \"subproject 'xyzzy' differs\" in the diff, for \n>> example, and have some rudimentary support for \"git status\" etc talking \n>> about subprojects that need to be committed.\n> \n> I agree with the static \"diff\" part, and probably \"checkout\" and\n> \"merge\" are not all that difficult.\n> \n> However, if I recall correctly, it was rather nightmarish to\n> make this also work for reachability traversal necessary for\n> pack generation. It was painful enough even when the bind was\n> at the commit level (which was way simpler to handle), but to do\n> this the right way, the bind needs to be done at the tree level,\n> and \"rev-list --objects foo..bar\" would need some way to limit\n> the commit ancestry chain of subproject at the same time, by\n> computing the commit ancestry of the embedded commits in the\n> trees.\n\nPerhaps it would be best to join those two subproject support\nsolutions together: \"bind\" tree/commit mount header in commit\nobject, and \"commit\" entry in a tree. But I agree that revision\nwalking needs to be rewamped... well, unless you always have\nproject and subproject in the same repository, and subprojects\nare branches in the project too... \n\n-- \nJakub Narebski\nWarsaw, Poland\nShadeHawk on #git\n\n"},{"id":"298379","messageId":"4562570D.90406@vilain.net","threadId":"43106","inReplyTo":"ejtc3e$tod$2@sea.gmane.org","subject":"Re: [RFC] Submodules in GIT","fromName":"Sam Vilain","fromEmail":"sam@vilain.net","sentAt":"2006-11-21T01:31:57Z","receivedAt":"2006-11-21T01:31:57Z","isPatch":false,"sender":{"key":"sam@vilain.net","avatar":"https://gravatar.com/avatar/8fc840ca854dbf6f7065b4335e3b934951c1dca3b11db688e95e471901f8f4a8?d=mp&s=160"},"body":"Jakub Narebski wrote:\n> I wonder if it makes sense to be able to add tag objects instead\n> of commit objects to trees (depeel to tree or blob)...\n> \n\nI'd say \"as well as\", and the semantics should be that to something\nbrowsing the filesystem, a tag looks like the type of object it refers\nto. eg, tag a tree, it's a tree, tag a commit, it's a sub-project/tree,\ntag a blob, it's a file.\n\nThe use case I'm thinking of is semi-transparent storing of archives;\ninstead of storing the archive body, store a tag which contains the\n\"extra\" information - like the gzip headers for a gz and which\ncompression options are needed to reproduce the same output stream. For\na tar, the per-file information such as the filestamps, owner and\npermissions are recorded, and it points to a tree. A clever porcelain\ncould detect these file types, and make sure the uncompressed streams\nare stored.\n\nPeople who are using clients which don't understand these tag objects in\nbetween will get the contents of the node checked out instead, so\ninstead of getting \"foo.tar.gz\" as a file, I got a \"foo.tar.gz/\" directory.\n\n"},{"id":"296961","messageId":"20061121062158.GF20736@admingilde.org","threadId":"43106","inReplyTo":"ejthuh$fn8$1@sea.gmane.org","subject":"Re: [RFC] Submodules in GIT","fromName":"Martin Waitz","fromEmail":"tali@admingilde.org","sentAt":"2006-11-21T06:21:58Z","receivedAt":"2006-11-21T06:21:58Z","isPatch":false,"sender":{"key":"tali@admingilde.org","avatar":"https://gravatar.com/avatar/3f89b03eee362187effabe257898735b475673a12265c398ea9161259ae91553?d=mp&s=160"},"body":"hoi :)\n\nOn Tue, Nov 21, 2006 at 01:42:22AM +0100, Jakub Narebski wrote:\n> Perhaps it would be best to join those two subproject support\n> solutions together: \"bind\" tree/commit mount header in commit\n> object, and \"commit\" entry in a tree.\n\nBut which is the autoritative source then?\nDoes it give any more information?\n\nThe advantage in your proposal would be that submodules would\nbe visible immediately when looking at the commit,\nwithout having to traverse the entire tree.\nThis may be worthwhile when showing the combined history of parent\nand submodules.\n\nBut still this looks like \"caching submodule information in the\ncommit object\" and I do not know if we really want to do that.\n\n-- \nMartin Waitz\n"},{"id":"294803","messageId":"20061121062753.GG20736@admingilde.org","threadId":"43106","inReplyTo":"7v4pstzmk5.fsf@assigned-by-dhcp.cox.net","subject":"Re: [RFC] Submodules in GIT","fromName":"Martin Waitz","fromEmail":"tali@admingilde.org","sentAt":"2006-11-21T06:27:53Z","receivedAt":"2006-11-21T06:27:53Z","isPatch":false,"sender":{"key":"tali@admingilde.org","avatar":"https://gravatar.com/avatar/3f89b03eee362187effabe257898735b475673a12265c398ea9161259ae91553?d=mp&s=160"},"body":"hoi :)\n\nOn Mon, Nov 20, 2006 at 04:10:50PM -0800, Junio C Hamano wrote:\n> However, if I recall correctly, it was rather nightmarish to\n> make this also work for reachability traversal necessary for\n> pack generation. It was painful enough even when the bind was\n> at the commit level (which was way simpler to handle), but to do\n> this the right way, the bind needs to be done at the tree level,\n> and \"rev-list --objects foo..bar\" would need some way to limit\n> the commit ancestry chain of subproject at the same time, by\n> computing the commit ancestry of the embedded commits in the\n> trees.\n\nThis at least seems to work already.\nThe UNINTERESTING flag is recursively set for the submodule\ncommits while walking the object chain.\n\nBut I must admit that I only did very simple tests up to now.\nDo you have any special constellations in mind which were\ndifficult to support?\n\n-- \nMartin Waitz\n"},{"id":"298248","messageId":"7vr6vxxnc8.fsf@assigned-by-dhcp.cox.net","threadId":"43106","inReplyTo":"20061121062753.GG20736@admingilde.org","subject":"Re: [RFC] Submodules in GIT","fromName":"Junio C Hamano","fromEmail":"junkio@cox.net","sentAt":"2006-11-21T07:36:55Z","receivedAt":"2006-11-21T07:36:55Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Martin Waitz writes:\n\n> On Mon, Nov 20, 2006 at 04:10:50PM -0800, Junio C Hamano wrote:\n>\n>> However, if I recall correctly, it was rather nightmarish to\n>> make this also work for reachability traversal necessary for\n>> pack generation. It was painful enough even when the bind was\n>> at the commit level (which was way simpler to handle), but to do\n>> this the right way, the bind needs to be done at the tree level,\n>> and \"rev-list --objects foo..bar\" would need some way to limit\n>> the commit ancestry chain of subproject at the same time, by\n>> computing the commit ancestry of the embedded commits in the\n>> trees.\n>\n> This at least seems to work already.\n> The UNINTERESTING flag is recursively set for the submodule\n> commits while walking the object chain.\n\nI think that is fine as long as we somehow enforce the topology\nof submodule to be similar to the toplevel topology. Otherwise\nI suspect it leads to unintuitive behaviour.\n\nSuppose that the ancestry chain for the toplevel are A, A~1, A~2\nand you asked for \"A~2..A\". A submodule is bound at tree \"sub/\"\nand suppose A:sub/ == B, A~1:sub/ == C, and A~2:sub/ == D.\n\nNow further suppose the ancestry chain for B, C and D are like\nthis:\n\n o---C\n / \\\n ...o---o---D---B\n\nA naive implementation of \"--objects A~2..A\" would propagate\nUNINTERESTING to D and mark B and C unmarked. Would it however\nbe reasonable to include commits marked as 'o'?\n\nI am not trying to be negative here, but just raising things\nthat I did not think through when I tried to tackle it the last\ntime...\n"},{"id":"294382","messageId":"20061121075538.GH20736@admingilde.org","threadId":"43106","inReplyTo":"7vr6vxxnc8.fsf@assigned-by-dhcp.cox.net","subject":"Re: [RFC] Submodules in GIT","fromName":"Martin Waitz","fromEmail":"tali@admingilde.org","sentAt":"2006-11-21T07:55:38Z","receivedAt":"2006-11-21T07:55:38Z","isPatch":false,"sender":{"key":"tali@admingilde.org","avatar":"https://gravatar.com/avatar/3f89b03eee362187effabe257898735b475673a12265c398ea9161259ae91553?d=mp&s=160"},"body":"hoi :)\n\nOn Mon, Nov 20, 2006 at 11:36:55PM -0800, Junio C Hamano wrote:\n> I think that is fine as long as we somehow enforce the topology\n> of submodule to be similar to the toplevel topology. Otherwise\n> I suspect it leads to unintuitive behaviour.\n> \n> Suppose that the ancestry chain for the toplevel are A, A~1, A~2\n> and you asked for \"A~2..A\". A submodule is bound at tree \"sub/\"\n> and suppose A:sub/ == B, A~1:sub/ == C, and A~2:sub/ == D.\n> \n> Now further suppose the ancestry chain for B, C and D are like\n> this:\n> \n> o---C\n> / \\\n> ...o---o---D---B\n> \n> A naive implementation of \"--objects A~2..A\" would propagate\n> UNINTERESTING to D and mark B and C unmarked. Would it however\n> be reasonable to include commits marked as 'o'?\n\nI think it is reasonable to just go on as in a normal repository.\nThat is, pretend we want to list D..B and mark all commits which\nare reachable.\n\n-- \nMartin Waitz\n"},{"id":"297328","messageId":"ejuit4$mg$1@sea.gmane.org","threadId":"43106","inReplyTo":"20061121062158.GF20736@admingilde.org","subject":"Re: [RFC] Submodules in GIT","fromName":"Jakub Narebski","fromEmail":"jnareb@gmail.com","sentAt":"2006-11-21T10:04:46Z","receivedAt":"2006-11-21T10:04:46Z","isPatch":false,"sender":{"key":"jnareb@gmail.com","avatar":"https://avatars.githubusercontent.com/u/2706?v=4"},"body":"Martin Waitz wrote:\n\n> On Tue, Nov 21, 2006 at 01:42:22AM +0100, Jakub Narebski wrote:\n>> Perhaps it would be best to join those two subproject support\n>> solutions together: \"bind\" tree/commit mount header in commit\n>> object, and \"commit\" entry in a tree.\n> \n> But which is the autoritative source then?\n> Does it give any more information?\n\nBoth should contain the same information, otherwise repository is corrupt\n(is in inconsistent state).\n\n\"bind\" header in commit objects is meant as a kind of shortcut, to ease\nreachability checking (you don't need to recurse into directories).\n\n> The advantage in your proposal would be that submodules would\n> be visible immediately when looking at the commit,\n> without having to traverse the entire tree.\n> This may be worthwhile when showing the combined history of parent\n> and submodules.\n\nThat was the idea.\n\n> But still this looks like \"caching submodule information in the\n> commit object\" and I do not know if we really want to do that.\n\nWell, we would be repeating information, sure. But we can put additional\ninformation in \"bind\" header except sha1 of commit and mount point...\nalthough I cannot think what... :)\n\n-- \nJakub Narebski\nWarsaw, Poland\nShadeHawk on #git\n\n"},{"id":"294583","messageId":"20061121114938.GI20736@admingilde.org","threadId":"43106","inReplyTo":"ejuit4$mg$1@sea.gmane.org","subject":"Re: [RFC] Submodules in GIT","fromName":"Martin Waitz","fromEmail":"tali@admingilde.org","sentAt":"2006-11-21T11:49:39Z","receivedAt":"2006-11-21T11:49:39Z","isPatch":false,"sender":{"key":"tali@admingilde.org","avatar":"https://gravatar.com/avatar/3f89b03eee362187effabe257898735b475673a12265c398ea9161259ae91553?d=mp&s=160"},"body":"hoi :)\n\nOn Tue, Nov 21, 2006 at 11:04:46AM +0100, Jakub Narebski wrote:\n> \"bind\" header in commit objects is meant as a kind of shortcut, to ease\n> reachability checking (you don't need to recurse into directories).\n\nWell, but you already have to recurse to find all objects which are\nreachable by a commit, so you don't loose anything.\n\n> > The advantage in your proposal would be that submodules would\n> > be visible immediately when looking at the commit,\n> > without having to traverse the entire tree.\n> > This may be worthwhile when showing the combined history of parent\n> > and submodules.\n> \n> That was the idea.\n\nOn the other hand that only has to be done once anyway.\nAfter you traversed the tree once you can create your own\n(in memory) cache of submodules connected to the tree.\nWhile walking the commits backwards, you only have to check those\nparts of the tree which have changed.\nSo it may even be suitable for larger repositories.\nBut clearly it is not as low as with the in-commit cache.\nSo we have to weight complexity of the data storage with\nruntime complexity. Opinions?\n\n-- \nMartin Waitz\n"},{"id":"298306","messageId":"20061121180127.GB27221@fieldses.org","threadId":"43106","inReplyTo":"20061120233333.GD20736@admingilde.org","subject":"Re: [RFC] Submodules in GIT","fromName":"J. Bruce Fields","fromEmail":"bfields@fieldses.org","sentAt":"2006-11-21T18:01:27Z","receivedAt":"2006-11-21T18:01:27Z","isPatch":false,"sender":{"key":"bfields@citi.umich.edu","avatar":null},"body":"On Tue, Nov 21, 2006 at 12:33:34AM +0100, Martin Waitz wrote:\n> On Mon, Nov 20, 2006 at 06:25:07PM -0500, J. Bruce Fields wrote:\n> > Would it also be possible to allow the \"Tree:\" line in the commit object\n> > to refer to a commit, or does the root of the project need to be a\n> > special case?\n> \n> this would then be something like the branch-archival proposal.\n\nDo you have any pointers to previous discussion? (A couple obvious\nsearches don't turn up anything for me.)\n\n"},{"id":"296664","messageId":"20061121193248.GJ20736@admingilde.org","threadId":"43106","inReplyTo":"20061121180127.GB27221@fieldses.org","subject":"Re: [RFC] Submodules in GIT","fromName":"Martin Waitz","fromEmail":"tali@admingilde.org","sentAt":"2006-11-21T19:32:48Z","receivedAt":"2006-11-21T19:32:48Z","isPatch":false,"sender":{"key":"tali@admingilde.org","avatar":"https://gravatar.com/avatar/3f89b03eee362187effabe257898735b475673a12265c398ea9161259ae91553?d=mp&s=160"},"body":"On Tue, Nov 21, 2006 at 01:01:27PM -0500, J. Bruce Fields wrote:\n> On Tue, Nov 21, 2006 at 12:33:34AM +0100, Martin Waitz wrote:\n> > On Mon, Nov 20, 2006 at 06:25:07PM -0500, J. Bruce Fields wrote:\n> > > Would it also be possible to allow the \"Tree:\" line in the commit object\n> > > to refer to a commit, or does the root of the project need to be a\n> > > special case?\n> > \n> > this would then be something like the branch-archival proposal.\n> \n> Do you have any pointers to previous discussion? (A couple obvious\n> searches don't turn up anything for me.)\n\nAug 04 Eric W. Biederman [RFC][PATCH] Branch history\n\nI really think that using subprojects can be used for this workflow, too.\nBut adding a submodule directly to the root is not really possible,\nwe'd have to use special user interfaces for that, even when the\ngit-core might be able to handle it.\nBut what might be possible is to have one toplevel history-tracking\nrepository in e.g. ~/src and then add all the repositories you work\nwith as a submodule. Whenever you want to record the history of\nsome project, you can simply commit it to ~/src.\n\n-- \nMartin Waitz\n"},{"id":"296039","messageId":"20061121223130.GA24909@nan92-1-81-57-214-146.fbx.proxad.net","threadId":"43106","inReplyTo":"Pine.LNX.4.64.0611201501230.3338@woody.osdl.org","subject":"Re: [RFC] Submodules in GIT","fromName":"Yann Dirson","fromEmail":"ydirson@altern.org","sentAt":"2006-11-21T22:31:30Z","receivedAt":"2006-11-21T22:31:30Z","isPatch":false,"sender":{"key":"ydirson@altern.org","avatar":"https://avatars.githubusercontent.com/u/1190950?v=4"},"body":"On Mon, Nov 20, 2006 at 03:05:47PM -0800, Linus Torvalds wrote:\n> On Mon, 20 Nov 2006, Junio C Hamano wrote:\n> > \n> > That was shot down by Linus and I agree with him. \"bind\" was a\n> > bad idea because binding of a particular subproject commit into\n> > a tree is a property of the tree, not one of the commits that\n> > happen to have that tree.\n> \n> Yes. I think it would be a _fine_ idea to have a new tree-entry type that \n> points to a sub-commit, but it really does need to be on a \"tree level\", \n> not a commit level.\n\nI'm not sure I get the reason why the submodule should not be recorded\non \"commit level\".\n\nWhat I'm thinking of would be that the submodule tree would just be a\nstandard antry of a tree in the supermodule, and we could record the\nsubmodule commit (pointing to the submodule tree) in the supermodule\ncommit.\n\nThis idea came when thinking about implementing partial merges. That\nis, when different people are responsible for different parts of the\ntree, and thus when merging a given branch, each dev has to make only\na partial merge of the full tree.\nHaving submodule commits referenced directly from the supercommit would\nmake it much easier to finalize the merge (ie. merging the full project\nwhile taking into account that some subtrees have been merged already).\n\nBest regards,\n-- \n"},{"id":"296825","messageId":"Pine.LNX.4.64.0611211437430.3338@woody.osdl.org","threadId":"43106","inReplyTo":"20061121223130.GA24909@nan92-1-81-57-214-146.fbx.proxad.net","subject":"Re: [RFC] Submodules in GIT","fromName":"Linus Torvalds","fromEmail":"torvalds@osdl.org","sentAt":"2006-11-21T22:51:56Z","receivedAt":"2006-11-21T22:51:56Z","isPatch":false,"sender":{"key":"torvalds@linux-foundation.org","avatar":"https://avatars.githubusercontent.com/u/1024025?v=4"},"body":"\n\nOn Tue, 21 Nov 2006, Yann Dirson wrote:\n> \n> I'm not sure I get the reason why the submodule should not be recorded\n> on \"commit level\".\n\nBecause that would be STUPID.\n\nWhat does the submodules have to do with the commit level? Nothing. Nada. \nZero.\n\nSubmodules are _directories_. They can be anywhere in the directory tree. \nIf you try to encode that in a commit message, you're going to totally \nbreak the whole notion of trying to \"diff\" two trees. \n\nAll of git is designed around the notion that a tree is the directory \nstructure. If you put directory structure somewhere else, you totally \nscrew all abstractions.\n\nNow, if that weren't enough, let me enumerate _another_ reason why it's \nidiotic and wrong, namely the fact that a \"commit\" is fundamnetally the \nwrong place to add something like that _anyway_. Quite apart from the fact \nthat we describe directory trees with (wait for it): \"tree objects\", the \nthing is, a commit is about a totally different _dimension_ altogether. \n\nThe only and _whole_ point of a \"commit\" is to describe the \"time \ndimension\". Something that doesn't always change in time should not be in \na commit object, because it is by definition not what a commit is all \nabout. A commit should describe the relationship of itself to other \ncommits, ie it's a \"how did this change\".\n\nAnd a sub-project simply doesn't even _do_ that. Much of the time, a \nsubproject stays constant, and is not something that comes and goes on an \nindividual commit basis. \n\nI don't understand why people are so fixated with putting things in the \nwrong object. WHY do people want to put crap in the \"commit\" object? \nPeople have wanted to put \"rename\" information there (which is stupid for \nall the same reasons: renames _remain_. They aren't a one-time event. If \nsomething was renamed in commit X, it will _remain_ renamed in commit X+1, \nso it's clearly not really a \"commit X\" thing)\n\nThink of it this way:\n\n - if something _only_ makes sense on an _individual commit_ level, it \n goes into the \"commit object\". But if it makes sense for \"git diff\",\n then it MUST NOT be in a commit object, because you do \"git diff\" over\n a big _range_ of commit objects.\n\nThink \"git show\". The \"author\" of a commit is only associated with a \n_single_ commit. It thus goes into the commit object, and nowhere else. \nSame goes for time, and commit message. A commit message is fundamentally \na \"this explains this _one_ commit\".\n\nBut anything that you expect to have in a \"range\" of commits MUST NOT be \nin a \"commit object\". If I do \"git diff v2.6.13..v2.6.14\", and I expect \nthe behaviour you want to encode to show up (and dammit, subprojects very \nmuch fall under that heading - exactly the same way renames must have \nmeaning _outside_ of a single commit) then clearly it is NOT something \nthat is associated with any individual commits. It's something that is \nassociated with the _state_ of the project.\n\nAnd the _state_ of the project is the \"tree\". Not the commit. The commit \nis about the _history_ of the project.\n\nSo please understand this: \"commit\" is about the time-dimension \n(\"history\"). \"tree\" is about the space-dimension (\"state\"). The two are \n_related_, but they are also very much different concepts, and \"related\" \ndoes not mean \"you can mix them up\".\n\nSub-projects are clearly not about \"time\". They are about \"state\".\n\n"},{"id":"294996","messageId":"Pine.LNX.4.64.0611211456520.3338@woody.osdl.org","threadId":"43106","inReplyTo":"Pine.LNX.4.64.0611211437430.3338@woody.osdl.org","subject":"Re: [RFC] Submodules in GIT","fromName":"Linus Torvalds","fromEmail":"torvalds@osdl.org","sentAt":"2006-11-21T22:59:00Z","receivedAt":"2006-11-21T22:59:00Z","isPatch":false,"sender":{"key":"torvalds@linux-foundation.org","avatar":"https://avatars.githubusercontent.com/u/1024025?v=4"},"body":"\n\nOn Tue, 21 Nov 2006, Linus Torvalds wrote:\n> \n> Submodules are _directories_.\n\nSide note - you can do submodules other ways, but if you do, you'll almost \ncertainly go crazy. \n\nYou could, for example, make submodules be some kind of \"union \nfilesystem\", where you allow overlapping trees. It's conceptually \npossible. It's also horribly horribly wrong, if only because I guarantee \nthat you'll have so many problems with it that you will only end up with a \nmess that is even worse than \"branches\" in CVS.\n\n"},{"id":"297144","messageId":"20061121235429.GH5443@nan92-1-81-57-214-146.fbx.proxad.net","threadId":"43106","inReplyTo":"Pine.LNX.4.64.0611211437430.3338@woody.osdl.org","subject":"Re: [RFC] Submodules in GIT","fromName":"Yann Dirson","fromEmail":"ydirson@altern.org","sentAt":"2006-11-21T23:54:29Z","receivedAt":"2006-11-21T23:54:29Z","isPatch":false,"sender":{"key":"ydirson@altern.org","avatar":"https://avatars.githubusercontent.com/u/1190950?v=4"},"body":"On Tue, Nov 21, 2006 at 02:51:56PM -0800, Linus Torvalds wrote:\n> \n> \n> On Tue, 21 Nov 2006, Yann Dirson wrote:\n> > \n> > I'm not sure I get the reason why the submodule should not be recorded\n> > on \"commit level\".\n> \n> Because that would be STUPID.\n> \n> What does the submodules have to do with the commit level? Nothing. Nada. \n> Zero.\n\nOh, I see I may have expressed something in the wrong way :)\nNamely, I brought an idea coming from partial merges into a discussion\non submodules, because when thinking about the former, I realized\nwe could maybe use similar mechanisms for both.\n\nNote that the proposal I outlined did not break the tree, in that the\nsumodule tree is still in the same place. In the case of a partial\nmerge, the info that a subtree has been merged in this commit is indeeed\npart of the commit itself.\n\nI agree that the subtree case is somewhat different, and my idea may not\napply to submodules after all :)\n\nA question would be, do \"submodules\" have to be permanent objects ?\nI suppose it depends on what people want to use them for. Indeed, the\n\"submodule\" names strongly carries the idea of a permanent subset of the\nrepository. My proposal partial merges could be seen as using transient\nsubmodules: they do not matter much during most of the repo life.\n\nPut it another way, I see the proposal of allowing tree entries to be\ncommits in addition to trees and blobs, akin to recording the submodule\n_history_ inside the _tree_, which I feel precisely violates the\ndistinction you want to keep between those 2 concepts.\n\n\n> And a sub-project simply doesn't even _do_ that. Much of the time, a \n> subproject stays constant, and is not something that comes and goes on an \n> individual commit basis. \n\nWhat about the case of a subproject that would evolve fast, and for\nwhich we may not want intermediate versions to be part of the\nsupermodule ? (just exploring an idea without real connection to the\none discussed above)\n\nI mean, I have a tree in which the whole software for an embedded\nplatform is stored, including kernel, apps, etc. While working on the\nkernel, I may want to do several commits to that submodule, and may not want\nto commit to the supermodule for each kernel commit, only when I feel the\nkernel is stable enough.\n\nOne may argue I just have to use a branch. Anyway, there will be a need\nfor submodule-specific branches - eg. kernel.org ones in my case.\n\nAn alternative would be to allow committing to the submodule without\ncreating matching supermodule commits, and let the user decide when he\nwants to commit at the higher level. That way, 2 successive supermodule\ncommits could have non-successive \"subcommits\".\n\nBest regards,\n-- \n"},{"id":"294112","messageId":"20061122034056.GB23856@spearce.org","threadId":"43106","inReplyTo":"20061121235429.GH5443@nan92-1-81-57-214-146.fbx.proxad.net","subject":"Re: [RFC] Submodules in GIT","fromName":"Shawn Pearce","fromEmail":"spearce@spearce.org","sentAt":"2006-11-22T03:40:56Z","receivedAt":"2006-11-22T03:40:56Z","isPatch":false,"sender":{"key":"spearce@spearce.org","avatar":"https://avatars.githubusercontent.com/u/34844?v=4"},"body":"Yann Dirson wrote:\n> Put it another way, I see the proposal of allowing tree entries to be\n> commits in addition to trees and blobs, akin to recording the submodule\n> _history_ inside the _tree_, which I feel precisely violates the\n> distinction you want to keep between those 2 concepts.\n\nNo. Linus is right. Submodule commits belong in the tree.\n\nWe want to record a specific subtree within a larger tree. There are\nthree ways we can refer to a tree: by its tree SHA1, by a commit\nwhich points at the tree SHA1, or by a tag which points at a\ncommit which points at the tree SHA1, or by a tag which points\nat a tag which points at a commit which points at a tree SHA1.\nWhich is basically a tree-ish.\n\nThe advantage of linking to the commit-ish (commit or tag) and\nnot the tree-ish for a submodule is that it also provides you quick\naccess to answer the \"how did this tree arive at this state\" question\nas the answer cannot come solely from the top level commit chain.\nThe reason... keep reading...\n \n> What about the case of a subproject that would evolve fast, and for\n> which we may not want intermediate versions to be part of the\n> supermodule ? (just exploring an idea without real connection to the\n> one discussed above)\n\nRight. The submodule is free to be committed to an infinite number\nof times for any given commit in the supermodule.\n\nIt is expected that users will commit to a submodule say hundreds of\ntimes for every commit they make to the supermodule. Or thousands.\nThis is especially true if the submodule is some very large project,\ne.g. the Linux kernel, and the supermodule \"upgrades\" the kernel it\nis using after 3 months of staying on the same version. Suddenly the\nsupermodule has only 1 commit which covers maybe 10,000 commits in\nthe submodule.\n\nYet we still want to be able to efficiently perform operations like\n\"git bisect\" within the scope of that submodule, to help narrow down\na particular bug that is within that submodule. To do that we need\nthe commit chain (all 10,000 of those commits) in the submodule.\nTo get those we really need a commit-ish and not a tree-ish, as\ngoing from a tree-ish to a commit-ish is not only not unique but\nis also pretty infeasible to do (you need to scan *every* commit).\n\n-- \n"},{"id":"296060","messageId":"20061123232313.GB24909@nan92-1-81-57-214-146.fbx.proxad.net","threadId":"43106","inReplyTo":"20061122034056.GB23856@spearce.org","subject":"Re: [RFC] Submodules in GIT","fromName":"Yann Dirson","fromEmail":"ydirson@altern.org","sentAt":"2006-11-23T23:23:13Z","receivedAt":"2006-11-23T23:23:13Z","isPatch":false,"sender":{"key":"ydirson@altern.org","avatar":"https://avatars.githubusercontent.com/u/1190950?v=4"},"body":"On Tue, Nov 21, 2006 at 10:40:56PM -0500, Shawn Pearce wrote:\n> Yet we still want to be able to efficiently perform operations like\n> \"git bisect\" within the scope of that submodule, to help narrow down\n> a particular bug that is within that submodule. To do that we need\n> the commit chain (all 10,000 of those commits) in the submodule.\n> To get those we really need a commit-ish and not a tree-ish, as\n> going from a tree-ish to a commit-ish is not only not unique but\n> is also pretty infeasible to do (you need to scan *every* commit).\n\nWe don't need to have commits in the tree for this. We'll just have\nsubmodule commits which are not attached to a supermodule commit, and we\ncan access the whole submodule history through the submodule .git/HEAD,\njust like we do for a standard git project.\n\nOr do I miss something else ?\n\nBest regards,\n-- \n"},{"id":"297024","messageId":"20061125065338.GC4528@spearce.org","threadId":"43106","inReplyTo":"20061123232313.GB24909@nan92-1-81-57-214-146.fbx.proxad.net","subject":"Re: [RFC] Submodules in GIT","fromName":"Shawn Pearce","fromEmail":"spearce@spearce.org","sentAt":"2006-11-25T06:53:38Z","receivedAt":"2006-11-25T06:53:38Z","isPatch":false,"sender":{"key":"spearce@spearce.org","avatar":"https://avatars.githubusercontent.com/u/34844?v=4"},"body":"Yann Dirson wrote:\n> On Tue, Nov 21, 2006 at 10:40:56PM -0500, Shawn Pearce wrote:\n> > Yet we still want to be able to efficiently perform operations like\n> > \"git bisect\" within the scope of that submodule, to help narrow down\n> > a particular bug that is within that submodule. To do that we need\n> > the commit chain (all 10,000 of those commits) in the submodule.\n> > To get those we really need a commit-ish and not a tree-ish, as\n> > going from a tree-ish to a commit-ish is not only not unique but\n> > is also pretty infeasible to do (you need to scan *every* commit).\n> \n> We don't need to have commits in the tree for this. We'll just have\n> submodule commits which are not attached to a supermodule commit, and we\n> can access the whole submodule history through the submodule .git/HEAD,\n> just like we do for a standard git project.\n\nNo. You cannot do that.\n\nHow do we setup .git/HEAD when bisecting the supermodule?\nOr merging it? Or doing anything else with it?\n\nIdeally the .git/HEAD of every submodule should seek to the commit\nthat points at the tree of the submodule which the supermodule\nis referencing. This lets you then perform a bisect within the\nsubmodule when you identify the supermodule commit which caused\nthe breakage.\n\nWe need the submodule commits to do this. Doing it without is\ntoo expensive.\n\n-- \n"},{"id":"294676","messageId":"20061125111235.GO5443@nan92-1-81-57-214-146.fbx.proxad.net","threadId":"43106","inReplyTo":"20061125065338.GC4528@spearce.org","subject":"Re: [RFC] Submodules in GIT","fromName":"Yann Dirson","fromEmail":"ydirson@altern.org","sentAt":"2006-11-25T11:12:35Z","receivedAt":"2006-11-25T11:12:35Z","isPatch":false,"sender":{"key":"ydirson@altern.org","avatar":"https://avatars.githubusercontent.com/u/1190950?v=4"},"body":"On Sat, Nov 25, 2006 at 01:53:38AM -0500, Shawn Pearce wrote:\n> Yann Dirson wrote:\n> > We don't need to have commits in the tree for this. We'll just have\n> > submodule commits which are not attached to a supermodule commit, and we\n> > can access the whole submodule history through the submodule .git/HEAD,\n> > just like we do for a standard git project.\n> \n> No. You cannot do that.\n> \n> How do we setup .git/HEAD when bisecting the supermodule?\n> Or merging it? Or doing anything else with it?\n\nWould there be any problem assuming git-update-ref would take care of\nupdating it ?\n\n> Ideally the .git/HEAD of every submodule should seek to the commit\n> that points at the tree of the submodule which the supermodule\n> is referencing.\n\nYou mean, whenever we seek the HEAD of the supermodule, right ?\n\n> This lets you then perform a bisect within the\n> submodule when you identify the supermodule commit which caused\n> the breakage.\n \nThat is, first bisect the supermodule (which naturally bisects the\nsubmodule with rough granularity, assuming there are many submodule\ncommits for at least some supermodule commits), then bisect the submodule\nbetween the two commits identified at supermodule level, right ?\n\n> We need the submodule commits to do this. Doing it without is\n> too expensive.\n\nMaybe I missed something again, but I'm still not convinced :)\n-- \n"},{"id":"295158","messageId":"Pine.LNX.4.64.0611251037000.6991@woody.osdl.org","threadId":"43106","inReplyTo":"20061125111235.GO5443@nan92-1-81-57-214-146.fbx.proxad.net","subject":"Re: [RFC] Submodules in GIT","fromName":"Linus Torvalds","fromEmail":"torvalds@osdl.org","sentAt":"2006-11-25T18:57:59Z","receivedAt":"2006-11-25T18:57:59Z","isPatch":false,"sender":{"key":"torvalds@linux-foundation.org","avatar":"https://avatars.githubusercontent.com/u/1024025?v=4"},"body":"\n\nOn Sat, 25 Nov 2006, Yann Dirson wrote:\n> \n> > This lets you then perform a bisect within the\n> > submodule when you identify the supermodule commit which caused\n> > the breakage.\n> \n> That is, first bisect the supermodule (which naturally bisects the\n> submodule with rough granularity, assuming there are many submodule\n> commits for at least some supermodule commits), then bisect the submodule\n> between the two commits identified at supermodule level, right ?\n\nRight. That is how you _must_ do it.\n\nThe reason is:\n\n - the supermodule will not track every release of the submodule. One of \n the biggest reasons for using submodules in the first place is that the \n submodules have their own development _independently_ of the \n supermodule, and usually the supermodule will import new versions of \n submodules only occasionally (eg the supermodule might choose to track \n only major releases of the submodule, for example)\n\n (And yes, I realize that this is not necessarily the only submodule \n usage: sometimes the submodules are literally _only_ developed as \n submodules, and you'd never develop them independently. It depends on \n the situation)\n\n - As a resule of the above, you MUST NOT do bisection at the submodule \n level at first: it's entirely possible that the supermodule never ever \n actually used the submodule state at a finer granularity, and \n \"bisecting\" into such state would be idiotic (it's really no different \n from \"bisecting\" a regular commit by splitting up a commit into patches \n against individual files - sure, it's a smaller granularity, but it's a \n granularity that never _existed_, and was never tested or intended to \n work!)\n\nSo yes, you should expect that\n\n (a) submodule changes \"jump around\" in the supermodule - even to the \n point of going backwards in time as far as the submodule is concerned \n (ie the supermodule might have tested a new release of a submodule, \n committed that, found a problem, and decided to just go back to an \n earlier version of the submodule again, and committed that again)\n\n (b) This implies very much that there can be a n:m relationship between \n submodule and supermodule commits. A supermodule commit does _not_ \n imply a commit in the submodule (it might commit changes to the \n top-level makefile or to _another_ submodule), but equally, a \n submodule commit does _not_ imply a commit in the supermodule \n (because the submodule might be independently changed in some other \n repository where it's the _primary_ development, not a submodule)\n\nSo you shouldn't expect submodules to be very \"tightly\" coupled, and I \ndon't think you even want the workflow to _be_ that tight. I think it's ok \nif submodules show up as such, and that \"git diff\" etc don't try to make \nit all \"seamless\".\n\nIt often _shouldn't_ be seamless: you should be able to commit to a \nsupermodule without committing the submodule state: it's really no \ndifferent from committing individual files (it migth be somethign that is \n_discouraged_ as a workflow for some project, the same way you might \ndiscourage using \"git commit one/file\" over \"git commit -a\", and for the \nsame reason: you're committing some state that doesn't match what your \ntree actually looks like).\n\nSimilarly, doing a \"git commit -a\" within a submodule should really just \ncommit _that_ submodule, and not even _try_ to know about supermodules \netc, because the submodule really should be a totally independent git \nrepository.\n\n[ Side note: you may well want to set up submodules so that they share the \n object store with the supermodule: that may be the simplest way to make \n operations that traverse things recursively work out, since it means \n that you can do object lookups for everythign you traverse without \n having to even think about it.\n\n On the other hand, this could equally easily be done by just making \n every submodule an \"alternates\" directory in the supermodule: that keeps \n the object databases separate, but means that anybody in the supermodule \n will always be able to look up all the objects in the submodules. So \n even here, we certainly _can_ keep things separated, without even \n introducing any new concepts. ]\n\nSo I actually think that submodules should at least start out as something \nrather independent, where a \"commit -a\" in the supermodule will _only_ \ncommit the supermodule itself - and if you haven't committed the submodule \nyet, you'll just get the current HEAD state of the submodule.\n\nAdd some trivial help in \"git status\" to _warn_ about the fact that \nsubmodules haven't been committed and are dirty, but I really think that \nit should be a very explicit thing where you really do see things as \nsubmodules, not as \"one big module\".\n\n"},{"id":"294331","messageId":"45689747.3020403@midwinter.com","threadId":"43106","inReplyTo":"Pine.LNX.4.64.0611251037000.6991@woody.osdl.org","subject":"Re: [RFC] Submodules in GIT","fromName":"Steven Grimm","fromEmail":"koreth@midwinter.com","sentAt":"2006-11-25T19:19:35Z","receivedAt":"2006-11-25T19:19:35Z","isPatch":false,"sender":{"key":"koreth@midwinter.com","avatar":"https://gravatar.com/avatar/71b4d2e8b62f168bdc9e9205341159e3567003b4f9e2127c617c5fa0a1f5bad2?d=mp&s=160"},"body":"Linus Torvalds wrote:\n> So I actually think that submodules should at least start out as something \n> rather independent, where a \"commit -a\" in the supermodule will _only_ \n> commit the supermodule itself - and if you haven't committed the submodule \n> yet, you'll just get the current HEAD state of the submodule.\n\nThat would make it impossible to atomically commit a change that affects \ntwo submodules, yes? I think cross-submodule commit is highly desirable \nand will be a fairly common use case for submodules if it's supported. \nFor example, if you have \"client\" and \"server\" submodules and someone \nmakes a protocol change, you don't want some unwitting developer to pull \njust half of the change and end up with incompatible code in the two \nsubmodules.\n\nI have no problem with making the \"only commit the supermodule\" behavior \nthe default and requiring a command-line option for the \"commit \neverything\" case, but I think \"commit everything\" is useful. And \nhonestly IMO it should be the default since it'll behave in a less \nsurprising way; when I do a \"commit -a\" I expect all my changes to be \ncommitted, whether they're in submodules or not.\n\n"},{"id":"298321","messageId":"Pine.LNX.4.64.0611251128170.3483@woody.osdl.org","threadId":"43106","inReplyTo":"45689747.3020403@midwinter.com","subject":"Re: [RFC] Submodules in GIT","fromName":"Linus Torvalds","fromEmail":"torvalds@osdl.org","sentAt":"2006-11-25T19:30:47Z","receivedAt":"2006-11-25T19:30:47Z","isPatch":false,"sender":{"key":"torvalds@linux-foundation.org","avatar":"https://avatars.githubusercontent.com/u/1024025?v=4"},"body":"\n\nOn Sat, 25 Nov 2006, Steven Grimm wrote:\n\n> Linus Torvalds wrote:\n> > So I actually think that submodules should at least start out as something\n> > rather independent, where a \"commit -a\" in the supermodule will _only_\n> > commit the supermodule itself - and if you haven't committed the submodule\n> > yet, you'll just get the current HEAD state of the submodule.\n> \n> That would make it impossible to atomically commit a change that affects two\n> submodules, yes?\n\nNo. Quite the reverse. What you do is:\n\n (a) commit both submodules INDEPENDENTLY.\n\n (b) then commit the supermodule that contains the submodules.\n\nAnd note how the important part here is that committing in a submodule \nDOES NOT AFFECT THE SUPERMODULE AT ALL!\n\nThe git trees are _independent_. That's important. You should _not_ try to \nmix them up and make a commit in one commit anything AT ALL in some other \ntree, exctly because it gets impossible to do (a) interesting things and \n(b) atomic commits otherwise.\n\nNote that this is true also in the case of a submodule that itself \ncontains a submodule. That doesn't change anything - you still need to be \nable to view _each_ layer as an independent thing.\n\n"},{"id":"296005","messageId":"20061125234908.GC24909@nan92-1-81-57-214-146.fbx.proxad.net","threadId":"43106","inReplyTo":"Pine.LNX.4.64.0611251128170.3483@woody.osdl.org","subject":"Re: [RFC] Submodules in GIT","fromName":"Yann Dirson","fromEmail":"ydirson@altern.org","sentAt":"2006-11-25T23:49:08Z","receivedAt":"2006-11-25T23:49:08Z","isPatch":false,"sender":{"key":"ydirson@altern.org","avatar":"https://avatars.githubusercontent.com/u/1190950?v=4"},"body":"On Sat, Nov 25, 2006 at 11:30:47AM -0800, Linus Torvalds wrote:\n> The git trees are _independent_. That's important.\n\nI'm not sure how independant you mean them to be. The approach I've\ntried to describe so far assumes that, although you can look at each\nsubmodule independently from the supermodule or any other submodule, you\ncan still look at the supermodule as a single tree of it own.\n\nEg, so that if one part of an appliance/ modules ends up promoted to a\nlib/ module, GIT can still show that as a move within the supermodule.\nIf we insist that the submodules get committed independently before we\nmake a supermodule commit tying those together, I fear it may make things\nlike such \"move/copy detection\" more tricky ?\n\nAlso, I'd rather expect \"git-commit -a\" outside of any submodule to\ncommit everything in the supermodule, triggering submodule commits as an\nintermediate step when needed - just like \"git-commit -a\" does not\nrequire to manually specify subdirectories to inclue in the commit. I'd\nrather expect a special flag to exclude submodules from a commit.\n\nBest regards,\n--\n"},{"id":"294215","messageId":"20061126011420.GI20094MdfPADPa@greensroom.kotnet.org","threadId":"43106","inReplyTo":"20061125234908.GC24909@nan92-1-81-57-214-146.fbx.proxad.net","subject":"Re: [RFC] Submodules in GIT","fromName":"Sven Verdoolaege","fromEmail":"skimo@kotnet.org","sentAt":"2006-11-26T01:14:20Z","receivedAt":"2006-11-26T01:14:20Z","isPatch":false,"sender":{"key":"skimo@kotnet.org","avatar":null},"body":"FWIW, here's my view on this issue.\n\nOn Sun, Nov 26, 2006 at 12:49:08AM +0100, Yann Dirson wrote:\n> Also, I'd rather expect \"git-commit -a\" outside of any submodule to\n> commit everything in the supermodule, triggering submodule commits as an\n> intermediate step when needed - just like \"git-commit -a\" does not\n> require to manually specify subdirectories to inclue in the commit. I'd\n> rather expect a special flag to exclude submodules from a commit.\n\nA commit should record the content changes that have been made, not change\nany content itself. Some VCSs change the contents of a file when you\ncommit them (e.g., keyword substitution). Git, rightly, doesn't do that.\nLikewise, when you commit in the superproject, it should simply record\nthe changes to the \"content\" of the subproject and not change it.\nAnd the content of the subproject is a commit, so a commit in the\nsuperproject should not change the content of the subproject by creating\nanother commit in the subproject.\n\n"},{"id":"296936","messageId":"20061126013220.GD24909@nan92-1-81-57-214-146.fbx.proxad.net","threadId":"43106","inReplyTo":"20061126011420.GI20094MdfPADPa@greensroom.kotnet.org","subject":"Re: [RFC] Submodules in GIT","fromName":"Yann Dirson","fromEmail":"ydirson@altern.org","sentAt":"2006-11-26T01:32:20Z","receivedAt":"2006-11-26T01:32:20Z","isPatch":false,"sender":{"key":"ydirson@altern.org","avatar":"https://avatars.githubusercontent.com/u/1190950?v=4"},"body":"On Sun, Nov 26, 2006 at 02:14:20AM +0100, Sven Verdoolaege wrote:\n> Likewise, when you commit in the superproject, it should simply record\n> the changes to the \"content\" of the subproject and not change it.\n> And the content of the subproject is a commit, so a commit in the\n> superproject should not change the content of the subproject by creating\n> another commit in the subproject.\n\nI've realized after suggesting that how much that idea was inadequate -\nsorry for the noise.\n\nHowever, I'm not yet buying the idea that \"the content of the subproject\nis a commit\" :)\n\nBest regards,\n-- \n"},{"id":"297142","messageId":"Pine.LNX.4.64.0611251936590.3483@woody.osdl.org","threadId":"43106","inReplyTo":"20061125234908.GC24909@nan92-1-81-57-214-146.fbx.proxad.net","subject":"Re: [RFC] Submodules in GIT","fromName":"Linus Torvalds","fromEmail":"torvalds@osdl.org","sentAt":"2006-11-26T03:39:35Z","receivedAt":"2006-11-26T03:39:35Z","isPatch":false,"sender":{"key":"torvalds@linux-foundation.org","avatar":"https://avatars.githubusercontent.com/u/1024025?v=4"},"body":"\n\nOn Sun, 26 Nov 2006, Yann Dirson wrote:\n> \n> Also, I'd rather expect \"git-commit -a\" outside of any submodule to\n> commit everything in the supermodule, triggering submodule commits as an\n> intermediate step when needed - just like \"git-commit -a\" does not\n> require to manually specify subdirectories to inclue in the commit. I'd\n> rather expect a special flag to exclude submodules from a commit.\n\nSo, how do you do commit messages? It generally doesn't make sense to \nshare the same commit message for submodules - the sub-commits generally \ndo different things.\n\nI'd actually suggest that \"git commit -a\" with non-clean submodules error \nout for that reason, with something like\n\n\tsubmodule 'src/xyzzy' is not up-to-date, please commit changes to \n\tthat first.\n\nexactly because you really generally should consider the submodule commits \nto be a separate phase.\n\n"},{"id":"293980","messageId":"Pine.LNX.4.64.0611260241320.20138@iabervon.org","threadId":"43106","inReplyTo":"Pine.LNX.4.64.0611251936590.3483@woody.osdl.org","subject":"Re: [RFC] Submodules in GIT","fromName":"Daniel Barkalow","fromEmail":"barkalow@iabervon.org","sentAt":"2006-11-26T08:05:15Z","receivedAt":"2006-11-26T08:05:15Z","isPatch":false,"sender":{"key":"barkalow@iabervon.org","avatar":"https://avatars.githubusercontent.com/u/55364219?v=4"},"body":"On Sat, 25 Nov 2006, Linus Torvalds wrote:\n\n> On Sun, 26 Nov 2006, Yann Dirson wrote:\n> > \n> > Also, I'd rather expect \"git-commit -a\" outside of any submodule to\n> > commit everything in the supermodule, triggering submodule commits as an\n> > intermediate step when needed - just like \"git-commit -a\" does not\n> > require to manually specify subdirectories to inclue in the commit. I'd\n> > rather expect a special flag to exclude submodules from a commit.\n> \n> So, how do you do commit messages? It generally doesn't make sense to \n> share the same commit message for submodules - the sub-commits generally \n> do different things.\n\nThe same way you do the first commit message. Ask independantly for each \ncommit message in sequence with enough context in the comment section that \nyou know what you're talking about.\n\n> I'd actually suggest that \"git commit -a\" with non-clean submodules error \n> out for that reason, with something like\n> \n> \tsubmodule 'src/xyzzy' is not up-to-date, please commit changes to \n> \tthat first.\n> \n> exactly because you really generally should consider the submodule commits \n> to be a separate phase.\n\nI think this is getting close to the classic usability blunder of having \nthe program tell you what you should have done instead of what you did, \nand then making you do it yourself, rather than just doing it.\n\nJust have it run \"git commit -a\" in each dirty submodule recursively as \npart of preparing the index, since that's what the user wants to do \nanyway, and nothing already done would be affected.\n\n\"git commit -a -m \" should probably fail, of course.\n\n\t-Daniel\n"},{"id":"295974","messageId":"456C0313.3020308@op5.se","threadId":"43106","inReplyTo":"Pine.LNX.4.64.0611260241320.20138@iabervon.org","subject":"Re: [RFC] Submodules in GIT","fromName":"Andreas Ericsson","fromEmail":"ae@op5.se","sentAt":"2006-11-28T09:36:19Z","receivedAt":"2006-11-28T09:36:19Z","isPatch":false,"sender":{"key":"ae@op5.se","avatar":"https://gravatar.com/avatar/426e89595c75a8f5252dd0c989e5fabe5bcac616e68557427ad9aef6b0ca342a?d=mp&s=160"},"body":"Daniel Barkalow wrote:\n> On Sat, 25 Nov 2006, Linus Torvalds wrote:\n> \n>> On Sun, 26 Nov 2006, Yann Dirson wrote:\n>>> Also, I'd rather expect \"git-commit -a\" outside of any submodule to\n>>> commit everything in the supermodule, triggering submodule commits as an\n>>> intermediate step when needed - just like \"git-commit -a\" does not\n>>> require to manually specify subdirectories to inclue in the commit. I'd\n>>> rather expect a special flag to exclude submodules from a commit.\n>> So, how do you do commit messages? It generally doesn't make sense to \n>> share the same commit message for submodules - the sub-commits generally \n>> do different things.\n> \n> The same way you do the first commit message. Ask independantly for each \n> commit message in sequence with enough context in the comment section that \n> you know what you're talking about.\n> \n>> I'd actually suggest that \"git commit -a\" with non-clean submodules error \n>> out for that reason, with something like\n>>\n>> \tsubmodule 'src/xyzzy' is not up-to-date, please commit changes to \n>> \tthat first.\n>>\n>> exactly because you really generally should consider the submodule commits \n>> to be a separate phase.\n> \n> I think this is getting close to the classic usability blunder of having \n> the program tell you what you should have done instead of what you did, \n> and then making you do it yourself, rather than just doing it.\n> \n> Just have it run \"git commit -a\" in each dirty submodule recursively as \n> part of preparing the index, since that's what the user wants to do \n> anyway, and nothing already done would be affected.\n> \n\nRunning \"commit -a\" is definitely the wrong thing to do, as it prevents \none from using the index at all. Erroring out if the submodules are \ndirty, or just accepting the fact that they are and taking whatever \ncommit HEAD points to is *always* preferrable.\n\nI'd actually prefer the second solution here and let git print a list of \nsubmodules with dirty state and ask for some sort of user-response \nbefore creating the actual commit. As non-interactive commits should \nalways be clean, requiring user intervention on non-clean state should \nbe a safe thing to do.\n\n> \"git commit -a -m \" should probably fail, of course.\n> \n\nWhy? There's no reason to rob this command of its power just because \nwe're using submodules.\n\n> \t-Daniel\n> *This .sig left intentionally blank*\n> -\n> To unsubscribe from this list: send the line \"unsubscribe git\" in\n> the body of a message to majordomo@vger.kernel.org\n> More majordomo info at http://vger.kernel.org/majordomo-info.html\n> \n\n-- \nAndreas Ericsson andreas.ericsson@op5.se\nOP5 AB www.op5.se\n"},{"id":"294255","messageId":"Pine.LNX.4.64.0611281218290.20138@iabervon.org","threadId":"43106","inReplyTo":"456C0313.3020308@op5.se","subject":"Re: [RFC] Submodules in GIT","fromName":"Daniel Barkalow","fromEmail":"barkalow@iabervon.org","sentAt":"2006-11-28T17:28:47Z","receivedAt":"2006-11-28T17:28:47Z","isPatch":false,"sender":{"key":"barkalow@iabervon.org","avatar":"https://avatars.githubusercontent.com/u/55364219?v=4"},"body":"On Tue, 28 Nov 2006, Andreas Ericsson wrote:\n\n> Daniel Barkalow wrote:\n> > On Sat, 25 Nov 2006, Linus Torvalds wrote:\n> > > I'd actually suggest that \"git commit -a\" with non-clean submodules error\n> > > out for that reason\n> > \n> > Just have it run \"git commit -a\" in each dirty submodule recursively as part\n> > of preparing the index, since that's what the user wants to do anyway, and\n> > nothing already done would be affected.\n> > \n> \n> Running \"commit -a\" is definitely the wrong thing to do, as it prevents one\n> from using the index at all. Erroring out if the submodules are dirty, or just\n> accepting the fact that they are and taking whatever commit HEAD points to is\n> *always* preferrable.\n\nI don't think anyone would actually use the index in submodules but not in \nthe supermodule. If submodules are seen mostly as ordinary directories as \nfar as the supermodule's working directory is concerned, it wouldn't make \nsense to not commit dirty state in a subdirectory with -a just because \nit's a submodule.\n\nIt would be wrong to do \"commit -a\" in submodules if the supermodule \nweren't being committed with -a, of course.\n\n> > \"git commit -a -m \" should probably fail, of course.\n> > \n> \n> Why? There's no reason to rob this command of its power just because we're\n> using submodules.\n\nIt should fail if there are dirty submodules, because the user needs to \nprovide a commit message for each of them, and only one commit message can \nbe provided this way, and -m inhibits invoking an editor.\n\n\t-Daniel\n"},{"id":"296071","messageId":"20061128180817.GA12463MdfPADPa@greensroom.kotnet.org","threadId":"43106","inReplyTo":"Pine.LNX.4.64.0611281218290.20138@iabervon.org","subject":"Re: [RFC] Submodules in GIT","fromName":"Sven Verdoolaege","fromEmail":"skimo@kotnet.org","sentAt":"2006-11-28T18:08:18Z","receivedAt":"2006-11-28T18:08:18Z","isPatch":false,"sender":{"key":"skimo@kotnet.org","avatar":null},"body":"On Tue, Nov 28, 2006 at 12:28:47PM -0500, Daniel Barkalow wrote:\n> It would be wrong to do \"commit -a\" in submodules if the supermodule \n> weren't being committed with -a, of course.\n\nWhat if you say \"git commit submodule\" ?\nI sure hope you wouldn't want to do a \"commit -a\" in the submodule.\nOne of the nice features of git is that you can still perform most\noperations if you have a dirty state and I would very much want to\nbe able to commit only some changes in the submodule and then only\ncommit that change in submodule commits in the supermodule without\nhaving my other changes in the submodule committed as well.\n\nIf you agree with the above, then why should \"git commit -a\"\ndo any different from \"git commit submodule\" if submodule was\nthe only thing that got changed ?\n\n"},{"id":"294655","messageId":"Pine.LNX.4.64.0611281315020.20138@iabervon.org","threadId":"43106","inReplyTo":"20061128180817.GA12463MdfPADPa@greensroom.kotnet.org","subject":"Re: [RFC] Submodules in GIT","fromName":"Daniel Barkalow","fromEmail":"barkalow@iabervon.org","sentAt":"2006-11-28T18:37:54Z","receivedAt":"2006-11-28T18:37:54Z","isPatch":false,"sender":{"key":"barkalow@iabervon.org","avatar":"https://avatars.githubusercontent.com/u/55364219?v=4"},"body":"On Tue, 28 Nov 2006, Sven Verdoolaege wrote:\n\n> On Tue, Nov 28, 2006 at 12:28:47PM -0500, Daniel Barkalow wrote:\n> > It would be wrong to do \"commit -a\" in submodules if the supermodule \n> > weren't being committed with -a, of course.\n> \n> What if you say \"git commit submodule\" ?\n\nObviously no -a, as I said.\n\n> If you agree with the above, then why should \"git commit -a\"\n> do any different from \"git commit submodule\" if submodule was\n> the only thing that got changed ?\n\nIf submodule was the only thing that got changed, it's not dirty; if it \nwere dirty, some of its contents would also have gotten changed. Surely:\n\n\"git commit submodule/foo bar\"\n\nshould do \"git commit foo\" in submodule, and then commit the supermodule \nwith the new commit for the submodule and the change to bar. And so\n\"submodule/foo\" is something you could commit changes to, so it should get \npicked up by -a.\n\nOf course, if submodule *is* the *only* thing that changed (e.g., you did \na fast-forward merge in it, or you've previously committed it completely), \nthere won't be a \"commit -a\" in it, because that would just generate a \ngratuitous commit.\n\n\t-Daniel\n"},{"id":"294887","messageId":"20061128190618.GB12463MdfPADPa@greensroom.kotnet.org","threadId":"43106","inReplyTo":"Pine.LNX.4.64.0611281315020.20138@iabervon.org","subject":"Re: [RFC] Submodules in GIT","fromName":"Sven Verdoolaege","fromEmail":"skimo@kotnet.org","sentAt":"2006-11-28T19:06:18Z","receivedAt":"2006-11-28T19:06:18Z","isPatch":false,"sender":{"key":"skimo@kotnet.org","avatar":null},"body":"On Tue, Nov 28, 2006 at 01:37:54PM -0500, Daniel Barkalow wrote:\n> If submodule was the only thing that got changed, it's not dirty; if it \n> were dirty, some of its contents would also have gotten changed.\n\nFor me, the commit is the only \"content\" of the subproject that the\nsuperproject should care about, so the submodule being dirty or not\nis completely irrelevant (for committing), but it seems you see the\nsubproject more as a (working) tree than as a commit. Of course, as\nLinus already mentioned, a \"git commit\" could still warn you if the\nsubproject was dirty.\n\n> Surely:\n> \n> \"git commit submodule/foo bar\"\n\nI wouldn't dream of doing such an operation, because it doesn't make\nsense to me. (So as far as I'm concerned, you can make it do whatever\nyou'd like it to do.) You can only commit the subproject as a whole.\n\n> should do \"git commit foo\" in submodule, and then commit the supermodule \n> with the new commit for the submodule and the change to bar. And so\n> \"submodule/foo\" is something you could commit changes to, so it should get \n> picked up by -a.\n\n"},{"id":"295759","messageId":"Pine.LNX.4.64.0611281407370.20138@iabervon.org","threadId":"43106","inReplyTo":"20061128190618.GB12463MdfPADPa@greensroom.kotnet.org","subject":"Re: [RFC] Submodules in GIT","fromName":"Daniel Barkalow","fromEmail":"barkalow@iabervon.org","sentAt":"2006-11-28T20:41:01Z","receivedAt":"2006-11-28T20:41:01Z","isPatch":false,"sender":{"key":"barkalow@iabervon.org","avatar":"https://avatars.githubusercontent.com/u/55364219?v=4"},"body":"On Tue, 28 Nov 2006, Sven Verdoolaege wrote:\n\n> On Tue, Nov 28, 2006 at 01:37:54PM -0500, Daniel Barkalow wrote:\n> > If submodule was the only thing that got changed, it's not dirty; if it \n> > were dirty, some of its contents would also have gotten changed.\n> \n> For me, the commit is the only \"content\" of the subproject that the\n> superproject should care about, so the submodule being dirty or not\n> is completely irrelevant (for committing), but it seems you see the\n> subproject more as a (working) tree than as a commit.\n\nI think we agree on the tree/commit/object database model part.\n\nI think we disagree on how the working *directories* relate. I see the \nchecked-out state of a submodule as being relevant to the checked-out \nstate of the supermodule, such that dirty state in the submodule directory \nis dirty state in the supermodule directory.\n\n> > Surely:\n> > \n> > \"git commit submodule/foo bar\"\n> \n> I wouldn't dream of doing such an operation, because it doesn't make\n> sense to me. (So as far as I'm concerned, you can make it do whatever\n> you'd like it to do.) You can only commit the subproject as a whole.\n\nI'm thinking that users of subprojects will often want to work on the\nsubprojects rather than exclusively using commits prepared by other \npeople, and it's too much trouble to have to do the work in a repository \nfor just the subproject and pull it into the superproject's submodule to \ntest it. So the submodule working directory needs to function as a working \ndirectory for the subproject. Then\n\n \"cd submodule; git commit foo\"\n\ndoes the obvious thing, but that should be the same as\n\n \"git commit submodule/foo\" (since it normally is)\n\nand then it makes sense to let you do multiple commits with a single \ncommand when the paths end in different modules, since that's obviously \nwhat you're requesting, and then -a must do all of them.\n\n\t-Daniel\n"},{"id":"296172","messageId":"20061128211012.GJ28337@spearce.org","threadId":"43106","inReplyTo":"Pine.LNX.4.64.0611281407370.20138@iabervon.org","subject":"Re: [RFC] Submodules in GIT","fromName":"Shawn Pearce","fromEmail":"spearce@spearce.org","sentAt":"2006-11-28T21:10:12Z","receivedAt":"2006-11-28T21:10:12Z","isPatch":false,"sender":{"key":"spearce@spearce.org","avatar":"https://avatars.githubusercontent.com/u/34844?v=4"},"body":"Daniel Barkalow wrote:\n> \"cd submodule; git commit foo\"\n> \n> does the obvious thing, but that should be the same as\n> \n> \"git commit submodule/foo\" (since it normally is)\n> \n> and then it makes sense to let you do multiple commits with a single \n> command when the paths end in different modules, since that's obviously \n> what you're requesting, and then -a must do all of them.\n\nExcept what if the submodules have different commit message\nstandards? E.g. one requires signoff and another doesn't? Or one\nallows privately held information (e.g. its your coporate project)\nand one doesn't (e.g. its an open source project you use/contribute\nto)?\n\nBut slightly more practical: the change message for the superproject\nmight simply be \"resolved bug X, caused by ...\". Which may make a\nlot of sense to the top level project, but makes no sense at all\nin a submodule involved in the fix as the submodule's developer\ncommunity doesn't even know what \"X\" is, let alone how \"...\" could\nhave caused it.\n\nSo you really need to think twice before you apply the same commit\nmessage to every project, as each commit message needs to make sense\nwith that one submodule's limited scope, or within the supermodule's\nlarger scope.\n\nBut if you really still think that the same commit message makes\nsense everywhere, we have 'git commit -F'. Write it out in a file\nand hand it off to -F in each module. This would be easier if\ngit-ls-files grew a new option:\n\n\tvi ~/msg\n\tfor m in $(git ls-files --submodules); do git commit -F ~/msg; done\n\tgit commit -F ~/msg\n\n-- \n"},{"id":"298060","messageId":"Pine.LNX.4.64.0611281614450.20138@iabervon.org","threadId":"43106","inReplyTo":"20061128211012.GJ28337@spearce.org","subject":"Re: [RFC] Submodules in GIT","fromName":"Daniel Barkalow","fromEmail":"barkalow@iabervon.org","sentAt":"2006-11-28T21:32:21Z","receivedAt":"2006-11-28T21:32:21Z","isPatch":false,"sender":{"key":"barkalow@iabervon.org","avatar":"https://avatars.githubusercontent.com/u/55364219?v=4"},"body":"On Tue, 28 Nov 2006, Shawn Pearce wrote:\n\n> Daniel Barkalow wrote:\n> > and then it makes sense to let you do multiple commits with a single \n> > command when the paths end in different modules, since that's obviously \n> > what you're requesting, and then -a must do all of them.\n> \n> Except what if the submodules have different commit message\n> standards? E.g. one requires signoff and another doesn't? Or one\n> allows privately held information (e.g. its your coporate project)\n> and one doesn't (e.g. its an open source project you use/contribute\n> to)?\n\nI don't think you'd ever want the same commit message for commits in two \nprojects. In any case where you'd commit a submodule in the process of \ncommitting a supermodule, git would do this by recursively calling \ngit-commit, which would prompt for separate commit messages.\n\n\t-Daniel\n"},{"id":"297193","messageId":"Pine.LNX.4.64.0611281349540.4244@woody.osdl.org","threadId":"43106","inReplyTo":"Pine.LNX.4.64.0611281614450.20138@iabervon.org","subject":"Re: [RFC] Submodules in GIT","fromName":"Linus Torvalds","fromEmail":"torvalds@osdl.org","sentAt":"2006-11-28T21:53:29Z","receivedAt":"2006-11-28T21:53:29Z","isPatch":false,"sender":{"key":"torvalds@linux-foundation.org","avatar":"https://avatars.githubusercontent.com/u/1024025?v=4"},"body":"\n\nOn Tue, 28 Nov 2006, Daniel Barkalow wrote:\n> \n> I don't think you'd ever want the same commit message for commits in two \n> projects.\n\nI don't know about \"ever\", but yes, I do think submodule commits are \ngenerally totally separate things from supermodule commits.\n\n> In any case where you'd commit a submodule in the process of \n> committing a supermodule, git would do this by recursively calling \n> git-commit, which would prompt for separate commit messages.\n\nThat certainly works, although I'm not convinved that it's necessarily a \nhugely important detail.\n\nI suspect there may well be more important things UI-wise wrt submodules \nthan the \"you may have to commit submodules separately\" question.\n\nFor example, doing a \"git pull\" is a lot more interesting, since that \nactually has the potential of having to resolve conflicts in submodules \nbefore the supermodule can be committed. Getting all the \"git reset\" \nbehaviour right for when you decide \"oops, that was too complicated\" is \nprobably a lot more important than whether you have to have a separate \n\"commit subproject\" phase for the simple cases of doing a bog-standard \n\"git commit -a\".\n\n"},{"id":"296455","messageId":"456EE3F1.5070101@b-i-t.de","threadId":"43106","inReplyTo":"200611301255.41733.andyparkins@gmail.com","subject":"Re: [RFC] Submodules in GIT","fromName":"Stephan Feder","fromEmail":"sf@b-i-t.de","sentAt":"2006-11-30T14:00:17Z","receivedAt":"2006-11-30T14:00:17Z","isPatch":false,"sender":{"key":"sf@b-i-t.de","avatar":null},"body":"Andy Parkins wrote:\n> On Thursday 2006 November 30 11:57, sf wrote:\n> \n>> > Worse, if you allow that to happen, the supermodule can commit a state\n>> > that cannot be retrieved from the submodule's repository. The ONLY thing\n>> > a supermodule can record about a submodule is a commit.\n>>\n>> So what? You have a submodule commit that only exists in the\n>> supermodule. I fail to see the problem. The changes you made to the\n>> submodule _in the supermodule_ can later be pulled from wherever you want.\n> \n> Eh? The files aren't stored in the supermodule, they're stored in the \n> submodule. The ONLY thing in the supermodule is the commit hash. The \n> objects for the submodule are still /in/ the submodule.\n\nBut you have got the submodule on your local disk anyway. So just setup \nalternates and the supermodule contains all of the submodule.\n\n> It sounds like you're suggesting that the supermodule commit includes files \n> from the submodule? How can that work? The two aren't separate entities \n> then, it's just one big repository. \n\nIt works as it always works in git: The supermodule commit contains the \nsubmodule commit, the submodule commit contains the submodule files, so \nthe supermodule contains the submodule (at least the part of the \nsubmodule that is visible). It _must_ be one repository but it need not \nbe big (once more, use alternates).\n\n> I mean, what would this supermodule commit look like? Would it include a \n> commit message? Which module should that commit message be about? Should \n> the commit's parents be stored? Which parents, the submodule HEAD or the \n> supermodule HEAD? Which tree object should it link to? The one in the \n> submodule doesn't exist, so it'll have to be a freshly made up one for the \n> supermodule - except now you've put submodule paths in the supermodule. \n> Nope. That's never going to work.\n\nAgain I do not see the problem. Probably I have a much simpler picture \nof submodules: They are just commits in the supermodule's tree. \nEverything else follows naturally from how git currently behaves.\n\nOf course it works. It is simple, it is the git way.\n\nAm I missing the point?\n\nRegards\n\nStephan\n\n-- \nb.i.t.\nberatungsgesellschaft für informations-technologie mbh\nStephan Feder\nelisabethenstr. 62 fon: +49(0)6151/827575\n64283 darmstadt fax: +49(0)6151/827576\nmailto:sf@b-i-t.de www: http://www.b-i-t.de\n"},{"id":"297483","messageId":"200611301449.55171.andyparkins@gmail.com","threadId":"43106","inReplyTo":"456EE3F1.5070101@b-i-t.de","subject":"Re: [RFC] Submodules in GIT","fromName":"Andy Parkins","fromEmail":"andyparkins@gmail.com","sentAt":"2006-11-30T14:49:53Z","receivedAt":"2006-11-30T14:49:53Z","isPatch":false,"sender":{"key":"andyparkins@gmail.com","avatar":null},"body":"On Thursday 2006 November 30 14:00, Stephan Feder wrote:\n\n> Again I do not see the problem. Probably I have a much simpler picture\n> of submodules: They are just commits in the supermodule's tree.\n> Everything else follows naturally from how git currently behaves.\n\nHow are these commits any different from just having one big repository? If \nsome of the development of the submodule is contained in the supermodule then \nit's not a submodule anymore.\n\nWhy bother with all the effort to make a separation between submodule and \nsupermodule and then store the submodule commits in the supermodule. That's \nnot supermodule/submodule git - that's just normal git.\n\nSurely the whole point of having submodule's is so that you can take the \nsubmodule away. Let me give you an example. Let's say I have a project that \nuses the libxcb library (some random project out in the world that uses git). \nI've arranged it something like this:\n\nmyproject (git root)\n |----- src\n |----- doc\n `----- libxcb (git root)\n\nThis works fine; with one problem. When I make a commit in myproject, there \nis no link into the particular snapshot of the libxcb that I used at that \nmoment. If libxcb moves on, and makes incompatible changes, then when I \ncheckout an old version of myproject, it won't compile any more because I'll \nneed to find out which commit of libxcb I used at the time.\n\nSubmodules will solve this problem. In the future I'll be able to check out \nany commit of myproject and it will automatically checkout the right commit \nfrom the libxcb repository. Now let's say I'm working away and find a bug in \nlibxcb; I fix it, commit it. That change had better be stored in the libxcb \nrepository, and had better make no reference to the myproject repository. If \nit doesn't, I'm going to have to pollute the libxcb upstream repository with \nmyproject if I want to share those fixes.\n\n\n\nAndy\n-- \nDr Andy Parkins, M Eng (hons), MIEE\n"},{"id":"295306","messageId":"20061130152011.GM12463MdfPADPa@greensroom.kotnet.org","threadId":"43106","inReplyTo":"200611301449.55171.andyparkins@gmail.com","subject":"Re: [RFC] Submodules in GIT","fromName":"Sven Verdoolaege","fromEmail":"skimo@kotnet.org","sentAt":"2006-11-30T15:20:11Z","receivedAt":"2006-11-30T15:20:11Z","isPatch":false,"sender":{"key":"skimo@kotnet.org","avatar":null},"body":"On Thu, Nov 30, 2006 at 02:49:53PM +0000, Andy Parkins wrote:\n> How are these commits any different from just having one big repository? If \n\nYou can work on the submodule independently.\n\n> some of the development of the submodule is contained in the supermodule then \n> it's not a submodule anymore.\n\nOn the contrary, that's exactly what a submodule is supposed to be.\n\n> Why bother with all the effort to make a separation between submodule and \n> supermodule and then store the submodule commits in the supermodule. That's \n> not supermodule/submodule git - that's just normal git.\n\n[..]\n\n> myproject (git root)\n> |----- src\n> |----- doc\n> `----- libxcb (git root)\n> \n[..]\n> \n> Submodules will solve this problem. In the future I'll be able to check out \n> any commit of myproject and it will automatically checkout the right commit \n> from the libxcb repository.\n\nHow are you going to checkout the right commit of the lixcb repo if\nyou didn't store it in the supermodule ?\n\n"},{"id":"296786","messageId":"200611301530.51171.andyparkins@gmail.com","threadId":"43106","inReplyTo":"20061130152011.GM12463MdfPADPa@greensroom.kotnet.org","subject":"Re: [RFC] Submodules in GIT","fromName":"Andy Parkins","fromEmail":"andyparkins@gmail.com","sentAt":"2006-11-30T15:30:49Z","receivedAt":"2006-11-30T15:30:49Z","isPatch":false,"sender":{"key":"andyparkins@gmail.com","avatar":null},"body":"On Thursday 2006 November 30 15:20, Sven Verdoolaege wrote:\n\n> You can work on the submodule independently.\n\nIt's not independent if any part of it is in the supermodule.\n\n> > some of the development of the submodule is contained in the supermodule\n> > then it's not a submodule anymore.\n>\n> On the contrary, that's exactly what a submodule is supposed to be.\n\nI don't think so. I think it's just made some complicated normal repository.\n\n> How are you going to checkout the right commit of the lixcb repo if\n> you didn't store it in the supermodule ?\n\nWell, I know what the commit is /that/ was all that was stored. So I \n(actually supermodule-git does):\n\ncd $DIRECTORY_ASSOCIATED_WITH_SUBMODULE\ngit checkout -f $COMMIT_FROM_SUPERMODULE\n\nObviously, this is grossly simplified. It also requires that HEAD be allowed \nto be an arbitrary commit rather than a branch, but that's already been \ngenerally agreed upon as a good thing.\n\nAndy\n-- \nDr Andy Parkins, M Eng (hons), MIEE\n"},{"id":"296478","messageId":"456EFDDE.9010705@op5.se","threadId":"43106","inReplyTo":"200611301530.51171.andyparkins@gmail.com","subject":"Re: [RFC] Submodules in GIT","fromName":"Andreas Ericsson","fromEmail":"ae@op5.se","sentAt":"2006-11-30T15:50:54Z","receivedAt":"2006-11-30T15:50:54Z","isPatch":false,"sender":{"key":"ae@op5.se","avatar":"https://gravatar.com/avatar/426e89595c75a8f5252dd0c989e5fabe5bcac616e68557427ad9aef6b0ca342a?d=mp&s=160"},"body":"Andy Parkins wrote:\n> On Thursday 2006 November 30 15:20, Sven Verdoolaege wrote:\n> \n>> You can work on the submodule independently.\n> \n> It's not independent if any part of it is in the supermodule.\n> \n>>> some of the development of the submodule is contained in the supermodule\n>>> then it's not a submodule anymore.\n>> On the contrary, that's exactly what a submodule is supposed to be.\n> \n> I don't think so. I think it's just made some complicated normal repository.\n> \n\nI believe that Andy meant \"development history\" in his above scentence. \nNaturally, using the code from the submodule while being capable of \ndeveloping the submodule separately from the supermodule is what \nsubmodules are all about.\n\n>> How are you going to checkout the right commit of the lixcb repo if\n>> you didn't store it in the supermodule ?\n> \n> Well, I know what the commit is /that/ was all that was stored. So I \n> (actually supermodule-git does):\n> \n> cd $DIRECTORY_ASSOCIATED_WITH_SUBMODULE\n> git checkout -f $COMMIT_FROM_SUPERMODULE\n> \n> Obviously, this is grossly simplified. It also requires that HEAD be allowed \n> to be an arbitrary commit rather than a branch, but that's already been \n> generally agreed upon as a good thing.\n> \n\nIt has? We're not talking supermodule specific things anymore, are we?\n\n-- \nAndreas Ericsson andreas.ericsson@op5.se\nOP5 AB www.op5.se\n"},{"id":"295876","messageId":"456F0153.5000107@b-i-t.de","threadId":"43106","inReplyTo":"200611301449.55171.andyparkins@gmail.com","subject":"Re: [RFC] Submodules in GIT","fromName":"sf","fromEmail":"sf@b-i-t.de","sentAt":"2006-11-30T16:05:39Z","receivedAt":"2006-11-30T16:05:39Z","isPatch":false,"sender":{"key":"sf@b-i-t.de","avatar":null},"body":"Andy Parkins wrote:\n> On Thursday 2006 November 30 14:00, Stephan Feder wrote:\n> \n>> Again I do not see the problem. Probably I have a much simpler picture\n>> of submodules: They are just commits in the supermodule's tree.\n>> Everything else follows naturally from how git currently behaves.\n> \n> How are these commits any different from just having one big repository? If \n> some of the development of the submodule is contained in the supermodule then \n> it's not a submodule anymore.\n\nRight now you only have commits of the top directory aka the super \nproject. Every subdirectory is just that: a directory (which git stores \nas trees).\n\nNow, if you have a subdirectory that git stores as a commit, not a tree, \nyou have a subproject. It is a directory with history, and because the \ncommit is part of your superprject, you have access to this history.\n\n> Why bother with all the effort to make a separation between submodule and \n> supermodule and then store the submodule commits in the supermodule. That's \n> not supermodule/submodule git - that's just normal git.\n\nNo, it is not. Currently, there is no way to store a commit within the \ncontents of another commit. You can only store trees and blobs.\n\n> Surely the whole point of having submodule's is so that you can take the \n> submodule away. Let me give you an example. Let's say I have a project that \n> uses the libxcb library (some random project out in the world that uses git). \n> I've arranged it something like this:\n> \n> myproject (git root)\n> |----- src\n> |----- doc\n> `----- libxcb (git root)\n> \n> This works fine; with one problem. When I make a commit in myproject, there \n> is no link into the particular snapshot of the libxcb that I used at that \n> moment. If libxcb moves on, and makes incompatible changes, then when I \n> checkout an old version of myproject, it won't compile any more because I'll \n> need to find out which commit of libxcb I used at the time.\n\nOK.\n\n> Submodules will solve this problem. In the future I'll be able to check out \n> any commit of myproject and it will automatically checkout the right commit \n> from the libxcb repository.\n\nOK, I am still with you so far.\n\n> Now let's say I'm working away and find a bug in \n> libxcb; I fix it, commit it. That change had better be stored in the libxcb \n> repository, and had better make no reference to the myproject repository. If \n> it doesn't, I'm going to have to pollute the libxcb upstream repository with \n> myproject if I want to share those fixes.\n\nHere comes the part where we did not meet before.\n\nOf course you do not make any reference from your subproject to your \nsuperproject. You do exactly what you do in git today when you work with \ndifferent branches:\n\nStep 1: You fix a bug in myproject's subdirectory libxcb.\n\nStep 2: You commit to myproject. myproject now contains a new commit \nobject in path libxcb. (How to do that is up to the UI but at the \nrepository level the outcome should be obvious). This commit is local to \nyour repository.\n\nStep 3: You propose your changes to the libxcb upstream (it might not be \na repository you have write access to). I use the following made up \nsyntax (see man git-rev-parse):\n\nA suffix : followed by a path, _followed by a suffix //::_ names the \n_revision_ at the given path in the tree-ish object named by the part \nbefore the colon.\n\nStep 3a: Generate a patch\n\ngit diff libxcb//^..libxcb//\n\nStep 3b: Push your changes\n\ngit push HEAD:libxcb//:\n\nStep 3c: Let your changes be pulled\n\n\"Hello, please pull HEAD:libxcb//:\"\n\nStep 4: Pull upstream version (hopefully with your changes, otherwise \nyou have to merge)\n\ngit pull ::HEAD:libxcb//\n\nSee, it works.\n\n From what I understand you want to do the commit and push steps in one \ngo. How do you want to record local (to your superproject) changes to \nthe subproject?\n\nRegards\n\n"},{"id":"294161","messageId":"200611301608.28560.andyparkins@gmail.com","threadId":"43106","inReplyTo":"456EFDDE.9010705@op5.se","subject":"Re: [RFC] Submodules in GIT","fromName":"Andy Parkins","fromEmail":"andyparkins@gmail.com","sentAt":"2006-11-30T16:08:27Z","receivedAt":"2006-11-30T16:08:27Z","isPatch":false,"sender":{"key":"andyparkins@gmail.com","avatar":null},"body":"On Thursday 2006 November 30 15:50, Andreas Ericsson wrote:\n\n> > Obviously, this is grossly simplified. It also requires that HEAD be\n> > allowed to be an arbitrary commit rather than a branch, but that's\n> > already been generally agreed upon as a good thing.\n>\n> It has? We're not talking supermodule specific things anymore, are we?\n\nNot entirely, although I think it's going to be handy for submodules. It was \nin a thread about remotes branches. By allowing checkout of any commit \nrather than only those that have a ref/heads/ entry, you effectively have a \nread-only checkout. You obviously couldn't commit to a repository like this, \nbecause HEAD wouldn't point at anything that is changeable. It would be very \neasy to just git-branch from there and start work though.\n\nI think it's going to be necessary for the submodule work, because without it \nthe supermodule will have to create it's own temporary branches in the \nsubmodule in order to checkout an arbitrary commit.\n\n\nAndy\n\n-- \nDr Andy Parkins, M Eng (hons), MIEE\n"},{"id":"295961","messageId":"456F02FB.6010901@b-i-t.de","threadId":"43106","inReplyTo":"456F0153.5000107@b-i-t.de","subject":"Re: [RFC] Submodules in GIT","fromName":"sf","fromEmail":"sf@b-i-t.de","sentAt":"2006-11-30T16:12:43Z","receivedAt":"2006-11-30T16:12:43Z","isPatch":false,"sender":{"key":"sf@b-i-t.de","avatar":null},"body":"sf wrote:\n...\n> A suffix : followed by a path, _followed by a suffix //::_ names the \n> _revision_ at the given path in the tree-ish object named by the part \n> before the colon.\n\nSorry, that was supposed to read: followed by a suffix //\n\n"},{"id":"297751","messageId":"20061130163304.GN12463MdfPADPa@greensroom.kotnet.org","threadId":"43106","inReplyTo":"200611301530.51171.andyparkins@gmail.com","subject":"Re: [RFC] Submodules in GIT","fromName":"Sven Verdoolaege","fromEmail":"skimo@kotnet.org","sentAt":"2006-11-30T16:33:04Z","receivedAt":"2006-11-30T16:33:04Z","isPatch":false,"sender":{"key":"skimo@kotnet.org","avatar":null},"body":"On Thu, Nov 30, 2006 at 03:30:49PM +0000, Andy Parkins wrote:\n> On Thursday 2006 November 30 15:20, Sven Verdoolaege wrote:\n> > How are you going to checkout the right commit of the lixcb repo if\n> > you didn't store it in the supermodule ?\n> \n> Well, I know what the commit is /that/ was all that was stored. So I \n\nThen I have no idea what you are talking about.\nA commit _contains_ all the history that lead up to that commit,\nso if you have the commit, then you also have the history.\n\n"},{"id":"296148","messageId":"20061130171949.GI18810@admingilde.org","threadId":"43106","inReplyTo":"200611301530.51171.andyparkins@gmail.com","subject":"Re: [RFC] Submodules in GIT","fromName":"Martin Waitz","fromEmail":"tali@admingilde.org","sentAt":"2006-11-30T17:19:49Z","receivedAt":"2006-11-30T17:19:49Z","isPatch":false,"sender":{"key":"tali@admingilde.org","avatar":"https://gravatar.com/avatar/3f89b03eee362187effabe257898735b475673a12265c398ea9161259ae91553?d=mp&s=160"},"body":"hoi :)\n\nOn Thu, Nov 30, 2006 at 03:30:49PM +0000, Andy Parkins wrote:\n> Well, I know what the commit is /that/ was all that was stored. So I \n> (actually supermodule-git does):\n> \n> cd $DIRECTORY_ASSOCIATED_WITH_SUBMODULE\n> git checkout -f $COMMIT_FROM_SUPERMODULE\n> \n> Obviously, this is grossly simplified. It also requires that HEAD be allowed \n> to be an arbitrary commit rather than a branch, but that's already been \n> generally agreed upon as a good thing.\n\nIt's not that easy.\n\nYou also have to make sure that all your submodule commits that _ever_\nhave been part of your submodule have be stay in your repository\nforever.\nConsider that your submodule switches to an other branch and some\nold commits are not referenced by the current version any more.\nThese old commits still have to survive a git-prune, if they have been\npart of some old supermodule version.\nSo you really have to connect both object databases and it's not enough\nto just store the commit sha1 without actually parsing it by the GIT\ncore.\n\n-- \nMartin Waitz\n"},{"id":"297106","messageId":"200612010001.57111.andyparkins@gmail.com","threadId":"43106","inReplyTo":"20061130163304.GN12463MdfPADPa@greensroom.kotnet.org","subject":"Re: [RFC] Submodules in GIT","fromName":"Andy Parkins","fromEmail":"andyparkins@gmail.com","sentAt":"2006-12-01T00:01:54Z","receivedAt":"2006-12-01T00:01:54Z","isPatch":false,"sender":{"key":"andyparkins@gmail.com","avatar":null},"body":"On Thursday 2006, November 30 16:33, Sven Verdoolaege wrote:\n> > Well, I know what the commit is /that/ was all that was stored. So I\n>\n> Then I have no idea what you are talking about.\n> A commit _contains_ all the history that lead up to that commit,\n> so if you have the commit, then you also have the history.\n\nIt's not so much an actual commit, as a reference to a commit in another \nrepository.\n\nAndy\n\n-- \nDr Andrew Parkins, M Eng (Hons), AMIEE\n"},{"id":"294120","messageId":"eknrr8$n27$2@sea.gmane.org","threadId":"43106","inReplyTo":"200612010001.57111.andyparkins@gmail.com","subject":"Re: [RFC] Submodules in GIT","fromName":"Jakub Narebski","fromEmail":"jnareb@gmail.com","sentAt":"2006-12-01T00:11:07Z","receivedAt":"2006-12-01T00:11:07Z","isPatch":false,"sender":{"key":"jnareb@gmail.com","avatar":"https://avatars.githubusercontent.com/u/2706?v=4"},"body":"Andy Parkins wrote:\n\n> On Thursday 2006, November 30 16:33, Sven Verdoolaege wrote:\n>>>\n>>> Well, I know what the commit is /that/ was all that was stored. So I\n>>\n>> Then I have no idea what you are talking about.\n>> A commit _contains_ all the history that lead up to that commit,\n>> so if you have the commit, then you also have the history.\n> \n> It's not so much an actual commit, as a reference to a commit in another \n> repository.\n\nHmmm... I thought the idea was that submodule commit is available in the\nobject repository, be it via alternates mechanism pointing to the submodule\nrepository for alternate storage, or submodule being in \"unrelated\"\nbranch/tracking branch.\n-- \nJakub Narebski\nWarsaw, Poland\nShadeHawk on #git\n\n"},{"id":"297459","messageId":"200612010849.11176.andyparkins@gmail.com","threadId":"43106","inReplyTo":"456F29A2.1050205@op5.se","subject":"Re: [RFC] Submodules in GIT","fromName":"Andy Parkins","fromEmail":"andyparkins@gmail.com","sentAt":"2006-12-01T08:49:09Z","receivedAt":"2006-12-01T08:49:09Z","isPatch":false,"sender":{"key":"andyparkins@gmail.com","avatar":null},"body":"On Thursday 2006 November 30 18:57, Andreas Ericsson wrote:\n\n(agree with everything in your mail)\n\n> The only problem I'm seeing atm is that the supermodule somehow has to\n> mark whatever commits it's using from the submodule inside the submodule\n> repo so that they effectively become un-prunable, otherwise the\n> supermodule may some day find itself with a history that it can't restore.\n\nWhat about submodule/.git/refs/supermodule/commit12345678, where \"12345678\" is \nthe hash of the supermodule commit? This gives a convenient route in the \nsubmodule to which commit contains that commit from the submodule; but \ndoesn't write anything into the submodule repository itself. It's just a tag \nwith a different intent.\n\n\nAndy\n-- \nDr Andy Parkins, M Eng (hons), MIEE\n"},{"id":"294429","messageId":"200612010919.06030.andyparkins@gmail.com","threadId":"43106","inReplyTo":"456F0153.5000107@b-i-t.de","subject":"Re: [RFC] Submodules in GIT","fromName":"Andy Parkins","fromEmail":"andyparkins@gmail.com","sentAt":"2006-12-01T09:19:04Z","receivedAt":"2006-12-01T09:19:04Z","isPatch":false,"sender":{"key":"andyparkins@gmail.com","avatar":null},"body":"On Thursday 2006 November 30 16:05, sf wrote:\n\n> Step 2: You commit to myproject. myproject now contains a new commit\n> object in path libxcb. (How to do that is up to the UI but at the\n> repository level the outcome should be obvious). This commit is local to\n> your repository.\n\nLet's imagine a supermodule repository, and guess at it in more detail (I'll \nabbreviate some of the less interesting output):\n\n$ git-cat-file -p HEAD\ntree fb02e78085ecf2f29045603df858b5362e5bf8a4\nparent 4f2dba685507e4a8e07dac298c4024feaec6bd7d\nauthor Andy Parkins\ncommitter Andy Parkins \n$ git-cat-file -p fb02e78085ecf2f29045603df858b5362e5bf8a4\n100644 blob 46bd4e284a57e2faa539e7b72d62a38867075af5 Makefile\n040000 tree 49ea01373a986a3db44d66702714aa75059ffa2c doc\n040000 subm d0a877464dc0198667a3e27ed3af8448ddacf947 libxcb\n\nThe \"subm\" type is our new ODB object that's going to store whatever we will \nneed to access the submodule. \"libxcb\" has already told us where this \nsubmodule is in the supermodule tree.\n\n$ git-cat-file -p d0a877464dc0198667a3e27ed3af8448ddacf947\nsubmodulecommithash ccddf1d4b0cf7fd3a699d8b33cf5bc4c5c4435b7\nsubmoduleurlhint git://anongit.freedesktop.org/git/xcb/libxcb\n\nHere \"submodulecommithash\" is telling us what commit in the submodule is \nstored in this supermodule tree. The \"submoduleurlhint\" is to help when \ngit-clone is used to clone this supermodule.\n\nThey key thing I wanted to point out here is the line:\n submodulecommithash ccddf1d4b0cf7fd3a699d8b33cf5bc4c5c4435b7\nThis is the ONLY link you have to the submodule. I think this line represents \nthe fundamental difference between our thinking on submodules.\n\nI say:\n submodulecommithash points at a commit /in the submodule/\nYou say:\n \"This commit is local to your repository\". i.e. it points at a commit in\n the supermodule, which in turn implies that the local commit object points\n at a local tree and local parents.\n\nMy question is therefore: tell me what that local commit's tree and parent's \nare? At the moment I am having difficulty understanding what meaningful \nthings you could have in those fields.\n\n\n\nAndy\n-- \nDr Andy Parkins, M Eng (hons), MIEE\n"},{"id":"295854","messageId":"20061201093244.GP12463MdfPADPa@greensroom.kotnet.org","threadId":"43106","inReplyTo":"200612010001.57111.andyparkins@gmail.com","subject":"Re: [RFC] Submodules in GIT","fromName":"Sven Verdoolaege","fromEmail":"skimo@kotnet.org","sentAt":"2006-12-01T09:32:44Z","receivedAt":"2006-12-01T09:32:44Z","isPatch":false,"sender":{"key":"skimo@kotnet.org","avatar":null},"body":"On Fri, Dec 01, 2006 at 12:01:54AM +0000, Andy Parkins wrote:\n> On Thursday 2006, November 30 16:33, Sven Verdoolaege wrote:\n> > > Well, I know what the commit is /that/ was all that was stored. So I\n> >\n> > Then I have no idea what you are talking about.\n> > A commit _contains_ all the history that lead up to that commit,\n> > so if you have the commit, then you also have the history.\n> \n> It's not so much an actual commit, as a reference to a commit in another \n> repository.\n\nThis is heresy. Any object referenced in a tree should be in the repo\n(possibly via alternates).\n\n"},{"id":"296537","messageId":"456FF6D1.4040500@op5.se","threadId":"43106","inReplyTo":"200612010849.11176.andyparkins@gmail.com","subject":"Re: [RFC] Submodules in GIT","fromName":"Andreas Ericsson","fromEmail":"ae@op5.se","sentAt":"2006-12-01T09:33:05Z","receivedAt":"2006-12-01T09:33:05Z","isPatch":false,"sender":{"key":"ae@op5.se","avatar":"https://gravatar.com/avatar/426e89595c75a8f5252dd0c989e5fabe5bcac616e68557427ad9aef6b0ca342a?d=mp&s=160"},"body":"Andy Parkins wrote:\n> On Thursday 2006 November 30 18:57, Andreas Ericsson wrote:\n> \n> (agree with everything in your mail)\n> \n>> The only problem I'm seeing atm is that the supermodule somehow has to\n>> mark whatever commits it's using from the submodule inside the submodule\n>> repo so that they effectively become un-prunable, otherwise the\n>> supermodule may some day find itself with a history that it can't restore.\n> \n> What about submodule/.git/refs/supermodule/commit12345678, where \"12345678\" is \n> the hash of the supermodule commit? This gives a convenient route in the \n> submodule to which commit contains that commit from the submodule; but \n> doesn't write anything into the submodule repository itself. It's just a tag \n> with a different intent.\n> \n\nTrue, but this makes one repo of the submodule special. Let's say you \nhave this layout\n\nmozilla/.git\nmozilla/openssl/.git\nmozilla/xlat/.git\n\nNow, we can be reasonably sure that the 'xlat' repo is something the \nmozilla core team can push to, or at least we can consider the core repo \nowners an official \"vendor\" of tags for the submodule repo. I'm fairly \ncertain openssl authors won't be too happy with allowing the thousands \nof projects using its code to push tags to its official repo though.\n\nNow that I think about it more, I realize this is completely irrelevant \nas the ui can create the tags in the submodule with info only from the \nthe supermodule, which means the submodule repo will only be special if \nit's connected to the supermodule. We just need a command for creating \nthose tags in the submodule repo so people who use the same submodule \ncode for several projects can use the alternates mechanism effectively.\n\n-- \nAndreas Ericsson andreas.ericsson@op5.se\nOP5 AB www.op5.se\n"},{"id":"296389","messageId":"20061201095751.GK18810@admingilde.org","threadId":"43106","inReplyTo":"200612010919.06030.andyparkins@gmail.com","subject":"Re: [RFC] Submodules in GIT","fromName":"Martin Waitz","fromEmail":"tali@admingilde.org","sentAt":"2006-12-01T09:57:51Z","receivedAt":"2006-12-01T09:57:51Z","isPatch":false,"sender":{"key":"tali@admingilde.org","avatar":"https://gravatar.com/avatar/3f89b03eee362187effabe257898735b475673a12265c398ea9161259ae91553?d=mp&s=160"},"body":"hoi :)\n\nOn Fri, Dec 01, 2006 at 09:19:04AM +0000, Andy Parkins wrote:\n> Let's imagine a supermodule repository, and guess at it in more detail (I'll \n> abbreviate some of the less interesting output):\n> \n> $ git-cat-file -p HEAD\n> tree fb02e78085ecf2f29045603df858b5362e5bf8a4\n> parent 4f2dba685507e4a8e07dac298c4024feaec6bd7d\n> author Andy Parkins\n> committer Andy Parkins \n> $ git-cat-file -p fb02e78085ecf2f29045603df858b5362e5bf8a4\n> 100644 blob 46bd4e284a57e2faa539e7b72d62a38867075af5 Makefile\n> 040000 tree 49ea01373a986a3db44d66702714aa75059ffa2c doc\n> 040000 subm d0a877464dc0198667a3e27ed3af8448ddacf947 libxcb\n\nat the moment, it is:\n 140000 commit ccddf1d4b0cf7fd3a699d8b33cf5bc4c5c4435b7 libxcb\n\n> The \"subm\" type is our new ODB object that's going to store whatever we will \n> need to access the submodule. \"libxcb\" has already told us where this \n> submodule is in the supermodule tree.\n> \n> $ git-cat-file -p d0a877464dc0198667a3e27ed3af8448ddacf947\n> submodulecommithash ccddf1d4b0cf7fd3a699d8b33cf5bc4c5c4435b7\n> submoduleurlhint git://anongit.freedesktop.org/git/xcb/libxcb\n\nSo why do you need the url hint committed to the supermodule?\nWe don't store remote information in the object database, too.\nRemember: this is still a distributed project, there is no one URL to\nany submodule.\n\n> I say:\n> submodulecommithash points at a commit /in the submodule/\n\nBut unluckily, this does not work.\nYou really have to be able to traverse the entire commit chain\nfrom the supermodule into all submodules.\n\n-- \nMartin Waitz\n"},{"id":"297199","messageId":"200612011019.40548.andyparkins@gmail.com","threadId":"43106","inReplyTo":"20061201093244.GP12463MdfPADPa@greensroom.kotnet.org","subject":"Re: [RFC] Submodules in GIT","fromName":"Andy Parkins","fromEmail":"andyparkins@gmail.com","sentAt":"2006-12-01T10:19:39Z","receivedAt":"2006-12-01T10:19:39Z","isPatch":false,"sender":{"key":"andyparkins@gmail.com","avatar":null},"body":"On Friday 2006 December 01 09:32, Sven Verdoolaege wrote:\n\n> This is heresy. Any object referenced in a tree should be in the repo\n> (possibly via alternates).\n\nThe \"submodule\" object would be in the local repository. That would refer to \nanother object, and is merely part of the submodule object. Just as \nthe \"Author\" and \"Commiter\" fields are part of the commit object but aren't \nactual objects in the tree.\n\nAndy\n\n-- \nDr Andy Parkins, M Eng (hons), MIEE\n"},{"id":"294148","messageId":"200612011029.28059.andyparkins@gmail.com","threadId":"43106","inReplyTo":"20061201095751.GK18810@admingilde.org","subject":"Re: [RFC] Submodules in GIT","fromName":"Andy Parkins","fromEmail":"andyparkins@gmail.com","sentAt":"2006-12-01T10:29:26Z","receivedAt":"2006-12-01T10:29:26Z","isPatch":false,"sender":{"key":"andyparkins@gmail.com","avatar":null},"body":"On Friday 2006 December 01 09:57, Martin Waitz wrote:\n\n> So why do you need the url hint committed to the supermodule?\n> We don't store remote information in the object database, too.\n\nThat's why it was a hint, probably configured when you first create the \nsubmodule connection.\n\n> Remember: this is still a distributed project, there is no one URL to\n> any submodule.\n\nThat point applies equally to your \"tracking a submodule branch\" point, except \nmine is only a URL hint, to help when first cloning that supermodule. In \ntruth, the clone will be perfectly able to get the submodule objects from the \nupstream supermodule, maintaining the distributed nature easily.\n\n> > I say:\n> > submodulecommithash points at a commit /in the submodule/\n>\n> But unluckily, this does not work.\n\nEh? \"Not work\", we're talking about code that doesn't even exist, of course \nit doesn't \"work\". Do you mean \"doesn't work if we're using my \nimplementation of submodules\"? Well that hardly seems like a fair attack.\n\n> You really have to be able to traverse the entire commit chain\n> from the supermodule into all submodules.\n\nYou can: when you hit a submodule tree object you set GIT_DIR to that \nsubmodule and continue. If you don't do it like that then you have stored \nsubmodule trees in the supermodule and it's no longer a separate repository. \nWhy you'd want to - I have no idea. What purpose would you have for \ntraversing the commit chain into the submodules? The commit in the submodule \nis just a note of where that submodule was during the supermodule commit in \nquestion.\n\nI notice though that you avoided my question: what does YOUR submodule object \ncontain? I really do want to know, as there is obviously a fundamental \ndifference in what I think a submodule does and what you (and maybe everybody \nelse) thinks a submodule does. I'm perfectly willing to accept I'm wrong, \nbut not without understanding how your method is going to work.\n\n\nAndy\n-- \nDr Andy Parkins, M Eng (hons), MIEE\n"},{"id":"296380","messageId":"200612011038.29193.andyparkins@gmail.com","threadId":"43106","inReplyTo":"456FF6D1.4040500@op5.se","subject":"Re: [RFC] Submodules in GIT","fromName":"Andy Parkins","fromEmail":"andyparkins@gmail.com","sentAt":"2006-12-01T10:38:27Z","receivedAt":"2006-12-01T10:38:27Z","isPatch":false,"sender":{"key":"andyparkins@gmail.com","avatar":null},"body":"On Friday 2006 December 01 09:33, Andreas Ericsson wrote:\n\n> True, but this makes one repo of the submodule special. Let's say you\n> have this layout\n\nIn a way, but it's information that doesn't need to be transmitted.\n\n> mozilla/.git\n> mozilla/openssl/.git\n> mozilla/xlat/.git\n>\n> Now, we can be reasonably sure that the 'xlat' repo is something the\n> mozilla core team can push to, or at least we can consider the core repo\n> owners an official \"vendor\" of tags for the submodule repo. I'm fairly\n> certain openssl authors won't be too happy with allowing the thousands\n> of projects using its code to push tags to its official repo though.\n\nNo need, when cloning a supermodule, it will make those special tags \nautomatically in the submodule repo. They are only there to prevent prune \nfrom destroying those referenced commits after all. If the submodule is \ncloned directly, they aren't needed anyway, and those objects won't be part \nof the dependency chain so wouldn't be downloaded.\n\n> Now that I think about it more, I realize this is completely irrelevant\n> as the ui can create the tags in the submodule with info only from the\n> the supermodule, which means the submodule repo will only be special if\n> it's connected to the supermodule. We just need a command for creating\n> those tags in the submodule repo so people who use the same submodule\n> code for several projects can use the alternates mechanism effectively.\n\nIs that even necessary? git-clone of a supermodule will make those tags \nautomatically. If a submodule was alternative-cloned into a different \nsupermodule, well then THAT supermodule would make the right tags for itself. \nAh, I think I see what you mean now though, a method would be needed for \ncreating those tags if we managed to manually get a submodule repository in \nto the supermodule - then supermodule-clone wouldn't have run. Perhaps they \ncould be checked for at commit time and recreated then?\n\n\nAndy\n-- \nDr Andy Parkins, M Eng (hons), MIEE\n"},{"id":"293852","messageId":"20061201104256.GQ12463MdfPADPa@greensroom.kotnet.org","threadId":"43106","inReplyTo":"200612011029.28059.andyparkins@gmail.com","subject":"Re: [RFC] Submodules in GIT","fromName":"Sven Verdoolaege","fromEmail":"skimo@kotnet.org","sentAt":"2006-12-01T10:42:56Z","receivedAt":"2006-12-01T10:42:56Z","isPatch":false,"sender":{"key":"skimo@kotnet.org","avatar":null},"body":"On Fri, Dec 01, 2006 at 10:29:26AM +0000, Andy Parkins wrote:\n> I notice though that you avoided my question: what does YOUR submodule object \n> contain?\n\nHe showed it to you in the example. The \"submodule object\" is the COMMIT\nof the submodule itself.\n\n"},{"id":"294034","messageId":"200612011102.17079.andyparkins@gmail.com","threadId":"43106","inReplyTo":"20061201104256.GQ12463MdfPADPa@greensroom.kotnet.org","subject":"Re: [RFC] Submodules in GIT","fromName":"Andy Parkins","fromEmail":"andyparkins@gmail.com","sentAt":"2006-12-01T11:02:15Z","receivedAt":"2006-12-01T11:02:15Z","isPatch":false,"sender":{"key":"andyparkins@gmail.com","avatar":null},"body":"On Friday 2006 December 01 10:42, Sven Verdoolaege wrote:\n\n> He showed it to you in the example. The \"submodule object\" is the COMMIT\n> of the submodule itself.\n\nThat's no different from mine. I need more detail than that.\n\nIs that commit in the submodule or the supermodule? If it's in the submodule \nthen we're talking about the same thing, as that's all I want. If it's in \nthe supermodule then I want to know what the tree object that that commit \npoints to contains. I also want to know how we tell the difference between a \ncommit-in-supermodule and a \ncommit-in-supermodule-which-is-actually-in-submodule.\n\n\nAndy\n-- \nDr Andy Parkins, M Eng (hons), MIEE\n"},{"id":"294348","messageId":"20061201111027.GR12463MdfPADPa@greensroom.kotnet.org","threadId":"43106","inReplyTo":"200612011102.17079.andyparkins@gmail.com","subject":"Re: [RFC] Submodules in GIT","fromName":"Sven Verdoolaege","fromEmail":"skimo@kotnet.org","sentAt":"2006-12-01T11:10:27Z","receivedAt":"2006-12-01T11:10:27Z","isPatch":false,"sender":{"key":"skimo@kotnet.org","avatar":null},"body":"On Fri, Dec 01, 2006 at 11:02:15AM +0000, Andy Parkins wrote:\n> On Friday 2006 December 01 10:42, Sven Verdoolaege wrote:\n> \n> > He showed it to you in the example. The \"submodule object\" is the COMMIT\n> > of the submodule itself.\n> \n> That's no different from mine. I need more detail than that.\n\nYou were proposing to create an extra object containing some random value\nthat is disconnected from the repo.\n\n> Is that commit in the submodule or the supermodule?\n\nIt's in BOTH. That's why it's a *sub*module.\n\nSomeone else can try to expain it you.\n\n"},{"id":"294481","messageId":"20061201113103.GM18810@admingilde.org","threadId":"43106","inReplyTo":"200612011029.28059.andyparkins@gmail.com","subject":"Re: [RFC] Submodules in GIT","fromName":"Martin Waitz","fromEmail":"tali@admingilde.org","sentAt":"2006-12-01T11:31:03Z","receivedAt":"2006-12-01T11:31:03Z","isPatch":false,"sender":{"key":"tali@admingilde.org","avatar":"https://gravatar.com/avatar/3f89b03eee362187effabe257898735b475673a12265c398ea9161259ae91553?d=mp&s=160"},"body":"hoi :)\n\nOn Fri, Dec 01, 2006 at 10:29:26AM +0000, Andy Parkins wrote:\n> On Friday 2006 December 01 09:57, Martin Waitz wrote:\n> \n> > So why do you need the url hint committed to the supermodule?\n> > We don't store remote information in the object database, too.\n> \n> That's why it was a hint, probably configured when you first create the \n> submodule connection.\n> \n> In truth, the clone will be perfectly able to get the submodule\n> objects from the upstream supermodule, maintaining the distributed\n> nature easily.\n\nthat's exactly the reason why the hint is not needed.\nAlthogh you need to have one common project object database, storing the\nobjects of all modules.\n\n> > > I say:\n> > > submodulecommithash points at a commit /in the submodule/\n> >\n> > But unluckily, this does not work.\n> \n> Eh? \"Not work\", we're talking about code that doesn't even exist, of\n> course it doesn't \"work\". Do you mean \"doesn't work if we're using\n> my implementation of submodules\"? Well that hardly seems like a fair\n> attack.\n\nWell, at first I started exactly as you described: only store the\nsubmodule commit sha1 in the parent somewhere, but don't traverse it.\nSo this is a fair attack: your implementation already exists in\nhttp://git.admingilde.org/tali/git.git/module ;-)\n(ok, yes, it really is different to what you described as I stored the\nsha1 differently, but I really learned that it is important to be able\nto traverse the entire commit chain, from the root of the project to the\ndeepest submodule.)\n\n\n> > You really have to be able to traverse the entire commit chain\n> > from the supermodule into all submodules.\n> \n> You can: when you hit a submodule tree object you set GIT_DIR to that\n> submodule and continue. If you don't do it like that then you have\n> stored submodule trees in the supermodule and it's no longer a\n> separate repository.\n\nWell, a submodule repository _is_ special in some ways:\nfsck and prune have to take the references from the supermodule into\naccount. In this sense it is _not_ separate from the supermodule.\n\nI think that is important for the submodule repository to be independent\nin other ways than its object database: you should be able to exchange\ncommits with other repositories (be they stand-alone or a submodule in\nanother supermodule). You should be able to use log/diff/blame/whatever\ninside the submodule.\n\nAll this does not need an object database of its own.\nSo I chose to do it the easy way and use one object database for the\nentire project - and disallow git-prune in a submodule.\nThere may be other/better ways to do this, but you have to be able\nto access all objects which belong the project inside the toplevel\nproject repository.\n\n> Why you'd want to - I have no idea. What\n> purpose would you have for traversing the commit chain into the\n> submodules? The commit in the submodule is just a note of where that\n> submodule was during the supermodule commit in question.\n\nThings get much simpler if you have one big graph of objects.\n\nclone and especially fetch/pull naturally work at once.\nYou can ask for all objects inside the whole project which are needed to\nbe transferred between project version A and B, including all submodules.\n\nYou can even have one bare repository for the whole project.\n\n> I notice though that you avoided my question: what does YOUR submodule\n> object contain? I really do want to know, as there is obviously a\n> fundamental difference in what I think a submodule does and what you\n> (and maybe everybody else) thinks a submodule does.\n\nIt really only stores the commit of the submodule directly.\nSo there is no new submodule object type. The parent has a direct link\nto the submodule commit in his tree object and in its index. In order\nto separate them from normal files or normal subdirectories, they get a\nspecial mode: they are represented as socket.\n\n-- \nMartin Waitz\n"},{"id":"298608","messageId":"ekp4l0$idb$1@sea.gmane.org","threadId":"43106","inReplyTo":"20061201111027.GR12463MdfPADPa@greensroom.kotnet.org","subject":"Re: [RFC] Submodules in GIT","fromName":"sf","fromEmail":"sf@b-i-t.de","sentAt":"2006-12-01T11:45:31Z","receivedAt":"2006-12-01T11:45:31Z","isPatch":false,"sender":{"key":"sf@b-i-t.de","avatar":null},"body":"Sven Verdoolaege wrote:\n> On Fri, Dec 01, 2006 at 11:02:15AM +0000, Andy Parkins wrote:\n>> On Friday 2006 December 01 10:42, Sven Verdoolaege wrote:\n>> \n>> > He showed it to you in the example. The \"submodule object\" is the COMMIT\n>> > of the submodule itself.\n>> \n>> That's no different from mine. I need more detail than that.\n> \n> You were proposing to create an extra object containing some random value\n> that is disconnected from the repo.\n> \n>> Is that commit in the submodule or the supermodule?\n> \n> It's in BOTH. That's why it's a *sub*module.\n\nI would say it is only in the supermodule because that is the branch you \nare working on. If you are working on the submodule in an independent \nbranch then you can pull from the submodule commit. But you do not want \nto pull the supermodule commit itself but only the commit in path libxcb \n(see my proposed syntax).\n\nRegards\n\nStephan\n"},{"id":"297755","messageId":"20061201114607.GN18810@admingilde.org","threadId":"43106","inReplyTo":"200612011102.17079.andyparkins@gmail.com","subject":"Re: [RFC] Submodules in GIT","fromName":"Martin Waitz","fromEmail":"tali@admingilde.org","sentAt":"2006-12-01T11:46:07Z","receivedAt":"2006-12-01T11:46:07Z","isPatch":false,"sender":{"key":"tali@admingilde.org","avatar":"https://gravatar.com/avatar/3f89b03eee362187effabe257898735b475673a12265c398ea9161259ae91553?d=mp&s=160"},"body":"hoi :)\n\nOn Fri, Dec 01, 2006 at 11:02:15AM +0000, Andy Parkins wrote:\n> On Friday 2006 December 01 10:42, Sven Verdoolaege wrote:\n> \n> > He showed it to you in the example. The \"submodule object\" is the COMMIT\n> > of the submodule itself.\n> \n> That's no different from mine.\n\nWell, there simply is no proxy object inbetween.\n\n> Is that commit in the submodule or the supermodule?\n\nWell, logically that commit belongs to the submodule and is referenced\nby the tree in the supermodule.\nPhyisically it is stored in the projects object database which is\nshared between the supermodule and all submodules (at least in my\nimplementation).\n\n> I also want to know how we tell the difference between a\n> commit-in-supermodule and a\n> commit-in-supermodule-which-is-actually-in-submodule.\n\nThere is no difference.\n\n-- \nMartin Waitz\n"},{"id":"295799","messageId":"200612011212.35656.andyparkins@gmail.com","threadId":"43106","inReplyTo":"20061201111027.GR12463MdfPADPa@greensroom.kotnet.org","subject":"Re: [RFC] Submodules in GIT","fromName":"Andy Parkins","fromEmail":"andyparkins@gmail.com","sentAt":"2006-12-01T12:12:34Z","receivedAt":"2006-12-01T12:12:34Z","isPatch":false,"sender":{"key":"andyparkins@gmail.com","avatar":null},"body":"On Friday 2006 December 01 11:10, Sven Verdoolaege wrote:\n\n> You were proposing to create an extra object containing some random value\n> that is disconnected from the repo.\n\nRight, I think I've finally understood what Martin (and you) are proposing. \nYou want every commit in the submodule to be propagated up to the supermodule \nas well. Okay.\n\nI don't think it's right, but at least I understand.\n\nIt seems wrong because it's making commits in the supermodule that aren't \ncommits to do with that project. In my libxcb example; why should every \nproject use libxcb in have to store the entire history of libxcb? When \nexamining the supermodule history, I won't care about how libxcb got to the \nstate its in, and it's just noise in the supermodule history. What if I use \n10 submodules, the supermodule history won't show you anything useful - it's \njust unrelated submodule commits.\n\nIt gets worse, this is why I was asking for more detail: this commit that \nyou're storing in the supermodule. It's the same commit as is in the \nsubmodule? What would the parent commit of that commit be? It has to be the \nsame in both, because the commit-hash forces it to be.\n\nThe only possibility would be that it's NOT the same hash in both, because the \nparents in the supermodule are inapplicable to the submodule, and the parent \nin the submodule is independent from the supermodule. That means you have to \nstore two commits: one for the submodule commit and one for the supermodule \ncommit. So what are you going to write in the supermodule commit? Answer: a \nsubmodule commit hash - exactly as I said.\n\n> > Is that commit in the submodule or the supermodule?\n>\n> It's in BOTH. That's why it's a *sub*module.\n\nIf it's in BOTH then the supermodule is a normal git repository. You aren't \ntracking the submodule, you're just including it en masse. Using semantics \nto justify a position isn't a very strong argument, calling it a \"sub\" module \nis just an easy bit of naming for us to hang the discussion on, it isn't \nnecessarily a mathematical subset and superset.\n\n\nAndy\n\n-- \nDr Andy Parkins, M Eng (hons), MIEE\n"},{"id":"298165","messageId":"200612011216.04555.andyparkins@gmail.com","threadId":"43106","inReplyTo":"20061201114607.GN18810@admingilde.org","subject":"Re: [RFC] Submodules in GIT","fromName":"Andy Parkins","fromEmail":"andyparkins@gmail.com","sentAt":"2006-12-01T12:16:00Z","receivedAt":"2006-12-01T12:16:00Z","isPatch":false,"sender":{"key":"andyparkins@gmail.com","avatar":null},"body":"On Friday 2006 December 01 11:46, Martin Waitz wrote:\n\n> > That's no different from mine.\n>\n> Well, there simply is no proxy object inbetween.\n\nThat's fine, I was only using the proxy object to allow additional information \ninto the submodule object. Actually, I think it would always be better to \nuse a proxy object otherwise you have an error in the tree object, because it \nwill refer to an object that does not exist. The proxy object is allowed to \nrefer to objects that don't exist because it's not a tree object.\n\n> > Is that commit in the submodule or the supermodule?\n>\n> Well, logically that commit belongs to the submodule and is referenced\n> by the tree in the supermodule.\n> Phyisically it is stored in the projects object database which is\n> shared between the supermodule and all submodules (at least in my\n> implementation).\n\nHmmm, \"shared\"? It must still be in the submodule physically though, and \npresumably the supermodule uses alternatives to get access to it? Otherwise \nthe submodule will be impossible to separate from the supermodule.\n\n> > I also want to know how we tell the difference between a\n> > commit-in-supermodule and a\n> > commit-in-supermodule-which-is-actually-in-submodule.\n>\n> There is no difference.\n\nOkay. I think I'm still a bit lost then. I suppose I'll wait for your \npatches to understand.\n\n\nAndy\n-- \nDr Andy Parkins, M Eng (hons), MIEE\n"},{"id":"298150","messageId":"200612011220.43813.andyparkins@gmail.com","threadId":"43106","inReplyTo":"20061201113103.GM18810@admingilde.org","subject":"Re: [RFC] Submodules in GIT","fromName":"Andy Parkins","fromEmail":"andyparkins@gmail.com","sentAt":"2006-12-01T12:20:42Z","receivedAt":"2006-12-01T12:20:42Z","isPatch":false,"sender":{"key":"andyparkins@gmail.com","avatar":null},"body":"On Friday 2006 December 01 11:31, Martin Waitz wrote:\n\n> It really only stores the commit of the submodule directly.\n> So there is no new submodule object type. The parent has a direct link\n> to the submodule commit in his tree object and in its index. In order\n> to separate them from normal files or normal subdirectories, they get a\n> special mode: they are represented as socket.\n\nOkay. I think I've got it now. I'm not convinced that the way you've chosen \nis the correct way, primarily because the separation between supermodule and \nsubmodule is not strong. Regardless, as you're doing it, you get to pick :-) \nIs there a public repository I can look at to see what you've done? I'm \ninterested in the sort of plumbing changes needed to make something like this \nwork.\n\n\nAndy\n\n-- \nDr Andy Parkins, M Eng (hons), MIEE\n"},{"id":"294357","messageId":"20061201122854.GR18810@admingilde.org","threadId":"43106","inReplyTo":"200612011212.35656.andyparkins@gmail.com","subject":"Re: [RFC] Submodules in GIT","fromName":"Martin Waitz","fromEmail":"tali@admingilde.org","sentAt":"2006-12-01T12:28:54Z","receivedAt":"2006-12-01T12:28:54Z","isPatch":false,"sender":{"key":"tali@admingilde.org","avatar":"https://gravatar.com/avatar/3f89b03eee362187effabe257898735b475673a12265c398ea9161259ae91553?d=mp&s=160"},"body":"hoi :)\n\nOn Fri, Dec 01, 2006 at 12:12:34PM +0000, Andy Parkins wrote:\n> On Friday 2006 December 01 11:10, Sven Verdoolaege wrote:\n> \n> > You were proposing to create an extra object containing some random value\n> > that is disconnected from the repo.\n> \n> Right, I think I've finally understood what Martin (and you) are\n> proposing. You want every commit in the submodule to be propagated up\n> to the supermodule as well. Okay.\n> \n> I don't think it's right, but at least I understand.\n\nPlease note that the submodule commits are not part of the supermodule\ncommit chain, they are part of the supermodule _tree_.\n\n> It seems wrong because it's making commits in the supermodule that aren't \n> commits to do with that project.\n\nOf course they are part of your project, just like all the tree and blob\nobjects, too.\n\n> In my libxcb example; why should every project use libxcb in have to\n> store the entire history of libxcb?\n\nBecause you want to be able to use the submodule as a repository of its\nown, too. Be able to look at its history if you want to.\nBe able to merge with new versions of the submodule.\nThis is what distiguishes a submodule from a pure file-based import of\nanother project.\n\n> When examining the supermodule history, I won't care about how libxcb\n> got to the state its in, and it's just noise in the supermodule\n> history. What if I use 10 submodules, the supermodule history won't\n> show you anything useful - it's just unrelated submodule commits.\n\nAgain: the submodules are part of your supermodule _tree_, not it's\ncommit chain. So you won't see the submodule commits when you invoke\ngit-log in the supermodule.\n\n> It gets worse, this is why I was asking for more detail: this commit\n> that you're storing in the supermodule. It's the same commit as is in\n> the submodule?\n\nIt is _the_ commit from the submodule, yes.\n\n> What would the parent commit of that commit be? It has to be the same\n> in both, because the commit-hash forces it to be.\n\nIt is the commit of the submodule, so its parents point to the submodule\nhistory.\n\n> > > Is that commit in the submodule or the supermodule?\n> >\n> > It's in BOTH. That's why it's a *sub*module.\n> \n> If it's in BOTH then the supermodule is a normal git repository. You aren't \n> tracking the submodule, you're just including it en masse.\n\nThe submodule is part of the entire project, so yes, it is included.\nAnd the supermodule tracks submodule development by storing references\nto the submodule history that was used at that time.\n\n\nLets try to paint a little diagram:\n\n\nbelongint to:\n/--------- supermodule -------\\ /---- submodule -------\\\n\ncommit -> tree +-> blob\n | +-> tree -> ...\n | +-----------------> commit -> tree -> ...\n v |\ncommit -> tree +-> ... v\n | +-----------------> commit -> ...\n | |\n | v\n | commit -> ...\n v |\ncommit -> tree +-> ... v\n +-----------------> commit\n\n\nBoth have their independent history, but they are linked as some\nsubmodule versions are part of the supermodule tree.\n\n-- \nMartin Waitz\n"},{"id":"294135","messageId":"20061201123447.GS18810@admingilde.org","threadId":"43106","inReplyTo":"200612011216.04555.andyparkins@gmail.com","subject":"Re: [RFC] Submodules in GIT","fromName":"Martin Waitz","fromEmail":"tali@admingilde.org","sentAt":"2006-12-01T12:34:47Z","receivedAt":"2006-12-01T12:34:47Z","isPatch":false,"sender":{"key":"tali@admingilde.org","avatar":"https://gravatar.com/avatar/3f89b03eee362187effabe257898735b475673a12265c398ea9161259ae91553?d=mp&s=160"},"body":"hoi :)\n\nOn Fri, Dec 01, 2006 at 12:16:00PM +0000, Andy Parkins wrote:\n> That's fine, I was only using the proxy object to allow additional\n> information into the submodule object. Actually, I think it would\n> always be better to use a proxy object otherwise you have an error in\n> the tree object, because it will refer to an object that does not\n> exist. The proxy object is allowed to refer to objects that don't\n> exist because it's not a tree object.\n\nIt is exactly the aim of my implementation to not have any reference to\nsomething that is not accessible in the supermodule repository.\n\n> > > Is that commit in the submodule or the supermodule?\n> >\n> > Well, logically that commit belongs to the submodule and is referenced\n> > by the tree in the supermodule.\n> > Phyisically it is stored in the projects object database which is\n> > shared between the supermodule and all submodules (at least in my\n> > implementation).\n> \n> Hmmm, \"shared\"? It must still be in the submodule physically though,\n> and presumably the supermodule uses alternatives to get access to it?\n> Otherwise the submodule will be impossible to separate from the\n> supermodule.\n\nYes, you can't separate it my just moving it out of the supermodule,\nbut you can always clone the submodule alone.\n\n> Okay. I think I'm still a bit lost then. I suppose I'll wait for your\n> patches to understand.\n\nhave a look at http://git.admingilde.org/tali/git.git/module2.\nIf you want to try it out, have a look at t/t7500-submodule.sh on how to\ncreate submodules.\n\n-- \nMartin Waitz\n"},{"id":"296930","messageId":"20061201123739.GT18810@admingilde.org","threadId":"43106","inReplyTo":"200612011220.43813.andyparkins@gmail.com","subject":"Re: [RFC] Submodules in GIT","fromName":"Martin Waitz","fromEmail":"tali@admingilde.org","sentAt":"2006-12-01T12:37:39Z","receivedAt":"2006-12-01T12:37:39Z","isPatch":false,"sender":{"key":"tali@admingilde.org","avatar":"https://gravatar.com/avatar/3f89b03eee362187effabe257898735b475673a12265c398ea9161259ae91553?d=mp&s=160"},"body":"hoi :)\n\nOn Fri, Dec 01, 2006 at 12:20:42PM +0000, Andy Parkins wrote:\n> Is there a public repository I can look at to see what you've done?\n> I'm interested in the sort of plumbing changes needed to make\n> something like this work.\n\nlink is in the mail that started this thread ;-).\n\n-- \nMartin Waitz\n"},{"id":"294226","messageId":"200612011400.00262.andyparkins@gmail.com","threadId":"43106","inReplyTo":"20061201123447.GS18810@admingilde.org","subject":"Re: [RFC] Submodules in GIT","fromName":"Andy Parkins","fromEmail":"andyparkins@gmail.com","sentAt":"2006-12-01T13:59:58Z","receivedAt":"2006-12-01T13:59:58Z","isPatch":false,"sender":{"key":"andyparkins@gmail.com","avatar":null},"body":"On Friday 2006 December 01 12:34, Martin Waitz wrote:\n\n> It is exactly the aim of my implementation to not have any reference to\n> something that is not accessible in the supermodule repository.\n\nOkay - I think you've put me right in another reply on this point - the \nsubmodule commit is in the supermodule; that was the part I hadn't got.\n\n> Yes, you can't separate it my just moving it out of the supermodule,\n> but you can always clone the submodule alone.\n\nAh - now that clarifies things a lot. The fact that you can't separate it by \nmoving it implies lots of things that take away many of my earlier worries.\n\n> have a look at http://git.admingilde.org/tali/git.git/module2.\n> If you want to try it out, have a look at t/t7500-submodule.sh on how to\n> create submodules.\n\nThanks. I will look hard at this :-) My apologies for bothering you so much \nwith all these questions. I just got a bit interested in it all :-)\n\n\nAndy\n\n-- \nDr Andy Parkins, M Eng (hons), MIEE\n"},{"id":"295027","messageId":"20061201140758.GX18810@admingilde.org","threadId":"43106","inReplyTo":"200612011400.00262.andyparkins@gmail.com","subject":"Re: [RFC] Submodules in GIT","fromName":"Martin Waitz","fromEmail":"tali@admingilde.org","sentAt":"2006-12-01T14:07:58Z","receivedAt":"2006-12-01T14:07:58Z","isPatch":false,"sender":{"key":"tali@admingilde.org","avatar":"https://gravatar.com/avatar/3f89b03eee362187effabe257898735b475673a12265c398ea9161259ae91553?d=mp&s=160"},"body":"hoi :)\n\nOn Fri, Dec 01, 2006 at 01:59:58PM +0000, Andy Parkins wrote:\n> My apologies for bothering you so much with all these questions.\n\nYou were not bothering me.\nThose were really interesting and valid questions.\nIn fact, it was a long way for me to come to the implementation I have\nnow. And I really did ask many of those questions to me, too.\n\nI should really write a nice paper about all of that, I think.\n\n> I just got a bit interested in it all :-)\n\nGood :-)\n\n-- \nMartin Waitz\n"},{"id":"298226","messageId":"200612011411.20113.andyparkins@gmail.com","threadId":"43106","inReplyTo":"20061201122854.GR18810@admingilde.org","subject":"Re: [RFC] Submodules in GIT","fromName":"Andy Parkins","fromEmail":"andyparkins@gmail.com","sentAt":"2006-12-01T14:11:19Z","receivedAt":"2006-12-01T14:11:19Z","isPatch":false,"sender":{"key":"andyparkins@gmail.com","avatar":null},"body":"On Friday 2006 December 01 12:28, Martin Waitz wrote:\n\n> > It seems wrong because it's making commits in the supermodule that aren't\n> > commits to do with that project.\n>\n> Of course they are part of your project, just like all the tree and blob\n> objects, too.\n\nI wouldn't go as far as that; just because I use libxcb doesn't mean I want \nit's history merged with mine. However, I think my worries are unfounded, \nyour comment about being able to independently clone the libxcb tree helped \nme there. If I've understood; while the objects themselves are stored in the \nsupermodule ODB, they are still independent. In fact, they're only in the \nsupermodule tree because it's most convenient to keep them there; it sounds \nlike it's very easy to strip them out again.\n\n> It is the commit of the submodule, so its parents point to the submodule\n> history.\n\nAgain, if I'm understanding, it's a bit like when you have an additional root \nin a normal git repository, for example:\n \n * -- * -- * -- * (project1)\n \\\n * -- * -- * (project1/stable)\n \n * -- * -- * -- * (project2)\n\nThen to make project2 a submodule of project1, one of the project1 trees \nsimply refers to a commit in project2.\n\nI think my original idea for how this works was correct with one minor flaw, \nand from that flaw all the other concerns flowed. I imagined that there were \ntwo object databases - one for the supermodule and one for the submodule. \nThe fault was that there aren't two ODBs there are two roots. Which of \ncourse is a far easier way to blend to repositories. Apart from that, I \nthink I'm entirely in sync, and it was merely my wanting to put each of these \nroots in their own repository that caused all the confusion.\n\n\n\n\nAndy\n-- \nDr Andy Parkins, M Eng (hons), MIEE\n"},{"id":"295650","messageId":"20061201151223.GB18810@admingilde.org","threadId":"43106","inReplyTo":"200612011411.20113.andyparkins@gmail.com","subject":"Re: [RFC] Submodules in GIT","fromName":"Martin Waitz","fromEmail":"tali@admingilde.org","sentAt":"2006-12-01T15:12:23Z","receivedAt":"2006-12-01T15:12:23Z","isPatch":false,"sender":{"key":"tali@admingilde.org","avatar":"https://gravatar.com/avatar/3f89b03eee362187effabe257898735b475673a12265c398ea9161259ae91553?d=mp&s=160"},"body":"hoi :)\n\nOn Fri, Dec 01, 2006 at 02:11:19PM +0000, Andy Parkins wrote:\n> If I've understood; while the objects themselves are stored in the\n> supermodule ODB, they are still independent. In fact, they're only in\n> the supermodule tree because it's most convenient to keep them there;\n> it sounds like it's very easy to strip them out again.\n\nYes.\n\n> Again, if I'm understanding, it's a bit like when you have an\n> additional root in a normal git repository, for example:\n>\n> * -- * -- * -- * (project1)\n> \\\n> * -- * -- * (project1/stable)\n>\n> * -- * -- * -- * (project2)\n> \n> Then to make project2 a submodule of project1, one of the project1\n> trees simply refers to a commit in project2.\n\nExactly.\n\n\n-- \nMartin Waitz\n"},{"id":"294703","messageId":"eks58k$obn$1@sea.gmane.org","threadId":"43106","inReplyTo":"20061201123739.GT18810@admingilde.org","subject":"Re: [RFC] Submodules in GIT","fromName":"Jakub Narebski","fromEmail":"jnareb@gmail.com","sentAt":"2006-12-02T15:16:30Z","receivedAt":"2006-12-02T15:16:30Z","isPatch":false,"sender":{"key":"jnareb@gmail.com","avatar":"https://avatars.githubusercontent.com/u/2706?v=4"},"body":"Martin Waitz wrote:\n\n> hoi :)\n> \n> On Fri, Dec 01, 2006 at 12:20:42PM +0000, Andy Parkins wrote:\n>> Is there a public repository I can look at to see what you've done?\n>> I'm interested in the sort of plumbing changes needed to make\n>> something like this work.\n> \n> link is in the mail that started this thread ;-).\n\nAnd on GitWiki as well:\n http://git.or.cz/gitwiki/SubprojectSupport\n\n-- \nJakub Narebski\nWarsaw, Poland\nShadeHawk on #git\n\n"}]}