{"thread":{"id":"18748","subject":"Submodules can't work recursively because Git implements policy?","startedAt":"2009-04-06T13:42:31Z","lastAt":"2009-04-06T16:29:30Z","messageCount":5,"participants":["Klas Lindberg","Finn Arne Gangstad","Shawn O. Pearce"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"110578","messageId":"33f4f4d70904060642m25b2cff8nafed433eeabfb6c4@mail.gmail.com","threadId":"18748","inReplyTo":null,"subject":"Submodules can't work recursively because Git implements policy?","fromName":"Klas Lindberg","fromEmail":"klas.lindberg@gmail.com","sentAt":"2009-04-06T13:42:31Z","receivedAt":"2009-04-06T13:42:31Z","isPatch":false,"sender":{"key":"klas.lindberg@gmail.com","avatar":null},"body":"On Mon, Apr 6, 2009 at 3:16 PM, Finn Arne Gangstad wrote:\n\n> git submodule update just does \"git fetch\" and hopes that the required\n> commit appears. In practice this means that you (may) need to invent a\n> tag or a branch for all the submodules, otherwise they are not\n> fetchable.\n>\n> This bit us pretty hard when we tried to use submodules earlier, so we\n> gave up. Maybe some day...\n\nIt \"hopes\" to find them? This is actually my other reason for bringing\nthe whole SHA key fetching thing up. From what I can see, it is not\npossible to implement submodules sensibly without support for fetching\nSHA keys. I.e. I want fetch, checkout and every other command to\nrecurse as needed in the presence of submodules. This limitation\nforces me to implement a whole CM tool where none should be necessary.\n\nIt appears to me that the security concern (being able to hide commits\nby making them unreachable from a named reference) is actually a\npolicy decision and not a technical one. On what grounds does Git\ndecide for me how to handle security concerns? It just seems more\nimportant to be able to have recursive submodule behaviour than to\nprovide band aid for careless users.\n\nOut of curiosity: Is it really possible to change the value of an\nalready pushed tag? Can you only do the hiding trick with branches?\n\nBR / Klas\n"},{"id":"110581","messageId":"20090406135618.GA17793@pvv.org","threadId":"18748","inReplyTo":"33f4f4d70904060642m25b2cff8nafed433eeabfb6c4@mail.gmail.com","subject":"Re: Submodules can't work recursively because Git implements policy?","fromName":"Finn Arne Gangstad","fromEmail":"finnag@pvv.org","sentAt":"2009-04-06T13:56:18Z","receivedAt":"2009-04-06T13:56:18Z","isPatch":false,"sender":{"key":"finnag@pvv.org","avatar":"https://gravatar.com/avatar/b421ddd58c3f0f93aa473e17b98bb8d53c221fef741746bc8cb59fae4ec6d95e?d=mp&s=160"},"body":"On Mon, Apr 06, 2009 at 03:42:31PM +0200, Klas Lindberg wrote:\n> On Mon, Apr 6, 2009 at 3:16 PM, Finn Arne Gangstad wrote:\n> \n> > git submodule update just does \"git fetch\" and hopes that the required\n> > commit appears. In practice this means that you (may) need to invent a\n> > tag or a branch for all the submodules, otherwise they are not\n> > fetchable.\n> >\n> > This bit us pretty hard when we tried to use submodules earlier, so we\n> > gave up. Maybe some day...\n> \n> It \"hopes\" to find them?\n\nPerhaps \"hopes\" is a loaded word, \"expects\" at least. The code just\ndoes the equivalent of \"git fetch; git checkout || die .. \"\n\n> This is actually my other reason for bringing\n> the whole SHA key fetching thing up. From what I can see, it is not\n> possible to implement submodules sensibly without support for fetching\n> SHA keys. I.e. I want fetch, checkout and every other command to\n> recurse as needed in the presence of submodules. This limitation\n> forces me to implement a whole CM tool where none should be necessary.\n\nYes, I could not agree more. You may also end up writing some really\ncomplicated wrappers around git push to get things going (where do you\npush, for example). We made some interesting \"concept art\" around this\nlast year at $dayjob, but decided to drop it.\n\n> It appears to me that the security concern (being able to hide commits\n> by making them unreachable from a named reference) is actually a\n> policy decision and not a technical one. On what grounds does Git\n> decide for me how to handle security concerns? It just seems more\n> important to be able to have recursive submodule behaviour than to\n> provide band aid for careless users.\n\nMaybe the security concerns could be handled by adding some\nfunctionality to (quickly) get rid of unwanted commits?\n\n> Out of curiosity: Is it really possible to change the value of an\n> already pushed tag? Can you only do the hiding trick with branches?\n\nYes, but if you modify a tag, you get additional complications. In\nparticular, no one will ever try to refetch the tag, so everyone who\nhas already fetched it will have a permanently broken tag.\n\n- Finn Arne\n"},{"id":"110589","messageId":"33f4f4d70904060747h72019846gca18255bd71adc22@mail.gmail.com","threadId":"18748","inReplyTo":"20090406135618.GA17793@pvv.org","subject":"Re: Submodules can't work recursively because Git implements policy?","fromName":"Klas Lindberg","fromEmail":"klas.lindberg@gmail.com","sentAt":"2009-04-06T14:47:22Z","receivedAt":"2009-04-06T14:47:22Z","isPatch":false,"sender":{"key":"klas.lindberg@gmail.com","avatar":null},"body":"On Mon, Apr 6, 2009 at 3:56 PM, Finn Arne Gangstad wrote:\n\n> Yes, I could not agree more. You may also end up writing some really\n> complicated wrappers around git push to get things going (where do you\n> push, for example). We made some interesting \"concept art\" around this\n> last year at $dayjob, but decided to drop it.\n\nI don't see how pushing could work at all without recursion.\n\n> Maybe the security concerns could be handled by adding some\n> functionality to (quickly) get rid of unwanted commits?\n\nWhy not simply allow users with write permissions to \"pop\" revisions\nfrom the top of the history DAG in a way that actually really deletes\nthe them? Or at least moves those commits to a separate, locked down\nDAG that cannot be read by people without write permissions?\n\nBut anyway: If I implement support for fetching SHA keys and full\nrecursive behaviour in the presence of submodules; would my patches\nautomatically be rejected because of the rationale for the current\nbehaviour?\n\n/Klas\n"},{"id":"110591","messageId":"20090406145140.GG23604@spearce.org","threadId":"18748","inReplyTo":"33f4f4d70904060747h72019846gca18255bd71adc22@mail.gmail.com","subject":"Re: Submodules can't work recursively because Git implements policy?","fromName":"Shawn O. Pearce","fromEmail":"spearce@spearce.org","sentAt":"2009-04-06T14:51:40Z","receivedAt":"2009-04-06T14:51:40Z","isPatch":false,"sender":{"key":"spearce@spearce.org","avatar":"https://avatars.githubusercontent.com/u/34844?v=4"},"body":"Klas Lindberg wrote:\n> On Mon, Apr 6, 2009 at 3:56 PM, Finn Arne Gangstad wrote:\n> > Maybe the security concerns could be handled by adding some\n> > functionality to (quickly) get rid of unwanted commits?\n> \n> Why not simply allow users with write permissions to \"pop\" revisions\n> from the top of the history DAG in a way that actually really deletes\n> the them? Or at least moves those commits to a separate, locked down\n> DAG that cannot be read by people without write permissions?\n\nWhat, like a secret shadow repository that you move the objects into?\n\nThat could be very expensive in terms of disk IO if those objects\nare in large packs. You'd need to break the pack apart into the\n\"ok\" and \"sekret\" parts. Ick.\n \n> But anyway: If I implement support for fetching SHA keys and full\n> recursive behaviour in the presence of submodules; would my patches\n> automatically be rejected because of the rationale for the current\n> behaviour?\n\nSee my recent email (like ~10-15 minutes ago). It will be rejected\ndue to the issue that unreachable objects are subjected to GC and\nyou'd easily see your repository delete that data on the next \"git\ngc\" invocation. Automatic data destruction is not something that\nusers come to git for.\n\n-- \nShawn.\n"},{"id":"110600","messageId":"33f4f4d70904060929r31aa5870s3d9880c8ec9afc67@mail.gmail.com","threadId":"18748","inReplyTo":"20090406145140.GG23604@spearce.org","subject":"Re: Submodules can't work recursively because Git implements policy?","fromName":"Klas Lindberg","fromEmail":"klas.lindberg@gmail.com","sentAt":"2009-04-06T16:29:30Z","receivedAt":"2009-04-06T16:29:30Z","isPatch":false,"sender":{"key":"klas.lindberg@gmail.com","avatar":null},"body":"On Mon, Apr 6, 2009 at 4:51 PM, Shawn O. Pearce wrote:\n\n> What, like a secret shadow repository that you move the objects into?\n>\n> That could be very expensive in terms of disk IO if those objects\n> are in large packs. You'd need to break the pack apart into the\n> \"ok\" and \"sekret\" parts. Ick.\n\nWell, ok. Just popping then?\nOr adding the wrong publication to a forbidden fetch list?\n\n>> But anyway: If I implement support for fetching SHA keys and full\n>> recursive behaviour in the presence of submodules; would my patches\n>> automatically be rejected because of the rationale for the current\n>> behaviour?\n>\n> See my recent email (like ~10-15 minutes ago). It will be rejected\n> due to the issue that unreachable objects are subjected to GC and\n> you'd easily see your repository delete that data on the next \"git\n> gc\" invocation. Automatic data destruction is not something that\n> users come to git for.\n\nIndeed not. But I don't want to suggest that named references\nshouldn't be necessary. Just that it should be possible to fetch based\non the SHA key if that commit is (still) available on the remote end.\n\nBR / Klas\n"}]}