{"thread":{"id":"21989","subject":"Query remote repository files, blobs","startedAt":"2009-12-19T07:45:55Z","lastAt":"2009-12-19T20:50:39Z","messageCount":6,"participants":["Shakthi Kannan","Johannes Schindelin","Bill Lear","Shawn O. Pearce","A Large Angry SCM"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"130128","messageId":"d16b1c80912182345h4163430bv810adb860575d67a@mail.gmail.com","threadId":"21989","inReplyTo":null,"subject":"Query remote repository files, blobs","fromName":"Shakthi Kannan","fromEmail":"shakthimaan@gmail.com","sentAt":"2009-12-19T07:45:55Z","receivedAt":"2009-12-19T07:45:55Z","isPatch":false,"sender":{"key":"shakthimaan@gmail.com","avatar":null},"body":"Hi,\n\n>From git-ls-remote:\nhttp://www.kernel.org/pub/software/scm/git/docs/git-ls-remote.html\n\nI am able to query for list of remote heads, and tags. I would like to\nknow if it is possible to query for information on remote files, or\nblobs? Sometimes, I am on a slow Internet connection, and I just want\nto be able to see the remote file diffs, or new file, branch\nadditions, or changes without having to clone the repository.\n\nAppreciate any inputs on this regard,\n\nThanks!\n\nSK\n\n-- \nShakthi Kannan\nhttp://www.shakthimaan.com\n"},{"id":"130132","messageId":"alpine.DEB.1.00.0912191159280.4985@pacific.mpi-cbg.de","threadId":"21989","inReplyTo":"d16b1c80912182345h4163430bv810adb860575d67a@mail.gmail.com","subject":"Re: Query remote repository files, blobs","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2009-12-19T11:02:02Z","receivedAt":"2009-12-19T11:02:02Z","isPatch":false,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi,\n\nOn Sat, 19 Dec 2009, Shakthi Kannan wrote:\n\n> Hi,\n> \n> >From git-ls-remote:\n> http://www.kernel.org/pub/software/scm/git/docs/git-ls-remote.html\n> \n> I am able to query for list of remote heads, and tags. I would like to\n> know if it is possible to query for information on remote files, or\n> blobs?\n\nThis has been discussed a number of times, but we cannot allow that for \nsecurity reasons.  A blob might contain confidential information, in which \ncase the branch has to be rewritten and force-pushed.  However, that does \nnot make the blob go away, but makes it only unreachable.  Until the next \ngarbage collection kicks in, that is (which you typically cannot control).\n\nSo: sorry, your feature request cannot be granted.\n\nCiao,\nDscho\n"},{"id":"130138","messageId":"19244.54684.250532.801766@lisa.zopyra.com","threadId":"21989","inReplyTo":"alpine.DEB.1.00.0912191159280.4985@pacific.mpi-cbg.de","subject":"Re: Query remote repository files, blobs","fromName":"Bill Lear","fromEmail":"rael@zopyra.com","sentAt":"2009-12-19T13:31:08Z","receivedAt":"2009-12-19T13:31:08Z","isPatch":false,"sender":{"key":"rael@zopyra.com","avatar":"https://gravatar.com/avatar/c4f2d2790ca3828d3b4e7dfebabf61d2fe94fd82fa49cdac2a5295dd2d46a874?d=mp&s=160"},"body":"On Saturday, December 19, 2009 at 12:02:02 (+0100) Johannes Schindelin writes:\n>On Sat, 19 Dec 2009, Shakthi Kannan wrote:\n>...\n>> I am able to query for list of remote heads, and tags. I would like to\n>> know if it is possible to query for information on remote files, or\n>> blobs?\n>\n>This has been discussed a number of times, but we cannot allow that for \n>security reasons.  A blob might contain confidential information, in which \n>case the branch has to be rewritten and force-pushed.  However, that does \n>not make the blob go away, but makes it only unreachable.  Until the next \n>garbage collection kicks in, that is (which you typically cannot control).\n\nHmm, I thought this had been addressed by git in a different way (removing\nconfidential information).  A company will not be satisfied that its\nproprietary information is \"unreachable\" in your software repository.\nThey want absolute assurance that the information is completely\nremoved.\n\nHave I remembered wrongly --- is this still not possible with git?\n\n\nBill\n"},{"id":"130157","messageId":"20091219192415.GC10687@spearce.org","threadId":"21989","inReplyTo":"19244.54684.250532.801766@lisa.zopyra.com","subject":"Re: Query remote repository files, blobs","fromName":"Shawn O. Pearce","fromEmail":"spearce@spearce.org","sentAt":"2009-12-19T19:24:15Z","receivedAt":"2009-12-19T19:24:15Z","isPatch":false,"sender":{"key":"spearce@spearce.org","avatar":"https://avatars.githubusercontent.com/u/34844?v=4"},"body":"Bill Lear <rael@zopyra.com> wrote:\n> On Saturday, December 19, 2009 at 12:02:02 (+0100) Johannes Schindelin writes:\n> >On Sat, 19 Dec 2009, Shakthi Kannan wrote:\n> >...\n> >> I am able to query for list of remote heads, and tags. I would like to\n> >> know if it is possible to query for information on remote files, or\n> >> blobs?\n> >\n> >This has been discussed a number of times, but we cannot allow that for \n> >security reasons.  A blob might contain confidential information, in which \n> >case the branch has to be rewritten and force-pushed.  However, that does \n> >not make the blob go away, but makes it only unreachable.  Until the next \n> >garbage collection kicks in, that is (which you typically cannot control).\n> \n> Hmm, I thought this had been addressed by git in a different way (removing\n> confidential information).  A company will not be satisfied that its\n> proprietary information is \"unreachable\" in your software repository.\n> They want absolute assurance that the information is completely\n> removed.\n> \n> Have I remembered wrongly --- is this still not possible with git?\n\nIts still possible, but you have to wipe out the reflog record(s)\nthat had the object in it, and you have to repack to evict it from\nthe pack files, and you have to run `git prune --expire=0` to force\nit to wipe out the object immediately.\n\nWe already support dumping back random commits via upload-archive, if\nits enabled in the daemon, and I think a lot of people do turn it on.\nThere is no validation that the requested tree-ish is reachable.\n\nI think gitweb winds up doing the same thing, it doesn't actually\ntry to validate the object is reachable, it just serves whatever\nit was asked for, if its present in the repository.\n\n\nI'm getting some mild suggestions over here at $DAY_JOB to implement\nshallow clones by lazily downloading large blobs on demand.\nWe've resisted doing this in git because of the reachability test\nDscho mentioned above... but many people skip that anyway due\nto gitweb and upload-archive being enabled.  Which is making me\nstart to question who is broken... upload-pack for not being more\nwilling to serve arbitrary content, or gitweb/upload-archive for\nnot validating their requests are reachable.\n\n-- \nShawn.\n"},{"id":"130158","messageId":"4B2D3AB1.5000504@gmail.com","threadId":"21989","inReplyTo":"20091219192415.GC10687@spearce.org","subject":"Re: Query remote repository files, blobs","fromName":"A Large Angry SCM","fromEmail":"gitzilla@gmail.com","sentAt":"2009-12-19T20:42:25Z","receivedAt":"2009-12-19T20:42:25Z","isPatch":false,"sender":{"key":"gitzilla@gmail.com","avatar":"https://gravatar.com/avatar/354625c442439908ff3dd99757dee330e29e9df7847472384faf7a00add247fb?d=mp&s=160"},"body":"Shawn O. Pearce wrote:\n> Bill Lear <rael@zopyra.com> wrote:\n>> On Saturday, December 19, 2009 at 12:02:02 (+0100) Johannes Schindelin writes:\n>>> On Sat, 19 Dec 2009, Shakthi Kannan wrote:\n>>> ...\n>>>> I am able to query for list of remote heads, and tags. I would like to\n>>>> know if it is possible to query for information on remote files, or\n>>>> blobs?\n>>> This has been discussed a number of times, but we cannot allow that for \n>>> security reasons.  A blob might contain confidential information, in which \n>>> case the branch has to be rewritten and force-pushed.  However, that does \n>>> not make the blob go away, but makes it only unreachable.  Until the next \n>>> garbage collection kicks in, that is (which you typically cannot control).\n>> Hmm, I thought this had been addressed by git in a different way (removing\n>> confidential information).  A company will not be satisfied that its\n>> proprietary information is \"unreachable\" in your software repository.\n>> They want absolute assurance that the information is completely\n>> removed.\n>>\n>> Have I remembered wrongly --- is this still not possible with git?\n> \n> Its still possible, but you have to wipe out the reflog record(s)\n> that had the object in it, and you have to repack to evict it from\n> the pack files, and you have to run `git prune --expire=0` to force\n> it to wipe out the object immediately.\n> \n> We already support dumping back random commits via upload-archive, if\n> its enabled in the daemon, and I think a lot of people do turn it on.\n> There is no validation that the requested tree-ish is reachable.\n> \n> I think gitweb winds up doing the same thing, it doesn't actually\n> try to validate the object is reachable, it just serves whatever\n> it was asked for, if its present in the repository.\n> \n> \n> I'm getting some mild suggestions over here at $DAY_JOB to implement\n> shallow clones by lazily downloading large blobs on demand.\n> We've resisted doing this in git because of the reachability test\n> Dscho mentioned above... but many people skip that anyway due\n> to gitweb and upload-archive being enabled.  Which is making me\n> start to question who is broken... upload-pack for not being more\n> willing to serve arbitrary content, or gitweb/upload-archive for\n> not validating their requests are reachable.\n\nThe security argument hasn't been true for a while if any of the \nfollowing are enabled on the server:\n\tGitweb\n\thttp transport\n\trsync transport\n\tftp transport\n\talternates (the alternates are the ones at risk)\n"},{"id":"130160","messageId":"alpine.DEB.1.00.0912192148360.4985@pacific.mpi-cbg.de","threadId":"21989","inReplyTo":"4B2D3AB1.5000504@gmail.com","subject":"Re: Query remote repository files, blobs","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2009-12-19T20:50:39Z","receivedAt":"2009-12-19T20:50:39Z","isPatch":false,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi,\n\nOn Sat, 19 Dec 2009, A Large Angry SCM wrote:\n\n> Shawn O. Pearce wrote:\n> > Bill Lear <rael@zopyra.com> wrote:\n> > > On Saturday, December 19, 2009 at 12:02:02 (+0100) Johannes Schindelin\n> > > writes:\n> > > > On Sat, 19 Dec 2009, Shakthi Kannan wrote:\n> > > > ...\n> > > > > I am able to query for list of remote heads, and tags. I would like to\n> > > > > know if it is possible to query for information on remote files, or\n> > > > > blobs?\n> > > > This has been discussed a number of times, but we cannot allow that for\n> > > > security reasons.  A blob might contain confidential information, in\n> > > > which case the branch has to be rewritten and force-pushed.  However,\n> > > > that does not make the blob go away, but makes it only unreachable.\n> > > > Until the next garbage collection kicks in, that is (which you typically\n> > > > cannot control).\n> > > Hmm, I thought this had been addressed by git in a different way (removing\n> > > confidential information).  A company will not be satisfied that its\n> > > proprietary information is \"unreachable\" in your software repository.\n> > > They want absolute assurance that the information is completely\n> > > removed.\n> > >\n> > > Have I remembered wrongly --- is this still not possible with git?\n> > \n> > Its still possible, but you have to wipe out the reflog record(s)\n> > that had the object in it, and you have to repack to evict it from\n> > the pack files, and you have to run `git prune --expire=0` to force\n> > it to wipe out the object immediately.\n> > \n> > We already support dumping back random commits via upload-archive, if\n> > its enabled in the daemon, and I think a lot of people do turn it on.\n> > There is no validation that the requested tree-ish is reachable.\n> > \n> > I think gitweb winds up doing the same thing, it doesn't actually\n> > try to validate the object is reachable, it just serves whatever\n> > it was asked for, if its present in the repository.\n> > \n> > \n> > I'm getting some mild suggestions over here at $DAY_JOB to implement\n> > shallow clones by lazily downloading large blobs on demand.\n> > We've resisted doing this in git because of the reachability test\n> > Dscho mentioned above... but many people skip that anyway due\n> > to gitweb and upload-archive being enabled.  Which is making me\n> > start to question who is broken... upload-pack for not being more\n> > willing to serve arbitrary content, or gitweb/upload-archive for\n> > not validating their requests are reachable.\n> \n> The security argument hasn't been true for a while if any of the following are\n> enabled on the server:\n> \tGitweb\n> \thttp transport\n> \trsync transport\n> \tftp transport\n> \talternates (the alternates are the ones at risk)\n\nRight.  But we must not break the security for careful administrators \n(read: who did _not_ enable any of the above) just out of a whim, right?  \nCertainly not if the only reason to do it was \"somebody thought it would \nbe neat, but most probably is not going to use it herself after having \nplayed with it for a few days\".\n\nCiao,\nDscho\n"}]}