{"thread":{"id":"11260","subject":"Efficient retrieval of commit log info","startedAt":"2007-12-12T14:36:15Z","lastAt":"2007-12-12T16:24:38Z","messageCount":8,"participants":["Eirik Bjørsnøs","Johannes Schindelin","Nicolas Pitre","Santi Béjar","Jon Smirl","Linus Torvalds"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"62901","messageId":"34660cca0712120636w149e2a82h84609f8ac7c958a9@mail.gmail.com","threadId":"11260","inReplyTo":null,"subject":"Efficient retrieval of commit log info","fromName":"Eirik Bjørsnøs","fromEmail":"eirbjo@gmail.com","sentAt":"2007-12-12T14:36:15Z","receivedAt":"2007-12-12T14:36:15Z","isPatch":false,"sender":{"key":"eirbjo@gmail.com","avatar":"https://gravatar.com/avatar/f4d63d977be4245a00e9cdc15c7f9f3c65c7167a22c667daf4558c3ef5809f5f?d=mp&s=160"},"body":"Hi,\n\nI'm developing a piece of software that grabs logs from various types\nof SCMs and presents (hopefully) useful information about the history.\n\nMy current approach with Git is do a \"git clone --n\" and then parse\nthe output of \"git log\". To check for updates I do a \"git pull\"\nfollowed by a new \"git log\".\n\nThis approach works fine, but cloning the whole repository just to get\nthe change log seems like a somewhat inefficient use of bandwidth and\nstorage.\n\nWhat I would like to do is to fetch just the change log information\nfrom the remote repository.\n\n(Using the \"CVS done right\" tool I can do this with: \"svn log --xml -v\n-r<last, HEAD> http://svn.example.com/\")\n\nI haven't found a way to do this using the Git command line tools.\n\nIs it at all possible using the git network protocols to fetch just\nthe commit log info, without transferring the contents?\n\nI'd be happy to hear from anyone who has some insight into this.\n\nThanks,\n\nEirik.\n"},{"id":"62903","messageId":"Pine.LNX.4.64.0712121453150.27959@racer.site","threadId":"11260","inReplyTo":"34660cca0712120636w149e2a82h84609f8ac7c958a9@mail.gmail.com","subject":"Re: Efficient retrieval of commit log info","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2007-12-12T14:55:41Z","receivedAt":"2007-12-12T14:55:41Z","isPatch":false,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi,\n\nOn Wed, 12 Dec 2007, Eirik Bj?rsn?s wrote:\n\n> I'm developing a piece of software that grabs logs from various types of \n> SCMs and presents (hopefully) useful information about the history.\n> \n> My current approach with Git is do a \"git clone --n\" and then parse the \n> output of \"git log\". To check for updates I do a \"git pull\" followed by \n> a new \"git log\".\n> \n> This approach works fine, but cloning the whole repository just to get \n> the change log seems like a somewhat inefficient use of bandwidth and \n> storage.\n> \n> What I would like to do is to fetch just the change log information\n> from the remote repository.\n> \n> (Using the \"CVS done right\" tool I can do this with: \"svn log --xml -v\n> -r<last, HEAD> http://svn.example.com/\")\n> \n> I haven't found a way to do this using the Git command line tools.\n\nIt is not possible to get just the metadata.  Remember, svn can do it only \nsince the repository is purely remote.  And git is a SCM (source code \nmanagement system), not a CMV (commit metadata viewer).\n\nYou might be able to cobble up something that works accessing gitweb, but \nit might be even more inefficient.\n\nCiao,\nDscho\n"},{"id":"62906","messageId":"alpine.LFD.0.99999.0712121001430.555@xanadu.home","threadId":"11260","inReplyTo":"Pine.LNX.4.64.0712121453150.27959@racer.site","subject":"Re: Efficient retrieval of commit log info","fromName":"Nicolas Pitre","fromEmail":"nico@cam.org","sentAt":"2007-12-12T15:04:07Z","receivedAt":"2007-12-12T15:04:07Z","isPatch":false,"sender":{"key":"nico@fluxnic.net","avatar":"https://avatars.githubusercontent.com/u/702790?v=4"},"body":"On Wed, 12 Dec 2007, Johannes Schindelin wrote:\n\n> Hi,\n> \n> On Wed, 12 Dec 2007, Eirik Bj?rsn?s wrote:\n> \n> > I'm developing a piece of software that grabs logs from various types of \n> > SCMs and presents (hopefully) useful information about the history.\n> > \n> > My current approach with Git is do a \"git clone --n\" and then parse the \n> > output of \"git log\". To check for updates I do a \"git pull\" followed by \n> > a new \"git log\".\n> > \n> > This approach works fine, but cloning the whole repository just to get \n> > the change log seems like a somewhat inefficient use of bandwidth and \n> > storage.\n> > \n> > What I would like to do is to fetch just the change log information\n> > from the remote repository.\n> > \n> > (Using the \"CVS done right\" tool I can do this with: \"svn log --xml -v\n> > -r<last, HEAD> http://svn.example.com/\")\n> > \n> > I haven't found a way to do this using the Git command line tools.\n> \n> It is not possible to get just the metadata.  Remember, svn can do it only \n> since the repository is purely remote.  And git is a SCM (source code \n> management system), not a CMV (commit metadata viewer).\n> \n> You might be able to cobble up something that works accessing gitweb, but \n> it might be even more inefficient.\n\nOn the other hand, this is pretty trivial to extend the protocol so only \ncommit objects are transferred.  Using 'git log' on the client side \nwould work, as long as you don't use any of the content walking options.\n\n\nNicolas\n"},{"id":"62909","messageId":"34660cca0712120719p2d7cda44s97ab770abd648742@mail.gmail.com","threadId":"11260","inReplyTo":"alpine.LFD.0.99999.0712121001430.555@xanadu.home","subject":"Re: Efficient retrieval of commit log info","fromName":"Eirik Bjørsnøs","fromEmail":"eirbjo@gmail.com","sentAt":"2007-12-12T15:19:05Z","receivedAt":"2007-12-12T15:19:05Z","isPatch":false,"sender":{"key":"eirbjo@gmail.com","avatar":"https://gravatar.com/avatar/f4d63d977be4245a00e9cdc15c7f9f3c65c7167a22c667daf4558c3ef5809f5f?d=mp&s=160"},"body":"> On the other hand, this is pretty trivial to extend the protocol so only\n> commit objects are transferred.  Using 'git log' on the client side\n> would work, as long as you don't use any of the content walking options.\n\nNicolas,\n\nSo what you're suggesting is that:\n\n1) git-clone could be changed such that the user could ask not to\nfetch content, only commit objects.\n2) git-daemon (or the protocol layer, again I'm not familiar with the\nimplementation) could be changed such that it only sends commit\nobjects on request.\n3) Changes to git-log is not necessary.\n\nAnybody have a clue how much work this would require and if this is\nsomething that would be generally useful for the community?\n\nEirik.\n"},{"id":"62912","messageId":"8aa486160712120734u7d77cc0ew7b3ab1663db1fa62@mail.gmail.com","threadId":"11260","inReplyTo":"Pine.LNX.4.64.0712121453150.27959@racer.site","subject":"Re: Efficient retrieval of commit log info","fromName":"Santi Béjar","fromEmail":"sbejar@gmail.com","sentAt":"2007-12-12T15:34:02Z","receivedAt":"2007-12-12T15:34:02Z","isPatch":false,"sender":{"key":"santi@agolina.net","avatar":null},"body":"On Dec 12, 2007 3:55 PM, Johannes Schindelin <Johannes.Schindelin@gmx.de> wrote:\n> Hi,\n>\n> On Wed, 12 Dec 2007, Eirik Bj?rsn?s wrote:\n>\n[...]\n> > What I would like to do is to fetch just the change log information\n> > from the remote repository.\n> >\n[ ... ]\n>\n> It is not possible to get just the metadata.  Remember, svn can do it only\n> since the repository is purely remote.  And git is a SCM (source code\n> management system), not a CMV (commit metadata viewer).\n>\n> You might be able to cobble up something that works accessing gitweb, but\n> it might be even more inefficient.\n\nYou could see how it is done in the git-browser:\n\nhttp://repo.or.cz/w/git-browser.git\n\nIt is a gitk-like browser written in javascript. But I suspect that it\nshould be enabled in the web server.\n\nSanti\n"},{"id":"62913","messageId":"9e4733910712120734r2618e882w1e6f930c74f3065a@mail.gmail.com","threadId":"11260","inReplyTo":"34660cca0712120719p2d7cda44s97ab770abd648742@mail.gmail.com","subject":"Re: Efficient retrieval of commit log info","fromName":"Jon Smirl","fromEmail":"jonsmirl@gmail.com","sentAt":"2007-12-12T15:34:33Z","receivedAt":"2007-12-12T15:34:33Z","isPatch":false,"sender":{"key":"jonsmirl@gmail.com","avatar":"https://gravatar.com/avatar/cff3bf5bfdfa6708b905712ff91f0f9b8aaca161659f38c02b787920d5d28b7e?d=mp&s=160"},"body":"On 12/12/07, Eirik Bjørsnøs <eirbjo@gmail.com> wrote:\n> > On the other hand, this is pretty trivial to extend the protocol so only\n> > commit objects are transferred.  Using 'git log' on the client side\n> > would work, as long as you don't use any of the content walking options.\n>\n> Nicolas,\n>\n> So what you're suggesting is that:\n\nWrite a script the runs the command locally on the remote sever and\nemails you the info. Or ssh to the remote server and run the commands\nlocally.\n\n>\n> 1) git-clone could be changed such that the user could ask not to\n> fetch content, only commit objects.\n> 2) git-daemon (or the protocol layer, again I'm not familiar with the\n> implementation) could be changed such that it only sends commit\n> objects on request.\n> 3) Changes to git-log is not necessary.\n>\n> Anybody have a clue how much work this would require and if this is\n> something that would be generally useful for the community?\n>\n> Eirik.\n> -\n> To unsubscribe from this list: send the line \"unsubscribe git\" in\n> the body of a message to majordomo@vger.kernel.org\n> More majordomo info at  http://vger.kernel.org/majordomo-info.html\n>\n\n\n-- \nJon Smirl\njonsmirl@gmail.com\n"},{"id":"62916","messageId":"34660cca0712120750p103e0989n80bd1a6a1f0ffa2c@mail.gmail.com","threadId":"11260","inReplyTo":"9e4733910712120734r2618e882w1e6f930c74f3065a@mail.gmail.com","subject":"Re: Efficient retrieval of commit log info","fromName":"Eirik Bjørsnøs","fromEmail":"eirbjo@gmail.com","sentAt":"2007-12-12T15:50:15Z","receivedAt":"2007-12-12T15:50:15Z","isPatch":false,"sender":{"key":"eirbjo@gmail.com","avatar":"https://gravatar.com/avatar/f4d63d977be4245a00e9cdc15c7f9f3c65c7167a22c667daf4558c3ef5809f5f?d=mp&s=160"},"body":"> Write a script the runs the command locally on the remote sever and\n> emails you the info. Or ssh to the remote server and run the commands\n> locally.\n\nHi Jon,\n\nUnfortunately that won't work for my use case since I'm fetching\nchange metadata from remote repositories that I don't have any kind of\ncontrol over.\n\nEirik.\n"},{"id":"62930","messageId":"alpine.LFD.0.9999.0712120819430.25032@woody.linux-foundation.org","threadId":"11260","inReplyTo":"34660cca0712120636w149e2a82h84609f8ac7c958a9@mail.gmail.com","subject":"Re: Efficient retrieval of commit log info","fromName":"Linus Torvalds","fromEmail":"torvalds@linux-foundation.org","sentAt":"2007-12-12T16:24:38Z","receivedAt":"2007-12-12T16:24:38Z","isPatch":false,"sender":{"key":"torvalds@linux-foundation.org","avatar":"https://avatars.githubusercontent.com/u/1024025?v=4"},"body":"\n\nOn Wed, 12 Dec 2007, Eirik Bj?rsn?s wrote:\n> \n> Is it at all possible using the git network protocols to fetch just\n> the commit log info, without transferring the contents?\n\n>From a _protocol_ angle that's trivial (the git network protocol doesn't \nreally care what it transfers, it just transfers objects), but no, we \ndon't actually expose any capability like that. If you were to just want \nthis privately, the quick hack is to just disable traversal of trees in \nbuiltin-pack-objects on the server side, but it sounds like you want to do \nthis in general (not just with git and not just with a server you \ncontrol), and if so, you're kind of screwed.\n\nYou could hack around it (very inefficiently) by parsing gitweb output, of \ncourse, but I bet that you'd probably get better performance just cloning \nthe whole thing (which allows streaming, rather than trying to click \nthrough commits on gitweb)\n\n\t\tLinus\n"}]}