# Efficient retrieval of commit log info 8 messages from 2007-12-12 to 2007-12-12. Participants: Eirik Bjørsnøs, Johannes Schindelin, Nicolas Pitre, Santi Béjar, Jon Smirl, Linus Torvalds. Thread: https://gitlist.dev/t/11260 ## Eirik Bjørsnøs, 2007-12-12 14:36 Subject: Efficient retrieval of commit log info Message-ID: <34660cca0712120636w149e2a82h84609f8ac7c958a9@mail.gmail.com> URL: https://gitlist.dev/e/34660cca0712120636w149e2a82h84609f8ac7c958a9%40mail.gmail.com ``` Hi, I'm developing a piece of software that grabs logs from various types of SCMs and presents (hopefully) useful information about the history. My current approach with Git is do a "git clone --n" and then parse the output of "git log". To check for updates I do a "git pull" followed by a new "git log". This approach works fine, but cloning the whole repository just to get the change log seems like a somewhat inefficient use of bandwidth and storage. What I would like to do is to fetch just the change log information from the remote repository. (Using the "CVS done right" tool I can do this with: "svn log --xml -v -r http://svn.example.com/") I haven't found a way to do this using the Git command line tools. Is it at all possible using the git network protocols to fetch just the commit log info, without transferring the contents? I'd be happy to hear from anyone who has some insight into this. Thanks, Eirik. ``` ## Johannes Schindelin, 2007-12-12 14:55 Subject: Re: Efficient retrieval of commit log info Message-ID: URL: https://gitlist.dev/e/Pine.LNX.4.64.0712121453150.27959%40racer.site In-Reply-To: <34660cca0712120636w149e2a82h84609f8ac7c958a9@mail.gmail.com> ``` Hi, On Wed, 12 Dec 2007, Eirik Bj?rsn?s wrote: > I'm developing a piece of software that grabs logs from various types of > SCMs and presents (hopefully) useful information about the history. > > My current approach with Git is do a "git clone --n" and then parse the > output of "git log". To check for updates I do a "git pull" followed by > a new "git log". > > This approach works fine, but cloning the whole repository just to get > the change log seems like a somewhat inefficient use of bandwidth and > storage. > > What I would like to do is to fetch just the change log information > from the remote repository. > > (Using the "CVS done right" tool I can do this with: "svn log --xml -v > -r http://svn.example.com/") > > I haven't found a way to do this using the Git command line tools. It is not possible to get just the metadata. Remember, svn can do it only since the repository is purely remote. And git is a SCM (source code management system), not a CMV (commit metadata viewer). You might be able to cobble up something that works accessing gitweb, but it might be even more inefficient. Ciao, Dscho ``` ## Nicolas Pitre, 2007-12-12 15:04 Subject: Re: Efficient retrieval of commit log info Message-ID: URL: https://gitlist.dev/e/alpine.LFD.0.99999.0712121001430.555%40xanadu.home In-Reply-To: ``` On Wed, 12 Dec 2007, Johannes Schindelin wrote: > Hi, > > On Wed, 12 Dec 2007, Eirik Bj?rsn?s wrote: > > > I'm developing a piece of software that grabs logs from various types of > > SCMs and presents (hopefully) useful information about the history. > > > > My current approach with Git is do a "git clone --n" and then parse the > > output of "git log". To check for updates I do a "git pull" followed by > > a new "git log". > > > > This approach works fine, but cloning the whole repository just to get > > the change log seems like a somewhat inefficient use of bandwidth and > > storage. > > > > What I would like to do is to fetch just the change log information > > from the remote repository. > > > > (Using the "CVS done right" tool I can do this with: "svn log --xml -v > > -r http://svn.example.com/") > > > > I haven't found a way to do this using the Git command line tools. > > It is not possible to get just the metadata. Remember, svn can do it only > since the repository is purely remote. And git is a SCM (source code > management system), not a CMV (commit metadata viewer). > > You might be able to cobble up something that works accessing gitweb, but > it might be even more inefficient. On the other hand, this is pretty trivial to extend the protocol so only commit objects are transferred. Using 'git log' on the client side would work, as long as you don't use any of the content walking options. Nicolas ``` ## Eirik Bjørsnøs, 2007-12-12 15:19 Subject: Re: Efficient retrieval of commit log info Message-ID: <34660cca0712120719p2d7cda44s97ab770abd648742@mail.gmail.com> URL: https://gitlist.dev/e/34660cca0712120719p2d7cda44s97ab770abd648742%40mail.gmail.com In-Reply-To: ``` > On the other hand, this is pretty trivial to extend the protocol so only > commit objects are transferred. Using 'git log' on the client side > would work, as long as you don't use any of the content walking options. Nicolas, So what you're suggesting is that: 1) git-clone could be changed such that the user could ask not to fetch content, only commit objects. 2) git-daemon (or the protocol layer, again I'm not familiar with the implementation) could be changed such that it only sends commit objects on request. 3) Changes to git-log is not necessary. Anybody have a clue how much work this would require and if this is something that would be generally useful for the community? Eirik. ``` ## Santi Béjar, 2007-12-12 15:34 Subject: Re: Efficient retrieval of commit log info Message-ID: <8aa486160712120734u7d77cc0ew7b3ab1663db1fa62@mail.gmail.com> URL: https://gitlist.dev/e/8aa486160712120734u7d77cc0ew7b3ab1663db1fa62%40mail.gmail.com In-Reply-To: ``` On Dec 12, 2007 3:55 PM, Johannes Schindelin wrote: > Hi, > > On Wed, 12 Dec 2007, Eirik Bj?rsn?s wrote: > [...] > > What I would like to do is to fetch just the change log information > > from the remote repository. > > [ ... ] > > It is not possible to get just the metadata. Remember, svn can do it only > since the repository is purely remote. And git is a SCM (source code > management system), not a CMV (commit metadata viewer). > > You might be able to cobble up something that works accessing gitweb, but > it might be even more inefficient. You could see how it is done in the git-browser: http://repo.or.cz/w/git-browser.git It is a gitk-like browser written in javascript. But I suspect that it should be enabled in the web server. Santi ``` ## Jon Smirl, 2007-12-12 15:34 Subject: Re: Efficient retrieval of commit log info Message-ID: <9e4733910712120734r2618e882w1e6f930c74f3065a@mail.gmail.com> URL: https://gitlist.dev/e/9e4733910712120734r2618e882w1e6f930c74f3065a%40mail.gmail.com In-Reply-To: <34660cca0712120719p2d7cda44s97ab770abd648742@mail.gmail.com> ``` On 12/12/07, Eirik Bjørsnøs wrote: > > On the other hand, this is pretty trivial to extend the protocol so only > > commit objects are transferred. Using 'git log' on the client side > > would work, as long as you don't use any of the content walking options. > > Nicolas, > > So what you're suggesting is that: Write a script the runs the command locally on the remote sever and emails you the info. Or ssh to the remote server and run the commands locally. > > 1) git-clone could be changed such that the user could ask not to > fetch content, only commit objects. > 2) git-daemon (or the protocol layer, again I'm not familiar with the > implementation) could be changed such that it only sends commit > objects on request. > 3) Changes to git-log is not necessary. > > Anybody have a clue how much work this would require and if this is > something that would be generally useful for the community? > > Eirik. > - > To unsubscribe from this list: send the line "unsubscribe git" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html > -- Jon Smirl jonsmirl@gmail.com ``` ## Eirik Bjørsnøs, 2007-12-12 15:50 Subject: Re: Efficient retrieval of commit log info Message-ID: <34660cca0712120750p103e0989n80bd1a6a1f0ffa2c@mail.gmail.com> URL: https://gitlist.dev/e/34660cca0712120750p103e0989n80bd1a6a1f0ffa2c%40mail.gmail.com In-Reply-To: <9e4733910712120734r2618e882w1e6f930c74f3065a@mail.gmail.com> ``` > Write a script the runs the command locally on the remote sever and > emails you the info. Or ssh to the remote server and run the commands > locally. Hi Jon, Unfortunately that won't work for my use case since I'm fetching change metadata from remote repositories that I don't have any kind of control over. Eirik. ``` ## Linus Torvalds, 2007-12-12 16:24 Subject: Re: Efficient retrieval of commit log info Message-ID: URL: https://gitlist.dev/e/alpine.LFD.0.9999.0712120819430.25032%40woody.linux-foundation.org In-Reply-To: <34660cca0712120636w149e2a82h84609f8ac7c958a9@mail.gmail.com> ``` On Wed, 12 Dec 2007, Eirik Bj?rsn?s wrote: > > Is it at all possible using the git network protocols to fetch just > the commit log info, without transferring the contents? >From a _protocol_ angle that's trivial (the git network protocol doesn't really care what it transfers, it just transfers objects), but no, we don't actually expose any capability like that. If you were to just want this privately, the quick hack is to just disable traversal of trees in builtin-pack-objects on the server side, but it sounds like you want to do this in general (not just with git and not just with a server you control), and if so, you're kind of screwed. You could hack around it (very inefficiently) by parsing gitweb output, of course, but I bet that you'd probably get better performance just cloning the whole thing (which allows streaming, rather than trying to click through commits on gitweb) Linus ```