{"thread":{"id":"9511","subject":"gitk performance questions/issues","startedAt":"2007-08-13T09:48:54Z","lastAt":"2007-08-14T11:25:59Z","messageCount":5,"participants":["David Tweed","Linus Torvalds","Paul Mackerras"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"50626","messageId":"e1dab3980708130248g1cbab0cej18e260c8bfa2b315@mail.gmail.com","threadId":"9511","inReplyTo":null,"subject":"gitk performance questions/issues","fromName":"David Tweed","fromEmail":"david.tweed@gmail.com","sentAt":"2007-08-13T09:48:54Z","receivedAt":"2007-08-13T09:48:54Z","isPatch":false,"sender":{"key":"david.tweed@gmail.com","avatar":null},"body":"Since the gitk is being discussed, I'd like to see if I'm\nunderstanding correctly and hitting problems or just\nnot doing things correctly. (This is using a\nself-compiled \"git version 1.5.2.2\".)\n\n[Caveat: this is on a repository that is evolves using\nmy own scripts which mimic the git shell scripts in\ncalling the low-level programs. I'm 99.999% sure it's\ngenerating a fully valid repo, but I might be missing\nsomething. Actual generating script is at\nhttp://www.personal.rdg.ac.uk/~sis05dst/chronoversion.htm\n]\n\n1. With gitk --all -n 256 on my repo it consistently\ntakes 12s for the window to appear and 21s for the\n\"I'm working\" cursor to change to a normal cursor.\nThis is on a dual Xeon machine, /proc/cpuinfo\nexcerpt\n\nmodel name : Intel(R) Xeon(TM) CPU 3.00GHz\nstepping : 3\ncpu MHz : 2992.496\ncache size : 2048 KB\n\nrunning x86-64 Fedora 7, packed repo has a 12M\n.git dir. Is this the expected start-up time for such a\nconfiguration? The output of\ntime git-rev-list --parents --topo-order --all>/dev/null\nis\n\nreal 0m0.048s\nuser 0m0.044s\nsys 0m0.005s\n\n2. Startup time isn't an excessive problem. However,\nI often don't seem to be getting the \"diff with parent\"\nbeing displayed once I click on the another commit. If\nI click on several in sequence I get the error\nmessage\n\n\"error getting diffs: couldn't fork child process: not enough memory\"\n\nThe diffs are relatively small (consecutive commits\nare generated hourly automatically) and a command\nline git-diff generates them instantly.\n\n3. Is \"gitk --all -n 256\" the best way to say \"show me\nrelatively recent stuff\" or should I be using different\noptions for limiting things?\n\nMany thanks for any insight,\n\n-- \ncheers, dave tweed__________________________\ndavid.tweed@gmail.com\nRm 124, School of Systems Engineering, University of Reading.\n\"we had no idea that when we added templates we were adding a Turing-\ncomplete compile-time language.\" -- C++ standardisation committee\n"},{"id":"50642","messageId":"alpine.LFD.0.999.0708130945420.30176@woody.linux-foundation.org","threadId":"9511","inReplyTo":"e1dab3980708130248g1cbab0cej18e260c8bfa2b315@mail.gmail.com","subject":"Re: gitk performance questions/issues","fromName":"Linus Torvalds","fromEmail":"torvalds@linux-foundation.org","sentAt":"2007-08-13T16:59:11Z","receivedAt":"2007-08-13T16:59:11Z","isPatch":false,"sender":{"key":"torvalds@linux-foundation.org","avatar":"https://avatars.githubusercontent.com/u/1024025?v=4"},"body":"\n\nOn Mon, 13 Aug 2007, David Tweed wrote:\n> \n> 1. With gitk --all -n 256 on my repo it consistently\n> takes 12s for the window to appear and 21s for the\n> \"I'm working\" cursor to change to a normal cursor.\n\nThere is something seriously wrong there.\n\nFor me, on the kernel archive (which is much bigger than yours), the \nwindow appears immediately, and the \"working cursor\" takes about two \nseconds to disappear.\n\n> This is on a dual Xeon machine, /proc/cpuinfo\n> excerpt\n> \n> model name : Intel(R) Xeon(TM) CPU 3.00GHz\n> stepping : 3\n> cpu MHz : 2992.496\n> cache size : 2048 KB\n\nYour CPU is fast enough..\n\n> running x86-64 Fedora 7, packed repo has a 12M\n> .git dir.\n\n.. and you're apparently fully packed, and do not even have all that big a \nrepository.\n\nSo:\n\n> Is this the expected start-up time for such a\n> configuration?\n\nAbsolutely not.\n\n> The output of\n>\n> time git-rev-list --parents --topo-order --all>/dev/null\n>\n> is\n> \n> real 0m0.048s\n> user 0m0.044s\n> sys 0m0.005s\n\nOk, so it's not the revision generation that takes time. \n\n> 2. Startup time isn't an excessive problem. However,\n> I often don't seem to be getting the \"diff with parent\"\n> being displayed once I click on the another commit. If\n> I click on several in sequence I get the error\n> message\n> \n> \"error getting diffs: couldn't fork child process: not enough memory\"\n\nYou have some *serious* problem somewhere. I think your tcl/tk is screwed \nup somehow. \n\nAlternatively, maybe your repo has some odd structure (do you have tons \nand tons of branches or tags?) and \"gitk\" ends up having some problem with \nthat. What does\n\n\tgit rev-parse --all | wc -l\n\nsay? Various git tools have had performance problems with thousands of \nbranches or tags in the past.\n\nOne thing to try: I think that \"git gc\" back in the 1.5.2.2 days didn't \npack branches by default - and if you have lots and lots of branches, you \nshould try to make sure that they are packed, using\n\n\tgit pack-refs --all --prune\n\n(newer git versions will do this automatically in \"git gc\" - while \nv1.5.2.2 will only pack the tags by default, iirc).\n\n> The diffs are relatively small (consecutive commits\n> are generated hourly automatically) and a command\n> line git-diff generates them instantly.\n\nYeah, it sounds like git itself has no problems, and it's literally gitk \nthat shows some strange behaviour.\n\n> 3. Is \"gitk --all -n 256\" the best way to say \"show me\n> relatively recent stuff\" or should I be using different\n> options for limiting things?\n\nThat's a perfectly good thing to do. You don't need the \"-n\", you can \njust do \"-256\". That said, if you actually want commits to be in date \norder, use\n\n\tgitk -d --all -256\n\ninstead, which tells gitk to use \"--date-order\" instead, which is often \nnicer (the graph ends up being a bit messier, but it gives you a more \ncorrect view of relative dates when you have multiple branches!\n\n\t\tLinus\n"},{"id":"50644","messageId":"e1dab3980708131018h495c5cf2m76cb8f6ffc4df6dc@mail.gmail.com","threadId":"9511","inReplyTo":"alpine.LFD.0.999.0708130945420.30176@woody.linux-foundation.org","subject":"Re: gitk performance questions/issues","fromName":"David Tweed","fromEmail":"david.tweed@gmail.com","sentAt":"2007-08-13T17:18:24Z","receivedAt":"2007-08-13T17:18:24Z","isPatch":false,"sender":{"key":"david.tweed@gmail.com","avatar":null},"body":"On 8/13/07, Linus Torvalds wrote:\n> Alternatively, maybe your repo has some odd structure (do you have tons\n> and tons of branches or tags?) and \"gitk\" ends up having some problem with\n> that. What does\n>\n> git rev-parse --all | wc -l\n>\n> say? Various git tools have had performance problems with thousands of\n> branches or tags in the past.\n\nAh, that's it: I've got\n$ wc .git/packed-refs\n 1915 3832 130178 .git/packed-refs\nand 35 currently unpacked tags and temporarily moving that\nfile away, gitk now starts up virtually instantly. My usage generates\na tag per commit which is probably excessive. For completeness,\n$ git rev-parse --all | wc -l\n1957\n\nThinking about it, I suppose even if you're only looking at the last\n256 commits you've got to look through all the tags to see whether\nor not they refer to something within that window, so it's not\nunreasonable for it to affect startup time.\n\nThe bit that is a quite surprising to me is that once the window has\nfinished initialising, clicking on one of the blue dots now (ie\nwithout .git/packed-refs file) now generates the diffs instantly\nas well. It's not obvious to me why the existence/non-existence\nof tags should affect displaying-diff performance.\n\nAnyway, now I know I can rearrange things not to need anything\nlike as many tags.\n\nMany thanks for the help,\n\n-- \ncheers, dave tweed__________________________\ndavid.tweed@gmail.com\nRm 124, School of Systems Engineering, University of Reading.\n\"we had no idea that when we added templates we were adding a Turing-\ncomplete compile-time language.\" -- C++ standardisation committee\n"},{"id":"50675","messageId":"18113.12777.164103.302185@cargo.ozlabs.ibm.com","threadId":"9511","inReplyTo":"e1dab3980708131018h495c5cf2m76cb8f6ffc4df6dc@mail.gmail.com","subject":"Re: gitk performance questions/issues","fromName":"Paul Mackerras","fromEmail":"paulus@samba.org","sentAt":"2007-08-14T04:39:05Z","receivedAt":"2007-08-14T04:39:05Z","isPatch":false,"sender":{"key":"paulus@samba.org","avatar":"https://avatars.githubusercontent.com/u/1606439?v=4"},"body":"David Tweed writes:\n\n> Thinking about it, I suppose even if you're only looking at the last\n> 256 commits you've got to look through all the tags to see whether\n> or not they refer to something within that window, so it's not\n> unreasonable for it to affect startup time.\n\nGitk uses a Tcl associative array for that, which should be very\nfast.\n\nCould you try this: go to the Edit->Preferences window and turn off\n\"Display nearby tags\". Then with your ~2000 refs in place, see how\nlong gitk takes to start up and to display a diff.\n\nIf doing that makes it fast, then I suspect that my topology caching\npatch will also help. That's waiting on an answer from Junio or Linus\nto my question about making git rev-list ignore negated non-existent\nrefs.\n\nAlso, if it's possible to give me a copy of your repo, that would\nhelp.\n\nPaul.\n"},{"id":"50699","messageId":"e1dab3980708140425r72733f1bn4bc0d2e6cc68fb4c@mail.gmail.com","threadId":"9511","inReplyTo":"18113.12777.164103.302185@cargo.ozlabs.ibm.com","subject":"Re: gitk performance questions/issues","fromName":"David Tweed","fromEmail":"david.tweed@gmail.com","sentAt":"2007-08-14T11:25:59Z","receivedAt":"2007-08-14T11:25:59Z","isPatch":false,"sender":{"key":"david.tweed@gmail.com","avatar":null},"body":"On 8/14/07, Paul Mackerras wrote:\n> David Tweed writes:\n> Could you try this: go to the Edit->Preferences window and turn off\n> \"Display nearby tags\". Then with your ~2000 refs in place, see how\n> long gitk takes to start up and to display a diff.\n\nRight, I've restored the file of 1915 packed tags (but not the 35 loose\ntags). Using DNT for \"display nearby tags\"\n\nDNT on: consistently 11s to window appear, 20s to normal cursor\nDNT off: consistently 11s to window appear but normal cursor immediately\n\nRegarding getting the diffs, I think that was \"user misunderstanding\" and\nI'm doubting if I definitely did click on the blue circles yesterday. Clicking\non the any part yellow tag marker displays the SHA of the tag object\n(which I'd confused with the start of an unfinished commit diff);\nfor some reason I'd thought it would give the diff-with-parent. Clicking on\neither the blue circle or \"commit headline text\" does bring up the diff\ninstantly even with the packed refs file. Sorry for the confusion.\n\n> Also, if it's possible to give me a copy of your repo, that would\n> help.\n\nI'll privately mail Paul about getting it; if anyone else wants a copy\nof the repository\nin order look at startup performance issues mail me. (The most embarassing\nthing in there is just some astonishingly crap code, but I'd prefer not to\nblanket publish it.)\n\n-- \ncheers, dave tweed__________________________\ndavid.tweed@gmail.com\nRm 124, School of Systems Engineering, University of Reading.\n\"we had no idea that when we added templates we were adding a Turing-\ncomplete compile-time language.\" -- C++ standardisation committee\n"}]}