{"thread":{"id":"10626","subject":"[RFC PATCH] Make gitk use --early-output","startedAt":"2007-11-03T23:49:01Z","lastAt":"2007-11-04T18:28:39Z","messageCount":8,"participants":["Paul Mackerras","Michael J. Cohen","Linus Torvalds","Marco Costalba","David Kastrup"],"isPatch":true,"patchVersion":1,"patchTotal":null},"messages":[{"id":"58188","messageId":"18221.2285.259487.655684@cargo.ozlabs.ibm.com","threadId":"10626","inReplyTo":null,"subject":"[RFC PATCH] Make gitk use --early-output","fromName":"Paul Mackerras","fromEmail":"paulus@samba.org","sentAt":"2007-11-03T23:49:01Z","receivedAt":"2007-11-03T23:49:01Z","isPatch":true,"sender":{"key":"paulus@samba.org","avatar":"https://avatars.githubusercontent.com/u/1606439?v=4"},"body":"This makes gitk use the --early-output flag on the git log command.\n\nWhen gitk sees the \"Final output:\" line from git log, it goes into a\nmode where it basically just checks that it is getting the commits\nagain in the same order as before.  If they are, well and good; if\nnot, it truncates its internal list at the point of difference and\nproceeds to read in the commits in the new order from there on, and\nre-does the graph layout if necessary.\n\nThis gives a much more immediate feel to the startup; gitk shows its\nwindow with the first screenful of commits displayed very quickly this\nway.\n\nSigned-off-by: Paul Mackerras <paulus@samba.org>\n---\ndiff --git a/gitk b/gitk\nindex 1da0b0a..7d9a2f2 100755\n--- a/gitk\n+++ b/gitk\n@@ -84,25 +84,27 @@ proc start_rev_list {view} {\n     global commfd leftover tclencoding datemode\n     global viewargs viewfiles commitidx viewcomplete vnextroot\n     global showlocalchanges commitinterest mainheadid\n-    global progressdirn progresscoords proglastnc curview\n+    global progressdirn progresscoords proglastnc curview rereading\n \n     set startmsecs [clock clicks -milliseconds]\n     set commitidx($view) 0\n     set viewcomplete($view) 0\n     set vnextroot($view) 0\n-    set order \"--topo-order\"\n+    set order \"--early-output=50\"\n     if {$datemode} {\n-\tset order \"--date-order\"\n+\tlappend order \"--date-order\"\n     }\n     if {[catch {\n-\tset fd [open [concat | git log --no-color -z --pretty=raw $order --parents \\\n-\t\t\t --boundary $viewargs($view) \"--\" $viewfiles($view)] r]\n+\tset fd [open [concat | git log --no-color -z --pretty=raw \\\n+\t\t\t  $order --parents --boundary \\\n+\t\t\t  $viewargs($view) \"--\" $viewfiles($view)] r]\n     } err]} {\n \terror_popup \"Error executing git rev-list: $err\"\n \texit 1\n     }\n     set commfd($view) $fd\n     set leftover($view) {}\n+    set rereading($view) -1\n     if {$showlocalchanges} {\n \tlappend commitinterest($mainheadid) {dodiffindex}\n     }\n@@ -161,6 +163,7 @@ proc getcommitlines {fd view}  {\n     global parentlist children curview hlview\n     global vparentlist vdisporder vcmitlisted\n     global ordertok vnextroot idpending\n+    global rereading nullid nullid2\n \n     set stuff [read $fd 500000]\n     # git log doesn't terminate the last commit with a null...\n@@ -236,6 +239,15 @@ proc getcommitlines {fd view}  {\n \t}\n \tset start [expr {$i + 1}]\n \tset j [string first \"\\n\" $cmit]\n+\tif {$j >= 0 && [string match \"Final output:*\" $cmit]} {\n+\t    set rereading($view) 0\n+\t    set cmit [string range $cmit [expr {$j + 1}] end]\n+\t    set j [string first \"\\n\" $cmit]\n+\t    if {$view == $curview} {\n+\t\tlayoutmore\n+\t\tupdate\n+\t    }\n+\t}\n \tset ok 0\n \tset listed 1\n \tif {$j >= 0 && [string match \"commit *\" $cmit]} {\n@@ -255,6 +267,7 @@ proc getcommitlines {fd view}  {\n \t\t    break\n \t\t}\n \t    }\n+\t    set cmit [string range $cmit [expr {$j + 1}] end]\n \t}\n \tif {!$ok} {\n \t    set shortcmit $cmit\n@@ -265,13 +278,31 @@ proc getcommitlines {fd view}  {\n \t    exit 1\n \t}\n \tset id [lindex $ids 0]\n+\tif {$rereading($view) >= 0} {\n+\t    set r $rereading($view)\n+\t    set oldid [lindex $displayorder $r]\n+\t    while {$oldid eq $nullid || $oldid eq $nullid2} {\n+\t\tset oldid [lindex $displayorder [incr r]]\n+\t    }\n+\t    if {$oldid eq $id} {\n+\t\t# commits are still in the same order; just skip to the next\n+\t\tset rereading($view) [expr {$r + 1}]\n+\t\tcontinue\n+\t    }\n+\t    if {$r < $commitidx($view)} {\n+\t\t# commits are in a different order now;\n+\t\t# truncate the list and redisplay\n+\t\ttruncate_view $view $r\n+\t    }\n+\t    set rereading($view) -1\n+\t}\n \tif {![info exists ordertok($view,$id)]} {\n \t    set otok \"o[strrep $vnextroot($view)]\"\n \t    incr vnextroot($view)\n \t    set ordertok($view,$id) $otok\n \t} else {\n \t    set otok $ordertok($view,$id)\n-\t    unset idpending($view,$id)\n+\t    catch {unset idpending($view,$id)}\n \t}\n \tif {$listed} {\n \t    set olds [lrange $ids 1 end]\n@@ -301,7 +332,7 @@ proc getcommitlines {fd view}  {\n \tif {![info exists children($view,$id)]} {\n \t    set children($view,$id) {}\n \t}\n-\tset commitdata($id) [string range $cmit [expr {$j + 1}] end]\n+\tset commitdata($id) $cmit\n \tset commitrow($view,$id) $commitidx($view)\n \tincr commitidx($view)\n \tif {$view == $curview} {\n@@ -323,7 +354,7 @@ proc getcommitlines {fd view}  {\n     }\n     if {$gotsome} {\n \trun chewcommits $view\n-\tif {$view == $curview} {\n+\tif {0 && $view == $curview} {\n \t    # update progress bar\n \t    global progressdirn progresscoords proglastnc\n \t    set inc [expr {($commitidx($view) - $proglastnc) * 0.0002}]\n@@ -354,6 +385,43 @@ proc getcommitlines {fd view}  {\n     return 2\n }\n \n+proc truncate_view {view row} {\n+    global curview commitidx displayorder parentlist commitlisted\n+    global vdisporder vparentlist vcmitlisted commitrow children\n+    global numcommits localfrow localirow\n+\n+    set rm1 [expr {$row - 1}]\n+    if {$view == $curview} {\n+\tset disporder $displayorder\n+\tset displayorder [lrange $disporder 0 $rm1]\n+\tset parents $parentlist\n+\tset parentlist [lrange $parents 0 $rm1]\n+\tset commitlisted [lrange $commitlisted 0 $rm1]\n+    } else {\n+\tset disporder $vdisporder($view)\n+\tset vdisporder($view) [lrange $disporder 0 $rm1]\n+\tset parents $vparentlist($view)\n+\tset vparentlist($view) [lrange $parents 0 $rm1]\n+\tset vcmitlisted($view) [lrange $vcmitlisted($view) 0 $rm1]\n+    }\n+    for {set r $commitidx($view)} {[incr r -1] >= $row} {} {\n+\tset id [lindex $disporder $r]\n+\tforeach p [lindex $parents $r] {\n+\t    if {[lindex $children($view,$p) end] eq $id} {\n+\t\tset children($view,$p) [lrange $children($view,$p) 0 end-1]\n+\t    }\n+\t}\n+\tunset commitrow($view,$id)\n+    }\n+    set commitidx($view) $row\n+    if {$view == $curview} {\n+\ttruncate_localchanges $row\n+\tif {$row < $numcommits} {\n+\t    undolayout $row\n+\t}\n+    }\n+}\n+\n proc chewcommits {view} {\n     global curview hlview viewcomplete\n     global selectedline pending_select\n@@ -2843,6 +2911,20 @@ proc dohidelocalchanges {} {\n     incr lserial\n }\n \n+proc truncate_localchanges {row} {\n+    global localfrow localirow\n+\n+    if {$localfrow >= $row} {\n+\tset localfrow -1\n+    }\n+    if {$localirow >= $row} {\n+\tset localirow -1\n+    }\n+    if {$localfrow == $row - 1 || $localirow == $row - 1} {\n+\tdohidelocalchanges\n+    }\n+}\n+\n # spawn off a process to do git diff-index --cached HEAD\n proc dodiffindex {} {\n     global localirow localfrow lserial showlocalchanges\n@@ -3840,6 +3922,23 @@ proc drawcommits {row {endrow {}}} {\n     }\n }\n \n+proc undolayout {row} {\n+    global uparrowlen mingaplen downarrowlen\n+    global rowidlist rowisopt rowfinal need_redisplay\n+\n+    set r [expr {$row - ($uparrowlen + $mingaplen + $downarrowlen)}]\n+    if {$r < 0} {\n+\tset r 0\n+    }\n+    if {[llength $rowidlist] > $r} {\n+\tset rowidlist [lrange $rowidlist 0 $r]\n+\tset rowfinal [lrange $rowfinal 0 $r]\n+\tset rowisopt [lrange $rowisopt 0 $r]\n+\tset need_redisplay 1\n+\trun drawvisible\n+    }\n+}\n+\n proc drawfrac {f0 f1} {\n     global canv linespc\n \n"},{"id":"58199","messageId":"00989975-E385-4208-9E7E-1A2A3C737F61@mac.com","threadId":"10626","inReplyTo":"18221.2285.259487.655684@cargo.ozlabs.ibm.com","subject":"Re: [RFC PATCH] Make gitk use --early-output","fromName":"Michael J. Cohen","fromEmail":"michaeljosephcohen@mac.com","sentAt":"2007-11-04T02:07:57Z","receivedAt":"2007-11-04T02:07:57Z","isPatch":true,"sender":{"key":"michaeljosephcohen@mac.com","avatar":null},"body":"On Nov 3, 2007, at 7:49 PM, Paul Mackerras wrote:\n\n> This makes gitk use the --early-output flag on the git log command.\n\n\nOn Nov 3, 2007, at 2:11 PM, Linus Torvalds wrote:\n\n> Try it out, with\n>\n> \tgit log --early-output=2\n>\n> and look at what happens\n\nThis is awesome, guys.\n\nI was initially doing some research to see what makes gitk/qgit slow  \nover samba, but this seems to have solved it. :P\n\n-mjc\n"},{"id":"58214","messageId":"alpine.LFD.0.999.0711032227420.15101@woody.linux-foundation.org","threadId":"10626","inReplyTo":"18221.2285.259487.655684@cargo.ozlabs.ibm.com","subject":"Re: [RFC PATCH] Make gitk use --early-output","fromName":"Linus Torvalds","fromEmail":"torvalds@linux-foundation.org","sentAt":"2007-11-04T05:30:06Z","receivedAt":"2007-11-04T05:30:06Z","isPatch":true,"sender":{"key":"torvalds@linux-foundation.org","avatar":"https://avatars.githubusercontent.com/u/1024025?v=4"},"body":"\n\nOn Sun, 4 Nov 2007, Paul Mackerras wrote:\n>\n> This makes gitk use the --early-output flag on the git log command.\n> \n> When gitk sees the \"Final output:\" line from git log, it goes into a\n> mode where it basically just checks that it is getting the commits\n> again in the same order as before.  If they are, well and good; if\n> not, it truncates its internal list at the point of difference and\n> proceeds to read in the commits in the new order from there on, and\n> re-does the graph layout if necessary.\n> \n> This gives a much more immediate feel to the startup; gitk shows its\n> window with the first screenful of commits displayed very quickly this\n> way.\n\nGoodie. Seems to work for me. I'll tweak the behaviour of --early-output a \nbit more, because right now if things are really cold in the cache, the \n\"--early-output\" logic will often trigger with just a single commit in the \nlist (because the timeout is so short), but it already seems to work \npretty well.\n\n\t\tLinus\n"},{"id":"58242","messageId":"e5bfff550711040237s250bcec0iddf1ebdc616e0bbf@mail.gmail.com","threadId":"10626","inReplyTo":"18221.2285.259487.655684@cargo.ozlabs.ibm.com","subject":"Re: [RFC PATCH] Make gitk use --early-output","fromName":"Marco Costalba","fromEmail":"mcostalba@gmail.com","sentAt":"2007-11-04T10:37:43Z","receivedAt":"2007-11-04T10:37:43Z","isPatch":true,"sender":{"key":"mcostalba@gmail.com","avatar":null},"body":"On 11/4/07, Paul Mackerras <paulus@samba.org> wrote:\n>\n>      set vnextroot($view) 0\n> -    set order \"--topo-order\"\n> +    set order \"--early-output=50\"\n\nBut --early-output does not imply --topo-order, I guess...\n\nI would think you need _both_ in git log:\n\ngit log --early-output --topo-order <...remaining stuff...>\n\nAm I missing something?\n\nMarco\n\n\nP.S: Why did you choose not let git log (i.e. Linus) to handle the\ndefault number of commits?\n\n\"--early-output=50\" instead of just \"--early-output\"\n\nI would say, he added this feature mainly for his personal use, so why\ndon't let him to tweak git-log defaults to his wishes ;-)\n"},{"id":"58245","messageId":"18221.42793.38389.359621@cargo.ozlabs.ibm.com","threadId":"10626","inReplyTo":"e5bfff550711040237s250bcec0iddf1ebdc616e0bbf@mail.gmail.com","subject":"Re: [RFC PATCH] Make gitk use --early-output","fromName":"Paul Mackerras","fromEmail":"paulus@samba.org","sentAt":"2007-11-04T11:04:09Z","receivedAt":"2007-11-04T11:04:09Z","isPatch":true,"sender":{"key":"paulus@samba.org","avatar":"https://avatars.githubusercontent.com/u/1606439?v=4"},"body":"Marco Costalba writes:\n\n> On 11/4/07, Paul Mackerras <paulus@samba.org> wrote:\n> >\n> >      set vnextroot($view) 0\n> > -    set order \"--topo-order\"\n> > +    set order \"--early-output=50\"\n> \n> But --early-output does not imply --topo-order, I guess...\n\nLook here in Linus' patch:\n\n+\t\t\tif (!prefixcmp(arg, \"--early-output\")) {\n+\t\t\t\tint count = 100;\n+\t\t\t\tswitch (arg[14]) {\n+\t\t\t\tcase '=':\n+\t\t\t\t\tcount = atoi(arg+15);\n+\t\t\t\t\t/* Fallthrough */\n+\t\t\t\tcase 0:\n+\t\t\t\t\trevs->topo_order = 1;\n+\t\t\t\t\trevs->early_output = count;\n+\t\t\t\t\tcontinue;\n+\t\t\t\t}\n+\t\t\t}\n\nSo yes, --early-output does imply --topo-order.\n\n> P.S: Why did you choose not let git log (i.e. Linus) to handle the\n> default number of commits?\n> \n> \"--early-output=50\" instead of just \"--early-output\"\n\nBecause I was thinking of adding a control in the edit/preferences\nwindow for it later on.\n\nPaul.\n"},{"id":"58258","messageId":"e5bfff550711040357w77e85ecfl3f840502a8f2ec38@mail.gmail.com","threadId":"10626","inReplyTo":"18221.42793.38389.359621@cargo.ozlabs.ibm.com","subject":"Re: [RFC PATCH] Make gitk use --early-output","fromName":"Marco Costalba","fromEmail":"mcostalba@gmail.com","sentAt":"2007-11-04T11:57:36Z","receivedAt":"2007-11-04T11:57:36Z","isPatch":true,"sender":{"key":"mcostalba@gmail.com","avatar":null},"body":"On 11/4/07, Paul Mackerras <paulus@samba.org> wrote:\n>\n> So yes, --early-output does imply --topo-order.\n>\n\nThanks, I should have checked myself.\n\n> > P.S: Why did you choose not let git log (i.e. Linus) to handle the\n> > default number of commits?\n> >\n> > \"--early-output=50\" instead of just \"--early-output\"\n>\n> Because I was thinking of adding a control in the edit/preferences\n> window for it later on.\n>\n\nI see. Perhaps this default number could become obsolete very quickly\nif Linus implements what he has suggested in a similar thread:\n\n\"One other thing I was thinking of was also to perhaps allow multiple\npartial early-output things, in case we get just 5 commits in the first\n0.1 seconds, then 50 in the first second, and 200 after 2 seconds.. I can\nwell imagine getting the full list taking a long time over a network\nfilesystem (somebody mentioned samba), and maybe having just a single\ntrigger is too inflexible.\"\n\n\nOne thing I see playing with this new --early-output feature in qgit\nis that for small /warm cache repos the list of revisions is already\nthe final one, i.e. the line\n\n\"Final output\"\n\nappears as the first (and useless in this case) line of the git-log\noutput stream.\n\nIf my proposal to teach git-log to check the final output revisions\nagainst the already outputted one is accepted then the handling of the\nabove case would come free.\n\nThe proposal is that in case early-output has already streamed out 'n'\nrevisions, when the final ones are ready git-log checks the firsts 'n'\nfinal output revisions and if they exactly match with the already\noutputted ones then \"Final output\" line is skipped and final output\nstream starts directly from revisions 'n+1'.\n\nGiven the statistically very low number of out of order revisions in\nbig repos the above could end up being the common case.\n\nMarco\n"},{"id":"58286","messageId":"alpine.LFD.0.999.0711040947110.15101@woody.linux-foundation.org","threadId":"10626","inReplyTo":"e5bfff550711040237s250bcec0iddf1ebdc616e0bbf@mail.gmail.com","subject":"Re: [RFC PATCH] Make gitk use --early-output","fromName":"Linus Torvalds","fromEmail":"torvalds@linux-foundation.org","sentAt":"2007-11-04T17:53:28Z","receivedAt":"2007-11-04T17:53:28Z","isPatch":true,"sender":{"key":"torvalds@linux-foundation.org","avatar":"https://avatars.githubusercontent.com/u/1024025?v=4"},"body":"\n\nOn Sun, 4 Nov 2007, Marco Costalba wrote:\n> \n> But --early-output does not imply --topo-order, I guess...\n\nWell, it does right now, because I imagined that the primary users would \nalways want the topological sort.\n\nHowever, I have to admit that --early-output *could* be used even without \nthe topological sort, because it also works for other cases that require \nup-front limiter logic - things like ranges of commits also have to be \nfully evaluated before they are totally certain, so I could imagine seeing \nsome visualizer some day that doesn't need the topo-order sort, but does \nwant to get a \"preliminary\" list.\n\nThat said, it does seem unlikely. Anybody who asks for --early-output is \npretty much invariably going to be an interactive visulizer: the whole \nnotion doesn't make much sense otherwise. So I think I made the right \nchoice in making --early-output imply topo-order, and if somebody ever \nwants to not get the output topologically sorted (unlikely), we could add \na \"--no-topo-order\" flag.\n\nSide note: if you want the \"--date-order\", you do need to specify *both* \n--early-output and --date-order, and it will do the right thing (ie both \nthe preliminary output and the final one will be topologically sorted, but \nwithin that topo-sort it will be in date order rather than clumped by \nthe \"shape\" of the history).\n\n\t\t\tLinus\n"},{"id":"58289","messageId":"85fxzlrhk8.fsf@lola.goethe.zz","threadId":"10626","inReplyTo":"18221.2285.259487.655684@cargo.ozlabs.ibm.com","subject":"Re: [RFC PATCH] Make gitk use --early-output","fromName":"David Kastrup","fromEmail":"dak@gnu.org","sentAt":"2007-11-04T18:28:39Z","receivedAt":"2007-11-04T18:28:39Z","isPatch":true,"sender":{"key":"dak@gnu.org","avatar":"https://avatars.githubusercontent.com/u/52141349?v=4"},"body":"Paul Mackerras <paulus@samba.org> writes:\n\n> This makes gitk use the --early-output flag on the git log command.\n>\n> When gitk sees the \"Final output:\" line from git log, it goes into a\n> mode where it basically just checks that it is getting the commits\n> again in the same order as before.  If they are, well and good; if\n> not, it truncates its internal list at the point of difference and\n> proceeds to read in the commits in the new order from there on, and\n> re-does the graph layout if necessary.\n>\n> This gives a much more immediate feel to the startup; gitk shows its\n> window with the first screenful of commits displayed very quickly this\n> way.\n\nThis is not strictly related with the patch: would it be possible to let\ngitk just stall reading from git-rev-list if it has rendered enough\ncontent on-screen?  The behavior I have with gitk on enormous\nrepositories now is that it starts up reasonably fast and nice and then\nproceeds to suck up all memory in the background.\n\nParticularly annoying is that closing its window appears to work, but\nwish will still proceed sucking up all the pending git-rev-list output\nand allocating memory for it before it will actually exit.\n\n-- \nDavid Kastrup, Kriemhildstr. 15, 44793 Bochum\n"}]}