{"thread":{"id":"11245","subject":"git blame with valgrind massif","startedAt":"2007-12-11T20:57:24Z","lastAt":"2007-12-12T00:50:53Z","messageCount":6,"participants":["Jon Smirl","Pierre Habouzit","Linus Torvalds","André Goddard Rosa"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"62799","messageId":"9e4733910712111257h20a4a916gd4747e816e4706ff@mail.gmail.com","threadId":"11245","inReplyTo":null,"subject":"git blame with valgrind massif","fromName":"Jon Smirl","fromEmail":"jonsmirl@gmail.com","sentAt":"2007-12-11T20:57:24Z","receivedAt":"2007-12-11T20:57:24Z","isPatch":false,"sender":{"key":"jonsmirl@gmail.com","avatar":"https://gravatar.com/avatar/cff3bf5bfdfa6708b905712ff91f0f9b8aaca161659f38c02b787920d5d28b7e?d=mp&s=160"},"body":"I ran:\n valgrind --tool=massif --heap=yes git blame gcc/ChangeLog\nit used about 2.25GB\n\nHow do you interpret the massif output?\n\nCommand: git blame gcc/ChangeLog\n\n== 0 ===========================\nHeap allocation functions accounted for 98.2% of measured spacetime\n\nCalled from:\n  34.3% : 0x44EFC2: patch_delta (git-compat-util.h:235)\n\n  24.7% : 0x45DD38: cache_or_unpack_entry (git-compat-util.h:235)\n\n  13.1% : 0x4909FC: xdl_cha_alloc (xutils.c:113)\n\n  12.0% : 0x461277: strbuf_grow (git-compat-util.h:268)\n\n   5.9% : 0x40EF5A: cmd_blame (git-compat-util.h:235)\n\n   2.6% : 0x48FA5E: xdl_prepare_env (xprepare.c:79)\n\n   1.9% : 0x45D775: unpack_compressed_entry (git-compat-util.h:235)\n\n   1.5% : 0x452FE5: read_index_from (git-compat-util.h:284)\n\n   1.3% : 0x48F672: xdl_prepare_ctx (xprepare.c:162)\n\n  and 61 other insignificant places\n\n== 1 ===========================\nContext accounted for 34.3% of measured spacetime\n  0x44EFC2: patch_delta (git-compat-util.h:235)\n\nCalled from:\n  34.3% : 0x45DA03: unpack_entry (sha1_file.c:1578)\n\n---------------------------------\nContext accounted for 24.7% of measured spacetime\n  0x45DD38: cache_or_unpack_entry (git-compat-util.h:235)\n\nCalled from:\n  24.7% : 0x45F175: read_packed_sha1 (sha1_file.c:1815)\n\n---------------------------------\nContext accounted for 13.1% of measured spacetime\n  0x4909FC: xdl_cha_alloc (xutils.c:113)\n\nCalled from:\n   6.6% : 0x48F75B: xdl_prepare_ctx (xprepare.c:192)\n\n   4.0% : 0x48F83A: xdl_prepare_ctx (xprepare.c:115)\n\n---------------------------------\nContext accounted for 12.0% of measured spacetime\n  0x461277: strbuf_grow (git-compat-util.h:268)\n\nCalled from:\n  12.0% : 0x4617F4: strbuf_read (strbuf.c:192)\n\n  and 2 other insignificant places\n\n---------------------------------\nContext accounted for  5.9% of measured spacetime\n  0x40EF5A: cmd_blame (git-compat-util.h:235)\n\nCalled from:\n   5.9% : 0x40405A: handle_internal_command (git.c:266)\n\n---------------------------------\nContext accounted for  2.6% of measured spacetime\n  0x48FA5E: xdl_prepare_env (xprepare.c:79)\n\nCalled from:\n   2.6% : 0x48F21A: xdl_do_diff (xdiffi.c:332)\n\n---------------------------------\nContext accounted for  1.9% of measured spacetime\n  0x45D775: unpack_compressed_entry (git-compat-util.h:235)\n\nCalled from:\n   1.9% : 0x45D945: unpack_entry (sha1_file.c:1606)\n\n  and 1 other insignificant place\n\n---------------------------------\nContext accounted for  1.5% of measured spacetime\n  0x452FE5: read_index_from (git-compat-util.h:284)\n\nCalled from:\n   1.5% : 0x40FAD7: cmd_blame (builtin-blame.c:2075)\n\n---------------------------------\nContext accounted for  1.3% of measured spacetime\n  0x48F672: xdl_prepare_ctx (xprepare.c:162)\n\nCalled from:\n   0.6% : 0x48FAAB: xdl_prepare_env (xprepare.c:280)\n\n  and 1 other insignificant place\n\n== 2 ===========================\nContext accounted for 34.3% of measured spacetime\n  0x44EFC2: patch_delta (git-compat-util.h:235)\n  0x45DA03: unpack_entry (sha1_file.c:1578)\n\nCalled from:\n  41.7% : 0x45D9BE: unpack_entry (sha1_file.c:1567)\n\n  33.3% : 0x45F175: read_packed_sha1 (sha1_file.c:1815)\n\n---------------------------------\nContext accounted for 24.7% of measured spacetime\n  0x45DD38: cache_or_unpack_entry (git-compat-util.h:235)\n  0x45F175: read_packed_sha1 (sha1_file.c:1815)\n\nCalled from:\n  24.7% : 0x45F55D: read_sha1_file (sha1_file.c:1881)\n\n---------------------------------\nContext accounted for  6.6% of measured spacetime\n  0x4909FC: xdl_cha_alloc (xutils.c:113)\n  0x48F75B: xdl_prepare_ctx (xprepare.c:192)\n\nCalled from:\n   2.9% : 0x48FAAB: xdl_prepare_env (xprepare.c:280)\n\n   1.6% : 0x48FAD0: xdl_prepare_env (xprepare.c:285)\n\n---------------------------------\nContext accounted for  4.0% of measured spacetime\n  0x4909FC: xdl_cha_alloc (xutils.c:113)\n  0x48F83A: xdl_prepare_ctx (xprepare.c:115)\n\nCalled from:\n   4.0% : 0x48FAAB: xdl_prepare_env (xprepare.c:280)\n\n  and 1 other insignificant place\n\n---------------------------------\nContext accounted for 12.0% of measured spacetime\n  0x461277: strbuf_grow (git-compat-util.h:268)\n  0x4617F4: strbuf_read (strbuf.c:192)\n\nCalled from:\n  12.0% : 0x461881: strbuf_read_file (strbuf.c:228)\n\n---------------------------------\nContext accounted for  5.9% of measured spacetime\n  0x40EF5A: cmd_blame (git-compat-util.h:235)\n  0x40405A: handle_internal_command (git.c:266)\n\nCalled from:\n   5.9% : 0x4045C9: main (git.c:456)\n\n---------------------------------\nContext accounted for  2.6% of measured spacetime\n  0x48FA5E: xdl_prepare_env (xprepare.c:79)\n  0x48F21A: xdl_do_diff (xdiffi.c:332)\n\nCalled from:\n   2.6% : 0x48F3B0: xdl_diff (xdiffi.c:542)\n\n---------------------------------\nContext accounted for  1.9% of measured spacetime\n  0x45D775: unpack_compressed_entry (git-compat-util.h:235)\n  0x45D945: unpack_entry (sha1_file.c:1606)\n\nCalled from:\n   2.4% : 0x45F175: read_packed_sha1 (sha1_file.c:1815)\n\n  and 1 other insignificant place\n\n---------------------------------\nContext accounted for  1.5% of measured spacetime\n  0x452FE5: read_index_from (git-compat-util.h:284)\n  0x40FAD7: cmd_blame (builtin-blame.c:2075)\n\nCalled from:\n   1.5% : 0x40405A: handle_internal_command (git.c:266)\n\n---------------------------------\nContext accounted for  0.6% of measured spacetime\n  0x48F672: xdl_prepare_ctx (xprepare.c:162)\n  0x48FAAB: xdl_prepare_env (xprepare.c:280)\n\nCalled from:\n   0.6% : 0x48F21A: xdl_do_diff (xdiffi.c:332)\n\n=================================\n\nEnd of information.  Rerun with a bigger --depth value for more.\n\n-- \nJon Smirl\njonsmirl@gmail.com\n"},{"id":"62807","messageId":"20071211212052.GC29110@artemis.madism.org","threadId":"11245","inReplyTo":"9e4733910712111257h20a4a916gd4747e816e4706ff@mail.gmail.com","subject":"Re: git blame with valgrind massif","fromName":"Pierre Habouzit","fromEmail":"madcoder@debian.org","sentAt":"2007-12-11T21:20:52Z","receivedAt":"2007-12-11T21:20:52Z","isPatch":false,"sender":{"key":"madcoder@debian.org","avatar":"https://avatars.githubusercontent.com/u/44708?v=4"},"body":"On Tue, Dec 11, 2007 at 08:57:24PM +0000, Jon Smirl wrote:\n> I ran:\n>  valgrind --tool=massif --heap=yes git blame gcc/ChangeLog\n> it used about 2.25GB\n> \n> How do you interpret the massif output?\n\n  would you mind putting the postscript it generated somewhere too ?\nit's usually pretty informative, because the amount of data allocated is\nnot all, its liveness is an important information too.\n\n-- \n·O·  Pierre Habouzit\n··O                                                madcoder@debian.org\nOOO                                                http://www.madism.org\n"},{"id":"62808","messageId":"alpine.LFD.0.9999.0712111315060.25032@woody.linux-foundation.org","threadId":"11245","inReplyTo":"9e4733910712111257h20a4a916gd4747e816e4706ff@mail.gmail.com","subject":"Re: git blame with valgrind massif","fromName":"Linus Torvalds","fromEmail":"torvalds@linux-foundation.org","sentAt":"2007-12-11T21:22:53Z","receivedAt":"2007-12-11T21:22:53Z","isPatch":false,"sender":{"key":"torvalds@linux-foundation.org","avatar":"https://avatars.githubusercontent.com/u/1024025?v=4"},"body":"\n\nOn Tue, 11 Dec 2007, Jon Smirl wrote:\n> \n> How do you interpret the massif output?\n\nNot very easy, since massif will tell you what *allocated* it, but then \ntrying to see who was supposed to free it is another issue altogether.\n\nI also find the textual output to be very confusing. But what massif is \nreally good at is to look at the memory usage over time in the postscript \nfile it generates, and that gives you a much better feel for what \nparticular allocation is a problem.\n\nIn this case, it's patch_delta that generates all the memory usage (well, \n98% of it ;^), but that's not that helpful unless you know git internals, \nand realize that with deep delta chains, that only means that the memory \nis kept around just for random object data. The question is why that \nobject data stays around and isn't free'd.\n\nThere's two answers to that:\n - the non-buggy use of the object data in the delta base cache (limited \n   by the delta_base_cache_limit, which defaults to 16MB, although you can \n   tweak it with core.deltabasecachelimit)\n - the (possibly buggy) callers that keep the data.\n\nSee my previous email with a patch to \"git blame\" to make it release the \nobject data. That should fix it, I think.\n\n\t\tLinus\n"},{"id":"62810","messageId":"20071211212743.GD29110@artemis.madism.org","threadId":"11245","inReplyTo":"alpine.LFD.0.9999.0712111315060.25032@woody.linux-foundation.org","subject":"Re: git blame with valgrind massif","fromName":"Pierre Habouzit","fromEmail":"madcoder@debian.org","sentAt":"2007-12-11T21:27:43Z","receivedAt":"2007-12-11T21:27:43Z","isPatch":false,"sender":{"key":"madcoder@debian.org","avatar":"https://avatars.githubusercontent.com/u/44708?v=4"},"body":"On Tue, Dec 11, 2007 at 09:22:53PM +0000, Linus Torvalds wrote:\n> \n> \n> On Tue, 11 Dec 2007, Jon Smirl wrote:\n> > \n> > How do you interpret the massif output?\n> \n> Not very easy, since massif will tell you what *allocated* it, but then \n> trying to see who was supposed to free it is another issue altogether.\n> \n> I also find the textual output to be very confusing. But what massif is \n> really good at is to look at the memory usage over time in the postscript \n> file it generates, and that gives you a much better feel for what \n> particular allocation is a problem.\n> \n> In this case, it's patch_delta that generates all the memory usage (well, \n> 98% of it ;^), but that's not that helpful unless you know git internals, \n\n  No 37%, the 98% list is the list of calls that generate 98% of the\ntotal allocations git blame does, and then massif lists each individual\ncall, and patch_delta generates 37% of the total.\n\n  Though it's meaningless without knowing how much time this memory\nstayed allocated>\n\n-- \n·O·  Pierre Habouzit\n··O                                                madcoder@debian.org\nOOO                                                http://www.madism.org\n"},{"id":"62813","messageId":"9e4733910712111345i23d86a85jfedb4f37f20a5b0@mail.gmail.com","threadId":"11245","inReplyTo":"20071211212052.GC29110@artemis.madism.org","subject":"Re: git blame with valgrind massif","fromName":"Jon Smirl","fromEmail":"jonsmirl@gmail.com","sentAt":"2007-12-11T21:45:48Z","receivedAt":"2007-12-11T21:45:48Z","isPatch":false,"sender":{"key":"jonsmirl@gmail.com","avatar":"https://gravatar.com/avatar/cff3bf5bfdfa6708b905712ff91f0f9b8aaca161659f38c02b787920d5d28b7e?d=mp&s=160"},"body":"On 12/11/07, Pierre Habouzit <madcoder@debian.org> wrote:\n> On Tue, Dec 11, 2007 at 08:57:24PM +0000, Jon Smirl wrote:\n> > I ran:\n> >  valgrind --tool=massif --heap=yes git blame gcc/ChangeLog\n> > it used about 2.25GB\n> >\n> > How do you interpret the massif output?\n>\n>   would you mind putting the postscript it generated somewhere too ?\n> it's usually pretty informative, because the amount of data allocated is\n> not all, its liveness is an important information too.\n\nIt was very boring. A diagonal line from 0 to 2GB.\n\n>\n> --\n> ·O·  Pierre Habouzit\n> ··O                                                madcoder@debian.org\n> OOO                                                http://www.madism.org\n>\n>\n\n\n-- \nJon Smirl\njonsmirl@gmail.com\n"},{"id":"62838","messageId":"b8bf37780712111650i1b291cbaneea5efd16b92cb8b@mail.gmail.com","threadId":"11245","inReplyTo":"9e4733910712111345i23d86a85jfedb4f37f20a5b0@mail.gmail.com","subject":"Re: git blame with valgrind massif","fromName":"André Goddard Rosa","fromEmail":"andre.goddard@gmail.com","sentAt":"2007-12-12T00:50:53Z","receivedAt":"2007-12-12T00:50:53Z","isPatch":false,"sender":{"key":"andre.goddard@gmail.com","avatar":null},"body":"On Dec 11, 2007 7:45 PM, Jon Smirl <jonsmirl@gmail.com> wrote:\n> On 12/11/07, Pierre Habouzit <madcoder@debian.org> wrote:\n> > On Tue, Dec 11, 2007 at 08:57:24PM +0000, Jon Smirl wrote:\n> > > I ran:\n> > >  valgrind --tool=massif --heap=yes git blame gcc/ChangeLog\n> > > it used about 2.25GB\n> > >\n> > > How do you interpret the massif output?\n\nA bit unrelated, but there's a new valgrind release:\nhttp://www.valgrind.org/docs/manual/dist.news.html\n\n\"The main excitement in 3.3.0 is new and improved tools.  Helgrind\nworks again, Massif has been completely overhauled and much improved,\nCachegrind now does branch-misprediction profiling, and a new category\nof experimental tools has been created, containing two new tools:\nOmega and DRD.  There are many other smaller improvements.\"\n\nBest regards,\n-- \n[]s,\nAndré Goddard\n"}]}