{"thread":{"id":"20465","subject":"[PATCH 0/6 (v2)] Suggested for PU: revision caching system to significantly speed up packing/walking","startedAt":"2009-08-08T07:31:29Z","lastAt":"2009-08-17T08:16:41Z","messageCount":5,"participants":["Nick Edelen","Sam Vilain","Johannes Schindelin"],"isPatch":true,"patchVersion":2,"patchTotal":6},"messages":[{"id":"119962","messageId":"op.uyb1uryftdk399@sirnot.private","threadId":"20465","inReplyTo":null,"subject":"[PATCH 0/6 (v2)] Suggested for PU: revision caching system to significantly speed up packing/walking","fromName":"Nick Edelen","fromEmail":"sirnot@gmail.com","sentAt":"2009-08-08T07:31:29Z","receivedAt":"2009-08-08T07:31:29Z","isPatch":true,"sender":{"key":"sirnot@gmail.com","avatar":null},"body":"SUGGESTED FOR 'PU': (patchset v2)\n\nTraversing objects is currently very costly, as every commit and tree must be \nloaded and parsed.  Much time and energy could be saved by caching metadata and \ntopological info in an efficient, easily accessible manner.  Furthermore, this \ncould improve git's interfacing potential, by providing a condensed summary of \na repository's commit tree.\n\nThis is a series to implement such a revision caching mechanism, aptly named \nrev-cache.  The series will provide:\n - a core API to manipulate and traverse caches\n - an integration into the internal revision walker\n - a porcelain front-end providing access to users and (shell) applications\n - a series of tests to verify/demonstrate correctness\n - documentation of the API, porcelain and core concepts\n\nIn cold starts rev-cache has sped up packing and walking by a factor of 4, and \nover twice that on warm starts.  Some times on slax for the linux repository:\n\nrev-list --all --objects >/dev/null\n default\n   cold    1:13\n   warm    0:43\n rev-cache'd\n   cold    0:19\n   warm    0:02\n\npack-objects --revs --all --stdout >/dev/null\n default\n   cold    2:44\n   warm    1:21\n rev-cache'd\n   cold    0:44\n   warm    0:10\n\nThe mechanism is minimally intrusive: most of the changes take place in \nseperate files, and only a handful of git's existing functions are modified.\n\nHope you find this useful.\n\n - Nick\n\n---\nThis is a revised patchset; the biggest changes are:\n - elimination of python dependency\n - revision of documentation\n - addition of name caching (patch 6)\n\n Documentation/git-rev-cache.txt       |   96 ++\n Documentation/technical/rev-cache.txt |  379 ++++++\n Makefile                              |    2 +\n blob.c                                |    1 +\n blob.h                                |    2 +\n builtin-rev-cache.c                   |  309 +++++\n builtin.h                             |    1 +\n commit.c                              |    3 +\n commit.h                              |    2 +\n git.c                                 |    1 +\n list-objects.c                        |   49 +-\n rev-cache.c                           | 2041 +++++++++++++++++++++++++++++++++\n revision.c                            |   89 ++-\n revision.h                            |   47 +-\n t/t6015-rev-cache-list.sh             |  246 ++++\n tree.h                                |    1 +\n 16 files changed, 3244 insertions(+), 25 deletions(-)\n create mode 100755 Documentation/git-rev-cache.txt\n create mode 100755 Documentation/technical/rev-cache.txt\n create mode 100755 builtin-rev-cache.c\n create mode 100755 rev-cache.c\n create mode 100755 t/t6015-rev-cache-list.sh\n"},{"id":"120798","messageId":"c77435a80908161547x4d2a9087qf439d1c30cc4fa99@mail.gmail.com","threadId":"20465","inReplyTo":"op.uyb1uryftdk399@sirnot.private","subject":"Re: [PATCH 0/6 (v2)] Suggested for PU: revision caching system to significantly speed up packing/walking","fromName":"Nick Edelen","fromEmail":"sirnot@gmail.com","sentAt":"2009-08-16T22:47:28Z","receivedAt":"2009-08-16T22:47:28Z","isPatch":true,"sender":{"key":"sirnot@gmail.com","avatar":null},"body":"Well crap, in my infinite wisdom I replied with the final updated\npatchset to the wrong version :-/  I don't want to clog up the list\nwith my stupid mistakes, but if you need me to I can resend them to\nv3.\n\nBut yeah, I've uploaded an updated version.  This patchset:\n - fixes whitespace errors\n - adds _ondisk versions of structures to ensure portability\n - cleans up definitions/fix a typo\n\nHope it works out for you all.  Sorry about the mismatched reply.\n\n - Nick\n"},{"id":"120801","messageId":"4A889412.9090209@vilain.net","threadId":"20465","inReplyTo":"c77435a80908161547x4d2a9087qf439d1c30cc4fa99@mail.gmail.com","subject":"Re: [PATCH 0/6 (v2)] Suggested for PU: revision caching system to significantly speed up packing/walking","fromName":"Sam Vilain","fromEmail":"sam@vilain.net","sentAt":"2009-08-16T23:19:46Z","receivedAt":"2009-08-16T23:19:46Z","isPatch":true,"sender":{"key":"sam@vilain.net","avatar":"https://gravatar.com/avatar/8fc840ca854dbf6f7065b4335e3b934951c1dca3b11db688e95e471901f8f4a8?d=mp&s=160"},"body":"Nick Edelen wrote:\n> Well crap, in my infinite wisdom I replied with the final updated\n> patchset to the wrong version :-/  I don't want to clog up the list\n> with my stupid mistakes, but if you need me to I can resend them to\n> v3.\n>   \n\nHeh, that sure reads like a sentence written after midnight :-)\n\nThese all say v2, and the last lot was labeled as v3.  And I see an\nolder submission labeled as v2 (8th August) and got duplicates of those\nlabeled as v3.\n\nAre the ones attached to this the ones that are the best?\n\nSam\n\n> But yeah, I've uploaded an updated version.  This patchset:\n>  - fixes whitespace errors\n>  - adds _ondisk versions of structures to ensure portability\n>  - cleans up definitions/fix a typo\n>\n> Hope it works out for you all.  Sorry about the mismatched reply.\n>\n>  - Nick\n>   \n"},{"id":"120807","messageId":"c77435a80908161756n88eb967wbc82058fc4975c04@mail.gmail.com","threadId":"20465","inReplyTo":"4A889412.9090209@vilain.net","subject":"Re: [PATCH 0/6 (v2)] Suggested for PU: revision caching system to significantly speed up packing/walking","fromName":"Nick Edelen","fromEmail":"sirnot@gmail.com","sentAt":"2009-08-17T00:56:27Z","receivedAt":"2009-08-17T00:56:27Z","isPatch":true,"sender":{"key":"sirnot@gmail.com","avatar":null},"body":"> Heh, that sure reads like a sentence written after midnight :-)\n>\n> These all say v2, and the last lot was labeled as v3.  And I see an\n> older submission labeled as v2 (8th August) and got duplicates of those\n> labeled as v3.\n>\n> Are the ones attached to this the ones that are the best?\n\nYeah, the latest ones Re: v2, uploaded a few hours ago, are the best.\nThey should be Re: v3 but they're not...  If I'm being confusing I can\njust upload them to v3, but I didn't want to bombard everyone with\nduplicates.\n"},{"id":"120844","messageId":"alpine.DEB.1.00.0908171012210.4991@intel-tinevez-2-302","threadId":"20465","inReplyTo":"c77435a80908161756n88eb967wbc82058fc4975c04@mail.gmail.com","subject":"Re: [PATCH 0/6 (v2)] Suggested for PU: revision caching system to significantly speed up packing/walking","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2009-08-17T08:16:41Z","receivedAt":"2009-08-17T08:16:41Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi,\n\nOn Mon, 17 Aug 2009, Nick Edelen wrote:\n\n> If I'm being confusing I can just upload them to v3, but I didn't want \n> to bombard everyone with duplicates.\n\nI think by now people _are_ confused, so it's probably better to \"bomb\" \neveryone with a clean current version of your patch series.\n\nAlthough I have to admit that I am scared away by the sheer size of the \nindividual patches (together with bad feelings from earlier discussions \nthat I want to avoid).\n\nCiao,\nDscho\n"}]}