{"thread":{"id":"20627","subject":"[PATCH 0/6 (v4)] revision caching mechanism to significantly speed up packing/walking (last version)","startedAt":"2009-08-17T12:31:15Z","lastAt":"2009-08-18T11:51:46Z","messageCount":2,"participants":["Nick Edelen"],"isPatch":true,"patchVersion":4,"patchTotal":6},"messages":[{"id":"120870","messageId":"op.uys3qdnntdk399@sirnot.private","threadId":"20627","inReplyTo":null,"subject":"[PATCH 0/6 (v4)] revision caching mechanism to significantly speed up packing/walking (last version)","fromName":"Nick Edelen","fromEmail":"sirnot@gmail.com","sentAt":"2009-08-17T12:31:15Z","receivedAt":"2009-08-17T12:31:15Z","isPatch":true,"sender":{"key":"sirnot@gmail.com","avatar":null},"body":"SUGGESTED FOR 'PU':\n\nTraversing objects is currently very costly, as every commit and tree must be \nloaded and parsed.  Much time and energy could be saved by caching metadata and \ntopological info in an efficient, easily accessible manner.  Furthermore, this \ncould improve git's interfacing potential, by providing a condensed summary of \na repository's commit tree.\n\nThis is a series to implement such a revision caching mechanism, aptly named \nrev-cache.  The series will provide:\n - a core API to manipulate and traverse caches\n - an integration into the internal revision walker\n - a porcelain front-end providing access to users and (shell) applications\n - a series of tests to verify/demonstrate correctness\n - documentation of the API, porcelain and core concepts\n\nIn cold starts rev-cache has sped up packing and walking by a factor of 4, and \nover twice that on warm starts.  Some times on slax for the linux repository:\n\nrev-list --all --objects >/dev/null\n default\n   cold    1:13\n   warm    0:43\n rev-cache'd\n   cold    0:19\n   warm    0:02\n\npack-objects --revs --all --stdout >/dev/null\n default\n   cold    2:44\n   warm    1:21\n rev-cache'd\n   cold    0:44\n   warm    0:10\n\nThe mechanism is minimally intrusive: most of the changes take place in \nseperate files, and only a handful of git's existing functions are modified.\n\nHope you find this useful.\n\n - Nick\n---\nBOMBS AWAY!!!\n\nTo clarify my magnificent muddying of this final update I'm flash flooding y'all with a mint patchset!  This probably isn't deserving of it's own version, but it's the last one and I've confuddled everyone enough already.\n\nSO, what's changed from the last one?\n - fixed whitespace errors\n - cleaned up header definitions\n - implemented _ondisk versions of structs for ensured portability\n - fixed typo\n\nSorry again for the size of the patches, but like I said earlier they're not so much patches as files -- the changes in existing git files are actually pretty small.  Since most everything in the patches is intertwined I couldn't split it up a bunch more without having non-functioning patches.\n\nI promise this is the last one :-P\n\n Documentation/git-rev-cache.txt       |  144 ++\n Documentation/technical/rev-cache.txt |  614 +++++++++\n Makefile                              |    2 +\n builtin-gc.c                          |    9 +\n builtin-rev-cache.c                   |  322 +++++\n builtin.h                             |    1 +\n commit.c                              |    2 +\n git.c                                 |    1 +\n list-objects.c                        |   49 +-\n rev-cache.c                           | 2314 +++++++++++++++++++++++++++++++++\n rev-cache.h                           |  124 ++\n revision.c                            |   90 +-\n revision.h                            |   44 +-\n t/t6015-rev-cache-list.sh             |  251 ++++\n 14 files changed, 3942 insertions(+), 25 deletions(-)\n create mode 100644 Documentation/git-rev-cache.txt\n create mode 100644 Documentation/technical/rev-cache.txt\n create mode 100644 builtin-rev-cache.c\n create mode 100644 rev-cache.c\n create mode 100644 rev-cache.h\n create mode 100755 t/t6015-rev-cache-list.sh\n"},{"id":"121083","messageId":"op.uyuwkkq9tdk399@sirnot.private","threadId":"20627","inReplyTo":"op.uys3qdnntdk399@sirnot.private","subject":"Re: [PATCH 0/6 (v4)] revision caching mechanism to significantly speed up packing/walking (last version)","fromName":"Nick Edelen","fromEmail":"sirnot@gmail.com","sentAt":"2009-08-18T11:51:46Z","receivedAt":"2009-08-18T11:51:46Z","isPatch":true,"sender":{"key":"sirnot@gmail.com","avatar":null},"body":"SUGGESTED FOR 'PU':\n\nTraversing objects is currently very costly, as every commit and tree must be\nloaded and parsed.  Much time and energy could be saved by caching metadata and\ntopological info in an efficient, easily accessible manner.  Furthermore, this\ncould improve git's interfacing potential, by providing a condensed summary of\na repository's commit tree.\n\nThis is a series to implement such a revision caching mechanism, aptly named\nrev-cache.  The series will provide:\n - a core API to manipulate and traverse caches\n - an integration into the internal revision walker\n - a porcelain front-end providing access to users and (shell) applications\n - a series of tests to verify/demonstrate correctness\n - documentation of the API, porcelain and core concepts\n\nIn cold starts rev-cache has sped up packing and walking by a factor of 4, and\nover twice that on warm starts.  Some times on slax for the linux repository:\n\nrev-list --all --objects >/dev/null\n default\n   cold    1:13\n   warm    0:43\n rev-cache'd\n   cold    0:19\n   warm    0:02\n\npack-objects --revs --all --stdout >/dev/null\n default\n   cold    2:44\n   warm    1:21\n rev-cache'd\n   cold    0:44\n   warm    0:10\n\nThe mechanism is minimally intrusive: most of the changes take place in\nseperate files, and only a handful of git's existing functions are modified.\n\nHope you find this useful.\n\n - Nick\n\n---\nOk, this really is the last time I send all these.\n\n Documentation/git-rev-cache.txt       |  162 +++\n Documentation/technical/rev-cache.txt |  614 +++++++++\n Makefile                              |    2 +\n builtin-gc.c                          |    9 +\n builtin-rev-cache.c                   |  323 +++++\n builtin.h                             |    1 +\n commit.c                              |    2 +\n git.c                                 |    1 +\n list-objects.c                        |   49 +-\n rev-cache.c                           | 2316 +++++++++++++++++++++++++++++++++\n rev-cache.h                           |  124 ++\n revision.c                            |   90 +-\n revision.h                            |   44 +-\n t/t6015-rev-cache-list.sh             |  251 ++++\n 14 files changed, 3963 insertions(+), 25 deletions(-)\n create mode 100644 Documentation/git-rev-cache.txt\n create mode 100644 Documentation/technical/rev-cache.txt\n create mode 100644 builtin-rev-cache.c\n create mode 100644 rev-cache.c\n create mode 100644 rev-cache.h\n create mode 100755 t/t6015-rev-cache-list.sh\n"}]}