{"thread":{"id":"3722","subject":"[PATCH] Add git-explode-packs","startedAt":"2006-03-25T12:02:24Z","lastAt":"2006-03-27T03:53:31Z","messageCount":4,"participants":["Martin Atukunda","Junio C Hamano","Jan-Benedict Glaw"],"isPatch":true,"patchVersion":1,"patchTotal":null},"messages":[{"id":"17934","messageId":"11432881443149-git-send-email-matlads@dsmagic.com","threadId":"3722","inReplyTo":null,"subject":"[PATCH] Add git-explode-packs","fromName":"Martin Atukunda","fromEmail":"matlads@dsmagic.com","sentAt":"2006-03-25T12:02:24Z","receivedAt":"2006-03-25T12:02:24Z","isPatch":true,"sender":{"key":"matlads@dsmagic.com","avatar":null},"body":"This script does the opposite of git repack -a -d.\n\nSigned-Off-By: Martin Atukunda \n\n---\n\n .gitignore | 1 +\n Documentation/git-explode-packs.txt | 45 +++++++++++++++++++++++++++++++++++\n Makefile | 2 +-\n git-explode-packs.sh | 26 ++++++++++++++++++++\n 4 files changed, 73 insertions(+), 1 deletions(-)\n create mode 100644 Documentation/git-explode-packs.txt\n create mode 100755 git-explode-packs.sh\n\n277352dd9a0549cd626242b14454da37acbc72f3\ndiff --git a/.gitignore b/.gitignore\nindex b4355b9..0ac74e3 100644\n--- a/.gitignore\n+++ b/.gitignore\n@@ -133,3 +133,4 @@ libgit.a\n *.py[co]\n config.mak\n git-blame\n+git-explode-packs\ndiff --git a/Documentation/git-explode-packs.txt b/Documentation/git-explode-packs.txt\nnew file mode 100644\nindex 0000000..9651a4e\n--- /dev/null\n+++ b/Documentation/git-explode-packs.txt\n@@ -0,0 +1,45 @@\n+git-explode-packs(1)\n+====================\n+\n+NAME\n+----\n+git-explode-packs - Script used to explode packs in a repository into objects\n+\n+\n+SYNOPSIS\n+--------\n+'git-explode-packs'\n+\n+DESCRIPTION\n+-----------\n+\n+This script is used to explode all packs into the constituent objects.\n+\n+A pack is a collection of objects, individually compressed, with\n+delta compression applied, stored in a single file, with an\n+associated index file.\n+\n+Packs are used to reduce the load on mirror systems, backup\n+engines, disk storage, etc.\n+\n+This script removes all the packs in the repository, replacing them with the\n+objects that are stored inside them.\n+\n+Author\n+------\n+Written by Martin Atukunda \n+\n+Documentation\n+--------------\n+Documentation by Martin Atukunda \n+\n+See Also\n+--------\n+gitlink:git-pack-objects[1]\n+gitlink:git-prune-packed[1]\n+gitlink:git-repack[1]\n+\n+GIT\n+---\n+Part of the gitlink:git[7] suite\n+\ndiff --git a/Makefile b/Makefile\nindex 8d45378..71e31f0 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -125,7 +125,7 @@ SCRIPT_SH = \\\n \tgit-applymbox.sh git-applypatch.sh git-am.sh \\\n \tgit-merge.sh git-merge-stupid.sh git-merge-octopus.sh \\\n \tgit-merge-resolve.sh git-merge-ours.sh git-grep.sh \\\n-\tgit-lost-found.sh\n+\tgit-lost-found.sh git-explode-packs.sh\n \n SCRIPT_PERL = \\\n \tgit-archimport.perl git-cvsimport.perl git-relink.perl \\\ndiff --git a/git-explode-packs.sh b/git-explode-packs.sh\nnew file mode 100755\nindex 0000000..a7e9761\n--- /dev/null\n+++ b/git-explode-packs.sh\n@@ -0,0 +1,26 @@\n+#!/bin/sh\n+#\n+# Copyright (c) 2006 Martin Atukunda\n+#\n+\n+USAGE=''\n+. git-sh-setup\n+\n+PACKDIR=\"$GIT_OBJECT_DIRECTORY/pack\"\n+PRESDIR=\"./++preserve\"\n+\n+mkdir \"$PRESDIR\" && (\n+\tfor p in \"$GIT_OBJECT_DIRECTORY\"/pack/pack-*.pack; do\n+\t\tif test -f \"$p\"; then\n+\t\t\tmv \"$p\" \"$PRESDIR\"\n+\t\tfi\n+\tdone\n+\n+\tfor p in \"$PRESDIR\"/pack-*.pack; do\n+\t\tif test -f \"$p\"; then\n+\t\t\tgit-unpack-objects <$p\n+\t\t\trm -- $p\n+\t\tfi\n+\tdone\n+\trmdir \"$PRESDIR\"\n+)\n-- \n1.2.4.gd3e1\n"},{"id":"17980","messageId":"7vwtehpwdd.fsf@assigned-by-dhcp.cox.net","threadId":"3722","inReplyTo":"11432881443149-git-send-email-matlads@dsmagic.com","subject":"Re: [PATCH] Add git-explode-packs","fromName":"Junio C Hamano","fromEmail":"junkio@cox.net","sentAt":"2006-03-26T06:12:46Z","receivedAt":"2006-03-26T06:12:46Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Martin Atukunda writes:\n\n> This script does the opposite of git repack -a -d.\n\nThe script seems to do what it claims to, but now why would one\nneed to use this? In other words what's the situation one would\nfind this useful?\n"},{"id":"17989","messageId":"20060326125450.GT31387@lug-owl.de","threadId":"3722","inReplyTo":"7vwtehpwdd.fsf@assigned-by-dhcp.cox.net","subject":"Re: [PATCH] Add git-explode-packs","fromName":"Jan-Benedict Glaw","fromEmail":"jbglaw@lug-owl.de","sentAt":"2006-03-26T12:54:50Z","receivedAt":"2006-03-26T12:54:50Z","isPatch":true,"sender":{"key":"jbglaw@lug-owl.de","avatar":null},"body":"On Sat, 2006-03-25 22:12:46 -0800, Junio C Hamano wrote:\n> Martin Atukunda writes:\n> > This script does the opposite of git repack -a -d.\n> \n> The script seems to do what it claims to, but now why would one\n> need to use this? In other words what's the situation one would\n> find this useful?\n\nIt's possibly useful if you oftenly access old objects with\ngit-cat-file or git-ls-tree.\n\nNot being a Perl hacker, a friend and I eg. started to hack GIT\nsupport into LXR. I've just posted some very early patches on the LXR\nmailing list\n(http://sourceforge.net/mailarchive/forum.php?forum_id=1734). What\nwould be even more interesting is to not unpack _all_ objects, but\nonly those belonging to specifically mentioned commits or tags. I\nthink LXR could make _good_ use of that.\n\nMfG, JBG\n\n-- \nJan-Benedict Glaw jbglaw@lug-owl.de . +49-172-7608481 _ O _\n\"Eine Freie Meinung in einem Freien Kopf | Gegen Zensur | Gegen Krieg _ _ O\n für einen Freien Staat voll Freier Bürger\" | im Internet! | im Irak! O O O\nret = do_actions((curr | FREE_SPEECH) & ~(NEW_COPYRIGHT_LAW | DRM | TCPA));\n"},{"id":"18021","messageId":"7vd5g8pmpw.fsf@assigned-by-dhcp.cox.net","threadId":"3722","inReplyTo":"20060326125450.GT31387@lug-owl.de","subject":"Re: [PATCH] Add git-explode-packs","fromName":"Junio C Hamano","fromEmail":"junkio@cox.net","sentAt":"2006-03-27T03:53:31Z","receivedAt":"2006-03-27T03:53:31Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jan-Benedict Glaw writes:\n\n> On Sat, 2006-03-25 22:12:46 -0800, Junio C Hamano wrote:\n>> The script seems to do what it claims to, but now why would one\n>> need to use this? In other words what's the situation one would\n>> find this useful?\n>\n> It's possibly useful if you oftenly access old objects with\n> git-cat-file or git-ls-tree.\n\nBenchmarks?\n\nI created two cloned repositories from git.git. victim03\nrepository is fully packed with the default pack parameter of\ndepth and window set both to 10. victim04 repository has the\nsame set of objects and refs but the pack is expanded (16232\nloose objects).\n\nNow in victim03 repository, 657 blobs have depth 10 (i.e. you\nneed to inflate and apply delta 10 times to get to the object).\nSo I made the list of these \"expensive to access\" objects and\nrun this:\n\n\t$ cd victim03\n\t$ /usr/bin/time sh -c '\n while read sha1; do git cat-file blob $sha1;\n done >/dev/null /dev/null <../victim03/list\n\t'\n\n3.29user 2.98system 0:06.33elapsed 98%CPU (0avgtext+0avgdata 0maxresident)k\n0inputs+0outputs (0major+348786minor)pagefaults 0swaps\n3.26user 2.88system 0:06.63elapsed 92%CPU (0avgtext+0avgdata 0maxresident)k\n0inputs+0outputs (0major+347512minor)pagefaults 0swaps\n3.16user 2.98system 0:06.20elapsed 99%CPU (0avgtext+0avgdata 0maxresident)k\n0inputs+0outputs (0major+347489minor)pagefaults 0swaps\n\nSo you are getting slight performance gain out of this by\nexploding the pack, but on the other hand you are taxing the\nbuffer cache quite heavily by reading the loose objects (in both\nof the experiments above, I discarded numbers from the very\nfirst run). The size of object databases in these cases are:\n\n $ du -sh victim0[34]/.git/objects\n 6.2M victim03/.git/objects\n 84M victim04/.git/objects\n\nSo I am still not convinced it would be useful in general. It\nused to be that exploding everything and repacking was the only\nway to clean out garbage from packs, but after \"repack -a -d\"\nwas invented by Frank Sorenson that became more convenient way.\nEspecially with the recent \"delta reusing\" pack-objects, doing\n\"repack -a -d\" has become quite cheap, so...\n"}]}