{"thread":{"id":"16801","subject":"How to extract files out of a \"git bundle\", no matter what?","startedAt":"2008-12-19T19:29:00Z","lastAt":"2009-02-04T22:44:42Z","messageCount":32,"participants":["jidanni@jidanni.org","Shawn O. Pearce","Mark Levedahl","Junio C Hamano","Jeff King","Johannes Schindelin","Alex Riesen","Nanako Shiraishi"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"98366","messageId":"87iqpgc6bn.fsf@jidanni.org","threadId":"16801","inReplyTo":null,"subject":"How to extract files out of a \"git bundle\", no matter what?","fromName":"","fromEmail":"jidanni@jidanni.org","sentAt":"2008-12-19T19:29:00Z","receivedAt":"2008-12-19T19:29:00Z","isPatch":false,"sender":{"key":"jidanni@jidanni.org","avatar":"https://gravatar.com/avatar/36568d4af4c8d3e71627ef3b8c8d00e39065b12f29676cccd38ced75e68fa2a6?d=mp&s=160"},"body":"Someone has handed you a \"git bundle\".\nHow do you get the files out of it?\nIf it were cpio, you would use -i, if it were tar, you would use -x...\nYou read the git-bundle man page.\nYou only get as far as\n# git-bundle verify bundle.bdl\nThe bundle contains 1 ref\nd01... /heads/master\nThe bundle requires these 0 ref\nbundle.bdl is okay\n\nThe rest is mish-mosh. There should be an emergency example for non\ngit club members, even starting from apt-get install git-core, of the\nall the real steps needed _to get the files out of the bundle_.\n\nAssume the user _just wants to get the files out of the bundle_ and\nnot learn about or participate in some project.\n"},{"id":"98367","messageId":"20081219193256.GU32487@spearce.org","threadId":"16801","inReplyTo":"87iqpgc6bn.fsf@jidanni.org","subject":"Re: How to extract files out of a \"git bundle\", no matter what?","fromName":"Shawn O. Pearce","fromEmail":"spearce@spearce.org","sentAt":"2008-12-19T19:32:56Z","receivedAt":"2008-12-19T19:32:56Z","isPatch":false,"sender":{"key":"spearce@spearce.org","avatar":"https://avatars.githubusercontent.com/u/34844?v=4"},"body":"jidanni@jidanni.org wrote:\n> Someone has handed you a \"git bundle\".\n> How do you get the files out of it?\n> If it were cpio, you would use -i, if it were tar, you would use -x...\n> You read the git-bundle man page.\n> You only get as far as\n> # git-bundle verify bundle.bdl\n> The bundle contains 1 ref\n> d01... /heads/master\n> The bundle requires these 0 ref\n> bundle.bdl is okay\n> \n> The rest is mish-mosh. There should be an emergency example for non\n> git club members, even starting from apt-get install git-core, of the\n> all the real steps needed _to get the files out of the bundle_.\n> \n> Assume the user _just wants to get the files out of the bundle_ and\n> not learn about or participate in some project.\n\nYou can't just \"get the files out\".  A bundle contains deltas,\nwhere you need the base in order to recreate the file content.\nIt can't be unpacked in a vacuum.\n\nTo unpack a bundle you need to clone the project and then fetch\nfrom it:\n\n\tgit clone src...\n\tgit pull bundle.bdl master\n\nIf the bundle requires 0 refs (like above) then you can init a\nnew repository and should be able to fetch from it:\n\n\tgit init\n\tgit pull bundle.bdl master\n\n-- \nShawn.\n"},{"id":"98368","messageId":"494BFCAF.9060703@verizon.net","threadId":"16801","inReplyTo":"20081219193256.GU32487@spearce.org","subject":"Re: How to extract files out of a \"git bundle\", no matter what?","fromName":"Mark Levedahl","fromEmail":"mdl123@verizon.net","sentAt":"2008-12-19T19:57:35Z","receivedAt":"2008-12-19T19:57:35Z","isPatch":false,"sender":{"key":"mdl123@verizon.net","avatar":"https://avatars.githubusercontent.com/u/5302462?v=4"},"body":"Shawn O. Pearce wrote:\n>\n> If the bundle requires 0 refs (like above) then you can init a\n> new repository and should be able to fetch from it:\n>\n> \tgit init\n> \tgit pull bundle.bdl master\n>\n>   \n\nWith relatively recent git (not sure the version), you can just do\n\n    git clone bundle.bdl\n\nMark\n"},{"id":"98369","messageId":"7vtz906iaa.fsf@gitster.siamese.dyndns.org","threadId":"16801","inReplyTo":"87iqpgc6bn.fsf@jidanni.org","subject":"Re: How to extract files out of a \"git bundle\", no matter what?","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2008-12-19T20:07:09Z","receivedAt":"2008-12-19T20:07:09Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"jidanni@jidanni.org writes:\n\n> Someone has handed you a \"git bundle\".\n> How do you get the files out of it?\n> If it were cpio, you would use -i, if it were tar, you would use -x...\n> You read the git-bundle man page.\n> You only get as far as\n> # git-bundle verify bundle.bdl\n> The bundle contains 1 ref\n> d01... /heads/master\n> The bundle requires these 0 ref\n> bundle.bdl is okay\n>\n> The rest is mish-mosh.\n\nThe last example in the git-bundle man page might be a bit cryptic but\nthat is how bundles are expected to be used.  To give people repository\naccess who do not have real network connection other than Sneakernet.\n\nFor one shot extraction, defining a remote in the config is overkill and\nyou could just say:\n\n\tgit ls-remote bundle.bdl\n\nto see what branches it contains and if you are interested in its\nmaster branch and want to merge it to your history, then\n\n\tgit pull bundle.bdl master\n\nshould do that.\n"},{"id":"98370","messageId":"87zlirc49l.fsf@jidanni.org","threadId":"16801","inReplyTo":"494BFCAF.9060703@verizon.net","subject":"Re: How to extract files out of a \"git bundle\", no matter what?","fromName":"","fromEmail":"jidanni@jidanni.org","sentAt":"2008-12-19T20:13:26Z","receivedAt":"2008-12-19T20:13:26Z","isPatch":false,"sender":{"key":"jidanni@jidanni.org","avatar":"https://gravatar.com/avatar/36568d4af4c8d3e71627ef3b8c8d00e39065b12f29676cccd38ced75e68fa2a6?d=mp&s=160"},"body":"SOP> If the bundle requires 0 refs (like above) then you can init a\nSOP> new repository and should be able to fetch from it:\n\nSOP> \tgit init\nSOP> \tgit pull bundle.bdl master\n\nPhew, that worked. Thank you!\n\nML> With relatively recent git (not sure the version), you can just do\nML>    git clone bundle.bdl\nNot with git version 1.5.6.5, Debian sid.\n\nAnyway, for man page completeness, I still see the day when:\n\nSOP> You can't just \"get the files out\".  A bundle contains deltas,\nSOP> where you need the base in order to recreate the file content.\nSOP> It can't be unpacked in a vacuum.\n\nThat is nice by we here at the forensics department of XYZ police\nforce just need to get the files out. We tried \"PK UNZIP\" but that\ndidn't extract them. We contacted the Computer Science Dept. but\nthat's who they're holding hostage.\n\nSOP> To unpack a bundle you need to clone the project and then fetch\nSOP> from it:\n\nSOP> \tgit clone src...\nSOP> \tgit pull bundle.bdl master\n\nThat is nice but the perpetrators have destroyed everything except for\nthat one bundle.bdl file, which contains the password to defuse the\ntime bomb.\n\nThere must be a way to make a \"phony tree\" or whatever to \"attach to\"\nso extraction can proceed. Be sure to spell it all out on the\ngit-bundle man page as a reference in case some non-computer people\nneed to do aforementioned emergency extraction one day.\n"},{"id":"98371","messageId":"20081219202118.GA26513@coredump.intra.peff.net","threadId":"16801","inReplyTo":"87zlirc49l.fsf@jidanni.org","subject":"Re: How to extract files out of a \"git bundle\", no matter what?","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2008-12-19T20:21:19Z","receivedAt":"2008-12-19T20:21:19Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Sat, Dec 20, 2008 at 04:13:26AM +0800, jidanni@jidanni.org wrote:\n\n> There must be a way to make a \"phony tree\" or whatever to \"attach to\"\n> so extraction can proceed. Be sure to spell it all out on the\n> git-bundle man page as a reference in case some non-computer people\n> need to do aforementioned emergency extraction one day.\n\nNo, that information may not even be in the bundle at all (unless it is\na bundle that has a 0-ref basis). In particular, if a bundle contains\nchanges between some commit A and some commit B, then:\n\n  - files that were not changed between A and B will not be included at\n    all\n\n  - the object pack in the bundle is \"thin\", meaning it may contain\n    deltas against objects that are reachable from A, but not B. So even\n    _within_ a changed file, you may see only the changes from A to B.\n\nIf the bundle has a 0-ref basis, then you can clone straight from the\nbundle, which must have everything.\n\n-Peff\n"},{"id":"98372","messageId":"87vdtfc389.fsf@jidanni.org","threadId":"16801","inReplyTo":"20081219202118.GA26513@coredump.intra.peff.net","subject":"Re: How to extract files out of a \"git bundle\", no matter what?","fromName":"","fromEmail":"jidanni@jidanni.org","sentAt":"2008-12-19T20:35:50Z","receivedAt":"2008-12-19T20:35:50Z","isPatch":false,"sender":{"key":"jidanni@jidanni.org","avatar":"https://gravatar.com/avatar/36568d4af4c8d3e71627ef3b8c8d00e39065b12f29676cccd38ced75e68fa2a6?d=mp&s=160"},"body":"JK> In particular, if a bundle contains changes between some commit A\nJK> and some commit B, then:\n\nJK>   - files that were not changed between A and B will not be included at\nJK>     all\n\nJK>   - the object pack in the bundle is \"thin\", meaning it may contain\nJK>     deltas against objects that are reachable from A, but not B. So even\nJK>     _within_ a changed file, you may see only the changes from A to B.\n\nOK, we here at the police forensics department would be very happy if\nwe could at least get some ASCII out of that .BDL file, even if it is\njust a diff shred,\n-       The password to the time bomb was BLORFZ\n+       The password to the time bomb is  NORFLZ\nthat would be fine. All we know is after the work PACK it is all\nbinary, and git-unpack-objects and git-unpack-file don't work on it.\n"},{"id":"98374","messageId":"20081219205100.GA26671@coredump.intra.peff.net","threadId":"16801","inReplyTo":"87vdtfc389.fsf@jidanni.org","subject":"Re: How to extract files out of a \"git bundle\", no matter what?","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2008-12-19T20:51:00Z","receivedAt":"2008-12-19T20:51:00Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Sat, Dec 20, 2008 at 04:35:50AM +0800, jidanni@jidanni.org wrote:\n\n> JK>   - the object pack in the bundle is \"thin\", meaning it may contain\n> JK>     deltas against objects that are reachable from A, but not B. So even\n> JK>     _within_ a changed file, you may see only the changes from A to B.\n> \n> OK, we here at the police forensics department would be very happy if\n> we could at least get some ASCII out of that .BDL file, even if it is\n> just a diff shred,\n> -       The password to the time bomb was BLORFZ\n> +       The password to the time bomb is  NORFLZ\n> that would be fine. All we know is after the work PACK it is all\n> binary, and git-unpack-objects and git-unpack-file don't work on it.\n\nAFAIK, there is no tool to try salvaging strings from an incomplete pack\n(and you can't just run \"strings\" because the deltas are zlib\ncompressed). So if I were in the police forensics department, I think I\nwould read Documentation/technical/pack-format.txt and start hacking a\nsolution as quickly as possible.\n\n-Peff\n"},{"id":"99076","messageId":"87prj7mz50.fsf_-_@jidanni.org","threadId":"16801","inReplyTo":"20081219205100.GA26671@coredump.intra.peff.net","subject":"[PATCH] Documentation/git-bundle.txt: Dumping contents of any bundle","fromName":"","fromEmail":"jidanni@jidanni.org","sentAt":"2009-01-01T04:24:59Z","receivedAt":"2009-01-01T04:24:59Z","isPatch":true,"sender":{"key":"jidanni@jidanni.org","avatar":"https://gravatar.com/avatar/36568d4af4c8d3e71627ef3b8c8d00e39065b12f29676cccd38ced75e68fa2a6?d=mp&s=160"},"body":">>>>> \"JK\" == Jeff King <peff@peff.net> writes:\n\nJK> AFAIK, there is no tool to try salvaging strings from an incomplete pack\nJK> (and you can't just run \"strings\" because the deltas are zlib\nJK> compressed). So if I were in the police forensics department, I think I\nJK> would read Documentation/technical/pack-format.txt and start hacking a\nJK> solution as quickly as possible.\n\nHogwash. Patch follows. Maybe even better methods are available.\n\nSigned-off-by: jidanni <jidanni@jidanni.org>\n---\n Documentation/git-bundle.txt |   22 ++++++++++++++++++++++\n 1 files changed, 22 insertions(+), 0 deletions(-)\n\ndiff --git a/Documentation/git-bundle.txt b/Documentation/git-bundle.txt\nindex 1b66ab7..80248f5 100644\n--- a/Documentation/git-bundle.txt\n+++ b/Documentation/git-bundle.txt\n@@ -164,6 +164,28 @@ $ git pull bundle\n would treat it as if it is talking with a remote side over the\n network.\n \n+DUMPING CONTENTS OF ANY BUNDLE\n+-----------------------\n+\n+Even if we cannot unbundle a bundle,\n+\n+------------\n+$ git init\n+$ git bundle unbundle mybundle.bun\n+error: Repository lacks these prerequisite commits...\n+------------\n+\n+We can still examine all the data contained within,\n+\n+------------\n+$ sed '/^PACK/,$!d' mybundle.bun > mybundle.pack\n+$ git unpack-objects < mybundle.pack\n+$ cd .git/objects\n+$ ls ??/*|tr -d /|git cat-file --batch-check\n+$ ls ??/*|tr -d /|git cat-file --batch\n+------------\n+\n+\n Author\n ------\n Written by Mark Levedahl <mdl123@verizon.net>\n-- \n1.6.0.6\n"},{"id":"99090","messageId":"alpine.DEB.1.00.0901011802350.30769@pacific.mpi-cbg.de","threadId":"16801","inReplyTo":"87prj7mz50.fsf_-_@jidanni.org","subject":"Re: [PATCH] Documentation/git-bundle.txt: Dumping contents of any bundle","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2009-01-01T17:03:25Z","receivedAt":"2009-01-01T17:03:25Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi,\n\nOn Thu, 1 Jan 2009, jidanni@jidanni.org wrote:\n\n> >>>>> \"JK\" == Jeff King <peff@peff.net> writes:\n> \n> JK> AFAIK, there is no tool to try salvaging strings from an incomplete pack\n> JK> (and you can't just run \"strings\" because the deltas are zlib\n> JK> compressed). So if I were in the police forensics department, I think I\n> JK> would read Documentation/technical/pack-format.txt and start hacking a\n> JK> solution as quickly as possible.\n> \n> Hogwash. Patch follows. Maybe even better methods are available.\n> \n> Signed-off-by: jidanni <jidanni@jidanni.org>\n> ---\n\nJust for the record: this is in so many ways not a commit message I want \nto have in git.git.  I hope it is not applied.\n\nCiao,\nDscho\n"},{"id":"99096","messageId":"20090101192153.GA6536@coredump.intra.peff.net","threadId":"16801","inReplyTo":"87prj7mz50.fsf_-_@jidanni.org","subject":"Re: [PATCH] Documentation/git-bundle.txt: Dumping contents of any bundle","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2009-01-01T19:21:54Z","receivedAt":"2009-01-01T19:21:54Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Thu, Jan 01, 2009 at 12:24:59PM +0800, jidanni@jidanni.org wrote:\n\n> JK> AFAIK, there is no tool to try salvaging strings from an incomplete pack\n> JK> (and you can't just run \"strings\" because the deltas are zlib\n> JK> compressed). So if I were in the police forensics department, I think I\n> JK> would read Documentation/technical/pack-format.txt and start hacking a\n> JK> solution as quickly as possible.\n> \n> Hogwash. Patch follows. Maybe even better methods are available.\n> [...]\n> +$ sed '/^PACK/,$!d' mybundle.bun > mybundle.pack\n> +$ git unpack-objects < mybundle.pack\n> +$ cd .git/objects\n> +$ ls ??/*|tr -d /|git cat-file --batch-check\n> +$ ls ??/*|tr -d /|git cat-file --batch\n\nSorry, no, but your method does not work in the case I described: a thin\npack with deltas. In that case, git unpack-objects cannot unpack the\nobject since it lacks the delta, and will skip it. For example:\n\n  # create a bundle with a thin delta blob\n  mkdir one && cd one && git init\n  cp /usr/share/dict/words . && git add words && git commit -m one\n  echo SECRET MESSAGE >>words && git add words && git commit -m two\n  git bundle create ../mybundle.bun HEAD^..\n\n  # now try to fetch from it\n  mkdir ../two && cd ../two && git init\n  git bundle unbundle ../mybundle.bun\n  # produces:\n  # error: Repository lacks these prerequisite commits:\n  # error: b7d1a0ca98ca0e997d4222459d6fc1c9edae6a3f one\n\n  # so try to recover\n  sed '/^PACK/,$!d' ../mybundle.bun > mybundle.pack\n  git unpack-objects < mybundle.pack\n  # Unpacking objects: 100% (3/3), done.\n  # fatal: unresolved deltas left after unpacking\n  cd .git/objects\n  # this will show just two objects: the commit and the tree\n  ls ??/* | tr -d /\n  # confirm that we don't have the blob or the string of interest\n  ls ??/* | tr -d / | git cat-file --batch | grep SECRET\n\nIt is nice that unpack-objects continues at all thanks to the recent\nimprovements by Nicolas, so you may be able to get some of the data out.\nBut it just skips over any unresolvable deltas, since we can't make a\nuseful object from them. Maybe it would be worth adding an option to\ndump the uncompressed deltas to a file or directory so you could run\n\"strings\" on them to recover some of the data.\n\n-Peff\n"},{"id":"99120","messageId":"87fxk2u13r.fsf@jidanni.org","threadId":"16801","inReplyTo":"20090101192153.GA6536@coredump.intra.peff.net","subject":"Re: [PATCH] Documentation/git-bundle.txt: Dumping contents of any bundle","fromName":"","fromEmail":"jidanni@jidanni.org","sentAt":"2009-01-01T22:12:56Z","receivedAt":"2009-01-01T22:12:56Z","isPatch":true,"sender":{"key":"jidanni@jidanni.org","avatar":"https://gravatar.com/avatar/36568d4af4c8d3e71627ef3b8c8d00e39065b12f29676cccd38ced75e68fa2a6?d=mp&s=160"},"body":"JK> Maybe it would be worth adding an option to dump the uncompressed\nJK> deltas to a file or directory so you could run \"strings\" on them\nJK> to recover some of the data.\n\nI got as far as these wheezy little bytes,\n$ ls ??/*|tr -d /|sed q|xargs git cat-file tree|perl -pwe 's/[^\\0]+[\\0]//'|hd\n00000000  ae 83 2f 22 45 89 2d dd  e5 22 13 57 46 64 48 b4  |../\"E.-..\".WFdH.|\n00000010  09 77 51 42                                       |.wQB|\nbefore I ran out of tools to crack it. It must be in some standard git\ngzip format. There should be a command line tool to crack it with\nprovided in the git suite.\n\nAnyways, one day some forensics department will need to crack one of\nthese things, and I want the instructions available.\n\nJS> Just for the record: this is in so many ways not a commit message I want\nJS> to have in git.git.  I hope it is not applied.\nIs that where they end up? Oops, please reword it for me, anybody.\n"},{"id":"99126","messageId":"87eizmty25.fsf_-_@jidanni.org","threadId":"16801","inReplyTo":"20090101192153.GA6536@coredump.intra.peff.net","subject":"git ls-tree prints wacko file sizes if it can't find the blob","fromName":"","fromEmail":"jidanni@jidanni.org","sentAt":"2009-01-01T23:18:42Z","receivedAt":"2009-01-01T23:18:42Z","isPatch":false,"sender":{"key":"jidanni@jidanni.org","avatar":"https://gravatar.com/avatar/36568d4af4c8d3e71627ef3b8c8d00e39065b12f29676cccd38ced75e68fa2a6?d=mp&s=160"},"body":"git ls-tree prints wacko file sizes if it can't find the blob:\n$ git ls-tree --abbrev=4 -l 76e4\nerror: unable to find ae832f2245892ddde5221357466448b409775142\n100644 blob ae83 3220821896     words\n\nIt is even affected by --abbrev:\n$ for i in 4 5 40 999; do git ls-tree --abbrev=$i -l 76e4; done 2>&-|\nperl -nwale 'print $F[3]'\n3214344536\n3219092952\n3216251688\n3217198088\n$ git version\ngit version 1.6.0.6\n"},{"id":"99128","messageId":"87d4f6twq7.fsf_-_@jidanni.org","threadId":"16801","inReplyTo":"87eizmty25.fsf_-_@jidanni.org","subject":"Re: git ls-tree prints wacko file sizes if it can't find the blob","fromName":"","fromEmail":"jidanni@jidanni.org","sentAt":"2009-01-01T23:47:28Z","receivedAt":"2009-01-01T23:47:28Z","isPatch":false,"sender":{"key":"jidanni@jidanni.org","avatar":"https://gravatar.com/avatar/36568d4af4c8d3e71627ef3b8c8d00e39065b12f29676cccd38ced75e68fa2a6?d=mp&s=160"},"body":"> It is even affected by --abbrev:\nIt's not. It is just randomly grabbing digits even without --abbrev.\n"},{"id":"99129","messageId":"20090101234815.GA9049@coredump.intra.peff.net","threadId":"16801","inReplyTo":"87fxk2u13r.fsf@jidanni.org","subject":"Re: [PATCH] Documentation/git-bundle.txt: Dumping contents of any bundle","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2009-01-01T23:48:15Z","receivedAt":"2009-01-01T23:48:15Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Fri, Jan 02, 2009 at 06:12:56AM +0800, jidanni@jidanni.org wrote:\n\n> I got as far as these wheezy little bytes,\n> $ ls ??/*|tr -d /|sed q|xargs git cat-file tree|perl -pwe 's/[^\\0]+[\\0]//'|hd\n> 00000000  ae 83 2f 22 45 89 2d dd  e5 22 13 57 46 64 48 b4  |../\"E.-..\".WFdH.|\n> 00000010  09 77 51 42                                       |.wQB|\n\nThose are just the bytes of the sha1 of the blob object, which is\npointed to by the tree object. You have the tree object correctly\nunpacked, but not the blob, as I said before. So no amount of looking\nin .git/objects is going to help you: git-unpack-objects didn't unpack\nit, and the data isn't there in any form.\n\nThe data is in the pack, but as a delta, and that delta has further been\ngzipped. So you can either write a custom parser based on the pack\nformat (which, as I mentioned, is described in\nDocumentation/technical/pack-format.txt), or you can add a switch to\nunpack-objects, which is already parsing that format, to dump the\nunresolved deltas. Which is what I was suggesting before.\n\nHere's a very rough patch to do the latter. Try:\n\n  git unpack-objects --dump-delta <mybundle.pack\n  strings .git/lost-found/delta/*\n\nProbably one could also write some tool to decode the delta format into\nsomething more human readable.\n\n---\ndiff --git a/builtin-unpack-objects.c b/builtin-unpack-objects.c\nindex 47ed610..ab33ab1 100644\n--- a/builtin-unpack-objects.c\n+++ b/builtin-unpack-objects.c\n@@ -13,6 +13,7 @@\n #include \"fsck.h\"\n \n static int dry_run, quiet, recover, has_errors, strict;\n+static int dump_deltas;\n static const char unpack_usage[] = \"git unpack-objects [-n] [-q] [-r] [--strict] < pack-file\";\n \n /* We always read in 4kB chunks. */\n@@ -462,6 +463,36 @@ static void unpack_one(unsigned nr)\n \t}\n }\n \n+static void dump_delta_list(void)\n+{\n+\tstruct delta_info *d;\n+\n+\tfor (d = delta_list; d; d = d->next) {\n+\t\tgit_SHA_CTX c;\n+\t\tunsigned char sha1[20];\n+\t\tchar *path;\n+\t\tint fd;\n+\n+\t\tgit_SHA1_Init(&c);\n+\t\tgit_SHA1_Update(&c, d->delta, d->size);\n+\t\tgit_SHA1_Final(sha1, &c);\n+\t\tpath = git_path(\"lost-found/delta/%s\", sha1_to_hex(sha1));\n+\n+\t\tif (safe_create_leading_directories(path) < 0)\n+\t\t\tdie(\"could not create lost-found directory\");\n+\n+\t\tfd = open(path, O_CREAT|O_WRONLY, 0666);\n+\t\tif (fd < 0)\n+\t\t\tdie(\"unable to open %s: %s\", path, strerror(errno));\n+\t\tif (write_in_full(fd, d->delta, d->size) < 0)\n+\t\t\tdie(\"error writing to %s: %s\", path, strerror(errno));\n+\t\tif (close(fd) < 0)\n+\t\t\tdie(\"error writing to %s: %s\", path, strerror(errno));\n+\n+\t\tfprintf(stderr, \"dumped delta %s\\n\", sha1_to_hex(sha1));\n+\t}\n+}\n+\n static void unpack_all(void)\n {\n \tint i;\n@@ -486,8 +517,11 @@ static void unpack_all(void)\n \t}\n \tstop_progress(&progress);\n \n-\tif (delta_list)\n+\tif (delta_list) {\n+\t\tif (dump_deltas)\n+\t\t\tdump_delta_list();\n \t\tdie(\"unresolved deltas left after unpacking\");\n+\t}\n }\n \n int cmd_unpack_objects(int argc, const char **argv, const char *prefix)\n@@ -534,6 +568,10 @@ int cmd_unpack_objects(int argc, const char **argv, const char *prefix)\n \t\t\t\tlen = sizeof(*hdr);\n \t\t\t\tcontinue;\n \t\t\t}\n+\t\t\tif (!strcmp(arg, \"--dump-deltas\")) {\n+\t\t\t\tdump_deltas = 1;\n+\t\t\t\tcontinue;\n+\t\t\t}\n \t\t\tusage(unpack_usage);\n \t\t}\n \n"},{"id":"99130","messageId":"20090101235200.GA14713@blimp.localdomain","threadId":"16801","inReplyTo":"87eizmty25.fsf_-_@jidanni.org","subject":"[PATCH] Handle sha1_object_info failures in ls-tree -l","fromName":"Alex Riesen","fromEmail":"raa.lkml@gmail.com","sentAt":"2009-01-01T23:52:00Z","receivedAt":"2009-01-01T23:52:00Z","isPatch":true,"sender":{"key":"raa.lkml@gmail.com","avatar":"https://avatars.githubusercontent.com/u/324101?v=4"},"body":"Printing 0 as the size of the blob seem to be the safest. The error\nmessage is already printed by sha1_object_info itself.\n\nSigned-off-by: Alex Riesen <raa.lkml@gmail.com>\n---\njidanni@jidanni.org, Fri, Jan 02, 2009 00:18:42 +0100:\n> git ls-tree prints wacko file sizes if it can't find the blob:\n> $ git ls-tree --abbrev=4 -l 76e4\n> error: unable to find ae832f2245892ddde5221357466448b409775142\n> 100644 blob ae83 3220821896     words\n\nNot tested, but should print size of 0 if this happens.\nI actually would prefer ls-tree finish listing and exit(1) in this case,\nbut ... am a little lazy (or scared of a \"static int exit_code;\").\n\n builtin-ls-tree.c |    3 ++-\n 1 files changed, 2 insertions(+), 1 deletions(-)\n\ndiff --git a/builtin-ls-tree.c b/builtin-ls-tree.c\nindex cb61717..234df50 100644\n--- a/builtin-ls-tree.c\n+++ b/builtin-ls-tree.c\n@@ -96,7 +96,8 @@ static int show_tree(const unsigned char *sha1, const char *base, int baselen,\n \tif (!(ls_options & LS_NAME_ONLY)) {\n \t\tif (ls_options & LS_SHOW_SIZE) {\n \t\t\tif (!strcmp(type, blob_type)) {\n-\t\t\t\tsha1_object_info(sha1, &size);\n+\t\t\t\tif (sha1_object_info(sha1, &size))\n+\t\t\t\t\tsize = 0;\n \t\t\t\tprintf(\"%06o %s %s %7lu\\t\", mode, type,\n \t\t\t\t       abbrev ? find_unique_abbrev(sha1, abbrev)\n \t\t\t\t              : sha1_to_hex(sha1),\n-- \n1.6.1.73.g7450\n"},{"id":"99133","messageId":"878wputvnt.fsf@jidanni.org","threadId":"16801","inReplyTo":"20090101234815.GA9049@coredump.intra.peff.net","subject":"Re: [PATCH] Documentation/git-bundle.txt: Dumping contents of any bundle","fromName":"","fromEmail":"jidanni@jidanni.org","sentAt":"2009-01-02T00:10:30Z","receivedAt":"2009-01-02T00:10:30Z","isPatch":true,"sender":{"key":"jidanni@jidanni.org","avatar":"https://gravatar.com/avatar/36568d4af4c8d3e71627ef3b8c8d00e39065b12f29676cccd38ced75e68fa2a6?d=mp&s=160"},"body":"JK> diff --git a/builtin-unpack-objects.c b/builtin-unpack-objects.c\nOK, I wish you luck in the fruition of the new --dump-delta option, and\ncan proofread the man pages involved, otherwise this is no area for\njunior programmer me.\n"},{"id":"99148","messageId":"20090102071519.GA14472@spearce.org","threadId":"16801","inReplyTo":"878wputvnt.fsf@jidanni.org","subject":"Re: [PATCH] Documentation/git-bundle.txt: Dumping contents of any bundle","fromName":"Shawn O. Pearce","fromEmail":"spearce@spearce.org","sentAt":"2009-01-02T07:15:19Z","receivedAt":"2009-01-02T07:15:19Z","isPatch":true,"sender":{"key":"spearce@spearce.org","avatar":"https://avatars.githubusercontent.com/u/34844?v=4"},"body":"jidanni@jidanni.org wrote:\n> JK> diff --git a/builtin-unpack-objects.c b/builtin-unpack-objects.c\n> OK, I wish you luck in the fruition of the new --dump-delta option, and\n> can proofread the man pages involved, otherwise this is no area for\n> junior programmer me.\n\nThis is rather insane.  There's very little data inside of a delta.\nThat's sort of the point of that level of compression, it takes\nup very little disk space and yet describes the change made.\nAlmost nobody is going to want the delta without the base object\nit applies onto.  No user of git is going to need that.  I'd rather\nnot carry dead code around in the tree for something nobody will\never use.\n\nFWIW, most Git deltas are \"copy\" instructions, they list a position\nand count in the base to copy data *from*.  These take up less\nspace then \"insert\" instructions, where new text is placed into\nthe file.  As the delta generator favors a smaller delta, it tends\nto create deltas that use the \"copy\" instruction more often than the\n\"insert\" instruction.  So there is *very* little data in the delta,\njust ranges to copy from somewhere else.  Without that other place\n(the delta base) all you can do is guess about those bits.  Which you\ncan do just as well with a few flips of a fair coin.  :-)\n\n-- \nShawn.\n"},{"id":"99150","messageId":"20090102082709.GA3498@coredump.intra.peff.net","threadId":"16801","inReplyTo":"20090102071519.GA14472@spearce.org","subject":"Re: [PATCH] Documentation/git-bundle.txt: Dumping contents of any bundle","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2009-01-02T08:27:09Z","receivedAt":"2009-01-02T08:27:09Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Thu, Jan 01, 2009 at 11:15:19PM -0800, Shawn O. Pearce wrote:\n\n> > OK, I wish you luck in the fruition of the new --dump-delta option, and\n> > can proofread the man pages involved, otherwise this is no area for\n> > junior programmer me.\n> \n> This is rather insane.  There's very little data inside of a delta.\n> That's sort of the point of that level of compression, it takes\n> up very little disk space and yet describes the change made.\n> Almost nobody is going to want the delta without the base object\n> it applies onto.  No user of git is going to need that.  I'd rather\n> not carry dead code around in the tree for something nobody will\n> ever use.\n\nI somewhat agree. Obviously we can come up with contrived cases where\nthe delta is a pure \"add\" and this option magically lets you recover\nsome text via \"strings\" on the resulting delta dump. But in practice,\nit's hard to say exactly how useful it would be, especially since the\n\"motivation\" here seems to be more academic than any actual real-world\nproblem. We can approximate with something like:\n\n  git clone git://git.kernel.org/pub/scm/git/git.git\n  cd git\n  git bundle create ../bundle.git v1.6.0..v1.6.1\n  mkdir ../broken && cd ../broken\n  sed '/^PACK/,$!d' ../bundle.git >pack\n  git init\n  git unpack-objects --dump-deltas <pack\n  strings .git/lost-found/delta/* | less\n\nwhere maybe you lost your actual repository, but you still have a backup\nof a bundle you sneaker-netted between major versions. In this instance\nwe have 6000 objects in the bundle, 2681 of which are blobs (and\ntherefore presumably the most interesting things to recover). Of those,\n1070 were non-delta and can be recovered completely. For the remainder,\nour strings command shows us snippets of what was there. There are\ndefinitely recognizable pieces of code. But likewise there are pieces of\ncode that are missing subtle parts. E.g.:\n\n                  if (textconv_one) {\n                        size_t size;\n                        mf1.ptr = run_textconv(textconv_one, one, &size);\n                        if (!mf1.\nptr)\n                        mf1.size = size;\n                if (textconv_two) {\n                        size_t size;\n                        mf2.ptr = run_textconv(textconv_two, two, &size);\n                        if (!mf2.\nptr)\n                        mf2.size = size;\n\nSo while there is _something_ to be recovered there, it is basically as\neasy to rewrite the code as it is to piece together whatever fragments\nare available into something comprehensible.\n\nSo in practice, the delta dump would only be useful if:\n\n  1. You have an incomplete thin pack, which generally means you are\n     using bundles (or you interrupted a fetch and kept the tmp_pack).\n\n  2. There is _no_ other copy of the basis. The results you get from\n     this method are so awful that it should really only be last-ditch.\n     I think you would be insane to say \"Oh, I don't have net access\n     right now. Let me just spend hours picking through these deltas to\n     find a scrap of something useful instead of just waiting until I\n     get access again.\"\n\n  3. The changes in the pack tend to produce deltas rather than full\n     blobs, but the deltas tend to be very add-heavy.\n\nI don't know how popular bundles are, but I would expect (1) puts us\nvery much in the minority. On top of that, given the nature of git, I\nfind (2) to be pretty unlikely. If you're sneaker-netting data with a\nbundle, then it seems rare that both ends of the net will be lost at\nonce. As for (3), it seems source code is not a good candidate here.\nPerhaps if you were writing a novel in a single file, you might salvage\nwhole paragraphs or even chapters.\n\nSo I am inclined to leave it as-is: a patch in the list archive. If and\nwhen the day comes when somebody loses some super-important data and\nsomehow matches all of these criteria, then they can consult whatever\naged and senile git gurus still exist to pull the patch out and see if\nanything can be recovered.\n\n-Peff\n"},{"id":"99214","messageId":"87mye9wekg.fsf@jidanni.org","threadId":"16801","inReplyTo":"20090102082709.GA3498@coredump.intra.peff.net","subject":"Re: [PATCH] Documentation/git-bundle.txt: Dumping contents of any bundle","fromName":"","fromEmail":"jidanni@jidanni.org","sentAt":"2009-01-02T22:03:43Z","receivedAt":"2009-01-02T22:03:43Z","isPatch":true,"sender":{"key":"jidanni@jidanni.org","avatar":"https://gravatar.com/avatar/36568d4af4c8d3e71627ef3b8c8d00e39065b12f29676cccd38ced75e68fa2a6?d=mp&s=160"},"body":"Some options are:\n\n1) just add a line or two to my man page patch showing\nwhat recovery can and can't presently be done. (No need for my\ntemporary file, use a pipe too.)\n\n2) Also implement that step where everything is uncompressed and put\ninto lost+found, and document that they should expect to just see a\nlot of connector markings, and if there are useful strings in there\nthen they are just lucky. We did the job asked: recovered to the best\nextent of what they gave us.\n\nJK> So I am inclined to leave it as-is: a patch in the list archive. If and\nJK> when the day comes when somebody loses some super-important data and\nJK> somehow matches all of these criteria, then they can consult whatever\nJK> aged and senile git gurus still exist to pull the patch out and see if\nJK> anything can be recovered.\n\nI've read too many cases in RISKS Digest, news:comp.risks, about years\nlater organizations trying to recover some weird format or media.\nTherefore I urge you to strike while the iron is hot and hook up the\nfunction into the code.\n\nMaybe some have never tried to recover data, but for those that one\nday might, they will be thanking you over and over for taking this\nopportunity to give them a chance. In many cases the few shreds they\ncan recover might be all they need.\n\nAlso one can see the innards of git -- no more black box.\n\nIf I were creating a new binary format, I would be sure to also\nprovide decoder tools. Otherwise it is just like it requires its own\nproprietary environment to reveal any of its innards. Sure, you can\nsay well that data is mainly useless... but it is better than nothing\n-- we did the best with what they gave us.\n"},{"id":"102021","messageId":"87d4e97uxb.fsf_-_@jidanni.org","threadId":"16801","inReplyTo":"87zlirc49l.fsf@jidanni.org","subject":"[PATCH] git-bundle(1): add no references required simplest case","fromName":"","fromEmail":"jidanni@jidanni.org","sentAt":"2009-01-26T19:02:08Z","receivedAt":"2009-01-26T19:02:08Z","isPatch":true,"sender":{"key":"jidanni@jidanni.org","avatar":"https://gravatar.com/avatar/36568d4af4c8d3e71627ef3b8c8d00e39065b12f29676cccd38ced75e68fa2a6?d=mp&s=160"},"body":"Signed-off-by: jidanni <jidanni@jidanni.org>\n---\nSee http://article.gmane.org/gmane.comp.version-control.git/103576\n Documentation/git-bundle.txt |    7 +++++++\n 1 files changed, 7 insertions(+), 0 deletions(-)\n\ndiff --git a/Documentation/git-bundle.txt b/Documentation/git-bundle.txt\nindex 1b66ab7..7c1e990 100644\n--- a/Documentation/git-bundle.txt\n+++ b/Documentation/git-bundle.txt\n@@ -164,6 +164,13 @@ $ git pull bundle\n would treat it as if it is talking with a remote side over the\n network.\n \n+If a bundle requires no references, one may simply use:\n+\n+------------\n+$ git init\n+$ git pull bundle.bdl master\n+------------\n+\n Author\n ------\n Written by Mark Levedahl <mdl123@verizon.net>\n-- \n1.6.0.6\n"},{"id":"102033","messageId":"7vljsx6dzi.fsf@gitster.siamese.dyndns.org","threadId":"16801","inReplyTo":"87d4e97uxb.fsf_-_@jidanni.org","subject":"Re: [PATCH] git-bundle(1): add no references required simplest case","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2009-01-26T19:53:21Z","receivedAt":"2009-01-26T19:53:21Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"jidanni@jidanni.org writes:\n\n> Signed-off-by: jidanni <jidanni@jidanni.org>\n> ---\n> See http://article.gmane.org/gmane.comp.version-control.git/103576\n>  Documentation/git-bundle.txt |    7 +++++++\n>  1 files changed, 7 insertions(+), 0 deletions(-)\n>\n> diff --git a/Documentation/git-bundle.txt b/Documentation/git-bundle.txt\n> index 1b66ab7..7c1e990 100644\n> --- a/Documentation/git-bundle.txt\n> +++ b/Documentation/git-bundle.txt\n> @@ -164,6 +164,13 @@ $ git pull bundle\n>  would treat it as if it is talking with a remote side over the\n>  network.\n>  \n> +If a bundle requires no references, one may simply use:\n\nTwo nits.\n\n 1. Bundle does not require reference;it requires commits.\n\n 2. \"One may simply use:\" with a recipe without saying what the recipe is\n    useful for is not very helpful.\n\nThe second point needs to be stressed.  For example, you could say\nsomething like this:\n\n        With any bundle, you may simply say:\n\n                $ git ls-remote bundle.bdl\n\nand it is a correct description if it is to see the refs in the bundle is\nwhat you want to do, but it does not help when cloning from it is what you\nwant.\n\nIt would be a good practice to make the new part go with the flow of the\nexisting examples.  Adding the following at the end might be a better way\nto do this than your \"init then pull\" example:\n\n\tA complete bundle is one that does not require you to have any\n\tprerequiste object for you to extract its contents.  Not only you\n\tcan fetch/pull from a bundle, you can clone from a complete bundle\n\tas if it is a remote repository, like this:\n\n\t----------------\n        $ git clone /home/me/tmp/file.bdl mine.git\n        ----------------\n\n\tThis will define a remote called \"origin\" in the resulting\n\trepository that lets you fetch and pull from the bundle, just\n\tlike the previous example lets you do with the remote called\n\t\"bundle\", and from then on you can fetch/pull to update the\n\tresulting mine.git repository after replacing the bundle you store\n\tat /home/me/tmp/file.bdl with incremental updates.\n"},{"id":"102466","messageId":"87tz7i6scg.fsf_-_@jidanni.org","threadId":"16801","inReplyTo":"7vljsx6dzi.fsf@gitster.siamese.dyndns.org","subject":"[PATCH,v2] git-bundle(1): add no references required simplest case","fromName":"","fromEmail":"jidanni@jidanni.org","sentAt":"2009-01-29T15:32:15Z","receivedAt":"2009-01-29T15:32:15Z","isPatch":true,"sender":{"key":"jidanni@jidanni.org","avatar":"https://gravatar.com/avatar/36568d4af4c8d3e71627ef3b8c8d00e39065b12f29676cccd38ced75e68fa2a6?d=mp&s=160"},"body":"Signed-off-by: jidanni <jidanni@jidanni.org>\n---\nWords totally by Junio C Hamano.\n Documentation/git-bundle.txt |   16 ++++++++++++++++\n 1 files changed, 16 insertions(+), 0 deletions(-)\n\ndiff --git a/Documentation/git-bundle.txt b/Documentation/git-bundle.txt\nindex 1b66ab7..42c2abc 100644\n--- a/Documentation/git-bundle.txt\n+++ b/Documentation/git-bundle.txt\n@@ -164,6 +164,22 @@ $ git pull bundle\n would treat it as if it is talking with a remote side over the\n network.\n \n+A complete bundle is one that does not require you to have any\n+prerequisite object for you to extract its contents.  Not only you\n+can fetch/pull from a bundle, you can clone from a complete bundle\n+as if it was a remote repository, like this:\n+\n+----------------\n+$ git clone /home/me/tmp/file.bdl mine.git\n+----------------\n+\n+This will define a remote called \"origin\" in the resulting\n+repository that lets you fetch and pull from the bundle, just\n+like the previous example lets you do with the remote called\n+\"bundle\", and from then on you can fetch/pull to update the\n+resulting mine.git repository after replacing the bundle you store\n+at /home/me/tmp/file.bdl with incremental updates.\n+\n Author\n ------\n Written by Mark Levedahl <mdl123@verizon.net>\n-- \n1.6.0.6\n"},{"id":"102799","messageId":"87pri12078.fsf@jidanni.org","threadId":"16801","inReplyTo":"87tz7i6scg.fsf_-_@jidanni.org","subject":"Re: [PATCH,v2] git-bundle(1): add no references required simplest case","fromName":"","fromEmail":"jidanni@jidanni.org","sentAt":"2009-02-01T23:42:51Z","receivedAt":"2009-02-01T23:42:51Z","isPatch":true,"sender":{"key":"jidanni@jidanni.org","avatar":"https://gravatar.com/avatar/36568d4af4c8d3e71627ef3b8c8d00e39065b12f29676cccd38ced75e68fa2a6?d=mp&s=160"},"body":"Words totally by Junio C Hamano.\nSigned-off-by: jidanni <jidanni@jidanni.org>\n---\n\nJunio: I used your words.\nYou might have missed this patch. Resending.\n\n\n Documentation/git-bundle.txt |   16 ++++++++++++++++\n 1 files changed, 16 insertions(+), 0 deletions(-)\n\ndiff --git a/Documentation/git-bundle.txt b/Documentation/git-bundle.txt\nindex 1b66ab7..42c2abc 100644\n--- a/Documentation/git-bundle.txt\n+++ b/Documentation/git-bundle.txt\n@@ -164,6 +164,22 @@ $ git pull bundle\n would treat it as if it is talking with a remote side over the\n network.\n \n+A complete bundle is one that does not require you to have any\n+prerequisite object for you to extract its contents.  Not only you\n+can fetch/pull from a bundle, you can clone from a complete bundle\n+as if it was a remote repository, like this:\n+\n+----------------\n+$ git clone /home/me/tmp/file.bdl mine.git\n+----------------\n+\n+This will define a remote called \"origin\" in the resulting\n+repository that lets you fetch and pull from the bundle, just\n+like the previous example lets you do with the remote called\n+\"bundle\", and from then on you can fetch/pull to update the\n+resulting mine.git repository after replacing the bundle you store\n+at /home/me/tmp/file.bdl with incremental updates.\n+\n Author\n ------\n Written by Mark Levedahl <mdl123@verizon.net>\n-- \n1.6.0.6\n"},{"id":"102801","messageId":"alpine.DEB.1.00.0902020056520.3586@pacific.mpi-cbg.de","threadId":"16801","inReplyTo":"87pri12078.fsf@jidanni.org","subject":"Re: [PATCH,v2] git-bundle(1): add no references required simplest case","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2009-02-02T00:04:13Z","receivedAt":"2009-02-02T00:04:13Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi,\n\nOn Mon, 2 Feb 2009, jidanni@jidanni.org wrote:\n\n> Words totally by Junio C Hamano.\n> Signed-off-by: jidanni <jidanni@jidanni.org>\n> ---\n> \n> Junio: I used your words.\n> You might have missed this patch. Resending.\n\nYou are not serious, are you?  People have explained time and time again \nwhat is required by a commit message.\n\nNow, I am not a native speaker, but the commit subject seems to contain \ngrammatical errors.  Even if it weren't, it is not understandable.\n\nSo the only thing that is in your complete commit message remotely \npurporting to explain what the patch is about and why it is good, fails to \ndo so.\n\nAlso, we always have an empty line before SOB lines.\n\n> +A complete bundle is one that does not require you to have any\n\nI have not heard of any \"complete\" bundle before, and I do not understand \nthe need for such a definition, either.\n\n> +prerequisite object for you to extract its contents.  Not only you\n> +can fetch/pull from a bundle, you can clone from a complete bundle\n\n\"Not only you can\" violates grammar in my book.\n\n> +as if it was a remote repository, like this:\n> +\n> +----------------\n> +$ git clone /home/me/tmp/file.bdl mine.git\n> +----------------\n> +\n> +This will define a remote called \"origin\" in the resulting\n> +repository that lets you fetch and pull from the bundle, just\n> +like the previous example lets you do with the remote called\n> +\"bundle\", and from then on you can fetch/pull to update the\n> +resulting mine.git repository after replacing the bundle you store\n> +at /home/me/tmp/file.bdl with incremental updates.\n\nIMO this paragraph just adds words, not anything the user does not know \nalready by that stage.\n\nCiao,\nDscho\n"},{"id":"102803","messageId":"7vab95r7j4.fsf@gitster.siamese.dyndns.org","threadId":"16801","inReplyTo":"alpine.DEB.1.00.0902020056520.3586@pacific.mpi-cbg.de","subject":"Re: [PATCH,v2] git-bundle(1): add no references required simplest case","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2009-02-02T00:45:19Z","receivedAt":"2009-02-02T00:45:19Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:\n\n>> +A complete bundle is one that does not require you to have any\n>\n> I have not heard of any \"complete\" bundle before, and I do not understand \n> the need for such a definition, either.\n\nSorry, that's mine, not Jidanni's fault.  I agree that we do not\nnecessarily have to introduce a new term.\n\n>> +as if it was a remote repository, like this:\n>> +\n>> +----------------\n>> +$ git clone /home/me/tmp/file.bdl mine.git\n>> +----------------\n>> +\n>> +This will define a remote called \"origin\" in the resulting\n>> +repository that lets you fetch and pull from the bundle, just\n>> +like the previous example lets you do with the remote called\n>> +\"bundle\", and from then on you can fetch/pull to update the\n>> +resulting mine.git repository after replacing the bundle you store\n>> +at /home/me/tmp/file.bdl with incremental updates.\n>\n> IMO this paragraph just adds words, not anything the user does not know \n> already by that stage.\n\nTrue again.\n\nThe only justification that an example of cloning from a complete (or\n\"baseless\" or \"full\" or whatever new term we have already agreed that is\nnot needed ;-)) bundle in the example I can think of is that by having\nsuch an example way earlier in the example sequence, we could show a full\ncycle of sneakernetting into a repository.  You bootstrap it by cloning\nfrom a complete bundle, so that the clone has remotes set up to facilitate\nfurther updates via fetch/pull pointing at a known location.  Then you\ndrop a new bundle to the same location that is relative to an earlier one,\nand pull from it to incrementally keep the repository up-to-date.\n\nIn other words, we currently have a very cursory description that says you\ncan ls-remote and fetch from a bundle at the end, and mention that the\nremote configuration can be defined to facilitate repeated sneakernet\noperation.  But we could reorganize the example this way (the ones with\nasterisk are already in our example section, the ones with plus are\nadditions):\n\n * you first create a full bundle without basis\n\n\t$ git bundle create mybundle master\n\n * you make note of the current tip to optimize later bundles\n\n        $ git tag -f lastR2bundle master\n\n + sneakernet it and clone it to prime the recipient\n\n\t... sneakernet mybundle to /home/me/tmp/mybundle\n \t$ git clone /home/me/tmp/mybundle mine.git\n\n + after working more in the original, create an incremental bundle\n\n\t$ git bundle create mybundle lastR2bundle..master\n\t$ git tag -f lastR2bundle master\n\n + sneakernet it again, and use it to update the recipient\n \n\t... sneakernet the new mybundle to /home/me/tmp/mybundle\n \t$ git pull /home/me/tmp/mybundle mine.git\n\nto show the simplest \"full cycle\" of sneakernet workflow.  And then show\nvarious variations we already have in the existing examples.\n\nSomething like:\n\n    In addition, if you know up to what commit the intended recipient\n    repository should have the necessary objects for, you can use that\n    knowledge to specify the basis, giving a cut-off point to limit the\n    revisions and objects that go in to the resulting bundle.  Here are the\n    examples:\n\n     * using a tag present in both to optimize the bundle\n\n            $ git bundle create mybundle master ^v1.0.0\n\n     * using a basis based on time to optimize the bundle\n\n            $ git bundle create mybundle master --since=10.days\n\n     * using the number of commits to optimize the bundle\n\n            $ git bundle create mybundle master -n 10 \n\n    A bundle from a recipient repository's point of view is just like a\n    regular repository it fetches/pulls from.  You can for example map\n    refs, like this example, when fetching.\n\n            $ git fetch mybundle master:localRef\n\n    Or see what refs it offers\n\n            $ git ls-remote mybundle\n"},{"id":"103052","messageId":"87y6wnnjvl.fsf@jidanni.org","threadId":"16801","inReplyTo":"7vab95r7j4.fsf@gitster.siamese.dyndns.org","subject":"Re: [PATCH,v2] git-bundle(1): add no references required simplest case","fromName":"","fromEmail":"jidanni@jidanni.org","sentAt":"2009-02-04T00:09:02Z","receivedAt":"2009-02-04T00:09:02Z","isPatch":true,"sender":{"key":"jidanni@jidanni.org","avatar":"https://gravatar.com/avatar/36568d4af4c8d3e71627ef3b8c8d00e39065b12f29676cccd38ced75e68fa2a6?d=mp&s=160"},"body":"Junio, could you combine your two recent versions,\nhttp://news.gmane.org/group/gmane.comp.version-control.git/thread=103575/force_load=t/focus=108030\ninto a final one and commit it. No need to credit me. This is already\nway over my head.  Note however that the git clone example disappeared\nfrom your final version. Also perhaps give a simplest example of git\npull. Indeed, much of your discussion is valuable and should be\nincluded on the man page. Whatever you commit is fine. I would just\nlike to \"close this bug\" without having all the valuable documentation\nyou wrote for it just go down the drain, which will certainly happen\nif I didn't send this message...\n"},{"id":"103060","messageId":"7vbptj9cp8.fsf@gitster.siamese.dyndns.org","threadId":"16801","inReplyTo":"87y6wnnjvl.fsf@jidanni.org","subject":"Re: [PATCH,v2] git-bundle(1): add no references required simplest case","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2009-02-04T02:07:47Z","receivedAt":"2009-02-04T02:07:47Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"jidanni@jidanni.org writes:\n\n> Junio, could you combine your two recent versions,\n> http://news.gmane.org/group/gmane.comp.version-control.git/thread=103575/force_load=t/focus=108030\n> into a final one and commit it. No need to credit me. This is already\n> way over my head.  Note however that the git clone example disappeared\n> from your final version. Also perhaps give a simplest example of git\n> pull. Indeed, much of your discussion is valuable and should be\n> included on the man page. Whatever you commit is fine. I would just\n> like to \"close this bug\" without having all the valuable documentation\n> you wrote for it just go down the drain, which will certainly happen\n> if I didn't send this message...\n\nThe former was shot down by Johannes and I agree with his reasoning, and\nthe latter is merely \"something like\" outline that is not good enough for\ninclusion.  I personally do not consider there is a *bug* in the current\ndocumentation so it is not much of my itch to scratch either.\n\nCould you convince me that I should spend more time on that filling the\nblanks in \"something line\" outline myself, instead of spending my git time\non some other areas, please?\n"},{"id":"103062","messageId":"87ab93ndvz.fsf@jidanni.org","threadId":"16801","inReplyTo":"7vbptj9cp8.fsf@gitster.siamese.dyndns.org","subject":"Re: [PATCH,v2] git-bundle(1): add no references required simplest case","fromName":"","fromEmail":"jidanni@jidanni.org","sentAt":"2009-02-04T02:18:24Z","receivedAt":"2009-02-04T02:18:24Z","isPatch":true,"sender":{"key":"jidanni@jidanni.org","avatar":"https://gravatar.com/avatar/36568d4af4c8d3e71627ef3b8c8d00e39065b12f29676cccd38ced75e68fa2a6?d=mp&s=160"},"body":"JCH> Could you convince me that I should spend more time on that\n\nI can't. I will however at least for myself bookmark this thread as a\nvaluable git bundle documentation supplement. OK, thanks.\n"},{"id":"103091","messageId":"20090204181529.6117@nanako3.lavabit.com","threadId":"16801","inReplyTo":"7vbptj9cp8.fsf@gitster.siamese.dyndns.org","subject":"[PATCH] git-bundle doc: update examples","fromName":"Nanako Shiraishi","fromEmail":"nanako3@lavabit.com","sentAt":"2009-02-04T09:15:29Z","receivedAt":"2009-02-04T09:15:29Z","isPatch":true,"sender":{"key":"nanako3@lavabit.com","avatar":"https://gravatar.com/avatar/3777b9e201c5883a62b1a6fdf7c53f2d712d1d80989146063ea861e33aad72a8?d=mp&s=160"},"body":"This rewrites the example part of the bundle doucmentation to follow\nthe suggestion made by Junio during a recent discussion (gmane 108030).\n\nInstead of just showing different ways to create and use bundles in a\ndisconnected fashion, the rewritten example first shows the simplest\n\"full cycle\" of sneakernet workflow, and then introduces various\nvariations.\n\nThe words are mostly taken from Junio's outline. I only reformatted\nthem and proofread to make sure the end result flows naturally.\n\nSigned-off-by: Nanako Shiraishi <nanako3@lavabit.com>\n---\n\nI didn't want your improvement suggestion to go to waste either, so\nhere is a proposed conclusion of this topic in a patch form, hopefully\nin a good enough quality.\n\nAfter the maintainer spent a lot of time to suggest how to improve a\nproposed patch for inclusion, it is rude for a contributor to walk\naway without following through the review process. Such a proposed\npatch is not contributing to the development process but only stealing\nmaintainer's and reviewers' time from the community. But others like I\ncan at least try to help (^_^;).\n\n Documentation/git-bundle.txt |  132 ++++++++++++++++++++++++++---------------\n 1 files changed, 84 insertions(+), 48 deletions(-)\n\ndiff --git a/Documentation/git-bundle.txt b/Documentation/git-bundle.txt\nindex 1b66ab7..ea0f6a0 100644\n--- a/Documentation/git-bundle.txt\n+++ b/Documentation/git-bundle.txt\n@@ -84,7 +84,7 @@ defining the basis.  More than one reference may be packaged, and more\n than one basis can be specified.  The objects packaged are those not\n contained in the union of the given bases.  Each basis can be\n specified explicitly (e.g., ^master~10), or implicitly (e.g.,\n-master~10..master, master --since=10.days.ago).\n+master~10..master, --since=10.days.ago master).\n \n It is very important that the basis used be held by the destination.\n It is okay to err on the side of conservatism, causing the bundle file\n@@ -94,75 +94,111 @@ when unpacking at the destination.\n EXAMPLE\n -------\n \n-Assume two repositories exist as R1 on machine A, and R2 on machine B.\n+Assume you want to transfer the history from a repository R1 on machine A\n+to another repository R2 on machine B.\n For whatever reason, direct connection between A and B is not allowed,\n but we can move data from A to B via some mechanism (CD, email, etc).\n We want to update R2 with developments made on branch master in R1.\n \n-To create the bundle you have to specify the basis. You have some options:\n+To bootstrap the process, you can first create a bundle that doesn't have\n+any basis. You can use a tag to remember up to what commit you sent out\n+in order to make it easy to later update the other repository with\n+incremental bundle,\n \n-- Without basis.\n-+\n-This is useful when sending the whole history.\n+----------------\n+machineA$ cd R1\n+machineA$ git bundle create file.bdl master\n+machineA$ git tag -f lastR2bundle master\n+----------------\n \n-------------\n-$ git bundle create mybundle master\n-------------\n+Then you sneakernet file.bdl to the target machine B. Because you don't\n+have to have any object to extract objects from such a bundle, not only\n+you can fetch/pull from a bundle, you can clone from it as if it was a\n+remote repository.\n \n-- Using temporally tags.\n-+\n-We set a tag in R1 (lastR2bundle) after the previous such transport,\n-and move it afterwards to help build the bundle.\n+----------------\n+machineB$ git clone /home/me/tmp/file.bdl R2\n+----------------\n \n-------------\n-$ git bundle create mybundle master ^lastR2bundle\n-$ git tag -f lastR2bundle master\n-------------\n+This will define a remote called \"origin\" in the resulting repository that\n+lets you fetch and pull from the bundle. $GIT_DIR/config file in R2 may\n+have an entry like this:\n \n-- Using a tag present in both repositories\n+------------------------\n+[remote \"origin\"]\n+    url = /home/me/tmp/file.bdl\n+    fetch = refs/heads/*:refs/remotes/origin/*\n+------------------------\n+\n+You can fetch/pull to update the resulting mine.git repository after\n+replacing the bundle you store at /home/me/tmp/file.bdl with incremental\n+updates from here on.\n+\n+After working more in the original repository, you can create an\n+incremental bundle to update the other:\n+\n+----------------\n+machineA$ cd R1\n+machineA$ git bundle create file.bdl lastR2bundle..master\n+machineA$ git tag -f lastR2bundle master\n+----------------\n+\n+and sneakernet it to the other machine to replace /home/me/tmp/file.bdl,\n+and pull from it.\n+\n+----------------\n+machineB$ cd R2\n+machineB$ git pull\n+----------------\n \n-------------\n-$ git bundle create mybundle master ^v1.0.0\n-------------\n+If you know up to what commit the intended recipient repository should\n+have the necessary objects for, you can use that knowledge to specify the\n+basis, giving a cut-off point to limit the revisions and objects that go\n+in the resulting bundle. The previous example used lastR2bundle tag\n+for this purpose, but you can use other options you would give to\n+the linkgit:git-log[1] command. Here are more examples:\n \n-- A basis based on time.\n+You can use a tag that is present in both.\n \n-------------\n-$ git bundle create mybundle master --since=10.days.ago\n-------------\n+----------------\n+$ git bundle create mybundle v1.0.0..master\n+----------------\n \n-- With a limit on the number of commits\n+You can use a basis based on time.\n \n-------------\n-$ git bundle create mybundle master -n 10\n-------------\n+----------------\n+$ git bundle create mybundle --since=10.days master\n+----------------\n \n-Then you move mybundle from A to B, and in R2 on B:\n+Or you can use the number of commits.\n \n-------------\n+----------------\n+$ git bundle create mybundle -10 master\n+----------------\n+\n+You can run `git-bundle verify` to see if you can extract from a bundle\n+that was created with a basis.\n+\n+----------------\n $ git bundle verify mybundle\n-$ git fetch mybundle master:localRef\n-------------\n+----------------\n \n-With something like this in the config in R2:\n+This will list what commits you must have in order to extract from the\n+bundle and will error out if you don't have them.\n \n-------------------------\n-[remote \"bundle\"]\n-    url = /home/me/tmp/file.bdl\n-    fetch = refs/heads/*:refs/remotes/origin/*\n-------------------------\n+A bundle from a recipient repository's point of view is just like a\n+regular repository it fetches/pulls from. You can for example map\n+refs, like this example, when fetching:\n \n-You can first sneakernet the bundle file to ~/tmp/file.bdl and\n-then these commands on machine B:\n+----------------\n+$ git fetch mybundle master:localRef\n+----------------\n \n-------------\n-$ git ls-remote bundle\n-$ git fetch bundle\n-$ git pull bundle\n-------------\n+Or see what refs it offers.\n \n-would treat it as if it is talking with a remote side over the\n-network.\n+----------------\n+$ git ls-remote mybundle\n+----------------\n \n Author\n ------\n-- \n1.6.1.2\n\n-- \nNanako Shiraishi\nhttp://ivory.ap.teacup.com/nanako3/\n"},{"id":"103148","messageId":"20090204152605.GB6896@sigill.intra.peff.net","threadId":"16801","inReplyTo":"20090204181529.6117@nanako3.lavabit.com","subject":"Re: [PATCH] git-bundle doc: update examples","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2009-02-04T15:26:05Z","receivedAt":"2009-02-04T15:26:05Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Wed, Feb 04, 2009 at 06:15:29PM +0900, Nanako Shiraishi wrote:\n\n> I didn't want your improvement suggestion to go to waste either, so\n> here is a proposed conclusion of this topic in a patch form, hopefully\n> in a good enough quality.\n> \n> After the maintainer spent a lot of time to suggest how to improve a\n> proposed patch for inclusion, it is rude for a contributor to walk\n> away without following through the review process. Such a proposed\n> patch is not contributing to the development process but only stealing\n> maintainer's and reviewers' time from the community. But others like I\n> can at least try to help (^_^;).\n\nNanako,\n\nI often see you doing small patch cleanups, reposts, gentle reminders,\nand other work like this that really helps the community process run\nsmoothly. I just wanted to say \"thank you\" so that you know that your\nefforts are not going unnoticed.\n\n-Peff\n"},{"id":"103221","messageId":"7v8woln7ol.fsf@gitster.siamese.dyndns.org","threadId":"16801","inReplyTo":"20090204181529.6117@nanako3.lavabit.com","subject":"Re: [PATCH] git-bundle doc: update examples","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2009-02-04T22:44:42Z","receivedAt":"2009-02-04T22:44:42Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Nanako Shiraishi <nanako3@lavabit.com> writes:\n\n> This rewrites the example part of the bundle doucmentation to follow\n> the suggestion made by Junio during a recent discussion (gmane 108030).\n> \n> Instead of just showing different ways to create and use bundles in a\n> disconnected fashion, the rewritten example first shows the simplest\n> \"full cycle\" of sneakernet workflow, and then introduces various\n> variations.\n> \n> The words are mostly taken from Junio's outline. I only reformatted\n> them and proofread to make sure the end result flows naturally.\n> \n> Signed-off-by: Nanako Shiraishi <nanako3@lavabit.com>\n> ---\n> I didn't want your improvement suggestion to go to waste either, so\n> here is a proposed conclusion of this topic in a patch form, hopefully\n> in a good enough quality.\n\nI appreciate your help like this patch, and your other contributions of\n\"project secretary\" kind, pointing out old threads, prodding about\nunapplied patches, etc., because I do not have infinite amount of time.\n\nThe text seems to follow my \"this might flow more naturally and easier to\nread\" outline exactly, and I do not have a problem with the patch itself.\nAmong the people who were involved in the review, Jidanni seemed to be of\nthe same opinion, but I haven't heard from Dscho one way or another.  So\nI'd keep this on hold for now but I think the examples are organized much\nbetter with this version and we should take it.\n\nHOWEVER.\n\n> After the maintainer spent a lot of time to suggest how to improve a\n> proposed patch for inclusion, it is rude for a contributor to walk\n> away without following through the review process. Such a proposed\n> patch is not contributing to the development process but only stealing\n> maintainer's and reviewers' time from the community. But others like I\n> can at least try to help (^_^;).\n\nI see a smiley, but what's with the animosity?  One thing I've always\nliked about your messages to this list is that they have an exceptional\nsignal to noise ratio, certainly much better than mine [*1*].\n\nI saw you were annoyed by his recent \"bug tracker\" remark in another\nthread, and I do appreciate that you are showing a better way to help by\nsetting an example, but I think this comment is counterproductive.\n\n[Footnote]\n\n*1* You certainly never said anything like giving furniture to somebody\nelse ;-)\n"}]}