{"thread":{"id":"6818","subject":"[PATCH] git-bundle - bundle objects and references for disconnected transfer.","startedAt":"2007-02-14T14:10:25Z","lastAt":"2007-02-16T03:23:59Z","messageCount":32,"participants":["Mark Levedahl","Matthieu Moy","Johannes Schindelin","Junio C Hamano","Shawn O. Pearce","Nicolas Pitre"],"isPatch":true,"patchVersion":1,"patchTotal":null},"messages":[{"id":"34529","messageId":"11714622292110-git-send-email-mdl123@verizon.net","threadId":"6818","inReplyTo":null,"subject":"Scripts to use \"bundles\" for moving data between repositories","fromName":"Mark Levedahl","fromEmail":"mdl123@verizon.net","sentAt":"2007-02-14T14:10:25Z","receivedAt":"2007-02-14T14:10:25Z","isPatch":false,"sender":{"key":"mdl123@verizon.net","avatar":"https://avatars.githubusercontent.com/u/5302462?v=4"},"body":"I am working a project using git where we have many repositories on machines\nthat can never be directly connected, but which need to have the same objects\nand development history.  Existing git protocols offer limited support: we can\neither a) publish and apply patch files branch by branch, or b) copy an entire\nrepository from one machine to another and then do local push or fetch.  While\nboth are workable, neither is a completely satisfactory solution, so I wrote the\nattached scripts that support a \"bundle\" transfer mechanism.  A bundle is a zip\narchive having two files: a list of references as given by git-show-ref and a\npack file of objects from git-pack-objects.  git-bundle creates the bundle,\ngit-unbundle unpacks and applies at the receiving end.  The means of\ntransporting the bundle file between the machines is arbitrary (sneaker net,\nemail, etc all can work).\n\nThis transfer protocol leaves it to the user to assure that the objects in the \nbundle are sufficient to update the target machine.  This is a direct \nconsequence of the prohibition on direct communication between the machines.  \nThe approach supported here is to use normal git-rev-list format to specify what \nto include, e.g.  master~10..master, or ^master pu next, etc.  Having too many \nobjects in the pack file is fine: git-unpack-objects at the receiving end \nhappily ignores things not needed. git-unbundle normally checks that the updated \nreferences are fast-forward (--force to override), and that all required objects \nexist (--shallow to override). This latter option supports a disconnected \nshallow clone.\n\nI offer this for inclusion in the main distribution, comments and suggestions\nfor improvement are welcome regardless. The scripts are working for me today\nand I find them very useful.\n\nMark Levedahl\n"},{"id":"34526","messageId":"11714622292295-git-send-email-mdl123@verizon.net","threadId":"6818","inReplyTo":"11714622292110-git-send-email-mdl123@verizon.net","subject":"[PATCH] git-bundle - bundle objects and references for disconnected transfer.","fromName":"Mark Levedahl","fromEmail":"mdl123@verizon.net","sentAt":"2007-02-14T14:10:26Z","receivedAt":"2007-02-14T14:10:26Z","isPatch":true,"sender":{"key":"mdl123@verizon.net","avatar":"https://avatars.githubusercontent.com/u/5302462?v=4"},"body":"Some workflows require coordinated development between repositories on\nmachines that can never be connected. This utility creates a bundle\ncontaining a pack of objects and associated references (heads or tags)\nthat can be independently transferred to another machine, effectively\nsupporting git-push like operations between disconnected systems.\n\nSigned-off-by: Mark Levedahl <mdl123@verizon.net>\n---\n git-bundle |   85 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++\n 1 files changed, 85 insertions(+), 0 deletions(-)\n create mode 100755 git-bundle\n\ndiff --git a/git-bundle b/git-bundle\nnew file mode 100755\nindex 0000000..1341885\n--- /dev/null\n+++ b/git-bundle\n@@ -0,0 +1,85 @@\n+#!/bin/sh\n+# Create a bundle to carry from one git repo to another (e.g., \"sneaker-net\" based push)\n+# git-bundle <git-rev-list args>\n+# git-bundle --bare <git-rev-list args>\n+# creates bundle.zip in current directory (can rename of course)\n+#\n+# The bundle includes all refs given (--all selects every ref in the repo).\n+# and all of the commit objects needed subject to the list given.\n+#\n+# Objects to be packed are limited by specifying one or more of\n+#   ^commit-ish    - indicated commits already at the target (can have more than one ^commit-ish)\n+#   --since=xxx    - Assume target repo has all relevant commits earlier than xxx\n+\n+USAGE='git-bundle <options> <git-rev-list arguments>\n+\n+Creates a bundle of objects to be carried to a disconnected repository, bringing the target\n+repository''s definition of one or more references up to date as selected by the\n+<git-rev-list arguments>\n+\n+Options:\n+    -h, --help        Print this help screen\n+        --bare        Work in a bare repository\n+    --output=f        Output to file f (default is bundle.zip)\n+\n+    examples\n+        git-bundle master~10..master\n+        git-bundle master next pu ^master~30 --output=mybundle.zip --bare'\n+\n+die() {\n+\techo >&2 \"$@\"\n+\texit 1\n+}\n+\n+# pull out rev-list args vs program args, parse the latter\n+gitrevargs=$(git-rev-parse --symbolic --revs-only $*) || exit 1\n+myargs=$(git-rev-parse --no-revs $*) || exit 1\n+\n+bfile=bundle.zip\n+for arg in $myargs ; do\n+\tcase \"$arg\" in\n+\t\t--bare)\n+\t\t\texport GIT_DIR=.;;\n+\t\t-h|--h|--he|--hel|--help)\n+\t\t\techo \"$USAGE\"\n+\t\t\texit;;\n+\t\t--output=*)\n+\t\t\tbfile=${arg##--output=};;\n+\t\t-*)\n+\t\t\tdie \"unknown option: $arg\";;\n+\t\t*)\n+\tesac\n+done\n+\n+GIT_DIR=$(git-rev-parse --git-dir) || die \"Not in a git directory\"\n+\n+# find the refs to carry along and get sha1s for each.\n+refs=\n+for arg in $gitrevargs ; do\n+\t#ignore options and basis refs\n+\tcase \"$arg\" in\n+\t\t^*) ;;\n+\t\t-*) ;;\n+\t\t*)\n+\t\t\tn=$(git-show-ref \"$arg\" | wc -l)\n+\t\t\t[ $n -eq 1 ] || die \"ambiguous reference: $arg\"\n+\t\t\trefs=\"$refs $arg\"\n+\t\t\t;;\n+\tesac\n+done\n+[ -z \"$refs\" ] && die \"No references specified, I don't know what to bundle.\"\n+\n+# put the refs into the bundle file\n+[ -e \"$bfile\" ] && rm -f \"$bfile\" 2>/dev/null\n+git-show-ref $refs > .gitBundleReferences\n+zip -m \"$bfile\" .gitBundleReferences\n+\n+# add the pack file\n+(git-rev-list --objects $gitrevargs | \\\n+\tcut -b -40 | \\\n+\tgit pack-objects --all-progress --progress --stdout >.gitBundlePack) \\\n+\t|| (rm -f \"$bfile\" ; exit)\n+zip -m \"$bfile\" .gitBundlePack\n+\n+# done\n+echo \"Created $bfile\"\n-- \n1.5.0.rc3.24.g0c5e\n"},{"id":"34527","messageId":"11714622293142-git-send-email-mdl123@verizon.net","threadId":"6818","inReplyTo":"11714622292295-git-send-email-mdl123@verizon.net","subject":"[PATCH] git-unbundle - unbundle objects and references for disconnected transfer.","fromName":"Mark Levedahl","fromEmail":"mdl123@verizon.net","sentAt":"2007-02-14T14:10:27Z","receivedAt":"2007-02-14T14:10:27Z","isPatch":true,"sender":{"key":"mdl123@verizon.net","avatar":"https://avatars.githubusercontent.com/u/5302462?v=4"},"body":"Some workflows require coordinated development between repositories on\nmachines that can never be connected. This utility unpacks a bundle\ncontaining objects and associated references (heads or tags) into the\ncurrent repository, effectively supporting git-push like operations\nbetween disconnected systems.\n\nSigned-off-by: Mark Levedahl <mdl123@verizon.net>\n---\n git-unbundle |   75 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++\n 1 files changed, 75 insertions(+), 0 deletions(-)\n create mode 100755 git-unbundle\n\ndiff --git a/git-unbundle b/git-unbundle\nnew file mode 100755\nindex 0000000..5ea4ae6\n--- /dev/null\n+++ b/git-unbundle\n@@ -0,0 +1,75 @@\n+#!/bin/sh\n+# unpack a git-bundle file into current repository\n+#\n+# See git-bundle.\n+\n+die() {\n+\techo >&2 \"$@\"\n+\texit 1\n+}\n+\n+bfile=bundle.zip\n+force=\n+shallow=\n+while case \"$#\" in 0) break ;; esac\n+do\n+\tcase \"$1\" in\n+\t--bare)\n+\t\texport GIT_DIR=.;;\n+\t-f|--f|--fo|--for|--forc|--force)\n+\t\tforce=1;;\n+\t-h|--h|--he|--hel|--help)\n+\t\techo \"usage: git-unbundle [--bare] [-f|--force] [--shallow] [bundle (default is bundle.zip)]\"\n+\t\texit;;\n+\t-s|--s|--sh|--sha|--shal|--shall|--shallo|--shallow)\n+\t\tshallow=1;;\n+\t-*)\n+\t\tdie \"unknown option: $1\";;\n+\t*)\n+\t\tbfile=\"$1\";;\n+\tesac\n+\tshift\n+done\n+\n+[ -e \"$bfile\" ] || die \"cannot find $bfile\"\n+GIT_DIR=$(git-rev-parse --git-dir) || die \"Not in a git directory\"\n+\n+# get the objects\n+unzip -p \"$bfile\" .gitBundlePack | git-unpack-objects\n+\n+# check each reference, assure that the result would be valid before updating local ref\n+unzip -p \"$bfile\" .gitBundleReferences | while read sha1 ref ; do\n+\tif [ -z \"$shallow\" ] ; then\n+\t\tresult=$(git fsck $sha1)\n+\t\thavemissing=$(echo \"$result\" | grep '^missing')\n+\telse\n+\t\t# accept a shallow transfer\n+\t\thavemissing=\n+\tfi\n+\tok=\n+\tif [ ! -z \"$havemissing\" ] ; then\n+\t\techo \"Not updating: $ref to $sha1\"\n+\t\techo \"Bundle does not contain all required objects. (possibly partial) errors:\"\n+\t\techo \"$result\" | head\n+\telif [ -z \"$force\" ] ; then\n+\t\t# update only if non-fastforward\n+\t\tlocal=$(git-rev-parse --verify \"$ref^0\" 2>/dev/null)\n+\t\tif [ ! -z \"$local\" ] ; then\n+\t\t\tmb=$(git-merge-base $local $sha1)\n+\t\t\tif [ \"$mb\" != \"$local\" ] ; then\n+\t\t\t\techo \"Not applying non-fast forward update: $ref\"\n+\t\t\telse\n+\t\t\t\tok=1\n+\t\t\tfi\n+\t\telse\n+\t\t\tok=1\n+\t\tfi\n+\telse\n+\t\t#forced, accept non-fast forward update\n+\t\tok=1\n+\tfi\n+\tif [ ! -z \"$ok\" ] ; then\n+\t\techo \"updating: $ref to $sha1\"\n+\t\tgit-update-ref -m \"git-unbundle update\" $ref $sha1\n+\tfi\n+done\n-- \n1.5.0.rc3.24.g0c5e\n"},{"id":"34530","messageId":"11714622291709-git-send-email-mdl123@verizon.net","threadId":"6818","inReplyTo":"11714622293142-git-send-email-mdl123@verizon.net","subject":"[PATCH] Create a man page for git-bundle.","fromName":"Mark Levedahl","fromEmail":"mdl123@verizon.net","sentAt":"2007-02-14T14:10:28Z","receivedAt":"2007-02-14T14:10:28Z","isPatch":true,"sender":{"key":"mdl123@verizon.net","avatar":"https://avatars.githubusercontent.com/u/5302462?v=4"},"body":"Signed-off-by: Mark Levedahl <mdl123@verizon.net>\n---\n Documentation/git-bundle.txt |   92 ++++++++++++++++++++++++++++++++++++++++++\n 1 files changed, 92 insertions(+), 0 deletions(-)\n create mode 100644 Documentation/git-bundle.txt\n\ndiff --git a/Documentation/git-bundle.txt b/Documentation/git-bundle.txt\nnew file mode 100644\nindex 0000000..2cec0f9\n--- /dev/null\n+++ b/Documentation/git-bundle.txt\n@@ -0,0 +1,92 @@\n+git-bundle(1)\n+================\n+\n+NAME\n+----\n+git-bundle - Package objects and refs to update a disconnected repository\n+\n+\n+SYNOPSIS\n+--------\n+'git-bundle' [ --bare ] [--output=file] <git-rev-list args>\n+\n+DESCRIPTION\n+-----------\n+\n+Some workflows require that one or more branches of development on one machine\n+be replicated on another machine, but the two machines cannot be directly\n+connected so the git-fetch protocol cannot be used.  This command creates a\n+bundle file containing objects and references that can be used to update another\n+repository (using gitlink:git-unbundle[1]) without phsyically connecting the\n+two.  As no direct connection exists, the user must specify a basis for the\n+bundle that is held by the destination repository: the bundle assumes that all\n+objects in the basis are already in the destination repository.\n+\n+OPTIONS\n+-------\n+\n+--bare::\n+\tAssume operation in a bare repository.\n+\n+--output=file::\n+\tSpecifies the name of the bundle file. Default is \"bundle.zip\" in the\n+\tcurrent directory.\n+\n+\n+<git-rev-list args>::\n+\n+        A list of arguments, accepatble to git-rev-parse and git-rev-list, that\n+        specify the specific objects and references to transport. For example,\n+        \"master~10..master\" causes the current master reference to be packaged\n+        along with all objects added since its 10th ancestor commit. There is no\n+        explicit limit to the number of references and objects that may be\n+        packaged.\n+\n+\n+SPECIFYING REFERENCES\n+--------------------\n+\n+git-bundle will only package references that are shown by git-show-ref: this\n+includes heads, tags, and remote heads. References such as master~1 cannot be\n+packaged, but are perfectly suitable for defining the basis. More than one\n+reference may be packaged, and more than one basis can be specified. The objects\n+packaged are those not contained in the union of the given bases. Each basis can\n+be specified explicitly (e.g., ^master~10), or implicitly (e.g.,\n+master~10..master).\n+\n+In general, it is very important that the basis used be held by the destination.\n+It is ok to err on the side of conservatism, causing the bundle file to contain\n+objects already in the destination as these are ignored when unpacking at the\n+destination.\n+\n+A shallow copy or clone can be done which has fewer than all required objects.\n+Typically, this would use \"git-bundle --since=<some date> master\", and use the\n+--shallow option to git-unbundle at the receiving end.\n+\n+EXAMPLE\n+-------\n+\n+Assume two repositories exist as R1 on machine A, and R2 on machine B.  For\n+whatever reason, direct connection between A and B is not allowed, but we can\n+move data from A to B via some mechanism (CD, email, etc).  We want to update R2\n+with developments made on branch master in R1.  We set a tag in R1\n+(lastR2bundle) after the previous such transport, and move it afterwards to help\n+build the bundle.\n+\n+in R1 on A:\n+git-bundle master ^lastR2bundle\n+git tag -f lastR2bundle master\n+\n+[move bundle.zip from A to B by some mechanism]\n+\n+in R2 on B:\n+git-unbundle bundle.zip (3)\n+\n+\n+Author\n+------\n+Written by Mark Levedahl <mdl123@verizon.net>\n+\n+GIT\n+---\n+Part of the gitlink:git[7] suite\n-- \n1.5.0.rc3.24.g0c5e\n"},{"id":"34528","messageId":"1171462229415-git-send-email-mdl123@verizon.net","threadId":"6818","inReplyTo":"11714622291709-git-send-email-mdl123@verizon.net","subject":"[PATCH] Create a man page for git-unbundle.","fromName":"Mark Levedahl","fromEmail":"mdl123@verizon.net","sentAt":"2007-02-14T14:10:29Z","receivedAt":"2007-02-14T14:10:29Z","isPatch":true,"sender":{"key":"mdl123@verizon.net","avatar":"https://avatars.githubusercontent.com/u/5302462?v=4"},"body":"Signed-off-by: Mark Levedahl <mdl123@verizon.net>\n---\n Documentation/git-unbundle.txt |   55 ++++++++++++++++++++++++++++++++++++++++\n 1 files changed, 55 insertions(+), 0 deletions(-)\n create mode 100644 Documentation/git-unbundle.txt\n\ndiff --git a/Documentation/git-unbundle.txt b/Documentation/git-unbundle.txt\nnew file mode 100644\nindex 0000000..8dbfccb\n--- /dev/null\n+++ b/Documentation/git-unbundle.txt\n@@ -0,0 +1,55 @@\n+git-unbundle(1)\n+================\n+\n+NAME\n+----\n+git-unbundle - Unpackage objects and refs to update a disconnected repository\n+\n+\n+SYNOPSIS\n+--------\n+'git-unbundle' [--bare ] [--force] [--shallow] file\n+\n+DESCRIPTION\n+-----------\n+\n+Some workflows require that one or more branches of development on one machine\n+be replicated on another machine, but the two machines cannot be directly\n+connected so the gitlink:git-fetch[1] protocol cannot be used.  This command\n+unpacks a bundle file created by gitlink:git-bundle[1] on another repository,\n+adding the objects and updating references as defined by the donor repository.\n+\n+OPTIONS\n+-------\n+\n+--bare::\n+\tAssume operation in a bare repository.\n+\n+--force::\n+        Normally only fast-forward reference updates are performed. Specifying\n+        this option allows non-fast forward updates.\n+\n+--shallow::\n+        Normally, git-fsck is invoked on each reference to assure there are no\n+        missing objects. This option bypasses that checking, allowing shallow\n+        copies. Use with caution, many git operations are not supported on\n+        shallow repositories.\n+\n+file::\n+        Bundle file created by gitlink:git-bundle[1]. Default is bundle.zip.\n+\n+ERROR CHECKING\n+--------------\n+\n+In addition to the checks mentioned under --force and --shallow above,\n+git-unbundle uses gitlink:git-unpack-objects[1] to update objects, and\n+gitlink:git-update-ref to update all references, and thus all the inherent\n+safety checks provided by those functions are in force.\n+\n+Author\n+------\n+Written by Mark Levedahl <mdl123@verizon.net>\n+\n+GIT\n+---\n+Part of the gitlink:git[7] suite\n-- \n1.5.0.rc3.24.g0c5e\n"},{"id":"34531","messageId":"vpq1wksbzf5.fsf@olympe.imag.fr","threadId":"6818","inReplyTo":"11714622292110-git-send-email-mdl123@verizon.net","subject":"Re: Scripts to use \"bundles\" for moving data between repositories","fromName":"Matthieu Moy","fromEmail":"matthieu.moy@imag.fr","sentAt":"2007-02-14T14:13:02Z","receivedAt":"2007-02-14T14:13:02Z","isPatch":false,"sender":{"key":"git@matthieu-moy.fr","avatar":"https://avatars.githubusercontent.com/u/14709?v=4"},"body":"Mark Levedahl <mdl123@verizon.net> writes:\n\n> I offer this for inclusion in the main distribution, comments and suggestions\n> for improvement are welcome regardless. The scripts are working for me today\n> and I find them very useful.\n\nDid you also have a look at\n\nhttp://kernel.org/git/?p=cogito/cogito-bundle.git\n\n-- \nMatthieu\n"},{"id":"34534","messageId":"45D31EBE.9090106@verizon.net","threadId":"6818","inReplyTo":"vpq1wksbzf5.fsf@olympe.imag.fr","subject":"Re: Scripts to use \"bundles\" for moving data between repositories","fromName":"Mark Levedahl","fromEmail":"mdl123@verizon.net","sentAt":"2007-02-14T14:37:50Z","receivedAt":"2007-02-14T14:37:50Z","isPatch":false,"sender":{"key":"mdl123@verizon.net","avatar":"https://avatars.githubusercontent.com/u/5302462?v=4"},"body":"Matthieu Moy wrote:\n>> Did you also have a look at\n>>\n>> http://kernel.org/git/?p=cogito/cogito-bundle.git\n>>\n>>     \nYes, I did. I rejected that for my use as it seemed much too \nrestrictive: 1 branch at a time, no tags. What I wrote can pack up \neverything in a repository in one go if so desired, or any subpiece. \nAlso, this requires nothing beyond git-core (no dependency upon cg), and \ncg and git-core do not interoperate well regarding remote branch \ndefinitions.\n\nMark\n"},{"id":"34538","messageId":"Pine.LNX.4.63.0702141645270.22628@wbgn013.biozentrum.uni-wuerzburg.de","threadId":"6818","inReplyTo":"11714622292110-git-send-email-mdl123@verizon.net","subject":"Re: Scripts to use \"bundles\" for moving data between repositories","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2007-02-14T15:46:58Z","receivedAt":"2007-02-14T15:46:58Z","isPatch":false,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi,\n\nOn Wed, 14 Feb 2007, Mark Levedahl wrote:\n\n> I am working a project using git where we have many repositories on \n> machines that can never be directly connected, but which need to have \n> the same objects and development history.\n\nI had the same problem some time ago. My network is a sneaker \nnet, and the transport medium is a USB stick.\n\nThen, I just push onto the stick and pull from the stick as needed.\n\nCiao,\nDscho\n"},{"id":"34572","messageId":"7vlkj0r7f0.fsf@assigned-by-dhcp.cox.net","threadId":"6818","inReplyTo":"11714622292110-git-send-email-mdl123@verizon.net","subject":"Re: Scripts to use \"bundles\" for moving data between repositories","fromName":"Junio C Hamano","fromEmail":"junkio@cox.net","sentAt":"2007-02-14T17:11:15Z","receivedAt":"2007-02-14T17:11:15Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"I think something like this is a good addition but I do not want\nus to require zip/unzip to use git.  I think saner alternative\nwould be to use tar.\n"},{"id":"34588","messageId":"45D34D58.8050701@verizon.net","threadId":"6818","inReplyTo":"7vlkj0r7f0.fsf@assigned-by-dhcp.cox.net","subject":"Re: Scripts to use \"bundles\" for moving data between repositories","fromName":"Mark Levedahl","fromEmail":"mdl123@verizon.net","sentAt":"2007-02-14T17:56:40Z","receivedAt":"2007-02-14T17:56:40Z","isPatch":false,"sender":{"key":"mdl123@verizon.net","avatar":"https://avatars.githubusercontent.com/u/5302462?v=4"},"body":"Junio C Hamano wrote:\n> I think something like this is a good addition but I do not want\n> us to require zip/unzip to use git.  I think saner alternative\n> would be to use tar.\n>\n>\n>   \nThat is any easy change. As the dominant content is an already \ncompressed pack file, is tar sufficient or should it be a gzip or bzip tar?\n\nMark\n"},{"id":"34591","messageId":"7vhctopqkw.fsf@assigned-by-dhcp.cox.net","threadId":"6818","inReplyTo":"45D34D58.8050701@verizon.net","subject":"Re: Scripts to use \"bundles\" for moving data between repositories","fromName":"Junio C Hamano","fromEmail":"junkio@cox.net","sentAt":"2007-02-14T18:00:15Z","receivedAt":"2007-02-14T18:00:15Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Mark Levedahl <mdl123@verizon.net> writes:\n\n> That is any easy change. As the dominant content is an already\n> compressed pack file, is tar sufficient or should it be a gzip or bzip\n> tar?\n\nPlain vanilla would do.  Have you noticed how well they deflate\nwith your implementation that uses zip?\n\nAlso we _might_ want to uuencode (or base85) so that you can\neven e-mail a bundle easily.\n\nI am 75% kidding ;-).\n"},{"id":"34601","messageId":"7v8xf0ppmf.fsf@assigned-by-dhcp.cox.net","threadId":"6818","inReplyTo":"45D34D58.8050701@verizon.net","subject":"Re: Scripts to use \"bundles\" for moving data between repositories","fromName":"Junio C Hamano","fromEmail":"junkio@cox.net","sentAt":"2007-02-14T18:20:56Z","receivedAt":"2007-02-14T18:20:56Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Mark Levedahl <mdl123@verizon.net> writes:\n\n> Junio C Hamano wrote:\n>> I think something like this is a good addition but I do not want\n>> us to require zip/unzip to use git.  I think saner alternative\n>> would be to use tar.\n>>\n>>\n>>\n> That is any easy change. As the dominant content is an already\n> compressed pack file, is tar sufficient or should it be a gzip or bzip\n> tar?\n\nIf you are re-spinning the patch, please do not forget that you\nwould want to link it in the main Makefile and link the docs to\ngit.7 by adding them to Documentation/cmd-list.perl.\n\nThe USAGE string should fit comfortably on 80-column terminal.\nThe same goes for AsciiDoc text documentation.  Your lines are\ntoo long.\n"},{"id":"34612","messageId":"20070214194237.GC28290@spearce.org","threadId":"6818","inReplyTo":"11714622292295-git-send-email-mdl123@verizon.net","subject":"Re: [PATCH] git-bundle - bundle objects and references for disconnected transfer.","fromName":"Shawn O. Pearce","fromEmail":"spearce@spearce.org","sentAt":"2007-02-14T19:42:37Z","receivedAt":"2007-02-14T19:42:37Z","isPatch":true,"sender":{"key":"spearce@spearce.org","avatar":"https://avatars.githubusercontent.com/u/34844?v=4"},"body":"Mark Levedahl <mdl123@verizon.net> wrote:\n> +# add the pack file\n> +(git-rev-list --objects $gitrevargs | \\\n> +\tcut -b -40 | \\\n> +\tgit pack-objects --all-progress --progress --stdout >.gitBundlePack) \\\n> +\t|| (rm -f \"$bfile\" ; exit)\n\npack-objects can run a rev-list internally; which means this\ncan be written as:\n\n echo $gitrevargs | \\\n git pack-objects --all-progress --progress --stdout --revs >.gitBundlePack \\\n || (rm -f \"$bfile\" ; exit)\n\n-- \nShawn.\n"},{"id":"34614","messageId":"20070214194537.GD28290@spearce.org","threadId":"6818","inReplyTo":"11714622293142-git-send-email-mdl123@verizon.net","subject":"Re: [PATCH] git-unbundle - unbundle objects and references for disconnected transfer.","fromName":"Shawn O. Pearce","fromEmail":"spearce@spearce.org","sentAt":"2007-02-14T19:45:37Z","receivedAt":"2007-02-14T19:45:37Z","isPatch":true,"sender":{"key":"spearce@spearce.org","avatar":"https://avatars.githubusercontent.com/u/34844?v=4"},"body":"Mark Levedahl <mdl123@verizon.net> wrote:\n> +# get the objects\n> +unzip -p \"$bfile\" .gitBundlePack | git-unpack-objects\n\nSince you are transporting a packfile by sneakernet it might\nbe reasonable to assume this transfer happens infrequently.\nConsequently we might assume its object count exceeds\ntransfer.unpackLimit, which means a standard fetch or push would\nhave kept the packfile rather than unpacking it to loose objects.\n\nSo maybe use git-index-pack here to index the packfile and\nretain it as-is, rather than unpacking it?\n\n-- \nShawn.\n"},{"id":"34625","messageId":"45D377A8.1080404@verizon.net","threadId":"6818","inReplyTo":"20070214194537.GD28290@spearce.org","subject":"Re: [PATCH] git-unbundle - unbundle objects and references for disconnected transfer.","fromName":"Mark Levedahl","fromEmail":"mdl123@verizon.net","sentAt":"2007-02-14T20:57:12Z","receivedAt":"2007-02-14T20:57:12Z","isPatch":true,"sender":{"key":"mdl123@verizon.net","avatar":"https://avatars.githubusercontent.com/u/5302462?v=4"},"body":"Shawn O. Pearce wrote:\n> Mark Levedahl <mdl123@verizon.net> wrote:\n>   \n>> +# get the objects\n>> +unzip -p \"$bfile\" .gitBundlePack | git-unpack-objects\n>>     \n>\n> Since you are transporting a packfile by sneakernet it might\n> be reasonable to assume this transfer happens infrequently.\n> Consequently we might assume its object count exceeds\n> transfer.unpackLimit, which means a standard fetch or push would\n> have kept the packfile rather than unpacking it to loose objects.\n>\n> So maybe use git-index-pack here to index the packfile and\n> retain it as-is, rather than unpacking it?\n>\n>   \nMany of my uses of this result in 10-20 objects being transferred, so \nI'm not sure keeping each pack is a real benefit. In particular, one use \nis for daily updates between two sites via email where we tend to have a \nlot of extra objects in the packs as we assume that not every bundle \nactually gets applied, while the number of real new objects tends to be \nsmall. On the other hand, given the manual nature of this operation, we \ncould always just follow up with repack -a -d, possibly guarded by a git \ncount. Thoughts?\n\nMark\n"},{"id":"34626","messageId":"20070214210300.GF28290@spearce.org","threadId":"6818","inReplyTo":"45D377A8.1080404@verizon.net","subject":"Re: [PATCH] git-unbundle - unbundle objects and references for disconnected transfer.","fromName":"Shawn O. Pearce","fromEmail":"spearce@spearce.org","sentAt":"2007-02-14T21:03:00Z","receivedAt":"2007-02-14T21:03:00Z","isPatch":true,"sender":{"key":"spearce@spearce.org","avatar":"https://avatars.githubusercontent.com/u/34844?v=4"},"body":"Mark Levedahl <mdl123@verizon.net> wrote:\n> Many of my uses of this result in 10-20 objects being transferred, so \n> I'm not sure keeping each pack is a real benefit. In particular, one use \n> is for daily updates between two sites via email where we tend to have a \n> lot of extra objects in the packs as we assume that not every bundle \n> actually gets applied, while the number of real new objects tends to be \n> small. On the other hand, given the manual nature of this operation, we \n> could always just follow up with repack -a -d, possibly guarded by a git \n> count. Thoughts?\n\nI don't really have an opinion here, as I'm fortunate enough that\nI can use an SSH or an anonymous git connection between all of my\nrepositories, and thus don't really have a need for bundle/unbundle.\n\nIts just one of those operations which I thought would not happen\noften, and when it did, probably would be big.  In which case keeping\nthe packfile would make the unbundle run faster, as you don't need\nto create a huge mess of loose objects.\n\n-- \nShawn.\n"},{"id":"34632","messageId":"Pine.LNX.4.64.0702141615570.1757@xanadu.home","threadId":"6818","inReplyTo":"45D377A8.1080404@verizon.net","subject":"Re: [PATCH] git-unbundle - unbundle objects and references for disconnected transfer.","fromName":"Nicolas Pitre","fromEmail":"nico@cam.org","sentAt":"2007-02-14T21:18:46Z","receivedAt":"2007-02-14T21:18:46Z","isPatch":true,"sender":{"key":"nico@fluxnic.net","avatar":"https://avatars.githubusercontent.com/u/702790?v=4"},"body":"On Wed, 14 Feb 2007, Mark Levedahl wrote:\n\n> Shawn O. Pearce wrote:\n> > Mark Levedahl <mdl123@verizon.net> wrote:\n> >   \n> > > +# get the objects\n> > > +unzip -p \"$bfile\" .gitBundlePack | git-unpack-objects\n> > >     \n> >\n> > Since you are transporting a packfile by sneakernet it might\n> > be reasonable to assume this transfer happens infrequently.\n> > Consequently we might assume its object count exceeds\n> > transfer.unpackLimit, which means a standard fetch or push would\n> > have kept the packfile rather than unpacking it to loose objects.\n> >\n> > So maybe use git-index-pack here to index the packfile and\n> > retain it as-is, rather than unpacking it?\n> >\n> >   \n> Many of my uses of this result in 10-20 objects being transferred, so I'm not\n> sure keeping each pack is a real benefit. In particular, one use is for daily\n> updates between two sites via email where we tend to have a lot of extra\n> objects in the packs as we assume that not every bundle actually gets applied,\n> while the number of real new objects tends to be small. On the other hand,\n> given the manual nature of this operation, we could always just follow up with\n> repack -a -d, possibly guarded by a git count. Thoughts?\n\nSince this is meant for manual operation and therefore is not meant to \nhappen multiple times per minute, I'd suggest you still use index-pack \nunconditionally instead of unpack-objects despite having a small number \nof objects.\n\n\nNicolas\n"},{"id":"34634","messageId":"45D37DF0.70708@verizon.net","threadId":"6818","inReplyTo":"7vhctopqkw.fsf@assigned-by-dhcp.cox.net","subject":"Re: Scripts to use \"bundles\" for moving data between repositories","fromName":"Mark Levedahl","fromEmail":"mdl123@verizon.net","sentAt":"2007-02-14T21:24:00Z","receivedAt":"2007-02-14T21:24:00Z","isPatch":false,"sender":{"key":"mdl123@verizon.net","avatar":"https://avatars.githubusercontent.com/u/5302462?v=4"},"body":"Junio C Hamano wrote:\n> Also we _might_ want to uuencode (or base85) so that you can\n> even e-mail a bundle easily.\n>\n> I am 75% kidding ;-).\n>   \nWouldn't such encoding more logically be a part of whatever is used for \nthe transport? My experience (other than emailing patches to the git \nlist :-[ ) in this area is that encode / decode of attachments is \nhandled transparently by email clients. My email programs (including \ncommand line scripts) all know how to mime encode / decode arbitrary \nattachments, so I at least would gain nothing by such encoding.\n\nMark\n"},{"id":"34636","messageId":"7vejosmnvw.fsf@assigned-by-dhcp.cox.net","threadId":"6818","inReplyTo":"45D37DF0.70708@verizon.net","subject":"Re: Scripts to use \"bundles\" for moving data between repositories","fromName":"Junio C Hamano","fromEmail":"junkio@cox.net","sentAt":"2007-02-14T21:26:43Z","receivedAt":"2007-02-14T21:26:43Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Exactly, that is why I said I was 75% kidding.\n"},{"id":"34642","messageId":"Pine.LNX.4.63.0702142238310.22628@wbgn013.biozentrum.uni-wuerzburg.de","threadId":"6818","inReplyTo":"11714622292295-git-send-email-mdl123@verizon.net","subject":"Re: [PATCH] git-bundle - bundle objects and references for disconnected transfer.","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2007-02-14T21:58:15Z","receivedAt":"2007-02-14T21:58:15Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi,\n\nOn Wed, 14 Feb 2007, Mark Levedahl wrote:\n\n> +bfile=bundle.zip\n> +for arg in $myargs ; do\n> +\tcase \"$arg\" in\n> +\t\t--bare)\n> +\t\t\texport GIT_DIR=.;;\n\nThis is not necessary. You should do this instead:\n\n\t. git-sh-setup\n\nIt should autodetect if you are running in a bare repo. Also, it gives you \nthe nice die and help functions.\n\n> +\t\t-h|--h|--he|--hel|--help)\n> +\t\t\techo \"$USAGE\"\n> +\t\t\texit;;\n> +\t\t--output=*)\n> +\t\t\tbfile=${arg##--output=};;\n\nThroughout git, we seem to do both \"--output=<bla>\" _and_ \"--output <bla>\" \nforms, or just the latter.\n\n> +GIT_DIR=$(git-rev-parse --git-dir) || die \"Not in a git directory\"\n\nAgain, this is done by git-sh-setup\n\n> +git-show-ref $refs > .gitBundleReferences\n\nWould it not be better to say explicitely which refs are expected to be \npresent already (they start with \"^\" in the output of `git-rev-parse`, but \nyou would need to do a bit more work, since you cannot just take the \nsymbolic names).\n\nSome general remarks:\n\nIt would be so much nicer if you worked without temporary files (you could \ndo that by starting the file with the refs, then have an empty line, and \nthen just pipe the pack after that).\n\nIMHO reliance on $(git fsck | grep ^missing) is not good. The file check \nmight take very, very long, or use much memory. And you _can_ do better \n[*1*].\n\nAlso, your use of shallow is incorrect. If the boundary commits are \npresent, you might just leave them as-are, but if they are not present, \nyou have to mark them as shallow. Otherwise, you end up with a corrupt \n(not shallow) repository.\n\nCiao,\nDscho\n\n[*1*] Instead of providing a list \"<hash> <refname>\" with just the refs to \nbe updated, append a list \"<hash> ^<refname>\" with the refs which _have_ \nto be present in order to succeed. You get this list by\n\n\tgitrevnotargs=$(git-rev-parse --symbolic --revs-only --not $*)\n\tgit show-ref $gitrevnotargs | sed 's/^\\(.\\{41\\}\\)/&^/'\n"},{"id":"34651","messageId":"45D39081.2010709@verizon.net","threadId":"6818","inReplyTo":"20070214210300.GF28290@spearce.org","subject":"Re: [PATCH] git-unbundle - unbundle objects and references for disconnected transfer.","fromName":"Mark Levedahl","fromEmail":"mdl123@verizon.net","sentAt":"2007-02-14T22:43:13Z","receivedAt":"2007-02-14T22:43:13Z","isPatch":true,"sender":{"key":"mdl123@verizon.net","avatar":"https://avatars.githubusercontent.com/u/5302462?v=4"},"body":"Shawn O. Pearce wrote:\n> I don't really have an opinion here, as I'm fortunate enough that\n> I can use an SSH or an anonymous git connection between all of my\n> repositories, and thus don't really have a need for bundle/unbundle.\n>\n> Its just one of those operations which I thought would not happen\n> often, and when it did, probably would be big.  In which case keeping\n> the packfile would make the unbundle run faster, as you don't need\n> to create a huge mess of loose objects.\nFair enough - I've made that change, also your other suggestion about \npiping refs directly to pack-objects. Thanks for both.\n\nMark\n"},{"id":"34653","messageId":"45D398F9.6070205@verizon.net","threadId":"6818","inReplyTo":"Pine.LNX.4.63.0702142238310.22628@wbgn013.biozentrum.uni-wuerzburg.de","subject":"Re: [PATCH] git-bundle - bundle objects and references for disconnected transfer.","fromName":"Mark Levedahl","fromEmail":"mdl123@verizon.net","sentAt":"2007-02-14T23:19:21Z","receivedAt":"2007-02-14T23:19:21Z","isPatch":true,"sender":{"key":"mdl123@verizon.net","avatar":"https://avatars.githubusercontent.com/u/5302462?v=4"},"body":"Johannes Schindelin wrote:\n> This is not necessary. You should do this instead:\n>\n> \t. git-sh-setup\n>   \nI debated that, it seemed a wash but can do.\n> Throughout git, we seem to do both \"--output=<bla>\" _and_ \"--output <bla>\" \n> forms, or just the latter.\n>   \nPatches gratefully accepted for that. This exceeds my skills in bash: I \ncan do that in python, C, or other languages, but in bash I am working \nthrough a list that is a part of $* an arg at a time with no ability to \nlook at the next, which is what this needs. Unless of course bash arrays \nare part of portable shell (not sure on that).\n>> +git-show-ref $refs > .gitBundleReferences\n>>     \n>\n> Would it not be better to say explicitely which refs are expected to be \n> present already (they start with \"^\" in the output of `git-rev-parse`, but \n> you would need to do a bit more work, since you cannot just take the \n> symbolic names).\n>\n> Some general remarks:\n>\n> It would be so much nicer if you worked without temporary files (you could \n> do that by starting the file with the refs, then have an empty line, and \n> then just pipe the pack after that).\n>   \nOriginally, this was in python with zip file built in memory (no \ntemporaries). Sticking to portable shell makes many easy things really \nhard. I'll think about this.\n> IMHO reliance on $(git fsck | grep ^missing) is not good. The file check \n> might take very, very long, or use much memory. And you _can_ do better \n> [*1*].\n>   \nGood idea, but I think it is simpler to just keep the ^... output from \ngit-rev-parse and check that those exist. What you suggest below seems \nto presume all bases are themselves references, which is not the case \nwhen doing, for example, master~10..master.\n> Also, your use of shallow is incorrect. If the boundary commits are \n> present, you might just leave them as-are, but if they are not present, \n> you have to mark them as shallow. Otherwise, you end up with a corrupt \n> (not shallow) repository.\n>   \nI have to say I do not understand what \"mark them as shallow\" means: can \nyou please enlighten me further?\n\n\nThanks,\nMark\n"},{"id":"34656","messageId":"45D3A18E.40309@verizon.net","threadId":"6818","inReplyTo":"45D398F9.6070205@verizon.net","subject":"Re: [PATCH] git-bundle - bundle objects and references for disconnected transfer.","fromName":"Mark Levedahl","fromEmail":"mdl123@verizon.net","sentAt":"2007-02-14T23:55:58Z","receivedAt":"2007-02-14T23:55:58Z","isPatch":true,"sender":{"key":"mdl123@verizon.net","avatar":"https://avatars.githubusercontent.com/u/5302462?v=4"},"body":"Mark Levedahl wrote:\n> Johannes Schindelin wrote:\n>\n>>\n>> Would it not be better to say explicitely which refs are expected to \n>> be present already (they start with \"^\" in the output of \n>> `git-rev-parse`, but you would need to do a bit more work, since you \n>> cannot just take the symbolic names).\n>>\n>> IMHO reliance on $(git fsck | grep ^missing) is not good. The file \n>> check might take very, very long, or use much memory. And you _can_ \n>> do better [*1*].\n>>   \n> Good idea, but I think it is simpler to just keep the ^... output from \n> git-rev-parse and check that those exist. What you suggest below seems \n> to presume all bases are themselves references, which is not the case \n> when doing, for example, master~10..master.\nExamining further, I just don't know how to do this in shell. Basically, \nwhat I want is the list of parents of all bases, but those bases might \nnot be explicitly mentioned, e.g., master --since=10.days.ago and I \ndon't understand any direct plumbing call that will give me the list of \nparents in the general case. At the expense of extreme slowness I can do \nsome of this invoking sort and uniq with long lists of objects. The pain \nin doing that on the sender side is definitely worth the potential gain \non the receiver side (I now remember I tried that a while back, was able \nto do something reasonable in Python using sets, it died in bash).\n\nMark\n"},{"id":"34658","messageId":"Pine.LNX.4.63.0702150054210.22628@wbgn013.biozentrum.uni-wuerzburg.de","threadId":"6818","inReplyTo":"45D398F9.6070205@verizon.net","subject":"Re: [PATCH] git-bundle - bundle objects and references for disconnected transfer.","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2007-02-15T00:07:23Z","receivedAt":"2007-02-15T00:07:23Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi,\n\nOn Wed, 14 Feb 2007, Mark Levedahl wrote:\n\n> Johannes Schindelin wrote:\n> > This is not necessary. You should do this instead:\n> > \n> > \t. git-sh-setup\n> >   \n> I debated that, it seemed a wash but can do.\n\nIt makes things easier, doesn't it?\n\n> > Throughout git, we seem to do both \"--output=<bla>\" _and_ \"--output <bla>\"\n> > forms, or just the latter.\n> >   \n> Patches gratefully accepted for that. This exceeds my skills in bash: I can\n> do that in python, C, or other languages, but in bash I am working through a\n> list that is a part of $* an arg at a time with no ability to look at the\n> next, which is what this needs. Unless of course bash arrays are part of\n> portable shell (not sure on that).\n\nAh, I just realized that you do not shift. This is wrong. For example,\n\n\tgit bundle --output=a1 a..b\n\nwould pass \"--output=a1 a..b\" to git-rev-parse. While you say \n\"--revs-only\", this would work, but so would \"these are no refs\". You lose \nvaluable information that way (namely invalid parameters). The standard \nshell way is nicely visible in git-tag.sh (see the while loop). It is \nbasically\n\nwhile case \"$#\" in 0) break ;; esac\ndo\n\tcase \"$1\" in\n\t--output)\n\t\t# handle $1 (and check that you can write to it).\n\t\t;;\n\t-*)\n\t\tusage\n\t\t;;\n\t*)\n\t\tbreak\n\tesac\ndone\n\n> > > +git-show-ref $refs > .gitBundleReferences\n> > >     \n> > \n> > Would it not be better to say explicitely which refs are expected to be\n> > present already (they start with \"^\" in the output of `git-rev-parse`, but\n> > you would need to do a bit more work, since you cannot just take the\n> > symbolic names).\n> > \n> > Some general remarks:\n> > \n> > It would be so much nicer if you worked without temporary files (you \n> > could do that by starting the file with the refs, then have an empty \n> > line, and then just pipe the pack after that).\n> >   \n> Originally, this was in python with zip file built in memory (no \n> temporaries). Sticking to portable shell makes many easy things really \n> hard.\n\nNot if you just pipe the two parts (refs & pack) into the output. Piping \nalso allows for \"--output -\" meaning stdout...\n\n> > IMHO reliance on $(git fsck | grep ^missing) is not good. The file \n> > check might take very, very long, or use much memory. And you _can_ do \n> > better [*1*].\n>   \n> Good idea, but I think it is simpler to just keep the ^... output from\n> git-rev-parse and check that those exist. What you suggest below seems to\n> presume all bases are themselves references, which is not the case when\n> doing, for example, master~10..master.\n\nNot at all. I meant to verify that these _hashes_ exist as commits. Not \nnecessarily refs.\n\n> > Also, your use of shallow is incorrect. If the boundary commits are \n> > present, you might just leave them as-are, but if they are not \n> > present, you have to mark them as shallow. Otherwise, you end up with \n> > a corrupt (not shallow) repository.\n> >   \n> I have to say I do not understand what \"mark them as shallow\" means: can \n> you please enlighten me further?\n\nWe have shallow clones. This means that you can mark commits as \"fake \nroot\" commits, i.e. even if they have parents, they are treated as if they \nhad no parents. You do this by adding the hashes of the shallow commits to \n.git/shallow. For a short description, search for \"shallow\" in \nDocumentation/glossary.txt.\n\nCiao,\nDscho\n"},{"id":"34659","messageId":"Pine.LNX.4.63.0702150114230.22628@wbgn013.biozentrum.uni-wuerzburg.de","threadId":"6818","inReplyTo":"45D3A18E.40309@verizon.net","subject":"Re: [PATCH] git-bundle - bundle objects and references for disconnected transfer.","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2007-02-15T00:15:33Z","receivedAt":"2007-02-15T00:15:33Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi,\n\nOn Wed, 14 Feb 2007, Mark Levedahl wrote:\n\n> Mark Levedahl wrote:\n> > Johannes Schindelin wrote:\n> > \n> > > \n> > > Would it not be better to say explicitely which refs are expected to be\n> > > present already (they start with \"^\" in the output of `git-rev-parse`,\n> > > but you would need to do a bit more work, since you cannot just take the\n> > > symbolic names).\n> > > \n> > > IMHO reliance on $(git fsck | grep ^missing) is not good. The file check\n> > > might take very, very long, or use much memory. And you _can_ do better\n> > > [*1*].\n> > >   \n> > Good idea, but I think it is simpler to just keep the ^... output from\n> > git-rev-parse and check that those exist. What you suggest below seems to\n> > presume all bases are themselves references, which is not the case when\n> > doing, for example, master~10..master.\n>\n> Examining further, I just don't know how to do this in shell. Basically, \n> what I want is the list of parents of all bases,\n\nI don't think you need the bases. If you say \"master~10..master\" on the \nsender side, you want to update master on the receiving side, _after_ you \nverified that receiver already has \"master~10\".\n\nCiao,\nDscho\n"},{"id":"34671","messageId":"45D3C1DB.9020009@verizon.net","threadId":"6818","inReplyTo":"Pine.LNX.4.63.0702150114230.22628@wbgn013.biozentrum.uni-wuerzburg.de","subject":"Re: [PATCH] git-bundle - bundle objects and references for disconnected transfer.","fromName":"Mark Levedahl","fromEmail":"mdl123@verizon.net","sentAt":"2007-02-15T02:13:47Z","receivedAt":"2007-02-15T02:13:47Z","isPatch":true,"sender":{"key":"mdl123@verizon.net","avatar":"https://avatars.githubusercontent.com/u/5302462?v=4"},"body":"Johannes Schindelin wrote:\n> Hi,\n> \n> On Wed, 14 Feb 2007, Mark Levedahl wrote:\n> \n>> Mark Levedahl wrote:\n>>> Johannes Schindelin wrote:\n>>>\n> \n> I don't think you need the bases. If you say \"master~10..master\" on the \n> sender side, you want to update master on the receiving side, _after_ you \n> verified that receiver already has \"master~10\".\n> \n> Ciao,\n> Dscho\n> \ngit>git-rev-parse master~10..master\ndc0f74905bd94b88d3b1d477e79faef7e0308fbf\n^602598fd5d8f64028f84d2772725c5e3414a112f\n\nWhich shows the new head and the commit that the destination needs. That \nis fine. But:\n\ngit>git-rev-parse master --since=10.days.ago\ndc0f74905bd94b88d3b1d477e79faef7e0308fbf\n--max-age=1170641182\n\nis not helpful: it does not tell what is expected to be on the other \nend. And I find both forms absolutely useful in the ways I use \ngit-bundle. The latter one does not tell me what is needed. The only way \nI solved that was to walk all the commits from git-rev-list, one at a \ntime, to find the parents, and keep the results not otherwise in the \nlist. I found that so terribly slow in bash I gave up on it as \nunworkable: I have found in practice my current solution of git-fsck to \nbe much faster.\n\nMark\n"},{"id":"34672","messageId":"45D3C62D.6090609@verizon.net","threadId":"6818","inReplyTo":"Pine.LNX.4.63.0702150054210.22628@wbgn013.biozentrum.uni-wuerzburg.de","subject":"Re: [PATCH] git-bundle - bundle objects and references for disconnected transfer.","fromName":"Mark Levedahl","fromEmail":"mdl123@verizon.net","sentAt":"2007-02-15T02:32:13Z","receivedAt":"2007-02-15T02:32:13Z","isPatch":true,"sender":{"key":"mdl123@verizon.net","avatar":"https://avatars.githubusercontent.com/u/5302462?v=4"},"body":"Johannes Schindelin wrote:\n> Hi,\n> \n> On Wed, 14 Feb 2007, Mark Levedahl wrote:\n> \n> \n> Ah, I just realized that you do not shift. This is wrong. For example,\n> \n> \tgit bundle --output=a1 a..b\n> \n> would pass \"--output=a1 a..b\" to git-rev-parse. While you say \n> \"--revs-only\", this would work, but so would \"these are no refs\". You lose \n> valuable information that way (namely invalid parameters). The standard \n> shell way is nicely visible in git-tag.sh (see the while loop). It is \n> basically\n> \n> while case \"$#\" in 0) break ;; esac\n> do\n> \tcase \"$1\" in\n> \t--output)\n> \t\t# handle $1 (and check that you can write to it).\n> \t\t;;\n> \t-*)\n> \t\tusage\n> \t\t;;\n> \t*)\n> \t\tbreak\n> \tesac\n> done\n\nAnd that loop would always abort on things meant for git-rev-list. I \nwant to avoid making git-bundle have to understand everything that is \nlegal to git-rev-list. The current construct does this: it lets \ngit-rev-parse remove what that function knows, aborting if something is \namiss (or aborting later in git-rev-list), leaving git-bundle's parser \nto chew on the rest. I really don't see a way out of the dilemma: either \nallow --output foo but don't barf on bad arguments, or only accept \n--output=foo and be able to trap errors, or teach git-bundle everything \nthat is valid for the other two.  (Let me write this in python, the \ndilemma is gone).\n\n>>>   \n>> Originally, this was in python with zip file built in memory (no \n>> temporaries). Sticking to portable shell makes many easy things really \n>> hard.\n> \n> Not if you just pipe the two parts (refs & pack) into the output. Piping \n> also allows for \"--output -\" meaning stdout...\n\ngit-unbundle uses no temporary files: it pipes directly from tar (was \nzip, but I've changed to tar per Junio's request).\n\nThe problem is creating the tar: I know of no way to create a tar file \nwith two separately addressable items, both created by piping in to \nstdin. If there are not two streams, I don't know how to split the data \nin sh without mangling the pack file due to sh variable substitution \nrules. So, I think the temporary file solution is a reasonable compromise.\n> \n> Not at all. I meant to verify that these _hashes_ exist as commits. Not \n> necessarily refs.\n\nSee my other note.\n\n>\n> \n> We have shallow clones. This means that you can mark commits as \"fake \n> root\" commits, i.e. even if they have parents, they are treated as if they \n> had no parents. You do this by adding the hashes of the shallow commits to \n> ..git/shallow. For a short description, search for \"shallow\" in \n> Documentation/glossary.txt.\n\nThanks.\n\n> \n> Ciao,\n> Dscho\n> \n\nMark\n"},{"id":"34712","messageId":"Pine.LNX.4.63.0702151629130.22628@wbgn013.biozentrum.uni-wuerzburg.de","threadId":"6818","inReplyTo":"45D3C62D.6090609@verizon.net","subject":"Re: [PATCH] git-bundle - bundle objects and references for disconnected transfer.","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2007-02-15T15:32:20Z","receivedAt":"2007-02-15T15:32:20Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi,\n\nOn Wed, 14 Feb 2007, Mark Levedahl wrote:\n\n> Johannes Schindelin wrote:\n> > \n> > On Wed, 14 Feb 2007, Mark Levedahl wrote:\n> > \n> > Ah, I just realized that you do not shift. This is wrong. For example,\n> > \n> > \tgit bundle --output=a1 a..b\n> > \n> > would pass \"--output=a1 a..b\" to git-rev-parse. While you say \"--revs-only\",\n> > this would work, but so would \"these are no refs\". You lose valuable\n> > information that way (namely invalid parameters). The standard shell way is\n> > nicely visible in git-tag.sh (see the while loop). It is basically\n> > \n> > while case \"$#\" in 0) break ;; esac\n> > do\n> > \tcase \"$1\" in\n> > \t--output)\n> > \t\t# handle $1 (and check that you can write to it).\n> > \t\t;;\n> > \t-*)\n> > \t\tusage\n> > \t\t;;\n> > \t*)\n> > \t\tbreak\n> > \tesac\n> > done\n> \n> And that loop would always abort on things meant for git-rev-list. I \n> want to avoid making git-bundle have to understand everything that is \n> legal to git-rev-list. The current construct does this: it lets \n> git-rev-parse remove what that function knows, aborting if something is \n> amiss (or aborting later in git-rev-list), leaving git-bundle's parser \n> to chew on the rest.\n\nWhy not force unmixing? I.e. first the options for git-bundle, _then_ the \nrest? (In that case, you would leave out the \"-*)\" clause).\n\n> > > Originally, this was in python with zip file built in memory (no \n> > > temporaries). Sticking to portable shell makes many easy things \n> > > really hard.\n> > \n> > Not if you just pipe the two parts (refs & pack) into the output. \n> > Piping also allows for \"--output -\" meaning stdout...\n> \n> git-unbundle uses no temporary files: it pipes directly from tar (was \n> zip, but I've changed to tar per Junio's request).\n\nIt does not have to be tar. There is no good reason that the parts you put \ninto the bundle have to be files, rather than header and body.\n\nCiao,\nDscho\n"},{"id":"34713","messageId":"Pine.LNX.4.63.0702151632270.22628@wbgn013.biozentrum.uni-wuerzburg.de","threadId":"6818","inReplyTo":"45D3C1DB.9020009@verizon.net","subject":"Re: [PATCH] git-bundle - bundle objects and references for disconnected transfer.","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2007-02-15T15:35:42Z","receivedAt":"2007-02-15T15:35:42Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi,\n\nOn Wed, 14 Feb 2007, Mark Levedahl wrote:\n\n> Johannes Schindelin wrote:\n> > \n> > On Wed, 14 Feb 2007, Mark Levedahl wrote:\n> > \n> > > Mark Levedahl wrote:\n> > > > Johannes Schindelin wrote:\n> > > > \n> > \n> > I don't think you need the bases. If you say \"master~10..master\" on \n> > the sender side, you want to update master on the receiving side, \n> > _after_ you verified that receiver already has \"master~10\".\n> > \n> git>git-rev-parse master~10..master\n> dc0f74905bd94b88d3b1d477e79faef7e0308fbf\n> ^602598fd5d8f64028f84d2772725c5e3414a112f\n> \n> Which shows the new head and the commit that the destination needs. That \n> is fine. But:\n> \n> git>git-rev-parse master --since=10.days.ago\n> dc0f74905bd94b88d3b1d477e79faef7e0308fbf\n> --max-age=1170641182\n> \n> is not helpful: it does not tell what is expected to be on the other \n> end. And I find both forms absolutely useful in the ways I use \n> git-bundle.\n\nYou're right.\n\nBut instead of doing this with Python or by hand, why not make the \n\"--boundary\" option useful in that case?\n\n> I have found in practice my current solution of git-fsck to be much \n> faster.\n\nIt is only faster since you unpack the objects. Which makes almost every \nother operation slow.\n\nCiao,\nDscho\n"},{"id":"34746","messageId":"45D4F6D4.9070703@verizon.net","threadId":"6818","inReplyTo":"Pine.LNX.4.63.0702151629130.22628@wbgn013.biozentrum.uni-wuerzburg.de","subject":"Re: [PATCH] git-bundle - bundle objects and references for disconnected transfer.","fromName":"Mark Levedahl","fromEmail":"mdl123@verizon.net","sentAt":"2007-02-16T00:12:04Z","receivedAt":"2007-02-16T00:12:04Z","isPatch":true,"sender":{"key":"mdl123@verizon.net","avatar":"https://avatars.githubusercontent.com/u/5302462?v=4"},"body":"Johannes Schindelin wrote:\n> Why not force unmixing? I.e. first the options for git-bundle, _then_ the \n> rest? (In that case, you would leave out the \"-*)\" clause).\n>   \nThis would just trade one usability issue for another.\n> It does not have to be tar. There is no good reason that the parts you put \n> into the bundle have to be files, rather than header and body.\n>   \nsh does not handle binary files: there is no way to split header from \nbinary payload.\n\nMark\n"},{"id":"34754","messageId":"Pine.LNX.4.63.0702160137330.22628@wbgn013.biozentrum.uni-wuerzburg.de","threadId":"6818","inReplyTo":"45D4F6D4.9070703@verizon.net","subject":"Re: [PATCH] git-bundle - bundle objects and references for disconnected transfer.","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2007-02-16T00:40:19Z","receivedAt":"2007-02-16T00:40:19Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi,\n\nOn Thu, 15 Feb 2007, Mark Levedahl wrote:\n\n> Johannes Schindelin wrote:\n> > Why not force unmixing? I.e. first the options for git-bundle, _then_ the\n> > rest? (In that case, you would leave out the \"-*)\" clause).\n> >   \n> This would just trade one usability issue for another.\n\nIt is not a usability issue if you are cleanly separating things which do \nnot belong together.\n\n> > It does not have to be tar. There is no good reason that the parts you \n> > put into the bundle have to be files, rather than header and body.\n> >   \n> sh does not handle binary files: there is no way to split header from \n> binary payload.\n\nExample:\n\n#!/bin/sh\n\n(echo Hallo; echo Bello; echo; echo blabla) | \\\n(\n\twhile read line; do\n\t\techo \"$line\"\n\t\tif [ -z \"$line\" ]; then\n\t\t\tbreak\n\t\tfi\n\tdone\n\techo \"xxx\"\n\tcat\n)\n\nIn this case, shell reads the header until an empty line is encountered. \nThe rest is piped through cat. And it does not matter if \"blabla\" is text \nor binary.\n\nCiao,\nDscho\n"},{"id":"34762","messageId":"45D523CF.1060803@verizon.net","threadId":"6818","inReplyTo":"Pine.LNX.4.63.0702160137330.22628@wbgn013.biozentrum.uni-wuerzburg.de","subject":"Re: [PATCH] git-bundle - bundle objects and references for disconnected transfer.","fromName":"Mark Levedahl","fromEmail":"mdl123@verizon.net","sentAt":"2007-02-16T03:23:59Z","receivedAt":"2007-02-16T03:23:59Z","isPatch":true,"sender":{"key":"mdl123@verizon.net","avatar":"https://avatars.githubusercontent.com/u/5302462?v=4"},"body":"Johannes Schindelin wrote:\n> It is not a usability issue if you are cleanly separating things which do \n> not belong together.\n>\n>   \nThis introduces order dependency that is otherwise not there. The order \ndependency makes perfect sense to one who understands the details, but \notherwise seems arbitrary. (..still pondering what to do here).\n> Example:\n>\n> #!/bin/sh\n>\n> (echo Hallo; echo Bello; echo; echo blabla) | \\\n> (\n> \twhile read line; do\n> \t\techo \"$line\"\n> \t\tif [ -z \"$line\" ]; then\n> \t\t\tbreak\n> \t\tfi\n> \tdone\n> \techo \"xxx\"\n> \tcat\n> )\n>\n> In this case, shell reads the header until an empty line is encountered. \n> The rest is piped through cat. And it does not matter if \"blabla\" is text \n> or binary.\n>   \nDoh! (sometimes you just have to whack people over the head with a 2x4). \nThanks.\n\nMark\n"}]}