{"thread":{"id":"16853","subject":"[JGIT PATCH 3/5] Modify \"jgit daemon\" so it can run outside of a repository","startedAt":"2008-12-23T18:03:38Z","lastAt":"2009-01-06T20:55:23Z","messageCount":13,"participants":["Shawn O. Pearce","Robin Rosenberg","Nicolas Pitre"],"isPatch":true,"patchVersion":1,"patchTotal":5},"messages":[{"id":"98632","messageId":"1230055423-9944-1-git-send-email-spearce@spearce.org","threadId":"16853","inReplyTo":null,"subject":"[JGIT PATCH 0/5] Add jgit init, clone, receive-pack; transport fixes","fromName":"Shawn O. Pearce","fromEmail":"spearce@spearce.org","sentAt":"2008-12-23T18:03:38Z","receivedAt":"2008-12-23T18:03:38Z","isPatch":true,"sender":{"key":"spearce@spearce.org","avatar":"https://avatars.githubusercontent.com/u/34844?v=4"},"body":"Define a \"jgit init\", \"jgit clone\" commands from the command line.\n\nAdd \"jgit receive-pack\".  This now works from C git:\n\n  git push --receive-pack='jgit receive-pack' dst.git\n\nFix issue #22, causing jgit to fetch too many objects during an\ninitial clone/fetch of a repository.\n\nChange \"jgit daemon\" so it can run outside of a repository, making\nit slightly more useful.\n\n\nShawn O. Pearce (5):\n  Add \"jgit receive-pack\" and permit commands to start not in a\n    repository\n  Add \"jgit init\" command to create a new repository\n  Modify \"jgit daemon\" so it can run outside of a repository\n  Add \"jgit clone\" to support cloning off URLs that are JGit specific\n  Fix \"fetch pulled too many objects\" when auto-following tags\n\n .../services/org.spearce.jgit.pgm.TextBuiltin      |    3 +\n .../org/spearce/jgit/pgm/AbstractFetchCommand.java |  126 ++++++++++++++\n .../src/org/spearce/jgit/pgm/Clone.java            |  177 ++++++++++++++++++++\n .../src/org/spearce/jgit/pgm/Daemon.java           |   33 +---\n .../src/org/spearce/jgit/pgm/Fetch.java            |   83 +---------\n .../src/org/spearce/jgit/pgm/Init.java             |   60 +++++++\n .../src/org/spearce/jgit/pgm/Main.java             |   19 ++-\n .../src/org/spearce/jgit/pgm/ReceivePack.java      |   67 ++++++++\n .../src/org/spearce/jgit/pgm/TextBuiltin.java      |   24 +++-\n .../jgit/transport/BaseFetchConnection.java        |   26 ++-\n .../jgit/transport/BasePackFetchConnection.java    |   25 +++-\n .../spearce/jgit/transport/FetchConnection.java    |   21 ++-\n .../org/spearce/jgit/transport/FetchProcess.java   |    6 +-\n .../spearce/jgit/transport/TransportBundle.java    |    3 +-\n .../jgit/transport/WalkFetchConnection.java        |   14 ++-\n 15 files changed, 544 insertions(+), 143 deletions(-)\n create mode 100644 org.spearce.jgit.pgm/src/org/spearce/jgit/pgm/AbstractFetchCommand.java\n create mode 100644 org.spearce.jgit.pgm/src/org/spearce/jgit/pgm/Clone.java\n create mode 100644 org.spearce.jgit.pgm/src/org/spearce/jgit/pgm/Init.java\n create mode 100644 org.spearce.jgit.pgm/src/org/spearce/jgit/pgm/ReceivePack.java\n"},{"id":"98634","messageId":"1230055423-9944-2-git-send-email-spearce@spearce.org","threadId":"16853","inReplyTo":"1230055423-9944-1-git-send-email-spearce@spearce.org","subject":"[JGIT PATCH 1/5] Add \"jgit receive-pack\" and permit commands to start not in a repository","fromName":"Shawn O. Pearce","fromEmail":"spearce@spearce.org","sentAt":"2008-12-23T18:03:39Z","receivedAt":"2008-12-23T18:03:39Z","isPatch":true,"sender":{"key":"spearce@spearce.org","avatar":"https://avatars.githubusercontent.com/u/34844?v=4"},"body":"Signed-off-by: Shawn O. Pearce <spearce@spearce.org>\n---\n .../services/org.spearce.jgit.pgm.TextBuiltin      |    1 +\n .../src/org/spearce/jgit/pgm/Main.java             |   19 +++---\n .../src/org/spearce/jgit/pgm/ReceivePack.java      |   67 ++++++++++++++++++++\n .../src/org/spearce/jgit/pgm/TextBuiltin.java      |   24 ++++++-\n 4 files changed, 99 insertions(+), 12 deletions(-)\n create mode 100644 org.spearce.jgit.pgm/src/org/spearce/jgit/pgm/ReceivePack.java\n\ndiff --git a/org.spearce.jgit.pgm/src/META-INF/services/org.spearce.jgit.pgm.TextBuiltin b/org.spearce.jgit.pgm/src/META-INF/services/org.spearce.jgit.pgm.TextBuiltin\nindex 5fb0953..0e7e7d8 100644\n--- a/org.spearce.jgit.pgm/src/META-INF/services/org.spearce.jgit.pgm.TextBuiltin\n+++ b/org.spearce.jgit.pgm/src/META-INF/services/org.spearce.jgit.pgm.TextBuiltin\n@@ -9,6 +9,7 @@ org.spearce.jgit.pgm.LsRemote\n org.spearce.jgit.pgm.LsTree\n org.spearce.jgit.pgm.MergeBase\n org.spearce.jgit.pgm.Push\n+org.spearce.jgit.pgm.ReceivePack\n org.spearce.jgit.pgm.RevList\n org.spearce.jgit.pgm.Rm\n org.spearce.jgit.pgm.ShowRev\ndiff --git a/org.spearce.jgit.pgm/src/org/spearce/jgit/pgm/Main.java b/org.spearce.jgit.pgm/src/org/spearce/jgit/pgm/Main.java\nindex c8bade8..8de4ae9 100644\n--- a/org.spearce.jgit.pgm/src/org/spearce/jgit/pgm/Main.java\n+++ b/org.spearce.jgit.pgm/src/org/spearce/jgit/pgm/Main.java\n@@ -143,15 +143,18 @@ private void execute(final String[] argv) throws Exception {\n \t\t\tSystem.exit(1);\n \t\t}\n \n-\t\tif (gitdir == null)\n-\t\t\tgitdir = findGitDir();\n-\t\tif (gitdir == null || !gitdir.isDirectory()) {\n-\t\t\tSystem.err.println(\"error: can't find git directory\");\n-\t\t\tSystem.exit(1);\n-\t\t}\n-\n \t\tfinal TextBuiltin cmd = subcommand;\n-\t\tcmd.init(new Repository(gitdir));\n+\t\tif (cmd.requiresRepository()) {\n+\t\t\tif (gitdir == null)\n+\t\t\t\tgitdir = findGitDir();\n+\t\t\tif (gitdir == null || !gitdir.isDirectory()) {\n+\t\t\t\tSystem.err.println(\"error: can't find git directory\");\n+\t\t\t\tSystem.exit(1);\n+\t\t\t}\n+\t\t\tcmd.init(new Repository(gitdir), gitdir);\n+\t\t} else {\n+\t\t\tcmd.init(null, gitdir);\n+\t\t}\n \t\ttry {\n \t\t\tcmd.execute(arguments.toArray(new String[arguments.size()]));\n \t\t} finally {\ndiff --git a/org.spearce.jgit.pgm/src/org/spearce/jgit/pgm/ReceivePack.java b/org.spearce.jgit.pgm/src/org/spearce/jgit/pgm/ReceivePack.java\nnew file mode 100644\nindex 0000000..579f893\n--- /dev/null\n+++ b/org.spearce.jgit.pgm/src/org/spearce/jgit/pgm/ReceivePack.java\n@@ -0,0 +1,67 @@\n+/*\n+ * Copyright (C) 2008, Google Inc.\n+ *\n+ * All rights reserved.\n+ *\n+ * Redistribution and use in source and binary forms, with or\n+ * without modification, are permitted provided that the following\n+ * conditions are met:\n+ *\n+ * - Redistributions of source code must retain the above copyright\n+ *   notice, this list of conditions and the following disclaimer.\n+ *\n+ * - Redistributions in binary form must reproduce the above\n+ *   copyright notice, this list of conditions and the following\n+ *   disclaimer in the documentation and/or other materials provided\n+ *   with the distribution.\n+ *\n+ * - Neither the name of the Git Development Community nor the\n+ *   names of its contributors may be used to endorse or promote\n+ *   products derived from this software without specific prior\n+ *   written permission.\n+ *\n+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND\n+ * CONTRIBUTORS \"AS IS\" AND ANY EXPRESS OR IMPLIED WARRANTIES,\n+ * INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES\n+ * OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE\n+ * ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR\n+ * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,\n+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT\n+ * NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;\n+ * LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER\n+ * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT,\n+ * STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)\n+ * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF\n+ * ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.\n+ */\n+\n+package org.spearce.jgit.pgm;\n+\n+import java.io.File;\n+\n+import org.kohsuke.args4j.Argument;\n+import org.spearce.jgit.lib.Repository;\n+\n+@Command(common = false, usage = \"Server side backend for 'jgit push'\")\n+class ReceivePack extends TextBuiltin {\n+\t@Argument(index = 0, required = true, metaVar = \"DIRECTORY\", usage = \"Repository to receive into\")\n+\tFile gitdir;\n+\n+\t@Override\n+\tprotected final boolean requiresRepository() {\n+\t\treturn false;\n+\t}\n+\n+\t@Override\n+\tprotected void run() throws Exception {\n+\t\tfinal org.spearce.jgit.transport.ReceivePack rp;\n+\n+\t\tif (new File(gitdir, \".git\").isDirectory())\n+\t\t\tgitdir = new File(gitdir, \".git\");\n+\t\tdb = new Repository(gitdir);\n+\t\tif (!db.getObjectsDirectory().isDirectory())\n+\t\t\tthrow die(\"'\" + gitdir.getPath() + \"' not a git repository\");\n+\t\trp = new org.spearce.jgit.transport.ReceivePack(db);\n+\t\trp.receive(System.in, System.out, System.err);\n+\t}\n+}\ndiff --git a/org.spearce.jgit.pgm/src/org/spearce/jgit/pgm/TextBuiltin.java b/org.spearce.jgit.pgm/src/org/spearce/jgit/pgm/TextBuiltin.java\nindex 1f3a136..d3e32b3 100644\n--- a/org.spearce.jgit.pgm/src/org/spearce/jgit/pgm/TextBuiltin.java\n+++ b/org.spearce.jgit.pgm/src/org/spearce/jgit/pgm/TextBuiltin.java\n@@ -43,6 +43,7 @@\n import static org.spearce.jgit.lib.Constants.R_TAGS;\n \n import java.io.BufferedWriter;\n+import java.io.File;\n import java.io.IOException;\n import java.io.OutputStreamWriter;\n import java.io.PrintWriter;\n@@ -77,6 +78,9 @@\n \t/** Git repository the command was invoked within. */\n \tprotected Repository db;\n \n+\t/** Directory supplied via --git-dir command line option. */\n+\tprotected File gitdir;\n+\n \t/** RevWalk used during command line parsing, if it was required. */\n \tprotected RevWalk argWalk;\n \n@@ -84,10 +88,15 @@ final void setCommandName(final String name) {\n \t\tcommandName = name;\n \t}\n \n-\tvoid init(final Repository repo) {\n+\t/** @return true if {@link #db}/{@link #getRepository()} is required. */\n+\tprotected boolean requiresRepository() {\n+\t\treturn true;\n+\t}\n+\n+\tvoid init(final Repository repo, final File gd) {\n \t\ttry {\n-\t\t\tString outputEncoding = repo.getConfig().getString(\"i18n\", null,\n-\t\t\t\t\t\"logOutputEncoding\");\n+\t\t\tfinal String outputEncoding = repo != null ? repo.getConfig()\n+\t\t\t\t\t.getString(\"i18n\", null, \"logOutputEncoding\") : null;\n \t\t\tif (outputEncoding != null)\n \t\t\t\tout = new PrintWriter(new BufferedWriter(\n \t\t\t\t\t\tnew OutputStreamWriter(System.out, outputEncoding)));\n@@ -97,7 +106,14 @@ void init(final Repository repo) {\n \t\t} catch (IOException e) {\n \t\t\tthrow die(\"cannot create output stream\");\n \t\t}\n-\t\tdb = repo;\n+\n+\t\tif (repo != null) {\n+\t\t\tdb = repo;\n+\t\t\tgitdir = repo.getDirectory();\n+\t\t} else {\n+\t\t\tdb = null;\n+\t\t\tgitdir = gd;\n+\t\t}\n \t}\n \n \t/**\n-- \n1.6.1.rc4.301.g5497a\n"},{"id":"98633","messageId":"1230055423-9944-3-git-send-email-spearce@spearce.org","threadId":"16853","inReplyTo":"1230055423-9944-2-git-send-email-spearce@spearce.org","subject":"[JGIT PATCH 2/5] Add \"jgit init\" command to create a new repository","fromName":"Shawn O. Pearce","fromEmail":"spearce@spearce.org","sentAt":"2008-12-23T18:03:40Z","receivedAt":"2008-12-23T18:03:40Z","isPatch":true,"sender":{"key":"spearce@spearce.org","avatar":"https://avatars.githubusercontent.com/u/34844?v=4"},"body":"Signed-off-by: Shawn O. Pearce <spearce@spearce.org>\n---\n .../services/org.spearce.jgit.pgm.TextBuiltin      |    1 +\n .../src/org/spearce/jgit/pgm/Init.java             |   60 ++++++++++++++++++++\n 2 files changed, 61 insertions(+), 0 deletions(-)\n create mode 100644 org.spearce.jgit.pgm/src/org/spearce/jgit/pgm/Init.java\n\ndiff --git a/org.spearce.jgit.pgm/src/META-INF/services/org.spearce.jgit.pgm.TextBuiltin b/org.spearce.jgit.pgm/src/META-INF/services/org.spearce.jgit.pgm.TextBuiltin\nindex 0e7e7d8..4a2b605 100644\n--- a/org.spearce.jgit.pgm/src/META-INF/services/org.spearce.jgit.pgm.TextBuiltin\n+++ b/org.spearce.jgit.pgm/src/META-INF/services/org.spearce.jgit.pgm.TextBuiltin\n@@ -4,6 +4,7 @@ org.spearce.jgit.pgm.DiffTree\n org.spearce.jgit.pgm.Fetch\n org.spearce.jgit.pgm.Glog\n org.spearce.jgit.pgm.IndexPack\n+org.spearce.jgit.pgm.Init\n org.spearce.jgit.pgm.Log\n org.spearce.jgit.pgm.LsRemote\n org.spearce.jgit.pgm.LsTree\ndiff --git a/org.spearce.jgit.pgm/src/org/spearce/jgit/pgm/Init.java b/org.spearce.jgit.pgm/src/org/spearce/jgit/pgm/Init.java\nnew file mode 100644\nindex 0000000..197864d\n--- /dev/null\n+++ b/org.spearce.jgit.pgm/src/org/spearce/jgit/pgm/Init.java\n@@ -0,0 +1,60 @@\n+/*\n+ * Copyright (C) 2008, Google Inc.\n+ *\n+ * All rights reserved.\n+ *\n+ * Redistribution and use in source and binary forms, with or\n+ * without modification, are permitted provided that the following\n+ * conditions are met:\n+ *\n+ * - Redistributions of source code must retain the above copyright\n+ *   notice, this list of conditions and the following disclaimer.\n+ *\n+ * - Redistributions in binary form must reproduce the above\n+ *   copyright notice, this list of conditions and the following\n+ *   disclaimer in the documentation and/or other materials provided\n+ *   with the distribution.\n+ *\n+ * - Neither the name of the Git Development Community nor the\n+ *   names of its contributors may be used to endorse or promote\n+ *   products derived from this software without specific prior\n+ *   written permission.\n+ *\n+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND\n+ * CONTRIBUTORS \"AS IS\" AND ANY EXPRESS OR IMPLIED WARRANTIES,\n+ * INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES\n+ * OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE\n+ * ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR\n+ * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,\n+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT\n+ * NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;\n+ * LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER\n+ * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT,\n+ * STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)\n+ * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF\n+ * ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.\n+ */\n+\n+package org.spearce.jgit.pgm;\n+\n+import java.io.File;\n+\n+import org.spearce.jgit.lib.Repository;\n+\n+@Command(common = true, usage = \"Create an empty git repository\")\n+class Init extends TextBuiltin {\n+\t@Override\n+\tprotected final boolean requiresRepository() {\n+\t\treturn false;\n+\t}\n+\n+\t@Override\n+\tprotected void run() throws Exception {\n+\t\tif (gitdir == null)\n+\t\t\tgitdir = new File(\".git\");\n+\t\tdb = new Repository(gitdir);\n+\t\tdb.create();\n+\t\tout.println(\"Initialized empty Git repository in \"\n+\t\t\t\t+ gitdir.getAbsolutePath());\n+\t}\n+}\n-- \n1.6.1.rc4.301.g5497a\n"},{"id":"98629","messageId":"1230055423-9944-4-git-send-email-spearce@spearce.org","threadId":"16853","inReplyTo":"1230055423-9944-3-git-send-email-spearce@spearce.org","subject":"[JGIT PATCH 3/5] Modify \"jgit daemon\" so it can run outside of a repository","fromName":"Shawn O. Pearce","fromEmail":"spearce@spearce.org","sentAt":"2008-12-23T18:03:41Z","receivedAt":"2008-12-23T18:03:41Z","isPatch":true,"sender":{"key":"spearce@spearce.org","avatar":"https://avatars.githubusercontent.com/u/34844?v=4"},"body":"We no longer export the current repository, but instead export the\ndirectory trees given.  This behavior is more closely matches that\nof \"git daemon\".\n\nSigned-off-by: Shawn O. Pearce <spearce@spearce.org>\n---\n .../src/org/spearce/jgit/pgm/Daemon.java           |   33 +++++--------------\n 1 files changed, 9 insertions(+), 24 deletions(-)\n\ndiff --git a/org.spearce.jgit.pgm/src/org/spearce/jgit/pgm/Daemon.java b/org.spearce.jgit.pgm/src/org/spearce/jgit/pgm/Daemon.java\nindex aafc82e..39b43b2 100644\n--- a/org.spearce.jgit.pgm/src/org/spearce/jgit/pgm/Daemon.java\n+++ b/org.spearce.jgit.pgm/src/org/spearce/jgit/pgm/Daemon.java\n@@ -44,7 +44,6 @@\n \n import org.kohsuke.args4j.Argument;\n import org.kohsuke.args4j.Option;\n-import org.spearce.jgit.lib.Repository;\n import org.spearce.jgit.transport.DaemonService;\n \n @Command(common = true, usage = \"Export repositories over git://\")\n@@ -67,10 +66,15 @@\n \t@Option(name = \"--forbid-override\", metaVar = \"SERVICE\", usage = \"configure the service in daemon.servicename\", multiValued = true)\n \tfinal List<String> forbidOverride = new ArrayList<String>();\n \n-\t@Argument(metaVar = \"DIRECTORY\", usage = \"directories to export\")\n+\t@Argument(required = true, metaVar = \"DIRECTORY\", usage = \"directories to export\")\n \tfinal List<File> directory = new ArrayList<File>();\n \n \t@Override\n+\tprotected boolean requiresRepository() {\n+\t\treturn false;\n+\t}\n+\n+\t@Override\n \tprotected void run() throws Exception {\n \t\tfinal org.spearce.jgit.transport.Daemon d;\n \n@@ -88,13 +92,9 @@ protected void run() throws Exception {\n \t\tfor (final String n : forbidOverride)\n \t\t\tservice(d, n).setOverridable(false);\n \n-\t\tif (directory.isEmpty()) {\n-\t\t\texport(d, db);\n-\t\t} else {\n-\t\t\tfor (final File f : directory) {\n-\t\t\t\tout.println(\"Exporting \" + f.getAbsolutePath());\n-\t\t\t\td.exportDirectory(f);\n-\t\t\t}\n+\t\tfor (final File f : directory) {\n+\t\t\tout.println(\"Exporting \" + f.getAbsolutePath());\n+\t\t\td.exportDirectory(f);\n \t\t}\n \t\td.start();\n \t\tout.println(\"Listening on \" + d.getAddress());\n@@ -107,19 +107,4 @@ private DaemonService service(final org.spearce.jgit.transport.Daemon d,\n \t\t\tthrow die(\"Service '\" + n + \"' not supported\");\n \t\treturn svc;\n \t}\n-\n-\tprivate void export(final org.spearce.jgit.transport.Daemon daemon,\n-\t\t\tfinal Repository repo) {\n-\t\tFile d = repo.getDirectory();\n-\t\tString name = d.getName();\n-\t\twhile (name.equals(\".git\") || name.equals(\".\")) {\n-\t\t\td = d.getParentFile();\n-\t\t\tname = d.getName();\n-\t\t}\n-\t\tif (!name.endsWith(\".git\"))\n-\t\t\tname += \".git\";\n-\n-\t\tout.println(\"Exporting current repository as \\\"\" + name + \"\\\"\");\n-\t\tdaemon.exportRepository(name, repo);\n-\t}\n }\n-- \n1.6.1.rc4.301.g5497a\n"},{"id":"98630","messageId":"1230055423-9944-5-git-send-email-spearce@spearce.org","threadId":"16853","inReplyTo":"1230055423-9944-4-git-send-email-spearce@spearce.org","subject":"[JGIT PATCH 4/5] Add \"jgit clone\" to support cloning off URLs that are JGit specific","fromName":"Shawn O. Pearce","fromEmail":"spearce@spearce.org","sentAt":"2008-12-23T18:03:42Z","receivedAt":"2008-12-23T18:03:42Z","isPatch":true,"sender":{"key":"spearce@spearce.org","avatar":"https://avatars.githubusercontent.com/u/34844?v=4"},"body":"The amazon-s3:// protocol is unique to JGit, and is not supported\nby any other Git implementation.  The easiest way to clone off of\none of those URLs is to use \"jgit clone\".  This program also acts\nas a simple example for any IDE developers who want to implement\na clone feature, as it is the minimum effort required to make a\nuser reasonably happy with the resulting repository.\n\nSigned-off-by: Shawn O. Pearce <spearce@spearce.org>\n---\n .../services/org.spearce.jgit.pgm.TextBuiltin      |    1 +\n .../org/spearce/jgit/pgm/AbstractFetchCommand.java |  126 ++++++++++++++\n .../src/org/spearce/jgit/pgm/Clone.java            |  177 ++++++++++++++++++++\n .../src/org/spearce/jgit/pgm/Fetch.java            |   83 +---------\n 4 files changed, 306 insertions(+), 81 deletions(-)\n create mode 100644 org.spearce.jgit.pgm/src/org/spearce/jgit/pgm/AbstractFetchCommand.java\n create mode 100644 org.spearce.jgit.pgm/src/org/spearce/jgit/pgm/Clone.java\n\ndiff --git a/org.spearce.jgit.pgm/src/META-INF/services/org.spearce.jgit.pgm.TextBuiltin b/org.spearce.jgit.pgm/src/META-INF/services/org.spearce.jgit.pgm.TextBuiltin\nindex 4a2b605..40177f9 100644\n--- a/org.spearce.jgit.pgm/src/META-INF/services/org.spearce.jgit.pgm.TextBuiltin\n+++ b/org.spearce.jgit.pgm/src/META-INF/services/org.spearce.jgit.pgm.TextBuiltin\n@@ -1,4 +1,5 @@\n org.spearce.jgit.pgm.Branch\n+org.spearce.jgit.pgm.Clone\n org.spearce.jgit.pgm.Daemon\n org.spearce.jgit.pgm.DiffTree\n org.spearce.jgit.pgm.Fetch\ndiff --git a/org.spearce.jgit.pgm/src/org/spearce/jgit/pgm/AbstractFetchCommand.java b/org.spearce.jgit.pgm/src/org/spearce/jgit/pgm/AbstractFetchCommand.java\nnew file mode 100644\nindex 0000000..ea6f277\n--- /dev/null\n+++ b/org.spearce.jgit.pgm/src/org/spearce/jgit/pgm/AbstractFetchCommand.java\n@@ -0,0 +1,126 @@\n+/*\n+ * Copyright (C) 2008, Shawn O. Pearce <spearce@spearce.org>\n+ *\n+ * All rights reserved.\n+ *\n+ * Redistribution and use in source and binary forms, with or\n+ * without modification, are permitted provided that the following\n+ * conditions are met:\n+ *\n+ * - Redistributions of source code must retain the above copyright\n+ *   notice, this list of conditions and the following disclaimer.\n+ *\n+ * - Redistributions in binary form must reproduce the above\n+ *   copyright notice, this list of conditions and the following\n+ *   disclaimer in the documentation and/or other materials provided\n+ *   with the distribution.\n+ *\n+ * - Neither the name of the Git Development Community nor the\n+ *   names of its contributors may be used to endorse or promote\n+ *   products derived from this software without specific prior\n+ *   written permission.\n+ *\n+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND\n+ * CONTRIBUTORS \"AS IS\" AND ANY EXPRESS OR IMPLIED WARRANTIES,\n+ * INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES\n+ * OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE\n+ * ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR\n+ * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,\n+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT\n+ * NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;\n+ * LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER\n+ * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT,\n+ * STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)\n+ * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF\n+ * ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.\n+ */\n+\n+package org.spearce.jgit.pgm;\n+\n+import org.kohsuke.args4j.Option;\n+import org.spearce.jgit.lib.Constants;\n+import org.spearce.jgit.lib.RefUpdate;\n+import org.spearce.jgit.transport.FetchResult;\n+import org.spearce.jgit.transport.TrackingRefUpdate;\n+import org.spearce.jgit.transport.Transport;\n+\n+abstract class AbstractFetchCommand extends TextBuiltin {\n+\t@Option(name = \"--verbose\", aliases = { \"-v\" }, usage = \"be more verbose\")\n+\tprivate boolean verbose;\n+\n+\tprotected void showFetchResult(final Transport tn, final FetchResult r) {\n+\t\tboolean shownURI = false;\n+\t\tfor (final TrackingRefUpdate u : r.getTrackingRefUpdates()) {\n+\t\t\tif (!verbose && u.getResult() == RefUpdate.Result.NO_CHANGE)\n+\t\t\t\tcontinue;\n+\n+\t\t\tfinal char type = shortTypeOf(u.getResult());\n+\t\t\tfinal String longType = longTypeOf(u);\n+\t\t\tfinal String src = abbreviateRef(u.getRemoteName(), false);\n+\t\t\tfinal String dst = abbreviateRef(u.getLocalName(), true);\n+\n+\t\t\tif (!shownURI) {\n+\t\t\t\tout.print(\"From \");\n+\t\t\t\tout.print(tn.getURI());\n+\t\t\t\tout.println();\n+\t\t\t\tshownURI = true;\n+\t\t\t}\n+\n+\t\t\tout.format(\" %c %-17s %-10s -> %s\", type, longType, src, dst);\n+\t\t\tout.println();\n+\t\t}\n+\t}\n+\n+\tprivate String longTypeOf(final TrackingRefUpdate u) {\n+\t\tfinal RefUpdate.Result r = u.getResult();\n+\t\tif (r == RefUpdate.Result.LOCK_FAILURE)\n+\t\t\treturn \"[lock fail]\";\n+\n+\t\tif (r == RefUpdate.Result.IO_FAILURE)\n+\t\t\treturn \"[i/o error]\";\n+\n+\t\tif (r == RefUpdate.Result.NEW) {\n+\t\t\tif (u.getRemoteName().startsWith(Constants.R_HEADS))\n+\t\t\t\treturn \"[new branch]\";\n+\t\t\telse if (u.getLocalName().startsWith(Constants.R_TAGS))\n+\t\t\t\treturn \"[new tag]\";\n+\t\t\treturn \"[new]\";\n+\t\t}\n+\n+\t\tif (r == RefUpdate.Result.FORCED) {\n+\t\t\tfinal String aOld = u.getOldObjectId().abbreviate(db).name();\n+\t\t\tfinal String aNew = u.getNewObjectId().abbreviate(db).name();\n+\t\t\treturn aOld + \"...\" + aNew;\n+\t\t}\n+\n+\t\tif (r == RefUpdate.Result.FAST_FORWARD) {\n+\t\t\tfinal String aOld = u.getOldObjectId().abbreviate(db).name();\n+\t\t\tfinal String aNew = u.getNewObjectId().abbreviate(db).name();\n+\t\t\treturn aOld + \"..\" + aNew;\n+\t\t}\n+\n+\t\tif (r == RefUpdate.Result.REJECTED)\n+\t\t\treturn \"[rejected]\";\n+\t\tif (r == RefUpdate.Result.NO_CHANGE)\n+\t\t\treturn \"[up to date]\";\n+\t\treturn \"[\" + r.name() + \"]\";\n+\t}\n+\n+\tprivate static char shortTypeOf(final RefUpdate.Result r) {\n+\t\tif (r == RefUpdate.Result.LOCK_FAILURE)\n+\t\t\treturn '!';\n+\t\tif (r == RefUpdate.Result.IO_FAILURE)\n+\t\t\treturn '!';\n+\t\tif (r == RefUpdate.Result.NEW)\n+\t\t\treturn '*';\n+\t\tif (r == RefUpdate.Result.FORCED)\n+\t\t\treturn '+';\n+\t\tif (r == RefUpdate.Result.FAST_FORWARD)\n+\t\t\treturn ' ';\n+\t\tif (r == RefUpdate.Result.REJECTED)\n+\t\t\treturn '!';\n+\t\tif (r == RefUpdate.Result.NO_CHANGE)\n+\t\t\treturn '=';\n+\t\treturn ' ';\n+\t}\n+}\ndiff --git a/org.spearce.jgit.pgm/src/org/spearce/jgit/pgm/Clone.java b/org.spearce.jgit.pgm/src/org/spearce/jgit/pgm/Clone.java\nnew file mode 100644\nindex 0000000..51dd95d\n--- /dev/null\n+++ b/org.spearce.jgit.pgm/src/org/spearce/jgit/pgm/Clone.java\n@@ -0,0 +1,177 @@\n+/*\n+ * Copyright (C) 2008, Google Inc.\n+ *\n+ * All rights reserved.\n+ *\n+ * Redistribution and use in source and binary forms, with or\n+ * without modification, are permitted provided that the following\n+ * conditions are met:\n+ *\n+ * - Redistributions of source code must retain the above copyright\n+ *   notice, this list of conditions and the following disclaimer.\n+ *\n+ * - Redistributions in binary form must reproduce the above\n+ *   copyright notice, this list of conditions and the following\n+ *   disclaimer in the documentation and/or other materials provided\n+ *   with the distribution.\n+ *\n+ * - Neither the name of the Git Development Community nor the\n+ *   names of its contributors may be used to endorse or promote\n+ *   products derived from this software without specific prior\n+ *   written permission.\n+ *\n+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND\n+ * CONTRIBUTORS \"AS IS\" AND ANY EXPRESS OR IMPLIED WARRANTIES,\n+ * INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES\n+ * OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE\n+ * ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR\n+ * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,\n+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT\n+ * NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;\n+ * LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER\n+ * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT,\n+ * STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)\n+ * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF\n+ * ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.\n+ */\n+\n+package org.spearce.jgit.pgm;\n+\n+import java.io.File;\n+import java.io.IOException;\n+import java.net.URISyntaxException;\n+import java.util.ArrayList;\n+import java.util.Collections;\n+import java.util.List;\n+\n+import org.kohsuke.args4j.Argument;\n+import org.kohsuke.args4j.Option;\n+import org.spearce.jgit.errors.NotSupportedException;\n+import org.spearce.jgit.errors.TransportException;\n+import org.spearce.jgit.lib.Commit;\n+import org.spearce.jgit.lib.Constants;\n+import org.spearce.jgit.lib.GitIndex;\n+import org.spearce.jgit.lib.Ref;\n+import org.spearce.jgit.lib.RefComparator;\n+import org.spearce.jgit.lib.RefUpdate;\n+import org.spearce.jgit.lib.Repository;\n+import org.spearce.jgit.lib.TextProgressMonitor;\n+import org.spearce.jgit.lib.Tree;\n+import org.spearce.jgit.lib.WorkDirCheckout;\n+import org.spearce.jgit.transport.FetchResult;\n+import org.spearce.jgit.transport.RefSpec;\n+import org.spearce.jgit.transport.RemoteConfig;\n+import org.spearce.jgit.transport.Transport;\n+import org.spearce.jgit.transport.URIish;\n+\n+@Command(common = true, usage = \"Clone a repository into a new directory\")\n+class Clone extends AbstractFetchCommand {\n+\t@Option(name = \"--origin\", aliases = { \"-o\" }, metaVar = \"name\", usage = \"use <name> instead of 'origin' to track upstream\")\n+\tprivate String remoteName = \"origin\";\n+\n+\t@Argument(index = 0, required = true, metaVar = \"uri-ish\")\n+\tprivate String sourceUri;\n+\n+\t@Argument(index = 1, metaVar = \"directory\")\n+\tprivate String localName;\n+\n+\t@Override\n+\tprotected final boolean requiresRepository() {\n+\t\treturn false;\n+\t}\n+\n+\t@Override\n+\tprotected void run() throws Exception {\n+\t\tif (localName != null && gitdir != null)\n+\t\t\tthrow die(\"conflicting usage of --git-dir and arguments\");\n+\n+\t\tfinal URIish uri = new URIish(sourceUri);\n+\t\tif (localName == null) {\n+\t\t\tString p = uri.getPath();\n+\t\t\twhile (p.endsWith(\"/\"))\n+\t\t\t\tp = p.substring(0, p.length() - 1);\n+\t\t\tfinal int s = p.lastIndexOf('/');\n+\t\t\tif (s < 0)\n+\t\t\t\tthrow die(\"cannot guess local name from \" + sourceUri);\n+\t\t\tlocalName = p.substring(s + 1);\n+\t\t\tif (localName.endsWith(\".git\"))\n+\t\t\t\tlocalName = localName.substring(0, localName.length() - 4);\n+\t\t}\n+\t\tif (gitdir == null)\n+\t\t\tgitdir = new File(localName, \".git\");\n+\t\tdb = new Repository(gitdir);\n+\t\tdb.create();\n+\t\tout.println(\"Initialized empty Git repository in \"\n+\t\t\t\t+ gitdir.getAbsolutePath());\n+\t\tout.flush();\n+\n+\t\tsaveRemote(uri);\n+\t\tfinal FetchResult r = runFetch();\n+\t\tfinal Ref branch = guessHEAD(r);\n+\t\tdoCheckout(branch);\n+\t}\n+\n+\tprivate void saveRemote(final URIish uri) throws URISyntaxException,\n+\t\t\tIOException {\n+\t\tfinal RemoteConfig rc = new RemoteConfig(db.getConfig(), remoteName);\n+\t\trc.addURI(uri);\n+\t\trc.addFetchRefSpec(new RefSpec().setForceUpdate(true)\n+\t\t\t\t.setSourceDestination(Constants.R_HEADS + \"*\",\n+\t\t\t\t\t\tConstants.R_REMOTES + remoteName + \"/*\"));\n+\t\trc.update(db.getConfig());\n+\t\tdb.getConfig().save();\n+\t}\n+\n+\tprivate FetchResult runFetch() throws NotSupportedException,\n+\t\t\tURISyntaxException, TransportException {\n+\t\tfinal Transport tn = Transport.open(db, remoteName);\n+\t\tfinal FetchResult r;\n+\t\ttry {\n+\t\t\tr = tn.fetch(new TextProgressMonitor(), null);\n+\t\t} finally {\n+\t\t\ttn.close();\n+\t\t}\n+\t\tshowFetchResult(tn, r);\n+\t\treturn r;\n+\t}\n+\n+\tprivate Ref guessHEAD(final FetchResult result) {\n+\t\tfinal Ref idHEAD = result.getAdvertisedRef(Constants.HEAD);\n+\t\tfinal List<Ref> availableRefs = new ArrayList<Ref>();\n+\t\tRef head = null;\n+\t\tfor (final Ref r : result.getAdvertisedRefs()) {\n+\t\t\tfinal String n = r.getName();\n+\t\t\tif (!n.startsWith(Constants.R_HEADS))\n+\t\t\t\tcontinue;\n+\t\t\tavailableRefs.add(r);\n+\t\t\tif (idHEAD == null || head != null)\n+\t\t\t\tcontinue;\n+\t\t\tif (r.getObjectId().equals(idHEAD.getObjectId()))\n+\t\t\t\thead = r;\n+\t\t}\n+\t\tCollections.sort(availableRefs, RefComparator.INSTANCE);\n+\t\tif (idHEAD != null && head == null)\n+\t\t\thead = idHEAD;\n+\t\treturn head;\n+\t}\n+\n+\tprivate void doCheckout(final Ref branch) throws IOException {\n+\t\tif (branch == null)\n+\t\t\tthrow die(\"cannot checkout; no HEAD advertised by remote\");\n+\t\tif (!Constants.HEAD.equals(branch.getName()))\n+\t\t\tdb.writeSymref(Constants.HEAD, branch.getName());\n+\n+\t\tfinal Commit commit = db.mapCommit(branch.getObjectId());\n+\t\tfinal RefUpdate u = db.updateRef(Constants.HEAD);\n+\t\tu.setNewObjectId(commit.getCommitId());\n+\t\tu.forceUpdate();\n+\n+\t\tfinal GitIndex index = new GitIndex(db);\n+\t\tfinal Tree tree = commit.getTree();\n+\t\tfinal WorkDirCheckout co;\n+\n+\t\tco = new WorkDirCheckout(db, db.getWorkDir(), index, tree);\n+\t\tco.checkout();\n+\t\tindex.write();\n+\t}\n+}\ndiff --git a/org.spearce.jgit.pgm/src/org/spearce/jgit/pgm/Fetch.java b/org.spearce.jgit.pgm/src/org/spearce/jgit/pgm/Fetch.java\nindex e9d3260..8f3f7d5 100644\n--- a/org.spearce.jgit.pgm/src/org/spearce/jgit/pgm/Fetch.java\n+++ b/org.spearce.jgit.pgm/src/org/spearce/jgit/pgm/Fetch.java\n@@ -41,19 +41,13 @@\n \n import org.kohsuke.args4j.Argument;\n import org.kohsuke.args4j.Option;\n-import org.spearce.jgit.lib.Constants;\n-import org.spearce.jgit.lib.RefUpdate;\n import org.spearce.jgit.lib.TextProgressMonitor;\n import org.spearce.jgit.transport.FetchResult;\n import org.spearce.jgit.transport.RefSpec;\n-import org.spearce.jgit.transport.TrackingRefUpdate;\n import org.spearce.jgit.transport.Transport;\n \n @Command(common = true, usage = \"Update remote refs from another repository\")\n-class Fetch extends TextBuiltin {\n-\t@Option(name = \"--verbose\", aliases = { \"-v\" }, usage = \"be more verbose\")\n-\tprivate boolean verbose;\n-\n+class Fetch extends AbstractFetchCommand {\n \t@Option(name = \"--fsck\", usage = \"perform fsck style checks on receive\")\n \tprivate Boolean fsck;\n \n@@ -91,79 +85,6 @@ protected void run() throws Exception {\n \t\t} finally {\n \t\t\ttn.close();\n \t\t}\n-\n-\t\tboolean shownURI = false;\n-\t\tfor (final TrackingRefUpdate u : r.getTrackingRefUpdates()) {\n-\t\t\tif (!verbose && u.getResult() == RefUpdate.Result.NO_CHANGE)\n-\t\t\t\tcontinue;\n-\n-\t\t\tfinal char type = shortTypeOf(u.getResult());\n-\t\t\tfinal String longType = longTypeOf(u);\n-\t\t\tfinal String src = abbreviateRef(u.getRemoteName(), false);\n-\t\t\tfinal String dst = abbreviateRef(u.getLocalName(), true);\n-\n-\t\t\tif (!shownURI) {\n-\t\t\t\tout.print(\"From \");\n-\t\t\t\tout.print(tn.getURI());\n-\t\t\t\tout.println();\n-\t\t\t\tshownURI = true;\n-\t\t\t}\n-\n-\t\t\tout.format(\" %c %-17s %-10s -> %s\", type, longType, src, dst);\n-\t\t\tout.println();\n-\t\t}\n-\t}\n-\n-\tprivate String longTypeOf(final TrackingRefUpdate u) {\n-\t\tfinal RefUpdate.Result r = u.getResult();\n-\t\tif (r == RefUpdate.Result.LOCK_FAILURE)\n-\t\t\treturn \"[lock fail]\";\n-\n-\t\tif (r == RefUpdate.Result.IO_FAILURE)\n-\t\t\treturn \"[i/o error]\";\n-\n-\t\tif (r == RefUpdate.Result.NEW) {\n-\t\t\tif (u.getRemoteName().startsWith(Constants.R_HEADS))\n-\t\t\t\treturn \"[new branch]\";\n-\t\t\telse if (u.getLocalName().startsWith(Constants.R_TAGS))\n-\t\t\t\treturn \"[new tag]\";\n-\t\t\treturn \"[new]\";\n-\t\t}\n-\n-\t\tif (r == RefUpdate.Result.FORCED) {\n-\t\t\tfinal String aOld = u.getOldObjectId().abbreviate(db).name();\n-\t\t\tfinal String aNew = u.getNewObjectId().abbreviate(db).name();\n-\t\t\treturn aOld + \"...\" + aNew;\n-\t\t}\n-\n-\t\tif (r == RefUpdate.Result.FAST_FORWARD) {\n-\t\t\tfinal String aOld = u.getOldObjectId().abbreviate(db).name();\n-\t\t\tfinal String aNew = u.getNewObjectId().abbreviate(db).name();\n-\t\t\treturn aOld + \"..\" + aNew;\n-\t\t}\n-\n-\t\tif (r == RefUpdate.Result.REJECTED)\n-\t\t\treturn \"[rejected]\";\n-\t\tif (r == RefUpdate.Result.NO_CHANGE)\n-\t\t\treturn \"[up to date]\";\n-\t\treturn \"[\" + r.name() + \"]\";\n-\t}\n-\n-\tprivate static char shortTypeOf(final RefUpdate.Result r) {\n-\t\tif (r == RefUpdate.Result.LOCK_FAILURE)\n-\t\t\treturn '!';\n-\t\tif (r == RefUpdate.Result.IO_FAILURE)\n-\t\t\treturn '!';\n-\t\tif (r == RefUpdate.Result.NEW)\n-\t\t\treturn '*';\n-\t\tif (r == RefUpdate.Result.FORCED)\n-\t\t\treturn '+';\n-\t\tif (r == RefUpdate.Result.FAST_FORWARD)\n-\t\t\treturn ' ';\n-\t\tif (r == RefUpdate.Result.REJECTED)\n-\t\t\treturn '!';\n-\t\tif (r == RefUpdate.Result.NO_CHANGE)\n-\t\t\treturn '=';\n-\t\treturn ' ';\n+\t\tshowFetchResult(tn, r);\n \t}\n }\n-- \n1.6.1.rc4.301.g5497a\n"},{"id":"98631","messageId":"1230055423-9944-6-git-send-email-spearce@spearce.org","threadId":"16853","inReplyTo":"1230055423-9944-5-git-send-email-spearce@spearce.org","subject":"[JGIT PATCH 5/5] Fix \"fetch pulled too many objects\" when auto-following tags","fromName":"Shawn O. Pearce","fromEmail":"spearce@spearce.org","sentAt":"2008-12-23T18:03:43Z","receivedAt":"2008-12-23T18:03:43Z","isPatch":true,"sender":{"key":"spearce@spearce.org","avatar":"https://avatars.githubusercontent.com/u/34844?v=4"},"body":"If we don't take into consideration the objects obtained during\nthe first connection when we open a second to auto-follow tags\nwe will download a large chunk of the repository a second time.\nThis is very wasteful of network bandwidth, and is an abuse of\nthe server.\n\nBecause we delay all ref updates until the very end of the fetch\nprocess we need to hold onto the set of objects we requested in\nthe first connection, and pass that set into the subsequent one\nso it can be considered reachable.\n\nIssue: http://code.google.com/p/egit/issues/detail?id=22\nSigned-off-by: Shawn O. Pearce <spearce@spearce.org>\n---\n .../jgit/transport/BaseFetchConnection.java        |   26 ++++++++++++-------\n .../jgit/transport/BasePackFetchConnection.java    |   25 +++++++++++++++----\n .../spearce/jgit/transport/FetchConnection.java    |   21 +++++++++++----\n .../org/spearce/jgit/transport/FetchProcess.java   |    6 ++++-\n .../spearce/jgit/transport/TransportBundle.java    |    3 +-\n .../jgit/transport/WalkFetchConnection.java        |   14 ++++++++--\n 6 files changed, 69 insertions(+), 26 deletions(-)\n\ndiff --git a/org.spearce.jgit/src/org/spearce/jgit/transport/BaseFetchConnection.java b/org.spearce.jgit/src/org/spearce/jgit/transport/BaseFetchConnection.java\nindex 6709bfc..bb81296 100644\n--- a/org.spearce.jgit/src/org/spearce/jgit/transport/BaseFetchConnection.java\n+++ b/org.spearce.jgit/src/org/spearce/jgit/transport/BaseFetchConnection.java\n@@ -38,8 +38,10 @@\n package org.spearce.jgit.transport;\n \n import java.util.Collection;\n+import java.util.Set;\n \n import org.spearce.jgit.errors.TransportException;\n+import org.spearce.jgit.lib.ObjectId;\n import org.spearce.jgit.lib.ProgressMonitor;\n import org.spearce.jgit.lib.Ref;\n \n@@ -54,9 +56,10 @@\n abstract class BaseFetchConnection extends BaseConnection implements\n \t\tFetchConnection {\n \tpublic final void fetch(final ProgressMonitor monitor,\n-\t\t\tfinal Collection<Ref> want) throws TransportException {\n+\t\t\tfinal Collection<Ref> want, final Set<ObjectId> have)\n+\t\t\tthrows TransportException {\n \t\tmarkStartedOperation();\n-\t\tdoFetch(monitor, want);\n+\t\tdoFetch(monitor, want, have);\n \t}\n \n \t/**\n@@ -68,19 +71,22 @@ public boolean didFetchIncludeTags() {\n \t}\n \n \t/**\n-\t * Implementation of {@link #fetch(ProgressMonitor, Collection)} without\n-\t * checking for multiple fetch.\n+\t * Implementation of {@link #fetch(ProgressMonitor, Collection, Set)}\n+\t * without checking for multiple fetch.\n \t *\n \t * @param monitor\n-\t *            as in {@link #fetch(ProgressMonitor, Collection)}\n+\t *            as in {@link #fetch(ProgressMonitor, Collection, Set)}\n \t * @param want\n-\t *            as in {@link #fetch(ProgressMonitor, Collection)}\n+\t *            as in {@link #fetch(ProgressMonitor, Collection, Set)}\n+\t * @param have\n+\t *            as in {@link #fetch(ProgressMonitor, Collection, Set)}\n \t * @throws TransportException\n-\t *             as in {@link #fetch(ProgressMonitor, Collection)}, but\n+\t *             as in {@link #fetch(ProgressMonitor, Collection, Set)}, but\n \t *             implementation doesn't have to care about multiple\n-\t *             {@link #fetch(ProgressMonitor, Collection)} calls, as it is\n-\t *             checked in this class.\n+\t *             {@link #fetch(ProgressMonitor, Collection, Set)} calls, as it\n+\t *             is checked in this class.\n \t */\n \tprotected abstract void doFetch(final ProgressMonitor monitor,\n-\t\t\tfinal Collection<Ref> want) throws TransportException;\n+\t\t\tfinal Collection<Ref> want, final Set<ObjectId> have)\n+\t\t\tthrows TransportException;\n }\ndiff --git a/org.spearce.jgit/src/org/spearce/jgit/transport/BasePackFetchConnection.java b/org.spearce.jgit/src/org/spearce/jgit/transport/BasePackFetchConnection.java\nindex 542a8a9..2cb9b64 100644\n--- a/org.spearce.jgit/src/org/spearce/jgit/transport/BasePackFetchConnection.java\n+++ b/org.spearce.jgit/src/org/spearce/jgit/transport/BasePackFetchConnection.java\n@@ -41,10 +41,12 @@\n import java.io.IOException;\n import java.util.Collection;\n import java.util.Date;\n+import java.util.Set;\n \n import org.spearce.jgit.errors.TransportException;\n import org.spearce.jgit.lib.AnyObjectId;\n import org.spearce.jgit.lib.MutableObjectId;\n+import org.spearce.jgit.lib.ObjectId;\n import org.spearce.jgit.lib.ProgressMonitor;\n import org.spearce.jgit.lib.Ref;\n import org.spearce.jgit.revwalk.RevCommit;\n@@ -137,9 +139,10 @@ BasePackFetchConnection(final PackTransport packTransport) {\n \t}\n \n \tpublic final void fetch(final ProgressMonitor monitor,\n-\t\t\tfinal Collection<Ref> want) throws TransportException {\n+\t\t\tfinal Collection<Ref> want, final Set<ObjectId> have)\n+\t\t\tthrows TransportException {\n \t\tmarkStartedOperation();\n-\t\tdoFetch(monitor, want);\n+\t\tdoFetch(monitor, want, have);\n \t}\n \n \tpublic boolean didFetchIncludeTags() {\n@@ -151,10 +154,11 @@ public boolean didFetchTestConnectivity() {\n \t}\n \n \tprotected void doFetch(final ProgressMonitor monitor,\n-\t\t\tfinal Collection<Ref> want) throws TransportException {\n+\t\t\tfinal Collection<Ref> want, final Set<ObjectId> have)\n+\t\t\tthrows TransportException {\n \t\ttry {\n \t\t\tmarkRefsAdvertised();\n-\t\t\tmarkReachable(maxTimeWanted(want));\n+\t\t\tmarkReachable(have, maxTimeWanted(want));\n \n \t\t\tif (sendWants(want)) {\n \t\t\t\tnegotiate(monitor);\n@@ -193,7 +197,8 @@ private int maxTimeWanted(final Collection<Ref> wants) {\n \t\treturn maxTime;\n \t}\n \n-\tprivate void markReachable(final int maxTime) throws IOException {\n+\tprivate void markReachable(final Set<ObjectId> have, final int maxTime)\n+\t\t\tthrows IOException {\n \t\tfor (final Ref r : local.getAllRefs().values()) {\n \t\t\ttry {\n \t\t\t\tfinal RevCommit o = walk.parseCommit(r.getObjectId());\n@@ -204,6 +209,16 @@ private void markReachable(final int maxTime) throws IOException {\n \t\t\t}\n \t\t}\n \n+\t\tfor (final ObjectId id : have) {\n+\t\t\ttry {\n+\t\t\t\tfinal RevCommit o = walk.parseCommit(id);\n+\t\t\t\to.add(REACHABLE);\n+\t\t\t\treachableCommits.add(o);\n+\t\t\t} catch (IOException readError) {\n+\t\t\t\t// If we cannot read the value of the ref skip it.\n+\t\t\t}\n+\t\t}\n+\n \t\tif (maxTime > 0) {\n \t\t\t// Mark reachable commits until we reach maxTime. These may\n \t\t\t// wind up later matching up against things we want and we\ndiff --git a/org.spearce.jgit/src/org/spearce/jgit/transport/FetchConnection.java b/org.spearce.jgit/src/org/spearce/jgit/transport/FetchConnection.java\nindex a56ca6c..61ef219 100644\n--- a/org.spearce.jgit/src/org/spearce/jgit/transport/FetchConnection.java\n+++ b/org.spearce.jgit/src/org/spearce/jgit/transport/FetchConnection.java\n@@ -38,8 +38,10 @@\n package org.spearce.jgit.transport;\n \n import java.util.Collection;\n+import java.util.Set;\n \n import org.spearce.jgit.errors.TransportException;\n+import org.spearce.jgit.lib.ObjectId;\n import org.spearce.jgit.lib.ProgressMonitor;\n import org.spearce.jgit.lib.Ref;\n \n@@ -85,23 +87,29 @@\n \t * @param want\n \t *            one or more refs advertised by this connection that the caller\n \t *            wants to store locally.\n+\t * @param have\n+\t *            additional objects known to exist in the destination\n+\t *            repository, especially if they aren't yet reachable by the ref\n+\t *            database. Connections should take this set as an addition to\n+\t *            what is reachable through all Refs, not in replace of it.\n \t * @throws TransportException\n \t *             objects could not be copied due to a network failure,\n \t *             protocol error, or error on remote side, or connection was\n \t *             already used for fetch.\n \t */\n-\tpublic void fetch(final ProgressMonitor monitor, final Collection<Ref> want)\n+\tpublic void fetch(final ProgressMonitor monitor,\n+\t\t\tfinal Collection<Ref> want, final Set<ObjectId> have)\n \t\t\tthrows TransportException;\n \n \t/**\n-\t * Did the last {@link #fetch(ProgressMonitor, Collection)} get tags?\n+\t * Did the last {@link #fetch(ProgressMonitor, Collection, Set)} get tags?\n \t * <p>\n \t * Some Git aware transports are able to implicitly grab an annotated tag if\n \t * {@link TagOpt#AUTO_FOLLOW} or {@link TagOpt#FETCH_TAGS} was selected and\n \t * the object the tag peels to (references) was transferred as part of the\n-\t * last {@link #fetch(ProgressMonitor, Collection)} call. If it is possible\n-\t * for such tags to have been included in the transfer this method returns\n-\t * true, allowing the caller to attempt tag discovery.\n+\t * last {@link #fetch(ProgressMonitor, Collection, Set)} call. If it is\n+\t * possible for such tags to have been included in the transfer this method\n+\t * returns true, allowing the caller to attempt tag discovery.\n \t * <p>\n \t * By returning only true/false (and not the actual list of tags obtained)\n \t * the transport itself does not need to be aware of whether or not tags\n@@ -113,7 +121,8 @@ public void fetch(final ProgressMonitor monitor, final Collection<Ref> want)\n \tpublic boolean didFetchIncludeTags();\n \n \t/**\n-\t * Did the last {@link #fetch(ProgressMonitor, Collection)} validate graph?\n+\t * Did the last {@link #fetch(ProgressMonitor, Collection, Set)} validate\n+\t * graph?\n \t * <p>\n \t * Some transports walk the object graph on the client side, with the client\n \t * looking for what objects it is missing and requesting them individually\ndiff --git a/org.spearce.jgit/src/org/spearce/jgit/transport/FetchProcess.java b/org.spearce.jgit/src/org/spearce/jgit/transport/FetchProcess.java\nindex bb2d051..09718eb 100644\n--- a/org.spearce.jgit/src/org/spearce/jgit/transport/FetchProcess.java\n+++ b/org.spearce.jgit/src/org/spearce/jgit/transport/FetchProcess.java\n@@ -75,6 +75,9 @@\n \t/** Set of refs we will actually wind up asking to obtain. */\n \tprivate final HashMap<ObjectId, Ref> askFor = new HashMap<ObjectId, Ref>();\n \n+\t/** Objects we know we have locally. */\n+\tprivate final HashSet<ObjectId> have = new HashSet<ObjectId>();\n+\n \t/** Updates to local tracking branches (if any). */\n \tprivate final ArrayList<TrackingRefUpdate> localUpdates = new ArrayList<TrackingRefUpdate>();\n \n@@ -133,6 +136,7 @@ else if (tagopt == TagOpt.FETCH_TAGS)\n \t\t\t\t// There are more tags that we want to follow, but\n \t\t\t\t// not all were asked for on the initial request.\n \t\t\t\t//\n+\t\t\t\thave.addAll(askFor.keySet());\n \t\t\t\taskFor.clear();\n \t\t\t\tfor (final Ref r : additionalTags) {\n \t\t\t\t\tfinal ObjectId id = r.getPeeledObjectId();\n@@ -173,7 +177,7 @@ else if (tagopt == TagOpt.FETCH_TAGS)\n \n \tprivate void fetchObjects(final ProgressMonitor monitor)\n \t\t\tthrows TransportException {\n-\t\tconn.fetch(monitor, askFor.values());\n+\t\tconn.fetch(monitor, askFor.values(), have);\n \t\tif (transport.isCheckFetchedObjects()\n \t\t\t\t&& !conn.didFetchTestConnectivity() && !askForIsComplete())\n \t\t\tthrow new TransportException(transport.getURI(),\ndiff --git a/org.spearce.jgit/src/org/spearce/jgit/transport/TransportBundle.java b/org.spearce.jgit/src/org/spearce/jgit/transport/TransportBundle.java\nindex 7d38b02..1734d94 100644\n--- a/org.spearce.jgit/src/org/spearce/jgit/transport/TransportBundle.java\n+++ b/org.spearce.jgit/src/org/spearce/jgit/transport/TransportBundle.java\n@@ -171,7 +171,8 @@ public boolean didFetchTestConnectivity() {\n \n \t\t@Override\n \t\tprotected void doFetch(final ProgressMonitor monitor,\n-\t\t\t\tfinal Collection<Ref> want) throws TransportException {\n+\t\t\t\tfinal Collection<Ref> want, final Set<ObjectId> have)\n+\t\t\t\tthrows TransportException {\n \t\t\tverifyPrerequisites();\n \t\t\ttry {\n \t\t\t\tfinal IndexPack ip = newIndexPack();\ndiff --git a/org.spearce.jgit/src/org/spearce/jgit/transport/WalkFetchConnection.java b/org.spearce.jgit/src/org/spearce/jgit/transport/WalkFetchConnection.java\nindex d089f7b..91c5ea8 100644\n--- a/org.spearce.jgit/src/org/spearce/jgit/transport/WalkFetchConnection.java\n+++ b/org.spearce.jgit/src/org/spearce/jgit/transport/WalkFetchConnection.java\n@@ -195,8 +195,9 @@ public boolean didFetchTestConnectivity() {\n \n \t@Override\n \tprotected void doFetch(final ProgressMonitor monitor,\n-\t\t\tfinal Collection<Ref> want) throws TransportException {\n-\t\tmarkLocalRefsComplete();\n+\t\t\tfinal Collection<Ref> want, final Set<ObjectId> have)\n+\t\t\tthrows TransportException {\n+\t\tmarkLocalRefsComplete(have);\n \t\tqueueWants(want);\n \n \t\twhile (!monitor.isCancelled() && !workQueue.isEmpty()) {\n@@ -642,7 +643,7 @@ private void saveLooseObject(final AnyObjectId id, final byte[] compressed)\n \t\treturn null;\n \t}\n \n-\tprivate void markLocalRefsComplete() throws TransportException {\n+\tprivate void markLocalRefsComplete(final Set<ObjectId> have) throws TransportException {\n \t\tfor (final Ref r : local.getAllRefs().values()) {\n \t\t\ttry {\n \t\t\t\tmarkLocalObjComplete(revWalk.parseAny(r.getObjectId()));\n@@ -651,6 +652,13 @@ private void markLocalRefsComplete() throws TransportException {\n \t\t\t\t\t\t+ \" is missing object(s).\", readError);\n \t\t\t}\n \t\t}\n+\t\tfor (final ObjectId id : have) {\n+\t\t\ttry {\n+\t\t\t\tmarkLocalObjComplete(revWalk.parseAny(id));\n+\t\t\t} catch (IOException readError) {\n+\t\t\t\tthrow new TransportException(\"Missing assumed \"+id.name(), readError);\n+\t\t\t}\n+\t\t}\n \t}\n \n \tprivate void markLocalObjComplete(RevObject obj) throws IOException {\n-- \n1.6.1.rc4.301.g5497a\n"},{"id":"99040","messageId":"200812310812.15305.robin.rosenberg@dewire.com","threadId":"16853","inReplyTo":"1230055423-9944-5-git-send-email-spearce@spearce.org","subject":"Re: [JGIT PATCH 4/5] Add \"jgit clone\" to support cloning off URLs that are JGit specific","fromName":"Robin Rosenberg","fromEmail":"robin.rosenberg@dewire.com","sentAt":"2008-12-31T07:12:14Z","receivedAt":"2008-12-31T07:12:14Z","isPatch":true,"sender":{"key":"robin.rosenberg@dewire.com","avatar":"https://avatars.githubusercontent.com/u/46357?v=4"},"body":"Would it be hard to get the progress look better?\n\n-- robin\n\nEGIT.contrib/jgit clone git://repo.or.cz/libgit2.git LIBGIT2\nInitialized empty Git repository in /home/me/SW/LIBGIT2/.git                  \nCounting objects:       547                                                   \nCompressing objects:    100% (192/192)                                        \nts:                     100% (192/192)                                        \nCompressing objects:    100% (192/192)                                        \nng objects:             100% (192/192)                                        \nCompressing objects:    100% (192/192)                                        \nbjects:                 100% (192/192)                                        \nCompressing objects:    100% (192/192)\npressing objects:       100% (192/192)\nts:                     100% (192/192)\nCompressing objects:    100% (192/192)\nReceiving objects:      100% (547/547)\njects:                  100% (192/192)\nCompressing objects:    100% (192/192)\nressing objects:        100% (192/192)\ns:                      100% (192/192)\nCompressing objects:    100% (192/192)\nssing objects:          100% (192/192)\nts:                     100% (192/192)\nCompressing objects:    100% (192/192)\n objects:               100% (192/192)\nCompressing objects:    100% (192/192)\nessing objects:         100% (192/192)\ncts:                    100% (192/192)\nCompressing objects:    100% (192/192)\ng objects:              100% (192/192)\nCompressing objects:    100% (192/192)\nressing objects:        100% (192/192)\nResolving deltas:       100% (358/358)\nFrom git://repo.or.cz/libgit2.git\n * [new branch]      master     -> origin/master\n"},{"id":"99041","messageId":"20081231073505.GA22551@spearce.org","threadId":"16853","inReplyTo":"200812310812.15305.robin.rosenberg@dewire.com","subject":"Re: [JGIT PATCH 4/5] Add \"jgit clone\" to support cloning off URLs that are JGit specific","fromName":"Shawn O. Pearce","fromEmail":"spearce@spearce.org","sentAt":"2008-12-31T07:35:05Z","receivedAt":"2008-12-31T07:35:05Z","isPatch":true,"sender":{"key":"spearce@spearce.org","avatar":"https://avatars.githubusercontent.com/u/34844?v=4"},"body":"Robin Rosenberg <robin.rosenberg@dewire.com> wrote:\n> Would it be hard to get the progress look better?\n\nThis happens because we are scanning the progress channel wrong in\nsome cases.  Its random; most of the time we can get it right, but\nevery once in a while we get out of sync and get these short names.\n\nThis is a general problem with the progress client side code, not\nthe new server side code I've introduced.  We also see it going\nagainst a standard C git daemon.  I haven't been able to figure\nout a way to smooth out the messages.\n\nFWIW, git-gui also gets this sort of progress corruption sometimes\nwhen it does a clone.  So I've reproduced the same parsing bug in\ntwo different languages and two different implementations... ;-)\n\n \n> EGIT.contrib/jgit clone git://repo.or.cz/libgit2.git LIBGIT2\n> Initialized empty Git repository in /home/me/SW/LIBGIT2/.git                  \n> Counting objects:       547                                                   \n> Compressing objects:    100% (192/192)                                        \n> ts:                     100% (192/192)                                        \n> Compressing objects:    100% (192/192)                                        \n> ng objects:             100% (192/192)                                        \n> Compressing objects:    100% (192/192)                                        \n> bjects:                 100% (192/192)                                        \n> Compressing objects:    100% (192/192)\n> pressing objects:       100% (192/192)\n> ts:                     100% (192/192)\n> Compressing objects:    100% (192/192)\n> Receiving objects:      100% (547/547)\n> jects:                  100% (192/192)\n> Compressing objects:    100% (192/192)\n> ressing objects:        100% (192/192)\n> s:                      100% (192/192)\n> Compressing objects:    100% (192/192)\n> ssing objects:          100% (192/192)\n> ts:                     100% (192/192)\n> Compressing objects:    100% (192/192)\n>  objects:               100% (192/192)\n> Compressing objects:    100% (192/192)\n> essing objects:         100% (192/192)\n> cts:                    100% (192/192)\n> Compressing objects:    100% (192/192)\n> g objects:              100% (192/192)\n> Compressing objects:    100% (192/192)\n> ressing objects:        100% (192/192)\n> Resolving deltas:       100% (358/358)\n> From git://repo.or.cz/libgit2.git\n>  * [new branch]      master     -> origin/master\n\n-- \nShawn.\n"},{"id":"99070","messageId":"20081231190401.GI29071@spearce.org","threadId":"16853","inReplyTo":"20081231073505.GA22551@spearce.org","subject":"[JGIT PATCH] Improve the sideband progress scaper to be more accurate","fromName":"Shawn O. Pearce","fromEmail":"spearce@spearce.org","sentAt":"2008-12-31T19:04:01Z","receivedAt":"2008-12-31T19:04:01Z","isPatch":true,"sender":{"key":"spearce@spearce.org","avatar":"https://avatars.githubusercontent.com/u/34844?v=4"},"body":"By matching only whole lines we should be able to improve the\nprogress scaper so we avoid ugly output like we had been seeing:\n\n  EGIT.contrib/jgit clone git://repo.or.cz/libgit2.git LIBGIT2\n  Initialized empty Git repository in /home/me/SW/LIBGIT2/.git\n  Counting objects:       547\n  Compressing objects:    100% (192/192)\n  ts:                     100% (192/192)\n  Compressing objects:    100% (192/192)\n  ng objects:             100% (192/192)\n\nSigned-off-by: Shawn O. Pearce <spearce@spearce.org>\n---\n Robin Rosenberg <robin.rosenberg@dewire.com> wrote:\n > Would it be hard to get the progress look better?\n\n Maybe this does the trick.  Its hard to reproduce so its hard to\n come up with the condition that was giving us the problem before.\n I suspect its because we were getting line fragments on the sideband\n channel, but I'm not sure that was really the case.\n\n .../jgit/transport/SideBandInputStream.java        |   37 +++++++++++++++++--\n 1 files changed, 33 insertions(+), 4 deletions(-)\n\ndiff --git a/org.spearce.jgit/src/org/spearce/jgit/transport/SideBandInputStream.java b/org.spearce.jgit/src/org/spearce/jgit/transport/SideBandInputStream.java\nindex 3ec9bff..f0ba3d3 100644\n--- a/org.spearce.jgit/src/org/spearce/jgit/transport/SideBandInputStream.java\n+++ b/org.spearce.jgit/src/org/spearce/jgit/transport/SideBandInputStream.java\n@@ -73,10 +73,10 @@\n \tprivate static final int CH_ERROR = 3;\n \n \tprivate static Pattern P_UNBOUNDED = Pattern.compile(\n-\t\t\t\".*?([\\\\w ]+): (\\\\d+)(, done)?.*\", Pattern.DOTALL);\n+\t\t\t\"^([\\\\w ]+): (\\\\d+)( |, done)?.*\", Pattern.DOTALL);\n \n \tprivate static Pattern P_BOUNDED = Pattern.compile(\n-\t\t\t\".*?([\\\\w ]+):.*\\\\((\\\\d+)/(\\\\d+)\\\\).*\", Pattern.DOTALL);\n+\t\t\t\"^([\\\\w ]+):.*\\\\((\\\\d+)/(\\\\d+)\\\\).*\", Pattern.DOTALL);\n \n \tprivate final PacketLineIn pckIn;\n \n@@ -84,6 +84,8 @@\n \n \tprivate final ProgressMonitor monitor;\n \n+\tprivate String progressBuffer = \"\";\n+\n \tprivate String currentTask;\n \n \tprivate int lastCnt;\n@@ -160,7 +162,31 @@ private void needDataPacket() throws IOException {\n \t\t}\n \t}\n \n-\tprivate void progress(final String msg) {\n+\tprivate void progress(String pkt) {\n+\t\tpkt = progressBuffer + pkt;\n+\t\tfor (;;) {\n+\t\t\tfinal int lf = pkt.indexOf('\\n');\n+\t\t\tfinal int cr = pkt.indexOf('\\r');\n+\t\t\tfinal int s;\n+\t\t\tif (0 <= lf && 0 <= cr)\n+\t\t\t\ts = Math.min(lf, cr);\n+\t\t\telse if (0 <= lf)\n+\t\t\t\ts = lf;\n+\t\t\telse if (0 <= cr)\n+\t\t\t\ts = cr;\n+\t\t\telse\n+\t\t\t\tbreak;\n+\n+\t\t\tfinal String msg = pkt.substring(0, s);\n+\t\t\tif (doProgressLine(msg))\n+\t\t\t\tpkt = pkt.substring(s + 1);\n+\t\t\telse\n+\t\t\t\tbreak;\n+\t\t}\n+\t\tprogressBuffer = pkt;\n+\t}\n+\n+\tprivate boolean doProgressLine(final String msg) {\n \t\tMatcher matcher;\n \n \t\tmatcher = P_BOUNDED.matcher(msg);\n@@ -175,7 +201,7 @@ private void progress(final String msg) {\n \t\t\tfinal int cnt = Integer.parseInt(matcher.group(2));\n \t\t\tmonitor.update(cnt - lastCnt);\n \t\t\tlastCnt = cnt;\n-\t\t\treturn;\n+\t\t\treturn true;\n \t\t}\n \n \t\tmatcher = P_UNBOUNDED.matcher(msg);\n@@ -189,7 +215,10 @@ private void progress(final String msg) {\n \t\t\tfinal int cnt = Integer.parseInt(matcher.group(2));\n \t\t\tmonitor.update(cnt - lastCnt);\n \t\t\tlastCnt = cnt;\n+\t\t\treturn true;\n \t\t}\n+\n+\t\treturn false;\n \t}\n \n \tprivate String readString(final int len) throws IOException {\n-- \n1.6.1.59.g6f6746\n\n\n-- \nShawn.\n"},{"id":"99269","messageId":"200901032312.53148.robin.rosenberg@dewire.com","threadId":"16853","inReplyTo":"20081231190401.GI29071@spearce.org","subject":"Re: [JGIT PATCH] Improve the sideband progress scaper to be more accurate","fromName":"Robin Rosenberg","fromEmail":"robin.rosenberg@dewire.com","sentAt":"2009-01-03T22:12:52Z","receivedAt":"2009-01-03T22:12:52Z","isPatch":true,"sender":{"key":"robin.rosenberg@dewire.com","avatar":"https://avatars.githubusercontent.com/u/46357?v=4"},"body":"onsdag 31 december 2008 20:04:01 skrev Shawn O. Pearce:\n> By matching only whole lines we should be able to improve the\n> progress scaper so we avoid ugly output like we had been seeing:\n> \n>   EGIT.contrib/jgit clone git://repo.or.cz/libgit2.git LIBGIT2\n>   Initialized empty Git repository in /home/me/SW/LIBGIT2/.git\n>   Counting objects:       547\n>   Compressing objects:    100% (192/192)\n>   ts:                     100% (192/192)\n>   Compressing objects:    100% (192/192)\n>   ng objects:             100% (192/192)\n> \n> Signed-off-by: Shawn O. Pearce <spearce@spearce.org>\n> ---\n>  Robin Rosenberg <robin.rosenberg@dewire.com> wrote:\n>  > Would it be hard to get the progress look better?\n> \n>  Maybe this does the trick.  Its hard to reproduce so its hard to\n>  come up with the condition that was giving us the problem before.\n>  I suspect its because we were getting line fragments on the sideband\n>  channel, but I'm not sure that was really the case.\n\nNasty. I couldn't reproduce it myself. I'll hold onto this one for a while and\nsee if I'll get the opportunity to test it live with this problem.\n\n-- robin\n"},{"id":"99492","messageId":"alpine.LFD.2.00.0901061343000.26118@xanadu.home","threadId":"16853","inReplyTo":"200901032312.53148.robin.rosenberg@dewire.com","subject":"Re: [JGIT PATCH] Improve the sideband progress scaper to be more accurate","fromName":"Nicolas Pitre","fromEmail":"nico@cam.org","sentAt":"2009-01-06T19:23:02Z","receivedAt":"2009-01-06T19:23:02Z","isPatch":true,"sender":{"key":"nico@fluxnic.net","avatar":"https://avatars.githubusercontent.com/u/702790?v=4"},"body":"On Sat, 3 Jan 2009, Robin Rosenberg wrote:\n\n> onsdag 31 december 2008 20:04:01 skrev Shawn O. Pearce:\n> > By matching only whole lines we should be able to improve the\n> > progress scaper so we avoid ugly output like we had been seeing:\n> > \n> >   EGIT.contrib/jgit clone git://repo.or.cz/libgit2.git LIBGIT2\n> >   Initialized empty Git repository in /home/me/SW/LIBGIT2/.git\n> >   Counting objects:       547\n> >   Compressing objects:    100% (192/192)\n> >   ts:                     100% (192/192)\n> >   Compressing objects:    100% (192/192)\n> >   ng objects:             100% (192/192)\n> > \n> > Signed-off-by: Shawn O. Pearce <spearce@spearce.org>\n> > ---\n> >  Robin Rosenberg <robin.rosenberg@dewire.com> wrote:\n> >  > Would it be hard to get the progress look better?\n> > \n> >  Maybe this does the trick.  Its hard to reproduce so its hard to\n> >  come up with the condition that was giving us the problem before.\n> >  I suspect its because we were getting line fragments on the sideband\n> >  channel, but I'm not sure that was really the case.\n> \n> Nasty. I couldn't reproduce it myself. I'll hold onto this one for a while and\n> see if I'll get the opportunity to test it live with this problem.\n\nThere may indeed be line fragments sent over the sideband channel, as \nwell as the opposite which is multiple lines sent at once in a single \npacket.  If you look at sideband.c you'll find about all those cases.\n\nIn general, what you have to do is:\n\n - for each packet:\n   - split into multiple chunks on line breaks ('\\r' or '\\n')\n   - for each chunk:\n     - if last chunk didn't end with a line break, or if current \n       chunk is empty or only contains a line break, then skip printing \n       the \"remote:\" prefix.  Otherwise print it.\n     - print the current chunk up to any line break\n     - if current chunk contains a line break and other characters then\n       print a sequence to clear the remaining of the screen line\n     - print the line break if any\n\nThat should cover all cases.  Note that the case where a previous chunk \ndidn't end with a line break is always seen when moving from one packet \nto another, so that state cannot be local to the inner loop.\n\n\nNicolas\n"},{"id":"99493","messageId":"20090106192747.GD24578@spearce.org","threadId":"16853","inReplyTo":"alpine.LFD.2.00.0901061343000.26118@xanadu.home","subject":"Re: [JGIT PATCH] Improve the sideband progress scaper to be more accurate","fromName":"Shawn O. Pearce","fromEmail":"spearce@spearce.org","sentAt":"2009-01-06T19:27:47Z","receivedAt":"2009-01-06T19:27:47Z","isPatch":true,"sender":{"key":"spearce@spearce.org","avatar":"https://avatars.githubusercontent.com/u/34844?v=4"},"body":"Nicolas Pitre <nico@cam.org> wrote:\n> There may indeed be line fragments sent over the sideband channel, as \n> well as the opposite which is multiple lines sent at once in a single \n> packet.  If you look at sideband.c you'll find about all those cases.\n\nThanks, that's what I thought...\n \n> In general, what you have to do is:\n> \n>  - for each packet:\n>    - split into multiple chunks on line breaks ('\\r' or '\\n')\n>    - for each chunk:\n>      - if last chunk didn't end with a line break, or if current \n>        chunk is empty or only contains a line break, then skip printing \n>        the \"remote:\" prefix.  Otherwise print it.\n>      - print the current chunk up to any line break\n>      - if current chunk contains a line break and other characters then\n>        print a sequence to clear the remaining of the screen line\n>      - print the line break if any\n\nHmm.  I should note that C Git still screws this up sometimes.  I've\nseen 1.6.1 git fetch mess up the output from repo.or.cz's sideband.\nI'm sure Pasky isn't running JGit's daemon, its too damn fast. :-)\n\nI don't have a spew of it handy, but late last week I saw it screw\nup while doing a clone off repo.or.cz.\n\nRobin, maybe we should apply this patch?  Its along the lines\nNico is describing above, but because our progress code doesn't do\ndirect display its a little less complex, we don't have to worry\nabout clearing the current line.  But I think I'm doing the chunk\nsplitting and filtering he's talking about.\n\n-- \nShawn.\n"},{"id":"99510","messageId":"alpine.LFD.2.00.0901061546460.26118@xanadu.home","threadId":"16853","inReplyTo":"20090106192747.GD24578@spearce.org","subject":"Re: [JGIT PATCH] Improve the sideband progress scaper to be more accurate","fromName":"Nicolas Pitre","fromEmail":"nico@cam.org","sentAt":"2009-01-06T20:55:23Z","receivedAt":"2009-01-06T20:55:23Z","isPatch":true,"sender":{"key":"nico@fluxnic.net","avatar":"https://avatars.githubusercontent.com/u/702790?v=4"},"body":"On Tue, 6 Jan 2009, Shawn O. Pearce wrote:\n\n> Nicolas Pitre <nico@cam.org> wrote:\n> > There may indeed be line fragments sent over the sideband channel, as \n> > well as the opposite which is multiple lines sent at once in a single \n> > packet.  If you look at sideband.c you'll find about all those cases.\n> \n> Thanks, that's what I thought...\n>  \n> > In general, what you have to do is:\n> > \n> >  - for each packet:\n> >    - split into multiple chunks on line breaks ('\\r' or '\\n')\n> >    - for each chunk:\n> >      - if last chunk didn't end with a line break, or if current \n> >        chunk is empty or only contains a line break, then skip printing \n> >        the \"remote:\" prefix.  Otherwise print it.\n> >      - print the current chunk up to any line break\n> >      - if current chunk contains a line break and other characters then\n> >        print a sequence to clear the remaining of the screen line\n> >      - print the line break if any\n> \n> Hmm.  I should note that C Git still screws this up sometimes.  I've\n> seen 1.6.1 git fetch mess up the output from repo.or.cz's sideband.\n> I'm sure Pasky isn't running JGit's daemon, its too damn fast. :-)\n> \n> I don't have a spew of it handy, but late last week I saw it screw\n> up while doing a clone off repo.or.cz.\n\nNext time you see such things please send me a snapshot.\n\nWhat is still possible and IMHO not worth the effort to fix is some \ninteraction between local and remote displays which, if intermixed in \nsome unlucky way, do screw up the final appearance.  For example, if the \nremote has sent a partial line, and before the remaining of it is \nreceived/printed you actually have some local process also displaying a \nline of its own.  If that was to become a real issue, then the fix would \ninvolve caching any partial line sent from the remote until the \nassociated line break is received.\n\n\nNicolas\n"}]}