{"thread":{"id":"20543","subject":"[RFCv3 0/4] CVS remote helper","startedAt":"2009-08-12T00:13:47Z","lastAt":"2009-08-17T01:58:30Z","messageCount":22,"participants":["Johan Herland","David Aguilar","Sverre Rabbelier","Michael Haggerty","Junio C Hamano","Johannes Schindelin"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"120333","messageId":"1250036031-32272-1-git-send-email-johan@herland.net","threadId":"20543","inReplyTo":null,"subject":"[RFCv3 0/4] CVS remote helper","fromName":"Johan Herland","fromEmail":"johan@herland.net","sentAt":"2009-08-12T00:13:47Z","receivedAt":"2009-08-12T00:13:47Z","isPatch":false,"sender":{"key":"johan@herland.net","avatar":"https://avatars.githubusercontent.com/u/547031?v=4"},"body":"Hi,\n\nAnother iteration of the patch series implementing a CVS remote helper, as\npromised a couple of days ago. Changes from the previous iteration:\n\n- Rebased on top of Daniel Barkalow's latest foreign VCS helpers work\n (aka. 'db/vcs-helper' (early part)). This means that all the \"generic\"\n foreign-scm patches that were part of the previous iteration are no\n longer needed, and only the CVS remote helper patches remain.\n Also, this series applies cleanly to current 'pu'.\n\n- Replaced the \"git-vcs-cvs\" naming with \"git-remote-cvs\" throughout the code\n and documentation.\n\n- Minor updates to the git-remote-cvs implementation in order to more closely\n follow the current remote helper API.\n\n- Split up the patch series into somewhat smaller patches to (hopefully) be\n allowed onto git@vger.kernel.org.\n\n\nHave fun! :)\n\n...Johan\n\n\nJohan Herland (4):\n Basic build infrastructure for Python scripts\n Add Python support library for CVS remote helper\n Third draft of CVS remote helper program\n Add simple selftests of git-remote-cvs functionality\n\n Documentation/git-remote-cvs.txt | 85 ++++\n Makefile | 46 ++\n configure.ac | 3 +\n git-remote-cvs.py | 697 ++++++++++++++++++++++++++++\n git_remote_cvs/.gitignore | 2 +\n git_remote_cvs/Makefile | 27 ++\n git_remote_cvs/changeset.py | 114 +++++\n git_remote_cvs/commit_states.py | 52 +++\n git_remote_cvs/cvs.py | 884 ++++++++++++++++++++++++++++++++++++\n git_remote_cvs/cvs_revision_map.py | 362 +++++++++++++++\n git_remote_cvs/cvs_symbol_cache.py | 283 ++++++++++++\n git_remote_cvs/git.py | 586 ++++++++++++++++++++++++\n git_remote_cvs/setup.py | 12 +\n git_remote_cvs/util.py | 147 ++++++\n t/t9800-remote-cvs-basic.sh | 524 +++++++++++++++++++++\n t/t9801-remote-cvs-fetch.sh | 291 ++++++++++++\n t/test-lib.sh | 1 +\n 17 files changed, 4116 insertions(+), 0 deletions(-)\n create mode 100644 Documentation/git-remote-cvs.txt\n create mode 100755 git-remote-cvs.py\n create mode 100644 git_remote_cvs/.gitignore\n create mode 100644 git_remote_cvs/Makefile\n create mode 100644 git_remote_cvs/__init__.py\n create mode 100644 git_remote_cvs/changeset.py\n create mode 100644 git_remote_cvs/commit_states.py\n create mode 100644 git_remote_cvs/cvs.py\n create mode 100644 git_remote_cvs/cvs_revision_map.py\n create mode 100644 git_remote_cvs/cvs_symbol_cache.py\n create mode 100644 git_remote_cvs/git.py\n create mode 100644 git_remote_cvs/setup.py\n create mode 100644 git_remote_cvs/util.py\n create mode 100755 t/t9800-remote-cvs-basic.sh\n create mode 100755 t/t9801-remote-cvs-fetch.sh\n"},{"id":"120334","messageId":"1250036031-32272-2-git-send-email-johan@herland.net","threadId":"20543","inReplyTo":"1250036031-32272-1-git-send-email-johan@herland.net","subject":"[RFCv3 1/4] Basic build infrastructure for Python scripts","fromName":"Johan Herland","fromEmail":"johan@herland.net","sentAt":"2009-08-12T00:13:48Z","receivedAt":"2009-08-12T00:13:48Z","isPatch":false,"sender":{"key":"johan@herland.net","avatar":"https://avatars.githubusercontent.com/u/547031?v=4"},"body":"This patch adds basic boilerplate support (based on corresponding Perl\nsections) for enabling the building and installation Python scripts.\n\nThere are currently no Python scripts being built, and when Python\nscripts are added in future patches, their building and installation\ncan be disabled by defining NO_PYTHON.\n\nSigned-off-by: Johan Herland \n---\n Makefile | 13 +++++++++++++\n configure.ac | 3 +++\n t/test-lib.sh | 1 +\n 3 files changed, 17 insertions(+), 0 deletions(-)\n\ndiff --git a/Makefile b/Makefile\nindex 189aee5..969cef5 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -166,6 +166,8 @@ all::\n #\n # Define NO_PERL if you do not want Perl scripts or libraries at all.\n #\n+# Define NO_PYTHON if you do not want Python scripts or libraries at all.\n+#\n # Define NO_TCLTK if you do not want Tcl/Tk GUI.\n #\n # The TCL_PATH variable governs the location of the Tcl interpreter\n@@ -311,6 +313,7 @@ LIB_H =\n LIB_OBJS =\n PROGRAMS =\n SCRIPT_PERL =\n+SCRIPT_PYTHON =\n SCRIPT_SH =\n TEST_PROGRAMS =\n \n@@ -349,6 +352,7 @@ SCRIPT_PERL += git-svn.perl\n \n SCRIPTS = $(patsubst %.sh,%,$(SCRIPT_SH)) \\\n \t $(patsubst %.perl,%,$(SCRIPT_PERL)) \\\n+\t $(patsubst %.py,%,$(SCRIPT_PYTHON)) \\\n \t git-instaweb\n \n # Empty...\n@@ -404,8 +408,12 @@ endif\n ifndef PERL_PATH\n \tPERL_PATH = /usr/bin/perl\n endif\n+ifndef PYTHON_PATH\n+\tPYTHON_PATH = /usr/bin/python\n+endif\n \n export PERL_PATH\n+export PYTHON_PATH\n \n LIB_FILE=libgit.a\n XDIFF_LIB=xdiff/lib.a\n@@ -1259,6 +1267,10 @@ ifeq ($(PERL_PATH),)\n NO_PERL=NoThanks\n endif\n \n+ifeq ($(PYTHON_PATH),)\n+NO_PYTHON=NoThanks\n+endif\n+\n QUIET_SUBDIR0 = +$(MAKE) -C # space to separate -C and subdir\n QUIET_SUBDIR1 =\n \n@@ -1306,6 +1318,7 @@ prefix_SQ = $(subst ','\\'',$(prefix))\n \n SHELL_PATH_SQ = $(subst ','\\'',$(SHELL_PATH))\n PERL_PATH_SQ = $(subst ','\\'',$(PERL_PATH))\n+PYTHON_PATH_SQ = $(subst ','\\'',$(PYTHON_PATH))\n TCLTK_PATH_SQ = $(subst ','\\'',$(TCLTK_PATH))\n \n LIBS = $(GITLIBS) $(EXTLIBS)\ndiff --git a/configure.ac b/configure.ac\nindex 3f1922d..3749e5c 100644\n--- a/configure.ac\n+++ b/configure.ac\n@@ -241,6 +241,9 @@ GIT_ARG_SET_PATH(shell)\n # Define PERL_PATH to provide path to Perl.\n GIT_ARG_SET_PATH(perl)\n #\n+# Define PYTHON_PATH to provide path to Python.\n+GIT_ARG_SET_PATH(python)\n+#\n # Define ZLIB_PATH to provide path to zlib.\n GIT_ARG_SET_PATH(zlib)\n #\ndiff --git a/t/test-lib.sh b/t/test-lib.sh\nindex a5b8d03..01ea386 100644\n--- a/t/test-lib.sh\n+++ b/t/test-lib.sh\n@@ -714,6 +714,7 @@ case $(uname -s) in\n esac\n \n test -z \"$NO_PERL\" && test_set_prereq PERL\n+test -z \"$NO_PYTHON\" && test_set_prereq PYTHON\n \n # test whether the filesystem supports symbolic links\n ln -s x y 2>/dev/null && test -h y 2>/dev/null && test_set_prereq SYMLINKS\n-- \n1.6.4.rc3.138.ga6b98.dirty\n"},{"id":"120336","messageId":"1250036031-32272-3-git-send-email-johan@herland.net","threadId":"20543","inReplyTo":"1250036031-32272-1-git-send-email-johan@herland.net","subject":"[RFCv3 2/4] Add Python support library for CVS remote helper","fromName":"Johan Herland","fromEmail":"johan@herland.net","sentAt":"2009-08-12T00:13:49Z","receivedAt":"2009-08-12T00:13:49Z","isPatch":false,"sender":{"key":"johan@herland.net","avatar":"https://avatars.githubusercontent.com/u/547031?v=4"},"body":"This patch introduces a Python package called \"git_remote_cvs\" containing\nthe building blocks of the CVS remote helper. The CVS remote helper itself\nis NOT part of this patch.\n\nThe patch includes the necessary Makefile additions to build and install\nthe git_remote_cvs Python package along with the rest of Git.\n\nSigned-off-by: Johan Herland \n---\n Makefile | 9 +\n git_remote_cvs/.gitignore | 2 +\n git_remote_cvs/Makefile | 27 ++\n git_remote_cvs/changeset.py | 114 +++++\n git_remote_cvs/commit_states.py | 52 +++\n git_remote_cvs/cvs.py | 884 ++++++++++++++++++++++++++++++++++++\n git_remote_cvs/cvs_revision_map.py | 362 +++++++++++++++\n git_remote_cvs/cvs_symbol_cache.py | 283 ++++++++++++\n git_remote_cvs/git.py | 586 ++++++++++++++++++++++++\n git_remote_cvs/setup.py | 12 +\n git_remote_cvs/util.py | 147 ++++++\n 11 files changed, 2478 insertions(+), 0 deletions(-)\n create mode 100644 git_remote_cvs/.gitignore\n create mode 100644 git_remote_cvs/Makefile\n create mode 100644 git_remote_cvs/__init__.py\n create mode 100644 git_remote_cvs/changeset.py\n create mode 100644 git_remote_cvs/commit_states.py\n create mode 100644 git_remote_cvs/cvs.py\n create mode 100644 git_remote_cvs/cvs_revision_map.py\n create mode 100644 git_remote_cvs/cvs_symbol_cache.py\n create mode 100644 git_remote_cvs/git.py\n create mode 100644 git_remote_cvs/setup.py\n create mode 100644 git_remote_cvs/util.py\n\ndiff --git a/Makefile b/Makefile\nindex 969cef5..bb5cea2 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -1350,6 +1350,9 @@ endif\n ifndef NO_PERL\n \t$(QUIET_SUBDIR0)perl $(QUIET_SUBDIR1) PERL_PATH='$(PERL_PATH_SQ)' prefix='$(prefix_SQ)' all\n endif\n+ifndef NO_PYTHON\n+\t$(QUIET_SUBDIR0)git_remote_cvs $(QUIET_SUBDIR1) PYTHON_PATH='$(PYTHON_PATH_SQ)' prefix='$(prefix_SQ)' all\n+endif\n \t$(QUIET_SUBDIR0)templates $(QUIET_SUBDIR1)\n \n please_set_SHELL_PATH_to_a_more_modern_shell:\n@@ -1701,6 +1704,9 @@ install: all\n ifndef NO_PERL\n \t$(MAKE) -C perl prefix='$(prefix_SQ)' DESTDIR='$(DESTDIR_SQ)' install\n endif\n+ifndef NO_PYTHON\n+\t$(MAKE) -C git_remote_cvs prefix='$(prefix_SQ)' DESTDIR='$(DESTDIR_SQ)' install\n+endif\n ifndef NO_TCLTK\n \t$(MAKE) -C gitk-git install\n \t$(MAKE) -C git-gui gitexecdir='$(gitexec_instdir_SQ)' install\n@@ -1821,6 +1827,9 @@ ifndef NO_PERL\n \t$(RM) gitweb/gitweb.cgi\n \t$(MAKE) -C perl clean\n endif\n+ifndef NO_PYTHON\n+\t$(MAKE) -C git_remote_cvs clean\n+endif\n \t$(MAKE) -C templates/ clean\n \t$(MAKE) -C t/ clean\n ifndef NO_TCLTK\ndiff --git a/git_remote_cvs/.gitignore b/git_remote_cvs/.gitignore\nnew file mode 100644\nindex 0000000..2247d5f\n--- /dev/null\n+++ b/git_remote_cvs/.gitignore\n@@ -0,0 +1,2 @@\n+/build\n+/dist\ndiff --git a/git_remote_cvs/Makefile b/git_remote_cvs/Makefile\nnew file mode 100644\nindex 0000000..8dbf3fa\n--- /dev/null\n+++ b/git_remote_cvs/Makefile\n@@ -0,0 +1,27 @@\n+#\n+# Makefile for the git_remote_cvs python support modules\n+#\n+pysetupfile:=setup.py\n+\n+ifndef PYTHON_PATH\n+\tPYTHON_PATH = /usr/bin/python\n+endif\n+ifndef prefix\n+\tprefix = $(HOME)\n+endif\n+ifndef V\n+\tQUIET = @\n+\tQUIETSETUP = --quiet\n+endif\n+\n+PYLIBDIR=`$(PYTHON_PATH) -c \"import sys; print 'lib/python%i.%i/site-packages' % sys.version_info[:2]\"`\n+\n+all: $(pysetupfile)\n+\t$(QUIET)$(PYTHON_PATH) $(pysetupfile) $(QUIETSETUP) build\n+install: $(pysetupfile)\n+\t$(PYTHON_PATH) $(pysetupfile) install --prefix $(prefix)\n+instlibdir: $(pysetupfile)\n+\t@echo \"$(prefix)/$(PYLIBDIR)\"\n+clean:\n+\t$(QUIET)$(PYTHON_PATH) $(pysetupfile) $(QUIETSETUP) clean -a\n+\t$(RM) *.pyo *.pyc\ndiff --git a/git_remote_cvs/__init__.py b/git_remote_cvs/__init__.py\nnew file mode 100644\nindex 0000000..e69de29\ndiff --git a/git_remote_cvs/changeset.py b/git_remote_cvs/changeset.py\nnew file mode 100644\nindex 0000000..27c4129\n--- /dev/null\n+++ b/git_remote_cvs/changeset.py\n@@ -0,0 +1,114 @@\n+#!/usr/bin/env python\n+\n+\"\"\"Functionality for collecting individual CVS revisions into \"changesets\"\n+\n+A changeset is a collection of CvsRev objects that belong together in the same\n+\"commit\". This is a somewhat artificial construct on top of CVS, which only\n+stores changes at the per-file level. Normally, CVS users create several CVS\n+revisions simultaneously by applying the \"cvs commit\" command to several files\n+with related changes. This module tries to reconstruct this notion of related\n+revisions.\n+\"\"\"\n+\n+from util import *\n+\n+\n+class Changeset (object):\n+\t\"\"\"Encapsulate a single changeset/commit\"\"\"\n+\n+\t__slots__ = ('revs', 'date', 'author', 'message')\n+\n+\t# The maximum time between the changeset's date, and the date of a rev\n+\t# to included in that changeset.\n+\tMaxSecondsBetweenRevs = 8 * 60 * 60 # 8 hours\n+\n+\t@classmethod\n+\tdef from_rev (cls, rev):\n+\t\t\"\"\"Return a Changeset based on the given CvsRev object\"\"\"\n+\t\tc = cls(rev.date, rev.author, rev.message)\n+\t\tresult = c.add(rev)\n+\t\tassert result\n+\t\treturn c\n+\n+\tdef __init__ (self, date, author, message):\n+\t\tself.revs = {} # dict: path -> CvsRev object\n+\t\tself.date = date # CvsDate object\n+\t\tself.author = author\n+\t\tself.message = message # Lines of commit message\n+\n+\tdef __str__ (self):\n+\t\tmsg = self.message[0] # First line only\n+\t\tif len(msg) > 25: msg = msg[:22] + \"...\" # Max 25 chars long\n+\t\treturn \"\" % (\n+\t\t\tself.date, self.author, msg, len(self.revs))\n+\n+\tdef __iter__ (self):\n+\t\treturn self.revs.itervalues()\n+\n+\tdef __getitem__ (self, path):\n+\t\treturn self.revs[path]\n+\n+\tdef within_time_window (self, rev):\n+\t\t\"\"\"Return True iff the rev is within the time window of self\"\"\"\n+\t\treturn abs(rev.date.diff(self.date)) <= \\\n+\t\t self.MaxSecondsBetweenRevs\n+\n+\tdef add (self, rev):\n+\t\t\"\"\"Add the given CvsRev to this Changeset\n+\n+\t\tThe addition will only succeed if the following holds:\n+\t\t - rev.author == self.author\n+\t\t - rev.message == self.message\n+\t\t - rev.path is not in self.revs\n+\t\t - rev.date is within MaxSecondsBetweenRevs of self.date\n+\t\tIf the addition succeeds, True is returned; otherwise False.\n+\t\t\"\"\"\n+\t\tif rev.author != self.author: return False\n+\t\tif rev.message != self.message: return False\n+\t\tif rev.path in self.revs: return False\n+\t\tif not self.within_time_window(rev): return False\n+\n+\t\tself.revs[rev.path] = rev\n+\t\treturn True\n+\n+\n+def build_changesets_from_revs (cvs_revs):\n+\t\"\"\"Organize CvsRev objects into a chronological list of Changesets\"\"\"\n+\n+\t# Construct chronological list of CvsRev objects\n+\tchron_revs = []\n+\tfor path, d in cvs_revs.iteritems():\n+\t\ti = 0 # Current index into chronRevs\n+\t\tfor revnum, cvsrev in sorted(d.iteritems()):\n+\t\t\twhile i < len(chron_revs) \\\n+\t\t\t and cvsrev.date > chron_revs[i].date:\n+\t\t\t\ti += 1\n+\t\t\t# Insert cvsRev at position i in chronRevs\n+\t\t\tchron_revs.insert(i, cvsrev)\n+\t\t\ti += 1\n+\n+\tchangesets = [] # Chronological list of Changeset objects\n+\twhile len(chron_revs): # There are still revs to be added to Changesets\n+\t\t# Create Changeset based on the first rev in chronRevs\n+\t\tchangeset = Changeset.from_rev(chron_revs.pop(0))\n+\t\t# Keep adding revs chronologically until MaxSecondsBetweenRevs\n+\t\trejects = [] # Revs that cannot be added to this changeset\n+\t\twhile len(chron_revs):\n+\t\t\trev = chron_revs.pop(0)\n+\t\t\treject = False\n+\t\t\t# First, if we have one of rev's parents in rejects, we\n+\t\t\t# must also reject rev\n+\t\t\tfor r in rejects:\n+\t\t\t\tif r.path == rev.path:\n+\t\t\t\t\treject = True\n+\t\t\t\t\tbreak\n+\t\t\t# Next, add rev to changeset, reject if add fails\n+\t\t\tif not reject: reject = not changeset.add(rev)\n+\t\t\tif reject:\n+\t\t\t\trejects.append(rev)\n+\t\t\t\t# stop trying when rev is too far in the future\n+\t\t\t\tif not changeset.within_time_window(rev): break\n+\t\tchron_revs = rejects + chron_revs # Reconstruct remaining revs\n+\t\tchangesets.append(changeset)\n+\n+\treturn changesets\ndiff --git a/git_remote_cvs/commit_states.py b/git_remote_cvs/commit_states.py\nnew file mode 100644\nindex 0000000..f4b769b\n--- /dev/null\n+++ b/git_remote_cvs/commit_states.py\n@@ -0,0 +1,52 @@\n+#!/usr/bin/env python\n+\n+\"\"\"Functionality for relating Git commits to corresponding CvsState objects\"\"\"\n+\n+from util import *\n+from cvs import CvsState\n+\n+\n+class CommitStates (object):\n+\t\"\"\"Provide a mapping from Git commits to CvsState objects\n+\n+\tBehaves like a dictionary of Git commit names -> CvsState mappings.\n+\n+\tEvery Git commit converted from CVS has a corresponding CvsState, which\n+\tdescribes exactly which CVS revisions are present in a checkout of that\n+\tcommit.\n+\n+\tThis class provides the interface to map from a Git commit to its\n+\tcorresponding CvsState. The mapping uses GitNotes as a persistent\n+\tstorage backend.\n+\t\"\"\"\n+\n+\tdef __init__ (self, notes):\n+\t\tself.notes = notes\n+\t\tself._cache = {} # commitname -> CvsState\n+\n+\tdef _load (self, commit):\n+\t\tif commit is None: return None\n+\t\tif commit in self._cache: return self._cache[commit]\n+\t\tnotedata = self.notes.get(commit)\n+\t\tif notedata is None: # Given commit has no associated note\n+\t\t\treturn None\n+\t\tstate = CvsState()\n+\t\tstate.load_data(notedata)\n+\t\tself._cache[commit] = state\n+\t\treturn state\n+\n+\tdef add (self, commit, state, gfi):\n+\t\tassert commit not in self._cache\n+\t\tself._cache[commit] = state\n+\t\tself.notes.import_note(commit, str(state), gfi)\n+\n+\tdef __getitem__ (self, commit):\n+\t\tstate = self._load(commit)\n+\t\tif state is None:\n+\t\t\traise KeyError, \"Unknown commit '%s'\" % (commit)\n+\t\treturn state\n+\n+\tdef get (self, commit, default = None):\n+\t\tstate = self._load(commit)\n+\t\tif state is None: return default\n+\t\treturn state\ndiff --git a/git_remote_cvs/cvs.py b/git_remote_cvs/cvs.py\nnew file mode 100644\nindex 0000000..cc2e13f\n--- /dev/null\n+++ b/git_remote_cvs/cvs.py\n@@ -0,0 +1,884 @@\n+#!/usr/bin/env python\n+\n+\"\"\"Functionality for interacting with CVS repositories\n+\n+This module provides classes for interrogating a CVS repository via a CVS\n+working directory (aka. checkout), or via direct queries using the \"cvs rlog\"\n+command.\n+\n+Also, classes for encapsulating fundamental CVS concepts (like CVS numbers)\n+are provided.\n+\"\"\"\n+\n+import sys, time, shutil\n+from calendar import timegm\n+\n+from util import *\n+\n+\n+class CvsNum (object):\n+\t\"\"\"Encapsulate a single CVS revision/branch number\n+\n+\tProvides functionality for common operations on CVS numbers.\n+\n+\tA CVS number consists of a list of components separated by dots ('.'),\n+\twhere each component is a decimal number. Inspecting the components\n+\tfrom left to right, the odd-numbered (1st, 3rd, 5th, etc.) components\n+\trepresent branches in the CVS history tree, while the even-numbered\n+\t(2nd, 4th, 6th, etc.) components represent revisions on the branch\n+\tspecified in the previous position. Thus \"1.2\" denotes the second\n+\trevision on the first branch (aka. trunk), while \"1.2.4.6\" denotes the\n+\tsixth revision of the fourth branch started from revision \"1.2\".\n+\n+\tTherefore, in general, a CVS number with an even number of components\n+\tdenotes a revision (we call this a \"revision number\"), while an odd\n+\tnumber of components denotes a branch (called a \"branch number\").\n+\n+\tThere are a few complicating peculiarities: If there is an even number\n+\tof components, and the second-last component is 0, the number is not\n+\ta revision number, but is rather equivalent to the branch number we get\n+\tby removing the 0-component. I.e. \"1.2.0.4\" is equivalent to \"1.2.4\".\n+\n+\tA branch number (except the trunk: \"1\") always has a \"branch point\"\n+\trevision, i.e. the revision from which the branch was started. This\n+\trevision is found by removing the last component of the branch number.\n+\tFor example the branch point of \"1.2.4\" is \"1.2\".\n+\n+\tConversely, all revision numbers belong to a corresponding branch,\n+\twhose branch number is found by removing the last component. Examples:\n+\tThe \"1.2.4.6\" revision belong to the \"1.2.4\" branch, while the \"1.2\"\n+\trevision belongs to the \"1\" branch (the \"trunk\").\n+\n+\tFrom this we can programatically determine the ancestry of any revision\n+\tnumber, by decrementing the last revision component until it equals 1,\n+\tand then trim off the last two components to get to the branch point,\n+\tand repeat the process from there until we reach the initial revision\n+\t(typically \"1.1\"). For example, recursively enumerating the parent\n+\trevisions of \"1.2.4.6\" yields the following revisions:\n+\t \"1.2.4.5\", \"1.2.4.4\", \"1.2.4.3\", \"1.2.4.2\", \"1.2.4.1\", \"1.2\", \"1.1\"\n+\t\"\"\"\n+\n+\t__slots__ = ('l',)\n+\n+\t@staticmethod\n+\tdef decompose (cvsnum):\n+\t\t\"\"\"Split the given CVS number into a list of integer components\n+\n+\t\tBranch numbers are normalized to the odd-numbered components\n+\t\tform (i.e. removing the second last '0' component)\n+\n+\t\tExamples:\n+\t\t '1.2.4.8' -> [1, 2, 4, 8]\n+\t\t '1.2.3' -> [1, 2, 3]\n+\t\t '1.2.0.5' -> [1, 2, 5]\n+\t\t\"\"\"\n+\t\tif cvsnum: r = map(int, cvsnum.split('.'))\n+\t\telse: r = []\n+\t\tif len(r) >= 2 and r[-2] == 0: del r[-2]\n+\t\treturn tuple(r)\n+\n+\t@staticmethod\n+\tdef compose (l):\n+\t\t\"\"\"Join the given list of integer components into a CVS number\n+\n+\t\tE.g.: (1, 2, 4, 8) -> '1.2.4.8'\n+\t\t\"\"\"\n+\t\treturn \".\".join(map(str, l))\n+\n+\t@classmethod\n+\tdef from_components (cls, args):\n+\t\treturn cls(cls.compose(args))\n+\n+\tdef __init__ (self, cvsnum):\n+\t\tself.l = self.decompose(str(cvsnum))\n+\n+\tdef __repr__ (self):\n+\t\treturn self.compose(self.l)\n+\n+\tdef __str__ (self):\n+\t\treturn repr(self)\n+\n+\tdef __hash__ (self):\n+\t\treturn hash(repr(self))\n+\n+\tdef __len__ (self):\n+\t\treturn len(self.l)\n+\n+\tdef __cmp__ (self, other):\n+\t\ttry: return cmp(self.l, other.l)\n+\t\texcept AttributeError: return 1\n+\n+\tdef __getitem__ (self, key):\n+\t\treturn self.l[key]\n+\n+\tdef is_rev (self):\n+\t\t\"\"\"Return True iff this number is a CVS revision number\"\"\"\n+\t\treturn len(self.l) % 2 == 0 \\\n+\t and len(self.l) >= 2 \\\n+\t\t and self.l[-2] != 0\n+\n+\tdef is_branch (self):\n+\t\t\"\"\"Return True iff this number is a CVS branch number\"\"\"\n+\t\treturn len(self.l) % 2 != 0 \\\n+\t or (len(self.l) >= 2 and self.l[-2] == 0)\n+\n+\tdef enumerate (self):\n+\t\t\"\"\"Return a list of integer components in this CVS number\"\"\"\n+\t\treturn list(self.l)\n+\n+\tdef branch (self):\n+\t\t\"\"\"Return the branch on which the given number lives.\n+\n+\t\tRevisions: chop the last component to find the branch, e.g.:\n+\t\t\t1.2.4.6 -> 1.2.4\n+\t\t\t1.1 -> 1\n+\t\tBranches: unchanged\n+\t\t\"\"\"\n+\t\tif self.is_rev():\n+\t\t\treturn self.from_components(self.l[:-1])\n+\t\treturn self\n+\n+\tdef parent (self):\n+\t\t\"\"\"Return the parent/previous revision number to this number.\n+\n+\t\tFor revisions, this is the previous revision, e.g.:\n+\t\t\t1.2.4.6 -> 1.2.4.5\n+\t\t\t1.2.4.1 -> 1.2\n+\t\t\t1.1 -> None\n+\t\t\t2.1 -> None\n+\t\tFor branches, this is the branch point, e.g.:\n+\t\t\t1.2.4 -> 1.2\n+\t\t\t1 -> None\n+\t\t\t2 -> None\n+\t\t\"\"\"\n+\t\tif len(self.l) < 2: return None\n+\t\telif len(self.l) % 2: # branch number\n+\t\t\treturn self.from_components(self.l[:-1])\n+\t\telse: # revision number\n+\t\t\tassert self.l[-1] > 0\n+\t\t\tresult = self.enumerate()\n+\t\t\tresult[-1] -= 1 # Decrement final component\n+\t\t\tif result[-1] == 0: # We're at the start of the branch\n+\t\t\t\tdel result[-2:] # Make into branch point\n+\t\t\t\tif not result: return None\n+\t\t\treturn self.from_components(result)\n+\n+\tdef follows (self, other):\n+\t\t\"\"\"Return True iff self historically follows the given rev\n+\n+\t\tThis iterates through the parents of self, and returns True iff\n+\t\tany of them equals the given rev. Otherwise, False is returned.\n+\t\t\"\"\"\n+\t\tassert other.is_rev()\n+\t\tcur = self\n+\t\twhile cur:\n+\t\t\tif cur == other: return True\n+\t\t\tcur = cur.parent()\n+\t\treturn False\n+\n+\tdef on_branch (self, branch):\n+\t\t\"\"\"Return True iff this rev is on the given branch.\n+\n+\t\tThe revs considered to be \"on\" a branch X also includes the\n+\t\tbranch point of branch X.\n+\t\t\"\"\"\n+\t\treturn branch == self.branch() or branch.parent() == self\n+\n+\t@classmethod\n+\tdef disjoint (self, a, b):\n+\t\t\"\"\"Return True iff the CVS numbers are historically disjoint\n+\n+\t\tTwo CVS numbers are disjoint if they do not share the same\n+\t\thistorical line back to the initial revision. In other words:\n+\t\tthe two numbers are disjoint if the history (i.e. set of parent\n+\t\trevisions all the way back to the intial (1.1) revision) of\n+\t\tneither number is a superset of the other's history.\n+\t\tSee test_disjoint() for practical examples:\n+\t\t\"\"\"\n+\t\tif a.is_branch(): a = self.from_components(a.l + (1,))\n+\t\tif b.is_branch(): b = self.from_components(b.l + (1,))\n+\t\tif len(a.l) > len(b.l): a, b = b, a # a is now shortest\n+\t\tpairs = zip(a.l, b.l)\n+\t\tfor pa, pb in pairs[:-1]:\n+\t\t\tif pa != pb: return True\n+\t\tif len(a) == len(b): return False\n+\t\tcommon_len = len(a)\n+\t\tif a.l[common_len - 1] <= b.l[common_len - 1]: return False\n+\t\treturn True\n+\n+\t@classmethod\n+\tdef test_disjoint (cls):\n+\t\ttests = [\n+\t\t\t(\"1.2\", \"1.1\", False),\n+\t\t\t(\"1.2\", \"1.2\", False),\n+\t\t\t(\"1.2\", \"1.3\", False),\n+\t\t\t(\"1.2\", \"1.1.2\", True),\n+\t\t\t(\"1.2\", \"1.1.2.3\", True),\n+\t\t\t(\"1.2.4\", \"1.1\", False),\n+\t\t\t(\"1.2.4\", \"1.2\", False),\n+\t\t\t(\"1.2.4\", \"1.3\", True),\n+\t\t\t(\"1.2.4\", \"1.2.2\", True),\n+\t\t\t(\"1.2.4\", \"1.2.4\", False),\n+\t\t\t(\"1.2.4\", \"1.2.6\", True),\n+\t\t\t(\"1.2.4\", \"1.2.2.4\", True),\n+\t\t\t(\"1.2.4\", \"1.2.4.4\", False),\n+\t\t\t(\"1.2.4\", \"1.2.6.4\", True),\n+\t\t\t(\"1.2.4.6\", \"1.1\", False),\n+\t\t\t(\"1.2.4.6\", \"1.2\", False),\n+\t\t\t(\"1.2.4.6\", \"1.3\", True),\n+\t\t\t(\"1.2.4.6\", \"1.2.2\", True),\n+\t\t\t(\"1.2.4.6\", \"1.2.2.1\", True),\n+\t\t\t(\"1.2.4.6\", \"1.2.4\", False),\n+\t\t\t(\"1.2.4.6\", \"1.2.4.5\", False),\n+\t\t\t(\"1.2.4.6\", \"1.2.4.6\", False),\n+\t\t\t(\"1.2.4.6\", \"1.2.4.7\", False),\n+\t\t\t(\"1.2.4.6.8.10\", \"1.2.4.5\", False),\n+\t\t\t(\"1.2.4.6.8.10\", \"1.2.4.6\", False),\n+\t\t\t(\"1.2.4.6.8.10\", \"1.2.4.7\", True),\n+\t\t]\n+\t\tfor a, b, result in tests:\n+\t\t\ta, b = map(cls, (a, b))\n+\t\t\tassert cls.disjoint(a, b) == result, \\\n+\t\t\t\t\"disjoint(%s, %s) is not %s\" % (a, b, result)\n+\t\t\tassert cls.disjoint(b, a) == result, \\\n+\t\t\t\t\"disjoint(%s, %s) is not %s\" % (a, b, result)\n+\n+\t@classmethod\n+\tdef test (cls):\n+\t\tassert cls(\"1.2.4\") == cls(\"1.2.0.4\")\n+\t\tassert cls(\"1.2.4\").is_branch()\n+\t\tassert cls(\"1.2\").is_rev()\n+\t\tassert cls(\"1\").is_branch()\n+\t\tassert cls(\"1.2.4.6\").is_rev()\n+\t\tassert cls(\"1.2.4.6\").enumerate() == [1, 2, 4, 6]\n+\t\tassert cls.from_components([1, 2, 4, 6]) == cls(\"1.2.4.6\")\n+\t\tassert str(cls.from_components([1, 2, 4, 6])) == \"1.2.4.6\"\n+\t\tassert len(cls(\"1.2.4.6\")) == 4\n+\t\tassert cls(\"1.2.4.6\").branch() == cls(\"1.2.4\")\n+\t\tassert cls(\"1.2.4\").branch() == cls(\"1.2.4\")\n+\t\tassert cls(\"1.1\").branch() == cls(\"1\")\n+\t\tassert cls(\"1\").branch() == cls(\"1\")\n+\t\tassert cls(\"1.2.4.6\").parent() == cls(\"1.2.4.5\")\n+\t\tassert cls(\"1.2.4.1\").parent() == cls(\"1.2\")\n+\t\tassert cls(\"1.2\").parent() == cls(\"1.1\")\n+\t\tassert cls(\"1.1\").parent() == None\n+\t\tassert cls(\"2.1\").parent() == None\n+\t\tassert cls(\"1.2.4\").parent() == cls(\"1.2\")\n+\t\tassert cls(\"1\").parent() == None\n+\t\tassert cls(\"2\").parent() == None\n+\t\tassert cls(\"1.2.4.6\").follows(cls(\"1.1\"))\n+\t\tassert cls(\"1.2.4.6\").follows(cls(\"1.2\"))\n+\t\tassert cls(\"1.2.4.6\").follows(cls(\"1.2.4.1\"))\n+\t\tassert cls(\"1.2.4.6\").follows(cls(\"1.2.4.2\"))\n+\t\tassert cls(\"1.2.4.6\").follows(cls(\"1.2.4.3\"))\n+\t\tassert cls(\"1.2.4.6\").follows(cls(\"1.2.4.4\"))\n+\t\tassert cls(\"1.2.4.6\").follows(cls(\"1.2.4.5\"))\n+\t\tassert cls(\"1.2.4.6\").follows(cls(\"1.2.4.6\"))\n+\t\tassert not cls(\"1.2.4.6\").follows(cls(\"1.2.4.7\"))\n+\t\tassert not cls(\"1.2.4.6\").follows(cls(\"1.3\"))\n+\t\tassert not cls(\"1.1\").follows(cls(\"1.2.4.6\"))\n+\n+\t\tcls.test_disjoint()\n+\n+\n+class CvsState (object):\n+\t\"\"\"Encapsulate a historical state in CVS (a set of paths and nums)\n+\n+\tThis class is a container of CVS pathnames and associated CvsNum\n+\tobjects.\n+\n+\tNo communication with a CVS working directory or repository is done in\n+\tthis class, hence only basic sanity checks are performed:\n+\t - A path may only appear once in a CvsState.\n+\t - When adding a path:num pair, path may not already exist in self\n+\t - When replacing a path:num pair, path must already exist in self\n+\t - When removing a path:num pair, both path and num must be given\n+\n+\tIMPORTANT: Objects of this class are hash()able (to support being used\n+\tas keys in a dict), but they are also mutable. It is therefore up to\n+\tthe caller to make sure that the object is not changed after being\n+\tstored in a data structure indexed by its hash value.\n+\t\"\"\"\n+\n+\t__slots__ = ('revs', '_hash')\n+\n+\tdef __init__ (self):\n+\t\tself.revs = {} # dict: path -> CvsNum object\n+\t\tself._hash = None\n+\n+\tdef __iter__ (self):\n+\t\treturn self.revs.iteritems()\n+\n+\tdef __getitem__ (self, path):\n+\t\treturn self.revs[path]\n+\n+\tdef __cmp__ (self, other):\n+\t\treturn cmp(self.revs, other.revs)\n+\n+\tdef __str__ (self):\n+\t\treturn \"\".join([\"%s:%s\\n\" % (p, n) for p, n in sorted(self)])\n+\n+\tdef __hash__ (self):\n+\t\tif self._hash is None:\n+\t\t\tself._hash = hash(str(self))\n+\t\treturn self._hash\n+\n+\tdef paths (self):\n+\t\treturn self.revs.iterkeys()\n+\n+\tdef get (self, path, default = None):\n+\t\treturn self.revs.get(path, default)\n+\n+\tdef add (self, path, revnum):\n+\t\tassert path not in self.revs\n+\t\tself._hash = None\n+\t\tself.revs[path] = revnum\n+\n+\tdef replace (self, path, revnum):\n+\t\tassert path in self.revs\n+\t\tself._hash = None\n+\t\tself.revs[path] = revnum\n+\n+\tdef remove (self, path, revnum):\n+\t\tassert path in self.revs and self.revs[path] == revnum\n+\t\tself._hash = None\n+\t\tdel self.revs[path]\n+\n+\tdef copy (self):\n+\t\t\"\"\"Create and return a copy of this object\"\"\"\n+\t\tret = CvsState()\n+\t\tret.revs = self.revs.copy()\n+\t\tret._hash = self._hash\n+\t\treturn ret\n+\n+\tdef load_data (self, note_data):\n+\t\t\"\"\"Load note data as formatted by self.__str__()\"\"\"\n+\t\tfor line in note_data.split(\"\\n\"):\n+\t\t\tline = line.strip()\n+\t\t\tif not line: continue\n+\t\t\tpath, num = line.rsplit(':', 1)\n+\t\t\tself.add(path, CvsNum(num))\n+\t\tself._hash = hash(note_data)\n+\n+\tdef print_members (self, f = sys.stdout, prefix = \"\"):\n+\t\tfor path, num in sorted(self):\n+\t\t\tprint >>f, \"%s%s:%s\" % (prefix, path, num)\n+\n+\t@file_reader_method(missing_ok = True)\n+\tdef load (self, f):\n+\t\t\"\"\"Load CVS state from the given file name/object\"\"\"\n+\t\tif not f: return\n+\t\tself.load_data(f.read())\n+\n+\t@file_writer_method\n+\tdef save (self, f):\n+\t\t\"\"\"Save CVS state to the given file name/object\"\"\"\n+\t\tassert f\n+\t\tprint >>f, str(self),\n+\n+\n+class CvsDate (object):\n+\t\"\"\"Encapsulate a timestamp, as reported by CVS\n+\n+\tThe internal representation of a timestamp is two integers, the first\n+\trepresenting the timestamp as #seconds since epoch (UTC), and the\n+\tsecond representing the timezone as #minutes offset from UTC.\n+\tE.g.: \"2007-09-05 17:26:28 -0200\" is converted to (1189013188, -120)\n+\t\"\"\"\n+\n+\t__slots__ = ('ts', 'tz')\n+\n+\tdef __init__ (self, date_str = None, in_utc = False):\n+\t\t\"\"\"Convert CVS date string into a CvsDate object\n+\n+\t\tA CVS timestamp string has one of the following forms:\n+\t\t - \"YYYY-MM-DD hh:mm:ss SZZZZ\"\n+\t\t - \"YYYY/MM/DD hh:mm:ss\" (with timezone assumed to be UTC)\n+\t\tThe in_utc parameter determines whether the timestamp part of\n+\t\tthe given string (the \"YYYY-MM-DD hh:mm:ss\" part) is given in\n+\t\tlocal time or UTC (normally CVS dates are given in local time.\n+\t\tIf given in local time, the timezone offset is subtracted from\n+\t\tthe timestamp in order to make the time in UTC format.\n+\t\t\"\"\"\n+\t\tif date_str is None:\n+\t\t\tself.ts, self.tz = 0, 0\n+\t\t\treturn\n+\t\tif date_str == \"now\":\n+\t\t\tself.ts, self.tz = time.time(), 0\n+\t\t\treturn\n+\t\tdate_str = date_str.strip()\n+\t\t# Set up self.ts and self.tz\n+\t\tif date_str.count(\" \") == 2:\n+\t\t\t# Assume format \"YYYY-MM-DD hh:mm:ss SZZZZ\"\n+\t\t\tt, z = date_str.rsplit(\" \", 1)\n+\t\t\t# Convert timestamp to #secs since epoch (UTC)\n+\t\t\tself.ts = timegm(time.strptime(t, \"%Y-%m-%d %H:%M:%S\"))\n+\t\t\t# Convert timezone into #mins offset from UTC\n+\t\t\tself.tz = int(z[1:3]) * 60 + int(z[3:5])\n+\t\t\t# Incorporate timezone sign\n+\t\t\tif z[0] == '-': self.tz *= -1\n+\t\telse:\n+\t\t\tassert date_str.count(\" \") == 1\n+\t\t\t# Assume format \"YYYY/MM/DD hh:mm:ss\"\n+\t\t\tself.ts = timegm(time.strptime(\n+\t\t\t\tdate_str, \"%Y/%m/%d %H:%M:%S\"))\n+\t\t\tself.tz = 0\n+\t\t# Adjust timestamp if not already in UTC\n+\t\tif not in_utc: self.ts -= self.tz * 60\n+\n+\tdef tz_str (self):\n+\t\t\"\"\"Return timezone part of self in string format\"\"\"\n+\t\tsign = '+'\n+\t\tif self.tz < 0: sign = '-'\n+\t\t(hours, minutes) = divmod(abs(self.tz), 60)\n+\t\treturn \"%s%02d%02d\" % (sign, hours, minutes)\n+\n+\tdef __str__ (self):\n+\t\t\"\"\"Reconstruct date string from members\"\"\"\n+\t\ts = time.strftime(\"%Y-%m-%d %H:%M:%S\", time.gmtime(self.ts))\n+\t\treturn \"%s %s\" % (s, self.tz_str())\n+\n+\tdef __repr__ (self):\n+\t\treturn \"CvsDate('%s')\" % (str(self))\n+\n+\tdef __hash__ (self):\n+\t\treturn hash((self.ts, self.tz))\n+\n+\tdef __nonzero__ (self):\n+\t\treturn bool(self.ts or self.tz)\n+\n+\tdef __cmp__ (self, other):\n+\t\treturn cmp(self.ts, other.ts) or cmp(self.tz, other.tz)\n+\n+\tdef __eq__ (self, other):\n+\t\treturn self.ts == other.ts and self.tz == other.tz\n+\n+\tdef diff (self, other):\n+\t\t\"\"\"Return difference between self and other in #seconds\n+\n+\t\tInvariant: self == other.add(self.diff(other))\n+\t\t\"\"\"\n+\t\treturn self.ts - other.ts\n+\n+\t@classmethod\n+\tdef test (cls):\n+\t\ta = cls(\"2009-05-10 14:34:56 +0200\")\n+\t\tb = cls(\"2009/05/10 12:34:56\")\n+\t\tassert a\n+\t\tassert b\n+\t\tassert str(a) == \"2009-05-10 12:34:56 +0200\", str(a)\n+\t\tassert str(b) == \"2009-05-10 12:34:56 +0000\", str(b)\n+\t\tassert a != b\n+\t\tassert a.diff(b) == 0\n+\t\tc = cls(\"2009-05-10 16:34:56 +0200\")\n+\t\tassert c\n+\t\tassert str(c) == \"2009-05-10 14:34:56 +0200\", str(c)\n+\t\tassert c != a\n+\t\tassert c.diff(a) == 2 * 60 * 60\n+\t\tassert a.diff(c) == -2 * 60 * 60\n+\n+\n+class CvsRev (object):\n+\t\"\"\"Encapsulate metadata on a CVS revision\"\"\"\n+\t__slots__ = ('path', 'num', 'date', 'author', 'deleted', 'message')\n+\n+\tdef __init__ (self, path, num):\n+\t\tself.path = path\n+\t\tself.num = num\n+\t\tself.date = None # CvsDate object\n+\t\tself.author = \"\"\n+\t\tself.deleted = None # True or False\n+\t\tself.message = [] # Lines of commit message\n+\n+\tdef __str__ (self):\n+\t\treturn \"<%s:%s on %s by %s%s>\" % (\n+\t\t\tself.path, self.num, self.date, self.author,\n+\t\t\tself.deleted and \", deleted\" or \"\")\n+\n+\tdef __cmp__ (self, other):\n+\t\treturn cmp(self.path, other.path) or cmp(self.num, other.num)\n+\n+\n+class CvsWorkDir (object):\n+\t\"\"\"Encapsulate a CVS working directory\n+\n+\tThis class auto-creates a CVS workdir/checkout in the directory given\n+\tto the constructor, and provides various methods for interacting with\n+\tthis workdir.\n+\t\"\"\"\n+\n+\tdef __init__ (self, workdir, cvs_repo):\n+\t\t\"\"\"Create a new CvsWorkDir\n+\n+\t\tThe cvs_repo argument must be a (cvs_root, cvs_module) tuple\n+\t\t\"\"\"\n+\t\tself.d = workdir\n+\t\tself.cvs_root, self.cvs_module = cvs_repo\n+\t\tparent_dir = os.path.dirname(self.d)\n+\t\tif not os.path.isdir(parent_dir): os.makedirs(parent_dir)\n+\t\tself._valid = None\n+\n+\tdef makepath(self, *args):\n+\t\t\"\"\"Create path relative to working directory\"\"\"\n+\t\treturn os.path.join(self.d, *args)\n+\n+\tdef valid (self):\n+\t\t\"\"\"Return True iff this workdir is present and valid\"\"\"\n+\t\tif self._valid is not None: return True\n+\t\ttry:\n+\t\t\tf = open(self.makepath(\"CVS\", \"Root\"), 'r')\n+\t\t\tassert f.read().strip() == self.cvs_root\n+\t\t\tf.close()\n+\t\t\tf = open(self.makepath(\"CVS\", \"Repository\"), 'r')\n+\t\t\tassert f.read().strip() == self.cvs_module\n+\t\t\tf.close()\n+\t\t\tself._valid = True\n+\t\texcept:\n+\t\t\tself._valid = False\n+\t\treturn self._valid\n+\n+\tdef remove (self):\n+\t\t\"\"\"Remove this checkout\"\"\"\n+\t\tshutil.rmtree(self.d, True)\n+\t\tassert not os.path.exists(self.d)\n+\t\tself._valid = False\n+\n+\tdef checkout (self, revision = \"HEAD\"):\n+\t\t\"\"\"Create a checkout of the given revision\"\"\"\n+\t\tself.remove()\n+\t\tparent_dir, co_dir = os.path.split(self.d)\n+\t\targs = [\"cvs\", \"-f\", \"-Q\", \"-d\", self.cvs_root, \"checkout\"]\n+\t\tif str(revision) != \"HEAD\": args.extend([\"-r\", str(revision)])\n+\t\targs.extend([\"-d\", co_dir, self.cvs_module])\n+\t\texit_code, output, errors = run_command(args, cwd = parent_dir)\n+\t\tif exit_code:\n+\t\t\tdie(\"Failed to checkout CVS working directory\")\n+\t\tassert not errors\n+\t\tassert not output, \"output = '%s'\" % (output)\n+\t\tself._valid = None\n+\t\tassert self.valid()\n+\n+\tdef update (self, revision = \"HEAD\", paths = []):\n+\t\t\"\"\"Update the given paths to the given revision\"\"\"\n+\t\tif not self.valid(): self.checkout()\n+\t\targs = [\"cvs\", \"-f\", \"-Q\", \"update\", \"-kk\"]\n+\t\tif str(revision) == \"HEAD\": args.append(\"-A\")\n+\t\telse: args.extend([\"-r\", str(revision)])\n+\t\targs.extend(paths)\n+\t\texit_code, output, errors = run_command(args, cwd = self.d)\n+\t\tif exit_code:\n+\t\t\tdie(\"Failed to checkout CVS working directory\")\n+\t\tassert not errors\n+\t\tassert not output, \"output = '%s'\" % (output)\n+\n+\tdef get_revision_data (self, path, revision):\n+\t\t\"\"\"Return the contents of the given CVS path:revision\"\"\"\n+\t\tif not self.valid(): self.checkout()\n+\t\targs = [\"cvs\", \"-f\", \"-Q\", \"update\", \"-p\", \"-kk\"]\n+\t\tif str(revision) == \"HEAD\": args.append(\"-A\")\n+\t\telse: args.extend([\"-r\", str(revision)])\n+\t\targs.append(path)\n+\t\texit_code, output, errors = run_command(args, cwd = self.d)\n+\t\tif exit_code:\n+\t\t\tdie(\"Failed to checkout CVS working directory\")\n+\t\tassert not errors\n+\t\treturn output\n+\n+\tdef get_modeinfo (self, paths = []):\n+\t\t\"\"\"Return mode information for the given paths.\n+\n+\t\tReturns a dict of path -> mode number mappings. If paths are\n+\t\tnot specified, mode information for all files in the current\n+\t\tcheckout will be returned. No checkout/update will be done.\n+\t\t\"\"\"\n+\t\tresult = {}\n+\t\tif paths:\n+\t\t\tfor path in paths:\n+\t\t\t\tfullpath = os.path.join(self.d, path)\n+\t\t\t\tmode = 644\n+\t\t\t\tif os.access(fullpath, os.X_OK):\n+\t\t\t\t\tmode = 755\n+\t\t\t\tassert path not in result\n+\t\t\t\tresult[path] = mode\n+\t\telse: # Return mode information for all paths\n+\t\t\tfor dirpath, dirnames, filenames in os.walk(self.d):\n+\t\t\t\t# Don't descend into CVS subdirs\n+\t\t\t\ttry: dirnames.remove('CVS')\n+\t\t\t\texcept ValueError: pass\n+\n+\t\t\t\tassert dirpath.startswith(self.d)\n+\t\t\t\tdirectory = dirpath[len(self.d):].lstrip(\"/\")\n+\n+\t\t\t\tfor fname in filenames:\n+\t\t\t\t\tpath = os.path.join(directory, fname)\n+\t\t\t\t\tfullpath = os.path.join(dirpath, fname)\n+\t\t\t\t\tmode = 644\n+\t\t\t\t\tif os.access(fullpath, os.X_OK):\n+\t\t\t\t\t\tmode = 755\n+\t\t\t\t\tassert path not in result\n+\t\t\t\t\tresult[path] = mode\n+\t\treturn result\n+\n+\t@classmethod\n+\tdef parse_entries (cls, entries, prefix, directory = \"\"):\n+\t\t\"\"\"Recursively parse CVS/Entries files\n+\n+\t\tReturn a dict of CVS paths found by parsing the CVS/Entries\n+\t\tfiles rooted at the given directory.\n+\n+\t\tSee http://ximbiot.com/cvs/manual/feature/cvs_2.html#SEC19 for\n+\t\tinformation on the format of the CVS/Entries file.\n+\t\t\"\"\"\n+\t\tfname = os.path.join(prefix, directory, \"CVS\", \"Entries\")\n+\t\tsubdirs = []\n+\t\tf = open(fname, 'r')\n+\t\tfor line in f:\n+\t\t\tline = line.strip()\n+\t\t\tif line == \"D\": continue # There are no subdirectories\n+\t\t\tt, path, revnum, date, options, tag = line.split(\"/\")\n+\t\t\tif t == \"D\":\n+\t\t\t\tsubdirs.append(path)\n+\t\t\t\tcontinue\n+\t\t\tassert line.startswith(\"/\")\n+\t\t\tpath = os.path.join(directory, path)\n+\t\t\trevnum = CvsNum(revnum)\n+\t\t\tassert path not in entries\n+\t\t\tentries[path] = (revnum, date, options, tag)\n+\t\tf.close()\n+\n+\t\tfor d in subdirs:\n+\t\t\td = os.path.join(directory, d)\n+\t\t\tcls.parse_entries(entries, prefix, d)\n+\n+\tdef get_state (self):\n+\t\t\"\"\"Return CvsState reflecting current state of this checkout\n+\n+\t\tNote that the resulting CvsState will never contain any\n+\t\tdeleted/dead files. Other CvsStates to be compared to the one\n+\t\treturned from here should remove deleted/dead entries first.\n+\t\t\"\"\"\n+\t\tassert self.valid()\n+\t\tentries = {}\n+\t\tresult = CvsState()\n+\t\tself.parse_entries(entries, self.d)\n+\t\tfor path, info in entries.iteritems():\n+\t\t\tresult.add(path, info[0])\n+\t\treturn result\n+\n+\n+class CvsLogParser (object):\n+\t\"\"\"Encapsulate the execution of a \"cvs rlog\" command\"\"\"\n+\n+\tdef __init__ (self, cvs_repo):\n+\t\t\"\"\"Create a new CvsLogParser\n+\n+\t\tThe cvs_repo argument must be a (cvs_root, cvs_module) tuple\n+\t\t\"\"\"\n+\t\tself.cvs_root, self.cvs_module = cvs_repo\n+\n+\tdef cleanup_path (self, cvs_path):\n+\t\t\"\"\"Utility method for parsing CVS paths from CVS log\"\"\"\n+\t\tcvsprefix = \"/\".join((self.cvs_root[self.cvs_root.index(\"/\"):],\n+\t\t\t\tself.cvs_module))\n+\t\tassert cvs_path.startswith(cvsprefix)\n+\t\tassert cvs_path.endswith(\",v\")\n+\t\t# Drop cvsprefix and ,v-extension\n+\t\tcvs_path = cvs_path[len(cvsprefix):-2]\n+\t\t# Split the remaining path into components\n+\t\tcomponents = filter(None, cvs_path.strip().split('/'))\n+\t\t# Remove 'Attic' from CVS paths\n+\t\tif len(components) >= 2 and components[-2] == \"Attic\":\n+\t\t\tdel components[-2]\n+\t\t# Reconstruct resulting \"cleaned\" path\n+\t\treturn \"/\".join(components)\n+\n+\tdef __call__ (self, line):\n+\t\t\"\"\"Line parser; this method is invoked for each line in the log\n+\n+\t\tMust be reimplemented by subclass\n+\t\t\"\"\"\n+\t\tpass\n+\n+\tdef finish (self):\n+\t\t\"\"\"This method is invoked after the last line has been parsed\n+\n+\t\tMay be reimplemented by subclass\n+\t\t\"\"\"\n+\t\tpass\n+\n+\tdef run (self, paths = [], no_symbols = False, revisions = None):\n+\t\targs = [\"cvs\", \"-f\", \"-q\", \"-d\", self.cvs_root, \"rlog\"]\n+\t\tif no_symbols: args.append(\"-N\")\n+\t\tif revisions: args.append(\"-r%s\" % (revisions))\n+\t\tif paths:\n+\t\t\tfor p in paths:\n+\t\t\t\targs.append(\"%s/%s\" % (self.cvs_module, p))\n+\t\telse:\n+\t\t\targs.append(self.cvs_module)\n+\n+\t\tproc = start_command(args)\n+\t\tproc.stdin.close()\n+\t\twhile True:\n+\t\t\tfor line in proc.stdout:\n+\t\t\t\tself(line.rstrip()) # Call subclass line parser\n+\t\t\tif proc.poll() is not None:\n+\t\t\t\tbreak\n+\n+\t\tassert proc.stdout.read() == \"\"\n+\t\tself.finish() # Notify subclass that parsing is finished\n+\t\texit_code = proc.returncode\n+\t\tif exit_code:\n+\t\t\terror(\"'%s' returned exit code %i, and errors:\\n\" \\\n+\t\t\t\t\"---\\n%s---\", \" \".join(args), exit_code,\n+\t\t\t\tproc.stderr.read())\n+\t\treturn exit_code\n+\n+\n+class CvsRevLister (CvsLogParser):\n+\t\"\"\"Extract CvsRev objects (with revision metadata) from a CVS log\"\"\"\n+\n+\tdef __init__ (self, cvs_repo, show_progress = False):\n+\t\tsuper(CvsRevLister, self).__init__(cvs_repo)\n+\t\tself.cur_file = None # current CVS file being processed\n+\t\tself.cur_file_numrevs = 0 # #revs in current CVS file\n+\t\tself.cur_rev = None # Current CvsRev under construction\n+\n+\t\tself.progress = None\n+\t\tif show_progress:\n+\t\t\tself.progress = ProgressIndicator(\"\\t\", sys.stderr)\n+\n+\t\t# Store found revs in a two-level dict structure:\n+\t\t# filename -> revnum -> CvsRev\n+\t\tself.revs = {}\n+\n+\t\t# Possible states:\n+\t\t# - BeforeRevs - waiting for \"total revisions:\"\n+\t\t# - BetweenRevs - waiting for \"----------------------------\"\n+\t\t# - ReadingRev - reading CVS revision details\n+\t\tself.state = 'BeforeRevs'\n+\n+\tdef __call__ (self, line):\n+\t\tif self.state == 'BeforeRevs':\n+\t\t\tif line.startswith(\"RCS file: \"):\n+\t\t\t\tself.cur_file = self.cleanup_path(line[10:])\n+\t\t\t\tassert self.cur_file not in self.revs\n+\t\t\t\tself.revs[self.cur_file] = {}\n+\t\t\telif line.startswith(\"total revisions: \"):\n+\t\t\t\tassert self.cur_file\n+\t\t\t\ttotalrevs, selectedrevs = line.split(\";\")\n+\t\t\t\tself.cur_file_numrevs = \\\n+\t\t\t\t\tint(selectedrevs.split(\":\")[1].strip())\n+\t\t\t\tself.state = 'BetweenRevs'\n+\t\telif self.state == 'BetweenRevs':\n+\t\t\tif line == \"----------------------------\" or \\\n+\t\t\t line == \"======================================\" \\\n+\t\t\t \"=======================================\":\n+\t\t\t\tif self.cur_rev:\n+\t\t\t\t\t# Finished current revision\n+\t\t\t\t\tf = self.revs[self.cur_file]\n+\t\t\t\t\tassert self.cur_rev.num not in f\n+\t\t\t\t\tf[self.cur_rev.num] = self.cur_rev\n+\t\t\t\t\tself.cur_rev = None\n+\t\t\t\t\tif self.progress: self.progress()\n+\t\t\t\tif line == \"----------------------------\":\n+\t\t\t\t\tself.state = 'ReadingRev'\n+\t\t\t\telse:\n+\t\t\t\t\t# Finalize current CVS file\n+\t\t\t\t\tassert len(self.revs[self.cur_file]) \\\n+\t\t\t\t\t\t== self.cur_file_numrevs\n+\t\t\t\t\tself.cur_file = None\n+\t\t\t\t\tself.state = 'BeforeRevs'\n+\t\t\telif self.cur_rev:\n+\t\t\t\t# Currently in the middle of a revision.\n+\n+\t\t\t\tif line.startswith(\"branches: %s\" \\\n+\t\t\t\t % (self.cur_rev.num)) \\\n+\t\t\t\t and line.endswith(\";\"):\n+\t\t\t\t\treturn # Skip 'branches:' lines\n+\t\t\t\t# This line is part of the commit message.\n+\t\t\t\tself.cur_rev.message.append(line)\n+\t\telif self.state == 'ReadingRev':\n+\t\t\tif line.startswith(\"revision \"):\n+\t\t\t\tself.cur_rev = CvsRev(\n+\t\t\t\t\tself.cur_file, CvsNum(line.split()[1]))\n+\t\t\telse:\n+\t\t\t\tdate, author, state, dummy = line.split(\";\", 3)\n+\t\t\t\tassert date.startswith(\"date: \")\n+\t\t\t\tself.cur_rev.date = CvsDate(date[6:])\n+\t\t\t\tassert author.strip().startswith(\"author: \")\n+\t\t\t\tself.cur_rev.author = author.strip()[8:]\n+\t\t\t\tassert state.strip().startswith(\"state: \")\n+\t\t\t\tstate = state.strip()[7:]\n+\t\t\t\tself.cur_rev.deleted = state == \"dead\"\n+\t\t\t\tself.state = 'BetweenRevs'\n+\n+\tdef finish (self):\n+\t\tassert self.state == 'BeforeRevs'\n+\t\tif self.progress:\n+\t\t\tself.progress.finish(\"Parsed %i revs in %i files\" % (\n+\t\t\t\tself.progress.n, len(self.revs)))\n+\n+\n+def fetch_revs (path, from_rev, to_rev, symbol, cvs_repo):\n+\t\"\"\"Fetch CvsRev objects for each rev in CvsRev), where each CvsRev\n+\tencapsulates a CVS revision in the range from path:from_rev to\n+\tpath:symbol (inclusive). If symbol currently refers to from_rev (i.e.\n+\tnothing has happened since the last import), the returned dict will\n+\thave exactly one entry (from_rev). If there is no valid revision range\n+\tbetween from_rev and symbol, the returned dict will be empty.\n+\tSituations in which an empty dict is returned, include:\n+\t- symbol is no longer defined on this path\n+\t- symbol refers to a revision that is disjoint from from_rev\n+\n+\tfrom_rev may be None, meaning that all revisions from the initial\n+\tversion of path up to the revision currently referenced by symbol\n+\tshould be fetched.\n+\n+\tIf the revision currently referenced by symbol is disjoint from\n+\tfrom_rev, the returned dict will be empty.\n+\n+\tNote that there is lots of unexpected behaviour in the handling of the\n+\t'cvs rlog -r' parameter: Say you have a branch, called 'my_branch',\n+\tthat points to branch number 1.1.2 of a file. Say there are 3 revisions\n+\ton this branch: 1.1.2.1 -> 1.1.2.3. (in additions to the branch point\n+\t1.1). Now, observe the following 'cvs rlog' executions:\n+\t- cvs rlog -r0:my_branch ... returns 1.1, 1.1.2.1, 1.1.2.2, 1.1.2.3\n+\t- cvs rlog -r1.1:my_branch ... returns the same revs\n+\t- cvs rlog -rmy_branch ... returns 1.1.2.1, 1.1.2.2, 1.1.2.3\n+\t- cvs rlog -rmy_branch: ... returns the same revs\n+\t- cvs rlog -r:my_branch ... returns the same revs\n+\t- cvs rlog -r::my_branch ... returns the same revs\n+\t- cvs rlog -r1.1.2.1: ... returns the same revs\n+\tHere is where it gets really weird:\n+\t- cvs rlog -r1.1.2.1:my_branch ... returns 1.1.2.1 only\n+\t- cvs rlog -r1.1.2.2:my_branch ... returns 1.1.2.1, 1.1.2.2\n+\t- cvs rlog -r1.1.2.3:my_branch ... returns 1.1.2.1, 1.1.2.2, 1.1.2.3\n+\n+\tIn other words the 'cvs rlog -rfrom_rev:symbol' scheme that we normally\n+\tuse will not work in the case where from_rev is _on_ the branch pointed\n+\tat by the symbol.\n+\n+\tTherefore, we need an extra parameter, to_rev, which we can use to:\n+\t1. Detect when this situation is present.\n+\t2. Work around by using 'cvs rlog -rfrom_ref:to_rev' instead.\n+\t\"\"\"\n+\n+\tif from_rev is None: # Initial import\n+\t\tfrom_rev = \"0\" # cvs rlog -r0:X fetches from initial revision\n+\telif to_rev and to_rev.branch() == from_rev.branch():\n+\t\tsymbol = to_rev # Use to_rev instead of given symbol\n+\n+\t# Run 'cvs rlog' on range [from_rev, symbol] and parse CvsRev objects\n+\tparser = CvsRevLister(cvs_repo)\n+\tparser.run((path,), True, \"%s:%s\" % (from_rev, symbol))\n+\tassert len(parser.revs) == 1\n+\tassert path in parser.revs\n+\treturn parser.revs[path]\n+\n+\n+if __name__ == '__main__':\n+\t# Run selftests\n+\tCvsNum.test()\n+\tCvsDate.test()\ndiff --git a/git_remote_cvs/cvs_revision_map.py b/git_remote_cvs/cvs_revision_map.py\nnew file mode 100644\nindex 0000000..7d7810f\n--- /dev/null\n+++ b/git_remote_cvs/cvs_revision_map.py\n@@ -0,0 +1,362 @@\n+#!/usr/bin/env python\n+\n+\"\"\"Functionality for mapping CVS revisions to associated metainformation\"\"\"\n+\n+from util import *\n+from cvs import CvsNum, CvsDate\n+from git import GitFICommit, GitFastImport, GitObjectFetcher\n+\n+\n+class CvsPathInfo (object):\n+\t\"\"\"Information on a single CVS path\"\"\"\n+\t__slots__ = ('revs', 'mode')\n+\n+\tdef __init__ (self, mode = None):\n+\t\tself.revs = {}\n+\t\tself.mode = mode\n+\n+class CvsRevInfo (object):\n+\t\"\"\"Information on a single CVS revision\"\"\"\n+\t__slots__ = ('blob', 'commits')\n+\n+\tdef __init__ (self, blob):\n+\t\tself.blob = blob\n+\t\tself.commits = []\n+\n+class CvsRevisionMap (object):\n+\t\"\"\"Encapsulate the mapping of CVS revisions to associated metainfo\n+\n+\tThis container maps CVS revisions (a combination of a CVS path and a\n+\tCVS revision number) into Git blob names, Git commit names, and CVS\n+\tpath information. Git object (blob/commit) names are either 40-char\n+\tSHA1 strings, or \"git fast-import\" mark numbers if the form \":\".\n+\n+\tThe data structure is organized as follows:\n+\t- A CvsRevisionMap instance knows about a set of CVS paths.\n+\t For each CVS path, the following is known:\n+\t - The mode (permission bits) of that CVS path (644 or 755)\n+\t - The CVS revision numbers that exist for that CVS path.\n+\t For each revision number, the following is known:\n+\t - Exactly 1 blob name; the Git blob containing the contents of the\n+\t revision (the contents of the CVS path at that CVS revision).\n+\t - One or more commit names; the Git commits which encapsulate a\n+\t CVS state in which this CVS revision\n+\n+\tTo store this data structure persistently, this class uses a Git ref\n+\tthat points to a tree structure containing the above information. When\n+\tchanges to the structure are made, this class will produce\n+\tgit-fast-import commands to update that tree structure accordingly.\n+\n+\tIMPORTANT: No changes to the CvsRevisionMap will be stored unless\n+\t.commit() is called with a valid GitFastImport instance\n+\n+\tNOTES: Mark numbers are only transient references bound to the current\n+\t\"git fast-import\" process (assumed to be running alongside this\n+\tprocess). Therefore, when the \"git fast-import\" process ends, it must\n+\twrite out a mark number -> SHA1 mapping (see the \"--export-marks\"\n+\targument to \"git fast-import\"). Subsequently, this mapping must be\n+\tparsed, and the mark numbers in this CvsRevisionMap must be resolved\n+\tinto persistent SHA1 names. In order to quickly find all the unresolved\n+\tmark number entries in the data structure, and index mapping mark\n+\tnumbers to revisions is kept in a separate file in the tree structure.\n+\tSee the loadMarksFile() method for more details.\n+\t\"\"\"\n+\n+\tMarkNumIndexFile = \"CVS/marks\" # invalid CVS path name\n+\n+\tdef __init__ (self, git_ref, obj_fetcher):\n+\t\tself.git_ref = git_ref\n+\t\tself.obj_fetcher = obj_fetcher\n+\n+\t\t# The following data structure is a cache of the persistent\n+\t\t# data structure found at self.git_ref.\n+\t\t# It is structured as follows:\n+\t\t# - self.d is a dict, mapping CVS paths to CvsPathInfo objects.\n+\t\t# CvsPathInfo object have two fields: revs, mode:\n+\t\t# - mode is either 644 (non-executable) or 755 (executable).\n+\t\t# - revs is a dict, mapping CVS revision numbers (CvsNum\n+\t\t# instances) to CvsRevInfo objects. CvsRevInfo objects have\n+\t\t# two fields: blob, commits:\n+\t\t# - blob is the name of the Git blob object holding the\n+\t\t# contents of that revision.\n+\t\t# - commits is a collection of zero or more Git commit\n+\t\t# names where the commit contains this revision.\n+\t\tself.d = {} # dict: path -> CvsPathInfo\n+\t\tself.mods = set() # (path, revnum) pairs for all modified revs\n+\t\tself.marknum_idx = {} # dict: mark num -> [(path, revnum), ...]\n+\t\tself._load_marknum_idx()\n+\n+\tdef __del__ (self):\n+\t\tif self.mods:\n+\t\t\terror(\"Missing call to self.commit().\")\n+\t\t\terror(\"%i revision changes are lost!\", len(self.mods))\n+\n+\tdef __nonzero__ (self):\n+\t\t\"\"\"Return True iff any information is currently stored here\"\"\"\n+\t\treturn bool(self.d)\n+\n+\t# Private methods:\n+\n+\tdef _add_to_marknum_idx(self, marknum, path, revnum):\n+\t\t\"\"\"Add the given marknum -> (path, revnum) association\"\"\"\n+\t\tentry = self.marknum_idx.setdefault(marknum, [])\n+\t\tentry.append((path, revnum))\n+\n+\tdef _load_marknum_idx(self):\n+\t\t\"\"\"Load contents of MarkNumIndexFile into self.marknum_idx\"\"\"\n+\t\tblobspec = \"%s:%s\" % (self.git_ref, self.MarkNumIndexFile)\n+\t\ttry: f = self.obj_fetcher.open_obj(blobspec)\n+\t\texcept KeyError: return # missing object; nothing to do\n+\n+\t\tfor line in f:\n+\t\t\t# Format of line is \" :\"\n+\t\t\tmark, rest = line.strip().split(' ', 1)\n+\t\t\tpath, rev = rest.rsplit(':', 1)\n+\t\t\tself._add_to_marknum_idx(int(mark), path, CvsNum(rev))\n+\t\tf.close()\n+\n+\tdef _save_marknum_idx(self):\n+\t\t\"\"\"Prepare data for storage into MarkNumIndexFile\n+\n+\t\tThe returned string contains the current contents of\n+\t\tself.marknum_idx, formatted to be stored verbatim in\n+\t\tself.MarkNumIndexFile.\n+\t\t\"\"\"\n+\t\tlines = []\n+\t\tfor marknum, revs in sorted(self.marknum_idx.iteritems()):\n+\t\t\tfor path, revnum in revs:\n+\t\t\t\t# Format of line is \" :\"\n+\t\t\t\tline = \"%i %s:%s\\n\" % (marknum, path, revnum)\n+\t\t\t\tlines.append(line)\n+\t\treturn \"\".join(lines)\n+\n+\tdef _save_rev(self, path, revnum):\n+\t\t\"\"\"Return blob data for storing the given revision persistently\n+\n+\t\tGenerate the blob contents that will reconstitute the same\n+\t\trevision entry when read back in with _fetch_path().\n+\t\t\"\"\"\n+\t\tlines = []\n+\t\trev_info = self.d[path].revs[revnum]\n+\t\tlines.append(\"blob %s\\n\" % (rev_info.blob))\n+\t\tfor commit in rev_info.commits:\n+\t\t\tlines.append(\"commit %s\\n\" % (commit))\n+\t\treturn \"\".join(lines)\n+\n+\t@staticmethod\n+\tdef _valid_objname (objname):\n+\t\t\"\"\"Return the argument as a SHA1 (string) or mark num (int)\"\"\"\n+\t\t# Blob names are either 40-char SHA1 strings, or mark numbers\n+\t\tif isinstance(objname, int) or len(objname) != 40: # mark num\n+\t\t\treturn int(objname)\n+\t\treturn objname\n+\n+\tdef _load (self, path, mode, data):\n+\t\t\"\"\"GitObjectFetcher.walk_tree() callback\"\"\"\n+\t\tassert mode in (644, 755)\n+\t\tcvsPath, revnum = os.path.split(path)\n+\t\trevnum = CvsNum(revnum)\n+\t\tif cvsPath in self.d:\n+\t\t\tassert mode == self.d[cvsPath].mode\n+\t\telse:\n+\t\t\tself.d[cvsPath] = CvsPathInfo(mode)\n+\t\tassert revnum not in self.d[cvsPath].revs\n+\t\trev_info = None\n+\t\tfor line in data.split(\"\\n\"):\n+\t\t\tif not line: continue\n+\t\t\tt, objname = line.split()\n+\t\t\tobjname = self._valid_objname(objname)\n+\t\t\tif t == \"blob\":\n+\t\t\t\tassert rev_info is None\n+\t\t\t\trev_info = CvsRevInfo(objname)\n+\t\t\telif t == \"commit\":\n+\t\t\t\tassert rev_info is not None\n+\t\t\t\trev_info.commits.append(objname)\n+\t\t\telse:\n+\t\t\t\tassert False, \"Unknown type '%s'\" % (t)\n+\t\tassert rev_info.commits # each rev is in at least one commit\n+\t\tself.d[cvsPath].revs[revnum] = rev_info\n+\n+\tdef _fetch_path (self, path):\n+\t\t\"\"\"If the given path exists, create a path entry in self.d\"\"\"\n+\t\ttree_spec = \"%s:%s\" % (self.git_ref, path)\n+\t\tself.obj_fetcher.walk_tree(tree_spec, self._load, path) # *** Don't load entire tree at once?\n+\n+\t# Public methods:\n+\n+\tdef has_path (self, path):\n+\t\t\"\"\"Return True iff the given path is present\"\"\"\n+\t\tif path not in self.d: self._fetch_path(path)\n+\t\treturn path in self.d\n+\n+\tdef has_rev (self, path, revnum):\n+\t\t\"\"\"Return True iff the given path:revnum is present\"\"\"\n+\t\tif path not in self.d: self._fetch_path(path)\n+\t\treturn path in self.d and revnum in self.d[path].revs\n+\n+\tdef get_mode (self, path):\n+\t\t\"\"\"Return mode bits for the given path\"\"\"\n+\t\tif path not in self.d: self._fetch_path(path)\n+\t\treturn self.d[path].mode\n+\n+\tdef get_blob (self, path, revnum):\n+\t\t\"\"\"Return the blob name for the given revision\"\"\"\n+\t\tif path not in self.d: self._fetch_path(path)\n+\t\treturn self.d[path].revs[revnum].blob\n+\n+\tdef get_commits (self, path, revnum):\n+\t\t\"\"\"Return the commit names containing the given revision\"\"\"\n+\t\tif path not in self.d: self._fetch_path(path)\n+\t\treturn self.d[path].revs[revnum].commits\n+\n+\tdef has_unresolved_marks (self):\n+\t\t\"\"\"Return True iff there are mark nums in the data structure\"\"\"\n+\t\treturn self.marknum_idx\n+\n+\t# Public methods that change/add information:\n+\n+\tdef add_path (self, path, mode):\n+\t\t\"\"\"Add the given path and associated mode bits to this map\"\"\"\n+\t\tif path not in self.d: self._fetch_path(path)\n+\t\tif path in self.d:\n+\t\t\tif self.d[path].mode:\n+\t\t\t\tassert mode == self.d[path].mode, \\\n+\t\t\t\t\t\"The mode of %s has changed from %s \" \\\n+\t\t\t\t\t\"to %s since the last import. This \" \\\n+\t\t\t\t\t\"is not supported.\" % (\n+\t\t\t\t\t\tpath, self.d[path].mode, mode)\n+\t\t\telse:\n+\t\t\t\tself.d[path].mode = mode\n+\t\telse:\n+\t\t\tself.d[path] = CvsPathInfo(mode)\n+\t\t# Do not add to self.mods yet, as we expect revisions to be\n+\t\t# added before commit() is called\n+\n+\tdef add_blob (self, path, revnum, blobname):\n+\t\t\"\"\"Add the given path:revnum -> blobname association\"\"\"\n+\t\tassert blobname\n+\t\tif path not in self.d: self._fetch_path(path)\n+\t\tblobname = self._valid_objname(blobname)\n+\t\tif isinstance(blobname, int): # mark num\n+\t\t\tself._add_to_marknum_idx(blobname, path, revnum)\n+\t\tentry = self.d.setdefault(path, CvsPathInfo())\n+\t\tassert revnum not in entry.revs\n+\t\tentry.revs[revnum] = CvsRevInfo(blobname)\n+\t\tself.mods.add((path, revnum))\n+\n+\tdef add_commit (self, path, revnum, commitname):\n+\t\t\"\"\"Add the given path:revnum -> commitname association\"\"\"\n+\t\tif path not in self.d: self._fetch_path(path)\n+\t\tcommitname = self._valid_objname(commitname)\n+\t\tif isinstance(commitname, int): # mark num\n+\t\t\tself._add_to_marknum_idx(commitname, path, revnum)\n+\t\tassert revnum in self.d[path].revs\n+\t\tassert commitname not in self.d[path].revs[revnum].commits\n+\t\tself.d[path].revs[revnum].commits.append(commitname)\n+\t\tself.mods.add((path, revnum))\n+\n+\t@file_reader_method(missing_ok = True)\n+\tdef load_marks_file (self, f):\n+\t\t\"\"\"Load mark -> SHA1 mappings from git-fast-import marks file\n+\n+\t\tReplace all mark numbers with proper SHA1 names in this data\n+\t\tstructure (using self.marknum_idx to find them quickly).\n+\t\t\"\"\"\n+\t\tif not f: return 0\n+\t\tmarks = {}\n+\t\tlast_mark = 0\n+\t\tfor line in f:\n+\t\t\t(mark, sha1) = line.strip().split()\n+\t\t\tassert mark.startswith(\":\")\n+\t\t\tmark = int(mark[1:])\n+\t\t\tassert mark not in marks\n+\t\t\tmarks[mark] = sha1\n+\t\t\tif mark > last_mark: last_mark = mark\n+\t\tfor marknum, revs in self.marknum_idx.iteritems():\n+\t\t\tsha1 = marks[marknum]\n+\t\t\tfor path, revnum in revs:\n+\t\t\t\tif path not in self.d: self._fetch_path(path)\n+\t\t\t\trev_info = self.d[path].revs[revnum]\n+\t\t\t\tif rev_info.blob == marknum: # replace blobname\n+\t\t\t\t\trev_info.blob = sha1\n+\t\t\t\telse: # replace commitname\n+\t\t\t\t\tassert marknum in rev_info.commits\n+\t\t\t\t\ti = rev_info.commits.index(marknum)\n+\t\t\t\t\trev_info.commits[i] = sha1\n+\t\t\t\t\tassert marknum not in rev_info.commits\n+\t\t\t\tself.mods.add((path, revnum))\n+\t\tself.marknum_idx = {} # resolved all transient mark numbers\n+\t\treturn last_mark\n+\n+\tdef sync_modeinfo_from_cvs (self, cvs):\n+\t\t\"\"\"Update with mode information from current CVS checkout\n+\n+\t\tThis method will retrieve mode information on all paths in the\n+\t\tcurrent CVS checkout, and update this data structure\n+\t\tcorrespondingly. In the case where mode information is already\n+\t\tpresent for a given CVS path, this method will verify that such\n+\t\tinformation is correct.\n+\t\t\"\"\"\n+\t\tfor path, mode in cvs.get_modeinfo().iteritems():\n+\t\t\tself.add_path(path, mode)\n+\n+\tdef commit_map (self, gfi, author, message):\n+\t\t\"\"\"Produce git-fast-import commands for storing changes\n+\n+\t\tThe given GitFastImport object is used to produce a commit\n+\t\tmaking the changes done to this data structure persistent.\n+\t\t\"\"\"\n+\t\tnow = CvsDate(\"now\")\n+\t\tcommitdata = GitFICommit(\n+\t\t\tauthor[0], author[1], now.ts, now.tz_str(), message)\n+\n+\t\t# Add updated MarkNumIndexFile to commit\n+\t\tmark = gfi.blob(self._save_marknum_idx())\n+\t\tcommitdata.modify(644, mark, self.MarkNumIndexFile)\n+\n+\t\tfor path, revnum in self.mods:\n+\t\t\tmark = gfi.blob(self._save_rev(path, revnum))\n+\t\t\tmode = self.d[path].mode\n+\t\t\tassert mode in (644, 755)\n+\t\t\tcommitdata.modify(mode, mark, \"%s/%s\" % (path, revnum))\n+\n+\t\tgfi.commit(self.git_ref, commitdata)\n+\t\tself.mods = set()\n+\n+\n+class CvsStateMap (object):\n+\t\"\"\"Map CvsState object to the commit names which produces that state\"\"\"\n+\tdef __init__ (self, cvs_rev_map):\n+\t\tself.cvs_rev_map = cvs_rev_map\n+\n+\tdef get_commits (self, state):\n+\t\t\"\"\"Map the given CvsState to commits that contain this state\n+\n+\t\tReturn all commits where the given state is a subset of the\n+\t\tstate produced by that commit.\n+\n+\t\tReturns a set of commit names. The set may be empty.\n+\t\t\"\"\"\n+\t\tcandidate_commits = None\n+\t\tfor path, revnum in state:\n+\t\t\tcommits = self.cvs_rev_map.get_commits(path, revnum)\n+\t\t\tif candidate_commits is None:\n+\t\t\t\tcandidate_commits = set(commits)\n+\t\t\telse:\n+\t\t\t\tcandidate_commits.intersection_update(commits)\n+\t\treturn candidate_commits\n+\n+\tdef get_exact_commit (self, state, commit_map):\n+\t\t\"\"\"Map the given CvsState to the commit with this exact state\n+\n+\t\tThe given commit_map must be a CommitStates object.\n+\n+\t\tReturn the only commit (if any) that produces the exact given\n+\t\tCvsState.\n+\n+\t\tReturns a commit name, or None if no matching commit is found.\n+\t\t\"\"\"\n+\t\tcommits = self.get_commits(state)\n+\t\tfor c in commits:\n+\t\t\tif state == commit_map.get(c): return c\n+\t\treturn None\ndiff --git a/git_remote_cvs/cvs_symbol_cache.py b/git_remote_cvs/cvs_symbol_cache.py\nnew file mode 100644\nindex 0000000..a8ec18a\n--- /dev/null\n+++ b/git_remote_cvs/cvs_symbol_cache.py\n@@ -0,0 +1,283 @@\n+#!/usr/bin/env python\n+\n+\"\"\"Functionality for interacting with the local CVS symbol cache\"\"\"\n+\n+import os\n+\n+from util import *\n+from cvs import *\n+\n+\n+class CvsSymbolStateLister (CvsLogParser):\n+\t\"\"\"Extract current CvsStates for all CVS symbols from a CVS log\"\"\"\n+\n+\tdef __init__ (self, cvs_repo, show_progress = False):\n+\t\tsuper(CvsSymbolStateLister, self).__init__(cvs_repo)\n+\t\tself.symbols = {} # CVS symbol name -> CvsState object\n+\n+\t\tself.cur_file = None # current CVS file being processed\n+\t\tself.cur_file_numrevs = 0 # #revs in current CVS file\n+\t\tself.cur_revnum = None # current revision number\n+\t\tself.rev2syms = {} # CvsNum -> [CVS symbol names]\n+\t\tself.cur_revs = {} # CvsNum -> True/False (deleted)\n+\t\tself.head_num = None # CvsNum of the HEAD rev or branch\n+\n+\t\t# Possible states:\n+\t\t# - BeforeSymbols - waiting for \"symbolic names:\"\n+\t\t# - WithinSymbols - reading CVS symbol names\n+\t\t# - BeforeRevs - waiting for \"total revisions:\"\n+\t\t# - BetweenRevs - waiting for \"----------------------------\"\n+\t\t# - ReadingRev - reading CVS revision details\n+\t\tself.state = 'BeforeSymbols'\n+\n+\t\tself.progress = None\n+\t\tif show_progress:\n+\t\t\tself.progress = ProgressIndicator(\"\\t\", sys.stderr)\n+\n+\tdef finalize_symbol_states (self):\n+\t\t\"\"\"Adjust CvsStates in self.symbols based on revision data\n+\n+\t\tBased on the information found in self.rev2syms and\n+\t\tself.cur_revs, remove deleted revisions and turn branch numbers\n+\t\tinto corresponding revisions in the CvsStates found in\n+\t\tself.symbols.\n+\t\t\"\"\"\n+\t\t# Create a mapping from branch numbers to the last existing\n+\t\t# revision number on those branches\n+\t\tbranch2lastrev = {} # branch number -> revision number\n+\t\tfor revnum in self.cur_revs.iterkeys():\n+\t\t\tbranchnum = revnum.branch()\n+\t\t\tif (branchnum not in branch2lastrev) or \\\n+\t\t\t (revnum > branch2lastrev[branchnum]):\n+\t\t\t\tbranch2lastrev[branchnum] = revnum\n+\n+\t\tfor cvsnum, symbols in self.rev2syms.iteritems():\n+\t\t\tif cvsnum.is_branch():\n+\t\t\t\t# Turn into corresponding revision number\n+\t\t\t\trevnum = branch2lastrev.get(\n+\t\t\t\t\tcvsnum, cvsnum.parent())\n+\t\t\t\tfor s in symbols:\n+\t\t\t\t\tstate = self.symbols[s]\n+\t\t\t\t\tassert state[self.cur_file] == cvsnum\n+\t\t\t\t\tstate.replace(self.cur_file, revnum)\n+\t\t\t\tcvsnum = revnum\n+\t\t\tassert cvsnum.is_rev()\n+\t\t\tassert cvsnum in self.cur_revs\n+\t\t\tif self.cur_revs[cvsnum]: # cvsnum is a deleted rev\n+\t\t\t\t# Remove from CvsStates\n+\t\t\t\tfor s in symbols:\n+\t\t\t\t\tstate = self.symbols[s]\n+\t\t\t\t\tstate.remove(self.cur_file, cvsnum)\n+\n+\t\tself.rev2syms = {}\n+\t\tself.cur_revs= {}\n+\t\tself.cur_file = None\n+\n+\tdef __call__ (self, line):\n+\t\tif self.state == 'BeforeSymbols':\n+\t\t\tif line.startswith(\"RCS file: \"):\n+\t\t\t\tself.cur_file = self.cleanup_path(line[10:])\n+\t\t\t\tif self.progress:\n+\t\t\t\t\tself.progress(\"%5i symbols found - \" \\\n+\t\t\t\t\t\t\"Parsing CVS file #%i: %s \" \\\n+\t\t\t\t\t\t% (\n+\t\t\t\t\t\t\tlen(self.symbols),\n+\t\t\t\t\t\t\tself.progress.n,\n+\t\t\t\t\t\t\tself.cur_file,\n+\t\t\t\t\t\t))\n+\t\t\tif line.startswith(\"head: \"):\n+\t\t\t\tself.head_num = CvsNum(line[6:])\n+\t\t\tif line.startswith(\"branch: \"):\n+\t\t\t\tself.head_num = CvsNum(line[8:])\n+\t\t\telif line == \"symbolic names:\":\n+\t\t\t\tassert self.head_num\n+\t\t\t\ts = self.symbols.setdefault(\"HEAD\", CvsState())\n+\t\t\t\ts.add(self.cur_file, self.head_num)\n+\t\t\t\tr = self.rev2syms.setdefault(self.head_num, [])\n+\t\t\t\tr.append(\"HEAD\")\n+\t\t\t\tself.head_num = None\n+\t\t\t\tself.state = 'WithinSymbols'\n+\t\telif self.state == 'WithinSymbols':\n+\t\t\tif line.startswith(\"\\t\"):\n+\t\t\t\tsymbol, cvsnum = line.split(\":\", 1)\n+\t\t\t\tsymbol = symbol.strip()\n+\t\t\t\tcvsnum = CvsNum(cvsnum)\n+\t\t\t\ts = self.symbols.setdefault(symbol, CvsState())\n+\t\t\t\ts.add(self.cur_file, cvsnum)\n+\t\t\t\tr = self.rev2syms.setdefault(cvsnum, [])\n+\t\t\t\tr.append(symbol)\n+\t\t\telse:\n+\t\t\t\tself.state = 'BeforeRevs'\n+\t\telif self.state == 'BeforeRevs':\n+\t\t\tif line.startswith(\"total revisions: \"):\n+\t\t\t\tassert self.cur_file\n+\t\t\t\ttotalrevs, selectedrevs = line.split(\";\")\n+\t\t\t\tself.cur_file_numrevs = \\\n+\t\t\t\t\tint(selectedrevs.split(\":\")[1].strip())\n+\t\t\t\tself.state = 'BetweenRevs'\n+\t\telif self.state == 'BetweenRevs':\n+\t\t\tif line == \"----------------------------\" or \\\n+\t\t\t line == \"======================================\" \\\n+\t\t\t \"=======================================\":\n+\t\t\t\tif self.cur_revnum:\n+\t\t\t\t\tassert self.cur_revnum in self.cur_revs\n+\t\t\t\t\tself.cur_revnum = None\n+\t\t\t\tif line == \"----------------------------\":\n+\t\t\t\t\tself.state = 'ReadingRev'\n+\t\t\t\telse:\n+\t\t\t\t\t# Finalize current CVS file\n+\t\t\t\t\tassert len(self.cur_revs) \\\n+\t\t\t\t\t\t== self.cur_file_numrevs\n+\t\t\t\t\tself.finalize_symbol_states()\n+\t\t\t\t\tself.state = 'BeforeSymbols'\n+\t\telif self.state == 'ReadingRev':\n+\t\t\tif line.startswith(\"revision \"):\n+\t\t\t\tself.cur_revnum = CvsNum(line.split()[1])\n+\t\t\telse:\n+\t\t\t\tdate, author, state, dummy = line.split(\";\", 3)\n+\t\t\t\tassert date.startswith(\"date: \")\n+\t\t\t\tassert author.strip().startswith(\"author: \")\n+\t\t\t\tassert state.strip().startswith(\"state: \")\n+\t\t\t\tstate = state.strip()[7:]\n+\t\t\t\tassert self.cur_revnum not in self.cur_revs\n+\t\t\t\tdeleted = state == \"dead\"\n+\t\t\t\tself.cur_revs[self.cur_revnum] = deleted\n+\t\t\t\tself.state = 'BetweenRevs'\n+\n+\tdef finish (self):\n+\t\tassert self.state == 'BeforeSymbols'\n+\t\tif self.progress:\n+\t\t\tself.progress.finish(\"Parsed %i symbols in %i files\" \\\n+\t\t\t\t% (len(self.symbols), self.progress.n))\n+\n+\n+class CvsSymbolCache (object):\n+\t\"\"\"Local cache of the current CvsState of CVS symbols\n+\n+\tBehaves like a dictionary of CVS symbol -> CvsState mappings.\n+\t\"\"\"\n+\n+\tdef __init__ (self, symbols_dir):\n+\t\tself.symbols_dir = symbols_dir\n+\t\tif not os.path.isdir(self.symbols_dir):\n+\t\t\tos.makedirs(self.symbols_dir)\n+\n+\tdef __len__ (self):\n+\t\treturn len(os.listdir(self.symbols_dir))\n+\n+\tdef __iter__ (self):\n+\t\tfor filename in os.listdir(self.symbols_dir):\n+\t\t\tyield filename\n+\n+\tdef items (self):\n+\t\tfor filename in self:\n+\t\t\tyield (filename, self[filename])\n+\n+\tdef __contains__ (self, symbol):\n+\t\t\"\"\"Return True if the given symbol is present in this cache\"\"\"\n+\t\treturn os.access(os.path.join(self.symbols_dir, symbol),\n+\t\t os.F_OK | os.R_OK)\n+\n+\tdef __getitem__ (self, symbol):\n+\t\t\"\"\"Return the cached CvsState of the given CVS symbol\"\"\"\n+\t\ttry:\n+\t\t\tf = open(os.path.join(self.symbols_dir, symbol), 'r')\n+\t\texcept IOError:\n+\t\t\traise KeyError, \"'%s'\" % (symbol)\n+\t\tret = CvsState()\n+\t\tret.load(f)\n+\t\tf.close()\n+\t\treturn ret\n+\n+\tdef __setitem__ (self, symbol, cvs_state):\n+\t\t\"\"\"Store the given CVS symbol and CvsState into the cache\"\"\"\n+\t\tcvs_state.save(os.path.join(self.symbols_dir, symbol))\n+\n+\tdef __delitem__ (self, symbol):\n+\t\t\"\"\"Remove the the given CVS symbol from the cache\"\"\"\n+\t\tos.remove(os.path.join(self.symbols_dir, symbol))\n+\n+\tdef get (self, symbol, default = None):\n+\t\ttry:\n+\t\t\treturn self[symbol]\n+\t\texcept KeyError:\n+\t\t\treturn default\n+\n+\tdef clear (self):\n+\t\t\"\"\"Remove all entries from the symbol cache\"\"\"\n+\t\tfor filename in os.listdir(self.symbols_dir):\n+\t\t\tos.remove(os.path.join(self.symbols_dir, filename))\n+\n+\tdef sync_symbol (self, symbol, cvs, progress):\n+\t\t\"\"\"Synchronize the given CVS symbol with the CVS server\n+\n+\t\tThe given CVS workdir is used for the synchronization.\n+\t\tThe retrieved CvsState is also returned\n+\t\t\"\"\"\n+\t\tprogress(\"Retrieving state of CVS symbol '%s'...\" % (symbol))\n+\t\tcvs.update(symbol)\n+\t\tstate = cvs.get_state()\n+\n+\t\tprogress(\"Saving state of '%s' to symbol cache...\" % (symbol))\n+\t\tself[symbol] = state\n+\n+\tdef sync_all_symbols (self, cvs_repo, progress, symbol_filter = None):\n+\t\t\"\"\"Synchronize this symbol cache with the CVS server\n+\n+\t\tThis may be very expensive if the CVS repository is large, or\n+\t\thas many symbols. After this method returns, the symbol cache\n+\t\twill be in sync with the current state on the server.\n+\n+\t\tThis method returns a dict with the keys 'unchanged',\n+\t\t'changed', 'added', and 'deleted', where each map to a list of\n+\t\tCVS symbols. Each CVS symbol appears in exactly one of these\n+\t\tlists.\n+\n+\t\tIf symbol_filter is given, it specifies functions that takes\n+\t\tone parameter - a CVS symbol name - and returns True if that\n+\t\tsymbol should be synchronized, and False if that symbol should\n+\t\tbe skipped. Otherwise all CVS symbols are synchronized.\n+\t\t\"\"\"\n+\t\tif symbol_filter is None: symbol_filter = lambda symbol: True\n+\n+\t\t# 1. Run cvs rlog to fetch current CvsState for all CVS symbols\n+\t\tprogress(\"Retrieving current state of all CVS symbols from \" \\\n+\t\t \"CVS server...\", lf = True)\n+\t\tparser = CvsSymbolStateLister(cvs_repo, True)\n+\t\tretcode = parser.run()\n+\t\tif retcode:\n+\t\t\traise EnvironmentError, (retcode,\n+\t\t\t\t\"cvs rlog returned exit code %i\" % (retcode))\n+\n+\t\t# 2. Update symbol cache with new states from the CVS server\n+\t\tprogress(\"Updating symbol cache with current CVS state...\")\n+\t\tresults = {}\n+\t\tresult_keys = (\"unchanged\", \"changed\", \"added\", \"deleted\")\n+\t\tfor k in result_keys: results[k] = []\n+\t\t# Classify existing symbols as unchanged, changed, or deleted\n+\t\tfor symbol in filter(symbol_filter, self):\n+\t\t\tif symbol not in parser.symbols: # deleted\n+\t\t\t\tresults[\"deleted\"].append(symbol)\n+\t\t\t\tdel self[symbol]\n+\t\t\telif self[symbol] != parser.symbols[symbol]: # changed\n+\t\t\t\tresults[\"changed\"].append(symbol)\n+\t\t\t\tself[symbol] = parser.symbols[symbol]\n+\t\t\telse: # unchanged\n+\t\t\t\tresults[\"unchanged\"].append(symbol)\n+\t\t\tprogress()\n+\t\t# Add symbols that are not in self\n+\t\tfor symbol, state in parser.symbols.iteritems():\n+\t\t\tif not symbol_filter(symbol):\n+\t\t\t\tdebug(\"Skipping CVS symbol '%s'...\", symbol)\n+\t\t\telif symbol in self:\n+\t\t\t\tassert state == self[symbol]\n+\t\t\telse: # added\n+\t\t\t\tresults[\"added\"].append(symbol)\n+\t\t\t\tself[symbol] = state\n+\t\t\tprogress()\n+\n+\t\tprogress(\"Synchronized local symbol cache (%s)\" % (\n+\t\t\t\", \".join([\"%i %s\" % (len(results[k]), k) \\\n+\t\t\t\tfor k in result_keys])), True)\n+\n+\t\treturn results\ndiff --git a/git_remote_cvs/git.py b/git_remote_cvs/git.py\nnew file mode 100644\nindex 0000000..0696962\n--- /dev/null\n+++ b/git_remote_cvs/git.py\n@@ -0,0 +1,586 @@\n+#!/usr/bin/env python\n+\n+\"\"\"Functionality for interacting with Git repositories\n+\n+This module provides classes for interfacing with a Git repository.\n+\"\"\"\n+\n+import os, re\n+from binascii import hexlify\n+from cStringIO import StringIO\n+\n+from util import *\n+\n+def get_git_dir ():\n+\t\"\"\"Return the path to the GIT_DIR for this repo\"\"\"\n+\targs = (\"git\", \"rev-parse\", \"--git-dir\")\n+\texit_code, output, errors = run_command(args)\n+\tif exit_code:\n+\t\tdie(\"Failed to retrieve git dir\")\n+\tassert not errors\n+\treturn output.strip()\n+\n+def parse_git_config ():\n+\t\"\"\"Return a dict containing the parsed version of 'git config -l'\"\"\"\n+\texit_code, output, errors = run_command((\"git\", \"config\", \"-z\", \"-l\"))\n+\tif exit_code:\n+\t\tdie(\"Failed to retrieve git configuration\")\n+\tassert not errors\n+\treturn dict([e.split('\\n', 1) for e in output.split(\"\\0\") if e])\n+\n+def git_config_bool (value):\n+\t\"\"\"Convert the given git config string value to either True or False\n+\n+\tRaise ValueError if the given string was not recognized as a boolean\n+\tvalue.\n+\t\"\"\"\n+\tnorm_value = str(value).strip().lower()\n+\tif norm_value in (\"true\", \"1\", \"yes\", \"on\", \"\"): return True\n+\tif norm_value in (\"false\", \"0\", \"no\", \"off\", \"none\"): return False\n+\traise ValueError, \"Failed to parse '%s' into a boolean value\" % (value)\n+\n+def valid_git_ref (ref_name):\n+\t\"\"\"Return True iff the given ref name is a valid git ref name\"\"\"\n+\t# The following is a reimplementation of the git check-ref-format\n+\t# command. The rules were derived from the git check-ref-format(1)\n+\t# manual page. This code should be replaced by a call to\n+\t# check_ref_format() in the git library, when such is available.\n+\tif ref_name.endswith('/') or \\\n+\t ref_name.startswith('.') or \\\n+\t ref_name.count('/.') or \\\n+\t ref_name.count('..') or \\\n+\t ref_name.endswith('.lock'):\n+\t\treturn False\n+\tfor c in ref_name:\n+\t\tif ord(c) < 0x20 or ord(c) == 0x7f or c in \" ~^:?*[\":\n+\t\t\treturn False\n+\treturn True\n+\n+\n+class GitObjectFetcher (object):\n+\t\"\"\"Provide parsed access to 'git cat-file --batch'\"\"\"\n+\tdef __init__ (self):\n+\t\t\"\"\"Initiate a 'git cat-file --batch' session\"\"\"\n+\t\tself.queue = [] # list of object names to be submitted\n+\t\tself.in_transit = None # object name currently in transit\n+\n+\t\t# 'git cat-file --batch' produces binary output which is likely\n+\t\t# to be corrupted by the default \"rU\"-mode pipe opened by\n+\t\t# start_command. (Mode == \"rU\" does universal new-line\n+\t\t# conversion, which mangles carriage returns.) Therefore, we\n+\t\t# open an explicitly binary-safe pipe for transferring the\n+\t\t# output from 'git cat-file --batch'.\n+\t\tpipe_r_fd, pipe_w_fd = os.pipe()\n+\t\tpipe_r = os.fdopen(pipe_r_fd, \"rb\")\n+\t\tpipe_w = os.fdopen(pipe_w_fd, \"wb\")\n+\t\tself.proc = start_command(\n+\t\t\t(\"git\", \"cat-file\", \"--batch\"), stdout = pipe_w)\n+\t\tself.f = pipe_r\n+\n+\tdef __del__ (self):\n+\t\tassert not self.queue\n+\t\tassert self.in_transit is None\n+\t\tself.proc.stdin.close()\n+\t\tassert self.proc.wait() == 0 # zero exit code\n+\t\tassert self.f.read() == \"\" # no remaining output\n+\n+\tdef _submit_next_object (self):\n+\t\t\"\"\"Submit queue items to the 'git cat-file --batch' process\n+\n+\t\tIf there are items in the queue, and there is currently no item\n+\t\tcurrently in 'transit', then pop the first item off the queue,\n+\t\tand submit it.\n+\t\t\"\"\"\n+\t\tif self.queue and self.in_transit is None:\n+\t\t\tself.in_transit = self.queue.pop(0)\n+\t\t\tprint >>self.proc.stdin, self.in_transit[0]\n+\n+\tdef push (self, obj, callback):\n+\t\t\"\"\"Push the given object name onto the queue\n+\n+\t\tThe given callback function will at some point in the future\n+\t\tbe called exactly once with the following arguments:\n+\t\t- self (this GitObjectFetcher instance)\n+\t\t- obj (the object name provided to push())\n+\t\t- sha1 (the SHA1 of the object, if 'None' obj is missing)\n+\t\t- t (the type of the object (tag/commit/tree/blob))\n+\t\t- size (the size of the object in bytes)\n+\t\t- data (the object contents)\n+\t\t\"\"\"\n+\t\tself.queue.append((obj, callback))\n+\t\tself._submit_next_object() # (re)start queue processing\n+\n+\tdef process_next_entry (self):\n+\t\t\"\"\"Read the next entry off the queue and invoke callback\"\"\"\n+\t\tobj, cb = self.in_transit\n+\t\tself.in_transit = None\n+\t\theader = self.f.readline()\n+\t\tif header == \"%s missing\\n\" % (obj):\n+\t\t\tcb(self, obj, None, None, None, None)\n+\t\t\treturn\n+\t\tsha1, t, size = header.split(\" \")\n+\t\tassert len(sha1) == 40\n+\t\tassert t in (\"tag\", \"commit\", \"tree\", \"blob\")\n+\t\tassert size.endswith(\"\\n\")\n+\t\tsize = int(size.strip())\n+\t\tdata = self.f.read(size)\n+\t\tassert self.f.read(1) == \"\\n\"\n+\t\tcb(self, obj, sha1, t, size, data)\n+\t\tself._submit_next_object()\n+\n+\tdef process (self):\n+\t\t\"\"\"Process the current queue until empty\"\"\"\n+\t\twhile self.in_transit is not None:\n+\t\t\tself.process_next_entry()\n+\n+\t# High-level convenience methods:\n+\n+\tdef get_sha1 (self, objspec):\n+\t\t\"\"\"Return the SHA1 of the object specified by 'objspec'\n+\n+\t\tReturn None if 'objspec' does not specify an existing object.\n+\t\t\"\"\"\n+\t\tclass ObjHandler (object):\n+\t\t\tdef __init__ (self, parser):\n+\t\t\t\tself.parser = parser\n+\t\t\t\tself.sha1 = None\n+\n+\t\t\tdef __call__ (self, parser, obj, sha1, t, size, data):\n+\t\t\t\tassert parser == self.parser\n+\t\t\t\tself.sha1 = sha1\n+\n+\t\thandler = ObjHandler(self)\n+\t\tself.push(objspec, handler)\n+\t\tself.process()\n+\t\treturn handler.sha1\n+\n+\tdef open_obj (self, objspec):\n+\t\t\"\"\"Return a file object wrapping the contents of a named object\n+\n+\t\tThe caller is responsible for calling .close() on the returned\n+\t\tfile object.\n+\n+\t\tRaise KeyError if 'objspec' does not exist in the repo.\n+\t\t\"\"\"\n+\t\tclass ObjHandler (object):\n+\t\t\tdef __init__ (self, parser):\n+\t\t\t\tself.parser = parser\n+\t\t\t\tself.contents = StringIO()\n+\t\t\t\tself.err = None\n+\n+\t\t\tdef __call__ (self, parser, obj, sha1, t, size, data):\n+\t\t\t\tassert parser == self.parser\n+\t\t\t\tif not sha1: # missing object\n+\t\t\t\t\tself.err = \"Missing object '%s'\" % obj\n+\t\t\t\t\treturn\n+\t\t\t\tassert size == len(data)\n+\t\t\t\tself.contents.write(data)\n+\n+\t\thandler = ObjHandler(self)\n+\t\tself.push(objspec, handler)\n+\t\tself.process()\n+\t\tif handler.err: raise KeyError, handler.err\n+\t\thandler.contents.seek(0)\n+\t\treturn handler.contents\n+\n+\tdef walk_tree (self, tree_objspec, callback, prefix = \"\"):\n+\t\t\"\"\"Recursively walk the given Git tree object\n+\n+\t\tRecursively walks all subtrees of the given tree object, and\n+\t\tinvokes the given callback passing three arguments:\n+\t\t(path, mode, data) with the path, permission bits, and contents\n+\t\tof all the blobs found in the entire tree structure.\n+\t\t\"\"\"\n+\n+\t\tclass ObjHandler (object):\n+\t\t\t\"\"\"Helper class for walking a git tree structure\"\"\"\n+\t\t\tdef __init__ (self, parser, cb, path, mode = None):\n+\t\t\t\tself.parser = parser\n+\t\t\t\tself.cb = cb\n+\t\t\t\tself.path = path\n+\t\t\t\tself.mode = mode\n+\t\t\t\tself.err = None\n+\n+\t\t\tdef parse_tree (self, treedata):\n+\t\t\t\t\"\"\"Parse tree object data, yield tree entries\n+\n+\t\t\t\tEach tree entry is a 3-tuple (mode, sha1, path)\n+\n+\t\t\t\tself.path is prepended to all paths yielded\n+\t\t\t\tfrom this method.\n+\t\t\t\t\"\"\"\n+\t\t\t\twhile treedata:\n+\t\t\t\t\tmode = int(treedata[:6], 10)\n+\t\t\t\t\t# Turn 100xxx into xxx\n+\t\t\t\t\tif mode > 100000: mode -= 100000\n+\t\t\t\t\tassert treedata[6] == \" \"\n+\t\t\t\t\ti = treedata.find(\"\\0\", 7)\n+\t\t\t\t\tassert i > 0\n+\t\t\t\t\tpath = treedata[7:i]\n+\t\t\t\t\tsha1 = hexlify(treedata[i + 1: i + 21])\n+\t\t\t\t\tyield (mode, sha1, self.path + path)\n+\t\t\t\t\ttreedata = treedata[i + 21:]\n+\n+\t\t\tdef __call__ (self, parser, obj, sha1, t, size, data):\n+\t\t\t\tassert parser == self.parser\n+\t\t\t\tif not sha1: # missing object\n+\t\t\t\t\tself.err = \"Missing object '%s'\" % obj\n+\t\t\t\t\treturn\n+\t\t\t\tassert size == len(data)\n+\t\t\t\tif t == \"tree\":\n+\t\t\t\t\tif self.path: self.path += \"/\"\n+\t\t\t\t\t# recurse into all blobs and subtrees\n+\t\t\t\t\tfor m, s, p in self.parse_tree(data):\n+\t\t\t\t\t\tparser.push(s, self.__class__(\n+\t\t\t\t\t\t\tself.parser, self.cb,\n+\t\t\t\t\t\t\tp, m))\n+\t\t\t\telif t == \"blob\":\n+\t\t\t\t\tself.cb(self.path, self.mode, data)\n+\t\t\t\telse:\n+\t\t\t\t\traise ValueError, \"Unknown object \" \\\n+\t\t\t\t\t\t\"type '%s'\" % (t)\n+\n+\t\tself.push(tree_objspec, ObjHandler(self, callback, prefix))\n+\t\tself.process()\n+\n+\n+class GitRefMap (object):\n+\t\"\"\"Map Git ref names to the Git object names they currently point to\n+\n+\tBehaves like a dictionary of Git ref names -> Git object names.\n+\t\"\"\"\n+\n+\tdef __init__ (self, obj_fetcher):\n+\t\tself.obj_fetcher = obj_fetcher\n+\t\tself._cache = {} # dict: refname -> objname\n+\n+\tdef _load (self, ref):\n+\t\tif ref not in self._cache:\n+\t\t\tself._cache[ref] = self.obj_fetcher.get_sha1(ref)\n+\t\treturn self._cache[ref]\n+\n+\tdef __contains__ (self, refname):\n+\t\t\"\"\"Return True if the given refname is present in this cache\"\"\"\n+\t\treturn bool(self._load(refname))\n+\n+\tdef __getitem__ (self, refname):\n+\t\t\"\"\"Return the CvsState of the given refname\"\"\"\n+\t\tcommit = self._load(refname)\n+\t\tif commit is None:\n+\t\t\traise KeyError, \"Unknown ref '%s'\" % (refname)\n+\t\treturn commit\n+\n+\tdef get (self, refname, default = None):\n+\t\tcommit = self._load(refname)\n+\t\tif commit is None: return default\n+\t\treturn commit\n+\n+\n+class GitFICommit (object):\n+\t\"\"\"Encapsulate the data in a Git fast-import commit command\"\"\"\n+\n+\tSHA1RE = re.compile(r'^[0-9a-f]{40}$')\n+\n+\t@classmethod\n+\tdef parse_mode (cls, mode):\n+\t\tassert mode in (644, 755, 100644, 100755, 120000)\n+\t\treturn \"%i\" % (mode)\n+\n+\t@classmethod\n+\tdef parse_objname (cls, objname):\n+\t\tif isinstance(objname, int): # object name is a mark number\n+\t\t\tassert objname > 0\n+\t\t\treturn \":%i\" % (objname)\n+\n+\t\t# No existence check is done, only checks for valid format\n+\t\tassert cls.SHA1RE.match(objname) # object name is valid SHA1\n+\t\treturn objname\n+\n+\t@classmethod\n+\tdef quote_path (cls, path):\n+\t\tpath = path.replace(\"\\\\\", \"\\\\\\\\\")\n+\t\tpath = path.replace(\"\\n\", \"\\\\n\")\n+\t\tpath = path.replace('\"', '\\\\\"')\n+\t\treturn '\"%s\"' % (path)\n+\n+\t@classmethod\n+\tdef parse_path (cls, path):\n+\t\tassert not isinstance(path, int) # cannot be a mark number\n+\n+\t\t# These checks verify the rules on the fast-import man page\n+\t\tassert not path.count(\"//\")\n+\t\tassert not path.endswith(\"/\")\n+\t\tassert not path.startswith(\"/\")\n+\t\tassert not path.count(\"/./\")\n+\t\tassert not path.count(\"/../\")\n+\t\tassert not path.endswith(\"/.\")\n+\t\tassert not path.endswith(\"/..\")\n+\t\tassert not path.startswith(\"./\")\n+\t\tassert not path.startswith(\"../\")\n+\n+\t\tif path.count('\"') + path.count('\\n') + path.count('\\\\'):\n+\t\t\treturn cls.quote_path(path)\n+\t\treturn path\n+\n+\t@classmethod\n+\tdef test (cls):\n+\t\tdef expect_fail (method, data):\n+\t\t\ttry: method(data)\n+\t\t\texcept AssertionError: return\n+\t\t\traise AssertionError, \"Failed test for invalid data \" \\\n+\t\t\t\t\"'%s(%s)'\" % (method.__name__, repr(data))\n+\n+\t\t# Test parse_mode()\n+\t\tassert cls.parse_mode(644) == \"644\"\n+\t\tassert cls.parse_mode(755) == \"755\"\n+\t\tassert cls.parse_mode(100644) == \"100644\"\n+\t\tassert cls.parse_mode(100755) == \"100755\"\n+\t\tassert cls.parse_mode(120000) == \"120000\"\n+\t\texpect_fail(cls.parse_mode, 0)\n+\t\texpect_fail(cls.parse_mode, 123)\n+\t\texpect_fail(cls.parse_mode, 600)\n+\t\texpect_fail(cls.parse_mode, \"644\")\n+\t\texpect_fail(cls.parse_mode, \"abc\")\n+\n+\t\t# Test parse_objname()\n+\t\tassert cls.parse_objname(1) == \":1\"\n+\t\texpect_fail(cls.parse_objname, 0)\n+\t\texpect_fail(cls.parse_objname, -1)\n+\t\tassert cls.parse_objname(\"0123456789\" * 4) == \"0123456789\" * 4\n+\t\tassert cls.parse_objname(\"2468abcdef\" * 4) == \"2468abcdef\" * 4\n+\t\texpect_fail(cls.parse_objname, \"abcdefghij\" * 4)\n+\n+\t\t# Test parse_path()\n+\t\tassert cls.parse_path(\"foo/bar\") == \"foo/bar\"\n+\t\tassert cls.parse_path(1) == \":1\"\n+\t\tassert cls.parse_path(\"path/with\\n and \\\" in it\") \\\n+\t\t\t== '\"path/with\\\\n and \\\\\" in it\"'\n+\t\texpect_fail(cls.parse_path, 0)\n+\t\texpect_fail(cls.parse_path, -1)\n+\t\texpect_fail(cls.parse_path, \"foo//bar\")\n+\t\texpect_fail(cls.parse_path, \"foo/bar/\")\n+\t\texpect_fail(cls.parse_path, \"/foo/bar\")\n+\t\texpect_fail(cls.parse_path, \"foo/./bar\")\n+\t\texpect_fail(cls.parse_path, \"foo/../bar\")\n+\t\texpect_fail(cls.parse_path, \"foo/bar/.\")\n+\t\texpect_fail(cls.parse_path, \"foo/bar/..\")\n+\t\texpect_fail(cls.parse_path, \"./foo/bar\")\n+\t\texpect_fail(cls.parse_path, \"../foo/bar\")\n+\n+\tdef __init__ (self, name, email, timestamp, timezone, message):\n+\t\tself.name = name\n+\t\tself.email = email\n+\t\tself.timestamp = timestamp\n+\t\tself.timezone = timezone\n+\t\tself.message = message\n+\t\tself.pathops = [] # List of path operations in this commit\n+\n+\tdef modify (self, mode, blobname, path):\n+\t\tself.pathops.append((\"M\",\n+\t\t\tself.parse_mode(mode),\n+\t\t\tself.parse_objname(blobname),\n+\t\t\tself.parse_path(path)))\n+\n+\tdef delete (self, path):\n+\t\tself.pathops.append((\"D\", self.parse_path(path)))\n+\n+\tdef copy (self, path, newpath):\n+\t\tself.pathops.append((\"C\",\n+\t\t\tself.parse_path(path),\n+\t\t\tself.parse_path(newpath)))\n+\n+\tdef rename (self, path, newpath):\n+\t\tself.pathops.append((\"R\",\n+\t\t\tself.parse_path(path),\n+\t\t\tself.parse_path(newpath)))\n+\n+\tdef note (self, blobname, commit):\n+\t\tself.pathops.append((\"N\",\n+\t\t\tself.parse_objname(blobname),\n+\t\t\tself.parse_objname(commit)))\n+\n+\tdef deleteall (self):\n+\t\tself.pathops.append(\"deleteall\")\n+\n+\n+class GitFastImport (object):\n+\t\"\"\"Encapsulate communication with git fast-import\"\"\"\n+\n+\tdef __init__ (self, f, obj_fetcher, last_mark = 0):\n+\t\tself.f = f # File object where fast-import stream is written\n+\t\tself.obj_fetcher = obj_fetcher # GitObjectFetcher instance\n+\t\tself.next_mark = last_mark + 1 # Next mark number\n+\t\tself.refs = set() # keep track of the refnames we've seen\n+\n+\tdef comment (self, s):\n+\t\t\"\"\"Write the given comment in the fast-import stream\"\"\"\n+\t\tassert \"\\n\" not in s, \"Malformed comment: '%s'\" % (s)\n+\t\tself.f.write(\"# %s\\n\" % (s))\n+\n+\tdef commit (self, ref, commitdata):\n+\t\t\"\"\"Make a commit on the given ref, with the given GitFICommit\n+\n+\t\tReturn the mark number identifying this commit.\n+\t\t\"\"\"\n+\t\tself.f.write(\"\"\"\\\n+commit %(ref)s\n+mark :%(mark)i\n+committer %(name)s <%(email)s> %(timestamp)i %(timezone)s\n+data %(msgLength)i\n+%(msg)s\n+\"\"\"% {\n+\t'ref': ref,\n+\t'mark': self.next_mark,\n+\t'name': commitdata.name,\n+\t'email': commitdata.email,\n+\t'timestamp': commitdata.timestamp,\n+\t'timezone': commitdata.timezone,\n+\t'msgLength': len(commitdata.message),\n+\t'msg': commitdata.message,\n+})\n+\n+\t\tif ref not in self.refs:\n+\t\t\tself.refs.add(ref)\n+\t\t\tparent = ref + \"^0\"\n+\t\t\tif self.obj_fetcher.get_sha1(parent):\n+\t\t\t\tself.f.write(\"from %s\\n\" % (parent))\n+\n+\t\tfor op in commitdata.pathops:\n+\t\t\tself.f.write(\" \".join(op))\n+\t\t\tself.f.write(\"\\n\")\n+\t\tself.f.write(\"\\n\")\n+\t\tretval = self.next_mark\n+\t\tself.next_mark += 1\n+\t\treturn retval\n+\n+\tdef blob (self, data):\n+\t\t\"\"\"Import the given blob\n+\n+\t\tReturn the mark number identifying this blob.\n+\t\t\"\"\"\n+\t\tself.f.write(\"blob\\nmark :%i\\ndata %i\\n%s\\n\" \\\n+\t\t\t% (self.next_mark, len(data), data))\n+\t\tretval = self.next_mark\n+\t\tself.next_mark += 1\n+\t\treturn retval\n+\n+\tdef reset (self, ref, objname):\n+\t\tself.f.write(\"reset %s\\nfrom %s\\n\\n\" % \\\n+\t\t\t(ref, GitFICommit.parse_objname(objname)))\n+\t\tif ref not in self.refs:\n+\t\t\tself.refs.add(ref)\n+\n+\n+class GitNotes (object):\n+\t\"\"\"Encapsulate access to Git notes\n+\n+\tBehaves like a dictionary of object name (SHA1) -> Git note mappings.\n+\t\"\"\"\n+\tdef __init__ (self, notes_ref, obj_fetcher):\n+\t\tself.notes_ref = notes_ref\n+\t\tself.obj_fetcher = obj_fetcher # used to get objects from repo\n+\t\tself.imports = [] # list: (objname, note data blob name) tuples\n+\n+\tdef __del__ (self):\n+\t\tif self.imports:\n+\t\t\terror(\"Missing call to self.commit_notes().\")\n+\t\t\terror(\"%i notes are not committed!\", len(self.imports))\n+\n+\tdef _load (self, objname):\n+\t\ttry:\n+\t\t\tf = self.obj_fetcher.open_obj(\n+\t\t\t\t\"%s:%s\" % (self.notes_ref, objname))\n+\t\t\tret = f.read()\n+\t\t\tf.close()\n+\t\texcept KeyError:\n+\t\t\tret = None\n+\t\treturn ret\n+\n+\tdef __getitem__ (self, objname):\n+\t\t\"\"\"Return the note contents associated with the given object\n+\n+\t\tRaise KeyError if given object has no associated note.\n+\t\t\"\"\"\n+\t\tblobdata = self._load(objname)\n+\t\tif blobdata is None:\n+\t\t\traise KeyError, \"Object '%s' has no note\" % (objname)\n+\t\treturn blobdata\n+\n+\tdef get (self, objname, default = None):\n+\t\t\"\"\"Return the note contents associated with the given object\n+\n+\t\tReturn given default if given object has no associated note.\n+\t\t\"\"\"\n+\t\tblobdata = self._load(objname)\n+\t\tif blobdata is None: return default\n+\t\treturn blobdata\n+\n+\tdef import_note (self, objname, data, gfi):\n+\t\t\"\"\"Tell git fast-import to store data as a note for objname\n+\n+\t\tThis method uses the given GitFastImport object to create a\n+\t\tblob containing the given note data. Also an entry mapping the\n+\t\tgiven object name to the created blob is stored until\n+\t\tcommit_notes() is called.\n+\n+\t\tNote that this method only works if later followed with a call\n+\t\tto commit_notes() (which produces the note commit that refers\n+\t\tto the blob produced here).\n+\t\t\"\"\"\n+\t\tif not data.endswith(\"\\n\"): data += \"\\n\"\n+\t\tgfi.comment(\"Importing note for object %s\" % (objname))\n+\t\tmark = gfi.blob(data)\n+\t\tself.imports.append((objname, mark))\n+\n+\tdef commit_notes (self, gfi, author, message):\n+\t\t\"\"\"Produce a git fast-import note commit for the imported notes\n+\n+\t\tThis method uses the given GitFastImport object to create a\n+\t\tcommit on the notes ref, introducing the notes previously\n+\t\tsubmitted to import_note().\n+\t\t\"\"\"\n+\t\tif not self.imports: return # Nothing to do\n+\n+\t\tfrom cvs import CvsDate\n+\t\tnow = CvsDate(\"now\")\n+\t\tcommitdata = GitFICommit(\n+\t\t\tauthor[0], author[1], now.ts, now.tz_str(), message)\n+\n+\t\tfor objname, blobname in self.imports:\n+\t\t\tassert isinstance(objname, int) and objname > 0\n+\t\t\tassert isinstance(blobname, int) and blobname > 0\n+\t\t\tcommitdata.note(blobname, objname)\n+\n+\t\tgfi.commit(self.notes_ref, commitdata)\n+\t\tself.imports = []\n+\n+\n+class GitCachedNotes (GitNotes):\n+\t\"\"\"Encapsulate access to Git notes (cached version)\n+\n+\tOnly use this class if no caching is done at a higher level.\n+\n+\tBehaves like a dictionary of object name (SHA1) -> Git note mappings.\n+\t\"\"\"\n+\tdef __init__ (self, notes_ref, obj_fetcher):\n+\t\tGitNotes.__init__(self, notes_ref, obj_fetcher)\n+\t\tself._cache = {} # cache: object name -> note data\n+\n+\tdef __del__ (self):\n+\t\tGitNotes.__del__(self)\n+\n+\tdef _load (self, objname):\n+\t\tif objname not in self._cache:\n+\t\t\tself._cache[objname] = GitNotes._load(self, objname)\n+\t\treturn self._cache[objname]\n+\n+\tdef import_note (self, objname, data, gfi):\n+\t\tif not data.endswith(\"\\n\"): data += \"\\n\"\n+\t\tassert objname not in self._cache\n+\t\tself._cache[objname] = data\n+\t\tGitNotes.import_note(self, objname, data, gfi)\n+\n+\n+if __name__ == '__main__':\n+\t# Run selftests\n+\tGitFICommit.test()\ndiff --git a/git_remote_cvs/setup.py b/git_remote_cvs/setup.py\nnew file mode 100644\nindex 0000000..64c9209\n--- /dev/null\n+++ b/git_remote_cvs/setup.py\n@@ -0,0 +1,12 @@\n+from distutils.core import setup\n+setup(\n+\tname = 'git_remote_cvs',\n+\tversion = '0.1.0',\n+\tdescription = 'Git remote helper program for CVS repositories',\n+\tlicense = 'GPLv2',\n+\tauthor = 'The Git Community',\n+\tauthor_email = 'git@vger.kernel.org',\n+\turl = 'http://www.git-scm.com/',\n+\tpackage_dir = {'git_remote_cvs': ''},\n+\tpackages = ['git_remote_cvs'],\n+)\ndiff --git a/git_remote_cvs/util.py b/git_remote_cvs/util.py\nnew file mode 100644\nindex 0000000..6877454\n--- /dev/null\n+++ b/git_remote_cvs/util.py\n@@ -0,0 +1,147 @@\n+#!/usr/bin/env python\n+\n+\"\"\"Misc. useful functionality used by the rest of this package\n+\n+This module provides common functionality used by the other modules in this\n+package.\n+\"\"\"\n+\n+import sys, os, subprocess\n+\n+\n+# Whether or not to show debug messages\n+Debug = False\n+\n+def debug (msg, *args):\n+\tif Debug: print >>sys.stderr, msg % args\n+\n+def error (msg, *args):\n+\tprint >>sys.stderr, \"ERROR:\", msg % args\n+\n+def die (msg, *args):\n+\terror(msg, *args)\n+\tsys.exit(1)\n+\n+\n+class ProgressIndicator (object):\n+\t\"\"\"Simple progress indicator\n+\n+\tDisplayed as a spinning character by default, but can be customized by\n+\tpassing custom messages that will override the spinning character.\n+\t\"\"\"\n+\n+\tStates = (\"|\", \"/\", \"-\", \"\\\\\")\n+\n+\tdef __init__ (self, prefix = \"\", f = sys.stdout):\n+\t\tself.n = 0 # Simple progress counter\n+\t\tself.f = f # Progress is written to this file object\n+\t\tself.prev_len = 0 # Length of previous msg (to be overwritten)\n+\t\tself.prefix = prefix\n+\t\tself.prefix_lens = []\n+\n+\tdef pushprefix (self, prefix):\n+\t\tself.prefix_lens.append(len(self.prefix))\n+\t\tself.prefix += prefix\n+\n+\tdef popprefix (self):\n+\t\tprev_len = self.prefix_lens.pop()\n+\t\tself.prefix = self.prefix[:prev_len]\n+\n+\tdef __call__ (self, msg = None, lf = False):\n+\t\tif msg is None: msg = self.States[self.n % len(self.States)]\n+\t\tmsg = self.prefix + msg\n+\t\tprint >>self.f, \"\\r%-*s\" % (self.prev_len, msg),\n+\t\tself.prev_len = len(msg.expandtabs())\n+\t\tif lf:\n+\t\t\tprint >>self.f\n+\t\t\tself.prev_len = 0\n+\t\tself.n += 1\n+\n+\tdef finish (self, msg = \"done\", noprefix = False):\n+\t\tif noprefix: self.prefix = \"\"\n+\t\tself(msg, True)\n+\n+\n+def start_command (args, cwd = None, shell = False, add_env = None,\n+ stdin = subprocess.PIPE,\n+ stdout = subprocess.PIPE,\n+ stderr = subprocess.PIPE):\n+\tenv = None\n+\tif add_env is not None:\n+\t\tenv = os.environ.copy()\n+\t\tenv.update(add_env)\n+\treturn subprocess.Popen(args, # run command\n+\t\tbufsize = 1, # line buffered\n+\t\tstdin = stdin, # write to process\n+\t\tstdout = stdout, # read from process\n+\t\tstderr = stderr,\n+\t\tcwd = cwd,\n+\t\tshell = shell,\n+\t\tenv = env,\n+\t\tuniversal_newlines = True,\n+\t)\n+\n+def run_command (args, cwd = None, shell = False, add_env = None,\n+ flag_error = True):\n+\tprocess = start_command(args, cwd, shell, add_env)\n+\t(output, errors) = process.communicate()\n+\texit_code = process.returncode\n+\tif flag_error and errors:\n+\t\terror(\"'%s' returned errors:\\n---\\n%s---\",\n+\t\t \" \".join(args), errors)\n+\tif flag_error and exit_code:\n+\t\terror(\"'%s' returned exit code %i\", \" \".join(args), exit_code)\n+\treturn (exit_code, output, errors)\n+\n+\n+def file_reader_method (missing_ok = False):\n+\t\"\"\"Decorator for simplifying reading of files.\n+\n+\tIf missing_ok is True, a failure to open a file for reading will not\n+\traise the usual IOError, but instead the wrapped method will be called\n+\twith f == None. The method must in this case properly handle f == None.\n+\t\"\"\"\n+\tdef wrap (method):\n+\t\t\"\"\"Teach given method to handle both filenames and file objects\n+\n+\t\tThe given method must take a file object as its second argument\n+\t\t(the first argument being 'self', of course). This decorator\n+\t\twill take a filename given as the second argument and promote\n+\t\tit to a file object.\n+\t\t\"\"\"\n+\t\tdef wrapped_method (self, filename, *args, **kwargs):\n+\t\t\tif isinstance(filename, file): f = filename # No-op\n+\t\t\telse:\n+\t\t\t\ttry:\n+\t\t\t\t\tf = open(filename, 'r')\n+\t\t\t\texcept IOError:\n+\t\t\t\t\tif missing_ok: f = None\n+\t\t\t\t\telse: raise\n+\t\t\ttry:\n+\t\t\t\treturn method(self, f, *args, **kwargs)\n+\t\t\tfinally:\n+\t\t\t\tif not isinstance(filename, file) and f:\n+\t\t\t\t\tf.close()\n+\t\treturn wrapped_method\n+\treturn wrap\n+\n+def file_writer_method (method):\n+\t\"\"\"Enable the given method to handle both filenames and file objects\n+\n+\tThe given method must take a file object as its second argument (the\n+\tfirst argument being 'self', of course). This decorator will take a\n+\tfilename given as the second argument and promote it to a file object.\n+\t\"\"\"\n+\tdef new_method (self, filename, *args, **kwargs):\n+\t\tif isinstance(filename, file): f = filename # Nothing to do\n+\t\telse:\n+\t\t\t# Make sure the containing directory exists\n+\t\t\tparent_dir = os.path.dirname(filename)\n+\t\t\tif not os.path.isdir(parent_dir):\n+\t\t\t\tos.makedirs(parent_dir)\n+\t\t\tf = open(filename, 'w')\n+\t\ttry:\n+\t\t\treturn method(self, f, *args, **kwargs)\n+\t\tfinally:\n+\t\t\tif not isinstance(filename, file): f.close()\n+\treturn new_method\n-- \n1.6.4.rc3.138.ga6b98.dirty\n"},{"id":"120335","messageId":"1250036031-32272-4-git-send-email-johan@herland.net","threadId":"20543","inReplyTo":"1250036031-32272-1-git-send-email-johan@herland.net","subject":"[RFCv3 3/4] Third draft of CVS remote helper program","fromName":"Johan Herland","fromEmail":"johan@herland.net","sentAt":"2009-08-12T00:13:50Z","receivedAt":"2009-08-12T00:13:50Z","isPatch":false,"sender":{"key":"johan@herland.net","avatar":"https://avatars.githubusercontent.com/u/547031?v=4"},"body":"Implements the import of objects from a local or remote CVS repository.\n\nThis helper program uses the \"git_remote_cvs\" Python package introduced\nearlier, and provides a working draft implementation of the remote helper\nAPI, as described in Documentation/git-remote-helpers.txt. Further details\nabout this specific helper are described in the new\nDocumentation/git-remote-cvs.txt.\n\nThis patch has been improved by the following contributions:\n- Daniel Barkalow: Updates reflecting changes in remote helper API\n\nSigned-off-by: Johan Herland \n---\n Documentation/git-remote-cvs.txt | 85 +++++\n Makefile | 24 ++\n git-remote-cvs.py | 697 ++++++++++++++++++++++++++++++++++++++\n 3 files changed, 806 insertions(+), 0 deletions(-)\n create mode 100644 Documentation/git-remote-cvs.txt\n create mode 100755 git-remote-cvs.py\n\ndiff --git a/Documentation/git-remote-cvs.txt b/Documentation/git-remote-cvs.txt\nnew file mode 100644\nindex 0000000..783d542\n--- /dev/null\n+++ b/Documentation/git-remote-cvs.txt\n@@ -0,0 +1,85 @@\n+git-remote-cvs(1)\n+==============\n+\n+NAME\n+----\n+git-remote-cvs - Helper program for interoperation with CVS repositories\n+\n+SYNOPSIS\n+--------\n+'git remote-cvs' \n+\n+DESCRIPTION\n+-----------\n+\n+Please see the linkgit:git-remote-helpers[1] documentation for general\n+information about remote helper programs.\n+\n+CONFIGURATION\n+-------------\n+\n+remote.*.cvsRoot::\n+\tThe URL of the CVS repository (as found in a `CVSROOT` variable, or\n+\tin a `CVS/Root` file).\n+\tExample: \"`:pserver:user@server/var/cvs/cvsroot`\".\n+\n+remote.*.cvsModule::\n+\tThe path of the CVS module (as found in a `CVS/Repository` file)\n+\twithin the CVS repository specified in `remote.*.cvsRoot`.\n+\tExample: \"`foo/bar`\"\n+\n+remote.*.cachedSymbolsOnly::\n+\tWhen 'true', a cache of CVS symbols is used instead of querying the\n+\tCVS server for all existing symbols (potentially expensive). In this\n+\tmode, git-remote-cvs will not discover new CVS symbols unless you add\n+\tthem explicitly with the \"`addsymbol `\" command (on\n+\tgit-remote-cvs's stdin), or request an explicit symbol cache update\n+\tfrom the CVS server with the \"`syncsymbols`\" command (on\n+\tgit-remote-cvs's stdin). When 'false' (the default), the CVS server\n+\twill be queried whenever a list of CVS symbols is required.\n+\n+remote.*.usernameMap::\n+\tThe path (absolute, or relative to the repository (NOT the worktree))\n+\tto the file that contains the mapping from CVS usernames to the\n+\tcorresponding full names and email addresses, as used by Git in the\n+\tAuthor and Committer fields of commit objects. When this config\n+\tvariable is set, CVS usernames will be resolved against this file.\n+\tIf no match is found in the file, or if this config variable is unset,\n+\tor if the variable points to a non-existing file, the original CVS\n+\tusername will be used as the Author/Committer name, and the\n+\tcorresponding email address will be set to \"`@example.com`\".\n++\n+The format of the usernameMap file is one entry per line, where each line is\n+of the form \"`username: Full Name `\".\n+Example: `johndoe: John Doe `\n+Blank lines and lines starting with '#' are ignored.\n+\n+COMMANDS\n+--------\n+\n+In addition to the commands that constitute the git-remote-helpers API, the\n+following extra commands are supported for managing the local symbol cache when\n+the `remote.*.cachedSymbolsOnly` config variable is true. The following\n+commands can be given on the standard input of git-remote-cvs:\n+\n+'addsymbol'::\n+\tTakes one CVS symbol name as argument. The given CVS symbol is\n+\tfetched from the CVS server and stored into the local CVS symbol\n+\tcache. If `remote.*.cachedSymbolsOnly` is enabled, this can be used\n+\tto introduce a new CVS symbol to the CVS helper application.\n+\n+'syncsymbols'::\n+\tAll CVS symbols that are available from the given remote are\n+\tfetched from the CVS server and stored into the local CVS symbol\n+\tcache. This is equivalent to disabling `remote.{asterisk}.cachedSymbolsOnly`,\n+\trunning the \"list\" command, and then finally re-enabling the\n+\t`remote.*.cachedSymbolsOnly` config variable. I.e. this command can\n+\tbe used to manually synchronize the CVS symbols available to the\n+\tCVS helper application.\n+\n+'verify'::\n+\tTakes one CVS symbol name as argument. Verifies that the CVS symbol\n+\thas been successfully imported be checking out the CVS symbol from\n+\tthe CVS server, and comparing the CVS working tree against the Git\n+\ttree object identified by `refs/cvs//`. This can be\n+\tused to verify the correctness of a preceding 'import' command.\ndiff --git a/Makefile b/Makefile\nindex bb5cea2..b2af678 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -350,6 +350,8 @@ SCRIPT_PERL += git-relink.perl\n SCRIPT_PERL += git-send-email.perl\n SCRIPT_PERL += git-svn.perl\n \n+SCRIPT_PYTHON += git-remote-cvs.py\n+\n SCRIPTS = $(patsubst %.sh,%,$(SCRIPT_SH)) \\\n \t $(patsubst %.perl,%,$(SCRIPT_PERL)) \\\n \t $(patsubst %.py,%,$(SCRIPT_PYTHON)) \\\n@@ -1474,6 +1476,28 @@ $(patsubst %.perl,%,$(SCRIPT_PERL)) git-instaweb: % : unimplemented.sh\n \tmv $@+ $@\n endif # NO_PERL\n \n+ifndef NO_PYTHON\n+$(patsubst %.py,%,$(SCRIPT_PYTHON)): % : %.py\n+\t$(QUIET_GEN)$(RM) $@ $@+ && \\\n+\tINSTLIBDIR=`MAKEFLAGS= $(MAKE) -C git_remote_cvs -s --no-print-directory instlibdir` && \\\n+\tsed -e '1{' \\\n+\t -e '\ts|#!.*python|#!$(PYTHON_PATH_SQ)|' \\\n+\t -e '}' \\\n+\t -e 's|^import sys.*|&; sys.path.insert(0, \"@@INSTLIBDIR@@\")|' \\\n+\t -e 's|@@INSTLIBDIR@@|'\"$$INSTLIBDIR\"'|g' \\\n+\t $@.py >$@+ && \\\n+\tchmod +x $@+ && \\\n+\tmv $@+ $@\n+else # NO_PYTHON\n+$(patsubst %.py,%,$(SCRIPT_PYTHON)): % : unimplemented.sh\n+\t$(QUIET_GEN)$(RM) $@ $@+ && \\\n+\tsed -e '1s|#!.*/sh|#!$(SHELL_PATH_SQ)|' \\\n+\t -e 's|@@REASON@@|NO_PYTHON=$(NO_PYTHON)|g' \\\n+\t unimplemented.sh >$@+ && \\\n+\tchmod +x $@+ && \\\n+\tmv $@+ $@\n+endif # NO_PYTHON\n+\n configure: configure.ac\n \t$(QUIET_GEN)$(RM) $@ $<+ && \\\n \tsed -e 's/@@GIT_VERSION@@/$(GIT_VERSION)/g' \\\ndiff --git a/git-remote-cvs.py b/git-remote-cvs.py\nnew file mode 100755\nindex 0000000..1720d4c\n--- /dev/null\n+++ b/git-remote-cvs.py\n@@ -0,0 +1,697 @@\n+#!/usr/bin/env python\n+\n+\"\"\"Usage: git-remote-cvs []\n+\n+Git remote helper for interacting with CVS repositories\n+\n+See git-remote-helpers documentation for details on external interface, usage,\n+etc. See git-remote-cvs documentation for specific configuration details of\n+this remote helper.\n+\"\"\"\n+\n+# PRINCIPLES:\n+# -----------\n+# - Importing same symbol twice (with no CVS changes in between) should yield\n+# the exact same Git state (and the second import should have no commits).\n+# - Importing several CVS symbols pointing to the same state should yield\n+# corresponding refs pointing to the _same_ commit in Git.\n+# - Importing a CVS symbol which has received only \"regular commits\" since\n+# last import should yield a fast-forward straight line of commits.\n+\n+# TODO / KNOWN PROBLEMS:\n+# ----------------------\n+# - Remove cachedSymbolsOnly config variable for now?\n+# - Author map handling; mapping CVS usernames to Git full name + email address\n+# - Handle files that have been created AND deleted since the last import\n+# - How to handle CVS tags vs. CVS branches. Turn CVS tags into Git tags?\n+# - Better CVS branch handling: When a branch as a super/subset of files/revs\n+# compared to another branch, find a way to base one branch on the other\n+# instead of creating parallel lines of development with roughly the same\n+# commits.\n+# - Profiling, optimizations...\n+\n+import sys, string, os\n+\n+from git_remote_cvs.util import *\n+from git_remote_cvs.cvs import *\n+from git_remote_cvs.git import *\n+from git_remote_cvs.cvs_symbol_cache import CvsSymbolCache\n+from git_remote_cvs.commit_states import CommitStates\n+from git_remote_cvs.cvs_revision_map import CvsRevisionMap, CvsStateMap\n+from git_remote_cvs.changeset import build_changesets_from_revs\n+\n+class Config (object):\n+\t# Author name/email tuple for commits created by this tool\n+\tAuthor = (\"git remote-cvs\", \"invalid@example.com\")\n+\n+\t# Git remote name\n+\tRemote = None\n+\n+\t# CVS symbols are imported into this refs namespace/directory\n+\tRefSpace = None\n+\n+\t# Git notes ref, the refname pointing to our git notes\n+\tNotesRef = None\n+\n+\t# CVS repository identifier, a 2-tuple (cvs_root, cvs_module), where\n+\t# cvs_root is the CVS server/repository URL (as found in $CVSROOT, or\n+\t# in a CVS/Root file), and cvs_module is the path to a CVS module\n+\t# relative to the CVS repository (as found in a CVS/Repository file)\n+\tCvsRepo = (None, None)\n+\n+\t# Path to the git-remote-cvs cache/work directory\n+\t# (normally \"info/cvs/$remote\" within $GIT_DIR)\n+\tWorkDir = None\n+\n+\t# If False, the list of CVS symbols will always be retrieved from the\n+\t# CVS server using 'cvs rlog'. If True, only the cached symbols within\n+\t# the \"symbols\" subdirectory of WorkDir are consulted.\n+\tCachedSymbolsOnly = False\n+\n+\t@classmethod\n+\tdef init (cls, remote):\n+\t\t\"\"\"Fetch configurations parameters for the given remote\"\"\"\n+\t\tgit_config = parse_git_config()\n+\t\tassert git_config[\"remote.%s.vcs\" % (remote)] == \"cvs\"\n+\n+\t\tcls.Author = (\n+\t\t\tgit_config[\"user.name\"], git_config[\"user.email\"])\n+\t\tcls.Remote = remote\n+\t\tcls.RefSpace = \"refs/cvs/%s/\" % (remote)\n+\t\tcls.NotesRef = \"refs/notes/cvs/%s\" % (remote)\n+\t\tcls.CvsRepo = (\n+\t\t\tgit_config[\"remote.%s.cvsroot\" % (remote)],\n+\t\t\tgit_config[\"remote.%s.cvsmodule\" % (remote)])\n+\t\tcls.WorkDir = os.path.join(get_git_dir(), \"info/cvs\", remote)\n+\t\tcls.CachedSymbolsOnly = git_config_bool(git_config.get(\n+\t\t\t\"remote.%s.cachedsymbolsonly\" % (remote), \"false\"))\n+\n+def work_path (*args):\n+\t\"\"\"Return the given path appended to git-remote-cvs's cache/work dir\"\"\"\n+\treturn os.path.join(Config.WorkDir, *args)\n+\n+def cvs_to_refname (cvsname):\n+\t\"\"\"Return the git ref name for the given CVS symbolic name\"\"\"\n+\tif cvsname.startswith(Config.RefSpace): # Already converted\n+\t\treturn cvsname\n+\treturn Config.RefSpace + cvsname\n+\n+def ref_to_cvsname (refname):\n+\t\"\"\"Return the CVS symbolic name for the given git ref name\"\"\"\n+\tif refname.startswith(Config.RefSpace):\n+\t\treturn refname[len(Config.RefSpace):]\n+\treturn refname\n+\n+def valid_cvs_symbol (symbol):\n+\t\"\"\"Return True iff the given CVS symbol can be imported into Git\"\"\"\n+\treturn valid_git_ref(cvs_to_refname(symbol))\n+\n+def die_usage (msg, *args):\n+\t# Use this file's docstring as a usage string\n+\tprint >>sys.stderr, __doc__\n+\tdie(msg, *args)\n+\n+def import_cvs_revs (symbol, prev_state, cur_state, progress):\n+\t\"\"\"Import the CVS revisions involved in importing the given CVS symbol\n+\n+\tThis method will determine the CVS revisions involved in moving from\n+\tthe given prev_state to the given cur_state. This includes looking at\n+\trevision metadata in CVS, and importing needed blobs from CVS.\n+\n+\tThe revision metadata is returned as a 2-level dict of CvsRev objects:\n+\tmapping path -> revnum -> CvsRev object.\n+\t\"\"\"\n+\n+\t# Calculate the revisions involved in moving from prev_state to\n+\t# cur_state, and fetch CvsRev objects for these revisions.\n+\tprogress.pushprefix(\"Importing CVS revisions: \")\n+\tpaths = set(prev_state.paths()).union(cur_state.paths())\n+\tnum_fetched_revs = 0 # Number of CvsRev objects involved\n+\tnum_imported_blobs = 0 # Number of blobs actually imported\n+\tcvs_revs = {} # path -> revnum -> CvsRev\n+\tfor i, path in enumerate(sorted(paths)):\n+\t\tprogress.pushprefix(\"(%i/%i) %s: \" % (i + 1, len(paths), path))\n+\t\tprogress(\"\")\n+\t\tprev_rev = prev_state.get(path)\n+\t\tcur_rev = cur_state.get(path)\n+\t\tif prev_rev and cur_rev and prev_rev == cur_rev:\n+\t\t\t# No changes since last import\n+\t\t\tprogress.popprefix()\n+\t\t\tcontinue\n+\n+\t\t# Fetch CvsRev objects for range [path:prev_rev, path:symbol]\n+\t\tpath_revs = fetch_revs(path, prev_rev, cur_rev, symbol,\n+\t\t\tConfig.CvsRepo)\n+\t\tif not path_revs:\n+\t\t\t# Failed to find revs between prev_rev and symbol\n+\t\t\tif cur_rev:\n+\t\t\t\tassert not cur_rev.follows(prev_rev)\n+\t\t\t\t# The CVS symbol has been moved/reset since the\n+\t\t\t\t# last import in such a way that we cannot\n+\t\t\t\t# deduce the history between the last import\n+\t\t\t\t# and the current import.\n+\t\t\t\t# FIXME: Can we can work around this?\n+\t\t\t\tdie(\"CVS symbol %s has been moved/reset from\" \\\n+\t\t\t\t\t\" %s:%s to %s:%s since the last\" \\\n+\t\t\t\t\t\" import. This is not supported\",\n+\t\t\t\t\tsymbol, path, prev_rev, path, cur_rev)\n+\t\t\telse:\n+\t\t\t\t# CVS symbol has been removed from this path.\n+\t\t\t\t# We cannot conclusively determine the history\n+\t\t\t\t# of this path following prev_rev.\n+\t\t\t\t# FIXME: Can we can work around this?\n+\t\t\t\tdie(\"CVS symbol %s has been removed from %s\" \\\n+\t\t\t\t\t\" since the last import. This is not\" \\\n+\t\t\t\t\t\" supported\", symbol, path)\n+\n+\t\t# OK. We've got the revs in range [prev_rev, symbol]\n+\n+\t\t# Verify/determine cur_rev\n+\t\treal_cur_rev = max(path_revs.keys())\n+\t\tif cur_rev: assert cur_rev == real_cur_rev\n+\t\telse: cur_rev = real_cur_rev\n+\n+\t\t# No need to re-import prev_rev if already imported\n+\t\tif prev_rev:\n+\t\t\tassert cur_rev.follows(prev_rev)\n+\t\t\tassert prev_rev in path_revs\n+\t\t\tdel path_revs[prev_rev]\n+\n+\t\tassert path_revs # There should be more revs than just prev_rev\n+\n+\t\t# Sanity checks:\n+\t\t# All revs from prev_rev to cur_rev are about to be imported\n+\t\tcheck_rev = cur_rev\n+\t\twhile check_rev and check_rev != prev_rev:\n+\t\t\tassert check_rev in path_revs\n+\t\t\tcheck_rev = check_rev.parent()\n+\t\t# All previous revs have already been imported\n+\t\tcheck_rev = prev_rev\n+\t\twhile check_rev:\n+\t\t\tassert Globals.CvsRevisionMap.has_rev(path, check_rev)\n+\t\t\tcheck_rev = check_rev.parent()\n+\n+\t\t# Import CVS revisions as Git blobs\n+\t\tj = 0\n+\t\tfor num, rev in sorted(path_revs.iteritems(), reverse = True):\n+\t\t\tj += 1\n+\t\t\tprogress(\"(%i/%i) %s\" % (j, len(path_revs), num))\n+\t\t\tassert num == rev.num\n+\n+\t\t\t# Skip if already imported\n+\t\t\tif Globals.CvsRevisionMap.has_rev(rev.path, rev.num):\n+\t\t\t\tcontinue\n+\t\t\t# ...or if rev is a deletion\n+\t\t\telif rev.deleted:\n+\t\t\t\tcontinue\n+\n+\t\t\t# Import blob for reals\n+\t\t\tdata = Globals.CvsWorkDir.get_revision_data(\n+\t\t\t\trev.path, rev.num)\n+\t\t\tGlobals.GitFastImport.comment(\n+\t\t\t\t\"Importing CVS revision %s:%s\" % (\n+\t\t\t\trev.path, rev.num))\n+\t\t\tmark = Globals.GitFastImport.blob(data)\n+\t\t\tGlobals.CvsRevisionMap.add_blob(\n+\t\t\t\trev.path, rev.num, mark)\n+\t\t\tnum_imported_blobs += 1\n+\n+\t\t# Add path_revs to the overall structure of revs to be imported\n+\t\tassert path not in cvs_revs\n+\t\tcvs_revs[path] = path_revs\n+\t\tnum_fetched_revs += len(path_revs)\n+\n+\t\tprogress.popprefix()\n+\n+\tprogress.popprefix()\n+\tprogress(\"Imported %i blobs (reused %i existing blobs)\" % (\n+\t\tnum_imported_blobs, num_fetched_revs - num_imported_blobs),\n+\t\tTrue)\n+\n+\treturn cvs_revs\n+\n+def advance_state (state, changeset):\n+\t\"\"\"Advance the given state by applying the given changeset\"\"\"\n+\t# Verify that the given changeset \"fits\" on top of the given state\n+\tfor rev in changeset:\n+\t\tprev_num = rev.num.parent()\n+\t\tstate_num = state.get(rev.path)\n+\t\tif prev_num is None and state_num is None:\n+\t\t\t# 'rev' is the first revision of this path being added\n+\t\t\tstate.add(rev.path, rev.num)\n+\t\telif prev_num and state_num and prev_num == state_num:\n+\t\t\tif rev.deleted: # rev deletes path from state\n+\t\t\t\tstate.remove(rev.path, prev_num)\n+\t\t\telse: # rev follows state's revision of this path\n+\t\t\t\tstate.replace(rev.path, rev.num)\n+\t\telse:\n+\t\t\terror(\"Cannot apply changeset with %s:%s on top of \" \\\n+\t\t\t \"CVS state with %s:%s.\",\n+\t\t\t rev.path, changeset[rev.path].num,\n+\t\t\t rev.path, state.get(rev.path))\n+\t\t\terror(\" changeset: %s\", changeset)\n+\t\t\terror(\" CVS state: \\n---\\n%s---\", state)\n+\t\t\tdie(\"Failed to apply changeset. Aborting.\")\n+\n+def revert_state (state, changeset):\n+\t\"\"\"Revert the given state to _before_ the given changeset is applied\n+\n+\tThis is the reverse of the above advance_state() function.\n+\t\"\"\"\n+\tfor rev in changeset:\n+\t\tprev_num = rev.num.parent()\n+\t\tstate_num = state.get(rev.path)\n+\t\tif state_num is None: # revert deletion of file\n+\t\t\tassert rev.deleted\n+\t\t\tstate.add(rev.path, prev_num)\n+\t\telse:\n+\t\t\tassert state_num == rev.num\n+\t\t\tif prev_num is None: # revert addition of file\n+\t\t\t\tstate.remove(rev.path, rev.num)\n+\t\t\telse: # regular revert to previous version\n+\t\t\t\tstate.replace(rev.path, prev_num)\n+\n+def import_changesets (ref, changesets, from_state, to_state, progress):\n+\t\"\"\"Apply the given list of Changeset objects to the given ref\n+\n+\tAlso verify that the changesets bring us from the given from_state to\n+\tthe given to_state.\n+\t\"\"\"\n+\tstate = from_state\n+\tfor i, c in enumerate(changesets):\n+\t\tadvance_state(state, c)\n+\t\tprogress(\"(%i/%i) Committing %s\" % (i + 1, len(changesets), c))\n+\t\t# Make a git commit from changeset c\n+\t\tcommitdata = GitFICommit(\n+\t\t\tc.author, # TODO: author_map handling\n+\t\t\tc.author + \"@example.com\", # TODO: author_map handling\n+\t\t\tc.date.ts,\n+\t\t\tc.date.tz_str(),\n+\t\t\t\"\".join([\"%s\\n\" % (line) for line in c.message]),\n+\t\t)\n+\n+\t\tfor rev in c:\n+\t\t\tp, n = rev.path, rev.num\n+\t\t\tif rev.deleted:\n+\t\t\t\tcommitdata.delete(p)\n+\t\t\t\tcontinue\n+\t\t\tblobname = Globals.CvsRevisionMap.get_blob(p, n)\n+\t\t\tmode = Globals.CvsRevisionMap.get_mode(p)\n+\t\t\tif mode is None: # Must retrieve mode from CVS checkout\n+\t\t\t\tdebug(\"Retrieving mode info for '%s'\" % (p))\n+\t\t\t\tGlobals.CvsWorkDir.update(n, [p])\n+\t\t\t\tmode = Globals.CvsWorkDir.get_modeinfo([p])[p]\n+\t\t\t\tGlobals.CvsRevisionMap.add_path(p, mode)\n+\t\t\tcommitdata.modify(mode, blobname, p)\n+\n+\t\tcommitname = Globals.GitFastImport.commit(ref, commitdata)\n+\t\tGlobals.CommitStates.add(\n+\t\t\tcommitname, state, Globals.GitFastImport)\n+\t\tfor path, revnum in state:\n+\t\t\tGlobals.CvsRevisionMap.add_commit(\n+\t\t\t\tpath, revnum, commitname)\n+\t\tassert commitname in Globals.CvsStateMap.get_commits(state)\n+\n+\tassert state == to_state\n+\treturn len(changesets)\n+\n+def import_cvs_symbol (cvs_symbol, progress):\n+\t\"\"\"Import the given CVS symbol from CVS to Git\n+\n+\tReturn False if nothing was imported, True otherwise.\n+\t\"\"\"\n+\tprogress.pushprefix(\"%s: \" % (cvs_symbol))\n+\n+\tgit_ref = cvs_to_refname(cvs_symbol)\n+\n+\t# Verify that we are asked to import valid git ref names\n+\tif not valid_git_ref(git_ref):\n+\t\tprogress(\"Invalid git ref '%s'. Skipping.\" % (git_ref), True)\n+\t\tprogress.popprefix()\n+\t\treturn False\n+\n+\t# Retrieve previously imported CVS state\n+\tprogress(\"Loading previously imported state of %s...\" % (git_ref))\n+\tprev_commit = Globals.GitRefMap.get(git_ref)\n+\tprev_state = Globals.CommitStates.get(prev_commit, CvsState())\n+\n+\t# Retrieve current CVS state of symbol\n+\t# Also: At some point we will need mode information for all CVS paths\n+\t# (stored in CvsRevisionMap). This information can be added for each\n+\t# path on demand (using CvsWorkDir.get_modeinfo()), but doing so may\n+\t# be an expensive process. It is much cheaper to load mode information\n+\t# for as many paths as possible in a _single_ operation. We do this\n+\t# below, by calling CvsRevisionMap.sync_modeinfo_from_cvs() in\n+\t# appropriate places\n+\tif Config.CachedSymbolsOnly:\n+\t\tprogress(\"Synchronizing local CVS symbol cache for symbol...\")\n+\t\t# The symbol cache is likely not up-to-date. Synchronize the\n+\t\t# given CVS symbol explicitly, to make sure we get the version\n+\t\t# current with the CVS server.\n+\t\tGlobals.CvsSymbolCache.sync_symbol(\n+\t\t\tcvs_symbol, Globals.CvsWorkDir, progress)\n+\n+\t\t# The above method updates the CVS workdir to the current CVS\n+\t\t# version. Hence, now is a convenient time to preload mode info\n+\t\t# from the currently checked-out CVS files. There may be more\n+\t\t# files for which we'll need mode information, but we'll deal\n+\t\t# with those when needed.\n+\t\tprogress(\"Updating path mode info from current CVS checkout.\")\n+\t\tGlobals.CvsRevisionMap.sync_modeinfo_from_cvs(\n+\t\t\tGlobals.CvsWorkDir)\n+\telif not Globals.CvsRevisionMap: # There is no info for any paths, yet\n+\t\t# Pure optimization: We didn't get to preload all the mode info\n+\t\t# above. Normally, the only alternative is load mode info for\n+\t\t# each path on-demand. However, if our CvsRevisionMap is\n+\t\t# currently empty, that's probably going to be very expensive.\n+\t\t# Therefore, in this case, do an explicit CVS update here, and\n+\t\t# preload mode info for all paths.\n+\t\tprogress(\"Updating CVS checkout to sync path mode info.\")\n+\t\tGlobals.CvsWorkDir.update(cvs_symbol)\n+\t\tGlobals.CvsRevisionMap.sync_modeinfo_from_cvs(\n+\t\t\tGlobals.CvsWorkDir)\n+\n+\tprogress(\"Loading current CVS state...\")\n+\ttry: cur_state = Globals.CvsSymbolCache[cvs_symbol]\n+\texcept KeyError:\n+\t\tprogress(\"Couldn't find symbol '%s'. Skipping.\" % (cvs_symbol),\n+\t\t\tTrue)\n+\t\tprogress.popprefix()\n+\t\treturn False\n+\n+\t# Optimization: Check if the previous import of this symbol is still\n+\t# up-to-date. If so, there's nothing more to be done.\n+\tprogress(\"Checking if we're already up-to-date...\")\n+\tif cur_state == prev_state:\n+\t\tprogress(\"Already up-to-date. Skipping.\", True)\n+\t\tprogress.popprefix()\n+\t\treturn False\n+\n+\tprogress(\"Fetching CVS revisions...\")\n+\tcvs_revs = import_cvs_revs(cvs_symbol, prev_state, cur_state, progress)\n+\n+\t# Organize CvsRevs into a chronological list of changesets\n+\tprogress(\"Organizing revisions into changesets...\")\n+\tchangesets = build_changesets_from_revs(cvs_revs)\n+\n+\t# When importing a new branch, try to optimize branch start point,\n+\t# instead of importing entire branch from scratch\n+\tif prev_commit is None:\n+\t\tprogress(\"Finding startpoint for new symbol...\")\n+\t\ti = len(changesets)\n+\t\tstate = cur_state.copy()\n+\t\tfor c in reversed(changesets):\n+\t\t\tcommit = Globals.CvsStateMap.get_exact_commit(\n+\t\t\t\tstate, Globals.CommitStates)\n+\t\t\tif commit is not None:\n+\t\t\t\t# We have found a commit that exactly matches the state after commit #i (changesets[i - 1])\n+\t\t\t\tGlobals.GitFastImport.reset(git_ref, commit)\n+\t\t\t\tchangesets = changesets[i:]\n+\t\t\t\tbreak\n+\t\t\trevert_state(state, c)\n+\t\t\ti -= 1\n+\n+\tnum_changesets = len(changesets)\n+\tnum_applied = 0\n+\n+\n+\t# Apply changesets, bringing git_ref from prev_state to cur_state\n+\tif num_changesets:\n+\t\tprogress(\"Importing changesets...\")\n+\t\tnum_applied = import_changesets(git_ref, changesets, prev_state,\n+\t\t\tcur_state, progress)\n+\n+\tprogress(\"Imported %i changesets (reused %i existing changesets)\" % (\n+\t\tnum_applied, num_changesets - num_applied), True)\n+\tprogress.popprefix()\n+\treturn True\n+\n+def do_import (*args):\n+\t\"\"\"Do the 'import' command; import refs from a remote\"\"\"\n+\tif not args: die_usage(\"'import' takes at least one parameter: ref...\")\n+\n+\tprogress = ProgressIndicator(\" \", sys.stderr)\n+\n+\tcvs_symbols = map(ref_to_cvsname, args)\n+\tempty_import = True\n+\n+\tfor symbol in cvs_symbols:\n+\t\tif import_cvs_symbol(symbol, progress):\n+\t\t\tempty_import = False\n+\n+\tif empty_import:\n+\t\tprogress.finish(\"Everything up-to-date\", True)\n+\t\treturn 0\n+\n+\tprogress.finish(\"Finished importing %i CVS symbols to Git\" % (\n+\t\tlen(cvs_symbols)), True)\n+\treturn 0\n+\n+def do_list (*args):\n+\t\"\"\"Do the 'list' command; list refs available from a CVS remote\"\"\"\n+\tif args: die_usage(\"'list' takes no parameters\")\n+\n+\tprogress = ProgressIndicator(\" \", sys.stderr)\n+\n+\tif Config.CachedSymbolsOnly:\n+\t\tprogress(\"Listing symbols in local symbol cache...\", True)\n+\t\tfor symbol in sorted(Globals.CvsSymbolCache):\n+\t\t\tprint cvs_to_refname(symbol)\n+\t\tprogress.finish()\n+\t\tprint # terminate output with blank line\n+\t\treturn 0\n+\n+\t# Synchronize local symbol cache with CVS server\n+\tprogress(\"Synchronizing local symbol cache with CVS server...\")\n+\tGlobals.CvsSymbolCache.sync_all_symbols(Config.CvsRepo, progress,\n+\t\tvalid_cvs_symbol)\n+\n+\t# Load current states of Git refs\n+\tprogress(\"Loading current state of Git refs...\")\n+\tchanged, unchanged = 0, 0\n+\tfor cvs_symbol, cvs_state in sorted(Globals.CvsSymbolCache.items()):\n+\t\tgit_ref = cvs_to_refname(cvs_symbol)\n+\t\tprogress(\"\\tChecking if Git ref is up-to-date: %s\" % (git_ref))\n+\t\tgit_commit = Globals.GitRefMap.get(git_ref)\n+\t\tgit_state = Globals.CommitStates.get(git_commit)\n+\t\tattrs = \"\"\n+\t\tif git_state and git_state == cvs_state:\n+\t\t\tattrs = \" unchanged\"\n+\t\t\tunchanged += 1\n+\t\telse:\n+\t\t\tgit_commit = \"?\"\n+\t\t\tchanged += 1\n+\t\tprint \"%s %s%s\" % (git_commit, git_ref, attrs)\n+\n+\tprogress.finish(\"Found %i CVS symbols (%i changed, %i unchanged)\" % (\n+\t changed + unchanged, changed, unchanged))\n+\tprint # terminate with blank line\n+\treturn 0\n+\n+def do_capabilities (*args):\n+\t\"\"\"Do the 'capabilities' command; report supported features\"\"\"\n+\tif args: die_usage(\"'capabilities' takes no parameters\")\n+\tprint \"import\"\n+\tprint \"marks %s\" % (work_path(\"marks\"))\n+#\tprint \"export\"\n+#\tprint \"export-branch\"\n+#\tprint \"export-merges\"\n+\tprint # terminate with blank line\n+\treturn 0\n+\n+def do_addsymbol (*args):\n+\t\"\"\"Do the 'addsymbol' command; add given CVS symbol to local cache\"\"\"\n+\tif len(args) != 1: die_usage(\"'addsymbol' takes one parameter: symbol\")\n+\tsymbol = args[0]\n+\n+\tprogress = ProgressIndicator(\" \", sys.stderr)\n+\tif valid_cvs_symbol(symbol):\n+\t\tGlobals.CvsSymbolCache.sync_symbol(\n+\t\t\tsymbol, Globals.CvsWorkDir, progress)\n+\t\tprogress.finish(\"Added '%s' to CVS symbol cache\" % (symbol),\n+\t\t\tTrue)\n+\telse:\n+\t\terror(\"Skipping CVS symbol '%s'; it is not a valid git ref\",\n+\t\t\tsymbol)\n+\n+\tprint # terminate with blank line\n+\treturn 0\n+\n+def do_syncsymbols (*args):\n+\t\"\"\"Do the 'syncsymbols' command; sync all symbols with CVS server\"\"\"\n+\tif args: die_usage(\"'syncsymbols' takes no parameters\")\n+\tprogress = ProgressIndicator(\" \", sys.stderr)\n+\tGlobals.CvsSymbolCache.sync_all_symbols(Config.CvsRepo, progress,\n+\t\tvalid_cvs_symbol)\n+\tprogress.finish()\n+\tprint # terminate with blank line\n+\treturn 0\n+\n+def do_verify (*args):\n+\t\"\"\"Do the 'verify' command; Compare CVS checkout and Git tree\"\"\"\n+\tif len(args) != 1: die_usage(\"'verify' takes one parameter: symbol\")\n+\tsymbol = args[0]\n+\tgitref = cvs_to_refname(symbol)\n+\n+\tprogress = ProgressIndicator(\" \", sys.stderr)\n+\tassert valid_cvs_symbol(symbol)\n+\n+\tprogress(\"Checking out '%s' from CVS...\" % (symbol))\n+\tGlobals.CvsWorkDir.update(symbol)\n+\n+\tadd_env = {\"GIT_INDEX_FILE\": os.path.abspath(work_path(\"temp_index\"))}\n+\tprogress(\"Creating Git index from tree object @ '%s'...\" % (gitref))\n+\tcmd = (\"git\", \"read-tree\", gitref)\n+\tassert run_command(cmd, add_env = add_env)[0] == 0\n+\n+\tprogress(\"Comparing CVS checkout to Git index...\", True)\n+\tcmd = (\"git\", \"--work-tree=%s\" % (os.path.abspath(work_path(\"cvs\"))),\n+\t\t\"ls-files\",\n+\t\t\"--exclude=CVS\", \"--deleted\", \"--modified\", \"--others\", \"-t\")\n+\texit_code, output, errors = run_command(cmd, add_env = add_env)\n+\tassert exit_code == 0 and not errors\n+\n+\tif output:\n+\t\tprogress.finish(\"Failed verification of '%s'\" % (symbol), True)\n+\t\terror(\"The '%s' command returned:\\n---\\n%s---\", \" \".join(cmd),\n+\t\t\toutput)\n+\telse:\n+\t\tprogress.finish(\"Successfully verified '%s'\" % (symbol), True)\n+\n+\tprint # terminate with blank line\n+\treturn exit_code\n+\n+def not_implemented (*args):\n+\tdie_usage(\"Command not implemented\")\n+\n+Commands = {\n+\t\"capabilities\": do_capabilities,\n+\t\"list\": do_list,\n+\t# Special handling of 'import' in main()\n+\t# \"import\": do_import,\n+\t\"export\": not_implemented,\n+\t# Custom commands\n+\t\"addsymbol\": do_addsymbol,\n+\t\"syncsymbols\": do_syncsymbols,\n+\t\"verify\": do_verify,\n+}\n+\n+class Globals (object):\n+\t\"\"\"Global variables are placed here at the start of main()\"\"\"\n+\tpass\n+\n+def main (*args):\n+\tdebug(\"Invoked '%s'\", \" \".join(args))\n+\n+\t### Initialization of subsystems\n+\n+\t# Read config for the given remote\n+\tassert len(args) >= 2\n+\tConfig.init(args[1])\n+\n+\t# Local CVS symbol cache (CVS symbol -> CVS state mapping)\n+\tGlobals.CvsSymbolCache = CvsSymbolCache(work_path(\"symbols\"))\n+\n+\t# Local CVS checkout\n+\tGlobals.CvsWorkDir = CvsWorkDir(work_path(\"cvs\"), Config.CvsRepo)\n+\n+\t# Interface to 'git cat-file --batch'\n+\tGlobals.GitObjectFetcher = GitObjectFetcher()\n+\n+\t# Interface to Git object notes\n+\tGlobals.GitNotes = GitNotes(Config.NotesRef, Globals.GitObjectFetcher)\n+\n+\t# Mapping from Git commit objects to CVS states\n+\tGlobals.CommitStates = CommitStates(Globals.GitNotes)\n+\n+\t# Mapping from Git ref names to Git object names\n+\tGlobals.GitRefMap = GitRefMap(Globals.GitObjectFetcher)\n+\n+\t# Mapping from CVS revision to Git blob and commit objects\n+\tGlobals.CvsRevisionMap = CvsRevisionMap(\n+\t\tcvs_to_refname(\"_metadata\"), Globals.GitObjectFetcher)\n+\tlast_mark = 0\n+\tif Globals.CvsRevisionMap.has_unresolved_marks():\n+\t\t# Update with marks from last import\n+\t\tlast_mark = Globals.CvsRevisionMap.load_marks_file(\n+\t\t\twork_path(\"marks\"))\n+\telse:\n+\t\t# Truncate marks file. We cannot automatically do this after\n+\t\t# .load_marks_file() above, since we cannot yet guarantee that\n+\t\t# we will be able to save the revision map persistently. (That\n+\t\t# can only happen if we are given one or more import commands\n+\t\t# below.) We can only truncate this file when we know there are\n+\t\t# no unresolved marks in the revision map.\n+\t\topen(work_path(\"marks\"), \"w\").close()\n+\n+\t# Mapping from CVS states to commit objects that contain said state\n+\tGlobals.CvsStateMap = CvsStateMap(Globals.CvsRevisionMap)\n+\n+\t### Main program loop\n+\n+\timport_refs = [] # accumulate import commands here\n+\t# cannot use \"for line in sys.stdin\" for buffering (?) reasons\n+\tline = sys.stdin.readline()\n+\twhile (line):\n+\t\tcmdline = line.strip().split()\n+\t\tif not cmdline: break # blank line means we're about to quit\n+\n+\t\tdebug(\"Got command '%s'\", \" \".join(cmdline))\n+\t\tcmd = cmdline.pop(0)\n+\n+\t\tif cmd == \"import\":\n+\t\t\timport_refs.extend(cmdline)\n+\t\telse:\n+\t\t\tif cmd not in Commands:\n+\t\t\t\tdie_usage(\"Unknown command '%s'\", cmd)\n+\t\t\tif Commands[cmd](*cmdline):\n+\t\t\t\tdie(\"Command '%s' failed\", line.strip())\n+\t\t\tsys.stdout.flush()\n+\t\tline = sys.stdin.readline()\n+\n+\tret = 0\n+\tif import_refs: # trigger import processing after last import command\n+\t\t# Init producer of output in the git-fast-import format\n+\t\tGlobals.GitFastImport = GitFastImport(\n+\t\t\tsys.stdout, Globals.GitObjectFetcher, last_mark)\n+\n+\t\t# Perform import of given refs\n+\t\tret = do_import(*import_refs)\n+\n+\t\t### Notes on persistent storage of subsystems' data structures:\n+\t\t#\n+\t\t# Because the \"import\" command has been called, we here _know_\n+\t\t# that there is a fast-import process running in parallel.\n+\t\t# (This is NOT the case when there are no \"import\" commands).\n+\t\t# We can therefore now (and only now) safely commit the extra\n+\t\t# information that we store in the Git repo.\n+\t\t# In other words, the data structures that we commit to\n+\t\t# persistent storage with the following calls will NOT be\n+\t\t# committed if there are no \"import\" commands. The data\n+\t\t# structures must handle this in one of two ways:\n+\t\t# - In the no-\"import\" scenario, there is simply nothing to\n+\t\t# commit, so it can safely be skipped.\n+\t\t# - Any information that should have been committed in the\n+\t\t# no-\"import\" scenario can be reconstructed repeatedly in\n+\t\t# subsequent executions of this program, until the next\n+\t\t# invocation of an \"import\" command provides an opportunity\n+\t\t# to commit the data structure to persistent storage.\n+\n+\t\t# Write out commit notes (mapping git commits to CvsStates)\n+\t\t# The following call would be a no-op in the no-\"import\" case\n+\t\tGlobals.GitNotes.commit_notes(\n+\t\t\tGlobals.GitFastImport, Config.Author,\n+\t\t\t'Annotate commits imported by \"git remote-cvs\"\\n')\n+\n+\t\t# Save CVS revision metadata\n+\t\t# This data structure can handle the no-\"import\" case as long\n+\t\t# as the marks file from the last fast-import run is still\n+\t\t# present upon the next execution of this program.\n+\t\tGlobals.CvsRevisionMap.commit_map(\n+\t\t\tGlobals.GitFastImport, Config.Author,\n+\t\t\t'Updated metadata used by \"git remote-cvs\"\\n')\n+\n+\treturn ret\n+\n+if __name__ == '__main__':\n+\tsys.exit(main(*sys.argv))\n-- \n1.6.4.rc3.138.ga6b98.dirty\n"},{"id":"120337","messageId":"1250036031-32272-5-git-send-email-johan@herland.net","threadId":"20543","inReplyTo":"1250036031-32272-1-git-send-email-johan@herland.net","subject":"[RFCv3 4/4] Add simple selftests of git-remote-cvs functionality","fromName":"Johan Herland","fromEmail":"johan@herland.net","sentAt":"2009-08-12T00:13:51Z","receivedAt":"2009-08-12T00:13:51Z","isPatch":false,"sender":{"key":"johan@herland.net","avatar":"https://avatars.githubusercontent.com/u/547031?v=4"},"body":"Add two new selftests:\n\n- t9800-remote-cvs-basic: Test the git-remote-cvs implementation of the\n remote helper API, by verifying the expected output of the git-remote-cvs\n program when invoking remote helper API commands while doing some simple\n CVS operations.\n\n- t9801-remote-cvs-fetch: A more high-level test of the fetch/import-side\n of the git-remote-cvs implementation, by verifying the expected repository\n state after doing \"git fetch\" from a CVS remote where a variety of CVS\n operations are being performed.\n\nSigned-off-by: Johan Herland \n---\n t/t9800-remote-cvs-basic.sh | 524 +++++++++++++++++++++++++++++++++++++++++++\n t/t9801-remote-cvs-fetch.sh | 291 ++++++++++++++++++++++++\n 2 files changed, 815 insertions(+), 0 deletions(-)\n create mode 100755 t/t9800-remote-cvs-basic.sh\n create mode 100755 t/t9801-remote-cvs-fetch.sh\n\ndiff --git a/t/t9800-remote-cvs-basic.sh b/t/t9800-remote-cvs-basic.sh\nnew file mode 100755\nindex 0000000..6ddec17\n--- /dev/null\n+++ b/t/t9800-remote-cvs-basic.sh\n@@ -0,0 +1,524 @@\n+#!/bin/sh\n+\n+test_description='git remote-cvs basic tests'\n+. ./test-lib.sh\n+\n+if ! test_have_prereq PYTHON; then\n+\tsay 'skipping CVS foreign-vcs helper tests, python not available'\n+\ttest_done\n+fi\n+\n+CVS_EXEC=cvs\n+CVS_OPTS=\"-f -q\"\n+CVS=\"$CVS_EXEC $CVS_OPTS\"\n+\n+CVSROOT=$(pwd)/cvsroot\n+export CVSROOT\n+unset CVS_SERVER\n+\n+CVSMODULE=cvsmodule\n+GITREMOTE=cvsremote\n+\n+if ! type $CVS_EXEC >/dev/null 2>&1\n+then\n+\tsay 'skipping remote-cvs tests, $CVS_EXEC not found'\n+\ttest_done\n+fi\n+\n+test_expect_success 'setup cvsroot' '$CVS init'\n+\n+test_expect_success '#1: setup a cvs module' '\n+\n+\tmkdir \"$CVSROOT/$CVSMODULE\" &&\n+\t$CVS co -d module-cvs $CVSMODULE &&\n+\t(\n+\t\tcd module-cvs &&\n+\t\tcat <o_fortuna &&\n+O Fortuna\n+velut luna\n+statu variabilis,\n+\n+semper crescis\n+aut decrescis;\n+vita detestabilis\n+\n+nunc obdurat\n+et tunc curat\n+ludo mentis aciem,\n+\n+egestatem,\n+potestatem\n+dissolvit ut glaciem.\n+EOF\n+\t\t$CVS add o_fortuna &&\n+\t\tcat <message &&\n+add \"O Fortuna\" lyrics\n+\n+These public domain lyrics make an excellent sample text.\n+EOF\n+\t\t$CVS commit -f -F message o_fortuna\n+\t)\n+'\n+\n+test_expect_success 'set up CVS repo as a foreign remote' '\n+\n+\tgit config \"user.name\" \"Test User\"\n+\tgit config \"user.email\" \"test@example.com\"\n+\tgit config \"remote.$GITREMOTE.vcs\" cvs\n+\tgit config \"remote.$GITREMOTE.cvsRoot\" \"$CVSROOT\"\n+\tgit config \"remote.$GITREMOTE.cvsModule\" \"$CVSMODULE\"\n+\tgit config \"remote.$GITREMOTE.fetch\" \\\n+\t\t\"+refs/cvs/$GITREMOTE/*:refs/remotes/$GITREMOTE/*\"\n+\n+'\n+\n+test_expect_success '#1: git-remote-cvs \"capabilities\" command' '\n+\n+\techo \"capabilities\" | git remote-cvs \"$GITREMOTE\" > actual &&\n+\tcat <expect &&\n+import\n+marks .git/info/cvs/$GITREMOTE/marks\n+\n+EOF\n+\ttest_cmp expect actual\n+\n+'\n+\n+test_expect_success '#1: git-remote-cvs \"list\" command' '\n+\n+\techo \"list\" | git remote-cvs \"$GITREMOTE\" > actual &&\n+\tcat <expect &&\n+? refs/cvs/$GITREMOTE/HEAD\n+\n+EOF\n+\ttest_cmp expect actual\n+\n+'\n+\n+test_expect_success '#1: git-remote-cvs \"import\" command' '\n+\n+\techo \"import refs/cvs/$GITREMOTE/HEAD\" | git remote-cvs \"$GITREMOTE\" > actual &&\n+\tcat <expect &&\n+# Importing CVS revision o_fortuna:1.1\n+blob\n+mark :1\n+data 180\n+O Fortuna\n+velut luna\n+statu variabilis,\n+\n+semper crescis\n+aut decrescis;\n+vita detestabilis\n+\n+nunc obdurat\n+et tunc curat\n+ludo mentis aciem,\n+\n+egestatem,\n+potestatem\n+dissolvit ut glaciem.\n+\n+commit refs/cvs/$GITREMOTE/HEAD\n+mark :2\n+data 82\n+add \"O Fortuna\" lyrics\n+\n+These public domain lyrics make an excellent sample text.\n+\n+M 644 :1 o_fortuna\n+\n+# Importing note for object 2\n+blob\n+mark :3\n+data 14\n+o_fortuna:1.1\n+\n+commit refs/notes/cvs/$GITREMOTE\n+mark :4\n+data 46\n+Annotate commits imported by \"git remote-cvs\"\n+\n+N :3 :2\n+\n+blob\n+mark :5\n+data 32\n+1 o_fortuna:1.1\n+2 o_fortuna:1.1\n+\n+blob\n+mark :6\n+data 16\n+blob 1\n+commit 2\n+\n+commit refs/cvs/$GITREMOTE/_metadata\n+mark :7\n+data 42\n+Updated metadata used by \"git remote-cvs\"\n+\n+M 644 :5 CVS/marks\n+M 644 :6 o_fortuna/1.1\n+\n+EOF\n+\tgrep -v \"^committer \" actual > actual.filtered &&\n+\ttest_cmp expect actual.filtered\n+\n+'\n+\n+test_expect_success '#1: Passing git-remote-cvs output to git-fast-import' '\n+\n+\tgit fast-import --quiet \\\n+\t\t--export-marks=\".git/info/cvs/$GITREMOTE/marks\" \\\n+\t\t< actual &&\n+\tgit gc\n+\n+'\n+\n+test_expect_success '#1: Verifying correctness of import' '\n+\n+\techo \"verify HEAD\" | git remote-cvs \"$GITREMOTE\"\n+\n+'\n+\n+test_expect_success '#2: update cvs module' '\n+\n+\t(\n+\t\tcd module-cvs &&\n+\t\tcat <o_fortuna &&\n+O Fortune,\n+like the moon\n+you are changeable,\n+\n+ever waxing\n+and waning;\n+hateful life\n+\n+first oppresses\n+and then soothes\n+as fancy takes it;\n+\n+poverty\n+and power\n+it melts them like ice.\n+EOF\n+\t\tcat <message &&\n+translate to English\n+\n+My Latin is terrible.\n+EOF\n+\t\t$CVS commit -f -F message o_fortuna\n+\t)\n+'\n+\n+test_expect_success '#2: git-remote-cvs \"capabilities\" command' '\n+\n+\techo \"capabilities\" | git remote-cvs \"$GITREMOTE\" > actual &&\n+\tcat <expect &&\n+import\n+marks .git/info/cvs/$GITREMOTE/marks\n+\n+EOF\n+\ttest_cmp expect actual\n+\n+'\n+\n+test_expect_success '#2: git-remote-cvs \"list\" command' '\n+\n+\techo \"list\" | git remote-cvs \"$GITREMOTE\" > actual &&\n+\tcat <expect &&\n+? refs/cvs/$GITREMOTE/HEAD\n+\n+EOF\n+\ttest_cmp expect actual\n+\n+'\n+\n+test_expect_success '#2: git-remote-cvs \"import\" command' '\n+\n+\techo \"import refs/cvs/$GITREMOTE/HEAD\" | git remote-cvs \"$GITREMOTE\" > actual &&\n+\tcat <expect &&\n+# Importing CVS revision o_fortuna:1.2\n+blob\n+mark :8\n+data 179\n+O Fortune,\n+like the moon\n+you are changeable,\n+\n+ever waxing\n+and waning;\n+hateful life\n+\n+first oppresses\n+and then soothes\n+as fancy takes it;\n+\n+poverty\n+and power\n+it melts them like ice.\n+\n+commit refs/cvs/$GITREMOTE/HEAD\n+mark :9\n+data 44\n+translate to English\n+\n+My Latin is terrible.\n+\n+from refs/cvs/$GITREMOTE/HEAD^0\n+M 644 :8 o_fortuna\n+\n+# Importing note for object 9\n+blob\n+mark :10\n+data 14\n+o_fortuna:1.2\n+\n+commit refs/notes/cvs/$GITREMOTE\n+mark :11\n+data 46\n+Annotate commits imported by \"git remote-cvs\"\n+\n+from refs/notes/cvs/$GITREMOTE^0\n+N :10 :9\n+\n+blob\n+mark :12\n+data 32\n+8 o_fortuna:1.2\n+9 o_fortuna:1.2\n+\n+blob\n+mark :13\n+data 94\n+\n+blob\n+mark :14\n+data 16\n+blob 8\n+commit 9\n+\n+commit refs/cvs/$GITREMOTE/_metadata\n+mark :15\n+data 42\n+Updated metadata used by \"git remote-cvs\"\n+\n+from refs/cvs/$GITREMOTE/_metadata^0\n+M 644 :12 CVS/marks\n+M 644 :13 o_fortuna/1.1\n+M 644 :14 o_fortuna/1.2\n+\n+EOF\n+\tgrep -v -e \"^committer \" -e \"\\b[0-9a-f]\\{40\\}\\b\" actual > actual.filtered &&\n+\ttest_cmp expect actual.filtered\n+\n+'\n+\n+test_expect_success '#2: Passing git-remote-cvs output to git-fast-import' '\n+\n+\tgit fast-import --quiet \\\n+\t\t--import-marks=\".git/info/cvs/$GITREMOTE/marks\" \\\n+\t\t--export-marks=\".git/info/cvs/$GITREMOTE/marks\" \\\n+\t\t< actual &&\n+\tgit gc\n+\n+'\n+\n+test_expect_success '#2: Verifying correctness of import' '\n+\n+\techo \"verify HEAD\" | git remote-cvs \"$GITREMOTE\"\n+\n+'\n+\n+test_expect_success '#3: update cvs module' '\n+\n+\t(\n+\t\tcd module-cvs &&\n+\t\techo 1 >tick &&\n+\t\t$CVS add tick &&\n+\t\t$CVS commit -f -m 1 tick\n+\t)\n+\n+'\n+\n+test_expect_success '#3: git-remote-cvs \"capabilities\" command' '\n+\n+\techo \"capabilities\" | git remote-cvs \"$GITREMOTE\" > actual &&\n+\tcat <expect &&\n+import\n+marks .git/info/cvs/$GITREMOTE/marks\n+\n+EOF\n+\ttest_cmp expect actual\n+\n+'\n+\n+test_expect_success '#3: git-remote-cvs \"list\" command' '\n+\n+\techo \"list\" | git remote-cvs \"$GITREMOTE\" > actual &&\n+\tcat <expect &&\n+? refs/cvs/$GITREMOTE/HEAD\n+\n+EOF\n+\ttest_cmp expect actual\n+\n+'\n+\n+test_expect_success '#3: git-remote-cvs \"import\" command' '\n+\n+\techo \"import refs/cvs/$GITREMOTE/HEAD\" | git remote-cvs \"$GITREMOTE\" > actual &&\n+\tcat <expect &&\n+# Importing CVS revision tick:1.1\n+blob\n+mark :16\n+data 2\n+1\n+\n+commit refs/cvs/$GITREMOTE/HEAD\n+mark :17\n+data 2\n+1\n+\n+from refs/cvs/$GITREMOTE/HEAD^0\n+M 644 :16 tick\n+\n+# Importing note for object 17\n+blob\n+mark :18\n+data 23\n+o_fortuna:1.2\n+tick:1.1\n+\n+commit refs/notes/cvs/$GITREMOTE\n+mark :19\n+data 46\n+Annotate commits imported by \"git remote-cvs\"\n+\n+from refs/notes/cvs/$GITREMOTE^0\n+N :18 :17\n+\n+blob\n+mark :20\n+data 41\n+16 tick:1.1\n+17 tick:1.1\n+17 o_fortuna:1.2\n+\n+blob\n+mark :21\n+data 104\n+commit 17\n+\n+blob\n+mark :22\n+data 18\n+blob 16\n+commit 17\n+\n+commit refs/cvs/$GITREMOTE/_metadata\n+mark :23\n+data 42\n+Updated metadata used by \"git remote-cvs\"\n+\n+from refs/cvs/$GITREMOTE/_metadata^0\n+M 644 :20 CVS/marks\n+M 644 :21 o_fortuna/1.2\n+M 644 :22 tick/1.1\n+\n+EOF\n+\tgrep -v -e \"^committer \" -e \"\\b[0-9a-f]\\{40\\}\\b\" actual > actual.filtered &&\n+\ttest_cmp expect actual.filtered\n+\n+'\n+\n+test_expect_success '#3: Passing git-remote-cvs output to git-fast-import' '\n+\n+\tgit fast-import --quiet \\\n+\t\t--import-marks=\".git/info/cvs/$GITREMOTE/marks\" \\\n+\t\t--export-marks=\".git/info/cvs/$GITREMOTE/marks\" \\\n+\t\t< actual &&\n+\tgit gc\n+\n+'\n+\n+test_expect_success '#3: Verifying correctness of import' '\n+\n+\techo \"verify HEAD\" | git remote-cvs \"$GITREMOTE\"\n+\n+'\n+\n+test_expect_success '#4: git-remote-cvs \"capabilities\" command' '\n+\n+\techo \"capabilities\" | git remote-cvs \"$GITREMOTE\" > actual &&\n+\tcat <expect &&\n+import\n+marks .git/info/cvs/$GITREMOTE/marks\n+\n+EOF\n+\ttest_cmp expect actual\n+\n+'\n+\n+commit=$(git rev-parse \"refs/cvs/$GITREMOTE/HEAD\")\n+\n+test_expect_success '#4: git-remote-cvs \"list\" command' '\n+\n+\techo \"list\" | git remote-cvs \"$GITREMOTE\" > actual &&\n+\tcat <expect &&\n+$commit refs/cvs/$GITREMOTE/HEAD unchanged\n+\n+EOF\n+\ttest_cmp expect actual\n+\n+'\n+\n+test_expect_success '#4: git-remote-cvs \"import\" command' '\n+\n+\techo \"import refs/cvs/$GITREMOTE/HEAD\" | git remote-cvs \"$GITREMOTE\" > actual &&\n+\tcat <expect &&\n+blob\n+mark :24\n+data 0\n+\n+blob\n+mark :25\n+data 142\n+\n+blob\n+mark :26\n+data 94\n+\n+commit refs/cvs/$GITREMOTE/_metadata\n+mark :27\n+data 42\n+Updated metadata used by \"git remote-cvs\"\n+\n+from refs/cvs/$GITREMOTE/_metadata^0\n+M 644 :24 CVS/marks\n+M 644 :25 o_fortuna/1.2\n+M 644 :26 tick/1.1\n+\n+EOF\n+\tgrep -v -e \"^committer \" -e \"\\b[0-9a-f]\\{40\\}\\b\" actual > actual.filtered &&\n+\ttest_cmp expect actual.filtered\n+\n+'\n+\n+test_expect_success '#4: Passing git-remote-cvs output to git-fast-import' '\n+\n+\tgit fast-import --quiet \\\n+\t\t--import-marks=\".git/info/cvs/$GITREMOTE/marks\" \\\n+\t\t--export-marks=\".git/info/cvs/$GITREMOTE/marks\" \\\n+\t\t< actual &&\n+\tgit gc\n+\n+'\n+\n+test_expect_success '#4: Verifying correctness of import' '\n+\n+\techo \"verify HEAD\" | git remote-cvs \"$GITREMOTE\"\n+\n+'\n+\n+test_done\ndiff --git a/t/t9801-remote-cvs-fetch.sh b/t/t9801-remote-cvs-fetch.sh\nnew file mode 100755\nindex 0000000..93d44a7\n--- /dev/null\n+++ b/t/t9801-remote-cvs-fetch.sh\n@@ -0,0 +1,291 @@\n+#!/bin/sh\n+\n+test_description='git remote-cvs basic tests'\n+. ./test-lib.sh\n+\n+if ! test_have_prereq PYTHON; then\n+\tsay 'skipping CVS foreign-vcs helper tests, python not available'\n+\ttest_done\n+fi\n+\n+CVS_EXEC=cvs\n+CVS_OPTS=\"-f -q\"\n+CVS=\"$CVS_EXEC $CVS_OPTS\"\n+\n+CVSROOT=$(pwd)/cvsroot\n+export CVSROOT\n+unset CVS_SERVER\n+\n+CVSMODULE=cvsmodule\n+GITREMOTE=cvsremote\n+\n+if ! type $CVS_EXEC >/dev/null 2>&1\n+then\n+\tsay 'skipping remote-cvs tests, $CVS_EXEC not found'\n+\ttest_done\n+fi\n+\n+verify () {\n+\tgit log --reverse --format=\"--- %T%n%s%n%n%b\" \"$GITREMOTE/$1\" >actual &&\n+\ttest_cmp \"expect.$1\" actual &&\n+\techo \"verify $1\" | git remote-cvs \"$GITREMOTE\"\n+}\n+\n+test_expect_success 'setup CVS repo' '$CVS init'\n+\n+test_expect_success 'create CVS module with initial commit' '\n+\n+\tmkdir \"$CVSROOT/$CVSMODULE\" &&\n+\t$CVS co -d module-cvs $CVSMODULE &&\n+\t(\n+\t\tcd module-cvs &&\n+\t\tcat <o_fortuna &&\n+O Fortuna\n+velut luna\n+statu variabilis,\n+\n+semper crescis\n+aut decrescis;\n+vita detestabilis\n+\n+nunc obdurat\n+et tunc curat\n+ludo mentis aciem,\n+\n+egestatem,\n+potestatem\n+dissolvit ut glaciem.\n+EOF\n+\t\t$CVS add o_fortuna &&\n+\t\tcat <message &&\n+add \"O Fortuna\" lyrics\n+\n+These public domain lyrics make an excellent sample text.\n+EOF\n+\t\t$CVS commit -f -F message o_fortuna\n+\t)\n+'\n+\n+test_expect_success 'set up CVS repo/module as a foreign remote' '\n+\n+\tgit config \"user.name\" \"Test User\"\n+\tgit config \"user.email\" \"test@example.com\"\n+\tgit config \"remote.$GITREMOTE.vcs\" cvs\n+\tgit config \"remote.$GITREMOTE.cvsRoot\" \"$CVSROOT\"\n+\tgit config \"remote.$GITREMOTE.cvsModule\" \"$CVSMODULE\"\n+\tgit config \"remote.$GITREMOTE.fetch\" \\\n+\t\t\"+refs/cvs/$GITREMOTE/*:refs/remotes/$GITREMOTE/*\"\n+\n+'\n+\n+test_expect_success 'initial fetch from CVS remote' '\n+\n+\tcat <expect.HEAD &&\n+--- 0e06d780dedab23e683c686fb041daa9a84c936c\n+add \"O Fortuna\" lyrics\n+\n+These public domain lyrics make an excellent sample text.\n+\n+EOF\n+\tgit fetch \"$GITREMOTE\" &&\n+\tverify HEAD\n+\n+'\n+\n+test_expect_success 'CVS commit' '\n+\n+\t(\n+\t\tcd module-cvs &&\n+\t\tcat <o_fortuna &&\n+O Fortune,\n+like the moon\n+you are changeable,\n+\n+ever waxing\n+and waning;\n+hateful life\n+\n+first oppresses\n+and then soothes\n+as fancy takes it;\n+\n+poverty\n+and power\n+it melts them like ice.\n+EOF\n+\t\tcat <message &&\n+translate to English\n+\n+My Latin is terrible.\n+EOF\n+\t\t$CVS commit -f -F message o_fortuna\n+\t) &&\n+\tcat <>expect.HEAD &&\n+--- daa87269a5e00388135ad9542dc16ab6754466e5\n+translate to English\n+\n+My Latin is terrible.\n+\n+EOF\n+\tgit fetch \"$GITREMOTE\" &&\n+\tverify HEAD\n+\n+'\n+\n+test_expect_success 'CVS commit with new file' '\n+\n+\t(\n+\t\tcd module-cvs &&\n+\t\techo 1 >tick &&\n+\t\t$CVS add tick &&\n+\t\t$CVS commit -f -m 1 tick\n+\t) &&\n+\tcat <>expect.HEAD &&\n+--- 486935b4fccecea9b64cbed3a797ebbcbe2b7461\n+1\n+\n+\n+EOF\n+\tgit fetch \"$GITREMOTE\" &&\n+\tverify HEAD\n+\n+'\n+\n+test_expect_success 'fetch without CVS changes' '\n+\n+\tgit fetch \"$GITREMOTE\" &&\n+\tverify HEAD\n+\n+'\n+\n+test_expect_success 'add 2 CVS commits' '\n+\n+\t(\n+\t\tcd module-cvs &&\n+\t\techo 2 >tick &&\n+\t\t$CVS commit -f -m 2 tick &&\n+\t\techo 3 >tick &&\n+\t\t$CVS commit -f -m 3 tick\n+\t) &&\n+\tcat <>expect.HEAD &&\n+--- 83437ab3e57bf0a42915de5310e3419792b5a36f\n+2\n+\n+\n+--- 60fc50406a82dc6bd32dc6e5f7bd23e4c3cdf7ef\n+3\n+\n+\n+EOF\n+\tgit fetch \"$GITREMOTE\" &&\n+\tverify HEAD\n+\n+'\n+\n+test_expect_success 'CVS commit with removed file' '\n+\n+\t(\n+\t\tcd module-cvs &&\n+\t\t$CVS remove -f tick &&\n+\t\t$CVS commit -f -m \"remove file\" tick\n+\t) &&\n+\tcat <>expect.HEAD &&\n+--- daa87269a5e00388135ad9542dc16ab6754466e5\n+remove file\n+\n+\n+EOF\n+\tgit fetch \"$GITREMOTE\" &&\n+\tverify HEAD\n+\n+'\n+\n+test_expect_success 'CVS commit with several new files' '\n+\n+\t(\n+\t\tcd module-cvs &&\n+\t\techo spam >spam &&\n+\t\techo sausage >sausage &&\n+\t\techo eggs >eggs &&\n+\t\t$CVS add spam sausage eggs &&\n+\t\t$CVS commit -f -m \"spam, sausage, and eggs\" spam sausage eggs\n+\t) &&\n+\tcat <>expect.HEAD &&\n+--- 3190dfce44a6d5e9916b4870dbf8f37d1ca4ddaf\n+spam, sausage, and eggs\n+\n+\n+EOF\n+\tgit fetch \"$GITREMOTE\" &&\n+\tverify HEAD\n+\n+'\n+\n+test_expect_success 'new CVS branch' '\n+\n+\t(\n+\t\tcd module-cvs &&\n+\t\t$CVS tag -b foo\n+\t) &&\n+\tcp expect.HEAD expect.foo &&\n+\tgit fetch \"$GITREMOTE\" &&\n+\tverify HEAD &&\n+\tverify foo\n+\n+'\n+\n+test_expect_success 'CVS commit on branch' '\n+\n+\t(\n+\t\tcd module-cvs &&\n+\t\t$CVS up -r foo &&\n+\t\techo \"spam spam spam\" >spam &&\n+\t\t$CVS commit -f -m \"commit on branch foo\" spam\n+\t) &&\n+\tcat <>expect.foo &&\n+--- 1aba123e5c83898ce3a8b976cc6064d60246aef4\n+commit on branch foo\n+\n+\n+EOF\n+\tgit fetch \"$GITREMOTE\" &&\n+\tverify HEAD &&\n+\tverify foo\n+\n+'\n+\n+test_expect_success 'create CVS tag' '\n+\n+\t(\n+\t\tcd module-cvs &&\n+\t\t$CVS tag bar\n+\t) &&\n+\tcp expect.foo expect.bar &&\n+\tgit fetch \"$GITREMOTE\" &&\n+\tverify HEAD &&\n+\tverify foo &&\n+\tverify bar\n+\n+'\n+\n+test_expect_success 'another CVS commit on branch' '\n+\n+\t(\n+\t\tcd module-cvs &&\n+\t\techo \"spam spam spam spam spam spam\" >> spam &&\n+\t\t$CVS commit -f -m \"another commit on branch foo\" spam\n+\t) &&\n+\tcat <>expect.foo &&\n+--- 15a2635e76e8e5a5a8746021643de317452f2340\n+another commit on branch foo\n+\n+\n+EOF\n+\tgit fetch \"$GITREMOTE\" &&\n+\tverify HEAD &&\n+\tverify foo &&\n+\tverify bar\n+\n+'\n+\n+test_done\n-- \n1.6.4.rc3.138.ga6b98.dirty\n"},{"id":"120349","messageId":"20090812021017.GB62301@gmail.com","threadId":"20543","inReplyTo":"1250036031-32272-3-git-send-email-johan@herland.net","subject":"Re: [RFCv3 2/4] Add Python support library for CVS remote helper","fromName":"David Aguilar","fromEmail":"davvid@gmail.com","sentAt":"2009-08-12T02:10:27Z","receivedAt":"2009-08-12T02:10:27Z","isPatch":false,"sender":{"key":"davvid@gmail.com","avatar":"https://avatars.githubusercontent.com/u/13196?v=4"},"body":"On Wed, Aug 12, 2009 at 02:13:49AM +0200, Johan Herland wrote:\n> This patch introduces a Python package called \"git_remote_cvs\" containing\n> the building blocks of the CVS remote helper. The CVS remote helper itself\n> is NOT part of this patch.\n\nInteresting...\n\n> diff --git a/git_remote_cvs/changeset.py b/git_remote_cvs/changeset.py\n> new file mode 100644\n> index 0000000..27c4129\n> --- /dev/null\n> +++ b/git_remote_cvs/changeset.py\n> @@ -0,0 +1,114 @@\n> +#!/usr/bin/env python\n> +\n> +\"\"\"Functionality for collecting individual CVS revisions into \"changesets\"\n> +\n> +A changeset is a collection of CvsRev objects that belong together in the same\n> +\"commit\". This is a somewhat artificial construct on top of CVS, which only\n> +stores changes at the per-file level. Normally, CVS users create several CVS\n> +revisions simultaneously by applying the \"cvs commit\" command to several files\n> +with related changes. This module tries to reconstruct this notion of related\n> +revisions.\n> +\"\"\"\n> +\n> +from util import *\n\n\nImporting * is frowned upon in Python.\n\nIt's much easier to see where things are coming from if you\n'import util' and use the namespaced util.foo() way of accessing\nthe functions.\n\nFurthermore, you're going to want to use absolute imports.\nAnyone can create 'util.py' and blindly importing 'util' is\nasking for trouble.\n\nInstead use:\nfrom git_remote_cvs import util\n\n\n> +class Changeset (object):\n> +\t\"\"\"Encapsulate a single changeset/commit\"\"\"\n\nI think it reads better as Changeset(object)\n(drop the spaces before the parens).\n\nThat applies to the rest of this patch as well.\n\n\nThis also had me wondering about the following:\n\tgit uses tabs for indentation\n\nBUT, the python convention is to use 4-space indents ala PEP-8\nhttp://www.python.org/dev/peps/pep-0008/\n\n\nIt might be appealing to when-in-Rome (Rome being Python) here\nand do things the python way when we code in Python.\n\nConsistency with pep8 is good if we expect to get python hackers\nto contribute to git_remote_cvs.\n\n\n> +\n> +\t__slots__ = ('revs', 'date', 'author', 'message')\n\n\n__slots__ is pretty esoteric in Python-land.\n\nBut, if your justification is to minimize memory usage, then\nyes, this is a good thing to do.\n\n\n> +\tdef __init__ (self, date, author, message):\n> +\t\tself.revs = {} # dict: path -> CvsRev object\n> +\t\tself.date = date # CvsDate object\n> +\t\tself.author = author\n> +\t\tself.message = message # Lines of commit message\n\npep8 and other parts of the git codebase recommend against\nlining up the equals signs like that. Ya, sorry for the nits\nbeing that they're purely stylistic.\n\n> +\t\tif len(msg) > 25: msg = msg[:22] + \"...\" # Max 25 chars long\n> +\t\treturn \"\" % (\n> +\t\t\tself.date, self.author, msg, len(self.revs))\n\nSimilar to the git coding style, this might be better written:\n\n...\nif len(msg) > 25:\n msg = msg[:22] + '...' # Max 25 chars long\n...\n\n(aka avoid single-line ifs)\n\nThere's a few other instances of this in the patch as well.\n\n\n\n> diff --git a/git_remote_cvs/cvs.py b/git_remote_cvs/cvs.py\n> new file mode 100644\n> index 0000000..cc2e13f\n> --- /dev/null\n> +++ b/git_remote_cvs/cvs.py\n> @@ -0,0 +1,884 @@\n> [...]\n> +\n> +\tdef enumerate (self):\n> +\t\t\"\"\"Return a list of integer components in this CVS number\"\"\"\n> +\t\treturn list(self.l)\n\nenumerate has special meaning in Python.\n\nitems = (1, 2, 3, 4)\nfor idx, item in enumerate(items):\n print idx, item\n\n\nI'm not sure if this would cause confusion...\n\n\n> [...]\n> +\t\telse: # revision number\n> +\t\t\tassert self.l[-1] > 0\n\nasserts go away when running with PYTHONOPTIMIZE.\n\nIf this is really an error then we should we raise an exception\ninstead?\n\n\n> +\t@classmethod\n> +\tdef test (cls):\n> +\t\tassert cls(\"1.2.4\") == cls(\"1.2.0.4\")\n\nHmm.. Does it make more sense to use the unittest module?\n\ne.g. self.assertEqual(foo, bar)\n\n> diff --git a/git_remote_cvs/cvs_revision_map.py b/git_remote_cvs/cvs_revision_map.py\n> new file mode 100644\n> index 0000000..7d7810f\n> --- /dev/null\n> +++ b/git_remote_cvs/cvs_revision_map.py\n> @@ -0,0 +1,362 @@\n> +#!/usr/bin/env python\n> +\n> +\"\"\"Functionality for mapping CVS revisions to associated metainformation\"\"\"\n> +\n> +from util import *\n> +from cvs import CvsNum, CvsDate\n> +from git import GitFICommit, GitFastImport, GitObjectFetcher\n\nWe definitely need absolute imports here.\n\n'import git' could find the git-python project's git module.\n\n\nNonetheless, interesting stuff.\n\n\n-- \n\t\tDavid\n"},{"id":"120392","messageId":"200908121108.28714.johan@herland.net","threadId":"20543","inReplyTo":"20090812021017.GB62301@gmail.com","subject":"Re: [RFCv3 2/4] Add Python support library for CVS remote helper","fromName":"Johan Herland","fromEmail":"johan@herland.net","sentAt":"2009-08-12T09:08:28Z","receivedAt":"2009-08-12T09:08:28Z","isPatch":false,"sender":{"key":"johan@herland.net","avatar":"https://avatars.githubusercontent.com/u/547031?v=4"},"body":"First, thank you very much for the review. It is very helpful, and I really \nappreciate it.\n\nOn Wednesday 12 August 2009, David Aguilar wrote:\n> On Wed, Aug 12, 2009 at 02:13:49AM +0200, Johan Herland wrote:\n> > This patch introduces a Python package called \"git_remote_cvs\"\n> > containing the building blocks of the CVS remote helper. The CVS remote\n> > helper itself is NOT part of this patch.\n>\n> Interesting...\n>\n> > diff --git a/git_remote_cvs/changeset.py b/git_remote_cvs/changeset.py\n> > new file mode 100644\n> > index 0000000..27c4129\n> > --- /dev/null\n> > +++ b/git_remote_cvs/changeset.py\n> > @@ -0,0 +1,114 @@\n> > +#!/usr/bin/env python\n> > +\n> > +\"\"\"Functionality for collecting individual CVS revisions into\n> > \"changesets\" +\n> > +A changeset is a collection of CvsRev objects that belong together in\n> > the same +\"commit\". This is a somewhat artificial construct on top of\n> > CVS, which only +stores changes at the per-file level. Normally, CVS\n> > users create several CVS +revisions simultaneously by applying the \"cvs\n> > commit\" command to several files +with related changes. This module\n> > tries to reconstruct this notion of related +revisions.\n> > +\"\"\"\n> > +\n> > +from util import *\n>\n> Importing * is frowned upon in Python.\n>\n> It's much easier to see where things are coming from if you\n> 'import util' and use the namespaced util.foo() way of accessing\n> the functions.\n\nI'd rather do \"from util import X Y Z\", as the util stuff is used all over \nthe place.\n\n> Furthermore, you're going to want to use absolute imports.\n> Anyone can create 'util.py' and blindly importing 'util' is\n> asking for trouble.\n>\n> Instead use:\n> from git_remote_cvs import util\n\nI thought the python import rules specified that the current package was \nconsulted first, and therefore the 'util' package would always come from the \ncurrent package. However, I must confess that I don't know these rules very \nwell, so I'll take your word for it and use absolute imports instead.\n\n> > +class Changeset (object):\n> > +\t\"\"\"Encapsulate a single changeset/commit\"\"\"\n>\n> I think it reads better as Changeset(object)\n> (drop the spaces before the parens).\n>\n> That applies to the rest of this patch as well.\n\nOk. Will change.\n\n> This also had me wondering about the following:\n> \tgit uses tabs for indentation\n>\n> BUT, the python convention is to use 4-space indents ala PEP-8\n> http://www.python.org/dev/peps/pep-0008/\n\nInteresting. I have (obviously) never looked at PEP 8... :)\n\n> It might be appealing to when-in-Rome (Rome being Python) here\n> and do things the python way when we code in Python.\n>\n> Consistency with pep8 is good if we expect to get python hackers\n> to contribute to git_remote_cvs.\n\nI see your point, but I believe that since git_remote_cvs is not an \nindependent project (but very much coupled to git), its allegiance is with \nGit, and it should therefore follow the Git coding style. In other words, I \nclaim exception (2) in PEP 8\n\n> > +\n> > +\t__slots__ = ('revs', 'date', 'author', 'message')\n>\n> __slots__ is pretty esoteric in Python-land.\n>\n> But, if your justification is to minimize memory usage, then\n> yes, this is a good thing to do.\n\nYes, I only use __slots__ for classes that potentially have a large number \nof instances.\n\n> > +\tdef __init__ (self, date, author, message):\n> > +\t\tself.revs = {} # dict: path -> CvsRev object\n> > +\t\tself.date = date # CvsDate object\n> > +\t\tself.author = author\n> > +\t\tself.message = message # Lines of commit message\n>\n> pep8 and other parts of the git codebase recommend against\n> lining up the equals signs like that. Ya, sorry for the nits\n> being that they're purely stylistic.\n\nI can't find a good rationale for this rule in PEP8 (other than Guido's \npersonal style), and I personally find the above much more readable \n(otherwise I wouldn't go through the trouble of lining them all up...). Can \nI claim exception (1) (readability)?\n\n> > +\t\tif len(msg) > 25: msg = msg[:22] + \"...\" # Max 25 chars long\n> > +\t\treturn \"\" % (\n> > +\t\t\tself.date, self.author, msg, len(self.revs))\n>\n> Similar to the git coding style, this might be better written:\n>\n> ...\n> if len(msg) > 25:\n> msg = msg[:22] + '...' # Max 25 chars long\n> ...\n>\n> (aka avoid single-line ifs)\n>\n> There's a few other instances of this in the patch as well.\n\nOk. Will try to eliminate single-line ifs.\n\n> > diff --git a/git_remote_cvs/cvs.py b/git_remote_cvs/cvs.py\n> > new file mode 100644\n> > index 0000000..cc2e13f\n> > --- /dev/null\n> > +++ b/git_remote_cvs/cvs.py\n> > @@ -0,0 +1,884 @@\n> > [...]\n> > +\n> > +\tdef enumerate (self):\n> > +\t\t\"\"\"Return a list of integer components in this CVS number\"\"\"\n> > +\t\treturn list(self.l)\n>\n> enumerate has special meaning in Python.\n>\n> items = (1, 2, 3, 4)\n> for idx, item in enumerate(items):\n> print idx, item\n>\n>\n> I'm not sure if this would cause confusion...\n\nGood point, I should probably rename this method.\n\n> > [...]\n> > +\t\telse: # revision number\n> > +\t\t\tassert self.l[-1] > 0\n>\n> asserts go away when running with PYTHONOPTIMIZE.\n>\n> If this is really an error then we should we raise an exception\n> instead?\n\nI use asserts to verify pre/post-conditions and other invariants. I believe \nthat if this assert fails, it is indicative of something horribly wrong with \nthe code itself. However, I now see that one can also trigger this case with \nbad input (e.g. CvsNum(\"1.2.3.0\").parent()). I will keep the assert here, \nbut will also add some input verification to the CvsNum class.\n\n> > +\t@classmethod\n> > +\tdef test (cls):\n> > +\t\tassert cls(\"1.2.4\") == cls(\"1.2.0.4\")\n>\n> Hmm.. Does it make more sense to use the unittest module?\n>\n> e.g. self.assertEqual(foo, bar)\n\nProbably. I'm not familiar with 'unittest', but will take a look.\n\n> > diff --git a/git_remote_cvs/cvs_revision_map.py\n> > b/git_remote_cvs/cvs_revision_map.py new file mode 100644\n> > index 0000000..7d7810f\n> > --- /dev/null\n> > +++ b/git_remote_cvs/cvs_revision_map.py\n> > @@ -0,0 +1,362 @@\n> > +#!/usr/bin/env python\n> > +\n> > +\"\"\"Functionality for mapping CVS revisions to associated\n> > metainformation\"\"\" +\n> > +from util import *\n> > +from cvs import CvsNum, CvsDate\n> > +from git import GitFICommit, GitFastImport, GitObjectFetcher\n>\n> We definitely need absolute imports here.\n>\n> 'import git' could find the git-python project's git module.\n\nOk. Will fix.\n\n> Nonetheless, interesting stuff.\n\nThanks for the review!\n\n\nHave fun! :)\n\n...Johan\n\n-- \nJohan Herland, \nwww.herland.net\n"},{"id":"120426","messageId":"fabb9a1e0908121043p390d8532tc3a661b42b3bfd1a@mail.gmail.com","threadId":"20543","inReplyTo":"200908121108.28714.johan@herland.net","subject":"Re: [RFCv3 2/4] Add Python support library for CVS remote helper","fromName":"Sverre Rabbelier","fromEmail":"srabbelier@gmail.com","sentAt":"2009-08-12T17:43:42Z","receivedAt":"2009-08-12T17:43:42Z","isPatch":false,"sender":{"key":"srabbelier@gmail.com","avatar":"https://avatars.githubusercontent.com/u/3098?v=4"},"body":"Heya,\n\nOn Wed, Aug 12, 2009 at 02:08, Johan Herland wrote:\n> I can't find a good rationale for this rule in PEP8 (other than Guido's\n> personal style), and I personally find the above much more readable\n> (otherwise I wouldn't go through the trouble of lining them all up...). Can\n> I claim exception (1) (readability)?\n\nWorking with python a lot myself, if you want to claim readability _to\npython hackers_, then you should follow PEP8. If you want to follow\nyour own personal style which is easily readable to you, then by all\nmeans; just know that people that read python a lot will have a hard\ntime reading your code ;).\n\n-- \nCheers,\n\nSverre Rabbelier\n"},{"id":"120465","messageId":"4A83579E.70302@alum.mit.edu","threadId":"20543","inReplyTo":"200908121108.28714.johan@herland.net","subject":"Re: [RFCv3 2/4] Add Python support library for CVS remote helper","fromName":"Michael Haggerty","fromEmail":"mhagger@alum.mit.edu","sentAt":"2009-08-13T00:00:30Z","receivedAt":"2009-08-13T00:00:30Z","isPatch":false,"sender":{"key":"mhagger@alum.mit.edu","avatar":"https://avatars.githubusercontent.com/u/119718?v=4"},"body":"Johan Herland wrote:\n> On Wednesday 12 August 2009, David Aguilar wrote:\n>> On Wed, Aug 12, 2009 at 02:13:49AM +0200, Johan Herland wrote:\n>>> +\tdef __init__ (self, date, author, message):\n>>> +\t\tself.revs = {} # dict: path -> CvsRev object\n>>> +\t\tself.date = date # CvsDate object\n>>> +\t\tself.author = author\n>>> +\t\tself.message = message # Lines of commit message\n>> pep8 and other parts of the git codebase recommend against\n>> lining up the equals signs like that. Ya, sorry for the nits\n>> being that they're purely stylistic.\n> \n> I can't find a good rationale for this rule in PEP8 (other than Guido's \n> personal style), and I personally find the above much more readable \n> (otherwise I wouldn't go through the trouble of lining them all up...). Can \n> I claim exception (1) (readability)?\n\nI think you are missing the point. It may be true that the rules in\nPEP8 were *originally* written according to the unjustified whims of the\nBDFL, but now that they are established the reason for following them is\nnot that Guido likes them but rather to be consistent with the bulk of\nother Python code on the planet.\n\nWith respect to the rule to use 4-space indents, there are serious\npractical problems with using tabs *in addition to* the consistency\nargument.\n\nMichael\n"},{"id":"120466","messageId":"200908130220.02517.johan@herland.net","threadId":"20543","inReplyTo":"4A83579E.70302@alum.mit.edu","subject":"Re: [RFCv3 2/4] Add Python support library for CVS remote helper","fromName":"Johan Herland","fromEmail":"johan@herland.net","sentAt":"2009-08-13T00:20:02Z","receivedAt":"2009-08-13T00:20:02Z","isPatch":false,"sender":{"key":"johan@herland.net","avatar":"https://avatars.githubusercontent.com/u/547031?v=4"},"body":"On Thursday 13 August 2009, Michael Haggerty wrote:\n> Johan Herland wrote:\n> > On Wednesday 12 August 2009, David Aguilar wrote:\n> >> On Wed, Aug 12, 2009 at 02:13:49AM +0200, Johan Herland wrote:\n> >>> +\tdef __init__ (self, date, author, message):\n> >>> +\t\tself.revs = {} # dict: path -> CvsRev object\n> >>> +\t\tself.date = date # CvsDate object\n> >>> +\t\tself.author = author\n> >>> +\t\tself.message = message # Lines of commit message\n> >>\n> >> pep8 and other parts of the git codebase recommend against\n> >> lining up the equals signs like that. Ya, sorry for the nits\n> >> being that they're purely stylistic.\n> >\n> > I can't find a good rationale for this rule in PEP8 (other than Guido's\n> > personal style), and I personally find the above much more readable\n> > (otherwise I wouldn't go through the trouble of lining them all up...).\n> > Can I claim exception (1) (readability)?\n>\n> I think you are missing the point. It may be true that the rules in\n> PEP8 were *originally* written according to the unjustified whims of the\n> BDFL, but now that they are established the reason for following them is\n> not that Guido likes them but rather to be consistent with the bulk of\n> other Python code on the planet.\n\nOk. I will try to follow PEP8 as closely as possible.\n\n> With respect to the rule to use 4-space indents, there are serious\n> practical problems with using tabs *in addition to* the consistency\n> argument.\n\nThere are? What arguments? Assuming I don't mix spaces and tabs (which I \ncertainly don't), I can't see any \"practical problems\" with using tabs \n(except for the PEP8/consistency issue).\n\n\n...Johan\n\n-- \nJohan Herland, \nwww.herland.net\n"},{"id":"120469","messageId":"7vzla4r0qc.fsf@alter.siamese.dyndns.org","threadId":"20543","inReplyTo":"20090812021017.GB62301@gmail.com","subject":"Re: [RFCv3 2/4] Add Python support library for CVS remote helper","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2009-08-13T00:55:39Z","receivedAt":"2009-08-13T00:55:39Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"David Aguilar writes:\n\n> This also had me wondering about the following:\n> \tgit uses tabs for indentation\n\nNot relevant. That is a rule for our \"C\" source code. We also use it in\nour Perl scripts and shell scripts because there is no single \"one right\nway\" that is strongly defined and everybody adheres to, like the 4-space\nrule Python folks have.\n\n> BUT, the python convention is to use 4-space indents ala PEP-8\n> http://www.python.org/dev/peps/pep-0008/\n>\n> It might be appealing to when-in-Rome (Rome being Python) here\n> and do things the python way when we code in Python.\n\nYes, this is more important.\n\n>> +\t\tif len(msg) > 25: msg = msg[:22] + \"...\" # Max 25 chars long\n>> +\t\treturn \"\" % (\n>> +\t\t\tself.date, self.author, msg, len(self.revs))\n>\n> Similar to the git coding style, this might be better written:\n\nSo is this one. If experienced Python folks also frown on single-line\nconditionals, then by all means please update this. But if this\nsuggestion is solely because we don't do a single-line conditional in our\nC source code, then please do not insist on it too strongly. The code\nshould look familiar to Pythonistas with good tastes (if such a class of\npeople exist, that is ;-)).\n"},{"id":"120470","messageId":"200908130327.28571.johan@herland.net","threadId":"20543","inReplyTo":"7vzla4r0qc.fsf@alter.siamese.dyndns.org","subject":"Re: [RFCv3 2/4] Add Python support library for CVS remote helper","fromName":"Johan Herland","fromEmail":"johan@herland.net","sentAt":"2009-08-13T01:27:28Z","receivedAt":"2009-08-13T01:27:28Z","isPatch":false,"sender":{"key":"johan@herland.net","avatar":"https://avatars.githubusercontent.com/u/547031?v=4"},"body":"On Thursday 13 August 2009, Junio C Hamano wrote:\n> David Aguilar writes:\n> > This also had me wondering about the following:\n> > \tgit uses tabs for indentation\n>\n> Not relevant. That is a rule for our \"C\" source code. We also use it in\n> our Perl scripts and shell scripts because there is no single \"one right\n> way\" that is strongly defined and everybody adheres to, like the 4-space\n> rule Python folks have.\n>\n> > BUT, the python convention is to use 4-space indents ala PEP-8\n> > http://www.python.org/dev/peps/pep-0008/\n> >\n> > It might be appealing to when-in-Rome (Rome being Python) here\n> > and do things the python way when we code in Python.\n>\n> Yes, this is more important.\n>\n> >> +\t\tif len(msg) > 25: msg = msg[:22] + \"...\" # Max 25 chars long\n> >> +\t\treturn \"\" % (\n> >> +\t\t\tself.date, self.author, msg, len(self.revs))\n> >\n> > Similar to the git coding style, this might be better written:\n>\n> So is this one. If experienced Python folks also frown on single-line\n> conditionals, then by all means please update this. But if this\n> suggestion is solely because we don't do a single-line conditional in our\n> C source code, then please do not insist on it too strongly. The code\n> should look familiar to Pythonistas with good tastes (if such a class of\n> people exist, that is ;-)).\n\nOk. Thanks. I will follow PEP8 as closely as possible, including the 4-space \nindent.\n\n\n...Johan\n\n-- \nJohan Herland, \nwww.herland.net\n"},{"id":"120777","messageId":"7v7hx35ym1.fsf@alter.siamese.dyndns.org","threadId":"20543","inReplyTo":"1250036031-32272-3-git-send-email-johan@herland.net","subject":"Re: [RFCv3 2/4] Add Python support library for CVS remote helper","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2009-08-16T19:48:22Z","receivedAt":"2009-08-16T19:48:22Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"It appears that the \"make install\" step with this patch is broken, trying\nto write into /usr/lib/python2.6/ without honoring DESTDIR.\n\nIt needs to be resolved before the series nears 'master', preferrably\nbefore it hits 'next', as \"make rpm\" step is one of the things that is\nbroken by this.\n\nI am sure people who are more savvy on Python can offer help.\n\nThanks.\n"},{"id":"120780","messageId":"1250455088-23457-1-git-send-email-davvid@gmail.com","threadId":"20543","inReplyTo":"7v7hx35ym1.fsf@alter.siamese.dyndns.org","subject":"[PATCH 1/2] git_remote_cvs: Honor DESTDIR in the Makefile","fromName":"David Aguilar","fromEmail":"davvid@gmail.com","sentAt":"2009-08-16T20:38:07Z","receivedAt":"2009-08-16T20:38:07Z","isPatch":true,"sender":{"key":"davvid@gmail.com","avatar":"https://avatars.githubusercontent.com/u/13196?v=4"},"body":"This adds the --root= flag to setup.py so that the\nuser-provided DESTDIR is honored.\n\nSigned-off-by: David Aguilar \n---\n git_remote_cvs/Makefile | 14 +++++++++++++-\n 1 files changed, 13 insertions(+), 1 deletions(-)\n\ndiff --git a/git_remote_cvs/Makefile b/git_remote_cvs/Makefile\nindex 8dbf3fa..f52c096 100644\n--- a/git_remote_cvs/Makefile\n+++ b/git_remote_cvs/Makefile\n@@ -3,6 +3,15 @@\n #\n pysetupfile:=setup.py\n \n+# Setup the DESTDIR for Python.\n+ifeq ($(DESTDIR),)\n+PYTHON_DESTDIR = /\n+else\n+PYTHON_DESTDIR = $(DESTDIR)\n+endif\n+# Shell quote (do not use $(call) to accommodate ancient setups);\n+PYTHON_DESTDIR_SQ = $(subst ','\\'',$(PYTHON_DESTDIR))\n+\n ifndef PYTHON_PATH\n \tPYTHON_PATH = /usr/bin/python\n endif\n@@ -19,7 +28,10 @@ PYLIBDIR=`$(PYTHON_PATH) -c \"import sys; print 'lib/python%i.%i/site-packages' %\n all: $(pysetupfile)\n \t$(QUIET)$(PYTHON_PATH) $(pysetupfile) $(QUIETSETUP) build\n install: $(pysetupfile)\n-\t$(PYTHON_PATH) $(pysetupfile) install --prefix $(prefix)\n+\t$(PYTHON_PATH) $(pysetupfile) install \\\n+\t\t--prefix $(prefix) \\\n+\t\t--root $(PYTHON_DESTDIR_SQ)\n+\n instlibdir: $(pysetupfile)\n \t@echo \"$(prefix)/$(PYLIBDIR)\"\n clean:\n-- \n1.6.4.169.g64d5\n"},{"id":"120779","messageId":"1250455088-23457-2-git-send-email-davvid@gmail.com","threadId":"20543","inReplyTo":"1250455088-23457-1-git-send-email-davvid@gmail.com","subject":"[PATCH 2/2] git_remote_cvs: Use $(shell) in the Makefile","fromName":"David Aguilar","fromEmail":"davvid@gmail.com","sentAt":"2009-08-16T20:38:08Z","receivedAt":"2009-08-16T20:38:08Z","isPatch":true,"sender":{"key":"davvid@gmail.com","avatar":"https://avatars.githubusercontent.com/u/13196?v=4"},"body":"This updates the git_remote_cvs Makefile to use the same\n$(shell ) style used by the top-level git Makefile.\n\nSigned-off-by: David Aguilar \n---\n git_remote_cvs/Makefile | 6 +++++-\n 1 files changed, 5 insertions(+), 1 deletions(-)\n\ndiff --git a/git_remote_cvs/Makefile b/git_remote_cvs/Makefile\nindex f52c096..d281d48 100644\n--- a/git_remote_cvs/Makefile\n+++ b/git_remote_cvs/Makefile\n@@ -23,10 +23,13 @@ ifndef V\n \tQUIETSETUP = --quiet\n endif\n \n-PYLIBDIR=`$(PYTHON_PATH) -c \"import sys; print 'lib/python%i.%i/site-packages' % sys.version_info[:2]\"`\n+PYLIBDIR=$(shell $(PYTHON_PATH) -c \\\n+\t \"import sys; \\\n+\t print 'lib/python%i.%i/site-packages' % sys.version_info[:2]\")\n \n all: $(pysetupfile)\n \t$(QUIET)$(PYTHON_PATH) $(pysetupfile) $(QUIETSETUP) build\n+\n install: $(pysetupfile)\n \t$(PYTHON_PATH) $(pysetupfile) install \\\n \t\t--prefix $(prefix) \\\n@@ -34,6 +37,7 @@ install: $(pysetupfile)\n \n instlibdir: $(pysetupfile)\n \t@echo \"$(prefix)/$(PYLIBDIR)\"\n+\n clean:\n \t$(QUIET)$(PYTHON_PATH) $(pysetupfile) $(QUIETSETUP) clean -a\n \t$(RM) *.pyo *.pyc\n-- \n1.6.4.169.g64d5\n"},{"id":"120781","messageId":"20090816204658.GA23522@gmail.com","threadId":"20543","inReplyTo":"1250455088-23457-2-git-send-email-davvid@gmail.com","subject":"Re: [PATCH 2/2] git_remote_cvs: Use $(shell) in the Makefile","fromName":"David Aguilar","fromEmail":"davvid@gmail.com","sentAt":"2009-08-16T20:47:00Z","receivedAt":"2009-08-16T20:47:00Z","isPatch":true,"sender":{"key":"davvid@gmail.com","avatar":"https://avatars.githubusercontent.com/u/13196?v=4"},"body":"On Sun, Aug 16, 2009 at 01:38:08PM -0700, David Aguilar wrote:\n> This updates the git_remote_cvs Makefile to use the same\n> $(shell ) style used by the top-level git Makefile.\n> \n> Signed-off-by: David Aguilar \n> ---\n\nI should have mentioned here that I also spaced stuff out and\nchopped the long line so that it fits within 78 chars.\n\nI intentionally broke this out as a 2nd patch in case using\n$(shell ...) was not the right thing to do.\n\n\n\n\n> git_remote_cvs/Makefile | 6 +++++-\n> 1 files changed, 5 insertions(+), 1 deletions(-)\n> \n> diff --git a/git_remote_cvs/Makefile b/git_remote_cvs/Makefile\n> index f52c096..d281d48 100644\n> --- a/git_remote_cvs/Makefile\n> +++ b/git_remote_cvs/Makefile\n> @@ -23,10 +23,13 @@ ifndef V\n> \tQUIETSETUP = --quiet\n> endif\n> \n> -PYLIBDIR=`$(PYTHON_PATH) -c \"import sys; print 'lib/python%i.%i/site-packages' % sys.version_info[:2]\"`\n> +PYLIBDIR=$(shell $(PYTHON_PATH) -c \\\n> +\t \"import sys; \\\n> +\t print 'lib/python%i.%i/site-packages' % sys.version_info[:2]\")\n> \n> all: $(pysetupfile)\n> \t$(QUIET)$(PYTHON_PATH) $(pysetupfile) $(QUIETSETUP) build\n> +\n> install: $(pysetupfile)\n> \t$(PYTHON_PATH) $(pysetupfile) install \\\n> \t\t--prefix $(prefix) \\\n> @@ -34,6 +37,7 @@ install: $(pysetupfile)\n> \n> instlibdir: $(pysetupfile)\n> \t@echo \"$(prefix)/$(PYLIBDIR)\"\n> +\n> clean:\n> \t$(QUIET)$(PYTHON_PATH) $(pysetupfile) $(QUIETSETUP) clean -a\n> \t$(RM) *.pyo *.pyc\n> -- \n> 1.6.4.169.g64d5\n> \n\n-- \n\t\tDavid\n"},{"id":"120782","messageId":"alpine.DEB.1.00.0908162251360.8306@pacific.mpi-cbg.de","threadId":"20543","inReplyTo":"1250455088-23457-1-git-send-email-davvid@gmail.com","subject":"Re: [PATCH 1/2] git_remote_cvs: Honor DESTDIR in the Makefile","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2009-08-16T20:55:29Z","receivedAt":"2009-08-16T20:55:29Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi,\n\nOn Sun, 16 Aug 2009, David Aguilar wrote:\n\n> diff --git a/git_remote_cvs/Makefile b/git_remote_cvs/Makefile\n> index 8dbf3fa..f52c096 100644\n> --- a/git_remote_cvs/Makefile\n> +++ b/git_remote_cvs/Makefile\n> @@ -3,6 +3,15 @@\n> #\n> pysetupfile:=setup.py\n> \n> +# Setup the DESTDIR for Python.\n> +ifeq ($(DESTDIR),)\n> +PYTHON_DESTDIR = /\n\nHmm. I think this would break on msysGit. Not that anybody worked on \ngetting Python to compile on msysGit.\n\n(Just to make sure you understand the issue: on msysGit, we set prefix to \n\"\" (and I think DESTDIR somehow ends up taking on the same value). Now, \nwhen DESTDIR is set to \"/\" and something wants to be copied to \n$(DESTDIR)/something, the latter expands to //something, which tells MSys \nnot to expand //something to the correct Windows path.\n\nCiao,\nDscho\n"},{"id":"120783","messageId":"20090816210300.GB23522@gmail.com","threadId":"20543","inReplyTo":"alpine.DEB.1.00.0908162251360.8306@pacific.mpi-cbg.de","subject":"Re: [PATCH 1/2] git_remote_cvs: Honor DESTDIR in the Makefile","fromName":"David Aguilar","fromEmail":"davvid@gmail.com","sentAt":"2009-08-16T21:03:01Z","receivedAt":"2009-08-16T21:03:01Z","isPatch":true,"sender":{"key":"davvid@gmail.com","avatar":"https://avatars.githubusercontent.com/u/13196?v=4"},"body":"On Sun, Aug 16, 2009 at 10:55:29PM +0200, Johannes Schindelin wrote:\n> Hi,\n> \n> On Sun, 16 Aug 2009, David Aguilar wrote:\n> \n> > diff --git a/git_remote_cvs/Makefile b/git_remote_cvs/Makefile\n> > index 8dbf3fa..f52c096 100644\n> > --- a/git_remote_cvs/Makefile\n> > +++ b/git_remote_cvs/Makefile\n> > @@ -3,6 +3,15 @@\n> > #\n> > pysetupfile:=setup.py\n> > \n> > +# Setup the DESTDIR for Python.\n> > +ifeq ($(DESTDIR),)\n> > +PYTHON_DESTDIR = /\n> \n> Hmm. I think this would break on msysGit. Not that anybody worked on \n> getting Python to compile on msysGit.\n> \n> (Just to make sure you understand the issue: on msysGit, we set prefix to \n> \"\" (and I think DESTDIR somehow ends up taking on the same value). Now, \n> when DESTDIR is set to \"/\" and something wants to be copied to \n> $(DESTDIR)/something, the latter expands to //something, which tells MSys \n> not to expand //something to the correct Windows path.\n\n\nI see. Hmm.. setup.py is a real pain.\n\nI'll see if we rework this so that we end up passing \"\" to\n--root instead of /. I'm going to be gone for a few hours so\nprobably won't be able to try it out until tonight.\n\nAnother thing to consider --\n\nDebian once submitted a bug against another Python app asking\nthat we not place modules in site-packages unless we\nplan on having other applications importing those modules.\n\nThe more appropriate place for them if we don't plan on that is\n$(prefix)/share/git-core/git_remote_cvs or something like that.\n\nI guess that's another thing to think about.\n\n-- \n\t\tDavid\n"},{"id":"120784","messageId":"alpine.DEB.1.00.0908162316030.8306@pacific.mpi-cbg.de","threadId":"20543","inReplyTo":"20090816210300.GB23522@gmail.com","subject":"Re: [PATCH 1/2] git_remote_cvs: Honor DESTDIR in the Makefile","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2009-08-16T21:21:08Z","receivedAt":"2009-08-16T21:21:08Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi,\n\nOn Sun, 16 Aug 2009, David Aguilar wrote:\n\n> On Sun, Aug 16, 2009 at 10:55:29PM +0200, Johannes Schindelin wrote:\n> \n> > On Sun, 16 Aug 2009, David Aguilar wrote:\n> > \n> > > diff --git a/git_remote_cvs/Makefile b/git_remote_cvs/Makefile\n> > > index 8dbf3fa..f52c096 100644\n> > > --- a/git_remote_cvs/Makefile\n> > > +++ b/git_remote_cvs/Makefile\n> > > @@ -3,6 +3,15 @@\n> > > #\n> > > pysetupfile:=setup.py\n> > > \n> > > +# Setup the DESTDIR for Python.\n> > > +ifeq ($(DESTDIR),)\n> > > +PYTHON_DESTDIR = /\n> > \n> > Hmm. I think this would break on msysGit. Not that anybody worked on \n> > getting Python to compile on msysGit.\n> > \n> > (Just to make sure you understand the issue: on msysGit, we set prefix to \n> > \"\" (and I think DESTDIR somehow ends up taking on the same value). Now, \n> > when DESTDIR is set to \"/\" and something wants to be copied to \n> > $(DESTDIR)/something, the latter expands to //something, which tells MSys \n> > not to expand //something to the correct Windows path.\n> \n> \n> I see. Hmm.. setup.py is a real pain.\n> \n> I'll see if we rework this so that we end up passing \"\" to\n> --root instead of /. I'm going to be gone for a few hours so\n> probably won't be able to try it out until tonight.\n\nThinking about it a bit more: you might not need to do anything. I think \nthat msysGit should move to prefix = /mingw anyway.\n\nI hesitated to do that earlier, as all .perl and .sh scripts need MSys. \nBut the Git .exe files are MinGW programs, so technically they belong into \n/mingw/bin/ anyway.\n\nBut the issue would only really become relevant if anybody supported \nPython on msysGit (and thereby git-remote-cvs, before that, we cannot make \nuse of it).\n\nJust a thing to keep in mind.\n\nCiao,\nDscho\n"},{"id":"120786","messageId":"1250457908-27376-1-git-send-email-davvid@gmail.com","threadId":"20543","inReplyTo":"alpine.DEB.1.00.0908162251360.8306@pacific.mpi-cbg.de","subject":"[PATCH v2 1/2] git_remote_cvs: Honor DESTDIR in the Makefile","fromName":"David Aguilar","fromEmail":"davvid@gmail.com","sentAt":"2009-08-16T21:25:07Z","receivedAt":"2009-08-16T21:25:07Z","isPatch":true,"sender":{"key":"davvid@gmail.com","avatar":"https://avatars.githubusercontent.com/u/13196?v=4"},"body":"This modifies the setup.py invocation so that user-defined\nDESTDIRs are taken into account.\n\nSigned-off-by: David Aguilar \n---\n\nsetup.py gets confused if we use --root like in v1 of this patch.\nI think this is simpler. Thoughts?\n\n git_remote_cvs/Makefile | 6 +++++-\n 1 files changed, 5 insertions(+), 1 deletions(-)\n\ndiff --git a/git_remote_cvs/Makefile b/git_remote_cvs/Makefile\nindex 8dbf3fa..2e26dbe 100644\n--- a/git_remote_cvs/Makefile\n+++ b/git_remote_cvs/Makefile\n@@ -3,6 +3,9 @@\n #\n pysetupfile:=setup.py\n \n+# Shell quote (do not use $(call) to accommodate ancient setups);\n+DESTDIR_SQ = $(subst ','\\'',$(DESTDIR))\n+\n ifndef PYTHON_PATH\n \tPYTHON_PATH = /usr/bin/python\n endif\n@@ -19,7 +22,8 @@ PYLIBDIR=`$(PYTHON_PATH) -c \"import sys; print 'lib/python%i.%i/site-packages' %\n all: $(pysetupfile)\n \t$(QUIET)$(PYTHON_PATH) $(pysetupfile) $(QUIETSETUP) build\n install: $(pysetupfile)\n-\t$(PYTHON_PATH) $(pysetupfile) install --prefix $(prefix)\n+\t$(PYTHON_PATH) $(pysetupfile) install --prefix $(DESTDIR_SQ)$(prefix)\n+\n instlibdir: $(pysetupfile)\n \t@echo \"$(prefix)/$(PYLIBDIR)\"\n clean:\n-- \n1.6.4.314.g034e1\n"},{"id":"120787","messageId":"1250457908-27376-2-git-send-email-davvid@gmail.com","threadId":"20543","inReplyTo":"1250457908-27376-1-git-send-email-davvid@gmail.com","subject":"[PATCH v2 2/2] git_remote_cvs: Use $(shell) in the Makefile","fromName":"David Aguilar","fromEmail":"davvid@gmail.com","sentAt":"2009-08-16T21:25:08Z","receivedAt":"2009-08-16T21:25:08Z","isPatch":true,"sender":{"key":"davvid@gmail.com","avatar":"https://avatars.githubusercontent.com/u/13196?v=4"},"body":"This updates the git_remote_cvs Makefile to use the same\n$(shell ) style used by the top-level git Makefile.\n\nSigned-off-by: David Aguilar \n---\n git_remote_cvs/Makefile | 6 +++++-\n 1 files changed, 5 insertions(+), 1 deletions(-)\n\ndiff --git a/git_remote_cvs/Makefile b/git_remote_cvs/Makefile\nindex 2e26dbe..061c247 100644\n--- a/git_remote_cvs/Makefile\n+++ b/git_remote_cvs/Makefile\n@@ -17,15 +17,19 @@ ifndef V\n \tQUIETSETUP = --quiet\n endif\n \n-PYLIBDIR=`$(PYTHON_PATH) -c \"import sys; print 'lib/python%i.%i/site-packages' % sys.version_info[:2]\"`\n+PYLIBDIR=$(shell $(PYTHON_PATH) -c \\\n+\t \"import sys; \\\n+\t print 'lib/python%i.%i/site-packages' % sys.version_info[:2]\")\n \n all: $(pysetupfile)\n \t$(QUIET)$(PYTHON_PATH) $(pysetupfile) $(QUIETSETUP) build\n+\n install: $(pysetupfile)\n \t$(PYTHON_PATH) $(pysetupfile) install --prefix $(DESTDIR_SQ)$(prefix)\n \n instlibdir: $(pysetupfile)\n \t@echo \"$(prefix)/$(PYLIBDIR)\"\n+\n clean:\n \t$(QUIET)$(PYTHON_PATH) $(pysetupfile) $(QUIETSETUP) clean -a\n \t$(RM) *.pyo *.pyc\n-- \n1.6.4.314.g034e1\n"},{"id":"120813","messageId":"200908170358.30347.johan@herland.net","threadId":"20543","inReplyTo":"20090816210300.GB23522@gmail.com","subject":"Re: [PATCH 1/2] git_remote_cvs: Honor DESTDIR in the Makefile","fromName":"Johan Herland","fromEmail":"johan@herland.net","sentAt":"2009-08-17T01:58:30Z","receivedAt":"2009-08-17T01:58:30Z","isPatch":true,"sender":{"key":"johan@herland.net","avatar":"https://avatars.githubusercontent.com/u/547031?v=4"},"body":"On Sunday 16 August 2009, David Aguilar wrote:\n> I'll see if we rework this so that we end up passing \"\" to\n> --root instead of /. I'm going to be gone for a few hours so\n> probably won't be able to try it out until tonight.\n\nThanks a lot for your work! I will send an updated series shortly which will \ninclude v2 of your DESTDIR/Makefile fixes, and also the fixes you suggested \nearlier (including _lots_ of PEP8 fixes).\n\n> Another thing to consider --\n>\n> Debian once submitted a bug against another Python app asking\n> that we not place modules in site-packages unless we\n> plan on having other applications importing those modules.\n>\n> The more appropriate place for them if we don't plan on that is\n> $(prefix)/share/git-core/git_remote_cvs or something like that.\n>\n> I guess that's another thing to think about.\n\nYes, Debian raises a valid point. I haven't thought much about making the \ngit_remote_cvs package into something that would be useful for other \napplications. (I just assumed that the Python convention was to install it \ninto site-packages regardless...) For now, I'll concentrate on git-remote-\ncvs, and leave it to others to figure out if anything in the git_remote_cvs \npackage is useful for other programs.\n\nNote that there's a small chicken-and-egg problem here as well: If Debian \nrefuses us to install into site-packages, it will be harder for other Python \nprograms to discover (and import) the git_remote_cvs package.\n\nBTW, when we're on the subject of packaging: There are some variables in \ngit_remote_cvs/setup.py where I'm not sure what the correct value should be:\n\n- version - should this follow Git's version number, or is it independent?\n- author (and author_email + url) - For now, I'm referring to the Git \ncommunity. Should this be more specific/\n\nFeedback welcome.\n\n\nHave fun! :)\n\n...Johan\n\n-- \nJohan Herland, \nwww.herland.net\n"}]}