{"thread":{"id":"11683","subject":"[PATCH] fast importer for SCCS files","startedAt":"2008-01-20T11:12:47Z","lastAt":"2008-01-22T04:10:48Z","messageCount":6,"participants":["James Youngman","Junio C Hamano","Miles Bader","Sam Vilain"],"isPatch":true,"patchVersion":1,"patchTotal":null},"messages":[{"id":"66031","messageId":"c5df85930801200312o7cd5d307v1a39fb35179249a9@mail.gmail.com","threadId":"11683","inReplyTo":null,"subject":"[PATCH] fast importer for SCCS files","fromName":"James Youngman","fromEmail":"jay@gnu.org","sentAt":"2008-01-20T11:12:47Z","receivedAt":"2008-01-20T11:12:47Z","isPatch":true,"sender":{"key":"jay@gnu.org","avatar":"https://gravatar.com/avatar/8b346a7ca8e7b6f8ca5db01dd6cdb5a5d886013dfbf23105186fb8b036534e67?d=mp&s=160"},"body":"The attached patch implements an importer for SCCS files using\ngit-fastimport.  It's unidirectional SCCS->git.  This is my first git\npatch, so apologies if I have done something the wrong way with regard\nto how one shoudl submit a patch.    I've attached the patch because\nmy mail client would probably screw the formatting up if I just pasted\nit :(\n\nThanks,\nJames.\n\n\nFrom 64c49a2ee864d50280df06b0f04d17d718f187c2 Mon Sep 17 00:00:00 2001\nFrom: James Youngman <jay@gnu.org>\nDate: Sun, 20 Jan 2008 11:02:10 +0000\nSubject: [PATCH] Add a fast importer for SCCS.\n\n---\n contrib/fast-import/git-sccsimport.py |  733 +++++++++++++++++++++++++++++++++\n 1 files changed, 733 insertions(+), 0 deletions(-)\n create mode 100755 contrib/fast-import/git-sccsimport.py\n\ndiff --git a/contrib/fast-import/git-sccsimport.py b/contrib/fast-import/git-sccsimport.py\nnew file mode 100755\nindex 0000000..06980d8\n--- /dev/null\n+++ b/contrib/fast-import/git-sccsimport.py\n@@ -0,1 +1,733 @@\n+#!/usr/bin/env python\n+#\n+# git-sccsimport.py -- Imports SCCS files into git\n+#\n+# Author: James Youngman <jay@gnu.org>\n+# Copyright: 2008 James Youngman <jay@gnu.org>\n+# License: GNU GPL version 2 or later <http://www.gnu.org/licenses/old-licenses/gpl-2.0.html>\n+#\n+\"\"\"\n+A fast git importer for SCCS files.\n+\n+How to use this program:\n+\n+Let's assume you have some SCCS files in $PROJECTDIR and want to\n+convert them to a git repository in the directory $NEWGIT.\n+\n+First, make sure the SCCS (or CSSC) binaries are on your $PATH.  Then\n+do this:\n+\n+  cd \"$PROJECTDIR\"\n+  python git-sccsimport.py --init --git-dir=\"$NEWGIT\" --dirs .\n+\n+Notice that git-sccsimport is being run with its working\n+directory as the root of the tree of SCCS files.  This is\n+becaue the relative path from your working directory to the\n+SCCS file is also used as the name of the file in the resulting\n+git repository.\n+\n+I tried this on a 32M code repository in SCCS and it produced a\n+36M git repository. \n+\"\"\"\n+import datetime\n+import errno\n+import optparse\n+import os\n+import os.path\n+import re\n+import stat\n+import subprocess\n+import sys\n+import time\n+\n+GET = \"get\"\n+PRS = \"prs\"\n+SCCS_ESCAPE = chr(1)\n+TIMEZONE = None\n+MAIL_DOMAIN = None\n+UNIX_EPOCH = time.mktime(datetime.datetime(1970, 1, 1,\n+\t\t\t\t\t\t\t\t\t\t\t0, 0, 0, 0,\n+\t\t\t\t\t\t\t\t\t\t\tNone).timetuple())\n+IMPORT_REF = None\n+\n+\n+# Two checkins separated by more than FUZZY_WINDOW\n+# will never be considered part of the same commit.\n+FUZZY_WINDOW = 300.0\n+\n+\n+verbose = False\n+\n+\n+class ImportFailure(Exception):\n+  pass\n+\n+class UsageError(Exception):\n+  pass\n+\n+class AbstractClassError(Exception):\n+  pass\n+\n+def Usage(who, retval, f, e):\n+  if e:\n+\t print >>f, e\n+  print >>f, (\"usage: %s sccs-file [sccs-file...]\"\n+\t\t\t   % (who,))\n+  if retval:\n+\t sys.exit(retval)\n+  return retval\n+\n+\n+def NotImporting(file, sid, reason):\n+  if sid:\n+\t msg = (\"%s: not importing SID %s: %s\"\n+\t\t\t% (file, sid, reason))\n+  else:\n+\t msg = (\"%s: not importing this file: %s\"\n+\t\t\t% (file, reason))\n+  print >> sys.stderr, msg\n+\n+\n+def DiagnoseCommandFailure(command, returncode, errors):\n+  if not errors:\n+\t errors = \"\"\n+  if returncode < 0:\n+\t msg = (\"%s %s killed by signal %s\" % command, -(returncode))\n+  else:\n+\t msg = (\"%s %s returned exit status %s\" % command, returncode)\n+  raise ImportFailure, \"%s\\n%s\" % (errors, msg)\n+\t \n+\n+def RunCommand(commandline):\n+  try:\n+\t if verbose:\n+\t   msg = (\"Running command: %s\\n\"\n+\t\t\t  % (\" \".join(commandline),))\n+\t   sys.stderr.write(msg)\n+\t child = subprocess.Popen(commandline,\n+\t\t\t\t\t\t\t  close_fds = True,\n+\t\t\t\t\t\t\t  stdin =subprocess.PIPE,\n+\t\t\t\t\t\t\t  stdout=subprocess.PIPE,\n+\t\t\t\t\t\t\t  stderr=subprocess.PIPE)\n+\t output, errors = child.communicate(None)\n+\t if child.returncode != 0:\n+\t   # Some stderr output is normal (warnings, etc.)\n+\t   DiagnoseCommandFailure(commandline[0], child.returncode, errors)\n+\t else:\n+\t   return output\n+  except OSError, oe:\n+\t msg = (\"Failed to run '%s': %s (%s)\"\n+\t\t\t% (commandline[0], oe,\n+\t\t\t   errno.errorcode[oe.errno]))\n+\t raise ImportFailure, msg\n+\t\t \n+\n+\t \n+\n+def GetBody(sfile, seqno, expand_keywords):\n+  options = [GET, \"-p\", \"-s\", (\"-a%d\" % seqno)]\n+  if not expand_keywords:\n+\t options.append(\"-k\")\n+  options.append(sfile)\n+  return RunCommand(options)\n+\t \n+\n+def FileMode(filename):\n+  return stat.S_IMODE(os.stat(filename).st_mode)\n+\n+\n+class SccsFileQueryBase(object):\n+  @staticmethod\n+  def HeaderLines(filename):\n+\t header_end = \"^%cT\" % (SCCS_ESCAPE,)\n+\t result = []\n+\t try:\n+\t   f = open(filename, \"rb\")\n+\t   for line in f.readlines():\n+\t\t if line.startswith(header_end):\n+\t\t   break\n+\t\t else:\n+\t\t   result.append(line)\n+\t   return result\n+\t finally:\n+\t   f.close()\n+\n+\t\t \n+\n+#class SccsFileQuerySlow(SccsFileQueryBase):\n+#\t\"\"\"Extract information from SCCS files by running prs.\"\"\"\n+#\t@staticmethod\n+#\tdef RunPrs(options):\n+#\t  commandline = [PRS]\n+#\t  commandline.extend(options)\n+#\t  return RunCommand(commandline)\n+#\t\n+#\t@staticmethod\n+#\tdef FetchDeltaProperties(sid, filename):\n+#\t  fmt = (\":Dy:/:Dm:/:Dd:%(esc)c\"  # 0 delta creation date\n+#\t\t\t \":Th:::Tm:::Ts:%(esc)c\"  # 1 delta creation time (24h)\n+#\t\t\t \":C:%(esc)c\"\t\t\t  # 2 checkin comments\n+#\t\t\t \":DS:%(esc)c\"\t\t\t  # 3 seqno\n+#\t\t\t \":DP:%(esc)c\"\t\t\t  # 4 parent seqno\n+#\t\t\t \":DT:%(esc)c\"\t\t\t  # 5 delta type (R or D)\n+#\t\t\t \":I:%(esc)c\"\t\t\t  # 6 SID\n+#\t\t\t \":MR:%(esc)c\"\t\t\t  # 7 MR numbers\n+#\t\t\t \":P:%(esc)c\"\t\t\t  # 8 Perpetrator (committer)\n+#\t\t\t % { 'esc': SCCS_ESCAPE })\n+#\t  cmdline = [(\"-d%s\" % fmt), (\"-r%s\" % sid), filename]\n+#\t  propdata = SccsFileQuerySlow.RunPrs(cmdline)\n+#\t  return propdata.split(SCCS_ESCAPE)\n+#\n+#\t@staticmethod\n+#\tdef IsValidSccsFile(filename):\n+#\t  try:\n+#\t\toutput = SccsFileQuerySlow.RunPrs([\"-d:F: \", filename])\n+#\t\treturn True\n+#\t  except ImportFailure:\n+#\t\treturn False\n+#\t  \n+#\t@staticmethod\n+#\tdef GetRevisionList(filename):\n+#\t  revisions = SccsFileQuerySlow.RunPrs([\"-d:I: \", \"-e\", filename])\n+#\t  return revisions.split()\n+\n+\n+class SccsFileQueryFast(SccsFileQueryBase):\n+  \"\"\"Extract information from SCCS files by parsing them directly.\"\"\"\n+  DELTA_RE = re.compile(\"^%cd D ([.0-9]*)\" % (SCCS_ESCAPE,))\n+\n+  @staticmethod\n+  def IsValidSccsFile(filename):\n+\t try:\n+\t   f = open(filename, \"rb\")\n+\t   header = f.read(2)\n+\t   return header[0] == SCCS_ESCAPE and header[1] == 'h'\n+\t finally:\n+\t   f.close()\n+\n+  @staticmethod\n+  def GetRevisionList(filename):\n+\t result = []\n+\t header_end = \"^%cT\" % (SCCS_ESCAPE,)\n+\t for line in SccsFileQueryBase.HeaderLines(filename):\n+\t   m = SccsFileQueryFast.DELTA_RE.match(line)\n+\t   if m:\n+\t\t result.append(m.group(1))\n+\t return result\n+\n+  @staticmethod\n+  def FetchDeltaProperties(sid, filename):\n+\t found = False\n+\t props = []\n+\t comments = []\n+\t comment_leader = \"%cc\" % (SCCS_ESCAPE,)\n+\t for line in SccsFileQueryBase.HeaderLines(filename):\n+\t   if not found:\n+\t\t m = SccsFileQueryFast.DELTA_RE.match(line)\n+\t\t if m and (sid == m.group(1)):\n+\t\t   found = True\n+\t\t   fields = line.split()\n+\t\t   props = [ fields[3],\n+\t\t\t\t\t fields[4],\n+\t\t\t\t\t None, # checkin comment\n+\t\t\t\t\t fields[6],\n+\t\t\t\t\t fields[7],\n+\t\t\t\t\t 'D',\n+\t\t\t\t\t fields[2],\n+\t\t\t\t\t \"\",\t   # MR list\n+\t\t\t\t\t fields[5],\n+\t\t\t\t\t \"yodel\"\n+\t\t\t\t\t ]\n+\t\t   continue\n+\t\t else:\n+\t\t   if found:\n+\t\t\t if line.startswith(comment_leader):\n+\t\t\t   comments.append(line[3:])\n+\t\t\t else:\n+\t\t\t   break\n+\t if found:\n+\t   props[2] = \"\\n\".join(comments)\n+\t   # print >> sys.stderr, \"props: %s\" % (props,)\n+\t   return props\n+\t else:\n+\t   return None\n+\n+\n+def SccsFileQuery():\n+  \"\"\"Factory method for objects that query SccsFileQuery objects\"\"\"\n+  return SccsFileQueryFast()\n+\n+\n+class Delta(object):\n+  \"\"\"Represents the properties of an SCCS delta that we\n+  import into git.\"\"\"\n+  def __init__(self, sccsfile, sid, qif, *args, **kwargs):\n+\t super(Delta, self).__init__(*args, **kwargs)\n+\t self._sccsfile = sccsfile\n+\t self._sid\t\t= sid\n+\t self._qif = qif\n+\t self.FetchDeltaProperties()\n+\n+  def __repr__(self):\n+\t return 'Delta(%s, \"%s\")' % (repr(self._sccsfile), self._sid)\n+\n+  def FetchDeltaProperties(self):\n+\t \"\"\"Query the properties of this delta from the SCCS file.\"\"\"\n+\t props = self._qif.FetchDeltaProperties(self._sid,\n+\t\t\t\t\t\t\t\t\t\t\tself._sccsfile._filename)\n+\t assert len(props)>1, \"%s %s %s\" % (self._sccsfile._filename, self._sid, props)\n+\t self.SetTimestamp(props[0], props[1])\n+\t (self._comment, self._seqno, self._parent_seqno, self._type,\n+\t  sidcheck, mrlist, self._comitter, _) = props[2:]\n+\t\t \n+\t self._seqno = int(self._seqno)\n+\t self._parent_seqno = int(self._parent_seqno)\n+\t if self._comment == \"\\n\":\n+\t\t self._comment = None\n+\t assert sidcheck==self._sid\n+\t self._mrs = mrlist.split()\n+\n+  def SameFuzzyCommit(self, other):\n+\t #print >>sys.stderr, (\"SameFuzzyCommit: comparing\\n1: %s with\\n2: %s\"\n+\t #\t\t\t\t\t   % (self, other))\n+\t if self._comment != other._comment:\n+\t   return False\n+\t elif self._comitter != other._comitter:\n+\t   return False\n+\t elif self._mrs != other._mrs:\n+\t   return False\n+\t else:\n+\t   delta = abs(other._timestamp - self._timestamp)\n+\t   if delta > FUZZY_WINDOW:\n+\t\t return False\n+\t   else:\n+\t\t return True\n+\n+  def SetTimestamp(self, checkin_date, checkin_time):\n+\t   try:\n+\t\t year, month, monthday = [int(f) for f in checkin_date.split(\"/\")]\n+\t   except ValueError:\n+\t\t raise ImportFailure, (\"Unexpected date format: %s\"\n+\t\t\t\t\t\t\t   % (checkin_date,))\n+\t   try:\n+\t\t h,m,s = [int(f) for f in checkin_time.split(\":\")]\n+\t   except ValueError:\n+\t\t raise ImportFailure, (\"Unexpected time format: %s\"\n+\t\t\t\t\t\t\t   % (checkin_time,))\n+\t   microsec = 0\n+\n+\t   if year < 100:\n+\t\t # Apply the y2k rule (see the \"Year 2000 Issues\" section\n+\t\t # of the CSSC documentation).\n+\t\t if year < 69:\n+\t\t   year += 2000\n+\t\t else:\n+\t\t   year += 1900\n+\n+\t   dt = datetime.datetime(year, month, monthday,\n+\t\t\t\t\t\t\t  h, m, s,\n+\t\t\t\t\t\t\t  microsec, TIMEZONE)\n+\t   epoch_offset = time.mktime(dt.timetuple())\n+\t   # We subtract UNIX_EPOCH to take account of the fact\n+\t   # that the system epoch may in fact not be the same as the Unix\n+\t   # epoch.\t git fast-import requires the timestamp to be measured in\n+\t   # seconds since the Unix epoch.\n+\t   self._timestamp = epoch_offset - UNIX_EPOCH\n+\t\t \n+  def GitTimestamp(self):\n+\t n = int(self._timestamp)\n+\t return \"%d +0000\" % (n,)\n+\n+\n+class SccsFile(object):\n+  def __init__(self, name, *args, **kwargs):\n+\t super(SccsFile, self).__init__(*args, **kwargs)\n+\t self._filename = name\n+\t qif = SccsFileQuery()\n+\t revisions = filter(self.GoodRevision, qif.GetRevisionList(name))\n+\t self._deltas = [Delta(self, sid, qif) for sid in revisions]\n+\t self._gitname = self.GitFriendlyName(self._GottenName())\n+\t if FileMode(self._filename) & 0111:\n+\t   self._gitmode = \"755\"\n+\t else:\n+\t   self._gitmode = \"644\"\n+\t \n+  gitname = property(lambda self: self._gitname)\n+  gitmode = property(lambda self: self._gitmode)\n+  filename = property(lambda self: self._filename)\n+\n+  @staticmethod\n+  def IsValidSccsFile(filename):\n+\t qif = SccsFileQuery()\n+\t return qif.IsValidSccsFile(filename)\n+\n+  def __repr__(self):\n+\t return \"SccsFile(r'%s')\" % self._filename\n+\n+  @staticmethod\n+  def GitFriendlyName(name):\n+\t \"\"\"git-fastimport does not like leading or trailing slashes,\n+\t or . or .. in file names.\tThis method removes such things.\"\"\"\n+\t if os.path.isabs(name):\n+\t   raise ImportFailure, \"%s is an absolute path name\" % (name,)\n+\t drive, path = os.path.splitdrive(name)\n+\t return os.path.normpath(path)\n+\n+  def _GottenName(self):\n+\t \"\"\"Convert the name of an SCCS file to the name of the\n+\t working file as would be assumed by make.\tSpecifically,\n+\t the s. prefix is stipped, and if the s.file is in a directory\n+\t called SCCS, that directory is stripped out of the path also.\n+\t \"\"\"\n+\t head, tail = os.path.split(self._filename)\n+\t if tail.startswith(\"s.\"):\n+\t   tail = tail[2:]\n+\t dirhead, dirtail = os.path.split(head)\n+\t if dirtail == \"SCCS\":\n+\t   head = dirhead\n+\t return os.path.join(head, tail)\n+\n+\n+  def GoodRevision(self, sid):\n+\t comps = sid.split(\".\")\n+\t if len(comps) > 1:\n+\t   return True\n+\t else:\n+\t   NotImporting(self._filename, sid,\n+\t\t\t\t\t\"SID contains no level component\")\n+\t   return False\n+\n+\t\t \n+\t\t \n+\n+class GitImporter(object):\n+  \"\"\"Handles communications with git-fast-import.\"\"\"\n+  \n+  def __init__(self, *args, **kwargs):\n+\t super(GitImporter, self).__init__(*args, **kwargs)\n+\t self._next_mark = 1\n+\t self._command = None\n+\t self._importer = None\n+\t self._write_to_stdout = False\n+\n+  def StartImporter(self):\n+\t args = [\"git\",\"fast-import\"]\n+\t self._command = \" \".join(args)\n+\t self._importer = subprocess.Popen(args,\n+\t\t\t\t\t\t\t\t\t   close_fds=True,\n+\t\t\t\t\t\t\t\t\t   stdin=subprocess.PIPE)\n+\n+  def SendToStdout(self):\n+\t self._command = None\n+\t self._importer = None\n+\t self._write_to_stdout = True\n+\n+  def GetNextMark(self):\n+\t \"\"\"Get the next unused idnum for a mark statement.\"\"\"\n+\t result = self._next_mark\n+\t self._next_mark += 1\n+\t return result\n+\n+  def Write(self, s):\n+\t \"\"\"Write some data to the importer.\"\"\"\n+\t if self._write_to_stdout:\n+\t   sys.stdout.write(s)\n+\t else:\n+\t   assert self._importer\n+\t   self._importer.stdin.write(s)\n+\t   \n+  def Done(self):\n+\t if self._importer:\n+\t   self._importer.stdin.close()\n+\t   returncode = self._importer.wait()\n+\t   if returncode != 0:\n+\t\t DiagnoseCommandFailure(self._command, returncode, None)\n+\t   else:\n+\t\t print >>sys.stderr, (\"%s completed successfully\"\n+\t\t\t\t\t\t\t  % (self._command,))\n+\t\t \n+  def ProgressMsg(self, msg):\n+\t \"\"\"Emit a progress message for the user.\"\"\"\n+\t sys.stderr.write(msg)\n+\n+  def Progress(self, done, items):\n+\t \"\"\"Inform the user of our current progress.\"\"\"\n+\t percent = (100.0 * float(done)) / float(items)\n+\t if done == items:\n+\t   tail = \" done\\n\"\n+\t else:\n+\t   tail = \"\"\n+\t msg = \"\\r %3.0f%% (%d/%d)%s\" % (percent, done, items, tail)\n+\t self.ProgressMsg(msg)\n+\n+  def GetUserEmailAddress(self, login_name):\n+\t \"\"\"Get an email address corresponding to the given login name.\"\"\"\n+\t if MAIL_DOMAIN:\n+\t   return \"%s@$s\" % (login_name,MAIL_DOMAIN)\n+\t else:\n+\t   return login_name\n+\n+  def WriteData(self, data):\n+\t \"\"\"Emit a data command followd by a blob of data.\"\"\"\n+\t self.Write(\"data %d\\n\" % (len(data),))\n+\t self.Write(data)\n+\t self.Write(\"\\n\")\n+\t\t\t \n+  def BeginCommit(self, delta, parent):\n+\t \"\"\"Start a new commit (having the indicated parent).\"\"\"\n+\t mark = self.GetNextMark()\n+\t self.Write(\"commit %s\\nmark :%d\\n\" % (IMPORT_REF, mark))\n+\t ts = delta.GitTimestamp()\n+\t self.Write(\"committer %s <%s> %s\\n\"\n+\t\t\t\t% (delta._comitter,\n+\t\t\t\t   self.GetUserEmailAddress(delta._comitter),\n+\t\t\t\t   ts))\n+\t if delta._comment:\n+\t   self.WriteData(delta._comment)\n+\t else:\n+\t   self.WriteData(\"\")\t\t   # commit comment is mandatory\n+\t if parent:\n+\t   self.Write(\"from :%d\\n\" % (parent,))\n+\t return mark\n+\n+  def Filemodify(self, sfile, body):\n+\t \"\"\"Write a filemodify section of a commit.\"\"\"\n+\t self.Write(\"M %s inline %s\\n\" % (sfile.gitmode, sfile.gitname))\n+\t self.WriteData(body)\n+\n+  def CompleteCommit(self):\n+\t \"\"\"Write the final part of a commit.\"\"\"\n+\t self.Write(\"\\n\")\n+\t\t \n+# TODO: if the fuzzy commit logic puts subsequent deltas into the same\n+# commit, the timestamp of the commit is that of the first delta.\tOne\n+# could argue that the timestamp of the last one would be a better\n+# choice.\n+\n+def ImportDeltas(imp, deltas):\n+  if not deltas:\n+\t raise ImportFailure, \"No deltas to import\"\n+  first_delta_in_commit = None\n+  done = 0\n+  imp.ProgressMsg(\"Creating commits...\\n\")\n+  parent = None\n+  commit_count = 0\n+  for d in deltas:\n+\t imp.Progress(done, len(deltas))\n+\t done += 1\n+\t # Figure out if we need to start a new commit.\n+\t if first_delta_in_commit:\n+\t   if not first_delta_in_commit.SameFuzzyCommit(d):\n+\t\t imp.CompleteCommit()\n+\t\t first_delta_in_commit = None\n+\t if first_delta_in_commit is None:\n+\t   first_delta_in_commit = d\n+\t   current = imp.BeginCommit(d, parent)\n+\t   commit_count += 1\n+\t   parent = current\n+\t # We're now in a commit.  Emit the body for this delta.\n+\t body = GetBody(d._sccsfile._filename, d._seqno, False)\n+\t imp.Filemodify(d._sccsfile, body)\n+  # Finished looping over deltas\n+  if parent:\n+\t   imp.CompleteCommit()\n+  imp.Progress(done, len(deltas))\n+  print >>sys.stderr, (\"%d SCCS deltas in %d git commits\"\n+\t\t\t\t\t\t% (len(deltas), commit_count))\n+\n+\n+def Import(filenames, stdout):\n+  \"\"\"Import the indicated SCCS files into git.\"\"\"\n+  for filename in filenames:\n+\t if not os.access(filename, os.R_OK):\n+\t   msg = \"%s is not readable\" % (filename,)\n+\t   raise ImportFailure, msg\n+\t if not os.path.isfile(filename):\n+\t   msg = \"%s is not a file\" % (filename,)\n+\t   raise ImportFailure, msg\n+  sccsfiles = []\n+  done = 0\n+  imp = GitImporter()\n+  imp.ProgressMsg(\"Reading metadata from SCCS files...\\n\")\n+  for filename in filenames:\n+\t try:\n+\t   imp.Progress(done, len(filenames))\n+\t   if SccsFile.IsValidSccsFile(filename):\n+\t\t sf = SccsFile(filename)\n+\t\t sccsfiles.append(sf)\n+\t   else:\n+\t\t NotImporting(filename, None, \"not a valid SCCS file\")\n+\t   done += 1\n+\t except ImportFailure:\n+\t   msg = (\"\\nAn import failure occurred while processing %s\"\n+\t\t\t  % (filename,))\n+\t   print >> sys.stderr, msg\n+\t   raise\n+  imp.Progress(done, len(filenames))\n+  # Now we have all the metadata; sort the deltas by timestamp\n+  # and import the deltas in time order.\n+  delta_list = []\n+  for sfile in sccsfiles:\n+\t for delta in sfile._deltas:\n+\t   ts = \"%15f\" % (delta._timestamp,)\n+\t   t = (ts, delta)\n+\t   delta_list.append(t)\n+  delta_list.sort()\n+\n+  if stdout:\n+\t imp.SendToStdout()\n+  else:\n+\t imp.StartImporter()\n+  try:\n+\t ImportDeltas(imp, [d[1] for d in delta_list])\n+  finally:\n+\t imp.Done()\n+\n+\n+def IsValidGitDir(path):\n+  \"\"\"Returns True IFF path is a valid git repo.\n+\n+  Copied from git-p4.\n+  \"\"\"\n+  if (os.path.exists(path + \"/HEAD\")\n+\t   and os.path.exists(path + \"/refs\")\n+\t   and os.path.exists(path + \"/objects\")):\n+\t return True;\n+  return False\n+\n+\n+def FindGitDir(gitdir, init):\n+  \"\"\"Locate the git repository.\"\"\"\n+  if not gitdir:\n+\t gitdir = RunCommand([\"git\", \"rev-parse\", \"--git-dir\"])\n+\t if gitdir:\n+\t   gitdir = gitdir.strip()\n+\t else:\n+\t   cdup = RunCommand([\"git\", \"rev-parse\", \"--show-cdup\"])\n+\t   if cdup:\n+\t\t gitdir = os.path.abspath(cdup.strip())\n+\t   else:\n+\t\t gitdir = \".\"\n+  if init:\n+\t # Do a sanity check that we're not about to create /.git.\n+\t assert gitdir\n+\t if not gitdir.endswith(\"/.git\"):\n+\t   gitdir += \"/.git\"\n+\t if IsValidGitDir(gitdir):\n+\t   raise ImportFailure, (\"Git repository %s was already initialised\"\n+\t\t\t\t\t\t\t % (gitdir,))\n+\t else:\n+\t   RunCommand([\"git\", \"--git-dir=%s\" % (gitdir,), \"init\"])\n+\t   \n+  if not IsValidGitDir(gitdir):\n+\t gitdir += \"/.git\"\n+\t if not IsValidGitDir(gitdir):\n+\t   raise ImportFailure, (\"cannot locate git repository at %s\" %\n+\t\t\t\t\t\t\t (gitdir,))\n+  print >> sys.stderr, \"git repository is at\", gitdir\n+  return gitdir\n+\n+\n+def MakeDirWorklist(dirs):\n+  result = []\n+  for dirname in dirs:\n+\t for root, subdirs, files in os.walk(dirname):\n+\t   for f in files:\n+\t\t if f.startswith(\"s.\"):\n+\t\t   physpath = os.path.join(root, f)\n+\t\t   if os.access(physpath, os.R_OK):\n+\t\t\t result.append(physpath)\n+  if not result:\n+\t print >>sys.stderr, (\"Warning: No SCCS files were found in %s\"\n+\t\t\t\t\t\t  % (\" \".join(dirs)))\n+  return result\n+\n+\n+def ParseOptions(argv):\n+  global IMPORT_REF\n+  global MAIL_DOMAIN\n+  global FUZZY_WINDOW\n+  global verbose\n+  parser = optparse.OptionParser()\n+  parser.add_option(\"--branch\",\n+\t\t\t\t\t help=\"branch to populate\",\n+\t\t\t\t\t default=\"master\")\n+  parser.add_option(\"--maildomain\",\n+\t\t\t\t\t help=\"Mail domain for usernames taken from SCCS files\")\n+  parser.add_option(\"--fuzzy-commit-window\",\n+\t\t\t\t\t default=FUZZY_WINDOW,\n+\t\t\t\t\t help=(\"Deltas more than this many seconds apart \"\n+\t\t\t\t\t\t   \"are always considered to be in different commits\"))\n+  parser.add_option(\"--git-dir\",\n+\t\t\t\t\t help=\"Directory containing the git repository\")\n+  parser.add_option(\"--init\", default=False, action=\"store_true\",\n+\t\t\t\t\t help=\"Initialise the git repository first\")\n+  parser.add_option(\"--verbose\", default=False, action=\"store_true\",\n+\t\t\t\t\t help=\"Print verbose status messages\")\n+  parser.add_option(\"--stdout\", default=False, action=\"store_true\",\n+\t\t\t\t\t help=(\"Send git-fast-import data to stdout \"\n+\t\t\t\t\t\t   \"rather than to git-fast-import\"))\n+  parser.add_option(\"--dirs\",\n+\t\t\t\t\t action=\"store_true\",\n+\t\t\t\t\t help=(\"Command-line arguments are a list \"\n+\t\t\t\t\t\t   \"of directories to automatically search \"\n+\t\t\t\t\t\t   \"rather than a list of SCCS files\"))\n+  (options, args) = parser.parse_args(argv)\n+  verbose = options.verbose\n+  if options.maildomain:\n+\t MAIL_DOMAIN = options.maildomain\n+  try:\n+\t FUZZY_WINDOW = float(options.fuzzy_commit_window)\n+  except ValueError:\n+\t raise UsageError(\"The argument for the --fuzzy-commit-window option \"\n+\t\t\t\t\t  \"should be a number, but you specified '%s'\"\n+\t\t\t\t\t  % (options.fuzzy_commit_window,))\n+  IMPORT_REF = \"refs/heads/%s\" % (options.branch,)\n+  return options, args\n+\n+\n+def main(argv):\n+  progname = argv[0] or \"git-sccsimport\"\n+  try:\n+\t options, args = ParseOptions(argv)\n+\t print >>sys.stderr, \"Positional args:\", \" \".join(args)\n+\t args = args[1:]\n+\n+\t if not args:\n+\t   if options.dirs:\n+\t\t raise UsageError(\"You did not specify any directories to import from\")\n+\t   else:\n+\t\t raise UsageError(\"You did not specify any SCCS files to import\")\n+\t \n+\t if options.stdout and options.init:\n+\t   raise UsageError(\"The --init option is incompatible \"\n+\t\t\t\t\t\t\"with the --stdout option\")\n+\n+\t if not options.stdout:\n+\t   try:\n+\t\t os.environ[\"GIT_DIR\"] = FindGitDir(options.git_dir, options.init)\n+\t   except ImportFailure, init_failure:\n+\t\t if options.init:\n+\t\t   action = \"initialise\"\n+\t\t else:\n+\t\t   action = \"locate\"\n+\t\t   print >>sys.stderr, (\"Failed to %s the git repository: %s\"\n+\t\t\t\t\t\t\t\t% (action, init_failure))\n+\t\t   return 1\n+\n+\t if options.dirs:\n+\t   items = MakeDirWorklist(args)\n+\t else:\n+\t   items = args\n+\t print >>sys.stderr, \"Importing items:\", \" \".join(items)\n+\t return Import(items, options.stdout)\n+\n+  except UsageError, usage_err:\n+\t return Usage(progname, 1, sys.stderr, usage_err)\n+  except ImportFailure, imp_failure:\n+\t print >> sys.stderr, \"Import failed: %s\" % (imp_failure,)\n+\t return 1\n+\n+\n+if __name__ == '__main__':\n+  sys.exit(main(sys.argv))\n-- \n1.5.3.8\n\n"},{"id":"66065","messageId":"7vsl0s57e8.fsf@gitster.siamese.dyndns.org","threadId":"11683","inReplyTo":"c5df85930801200312o7cd5d307v1a39fb35179249a9@mail.gmail.com","subject":"Re: [PATCH] fast importer for SCCS files","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2008-01-21T00:50:23Z","receivedAt":"2008-01-21T00:50:23Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"\"James Youngman\" <jay@gnu.org> writes:\n\n> The attached patch implements an importer for SCCS files using\n> git-fastimport.  It's unidirectional SCCS->git.  This is my first git\n> patch, so apologies if I have done something the wrong way with regard\n> to how one shoudl submit a patch.    I've attached the patch because\n> my mail client would probably screw the formatting up if I just pasted\n> it :(\n\nWelcome.\n\n> From 64c49a2ee864d50280df06b0f04d17d718f187c2 Mon Sep 17 00:00:00 2001\n> From: James Youngman <jay@gnu.org>\n> Date: Sun, 20 Jan 2008 11:02:10 +0000\n> Subject: [PATCH] Add a fast importer for SCCS.\n>\n> ---\n\nLacks sign-off (please see Documentation/SubmittingPatches).\n\nA description?  For example, \"It's unidirectional\" should be in\nthe commit message.  If it allows incremental or just one shot\nshould also be said.  There perhaps are other things, including\nfuture directions.\n\nAs to future directions, I think unidirectional importers should\nallow incremental updates, and try to be usable as a drop-in\nreplacement of \"git fetch\".  Among the existing ones, cvsimport\nis almost there (instead of doing \"git fetch $url && git rebase\nFETCH_HEAD\", you can do \"git cvsimport $args $cvs_repository &&\ngit rebase $cvs_tracking_branch\"), and I think it is a good\nmodel.\n\n> +# Author: James Youngman <jay@gnu.org>\n> +# Copyright: 2008 James Youngman <jay@gnu.org>\n> +# License: GNU GPL version 2 or later <http://www.gnu.org/licenses/old-licenses/gpl-2.0.html>\n> +#\n\nThis is quite a long line isn't it?  Besides, we ship the copy\nof GPLv2 in git.git already.\n\n> +\"\"\"\n> +A fast git importer for SCCS files.\n> +\n> +How to use this program:\n> +\n> +Let's assume you have some SCCS files in $PROJECTDIR and want to\n> +convert them to a git repository in the directory $NEWGIT.\n> +\n> +First, make sure the SCCS (or CSSC) binaries are on your $PATH.  Then\n> +do this:\n> +\n> +  cd \"$PROJECTDIR\"\n> +  python git-sccsimport.py --init --git-dir=\"$NEWGIT\" --dirs .\n\nSo, from the point of view to imitate git-fetch, this is a\nsuboptimal interface.  Rather:\n\n\tmkdir my-project-converted-to-git\n        cd my-project-converted-to-git\n        git init\n        git-sccsimport $PROJECTDIR\n\nand later incremental option would allow the user to say:\n\n        cd my-project-converted-to-git\n        git-sccsimport $some_incremental_options $PROJECTDIR\n\n> +TIMEZONE = None\n> +MAIL_DOMAIN = None\n> +UNIX_EPOCH = time.mktime(datetime.datetime(1970, 1, 1,\n> +\t\t\t\t\t\t\t\t\t\t\t0, 0, 0, 0,\n> +\t\t\t\t\t\t\t\t\t\t\tNone).timetuple())\n\nYou seem to be using a tab-width that is not 8, and it shows\neverywhere.  Please don't.\n"},{"id":"66080","messageId":"buo7ii3pvwb.fsf@dhapc248.dev.necel.com","threadId":"11683","inReplyTo":"7vsl0s57e8.fsf@gitster.siamese.dyndns.org","subject":"Re: [PATCH] fast importer for SCCS files","fromName":"Miles Bader","fromEmail":"miles.bader@necel.com","sentAt":"2008-01-21T05:53:08Z","receivedAt":"2008-01-21T05:53:08Z","isPatch":true,"sender":{"key":"miles.bader@necel.com","avatar":"https://gravatar.com/avatar/be062d4050eb88e04229cbdb60f803e1bd647923a015996c2439e76f23e336a7?d=mp&s=160"},"body":"Junio C Hamano <gitster@pobox.com> writes:\n> You seem to be using a tab-width that is not 8, and it shows\n> everywhere.\n\nIn a python script, too...\n\n-miles\n\n-- \nAlliance, n. In international politics, the union of two thieves who have\ntheir hands so deeply inserted in each other's pockets that they cannot\nseparately plunder a third.\n"},{"id":"66085","messageId":"7v7ii33d82.fsf@gitster.siamese.dyndns.org","threadId":"11683","inReplyTo":"buo7ii3pvwb.fsf@dhapc248.dev.necel.com","subject":"Re: [PATCH] fast importer for SCCS files","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2008-01-21T06:27:25Z","receivedAt":"2008-01-21T06:27:25Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Miles Bader <miles.bader@necel.com> writes:\n\n> Junio C Hamano <gitster@pobox.com> writes:\n>> You seem to be using a tab-width that is not 8, and it shows\n>> everywhere.\n>\n> In a python script, too...\n\nWe already seem to have two .py file under contrib/ and I do not\nparticularly mind scripts written in Python in contrib/ area.\n"},{"id":"66096","messageId":"fc339e4a0801210001t463360a8t5ff91a1729d87b4b@mail.gmail.com","threadId":"11683","inReplyTo":"7v7ii33d82.fsf@gitster.siamese.dyndns.org","subject":"Re: [PATCH] fast importer for SCCS files","fromName":"Miles Bader","fromEmail":"miles@gnu.org","sentAt":"2008-01-21T08:01:14Z","receivedAt":"2008-01-21T08:01:14Z","isPatch":true,"sender":{"key":"miles@gnu.org","avatar":"https://gravatar.com/avatar/01069b69593af7bff28e2f97afeb3644ae6fe2f5f56cb3a8cf34c5fb8c36efe5?d=mp&s=160"},"body":"On 1/21/08, Junio C Hamano <gitster@pobox.com> wrote:\n> >> You seem to be using a tab-width that is not 8, and it shows\n> >> everywhere.\n> >\n> > In a python script, too...\n>\n> We already seem to have two .py file under contrib/ and I do not\n> particularly mind scripts written in Python in contrib/ area.\n\nActually I was just thinking that \"messed up whitespace\" is a much\ndeadlier sin in Python than in other languages...\n\n-Miles\n\n-- \nDo not taunt Happy Fun Ball.\n"},{"id":"66272","messageId":"47956CC8.3000307@vilain.net","threadId":"11683","inReplyTo":"c5df85930801200312o7cd5d307v1a39fb35179249a9@mail.gmail.com","subject":"Re: [PATCH] fast importer for SCCS files","fromName":"Sam Vilain","fromEmail":"sam@vilain.net","sentAt":"2008-01-22T04:10:48Z","receivedAt":"2008-01-22T04:10:48Z","isPatch":true,"sender":{"key":"sam@vilain.net","avatar":"https://gravatar.com/avatar/8fc840ca854dbf6f7065b4335e3b934951c1dca3b11db688e95e471901f8f4a8?d=mp&s=160"},"body":"James Youngman wrote:\n> The attached patch implements an importer for SCCS files using\n> git-fastimport.\n\nFWIW, here's another;\n\nhttp://search.cpan.org/src/HMBRAND/VCS-SCCS-0.11/examples/sccs2git-gfi\n\nSam.\n"}]}