threads / patch / 12081

patch, 2 partsPatch stack logging

Subject: [StGit PATCH 0/2] Patch stack logging

## tl;dr

8 messages between Feb 14, 2008 and Feb 22, 2008. Diffs are folded; open one to read it.

replies: 7people: 2as markdown or json

Karl Hasselström· Feb 14, 2008, 01:25 UTC · lore

So here's what I have this far. It writes a log that's close to the format I want, but doesn't do anything with it yet.

It's not in kha/experimental yet; it seems to work OK, but it leaves a log behind that'll have to be cleaned up manually when the format changes, which it will.

---
Karl Hasselström (2):
      Write to a stack log when stack is modified
      Library functions for tree and blob manipulation
 stgit/commands/branch.py   |   18 ++++--
 stgit/commands/clone.py    |    3 +
 stgit/commands/float.py    |    3 +
 stgit/commands/fold.py     |    3 +
 stgit/commands/imprt.py    |    3 +
 stgit/commands/init.py     |    3 +
 stgit/commands/pick.py     |    3 +
 stgit/commands/pop.py      |    3 +
 stgit/commands/pull.py     |    3 +
 stgit/commands/push.py     |    6 +-
 stgit/commands/rebase.py   |    4 +
 stgit/commands/refresh.py  |    4 +
 stgit/commands/rename.py   |    3 +
 stgit/commands/resolved.py |    4 +
 stgit/commands/sink.py     |    3 +
 stgit/commands/sync.py     |    5 +-
 stgit/lib/git.py           |  139 +++++++++++++++++++++++++++++++++++++-------
 stgit/lib/log.py           |   60 +++++++++++++++++++
 stgit/lib/transaction.py   |    3 +
 19 files changed, 229 insertions(+), 44 deletions(-)
 create mode 100644 stgit/lib/log.py
-- 
Karl Hasselström, kha@treskal.com
      www.treskal.com/kalle
Karl Hasselström· Feb 14, 2008, 01:29 UTC · re: Karl Hasselström · lore

[StGit PATCH 1/2] Library functions for tree and blob manipulation

Wrap trees and blobs in Python objects (just like commits were already wrapped), so that StGit code can read and write them.

Signed-off-by: Karl Hasselström <kha@treskal.com>
---

I'm quite pleased with the Foo/Foodata distinction (the former being a blob, tree, or commit in the git repository, the latter an immutable specification for such an object). It's convenient to work with, without loss of efficiency.

 stgit/lib/git.py |  139 ++++++++++++++++++++++++++++++++++++++++++++++--------
 1 files changed, 118 insertions(+), 21 deletions(-)
Show changes to stgit/lib/git.py +118 −21
diff --git a/stgit/lib/git.py b/stgit/lib/git.py
index 6ee8a71..8184fec 100644
--- a/stgit/lib/git.py
+++ b/stgit/lib/git.py
@@ -146,13 +146,100 @@ class Person(Repr):
                 defaults = cls.user())
         return cls.__committer
 
-class Tree(Repr):
+class GitObject(Repr):
+    """Base class for all git objects."""
+
+class Blobdata(Repr):
+    def __init__(self, string):
+        self.__string = str(string)
+    str = property(lambda self: self.__string)
+    def commit(self, repository):
+        sha1 = repository.run(['git', 'hash-object', '-w', '--stdin']
+                              ).raw_input(self.str).output_one_line()
+        return repository.get_blob(sha1)
+
+class Blob(GitObject):
     """Immutable."""
-    def __init__(self, sha1):
+    typename = 'blob'
+    default_perm = '100644'
+    def __init__(self, repository, sha1):
+        self.__repository = repository
         self.__sha1 = sha1
     sha1 = property(lambda self: self.__sha1)
     def __str__(self):
-        return 'Tree<%s>' % self.sha1
+        return 'Blob<%s>' % self.sha1
+    @property
+    def data(self):
+        return Blobdata(self.__repository.cat_object(self.sha1))
+
+class ImmutableDict(dict):
+    def error(*args, **kwargs):
+        raise TypeError('Cannot modify immutable dict')
+    __delitem__ = error
+    __setitem__ = error
+    clear = error
+    pop = error
+    popitem = error
+    setdefault = error
+    update = error
+
+class Treedata(Repr):
+    """Immutable."""
+    @staticmethod
+    def __x(po):
+        if isinstance(po, GitObject):
+            perm, object = po.default_perm, po
+        else:
+            perm, object = po
+        return perm, object
+    def __init__(self, entries):
+        self.__entries = ImmutableDict((name, self.__x(po))
+                                       for (name, po) in entries.iteritems())
+    entries = property(lambda self: self.__entries)
+    def set_entry(self, name, po):
+        e = dict(self.entries)
+        e[name] = self.__x(po)
+        return type(self)(e)
+    def del_entry(self, name):
+        e = dict(self.entries)
+        del e[name]
+        return type(self)(e)
+    def commit(self, repository):
+        listing = ''.join(
+            '%s %s %s\t%s\0' % (mode, obj.typename, obj.sha1, name)
+            for (name, (mode, obj)) in self.entries.iteritems())
+        sha1 = repository.run(['git', 'mktree', '-z']
+                              ).raw_input(listing).output_one_line()
+        return repository.get_tree(sha1)
+    @classmethod
+    def parse(cls, repository, s):
+        entries = {}
+        for line in s.split('\0')[:-1]:
+            m = re.match(r'^([0-7]{6}) ([a-z]) ([0-9a-f]{40})\t(.*)$', line)
+            assert m
+            perm, type, sha1, name = m.groups()
+            entries[name] = (perm, repository.get_object(type, sha1))
+        return cls(entries)
+
+class Tree(GitObject):
+    """Immutable."""
+    typename = 'tree'
+    default_perm = '040000'
+    def __init__(self, repository, sha1):
+        self.__sha1 = sha1
+        self.__repository = repository
+        self.__data = None
+    sha1 = property(lambda self: self.__sha1)
+    @property
+    def data(self):
+        if self.__data == None:
+            self.__data = Treedata.parse(
+                self.__repository,
+                self.__repository.run(['git', 'ls-tree', '-z', self.sha1]
+                                      ).raw_output())
+        return self.__data
+    def __str__(self):
+        return 'Tree<sha1: %s>' % self.sha1
 
 class Commitdata(Repr):
     """Immutable."""
@@ -202,6 +289,22 @@ class Commitdata(Repr):
         return ('Commitdata<tree: %s, parents: %s, author: %s,'
                 ' committer: %s, message: "%s">'
                 ) % (tree, parents, self.author, self.committer, self.message)
+    def commit(self, repository):
+        c = ['git', 'commit-tree', self.tree.sha1]
+        for p in self.parents:
+            c.append('-p')
+            c.append(p.sha1)
+        env = {}
+        for p, v1 in ((self.author, 'AUTHOR'),
+                       (self.committer, 'COMMITTER')):
+            if p != None:
+                for attr, v2 in (('name', 'NAME'), ('email', 'EMAIL'),
+                                 ('date', 'DATE')):
+                    if getattr(p, attr) != None:
+                        env['GIT_%s_%s' % (v1, v2)] = str(getattr(p, attr))
+        sha1 = repository.run(c, env = env).raw_input(self.message
+                                                      ).output_one_line()
+        return repository.get_commit(sha1)
     @classmethod
     def parse(cls, repository, s):
         cd = cls()
@@ -223,8 +326,9 @@ class Commitdata(Repr):
                 assert False
         assert False
 
-class Commit(Repr):
+class Commit(GitObject):
     """Immutable."""
+    typename = 'commit'
     def __init__(self, repository, sha1):
         self.__sha1 = sha1
         self.__repository = repository
@@ -302,7 +406,8 @@ class Repository(RunWithEnv):
     def __init__(self, directory):
         self.__git_dir = directory
         self.__refs = Refs(self)
-        self.__trees = ObjectCache(lambda sha1: Tree(sha1))
+        self.__blobs = ObjectCache(lambda sha1: Blob(self, sha1))
+        self.__trees = ObjectCache(lambda sha1: Tree(self, sha1))
         self.__commits = ObjectCache(lambda sha1: Commit(self, sha1))
         self.__default_index = None
         self.__default_worktree = None
@@ -353,26 +458,18 @@ class Repository(RunWithEnv):
                     ).output_one_line())
         except run.RunException:
             raise RepositoryException('%s: No such revision' % rev)
+    def get_blob(self, sha1):
+        return self.__blobs[sha1]
     def get_tree(self, sha1):
         return self.__trees[sha1]
     def get_commit(self, sha1):
         return self.__commits[sha1]
-    def commit(self, commitdata):
-        c = ['git', 'commit-tree', commitdata.tree.sha1]
-        for p in commitdata.parents:
-            c.append('-p')
-            c.append(p.sha1)
-        env = {}
-        for p, v1 in ((commitdata.author, 'AUTHOR'),
-                       (commitdata.committer, 'COMMITTER')):
-            if p != None:
-                for attr, v2 in (('name', 'NAME'), ('email', 'EMAIL'),
-                                 ('date', 'DATE')):
-                    if getattr(p, attr) != None:
-                        env['GIT_%s_%s' % (v1, v2)] = str(getattr(p, attr))
-        sha1 = self.run(c, env = env).raw_input(commitdata.message
-                                                ).output_one_line()
-        return self.get_commit(sha1)
+    def get_object(self, type, sha1):
+        return { Blob.typename: self.get_blob,
+                 Tree.typename: self.get_tree,
+                 Commit.typename: self.get_commit }[type](sha1)
+    def commit(self, objectdata):
+        return objectdata.commit(self)
     @property
     def head(self):
         try:
Karl Hasselström· Feb 14, 2008, 01:32 UTC · re: Karl Hasselström · lore

[StGit PATCH 2/2] Write to a stack log when stack is modified

Create a log branch (called <branchname>.stgit) for each StGit branch, and write to it whenever the stack is modified.

As of yet, nothing can be done with this log.
Signed-off-by: Karl Hasselström <kha@treskal.com>
---

Lots of noise here since all the old-infrastructure commands update the log manually. But note how few lines it takes to actually write the blobs, trees, and commits (in log.py, near the end).

 stgit/commands/branch.py   |   18 ++++++++++---
 stgit/commands/clone.py    |    3 +-
 stgit/commands/float.py    |    3 ++
 stgit/commands/fold.py     |    3 +-
 stgit/commands/imprt.py    |    3 +-
 stgit/commands/init.py     |    3 +-
 stgit/commands/pick.py     |    3 +-
 stgit/commands/pop.py      |    3 +-
 stgit/commands/pull.py     |    3 +-
 stgit/commands/push.py     |    6 ++--
 stgit/commands/rebase.py   |    4 +--
 stgit/commands/refresh.py  |    4 ++-
 stgit/commands/rename.py   |    3 +-
 stgit/commands/resolved.py |    4 ++-
 stgit/commands/sink.py     |    3 +-
 stgit/commands/sync.py     |    5 +++-
 stgit/lib/log.py           |   60 ++++++++++++++++++++++++++++++++++++++++++++
 stgit/lib/transaction.py   |    3 +-
 18 files changed, 111 insertions(+), 23 deletions(-)
 create mode 100644 stgit/lib/log.py
Show changes to 18 files +111 −23

stgit/commands/branch.py, stgit/commands/clone.py, stgit/commands/float.py, stgit/commands/fold.py, stgit/commands/imprt.py, stgit/commands/init.py, stgit/commands/pick.py, stgit/commands/pop.py, stgit/commands/pull.py, stgit/commands/push.py, stgit/commands/rebase.py, stgit/commands/refresh.py, stgit/commands/rename.py, stgit/commands/resolved.py, stgit/commands/sink.py, stgit/commands/sync.py, stgit/lib/log.py, stgit/lib/transaction.py

diff --git a/stgit/commands/branch.py b/stgit/commands/branch.py
index 50684bb..131dfea 100644
--- a/stgit/commands/branch.py
+++ b/stgit/commands/branch.py
@@ -25,7 +25,7 @@ from stgit.commands.common import *
 from stgit.utils import *
 from stgit.out import *
 from stgit import stack, git, basedir
-
+from stgit.lib import log
 
 help = 'manage patch stacks'
 usage = """%prog [options] branch-name [commit-id]
@@ -161,6 +161,7 @@ def func(parser, options, args):
                                    parent_branch = parentbranch)
 
         out.info('Branch "%s" created' % args[0])
+        log.log_entry(log.default_stack(), 'stg branch --create')
         return
 
     elif options.clone:
@@ -181,6 +182,8 @@ def func(parser, options, args):
         crt_series.clone(clone)
         out.done()
 
+        log.copy_log(log.default_repo(), crt_series.get_name(), clone,
+                     'stg branch --clone')
         return
 
     elif options.delete:
@@ -188,6 +191,7 @@ def func(parser, options, args):
         if len(args) != 1:
             parser.error('incorrect number of arguments')
         __delete_branch(args[0], options.force)
+        log.delete_log(log.default_repo(), args[0])
         return
 
     elif options.list:
@@ -195,13 +199,16 @@ def func(parser, options, args):
         if len(args) != 0:
             parser.error('incorrect number of arguments')
 
-        branches = git.get_heads()
-        branches.sort()
+        branches = set(git.get_heads())
+        for br in set(branches):
+            m = re.match(r'^(.*)\.stgit$', br)
+            if m and m.group(1) in branches:
+                branches.remove(br)
 
         if branches:
             out.info('Available branches:')
             max_len = max([len(i) for i in branches])
-            for i in branches:
+            for i in sorted(branches):
                 __print_branch(i, max_len)
         else:
             out.info('No branches')
@@ -238,7 +245,8 @@ def func(parser, options, args):
         stack.Series(args[0]).rename(args[1])
 
         out.info('Renamed branch "%s" to "%s"' % (args[0], args[1]))
-
+        log.rename_log(log.default_repo(), args[0], args[1],
+                       'stg branch --rename')
         return
 
     elif options.unprotect:
diff --git a/stgit/commands/clone.py b/stgit/commands/clone.py
index c3b0bbe..8c3cccd 100644
--- a/stgit/commands/clone.py
+++ b/stgit/commands/clone.py
@@ -21,7 +21,7 @@ from optparse import OptionParser, make_option
 from stgit.commands.common import *
 from stgit.utils import *
 from stgit import stack, git
-
+from stgit.lib import log
 
 help = 'make a local clone of a remote repository'
 usage = """%prog [options] <repository> <dir>
@@ -56,5 +56,6 @@ def func(parser, options, args):
     # to work on a brand new repository
     basedir.clear_cache()
     stack.Series().init()
+    log.log_entry(log.default_stack(), 'stg clone')
 
     print 'done'
diff --git a/stgit/commands/float.py b/stgit/commands/float.py
index 0ba4446..5fec87e 100644
--- a/stgit/commands/float.py
+++ b/stgit/commands/float.py
@@ -22,6 +22,7 @@ from optparse import OptionParser, make_option
 from stgit.commands.common import *
 from stgit.utils import *
 from stgit import stack, git
+from stgit.lib import log
 
 help = 'push patches to the top, even if applied'
 usage = """%prog [<patches> | -s [<series>] ]
@@ -89,3 +90,5 @@ def func(parser, options, args):
         if topush:
             topush.reverse()
             push_patches(crt_series, topush)
+
+    log.log_entry(log.default_stack(), 'stg float')
diff --git a/stgit/commands/fold.py b/stgit/commands/fold.py
index 3930a1f..105f54a 100644
--- a/stgit/commands/fold.py
+++ b/stgit/commands/fold.py
@@ -22,7 +22,7 @@ from stgit.commands.common import *
 from stgit.utils import *
 from stgit.out import *
 from stgit import stack, git
-
+from stgit.lib import log
 
 help = 'integrate a GNU diff patch into the current patch'
 usage = """%prog [options] [<file>]
@@ -80,3 +80,4 @@ def func(parser, options, args):
         git.apply_patch(filename = filename)
 
     out.done()
+    log.log_entry(log.default_stack(), 'stg fold')
diff --git a/stgit/commands/imprt.py b/stgit/commands/imprt.py
index 4a4b792..d90e45d 100644
--- a/stgit/commands/imprt.py
+++ b/stgit/commands/imprt.py
@@ -24,7 +24,7 @@ from stgit.commands.common import *
 from stgit.utils import *
 from stgit.out import *
 from stgit import stack, git
-
+from stgit.lib import log
 
 help = 'import a GNU diff file as a new patch'
 usage = """%prog [options] [<file>|<url>]
@@ -289,3 +289,4 @@ def func(parser, options, args):
         __import_file(filename, options)
 
     print_crt_patch(crt_series)
+    log.log_entry(log.default_stack(), 'stg import')
diff --git a/stgit/commands/init.py b/stgit/commands/init.py
index 475a4ce..b49b7b2 100644
--- a/stgit/commands/init.py
+++ b/stgit/commands/init.py
@@ -22,7 +22,7 @@ from optparse import OptionParser, make_option
 from stgit.commands.common import *
 from stgit.utils import *
 from stgit import stack, git
-
+from stgit.lib import log
 
 help = 'initialise the current branch for use with StGIT'
 usage = """%prog [options]
@@ -42,3 +42,4 @@ def func(parser, options, args):
         parser.error('incorrect number of arguments')
 
     crt_series.init()
+    log.log_entry(log.default_stack(), 'stg init')
diff --git a/stgit/commands/pick.py b/stgit/commands/pick.py
index e011a84..4cc8944 100644
--- a/stgit/commands/pick.py
+++ b/stgit/commands/pick.py
@@ -23,7 +23,7 @@ from stgit.utils import *
 from stgit.out import *
 from stgit import stack, git
 from stgit.stack import Series
-
+from stgit.lib import log
 
 help = 'import a patch from a different branch or a commit object'
 usage = """%prog [options] [<patch@branch>|<commit>]
@@ -163,4 +163,5 @@ def func(parser, options, args):
         else:
             out.done()
 
+    log.log_entry(log.default_stack(), 'stg pick')
     print_crt_patch(crt_series)
diff --git a/stgit/commands/pop.py b/stgit/commands/pop.py
index dedf41b..e60361e 100644
--- a/stgit/commands/pop.py
+++ b/stgit/commands/pop.py
@@ -22,7 +22,7 @@ from optparse import OptionParser, make_option
 from stgit.commands.common import *
 from stgit.utils import *
 from stgit import stack, git
-
+from stgit.lib import log
 
 help = 'pop one or more patches from the stack'
 usage = """%prog [options] [<patch1>] [<patch2>] [<patch3>..<patch4>]
@@ -87,4 +87,5 @@ def func(parser, options, args):
     if topush:
         push_patches(crt_series, topush)
 
+    log.log_entry(log.default_stack(), 'stg pop')
     print_crt_patch(crt_series)
diff --git a/stgit/commands/pull.py b/stgit/commands/pull.py
index bec7fd7..8be5a14 100644
--- a/stgit/commands/pull.py
+++ b/stgit/commands/pull.py
@@ -23,7 +23,7 @@ from stgit.utils import *
 from stgit.out import *
 from stgit.config import GitConfigException
 from stgit import stack, git
-
+from stgit.lib import log
 
 help = 'pull the changes from the remote repository'
 usage = """%prog [options] [<repository>]
@@ -106,4 +106,5 @@ def func(parser, options, args):
     if config.get('stgit.keepoptimized') == 'yes':
         git.repack()
 
+    log.log_entry(log.default_stack(), 'stg pull')
     print_crt_patch(crt_series)
diff --git a/stgit/commands/push.py b/stgit/commands/push.py
index 979835b..148a0b5 100644
--- a/stgit/commands/push.py
+++ b/stgit/commands/push.py
@@ -23,7 +23,7 @@ from stgit.commands.common import *
 from stgit.utils import *
 from stgit.out import *
 from stgit import stack, git
-
+from stgit.lib import log
 
 help = 'push one or more patches onto of the stack'
 usage = """%prog [options] [<patch1>] [<patch2>] [<patch3>..<patch4>]
@@ -69,11 +69,11 @@ def func(parser, options, args):
         out.start('Undoing push of "%s"' % patch)
         resolved_all()
         if crt_series.undo_push():
+            log.log_entry(log.default_stack(), 'stg push --undo')
             out.done()
         else:
             out.done('patch unchanged')
         print_crt_patch(crt_series)
-
         return
 
     check_local_changes()
@@ -100,5 +100,5 @@ def func(parser, options, args):
         patches.reverse()
 
     push_patches(crt_series, patches, options.merged)
-
+    log.log_entry(log.default_stack(), 'stg push')
     print_crt_patch(crt_series)
diff --git a/stgit/commands/rebase.py b/stgit/commands/rebase.py
index 12faaf8..41e93ce 100644
--- a/stgit/commands/rebase.py
+++ b/stgit/commands/rebase.py
@@ -21,7 +21,7 @@ from optparse import OptionParser, make_option
 from stgit.commands.common import *
 from stgit.utils import *
 from stgit import stack, git
-
+from stgit.lib import log
 
 help = 'move the stack base to another point in history'
 usage = """%prog [options] <new-base-id>
@@ -57,5 +57,5 @@ def func(parser, options, args):
     applied = prepare_rebase(crt_series)
     rebase(crt_series, args[0])
     post_rebase(crt_series, applied, options.nopush, options.merged)
-
+    log.log_entry(log.default_stack(), 'stg rebase')
     print_crt_patch(crt_series)
diff --git a/stgit/commands/refresh.py b/stgit/commands/refresh.py
index 4695c62..017ec5b 100644
--- a/stgit/commands/refresh.py
+++ b/stgit/commands/refresh.py
@@ -24,7 +24,7 @@ from stgit.utils import *
 from stgit.out import *
 from stgit import stack, git
 from stgit.config import config
-
+from stgit.lib import log
 
 help = 'generate a new commit for the current patch'
 usage = """%prog [options] [<files or dirs>]
@@ -91,6 +91,7 @@ def func(parser, options, args):
     if options.undo:
         out.start('Undoing the refresh of "%s"' % patch)
         crt_series.undo_refresh()
+        log.log_entry(log.default_stack(), 'stg refresh --undo')
         out.done()
         return
 
@@ -131,6 +132,7 @@ def func(parser, options, args):
         if options.patch:
             between.reverse()
             push_patches(crt_series, between)
+        log.log_entry(log.default_stack(), 'stg refresh')
     elif options.annotate:
         # only annotate the top log entry as there is no need to
         # refresh the patch and generate a full commit
diff --git a/stgit/commands/rename.py b/stgit/commands/rename.py
index e2b0fa4..4e9aac0 100644
--- a/stgit/commands/rename.py
+++ b/stgit/commands/rename.py
@@ -22,7 +22,7 @@ from stgit.commands.common import *
 from stgit.utils import *
 from stgit.out import *
 from stgit import stack, git
-
+from stgit.lib import log
 
 help = 'rename a patch in the series'
 usage = """%prog [options] <oldpatch> <newpatch>
@@ -42,4 +42,5 @@ def func(parser, options, args):
 
     out.start('Renaming patch "%s" to "%s"' % (args[0], args[1]))
     crt_series.rename_patch(args[0], args[1])
+    log.log_entry(log.default_stack(), 'stg rename')
     out.done()
diff --git a/stgit/commands/resolved.py b/stgit/commands/resolved.py
index 4ee75b8..cb21d84 100644
--- a/stgit/commands/resolved.py
+++ b/stgit/commands/resolved.py
@@ -24,7 +24,7 @@ from stgit.utils import *
 from stgit import stack, git, basedir
 from stgit.config import config, file_extensions
 from stgit.gitmergeonefile import interactive_merge
-
+from stgit.lib import log
 
 help = 'mark a file conflict as solved'
 usage = """%prog [options] [<files...>]
@@ -55,6 +55,7 @@ def func(parser, options, args):
 
     if options.all and not options.interactive:
         resolved_all(options.reset)
+        log.log_entry(log.default_stack(), 'stg resolved')
         return
 
     conflicts = git.get_conflicts()
@@ -82,3 +83,4 @@ def func(parser, options, args):
             git.resolved([filename])
     else:
         git.resolved(files, options.reset)
+    log.log_entry(log.default_stack(), 'stg resolved')
diff --git a/stgit/commands/sink.py b/stgit/commands/sink.py
index 2167d87..f3a9ada 100644
--- a/stgit/commands/sink.py
+++ b/stgit/commands/sink.py
@@ -22,7 +22,7 @@ from optparse import OptionParser, make_option
 from stgit.commands.common import *
 from stgit.utils import *
 from stgit import stack, git
-
+from stgit.lib import log
 
 help = 'send patches deeper down the stack'
 usage = """%prog [-t <target patch>] [-n] [<patches>]
@@ -68,3 +68,4 @@ def func(parser, options, args):
         def not_reapplied_yet(p):
             return not p in newapplied
         push_patches(crt_series, filter(not_reapplied_yet, oldapplied))
+    log.log_entry(log.default_stack(), 'stg sink')
diff --git a/stgit/commands/sync.py b/stgit/commands/sync.py
index 99ab7de..168199d 100644
--- a/stgit/commands/sync.py
+++ b/stgit/commands/sync.py
@@ -23,7 +23,7 @@ from stgit.commands.common import *
 from stgit.utils import *
 from stgit.out import *
 from stgit import stack, git
-
+from stgit.lib import log
 
 help = 'synchronise patches with a branch or a series'
 usage = """%prog [options] [<patch1>] [<patch2>] [<patch3>..<patch4>]
@@ -77,6 +77,7 @@ def func(parser, options, args):
         out.start('Undoing the sync of "%s"' % crt_series.get_current())
         crt_series.undo_refresh()
         git.reset()
+        log.log_entry(log.default_stack(), 'stg sync --undo')
         out.done()
         return
 
@@ -166,3 +167,5 @@ def func(parser, options, args):
     # push the remaining patches
     if popped:
         push_patches(crt_series, popped)
+
+    log.log_entry(log.default_stack(), 'stg sync')
diff --git a/stgit/lib/log.py b/stgit/lib/log.py
new file mode 100644
index 0000000..1a9a326
--- /dev/null
+++ b/stgit/lib/log.py
@@ -0,0 +1,60 @@
+from stgit.lib import git, stack
+
+def patch_tree(repository, cd):
+    return repository.commit(git.Treedata({ 'a': cd.parent.data.tree,
+                                            'b': cd.tree }))
+
+def order_blob(repository, stack, kind):
+    return repository.commit(git.Blobdata(''.join(
+                '%s\n' % pn for pn in getattr(stack.patchorder, kind))))
+
+def log_entry_tree(repository, stack):
+    patches = repository.commit(
+        git.Treedata(dict((pn, patch_tree(repository,
+                                          stack.patches.get(pn).commit.data))
+                          for pn in stack.patchorder.all)))
+    return repository.commit(git.Treedata({
+                'version': repository.commit(git.Blobdata('0')),
+                'patches': patches,
+                'applied': order_blob(repository, stack, 'applied'),
+                'unapplied': order_blob(repository, stack, 'unapplied'),
+                }))
+
+def log_ref(branch):
+    return 'refs/heads/%s.stgit' % branch
+
+def log_entry(stack, msg):
+    ref = log_ref(stack.name)
+    try:
+        last = [stack.repository.refs.get(ref)]
+    except KeyError:
+        last = []
+    commit = stack.repository.commit(
+        git.Commitdata(tree = log_entry_tree(stack.repository, stack),
+                       parents = last, message = msg))
+    stack.repository.refs.set(ref, commit, msg)
+
+def delete_log(repo, branch):
+    ref = log_ref(branch)
+    if repo.refs.exists(ref):
+        repo.refs.delete(ref)
+
+def rename_log(repo, old_branch, new_branch, msg):
+    old_ref = log_ref(old_branch)
+    new_ref = log_ref(new_branch)
+    if repo.refs.exists(old_ref):
+        repo.refs.set(new_ref, repo.refs.get(old_ref), msg)
+        repo.refs.delete(old_ref)
+
+def copy_log(repo, src_branch, dst_branch, msg):
+    src_ref = log_ref(src_branch)
+    dst_ref = log_ref(dst_branch)
+    if repo.refs.exists(src_ref):
+        repo.refs.set(dst_ref, repo.refs.get(src_ref), msg)
+
+def default_repo():
+    return stack.Repository.default()
+
+def default_stack(branch = None):
+    repo = default_repo()
+    return repo.get_stack(branch or repo.current_branch)
diff --git a/stgit/lib/transaction.py b/stgit/lib/transaction.py
index 3613b15..0b0b52d 100644
--- a/stgit/lib/transaction.py
+++ b/stgit/lib/transaction.py
@@ -1,6 +1,6 @@
 from stgit import exception, utils
 from stgit.out import *
-from stgit.lib import git
+from stgit.lib import git, log
 
 class TransactionException(exception.StgException):
     pass
@@ -117,6 +117,7 @@ class StackTransaction(object):
         _print_current_patch(self.__stack.patchorder.applied, self.__applied)
         self.__stack.patchorder.applied = self.__applied
         self.__stack.patchorder.unapplied = self.__unapplied
+        log.log_entry(self.__stack, self.__msg)
 
         if self.__error:
             return utils.STGIT_CONFLICT
Catalin Marinas· Feb 20, 2008, 22:46 UTC · re: Karl Hasselström · lore

Re: [StGit PATCH 2/2] Write to a stack log when stack is modified

On 14/02/2008, Karl Hasselström <kha@treskal.com> wrote:
> Create a log branch (called <branchname>.stgit) for each StGit branch,
>  and write to it whenever the stack is modified.

The abstractions are really nice (and I still wonder how StGIT codebase increased that much when all I needed two years ago was a simple script-like application to reorder commits :-)).

Anyway, I don't really like the idea of an additional commit (I don't even like the old patch log implementation) when the stack is modified. It needs some profiling but it has a visible impact on stacks with a big number of patches (my last kernel release at www.linux-arm.org/git had 80 patches and it takes a lot of time to push them).

Can we not use some of the automatic reflog recording that GIT does instead of writing a commit? It's cheaper to write a text file than generating a commit. In my kernel repository I have several branches with many patches and, even after "git gc" and repacking, it is still slow (mainly because of git-read-tree but I'd like to reduce the number of calls to GIT).

-- 
Catalin
Karl Hasselström· Feb 21, 2008, 07:18 UTC · re: Catalin Marinas · lore

Re: [StGit PATCH 2/2] Write to a stack log when stack is modified

On 2008-02-20 22:46:48 +0000, Catalin Marinas wrote:
Show 8 quoted lines
> On 14/02/2008, Karl Hasselström <kha@treskal.com> wrote:
>
> > Create a log branch (called <branchname>.stgit) for each StGit
> > branch, and write to it whenever the stack is modified.
>
> The abstractions are really nice (and I still wonder how StGIT
> codebase increased that much when all I needed two years ago was a
> simple script-like application to reorder commits :-)).

:-) I'll take some of the blame, but StGit was quite large already when I started submitting patches to it.

Show 6 quoted lines
> Anyway, I don't really like the idea of an additional commit (I
> don't even like the old patch log implementation) when the stack is
> modified. It needs some profiling but it has a visible impact on
> stacks with a big number of patches (my last kernel release at
> www.linux-arm.org/git had 80 patches and it takes a lot of time to
> push them).

One thing to remember is that my patch stack log gets one new commit per StGit operation, not one per changed patch. So if you push 80 patches with one command, that will give you just one commit on the log. (If someone makes a scripts that calls push 80 times this won't help, of course.)

> Can we not use some of the automatic reflog recording that GIT does
> instead of writing a commit? It's cheaper to write a text file than
> generating a commit.

As I said, I'm not convinced that it's a huge difference when you have just one log commit per operation, not one per changed patch. But let's stop arguing about it until I have some numbers to back my claim up. ;-)

> In my kernel repository I have several branches with many patches
> and, even after "git gc" and repacking, it is still slow (mainly
> because of git-read-tree but I'd like to reduce the number of calls
> to GIT).

When you say "it's still slow", are you referring to the existing per-patch log, my per-branch log, or just StGit in general?

Have you noticed any difference between commands using the old and new infrastructure (say, stg push vs. stg goto)? The latter should be taking less time, due to touching the worktree only when necessary.

-- 
Karl Hasselström, kha@treskal.com
      www.treskal.com/kalle
Catalin Marinas· Feb 22, 2008, 14:05 UTC · re: Karl Hasselström · lore

Re: [StGit PATCH 2/2] Write to a stack log when stack is modified

On 21/02/2008, Karl Hasselström <kha@treskal.com> wrote:
Show 9 quoted lines
> On 2008-02-20 22:46:48 +0000, Catalin Marinas wrote:
>
>  > The abstractions are really nice (and I still wonder how StGIT
>  > codebase increased that much when all I needed two years ago was a
>  > simple script-like application to reorder commits :-)).
>
>
> :-) I'll take some of the blame, but StGit was quite large already
>  when I started submitting patches to it.

Anyway, the new restructuring is much cleaner, though heavily OO and some people might not like it (me not included).

Show 15 quoted lines
>
>
>  > Anyway, I don't really like the idea of an additional commit (I
>  > don't even like the old patch log implementation) when the stack is
>  > modified. It needs some profiling but it has a visible impact on
>  > stacks with a big number of patches (my last kernel release at
>  > www.linux-arm.org/git had 80 patches and it takes a lot of time to
>  > push them).
>
>
> One thing to remember is that my patch stack log gets one new commit
>  per StGit operation, not one per changed patch. So if you push 80
>  patches with one command, that will give you just one commit on the
>  log. (If someone makes a scripts that calls push 80 times this won't
>  help, of course.)

Ah, I got it now. Indeed, it would be faster than the current per-patch log. I no longer that worried :-)

Show 8 quoted lines
>  > In my kernel repository I have several branches with many patches
>  > and, even after "git gc" and repacking, it is still slow (mainly
>  > because of git-read-tree but I'd like to reduce the number of calls
>  > to GIT).
>
>
> When you say "it's still slow", are you referring to the existing
>  per-patch log, my per-branch log, or just StGit in general?

I think it's more GIT in general. Checking the tree status takes some time and a 3-way merge on a 512MB RAM machine with GIT using over 200MB gets a bit slow.

>  Have you noticed any difference between commands using the old and new
>  infrastructure (say, stg push vs. stg goto)? The latter should be
>  taking less time, due to touching the worktree only when necessary.

In the patch pushing functions, it now first calls simple_merge() which is still a 3-way merge but without rename detection. The old StGIT implementation was using "git-diff | git-apply" and falling back to the recursive merge. Most of the patches apply cleanly and we don't need the three-way merge which uses more RAM.

The "(modified)" information was also displayed for a three-way merge (either read-tree or recursive-merge) but now it is only shown for the recursive one. Correctly, merging git-read-tree modifies a patch as well since it no longer applied cleanly. We could use this "modified" feature to automatically export patches (some people asked for this in the past, as means for backup in case of StGIT failures).

-- 
Catalin
Karl Hasselström· Feb 22, 2008, 14:56 UTC · re: Catalin Marinas · lore

Re: [StGit PATCH 2/2] Write to a stack log when stack is modified

On 2008-02-22 14:05:55 +0000, Catalin Marinas wrote:
Show 13 quoted lines
> On 21/02/2008, Karl Hasselström <kha@treskal.com> wrote:
>
> > On 2008-02-20 22:46:48 +0000, Catalin Marinas wrote:
> >
> >  > The abstractions are really nice (and I still wonder how StGIT
> >  > codebase increased that much when all I needed two years ago
> >  > was a simple script-like application to reorder commits :-)).
> >
> > :-) I'll take some of the blame, but StGit was quite large already
> >  when I started submitting patches to it.
>
> Anyway, the new restructuring is much cleaner, though heavily OO and
> some people might not like it (me not included).

That it's conceptually OO is unavoidable, I think (the only way to avoid that would be through obfuscation). And using Python's OO features to write code for such a model is the least bad you can do in Python. IMHO.

Show 6 quoted lines
> > When you say "it's still slow", are you referring to the existing
> > per-patch log, my per-branch log, or just StGit in general?
>
> I think it's more GIT in general. Checking the tree status takes
> some time and a 3-way merge on a 512MB RAM machine with GIT using
> over 200MB gets a bit slow.

I just upgraded my laptop from 512 MB to 2 GB, so I'll start neglecting this kind of problem, I fear. :-/

Show 10 quoted lines
> >  Have you noticed any difference between commands using the old
> >  and new infrastructure (say, stg push vs. stg goto)? The latter
> >  should be taking less time, due to touching the worktree only
> >  when necessary.
>
> In the patch pushing functions, it now first calls simple_merge()
> which is still a 3-way merge but without rename detection. The old
> StGIT implementation was using "git-diff | git-apply" and falling
> back to the recursive merge. Most of the patches apply cleanly and
> we don't need the three-way merge which uses more RAM.

I didn't include patch application because it wasn't necessary to get things working, and is easy to add at any later point.

I'd be curious to see how much of a difference it makes.
> The "(modified)" information was also displayed for a three-way
> merge (either read-tree or recursive-merge) but now it is only shown
> for the recursive one. Correctly, merging git-read-tree modifies a
> patch as well since it no longer applied cleanly.

Ah, so the precise meaning of a patch being "modified" is that git apply failed to apply it. I didn't know that.

> We could use this "modified" feature to automatically export patches
> (some people asked for this in the past, as means for backup in case
> of StGIT failures).

You mean automatically export only the changed patches? (Otherwise stg export would do the job -- I used to use that before I got confident enough with StGit.)

One cool thing we could do is export the patches as files in a git branch -- the log machinery I'm building should make it trivial. That way, you'd get the history of the patches too.

-- 
Karl Hasselström, kha@treskal.com
      www.treskal.com/kalle
Catalin Marinas· Feb 22, 2008, 15:58 UTC · re: Karl Hasselström · lore

Re: [StGit PATCH 2/2] Write to a stack log when stack is modified

On 22/02/2008, Karl Hasselström <kha@treskal.com> wrote:
Show 21 quoted lines
> On 2008-02-22 14:05:55 +0000, Catalin Marinas wrote:
>
>  > On 21/02/2008, Karl Hasselström <kha@treskal.com> wrote:
>  >
>  > > On 2008-02-20 22:46:48 +0000, Catalin Marinas wrote:
>  > >
>  > >  > The abstractions are really nice (and I still wonder how StGIT
>  > >  > codebase increased that much when all I needed two years ago
>  > >  > was a simple script-like application to reorder commits :-)).
>  > >
>  > > :-) I'll take some of the blame, but StGit was quite large already
>  > >  when I started submitting patches to it.
>  >
>  > Anyway, the new restructuring is much cleaner, though heavily OO and
>  > some people might not like it (me not included).
>
>
> That it's conceptually OO is unavoidable, I think (the only way to
>  avoid that would be through obfuscation). And using Python's OO
>  features to write code for such a model is the least bad you can do in
>  Python. IMHO.
I agree.
Show 10 quoted lines
>  > > When you say "it's still slow", are you referring to the existing
>  > > per-patch log, my per-branch log, or just StGit in general?
>  >
>  > I think it's more GIT in general. Checking the tree status takes
>  > some time and a 3-way merge on a 512MB RAM machine with GIT using
>  > over 200MB gets a bit slow.
>
>
> I just upgraded my laptop from 512 MB to 2 GB, so I'll start
>  neglecting this kind of problem, I fear. :-/
You can boot with mem=512MB (or even less) :-).
Show 16 quoted lines
>  > >  Have you noticed any difference between commands using the old
>  > >  and new infrastructure (say, stg push vs. stg goto)? The latter
>  > >  should be taking less time, due to touching the worktree only
>  > >  when necessary.
>  >
>  > In the patch pushing functions, it now first calls simple_merge()
>  > which is still a 3-way merge but without rename detection. The old
>  > StGIT implementation was using "git-diff | git-apply" and falling
>  > back to the recursive merge. Most of the patches apply cleanly and
>  > we don't need the three-way merge which uses more RAM.
>
>
> I didn't include patch application because it wasn't necessary to get
>  things working, and is easy to add at any later point.
>
>  I'd be curious to see how much of a difference it makes.

It makes a difference if you don't have much RAM available. I think once a patch falls back to three-way merge, Linux already makes room available for the subsequent merges. But, as I said, most of the time the patches just apply cleanly.

Show 6 quoted lines
>  > We could use this "modified" feature to automatically export patches
>  > (some people asked for this in the past, as means for backup in case
>  > of StGIT failures).
>
>
> You mean automatically export only the changed patches?
Yes, at refresh and pushing, but for the latter, only if a patch was modified.
>  One cool thing we could do is export the patches as files in a git
>  branch -- the log machinery I'm building should make it trivial. That
>  way, you'd get the history of the patches too.

Yes, I don't think it is difficult to do (it just need to be optional as some people might not need it).

-- 
Catalin

← back to recent threads