threads / announce / 1993

[ANNOUNCE] Stacked GIT 0.7

Subject: [ANNOUNCE] Stacked GIT 0.7

## tl;dr

5 messages between Oct 2, 2005 and Oct 3, 2005.

replies: 4people: 2as markdown or json

Catalin Marinas· Oct 2, 2005, 09:17 UTC · lore
Stacked GIT 0.7 release is available from http://www.procode.org/stgit/

StGIT is a Python application providing similar functionality to Quilt (i.e. pushing/popping patches to/from a stack) on top of GIT. These operations are performed using GIT commands and the patches are stored as GIT commit objects, allowing easy merging of the StGIT patches into other repositories using standard GIT functionality.

What's new in this release (the full ChangeLog is in the archive):
      * Uses the renamed GIT commands (requires GIT >= 0.99.7) 
      * 'pick' command to cherry-pick a commit object as an StGIT patch
        using a three-way merge. It also allows reverse-applying 
      * 'commit' command to permanently store the applied patches into
        the repository 
      * 'push' can now fast-forward the patches if the base tree object
        has not changed 
      * The configuration file supports the 'editor' field 
      * '--version' option for the 'mail' command to generate the e-mail
        subject in the form '[PATCH version nr/total]' 
      * '--showpatch' option for 'refresh' to show the patch content
        when editing the patch description 
      * '--branch' option for several commands to work on a given branch
        instead of the current one without switching 
      * the 'pull' command was updated to follow the 'git pull' API 
      * '--base' option for 'import' to import a diff file on a
        different base and perform a three-way merge with the HEAD
        afterwards 
      * Support for the '.gitignore' file 
      * Several other optimisations 
      * Bug fixes

Many thanks to Paolo 'Blaisorblade' Giarrusso, Chuck Lever, Pierre Ossman and Junio C Hamano for contributing patches to this release.

-- 
Catalin
Ingo Molnar· Oct 2, 2005, 12:10 UTC · re: Catalin Marinas · lore

Re: [ANNOUNCE] Stacked GIT 0.7

* Catalin Marinas <catalin.marinas@gmail.com> wrote:
Show 8 quoted lines
> Stacked GIT 0.7 release is available from 
> http://www.procode.org/stgit/
> 
> StGIT is a Python application providing similar functionality to Quilt 
> (i.e. pushing/popping patches to/from a stack) on top of GIT. These 
> operations are performed using GIT commands and the patches are stored 
> as GIT commit objects, allowing easy merging of the StGIT patches into 
> other repositories using standard GIT functionality.

i'm wondering - have you (or anyone else) done performance comparisons of quilt vs. stgit, using the same stack of patches? One of the most important features of quilt (for me) is that it's very fast at popping/pushing through hundreds of patches.

	Ingo
Catalin Marinas· Oct 2, 2005, 19:33 UTC · re: Ingo Molnar · lore

Re: [ANNOUNCE] Stacked GIT 0.7

On 02/10/05, Ingo Molnar <mingo@elte.hu> wrote:
> i'm wondering - have you (or anyone else) done performance comparisons
> of quilt vs. stgit, using the same stack of patches? One of the most
> important features of quilt (for me) is that it's very fast at
> popping/pushing through hundreds of patches.

I've done some basic comparison and I must admit that Quilt is much faster than StGIT. Popping is done in O(1) and it is equivalent to a switch between two branches with 'git checkout' (can be less than 2 seconds with warm caches). Pushing without merging required (i.e. fast-forwarding) is the same as popping.

What's time consuming is pushing when the base of the stack was changed and merging is required. If the base changes do not involve file removals/additions, a push operations takes ~1.5s per patch with warm caches. Most of the time is spent in "git-read-tree -m" (StGIT doesn't consume resources since it mainly calls GIT tools). If the base changes involve file removals/additions, it can take considerably longer since for each file GIT needs to call an external merger. This could be solved by providing a smarter git-merge-index to deal with file removals/additions but the time won't be shorter than ~1.5s per patch.

An optimisation would be to keep the patches in diff format and apply them but this would mean losing the advantages of the three-way merging. Another issue is that git-apply doesn't support fuzzy patches, though I could emulate it using patch. But I'm not sure that's a good approach since I don't intend to re-write Quilt in Python. My hope is that git-read-tree will get faster :-).

-- Catalin

Catalin Marinas· Oct 3, 2005, 13:56 UTC · re: Catalin Marinas · lore

Re: [ANNOUNCE] Stacked GIT 0.7

On 02/10/05, Catalin Marinas <catalin.marinas@gmail.com> wrote:
> An optimisation would be to keep the patches in diff format and apply
> them but this would mean losing the advantages of the three-way
> merging.

After some tests, it looks like 'git-diff-tree -p ... | git-apply --index' is about 3 times faster than 'git-read-tree -m' (in the best case for git-read-tree with minor modifications of the base). It now takes ~0.5s to push a single patch (compared to ~1.5s).

Since most of the patches, for routine updates of the base, don't generate conflicts, git-apply should work fine. I'll modify StGIT to use this method by default and fall back to the three-way merge in case it fails. This will also solve the problem with complex modifications of the base of the stack (i.e. file removals/additions) when there won't be any need to call the external index merger if git-apply succeeds.

-- Catalin

Catalin Marinas· Oct 3, 2005, 14:44 UTC · re: Catalin Marinas · lore

Re: [ANNOUNCE] Stacked GIT 0.7

Catalin Marinas <catalin.marinas@gmail.com> wrote:
> After some tests, it looks like 'git-diff-tree -p ... | git-apply
> --index' is about 3 times faster than 'git-read-tree -m' (in the best
> case for git-read-tree with minor modifications of the base). It now
> takes ~0.5s to push a single patch (compared to ~1.5s).

And that's the patch for whoever wants to try. I will also add it to the repository tonight:

Optimise 'push' to use git-apply instead of git-read-tree

With this patch, 'push' will use 'git-diff-tree | git-apply' first. If this operation fails, it will fall back to the three-way merge with git-read-tree.

Signed-off-by: Catalin Marinas <catalin.marinas@gmail.com>
---
 stgit/git.py   |    9 +++++++++
 stgit/stack.py |   19 +++++++++++--------
 2 files changed, 20 insertions(+), 8 deletions(-)
diff --git a/stgit/git.py b/stgit/git.py
--- a/stgit/git.py
+++ b/stgit/git.py
@@ -355,6 +355,15 @@ def commit(message, files = [], parents 
 
     return commit_id
 
+def apply_diff(rev1, rev2):
+    """Apply the diff between rev1 and rev2 onto the current
+    index. This function doesn't need to raise an exception since it
+    is only used for fast-pushing a patch. If this operation fails,
+    the pushing would fall back to the three-way merge.
+    """
+    return os.system('git-diff-tree -p %s %s | git-apply --index 2> /dev/null'
+                     % (rev1, rev2)) == 0
+
 def merge(base, head1, head2):
     """Perform a 3-way merge between base, head1 and head2 into the
     local tree
diff --git a/stgit/stack.py b/stgit/stack.py
--- a/stgit/stack.py
+++ b/stgit/stack.py
@@ -598,14 +598,17 @@ class Series:
             # The current patch is empty after merge.
             patch.set_bottom(head, backup = True)
             patch.set_top(head, backup = True)
-            # merge/refresh can fail but the patch needs to be pushed
-            try:
-                git.merge(bottom, head, top)
-            except git.GitException, ex:
-                print >> sys.stderr, \
-                      'The merge failed during "push". ' \
-                      'Use "refresh" after fixing the conflicts'
-                pass
+
+            # Try the fast applying first. If this fails, fall back to the
+            # three-way merge
+            if not git.apply_diff(bottom, top):
+                # merge can fail but the patch needs to be pushed
+                try:
+                    git.merge(bottom, head, top)
+                except git.GitException, ex:
+                    print >> sys.stderr, \
+                          'The merge failed during "push". ' \
+                          'Use "refresh" after fixing the conflicts'
 
         append_string(self.__applied_file, name)
 
-- 
Catalin

← back to recent threads