{"thread":{"id":"14886","subject":"[StGit] kha/{safe,experimental} updated","startedAt":"2008-08-08T08:07:04Z","lastAt":"2008-08-21T22:16:01Z","messageCount":7,"participants":["Karl Hasselström","Catalin Marinas"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"86464","messageId":"20080808080614.23424.28169.stgit@yoghurt","threadId":"14886","inReplyTo":"20080808082728.GA24017@diana.vm.bytemark.co.uk","subject":"[StGit PATCH] Read several objects at once with git cat-file --batch","fromName":"Karl Hasselström","fromEmail":"kha@treskal.com","sentAt":"2008-08-08T08:07:04Z","receivedAt":"2008-08-08T08:07:04Z","isPatch":true,"sender":{"key":"kha@treskal.com","avatar":"https://gravatar.com/avatar/f0120c734b5279b345075a28521e1ac66acb20c9913ffe9bf6ae97e53f7f3f13?d=mp&s=160"},"body":"Instead of spawning a separate cat-file process for every blob and\ncommit we want to read. This speeds things up slightly: about 6-8%\nwhen uncommitting and rebasing 1470 linux-kernel patches (perftest.py\nrebase-newrebase-add-file-linux).\n\nSigned-off-by: Karl Hasselström \n\n---\n\n stgit/lib/git.py | 34 +++++++++++++++++++++++++++++++++-\n stgit/run.py | 17 +++++++++++++++++\n 2 files changed, 50 insertions(+), 1 deletions(-)\n\n\ndiff --git a/stgit/lib/git.py b/stgit/lib/git.py\nindex 648e190..95efd9a 100644\n--- a/stgit/lib/git.py\n+++ b/stgit/lib/git.py\n@@ -520,6 +520,37 @@ class RunWithEnvCwd(RunWithEnv):\n @param args: Command and argument vector\"\"\"\n return RunWithEnv.run(self, args, self.env_in_cwd)\n \n+class CatFileProcess(object):\n+ def __init__(self, repo):\n+ self.__repo = repo\n+ self.__proc = None\n+ def __get_process(self):\n+ if not self.__proc:\n+ self.__proc = self.__repo.run(['git', 'cat-file', '--batch']\n+ ).run_background()\n+ return self.__proc\n+ def cat_file(self, sha1):\n+ p = self.__get_process()\n+ p.stdin.write('%s\\n' % sha1)\n+ p.stdin.flush()\n+\n+ # Read until we have the entire status line.\n+ s = ''\n+ while not '\\n' in s:\n+ s += os.read(p.stdout.fileno(), 4096)\n+ h, b = s.split('\\n', 1)\n+ if h == '%s missing' % sha1:\n+ raise SomeException()\n+ hash, type, length = h.split()\n+ assert hash == sha1\n+ length = int(length)\n+\n+ # Read until we have the entire object plus the trailing\n+ # newline.\n+ while len(b) < length + 1:\n+ b += os.read(p.stdout.fileno(), 4096)\n+ return type, b[:-1]\n+\n class Repository(RunWithEnv):\n \"\"\"Represents a git repository.\"\"\"\n def __init__(self, directory):\n@@ -531,6 +562,7 @@ class Repository(RunWithEnv):\n self.__default_index = None\n self.__default_worktree = None\n self.__default_iw = None\n+ self.__catfile = CatFileProcess(self)\n env = property(lambda self: { 'GIT_DIR': self.__git_dir })\n @classmethod\n def default(cls):\n@@ -580,7 +612,7 @@ class Repository(RunWithEnv):\n directory = property(lambda self: self.__git_dir)\n refs = property(lambda self: self.__refs)\n def cat_object(self, sha1):\n- return self.run(['git', 'cat-file', '-p', sha1]).raw_output()\n+ return self.__catfile.cat_file(sha1)[1]\n def rev_parse(self, rev):\n try:\n return self.get_commit(self.run(\ndiff --git a/stgit/run.py b/stgit/run.py\nindex 7493ed3..ccca059 100644\n--- a/stgit/run.py\n+++ b/stgit/run.py\n@@ -130,6 +130,19 @@ class Run:\n raise self.exc('%s failed: %s' % (self.__cmd[0], e))\n self.__log_end(self.exitcode)\n self.__check_exitcode()\n+ def __run_background(self):\n+ \"\"\"Run in background.\"\"\"\n+ assert self.__indata == None\n+ try:\n+ p = subprocess.Popen(self.__cmd, env = self.__env, cwd = self.__cwd,\n+ stdin = subprocess.PIPE,\n+ stdout = subprocess.PIPE,\n+ stderr = subprocess.PIPE)\n+ except OSError, e:\n+ raise self.exc('%s failed: %s' % (self.__cmd[0], e))\n+ self.stdin = p.stdin\n+ self.stdout = p.stdout\n+ self.stderr = p.stderr\n def returns(self, retvals):\n self.__good_retvals = retvals\n return self\n@@ -181,6 +194,10 @@ class Run:\n def run(self):\n \"\"\"Just run, with no IO redirection.\"\"\"\n self.__run_noio()\n+ def run_background(self):\n+ \"\"\"Run as a background process.\"\"\"\n+ self.__run_background()\n+ return self\n def xargs(self, xargs):\n \"\"\"Just run, with no IO redirection. The extra arguments are\n appended to the command line a few at a time; the command is\n"},{"id":"86463","messageId":"20080808082728.GA24017@diana.vm.bytemark.co.uk","threadId":"14886","inReplyTo":null,"subject":"[StGit] kha/{safe,experimental} updated","fromName":"Karl Hasselström","fromEmail":"kha@treskal.com","sentAt":"2008-08-08T08:27:28Z","receivedAt":"2008-08-08T08:27:28Z","isPatch":false,"sender":{"key":"kha@treskal.com","avatar":"https://gravatar.com/avatar/f0120c734b5279b345075a28521e1ac66acb20c9913ffe9bf6ae97e53f7f3f13?d=mp&s=160"},"body":"Catalin has pulled the safe and stable branches, but I've accumulated\nsome new stuff in kha/safe.\n\nThe stack log stuff (and nothing else) is still in kha/experimental.\nIt's unchanged since my last status mail, except for the addition of\nan optimization at the end (will post as a follow-up to this mail).\n\n\n -+-\n\n\nThe following changes since commit 36a06e0194e013552499677e431e528ecb2faee9:\n Karl Hasselström (1):\n Global performance logging\n\nare available in the git repository at:\n\n git://repo.or.cz/stgit/kha.git safe\n\nDaniel White (7):\n Fix Makefile to correctly pass prefix option\n Remove variables regarding section 7 man pages\n Fix default install location for manpages\n Add install-doc target to makefile\n Add install-html target to makefile\n Remove installation of documentation from setup.py\n Updated INSTALL with documentation of Makefile\n\nKarl Hasselström (1):\n Add some tests of refreshing removed files\n\nSamuel Tardieu (2):\n Do not insert an empty line before the diffstat info\n Do not mess-up with commit message formatting when sending email\n\n Documentation/Makefile | 25 +++++------\n INSTALL | 15 ++++--\n Makefile | 12 ++++-\n setup.py | 2 +-\n stgit/commands/mail.py | 4 +-\n t/t2702-refresh-rm.sh | 101 ++++++++++++++++++++++++++++++++++++++++++++\n templates/mailattch.tmpl | 1 -\n templates/patchexport.tmpl | 1 -\n templates/patchmail.tmpl | 1 -\n 9 files changed, 135 insertions(+), 27 deletions(-)\n create mode 100755 t/t2702-refresh-rm.sh\n\n\n -+-\n\n\nThe following changes since commit 42857cbe036ba5917eacc9dbb5644d395f638ed9:\n Samuel Tardieu (1):\n Do not mess-up with commit message formatting when sending email\n\nare available in the git repository at:\n\n git://repo.or.cz/stgit/kha.git experimental\n\nKarl Hasselström (17):\n Write to a stack log when stack is modified\n New command: stg reset\n Log conflicts separately\n Log conflicts separately for all commands\n Add a --hard flag to stg reset\n Don't write a log entry if there were no changes\n Move stack reset function to a shared location\n New command: stg undo\n New command: stg redo\n Log and undo external modifications\n Make \"stg log\" show stack log instead of patch log\n Convert \"stg refresh\" to the new infrastructure\n New refresh tests\n Remove --undo flags from stg commands and docs\n Refactor stgit.commands.edit\n Implement \"stg refresh --edit\" again\n Read several objects at once with git cat-file --batch\n\n Documentation/tutorial.txt | 4 +-\n TODO | 2 -\n stgit/commands/branch.py | 19 +-\n stgit/commands/clone.py | 2 +-\n stgit/commands/coalesce.py | 2 +-\n stgit/commands/common.py | 18 ++-\n stgit/commands/diff.py | 6 +-\n stgit/commands/edit.py | 82 +------\n stgit/commands/export.py | 2 +-\n stgit/commands/files.py | 6 +-\n stgit/commands/float.py | 2 +-\n stgit/commands/fold.py | 2 +-\n stgit/commands/goto.py | 3 +-\n stgit/commands/hide.py | 2 +-\n stgit/commands/id.py | 2 +-\n stgit/commands/imprt.py | 4 +-\n stgit/commands/log.py | 169 +++++----------\n stgit/commands/mail.py | 8 +-\n stgit/commands/new.py | 3 +-\n stgit/commands/patches.py | 2 +-\n stgit/commands/pick.py | 2 +-\n stgit/commands/pop.py | 4 +-\n stgit/commands/pull.py | 2 +-\n stgit/commands/push.py | 31 +--\n stgit/commands/rebase.py | 4 +-\n stgit/commands/redo.py | 52 ++++\n stgit/commands/refresh.py | 338 ++++++++++++++++++---------\n stgit/commands/rename.py | 2 +-\n stgit/commands/repair.py | 11 +-\n stgit/commands/reset.py | 57 +++++\n stgit/commands/resolved.py | 2 +-\n stgit/commands/show.py | 2 +-\n stgit/commands/sink.py | 2 +-\n stgit/commands/status.py | 3 +-\n stgit/commands/sync.py | 26 +--\n stgit/commands/undo.py | 49 ++++\n stgit/commands/unhide.py | 2 +-\n stgit/git.py | 4 -\n stgit/lib/edit.py | 99 ++++++++\n stgit/lib/git.py | 108 ++++++++-\n stgit/lib/log.py | 524 ++++++++++++++++++++++++++++++++++++++++++\n stgit/lib/stack.py | 25 ++\n stgit/lib/transaction.py | 125 +++++++----\n stgit/main.py | 8 +\n stgit/run.py | 17 ++\n stgit/stack.py | 45 +----\n stgit/utils.py | 18 +-\n t/t1200-push-modified.sh | 2 +-\n t/t1201-pull-trailing.sh | 2 +-\n t/t1202-push-undo.sh | 8 +-\n t/t1400-patch-history.sh | 103 --------\n t/t2300-refresh-subdir.sh | 29 +++-\n t/t2701-refresh-p.sh | 2 +-\n t/t3100-reset.sh | 160 +++++++++++++\n t/t3101-reset-hard.sh | 56 +++++\n t/t3102-undo.sh | 86 +++++++\n t/t3103-undo-hard.sh | 56 +++++\n t/t3104-redo.sh | 122 ++++++++++\n t/t3105-undo-external-mod.sh | 68 ++++++\n t/t3300-edit.sh | 4 +-\n 60 files changed, 1986 insertions(+), 614 deletions(-)\n create mode 100644 stgit/commands/redo.py\n create mode 100644 stgit/commands/reset.py\n create mode 100644 stgit/commands/undo.py\n create mode 100644 stgit/lib/edit.py\n create mode 100644 stgit/lib/log.py\n delete mode 100755 t/t1400-patch-history.sh\n create mode 100755 t/t3100-reset.sh\n create mode 100755 t/t3101-reset-hard.sh\n create mode 100755 t/t3102-undo.sh\n create mode 100755 t/t3103-undo-hard.sh\n create mode 100755 t/t3104-redo.sh\n create mode 100755 t/t3105-undo-external-mod.sh\n\n-- \nKarl Hasselström, kha@treskal.com\n www.treskal.com/kalle\n"},{"id":"86725","messageId":"b0943d9e0808101525w1e6f1e60h162a4b137d6dca6c@mail.gmail.com","threadId":"14886","inReplyTo":"20080808080614.23424.28169.stgit@yoghurt","subject":"Re: [StGit PATCH] Read several objects at once with git cat-file --batch","fromName":"Catalin Marinas","fromEmail":"catalin.marinas@gmail.com","sentAt":"2008-08-10T22:25:08Z","receivedAt":"2008-08-10T22:25:08Z","isPatch":true,"sender":{"key":"catalin.marinas@gmail.com","avatar":null},"body":"2008/8/8 Karl Hasselström :\n> Instead of spawning a separate cat-file process for every blob and\n> commit we want to read. This speeds things up slightly: about 6-8%\n> when uncommitting and rebasing 1470 linux-kernel patches (perftest.py\n> rebase-newrebase-add-file-linux).\n\nWhich version of Git got the --batch option to git-cat-file? It might\nbe possible that default Git in Debian (testing) or Ubuntu doesn't\nhave this option. Maybe we could still have the original behaviour as\na fallback.\n\nOtherwise, the patch looks allright. It took me a bit of time to see\nwhy we need the new run_background() function (but in my current Git,\n1.5.3.4.206.g58ba4, there wasn't such an option; I had to upgrade).\n\nThanks.\n\n-- \nCatalin\n"},{"id":"86737","messageId":"20080810235845.GA5106@diana.vm.bytemark.co.uk","threadId":"14886","inReplyTo":"b0943d9e0808101525w1e6f1e60h162a4b137d6dca6c@mail.gmail.com","subject":"Re: [StGit PATCH] Read several objects at once with git cat-file --batch","fromName":"Karl Hasselström","fromEmail":"kha@treskal.com","sentAt":"2008-08-10T23:58:45Z","receivedAt":"2008-08-10T23:58:45Z","isPatch":true,"sender":{"key":"kha@treskal.com","avatar":"https://gravatar.com/avatar/f0120c734b5279b345075a28521e1ac66acb20c9913ffe9bf6ae97e53f7f3f13?d=mp&s=160"},"body":"On 2008-08-10 23:25:08 +0100, Catalin Marinas wrote:\n\n> 2008/8/8 Karl Hasselström :\n>\n> > Instead of spawning a separate cat-file process for every blob and\n> > commit we want to read. This speeds things up slightly: about 6-8%\n> > when uncommitting and rebasing 1470 linux-kernel patches\n> > (perftest.py rebase-newrebase-add-file-linux).\n>\n> Which version of Git got the --batch option to git-cat-file? It\n> might be possible that default Git in Debian (testing) or Ubuntu\n> doesn't have this option. Maybe we could still have the original\n> behaviour as a fallback.\n\nHmm, I never realized it was so new. It's not in any 1.5.5 release,\nbut it is in 1.5.6-rc0. So realistically, we'll have to require\nversion 1.5.6 for this. (The patch should add a paragraph about the\nrequired git version to some doc file.)\n\nA fallback is certainly conceivable. The only nontrivial thing about\nit would be to detect when it's necessary. And to make sure both code\npaths are tested ...\n\n(I have a similar patch that uses diff-tree --stdin, but that needs a\ngit with the patches I posted some hours ago.)\n\n> Otherwise, the patch looks allright. It took me a bit of time to see\n> why we need the new run_background() function (but in my current\n> Git, 1.5.3.4.206.g58ba4, there wasn't such an option; I had to\n> upgrade).\n\nThanks. I'm not 100% pleased with the background running stuff yet. It\nworks, but could use some refactoring so that callers don't have to\nknow too much.\n\n-- \nKarl Hasselström, kha@treskal.com\n www.treskal.com/kalle\n"},{"id":"87116","messageId":"b0943d9e0808131454p16cb104bs1bcf67a6c660a88a@mail.gmail.com","threadId":"14886","inReplyTo":"20080808082728.GA24017@diana.vm.bytemark.co.uk","subject":"Re: [StGit] kha/{safe,experimental} updated","fromName":"Catalin Marinas","fromEmail":"catalin.marinas@gmail.com","sentAt":"2008-08-13T21:54:57Z","receivedAt":"2008-08-13T21:54:57Z","isPatch":false,"sender":{"key":"catalin.marinas@gmail.com","avatar":null},"body":"2008/8/8 Karl Hasselström :\n> Catalin has pulled the safe and stable branches, but I've accumulated\n> some new stuff in kha/safe.\n\nI merged it, thanks.\n\n> The stack log stuff (and nothing else) is still in kha/experimental.\n> It's unchanged since my last status mail, except for the addition of\n> an optimization at the end (will post as a follow-up to this mail).\n\nI'm on holiday until Monday and I haven't got the time to test the\nperformance but we can probably move the experimental stuff into\nmaster and start preparing some release candidates. I'll repost my\npatches with the new git id format as well and we need a 0.14.4 in the\nmeantime (hopefully over the next two weeks).\n\nThanks.\n\n-- \nCatalin\n"},{"id":"87454","messageId":"20080817201848.GA10889@diana.vm.bytemark.co.uk","threadId":"14886","inReplyTo":"b0943d9e0808131454p16cb104bs1bcf67a6c660a88a@mail.gmail.com","subject":"Re: [StGit] kha/{safe,experimental} updated","fromName":"Karl Hasselström","fromEmail":"kha@treskal.com","sentAt":"2008-08-17T20:18:48Z","receivedAt":"2008-08-17T20:18:48Z","isPatch":false,"sender":{"key":"kha@treskal.com","avatar":"https://gravatar.com/avatar/f0120c734b5279b345075a28521e1ac66acb20c9913ffe9bf6ae97e53f7f3f13?d=mp&s=160"},"body":"On 2008-08-13 22:54:57 +0100, Catalin Marinas wrote:\n\n> I'm on holiday until Monday and I haven't got the time to test the\n> performance but we can probably move the experimental stuff into\n> master and start preparing some release candidates.\n\nExcellent.\n\n> I'll repost my patches with the new git id format as well\n\nExcellent. I was about to merge them a while back, but I noticed you\nhadn't addressed the latest round of comments then.\n\n> and we need a 0.14.4 in the meantime (hopefully over the next two\n> weeks).\n\nExcellent.\n\n-- \nKarl Hasselström, kha@treskal.com\n www.treskal.com/kalle\n"},{"id":"88051","messageId":"b0943d9e0808211516j4c1f35c0gcbc3374a6b710797@mail.gmail.com","threadId":"14886","inReplyTo":"20080817201848.GA10889@diana.vm.bytemark.co.uk","subject":"Re: [StGit] kha/{safe,experimental} updated","fromName":"Catalin Marinas","fromEmail":"catalin.marinas@gmail.com","sentAt":"2008-08-21T22:16:01Z","receivedAt":"2008-08-21T22:16:01Z","isPatch":false,"sender":{"key":"catalin.marinas@gmail.com","avatar":null},"body":"2008/8/17 Karl Hasselström :\n> On 2008-08-13 22:54:57 +0100, Catalin Marinas wrote:\n>> I'll repost my patches with the new git id format as well\n>\n> Excellent. I was about to merge them a while back, but I noticed you\n> hadn't addressed the latest round of comments then.\n\nI actually decided to commit them since the comments were minor like\nfixing the commit log or using strip_prefix(). I implemented them and\nhopefully haven't forgotten any.\n\n-- \nCatalin\n"}]}