Re: git-revert is a memory hog

From: Junio C Hamano <gitster@pobox.com>
Date: Jan 29, 2008, 22:15 UTC
Message-ID: <7vk5lsmg6i.fsf@gitster.siamese.dyndns.org>
In-Reply-To: <alpine.LFD.1.00.0801300844170.28476@www.l.google.com>

Linus Torvalds <torvalds@linux-foundation.org> writes:

Show 31 quoted lines

> On Mon, 28 Jan 2008, Jeff King wrote:
>> 
>> I tried to reproduce this, but my peak heap allocation was only around
>> 20MB. Is your repository fully packed? Not packed at all? Can you use
>> valgrind/massif to figure out where the memory is going?
>
> I definitely can reproduce it, it's horrid.
>
> This is from "top" fairly late in the game, but with the thing not even 
> done yet. Current git, pretty much fully (and fairly aggressively) packed 
> current kernel repo, and using "diff.renamelmit=0".
>
> 	4751 torvalds  20   0  852m 446m  47m R   72 22.4   2:46.58 git-merge-recur
>
> It finally finished with time reporting:
>
> 	208.15user 3.50system 4:01.50elapsed 87%CPU (0avgtext+0avgdata 0maxresident)k
> 	238736inputs+4544outputs (8261major+280971minor)pagefaults 0swaps
>
> where those 280971 minor page faults are what largely indicates how much 
> memory it used (the technical term for that number is "metric buttload of 
> memory").
>
> But I'm in Melbourne right now on my laptop,and probably won't be able to 
> debug this much. 
>
>> In your case, the patch doesn't apply cleanly, so we end up doing a
>> 3-way merge (in my tests, it is git-merge-recursive which ends up taking
>> up the memory).
>
> It is indeed git-merge-recursive. It just shouldn't take that much memory.

Hmmmmm. Obviously this depends on where you start your revert from, but that is not what I am getting.

: gitster linux-2.6/test; git reset --hard HEAD is now at 0ba6c33... Merge git://git.kernel.org/pub/scm/linux/kernel/git/davem/net-2.6.25 : gitster linux-2.6/test; /usr/bin/time git-revert d19fbe8a7 Auto-merged drivers/input/input.c CONFLICT (content): Merge conflict in drivers/input/input.c Auto-merged include/linux/input.h CONFLICT (content): Merge conflict in include/linux/input.h Automatic revert failed. After resolving the conflicts, mark the corrected paths with 'git add <paths>' or 'git rm <paths>' and commit the result. Command exited with non-zero status 1 1.08user 0.06system 0:01.14elapsed 100%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+19867minor)pagefaults 0swaps

Now, a possible alternative (that produces an identical result) is not much better, because it ends up using merge-recursive as its core.

: gitster linux-2.6/test; git reset --hard HEAD is now at 0ba6c33... Merge git://git.kernel.org/pub/scm/linux/kernel/git/davem/net-2.6.25 : gitster linux-2.6/test; rm -fr .dotest : gitster linux-2.6/test; /usr/bin/time sh -c 'git format-patch -R --binary --stdout -1 d19fbe8a7 | git am -3' Applying Input: prepare to sysfs integration error: patch failed: drivers/input/input.c:27 error: drivers/input/input.c: patch does not apply error: patch failed: include/linux/input.h:12 error: include/linux/input.h: patch does not apply Using index info to reconstruct a base tree... Falling back to patching base and 3-way merge... Auto-merged drivers/input/input.c CONFLICT (content): Merge conflict in drivers/input/input.c Auto-merged include/linux/input.h CONFLICT (content): Merge conflict in include/linux/input.h Failed to merge in the changes. Patch failed at 0001. When you have resolved this problem run "git-am -3 --resolved". If you would prefer to skip this patch, instead run "git-am -3 --skip". Command exited with non-zero status 1 0.52user 0.25system 0:00.75elapsed 103%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+41208minor)pagefaults 0swaps