# Improve git-rev-list memory usage further

2 messages from 2005-09-16 to 2005-09-16. Participants: Linus Torvalds.
Thread: https://gitlist.dev/t/1833

## Linus Torvalds, 2005-09-16 21:32

Subject: Improve git-rev-list memory usage further
Message-ID: <Pine.LNX.4.58.0509161423400.26803@g5.osdl.org>
URL: https://gitlist.dev/e/Pine.LNX.4.58.0509161423400.26803%40g5.osdl.org

```

This avoids keeping tree entries around, and free's them as it traverses
the list. This avoids building up a huge memory footprint just for these 
small but very common allocations.

Before:

	[torvalds@g5 linux]$ /usr/bin/time git-rev-list --objects v2.6.12..HEAD | wc -l
	11.65user 0.38system 0:12.65elapsed 95%CPU (0avgtext+0avgdata 0maxresident)k
	0inputs+0outputs (0major+42934minor)pagefaults 0swaps
	59124

After:

	[torvalds@g5 linux]$ /usr/bin/time git-rev-list --objects v2.6.12..HEAD | wc -l
	12.28user 0.29system 0:12.57elapsed 99%CPU (0avgtext+0avgdata 0maxresident)k
	0inputs+0outputs (0major+26718minor)pagefaults 0swaps
	59124

note how the minor fault numbers - which ends up being how many pages we 
needed to map - go down from 42934 (167 MB) to 26718 (104 MB).

That's still a honking big memory footprint, but it's about half of what
it was just a day or two ago (and this is the object list for a pretty big
update - almost 60,000 objects. Smaller updates need less memory).

Signed-off-by: Linus Torvalds <torvalds@osdl.org>
---
diff --git a/rev-list.c b/rev-list.c
--- a/rev-list.c
+++ b/rev-list.c
@@ -147,11 +147,16 @@ static struct object_list **process_tree
 		die("bad tree object %s", sha1_to_hex(obj->sha1));
 	obj->flags |= SEEN;
 	p = add_object(obj, p, name);
-	for (entry = tree->entries ; entry ; entry = entry->next) {
+	entry = tree->entries;
+	tree->entries = NULL;
+	while (entry) {
+		struct tree_entry_list *next = entry->next;
 		if (entry->directory)
 			p = process_tree(entry->item.tree, p, entry->name);
 		else
 			p = process_blob(entry->item.blob, p, entry->name);
+		free(entry);
+		entry = next;
 	}
 	return p;
 }
@@ -218,12 +223,15 @@ static void mark_tree_uninteresting(stru
 	if (parse_tree(tree) < 0)
 		die("bad tree %s", sha1_to_hex(obj->sha1));
 	entry = tree->entries;
+	tree->entries = NULL;
 	while (entry) {
+		struct tree_entry_list *next = entry->next;
 		if (entry->directory)
 			mark_tree_uninteresting(entry->item.tree);
 		else
 			mark_blob_uninteresting(entry->item.blob);
-		entry = entry->next;
+		free(entry);
+		entry = next;
 	}
 }
 

```

## Linus Torvalds, 2005-09-16 21:42

Subject: Re: Improve git-rev-list memory usage further
Message-ID: <Pine.LNX.4.58.0509161434070.26803@g5.osdl.org>
URL: https://gitlist.dev/e/Pine.LNX.4.58.0509161434070.26803%40g5.osdl.org
In-Reply-To: <Pine.LNX.4.58.0509161423400.26803@g5.osdl.org>

```


On Fri, 16 Sep 2005, Linus Torvalds wrote:
>
> Before:
> 	42934 minor pagefaults
>
> After:
> 
> 	26718 minor pagefaults

In case it wasn't clear, this is all in _addition_ to the previous fixes. 
It used to be ~48,000 pagefaults.

Still some low-hanging fruit to go after this. Object refs next, that will 
bring it down to ~18,500 pages for this load.

		Linus

```