threads / patch / 24655

patchpack-object: trim memory usage a tiny bit

Subject: [PATCH] pack-object: trim memory usage a tiny bit

## tl;dr

7 messages between Aug 6, 2010 and Aug 7, 2010. Diffs are folded; open one to read it.

replies: 6people: 3as markdown or json

Nguyễn Thái Ngọc Duy· Aug 6, 2010, 17:00 UTC · lore

This shrinks struct object_entry from 88 bytes to 80 bytes on my 32 bit machine. So that would be 12M less on linux-2.6.git (1.6M objects)

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
 Don't know if it's really worth a patch..
 builtin/pack-objects.c |   18 ++++++++++--------
 1 files changed, 10 insertions(+), 8 deletions(-)
Show changes to builtin/pack-objects.c +10 −8
diff --git a/builtin/pack-objects.c b/builtin/pack-objects.c
index 0e81673..398c0bb 100644
--- a/builtin/pack-objects.c
+++ b/builtin/pack-objects.c
@@ -47,14 +47,14 @@ struct object_entry {
 	unsigned long delta_size;	/* delta data size (uncompressed) */
 	unsigned long z_delta_size;	/* delta data size (compressed) */
 	unsigned int hash;	/* name hint hash */
-	enum object_type type;
-	enum object_type in_pack_type;	/* could be delta */
+	enum object_type type:5;
+	enum object_type in_pack_type:5;	/* could be delta */
 	unsigned char in_pack_header_size;
-	unsigned char preferred_base; /* we do not pack this, but is available
-				       * to be used as the base object to delta
-				       * objects against.
-				       */
-	unsigned char no_try_delta;
+	unsigned char preferred_base:1; /* we do not pack this, but is available
+					 * to be used as the base object to delta
+					 * objects against.
+					 */
+	unsigned char no_try_delta:1;
 };
 
 /*
@@ -1001,6 +1001,7 @@ static void check_object(struct object_entry *entry)
 		unsigned int avail;
 		off_t ofs;
 		unsigned char *buf, c;
+		enum object_type in_pack_type;
 
 		buf = use_pack(p, &w_curs, entry->in_pack_offset, &avail);
 
@@ -1009,8 +1010,9 @@ static void check_object(struct object_entry *entry)
 		 * since non-delta representations could still be reused.
 		 */
 		used = unpack_object_header_buffer(buf, avail,
-						   &entry->in_pack_type,
+						   &in_pack_type,
 						   &entry->size);
+		entry->in_pack_type = in_pack_type;
 		if (used == 0)
 			goto give_up;
 
-- 
1.7.1.rc1.69.g24c2f7
Sverre Rabbelier· Aug 6, 2010, 23:58 UTC · re: Nguyễn Thái Ngọc Duy · lore

Re: [PATCH] pack-object: trim memory usage a tiny bit

Heya,
2010/8/6 Nguyễn Thái Ngọc Duy <pclouds@gmail.com>:
Show 6 quoted lines
> This shrinks struct object_entry from 88 bytes to 80 bytes on my 32
> bit machine. So that would be 12M less on linux-2.6.git (1.6M objects)
>
> Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
> ---
>  Don't know if it's really worth a patch..
Either way, some explanation would be nice.
-- 
Cheers,

Sverre Rabbelier
Nguyen Thai Ngoc Duy· Aug 7, 2010, 00:03 UTC · re: Sverre Rabbelier · lore

Re: [PATCH] pack-object: trim memory usage a tiny bit

2010/8/7 Sverre Rabbelier <srabbelier@gmail.com>:
Show 11 quoted lines
> Heya,
>
> 2010/8/6 Nguyễn Thái Ngọc Duy <pclouds@gmail.com>:
>> This shrinks struct object_entry from 88 bytes to 80 bytes on my 32
>> bit machine. So that would be 12M less on linux-2.6.git (1.6M objects)
>>
>> Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
>> ---
>>  Don't know if it's really worth a patch..
>
> Either way, some explanation would be nice.

struct object_entry is allocated for every object to be packed. So if you do a clone on linux-2.6, you would need 1.6M of those structs, 88 bytes each. I cut 8 bytes from each of them.

-- 
Duy
Sverre Rabbelier· Aug 7, 2010, 00:04 UTC · re: Nguyen Thai Ngoc Duy · lore

Re: [PATCH] pack-object: trim memory usage a tiny bit

Heya,
On Fri, Aug 6, 2010 at 19:03, Nguyen Thai Ngoc Duy <pclouds@gmail.com> wrote:
> struct object_entry is allocated for every object to be packed. So if
> you do a clone on linux-2.6, you would need 1.6M of those structs, 88
> bytes each. I cut 8 bytes from each of them.

Sure, that was clear enough. But _how_? What needs to be done to retain this? Are there any downsides?

-- 
Cheers,

Sverre Rabbelier
Nguyen Thai Ngoc Duy· Aug 7, 2010, 00:16 UTC · re: Sverre Rabbelier · lore

Re: [PATCH] pack-object: trim memory usage a tiny bit

On Sat, Aug 7, 2010 at 10:04 AM, Sverre Rabbelier <srabbelier@gmail.com> wrote:
Show 9 quoted lines
> Heya,
>
> On Fri, Aug 6, 2010 at 19:03, Nguyen Thai Ngoc Duy <pclouds@gmail.com> wrote:
>> struct object_entry is allocated for every object to be packed. So if
>> you do a clone on linux-2.6, you would need 1.6M of those structs, 88
>> bytes each. I cut 8 bytes from each of them.
>
> Sure, that was clear enough. But _how_? What needs to be done to
> retain this? Are there any downsides?

Ah, by packing it tighter using bitfields. There are two fields that act as boolean, but they use at least one byte each. And enum object_type, I recall, uses 4 bytes (int), while it only needs 5 bits (4 bits is enough if OBJ_BAD is not counted). No downsides I know of.

-- 
Duy
Shawn O. Pearce· Aug 7, 2010, 00:18 UTC · re: Nguyen Thai Ngoc Duy · lore

Re: [PATCH] pack-object: trim memory usage a tiny bit

Nguyen Thai Ngoc Duy <pclouds@gmail.com> wrote:
Show 5 quoted lines
> 
> Ah, by packing it tighter using bitfields. There are two fields that
> act as boolean, but they use at least one byte each. And enum
> object_type, I recall, uses 4 bytes (int), while it only needs 5 bits
> (4 bits is enough if OBJ_BAD is not counted). No downsides I know of.

Actually, object type is 3 bits in the pack file. So unless its using some additional magical values, 4 bits is sufficient to get you 3 bits and the -1 bad value.

-- 
Shawn.
Nguyen Thai Ngoc Duy· Aug 7, 2010, 00:21 UTC · re: Shawn O. Pearce · lore

Re: [PATCH] pack-object: trim memory usage a tiny bit

On Sat, Aug 7, 2010 at 10:18 AM, Shawn O. Pearce <spearce@spearce.org> wrote:
Show 10 quoted lines
> Nguyen Thai Ngoc Duy <pclouds@gmail.com> wrote:
>>
>> Ah, by packing it tighter using bitfields. There are two fields that
>> act as boolean, but they use at least one byte each. And enum
>> object_type, I recall, uses 4 bytes (int), while it only needs 5 bits
>> (4 bits is enough if OBJ_BAD is not counted). No downsides I know of.
>
> Actually, object type is 3 bits in the pack file.  So unless its
> using some additional magical values, 4 bits is sufficient to get
> you 3 bits and the -1 bad value.
Right. Obviously I miscounted.
-- 
Duy

← back to recent threads