Re: Reduce pack-objects memory footprint?

From: Duy Nguyen <pclouds@gmail.com>
Date: Mar 1, 2018, 09:00 UTC
Message-ID: <20180301090039.GA19167@ash>
In-Reply-To: <20180228182233.GC15256@dcvr>

On Wed, Feb 28, 2018 at 06:22:33PM +0000, Eric Wong wrote:

Show 9 quoted lines

> Duy Nguyen <pclouds@gmail.com> wrote:
> > which saves 12 bytes (or another 74 MB). 222 MB total is plenty of
> > space to keep some file cache from being evicted.
> 
> Nice!  I can definitely benefit from lower memory usage when
> packing.  Fwiw, I use pahole with other projects to help find
> packing opportunities:
> 
> 	git://git.kernel.org/pub/scm/devel/pahole/pahole.git

Yes it's a wonderful tool.

Show 28 quoted lines

> > @@ -14,11 +26,10 @@ struct object_entry {
> >  	void *delta_data;	/* cached delta (uncompressed) */
> >  	unsigned long delta_size;	/* delta data size (uncompressed) */
> >  	unsigned long z_delta_size;	/* delta data size (compressed) */
> > -	enum object_type type;
> > -	enum object_type in_pack_type;	/* could be delta */
> >  	uint32_t hash;			/* name hint hash */
> > -	unsigned int in_pack_pos;
> >  	unsigned char in_pack_header_size;
> > +	unsigned type:3;	 /* enum object_type */
> > +	unsigned in_pack_type:3; /* enum object_type - could be delta */
> 
> For C99 compilers, enums can be bitfields.  I introduced the
> following macro into Ruby a few weeks ago to remain compatible
> with non-C99 compilers:
> 
> /*
>  * For declaring bitfields out of non-unsigned int types:
>  *   struct date {
>  *      BITFIELD(enum months) month:4;
>  *      ...
>  *   };
>  */
> #if defined(__STDC_VERSION__) && (__STDC_VERSION__ >= 199901L)
> # define BITFIELD(type) type
> #else
> # define BITFIELD(type) unsigned int
> #endif

I tried this and got

In file included from builtin/pack-objects.c:20:0:
./pack-objects.h:49:19: l?i: ?type? is narrower than values of its type [-Werror]
  enum object_type type:TYPE_BITS;
                   ^~~~

The compiler is not wrong. What it does not realize is pack-objects code never uses out-of-range values (OBJ_BAD and OBJ_ANY) but I don't see how I could suppress this warning. So I went back to non-enum bitfields.

-- Duy