Re: Understanding version 4 packs

From: Nicolas Pitre <nico@cam.org>
Date: Mar 26, 2007, 14:27 UTC
Message-ID: <alpine.LFD.0.83.0703261015110.3041@xanadu.home>
In-Reply-To: <e5bfff550703260516q5da5f46et8aab2ebadcd9cceb@mail.gmail.com>

On Mon, 26 Mar 2007, Marco Costalba wrote:

Show 24 quoted lines

> Experimenting with file names cache in qgit I have found a big saving
> splitting the paths in base name and file name and indexing both:
> 
> drivers\usb\host\ehci.h
> drivers\usb\host\ehci-pci.c
> drivers\usb\host\ohci-pci.c
> kernel\sched.c
> 
> became:
> 
> dir names table
> 
> 0 drivers\usb\host
> 1 kernel
> 
> 
> file name table
> 
> 0 ehci.h
> 1 ehci-pci.c
> 2 ohci-pci.c
> 
> In this way a big saving is achieved in case of directories deep in
> the tree (long paths) and a lot of files.

Sure, but if you also consider drivers/usb/Makefile and drivers/Kconfig for example then you start losing on space saving. Maybe that makes sense for qgit but it has no advantage in a pack which contains every possible files.

> Regarding MODE field an observation could be that is almost always the
> same, so an idea could be to store a 'default mode' just after
> nr_entries and do not add the field any more except in case path mode
> is different from default mode.

If the mode is always the same, or most likely similar for many entries then it will compress very well. In fact in the current table format the tree byte sequence NULL+16-bit-mode will be quite common and likely to deflate accordingly. This is therefore not worth adding more complex handling at runtime for deciding which mode to use, and still you'd have to store a flag for each path component to decide if the default mode should be used or not anyway.

> In case this could bring to unaligned
> entries another idea could be to store _all_ mode fields at the
> beginning (or at the end and let deflate to remove almost everything
> more easily)

That's worth trying indeed.

Nicolas