Ramsay JonesSigned-off-by: Ramsay Jones <ramsay@ramsay1.demon.co.uk> ---
Whilst testing an msvc-built git on cygwin, I noticed that git-count-objects was producing different results to the cygwin version, viz:
$ ./git --version
git version 1.6.5.4.gf034d.MSVC
$ git --version
git version 1.6.5 $ ./git-count-objects
60 objects, 283 kilobytes $ git count-objects
60 objects, 210 kilobytes$ git config core.filemode false
$ git count-objects
60 objects, 297 kilobytes
$ Note also that the cygwin version of git gives two different answers, in the *same* repository, depending on the setting of core.filemode. (since the "win32 stat() functions" in compat/cygwin.c are used when core.filemode is false)
Having looked at the msvc code-path, I also noticed something else a bit odd; the value printed by the msvc version should be a *lower bound* for the amount of disk-space used (since it simply totals the actual file sizes). However, as you can see from the above, when core.filemode is true, the cygwin version of git is *underestimating* even this (210Kb vs 283kb).
A quick trip to the debugger confirmed that (st.st_blocks * 512) is less than st.st_size for several files. So, it looked like the block-size was not in units of 512 bytes; so if we look at /usr/include/sys/param.h we find:
#define DEV_BSIZE 1024
and in /usr/include/sys/stat.h we find:
#define S_BLKSIZE 1024
which seems to indicate a 1K block-size instead. Also, different filesystem types may use different block-sizes, which is why an st.st_blksize was added to struct stat; it seems that the Cygwin struct stat contains the st_blksize field, so lets try a quick test program:
$ cat junk.c
#include <stdio.h>
#include <sys/stat.h>
int main(int argc, char *argv[])
{
int i, bytes = 0, tot_5 = 0, tot_b = 0;
for (i=1; i< argc; i++) {
struct stat st;
if (lstat(argv[i], &st))
printf("can't lstat '%s'\n", argv[i]);
else {
int s = (int)st.st_size;
int b = (int)st.st_blocks;
int f = (int)st.st_blksize;
int d = (b * 512);
int e = (b * f);
printf("%8d, ", s);
printf("%2d*512=%-6d (%6d), ", b, d, d-s);
printf("%2d*%d=%-6d (%6d)\n", b, f, e, e-s);
bytes += s;
tot_5 += d;
tot_b += e;
}
}
printf("total bytes: %6d\n", bytes);
printf("b * 512: %6d (%d)\n", tot_5, tot_5 - bytes);
printf("b * blksize: %6d (%d)\n", tot_b, tot_b - bytes);
exit(0);
}
$ ls .git/objects/??/* | xargs ./junk.exe
148, 1*512=512 ( 364), 1*1024=1024 ( 876)
10058, 12*512=6144 ( -3914), 12*1024=12288 ( 2230)
3701, 4*512=2048 ( -1653), 4*1024=4096 ( 395)
463, 4*512=2048 ( 1585), 4*1024=4096 ( 3633)
148, 1*512=512 ( 364), 1*1024=1024 ( 876)
10056, 12*512=6144 ( -3912), 12*1024=12288 ( 2232)
198, 1*512=512 ( 314), 1*1024=1024 ( 826)
1782, 4*512=2048 ( 266), 4*1024=4096 ( 2314)
192, 1*512=512 ( 320), 1*1024=1024 ( 832)
3801, 4*512=2048 ( -1753), 4*1024=4096 ( 295)
14851, 16*512=8192 ( -6659), 16*1024=16384 ( 1533)
3761, 4*512=2048 ( -1713), 4*1024=4096 ( 335)
52, 1*512=512 ( 460), 1*1024=1024 ( 972)
956, 4*512=2048 ( 1092), 4*1024=4096 ( 3140)
956, 4*512=2048 ( 1092), 4*1024=4096 ( 3140)
3703, 4*512=2048 ( -1655), 4*1024=4096 ( 393)
10055, 12*512=6144 ( -3911), 12*1024=12288 ( 2233)
total bytes: 64881
b * 512: 45568 (-19313)
b * blksize: 91136 (26255)
$ Note that, in addition to confirming the 1K block-size, it appears that (on NTFS) files less than 1K are allocated a single block whereas larger files use a multiple of 4 blocks. Well, that is only a guess but it at least sounds plausible! ;-)
This patch implements a simple solution which has the useful property of returning a single answer, irrespective of the core.filemode setting.
ATB, Ramsay Jones
Makefile | 4 ++++ 1 files changed, 4 insertions(+), 0 deletions(-)
Show changes to Makefile +4 −0
diff --git a/Makefile b/Makefile index 5d5976f..5624563 100644 --- a/Makefile +++ b/Makefile @@ -783,6 +783,10 @@ ifeq ($(uname_O),Cygwin) NO_FAST_WORKING_DIRECTORY = UnfortunatelyYes NO_TRUSTABLE_FILEMODE = UnfortunatelyYes OLD_ICONV = UnfortunatelyYes + # The st_blocks field is not in units of 512 bytes, as the code + # expects, which leads to an under-estimate of the disk space used. + # In order to use an alternate algorithm, we claim to lack st_blocks. + NO_ST_BLOCKS_IN_STRUCT_STAT = YesPlease # There are conflicting reports about this. # On some boxes NO_MMAP is needed, and not so elsewhere. # Try commenting this out if you suspect MMAP is more efficient
-- 1.6.5
