Re: git status: small difference between stating whole repository and small subdirectory

From: Jeff King <peff@peff.net>
Date: Feb 16, 2012, 19:20 UTC
Message-ID: <20120216192001.GB4348@sigill.intra.peff.net>
In-Reply-To: <CAA01Cso5y23UMguEe0vwOc6kR3-DjuC8-LTMDsMeeOKU4rVGvg@mail.gmail.com>

On Thu, Feb 16, 2012 at 02:37:47PM +0100, Piotr Krukowiecki wrote:

Show 13 quoted lines

> >> $ time git status  -- .
> >> real    0m2.503s
> >> user    0m0.160s
> >> sys     0m0.096s
> >>
> >> $ time git status
> >> real    0m9.663s
> >> user    0m0.232s
> >> sys     0m0.556s
> >
> > Did you drop caches here, too?
> 
> Yes I did - with cache the status takes something like 0.1-0.3s on whole repo.

OK, then that makes sense. It's pretty much just I/O on the filesystem and on the object db.

You can break status down a little more to see which is which. Try "git update-index --refresh" to see just how expensive the lstat and index handling is.

And then try "git diff-index HEAD" for an idea of how expensive it is to just read the objects and compare to the index.

Show 6 quoted lines

> > Not really. You're showing an I/O problem, and repacking is git's way of
> > reducing I/O.
> 
> So if I understand correctly, the reason is because git must compare
> workspace files with packed objects - and the problem is
> reading/seeking/searching in the packs?

Mostly reading (we keep a sorted index and access the packfiles via mmap, so we only touch the pages we need). But you're also paying to lstat() the directory tree, too. And you're paying to load (probably) the whole index into memory, although it's relatively compact compared to the actual file data.

> Is there a way to make packs better? I think most operations are on
> workdir files - so maybe it'd be possible to tell gc/repack/whatever
> to optimize access to files which I currently have in workdir?

It already does optimize for that case. If you can make it even better, I'm sure people would be happy to see the numbers.

Mostly I think it is just the case that disk I/O is slow, and the operation you're asking for has to do a certain amount of it. What kind of disk/filesystem are you pulling off of?

It's not a fuse filesystem by any chance, is it? I have a repo on an encfs-mounted filesystem, and the lstat times are absolutely horrific.

-Peff