Re: Question: .idx without .pack causes performance issues?

From: Junio C Hamano <gitster@pobox.com>
Date: Aug 4, 2015, 01:27 UTC
Message-ID: <xmqqk2tb6dxc.fsf@gitster.dls.corp.google.com>
In-Reply-To: <CAEtYS8SNksc0m5rn_tRk8bGLBeq_8QcBLHyHo=cOfZ+aE6n0gA@mail.gmail.com>

Doug Kelly <dougk.ff7@gmail.com> writes:

> Here's a change to prune.c that at least addresses the issue by removing
> .idx files without an associated pack, but it's by no means pretty.  If anyone
> has any feedback before I turn this into a formal patch, it's more than welcome!

I'd hesitate to see removal of a file (for that matter, a creation too) inside a "while (de = readdir)" loop. As the original function is about temporary files, and the new thing is not about temporary files at all, I'd further prefer that we do not do it in the same loop.

I am wondering if we can add a new mode to report_pack_garbage() in sha1_file.c to allow it to remove stale and lone ".idx". Most of the time we are accessing packs read-only, and I do not want the function to unconditionally remove lone ".idx", but perhaps we can teach "prune" to set a custom report_garbage() routine and react to a call to its custom report_garbage()?

Perhaps that custom report_garbage() can make a list of ".idx" files, iterate over it to pick the lone one without ".pack" and remove them. Or the custom report_garbage() can make a list of lone ".idx" files, if you tweak the interface to report_garbage() to contain th seen_bits value, avoiding the need to check the existence of ".pack" for the second time.