Sensible way to see what objects are being fetched just-in-time in a partial clone?
- From
Tao Klerks <tao@klerks.biz>
- Date
- Aug 26, 2024, 16:38 UTC
- Message-ID
- <CAPMMpog7=ZnhJhrgZFwzRZibLtK1-LyOhsrp5c4O85ocRFDZxw@mail.gmail.com>
Hi folks,
In working with Partial / Filtered Clone repos, there are situations where objects get fetched just-in-time - eg during a "git blame", if you did a "blob:none" filtered clone, you can easily end up with hundreds of fetches as git iterates backwards through the file history.
I was trying to write a "git blame optimizer" to pre-fetch all the suitable blobs, and it wasn't working right, so the "git blame" was still fetching stuff - but I couldn't see what it was fetching (which made it hard to investigate the bug in my script).
I did end up getting a list of some just-in-time fetched blobs, by dumping a list of *all* the object IDs I had locally, before and after a still-fetching-stuff "git blame" run, and doing a before/after comparison of the resulting list of objects. To get the list of objects found locally I did:
git cat-file --batch-check='%(objectname)' --batch-all-objects --unordered
(ref: a conversation with Peff last year: https://lore.kernel.org/git/20230621064459.GA607974@coredump.intra.peff.net/ )
This was a sucky process though - and I was very surprised that I couldn't see what was being fetched (what the stdin content to the just-in-time fetch calls were) with any of the trace env vars that I was able to find documented: GIT_TRACE, GIT_CURL_VERBOSE, GIT_TRACE_PERFORMANCE, GIT_TRACE_PACK_ACCESS, GIT_TRACE_PACKET, GIT_TRACE_PACKFILE, GIT_TRACE_SETUP, GIT_TRACE_SHALLOW
The only thing I could easily see were the *args* passed to nested git processes.
Is there any way to see what a just-in-time fetch is fetching? Or any way to see the content passed around on stdin in nested git processes?
Thanks, Tao