Re: [PATCH v3 2/8] maintenance: add loose-objects task
- From
Derrick Stolee <stolee@gmail.com>
- Date
- Sep 24, 2020, 13:45 UTC
- Message-ID
- <446e805a-6b83-a6b7-18a3-f99ac85a6fe1@gmail.com>
- In-Reply-To
- <20200922230937.906065-1-jonathantanmy@google.com>
On 9/22/2020 7:09 PM, Jonathan Tan wrote:
Show 31 quoted lines
>> Create a 'loose-objects' task for the 'git maintenance run' command. >> This helps clean up loose objects without disrupting concurrent Git >> commands using the following sequence of events: >> >> 1. Run 'git prune-packed' to delete any loose objects that exist >> in a pack-file. Concurrent commands will prefer the packed >> version of the object to the loose version. (Of course, there >> are exceptions for commands that specifically care about the >> location of an object. These are rare for a user to run on >> purpose, and we hope a user that has selected background >> maintenance will not be trying to do foreground maintenance.) >> >> 2. Run 'git pack-objects' on a batch of loose objects. These >> objects are grouped by scanning the loose object directories in >> lexicographic order until listing all loose objects -or- >> reaching 50,000 objects. This is more than enough if the loose >> objects are created only by a user doing normal development. >> We noticed users with _millions_ of loose objects because VFS >> for Git downloads blobs on-demand when a file read operation >> requires populating a virtual file. > > [snip] > >> This has potential of >> happening in partial clones if someone runs 'git grep' or >> otherwise evades the batch-download feature for requesting >> promisor objects. > > This part is not strictly true, as even when Git lazy-fetches one > object, it fetches it in the form of a packfile - so maybe remove this > sentence.
This is a good point. I just did some testing and we do store these single-object downloads as pack-files. My misunderstanding is due to my own bias and experience with the GVFS protocol.
I have also heard that "git fetch" might explode some small pack-files into loose objects, and I guess I expected the same here. However, that is not the case for partial clone. I'll remove this.
Show 6 quoted lines
> This is nevertheless a good feature to have - loose objects may not be > created during lazy fetches, but they definitely are created during > normal operation (e.g. commits). Git, as a whole, prefers packfiles over > loose objects, and just packing the loose objects themselves instead of > running repack (which goes through all reachable objects) is definitely > better for large repositories.
Thanks, -Stolee