gc.linkedobjectdir discussion

Will Entriken· Feb 24, 2015, 21:56 UTC · lore

Hello,

This post presents an idea for storing git objects into a hard-linked shared directory which would allow performance gains and safety so that users can set it and forget it.

SETUP: Users run `git config --global gc.linkedobjectdir ~/.gitsharedobjects`

USAGE - CACHE MISS: During a pull, git will look for the needed object in LINKEDOBJECTDIR, similar to how it looks in GIT_ALTERNATE_OBJECT_DIRECTORIES. If it is not found, the object is received as normal and then hard linked into the object store at LINKEDOBJECTDIR.

USAGE - CACHE HIT: During another pull or clone on a different working directory, git will find the required object in LINKEDOBJECTDIR and hard link into the active repo's object store.

USAGE - DELETING: Whenever deleting an object, git will stat the file to see to see if its link count is 2, if so, both copies are deleted.

PERIODIC MAINTENANCE: Periodically, git will check all objects in LINKEDOBJECTDIR for a link count of 1 and delete such files. This will happen if a repository with linking is deleted with `rm -rf`.

This feature would be a benefit to organizations which do lots of cloning (think Travis CI) or users that clone the same project multiple times or have the same file used across different repos.

Please share your thoughts.

Thank you, Will

Junio C Hamano· Feb 24, 2015, 22:26 UTC · re: Will Entriken · lore

Re: gc.linkedobjectdir discussion

Will Entriken <fulldecent@gmail.com> writes:

> USAGE - CACHE HIT:
> During another pull or clone on a different working directory, git
> will find the required object in LINKEDOBJECTDIR and hard link into
> the active repo's object store.

What happens when the necessary objects are in a packfile, mixed with other objects that this repository does not need but may be necessary for other repositories?

> USAGE - DELETING:
> Whenever deleting an object, git will stat the file to see to see if
> its link count is 2, if so, both copies are deleted.

This is totally unworkable if you ever pack your repository. And packing loose objects is a lot more efficient way to reduce your disk footprint than sharing the object store among a handful of repositories.