threads / discuss / 28844

share object storage for multiple clones of different repositories

Subject: share object storage for multiple clones of different repositories

## tl;dr

5 messages between Nov 4, 2011 and Nov 5, 2011.

replies: 4people: 3as markdown or json

Gelonida N· Nov 4, 2011, 23:10 UTC · lore

Hi,

(Thunderbird frozewhile sending my previous message so here a resend)

I wondered whether it is possible, that all of my git repository clones share the same object storage whether they are cloned from they same remote repository or not.

In my case this might save a lot of diskspace and accelerate cloning as some huge files are part of several repositories' history and as I'd like to clone the same repository multiple times in order to have parallel working directories for parallel tests on different versions / branches / tags

Further this might reduce clone times.

My goal would be to:
- reduce disk space
- reduce clone time, as objects would be taken from the existing shared
  object storage

If possible it would be great if also all new created shared objects would end up in the new object storage.

If git doesn't support this natively, then would following approach work for reducing the disk space (clone time would not be reduced though)

SHARED_STORAGE=$HOME/shared_storage mkdir $SHARED_STORAGE

git clone remotehost1:repo1 cd repo1 rsync -av .git/objects $SHARED_REPO rm -rf .git/objects ln -s $SHARED_REPO/objects .git/

git clone remotehost2:repo2 cd repo2 rsync -av .git/objects $SHARED_REPO rm -rf .git/objects ln -s $SHARED_REPO/objects .git/

Thanks for any feedback or other suggestions.

Junio C Hamano· Nov 5, 2011, 02:26 UTC · re: Gelonida N · lore

Re: share object storage for multiple clones of different repositories

Gelonida N <gelonida@gmail.com> writes:

Show 6 quoted lines

> SHARED_STORAGE=$HOME/shared_storage
> mkdir $SHARED_STORAGE
>
> git clone remotehost1:repo1
> cd repo1
> rsync -av .git/objects $SHARED_REPO

Up to this part it is probably OK. Repeat that for all your local repositories to collect all objects in $HOME/shared_storage.

After doing that, do this in all of your local repositories:

	rm -rf .git/objects
        mkdir -p .git/objects/info
        echo $HOME/shared/storage >.git/objects/info/alternates

The reason why nobody should follow your original recipe is because any "git gc"/"git repack" in any of your local repositories would break others with that approach.

Frans Klaver· Nov 5, 2011, 07:37 UTC · re: Junio C Hamano · lore

Re: share object storage for multiple clones of different repositories

On Sat, 05 Nov 2011 03:26:11 +0100, Junio C Hamano <gitster@pobox.com> wrote:

Show 22 quoted lines

> Gelonida N <gelonida@gmail.com> writes:
>
>> SHARED_STORAGE=$HOME/shared_storage
>> mkdir $SHARED_STORAGE
>>
>> git clone remotehost1:repo1
>> cd repo1
>> rsync -av .git/objects $SHARED_REPO
>
> Up to this part it is probably OK.  Repeat that for all your local
> repositories to collect all objects in $HOME/shared_storage.
>
> After doing that, do this in all of your local repositories:
>
> 	rm -rf .git/objects
>         mkdir -p .git/objects/info
>         echo $HOME/shared/storage >.git/objects/info/alternates
>
> The reason why nobody should follow your original recipe is because any
> "git gc"/"git repack" in any of your local repositories would break  
> others
> with that approach.

Alternatively there's the git-new-workdir script in contrib/workdir in git.git. Haven't tested it, but it seems like it does what you want.

Frans

Gelonida N· Nov 5, 2011, 16:06 UTC · re: Frans Klaver · lore

Re: share object storage for multiple clones of different repositories

Thanks for both of your replies,

I'll probably mix both approaches and try out how it works. I didn't know about git.git What is the best url for an intrudoction into git.git?

On 11/05/2011 08:37 AM, Frans Klaver wrote:

Show 31 quoted lines

> On Sat, 05 Nov 2011 03:26:11 +0100, Junio C Hamano <gitster@pobox.com>
> wrote:
> 
>> Gelonida N <gelonida@gmail.com> writes:
>>
>>> SHARED_STORAGE=$HOME/shared_storage
>>> mkdir $SHARED_STORAGE
>>>
>>> git clone remotehost1:repo1
>>> cd repo1
>>> rsync -av .git/objects $SHARED_REPO
>>
>> Up to this part it is probably OK.  Repeat that for all your local
>> repositories to collect all objects in $HOME/shared_storage.
>>
>> After doing that, do this in all of your local repositories:
>>
>>     rm -rf .git/objects
>>         mkdir -p .git/objects/info
>>         echo $HOME/shared/storage >.git/objects/info/alternates
>>
>> The reason why nobody should follow your original recipe is because any
>> "git gc"/"git repack" in any of your local repositories would break
>> others
>> with that approach.
> 
> 
> Alternatively there's the git-new-workdir script in contrib/workdir in
> git.git. Haven't tested it, but it seems like it does what you want.
> 
> Frans

Gelonida N· Nov 5, 2011, 16:20 UTC · re: Gelonida N · lore

Re: share object storage for multiple clones of different repositories

On 11/05/2011 05:06 PM, Gelonida N wrote:

Show 15 quoted lines

> Thanks for both of your replies,
> 
> 
> I'll probably mix both approaches and try out how it works.
> I didn't know about git.git
> What is the best url for an intrudoction into git.git?
> 
> 
>>
>> Alternatively there's the git-new-workdir script in contrib/workdir in
>> git.git. Haven't tested it, but it seems like it does what you want.
>>
>> Frans
> 
>

In an installed git (at least on Ubuntu) the contrib directory can be found at /usr/share/doc/git/contrib