threads / discuss / 30548

Local cache for git objects

Subject: Local cache for git objects

## tl;dr

4 messages between May 17, 2012 and May 17, 2012.

replies: 3people: 2as markdown or json

Bartosz Gęza· May 17, 2012, 20:06 UTC · lore

I'd like to speed up fetching objects from remote repositiories when creating multiple clones on one or more machines in LAN.

I know it can be done by hand: git-fetch once and git-clone from local copy. I think about adding new capability to upload-pack which would allow making relay and object store. Client would supply url where "want"-ed objects should be available.

Next I'd add config option for git-fetch with url of this cache. fetch-pack would get refs from remote and request objects from cache.

I'm not sure if this hasn't been done. Would such feature be included in git?

Bartosz Gęza· May 17, 2012, 20:36 UTC · lore

Re: Local cache for git objects

On Thu, May 17, 2012 at 10:15 PM, Tomas Carnecky <tomas.carnecky@gmail.com> wrote:

> On Thu, 17 May 2012 22:06:51 +0200, Bartosz Gęza <bartoszgeza@gmail.com> wrote:
>> I'd like to speed up fetching objects from remote repositiories when creating
>> multiple clones on one or more machines in LAN.
> Does git clone --reference do what you want?

I didn't know about --reference, but that's not it. I've tried it just now:

$ git init --bare reference.git Initialized empty Git repository in /.../reference.git $ git clone --reference ./reference.git/ git://someurl/foo.git foo Cloning into 'foo'... [...] (fetches whole pack) $ git clone --reference ./reference.git/ git://someurl/foo.git foo2 Cloning into 'foo2'... [...] (fetches whole pack again)

Reference is read only (?) I'd like to set up one global cache repository and forget. Later as I fetch any objects they are stored there for fast future access.

Junio C Hamano· May 17, 2012, 20:47 UTC · re: Bartosz Gęza · lore

Re: Local cache for git objects

Bartosz Gęza <bartoszgeza@gmail.com> writes:

> Reference is read only (?)
> I'd like to set up one global cache repository and forget.
> Later as I fetch any objects they are stored there for fast future access.

Somebody needs to keep the "global cache" up to date, and that has to be somebody who has access to cron(7) on your system, and you will be a good candidate to do so. So as part of the procedure to "set up one global cache repository and forget (about it)", make a bare clone and then set up a scheduled task to periodically "git fetch" from its origin, and then you are pretty much done, no?

Bartosz Gęza· May 17, 2012, 21:11 UTC · re: Junio C Hamano · lore

Re: Local cache for git objects

On Thu, May 17, 2012 at 10:47 PM, Junio C Hamano <gitster@pobox.com> wrote:

Show 14 quoted lines

>
> Bartosz Gęza <bartoszgeza@gmail.com> writes:
>
> > Reference is read only (?)
> > I'd like to set up one global cache repository and forget.
> > Later as I fetch any objects they are stored there for fast future
> > access.
>
> Somebody needs to keep the "global cache" up to date, and that has to be
> somebody who has access to cron(7) on your system, and you will be a
> good candidate to do so.  So as part of the procedure to "set up one
> global cache repository and forget (about it)", make a bare clone and
> then set up a scheduled task to periodically "git fetch" from its
> origin, and then you are pretty much done, no?

That would work. It's a good practice for a single project with stable dependencies.

My only concern is that it requires effort from user of the cache to keep the list of tracked remotes up to date. After time user would have to recall how to update the list or bother somebody who remembers. I suppose that in a few months since set up it would only include repositories that take over 5-10 minutes to fetch.