threads / discuss / 32127

cloning a namespace downloads all the objects

Subject: cloning a namespace downloads all the objects

## tl;dr

3 messages between Nov 16, 2012 and Nov 16, 2012.

replies: 2people: 3as markdown or json

Sitaram Chamarty· Nov 16, 2012, 16:34 UTC · lore
Hi,

I have a repo on a server, which contains, as namespaces, the contents of several different repos of varying sizes. When I run a clone command for the smallest of the namespaces (I have a script that intercepts the clone and sets GIT_NAMESPACE appropriately), I get the correct set of refs, but *all* the objects from *all* the namespaces.

And since no refs from the other namespaces have come down, a 'git gc --prune=now', run immediately after, reduces the size of ".git/objects" to the size I would expect for just that small namespace.

In effect, it is bringing down data that is not reachable and will be wiped out on the next gc.

Is this expected?
-- 
Sitaram
Junio C Hamano· Nov 16, 2012, 16:43 UTC · re: Sitaram Chamarty · lore

Re: cloning a namespace downloads all the objects

Sitaram Chamarty <sitaramc@gmail.com> writes:
Show 17 quoted lines
> Hi,
>
> I have a repo on a server, which contains, as namespaces, the contents
> of several different repos of varying sizes.  When I run a clone
> command for the smallest of the namespaces (I have a script that
> intercepts the clone and sets GIT_NAMESPACE appropriately), I get the
> correct set of refs, but *all* the objects from *all* the namespaces.
>
> And since no refs from the other namespaces have come down, a 'git gc
> --prune=now', run immediately after, reduces the size of
> ".git/objects" to the size I would expect for just that small
> namespace.
>
> In effect, it is bringing down data that is not reachable and will be
> wiped out on the next gc.
>
> Is this expected?
I do not think so.

This was done with a series between a1bea2c (ref namespaces: infrastructure, 2011-07-05) and bf7930c (ref namespaces: tests, 2011-07-21); Josh, care to comment on and to look into it?

Jeff King· Nov 16, 2012, 17:34 UTC · re: Junio C Hamano · lore

Re: cloning a namespace downloads all the objects

On Fri, Nov 16, 2012 at 08:43:13AM -0800, Junio C Hamano wrote:
Show 21 quoted lines
> > I have a repo on a server, which contains, as namespaces, the contents
> > of several different repos of varying sizes.  When I run a clone
> > command for the smallest of the namespaces (I have a script that
> > intercepts the clone and sets GIT_NAMESPACE appropriately), I get the
> > correct set of refs, but *all* the objects from *all* the namespaces.
> >
> > And since no refs from the other namespaces have come down, a 'git gc
> > --prune=now', run immediately after, reduces the size of
> > ".git/objects" to the size I would expect for just that small
> > namespace.
> >
> > In effect, it is bringing down data that is not reachable and will be
> > wiped out on the next gc.
> >
> > Is this expected?
> 
> I do not think so.
> 
> This was done with a series between a1bea2c (ref namespaces:
> infrastructure, 2011-07-05) and bf7930c (ref namespaces: tests,
> 2011-07-21); Josh, care to comment on and to look into it?

I'd guess that the "create_full_pack" logic in create_pack_file is to blame. The client asked for everything we advertised, so we pass "--all" to pack-objects rather than giving it the specific list of tips.

We'd have to either fix that logic, or teach the pack-objects subprocess to respect GIT_NAMESPACE when processing --all.

-Peff

← back to recent threads