threads / discuss / 25190

Enforcing clone/fetch to use references.

Subject: Enforcing clone/fetch to use references.

## tl;dr

4 messages between Sep 21, 2010 and Sep 23, 2010.

replies: 3people: 3as markdown or json

David Brown· Sep 21, 2010, 20:44 UTC · lore

[resending with corrected subject]

Suppose I want to publish some changes to a tree. I have a server available where I can run a git daemon, but for one reason or another I want to force people to use the another git repo as a reference. The reason could be one of bandwidth, or someone who isn't comfortable making all of the other source available. Ideally, someone who already has the other git repo cloned, and just adds mine as a remote wouldn't notice the difference.

Is there a way to do this? I've tried various ways of using alternates to keep the blobs out of the repository I want to export, but the daemon just follows the alternates. If I remove the alternates, I then seem to have a broken repository. Most things I try, at least carry objects for all of the files in the HEAD tree, which most of the time is a large portion of the data.

If there isn't a way of doing this currently, is this something that others would find useful?

Thanks, David

-- 
Sent by an employee of the Qualcomm Innovation Center, Inc.
The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum.

Jeff King· Sep 21, 2010, 21:31 UTC · re: David Brown · lore

Re: Enforcing clone/fetch to use references.

On Tue, Sep 21, 2010 at 01:44:56PM -0700, David Brown wrote:

Show 7 quoted lines

> Suppose I want to publish some changes to a tree.  I have a server
> available where I can run a git daemon, but for one reason or another
> I want to force people to use the another git repo as a reference.
> The reason could be one of bandwidth, or someone who isn't comfortable
> making all of the other source available.  Ideally, someone who
> already has the other git repo cloned, and just adds mine as a remote
> wouldn't notice the difference.

I think the gentoo people were talking about doing something like this. They wanted you to use some faster and/or restartable protocol to clone initially, and so they wanted to reject initial clones. I'm not sure if they are doing that, and how (from the thread below, I suspect they run a patched git).

The simplest thing would be a pre-upload-pack hook. There was some discussion of that in this thread:

  http://article.gmane.org/gmane.comp.version-control.git/137007

but there are some security implications. For a remote site, where the user running upload-pack trusts the hook, it should be fine.

Once you have that hook, then you basically need to just check whether they are requesting objects in your "you should have this in your reference repository" set (probably by using merge-base to see if it is contained in some "everything our alternate has" ref), and then just

  echo >&2 Sorry, here's how to clone properly...
  exit 1

which should go to the user over the sideband.

And that would cover both the "clone --reference" and "git remote add" cases, as it is not looking at what the user invoked, but which objects they're requesting.

-Peff

David Brown· Sep 21, 2010, 22:12 UTC · re: Jeff King · lore

Re: Enforcing clone/fetch to use references.

On Tue, Sep 21, 2010 at 05:31:35PM -0400, Jeff King wrote:

Show 20 quoted lines

> On Tue, Sep 21, 2010 at 01:44:56PM -0700, David Brown wrote:
> 
> > Suppose I want to publish some changes to a tree.  I have a server
> > available where I can run a git daemon, but for one reason or another
> > I want to force people to use the another git repo as a reference.
> > The reason could be one of bandwidth, or someone who isn't comfortable
> > making all of the other source available.  Ideally, someone who
> > already has the other git repo cloned, and just adds mine as a remote
> > wouldn't notice the difference.
> 
> I think the gentoo people were talking about doing something like this.
> They wanted you to use some faster and/or restartable protocol to clone
> initially, and so they wanted to reject initial clones. I'm not sure if
> they are doing that, and how (from the thread below, I suspect they run
> a patched git).
> 
> The simplest thing would be a pre-upload-pack hook. There was some
> discussion of that in this thread:
> 
>   http://article.gmane.org/gmane.comp.version-control.git/137007

Arun, did you ever get a chance to rework the upload-pack hooks to work only from git daemon? If not, I'd like to take a look into implementing this.

Thanks, David

-- 
Sent by an employee of the Qualcomm Innovation Center, Inc.
The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum.

Arun Raghavan· Sep 23, 2010, 12:39 UTC · re: David Brown · lore

Re: Enforcing clone/fetch to use references.

Hey David,

On 22 September 2010 03:42, David Brown <davidb@codeaurora.org> wrote:

Show 25 quoted lines

> On Tue, Sep 21, 2010 at 05:31:35PM -0400, Jeff King wrote:
>> On Tue, Sep 21, 2010 at 01:44:56PM -0700, David Brown wrote:
>>
>> > Suppose I want to publish some changes to a tree.  I have a server
>> > available where I can run a git daemon, but for one reason or another
>> > I want to force people to use the another git repo as a reference.
>> > The reason could be one of bandwidth, or someone who isn't comfortable
>> > making all of the other source available.  Ideally, someone who
>> > already has the other git repo cloned, and just adds mine as a remote
>> > wouldn't notice the difference.
>>
>> I think the gentoo people were talking about doing something like this.
>> They wanted you to use some faster and/or restartable protocol to clone
>> initially, and so they wanted to reject initial clones. I'm not sure if
>> they are doing that, and how (from the thread below, I suspect they run
>> a patched git).
>>
>> The simplest thing would be a pre-upload-pack hook. There was some
>> discussion of that in this thread:
>>
>>   http://article.gmane.org/gmane.comp.version-control.git/137007
>
> Arun, did you ever get a chance to rework the upload-pack hooks to
> work only from git daemon?  If not, I'd like to take a look into
> implementing this.

I never did get a chance to finish work on this. The last IRC discussion we had about this is at http://dev.gentoo.org/~ford_prefect/git-hooks-discussion-log.txt

We still definitely do need this in Gentoo for when our git migration happens.

Cheers,

-- 
Arun Raghavan
http://arunraghavan.net/
(Ford_Prefect | Gentoo) & (arunsr | GNOME)