threads / discuss / 46164

proposal for how to share other refs as part of refs/tracking/*

Subject: proposal for how to share other refs as part of refs/tracking/*

## tl;dr

4 messages between Jun 12, 2017 and Jun 14, 2017.

replies: 3people: 2as markdown or json

Jacob Keller· Jun 12, 2017, 22:58 UTC · lore
Hi,

There's no actual code yet, (forgive me), but I've been thinking back to a while ago about attempting to find a way to share things like refs/notes, and similar refs which are usually not shared across a remote.

By default, those refs are not propagated when you do a push or a pull, and this makes using them in any project which has more then one repository quite difficult.

I'm going to focus the discussion primarily on refs/notes as this is what I am most interested in, but I think similar issues exist for refs/grafts and refs/replace, and maybe other sources?

For branches, we already have a system to share the status of remote branches, called "refs/remotes/<remote-name>/<branch-name>"

This hierarchy unfortunately does not keep space for non-branches, because you can't simply add a "refs/remotes/notes/<>" or "refs/remotes/<name>/notes" to this as it would be ambiguous.

Now, you might just decide to push the refs/notes directly, ie:
git push origin refs/notes/...:refs/notes/...

Unfortunately, this has problems because it leaves no standard way to distinguish your local work from what is on the remote, so you can't easily merge the remote work into yours.

For example, if Alice creates a new note and pushes it, then Bob creates a different note on the same commit, he needs to be able to merge Alice's changes into his note, just like one would do when two people commit to the same branch.

Today, he must pull the remote notes into a separate location, (since pulling directly into refs/notes will overwrite his work), and then update, and then push.

Because this is not standardized, it becomes unwieldy to actually perform on a day to day basis.

I propose that we add a new "refs/tracking" hierarchy which will be used to track these type of refs

We could even (long term) migrate refs/remotes into refs/tracking instead, or provide both with the refs/remotes being pointers or something like that..

Essentially, refs/notes would be pulled into refs/tracking/<remote>/notes/* or something along these lines.

Then, git notes would be modified to be able to have commands to "pull" and "push" notes which would fetch the remote, and then attempt a merge into the local notes, while a push would update the remote.

I chose "tracking" because it sort of fits the concept and does not include things like "remote-notes" or "remotes-v2" which are a bit weird.

I'm posting this on the mailing list prior to having code because I wanted to get a sense of how people felt about the question and whether others still felt it was an issue.

Essentially the goal is to standardize how any refs namespace can be shared in such a way that local copies can tell what the remote had at a fetch time, in order to allow easier handling of conflicts between local and remote changes, just like we do for branches (heads) but generalized so that other refs namespaces can get the same ability to handle conflicts.

Thanks, Jake

Marc Branchaud· Jun 13, 2017, 14:41 UTC · re: Jacob Keller · lore

Re: proposal for how to share other refs as part of refs/tracking/*

On 2017-06-12 06:58 PM, Jacob Keller wrote:
Show 14 quoted lines
> Hi,
> 
> There's no actual code yet, (forgive me), but I've been thinking back
> to a while ago about attempting to find a way to share things like
> refs/notes, and similar refs which are usually not shared across a
> remote.
> 
> By default, those refs are not propagated when you do a push or a
> pull, and this makes using them in any project which has more then one
> repository quite difficult.
> 
> I'm going to focus the discussion primarily on refs/notes as this is
> what I am most interested in, but I think similar issues exist for
> refs/grafts and refs/replace, and maybe other sources?

More formal support for custom ref namespaces would be a boon. For example, we have our nightly builds create a "build/w.x.y-z" ref (we only want to tag official releases). Sharing those refs is not hard, but a bit obscure.

Show 14 quoted lines
> For branches, we already have a system to share the status of remote
> branches, called "refs/remotes/<remote-name>/<branch-name>"
> 
> This hierarchy unfortunately does not keep space for non-branches,
> because you can't simply add a "refs/remotes/notes/<>" or
> "refs/remotes/<name>/notes" to this as it would be ambiguous.
> 
> Now, you might just decide to push the refs/notes directly, ie:
> 
> git push origin refs/notes/...:refs/notes/...
> 
> Unfortunately, this has problems because it leaves no standard way to
> distinguish your local work from what is on the remote, so you can't
> easily merge the remote work into yours.

There was a related discussion in the run-up to 1.8.0, about a "remote tag namespace" to support having different remotes with the same tag name for different objects. See these messages and their surrounding threads:

http://public-inbox.org/git/AANLkTikeqsg+qJ0z4iQ6ZmKL=_HB8YX_z20L=dFFApmA@mail.gmail.com/
http://public-inbox.org/git/AANLkTi=yFwOAQMHhvLsB1_xmYOE9HHP2YB4H4TQzwwc8@mail.gmail.com/
http://public-inbox.org/git/201102020322.00171.johan@herland.net/

The discussion explored, among other things, making refs/remotes/$remote/* a mirror of a remote's own refs/* hierarchy (well, within reason -- I think there are limits to what should be mirrored).

So I like your refs/tracking proposal, and hope that it aims for mirroring a remote's refs, to eventually replace refs/remotes entirely.

		M.
Show 44 quoted lines
> For example, if Alice creates a new note and pushes it, then Bob
> creates a different note on the same commit, he needs to be able to
> merge Alice's changes into his note, just like one would do when two
> people commit to the same branch.
> 
> Today, he must pull the remote notes into a separate location, (since
> pulling directly into refs/notes will overwrite his work), and then
> update, and then push.
> 
> Because this is not standardized, it becomes unwieldy to actually
> perform on a day to day basis.
> 
> I propose that we add a new "refs/tracking" hierarchy which will be
> used to track these type of refs
> 
> We could even (long term) migrate refs/remotes into refs/tracking
> instead, or provide both with the refs/remotes being pointers or
> something like that..
> 
> Essentially, refs/notes would be pulled into
> refs/tracking/<remote>/notes/* or something along these lines.
> 
> Then, git notes would be modified to be able to have commands to
> "pull" and "push" notes which would fetch the remote, and then attempt
> a merge into the local notes, while a push would update the remote.
> 
> I chose "tracking" because it sort of fits the concept and does not
> include things like "remote-notes" or "remotes-v2" which are a bit
> weird.
> 
> I'm posting this on the mailing list prior to having code because I
> wanted to get a sense of how people felt about the question and
> whether others still felt it was an issue.
> 
> Essentially the goal is to standardize how any refs namespace can be
> shared in such a way that local copies can tell what the remote had at
> a fetch time, in order to allow easier handling of conflicts between
> local and remote changes, just like we do for branches (heads) but
> generalized so that other refs namespaces can get the same ability to
> handle conflicts.
> 
> Thanks,
> Jake
> 
Marc Branchaud· Jun 13, 2017, 15:55 UTC · re: Marc Branchaud · lore

Re: proposal for how to share other refs as part of refs/tracking/*

On 2017-06-13 10:41 AM, Marc Branchaud wrote:
> 
> So I like your refs/tracking proposal, and hope that it aims for 
> mirroring a remote's refs, to eventually replace refs/remotes entirely.
To be extra-clear:

I think a refs/tracking hierarchy that starts with notes and maybe some other bits is a good first step.

I *hope* such a first step can eventually mirror all of a remote's refs, including heads and tags.

I in no way think that this effort should begin by tackling heads and tags.
		M.
Jacob Keller· Jun 14, 2017, 01:24 UTC · re: Marc Branchaud · lore

Re: proposal for how to share other refs as part of refs/tracking/*

On Tue, Jun 13, 2017 at 8:55 AM, Marc Branchaud <marcnarc@xiplink.com> wrote:
Show 19 quoted lines
> On 2017-06-13 10:41 AM, Marc Branchaud wrote:
>>
>>
>> So I like your refs/tracking proposal, and hope that it aims for mirroring
>> a remote's refs, to eventually replace refs/remotes entirely.
>
>
> To be extra-clear:
>
> I think a refs/tracking hierarchy that starts with notes and maybe some
> other bits is a good first step.
>
> I *hope* such a first step can eventually mirror all of a remote's refs,
> including heads and tags.
>
> I in no way think that this effort should begin by tackling heads and tags.
>
>                 M.
>

As far as I understand it, mirroring all refs should be as simple as adding something like

refs/*:refs/tracking/<name>/*
assuming that * begins working for multiple components..
Now actually making *use* of that feature would be different.
However, this was generally the direction I was thinking of heading.

Thanks, Jake

← back to recent threads