threads / discuss / 32935

Proposal: sharing .git/config

Subject: Proposal: sharing .git/config

## tl;dr

11 messages between Feb 19, 2013 and Mar 18, 2013.

replies: 10people: 5as markdown or json

Ramkumar Ramachandra· Feb 19, 2013, 09:25 UTC · lore
Hi,

I have this itch where I want to share my remotes config between machines. In my fork, I should be able to specify where my upstream sources are, so remotes get set up automatically when I clone. There are also other things in .git/config that would be nice to share, like whether to do a --word-diff (why isn't it a configuration variable yet?) on the repository. The only problem is that I have no clue how to implement this: I'm currently thinking a special remote ref?

Ram
Thomas Rast· Feb 19, 2013, 10:03 UTC · re: Ramkumar Ramachandra · lore

Re: Proposal: sharing .git/config

Ramkumar Ramachandra <artagnon@gmail.com> writes:
> I have this itch where I want to share my remotes config between
> machines.  In my fork, I should be able to specify where my upstream
> sources are, so remotes get set up automatically when I clone.

Note that you need to carefully pick only certain bits of the config, as otherwise there are big security headaches.

> There are also other things in .git/config that would be nice to
> share, like whether to do a --word-diff (why isn't it a configuration
> variable yet?)
Because that would break pretty much every script that uses git-diff?
-- 
Thomas Rast
trast@{inf,student}.ethz.ch
Ramkumar Ramachandra· Feb 19, 2013, 10:24 UTC · re: Thomas Rast · lore

Re: Proposal: sharing .git/config

Thomas Rast wrote:
Show 8 quoted lines
> Ramkumar Ramachandra <artagnon@gmail.com> writes:
>
>> I have this itch where I want to share my remotes config between
>> machines.  In my fork, I should be able to specify where my upstream
>> sources are, so remotes get set up automatically when I clone.
>
> Note that you need to carefully pick only certain bits of the config, as
> otherwise there are big security headaches.

Right. So, we can just start with remotes for the moment? Ideally, there should be a way to specify which configuration options to publish.

Show 5 quoted lines
>> There are also other things in .git/config that would be nice to
>> share, like whether to do a --word-diff (why isn't it a configuration
>> variable yet?)
>
> Because that would break pretty much every script that uses git-diff?

diff.c already makes a differentiation between git_diff_ui_config() and git_diff_basic_config(); there are configuration options that should only be applied when the command is called interactively.

Thomas Rast· Feb 19, 2013, 10:39 UTC · re: Ramkumar Ramachandra · lore

Re: Proposal: sharing .git/config

Ramkumar Ramachandra <artagnon@gmail.com> writes:
Show 11 quoted lines
> Thomas Rast wrote:
>> Ramkumar Ramachandra <artagnon@gmail.com> writes:
>>> There are also other things in .git/config that would be nice to
>>> share, like whether to do a --word-diff (why isn't it a configuration
>>> variable yet?)
>>
>> Because that would break pretty much every script that uses git-diff?
>
> diff.c already makes a differentiation between git_diff_ui_config()
> and git_diff_basic_config(); there are  configuration options that
> should only be applied when the command is called interactively.

It still breaks every other use of diff unless you make the diff output depend on whether the user runs directly at the terminal (possibly using git's own paging).

For example, if you just say something like 'git diff >file' for inclusion in an email, you expect that to be a git-apply compatible diff.

-- 
Thomas Rast
trast@{inf,student}.ethz.ch
Duy Nguyen· Feb 19, 2013, 10:34 UTC · re: Ramkumar Ramachandra · lore

Re: Proposal: sharing .git/config

On Tue, Feb 19, 2013 at 4:25 PM, Ramkumar Ramachandra <artagnon@gmail.com> wrote:

Show 9 quoted lines
> Hi,
>
> I have this itch where I want to share my remotes config between
> machines.  In my fork, I should be able to specify where my upstream
> sources are, so remotes get set up automatically when I clone.  There
> are also other things in .git/config that would be nice to share, like
> whether to do a --word-diff (why isn't it a configuration variable
> yet?) on the repository.  The only problem is that I have no clue how
> to implement this: I'm currently thinking a special remote ref?

If you check out the config file, then include.path should work. You could add include.ref to point to a ref, but you need to deal with the attached security implications. This has been proposed before (and turned down, I think).

-- 
Duy
Jeff King· Feb 19, 2013, 15:36 UTC · re: Duy Nguyen · lore

Re: Proposal: sharing .git/config

On Tue, Feb 19, 2013 at 05:34:43PM +0700, Nguyen Thai Ngoc Duy wrote:
Show 16 quoted lines
> On Tue, Feb 19, 2013 at 4:25 PM, Ramkumar Ramachandra
> <artagnon@gmail.com> wrote:
> > Hi,
> >
> > I have this itch where I want to share my remotes config between
> > machines.  In my fork, I should be able to specify where my upstream
> > sources are, so remotes get set up automatically when I clone.  There
> > are also other things in .git/config that would be nice to share, like
> > whether to do a --word-diff (why isn't it a configuration variable
> > yet?) on the repository.  The only problem is that I have no clue how
> > to implement this: I'm currently thinking a special remote ref?
> 
> If you check out the config file, then include.path should work. You
> could add include.ref to point to a ref, but you need to deal with the
> attached security implications. This has been proposed before (and
> turned down, I think).
Here's the patch:
  http://article.gmane.org/gmane.comp.version-control.git/189144
The basic argument against it is that you would _not_ want to do:
  $ git config include.ref origin/config

because it's unsafe (you immediately start using config fetched from the remote, before you even get a chance to inspect it). So the recommended way to use it is:

  $ git config include.ref config
  $ git show origin/config ;# make sure it looks reasonable
  $ git update-ref refs/config origin/config
  [time passes...]
  $ git fetch
  $ git diff config origin/config ;# inspect changes
  $ git update-ref refs/config origin/config
But it was pointed out that you could also just do:
  $ git config include.ref upstream-config
  $ git show origin/config ;# make sure it looks reasonable
  $ git show origin/config >.git/upstream-config

and so forth. There are some ways that a pure ref can be more convenient (e.g., if you are carrying local changes on top of the upstream config and want to merge), but ultimately, you can replicate any include.ref workflow with include.path by adding a "deploy" step where you copy the file into $GIT_DIR.

-Peff
Ramkumar Ramachandra· Mar 11, 2013, 19:31 UTC · re: Jeff King · lore

Re: Proposal: sharing .git/config

Jeff King wrote:
Show 52 quoted lines
> On Tue, Feb 19, 2013 at 05:34:43PM +0700, Nguyen Thai Ngoc Duy wrote:
>
>> On Tue, Feb 19, 2013 at 4:25 PM, Ramkumar Ramachandra
>> <artagnon@gmail.com> wrote:
>> > Hi,
>> >
>> > I have this itch where I want to share my remotes config between
>> > machines.  In my fork, I should be able to specify where my upstream
>> > sources are, so remotes get set up automatically when I clone.  There
>> > are also other things in .git/config that would be nice to share, like
>> > whether to do a --word-diff (why isn't it a configuration variable
>> > yet?) on the repository.  The only problem is that I have no clue how
>> > to implement this: I'm currently thinking a special remote ref?
>>
>> If you check out the config file, then include.path should work. You
>> could add include.ref to point to a ref, but you need to deal with the
>> attached security implications. This has been proposed before (and
>> turned down, I think).
>
> Here's the patch:
>
>   http://article.gmane.org/gmane.comp.version-control.git/189144
>
> The basic argument against it is that you would _not_ want to do:
>
>   $ git config include.ref origin/config
>
> because it's unsafe (you immediately start using config fetched from the
> remote, before you even get a chance to inspect it). So the recommended
> way to use it is:
>
>   $ git config include.ref config
>   $ git show origin/config ;# make sure it looks reasonable
>   $ git update-ref refs/config origin/config
>
>   [time passes...]
>
>   $ git fetch
>   $ git diff config origin/config ;# inspect changes
>   $ git update-ref refs/config origin/config
>
> But it was pointed out that you could also just do:
>
>   $ git config include.ref upstream-config
>   $ git show origin/config ;# make sure it looks reasonable
>   $ git show origin/config >.git/upstream-config
>
> and so forth. There are some ways that a pure ref can be more
> convenient (e.g., if you are carrying local changes on top of the
> upstream config and want to merge), but ultimately, you can replicate
> any include.ref workflow with include.path by adding a "deploy" step
> where you copy the file into $GIT_DIR.

This seems to be unnecessarily complex and inelegant. Maybe this functionality is best managed as a separate git repository: `repo` from depot_tools uses a manifest repository containing all the project metadata. Maybe we can extend it/ write an more general version?

Jeff King· Mar 12, 2013, 08:53 UTC · re: Ramkumar Ramachandra · lore

Re: Proposal: sharing .git/config

On Tue, Mar 12, 2013 at 01:01:08AM +0530, Ramkumar Ramachandra wrote:
Show 16 quoted lines
> > But it was pointed out that you could also just do:
> >
> >   $ git config include.ref upstream-config
> >   $ git show origin/config ;# make sure it looks reasonable
> >   $ git show origin/config >.git/upstream-config
> >
> > and so forth. There are some ways that a pure ref can be more
> > convenient (e.g., if you are carrying local changes on top of the
> > upstream config and want to merge), but ultimately, you can replicate
> > any include.ref workflow with include.path by adding a "deploy" step
> > where you copy the file into $GIT_DIR.
> 
> This seems to be unnecessarily complex and inelegant.  Maybe this
> functionality is best managed as a separate git repository: `repo`
> from depot_tools uses a manifest repository containing all the project
> metadata.  Maybe we can extend it/ write an more general version?

I don't think you can avoid the 3-step problem and retain the safety in the general case. Forgetting implementation details for a minute, you have either a 1-step system:

  1. Fetch and start using config from the remote.
which is subject to fetching and executing malicious config, or:
  1. Fetch config from remote.
  2. Inspect it.
  3. Integrate it into the current config.

We can automate the sequence to remove as much friction as possible, but fundamentally step 2 requires some effort from the user. Moving the config to a separate repo does not get rid of those steps. The user either does not look at the config before using it, in which case we are no better than the 1-step scenario, or they do, in which case they are replicating the 3-step scenario.

The other alternative is to automate step 2. The simplest way would be to have a whitelist of "ok to share" config, that would not include things like diff.external that can run arbitrary code. I don't know whether that would make the system too limited for what people want to do. Do we have a concrete example of what config people would like to share in this manner?

-Peff
Ramkumar Ramachandra· Mar 18, 2013, 09:00 UTC · re: Jeff King · lore

Re: Proposal: sharing .git/config

Jeff King wrote:
Show 11 quoted lines
> I don't think you can avoid the 3-step problem and retain the safety in
> the general case.  Forgetting implementation details for a minute, you
> have either a 1-step system:
>
>   1. Fetch and start using config from the remote.
>
> which is subject to fetching and executing malicious config, or:
>
>   1. Fetch config from remote.
>   2. Inspect it.
>   3. Integrate it into the current config.

I don't understand your emphasis on step 2. Isn't the configuration written by me? Why would it be malicious?

I've just started thinking about how to design something that will allow us to share configuration elegantly [1]. Essentially, the metadata repository will consist of *.layout files, one for each repository to clone, containing the .git/config to write after cloning that repository. So, a git.layout might look like:

[layout]
	directory = git
[remote "origin"]
	url = git://github.com/git/git
[remote "ram"]
	url = git@github.com:artagnon/git
[remote "junio"]
	url = git://github.com/gitster/git

As you can see the [layout] is a special section which will tell our fetcher where to place the repository. Everything else is meant to be inserted into the repository's .git/config. However, I can foresee a problem in scaling: when I ask a specific directory like a/b/c to be populated (equivalent of repo sync `a/b/c`), it'll have to parse the layout.directory variable of all the .layout files, and this can be slow. So, maybe we should have a special _manifest.layout listing all the paths?

Further, I see this as a way to work with projects that would otherwise require nested submodules like the Android project. What do you think?

[1]: https://github.com/artagnon/src.layout
Jeff King· Mar 18, 2013, 11:48 UTC · re: Ramkumar Ramachandra · lore

Re: Proposal: sharing .git/config

On Mon, Mar 18, 2013 at 02:30:23PM +0530, Ramkumar Ramachandra wrote:
Show 15 quoted lines
> Jeff King wrote:
> > I don't think you can avoid the 3-step problem and retain the safety in
> > the general case.  Forgetting implementation details for a minute, you
> > have either a 1-step system:
> >
> >   1. Fetch and start using config from the remote.
> >
> > which is subject to fetching and executing malicious config, or:
> >
> >   1. Fetch config from remote.
> >   2. Inspect it.
> >   3. Integrate it into the current config.
> 
> I don't understand your emphasis on step 2.  Isn't the configuration
> written by me?  Why would it be malicious?

Maybe I am misunderstanding the use case, but when people talk about share config, they are often talking about pushing project-wide config out to developers. So the config is not necessarily written by you, but by somebody who had write access to the upstream repository.

The obvious counterpoint is that people usually run "make" right after fetching, so they are trusting what they fetched already. And the counter-counterpoint is that yes, that's true, but at least with the "make" case they can use git to inspect the differences before running them. You may be able to tell that this is not the first time this discussion has happened. :)

Personally, I do not think it is the end of the world for people to opt into the "automatically fetch and respect config" method for certain repositories (and that's why I wrote include.ref support a while ago). It's a security tradeoff that the user may want to make. But I also respect the argument that we should not be endorsing risky behavior by advertising such a feature (especially when the risk is quite subtle, as many users may not realize that git config can execute arbitrary code).

Show 27 quoted lines
> I've just started thinking about how to design something that will
> allow us to share configuration elegantly [1].  Essentially, the
> metadata repository will consist of *.layout files, one for each
> repository to clone, containing the .git/config to write after cloning
> that repository.  So, a git.layout might look like:
> 
> [layout]
> 	directory = git
> [remote "origin"]
> 	url = git://github.com/git/git
> [remote "ram"]
> 	url = git@github.com:artagnon/git
> [remote "junio"]
> 	url = git://github.com/gitster/git
> 
> As you can see the [layout] is a special section which will tell our
> fetcher where to place the repository.  Everything else is meant to be
> inserted into the repository's .git/config.  However, I can foresee a
> problem in scaling: when I ask a specific directory like a/b/c to be
> populated (equivalent of repo sync `a/b/c`), it'll have to parse the
> layout.directory variable of all the .layout files, and this can be
> slow.  So, maybe we should have a special _manifest.layout listing all
> the paths?
> 
> Further, I see this as a way to work with projects that would
> otherwise require nested submodules like the Android project.  What do
> you think?

Yeah, reading your layout description, this is less about git config in particular, and more about managing hierarchies of repos. Which I think is a fine thing to do, and is a sensible place to put config management (since you are probably executing arbitrary code as part of the layout tool anyway). But I don't have a real opinion on the design of such a tool. I have used repo only once or twice to deal with Android. For my own menagerie of small repos, I have a hacky custom tool that is mostly about deciding when there are items to be committed, pushed, or fetched in each repo; I never found the need to handle git config at all.

-Peff
Adam Spiers· Feb 19, 2013, 14:38 UTC · re: Ramkumar Ramachandra · lore

Re: Proposal: sharing .git/config

On Tue, Feb 19, 2013 at 9:25 AM, Ramkumar Ramachandra <artagnon@gmail.com> wrote:

Show 9 quoted lines
> Hi,
>
> I have this itch where I want to share my remotes config between
> machines.  In my fork, I should be able to specify where my upstream
> sources are, so remotes get set up automatically when I clone.  There
> are also other things in .git/config that would be nice to share, like
> whether to do a --word-diff (why isn't it a configuration variable
> yet?) on the repository.  The only problem is that I have no clue how
> to implement this: I'm currently thinking a special remote ref?

I handle these kinds of configuration tasks out of band using mr, and it works pretty well:

  https://github.com/aspiers/mr-config/#readme
  https://github.com/aspiers/mr-config/blob/master/sh.d/git-remotes
Food for thought ...

← back to recent threads