threads / discuss / 30614
Subject: credential-helpers + remote-helper, starting point?
Hello,
I want to know if anyone can help me with git-credential-helpers we are trying to use git-credential-helpers in the git-mediawiki (implemented as a remote-helper). We need to ask for the login/pass of the wiki and it would be nice if we can use credential-helpers to manage this credentials.
Anyone can send me a starting point, like a url with the documentation of git-credential-helpers? To see how we can import credential-helpers for used with mediawiki.
Jeff KingRe: credential-helpers + remote-helper, starting point?
On Thu, May 24, 2012 at 05:14:35PM +0200, roucherj wrote:
> I want to know if anyone can help me with git-credential-helpers we > are trying to use git-credential-helpers in the git-mediawiki > (implemented as a remote-helper). > We need to ask for the login/pass of the wiki and it would be nice if > we can use credential-helpers to manage this credentials.
Yeah, I think it makes sense to use the credential-helpers.
> Anyone can send me a starting point, like a url with the > documentation of git-credential-helpers?
Try:
https://github.com/git/git/blob/master/Documentation/technical/api-credentials.txt
But that is the C API, and I assume you are building on the existing mediawiki helper that is written in perl. So I think what you really want is a "git credential" command that will let scripts hook into the credential API. Something like:
$ git credential get https://example.com username=bob password=secret
$ cat <<\EOF | git credential store https://example.com username=bob password=secret EOF
$ cat <<\EOF | git credential erase https://example.com username=bob password=secret EOF
I had planned eventually to do something like this for git-svn, but realized that it was more sane to just let svn library handle the credential storage.
Do you guys want to try writing "git credential" as above? It might be a fun side project, but I know you are also on a limited timeframe for your project. I can work on it if you don't have time.
-Peff
Matthieu MoyRe: credential-helpers + remote-helper, starting point?
Jeff King <peff@peff.net> writes:
> https://github.com/git/git/blob/master/Documentation/technical/api-credentials.txt
I just re-read that document (I had a quick glance only before), and started to understand. I think the document really lacks the "big picture". I took time to understand whether the API was to call credential helpers, or to write a new one. Actually, there are two "API": the C one, and the specification of what may flow on the pipe between the two processes.
Perhaps something like this should be added:
Subject: [PATCH] credential-helper documentation: show the big picture first
--- Documentation/technical/api-credentials.txt | 47 +++++++++++++++++++++++++-- 1 file changed, 44 insertions(+), 3 deletions(-)
diff --git a/Documentation/technical/api-credentials.txt b/Documentation/technical/api-credentials.txt index 21ca6a2..5a872c0 100644 --- a/Documentation/technical/api-credentials.txt +++ b/Documentation/technical/api-credentials.txt @@ -6,8 +6,40 @@ password credentials from the user (even though credentials in the wider world can take many forms, in this document the word "credential" always refers to a username and password pair). +When a function in Git or one of its remote-helpers needs to obtain +credentials (either by asking the user or by fetching from a store), +it can call the functions in the C API. These functions will fork a +new process, and communicate with it by passing command-line arguments +and then communicating through a pipe (see 'Credential Helpers' +below). The credential helper process will be in charge of actually +prompting the user and/or storing and fetching the credentials. + +For example, the execution of a command connecting to an HTTP server +and using the credential helper "cache" will have the following +structure: + +------------ ++-----+ -----> +-----------------+ +| git | pipe | git remote-http | --- to HTTP server ---> ++-----+ <----- +-----------------+ + ^ | + | pipe | + | v + +----------------------+ + | git credential-cache | + +----------------------+ +------------ + +git remote-http will take care of contacting the HTTP server, do the +actual authentication and see if it's accepted by the server. The +credential helper will deal with the credential store (which can be +done by contacting a keyring daemon) and the prompting if needed. + +C API +----- + Data Structures ---------------- +~~~~~~~~~~~~~~~ `struct credential`:: @@ -28,7 +60,7 @@ This struct should always be initialized with `CREDENTIAL_INIT` or Functions ---------- +~~~~~~~~~ `credential_init`:: @@ -72,7 +104,7 @@ Functions Parse a URL into broken-down credential fields. Example -------- +~~~~~~~ The example below shows how the functions of the credential API could be used to login to a fictitious "foo" service on a remote host: @@ -130,6 +162,9 @@ int foo_login(struct foo_connection *f) Credential Helpers ------------------ +Choosing the credential helper command +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + Credential helpers are programs executed by git to fetch or save credentials from and to long-term storage (where "long-term" is simply longer than a single git process; e.g., credentials may be stored @@ -176,6 +211,9 @@ users by naming their program "git-credential-$NAME", and putting it in the $PATH or $GIT_EXEC_PATH during installation, which will allow a user to enable it with `git config credential.helper $NAME`. +Credential helper command-line arguments +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + When a helper is executed, it will have one "operation" argument appended to its command line, which is one of: @@ -191,6 +229,9 @@ appended to its command line, which is one of: Remove a matching credential, if any, from the helper's storage. +Credential helper protocol +~~~~~~~~~~~~~~~~~~~~~~~~~~ + The details of the credential will be provided on the helper's stdin stream. The credential is split into a set of named attributes. Attributes are provided to the helper, one per line. Each attribute is
-- 1.7.10.363.g7fcd3d.dirty Also, shouldn't the documentation about choosing the command name be moved to git-config.txt, to document credential.helper? It really seems to be a user documentation, not a technical one meant for Git developers. > But that is the C API, and I assume you are building on the existing > mediawiki helper that is written in perl. Right. > So I think what you really want is a "git credential" command that > will let scripts hook into the credential API. Something like: > > $ git credential get https://example.com > username=bob > password=secret > [...] This is almost already done by test-credential.c indeed. But that's probably the simplest way to expose the C API to a perl program. > Do you guys want to try writing "git credential" as above? It might be a > fun side project, but I know you are also on a limited timeframe for > your project. I can work on it if you don't have time. I leave it up to the students. -- Matthieu Moy http://www-verimag.imag.fr/~moy/
Jeff KingRe: credential-helpers + remote-helper, starting point?
On Thu, May 24, 2012 at 09:20:08PM +0200, Matthieu Moy wrote:
> Jeff King <peff@peff.net> writes: > > > https://github.com/git/git/blob/master/Documentation/technical/api-credentials.txt > > I just re-read that document (I had a quick glance only before), and > started to understand. I think the document really lacks the "big > picture". I took time to understand whether the API was to call > credential helpers, or to write a new one. Actually, there are two > "API": the C one, and the specification of what may flow on the pipe > between the two processes.
Right. There are really two audiences for this document: people who are writing git code that wants to use credentials, and people who want to write a helper. And it probably makes sense to say that up front, and point people to the right section.
> diff --git a/Documentation/technical/api-credentials.txt b/Documentation/technical/api-credentials.txt > index 21ca6a2..5a872c0 100644 > --- a/Documentation/technical/api-credentials.txt > +++ b/Documentation/technical/api-credentials.txt > @@ -6,8 +6,40 @@ password credentials from the user (even though credentials in the wider > world can take many forms, in this document the word "credential" always > refers to a username and password pair). > > +When a function in Git or one of its remote-helpers needs to obtain > +credentials (either by asking the user or by fetching from a store), > +it can call the functions in the C API. These functions will fork a > +new process, and communicate with it by passing command-line arguments > +and then communicating through a pipe (see 'Credential Helpers' > +below).
We might or might not fork a new process. It depends on which helpers are defined.
> The credential helper process will be in charge of actually > +prompting the user and/or storing and fetching the credentials.
That's not exactly true; the C credential code will actually prompt the user if no helper provides it (in early versions, the helpers were responsible for prompting, but that is no longer the case).
> +For example, the execution of a command connecting to an HTTP server > +and using the credential helper "cache" will have the following > +structure: > + > +------------ > ++-----+ -----> +-----------------+ > +| git | pipe | git remote-http | --- to HTTP server ---> > ++-----+ <----- +-----------------+ > + ^ | > + | pipe | > + | v > + +----------------------+ > + | git credential-cache | > + +----------------------+ > +------------ > + > +git remote-http will take care of contacting the HTTP server, do the > +actual authentication and see if it's accepted by the server. The > +credential helper will deal with the credential store (which can be > +done by contacting a keyring daemon) and the prompting if needed.
I feel like adding remote-helpers into the mix just makes the situation more complex (and necessitates the diagram). I also don't want to get too much into "how it works inside" and would rather stay at the level of "here's how you use it".
What do you think of the patch below? It tries to route the reader to the most useful spot, and it would naturally extend to mentioning a "git credential" command when one exists (which would be separately documented in git-credential.txt).
---
diff --git a/Documentation/technical/api-credentials.txt b/Documentation/technical/api-credentials.txt index 21ca6a2..f6fa203 100644 --- a/Documentation/technical/api-credentials.txt +++ b/Documentation/technical/api-credentials.txt @@ -6,8 +6,27 @@ password credentials from the user (even though credentials in the wider world can take many forms, in this document the word "credential" always refers to a username and password pair). +This document describes two interfaces: the C API that the credential +subsystem provides to the rest of git, and the protocol that git uses to +communicate with system-specific "credential helpers". If you are +writing git code that wants to look up or prompt for credentials, see +the section "C API" below. If you want to write your own helper, see +the section on "Credential Helpers" below. + + +C API +----- + +The credential C API is meant to be called by git code which needs to +acquire or store a credential. It is centered around an object +representing a single credential and provides three basic operations: +fill (acquire credentials by calling helpers and/or prompting the user), +approve (mark a credential as successfully used so that it can be stored +for later use), and reject (mark a credential as unsuccessful so that it +can be erased from any persistent storage). + Data Structures ---------------- +~~~~~~~~~~~~~~~ `struct credential`:: @@ -28,7 +47,7 @@ This struct should always be initialized with `CREDENTIAL_INIT` or Functions ---------- +~~~~~~~~~ `credential_init`:: @@ -72,7 +91,7 @@ Functions Parse a URL into broken-down credential fields. Example -------- +~~~~~~~ The example below shows how the functions of the credential API could be used to login to a fictitious "foo" service on a remote host:
Matthieu MoyRe: credential-helpers + remote-helper, starting point?
Jeff King <peff@peff.net> writes:
>> +When a function in Git or one of its remote-helpers needs to obtain >> +credentials (either by asking the user or by fetching from a store), >> +it can call the functions in the C API. These functions will fork a >> +new process, and communicate with it by passing command-line arguments >> +and then communicating through a pipe (see 'Credential Helpers' >> +below). > > We might or might not fork a new process. It depends on which helpers > are defined.
OK, so s/will fork/may fork/ may be more appropriate.
>> +For example, the execution of a command connecting to an HTTP server >> +and using the credential helper "cache" will have the following >> +structure: >> + >> +------------ >> ++-----+ -----> +-----------------+ >> +| git | pipe | git remote-http | --- to HTTP server ---> >> ++-----+ <----- +-----------------+ >> + ^ | >> + | pipe | >> + | v >> + +----------------------+ >> + | git credential-cache | >> + +----------------------+ >> +------------ >> + >> +git remote-http will take care of contacting the HTTP server, do the >> +actual authentication and see if it's accepted by the server. The >> +credential helper will deal with the credential store (which can be >> +done by contacting a keyring daemon) and the prompting if needed. > > I feel like adding remote-helpers into the mix just makes the situation > more complex (and necessitates the diagram).
The helper is not strictly needed, but that seems to be a common situation, so I thought describing the common case made sense.
I think it's important to show the difference between the actual authentication process (e.g. HTTP) and the credential helper. I took time to understand the example later:
credential_fill(&c);
/* [...] */
status = send_foo_login(f, c.username, c.password);
switch (status) {then I understood that send_foo_login was the one doing the actual authentication, and that further calls to the credential API was needed to say how it went. Making explicit the fact that the Git code talks both the helper and the server on which we are authenticating would have helped me to understand at least.
I think mentionning the fact that a credential helper may connect to a keyring daemon was important also. I first read some discussions about credential helpers thinking that we would have to re-implement the storing mechanism, and really understood what it was all about when I saw patches for KDE support for example.
Other than that, I do like your wording. If it was just me, I'd take your introduction, but keep my example + diagram.
-- Matthieu Moy http://www-verimag.imag.fr/~moy/
Junio C HamanoRe: credential-helpers + remote-helper, starting point?
Jeff King <peff@peff.net> writes:
> On Thu, May 24, 2012 at 09:20:08PM +0200, Matthieu Moy wrote: > >> +------------ >> ++-----+ -----> +-----------------+ >> +| git | pipe | git remote-http | --- to HTTP server ---> >> ++-----+ <----- +-----------------+ >> + ^ | >> + | pipe | >> + | v >> + +----------------------+ >> + | git credential-cache | >> + +----------------------+ >> +------------ >> + >> +git remote-http will take care of contacting the HTTP server, do the >> +actual authentication and see if it's accepted by the server. The >> +credential helper will deal with the credential store (which can be >> +done by contacting a keyring daemon) and the prompting if needed. > > I feel like adding remote-helpers into the mix just makes the situation > more complex (and necessitates the diagram).
I would also prefer to see an example _without_ the remote helper, but at the same time it still helps to have an illustration.
> +This document describes two interfaces: the C API that the credential > +subsystem provides to the rest of git, and the protocol that git uses to > +communicate with system-specific "credential helpers". If you are > +writing git code that wants to look up or prompt for credentials, see
Don't you "prompt" yourself? The above sounds as if you are delegating both looking up and prompting to the helper.
> +the section "C API" below. If you want to write your own helper, see > +the section on "Credential Helpers" below.
Nice.
Matthieu MoyRe: credential-helpers + remote-helper, starting point?
Junio C Hamano <gitster@pobox.com> writes:
> Jeff King <peff@peff.net> writes: > >> On Thu, May 24, 2012 at 09:20:08PM +0200, Matthieu Moy wrote: >> >> +This document describes two interfaces: the C API that the credential >> +subsystem provides to the rest of git, and the protocol that git uses to >> +communicate with system-specific "credential helpers". If you are >> +writing git code that wants to look up or prompt for credentials, see > > Don't you "prompt" yourself? The above sounds as if you are delegating > both looking up and prompting to the helper.
It seems I misunderstood how it worked, indeed. I thought the prompting could also be done by the helper (it seems sensible if you're going to use your desktop's keyring to also allow the desktop's GUI for prompting for example).
Looking closely at the code, credential_fill either delegates fetching data to the remote helper, or does the prompting. IOW, the prompting is done behind the C API anyway, but most likely from credential.c than from the external helper. Is that understanding correct?
Thanks,
-- Matthieu Moy http://www-verimag.imag.fr/~moy/
Re: credential-helpers + remote-helper, starting point?
On Thu, 24 May 2012 14:21:10 -0400, Jeff King wrote:
> On Thu, May 24, 2012 at 05:14:35PM +0200, roucherj wrote: > >> I want to know if anyone can help me with git-credential-helpers we >> are trying to use git-credential-helpers in the git-mediawiki >> (implemented as a remote-helper). >> We need to ask for the login/pass of the wiki and it would be nice >> if >> we can use credential-helpers to manage this credentials. > > Yeah, I think it makes sense to use the credential-helpers. > >> Anyone can send me a starting point, like a url with the >> documentation of git-credential-helpers? > > Try: > > > > https://github.com/git/git/blob/master/Documentation/technical/api-credentials.txt > > But that is the C API, and I assume you are building on the existing > mediawiki helper that is written in perl. So I think what you really > want is a "git credential" command that will let scripts hook into > the > credential API. Something like: > > $ git credential get https://example.com > username=bob > password=secret > > $ cat <<\EOF | git credential store https://example.com > username=bob > password=secret > EOF > > $ cat <<\EOF | git credential erase https://example.com > username=bob > password=secret > EOF > > I had planned eventually to do something like this for git-svn, but > realized that it was more sane to just let svn library handle the > credential storage. > > Do you guys want to try writing "git credential" as above? It might > be a > fun side project, but I know you are also on a limited timeframe for > your project. I can work on it if you don't have time. > > -Peff
Jeff thanks you for helping me with use credentials-helpers with git-remote I will try to code your proposed solution. making something like that:
+-----+ -----> +----------------------+
| git | pipe | git-remote-mediawiki |
+-----+ <----- +----------------------+
/\ ||
|| pipe ||
|| \/
+----------------+ -----> O
| git-credential | <----- -|-
+----------------+ / \
/\ || User
|| pipe ||
|| \/
+-------------------------+
| git-credentials-helpers |
+-------------------------+Jeff KingRe: credential-helpers + remote-helper, starting point?
On Fri, May 25, 2012 at 03:28:39PM +0200, roucherj wrote:
> Jeff thanks you for helping me with use credentials-helpers with > git-remote > I will try to code your proposed solution. > making something like that: > > +-----+ -----> +----------------------+ > | git | pipe | git-remote-mediawiki | > +-----+ <----- +----------------------+ > /\ || > || pipe || > || \/ > +----------------+ -----> O > | git-credential | <----- -|- > +----------------+ / \ > /\ || User > || pipe || > || \/ > +-------------------------+ > | git-credentials-helpers | > +-------------------------+
Exactly. Let me know if you run into any problems.
-Peff