# [PATCH 1/4] imap-send: add wrapper to get server credentials if needed 8 messages from 2017-08-07 to 2017-08-07. Participants: Nicolas Morey-Chaisemartin, Martin Ågren, Jeff King, Junio C Hamano. Thread: https://gitlist.dev/t/46529 ## Nicolas Morey-Chaisemartin, 2017-08-07 14:03 Subject: [PATCH 1/4] imap-send: add wrapper to get server credentials if needed Message-ID: <9687e182-1e23-f84d-3fad-83fdf15dba4c@morey-chaisemartin.com> URL: https://gitlist.dev/e/9687e182-1e23-f84d-3fad-83fdf15dba4c%40morey-chaisemartin.com ``` Signed-off-by: Nicolas Morey-Chaisemartin --- imap-send.c | 38 ++++++++++++++++++++++++-------------- 1 file changed, 24 insertions(+), 14 deletions(-) diff --git a/imap-send.c b/imap-send.c index b2d0b849b..38b3c817e 100644 --- a/imap-send.c +++ b/imap-send.c @@ -926,6 +926,29 @@ static int auth_cram_md5(struct imap_store *ctx, struct imap_cmd *cmd, const cha return 0; } +static void server_fill_credential(struct imap_server_conf *srvc) +{ + struct credential cred = CREDENTIAL_INIT; + + if (srvc->user && srvc->pass) + return; + + cred.protocol = xstrdup(srvc->use_ssl ? "imaps" : "imap"); + cred.host = xstrdup(srvc->host); + + cred.username = xstrdup_or_null(srvc->user); + cred.password = xstrdup_or_null(srvc->pass); + + credential_fill(&cred); + + if (!srvc->user) + srvc->user = xstrdup(cred.username); + if (!srvc->pass) + srvc->pass = xstrdup(cred.password); + + credential_clear(&cred); +} + static struct imap_store *imap_open_store(struct imap_server_conf *srvc, char *folder) { struct credential cred = CREDENTIAL_INIT; @@ -1078,20 +1101,7 @@ static struct imap_store *imap_open_store(struct imap_server_conf *srvc, char *f } #endif imap_info("Logging in...\n"); - if (!srvc->user || !srvc->pass) { - cred.protocol = xstrdup(srvc->use_ssl ? "imaps" : "imap"); - cred.host = xstrdup(srvc->host); - - cred.username = xstrdup_or_null(srvc->user); - cred.password = xstrdup_or_null(srvc->pass); - - credential_fill(&cred); - - if (!srvc->user) - srvc->user = xstrdup(cred.username); - if (!srvc->pass) - srvc->pass = xstrdup(cred.password); - } + server_fill_credential(srvc); if (srvc->auth_method) { struct imap_cmd_cb cb; ``` ## Martin Ågren, 2017-08-07 16:30 Subject: Re: [PATCH 1/4] imap-send: add wrapper to get server credentials if needed Message-ID: URL: https://gitlist.dev/e/CAN0heSoA_Xm%2B14V49hjJBMT7PS_ge0wyhmuvKiw-MHD99rApYQ%40mail.gmail.com In-Reply-To: <9687e182-1e23-f84d-3fad-83fdf15dba4c@morey-chaisemartin.com> ``` On 7 August 2017 at 16:03, Nicolas Morey-Chaisemartin wrote: > Signed-off-by: Nicolas Morey-Chaisemartin > --- > imap-send.c | 38 ++++++++++++++++++++++++-------------- > 1 file changed, 24 insertions(+), 14 deletions(-) > > diff --git a/imap-send.c b/imap-send.c > index b2d0b849b..38b3c817e 100644 > --- a/imap-send.c > +++ b/imap-send.c > @@ -926,6 +926,29 @@ static int auth_cram_md5(struct imap_store *ctx, struct imap_cmd *cmd, const cha > return 0; > } > > +static void server_fill_credential(struct imap_server_conf *srvc) > +{ > + struct credential cred = CREDENTIAL_INIT; > + > + if (srvc->user && srvc->pass) > + return; > + > + cred.protocol = xstrdup(srvc->use_ssl ? "imaps" : "imap"); > + cred.host = xstrdup(srvc->host); > + > + cred.username = xstrdup_or_null(srvc->user); > + cred.password = xstrdup_or_null(srvc->pass); > + > + credential_fill(&cred); > + > + if (!srvc->user) > + srvc->user = xstrdup(cred.username); > + if (!srvc->pass) > + srvc->pass = xstrdup(cred.password); > + > + credential_clear(&cred); > +} > + > static struct imap_store *imap_open_store(struct imap_server_conf *srvc, char *folder) > { > struct credential cred = CREDENTIAL_INIT; > @@ -1078,20 +1101,7 @@ static struct imap_store *imap_open_store(struct imap_server_conf *srvc, char *f > } > #endif > imap_info("Logging in...\n"); > - if (!srvc->user || !srvc->pass) { > - cred.protocol = xstrdup(srvc->use_ssl ? "imaps" : "imap"); > - cred.host = xstrdup(srvc->host); > - > - cred.username = xstrdup_or_null(srvc->user); > - cred.password = xstrdup_or_null(srvc->pass); > - > - credential_fill(&cred); > - > - if (!srvc->user) > - srvc->user = xstrdup(cred.username); > - if (!srvc->pass) > - srvc->pass = xstrdup(cred.password); > - } > + server_fill_credential(srvc); > > if (srvc->auth_method) { > struct imap_cmd_cb cb; "cred.username" is checked further down, but now it will always be NULL, no? ``` ## Nicolas Morey-Chaisemartin, 2017-08-07 17:04 Subject: Re: [PATCH 1/4] imap-send: add wrapper to get server credentials if needed Message-ID: URL: https://gitlist.dev/e/fdd74c52-7d9a-b12e-2e1f-524479201701%40morey-chaisemartin.com In-Reply-To: ``` Le 07/08/2017 à 18:30, Martin Ågren a écrit : > On 7 August 2017 at 16:03, Nicolas Morey-Chaisemartin > wrote: >> Signed-off-by: Nicolas Morey-Chaisemartin >> --- >> imap-send.c | 38 ++++++++++++++++++++++++-------------- >> 1 file changed, 24 insertions(+), 14 deletions(-) >> >> diff --git a/imap-send.c b/imap-send.c >> index b2d0b849b..38b3c817e 100644 >> --- a/imap-send.c >> +++ b/imap-send.c >> @@ -926,6 +926,29 @@ static int auth_cram_md5(struct imap_store *ctx, struct imap_cmd *cmd, const cha >> return 0; >> } >> >> +static void server_fill_credential(struct imap_server_conf *srvc) >> +{ >> + struct credential cred = CREDENTIAL_INIT; >> + >> + if (srvc->user && srvc->pass) >> + return; >> + >> + cred.protocol = xstrdup(srvc->use_ssl ? "imaps" : "imap"); >> + cred.host = xstrdup(srvc->host); >> + >> + cred.username = xstrdup_or_null(srvc->user); >> + cred.password = xstrdup_or_null(srvc->pass); >> + >> + credential_fill(&cred); >> + >> + if (!srvc->user) >> + srvc->user = xstrdup(cred.username); >> + if (!srvc->pass) >> + srvc->pass = xstrdup(cred.password); >> + >> + credential_clear(&cred); >> +} >> + >> static struct imap_store *imap_open_store(struct imap_server_conf *srvc, char *folder) >> { >> struct credential cred = CREDENTIAL_INIT; >> @@ -1078,20 +1101,7 @@ static struct imap_store *imap_open_store(struct imap_server_conf *srvc, char *f >> } >> #endif >> imap_info("Logging in...\n"); >> - if (!srvc->user || !srvc->pass) { >> - cred.protocol = xstrdup(srvc->use_ssl ? "imaps" : "imap"); >> - cred.host = xstrdup(srvc->host); >> - >> - cred.username = xstrdup_or_null(srvc->user); >> - cred.password = xstrdup_or_null(srvc->pass); >> - >> - credential_fill(&cred); >> - >> - if (!srvc->user) >> - srvc->user = xstrdup(cred.username); >> - if (!srvc->pass) >> - srvc->pass = xstrdup(cred.password); >> - } >> + server_fill_credential(srvc); >> >> if (srvc->auth_method) { >> struct imap_cmd_cb cb; > "cred.username" is checked further down, but now it will always be NULL, > no? You're right I missed this. Not sure if this is needed though. From what I understand this means the username/password are store for the next access to credential. but in the current state, there is only one. Maybe the credential_approved can be dropped ? Nicolas ``` ## Martin Ågren, 2017-08-07 17:18 Subject: Re: [PATCH 1/4] imap-send: add wrapper to get server credentials if needed Message-ID: URL: https://gitlist.dev/e/CAN0heSqVmrFwP7LdjDJmH0JivoCc%2BDhGtUiTSBs%3D8nTppzG79A%40mail.gmail.com In-Reply-To: ``` On 7 August 2017 at 19:04, Nicolas Morey-Chaisemartin wrote: > > > Le 07/08/2017 à 18:30, Martin Ågren a écrit : >> On 7 August 2017 at 16:03, Nicolas Morey-Chaisemartin >> wrote: >>> Signed-off-by: Nicolas Morey-Chaisemartin >>> --- >>> imap-send.c | 38 ++++++++++++++++++++++++-------------- >>> 1 file changed, 24 insertions(+), 14 deletions(-) >>> >>> diff --git a/imap-send.c b/imap-send.c >>> index b2d0b849b..38b3c817e 100644 >>> --- a/imap-send.c >>> +++ b/imap-send.c >>> @@ -926,6 +926,29 @@ static int auth_cram_md5(struct imap_store *ctx, struct imap_cmd *cmd, const cha >>> return 0; >>> } >>> >>> +static void server_fill_credential(struct imap_server_conf *srvc) >>> +{ >>> + struct credential cred = CREDENTIAL_INIT; >>> + >>> + if (srvc->user && srvc->pass) >>> + return; >>> + >>> + cred.protocol = xstrdup(srvc->use_ssl ? "imaps" : "imap"); >>> + cred.host = xstrdup(srvc->host); >>> + >>> + cred.username = xstrdup_or_null(srvc->user); >>> + cred.password = xstrdup_or_null(srvc->pass); >>> + >>> + credential_fill(&cred); >>> + >>> + if (!srvc->user) >>> + srvc->user = xstrdup(cred.username); >>> + if (!srvc->pass) >>> + srvc->pass = xstrdup(cred.password); >>> + >>> + credential_clear(&cred); >>> +} >>> + >>> static struct imap_store *imap_open_store(struct imap_server_conf *srvc, char *folder) >>> { >>> struct credential cred = CREDENTIAL_INIT; >>> @@ -1078,20 +1101,7 @@ static struct imap_store *imap_open_store(struct imap_server_conf *srvc, char *f >>> } >>> #endif >>> imap_info("Logging in...\n"); >>> - if (!srvc->user || !srvc->pass) { >>> - cred.protocol = xstrdup(srvc->use_ssl ? "imaps" : "imap"); >>> - cred.host = xstrdup(srvc->host); >>> - >>> - cred.username = xstrdup_or_null(srvc->user); >>> - cred.password = xstrdup_or_null(srvc->pass); >>> - >>> - credential_fill(&cred); >>> - >>> - if (!srvc->user) >>> - srvc->user = xstrdup(cred.username); >>> - if (!srvc->pass) >>> - srvc->pass = xstrdup(cred.password); >>> - } >>> + server_fill_credential(srvc); >>> >>> if (srvc->auth_method) { >>> struct imap_cmd_cb cb; >> "cred.username" is checked further down, but now it will always be NULL, >> no? > > You're right I missed this. > Not sure if this is needed though. > From what I understand this means the username/password are store for the next access to credential. but in the current state, there is only one. > Maybe the credential_approved can be dropped ? I'm no credentials-expert, but api-credentials.txt says this: "Credential helpers are programs executed by Git to fetch or save credentials from and to long-term storage (where "long-term" is simply longer than a single Git process; e.g., credentials may be stored in-memory for a few minutes, or indefinitely on disk)." So the calls to approve/reject probably do matter in some scenarios. The current code is a bit non-obvious as we just discovered since it duplicates the strings (for good reasons, I believe) and then still refers to the originals (also for good reasons, I believe). I suppose your new function could be called like server_fill_credential(&cred, srvc); That should limit the impact of the change, but I'm not sure it's a brilliant interface. Just my 2c. Martin ``` ## Jeff King, 2017-08-07 19:42 Subject: Re: [PATCH 1/4] imap-send: add wrapper to get server credentials if needed Message-ID: <20170807194257.jrdmpvoseauz37uc@sigill.intra.peff.net> URL: https://gitlist.dev/e/20170807194257.jrdmpvoseauz37uc%40sigill.intra.peff.net In-Reply-To: ``` On Mon, Aug 07, 2017 at 07:18:32PM +0200, Martin Ågren wrote: > >> "cred.username" is checked further down, but now it will always be NULL, > >> no? > > > > You're right I missed this. > > Not sure if this is needed though. > > From what I understand this means the username/password are store for the next access to credential. but in the current state, there is only one. > > Maybe the credential_approved can be dropped ? > > I'm no credentials-expert, but api-credentials.txt says this: > > "Credential helpers are programs executed by Git to fetch or save > credentials from and to long-term storage (where "long-term" is simply > longer than a single Git process; e.g., credentials may be stored > in-memory for a few minutes, or indefinitely on disk)." > > So the calls to approve/reject probably do matter in some scenarios. Right, this is important. credential_fill() may actually prompt the user, and we'd want to then pass along that credential for storage. Or the user may have changed their password, and the credential_reject() is the only thing that prevents us from trying an out-dated password over and over. > The current code is a bit non-obvious as we just discovered since it > duplicates the strings (for good reasons, I believe) and then still > refers to the originals (also for good reasons, I believe). I suppose > your new function could be called like > > server_fill_credential(&cred, srvc); > > That should limit the impact of the change, but I'm not sure it's a > brilliant interface. Just my 2c. That would work. I'm also tempted to say that imap_server_conf could just store the "struct credential" inside it. We could even potentially drop its parallel user/pass fields to minimize confusion. Once upon a time imap-send was a fork of another program, which is why most of its code isn't well-integrated with Git. But I think at this point there's very little chance of merging changes back and forth between the two. On the other hand, if we're hoping to get rid of this code in favor of the curl-based approach, then it's not worth spending time on cosmetic refactoring, as long as it still behaves correctly in the interim. -Peff ``` ## Nicolas Morey-Chaisemartin, 2017-08-07 19:55 Subject: Re: [PATCH 1/4] imap-send: add wrapper to get server credentials if needed Message-ID: URL: https://gitlist.dev/e/a4761418-0464-baa1-b415-836f33503eb8%40morey-chaisemartin.com In-Reply-To: <20170807194257.jrdmpvoseauz37uc@sigill.intra.peff.net> ``` Le 07/08/2017 à 21:42, Jeff King a écrit : > On Mon, Aug 07, 2017 at 07:18:32PM +0200, Martin Ågren wrote: > >>>> "cred.username" is checked further down, but now it will always be NULL, >>>> no? >>> You're right I missed this. >>> Not sure if this is needed though. >>> From what I understand this means the username/password are store for the next access to credential. but in the current state, there is only one. >>> Maybe the credential_approved can be dropped ? >> I'm no credentials-expert, but api-credentials.txt says this: >> >> "Credential helpers are programs executed by Git to fetch or save >> credentials from and to long-term storage (where "long-term" is simply >> longer than a single Git process; e.g., credentials may be stored >> in-memory for a few minutes, or indefinitely on disk)." >> >> So the calls to approve/reject probably do matter in some scenarios. > Right, this is important. credential_fill() may actually prompt the > user, and we'd want to then pass along that credential for storage. Or > the user may have changed their password, and the credential_reject() is > the only thing that prevents us from trying an out-dated password over > and over. > >> The current code is a bit non-obvious as we just discovered since it >> duplicates the strings (for good reasons, I believe) and then still >> refers to the originals (also for good reasons, I believe). I suppose >> your new function could be called like >> >> server_fill_credential(&cred, srvc); >> >> That should limit the impact of the change, but I'm not sure it's a >> brilliant interface. Just my 2c. > That would work. I'm also tempted to say that imap_server_conf could > just store the "struct credential" inside it. We could even potentially > drop its parallel user/pass fields to minimize confusion. > > Once upon a time imap-send was a fork of another program, which is why > most of its code isn't well-integrated with Git. But I think at this > point there's very little chance of merging changes back and forth > between the two. > > On the other hand, if we're hoping to get rid of this code in favor of > the curl-based approach, then it's not worth spending time on > cosmetic refactoring, as long as it still behaves correctly in the > interim. > > -Peff Looking at the code, it seems the tunnel mode always uses the legacy imap approach. This would have to be ported to curl and stabilized before dropping the legacy code. In the meantime, it might be worth doing a bit of cleanup. And as I said in patch 4, I have a current IMAP account where it works without curl but not with curl (for unknown reason yet). Meaning legacy needs to stay for a while. But switching to curl as default to get out all the bugs (hence this slightly broken patch series) I think it would make sense to get patch 1 (fixed), 2 and 4 in to really test out the curl implementation and take some time to prepare another serie with code cleanups: dropping imap_server_conf parameters, storing cred therem etc.) Nicolas ``` ## Jeff King, 2017-08-07 20:07 Subject: Re: [PATCH 1/4] imap-send: add wrapper to get server credentials if needed Message-ID: <20170807200724.6pdfkmrrmmkdmmb6@sigill.intra.peff.net> URL: https://gitlist.dev/e/20170807200724.6pdfkmrrmmkdmmb6%40sigill.intra.peff.net In-Reply-To: ``` On Mon, Aug 07, 2017 at 09:55:56PM +0200, Nicolas Morey-Chaisemartin wrote: > > On the other hand, if we're hoping to get rid of this code in favor of > > the curl-based approach, then it's not worth spending time on > > cosmetic refactoring, as long as it still behaves correctly in the > > interim. > > Looking at the code, it seems the tunnel mode always uses the legacy imap approach. > This would have to be ported to curl and stabilized before dropping the legacy code. Urgh. That's an important mode, I'd think, and one that I have a feeling curl may not be interested in supporting, just because of it's complexity. And even if they did, it would take a while for that curl version to become available. So maybe the idea of deprecating the non-curl implementation is not something that can happen anytime soon. :( > In the meantime, it might be worth doing a bit of cleanup. In which case, yeah, it is definitely worth cleaning up the existing code. But I also agree with you that it's worth making sure the curl version behaves as similarly as possible. -Peff ``` ## Junio C Hamano, 2017-08-07 22:21 Subject: Re: [PATCH 1/4] imap-send: add wrapper to get server credentials if needed Message-ID: URL: https://gitlist.dev/e/xmqqd187nhmk.fsf%40gitster.mtv.corp.google.com In-Reply-To: <20170807200724.6pdfkmrrmmkdmmb6@sigill.intra.peff.net> ``` Jeff King writes: > On Mon, Aug 07, 2017 at 09:55:56PM +0200, Nicolas Morey-Chaisemartin wrote: > >> > On the other hand, if we're hoping to get rid of this code in favor of >> > the curl-based approach, then it's not worth spending time on >> > cosmetic refactoring, as long as it still behaves correctly in the >> > interim. >> >> Looking at the code, it seems the tunnel mode always uses the legacy imap approach. >> This would have to be ported to curl and stabilized before dropping the legacy code. > > Urgh. That's an important mode, I'd think, and one that I have a feeling > curl may not be interested in supporting, just because of it's > complexity. And even if they did, it would take a while for that curl > version to become available. > > So maybe the idea of deprecating the non-curl implementation is not > something that can happen anytime soon. :( > >> In the meantime, it might be worth doing a bit of cleanup. > > In which case, yeah, it is definitely worth cleaning up the existing > code. But I also agree with you that it's worth making sure the curl > version behaves as similarly as possible. Thanks for guiding this topic forward. I agree with all points you raised in your reviews. ```