threads / patch / 56861

patchbuiltin-commit: re-read file index before launching editor

Subject: [PATCH] builtin-commit: re-read file index before launching editor

## tl;dr

18 messages between Nov 9, 2021 and Nov 18, 2021. Diffs are folded; open one to read it.

replies: 17people: 5as markdown or json

Samuel Yvon via GitGitGadget· Nov 9, 2021, 02:06 UTC · lore
From: Samuel Yvon <samuelyvon9@gmail.com>

Changes made within a pre-commit hook are not reflected within the editor (for instance, with `git commit --verbose`) because the index is re-read after the editor has been used.

This has the consequence of not displaying the actual changes made by the pre-commit hook, but committing them anyways.

While it is often argued that the hook's purpose is not to automatically write content to the repository, it is acknowledged that using the pre-commit to apply mechanical fixes on top of the existing changes is a supported use case, along with verifying the content.

I think not seeing these mechanical fixes before commiting is incorrect. A developer might expect the commit to look a certain way but if the pre-commit goes wrong the invalid changes will go unnoticed until committed.

Signed-off-by: Samuel Yvon <samuelyvon9@gmail.com>
---
    builtin-commit: Re-read file index before launching editor
    
    Changes made within a pre-commit hook are not reflected within the
    editor (for instance, with git commit --verbose) because the index is
    re-read after the editor has been used.
    
    This has the consequence of not displaying the actual changes made by
    the pre-commit hook, but committing them anyways.
    
    While it is often argued that the hook's purpose is not to automatically
    write content to the repository, it is acknowledged that using the
    pre-commit to apply mechanical fixes on top of the existing changes is a
    supported use case, along with verifying the content.
    
    I think not seeing these mechanical fixes before commiting is incorrect.
    A developer might expect the commit to look a certain way but if the
    pre-commit goes wrong the invalid changes will go unnoticed until
    committed.
    
    I had a small exchange in the Google Group before submitting this Pr.
    Here is a link for cross-referencing:
    https://groups.google.com/g/git-mentoring/c/FsP83I9mN6c
    
    As a side note, I do not know who manages the Github Repo but the
    following description threw me off a little bit:
    
    Git Source Code Mirror - This is a publish-only repository and all pull requests are ignored. 
    
    
    since after looking deeper it seems the PRs are not ignored.
    
    Signed-off-by: Samuel Yvon samuelyvon9@gmail.com
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-git-1127%2FSamuelYvon%2Fmaint-v1
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-git-1127/SamuelYvon/maint-v1
Pull-Request: https://github.com/git/git/pull/1127
 builtin/commit.c | 21 +++++++++++----------
 1 file changed, 11 insertions(+), 10 deletions(-)
Show changes to builtin/commit.c +11 −10
diff --git a/builtin/commit.c b/builtin/commit.c
index 7c9b1e7be3a..e75b11d1c60 100644
--- a/builtin/commit.c
+++ b/builtin/commit.c
@@ -728,8 +728,17 @@ static int prepare_to_commit(const char *index_file, const char *prefix,
 	/* This checks and barfs if author is badly specified */
 	determine_author_info(author_ident);
 
-	if (!no_verify && run_commit_hook(use_editor, index_file, "pre-commit", NULL))
-		return 0;
+	if (!no_verify && find_hook("pre-commit")) {
+		if(run_commit_hook(use_editor, index_file, "pre-commit", NULL))
+			return 0;
+
+		/*
+		 * Re-read the index as pre-commit hook could have updated it,
+		 * and write it out as a tree.
+		 */
+		discard_cache();
+		read_cache_from(index_file);
+	}
 
 	if (squash_message) {
 		/*
@@ -1051,14 +1060,6 @@ static int prepare_to_commit(const char *index_file, const char *prefix,
 		return 0;
 	}
 
-	if (!no_verify && find_hook("pre-commit")) {
-		/*
-		 * Re-read the index as pre-commit hook could have updated it,
-		 * and write it out as a tree.  We must do this before we invoke
-		 * the editor and after we invoke run_status above.
-		 */
-		discard_cache();
-	}
 	read_cache_from(index_file);
 
 	if (update_main_cache_tree(0)) {

base-commit: 5fbd2fc5997dfa4d4593a862fe729b1e7a89bcf8
-- 
gitgitgadget
Ævar Arnfjörð Bjarmason· Nov 9, 2021, 02:32 UTC · re: Samuel Yvon via GitGitGadget · lore

Re: [PATCH] builtin-commit: re-read file index before launching editor

On Tue, Nov 09 2021, Samuel Yvon via GitGitGadget wrote:
Show 102 quoted lines
> From: Samuel Yvon <samuelyvon9@gmail.com>
>
> Changes made within a pre-commit hook are not reflected within
> the editor (for instance, with `git commit --verbose`) because
> the index is re-read after the editor has been used.
>
> This has the consequence of not displaying the actual changes made
> by the pre-commit hook, but committing them anyways.
>
> While it is often argued that the hook's purpose is not to automatically
> write content to the repository, it is acknowledged that using the
> pre-commit to apply mechanical fixes on top of the existing changes
> is a supported use case, along with verifying the content.
>
> I think not seeing these mechanical fixes before commiting is incorrect.
> A developer might expect the commit to look a certain way but if the
> pre-commit goes wrong the invalid changes will go unnoticed until
> committed.
>
> Signed-off-by: Samuel Yvon <samuelyvon9@gmail.com>
> ---
>     builtin-commit: Re-read file index before launching editor
>     
>     Changes made within a pre-commit hook are not reflected within the
>     editor (for instance, with git commit --verbose) because the index is
>     re-read after the editor has been used.
>     
>     This has the consequence of not displaying the actual changes made by
>     the pre-commit hook, but committing them anyways.
>     
>     While it is often argued that the hook's purpose is not to automatically
>     write content to the repository, it is acknowledged that using the
>     pre-commit to apply mechanical fixes on top of the existing changes is a
>     supported use case, along with verifying the content.
>     
>     I think not seeing these mechanical fixes before commiting is incorrect.
>     A developer might expect the commit to look a certain way but if the
>     pre-commit goes wrong the invalid changes will go unnoticed until
>     committed.
>     
>     I had a small exchange in the Google Group before submitting this Pr.
>     Here is a link for cross-referencing:
>     https://groups.google.com/g/git-mentoring/c/FsP83I9mN6c
>     
>     As a side note, I do not know who manages the Github Repo but the
>     following description threw me off a little bit:
>     
>     Git Source Code Mirror - This is a publish-only repository and all pull requests are ignored. 
>     
>     
>     since after looking deeper it seems the PRs are not ignored.
>     
>     Signed-off-by: Samuel Yvon samuelyvon9@gmail.com
>
> Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-git-1127%2FSamuelYvon%2Fmaint-v1
> Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-git-1127/SamuelYvon/maint-v1
> Pull-Request: https://github.com/git/git/pull/1127
>
>  builtin/commit.c | 21 +++++++++++----------
>  1 file changed, 11 insertions(+), 10 deletions(-)
>
> diff --git a/builtin/commit.c b/builtin/commit.c
> index 7c9b1e7be3a..e75b11d1c60 100644
> --- a/builtin/commit.c
> +++ b/builtin/commit.c
> @@ -728,8 +728,17 @@ static int prepare_to_commit(const char *index_file, const char *prefix,
>  	/* This checks and barfs if author is badly specified */
>  	determine_author_info(author_ident);
>  
> -	if (!no_verify && run_commit_hook(use_editor, index_file, "pre-commit", NULL))
> -		return 0;
> +	if (!no_verify && find_hook("pre-commit")) {
> +		if(run_commit_hook(use_editor, index_file, "pre-commit", NULL))
> +			return 0;
> +
> +		/*
> +		 * Re-read the index as pre-commit hook could have updated it,
> +		 * and write it out as a tree.
> +		 */
> +		discard_cache();
> +		read_cache_from(index_file);
> +	}
>  
>  	if (squash_message) {
>  		/*
> @@ -1051,14 +1060,6 @@ static int prepare_to_commit(const char *index_file, const char *prefix,
>  		return 0;
>  	}
>  
> -	if (!no_verify && find_hook("pre-commit")) {
> -		/*
> -		 * Re-read the index as pre-commit hook could have updated it,
> -		 * and write it out as a tree.  We must do this before we invoke
> -		 * the editor and after we invoke run_status above.
> -		 */
> -		discard_cache();
> -	}
>  	read_cache_from(index_file);
>  
>  	if (update_main_cache_tree(0)) {
>
> base-commit: 5fbd2fc5997dfa4d4593a862fe729b1e7a89bcf8

This hook logic is actively being changed these days, we're currently at batch 2/3 of 3/3 (or was it 4/4? I forget) to rewrite all these hook interfaces.

I haven't looked carefully at this, but I've got a patch that touches this exact logic here that'll be in-flight before too long: https://lore.kernel.org/git/patch-v5-36.36-fe056098534-20210902T125111Z-avarab@gmail.com/

Anyway, what you have here seems orthagonal, and it looks like these two could be combined, but on top of my linked patch we wouldn't be racy with the new behavior either.

But (and I've got to run) check out the commit message of that patch, perhaps it links to useful past discussions/commits.

I.e. a thing not covered in your commit message is a discussion of if the current behavior was explicitly desired by anyone, or if we just ended up with it by accident.

The code you're moving around has a comment which seems to suggest that what you want *is* the desired behavior, i.e. we re-read it before invoking the editor, so we should have the updated diff, but just don't?

Or maybe I'm misunderstanding it.
samuelyvon9@gmail.com· Nov 9, 2021, 03:08 UTC · re: Ævar Arnfjörð Bjarmason · lore

Re: [PATCH] builtin-commit: re-read file index before launching editor

From: Samuel Yvon <samuelyvon9@gmail.com>
> Ævar Arnfjörð Bjarmason <avarab@gmail.com> writes:
> The code you're moving around has a comment which seems to suggest that
> what you want *is* the desired behavior, i.e. we re-read it before
> invoking the editor, so we should have the updated diff, but just don't?

My understanding is that it was once the behaviour and has changed over time. I am saying this based on https://lore.kernel.org/git/xmqqk0yripca.fsf@gitster.c.googlers.com/t/#u

Specifically,
Show 21 quoted lines
> Junio C Hamano <gitster@pobox.com> writes:
> Even before ec84bd00 (git-commit: Refactor creation of log message.,
> 2008-02-05), the code anticipated that pre-commit may touch the index
> and tried to cope with it.
> However, ec84bd00 moved the place where we re-read the on-disk index
> in the sequence, and updated a message that used to read:
> 
> -    /*
> -     * Re-read the index as pre-commit hook could have updated it,
> -     * and write it out as a tree.
> -     */
> 
> to:
> 
> +    /*
> +     * Re-read the index as pre-commit hook could have updated it,
> +     * and write it out as a tree.  We must do this before we invoke
> +     * the editor and after we invoke run_status above.
> +     */
> 
> Unfortunately there is no mention of the reason why we "must" here.

Looking at ec84bd00 (git-commit: Refactor creation of log message., 2008-02-05), we can see that the editor is launched after the cache has been reset. The only part that troubles me is the line in the comment that specifies that "we must do this ... after we invoke run_status above". I have tested (with my limited knowledge of the internals of git) and it seems to be of no consequence of flushing before the call to run_status, but I might be missing something.

> The code you're moving around has a comment which seems to suggest that
> what you want *is* the desired behavior, i.e. we re-read it before
> invoking the editor, so we should have the updated diff, but just don't?
I think this is the case (based on the previously linked conversation).
Ævar Arnfjörð Bjarmason· Nov 9, 2021, 09:11 UTC · re: samuelyvon9@gmail.com · lore

Re: [PATCH] builtin-commit: re-read file index before launching editor

On Mon, Nov 08 2021, samuelyvon9@gmail.com wrote:
Show 47 quoted lines
> From: Samuel Yvon <samuelyvon9@gmail.com>
>
>> Ævar Arnfjörð Bjarmason <avarab@gmail.com> writes:
>> The code you're moving around has a comment which seems to suggest that
>> what you want *is* the desired behavior, i.e. we re-read it before
>> invoking the editor, so we should have the updated diff, but just don't?
>
> My understanding is that it was once the behaviour and has changed over time.
> I am saying this based on
> https://lore.kernel.org/git/xmqqk0yripca.fsf@gitster.c.googlers.com/t/#u
>
> Specifically,
>
>> Junio C Hamano <gitster@pobox.com> writes:
>> Even before ec84bd00 (git-commit: Refactor creation of log message.,
>> 2008-02-05), the code anticipated that pre-commit may touch the index
>> and tried to cope with it.
>> However, ec84bd00 moved the place where we re-read the on-disk index
>> in the sequence, and updated a message that used to read:
>> 
>> -    /*
>> -     * Re-read the index as pre-commit hook could have updated it,
>> -     * and write it out as a tree.
>> -     */
>> 
>> to:
>> 
>> +    /*
>> +     * Re-read the index as pre-commit hook could have updated it,
>> +     * and write it out as a tree.  We must do this before we invoke
>> +     * the editor and after we invoke run_status above.
>> +     */
>> 
>> Unfortunately there is no mention of the reason why we "must" here.
>
> Looking at ec84bd00 (git-commit: Refactor creation of log message., 2008-02-05),
> we can see that the editor is launched after the cache has been reset. The only
> part that troubles me is the line in the comment that specifies that "we must do
> this ... after we invoke run_status above". I have tested (with my limited knowledge
> of the internals of git) and it seems to be of no consequence of flushing before
> the call to run_status, but I might be missing something.
>
>> The code you're moving around has a comment which seems to suggest that
>> what you want *is* the desired behavior, i.e. we re-read it before
>> invoking the editor, so we should have the updated diff, but just don't?
>
> I think this is the case (based on the previously linked conversation).

*nod*, the implicit suggestion here being: Let's put more of that summary into the commit message. It helps when looking up/discovering older behavior.

The comment was first added in 2888605c649 (builtin-commit: fix partial-commit support, 2007-11-18), and quite suspicuous in timing is f5bbc3225c4 (Port git commit to C., 2007-11-08) where we moved from git-commit.sh.

It's a bit of a pain to build git that old, but my hunch is that perhaps this was tested with git-commit.sh, where the reading of the index would be another process.

Samuel Yvon· Nov 9, 2021, 15:22 UTC · re: Ævar Arnfjörð Bjarmason · lore

Re: [PATCH] builtin-commit: re-read file index before launching editor

> Ævar Arnfjörð Bjarmason <avarab@gmail.com> writes:
> *nod*, the implicit suggestion here being: Let's put more of that
> summary into the commit message. It helps when looking up/discovering
> older behavior.

Sorry! I will prepare an amended patch later. Exploring the linked commits has raised more questions on my end.

Show 5 quoted lines
> Ævar Arnfjörð Bjarmason <avarab@gmail.com> writes:
> The comment was first added in 2888605c649 (builtin-commit: fix
>         partial-commit support, 2007-11-18), and quite suspicuous in timing is
> f5bbc3225c4 (Port git commit to C., 2007-11-08) where we moved from
> git-commit.sh.

Exploring these commits, I fear I might have been confused; the editor was always launched after the reset (still today). The problem, as noted earlier, is the fact that the run_status is ran before the cache reset, which creates the outdated diff. I can't speculate to the intent of the author, but exploring the code makes this comment:

Show 5 quoted lines
> +    /*
> +     * Re-read the index as pre-commit hook could have updated it,
> +     * and write it out as a tree.  We must do this before we invoke
> +     * the editor and after we invoke run_status above.
> +     */

more confusing, because there is no point of invoking the editor after the reset if the status is written before, since the editor won't show anything different.

On one hand, flushing then showing the editor seems to indicate that we want the editor to be up-to-date, but because the status is prepared before the flushing, maybe not?

While it seems the current behaviour has been the behaviour since the start, I am inclined to continue pushing for this change. Unless I am missing something, the comment is contradictory and we should be coherent with the idea of accepting changes made within the pre-commit hook, as noted in https://lore.kernel.org/git/xmqqk0yripca.fsf@gitster.c.googlers.com/t/#u:

>> Junio C Hamano <gitster@pobox.com> writes:
>> Even before ec84bd00 (git-commit: Refactor creation of log message.,
>> 2008-02-05), the code anticipated that pre-commit may touch the index
>> and tried to cope with it.
Junio C Hamano· Nov 9, 2021, 18:36 UTC · re: Samuel Yvon · lore

Re: [PATCH] builtin-commit: re-read file index before launching editor

Samuel Yvon <samuelyvon9@gmail.com> writes:
Show 14 quoted lines
> On one hand, flushing then showing the editor seems to indicate that we want the
> editor to be up-to-date, but because the status is prepared before the flushing,
> maybe not?
>
> While it seems the current behaviour has been the behaviour since the start,
> I am inclined to continue pushing for this change. Unless I am missing something,
> the comment is contradictory and we should be coherent with the idea of
> accepting changes made within the pre-commit hook, as noted in 
> https://lore.kernel.org/git/xmqqk0yripca.fsf@gitster.c.googlers.com/t/#u:
>
>>> Junio C Hamano <gitster@pobox.com> writes:
>>> Even before ec84bd00 (git-commit: Refactor creation of log message.,
>>> 2008-02-05), the code anticipated that pre-commit may touch the index
>>> and tried to cope with it.

You seem to be quoting the thread over and over, but what you are quoting is somewhat different from the concluding part of what I said, which was:

    If I have to guess, I think the reason is because pre-commit
    automation is expected to be some sort of mechanical change and
    not part of the actual work that the end-user produced, it would
    become easier to perform the "final review" of "what have I done
    so far---does everything make sense?"  if such "extra" changes
    are excluded.
    So, in short, it is not "undefined", but rather it seems to be a
    designed behaviour that we are seeing.

I do not personally mind if we change the philosophy but because it has been a longstanding designed behaviour, it may need a careful transition plan.

Samuel Yvon· Nov 9, 2021, 20:01 UTC · re: Junio C Hamano · lore

Re: [PATCH] builtin-commit: re-read file index before launching editor

Junio C Hamano <gitster@pobox.com> writes:
Show 11 quoted lines
> You seem to be quoting the thread over and over, but what you are
> quoting is somewhat different from the concluding part of what I
> said, which was:
>> If I have to guess, I think the reason is because pre-commit
>> automation is expected to be some sort of mechanical change and
>> not part of the actual work that the end-user produced, it would
>> become easier to perform the "final review" of "what have I done
>> so far---does everything make sense?"  if such "extra" changes
>> are excluded.
> So, in short, it is not "undefined", but rather it seems to be a
> designed behaviour that we are seeing.

Apologies if you feel I misquoted you. I am confused by the comment in the original code w.r.t the location of the cache reset, which (from my understanding) is contradictory with the quoted conversation and so I focused on those parts.

Junio C Hamano <gitster@pobox.com> writes:
> I do not personally mind if we change the philosophy but because it
> has been a longstanding designed behaviour, it may need a careful
> transition plan.
Out of curiosity, what would that involve? 
Junio C Hamano· Nov 11, 2021, 22:09 UTC · re: Samuel Yvon · lore

Re: [PATCH] builtin-commit: re-read file index before launching editor

Samuel Yvon <samuelyvon9@gmail.com> writes:
Show 5 quoted lines
>> I do not personally mind if we change the philosophy but because it
>> has been a longstanding designed behaviour, it may need a careful
>> transition plan.
>
> Out of curiosity, what would that involve? 

We need to make sure we do not break workflows of existing users who relies on the current behaviour.

As Git is widely used, it is very likely that some of the workflow element existing users use depend on the current behaviour.

Letting pre-commit do its thing and letting run_status() compute "commitable" without its effect may be something they depend on. It is the same for any other behaviour we may be tempted to modify. We would be breaking Git for them by changing.

So, not breaking by not changing behaviour, if we can do so, would be ideal. Introducing a configuration option and hide the new and different behaviour behind it, so that only the folks who agree to take responsibility of adjusting to the new behaviour, would be a good way to isolate the existing users from the change.

If we come up with a new behaviour that would make the world a better place if adopted by everybody, we might be tempted to eventually make it the _only_ behaviour available to everybody. I am not sure if it is the case here, but if it were, the transition plan would become even more involved. We'd need to start by warning existing users (i.e. the code must detect a case where pre-commit mucks with the index and having an extra "re-read index file" would have made a difference---and tell the user that they must adjust to the change of behaviour in the future), wait for a few years and then flip the behaviour while removing the warning, or something like that.

Johannes Schindelin· Nov 9, 2021, 16:41 UTC · re: Samuel Yvon via GitGitGadget · lore

Description of github.com/git/git, was Re: [PATCH] builtin-commit: re-read file index before launching editor

Hi Samuel,
On Tue, 9 Nov 2021, Samuel Yvon via GitGitGadget wrote:
>     As a side note, I do not know who manages the Github Repo but the
>     following description threw me off a little bit:
>
>     Git Source Code Mirror - This is a publish-only repository and all pull requests are ignored.
I would be in favor of changing this description. How about:
	Git Source Code Mirror - Pull Requests can be submitted via GitGitGadget
Thoughts?

Ciao, Dscho

Samuel Yvon· Nov 9, 2021, 17:01 UTC · re: Johannes Schindelin · lore

Re: Description of github.com/git/git, was Re: [PATCH] builtin-commit: re-read file index before launching editor

Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
> I would be in favor of changing this description. How about:
> Git Source Code Mirror - Pull Requests can be submitted via GitGitGadget
> Thoughts?

I think it might be somewhat obscure for newcomers (What is GitGitGadget?). I know Github supports links in project descriptions. Maybe

    Git Source Code Mirror - Please read https://gitgitgadget.github.io/ before submitting Pull Requests
This would have helped me figure out if submitting a PR was gonna end up nowhere or not.
Junio C Hamano· Nov 9, 2021, 19:03 UTC · re: Johannes Schindelin · lore

Re: Description of github.com/git/git, was Re: [PATCH] builtin-commit: re-read file index before launching editor

Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
> I would be in favor of changing this description. How about:
>
> 	Git Source Code Mirror - Pull Requests can be submitted via GitGitGadget

Presense of GGG does not change the fact that the repository is publish only. But "all pull requests are ignored" is not correct anymore.

    Git Source Code Mirror - This is a publish-only repository but
    pull requests can be turned into patches to the mailing list via
    GitGitGadget. Please follow Documentation/SubmittingPatches
    procedure for any of your improvements.

I didn't double-check but I presume SubmittingPatches these days talk about GGG?

Taylor Blau· Nov 9, 2021, 19:23 UTC · re: Junio C Hamano · lore

Re: Description of github.com/git/git, was Re: [PATCH] builtin-commit: re-read file index before launching editor

On Tue, Nov 09, 2021 at 11:03:45AM -0800, Junio C Hamano wrote:
Show 17 quoted lines
> Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
>
> > I would be in favor of changing this description. How about:
> >
> > 	Git Source Code Mirror - Pull Requests can be submitted via GitGitGadget
>
> Presense of GGG does not change the fact that the repository is
> publish only.  But "all pull requests are ignored" is not correct
> anymore.
>
>     Git Source Code Mirror - This is a publish-only repository but
>     pull requests can be turned into patches to the mailing list via
>     GitGitGadget. Please follow Documentation/SubmittingPatches
>     procedure for any of your improvements.
>
> I didn't double-check but I presume SubmittingPatches these days
> talk about GGG?

I don't think so. Documentation/MyFirstContribution.txt does as of 76644e3268 (documentation: add tutorial for first contribution, 2019-05-17), but the same treatment was not applied elsewhere.

I would not be sad to see a reference to GGG be made in SubmittingPatches.

Thanks, Taylor

Samuel Yvon· Nov 9, 2021, 19:27 UTC · re: Junio C Hamano · lore

Re: Description of github.com/git/git, was Re: [PATCH] builtin-commit: re-read file index before launching editor

Junio C Hamano <gitster@pobox.com> writes:
> I didn't double-check but I presume SubmittingPatches these days
> talk about GGG?

It does not, but it links to Documentation/MyFirstContribution, which does.

Junio C Hamano <gitster@pobox.com> writes:
>     Git Source Code Mirror - This is a publish-only repository but
>     pull requests can be turned into patches to the mailing list via
>     GitGitGadget. Please follow Documentation/SubmittingPatches
>     procedure for any of your improvements.
I think this is descriptive enough so that newcomers can find their way.
Johannes Schindelin· Nov 10, 2021, 12:22 UTC · re: Samuel Yvon · lore

Re: Description of github.com/git/git, was Re: [PATCH] builtin-commit: re-read file index before launching editor

Hi,
On Tue, 9 Nov 2021, Samuel Yvon wrote:
Show 8 quoted lines
> Junio C Hamano <gitster@pobox.com> writes:
>
> >     Git Source Code Mirror - This is a publish-only repository but
> >     pull requests can be turned into patches to the mailing list via
> >     GitGitGadget. Please follow Documentation/SubmittingPatches
> >     procedure for any of your improvements.
>
> I think this is descriptive enough so that newcomers can find their way.

Excellent. Junio had made the adjustment, and I merely added a link to GitGitGadget's high-level description.

Thank you, all, Dscho

Samuel Yvon via GitGitGadget· Nov 11, 2021, 17:55 UTC · re: Samuel Yvon via GitGitGadget · lore

[PATCH v2] builtin-commit: re-read file index before run_status

From: Samuel Yvon <samuelyvon9@gmail.com>

Changes made within a pre-commit hook are not reflected within the editor (for instance, with `git commit --verbose`) because the index is re-read after the status has been written.

This has the consequence of not displaying the actual changes made by the pre-commit hook, but committing them anyways.

While it is often argued that the hook's purpose is not to automatically write content to the repository, it is acknowledged that using the pre-commit to apply mechanical fixes on top of the existing changes is a supported use case, along with verifying the content.

I think not seeing these mechanical fixes before commiting is incorrect. A developer might expect the commit to look a certain way but if the pre-commit goes wrong the invalid changes will go unnoticed until committed.

The comment suggesting that the cache must be reset after run_status and before the editor being launched was added in ec84bd00, (git-commit: Refactor creation of log message., 2008-02-05). It is unclear why the run_status must be called *after* the cache reset. However, calling run_status after the cache reset does not update the status line to state of the current index in the case a pre-commit hook is ran and changes files in the staging area.

Signed-off-by: Samuel Yvon <samuelyvon9@gmail.com>
---
    builtin-commit: re-read file index before run_status
    
    Changes made within a pre-commit hook are not reflected within the
    editor (for instance, with git commit --verbose) because the index is
    re-read after the editor has been used.
    
    This has the consequence of not displaying the actual changes made by
    the pre-commit hook, but committing them anyways.
    
    While it is often argued that the hook's purpose is not to automatically
    write content to the repository, it is acknowledged that using the
    pre-commit to apply mechanical fixes on top of the existing changes is a
    supported use case, along with verifying the content.
    
    I think not seeing these mechanical fixes before commiting is incorrect.
    A developer might expect the commit to look a certain way but if the
    pre-commit goes wrong the invalid changes will go unnoticed until
    committed.
    
    I had a small exchange in the Google Group before submitting this Pr.
    Here is a link for cross-referencing:
    https://groups.google.com/g/git-mentoring/c/FsP83I9mN6c
    
    As a side note, I do not know who manages the Github Repo but the
    following description threw me off a little bit:
    
    Git Source Code Mirror - This is a publish-only repository and all pull requests are ignored. 
    
    
    since after looking deeper it seems the PRs are not ignored.
    
    Changes since v1:
    
     * Edited the title to more accurately reflect the changes
     * More details in commit messages
    
    Signed-off-by: Samuel Yvon samuelyvon9@gmail.com
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-git-1127%2FSamuelYvon%2Fmaint-v2
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-git-1127/SamuelYvon/maint-v2
Pull-Request: https://github.com/git/git/pull/1127
Range-diff vs v1:
 1:  271bcf89d1e ! 1:  f09244a0ba3 builtin-commit: re-read file index before launching editor
     @@ Metadata
      Author: Samuel Yvon <samuelyvon9@gmail.com>
      
       ## Commit message ##
     -    builtin-commit: re-read file index before launching editor
     +    builtin-commit: re-read file index before run_status
      
          Changes made within a pre-commit hook are not reflected within
          the editor (for instance, with `git commit --verbose`) because
     -    the index is re-read after the editor has been used.
     +    the index is re-read after the status has been written.
      
          This has the consequence of not displaying the actual changes made
          by the pre-commit hook, but committing them anyways.
     @@ Commit message
          pre-commit goes wrong the invalid changes will go unnoticed until
          committed.
      
     +    The comment suggesting that the cache must be reset after run_status
     +    and before the editor being launched was added in ec84bd00,
     +    (git-commit: Refactor creation of log message., 2008-02-05). It is
     +    unclear why the run_status must be called *after* the cache reset.
     +    However, calling run_status after the cache reset does not update
     +    the status line to state of the current index in the case a
     +    pre-commit hook is ran and changes files in the staging area.
     +
          Signed-off-by: Samuel Yvon <samuelyvon9@gmail.com>
      
       ## builtin/commit.c ##
 builtin/commit.c | 21 +++++++++++----------
 1 file changed, 11 insertions(+), 10 deletions(-)
Show changes to builtin/commit.c +11 −10
diff --git a/builtin/commit.c b/builtin/commit.c
index 7c9b1e7be3a..e75b11d1c60 100644
--- a/builtin/commit.c
+++ b/builtin/commit.c
@@ -728,8 +728,17 @@ static int prepare_to_commit(const char *index_file, const char *prefix,
 	/* This checks and barfs if author is badly specified */
 	determine_author_info(author_ident);
 
-	if (!no_verify && run_commit_hook(use_editor, index_file, "pre-commit", NULL))
-		return 0;
+	if (!no_verify && find_hook("pre-commit")) {
+		if(run_commit_hook(use_editor, index_file, "pre-commit", NULL))
+			return 0;
+
+		/*
+		 * Re-read the index as pre-commit hook could have updated it,
+		 * and write it out as a tree.
+		 */
+		discard_cache();
+		read_cache_from(index_file);
+	}
 
 	if (squash_message) {
 		/*
@@ -1051,14 +1060,6 @@ static int prepare_to_commit(const char *index_file, const char *prefix,
 		return 0;
 	}
 
-	if (!no_verify && find_hook("pre-commit")) {
-		/*
-		 * Re-read the index as pre-commit hook could have updated it,
-		 * and write it out as a tree.  We must do this before we invoke
-		 * the editor and after we invoke run_status above.
-		 */
-		discard_cache();
-	}
 	read_cache_from(index_file);
 
 	if (update_main_cache_tree(0)) {

base-commit: 5fbd2fc5997dfa4d4593a862fe729b1e7a89bcf8
-- 
gitgitgadget
Junio C Hamano· Nov 12, 2021, 23:23 UTC · re: Samuel Yvon via GitGitGadget · lore

Re: [PATCH v2] builtin-commit: re-read file index before run_status

"Samuel Yvon via GitGitGadget" <gitgitgadget@gmail.com> writes:
> The comment suggesting that the cache must be reset after run_status
> and before the editor being launched was added in ec84bd00,
> (git-commit: Refactor creation of log message., 2008-02-05). It is
> unclear why the run_status must be called *after* the cache reset.

An older thread cited earlier suspected that it is to reflect the changes given to "git commit" proper, excluding whatever pre-commit did, to the return value of run_status(), which becomes the value of committable variable to answer "do we have anything to commit?".

And moving the call would affect both the contents of the status buffer (i.e. the list of paths got changed starts including what pre-commit did) and the "committable" bit by counting such a change as a true change, avoiding the "no empty commit by default" check, in a consistent way, hopefully. I wonder if we have test to demonstrate that, and if there isn't perhaps we would want to add one.

> However, calling run_status after the cache reset does not update
> the status line to state of the current index in the case a
> pre-commit hook is ran and changes files in the staging area.

And if this change also affects the "committable" assignment in a consistent way, it should probably want to be mentioned in this paragraph, too.

I am not convinced by the claim that there is no need for careful transition plans (yet), but I personally agree with the end state (with the above suggested tweaks, that is).

Thanks for working on the topic.
Samuel Yvon· Nov 17, 2021, 16:48 UTC · re: Junio C Hamano · lore

Re: [PATCH v2] builtin-commit: re-read file index before run_status

Apologies for the time I took to reply,
Junio C Hamano <gitster@pobox.com> writes:
Show 7 quoted lines
> And moving the call would affect both the contents of the status
> buffer (i.e. the list of paths got changed starts including what
>         pre-commit did) and the "committable" bit by counting such a change
> as a true change, avoiding the "no empty commit by default" check,
> in a consistent way, hopefully.  I wonder if we have test to
> demonstrate that, and if there isn't perhaps we would want to add
> one.

So, just to make sure I understand well, the concern is that an empty commit would trigger the commit routine, run the pre-commit hook, which may add files (thus making it an non-empty commit) and then push 100% automatic changes to a repo. I agree that this would be invalid behaviour and very odd, I will make sure no empty commit is allowed.

Junio C Hamano <gitster@pobox.com> writes:
Show 8 quoted lines
> Samuel Yvon <samuelyvon9@gmail.com> writes:
> > However, calling run_status after the cache reset does not update
> > the status line to state of the current index in the case a
> > pre-commit hook is ran and changes files in the staging area.
> 
> And if this change also affects the "committable" assignment in a
> consistent way, it should probably want to be mentioned in this
> paragraph, too.
What do you mean by "commitable assignment"? 
> I am not convinced by the claim that there is no need for careful
> transition plans (yet), but I personally agree with the end state
> (with the above suggested tweaks, that is).

With the last message, I agree the safest option is probably to leave this configurable for now and off by default.

So here's the next steps that I intend to take to get this merged in:
- Add a test for empty commit (if non-existent) and ensure the behaviour is the same
- Add a config option (or maybe a switch?) for migration purposes, with the default
  being the current behaviour.
Junio C Hamano· Nov 18, 2021, 23:51 UTC · re: Samuel Yvon · lore

Re: [PATCH v2] builtin-commit: re-read file index before run_status

Samuel Yvon <samuelyvon9@gmail.com> writes:
Show 5 quoted lines
>> And if this change also affects the "committable" assignment in a
>> consistent way, it should probably want to be mentioned in this
>> paragraph, too.
>
> What do you mean by "commitable assignment"? 

Assignment to the committable variable in the codepath you are looking at, was what I meant.

← back to recent threads