git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH v12 18/26] stash: convert push to builtin

From
Johannes Schindelin <johannes.schindelin@gmx.de>
Date
Feb 19, 2019, 10:47 UTC
Message-ID
<nycvar.QRO.7.76.6.1902191127420.41@tvgsbejvaqbjf.bet>
In-Reply-To
<20190219002336.GN1622@szeder.dev>
Hi Gábor & Thomas,
On Tue, 19 Feb 2019, SZEDER Gábor wrote:
Show 8 quoted lines
> On Tue, Feb 12, 2019 at 11:18:37PM +0000, Thomas Gummerer wrote:
> > Thanks.  I still didn't manage to reproduce it locally, but I was now
> > able to test it on Travis CI.
> > 
> > The diff below fixes the issue, but I still need to spend some time to
> > better understand why it does.
> 
> There is nothing like a fix that works, but you have no idea why :)
I know why. Now. See below for the analysis.
> FWIW, I'm at a couple of thousands of '--stress' repetitions with your
> patch below, and not a single failure yet.
Good, and yes, there is a problem.
Show 17 quoted lines
> > I'll hopefully be in a position to
> > send a patch with a proper log message why this is the right fix in
> > the next couple of days.
> > 
> > diff --git a/builtin/stash--helper.c b/builtin/stash--helper.c
> > index c77f62c895..3dab488bd6 100644
> > --- a/builtin/stash--helper.c
> > +++ b/builtin/stash--helper.c
> > @@ -231,6 +231,7 @@ static int reset_tree(struct object_id *i_tree, int update, int reset)
> >  	struct tree *tree;
> >  	struct lock_file lock_file = LOCK_INIT;
> >  
> > +	discard_cache();
> >  	read_cache_preload(NULL);
> >  	if (refresh_cache(REFRESH_QUIET))
> >  		return -1;
> >

So this is working, but it is not the correct spot for that `discard_cache()`, as it forces unnecessary cycles on code paths calling `reset_tree()` (which corresponds to `git read-tree`, admittedly a bit confusing) with a fully up to date index.

The real fix, I believe, is this:
-- snip --
diff --git a/builtin/stash.c b/builtin/stash.c
index 2d6dfce883..516dee0fa4 100644
--- a/builtin/stash.c
+++ b/builtin/stash.c
@@ -1372,6 +1372,7 @@ static int do_push_stash(struct pathspec ps, const char *stash_msg, int quiet,
 			}
 		} else {
 			struct child_process cp = CHILD_PROCESS_INIT;
+			discard_cache();
 			cp.git_cmd = 1;
 			argv_array_pushl(&cp.args, "reset", "--hard", "-q",
 					 NULL);
-- snap --

And the reason this is needed: we spawn a `git reset --hard` here, which
will change the index, but outside of the current process. So the
in-process copy is stale. And when the index' mtime does not help us
detect that, we run into that test breakage.

Now, I seriously believe that we missed the best time to move
ps/stash-in-c into `next` for cooking. The best time would have been just
after Paul submitted the latest patch series: we know for a fact that he
is too busy to really take care of this patch series, so keeping it in
`pu` puts everybody into that awkward spot where nobody wants to step on
Paul's toes messing with his patch series, but where Paul also lacks the
time to push it further, so everything is stuck in a limbo and is *so very
much* not cooking at all. You might say that it has turned bad because we
failed to stoke the fire appropriately.

Since it is now way too late in the v2.21.0 process, this problem is only
exacerbated, because it won't even enter `next` "better late than never".

To address this unfortunate situation, my current plan is to take over
from Paul (we had been chatting about this privately in the past, and he
is okay with this because of University eating all his time).

I will open the whole bag again, most likely squashing the late fixups
into the patches that introduced the problems, re-review with a much finer
comb than the patch series has enjoyed on the Git mailing list (even just
a quick look at `do_apply_stash()` revealed an unnecessary `reset_tree()`
call that *no* reviewer spotted, even I myself, but then, I am hardly
solely responsible for that review), and most likely I'll even take my
sweet little time changing the code to avoid more spawned Git processes.

It will take a long time, and the `stash` project that has been discussed
recently to be given to GSoC students is no longer available, as I will
take care of it before GSoC even starts, and I won't spend much time
reviewing other people's code in the meantime.

I will start that only after v2.21.0 final is out, obviously.

Once I submit a new iteration, it will look quite a bit different from
before, and reviewers will have to re-review *everything*, wasting
everybody's time even more. It will have to be re-reviewed in its entirety
anyway because it has been *such* a long time since the latest review, and
that's just the price we all have to pay for missing the right moment to
advance this to `next`. Thomas, I will ask you to review, and Gábor, I
will expect you to review that iteration, too, as you are now a bit
familiar with the code, and I will really need your help here.

Anyway, that's my plan for now.

Ciao,
Dscho
Previous: SZEDER GáborNext: Junio C Hamano
Message 34 of 106 in “Convert "git stash" to C builtin”
  1. 00/26 Convert "git stash" to C builtinPaul-Sebastian Ungureanu, Dec 20, 2018
  2. 01/26 sha1-name.c: add `get_oidf()` which acts like `get_oid()`Paul-Sebastian Ungureanu, Dec 20, 2018
  3. 02/26 strbuf.c: add `strbuf_join_argv()`Paul-Sebastian Ungureanu, Dec 20, 2018
  4. 03/26 strbuf.c: add `strbuf_insertf()` and `strbuf_vinsertf()`Paul-Sebastian Ungureanu, Dec 20, 2018
  5. 05/26 stash: improve option parsing test coveragePaul-Sebastian Ungureanu, Dec 20, 2018
  6. 04/26 ident: add the ability to provide a "fallback identity"Paul-Sebastian Ungureanu, Dec 20, 2018
  7. Junio C HamanoDec 26, 2018
  8. Johannes SchindelinDec 27, 2018
  9. Junio C HamanoDec 28, 2018
  10. 06/26 t3903: modernize stylePaul-Sebastian Ungureanu, Dec 20, 2018
  11. 08/26 stash: add tests for `git stash show` configPaul-Sebastian Ungureanu, Dec 20, 2018
  12. 09/26 stash: mention options in `show` synopsisPaul-Sebastian Ungureanu, Dec 20, 2018
  13. 07/26 stash: rename test cases to be more descriptivePaul-Sebastian Ungureanu, Dec 20, 2018
  14. 10/26 stash: convert apply to builtinPaul-Sebastian Ungureanu, Dec 20, 2018
  15. 12/26 stash: convert branch to builtinPaul-Sebastian Ungureanu, Dec 20, 2018
  16. 15/26 stash: convert show to builtinPaul-Sebastian Ungureanu, Dec 20, 2018
  17. 16/26 stash: convert store to builtinPaul-Sebastian Ungureanu, Dec 20, 2018
  18. 14/26 stash: convert list to builtinPaul-Sebastian Ungureanu, Dec 20, 2018
  19. 13/26 stash: convert pop to builtinPaul-Sebastian Ungureanu, Dec 20, 2018
  20. 11/26 stash: convert drop and clear to builtinPaul-Sebastian Ungureanu, Dec 20, 2018
  21. 19/26 stash: make push -q quietPaul-Sebastian Ungureanu, Dec 20, 2018
  22. 22/26 stash: replace all `write-tree` child processes with API callsPaul-Sebastian Ungureanu, Dec 20, 2018
  23. 21/26 stash: optimize `get_untracked_files()` and `check_changes()`Paul-Sebastian Ungureanu, Dec 20, 2018
  24. Thomas GummererJan 6, 2019
  25. 26/26 tests: add a special setup where stash.useBuiltin is offPaul-Sebastian Ungureanu, Dec 20, 2018
  26. 24/26 stash: add back the original, scripted `git stash`Paul-Sebastian Ungureanu, Dec 20, 2018
  27. 20/26 stash: convert save to builtinPaul-Sebastian Ungureanu, Dec 20, 2018
  28. 18/26 stash: convert push to builtinPaul-Sebastian Ungureanu, Dec 20, 2018
  29. SZEDER GáborFeb 8, 2019
  30. Thomas GummererFeb 10, 2019
  31. SZEDER GáborFeb 11, 2019
  32. Thomas GummererFeb 12, 2019
  33. SZEDER GáborFeb 19, 2019
  34. Johannes SchindelinFeb 19, 2019
  35. Junio C HamanoFeb 19, 2019
  36. Johannes SchindelinFeb 20, 2019
  37. Thomas GummererFeb 19, 2019
  38. Junio C HamanoFeb 20, 2019
  39. Johannes SchindelinFeb 20, 2019
  40. Thomas GummererFeb 20, 2019
  41. 00/27 Convert "git stash" to C builtinThomas Gummerer, Feb 25, 2019
  42. 01/27 sha1-name.c: add `get_oidf()` which acts like `get_oid()`Thomas Gummerer, Feb 25, 2019
  43. 02/27 strbuf.c: add `strbuf_join_argv()`Thomas Gummerer, Feb 25, 2019
  44. 03/27 strbuf.c: add `strbuf_insertf()` and `strbuf_vinsertf()`Thomas Gummerer, Feb 25, 2019
  45. 05/27 stash: improve option parsing test coverageThomas Gummerer, Feb 25, 2019
  46. 04/27 ident: add the ability to provide a "fallback identity"Thomas Gummerer, Feb 25, 2019
  47. 06/27 t3903: modernize styleThomas Gummerer, Feb 25, 2019
  48. 07/27 t3903: add test for --intent-to-add fileThomas Gummerer, Feb 25, 2019
  49. 08/27 stash: rename test cases to be more descriptiveThomas Gummerer, Feb 25, 2019
  50. 09/27 stash: add tests for `git stash show` configThomas Gummerer, Feb 25, 2019
  51. 10/27 stash: mention options in `show` synopsisThomas Gummerer, Feb 25, 2019
  52. 11/27 stash: convert apply to builtinThomas Gummerer, Feb 25, 2019
  53. regression in new built-in stash + fsmonitor (was Re: [PATCH v13 11/27] stash: convert apply to builtin)Ævar Arnfjörð Bjarmason, Mar 14, 2019
  54. Johannes SchindelinMar 14, 2019
  55. Ævar Arnfjörð BjarmasonMar 14, 2019
  56. Johannes SchindelinMar 14, 2019
  57. Ævar Arnfjörð BjarmasonMar 14, 2019
  58. Ben PeartMar 15, 2019
  59. 12/27 stash: convert drop and clear to builtinThomas Gummerer, Feb 25, 2019
  60. Jeff KingMar 7, 2019
  61. Thomas GummererMar 9, 2019
  62. Jeff KingMar 10, 2019
  63. Junio C HamanoMar 11, 2019
  64. Thomas GummererMar 11, 2019
  65. 13/27 stash: convert branch to builtinThomas Gummerer, Feb 25, 2019
  66. 14/27 stash: convert pop to builtinThomas Gummerer, Feb 25, 2019
  67. 15/27 stash: convert list to builtinThomas Gummerer, Feb 25, 2019
  68. 16/27 stash: convert show to builtinThomas Gummerer, Feb 25, 2019
  69. 17/27 stash: convert store to builtinThomas Gummerer, Feb 25, 2019
  70. 18/27 stash: convert create to builtinThomas Gummerer, Feb 25, 2019
  71. Jeff KingMar 7, 2019
  72. Johannes SchindelinMar 8, 2019
  73. Thomas GummererMar 9, 2019
  74. Junio C HamanoMar 11, 2019
  75. Junio C HamanoMar 11, 2019
  76. Thomas GummererMar 11, 2019
  77. stash: pass pathspec as pointerThomas Gummerer, Mar 11, 2019
  78. Junio C HamanoMar 12, 2019
  79. Johannes SchindelinMar 12, 2019
  80. Thomas GummererMar 12, 2019
  81. Junio C HamanoMar 13, 2019
  82. Johannes SchindelinMar 13, 2019
  83. Thomas GummererMar 15, 2019
  84. 19/27 stash: convert push to builtinThomas Gummerer, Feb 25, 2019
  85. 20/27 stash: make push -q quietThomas Gummerer, Feb 25, 2019
  86. 21/27 stash: convert save to builtinThomas Gummerer, Feb 25, 2019
  87. 22/27 stash: optimize `get_untracked_files()` and `check_changes()`Thomas Gummerer, Feb 25, 2019
  88. 23/27 stash: replace all `write-tree` child processes with API callsThomas Gummerer, Feb 25, 2019
  89. 24/27 stash: convert `stash--helper.c` into `stash.c`Thomas Gummerer, Feb 25, 2019
  90. 26/27 stash: optionally use the scripted version againThomas Gummerer, Feb 25, 2019
  91. 25/27 stash: add back the original, scripted `git stash`Thomas Gummerer, Feb 25, 2019
  92. 27/27 tests: add a special setup where stash.useBuiltin is offThomas Gummerer, Feb 25, 2019
  93. Johannes SchindelinFeb 26, 2019
  94. Thomas GummererFeb 26, 2019
  95. Ævar Arnfjörð BjarmasonFeb 26, 2019
  96. Johannes SchindelinFeb 26, 2019
  97. Junio C HamanoMar 3, 2019
  98. Junio C HamanoMar 3, 2019
  99. Thomas GummererMar 3, 2019
  100. 25/26 stash: optionally use the scripted version againPaul-Sebastian Ungureanu, Dec 20, 2018
  101. Thomas GummererJan 6, 2019
  102. 23/26 stash: convert `stash--helper.c` into `stash.c`Paul-Sebastian Ungureanu, Dec 20, 2018
  103. 17/26 stash: convert create to builtinPaul-Sebastian Ungureanu, Dec 20, 2018
  104. Junio C HamanoJan 3, 2019
  105. Johannes SchindelinJan 18, 2019
  106. Junio C HamanoJan 18, 2019

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.