git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH 1/1] vreportf(): avoid buffered write in favor of unbuffered one

From
Johannes Schindelin <johannes.schindelin@gmx.de>
Date
Oct 29, 2019, 12:38 UTC
Message-ID
<nycvar.QRO.7.76.6.1910291334210.46@tvgsbejvaqbjf.bet>
In-Reply-To
<20191029103842.GV4348@szeder.dev>
Hi Gábor,
On Tue, 29 Oct 2019, SZEDER Gábor wrote:
Show 15 quoted lines
> On Mon, Oct 28, 2019 at 03:00:59PM +0000, Johannes Schindelin via GitGitGadget wrote:
> > From: Johannes Schindelin <johannes.schindelin@gmx.de>
> >
> > The MSVC runtime behavior differs from glibc's with respect to
> > `fprintf(stderr, ...)` in that the former writes out the message
> > character by character.
> >
> > In t5516, this leads to a funny problem where a `git fetch` process as
> > well as the `git upload-pack` process spawned by it _both_ call `die()`
> > at the same time. The output can look like this:
> >
> > 	fatal: git uploadfata-lp: raemcokte :error:  upload-pnot our arcef k6: n4ot our ea4cr1e3f 36d45ea94fca1398e86a771eda009872d63adb28598f6a9
> > 	8e86a771eda009872d6ab2886
>
> Heh.
Yes ;-)
Show 7 quoted lines
> > Let's avoid this predicament altogether by rendering the entire message,
> > including the prefix and the trailing newline, into the buffer we
> > already have (and which is still fixed size) and then write it out via
> > `write_in_full()`.
>
> s/write_in_full/xwrite/ perhaps?  Both the cover letter and the patch
> below use xwrite().

Excellent eyes! I had originally used `write_in_full()` before realizing that `xwrite()` would be more appropriate.

Fixed.

Thank you for helping me improve the patch! Dscho

Show 61 quoted lines
> > The history of `vreportf()` with regard to this issue includes the
> > following commits:
> >
> > d048a96e (2007-11-09) - 'char msg[256]' is introduced to avoid interleaving
> > 389d1767 (2009-03-25) - Buffer size increased to 1024 to avoid truncation
> > 625a860c (2009-11-22) - Buffer size increased to 4096 to avoid truncation
> > f4c3edc0 (2015-08-11) - Buffer removed to avoid truncation
> > b5a9e435 (2017-01-11) - Reverts f4c3edc0 to be able to replace control
> >                         chars before sending to stderr
> > 9ac13ec9 (2006-10-11) - Another attempt to solve interleaving.
> >                         This is seemingly related to d048a96e.
> > 137a0d0e (2007-11-19) - Addresses out-of-order for display()
> > 34df8aba (2009-03-10) - Switches xwrite() to fprintf() in recv_sideband()
> >                         to support UTF-8 emulation
> > eac14f89 (2012-01-14) - Removes the need for fprintf() for UTF-8 emulation,
> >                         so it's safe to use xwrite() again
> > 5e5be9e2 (2016-06-28) - recv_sideband() uses xwrite() again
> >
> > Note that we need to be careful to handle the return value of
> > `vsnprintf()` that indicates the _desired_ byte count.
> >
> > Also please note that we `fflush(stderr)` here to help when running in a
> > Git Bash on Windows: in this case, `stderr` is not actually truly
> > unbuffered, and needs the extra help.
> >
> > Co-authored-by: Alexandr Miloslavskiy <alexandr.miloslavskiy@syntevo.com>
> > Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
> > ---
> >  usage.c | 12 +++++++++---
> >  1 file changed, 9 insertions(+), 3 deletions(-)
> >
> > diff --git a/usage.c b/usage.c
> > index 2fdb20086b..4328894dce 100644
> > --- a/usage.c
> > +++ b/usage.c
> > @@ -10,13 +10,19 @@ void vreportf(const char *prefix, const char *err, va_list params)
> >  {
> >  	char msg[4096];
> >  	char *p;
> > -
> > -	vsnprintf(msg, sizeof(msg), err, params);
> > +	size_t off = strlcpy(msg, prefix, sizeof(msg));
> > +	int ret = vsnprintf(msg + off, sizeof(msg) - off, err, params);
> >  	for (p = msg; *p; p++) {
> >  		if (iscntrl(*p) && *p != '\t' && *p != '\n')
> >  			*p = '?';
> >  	}
> > -	fprintf(stderr, "%s%s\n", prefix, msg);
> > +	if (ret > 0) {
> > +		if (off + ret > sizeof(msg) - 1)
> > +			ret = sizeof(msg) - 1 - off;
> > +		msg[off + ret] = '\n'; /* we no longer need a NUL */
> > +		fflush(stderr);
> > +		xwrite(2, msg, off + ret + 1);
> > +	}
> >  }
> >
> >  static NORETURN void usage_builtin(const char *err, va_list params)
> > --
> > gitgitgadget
>
Previous: SZEDER GáborNext: Jeff King
Message 12 of 34 in “Fix t5516 flakiness in Visual Studio builds”
  1. 0/1 Fix t5516 flakiness in Visual Studio buildsJohannes Schindelin via GitGitGadget, Oct 28, 2019
  2. 1/1 vreportf(): avoid buffered write in favor of unbuffered oneJohannes Schindelin via GitGitGadget, Oct 28, 2019
  3. Junio C HamanoOct 29, 2019
  4. Johannes SchindelinOct 29, 2019
  5. Jeff KingOct 29, 2019
  6. Johannes SchindelinOct 29, 2019
  7. Jeff KingOct 29, 2019
  8. Johannes SchindelinOct 29, 2019
  9. Junio C HamanoOct 30, 2019
  10. Junio C HamanoOct 29, 2019
  11. SZEDER GáborOct 29, 2019
  12. Johannes SchindelinOct 29, 2019
  13. Jeff KingOct 29, 2019
  14. Johannes SchindelinOct 29, 2019
  15. 0/1 Fix t5516 flakiness in Visual Studio buildsJohannes Schindelin via GitGitGadget, Oct 29, 2019
  16. 1/1 vreportf(): avoid relying on stdio bufferingJohannes Schindelin via GitGitGadget, Oct 29, 2019
  17. Alexandr MiloslavskiyOct 29, 2019
  18. Johannes SchindelinOct 29, 2019
  19. Jeff KingOct 29, 2019
  20. Alexandr MiloslavskiyOct 29, 2019
  21. Alexandr MiloslavskiyOct 29, 2019
  22. 0/1 Fix t5516 flakiness in Visual Studio buildsJohannes Schindelin via GitGitGadget, Oct 29, 2019
  23. 1/1 vreportf(): avoid relying on stdio bufferingJohannes Schindelin via GitGitGadget, Oct 29, 2019
  24. Jeff KingOct 29, 2019
  25. Johannes SchindelinOct 30, 2019
  26. Jeff KingOct 31, 2019
  27. Johannes SchindelinOct 31, 2019
  28. Jeff KingOct 31, 2019
  29. Johannes SchindelinNov 1, 2019
  30. Junio C HamanoOct 30, 2019
  31. Johannes SchindelinOct 30, 2019
  32. 0/1 Fix t5516 flakiness in Visual Studio buildsJohannes Schindelin via GitGitGadget, Oct 30, 2019
  33. 1/1 vreportf(): avoid relying on stdio bufferingJohannes Schindelin via GitGitGadget, Oct 30, 2019
  34. Junio C HamanoNov 2, 2019

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.