git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH 08/67] add reentrant variants of sha1_to_hex and find_unique_abbrev

From
Ramsay Jones <ramsay@ramsayjones.plus.com>
Date
Sep 15, 2015, 16:55 UTC
Message-ID
<55F84D9B.90004@ramsayjones.plus.com>
In-Reply-To
<20150915152629.GH29753@sigill.intra.peff.net>
On 15/09/15 16:26, Jeff King wrote:
Show 18 quoted lines
> The sha1_to_hex and find_unique_abbrev functions always
> write into reusable static buffers. There are a few problems
> with this:
>
>   - future calls overwrite our result. This is especially
>     annoying with find_unique_abbrev, which does not have a
>     ring of buffers, so you cannot even printf() a result
>     that has two abbreviated sha1s.
>
>   - if you want to put the result into another buffer, we
>     often strcpy, which looks suspicious when auditing for
>     overflows.
>
> This patch introduces sha1_to_hex_to and find_unique_abbrev_to,
> which write into a user-provided buffer. Of course this is
> just punting on the overflow-auditing, as the buffer
> obviously needs to be GIT_SHA1_HEXSZ + 1 bytes. But it is
> much easier to audit, since that is a well-known size.

Hmm, I haven't read any other patches yet (including those which use these new '_to' functions), but I can't help feeling they should be named something like 'sha1_to_hex_str()' and 'find_unique_abbrev_str()' instead. i.e. I don't get the '_to' thing - not that I'm any good at naming things ...

ATB, Ramsay Jones

Show 177 quoted lines
>
> We retain the non-reentrant forms, which just become thin
> wrappers around the reentrant ones. This patch also adds a
> strbuf variant of find_unique_abbrev, which will be handy in
> later patches.
>
> Signed-off-by: Jeff King <peff@peff.net>
> ---
> If we wanted to be really meticulous, these functions could
> take a size for the output buffer, and complain if it is not
> GIT_SHA1_HEXSZ+1 bytes. But that would bloat every call
> like:
>
>   sha1_to_hex_to(buf, sizeof(buf), sha1);
>
>  cache.h     | 27 ++++++++++++++++++++++++++-
>  hex.c       | 13 +++++++++----
>  sha1_name.c | 16 +++++++++++-----
>  strbuf.c    |  9 +++++++++
>  strbuf.h    |  8 ++++++++
>  5 files changed, 63 insertions(+), 10 deletions(-)
>
> diff --git a/cache.h b/cache.h
> index e231e47..cc59aba 100644
> --- a/cache.h
> +++ b/cache.h
> @@ -785,7 +785,21 @@ extern char *sha1_pack_name(const unsigned char *sha1);
>   */
>  extern char *sha1_pack_index_name(const unsigned char *sha1);
>  
> -extern const char *find_unique_abbrev(const unsigned char *sha1, int);
> +/*
> + * Return an abbreviated sha1 unique within this repository's object database.
> + * The result will be at least `len` characters long, and will be NUL
> + * terminated.
> + *
> + * The non-`_to` version returns a static buffer which will be overwritten by
> + * subsequent calls.
> + *
> + * The `_to` variant writes to a buffer supplied by the caller, which must be
> + * at least `GIT_SHA1_HEXSZ + 1` bytes. The return value is the number of bytes
> + * written (excluding the NUL terminator).
> + */
> +extern const char *find_unique_abbrev(const unsigned char *sha1, int len);
> +extern int find_unique_abbrev_to(char *hex, const unsigned char *sha1, int len);
> +
>  extern const unsigned char null_sha1[GIT_SHA1_RAWSZ];
>  
>  static inline int hashcmp(const unsigned char *sha1, const unsigned char *sha2)
> @@ -1067,6 +1081,17 @@ extern int for_each_abbrev(const char *prefix, each_abbrev_fn, void *);
>  extern int get_sha1_hex(const char *hex, unsigned char *sha1);
>  extern int get_oid_hex(const char *hex, struct object_id *sha1);
>  
> +/*
> + * Convert a binary sha1 to its hex equivalent. The `_to` variant writes
> + * the NUL-terminated output to the buffer `out`, which must be at least
> + * `GIT_SHA1_HEXSZ + 1` bytes, and returns a pointer to out for convenience.
> + *
> + * The non-`_to` variant returns a static buffer, but uses a ring of 4
> + * buffers, making it safe to make multiple calls for a single statement, like:
> + *
> + *   printf("%s -> %s", sha1_to_hex(one), sha1_to_hex(two));
> + */
> +extern char *sha1_to_hex_to(char *out, const unsigned char *sha1);
>  extern char *sha1_to_hex(const unsigned char *sha1);	/* static buffer result! */
>  extern char *oid_to_hex(const struct object_id *oid);	/* same static buffer as sha1_to_hex */
>  
> diff --git a/hex.c b/hex.c
> index 899b74a..004fdea 100644
> --- a/hex.c
> +++ b/hex.c
> @@ -61,12 +61,10 @@ int get_oid_hex(const char *hex, struct object_id *oid)
>  	return get_sha1_hex(hex, oid->hash);
>  }
>  
> -char *sha1_to_hex(const unsigned char *sha1)
> +char *sha1_to_hex_to(char *buffer, const unsigned char *sha1)
>  {
> -	static int bufno;
> -	static char hexbuffer[4][GIT_SHA1_HEXSZ + 1];
>  	static const char hex[] = "0123456789abcdef";
> -	char *buffer = hexbuffer[3 & ++bufno], *buf = buffer;
> +	char *buf = buffer;
>  	int i;
>  
>  	for (i = 0; i < GIT_SHA1_RAWSZ; i++) {
> @@ -79,6 +77,13 @@ char *sha1_to_hex(const unsigned char *sha1)
>  	return buffer;
>  }
>  
> +char *sha1_to_hex(const unsigned char *sha1)
> +{
> +	static int bufno;
> +	static char hexbuffer[4][GIT_SHA1_HEXSZ + 1];
> +	return sha1_to_hex_to(hexbuffer[3 & ++bufno], sha1);
> +}
> +
>  char *oid_to_hex(const struct object_id *oid)
>  {
>  	return sha1_to_hex(oid->hash);
> diff --git a/sha1_name.c b/sha1_name.c
> index da6874c..416e408 100644
> --- a/sha1_name.c
> +++ b/sha1_name.c
> @@ -368,14 +368,13 @@ int for_each_abbrev(const char *prefix, each_abbrev_fn fn, void *cb_data)
>  	return ds.ambiguous;
>  }
>  
> -const char *find_unique_abbrev(const unsigned char *sha1, int len)
> +int find_unique_abbrev_to(char *hex, const unsigned char *sha1, int len)
>  {
>  	int status, exists;
> -	static char hex[41];
>  
> -	memcpy(hex, sha1_to_hex(sha1), 40);
> +	sha1_to_hex_to(hex, sha1);
>  	if (len == 40 || !len)
> -		return hex;
> +		return 40;
>  	exists = has_sha1_file(sha1);
>  	while (len < 40) {
>  		unsigned char sha1_ret[20];
> @@ -384,10 +383,17 @@ const char *find_unique_abbrev(const unsigned char *sha1, int len)
>  		    ? !status
>  		    : status == SHORT_NAME_NOT_FOUND) {
>  			hex[len] = 0;
> -			return hex;
> +			return len;
>  		}
>  		len++;
>  	}
> +	return len;
> +}
> +
> +const char *find_unique_abbrev(const unsigned char *sha1, int len)
> +{
> +	static char hex[GIT_SHA1_HEXSZ + 1];
> +	find_unique_abbrev_to(hex, sha1, len);
>  	return hex;
>  }
>  
> diff --git a/strbuf.c b/strbuf.c
> index 29df55b..6c1b577 100644
> --- a/strbuf.c
> +++ b/strbuf.c
> @@ -743,3 +743,12 @@ void strbuf_addftime(struct strbuf *sb, const char *fmt, const struct tm *tm)
>  	}
>  	strbuf_setlen(sb, sb->len + len);
>  }
> +
> +void strbuf_add_unique_abbrev(struct strbuf *sb, const unsigned char *sha1,
> +			      int abbrev_len)
> +{
> +	int r;
> +	strbuf_grow(sb, GIT_SHA1_HEXSZ + 1);
> +	r = find_unique_abbrev_to(sb->buf + sb->len, sha1, abbrev_len);
> +	strbuf_setlen(sb, sb->len + r);
> +}
> diff --git a/strbuf.h b/strbuf.h
> index ba099cd..9aace36 100644
> --- a/strbuf.h
> +++ b/strbuf.h
> @@ -475,6 +475,14 @@ static inline struct strbuf **strbuf_split(const struct strbuf *sb,
>  extern void strbuf_list_free(struct strbuf **);
>  
>  /**
> + * Add the abbreviation, as generated by find_unique_abbrev, of `sha1` to
> + * the strbuf `sb`.
> + */
> +extern void strbuf_add_unique_abbrev(struct strbuf *sb,
> +				     const unsigned char *sha1,
> +				     int abbrev_len);
> +
> +/**
>   * Launch the user preferred editor to edit a file and fill the buffer
>   * with the file's contents upon the user completing their editing. The
>   * third argument can be used to set the environment which the editor is
Previous: Jeff KingNext: Jeff King
Message 20 of 154 in “war on sprintf, strcpy, etc”
  1. 0/67 war on sprintf, strcpy, etcJeff King, Sep 15, 2015
  2. 01/67 show-branch: avoid segfault with --reflog of unborn branchJeff King, Sep 15, 2015
  3. 02/67 mailsplit: fix FILE* leak in split_maildirJeff King, Sep 15, 2015
  4. 03/67 archive-tar: fix minor indentation violationJeff King, Sep 15, 2015
  5. 04/67 fsck: don't fsck alternates for connectivity-only checkJeff King, Sep 15, 2015
  6. Johannes SchindelinSep 15, 2015
  7. Junio C HamanoSep 16, 2015
  8. Jeff KingSep 16, 2015
  9. Junio C HamanoSep 16, 2015
  10. Eric SunshineSep 16, 2015
  11. Jeff KingSep 16, 2015
  12. 05/67 add xsnprintf helper functionJeff King, Sep 15, 2015
  13. 06/67 add git_path_buf helper functionJeff King, Sep 15, 2015
  14. 07/67 strbuf: make strbuf_complete_line more genericJeff King, Sep 15, 2015
  15. Eric SunshineSep 16, 2015
  16. Junio C HamanoSep 16, 2015
  17. Jeff KingSep 16, 2015
  18. Eric SunshineSep 16, 2015
  19. 08/67 add reentrant variants of sha1_to_hex and find_unique_abbrevJeff King, Sep 15, 2015
  20. Ramsay JonesSep 15, 2015
  21. Jeff KingSep 15, 2015
  22. Junio C HamanoSep 16, 2015
  23. Johannes SchindelinSep 16, 2015
  24. Jeff KingSep 16, 2015
  25. Junio C HamanoSep 16, 2015
  26. Jeff KingSep 16, 2015
  27. 09/67 fsck: use strbuf to generate alternate directoriesJeff King, Sep 15, 2015
  28. 10/67 mailsplit: make PATH_MAX buffers dynamicJeff King, Sep 15, 2015
  29. Eric SunshineSep 16, 2015
  30. Jeff KingSep 16, 2015
  31. Jeff KingSep 16, 2015
  32. Junio C HamanoSep 16, 2015
  33. Jeff KingSep 16, 2015
  34. 11/67 trace: use strbuf for quote_crnl outputJeff King, Sep 15, 2015
  35. Eric SunshineSep 16, 2015
  36. Jeff KingSep 16, 2015
  37. Eric SunshineSep 16, 2015
  38. 12/67 progress: store throughput display in a strbufJeff King, Sep 15, 2015
  39. 13/67 test-dump-cache-tree: avoid overflow of cache-tree nameJeff King, Sep 15, 2015
  40. 14/67 compat/inet_ntop: fix off-by-one in inet_ntop4Jeff King, Sep 15, 2015
  41. 15/67 convert trivial sprintf / strcpy calls to xsnprintfJeff King, Sep 15, 2015
  42. Ramsay JonesSep 15, 2015
  43. Jeff KingSep 15, 2015
  44. Ramsay JonesSep 15, 2015
  45. Stefan BellerSep 15, 2015
  46. Jeff KingSep 16, 2015
  47. Junio C HamanoSep 16, 2015
  48. Junio C HamanoSep 16, 2015
  49. Eric SunshineSep 16, 2015
  50. Jeff KingSep 16, 2015
  51. Junio C HamanoSep 16, 2015
  52. Jeff KingSep 16, 2015
  53. Junio C HamanoSep 16, 2015
  54. Stefan BellerSep 16, 2015
  55. Jeff KingSep 16, 2015
  56. Jeff KingSep 16, 2015
  57. 16/67 archive-tar: use xsnprintf for trivial formattingJeff King, Sep 15, 2015
  58. 17/67 use xsnprintf for generating git object headersJeff King, Sep 15, 2015
  59. Junio C HamanoSep 16, 2015
  60. 18/67 find_short_object_filename: convert sprintf to xsnprintfJeff King, Sep 15, 2015
  61. 19/67 stop_progress_msg: convert sprintf to xsnprintfJeff King, Sep 15, 2015
  62. 20/67 compat/hstrerror: convert sprintf to snprintfJeff King, Sep 15, 2015
  63. 21/67 grep: use xsnprintf to format failure messageJeff King, Sep 15, 2015
  64. 22/67 entry.c: convert strcpy to xsnprintfJeff King, Sep 15, 2015
  65. Ramsay JonesSep 15, 2015
  66. Stefan BellerSep 15, 2015
  67. 23/67 add_packed_git: convert strcpy into xsnprintfJeff King, Sep 15, 2015
  68. Junio C HamanoSep 16, 2015
  69. Jeff KingSep 16, 2015
  70. 24/67 http-push: replace strcat with xsnprintfJeff King, Sep 15, 2015
  71. 25/67 receive-pack: convert strncpy to xsnprintfJeff King, Sep 15, 2015
  72. 26/67 replace trivial malloc + sprintf /strcpy calls to xstrfmtJeff King, Sep 15, 2015
  73. Eric SunshineSep 16, 2015
  74. Jeff KingSep 16, 2015
  75. 27/67 config: use xstrfmt in normalize_valueJeff King, Sep 15, 2015
  76. 28/67 fetch: replace static buffer with xstrfmtJeff King, Sep 15, 2015
  77. 29/67 use strip_suffix and xstrfmt to replace suffixJeff King, Sep 15, 2015
  78. Eric SunshineSep 16, 2015
  79. Jeff KingSep 16, 2015
  80. Eric SunshineSep 16, 2015
  81. 30/67 ref-filter: drop sprintf and strcpy callsJeff King, Sep 15, 2015
  82. Junio C HamanoSep 16, 2015
  83. 31/67 help: drop prepend function in favor of xstrfmtJeff King, Sep 15, 2015
  84. 32/67 mailmap: replace strcpy with xstrdupJeff King, Sep 15, 2015
  85. 33/67 read_branches_file: replace strcpy with xstrdupJeff King, Sep 15, 2015
  86. Junio C HamanoSep 16, 2015
  87. Jeff KingSep 16, 2015
  88. Jeff KingSep 17, 2015
  89. Jeff KingSep 17, 2015
  90. Jeff KingSep 17, 2015
  91. Junio C HamanoSep 17, 2015
  92. Jeff KingSep 17, 2015
  93. Junio C HamanoSep 17, 2015
  94. 34/67 resolve_ref: use strbufs for internal buffersJeff King, Sep 15, 2015
  95. 35/67 upload-archive: convert sprintf to strbufJeff King, Sep 15, 2015
  96. 36/67 remote-ext: simplify git pkt-line generationJeff King, Sep 15, 2015
  97. Junio C HamanoSep 16, 2015
  98. Jeff KingSep 16, 2015
  99. 37/67 http-push: use strbuf instead of fwrite_bufferJeff King, Sep 15, 2015
  100. 38/67 http-walker: store url in a strbufJeff King, Sep 15, 2015
  101. 39/67 sha1_get_pack_name: use a strbufJeff King, Sep 15, 2015
  102. 40/67 init: use strbufs to store pathsJeff King, Sep 15, 2015
  103. 41/67 apply: convert root string to strbufJeff King, Sep 15, 2015
  104. 42/67 transport: use strbufs for status table "quickref" stringsJeff King, Sep 15, 2015
  105. 43/67 merge-recursive: convert malloc / strcpy to strbufJeff King, Sep 15, 2015
  106. 44/67 enter_repo: convert fixed-size buffers to strbufsJeff King, Sep 15, 2015
  107. 45/67 remove_leading_path: use a strbuf for internal storageJeff King, Sep 15, 2015
  108. 46/67 write_loose_object: convert to strbufJeff King, Sep 15, 2015
  109. Junio C HamanoSep 16, 2015
  110. Jeff KingSep 16, 2015
  111. 47/67 diagnose_invalid_index_path: use strbuf to avoid strcpy/strcatJeff King, Sep 15, 2015
  112. 48/67 fetch-pack: use argv_array for index-pack / unpack-objectsJeff King, Sep 15, 2015
  113. 49/67 http-push: use an argv_array for setup_revisionsJeff King, Sep 15, 2015
  114. 50/67 stat_tracking_info: convert to argv_arrayJeff King, Sep 15, 2015
  115. 51/67 daemon: use cld->env_array when re-spawningJeff King, Sep 15, 2015
  116. 52/67 use sha1_to_hex_to() instead of strcpyJeff King, Sep 15, 2015
  117. Junio C HamanoSep 16, 2015
  118. Jeff KingSep 16, 2015
  119. Junio C HamanoSep 16, 2015
  120. 53/67 drop strcpy in favor of raw sha1_to_hexJeff King, Sep 15, 2015
  121. Eric SunshineSep 18, 2015
  122. Jeff KingSep 18, 2015
  123. 54/67 color: add overflow checks for parsing colorsJeff King, Sep 15, 2015
  124. Eric SunshineSep 18, 2015
  125. Jeff KingSep 18, 2015
  126. Junio C HamanoSep 21, 2015
  127. 55/67 use alloc_ref rather than hand-allocating "struct ref"Jeff King, Sep 15, 2015
  128. 56/67 avoid sprintf and strcpy with flex arraysJeff King, Sep 15, 2015
  129. Eric SunshineSep 20, 2015
  130. Jeff KingSep 21, 2015
  131. Eric SunshineSep 21, 2015
  132. Jeff KingSep 21, 2015
  133. 57/67 receive-pack: simplify keep_arg computationJeff King, Sep 15, 2015
  134. Eric SunshineSep 18, 2015
  135. Jeff KingSep 18, 2015
  136. 58/67 help: clean up kfmclient mungingJeff King, Sep 15, 2015
  137. 59/67 prefer memcpy to strcpyJeff King, Sep 15, 2015
  138. 60/67 color: add color_set helper for copying raw colorsJeff King, Sep 15, 2015
  139. 61/67 notes: document length of fanout path with a constantJeff King, Sep 15, 2015
  140. 62/67 convert strncpy to memcpyJeff King, Sep 15, 2015
  141. 63/67 fsck: drop inode-sorting codeJeff King, Sep 15, 2015
  142. 64/67 Makefile: drop D_INO_IN_DIRENT build knobJeff King, Sep 15, 2015
  143. 65/67 fsck: use for_each_loose_file_in_objdirJeff King, Sep 15, 2015
  144. 66/67 use strbuf_complete to conditionally append slashJeff King, Sep 15, 2015
  145. Junio C HamanoSep 16, 2015
  146. Jeff KingSep 16, 2015
  147. Junio C HamanoSep 16, 2015
  148. Jeff KingSep 16, 2015
  149. Junio C HamanoSep 17, 2015
  150. Eric SunshineSep 21, 2015
  151. Jeff KingSep 21, 2015
  152. 67/67 name-rev: use strip_suffix to avoid magic numbersJeff King, Sep 15, 2015
  153. Junio C HamanoSep 16, 2015
  154. Jeff KingSep 16, 2015

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.