git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH 08/68] add reentrant variants of sha1_to_hex and find_unique_abbrev

From
Jeff King <peff@peff.net>
Date
Sep 24, 2015, 21:05 UTC
Message-ID
<20150924210545.GE30946@sigill.intra.peff.net>
In-Reply-To
<20150924210225.GA23624@sigill.intra.peff.net>

The sha1_to_hex and find_unique_abbrev functions always write into reusable static buffers. There are a few problems with this:

  - future calls overwrite our result. This is especially
    annoying with find_unique_abbrev, which does not have a
    ring of buffers, so you cannot even printf() a result
    that has two abbreviated sha1s.
  - if you want to put the result into another buffer, we
    often strcpy, which looks suspicious when auditing for
    overflows.

This patch introduces sha1_to_hex_r and find_unique_abbrev_r, which write into a user-provided buffer. Of course this is just punting on the overflow-auditing, as the buffer obviously needs to be GIT_SHA1_HEXSZ + 1 bytes. But it is much easier to audit, since that is a well-known size.

We retain the non-reentrant forms, which just become thin wrappers around the reentrant ones. This patch also adds a strbuf variant of find_unique_abbrev, which will be handy in later patches.

Signed-off-by: Jeff King <peff@peff.net>
---
 cache.h     | 31 ++++++++++++++++++++++++++++++-
 hex.c       | 13 +++++++++----
 sha1_name.c | 16 +++++++++++-----
 strbuf.c    |  9 +++++++++
 strbuf.h    |  8 ++++++++
 5 files changed, 67 insertions(+), 10 deletions(-)
diff --git a/cache.h b/cache.h
index e231e47..030b880 100644
--- a/cache.h
+++ b/cache.h
@@ -785,7 +785,24 @@ extern char *sha1_pack_name(const unsigned char *sha1);
  */
 extern char *sha1_pack_index_name(const unsigned char *sha1);
 
-extern const char *find_unique_abbrev(const unsigned char *sha1, int);
+/*
+ * Return an abbreviated sha1 unique within this repository's object database.
+ * The result will be at least `len` characters long, and will be NUL
+ * terminated.
+ *
+ * The non-`_r` version returns a static buffer which will be overwritten by
+ * subsequent calls.
+ *
+ * The `_r` variant writes to a buffer supplied by the caller, which must be at
+ * least `GIT_SHA1_HEXSZ + 1` bytes. The return value is the number of bytes
+ * written (excluding the NUL terminator).
+ *
+ * Note that while this version avoids the static buffer, it is not fully
+ * reentrant, as it calls into other non-reentrant git code.
+ */
+extern const char *find_unique_abbrev(const unsigned char *sha1, int len);
+extern int find_unique_abbrev_r(char *hex, const unsigned char *sha1, int len);
+
 extern const unsigned char null_sha1[GIT_SHA1_RAWSZ];
 
 static inline int hashcmp(const unsigned char *sha1, const unsigned char *sha2)
@@ -1067,6 +1084,18 @@ extern int for_each_abbrev(const char *prefix, each_abbrev_fn, void *);
 extern int get_sha1_hex(const char *hex, unsigned char *sha1);
 extern int get_oid_hex(const char *hex, struct object_id *sha1);
 
+/*
+ * Convert a binary sha1 to its hex equivalent. The `_r` variant is reentrant,
+ * and writes the NUL-terminated output to the buffer `out`, which must be at
+ * least `GIT_SHA1_HEXSZ + 1` bytes, and returns a pointer to out for
+ * convenience.
+ *
+ * The non-`_r` variant returns a static buffer, but uses a ring of 4
+ * buffers, making it safe to make multiple calls for a single statement, like:
+ *
+ *   printf("%s -> %s", sha1_to_hex(one), sha1_to_hex(two));
+ */
+extern char *sha1_to_hex_r(char *out, const unsigned char *sha1);
 extern char *sha1_to_hex(const unsigned char *sha1);	/* static buffer result! */
 extern char *oid_to_hex(const struct object_id *oid);	/* same static buffer as sha1_to_hex */
 
diff --git a/hex.c b/hex.c
index 899b74a..0519f85 100644
--- a/hex.c
+++ b/hex.c
@@ -61,12 +61,10 @@ int get_oid_hex(const char *hex, struct object_id *oid)
 	return get_sha1_hex(hex, oid->hash);
 }
 
-char *sha1_to_hex(const unsigned char *sha1)
+char *sha1_to_hex_r(char *buffer, const unsigned char *sha1)
 {
-	static int bufno;
-	static char hexbuffer[4][GIT_SHA1_HEXSZ + 1];
 	static const char hex[] = "0123456789abcdef";
-	char *buffer = hexbuffer[3 & ++bufno], *buf = buffer;
+	char *buf = buffer;
 	int i;
 
 	for (i = 0; i < GIT_SHA1_RAWSZ; i++) {
@@ -79,6 +77,13 @@ char *sha1_to_hex(const unsigned char *sha1)
 	return buffer;
 }
 
+char *sha1_to_hex(const unsigned char *sha1)
+{
+	static int bufno;
+	static char hexbuffer[4][GIT_SHA1_HEXSZ + 1];
+	return sha1_to_hex_r(hexbuffer[3 & ++bufno], sha1);
+}
+
 char *oid_to_hex(const struct object_id *oid)
 {
 	return sha1_to_hex(oid->hash);
diff --git a/sha1_name.c b/sha1_name.c
index da6874c..c58b477 100644
--- a/sha1_name.c
+++ b/sha1_name.c
@@ -368,14 +368,13 @@ int for_each_abbrev(const char *prefix, each_abbrev_fn fn, void *cb_data)
 	return ds.ambiguous;
 }
 
-const char *find_unique_abbrev(const unsigned char *sha1, int len)
+int find_unique_abbrev_r(char *hex, const unsigned char *sha1, int len)
 {
 	int status, exists;
-	static char hex[41];
 
-	memcpy(hex, sha1_to_hex(sha1), 40);
+	sha1_to_hex_r(hex, sha1);
 	if (len == 40 || !len)
-		return hex;
+		return 40;
 	exists = has_sha1_file(sha1);
 	while (len < 40) {
 		unsigned char sha1_ret[20];
@@ -384,10 +383,17 @@ const char *find_unique_abbrev(const unsigned char *sha1, int len)
 		    ? !status
 		    : status == SHORT_NAME_NOT_FOUND) {
 			hex[len] = 0;
-			return hex;
+			return len;
 		}
 		len++;
 	}
+	return len;
+}
+
+const char *find_unique_abbrev(const unsigned char *sha1, int len)
+{
+	static char hex[GIT_SHA1_HEXSZ + 1];
+	find_unique_abbrev_r(hex, sha1, len);
 	return hex;
 }
 
diff --git a/strbuf.c b/strbuf.c
index 29df55b..f3c44fb 100644
--- a/strbuf.c
+++ b/strbuf.c
@@ -743,3 +743,12 @@ void strbuf_addftime(struct strbuf *sb, const char *fmt, const struct tm *tm)
 	}
 	strbuf_setlen(sb, sb->len + len);
 }
+
+void strbuf_add_unique_abbrev(struct strbuf *sb, const unsigned char *sha1,
+			      int abbrev_len)
+{
+	int r;
+	strbuf_grow(sb, GIT_SHA1_HEXSZ + 1);
+	r = find_unique_abbrev_r(sb->buf + sb->len, sha1, abbrev_len);
+	strbuf_setlen(sb, sb->len + r);
+}
diff --git a/strbuf.h b/strbuf.h
index 43f27c3..0f9c8a7 100644
--- a/strbuf.h
+++ b/strbuf.h
@@ -475,6 +475,14 @@ static inline struct strbuf **strbuf_split(const struct strbuf *sb,
 extern void strbuf_list_free(struct strbuf **);
 
 /**
+ * Add the abbreviation, as generated by find_unique_abbrev, of `sha1` to
+ * the strbuf `sb`.
+ */
+extern void strbuf_add_unique_abbrev(struct strbuf *sb,
+				     const unsigned char *sha1,
+				     int abbrev_len);
+
+/**
  * Launch the user preferred editor to edit a file and fill the buffer
  * with the file's contents upon the user completing their editing. The
  * third argument can be used to set the environment which the editor is
-- 
2.6.0.rc3.454.g204ad51
Previous: Jeff KingNext: Jeff King
Message 9 of 93 in “war on sprintf”
  1. 0/68 war on sprintfJeff King, Sep 24, 2015
  2. 01/68 show-branch: avoid segfault with --reflog of unborn branchJeff King, Sep 24, 2015
  3. 02/68 mailsplit: fix FILE* leak in split_maildirJeff King, Sep 24, 2015
  4. 03/68 archive-tar: fix minor indentation violationJeff King, Sep 24, 2015
  5. 04/68 fsck: don't fsck alternates for connectivity-only checkJeff King, Sep 24, 2015
  6. 05/68 add xsnprintf helper functionJeff King, Sep 24, 2015
  7. 06/68 add git_path_buf helper functionJeff King, Sep 24, 2015
  8. 07/68 strbuf: make strbuf_complete_line more genericJeff King, Sep 24, 2015
  9. 08/68 add reentrant variants of sha1_to_hex and find_unique_abbrevJeff King, Sep 24, 2015
  10. 09/68 fsck: use strbuf to generate alternate directoriesJeff King, Sep 24, 2015
  11. 10/68 mailsplit: make PATH_MAX buffers dynamicJeff King, Sep 24, 2015
  12. 11/68 trace: use strbuf for quote_crnl outputJeff King, Sep 24, 2015
  13. 12/68 progress: store throughput display in a strbufJeff King, Sep 24, 2015
  14. 13/68 test-dump-cache-tree: avoid overflow of cache-tree nameJeff King, Sep 24, 2015
  15. 14/68 compat/inet_ntop: fix off-by-one in inet_ntop4Jeff King, Sep 24, 2015
  16. 15/68 convert trivial sprintf / strcpy calls to xsnprintfJeff King, Sep 24, 2015
  17. 16/68 archive-tar: use xsnprintf for trivial formattingJeff King, Sep 24, 2015
  18. 17/68 use xsnprintf for generating git object headersJeff King, Sep 24, 2015
  19. 18/68 find_short_object_filename: convert sprintf to xsnprintfJeff King, Sep 24, 2015
  20. 19/68 stop_progress_msg: convert sprintf to xsnprintfJeff King, Sep 24, 2015
  21. 20/68 compat/hstrerror: convert sprintf to snprintfJeff King, Sep 24, 2015
  22. 21/68 grep: use xsnprintf to format failure messageJeff King, Sep 24, 2015
  23. 22/68 entry.c: convert strcpy to xsnprintfJeff King, Sep 24, 2015
  24. 23/68 add_packed_git: convert strcpy into xsnprintfJeff King, Sep 24, 2015
  25. 24/68 http-push: replace strcat with xsnprintfJeff King, Sep 24, 2015
  26. 25/68 receive-pack: convert strncpy to xsnprintfJeff King, Sep 24, 2015
  27. 26/68 replace trivial malloc + sprintf / strcpy calls with xstrfmtJeff King, Sep 24, 2015
  28. 27/68 config: use xstrfmt in normalize_valueJeff King, Sep 24, 2015
  29. 28/68 fetch: replace static buffer with xstrfmtJeff King, Sep 24, 2015
  30. 29/68 use strip_suffix and xstrfmt to replace suffixJeff King, Sep 24, 2015
  31. 30/68 ref-filter: drop sprintf and strcpy callsJeff King, Sep 24, 2015
  32. 31/68 help: drop prepend function in favor of xstrfmtJeff King, Sep 24, 2015
  33. 32/68 mailmap: replace strcpy with xstrdupJeff King, Sep 24, 2015
  34. 33/68 read_branches_file: simplify string handlingJeff King, Sep 24, 2015
  35. 34/68 read_remotes_file: simplify string handlingJeff King, Sep 24, 2015
  36. 35/68 resolve_ref: use strbufs for internal buffersJeff King, Sep 24, 2015
  37. 36/68 upload-archive: convert sprintf to strbufJeff King, Sep 24, 2015
  38. 37/68 remote-ext: simplify git pkt-line generationJeff King, Sep 24, 2015
  39. 38/68 http-push: use strbuf instead of fwrite_bufferJeff King, Sep 24, 2015
  40. 39/68 http-walker: store url in a strbufJeff King, Sep 24, 2015
  41. 40/68 sha1_get_pack_name: use a strbufJeff King, Sep 24, 2015
  42. 41/68 init: use strbufs to store pathsJeff King, Sep 24, 2015
  43. Michael BlumeSep 29, 2015
  44. Jeff KingSep 30, 2015
  45. Junio C HamanoSep 30, 2015
  46. Jeff KingOct 1, 2015
  47. Torsten BögershausenOct 2, 2015
  48. Jeff KingOct 2, 2015
  49. Torsten BögershausenOct 3, 2015
  50. Junio C HamanoOct 3, 2015
  51. Torsten BögershausenOct 3, 2015
  52. Jeff KingOct 4, 2015
  53. Torsten BögershausenOct 4, 2015
  54. Jeff KingOct 5, 2015
  55. 1/3 precompose_utf8: drop unused variableJeff King, Oct 5, 2015
  56. Torsten BögershausenOct 6, 2015
  57. 2/3 probe_utf8_pathname_composition: use internal strbufJeff King, Oct 5, 2015
  58. 3/3 init: use strbufs to store pathsJeff King, Oct 5, 2015
  59. 42/68 apply: convert root string to strbufJeff King, Sep 24, 2015
  60. 43/68 transport: use strbufs for status table "quickref" stringsJeff King, Sep 24, 2015
  61. 44/68 merge-recursive: convert malloc / strcpy to strbufJeff King, Sep 24, 2015
  62. 45/68 enter_repo: convert fixed-size buffers to strbufsJeff King, Sep 24, 2015
  63. 46/68 remove_leading_path: use a strbuf for internal storageJeff King, Sep 24, 2015
  64. 47/68 write_loose_object: convert to strbufJeff King, Sep 24, 2015
  65. 48/68 diagnose_invalid_index_path: use strbuf to avoid strcpy/strcatJeff King, Sep 24, 2015
  66. 49/68 fetch-pack: use argv_array for index-pack / unpack-objectsJeff King, Sep 24, 2015
  67. 50/68 http-push: use an argv_array for setup_revisionsJeff King, Sep 24, 2015
  68. 51/68 stat_tracking_info: convert to argv_arrayJeff King, Sep 24, 2015
  69. 52/68 daemon: use cld->env_array when re-spawningJeff King, Sep 24, 2015
  70. 53/68 use sha1_to_hex_r() instead of strcpyJeff King, Sep 24, 2015
  71. 54/68 drop strcpy in favor of raw sha1_to_hexJeff King, Sep 24, 2015
  72. Eric SunshineSep 24, 2015
  73. Jeff KingSep 25, 2015
  74. 55/68 color: add overflow checks for parsing colorsJeff King, Sep 24, 2015
  75. 56/68 use alloc_ref rather than hand-allocating "struct ref"Jeff King, Sep 24, 2015
  76. 57/68 avoid sprintf and strcpy with flex arraysJeff King, Sep 24, 2015
  77. 58/68 receive-pack: simplify keep_arg computationJeff King, Sep 24, 2015
  78. 59/68 help: clean up kfmclient mungingJeff King, Sep 24, 2015
  79. 60/68 prefer memcpy to strcpyJeff King, Sep 24, 2015
  80. René ScharfeSep 27, 2015
  81. Torsten BögershausenSep 27, 2015
  82. René ScharfeSep 27, 2015
  83. René ScharfeSep 27, 2015
  84. Rasmus VillemoesSep 28, 2015
  85. 61/68 color: add color_set helper for copying raw colorsJeff King, Sep 24, 2015
  86. 62/68 notes: document length of fanout path with a constantJeff King, Sep 24, 2015
  87. 63/68 convert strncpy to memcpyJeff King, Sep 24, 2015
  88. 64/68 fsck: drop inode-sorting codeJeff King, Sep 24, 2015
  89. 65/68 Makefile: drop D_INO_IN_DIRENT build knobJeff King, Sep 24, 2015
  90. 66/68 fsck: use for_each_loose_file_in_objdirJeff King, Sep 24, 2015
  91. Jeff KingSep 26, 2015
  92. 67/68 use strbuf_complete to conditionally append slashJeff King, Sep 24, 2015
  93. 68/68 name-rev: use strip_suffix to avoid magic numbersJeff King, Sep 24, 2015

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.