git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH 9/9] prepare_packed_git_one: refactor duplicate-pack check

From
Jeff King <peff@peff.net>
Date
Jun 30, 2014, 17:04 UTC
Message-ID
<20140630170403.GI16637@sigill.intra.peff.net>
In-Reply-To
<20140630165526.GA15690@sigill.intra.peff.net>

When we are reloading the list of packs, we check whether a particular pack has been loaded. This is slightly tricky, because we load packs based on the presence of their ".idx" files, but record the name of the matching ".pack" file. Therefore we want to compare their bases.

The existing code stripped off ".idx" from a file we found, then compared that whole base length to strings containing the ".pack" version. This meant we could end up comparing bytes past what the ".pack" string contained, if the ".idx" file name was much longer.

In practice, it worked OK because memcmp would end up seeing a difference in the two strings and would return early before hitting the full length. However, memcmp may sometimes read extra bytes past a difference (e.g., because it is comparing 64-bit words), or is even free to compare in reverse order.

Furthermore, our memcmp made no guarantees that we matched the whole pack name, up to ".pack". So "foo.idx" would match "foo-bar.pack", which is wrong (but does not typically happen, because our pack names have a fixed size).

We can fix both issues, avoid magic numbers, and document that we expect to compare against a string with ".pack" by using strip_suffix.

Signed-off-by: Jeff King <peff@peff.net>
---
This is a teeny bit less efficient than it could be, because we are
verifying our assumption at run-time that each pack name ends in
".pack". I'd venture to say if we cared about efficiency here, the low
hanging fruit would be to avoid the O(n^2) loop to find duplicate pack
names in the first place.
 sha1_file.c | 9 +++++++--
 1 file changed, 7 insertions(+), 2 deletions(-)
diff --git a/sha1_file.c b/sha1_file.c
index 93b794f..129a4c5 100644
--- a/sha1_file.c
+++ b/sha1_file.c
@@ -1197,6 +1197,7 @@ static void prepare_packed_git_one(char *objdir, int local)
 	dirnamelen = path.len;
 	while ((de = readdir(dir)) != NULL) {
 		struct packed_git *p;
+		size_t base_len;
 
 		if (is_dot_or_dotdot(de->d_name))
 			continue;
@@ -1204,10 +1205,14 @@ static void prepare_packed_git_one(char *objdir, int local)
 		strbuf_setlen(&path, dirnamelen);
 		strbuf_addstr(&path, de->d_name);
 
-		if (ends_with(de->d_name, ".idx")) {
+		base_len = path.len;
+		if (strip_suffix_mem(path.buf, &base_len, ".idx")) {
 			/* Don't reopen a pack we already have. */
 			for (p = packed_git; p; p = p->next) {
-				if (!memcmp(path.buf, p->pack_name, path.len - 4))
+				size_t len;
+				if (strip_suffix(p->pack_name, ".pack", &len) &&
+				    len == base_len &&
+				    !memcmp(p->pack_name, path.buf, len))
 					break;
 			}
 			if (p == NULL &&
-- 
2.0.0.566.gfe3e6b2
Previous: Jeff KingNext: Junio C Hamano
Message 25 of 27 in “sha1_file: replace PATH_MAX buffer wirh strbuf in prepare_packed_git_one()”
  1. 1/2 sha1_file: replace PATH_MAX buffer wirh strbuf in prepare_packed_git_one()René Scharfe, Jun 28, 2014
  2. 2/4 sha1_file: use strncmp for string comparisonRené Scharfe, Jun 28, 2014
  3. Duy NguyenJun 29, 2014
  4. 1/2 sha1_file: replace PATH_MAX buffer wirh strbuf in, prepare_packed_git_one()René Scharfe, Jun 29, 2014
  5. 2/2 sha1_file: use strncmp for string comparisonRené Scharfe, Jun 29, 2014
  6. Jeff KingJun 30, 2014
  7. Duy NguyenJun 30, 2014
  8. Jeff KingJun 30, 2014
  9. René ScharfeJun 30, 2014
  10. Jeff KingJun 30, 2014
  11. René ScharfeJun 30, 2014
  12. Jeff KingJun 30, 2014
  13. 0/9 add strip_suffix as an alternative to ends_withJeff King, Jun 30, 2014
  14. 1/9 sha1_file: replace PATH_MAX buffer with strbuf in prepare_packed_git_one()Jeff King, Jun 30, 2014
  15. 2/9 add strip_suffix functionJeff King, Jun 30, 2014
  16. Junio C HamanoJul 2, 2014
  17. Jeff KingJul 2, 2014
  18. Junio C HamanoJul 2, 2014
  19. 3/9 implement ends_with via strip_suffixJeff King, Jun 30, 2014
  20. 4/9 replace has_extension with ends_withJeff King, Jun 30, 2014
  21. 5/9 use strip_suffix instead of ends_with in simple casesJeff King, Jun 30, 2014
  22. 6/9 index-pack: use strip_suffix to avoid magic numbersJeff King, Jun 30, 2014
  23. 7/9 strbuf: implement strbuf_strip_suffixJeff King, Jun 30, 2014
  24. 8/9 verify-pack: use strbuf_strip_suffixJeff King, Jun 30, 2014
  25. 9/9 prepare_packed_git_one: refactor duplicate-pack checkJeff King, Jun 30, 2014
  26. Junio C HamanoJul 1, 2014
  27. Jeff KingJun 30, 2014

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.