git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH v2] index-pack: always zero-initialize object_entry list

From
Jeff King <peff@peff.net>
Date
Mar 19, 2013, 16:17 UTC
Message-ID
<20130319161722.GA17445@sigill.intra.peff.net>
In-Reply-To
<20130319155244.GA16532@sigill.intra.peff.net>
On Tue, Mar 19, 2013 at 11:52:44AM -0400, Jeff King wrote:
Show 18 quoted lines
> > > > Commit 38a4556 (index-pack: start learning to emulate
> > > > "verify-pack -v", 2011-06-03) added a "delta_depth" counter
> > > > to each "struct object_entry". Initially, all object entries
> > > > have their depth set to 0; in resolve_delta, we then set the
> > > > depth of each delta to "base + 1". Base entries never have
> > > > their depth touched, and remain at 0.
> > > 
> > > This patch causes index-pack to fail on the pack that triggered the
> > > whole discussion.  More in a minute in another side thread, but
> > > meanwhile: NAK until we understand what is really going on here.
> > 
> > Odd; that's what I was testing with, and it worked fine.
> 
> Ah, interesting. I built the fix on top of d1a0ed1, the first commit
> that shows the problem. And it works fine there. But when it is
> forward-ported to the current master, it breaks as you saw.
> 
> More bisection fun.

So after bisecting, I realize that it is indeed broken on top of d1a0ed1. I have no idea why I didn't notice that before; I'm guessing it was because I was running it under valgrind and paying attention only to valgrind errors.

Anyway, the problem is simple and stupid. The original object array is not nr_objects item long; it is (nr_objects + 1) long, though I'm not clear why (1-indexing?). So my previous patch was zeroing the final entry, which was supposed to contain actual data. Oops.

Here's the corrected patch.
-- >8 --
Subject: [PATCH] index-pack: always zero-initialize object_entry list

Commit 38a4556 (index-pack: start learning to emulate "verify-pack -v", 2011-06-03) added a "delta_depth" counter to each "struct object_entry". Initially, all object entries have their depth set to 0; in resolve_delta, we then set the depth of each delta to "base + 1". Base entries never have their depth touched, and remain at 0.

To ensure that all depths start at 0, that commit changed calls to xmalloc the object_entry list into calls to xcalloc. However, it forgot that we grow the list with xrealloc later. These extra entries are used when we add an object from elsewhere pack to complete a thin pack. If we add a non-delta object, its depth value will just be uninitialized heap data.

This patch fixes it by zero-initializing entries we add to the objects list via the xrealloc.

Signed-off-by: Jeff King <peff@peff.net>
---
 builtin/index-pack.c | 2 ++
 1 file changed, 2 insertions(+)
diff --git a/builtin/index-pack.c b/builtin/index-pack.c
index 43d364b..5860085 100644
--- a/builtin/index-pack.c
+++ b/builtin/index-pack.c
@@ -1107,6 +1107,8 @@ static void conclude_pack(int fix_thin_pack, const char *curr_pack, unsigned cha
 		objects = xrealloc(objects,
 				   (nr_objects + nr_unresolved + 1)
 				   * sizeof(*objects));
+		memset(objects + nr_objects + 1, 0,
+		       nr_unresolved * sizeof(*objects));
 		f = sha1fd(output_fd, curr_pack);
 		fix_unresolved_deltas(f, nr_unresolved);
 		sprintf(msg, _("completed with %d local objects"),
-- 
1.8.2.22.g4863f63
Previous: Jeff KingNext: Thomas Rast
Message 17 of 46 in “regression in multi-threaded git-pack-index”
  1. Stefan ZagerMar 15, 2013
  2. Jeff KingMar 16, 2013
  3. Duy NguyenMar 16, 2013
  4. Thomas RastMar 19, 2013
  5. Jeff KingMar 19, 2013
  6. Jeff KingMar 19, 2013
  7. Jeff KingMar 19, 2013
  8. Jeff KingMar 19, 2013
  9. Thomas RastMar 19, 2013
  10. Jeff KingMar 19, 2013
  11. Thomas RastMar 19, 2013
  12. Jeff KingMar 19, 2013
  13. index-pack: always zero-initialize object_entry listJeff King, Mar 19, 2013
  14. Thomas RastMar 19, 2013
  15. Jeff KingMar 19, 2013
  16. Jeff KingMar 19, 2013
  17. index-pack: always zero-initialize object_entry listJeff King, Mar 19, 2013
  18. Thomas RastMar 19, 2013
  19. Junio C HamanoMar 19, 2013
  20. Eric SunshineMar 20, 2013
  21. Jeff KingMar 20, 2013
  22. Eric SunshineMar 20, 2013
  23. Duy NguyenMar 19, 2013
  24. index-pack: protect deepest_delta in multithread codeNguyễn Thái Ngọc Duy, Mar 19, 2013
  25. Jeff KingMar 19, 2013
  26. Thomas RastMar 19, 2013
  27. Duy NguyenMar 19, 2013
  28. index-pack: guard nr_resolved_deltas reads by lockThomas Rast, Mar 19, 2013
  29. Junio C HamanoMar 19, 2013
  30. Thomas RastMar 19, 2013
  31. Thomas RastMar 19, 2013
  32. Thomas RastMar 19, 2013
  33. Junio C HamanoMar 19, 2013
  34. sha1_file: remove recursion in packed_object_infoThomas Rast, Mar 19, 2013
  35. Junio C HamanoMar 20, 2013
  36. thomasMar 25, 2013
  37. 0/3 Recursion-free unpack_entry and packed_object_infoThomas Rast, Mar 25, 2013
  38. 1/3 sha1_file: remove recursion in packed_object_infoThomas Rast, Mar 25, 2013
  39. 2/3 Refactor parts of in_delta_base_cache/cache_or_unpack_entryThomas Rast, Mar 25, 2013
  40. Junio C HamanoMar 25, 2013
  41. thomasMar 26, 2013
  42. 3/3 sha1_file: remove recursion in unpack_entryThomas Rast, Mar 25, 2013
  43. Junio C HamanoMar 25, 2013
  44. Nicolas PitreMar 26, 2013
  45. Junio C HamanoMar 25, 2013
  46. Duy NguyenMar 20, 2013

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.