git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH v3 0/2] pack-bitmap.c: avoid exposing absolute paths

From
Jeff King <peff@peff.net>
Date
Nov 14, 2022, 22:23 UTC
Message-ID
<Y3K//kO3fxD7Pl3/@coredump.intra.peff.net>
In-Reply-To
<Y27MH2LtfVzKEM65@nand.local>
On Fri, Nov 11, 2022 at 05:26:39PM -0500, Taylor Blau wrote:
Show 11 quoted lines
> On Thu, Nov 10, 2022 at 03:10:10PM +0800, Teng Long wrote:
> > From: Teng Long <dyroneteng@gmail.com>
> >
> > Diff since v2:
> >
> > * remove unnecessary comments.
> > * use "GIT_TRACE2_EVENT" instead of "GIT_TRACE_PERF".
> > * improve commit message of [1/2].
> 
> Thanks, this version looks great. I'm very satisfied with where it ended
> up, and I'm feeling comfortable with merging it down.
Me too. If we wanted to go further, there are two obvious next steps.
One, we can break out of the bitmap loop early if we're not tracing:
diff --git a/pack-bitmap.c b/pack-bitmap.c
index aaa2d9a104..3b6c2f804a 100644
--- a/pack-bitmap.c
+++ b/pack-bitmap.c
@@ -527,8 +527,15 @@ static int open_pack_bitmap(struct repository *r,
 	assert(!bitmap_git->map);
 
 	for (p = get_all_packs(r); p; p = p->next) {
-		if (open_pack_bitmap_1(bitmap_git, p) == 0)
+		if (open_pack_bitmap_1(bitmap_git, p) == 0) {
 			ret = 0;
+			/*
+			 * The only reason to keep looking is to report
+			 * duplicates.
+			 */
+			if (!trace2_is_enabled())
+				break;
+		}
 	}
 
 	return ret;

I doubt this buys us much in practice. After patch 2, looking for extra
bitmaps is much cheaper. It's one open() call per pack (which will
return ENOENT normally) looking for a bitmap. And while it's only 2
lines of code, it does increase coupling of assumptions between the two
functions. So maybe not worth doing. I dunno.

And two, we could complain when there is both a midx and a pack bitmap,
like so:

diff --git a/pack-bitmap.c b/pack-bitmap.c
index 3b6c2f804a..44a80ed8f2 100644
--- a/pack-bitmap.c
+++ b/pack-bitmap.c
@@ -524,8 +524,6 @@ static int open_pack_bitmap(struct repository *r,
 	struct packed_git *p;
 	int ret = -1;
 
-	assert(!bitmap_git->map);
-
 	for (p = get_all_packs(r); p; p = p->next) {
 		if (open_pack_bitmap_1(bitmap_git, p) == 0) {
 			ret = 0;
@@ -559,11 +557,20 @@ static int open_midx_bitmap(struct repository *r,
 static int open_bitmap(struct repository *r,
 		       struct bitmap_index *bitmap_git)
 {
+	int found = 0;
+
 	assert(!bitmap_git->map);
 
-	if (!open_midx_bitmap(r, bitmap_git))
-		return 0;
-	return open_pack_bitmap(r, bitmap_git);
+	found = !open_midx_bitmap(r, bitmap_git);
+
+	/*
+	 * these will all be skipped if we opened a midx bitmap; but run it
+	 * anyway if tracing is enabled to report the duplicates
+	 */
+	if (!found || trace2_is_enabled())
+		found |= !open_pack_bitmap(r, bitmap_git);
+
+	return found ? 0 : -1;
 }
 
 struct bitmap_index *prepare_bitmap_git(struct repository *r)

But I'm not sure if that is even something we want. I know we retained
pack bitmaps as a quick recovery mechanism while test-deploying midx
bitmaps. OTOH, now that the feature is stable, I doubt anybody else
would ever do that.

It also suffers from the same coupling issues as the other.

So I don't know that either is worth pursuing (hence this message and
not fully prepared patches), but these are just the two leftover things
I noticed from the series, so I wanted to record them for posterity.

-Peff
Previous: Taylor BlauNext: Teng Long
Message 39 of 67 in “pack-bitmap.c: avoid exposing absolute paths”
  1. 0/1 pack-bitmap.c: avoid exposing absolute pathsTeng Long, Aug 26, 2022
  2. 1/1 pack-bitmap.c: avoid exposing absolute pathsTeng Long, Aug 26, 2022
  3. Junio C HamanoAug 26, 2022
  4. Teng LongAug 29, 2022
  5. Taylor BlauOct 26, 2022
  6. Ævar Arnfjörð BjarmasonOct 26, 2022
  7. Teng LongOct 31, 2022
  8. Jeff KingOct 27, 2022
  9. Taylor BlauOct 30, 2022
  10. Teng LongOct 31, 2022
  11. Teng LongNov 2, 2022
  12. Jeff KingNov 2, 2022
  13. Teng LongNov 2, 2022
  14. Teng LongOct 31, 2022
  15. Taylor BlauNov 3, 2022
  16. Ævar Arnfjörð BjarmasonNov 2, 2022
  17. Teng LongNov 2, 2022
  18. 0/1 pack-bitmap.c: avoid exposing absolute pathsTeng Long, Nov 2, 2022
  19. 1/1 pack-bitmap.c: avoid exposing absolute pathsTeng Long, Nov 2, 2022
  20. Taylor BlauNov 3, 2022
  21. Teng LongNov 3, 2022
  22. Taylor BlauNov 5, 2022
  23. Taylor BlauNov 3, 2022
  24. Teng LongNov 3, 2022
  25. 0/2 pack-bitmap.c: avoid exposing absolute pathsTeng Long, Nov 4, 2022
  26. 1/2 pack-bitmap.c: avoid exposing absolute pathsTeng Long, Nov 4, 2022
  27. Taylor BlauNov 4, 2022
  28. 2/2 pack-bitmap.c: remove unnecessary "open_pack_index()" callsTeng Long, Nov 4, 2022
  29. Taylor BlauNov 4, 2022
  30. Taylor BlauNov 4, 2022
  31. 0/2 pack-bitmap.c: avoid exposing absolute pathsTeng Long, Nov 10, 2022
  32. 1/2 pack-bitmap.c: remove unnecessary "open_pack_index()" callsTeng Long, Nov 10, 2022
  33. Jeff KingNov 14, 2022
  34. Taylor BlauNov 14, 2022
  35. Jeff KingNov 14, 2022
  36. Taylor BlauNov 14, 2022
  37. 2/2 pack-bitmap.c: avoid exposing absolute pathsTeng Long, Nov 10, 2022
  38. Taylor BlauNov 11, 2022
  39. Jeff KingNov 14, 2022
  40. Teng LongNov 17, 2022
  41. Jeff KingNov 17, 2022
  42. Taylor BlauNov 17, 2022
  43. Teng LongNov 21, 2022
  44. 0/4 pack-bitmap.c: avoid exposing absolute pathsTeng Long, Nov 21, 2022
  45. 1/4 pack-bitmap.c: remove unnecessary "open_pack_index()" callsTeng Long, Nov 21, 2022
  46. 2/4 pack-bitmap.c: avoid exposing absolute pathsTeng Long, Nov 21, 2022
  47. 3/4 pack-bitmap.c: break out of the bitmap loop early if not tracingTeng Long, Nov 21, 2022
  48. Junio C HamanoNov 21, 2022
  49. Teng LongNov 28, 2022
  50. 4/4 pack-bitmap.c: trace bitmap ignore logs when midx-bitmap is foundTeng Long, Nov 21, 2022
  51. Jeff KingNov 21, 2022
  52. Junio C HamanoNov 21, 2022
  53. Teng LongNov 28, 2022
  54. Teng LongNov 28, 2022
  55. Jeff KingNov 29, 2022
  56. Teng LongNov 29, 2022
  57. Jeff KingNov 21, 2022
  58. Teng LongNov 28, 2022
  59. 0/4 pack-bitmap.c: avoid exposing absolute pathsTeng Long, Nov 28, 2022
  60. 1/4 pack-bitmap.c: remove unnecessary "open_pack_index()" callsTeng Long, Nov 28, 2022
  61. 2/4 pack-bitmap.c: avoid exposing absolute pathsTeng Long, Nov 28, 2022
  62. 3/4 pack-bitmap.c: break out of the bitmap loop early if not tracingTeng Long, Nov 28, 2022
  63. Taylor BlauNov 28, 2022
  64. Teng LongNov 29, 2022
  65. 4/4 pack-bitmap.c: trace bitmap ignore logs when midx-bitmap is foundTeng Long, Nov 28, 2022
  66. Taylor BlauNov 28, 2022
  67. Teng LongNov 29, 2022

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.