git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH v3 0/2] pack-bitmap.c: avoid exposing absolute paths

From
Teng Long <dyroneteng@gmail.com>
Date
Nov 10, 2022, 07:10 UTC
Message-ID
<cover.1668063122.git.dyroneteng@gmail.com>
In-Reply-To
<cover.1667470481.git.dyroneteng@gmail.com>
From: Teng Long <dyroneteng@gmail.com>
Diff since v2:
* remove unnecessary comments.
* use "GIT_TRACE2_EVENT" instead of "GIT_TRACE_PERF".
* improve commit message of [1/2].
Thanks.
Teng Long (2):
  pack-bitmap.c: remove unnecessary "open_pack_index()" calls
  pack-bitmap.c: avoid exposing absolute paths
 pack-bitmap.c           | 13 ++++++-------
 t/t5310-pack-bitmaps.sh |  5 +++--
 2 files changed, 9 insertions(+), 9 deletions(-)
Range-diff against v2:
2:  7ac9b859f3 ! 1:  aaeb021538 pack-bitmap.c: remove unnecessary "open_pack_index()" calls
    @@ Metadata
      ## Commit message ##
         pack-bitmap.c: remove unnecessary "open_pack_index()" calls
     
    -    Everytime when calling "open_pack_bitmap_1()", we will firstly call
    -    "open_pack_index(packfile)" to check the index, then further check
    -    again in "is_pack_valid()" before mmap the bitmap file. So, let's
    -    remove the first check here.
    +    When trying to open a pack bitmap, we call open_pack_bitmap_1() in a
    +    loop, during which it tries to open up the pack index corresponding
    +    with each available pack.
     
    -    The relate discussion:
    -    https://public-inbox.org/git/Y2IiSU1L+bJPUioV@coredump.intra.peff.net/#t
    +    It's likely that we'll end up relying on objects in that pack later
    +    in the process (in which case we're doing the work of opening the
    +    pack index optimistically), but not guaranteed.
    +
    +    For instance, consider a repository with a large number of small
    +    packs, and one large pack with a bitmap. If we see that bitmap pack
    +    last in our loop which calls open_pack_bitmap_1(), the current code
    +    will have opened *all* pack index files in the repository. If the
    +    request can be served out of the bitmapped pack alone, then the time
    +    spent opening these idx files was wasted.S
    +
    +    Since open_pack_bitmap_1() calls is_pack_valid() later on (which in
    +    turns calls open_pack_index() itself), we can just drop the earlier
    +    call altogether.
     
         Signed-off-by: Teng Long <dyroneteng@gmail.com>
     
1:  de941f58f9 ! 2:  9d5a491887 pack-bitmap.c: avoid exposing absolute paths
    @@ pack-bitmap.c: static int open_midx_bitmap_1(struct bitmap_index *bitmap_git,
      		get_midx_filename(&buf, midx->object_dir);
     -		/* ignore extra bitmap file; we can only handle one */
     -		warning(_("ignoring extra bitmap file: '%s'"), buf.buf);
    -+		/* ignore extra midx bitmap files; we can only handle one */
     +		trace2_data_string("bitmap", the_repository,
     +				   "ignoring extra midx bitmap file", buf.buf);
      		close(fd);
    @@ pack-bitmap.c: static int open_pack_bitmap_1(struct bitmap_index *bitmap_git, st
      	if (bitmap_git->pack || bitmap_git->midx) {
     -		/* ignore extra bitmap file; we can only handle one */
     -		warning(_("ignoring extra bitmap file: '%s'"), packfile->pack_name);
    -+		/* ignore extra bitmap files; we can only handle one */
     +		trace2_data_string("bitmap", the_repository,
     +				   "ignoring extra bitmap file", packfile->pack_name);
      		close(fd);
    @@ t/t5310-pack-bitmaps.sh: test_bitmap_cases () {
      
     -			git rev-list --use-bitmap-index HEAD 2>err &&
     -			grep "ignoring extra bitmap file" err
    -+			GIT_TRACE2_PERF=$(pwd)/trace2.txt git rev-list --use-bitmap-index HEAD &&
    ++			GIT_TRACE2_EVENT=$(pwd)/trace2.txt git rev-list --use-bitmap-index HEAD &&
     +			grep "opened bitmap" trace2.txt &&
     +			grep "ignoring extra bitmap" trace2.txt
      		)
-- 
2.38.1.383.g7ac9b859f31.dirty
Previous: Taylor BlauNext: Teng Long
Message 31 of 67 in “pack-bitmap.c: avoid exposing absolute paths”
  1. 0/1 pack-bitmap.c: avoid exposing absolute pathsTeng Long, Aug 26, 2022
  2. 1/1 pack-bitmap.c: avoid exposing absolute pathsTeng Long, Aug 26, 2022
  3. Junio C HamanoAug 26, 2022
  4. Teng LongAug 29, 2022
  5. Taylor BlauOct 26, 2022
  6. Ævar Arnfjörð BjarmasonOct 26, 2022
  7. Teng LongOct 31, 2022
  8. Jeff KingOct 27, 2022
  9. Taylor BlauOct 30, 2022
  10. Teng LongOct 31, 2022
  11. Teng LongNov 2, 2022
  12. Jeff KingNov 2, 2022
  13. Teng LongNov 2, 2022
  14. Teng LongOct 31, 2022
  15. Taylor BlauNov 3, 2022
  16. Ævar Arnfjörð BjarmasonNov 2, 2022
  17. Teng LongNov 2, 2022
  18. 0/1 pack-bitmap.c: avoid exposing absolute pathsTeng Long, Nov 2, 2022
  19. 1/1 pack-bitmap.c: avoid exposing absolute pathsTeng Long, Nov 2, 2022
  20. Taylor BlauNov 3, 2022
  21. Teng LongNov 3, 2022
  22. Taylor BlauNov 5, 2022
  23. Taylor BlauNov 3, 2022
  24. Teng LongNov 3, 2022
  25. 0/2 pack-bitmap.c: avoid exposing absolute pathsTeng Long, Nov 4, 2022
  26. 1/2 pack-bitmap.c: avoid exposing absolute pathsTeng Long, Nov 4, 2022
  27. Taylor BlauNov 4, 2022
  28. 2/2 pack-bitmap.c: remove unnecessary "open_pack_index()" callsTeng Long, Nov 4, 2022
  29. Taylor BlauNov 4, 2022
  30. Taylor BlauNov 4, 2022
  31. 0/2 pack-bitmap.c: avoid exposing absolute pathsTeng Long, Nov 10, 2022
  32. 1/2 pack-bitmap.c: remove unnecessary "open_pack_index()" callsTeng Long, Nov 10, 2022
  33. Jeff KingNov 14, 2022
  34. Taylor BlauNov 14, 2022
  35. Jeff KingNov 14, 2022
  36. Taylor BlauNov 14, 2022
  37. 2/2 pack-bitmap.c: avoid exposing absolute pathsTeng Long, Nov 10, 2022
  38. Taylor BlauNov 11, 2022
  39. Jeff KingNov 14, 2022
  40. Teng LongNov 17, 2022
  41. Jeff KingNov 17, 2022
  42. Taylor BlauNov 17, 2022
  43. Teng LongNov 21, 2022
  44. 0/4 pack-bitmap.c: avoid exposing absolute pathsTeng Long, Nov 21, 2022
  45. 1/4 pack-bitmap.c: remove unnecessary "open_pack_index()" callsTeng Long, Nov 21, 2022
  46. 2/4 pack-bitmap.c: avoid exposing absolute pathsTeng Long, Nov 21, 2022
  47. 3/4 pack-bitmap.c: break out of the bitmap loop early if not tracingTeng Long, Nov 21, 2022
  48. Junio C HamanoNov 21, 2022
  49. Teng LongNov 28, 2022
  50. 4/4 pack-bitmap.c: trace bitmap ignore logs when midx-bitmap is foundTeng Long, Nov 21, 2022
  51. Jeff KingNov 21, 2022
  52. Junio C HamanoNov 21, 2022
  53. Teng LongNov 28, 2022
  54. Teng LongNov 28, 2022
  55. Jeff KingNov 29, 2022
  56. Teng LongNov 29, 2022
  57. Jeff KingNov 21, 2022
  58. Teng LongNov 28, 2022
  59. 0/4 pack-bitmap.c: avoid exposing absolute pathsTeng Long, Nov 28, 2022
  60. 1/4 pack-bitmap.c: remove unnecessary "open_pack_index()" callsTeng Long, Nov 28, 2022
  61. 2/4 pack-bitmap.c: avoid exposing absolute pathsTeng Long, Nov 28, 2022
  62. 3/4 pack-bitmap.c: break out of the bitmap loop early if not tracingTeng Long, Nov 28, 2022
  63. Taylor BlauNov 28, 2022
  64. Teng LongNov 29, 2022
  65. 4/4 pack-bitmap.c: trace bitmap ignore logs when midx-bitmap is foundTeng Long, Nov 28, 2022
  66. Taylor BlauNov 28, 2022
  67. Teng LongNov 29, 2022

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.