git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH 14/17] midx: use chunk-format read API

From
Taylor Blau <me@ttaylorr.com>
Date
Jan 27, 2021, 03:06 UTC
Message-ID
<YBDY0Me2QMcUMD2e@nand.local>
In-Reply-To
<cb145e0e32afed99b9bfa822c76f48bee18885ba.1611676886.git.gitgitgadget@gmail.com>
On Tue, Jan 26, 2021 at 04:01:23PM +0000, Derrick Stolee via GitGitGadget wrote:
Show 28 quoted lines
> From: Derrick Stolee <dstolee@microsoft.com>
>
> Instead of parsing the table of contents directly, use the chunk-format
> API methods read_table_of_contents() and pair_chunk(). In particular, we
> can use the return value of pair_chunk() to generate an error when a
> required chunk is missing.
>
> Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
> ---
>  midx.c                      | 103 ++++++++++++++++++++----------------
>  t/t5319-multi-pack-index.sh |   6 +--
>  2 files changed, 60 insertions(+), 49 deletions(-)
>
> diff --git a/midx.c b/midx.c
> index 0bfd2d802b6..dd019c00795 100644
> --- a/midx.c
> +++ b/midx.c
> @@ -54,6 +54,51 @@ static char *get_midx_filename(const char *object_dir)
>  	return xstrfmt("%s/pack/multi-pack-index", object_dir);
>  }
>
> +static int midx_read_pack_names(const unsigned char *chunk_start,
> +				size_t chunk_size, void *data)
> +{
> +	struct multi_pack_index *m = (struct multi_pack_index *)data;
> +	m->chunk_pack_names = chunk_start;
> +	return 0;
> +}

There are a lot of these callbacks that just assign some 'void **' to point at chunk_start.

Maybe a good use of the "pair_chunk" name would be something like:
    int pair_chunk(struct chunkfile *cf, uint32_t id, const unsigned char **p);

which does the same as what you wrote here. So instead of what you wrote, you could instead:

    pair_chunk(cf, MIDX_CHUNKID_PACKNAMES, &m->chunk_pack_names);

This would be in addition to the richer callback-style function which allows the caller greater flexibility (e.g., for the Bloom filter related readers in the commit-graph code).

Thanks, Taylor

Previous: Derrick Stolee via GitGitGadgetNext: Derrick Stolee
Message 15 of 120 in “Refactor chunk-format into an API”
  1. 00/17 Refactor chunk-format into an APIDerrick Stolee via GitGitGadget, Jan 26, 2021
  2. 02/17 chunk-format: create chunk format write APIDerrick Stolee via GitGitGadget, Jan 26, 2021
  3. Taylor BlauJan 27, 2021
  4. Derrick StoleeJan 27, 2021
  5. 01/17 commit-graph: anonymize data in chunk_write_fnDerrick Stolee via GitGitGadget, Jan 26, 2021
  6. Chris TorekJan 27, 2021
  7. Taylor BlauJan 27, 2021
  8. 05/17 midx: use context in write_midx_pack_names()Derrick Stolee via GitGitGadget, Jan 26, 2021
  9. 06/17 midx: add entries to write_midx_contextDerrick Stolee via GitGitGadget, Jan 26, 2021
  10. 04/17 midx: rename pack_info to write_midx_contextDerrick Stolee via GitGitGadget, Jan 26, 2021
  11. Taylor BlauJan 27, 2021
  12. 03/17 commit-graph: use chunk-format write APIDerrick Stolee via GitGitGadget, Jan 26, 2021
  13. Taylor BlauJan 27, 2021
  14. 14/17 midx: use chunk-format read APIDerrick Stolee via GitGitGadget, Jan 26, 2021
  15. Taylor BlauJan 27, 2021
  16. Derrick StoleeJan 27, 2021
  17. 17/17 chunk-format: add technical docsDerrick Stolee via GitGitGadget, Jan 26, 2021
  18. 11/17 midx: use chunk-format API in write_midx_internal()Derrick Stolee via GitGitGadget, Jan 26, 2021
  19. 16/17 chunk-format: restore duplicate chunk checksDerrick Stolee via GitGitGadget, Jan 26, 2021
  20. 15/17 midx: use 64-bit multiplication for chunk sizesDerrick Stolee via GitGitGadget, Jan 26, 2021
  21. 07/17 midx: add pack_perm to write_midx_contextDerrick Stolee via GitGitGadget, Jan 26, 2021
  22. 10/17 midx: drop chunk progress during writeDerrick Stolee via GitGitGadget, Jan 26, 2021
  23. 09/17 midx: return success/failure in chunk write methodsDerrick Stolee via GitGitGadget, Jan 26, 2021
  24. 08/17 midx: add num_large_offsets to write_midx_contextDerrick Stolee via GitGitGadget, Jan 26, 2021
  25. 13/17 commit-graph: use chunk-format read APIDerrick Stolee via GitGitGadget, Jan 26, 2021
  26. 12/17 chunk-format: create read chunk APIDerrick Stolee via GitGitGadget, Jan 26, 2021
  27. Taylor BlauJan 27, 2021
  28. Taylor BlauJan 27, 2021
  29. Junio C HamanoJan 26, 2021
  30. 00/17 Refactor chunk-format into an APIDerrick Stolee via GitGitGadget, Jan 27, 2021
  31. 05/17 midx: use context in write_midx_pack_names()Derrick Stolee via GitGitGadget, Jan 27, 2021
  32. 03/17 commit-graph: use chunk-format write APIDerrick Stolee via GitGitGadget, Jan 27, 2021
  33. 07/17 midx: add pack_perm to write_midx_contextDerrick Stolee via GitGitGadget, Jan 27, 2021
  34. 02/17 chunk-format: create chunk format write APIDerrick Stolee via GitGitGadget, Jan 27, 2021
  35. Junio C HamanoFeb 4, 2021
  36. Junio C HamanoFeb 4, 2021
  37. Derrick StoleeFeb 5, 2021
  38. Junio C HamanoFeb 5, 2021
  39. 15/17 midx: use 64-bit multiplication for chunk sizesDerrick Stolee via GitGitGadget, Jan 27, 2021
  40. Junio C HamanoFeb 5, 2021
  41. Chris TorekFeb 5, 2021
  42. Junio C HamanoFeb 5, 2021
  43. Chris TorekFeb 6, 2021
  44. Derrick StoleeFeb 5, 2021
  45. Junio C HamanoFeb 5, 2021
  46. SZEDER GáborFeb 7, 2021
  47. Junio C HamanoFeb 8, 2021
  48. 17/17 chunk-format: add technical docsDerrick Stolee via GitGitGadget, Jan 27, 2021
  49. Junio C HamanoFeb 5, 2021
  50. 14/17 midx: use chunk-format read APIDerrick Stolee via GitGitGadget, Jan 27, 2021
  51. 13/17 commit-graph: use chunk-format read APIDerrick Stolee via GitGitGadget, Jan 27, 2021
  52. 16/17 chunk-format: restore duplicate chunk checksDerrick Stolee via GitGitGadget, Jan 27, 2021
  53. Junio C HamanoFeb 5, 2021
  54. Derrick StoleeFeb 5, 2021
  55. 11/17 midx: use chunk-format API in write_midx_internal()Derrick Stolee via GitGitGadget, Jan 27, 2021
  56. 12/17 chunk-format: create read chunk APIDerrick Stolee via GitGitGadget, Jan 27, 2021
  57. Junio C HamanoFeb 4, 2021
  58. Derrick StoleeFeb 5, 2021
  59. Junio C HamanoFeb 5, 2021
  60. Junio C HamanoFeb 8, 2021
  61. Derrick StoleeFeb 9, 2021
  62. Junio C HamanoFeb 9, 2021
  63. 09/17 midx: return success/failure in chunk write methodsDerrick Stolee via GitGitGadget, Jan 27, 2021
  64. Junio C HamanoFeb 4, 2021
  65. Derrick StoleeFeb 5, 2021
  66. 10/17 midx: drop chunk progress during writeDerrick Stolee via GitGitGadget, Jan 27, 2021
  67. 06/17 midx: add entries to write_midx_contextDerrick Stolee via GitGitGadget, Jan 27, 2021
  68. 08/17 midx: add num_large_offsets to write_midx_contextDerrick Stolee via GitGitGadget, Jan 27, 2021
  69. 04/17 midx: rename pack_info to write_midx_contextDerrick Stolee via GitGitGadget, Jan 27, 2021
  70. 01/17 commit-graph: anonymize data in chunk_write_fnDerrick Stolee via GitGitGadget, Jan 27, 2021
  71. Taylor BlauJan 27, 2021
  72. Junio C HamanoFeb 5, 2021
  73. Derrick StoleeFeb 5, 2021
  74. 00/17 Refactor chunk-format into an APIDerrick Stolee via GitGitGadget, Feb 5, 2021
  75. 10/17 midx: drop chunk progress during writeDerrick Stolee via GitGitGadget, Feb 5, 2021
  76. 17/17 chunk-format: add technical docsDerrick Stolee via GitGitGadget, Feb 5, 2021
  77. 09/17 midx: return success/failure in chunk write methodsDerrick Stolee via GitGitGadget, Feb 5, 2021
  78. 15/17 midx: use 64-bit multiplication for chunk sizesDerrick Stolee via GitGitGadget, Feb 5, 2021
  79. 02/17 chunk-format: create chunk format write APIDerrick Stolee via GitGitGadget, Feb 5, 2021
  80. SZEDER GáborFeb 7, 2021
  81. Derrick StoleeFeb 8, 2021
  82. SZEDER GáborFeb 11, 2021
  83. 08/17 midx: add num_large_offsets to write_midx_contextDerrick Stolee via GitGitGadget, Feb 5, 2021
  84. 05/17 midx: use context in write_midx_pack_names()Derrick Stolee via GitGitGadget, Feb 5, 2021
  85. 14/17 midx: use chunk-format read APIDerrick Stolee via GitGitGadget, Feb 5, 2021
  86. 07/17 midx: add pack_perm to write_midx_contextDerrick Stolee via GitGitGadget, Feb 5, 2021
  87. 06/17 midx: add entries to write_midx_contextDerrick Stolee via GitGitGadget, Feb 5, 2021
  88. 16/17 chunk-format: restore duplicate chunk checksDerrick Stolee via GitGitGadget, Feb 5, 2021
  89. 13/17 commit-graph: use chunk-format read APIDerrick Stolee via GitGitGadget, Feb 5, 2021
  90. 01/17 commit-graph: anonymize data in chunk_write_fnDerrick Stolee via GitGitGadget, Feb 5, 2021
  91. 03/17 commit-graph: use chunk-format write APIDerrick Stolee via GitGitGadget, Feb 5, 2021
  92. 04/17 midx: rename pack_info to write_midx_contextDerrick Stolee via GitGitGadget, Feb 5, 2021
  93. 11/17 midx: use chunk-format API in write_midx_internal()Derrick Stolee via GitGitGadget, Feb 5, 2021
  94. 12/17 chunk-format: create read chunk APIDerrick Stolee via GitGitGadget, Feb 5, 2021
  95. SZEDER GáborFeb 7, 2021
  96. Derrick StoleeFeb 8, 2021
  97. 00/17 Refactor chunk-format into an APIDerrick Stolee via GitGitGadget, Feb 18, 2021
  98. 07/17 midx: add pack_perm to write_midx_contextDerrick Stolee via GitGitGadget, Feb 18, 2021
  99. 05/17 midx: use context in write_midx_pack_names()Derrick Stolee via GitGitGadget, Feb 18, 2021
  100. 03/17 commit-graph: use chunk-format write APIDerrick Stolee via GitGitGadget, Feb 18, 2021
  101. SZEDER GáborFeb 24, 2021
  102. Taylor BlauFeb 24, 2021
  103. Derrick StoleeFeb 24, 2021
  104. Junio C HamanoFeb 24, 2021
  105. 04/17 midx: rename pack_info to write_midx_contextDerrick Stolee via GitGitGadget, Feb 18, 2021
  106. 06/17 midx: add entries to write_midx_contextDerrick Stolee via GitGitGadget, Feb 18, 2021
  107. 02/17 chunk-format: create chunk format write APIDerrick Stolee via GitGitGadget, Feb 18, 2021
  108. 01/17 commit-graph: anonymize data in chunk_write_fnDerrick Stolee via GitGitGadget, Feb 18, 2021
  109. 08/17 midx: add num_large_offsets to write_midx_contextDerrick Stolee via GitGitGadget, Feb 18, 2021
  110. 09/17 midx: return success/failure in chunk write methodsDerrick Stolee via GitGitGadget, Feb 18, 2021
  111. 10/17 midx: drop chunk progress during writeDerrick Stolee via GitGitGadget, Feb 18, 2021
  112. 11/17 midx: use chunk-format API in write_midx_internal()Derrick Stolee via GitGitGadget, Feb 18, 2021
  113. 12/17 chunk-format: create read chunk APIDerrick Stolee via GitGitGadget, Feb 18, 2021
  114. 17/17 chunk-format: add technical docsDerrick Stolee via GitGitGadget, Feb 18, 2021
  115. Junio C HamanoFeb 18, 2021
  116. Derrick StoleeFeb 19, 2021
  117. 15/17 midx: use 64-bit multiplication for chunk sizesDerrick Stolee via GitGitGadget, Feb 18, 2021
  118. 14/17 midx: use chunk-format read APIDerrick Stolee via GitGitGadget, Feb 18, 2021
  119. 13/17 commit-graph: use chunk-format read APIDerrick Stolee via GitGitGadget, Feb 18, 2021
  120. 16/17 chunk-format: restore duplicate chunk checksDerrick Stolee via GitGitGadget, Feb 18, 2021

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.