git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH 14/15] commit-graph: restore duplicate chunk checks

From
Derrick Stolee <stolee@gmail.com>
Date
Dec 7, 2020, 13:43 UTC
Message-ID
<86e1d647-2242-8a3b-d5a3-832e37e79ff8@gmail.com>
In-Reply-To
<106dd51f75699fbf4fc1e46687124995f5ef0278.1607012215.git.gitgitgadget@gmail.com>
On 12/3/2020 11:16 AM, Derrick Stolee via GitGitGadget wrote:
Show 6 quoted lines
> From: Derrick Stolee <dstolee@microsoft.com>
> 
> The previous change introduced read_table_of_contents() in the
> chunk-format API, but dropped the duplicate chunk check from the
> commit-graph parsing logic. This was done to keep flexibility in the
> chunk-format API.

"keep flexibility" is bogus. This is the biggest YAGNI of this series. Instead, consider the patch below instead which restores duplicate checks for the commit-graph file AND adds them to the multi-pack-index file due to the shared API.

This is also roughly half of the added lines from the previous patch.

Thanks, -Stolee

-- >8 --
From 0df4959d59d7f9df3e9f6326bb0acb7b84f84980 Mon Sep 17 00:00:00 2001
From: Derrick Stolee <dstolee@microsoft.com>
Date: Mon, 7 Dec 2020 08:36:42 -0500
Subject: [PATCH] chunk-format: restore duplicate chunk checks

Before refactoring into the chunk-format API, the commit-graph parsing logic included checks for duplicate chunks. It is unlikely that we would desire a chunk-based file format that allows duplicate chunk IDs in the table of contents, so add duplicate checks into read_table_of_contents().

Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
---
 chunk-format.c | 15 ++++++++++++++-
 chunk-format.h |  3 +++
 2 files changed, 17 insertions(+), 1 deletion(-)
diff --git a/chunk-format.c b/chunk-format.c
index d888ef6ec73..a4891bbd28a 100644
--- a/chunk-format.c
+++ b/chunk-format.c
@@ -57,9 +57,13 @@ int read_table_of_contents(const unsigned char *mfile,
 			   int nr,
 			   void *data)
 {
+	int i;
 	uint32_t chunk_id;
 	const unsigned char *table_of_contents = mfile + toc_offset;
 
+	for (i = 0; i < nr; i++)
+		chunks[i].found = 0;
+
 	while (toc_length--) {
 		int i;
 		uint64_t chunk_offset, next_chunk_offset;
@@ -83,7 +87,16 @@ int read_table_of_contents(const unsigned char *mfile,
 		}
 		for (i = 0; i < nr; i++) {
 			if (chunks[i].id == chunk_id) {
-				int result = chunks[i].read_fn(
+				int result;
+
+				if (chunks[i].found) {
+					error(_("duplicate chunk ID %"PRIx32" found"),
+					      chunk_id);
+					return 1;
+				}
+
+				chunks[i].found = 1;
+				result = chunks[i].read_fn(
 						mfile + chunk_offset,
 						next_chunk_offset - chunk_offset,
 						data);
diff --git a/chunk-format.h b/chunk-format.h
index 7049800f734..de45797223a 100644
--- a/chunk-format.h
+++ b/chunk-format.h
@@ -56,6 +56,9 @@ typedef int (*chunk_read_fn)(const unsigned char *chunk_start,
 struct read_chunk_info {
 	uint32_t id;
 	chunk_read_fn read_fn;
+
+	/* used internally */
+	unsigned found:1;
 };
 
 int read_table_of_contents(const unsigned char *mfile,
-- 
2.29.0.vfs.0.0
Previous: Derrick Stolee via GitGitGadgetNext: Derrick Stolee via GitGitGadget
Message 30 of 40 in “Refactor chunk-format into an API”
  1. 00/15 Refactor chunk-format into an APIDerrick Stolee via GitGitGadget, Dec 3, 2020
  2. 02/15 chunk-format: add API for writing table of contentsDerrick Stolee via GitGitGadget, Dec 3, 2020
  3. Taylor BlauDec 8, 2020
  4. 01/15 commit-graph: anonymize data in chunk_write_fnDerrick Stolee via GitGitGadget, Dec 3, 2020
  5. 04/15 midx: use context in write_midx_pack_names()Derrick Stolee via GitGitGadget, Dec 3, 2020
  6. 03/15 midx: rename pack_info to write_midx_contextDerrick Stolee via GitGitGadget, Dec 3, 2020
  7. 05/15 midx: add entries to write_midx_contextDerrick Stolee via GitGitGadget, Dec 3, 2020
  8. Junio C HamanoDec 3, 2020
  9. Derrick StoleeDec 4, 2020
  10. Taylor BlauDec 8, 2020
  11. 07/15 midx: add num_large_offsets to write_midx_contextDerrick Stolee via GitGitGadget, Dec 3, 2020
  12. 06/15 midx: add pack_perm to write_midx_contextDerrick Stolee via GitGitGadget, Dec 3, 2020
  13. 09/15 midx: drop chunk progress during writeDerrick Stolee via GitGitGadget, Dec 3, 2020
  14. 08/15 midx: convert chunk write methods to return intDerrick Stolee via GitGitGadget, Dec 3, 2020
  15. Junio C HamanoDec 3, 2020
  16. Derrick StoleeDec 4, 2020
  17. 11/15 midx: use 64-bit multiplication for chunk sizesDerrick Stolee via GitGitGadget, Dec 3, 2020
  18. Junio C HamanoDec 3, 2020
  19. Taylor BlauDec 8, 2020
  20. 12/15 chunk-format: create write_chunks()Derrick Stolee via GitGitGadget, Dec 3, 2020
  21. Taylor BlauDec 8, 2020
  22. 13/15 chunk-format: create chunk reading APIDerrick Stolee via GitGitGadget, Dec 3, 2020
  23. Junio C HamanoDec 3, 2020
  24. Derrick StoleeDec 4, 2020
  25. Junio C HamanoDec 4, 2020
  26. Junio C HamanoDec 3, 2020
  27. Derrick StoleeDec 4, 2020
  28. 15/15 chunk-format: add technical docsDerrick Stolee via GitGitGadget, Dec 3, 2020
  29. 14/15 commit-graph: restore duplicate chunk checksDerrick Stolee via GitGitGadget, Dec 3, 2020
  30. Derrick StoleeDec 7, 2020
  31. 10/15 midx: use chunk-format API in write_midx_internal()Derrick Stolee via GitGitGadget, Dec 3, 2020
  32. Taylor BlauDec 8, 2020
  33. Derrick StoleeDec 10, 2020
  34. René ScharfeDec 4, 2020
  35. Derrick StoleeDec 4, 2020
  36. Junio C HamanoDec 4, 2020
  37. Taylor BlauDec 8, 2020
  38. René ScharfeDec 9, 2020
  39. Taylor BlauDec 10, 2020
  40. Derrick StoleeDec 10, 2020

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.