git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH 08/10] commit-graph: simplify parse_commit_graph() #2

From
SZEDER Gábor via GitGitGadget <gitgitgadget@gmail.com>
Date
Jun 5, 2020, 13:00 UTC
Message-ID
<83641b5e49e4f2abd2185fc1cde58eadad496cd4.1591362033.git.gitgitgadget@gmail.com>
In-Reply-To
<pull.650.git.1591362032.gitgitgadget@gmail.com>
From: =?UTF-8?q?SZEDER=20G=C3=A1bor?= <szeder.dev@gmail.com>

The Chunk Lookup table stores the chunks' starting offset in the commit-graph file, not their sizes. Consequently, the size of a chunk can only be calculated by subtracting its offset from the offset of the subsequent chunk (or that of the terminating label). This is currenly implemented in a bit complicated way: as we iterate over the entries of the Chunk Lookup table, we check the id of each chunk and store its starting offset, then we check the id of the last seen chunk and calculate its size using its previously saved offset. At the moment there is only one chunk for which we calculate its size, but this patch series will add more, and the repeated chunk id checks are not that pretty.

Instead let's read ahead the offset of the next chunk on each iteration, so we can calculate the size of each chunk right away, right where we store its starting offset.

Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
---
 commit-graph.c | 26 +++++++++-----------------
 1 file changed, 9 insertions(+), 17 deletions(-)
diff --git a/commit-graph.c b/commit-graph.c
index 9927762f18c..84206f0f512 100644
--- a/commit-graph.c
+++ b/commit-graph.c
@@ -230,8 +230,7 @@ struct commit_graph *parse_commit_graph(void *graph_map, int fd,
 	const unsigned char *data, *chunk_lookup;
 	uint32_t i;
 	struct commit_graph *graph;
-	uint64_t last_chunk_offset;
-	uint32_t last_chunk_id;
+	uint64_t next_chunk_offset;
 	uint32_t graph_signature;
 	unsigned char graph_version, hash_version;
 
@@ -281,18 +280,17 @@ struct commit_graph *parse_commit_graph(void *graph_map, int fd,
 		return NULL;
 	}
 
-	last_chunk_id = 0;
-	last_chunk_offset = 8;
 	chunk_lookup = data + 8;
-	for (i = 0; i <= graph->num_chunks; i++) {
+	next_chunk_offset = get_be64(chunk_lookup + 4);
+	for (i = 0; i < graph->num_chunks; i++) {
 		uint32_t chunk_id;
-		uint64_t chunk_offset;
+		uint64_t chunk_offset = next_chunk_offset;
 		int chunk_repeated = 0;
 
 		chunk_id = get_be32(chunk_lookup + 0);
-		chunk_offset = get_be64(chunk_lookup + 4);
 
 		chunk_lookup += GRAPH_CHUNKLOOKUP_WIDTH;
+		next_chunk_offset = get_be64(chunk_lookup + 4);
 
 		if (chunk_offset > graph_size - the_hash_algo->rawsz) {
 			error(_("commit-graph improper chunk offset %08x%08x"), (uint32_t)(chunk_offset >> 32),
@@ -312,8 +310,11 @@ struct commit_graph *parse_commit_graph(void *graph_map, int fd,
 		case GRAPH_CHUNKID_OIDLOOKUP:
 			if (graph->chunk_oid_lookup)
 				chunk_repeated = 1;
-			else
+			else {
 				graph->chunk_oid_lookup = data + chunk_offset;
+				graph->num_commits = (next_chunk_offset - chunk_offset)
+						     / graph->hash_len;
+			}
 			break;
 
 		case GRAPH_CHUNKID_DATA:
@@ -368,15 +369,6 @@ struct commit_graph *parse_commit_graph(void *graph_map, int fd,
 			free(graph);
 			return NULL;
 		}
-
-		if (last_chunk_id == GRAPH_CHUNKID_OIDLOOKUP)
-		{
-			graph->num_commits = (chunk_offset - last_chunk_offset)
-					     / graph->hash_len;
-		}
-
-		last_chunk_id = chunk_id;
-		last_chunk_offset = chunk_offset;
 	}
 
 	if (graph->chunk_bloom_indexes && graph->chunk_bloom_data) {
-- 
gitgitgadget
Previous: SZEDER Gábor via GitGitGadgetNext: SZEDER Gábor via GitGitGadget
Message 12 of 16 in “Szeder's commit-graph cleanups”
  1. 00/10 Szeder's commit-graph cleanupsDerrick Stolee via GitGitGadget, Jun 5, 2020
  2. 02/10 commit-graph: fix parsing the Chunk Lookup tableSZEDER Gábor via GitGitGadget, Jun 5, 2020
  3. 01/10 tree-walk.c: don't match submodule entries for 'submod/anything'SZEDER Gábor via GitGitGadget, Jun 5, 2020
  4. 03/10 commit-graph-format.txt: all multi-byte numbers are in network byte orderSZEDER Gábor via GitGitGadget, Jun 5, 2020
  5. 04/10 commit-slab: add a function to deep free entries on the slabSZEDER Gábor via GitGitGadget, Jun 5, 2020
  6. René ScharfeJun 18, 2020
  7. Derrick StoleeJun 19, 2020
  8. SZEDER GáborJun 27, 2020
  9. 05/10 diff.h: drop diff_tree_oid() & friends' return valueSZEDER Gábor via GitGitGadget, Jun 5, 2020
  10. 07/10 commit-graph: simplify parse_commit_graph() #1SZEDER Gábor via GitGitGadget, Jun 5, 2020
  11. 06/10 commit-graph: clean up #includesSZEDER Gábor via GitGitGadget, Jun 5, 2020
  12. 08/10 commit-graph: simplify parse_commit_graph() #2SZEDER Gábor via GitGitGadget, Jun 5, 2020
  13. 09/10 commit-graph: simplify write_commit_graph_file() #1SZEDER Gábor via GitGitGadget, Jun 5, 2020
  14. 10/10 commit-graph: simplify write_commit_graph_file() #2SZEDER Gábor via GitGitGadget, Jun 5, 2020
  15. Junio C HamanoJun 8, 2020
  16. Derrick StoleeJun 18, 2020

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.