git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH 2/2] Document the hairy gfi_unpack_entry part of fast-import

From
Shawn O. Pearce <spearce@spearce.org>
Date
Jan 21, 2008, 04:37 UTC
Message-ID
<20080121043701.GB27521@spearce.org>

Junio pointed out this part of fast-import wasn't very clear on initial read, and it took some time for someone who was new to fast-import's "dirty little tricks" to understand how this was even working. So a little bit of commentary in the proper place may help future readers.

Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
---
 fast-import.c |   33 +++++++++++++++++++++++++++++++++
 1 files changed, 33 insertions(+), 0 deletions(-)
diff --git a/fast-import.c b/fast-import.c
index f6872fe..a523b17 100644
--- a/fast-import.c
+++ b/fast-import.c
@@ -1125,6 +1125,24 @@ static int store_object(
 	return 0;
 }
 
+/* All calls must be guarded by find_object() or find_mark() to
+ * ensure the 'struct object_entry' passed was written by this
+ * process instance.  We unpack the entry by the offset, avoiding
+ * the need for the corresponding .idx file.  This unpacking rule
+ * works because we only use OBJ_REF_DELTA within the packfiles
+ * created by fast-import.
+ *
+ * oe must not be NULL.  Such an oe usually comes from giving
+ * an unknown SHA-1 to find_object() or an undefined mark to
+ * find_mark().  Callers must test for this condition and use
+ * the standard read_sha1_file() when it happens.
+ *
+ * oe->pack_id must not be MAX_PACK_ID.  Such an oe is usually from
+ * find_mark(), where the mark was reloaded from an existing marks
+ * file and is referencing an object that this fast-import process
+ * instance did not write out to a packfile.  Callers must test for
+ * this condition and use read_sha1_file() instead.
+ */
 static void *gfi_unpack_entry(
 	struct object_entry *oe,
 	unsigned long *sizep)
@@ -1132,7 +1150,22 @@ static void *gfi_unpack_entry(
 	enum object_type type;
 	struct packed_git *p = all_packs[oe->pack_id];
 	if (p == pack_data && p->pack_size < (pack_size + 20)) {
+		/* The object is stored in the packfile we are writing to
+		 * and we have modified it since the last time we scanned
+		 * back to read a previously written object.  If an old
+		 * window covered [p->pack_size, p->pack_size + 20) its
+		 * data is stale and is not valid.  Closing all windows
+		 * and updating the packfile length ensures we can read
+		 * the newly written data.
+		 */
 		close_pack_windows(p);
+
+		/* We have to offer 20 bytes additional on the end of
+		 * the packfile as the core unpacker code assumes the
+		 * footer is present at the file end and must promise
+		 * at least 20 bytes within any window it maps.  But
+		 * we don't actually create the footer here.
+		 */
 		p->pack_size = pack_size + 20;
 	}
 	return unpack_entry(p, oe->offset, &type, sizep);
-- 
1.5.4.rc4.1109.g30426
Message 1 of 1 in “Document the hairy gfi_unpack_entry part of fast-import”
  1. 2/2 Document the hairy gfi_unpack_entry part of fast-importShawn O. Pearce, Jan 21, 2008

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.