git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH 9/17] Document why header parsing won't exceed a window.

From
Shawn O. Pearce <spearce@spearce.org>
Date
Dec 23, 2006, 07:34 UTC
Message-ID
<20061223073418.GJ9837@spearce.org>
In-Reply-To
<53b67707929c7f051f6d384c5d96e653bfa8419c.1166857884.git.spearce@spearce.org>

When we parse the object header or the delta base reference we don't bother to loop over use_pack() calls. The reason we don't need to bother with calling use_pack for each byte accessed is that use_pack will always promise us at least 20 bytes (really the hash size) after the offset. This promise from use_pack simplifies a lot of code in the header parsing logic, as well as helps out the zlib library by ensuring there's always some data for it to consume during an inflate call.

Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
---
 sha1_file.c |   16 ++++++++++++----
 1 files changed, 12 insertions(+), 4 deletions(-)
diff --git a/sha1_file.c b/sha1_file.c
index 886e3b5..6c34482 100644
--- a/sha1_file.c
+++ b/sha1_file.c
@@ -901,10 +901,12 @@ static unsigned long get_delta_base(struct packed_git *p,
 	unsigned char *base_info = use_pack(p, w_curs, offset, NULL);
 	unsigned long base_offset;
 
-	/* there must be at least 20 bytes left regardless of delta type */
-	if (p->pack_size <= offset + 20)
-		die("truncated pack file");
-
+	/* use_pack() assured us we have [base_info, base_info + 20)
+	 * as a range that we can look at without walking off the
+	 * end of the mapped window.  Its actually the hash size
+	 * that is assured.  An OFS_DELTA longer than the hash size
+	 * is stupid, as then a REF_DELTA would be smaller to store.
+	 */
 	if (kind == OBJ_OFS_DELTA) {
 		unsigned used = 0;
 		unsigned char c = base_info[used++];
@@ -1007,6 +1009,12 @@ static unsigned long unpack_object_header(struct packed_git *p,
 	unsigned int left;
 	unsigned long used;
 
+	/* use_pack() assures us we have [base, base + 20) available
+	 * as a range that we can look at at.  (Its actually the hash
+	 * size that is assurred.)  With our object header encoding
+	 * the maximum deflated object size is 2^137, which is just
+	 * insane, so we know won't exceed what we have been given.
+	 */
 	base = use_pack(p, w_curs, offset, &left);
 	used = unpack_object_header_gently(base, left, type, sizep);
 	if (!used)
-- 
1.4.4.3.g87d8
Previous: Shawn O. PearceNext: Shawn O. Pearce
Message 9 of 25 in “Replace unpack_entry_gently with unpack_entry.”
  1. 1/17 Replace unpack_entry_gently with unpack_entry.Shawn O. Pearce, Dec 23, 2006
  2. 2/17 Introduce new config option for mmap limit.Shawn O. Pearce, Dec 23, 2006
  3. 3/17 Refactor packed_git to prepare for sliding mmap windows.Shawn O. Pearce, Dec 23, 2006
  4. 4/17 Use off_t for index and pack file lengths.Shawn O. Pearce, Dec 23, 2006
  5. 5/17 Create read_or_die utility routine.Shawn O. Pearce, Dec 23, 2006
  6. 6/17 Refactor how we open pack files to prepare for multiple windows.Shawn O. Pearce, Dec 23, 2006
  7. 7/17 Replace use_packed_git with window cursors.Shawn O. Pearce, Dec 23, 2006
  8. 8/17 Loop over pack_windows when inflating/accessing data.Shawn O. Pearce, Dec 23, 2006
  9. 9/17 Document why header parsing won't exceed a window.Shawn O. Pearce, Dec 23, 2006
  10. 10/17 Unmap individual windows rather than entire files.Shawn O. Pearce, Dec 23, 2006
  11. 11/17 Fully activate the sliding window pack access.Shawn O. Pearce, Dec 23, 2006
  12. Linus TorvaldsDec 23, 2006
  13. Eric BlakeDec 23, 2006
  14. Johannes SchindelinDec 24, 2006
  15. Junio C HamanoDec 23, 2006
  16. Linus TorvaldsDec 23, 2006
  17. Johannes SchindelinDec 24, 2006
  18. Shawn PearceDec 24, 2006
  19. Shawn PearceDec 24, 2006
  20. 12/17 Load core configuration in git-verify-pack.Shawn O. Pearce, Dec 23, 2006
  21. 13/17 Ensure core.packedGitWindowSize cannot be less than 2 pages.Shawn O. Pearce, Dec 23, 2006
  22. 14/17 Improve error message when packfile mmap fails.Shawn O. Pearce, Dec 23, 2006
  23. 15/17 Support unmapping windows on 'temporary' packfiles.Shawn O. Pearce, Dec 23, 2006
  24. 16/17 Create pack_report() as a debugging aid.Shawn O. Pearce, Dec 23, 2006
  25. 17/17 Test suite for sliding window mmap implementation.Shawn O. Pearce, Dec 23, 2006

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.