git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH 11/15] pack-objects: clarify the use of object_entry::size

From
Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Date
Apr 14, 2018, 15:35 UTC
Message-ID
<20180414153513.9902-12-pclouds@gmail.com>
In-Reply-To
<20180414153513.9902-1-pclouds@gmail.com>

While this field most of the time contains the canonical object size, there is one case it does not: when we have found that the base object of the delta in question is also to be packed, we will very happily reuse the delta by copying it over instead of regenerating the new delta.

"size" in this case will record the delta size, not canonical object size. Later on in write_reuse_object(), we reconstruct the delta header and "size" is used for this purpose. When this happens, the "type" field contains a delta type instead of a canonical type. Highlight this in the code since it could be tricky to see.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
 builtin/pack-objects.c | 11 ++++++++---
 pack-objects.h         |  4 +++-
 2 files changed, 11 insertions(+), 4 deletions(-)
diff --git a/builtin/pack-objects.c b/builtin/pack-objects.c
index e75693176e..779f14a45e 100644
--- a/builtin/pack-objects.c
+++ b/builtin/pack-objects.c
@@ -1418,6 +1418,7 @@ static void check_object(struct object_entry *entry)
 		off_t ofs;
 		unsigned char *buf, c;
 		enum object_type type;
+		unsigned long in_pack_size;
 
 		buf = use_pack(p, &w_curs, entry->in_pack_offset, &avail);
 
@@ -1427,7 +1428,7 @@ static void check_object(struct object_entry *entry)
 		 */
 		used = unpack_object_header_buffer(buf, avail,
 						   &type,
-						   &entry->size);
+						   &in_pack_size);
 		if (used == 0)
 			goto give_up;
 
@@ -1444,6 +1445,7 @@ static void check_object(struct object_entry *entry)
 		default:
 			/* Not a delta hence we've already got all we need. */
 			oe_set_type(entry, entry->in_pack_type);
+			entry->size = in_pack_size;
 			entry->in_pack_header_size = used;
 			if (oe_type(entry) < OBJ_COMMIT || oe_type(entry) > OBJ_BLOB)
 				goto give_up;
@@ -1500,6 +1502,7 @@ static void check_object(struct object_entry *entry)
 			 * circular deltas.
 			 */
 			oe_set_type(entry, entry->in_pack_type);
+			entry->size = in_pack_size; /* delta size */
 			SET_DELTA(entry, base_entry);
 			entry->delta_size = entry->size;
 			entry->delta_sibling_idx = base_entry->delta_child_idx;
@@ -1509,13 +1512,15 @@ static void check_object(struct object_entry *entry)
 		}
 
 		if (oe_type(entry)) {
+			off_t delta_pos;
+
 			/*
 			 * This must be a delta and we already know what the
 			 * final object type is.  Let's extract the actual
 			 * object size from the delta header.
 			 */
-			entry->size = get_size_from_delta(p, &w_curs,
-					entry->in_pack_offset + entry->in_pack_header_size);
+			delta_pos = entry->in_pack_offset + entry->in_pack_header_size;
+			entry->size = get_size_from_delta(p, &w_curs, delta_pos);
 			if (entry->size == 0)
 				goto give_up;
 			unuse_pack(&w_curs);
diff --git a/pack-objects.h b/pack-objects.h
index 9d0391c173..e4ea6a350c 100644
--- a/pack-objects.h
+++ b/pack-objects.h
@@ -32,7 +32,9 @@ enum dfs_state {
  *
  * "size" is the uncompressed object size. Compressed size of the raw
  * data for an object in a pack is not stored anywhere but is computed
- * and made available when reverse .idx is made.
+ * and made available when reverse .idx is made. Note that when a
+ * delta is reused, "size" is the uncompressed _delta_ size, not the
+ * canonical one after the delta has been applied.
  *
  * "hash" contains a path name hash which is used for sorting the
  * delta list and also during delta searching. Once prepare_pack()
-- 
2.17.0.367.g5dd2e386c3
Previous: Nguyễn Thái Ngọc DuyNext: Nguyễn Thái Ngọc Duy
Message 14 of 17 in “nd/pack-objects-pack-struct update”
  1. 00/15 nd/pack-objects-pack-struct updateNguyễn Thái Ngọc Duy, Apr 14, 2018
  2. 02/15 pack-objects: a bit of document about struct object_entryNguyễn Thái Ngọc Duy, Apr 14, 2018
  3. 01/15 read-cache.c: make $GIT_TEST_SPLIT_INDEX booleanNguyễn Thái Ngọc Duy, Apr 14, 2018
  4. Ævar Arnfjörð BjarmasonApr 14, 2018
  5. 04/15 pack-objects: use bitfield for object_entry::dfs_stateNguyễn Thái Ngọc Duy, Apr 14, 2018
  6. 05/15 pack-objects: use bitfield for object_entry::depthNguyễn Thái Ngọc Duy, Apr 14, 2018
  7. 07/15 pack-objects: move in_pack out of struct object_entryNguyễn Thái Ngọc Duy, Apr 14, 2018
  8. 08/15 pack-objects: refer to delta objects by index instead of pointerNguyễn Thái Ngọc Duy, Apr 14, 2018
  9. 09/15 pack-objects: shrink z_delta_size field in struct object_entryNguyễn Thái Ngọc Duy, Apr 14, 2018
  10. 15/15 ci: exercise the whole test suite with uncommon code in pack-objectsNguyễn Thái Ngọc Duy, Apr 14, 2018
  11. 14/15 pack-objects: reorder members to shrink struct object_entryNguyễn Thái Ngọc Duy, Apr 14, 2018
  12. 13/15 pack-objects: shrink delta_size field in struct object_entryNguyễn Thái Ngọc Duy, Apr 14, 2018
  13. 10/15 pack-objects: don't check size when the object is badNguyễn Thái Ngọc Duy, Apr 14, 2018
  14. 11/15 pack-objects: clarify the use of object_entry::sizeNguyễn Thái Ngọc Duy, Apr 14, 2018
  15. 06/15 pack-objects: move in_pack_pos out of struct object_entryNguyễn Thái Ngọc Duy, Apr 14, 2018
  16. 12/15 pack-objects: shrink size field in struct object_entryNguyễn Thái Ngọc Duy, Apr 14, 2018
  17. 03/15 pack-objects: turn type and in_pack_type to bitfieldsNguyễn Thái Ngọc Duy, Apr 14, 2018

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.