git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH 1/2] docs: document SHA-256 pack and indices

From
brian m. carlson <sandals@crustytoothpaste.net>
Date
Aug 13, 2020, 22:49 UTC
Message-ID
<20200813224901.2652387-2-sandals@crustytoothpaste.net>
In-Reply-To
<20200813224901.2652387-1-sandals@crustytoothpaste.net>

Now that we have SHA-256 support for packs and indices, let's document that in SHA-256 repositories, we use SHA-256 instead of SHA-1 for object names and checksums. Instead of duplicating this information throughout the document, let's just document that in SHA-1 repositories, we use SHA-1 for these purposes, and in SHA-256 repositories, we use SHA-256.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 Documentation/technical/pack-format.txt | 36 ++++++++++++++-----------
 1 file changed, 21 insertions(+), 15 deletions(-)
diff --git a/Documentation/technical/pack-format.txt b/Documentation/technical/pack-format.txt
index d3a142c652..f4c8d94f73 100644
--- a/Documentation/technical/pack-format.txt
+++ b/Documentation/technical/pack-format.txt
@@ -1,6 +1,12 @@
 Git pack format
 ===============
 
+== Checksums and object IDs
+
+In a repository using the traditional SHA-1, pack checksums, index checksums,
+and object IDs (object names) mentioned below are all computed using SHA-1.
+Similarly, in SHA-256 repositories, these values are computed using SHA-256.
+
 == pack-*.pack files have the following format:
 
    - A header appears at the beginning and consists of the following:
@@ -26,7 +32,7 @@ Git pack format
 
      (deltified representation)
      n-byte type and length (3-bit type, (n-1)*7+4-bit length)
-     20-byte base object name if OBJ_REF_DELTA or a negative relative
+     base object name if OBJ_REF_DELTA or a negative relative
 	 offset from the delta object's position in the pack if this
 	 is an OBJ_OFS_DELTA object
      compressed delta data
@@ -34,7 +40,7 @@ Git pack format
      Observation: length of each object is encoded in a variable
      length format and is not constrained to 32-bit or anything.
 
-  - The trailer records 20-byte SHA-1 checksum of all of the above.
+  - The trailer records a pack checksum of all of the above.
 
 === Object types
 
@@ -58,8 +64,8 @@ ofs-delta and ref-delta, which is only valid in a pack file.
 
 Both ofs-delta and ref-delta store the "delta" to be applied to
 another object (called 'base object') to reconstruct the object. The
-difference between them is, ref-delta directly encodes 20-byte base
-object name. If the base object is in the same pack, ofs-delta encodes
+difference between them is, ref-delta directly encodes base object
+name. If the base object is in the same pack, ofs-delta encodes
 the offset of the base object in the pack instead.
 
 The base object could also be deltified if it's in the same pack.
@@ -143,14 +149,14 @@ This is the instruction reserved for future expansion.
     object is stored in the packfile as the offset from the
     beginning.
 
-    20-byte object name.
+    one object name of the appropriate size.
 
   - The file is concluded with a trailer:
 
-    A copy of the 20-byte SHA-1 checksum at the end of
-    corresponding packfile.
+    A copy of the pack checksum at the end of the corresponding
+    packfile.
 
-    20-byte SHA-1-checksum of all of the above.
+    Index checksum of all of the above.
 
 Pack Idx file:
 
@@ -198,7 +204,7 @@ Pack file entry: <+
         If it is not DELTA, then deflated bytes (the size above
 		is the size before compression).
 	If it is REF_DELTA, then
-	  20-byte base object name SHA-1 (the size above is the
+	  base object name (the size above is the
 		size of the delta data that follows).
           delta data, deflated.
 	If it is OFS_DELTA, then
@@ -227,9 +233,9 @@ Pack file entry: <+
 
   - A 256-entry fan-out table just like v1.
 
-  - A table of sorted 20-byte SHA-1 object names.  These are
-    packed together without offset values to reduce the cache
-    footprint of the binary search for a specific object name.
+  - A table of sorted object names.  These are packed together
+    without offset values to reduce the cache footprint of the
+    binary search for a specific object name.
 
   - A table of 4-byte CRC32 values of the packed object data.
     This is new in v2 so compressed data can be copied directly
@@ -248,10 +254,10 @@ Pack file entry: <+
 
   - The same trailer as a v1 pack file:
 
-    A copy of the 20-byte SHA-1 checksum at the end of
+    A copy of the pack checksum at the end of
     corresponding packfile.
 
-    20-byte SHA-1-checksum of all of the above.
+    Index checksum of all of the above.
 
 == multi-pack-index (MIDX) files have the following format:
 
@@ -329,4 +335,4 @@ CHUNK DATA:
 
 TRAILER:
 
-	20-byte SHA1-checksum of the above contents.
+	Index checksum of the above contents.
Previous: brian m. carlsonNext: Derrick Stolee
Message 2 of 35 in “Documentation updates for SHA-256”
  1. 0/2 Documentation updates for SHA-256brian m. carlson, Aug 13, 2020
  2. 1/2 docs: document SHA-256 pack and indicesbrian m. carlson, Aug 13, 2020
  3. Derrick StoleeAug 14, 2020
  4. 2/2 docs: fix step in transition planbrian m. carlson, Aug 13, 2020
  5. Martin ÅgrenAug 14, 2020
  6. Derrick StoleeAug 14, 2020
  7. Junio C HamanoAug 14, 2020
  8. brian m. carlsonAug 14, 2020
  9. Junio C HamanoAug 14, 2020
  10. 0/5 more SHA-256 documentationMartin Ågren, Aug 14, 2020
  11. 1/5 http-protocol.txt: document SHA-256 "want"/"have" formatMartin Ågren, Aug 14, 2020
  12. Junio C HamanoAug 14, 2020
  13. brian m. carlsonAug 14, 2020
  14. Martin ÅgrenAug 14, 2020
  15. Junio C HamanoAug 14, 2020
  16. Junio C HamanoAug 14, 2020
  17. 4/5 shallow.txt: document SHA-256 shallow formatMartin Ågren, Aug 14, 2020
  18. 3/5 protocol-capabilities.txt: clarify "allow-x-sha1-in-want" re SHA-256Martin Ågren, Aug 14, 2020
  19. Derrick StoleeAug 14, 2020
  20. Martin ÅgrenAug 14, 2020
  21. Junio C HamanoAug 14, 2020
  22. Martin ÅgrenAug 14, 2020
  23. Junio C HamanoAug 14, 2020
  24. 5/5 commit-graph-format.txt: fix "Hash Version" descriptionMartin Ågren, Aug 14, 2020
  25. Derrick StoleeAug 14, 2020
  26. Martin ÅgrenAug 14, 2020
  27. 2/5 index-format.txt: document SHA-256 index formatMartin Ågren, Aug 14, 2020
  28. Derrick StoleeAug 14, 2020
  29. Martin ÅgrenAug 14, 2020
  30. brian m. carlsonAug 14, 2020
  31. 0/4 more SHA-256 documentationMartin Ågren, Aug 15, 2020
  32. 2/4 index-format.txt: document SHA-256 index formatMartin Ågren, Aug 15, 2020
  33. 3/4 protocol-capabilities.txt: clarify "allow-x-sha1-in-want" re SHA-256Martin Ågren, Aug 15, 2020
  34. 4/4 shallow.txt: document SHA-256 shallow formatMartin Ågren, Aug 15, 2020
  35. 1/4 http-protocol.txt: document SHA-256 "want"/"have" formatMartin Ågren, Aug 15, 2020

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.