git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH v6 12/12] packfile-uri.txt: support multiple object types

From
Teng Long <dyroneteng@gmail.com>
Date
Oct 19, 2021, 11:38 UTC
Message-ID
<79fc2c23cf053cb525d818b6b47e24c49b57f672.1634634814.git.tenglong@alibaba-inc.com>
In-Reply-To
<cover.1634634814.git.tenglong@alibaba-inc.com>
Signed-off-by: Teng Long <tenglong@alibaba-inc.com>
---
 Documentation/technical/packfile-uri.txt | 74 ++++++++++++++++++++----
 1 file changed, 63 insertions(+), 11 deletions(-)
diff --git a/Documentation/technical/packfile-uri.txt b/Documentation/technical/packfile-uri.txt
index f7eabc6c76..0e6d7c8dda 100644
--- a/Documentation/technical/packfile-uri.txt
+++ b/Documentation/technical/packfile-uri.txt
@@ -34,14 +34,69 @@ having it advertise `packfile-uris`, tolerating the client sending
 include some sort of non-trivial implementation in the Minimum Viable Product,
 at least so that we can test the client.
 
-This is the implementation: a feature, marked experimental, that allows the
-server to be configured by one or more `uploadpack.blobPackfileUri=<sha1>
-<uri>` entries. Whenever the list of objects to be sent is assembled, all such
-blobs are excluded, replaced with URIs. As noted in "Future work" below, the
-server can evolve in the future to support excluding other objects (or other
-implementations of servers could be made that support excluding other objects)
-without needing a protocol change, so clients should not expect that packfiles
-downloaded in this way only contain single blobs.
+This is the implementation: a feature, marked experimental, that allows
+the server to be configured by one or more entries with the format:
+
+    uploadpack.excludeobject=<object-hash> <level> <pack-hash> <uri>
+
+Value `<object-hash>` is the key of entry, and the object type can be
+blob, tree, commit, or tag. Value of entry has three parts,
+`<pack-hash>` is used to identify the packfile which contains the given
+`<object-hash>` object, and `<uri>` is the URI to download the packfile by
+client. For example, When a blob is configured with `uploadpack.excludeobject`
+that means whenever the blob to be send is assembled, the object will
+be excluded.
+
+In addition to excluding a single object like blob, sometimes it's
+hoped to exclude not only the object itself, but also all the related
+objects with it, like all the objects a tree contains or the ancestors
+that a commit can reach. In these cases, the `<level>` is designed to
+distinguish the scope of exclusion, it supports three levels:
+
+- Level 0: Excluding a single object itself, without any objects that
+  have a relationship with it. 
+
+- Level 1: Excluding object itself, and objects it contains.
+
+- Level 2: Excluding object itself, the objects it contains, and the
+  ancestors it can reach.
+
+If `<level>` is configured as 0, only the object itself will be
+excluded, no matter what the object type is. It is a common scenario
+for large size blobs, but it does much not sense for other object types
+(e.g. download a singe commit without downloading the blobs and tree
+in it).
+
+If `<level>` is configured as 1, not only the single object but also all
+the objects in it will be excluded. This applies to scenarios where
+it's wanted to exclude a specified non-blob object that includes some
+lage size objects.
+
+- If <object-hash> is a blob, the result is the same as level 0, because blob
+contains nothing just itself.
+
+- If <object-hash> is a tree, the tree itself, and all blobs and trees
+  in it will be excluded.
+
+- If <object-hash> is a commit, the commit itself, the referenced
+  root-tree, and all blobs and trees in the root-tree will be excluded.
+
+- If <object-hash> is a tag, the tag itself, the dereferenced commit
+  and all trees and blobs contained in its root-tree will be excluded.
+
+If `<level>` is configured as 2, not only the objects in the scope of
+level 1 , but also the reachable ancestors will be excluded if
+`<object-hash>` is commit or tag.
+
+Configuration compatibility
+---------------------------
+
+The old configuration of packfile-uri:
+
+    uploadpack.blobPackfileUri=<object-hash> <pack-hash> <uri>
+
+For the old configuration is compatible with the new one, but it only
+supports the exclusion of blob objects.
 
 Client design
 -------------
@@ -65,9 +120,6 @@ The protocol design allows some evolution of the server and client without any
 need for protocol changes, so only a small-scoped design is included here to
 form the MVP. For example, the following can be done:
 
- * On the server, more sophisticated means of excluding objects (e.g. by
-   specifying a commit to represent that commit and all objects that it
-   references).
  * On the client, resumption of clone. If a clone is interrupted, information
    could be recorded in the repository's config and a "clone-resume" command
    can resume the clone in progress. (Resumption of subsequent fetches is more
-- 
2.31.1.453.g945ddc3a74.dirty
Previous: Teng Long
Message 72 of 72 in “Packfile-uris support excluding commit objects”
  1. Packfile-uris support excluding commit objectsTeng Long, May 7, 2021
  2. Ævar Arnfjörð BjarmasonMay 10, 2021
  3. 0/3 packfile-uris: commit objects exclusionTeng Long, May 18, 2021
  4. 1/3 packfile-uris: support for excluding commit objectTeng Long, May 18, 2021
  5. Junio C HamanoMay 19, 2021
  6. Junio C HamanoMay 20, 2021
  7. 2/3 packfile-uris.txt: excluding commit objectTeng Long, May 18, 2021
  8. 3/3 t5702: excluding commits with packfile-urisTeng Long, May 18, 2021
  9. 0/3 packfile-uris: commit objects exclusioTeng Long, Jul 26, 2021
  10. 1/3 packfile-uris: support for excluding commit objectsTeng Long, Jul 26, 2021
  11. Junio C HamanoJul 26, 2021
  12. Felipe ContrerasJul 26, 2021
  13. Teng LongAug 11, 2021
  14. 2/3 t5702: support for excluding commit objectsTeng Long, Jul 26, 2021
  15. Ævar Arnfjörð BjarmasonJul 26, 2021
  16. Teng LongAug 11, 2021
  17. 3/3 packfile-uri.txt: support for excluding commit objectsTeng Long, Jul 26, 2021
  18. Junio C HamanoJul 26, 2021
  19. Teng LongAug 11, 2021
  20. Ævar Arnfjörð BjarmasonJul 26, 2021
  21. Teng LongAug 11, 2021
  22. 0/7 packfile-uris: commits and trees exclusionTeng Long, Aug 11, 2021
  23. 1/7 pack-objects.c: introduce new method `match_packfile_uri_exclusions`Teng Long, Aug 11, 2021
  24. 2/7 Add new parameter "carry_data" for "show_object" functionTeng Long, Aug 11, 2021
  25. 3/7 packfile-uri: support for excluding commit objectsTeng Long, Aug 11, 2021
  26. 4/7 packfile-uri: support for excluding tree objectsTeng Long, Aug 11, 2021
  27. 5/7 packfile-uri.txt: support for excluding commits and treesTeng Long, Aug 11, 2021
  28. Bagas SanjayaAug 11, 2021
  29. 6/7 t5702: replace with "test_when_finished" for cleanupTeng Long, Aug 11, 2021
  30. 7/7 t5702: support for excluding commit objectsTeng Long, Aug 11, 2021
  31. 00/14 packfile-uris: commits, trees and tags exclusionTeng Long, Aug 25, 2021
  32. 01/14 pack-objects.c: introduce new method `match_packfile_uri_exclusions`Teng Long, Aug 25, 2021
  33. 02/14 Add new parameter "carry_data" for "show_object" functionTeng Long, Aug 25, 2021
  34. Junio C HamanoAug 26, 2021
  35. Teng LongSep 2, 2021
  36. 03/14 packfile-uri: support for excluding commit objectsTeng Long, Aug 25, 2021
  37. Ævar Arnfjörð BjarmasonAug 25, 2021
  38. Teng LongSep 2, 2021
  39. Junio C HamanoAug 26, 2021
  40. Teng LongSep 2, 2021
  41. 04/14 packfile-uri: support for excluding tree objectsTeng Long, Aug 25, 2021
  42. 05/14 packfile-uri.txt: support for excluding commits and treesTeng Long, Aug 25, 2021
  43. Ævar Arnfjörð BjarmasonAug 25, 2021
  44. Teng LongSep 2, 2021
  45. 06/14 t5702: replace with "test_when_finished" for cleanupTeng Long, Aug 25, 2021
  46. Ævar Arnfjörð BjarmasonAug 25, 2021
  47. Teng LongSep 2, 2021
  48. 07/14 t5702: support for excluding commit objectsTeng Long, Aug 25, 2021
  49. 08/14 Add new parameter "carry_data" for "show_commit functionTeng Long, Aug 25, 2021
  50. 09/14 commit.h: add wrapped tags in commit structTeng Long, Aug 25, 2021
  51. Ævar Arnfjörð BjarmasonAug 25, 2021
  52. Teng LongSep 2, 2021
  53. ZheNing HuSep 2, 2021
  54. Teng LongSep 2, 2021
  55. 10/14 object.h: add referred tags in `referred_objects` structTeng Long, Aug 25, 2021
  56. 11/14 packfile-uri: support for excluding tag objectsTeng Long, Aug 25, 2021
  57. 12/14 packfile-uri.txt: support for excluding tag objectsTeng Long, Aug 25, 2021
  58. 13/14 t5702: add tag exclusion test caseTeng Long, Aug 25, 2021
  59. 14/14 pack-objects.c: introduce `want_exclude_object` functionTeng Long, Aug 25, 2021
  60. 00/12 packfile-uri: support excluding multiple object typesTeng Long, Oct 19, 2021
  61. 01/12 objects.c: introduce `exclude_level` enumTeng Long, Oct 19, 2021
  62. 02/12 Introduce function `match_packfile_uri_exclusions`Teng Long, Oct 19, 2021
  63. 03/12 Replace `show_data` with structure `show_info`Teng Long, Oct 19, 2021
  64. 04/12 Introduce `uploadpack.excludeobject` configurationTeng Long, Oct 19, 2021
  65. 05/12 t5702: test cases for `uploadpack.excludeobject`Teng Long, Oct 19, 2021
  66. 06/12 packfile-uri: support for excluding commitsTeng Long, Oct 19, 2021
  67. 07/12 t5702: test cases for excluding commitsTeng Long, Oct 19, 2021
  68. 08/12 packfile-uri: support for excluding treesTeng Long, Oct 19, 2021
  69. 09/12 t5702: test cases for excluding treesTeng Long, Oct 19, 2021
  70. 10/12 packfile-uri: support for excluding tagsTeng Long, Oct 19, 2021
  71. 11/12 t5702: test cases for excluding tagsTeng Long, Oct 19, 2021
  72. 12/12 packfile-uri.txt: support multiple object typesTeng Long, Oct 19, 2021

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.