git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH v4 00/28] First class shallow clone

From
Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Date
Dec 5, 2013, 13:02 UTC
Message-ID
<1386248575-10206-1-git-send-email-pclouds@gmail.com>
In-Reply-To
<1385351754-9954-1-git-send-email-pclouds@gmail.com>

This reroll should fix all comments I have received in v3. I reordered the shallow checks a bit so in common case it should be as cheap as a normal fetch or push. See 08/28 and 20/28 for the big picture. I'm not entirely happy with the hook issue in 20/28, but it looks good enough for me. There are a few XXXes for further improvement, but I'll keep them until this lands.

To recap, this series allows you to clone from a shallow repo, push to or fetch from any shallow repo. Normally it will reject new refs that introduce new shallow boundaries to your repository, so if you're in a full clone, it will always stay a full clone. Use "fetch --update-shallow" or set receive.shallowupdate to accept those refs.

Nguyễn Thái Ngọc Duy (28):
  transport.h: remove send_pack prototype, already defined in send-pack.h
  Replace struct extra_have_objects with struct sha1_array
  send-pack: forbid pushing from a shallow repository
  clone: prevent --reference to a shallow repository
  Make the sender advertise shallow commits to the receiver
  connect.c: teach get_remote_heads to parse "shallow" lines
  shallow.c: extend setup_*_shallow() to accept extra shallow commits
  shallow.c: the 8 steps to select new commits for .git/shallow
  shallow.c: steps 6 and 7 to select new commits for .git/shallow
  fetch-pack.c: move shallow update code out of fetch_pack()
  fetch-pack.h: one statement per bitfield declaration
  clone: support remote shallow repository
  fetch: support fetching from a shallow repository
  upload-pack: make sure deepening preserves shallow roots
  fetch: add --update-shallow to accept refs that update .git/shallow
  receive-pack: reorder some code in unpack()
  receive/send-pack: support pushing from a shallow clone
  New var GIT_SHALLOW_FILE to propagate --shallow-file to subprocesses
  connected.c: add new variant that runs with --shallow-file
  receive-pack: allow pushes that update .git/shallow
  send-pack: support pushing to a shallow clone
  remote-curl: pass ref SHA-1 to fetch-pack as well
  Support shallow fetch/clone over smart-http
  receive-pack: support pushing to a shallow clone via http
  send-pack: support pushing from a shallow clone via http
  clone: use git protocol for cloning shallow repo locally
  prune: clean .git/shallow after pruning objects
  git-clone.txt: remove shallow clone limitations
 Documentation/config.txt                  |   4 +
 Documentation/fetch-options.txt           |  14 +-
 Documentation/git-clone.txt               |   7 +-
 Documentation/git-prune.txt               |   2 +
 Documentation/gitremote-helpers.txt       |   7 +
 Documentation/technical/pack-protocol.txt |   7 +-
 builtin/clone.c                           |  18 +-
 builtin/fetch-pack.c                      |  23 +-
 builtin/fetch.c                           |  15 +-
 builtin/gc.c                              |   1 +
 builtin/prune.c                           |   4 +
 builtin/receive-pack.c                    | 314 +++++++++++++++----
 builtin/send-pack.c                       |   9 +-
 cache.h                                   |   3 +
 commit.h                                  |  39 ++-
 connect.c                                 |  22 +-
 connected.c                               |  42 ++-
 connected.h                               |   2 +
 environment.c                             |   6 +
 fetch-pack.c                              | 132 +++++++-
 fetch-pack.h                              |  29 +-
 git.c                                     |   2 +-
 remote-curl.c                             |  34 ++-
 remote.h                                  |   9 +-
 send-pack.c                               |  27 +-
 send-pack.h                               |   2 +-
 shallow.c                                 | 486 +++++++++++++++++++++++++++++-
 t/t5304-prune.sh                          |  10 +
 t/t5536-fetch-shallow.sh (new +x)         | 203 +++++++++++++
 t/t5537-push-shallow.sh (new +x)          | 183 +++++++++++
 t/t5601-clone.sh                          |   7 +
 trace.c                                   |   2 +-
 transport-helper.c                        |   6 +
 transport.c                               |  25 +-
 transport.h                               |  16 +-
 upload-pack.c                             |   8 +-
 36 files changed, 1555 insertions(+), 165 deletions(-)
 create mode 100755 t/t5536-fetch-shallow.sh
 create mode 100755 t/t5537-push-shallow.sh
-- 
1.8.5.1.25.g8667982
Previous: Nguyễn Thái Ngọc DuyNext: Nguyễn Thái Ngọc Duy
Message 48 of 79 in “First class shallow clone”
  1. 00/28 First class shallow cloneNguyễn Thái Ngọc Duy, Nov 25, 2013
  2. 01/28 transport.h: remove send_pack prototype, already defined in send-pack.hNguyễn Thái Ngọc Duy, Nov 25, 2013
  3. 02/28 send-pack: forbid pushing from a shallow repositoryNguyễn Thái Ngọc Duy, Nov 25, 2013
  4. 03/28 clone: prevent --reference to a shallow repositoryNguyễn Thái Ngọc Duy, Nov 25, 2013
  5. Eric SunshineNov 26, 2013
  6. 04/28 update-server-info: do not publish shallow clonesNguyễn Thái Ngọc Duy, Nov 25, 2013
  7. Junio C HamanoNov 25, 2013
  8. Duy NguyenNov 26, 2013
  9. 05/28 Advertise shallow graft information on the server endNguyễn Thái Ngọc Duy, Nov 25, 2013
  10. 06/28 connect.c: teach get_remote_heads to parse "shallow" linesNguyễn Thái Ngọc Duy, Nov 25, 2013
  11. Junio C HamanoNov 25, 2013
  12. Junio C HamanoNov 25, 2013
  13. Duy NguyenNov 27, 2013
  14. 07/28 shallow.c: add remove_reachable_shallow_points()Nguyễn Thái Ngọc Duy, Nov 25, 2013
  15. Junio C HamanoNov 25, 2013
  16. 08/28 shallow.c: add mark_new_shallow_refs()Nguyễn Thái Ngọc Duy, Nov 25, 2013
  17. Junio C HamanoNov 25, 2013
  18. Duy NguyenNov 26, 2013
  19. Junio C HamanoNov 26, 2013
  20. 09/28 shallow.c: extend setup_*_shallow() to accept extra shallow pointsNguyễn Thái Ngọc Duy, Nov 25, 2013
  21. Junio C HamanoNov 25, 2013
  22. 10/28 fetch-pack.c: move shallow update code out of fetch_pack()Nguyễn Thái Ngọc Duy, Nov 25, 2013
  23. 11/28 fetch-pack.h: one statement per bitfield declarationNguyễn Thái Ngọc Duy, Nov 25, 2013
  24. 12/28 clone: support remote shallow repositoryNguyễn Thái Ngọc Duy, Nov 25, 2013
  25. 13/28 fetch: support fetching from a shallow repositoryNguyễn Thái Ngọc Duy, Nov 25, 2013
  26. Eric SunshineNov 27, 2013
  27. 14/28 upload-pack: make sure deepening preserves shallow rootsNguyễn Thái Ngọc Duy, Nov 25, 2013
  28. 15/28 fetch: add --update-shallow to get refs that require updating .git/shallowNguyễn Thái Ngọc Duy, Nov 25, 2013
  29. Eric SunshineNov 27, 2013
  30. Duy NguyenNov 27, 2013
  31. Junio C HamanoNov 27, 2013
  32. 16/28 receive-pack: reorder some code in unpack()Nguyễn Thái Ngọc Duy, Nov 25, 2013
  33. Junio C HamanoDec 2, 2013
  34. 17/28 Support pushing from a shallow cloneNguyễn Thái Ngọc Duy, Nov 25, 2013
  35. Eric SunshineNov 26, 2013
  36. 18/28 New var GIT_SHALLOW_FILE to propagate --shallow-file to subprocessesNguyễn Thái Ngọc Duy, Nov 25, 2013
  37. 19/28 connected.c: add new variant that runs with --shallow-fileNguyễn Thái Ngọc Duy, Nov 25, 2013
  38. 20/28 receive-pack: allow pushing with new shallow rootsNguyễn Thái Ngọc Duy, Nov 25, 2013
  39. 21/28 send-pack: support pushing to a shallow cloneNguyễn Thái Ngọc Duy, Nov 25, 2013
  40. 22/28 remote-curl: pass ref SHA-1 to fetch-pack as wellNguyễn Thái Ngọc Duy, Nov 25, 2013
  41. 23/28 Support fetch/clone over httpNguyễn Thái Ngọc Duy, Nov 25, 2013
  42. 24/28 receive-pack: support pushing to a shallow clone via httpNguyễn Thái Ngọc Duy, Nov 25, 2013
  43. 25/28 send-pack: support pushing from a shallow clone via httpNguyễn Thái Ngọc Duy, Nov 25, 2013
  44. 26/28 git-clone.txt: remove shallow clone limitationsNguyễn Thái Ngọc Duy, Nov 25, 2013
  45. 27/28 clone: use git protocol for cloning shallow repo locallyNguyễn Thái Ngọc Duy, Nov 25, 2013
  46. Eric SunshineNov 27, 2013
  47. 28/28 prune: clean .git/shallow after pruning objectsNguyễn Thái Ngọc Duy, Nov 25, 2013
  48. 00/28 First class shallow cloneNguyễn Thái Ngọc Duy, Dec 5, 2013
  49. 01/28 transport.h: remove send_pack prototype, already defined in send-pack.hNguyễn Thái Ngọc Duy, Dec 5, 2013
  50. 02/28 Replace struct extra_have_objects with struct sha1_arrayNguyễn Thái Ngọc Duy, Dec 5, 2013
  51. 03/28 send-pack: forbid pushing from a shallow repositoryNguyễn Thái Ngọc Duy, Dec 5, 2013
  52. 04/28 clone: prevent --reference to a shallow repositoryNguyễn Thái Ngọc Duy, Dec 5, 2013
  53. 05/28 Make the sender advertise shallow commits to the receiverNguyễn Thái Ngọc Duy, Dec 5, 2013
  54. 06/28 connect.c: teach get_remote_heads to parse "shallow" linesNguyễn Thái Ngọc Duy, Dec 5, 2013
  55. 07/28 shallow.c: extend setup_*_shallow() to accept extra shallow commitsNguyễn Thái Ngọc Duy, Dec 5, 2013
  56. 08/28 shallow.c: the 8 steps to select new commits for .git/shallowNguyễn Thái Ngọc Duy, Dec 5, 2013
  57. 09/28 shallow.c: steps 6 and 7 to select new commits for .git/shallowNguyễn Thái Ngọc Duy, Dec 5, 2013
  58. 10/28 fetch-pack.c: move shallow update code out of fetch_pack()Nguyễn Thái Ngọc Duy, Dec 5, 2013
  59. 11/28 fetch-pack.h: one statement per bitfield declarationNguyễn Thái Ngọc Duy, Dec 5, 2013
  60. 12/28 clone: support remote shallow repositoryNguyễn Thái Ngọc Duy, Dec 5, 2013
  61. 13/28 fetch: support fetching from a shallow repositoryNguyễn Thái Ngọc Duy, Dec 5, 2013
  62. 14/28 upload-pack: make sure deepening preserves shallow rootsNguyễn Thái Ngọc Duy, Dec 5, 2013
  63. 15/28 fetch: add --update-shallow to accept refs that update .git/shallowNguyễn Thái Ngọc Duy, Dec 5, 2013
  64. 16/28 receive-pack: reorder some code in unpack()Nguyễn Thái Ngọc Duy, Dec 5, 2013
  65. 17/28 receive/send-pack: support pushing from a shallow cloneNguyễn Thái Ngọc Duy, Dec 5, 2013
  66. 18/28 New var GIT_SHALLOW_FILE to propagate --shallow-file to subprocessesNguyễn Thái Ngọc Duy, Dec 5, 2013
  67. 19/28 connected.c: add new variant that runs with --shallow-fileNguyễn Thái Ngọc Duy, Dec 5, 2013
  68. 20/28 receive-pack: allow pushes that update .git/shallowNguyễn Thái Ngọc Duy, Dec 5, 2013
  69. 21/28 send-pack: support pushing to a shallow cloneNguyễn Thái Ngọc Duy, Dec 5, 2013
  70. 22/28 remote-curl: pass ref SHA-1 to fetch-pack as wellNguyễn Thái Ngọc Duy, Dec 5, 2013
  71. 23/28 Support shallow fetch/clone over smart-httpNguyễn Thái Ngọc Duy, Dec 5, 2013
  72. Jeff KingJan 8, 2014
  73. t5537: fix incorrect expectation in test case 10Nguyễn Thái Ngọc Duy, Jan 8, 2014
  74. Jeff KingJan 9, 2014
  75. 24/28 receive-pack: support pushing to a shallow clone via httpNguyễn Thái Ngọc Duy, Dec 5, 2013
  76. 25/28 send-pack: support pushing from a shallow clone via httpNguyễn Thái Ngọc Duy, Dec 5, 2013
  77. 26/28 clone: use git protocol for cloning shallow repo locallyNguyễn Thái Ngọc Duy, Dec 5, 2013
  78. 27/28 prune: clean .git/shallow after pruning objectsNguyễn Thái Ngọc Duy, Dec 5, 2013
  79. 28/28 git-clone.txt: remove shallow clone limitationsNguyễn Thái Ngọc Duy, Dec 5, 2013

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.