git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH v3 00/28] First class shallow clone

From
Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Date
Nov 25, 2013, 03:55 UTC
Message-ID
<1385351754-9954-1-git-send-email-pclouds@gmail.com>

Compared to v2 [1], v3 grows a bit. The biggest difference is .git/shallow is not updated by default (except when you clone from a shallow repository). When you send something, the "safe" refs that do not need new shallow roots are accepted at the receiver end, the others rejected.

To accept those other refs, either use "fetch --update-shallow" or enable receive.shallowupdate on the receiver side of a push.

Filtering "safe" refs requires walking through some commits, so it'll be more expensive than normal full clones. This is especially true for the receiver of a push (see 07/28 and 17/28). I envision shallow repos are used as upstream to archive old history, so this is not a good news. Commit cache (or pack v4) might help. We might even be able to move some work from receive-pack to send-pack to reduce server load..

[1] http://mid.gmane.org/1374314290-5976-1-git-send-email-pclouds@gmail.com
Nguyễn Thái Ngọc Duy (28):
  transport.h: remove send_pack prototype, already defined in send-pack.h
  send-pack: forbid pushing from a shallow repository
  clone: prevent --reference to a shallow repository
  update-server-info: do not publish shallow clones
   This part is just cleanup.
  Advertise shallow graft information on the server end
  connect.c: teach get_remote_heads to parse "shallow" lines
  shallow.c: add remove_reachable_shallow_points()
  shallow.c: add mark_new_shallow_refs()
  shallow.c: extend setup_*_shallow() to accept extra shallow points
  fetch-pack.c: move shallow update code out of fetch_pack()
  fetch-pack.h: one statement per bitfield declaration
  clone: support remote shallow repository
  fetch: support fetching from a shallow repository
  upload-pack: make sure deepening preserves shallow roots
  fetch: add --update-shallow to get refs that require updating .git/shallow
   Basic shallow fetch/clone support on git protocol
  receive-pack: reorder some code in unpack()
  Support pushing from a shallow clone
  New var GIT_SHALLOW_FILE to propagate --shallow-file to subprocesses
  connected.c: add new variant that runs with --shallow-file
  receive-pack: allow pushing with new shallow roots
  send-pack: support pushing to a shallow clone
  remote-curl: pass ref SHA-1 to fetch-pack as well
  
   Push support
  Support fetch/clone over http
  receive-pack: support pushing to a shallow clone via http
  send-pack: support pushing from a shallow clone via http
   smart-http support
  git-clone.txt: remove shallow clone limitations
  clone: use git protocol for cloning shallow repo locally
  prune: clean .git/shallow after pruning objects
   miscellaneous
 Documentation/config.txt                  |   4 +
 Documentation/fetch-options.txt           |  14 +-
 Documentation/git-clone.txt               |   7 +-
 Documentation/gitremote-helpers.txt       |   7 +
 Documentation/technical/pack-protocol.txt |   7 +-
 builtin/clone.c                           |  18 +-
 builtin/fetch-pack.c                      |  23 +-
 builtin/fetch.c                           |  15 +-
 builtin/gc.c                              |   1 +
 builtin/prune.c                           |   4 +
 builtin/receive-pack.c                    | 248 +++++++++++++++++----
 builtin/send-pack.c                       |   5 +-
 cache.h                                   |   1 +
 commit.h                                  |  19 +-
 connect.c                                 |  14 +-
 connected.c                               |  42 +++-
 connected.h                               |   2 +
 environment.c                             |   6 +
 fetch-pack.c                              | 132 ++++++++++--
 fetch-pack.h                              |  29 +--
 git.c                                     |   2 +-
 remote-curl.c                             |  33 ++-
 remote.h                                  |   5 +-
 send-pack.c                               |  20 +-
 server-info.c                             |   9 +
 shallow.c                                 | 348 +++++++++++++++++++++++++++++-
 t/t5536-fetch-shallow.sh (new +x)         | 193 +++++++++++++++++
 t/t5537-push-shallow.sh (new +x)          | 184 ++++++++++++++++
 t/t5601-clone.sh                          |   7 +
 transport-helper.c                        |   6 +
 transport.c                               |  22 +-
 transport.h                               |  16 +-
 upload-pack.c                             |   8 +-
 33 files changed, 1323 insertions(+), 128 deletions(-)
 create mode 100755 t/t5536-fetch-shallow.sh
 create mode 100755 t/t5537-push-shallow.sh
-- 
1.8.2.83.gc99314b
Next: Nguyễn Thái Ngọc Duy
Message 1 of 79 in “First class shallow clone”
  1. 00/28 First class shallow cloneNguyễn Thái Ngọc Duy, Nov 25, 2013
  2. 01/28 transport.h: remove send_pack prototype, already defined in send-pack.hNguyễn Thái Ngọc Duy, Nov 25, 2013
  3. 02/28 send-pack: forbid pushing from a shallow repositoryNguyễn Thái Ngọc Duy, Nov 25, 2013
  4. 03/28 clone: prevent --reference to a shallow repositoryNguyễn Thái Ngọc Duy, Nov 25, 2013
  5. Eric SunshineNov 26, 2013
  6. 04/28 update-server-info: do not publish shallow clonesNguyễn Thái Ngọc Duy, Nov 25, 2013
  7. Junio C HamanoNov 25, 2013
  8. Duy NguyenNov 26, 2013
  9. 05/28 Advertise shallow graft information on the server endNguyễn Thái Ngọc Duy, Nov 25, 2013
  10. 06/28 connect.c: teach get_remote_heads to parse "shallow" linesNguyễn Thái Ngọc Duy, Nov 25, 2013
  11. Junio C HamanoNov 25, 2013
  12. Junio C HamanoNov 25, 2013
  13. Duy NguyenNov 27, 2013
  14. 07/28 shallow.c: add remove_reachable_shallow_points()Nguyễn Thái Ngọc Duy, Nov 25, 2013
  15. Junio C HamanoNov 25, 2013
  16. 08/28 shallow.c: add mark_new_shallow_refs()Nguyễn Thái Ngọc Duy, Nov 25, 2013
  17. Junio C HamanoNov 25, 2013
  18. Duy NguyenNov 26, 2013
  19. Junio C HamanoNov 26, 2013
  20. 09/28 shallow.c: extend setup_*_shallow() to accept extra shallow pointsNguyễn Thái Ngọc Duy, Nov 25, 2013
  21. Junio C HamanoNov 25, 2013
  22. 10/28 fetch-pack.c: move shallow update code out of fetch_pack()Nguyễn Thái Ngọc Duy, Nov 25, 2013
  23. 11/28 fetch-pack.h: one statement per bitfield declarationNguyễn Thái Ngọc Duy, Nov 25, 2013
  24. 12/28 clone: support remote shallow repositoryNguyễn Thái Ngọc Duy, Nov 25, 2013
  25. 13/28 fetch: support fetching from a shallow repositoryNguyễn Thái Ngọc Duy, Nov 25, 2013
  26. Eric SunshineNov 27, 2013
  27. 14/28 upload-pack: make sure deepening preserves shallow rootsNguyễn Thái Ngọc Duy, Nov 25, 2013
  28. 15/28 fetch: add --update-shallow to get refs that require updating .git/shallowNguyễn Thái Ngọc Duy, Nov 25, 2013
  29. Eric SunshineNov 27, 2013
  30. Duy NguyenNov 27, 2013
  31. Junio C HamanoNov 27, 2013
  32. 16/28 receive-pack: reorder some code in unpack()Nguyễn Thái Ngọc Duy, Nov 25, 2013
  33. Junio C HamanoDec 2, 2013
  34. 17/28 Support pushing from a shallow cloneNguyễn Thái Ngọc Duy, Nov 25, 2013
  35. Eric SunshineNov 26, 2013
  36. 18/28 New var GIT_SHALLOW_FILE to propagate --shallow-file to subprocessesNguyễn Thái Ngọc Duy, Nov 25, 2013
  37. 19/28 connected.c: add new variant that runs with --shallow-fileNguyễn Thái Ngọc Duy, Nov 25, 2013
  38. 20/28 receive-pack: allow pushing with new shallow rootsNguyễn Thái Ngọc Duy, Nov 25, 2013
  39. 21/28 send-pack: support pushing to a shallow cloneNguyễn Thái Ngọc Duy, Nov 25, 2013
  40. 22/28 remote-curl: pass ref SHA-1 to fetch-pack as wellNguyễn Thái Ngọc Duy, Nov 25, 2013
  41. 23/28 Support fetch/clone over httpNguyễn Thái Ngọc Duy, Nov 25, 2013
  42. 24/28 receive-pack: support pushing to a shallow clone via httpNguyễn Thái Ngọc Duy, Nov 25, 2013
  43. 25/28 send-pack: support pushing from a shallow clone via httpNguyễn Thái Ngọc Duy, Nov 25, 2013
  44. 26/28 git-clone.txt: remove shallow clone limitationsNguyễn Thái Ngọc Duy, Nov 25, 2013
  45. 27/28 clone: use git protocol for cloning shallow repo locallyNguyễn Thái Ngọc Duy, Nov 25, 2013
  46. Eric SunshineNov 27, 2013
  47. 28/28 prune: clean .git/shallow after pruning objectsNguyễn Thái Ngọc Duy, Nov 25, 2013
  48. 00/28 First class shallow cloneNguyễn Thái Ngọc Duy, Dec 5, 2013
  49. 01/28 transport.h: remove send_pack prototype, already defined in send-pack.hNguyễn Thái Ngọc Duy, Dec 5, 2013
  50. 02/28 Replace struct extra_have_objects with struct sha1_arrayNguyễn Thái Ngọc Duy, Dec 5, 2013
  51. 03/28 send-pack: forbid pushing from a shallow repositoryNguyễn Thái Ngọc Duy, Dec 5, 2013
  52. 04/28 clone: prevent --reference to a shallow repositoryNguyễn Thái Ngọc Duy, Dec 5, 2013
  53. 05/28 Make the sender advertise shallow commits to the receiverNguyễn Thái Ngọc Duy, Dec 5, 2013
  54. 06/28 connect.c: teach get_remote_heads to parse "shallow" linesNguyễn Thái Ngọc Duy, Dec 5, 2013
  55. 07/28 shallow.c: extend setup_*_shallow() to accept extra shallow commitsNguyễn Thái Ngọc Duy, Dec 5, 2013
  56. 08/28 shallow.c: the 8 steps to select new commits for .git/shallowNguyễn Thái Ngọc Duy, Dec 5, 2013
  57. 09/28 shallow.c: steps 6 and 7 to select new commits for .git/shallowNguyễn Thái Ngọc Duy, Dec 5, 2013
  58. 10/28 fetch-pack.c: move shallow update code out of fetch_pack()Nguyễn Thái Ngọc Duy, Dec 5, 2013
  59. 11/28 fetch-pack.h: one statement per bitfield declarationNguyễn Thái Ngọc Duy, Dec 5, 2013
  60. 12/28 clone: support remote shallow repositoryNguyễn Thái Ngọc Duy, Dec 5, 2013
  61. 13/28 fetch: support fetching from a shallow repositoryNguyễn Thái Ngọc Duy, Dec 5, 2013
  62. 14/28 upload-pack: make sure deepening preserves shallow rootsNguyễn Thái Ngọc Duy, Dec 5, 2013
  63. 15/28 fetch: add --update-shallow to accept refs that update .git/shallowNguyễn Thái Ngọc Duy, Dec 5, 2013
  64. 16/28 receive-pack: reorder some code in unpack()Nguyễn Thái Ngọc Duy, Dec 5, 2013
  65. 17/28 receive/send-pack: support pushing from a shallow cloneNguyễn Thái Ngọc Duy, Dec 5, 2013
  66. 18/28 New var GIT_SHALLOW_FILE to propagate --shallow-file to subprocessesNguyễn Thái Ngọc Duy, Dec 5, 2013
  67. 19/28 connected.c: add new variant that runs with --shallow-fileNguyễn Thái Ngọc Duy, Dec 5, 2013
  68. 20/28 receive-pack: allow pushes that update .git/shallowNguyễn Thái Ngọc Duy, Dec 5, 2013
  69. 21/28 send-pack: support pushing to a shallow cloneNguyễn Thái Ngọc Duy, Dec 5, 2013
  70. 22/28 remote-curl: pass ref SHA-1 to fetch-pack as wellNguyễn Thái Ngọc Duy, Dec 5, 2013
  71. 23/28 Support shallow fetch/clone over smart-httpNguyễn Thái Ngọc Duy, Dec 5, 2013
  72. Jeff KingJan 8, 2014
  73. t5537: fix incorrect expectation in test case 10Nguyễn Thái Ngọc Duy, Jan 8, 2014
  74. Jeff KingJan 9, 2014
  75. 24/28 receive-pack: support pushing to a shallow clone via httpNguyễn Thái Ngọc Duy, Dec 5, 2013
  76. 25/28 send-pack: support pushing from a shallow clone via httpNguyễn Thái Ngọc Duy, Dec 5, 2013
  77. 26/28 clone: use git protocol for cloning shallow repo locallyNguyễn Thái Ngọc Duy, Dec 5, 2013
  78. 27/28 prune: clean .git/shallow after pruning objectsNguyễn Thái Ngọc Duy, Dec 5, 2013
  79. 28/28 git-clone.txt: remove shallow clone limitationsNguyễn Thái Ngọc Duy, Dec 5, 2013

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.