git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH v2 15/25] upload-pack: add deepen-since to cut shallow repos based on time

From
Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Date
Feb 4, 2016, 09:03 UTC
Message-ID
<1454576641-29615-16-git-send-email-pclouds@gmail.com>
In-Reply-To
<1454576641-29615-1-git-send-email-pclouds@gmail.com>

This should allow the user to say "create a shallow clone containing the work from last year" (once the client side is fixed up, of course).

In theory deepen-since and deepen (aka --depth) can be used together to draw the shallow boundary (whether it's intersection or union is up to discussion, but if rev-list is used, it's likely intersection). However, because deepen goes with a custom commit walker, we can't mix the two yet.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
 Documentation/technical/pack-protocol.txt         |  3 +-
 Documentation/technical/protocol-capabilities.txt |  9 +++++
 upload-pack.c                                     | 45 ++++++++++++++++++++++-
 3 files changed, 54 insertions(+), 3 deletions(-)
diff --git a/Documentation/technical/pack-protocol.txt b/Documentation/technical/pack-protocol.txt
index c6977bb..9251df1 100644
--- a/Documentation/technical/pack-protocol.txt
+++ b/Documentation/technical/pack-protocol.txt
@@ -219,7 +219,8 @@ out of what the server said it could do with the first 'want' line.
 
   shallow-line      =  PKT-LINE("shallow" SP obj-id)
 
-  depth-request     =  PKT-LINE("deepen" SP depth)
+  depth-request     =  PKT-LINE("deepen" SP depth) /
+		       PKT-LINE("deepen-since" SP timestamp)
 
   first-want        =  PKT-LINE("want" SP obj-id SP capability-list)
   additional-want   =  PKT-LINE("want" SP obj-id)
diff --git a/Documentation/technical/protocol-capabilities.txt b/Documentation/technical/protocol-capabilities.txt
index eaab6b4..f08cc4e 100644
--- a/Documentation/technical/protocol-capabilities.txt
+++ b/Documentation/technical/protocol-capabilities.txt
@@ -179,6 +179,15 @@ This capability adds "deepen", "shallow" and "unshallow" commands to
 the  fetch-pack/upload-pack protocol so clients can request shallow
 clones.
 
+deepen-since
+------------
+
+This capability adds "deepen-since" command to fetch-pack/upload-pack
+protocol so the client can request shallow clones that are cut at a
+specific time, instead of depth. Internally it's equivalent of doing
+"rev-list --max-age=<timestamp>" on the server side. "deepen-since"
+cannot be used with "deepen".
+
 no-progress
 -----------
 
diff --git a/upload-pack.c b/upload-pack.c
index c8dafe8..794736c 100644
--- a/upload-pack.c
+++ b/upload-pack.c
@@ -14,6 +14,7 @@
 #include "sigchain.h"
 #include "version.h"
 #include "string-list.h"
+#include "argv-array.h"
 
 static const char upload_pack_usage[] = "git upload-pack [--strict] [--timeout=<n>] <dir>";
 
@@ -612,11 +613,25 @@ static void deepen(int depth, const struct object_array *shallows)
 	packet_flush(1);
 }
 
+static void deepen_by_rev_list(int ac, const char **av,
+			       struct object_array *shallows)
+{
+	struct commit_list *result;
+
+	result = get_shallow_commits_by_rev_list(ac, av, SHALLOW, NOT_SHALLOW);
+	send_shallow(result);
+	free_commit_list(result);
+	send_unshallow(shallows);
+	packet_flush(1);
+}
+
 static void receive_needs(void)
 {
 	struct object_array shallows = OBJECT_ARRAY_INIT;
 	int depth = 0;
 	int has_non_tip = 0;
+	unsigned long deepen_since = 0;
+	int deepen_rev_list = 0;
 
 	shallow_nr = 0;
 	for (;;) {
@@ -653,6 +668,16 @@ static void receive_needs(void)
 				die("Invalid deepen: %s", line);
 			continue;
 		}
+		if (skip_prefix(line, "deepen-since ", &arg)) {
+			char *end = NULL;
+			deepen_since = strtoul(arg, &end, 0);
+			if (!end || *end || !deepen_since ||
+			    /* revisions.c's max_age -1 is special */
+			    deepen_since == -1)
+				die("Invalid deepen-since: %s", line);
+			deepen_rev_list = 1;
+			continue;
+		}
 		if (!skip_prefix(line, "want ", &arg) ||
 		    get_sha1_hex(arg, sha1_buf))
 			die("git upload-pack: protocol error, "
@@ -704,10 +729,26 @@ static void receive_needs(void)
 	if (!use_sideband && daemon_mode)
 		no_progress = 1;
 
-	if (depth == 0 && shallows.nr == 0)
+	if (depth == 0 && !deepen_rev_list && shallows.nr == 0)
 		return;
+	if (depth > 0 && deepen_rev_list)
+		die("--depth and --shallow-since cannot be used together");
 	if (depth > 0)
 		deepen(depth, &shallows);
+	else if (deepen_rev_list) {
+		struct argv_array av = ARGV_ARRAY_INIT;
+		int i;
+
+		argv_array_push(&av, "rev-list");
+		if (deepen_since)
+			argv_array_pushf(&av, "--max-age=%lu", deepen_since);
+		for (i = 0; i < want_obj.nr; i++) {
+			struct object *o = want_obj.objects[i].item;
+			argv_array_push(&av, oid_to_hex(&o->oid));
+		}
+		deepen_by_rev_list(av.argc, av.argv, &shallows);
+		argv_array_clear(&av);
+	}
 	else
 		if (shallows.nr > 0) {
 			int i;
@@ -756,7 +797,7 @@ static int send_ref(const char *refname, const struct object_id *oid,
 		    int flag, void *cb_data)
 {
 	static const char *capabilities = "multi_ack thin-pack side-band"
-		" side-band-64k ofs-delta shallow no-progress"
+		" side-band-64k ofs-delta shallow deepen-since no-progress"
 		" include-tag multi_ack_detailed";
 	const char *refname_nons = strip_namespace(refname);
 	struct object_id peeled;
-- 
2.7.0.377.g4cd97dd
Previous: Duy NguyenNext: Junio C Hamano
Message 35 of 63 in “More flexibility in making shallow clones”
  1. 00/25 More flexibility in making shallow clonesNguyễn Thái Ngọc Duy, Feb 4, 2016
  2. 01/25 remote-curl.c: convert fetch_git() to use argv_arrayNguyễn Thái Ngọc Duy, Feb 4, 2016
  3. Junio C HamanoFeb 4, 2016
  4. 02/25 transport-helper.c: refactor set_helper_option()Nguyễn Thái Ngọc Duy, Feb 4, 2016
  5. Junio C HamanoFeb 4, 2016
  6. 03/25 transport-helper.c: do not send null option to remote helperNguyễn Thái Ngọc Duy, Feb 4, 2016
  7. Junio C HamanoFeb 4, 2016
  8. Duy NguyenFeb 6, 2016
  9. Junio C HamanoFeb 8, 2016
  10. 04/25 upload-pack: move shallow deepen code out of receive_needs()Nguyễn Thái Ngọc Duy, Feb 4, 2016
  11. Junio C HamanoFeb 4, 2016
  12. 05/25 upload-pack: move "shallow" sending code out of deepen()Nguyễn Thái Ngọc Duy, Feb 4, 2016
  13. 06/25 upload-pack: remove unused variable "backup"Nguyễn Thái Ngọc Duy, Feb 4, 2016
  14. Junio C HamanoFeb 4, 2016
  15. 07/25 upload-pack: move "unshallow" sending code out of deepen()Nguyễn Thái Ngọc Duy, Feb 4, 2016
  16. Junio C HamanoFeb 4, 2016
  17. 08/25 upload-pack: use skip_prefix() instead of starts_with() when possibleNguyễn Thái Ngọc Duy, Feb 4, 2016
  18. Junio C HamanoFeb 4, 2016
  19. 09/25 upload-pack: tighten number parsing at "deepen" linesNguyễn Thái Ngọc Duy, Feb 4, 2016
  20. Junio C HamanoFeb 4, 2016
  21. Duy NguyenFeb 15, 2016
  22. 10/25 upload-pack: move rev-list code out of check_non_tip()Nguyễn Thái Ngọc Duy, Feb 4, 2016
  23. 11/25 fetch-pack: use skip_prefix() instead of starts_with() when possibleNguyễn Thái Ngọc Duy, Feb 4, 2016
  24. Junio C HamanoFeb 4, 2016
  25. 12/25 fetch-pack: use a common function for verbose printingNguyễn Thái Ngọc Duy, Feb 4, 2016
  26. Junio C HamanoFeb 5, 2016
  27. Eric SunshineFeb 5, 2016
  28. 13/25 fetch-pack: use a separate flag for fetch in deepening modeNguyễn Thái Ngọc Duy, Feb 4, 2016
  29. Junio C HamanoFeb 5, 2016
  30. Eric SunshineFeb 5, 2016
  31. 14/25 shallow.c: implement a generic shallow boundary finder based on rev-listNguyễn Thái Ngọc Duy, Feb 4, 2016
  32. Junio C HamanoFeb 8, 2016
  33. Duy NguyenFeb 15, 2016
  34. Duy NguyenFeb 19, 2016
  35. 15/25 upload-pack: add deepen-since to cut shallow repos based on timeNguyễn Thái Ngọc Duy, Feb 4, 2016
  36. Junio C HamanoFeb 8, 2016
  37. 16/25 fetch: define shallow boundary with --shallow-sinceNguyễn Thái Ngọc Duy, Feb 4, 2016
  38. 17/25 clone: define shallow clone boundary based on time with --shallow-sinceNguyễn Thái Ngọc Duy, Feb 4, 2016
  39. Junio C HamanoFeb 8, 2016
  40. 18/25 t5500, t5539: tests for shallow depth since a specific dateNguyễn Thái Ngọc Duy, Feb 4, 2016
  41. Junio C HamanoFeb 8, 2016
  42. Duy NguyenFeb 15, 2016
  43. 19/25 refs: add expand_ref()Nguyễn Thái Ngọc Duy, Feb 4, 2016
  44. Junio C HamanoFeb 8, 2016
  45. 20/25 upload-pack: support define shallow boundary by excluding revisionsNguyễn Thái Ngọc Duy, Feb 4, 2016
  46. Eric SunshineFeb 5, 2016
  47. Junio C HamanoFeb 8, 2016
  48. Eric SunshineFeb 5, 2016
  49. Duy NguyenFeb 15, 2016
  50. 21/25 fetch: define shallow boundary with --shallow-excludeNguyễn Thái Ngọc Duy, Feb 4, 2016
  51. Eric SunshineFeb 5, 2016
  52. Duy NguyenFeb 15, 2016
  53. Eric SunshineFeb 15, 2016
  54. Eric SunshineFeb 15, 2016
  55. Duy NguyenFeb 15, 2016
  56. Eric SunshineFeb 19, 2016
  57. 22/25 clone: define shallow clone boundary with --shallow-excludeNguyễn Thái Ngọc Duy, Feb 4, 2016
  58. 23/25 t5500, t5539: tests for shallow depth excluding a refNguyễn Thái Ngọc Duy, Feb 4, 2016
  59. 24/25 upload-pack: make check_reachable_object() return unreachable list if askedNguyễn Thái Ngọc Duy, Feb 4, 2016
  60. Eric SunshineFeb 5, 2016
  61. 25/25 fetch, upload-pack: --deepen=N extends shallow boundary by N commitsNguyễn Thái Ngọc Duy, Feb 4, 2016
  62. Junio C HamanoFeb 8, 2016
  63. Duy NguyenFeb 12, 2016

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.