git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH 15/26] upload-pack: add deepen-since to cut shallow repos based on time

From
Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Date
Apr 13, 2016, 12:54 UTC
Message-ID
<1460552110-5554-16-git-send-email-pclouds@gmail.com>
In-Reply-To
<1460552110-5554-1-git-send-email-pclouds@gmail.com>

This should allow the user to say "create a shallow clone containing the work from last year" (once the client side is fixed up, of course).

In theory deepen-since and deepen (aka --depth) can be used together to draw the shallow boundary (whether it's intersection or union is up to discussion, but if rev-list is used, it's likely intersection). However, because deepen goes with a custom commit walker, we can't mix the two yet.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
 Documentation/technical/pack-protocol.txt         |  3 +-
 Documentation/technical/protocol-capabilities.txt |  9 +++++
 upload-pack.c                                     | 45 ++++++++++++++++++++++-
 3 files changed, 54 insertions(+), 3 deletions(-)
diff --git a/Documentation/technical/pack-protocol.txt b/Documentation/technical/pack-protocol.txt
index c6977bb..9251df1 100644
--- a/Documentation/technical/pack-protocol.txt
+++ b/Documentation/technical/pack-protocol.txt
@@ -219,7 +219,8 @@ out of what the server said it could do with the first 'want' line.
 
   shallow-line      =  PKT-LINE("shallow" SP obj-id)
 
-  depth-request     =  PKT-LINE("deepen" SP depth)
+  depth-request     =  PKT-LINE("deepen" SP depth) /
+		       PKT-LINE("deepen-since" SP timestamp)
 
   first-want        =  PKT-LINE("want" SP obj-id SP capability-list)
   additional-want   =  PKT-LINE("want" SP obj-id)
diff --git a/Documentation/technical/protocol-capabilities.txt b/Documentation/technical/protocol-capabilities.txt
index eaab6b4..f08cc4e 100644
--- a/Documentation/technical/protocol-capabilities.txt
+++ b/Documentation/technical/protocol-capabilities.txt
@@ -179,6 +179,15 @@ This capability adds "deepen", "shallow" and "unshallow" commands to
 the  fetch-pack/upload-pack protocol so clients can request shallow
 clones.
 
+deepen-since
+------------
+
+This capability adds "deepen-since" command to fetch-pack/upload-pack
+protocol so the client can request shallow clones that are cut at a
+specific time, instead of depth. Internally it's equivalent of doing
+"rev-list --max-age=<timestamp>" on the server side. "deepen-since"
+cannot be used with "deepen".
+
 no-progress
 -----------
 
diff --git a/upload-pack.c b/upload-pack.c
index c8dafe8..58c0936 100644
--- a/upload-pack.c
+++ b/upload-pack.c
@@ -14,6 +14,7 @@
 #include "sigchain.h"
 #include "version.h"
 #include "string-list.h"
+#include "argv-array.h"
 
 static const char upload_pack_usage[] = "git upload-pack [--strict] [--timeout=<n>] <dir>";
 
@@ -612,11 +613,25 @@ static void deepen(int depth, const struct object_array *shallows)
 	packet_flush(1);
 }
 
+static void deepen_by_rev_list(int ac, const char **av,
+			       struct object_array *shallows)
+{
+	struct commit_list *result;
+
+	result = get_shallow_commits_by_rev_list(ac, av, SHALLOW, NOT_SHALLOW);
+	send_shallow(result);
+	free_commit_list(result);
+	send_unshallow(shallows);
+	packet_flush(1);
+}
+
 static void receive_needs(void)
 {
 	struct object_array shallows = OBJECT_ARRAY_INIT;
 	int depth = 0;
 	int has_non_tip = 0;
+	unsigned long deepen_since = 0;
+	int deepen_rev_list = 0;
 
 	shallow_nr = 0;
 	for (;;) {
@@ -653,6 +668,16 @@ static void receive_needs(void)
 				die("Invalid deepen: %s", line);
 			continue;
 		}
+		if (skip_prefix(line, "deepen-since ", &arg)) {
+			char *end = NULL;
+			deepen_since = strtoul(arg, &end, 0);
+			if (!end || *end || !deepen_since ||
+			    /* revisions.c's max_age -1 is special */
+			    deepen_since == -1)
+				die("Invalid deepen-since: %s", line);
+			deepen_rev_list = 1;
+			continue;
+		}
 		if (!skip_prefix(line, "want ", &arg) ||
 		    get_sha1_hex(arg, sha1_buf))
 			die("git upload-pack: protocol error, "
@@ -704,10 +729,26 @@ static void receive_needs(void)
 	if (!use_sideband && daemon_mode)
 		no_progress = 1;
 
-	if (depth == 0 && shallows.nr == 0)
+	if (depth == 0 && !deepen_rev_list && shallows.nr == 0)
 		return;
+	if (depth > 0 && deepen_rev_list)
+		die("git upload-pack: deepen and deepen-since cannot be used together");
 	if (depth > 0)
 		deepen(depth, &shallows);
+	else if (deepen_rev_list) {
+		struct argv_array av = ARGV_ARRAY_INIT;
+		int i;
+
+		argv_array_push(&av, "rev-list");
+		if (deepen_since)
+			argv_array_pushf(&av, "--max-age=%lu", deepen_since);
+		for (i = 0; i < want_obj.nr; i++) {
+			struct object *o = want_obj.objects[i].item;
+			argv_array_push(&av, oid_to_hex(&o->oid));
+		}
+		deepen_by_rev_list(av.argc, av.argv, &shallows);
+		argv_array_clear(&av);
+	}
 	else
 		if (shallows.nr > 0) {
 			int i;
@@ -756,7 +797,7 @@ static int send_ref(const char *refname, const struct object_id *oid,
 		    int flag, void *cb_data)
 {
 	static const char *capabilities = "multi_ack thin-pack side-band"
-		" side-band-64k ofs-delta shallow no-progress"
+		" side-band-64k ofs-delta shallow deepen-since no-progress"
 		" include-tag multi_ack_detailed";
 	const char *refname_nons = strip_namespace(refname);
 	struct object_id peeled;
-- 
2.8.0.rc0.210.gd302cd2
Previous: Eric SunshineNext: Nguyễn Thái Ngọc Duy
Message 21 of 40 in “nd/shallow-deepen updates”
  1. 00/26 nd/shallow-deepen updatesNguyễn Thái Ngọc Duy, Apr 13, 2016
  2. 01/26 remote-curl.c: convert fetch_git() to use argv_arrayNguyễn Thái Ngọc Duy, Apr 13, 2016
  3. Eric SunshineMay 25, 2016
  4. 02/26 transport-helper.c: refactor set_helper_option()Nguyễn Thái Ngọc Duy, Apr 13, 2016
  5. Eric SunshineMay 25, 2016
  6. 03/26 upload-pack: move shallow deepen code out of receive_needs()Nguyễn Thái Ngọc Duy, Apr 13, 2016
  7. 04/26 upload-pack: move "shallow" sending code out of deepen()Nguyễn Thái Ngọc Duy, Apr 13, 2016
  8. 05/26 upload-pack: remove unused variable "backup"Nguyễn Thái Ngọc Duy, Apr 13, 2016
  9. 06/26 upload-pack: move "unshallow" sending code out of deepen()Nguyễn Thái Ngọc Duy, Apr 13, 2016
  10. 07/26 upload-pack: use skip_prefix() instead of starts_with()Nguyễn Thái Ngọc Duy, Apr 13, 2016
  11. Eric SunshineMay 25, 2016
  12. 08/26 upload-pack: tighten number parsing at "deepen" linesNguyễn Thái Ngọc Duy, Apr 13, 2016
  13. 09/26 upload-pack: move rev-list code out of check_non_tip()Nguyễn Thái Ngọc Duy, Apr 13, 2016
  14. Eric SunshineMay 25, 2016
  15. 10/26 fetch-pack: use skip_prefix() instead of starts_with()Nguyễn Thái Ngọc Duy, Apr 13, 2016
  16. 11/26 fetch-pack: use a common function for verbose printingNguyễn Thái Ngọc Duy, Apr 13, 2016
  17. 12/26 fetch-pack.c: mark strings for translatingNguyễn Thái Ngọc Duy, Apr 13, 2016
  18. 13/26 fetch-pack: use a separate flag for fetch in deepening modeNguyễn Thái Ngọc Duy, Apr 13, 2016
  19. 14/26 shallow.c: implement a generic shallow boundary finder based on rev-listNguyễn Thái Ngọc Duy, Apr 13, 2016
  20. Eric SunshineMay 27, 2016
  21. 15/26 upload-pack: add deepen-since to cut shallow repos based on timeNguyễn Thái Ngọc Duy, Apr 13, 2016
  22. 16/26 fetch: define shallow boundary with --shallow-sinceNguyễn Thái Ngọc Duy, Apr 13, 2016
  23. 17/26 clone: define shallow clone boundary based on time with --shallow-sinceNguyễn Thái Ngọc Duy, Apr 13, 2016
  24. 18/26 t5500, t5539: tests for shallow depth since a specific dateNguyễn Thái Ngọc Duy, Apr 13, 2016
  25. Eric SunshineJun 5, 2016
  26. 19/26 refs: add expand_ref()Nguyễn Thái Ngọc Duy, Apr 13, 2016
  27. 20/26 upload-pack: support define shallow boundary by excluding revisionsNguyễn Thái Ngọc Duy, Apr 13, 2016
  28. 21/26 fetch: define shallow boundary with --shallow-excludeNguyễn Thái Ngọc Duy, Apr 13, 2016
  29. Eric SunshineJun 5, 2016
  30. 22/26 clone: define shallow clone boundary with --shallow-excludeNguyễn Thái Ngọc Duy, Apr 13, 2016
  31. Eric SunshineJun 5, 2016
  32. 23/26 t5500, t5539: tests for shallow depth excluding a refNguyễn Thái Ngọc Duy, Apr 13, 2016
  33. Eric SunshineJun 5, 2016
  34. 24/26 upload-pack: split check_unreachable() in two, prep for get_reachable_list()Nguyễn Thái Ngọc Duy, Apr 13, 2016
  35. Eric SunshineJun 5, 2016
  36. Duy NguyenJun 10, 2016
  37. 25/26 upload-pack: add get_reachable_list()Nguyễn Thái Ngọc Duy, Apr 13, 2016
  38. Eric SunshineJun 5, 2016
  39. 26/26 fetch, upload-pack: --deepen=N extends shallow boundary by N commitsNguyễn Thái Ngọc Duy, Apr 13, 2016
  40. Junio C HamanoApr 14, 2016

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.