git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH 6/9] fetch: Check if all objects exist after fetching

From
AGAndreas Gruenbacher <agruen@suse.de>
Date
Mar 16, 2010, 18:48 UTC
Message-ID
<d3c00e2a27003dca196d5480007544610cc1e5b8.1268913163.git.agruen@suse.de>
In-Reply-To
<dbb0f92deca35a16c898af697dc41affc377ae0b.1268913163.git.agruen@suse.de>

Check if all objects reachable from the fetched refs exist after fetching instead of before: this allows us to distinguish between a repository which is not up to date and a corrupted repository, and to ensure that the repository is up to date and complete after the fetch.

In select_missing_refs() we must check which objects we have, instead of checking which refs are up to date: otherwise, we might fetch objects into the repository which exist in an alternate.

Signed-off-by: Andreas Gruenbacher <agruen@suse.de>
---
 builtin/fetch.c |   82 ++++++++++++++++++++++++++-----------------------------
 1 files changed, 39 insertions(+), 43 deletions(-)
diff --git a/builtin/fetch.c b/builtin/fetch.c
index 6dc75d4..0f292b8 100644
--- a/builtin/fetch.c
+++ b/builtin/fetch.c
@@ -440,20 +440,15 @@ static int store_updated_refs(const char *raw_url, const char *remote_name,
 }
 
 /*
- * We would want to bypass the object transfer altogether if
- * everything we are going to fetch already exists and is connected
- * locally.
- *
- * The refs we are going to fetch are in ref_map.  If running
+ * If running
  *
  *  $ git rev-list --objects --stdin --not --all
  *
  * (feeding all the refs in ref_map on its standard input)
  * does not error out, that means everything reachable from the
- * refs we are going to fetch exists and is connected to some of
- * our existing refs.
+ * refs we have fetched exists.
  */
-static int quickfetch(struct ref *ref_map)
+static int check_connectivity(struct ref *ref_map)
 {
 	struct child_process revlist;
 	struct ref *ref;
@@ -461,16 +456,6 @@ static int quickfetch(struct ref *ref_map)
 	const char *argv[] = {"rev-list",
 		"--quiet", "--objects", "--stdin", "--not", "--all", NULL};
 
-	/*
-	 * If we are deepening a shallow clone we already have these
-	 * objects reachable.  Running rev-list here will return with
-	 * a good (0) exit status and we'll bypass the fetch that we
-	 * really need to perform.  Claiming failure now will ensure
-	 * we perform the network exchange to deepen our history.
-	 */
-	if (depth)
-		return -1;
-
 	if (!ref_map)
 		return 0;
 
@@ -513,46 +498,57 @@ static int quickfetch(struct ref *ref_map)
 	return finish_command(&revlist) || err;
 }
 
-static void check_existing_refs(struct ref *ref_map) {
+static void select_missing_refs(struct ref *ref_map) {
 	struct ref *ref;
-	int done = 1;
 
-	if (!depth) {
-		for (ref = ref_map; ref; ref = ref->next) {
-			if (ref->peer_ref &&
-			    !is_null_sha1(ref->old_sha1) &&
-			    !hashcmp(ref->peer_ref->old_sha1, ref->old_sha1))
-				ref->dont_fetch = 1;
-			else
-				done = 0;
-		}
+	for (ref = ref_map; ref; ref = ref->next) {
+		if (has_sha1_file(ref->old_sha1))
+			ref->dont_fetch = 1;
 	}
-	if (done) {
-		/*
-		 * When deepening of a shallow repository is requested, local
-		 * and remote refs are likely to still be equal.  In addition,
-		 * we get here if all refs are up to date but quickfetch() did
-		 * not succeed.  In both cases, fetch all refs.
-		 */
-		for (ref = ref_map; ref; ref = ref->next)
+}
+
+static int select_all_refs(struct ref *ref_map) {
+	struct ref *ref;
+	int done = 1;
+
+	for (ref = ref_map; ref; ref = ref->next) {
+		if (ref->dont_fetch) {
 			ref->dont_fetch = 0;
+			done = 0;
+		}
 	}
+
+	return done;
 }
 
 static int fetch_refs(struct transport *transport, struct ref *ref_map)
 {
 	int ret;
 
-	ret = quickfetch(ref_map);
+	if (depth) {
+		/* When deepening a shallow clone, re-fetch all the refs:
+		 * it doesn't help to have the topmost commits of each ref.
+		 */
+		select_all_refs(ref_map);
+	} else
+		select_missing_refs(ref_map);
+	ret = transport_fetch_refs(transport, ref_map);
+	if (ret)
+		return ret;
+	ret = check_connectivity(ref_map);
 	if (ret) {
-		check_existing_refs(ref_map);
+		if (select_all_refs(ref_map))
+			return ret;
+		transport_unlock_pack(transport);
 		ret = transport_fetch_refs(transport, ref_map);
+		if (ret)
+			return ret;
 	}
-	if (!ret)
-		ret |= store_updated_refs(transport->url,
-				transport->remote->name,
-				ref_map);
+	ret = store_updated_refs(transport->url,
+				 transport->remote->name,
+				 ref_map);
 	transport_unlock_pack(transport);
+
 	return ret;
 }
 
-- 
1.7.0.2.273.gc2413
Previous: Andreas GruenbacherNext: Andreas Gruenbacher
Message 7 of 14 in “Multiple remotes without conflicts”
  1. 0/9 Multiple remotes without conflictsAndreas Gruenbacher, Mar 18, 2010
  2. 1/9 fetch: Check for a "^{}" suffix with suffixcmp()Andreas Gruenbacher, Mar 13, 2010
  3. 2/9 fetch: Properly initialize refspec on stackAndreas Gruenbacher, Mar 12, 2010
  4. 3/9 fetch: Fix minor memory leakAndreas Gruenbacher, Mar 15, 2010
  5. 4/9 fetch: Move deepening fetch check into builtin/fetch.cAndreas Gruenbacher, Mar 16, 2010
  6. 5/9 fetch: Move loop checking which refs we have alreadyAndreas Gruenbacher, Mar 16, 2010
  7. 6/9 fetch: Check if all objects exist after fetchingAndreas Gruenbacher, Mar 16, 2010
  8. 7/9 fetch: Use the same ref map for all branches and tagsAndreas Gruenbacher, Mar 17, 2010
  9. 8/9 fetch: Don't fetch tags twiceAndreas Gruenbacher, Mar 17, 2010
  10. 9/9 fetch: Make automatic tag following work with arbitrary refspecsAndreas Gruenbacher, Mar 17, 2010
  11. Shawn O. PearceMar 18, 2010
  12. Andreas GruenbacherMar 18, 2010
  13. Shawn O. PearceMar 18, 2010
  14. Andreas GruenbacherMar 18, 2010

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.