git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH 9/9] clone: run check_everything_connected

From
Jeff King <peff@peff.net>
Date
Mar 25, 2013, 20:26 UTC
Message-ID
<20130325202627.GI16019@sigill.intra.peff.net>
In-Reply-To
<20130325201427.GA15798@sigill.intra.peff.net>

When we fetch from a remote, we do a revision walk to make sure that what we received is connected to our existing history. We do not do the same check for clone, which should be able to check that we received an intact history graph.

The upside of this patch is that it will make clone more resilient against propagating repository corruption. The downside is that we will now traverse "rev-list --objects --all" down to the roots, which may take some time (it is especially noticeable for a "--local --bare" clone).

Note that we need to adjust t5710, which tries to make such a bogus clone. Rather than checking after the fact that our clone is bogus, we can simplify it to just make sure "git clone" reports failure.

Signed-off-by: Jeff King <peff@peff.net>
---
The slowdown is really quite terrible if you try "git clone --bare
linux-2.6.git". Even with this, the local-clone case already misses blob
corruption. So it probably makes sense to restrict it to just the
non-local clone case, which already has to do more work.

Even still, it adds a non-trivial amount of work (linux-2.6 takes something like a minute to check). I don't like the idea of declaring "git clone" non-safe unless you turn on transfer.fsckObjects, though. It should have the same safety as "git fetch".

 builtin/clone.c              | 26 ++++++++++++++++++++++++++
 t/t1060-object-corruption.sh |  2 +-
 t/t5710-info-alternate.sh    |  8 +-------
 3 files changed, 28 insertions(+), 8 deletions(-)
diff --git a/builtin/clone.c b/builtin/clone.c
index 7d48ef3..eceaa74 100644
--- a/builtin/clone.c
+++ b/builtin/clone.c
@@ -23,6 +23,7 @@
 #include "branch.h"
 #include "remote.h"
 #include "run-command.h"
+#include "connected.h"
 
 /*
  * Overall FIXMEs:
@@ -485,12 +486,37 @@ static void update_remote_refs(const struct ref *refs,
 	}
 }
 
+static int iterate_ref_map(void *cb_data, unsigned char sha1[20])
+{
+	struct ref **rm = cb_data;
+	struct ref *ref = *rm;
+
+	/*
+	 * Skip anything missing a peer_ref, which we are not
+	 * actually going to write a ref for.
+	 */
+	while (ref && !ref->peer_ref)
+		ref = ref->next;
+	/* Returning -1 notes "end of list" to the caller. */
+	if (!ref)
+		return -1;
+
+	hashcpy(sha1, ref->old_sha1);
+	*rm = ref->next;
+	return 0;
+}
+
 static void update_remote_refs(const struct ref *refs,
 			       const struct ref *mapped_refs,
 			       const struct ref *remote_head_points_at,
 			       const char *branch_top,
 			       const char *msg)
 {
+	const struct ref *rm = mapped_refs;
+
+	if (check_everything_connected(iterate_ref_map, 0, &rm))
+		die(_("remote did not send all necessary objects"));
+
 	if (refs) {
 		write_remote_refs(mapped_refs);
 		if (option_single_branch)
diff --git a/t/t1060-object-corruption.sh b/t/t1060-object-corruption.sh
index 05ba4e7..fd314ef 100755
--- a/t/t1060-object-corruption.sh
+++ b/t/t1060-object-corruption.sh
@@ -79,7 +79,7 @@ test_expect_success 'clone --no-local --bare detects missing object' '
 	test_must_fail git clone --no-local --bare missing missing-transport
 '
 
-test_expect_failure 'clone --no-local --bare detects misnamed object' '
+test_expect_success 'clone --no-local --bare detects misnamed object' '
 	test_must_fail git clone --no-local --bare misnamed misnamed-transport
 '
 
diff --git a/t/t5710-info-alternate.sh b/t/t5710-info-alternate.sh
index 5a6e49d..8956c21 100755
--- a/t/t5710-info-alternate.sh
+++ b/t/t5710-info-alternate.sh
@@ -58,13 +58,7 @@ git clone -l -s F G &&
 git clone -l -s D E &&
 git clone -l -s E F &&
 git clone -l -s F G &&
-git clone --bare -l -s G H'
-
-test_expect_success 'invalidity of deepest repository' \
-'cd H && {
-	test_valid_repo
-	test $? -ne 0
-}'
+test_must_fail git clone --bare -l -s G H'
 
 cd "$base_dir"
 
-- 
1.8.2.13.g0f18d3c
Previous: Jeff KingNext: Duy Nguyen
Message 55 of 60 in “propagating repo corruption across clone”
  1. Jeff KingMar 24, 2013
  2. Ævar Arnfjörð BjarmasonMar 24, 2013
  3. Jeff KingMar 24, 2013
  4. Jeff MitchellMar 25, 2013
  5. Jeff KingMar 25, 2013
  6. Duy NguyenMar 25, 2013
  7. Jeff KingMar 25, 2013
  8. Jeff MitchellMar 25, 2013
  9. Jeff KingMar 25, 2013
  10. Jeff MitchellMar 26, 2013
  11. Jeff KingMar 26, 2013
  12. Philip OakleyMar 26, 2013
  13. Jeff KingMar 26, 2013
  14. Rich FrommMar 26, 2013
  15. Jonathan NiederMar 27, 2013
  16. Rich FrommMar 27, 2013
  17. Jeff KingMar 27, 2013
  18. Jeff KingMar 27, 2013
  19. Junio C HamanoMar 27, 2013
  20. Sitaram ChamartyMar 27, 2013
  21. Junio C HamanoMar 27, 2013
  22. Sitaram ChamartyMar 27, 2013
  23. Rich FrommMar 27, 2013
  24. Junio C HamanoMar 27, 2013
  25. Jeff MitchellMar 28, 2013
  26. Jeff MitchellMar 28, 2013
  27. Duy NguyenMar 26, 2013
  28. Ilari LiusvaaraMar 24, 2013
  29. Junio C HamanoMar 25, 2013
  30. Jeff KingMar 25, 2013
  31. 0/9 corrupt object potpourriJeff King, Mar 25, 2013
  32. 1/9 stream_blob_to_fd: detect errors reading from streamJeff King, Mar 25, 2013
  33. Junio C HamanoMar 26, 2013
  34. 2/9 check_sha1_signature: check return value from read_istreamJeff King, Mar 25, 2013
  35. 3/9 read_istream_filtered: propagate read error from upstreamJeff King, Mar 25, 2013
  36. 4/9 avoid infinite loop in read_istream_looseJeff King, Mar 25, 2013
  37. 5/9 add test for streaming corrupt blobsJeff King, Mar 25, 2013
  38. Jonathan NiederMar 25, 2013
  39. Jeff KingMar 25, 2013
  40. Jeff KingMar 27, 2013
  41. Junio C HamanoMar 27, 2013
  42. 6/9 streaming_write_entry: propagate streaming errorsJeff King, Mar 25, 2013
  43. Eric SunshineMar 25, 2013
  44. Jeff KingMar 25, 2013
  45. Jonathan NiederMar 25, 2013
  46. 6/9 streaming_write_entry: propagate streaming errorsJeff King, Mar 25, 2013
  47. Jonathan NiederMar 25, 2013
  48. Junio C HamanoMar 26, 2013
  49. 7/9 add tests for cloning corrupted repositoriesJeff King, Mar 25, 2013
  50. 8/9 clone: die on errors from unpack_treesJeff King, Mar 25, 2013
  51. Junio C HamanoMar 26, 2013
  52. 10/9 clone: leave repo in place after checkout errorsJeff King, Mar 26, 2013
  53. Jonathan NiederMar 26, 2013
  54. Jeff KingMar 27, 2013
  55. 9/9 clone: run check_everything_connectedJeff King, Mar 25, 2013
  56. Duy NguyenMar 26, 2013
  57. Jeff KingMar 26, 2013
  58. Junio C HamanoMar 26, 2013
  59. Duy NguyenMar 28, 2013
  60. Duy NguyenMar 31, 2013

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.