git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH v2 1/4] fetch: extract writing to FETCH_HEAD

From
Patrick Steinhardt <ps@pks.im>
Date
Jan 8, 2021, 12:11 UTC
Message-ID
<d80dbc5a9c9520621651541e418ee5216d164053.1610107599.git.ps@pks.im>
In-Reply-To
<cover.1610107599.git.ps@pks.im>

When performing a fetch with the default `--write-fetch-head` option, we write all updated references to FETCH_HEAD while the updates are performed. Given that updates are not performed atomically, it means that we we write to FETCH_HEAD even if some or all of the reference updates fail.

Given that we simply update FETCH_HEAD ad-hoc with each reference, the
logic is completely contained in `store_update_refs` and thus quite hard
to extend. This can already be seen by the way we skip writing to the
FETCH_HEAD: instead of having a conditional which simply skips writing,
we instead open "/dev/null" and needlessly write all updates there.

We are about to extend git-fetch(1) to accept an `--atomic` flag which will make the fetch an all-or-nothing operation with regards to the reference updates. This will also require us to make the updates to FETCH_HEAD an all-or-nothing operation, but as explained doing so is not easy with the current layout. This commit thus refactors the wa we write to FETCH_HEAD and pulls out the logic to open, append to, commit and close the file. While this may seem rather over-the top at first, pulling out this logic will make it a lot easier to update the code in a subsequent commit. It also allows us to easily skip writing completely in case `--no-write-fetch-head` was passed.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 builtin/fetch.c | 80 ++++++++++++++++++++++++++++++++++++++-----------
 1 file changed, 62 insertions(+), 18 deletions(-)
diff --git a/builtin/fetch.c b/builtin/fetch.c
index ecf8537605..557ec130db 100644
--- a/builtin/fetch.c
+++ b/builtin/fetch.c
@@ -897,6 +897,56 @@ static int iterate_ref_map(void *cb_data, struct object_id *oid)
 	return 0;
 }
 
+struct fetch_head {
+	FILE *fp;
+};
+
+static int open_fetch_head(struct fetch_head *fetch_head)
+{
+	const char *filename = git_path_fetch_head(the_repository);
+
+	if (!write_fetch_head)
+		return 0;
+
+	fetch_head->fp = fopen(filename, "a");
+	if (!fetch_head->fp)
+		return error_errno(_("cannot open %s"), filename);
+
+	return 0;
+}
+
+static void append_fetch_head(struct fetch_head *fetch_head, const char *old_oid,
+			      const char *merge_status_marker, const char *note,
+			      const char *url, size_t url_len)
+{
+	size_t i;
+
+	if (!write_fetch_head)
+		return;
+
+	fprintf(fetch_head->fp, "%s\t%s\t%s",
+		old_oid, merge_status_marker, note);
+	for (i = 0; i < url_len; ++i)
+		if ('\n' == url[i])
+			fputs("\\n", fetch_head->fp);
+		else
+			fputc(url[i], fetch_head->fp);
+	fputc('\n', fetch_head->fp);
+}
+
+static void commit_fetch_head(struct fetch_head *fetch_head)
+{
+	/* Nothing to commit yet. */
+}
+
+static void close_fetch_head(struct fetch_head *fetch_head)
+{
+	if (!write_fetch_head)
+		return;
+
+	fclose(fetch_head->fp);
+}
+
 static const char warn_show_forced_updates[] =
 N_("Fetch normally indicates which branches had a forced update,\n"
    "but that check has been disabled. To re-enable, use '--show-forced-updates'\n"
@@ -909,22 +959,19 @@ N_("It took %.2f seconds to check forced updates. You can use\n"
 static int store_updated_refs(const char *raw_url, const char *remote_name,
 			      int connectivity_checked, struct ref *ref_map)
 {
-	FILE *fp;
+	struct fetch_head fetch_head;
 	struct commit *commit;
 	int url_len, i, rc = 0;
 	struct strbuf note = STRBUF_INIT;
 	const char *what, *kind;
 	struct ref *rm;
 	char *url;
-	const char *filename = (!write_fetch_head
-				? "/dev/null"
-				: git_path_fetch_head(the_repository));
 	int want_status;
 	int summary_width = transport_summary_width(ref_map);
 
-	fp = fopen(filename, "a");
-	if (!fp)
-		return error_errno(_("cannot open %s"), filename);
+	rc = open_fetch_head(&fetch_head);
+	if (rc)
+		return -1;
 
 	if (raw_url)
 		url = transport_anonymize_url(raw_url);
@@ -1016,16 +1063,10 @@ static int store_updated_refs(const char *raw_url, const char *remote_name,
 				merge_status_marker = "not-for-merge";
 				/* fall-through */
 			case FETCH_HEAD_MERGE:
-				fprintf(fp, "%s\t%s\t%s",
-					oid_to_hex(&rm->old_oid),
-					merge_status_marker,
-					note.buf);
-				for (i = 0; i < url_len; ++i)
-					if ('\n' == url[i])
-						fputs("\\n", fp);
-					else
-						fputc(url[i], fp);
-				fputc('\n', fp);
+				append_fetch_head(&fetch_head,
+						  oid_to_hex(&rm->old_oid),
+						  merge_status_marker,
+						  note.buf, url, url_len);
 				break;
 			default:
 				/* do not write anything to FETCH_HEAD */
@@ -1060,6 +1101,9 @@ static int store_updated_refs(const char *raw_url, const char *remote_name,
 		}
 	}
 
+	if (!rc)
+		commit_fetch_head(&fetch_head);
+
 	if (rc & STORE_REF_ERROR_DF_CONFLICT)
 		error(_("some local refs could not be updated; try running\n"
 		      " 'git remote prune %s' to remove any old, conflicting "
@@ -1077,7 +1121,7 @@ static int store_updated_refs(const char *raw_url, const char *remote_name,
  abort:
 	strbuf_release(&note);
 	free(url);
-	fclose(fp);
+	close_fetch_head(&fetch_head);
 	return rc;
 }
 
-- 
2.30.0
Previous: Patrick SteinhardtNext: Junio C Hamano
Message 9 of 41 in “fetch: implement support for atomic reference updates”
  1. 0/2 fetch: implement support for atomic reference updatesPatrick Steinhardt, Jan 7, 2021
  2. 1/2 fetch: allow passing a transaction to `s_update_ref()`Patrick Steinhardt, Jan 7, 2021
  3. Junio C HamanoJan 7, 2021
  4. Christian CouderJan 8, 2021
  5. Junio C HamanoJan 8, 2021
  6. 2/2 fetch: implement support for atomic reference updatesPatrick Steinhardt, Jan 7, 2021
  7. Junio C HamanoJan 8, 2021
  8. 0/4 fetch: implement support for atomic reference updatesPatrick Steinhardt, Jan 8, 2021
  9. 1/4 fetch: extract writing to FETCH_HEADPatrick Steinhardt, Jan 8, 2021
  10. Junio C HamanoJan 8, 2021
  11. Patrick SteinhardtJan 11, 2021
  12. Junio C HamanoJan 11, 2021
  13. 3/4 fetch: allow passing a transaction to `s_update_ref()`Patrick Steinhardt, Jan 8, 2021
  14. Junio C HamanoJan 8, 2021
  15. 2/4 fetch: refactor `s_update_ref` to use common exit pathPatrick Steinhardt, Jan 8, 2021
  16. Junio C HamanoJan 8, 2021
  17. Patrick SteinhardtJan 11, 2021
  18. 4/4 fetch: implement support for atomic reference updatesPatrick Steinhardt, Jan 8, 2021
  19. Junio C HamanoJan 9, 2021
  20. Patrick SteinhardtJan 11, 2021
  21. Junio C HamanoJan 11, 2021
  22. Patrick SteinhardtJan 12, 2021
  23. Patrick SteinhardtJan 12, 2021
  24. Junio C HamanoJan 12, 2021
  25. 0/5 fetch: implement support for atomic reference updatesPatrick Steinhardt, Jan 11, 2021
  26. 1/5 fetch: extract writing to FETCH_HEADPatrick Steinhardt, Jan 11, 2021
  27. Junio C HamanoJan 11, 2021
  28. 2/5 fetch: use strbuf to format FETCH_HEAD updatesPatrick Steinhardt, Jan 11, 2021
  29. Patrick SteinhardtJan 11, 2021
  30. Christian CouderJan 11, 2021
  31. Junio C HamanoJan 11, 2021
  32. 4/5 fetch: allow passing a transaction to `s_update_ref()`Patrick Steinhardt, Jan 11, 2021
  33. 3/5 fetch: refactor `s_update_ref` to use common exit pathPatrick Steinhardt, Jan 11, 2021
  34. 5/5 fetch: implement support for atomic reference updatesPatrick Steinhardt, Jan 11, 2021
  35. Junio C HamanoJan 12, 2021
  36. 0/5 fetch: implement support for atomic reference updatesPatrick Steinhardt, Jan 12, 2021
  37. 3/5 fetch: refactor `s_update_ref` to use common exit pathPatrick Steinhardt, Jan 12, 2021
  38. 1/5 fetch: extract writing to FETCH_HEADPatrick Steinhardt, Jan 12, 2021
  39. 2/5 fetch: use strbuf to format FETCH_HEAD updatesPatrick Steinhardt, Jan 12, 2021
  40. 4/5 fetch: allow passing a transaction to `s_update_ref()`Patrick Steinhardt, Jan 12, 2021
  41. 5/5 fetch: implement support for atomic reference updatesPatrick Steinhardt, Jan 12, 2021

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.