git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH 15/30] packed-backend: create abstraction for writing refs

From
Derrick Stolee via GitGitGadget <gitgitgadget@gmail.com>
Date
Nov 7, 2022, 18:35 UTC
Message-ID
<a3819f665977194a8f44061d4a52f86d44cf6e0a.1667846165.git.gitgitgadget@gmail.com>
In-Reply-To
<pull.1408.git.1667846164.gitgitgadget@gmail.com>
From: Derrick Stolee <derrickstolee@github.com>

The packed-refs file is a plaintext file format that starts with a header line, then each ref is given as one or two lines (two if there is a peeled value). These lines are written as part of a sequence of updates which are merged with the existing ref iterator in merge_iterator_and_updates(). That method is currently tied directly to write_packed_entry_v1().

When creating a new version of the packed-file format, it would be valuable to use this merging logic in an identical way. Create a new function pointer type, write_ref_fn, and use that type in merge_iterator_and_updates().

Notably, the function pointer type no longer depends on a FILE pointer, but instead takes an arbitrary "void *write_data" parameter. This flexibility will be critical in the future, since the planned v2 format will use the chunk-format API and need a more complicated structure than the output FILE.

Signed-off-by: Derrick Stolee <derrickstolee@github.com>
---
 refs/packed-backend.c   | 26 +++++++++++++++-----------
 refs/packed-backend.h   | 16 ++++++++++++++--
 refs/packed-format-v1.c |  7 +++++--
 3 files changed, 34 insertions(+), 15 deletions(-)
diff --git a/refs/packed-backend.c b/refs/packed-backend.c
index 0dff78f02c8..7ed9475812c 100644
--- a/refs/packed-backend.c
+++ b/refs/packed-backend.c
@@ -535,10 +535,11 @@ static void add_write_error(struct packed_ref_store *refs, struct strbuf *err)
 		    get_tempfile_path(refs->tempfile), strerror(errno));
 }
 
-static int merge_iterator_and_updates(struct packed_ref_store *refs,
-				      struct string_list *updates,
-				      struct strbuf *err,
-				      FILE *out)
+int merge_iterator_and_updates(struct packed_ref_store *refs,
+			       struct string_list *updates,
+			       struct strbuf *err,
+			       write_ref_fn write_fn,
+			       void *write_data)
 {
 	struct ref_iterator *iter = NULL;
 	int ok, i;
@@ -634,9 +635,10 @@ static int merge_iterator_and_updates(struct packed_ref_store *refs,
 			struct object_id peeled;
 			int peel_error = ref_iterator_peel(iter, &peeled);
 
-			if (write_packed_entry_v1(out, iter->refname,
-						  iter->oid,
-						  peel_error ? NULL : &peeled)) {
+			if (write_fn(iter->refname,
+				     iter->oid,
+				     peel_error ? NULL : &peeled,
+				     write_data)) {
 				add_write_error(refs, err);
 				goto error;
 			}
@@ -657,9 +659,10 @@ static int merge_iterator_and_updates(struct packed_ref_store *refs,
 			int peel_error = peel_object(&update->new_oid,
 						     &peeled);
 
-			if (write_packed_entry_v1(out, update->refname,
-						  &update->new_oid,
-						  peel_error ? NULL : &peeled)) {
+			if (write_fn(update->refname,
+				     &update->new_oid,
+				     peel_error ? NULL : &peeled,
+				     write_data)) {
 				add_write_error(refs, err);
 				goto error;
 			}
@@ -725,7 +728,8 @@ static int write_with_updates(struct packed_ref_store *refs,
 		goto error;
 	}
 
-	ok = merge_iterator_and_updates(refs, updates, err, out);
+	ok = merge_iterator_and_updates(refs, updates, err,
+					write_packed_entry_v1, out);
 
 	if (ok != ITER_DONE) {
 		strbuf_addstr(err, "unable to write packed-refs file: "
diff --git a/refs/packed-backend.h b/refs/packed-backend.h
index 143ed6d4f6c..b6908bb002c 100644
--- a/refs/packed-backend.h
+++ b/refs/packed-backend.h
@@ -192,6 +192,17 @@ struct packed_ref_iterator {
 	unsigned int flags;
 };
 
+typedef int (*write_ref_fn)(const char *refname,
+			    const struct object_id *oid,
+			    const struct object_id *peeled,
+			    void *write_data);
+
+int merge_iterator_and_updates(struct packed_ref_store *refs,
+			       struct string_list *updates,
+			       struct strbuf *err,
+			       write_ref_fn write_fn,
+			       void *write_data);
+
 /**
  * Parse the buffer at the given snapshot to verify that it is a
  * packed-refs file in version 1 format. Update the snapshot->peeled
@@ -227,8 +238,9 @@ void verify_buffer_safe_v1(struct snapshot *snapshot);
 void sort_snapshot_v1(struct snapshot *snapshot);
 int write_packed_file_header_v1(FILE *out);
 int next_record_v1(struct packed_ref_iterator *iter);
-int write_packed_entry_v1(FILE *fh, const char *refname,
+int write_packed_entry_v1(const char *refname,
 			  const struct object_id *oid,
-			  const struct object_id *peeled);
+			  const struct object_id *peeled,
+			  void *write_data);
 
 #endif /* REFS_PACKED_BACKEND_H */
diff --git a/refs/packed-format-v1.c b/refs/packed-format-v1.c
index ef9e6618c89..2d071567c02 100644
--- a/refs/packed-format-v1.c
+++ b/refs/packed-format-v1.c
@@ -441,10 +441,13 @@ int write_packed_file_header_v1(FILE *out)
  * error, return a nonzero value and leave errno set at the value left
  * by the failing call to `fprintf()`.
  */
-int write_packed_entry_v1(FILE *fh, const char *refname,
+int write_packed_entry_v1(const char *refname,
 			  const struct object_id *oid,
-			  const struct object_id *peeled)
+			  const struct object_id *peeled,
+			  void *write_data)
 {
+	FILE *fh = write_data;
+
 	if (fprintf(fh, "%s %s\n", oid_to_hex(oid), refname) < 0 ||
 	    (peeled && fprintf(fh, "^%s\n", oid_to_hex(peeled)) < 0))
 		return -1;
-- 
gitgitgadget
Previous: Derrick Stolee via GitGitGadgetNext: Derrick Stolee via GitGitGadget
Message 22 of 56 in “[RFC] extensions.refFormat and packed-refs v2 file format”
  1. 00/30 [RFC] extensions.refFormat and packed-refs v2 file formatDerrick Stolee via GitGitGadget, Nov 7, 2022
  2. 01/30 hashfile: allow skipping the hash functionDerrick Stolee via GitGitGadget, Nov 7, 2022
  3. 02/30 read-cache: add index.computeHash config optionDerrick Stolee via GitGitGadget, Nov 7, 2022
  4. Elijah NewrenNov 11, 2022
  5. Derrick StoleeNov 14, 2022
  6. Ævar Arnfjörð BjarmasonNov 17, 2022
  7. 03/30 extensions: add refFormat extensionDerrick Stolee via GitGitGadget, Nov 7, 2022
  8. Elijah NewrenNov 11, 2022
  9. Derrick StoleeNov 16, 2022
  10. 06/30 refs: allow loose files without packed-refsDerrick Stolee via GitGitGadget, Nov 7, 2022
  11. 07/30 chunk-format: number of chunks is optionalDerrick Stolee via GitGitGadget, Nov 7, 2022
  12. 04/30 config: fix multi-level bulleted listDerrick Stolee via GitGitGadget, Nov 7, 2022
  13. 05/30 repository: wire ref extensions to ref backendsDerrick Stolee via GitGitGadget, Nov 7, 2022
  14. 08/30 chunk-format: document trailing table of contentsDerrick Stolee via GitGitGadget, Nov 7, 2022
  15. 09/30 chunk-format: store chunk offset during writeDerrick Stolee via GitGitGadget, Nov 7, 2022
  16. 11/30 chunk-format: parse trailing table of contentsDerrick Stolee via GitGitGadget, Nov 7, 2022
  17. 10/30 chunk-format: allow trailing table of contentsDerrick Stolee via GitGitGadget, Nov 7, 2022
  18. 13/30 packed-backend: extract add_write_error()Derrick Stolee via GitGitGadget, Nov 7, 2022
  19. 12/30 refs: extract packfile format to new fileDerrick Stolee via GitGitGadget, Nov 7, 2022
  20. 14/30 packed-backend: extract iterator/updates mergeDerrick Stolee via GitGitGadget, Nov 7, 2022
  21. 16/30 config: add config values for packed-refs v2Derrick Stolee via GitGitGadget, Nov 7, 2022
  22. 15/30 packed-backend: create abstraction for writing refsDerrick Stolee via GitGitGadget, Nov 7, 2022
  23. 17/30 packed-backend: create shell of v2 writesDerrick Stolee via GitGitGadget, Nov 7, 2022
  24. 18/30 packed-refs: write file format version 2Derrick Stolee via GitGitGadget, Nov 7, 2022
  25. 19/30 packed-refs: read file format v2Derrick Stolee via GitGitGadget, Nov 7, 2022
  26. 20/30 packed-refs: read optional prefix chunksDerrick Stolee via GitGitGadget, Nov 7, 2022
  27. 21/30 packed-refs: write prefix chunksDerrick Stolee via GitGitGadget, Nov 7, 2022
  28. 22/30 packed-backend: create GIT_TEST_PACKED_REFS_VERSIONDerrick Stolee via GitGitGadget, Nov 7, 2022
  29. 24/30 t5312: allow packed-refs v2 formatDerrick Stolee via GitGitGadget, Nov 7, 2022
  30. 23/30 t1409: test with packed-refs v2Derrick Stolee via GitGitGadget, Nov 7, 2022
  31. 26/30 t3210: require packed-refs v1 for some testsDerrick Stolee via GitGitGadget, Nov 7, 2022
  32. 25/30 t5502: add PACKED_REFS_V1 prerequisiteDerrick Stolee via GitGitGadget, Nov 7, 2022
  33. 27/30 t*: skip packed-refs v2 over http testsDerrick Stolee via GitGitGadget, Nov 7, 2022
  34. 28/30 ci: run GIT_TEST_PACKED_REFS_VERSION=2 in some buildsDerrick Stolee via GitGitGadget, Nov 7, 2022
  35. 29/30 p1401: create performance test for ref operationsDerrick Stolee via GitGitGadget, Nov 7, 2022
  36. 30/30 refs: skip hashing when writing packed-refs v2Derrick Stolee via GitGitGadget, Nov 7, 2022
  37. Derrick StoleeNov 9, 2022
  38. Elijah NewrenNov 11, 2022
  39. Derrick StoleeNov 14, 2022
  40. Elijah NewrenNov 15, 2022
  41. Derrick StoleeNov 16, 2022
  42. Elijah NewrenNov 17, 2022
  43. Junio C HamanoNov 18, 2022
  44. Elijah NewrenNov 19, 2022
  45. Taylor BlauNov 19, 2022
  46. Derrick StoleeNov 30, 2022
  47. Han-Wen NienhuysNov 28, 2022
  48. Derrick StoleeNov 30, 2022
  49. Phillip WoodNov 30, 2022
  50. Taylor BlauNov 30, 2022
  51. Han-Wen NienhuysNov 30, 2022
  52. Sean AllredNov 30, 2022
  53. Derrick StoleeDec 1, 2022
  54. Han-Wen NienhuysDec 2, 2022
  55. Ævar Arnfjörð BjarmasonDec 2, 2022
  56. Junio C HamanoNov 30, 2022

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.