threads / patch / 66427

patchpacked-refs: use `fwrite()` when passing refs verbatim

Subject: [PATCH] packed-refs: use `fwrite()` when passing refs verbatim

## tl;dr

Karthik Nayak's patch avoids reformatting unchanged refs when updating packed-refs; review has focused on comment wording and commit message detail. Read the story.

replies: 15people: 4as markdown or json

Karthik Nayak· Sep 30, 2026, 15:15 UTC · lore

The `write_with_updates()` function uses a `struct ref_iterator` to iterate over all refs to write to the temporary packfile. It receives the iterator from `packed_ref_iterator_begin()` which takes a snapshot of the 'packed-refs' file.

While writing to the new packfile, writes are routed via `write_packed_entry()` which uses `fprintf()`. Even for references which haven't changed, we use the same mechanism. Instead, let's track the position of unchanged references in the snapshot iterator and directly use `fwrite()`.

This removes the unnecessary formatting operation involved. We can see a consistent ~20% performance improvement when deleting from packed references.

Benchmark 1: update-ref: delete ref (refcount = 100000, revision = master)
  Time (mean ± σ):      28.7 ms ±   1.7 ms    [User: 22.5 ms, System: 5.9 ms]
  Range (min … max):    26.7 ms …  33.3 ms    46 runs
Benchmark 2: update-ref: delete ref (refcount = 100000, revision = b4/kn-speedup-packed-refs)
  Time (mean ± σ):      23.8 ms ±   1.2 ms    [User: 17.5 ms, System: 6.0 ms]
  Range (min … max):    22.1 ms …  27.7 ms    56 runs
Summary
  update-ref: delete ref (refcount = 100000, revision = b4/kn-speedup-packed-refs) ran
    1.21 ± 0.09 times faster than update-ref: delete ref (refformat = files, refcount = 100000, revision = master)
Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
---
 refs/packed-backend.c | 43 ++++++++++++++++++++++++++++++++-----------
 1 file changed, 32 insertions(+), 11 deletions(-)
Show changes to refs/packed-backend.c +32 −12
diff --git a/refs/packed-backend.c b/refs/packed-backend.c
index a73fc6aca7..ef952cdba6 100644
--- a/refs/packed-backend.c
+++ b/refs/packed-backend.c
@@ -879,6 +879,12 @@ struct packed_ref_iterator {
 	/* The current position in the snapshot's buffer: */
 	const char *pos;
 
+	/*
+	 * Start of the current record, set when advancing `pos`. Used to
+	 * pass records verbatim to `fwrite()`.
+	 */
+	const char *record_start;
+
 	/* The end of the part of the buffer that will be iterated over: */
 	const char *eof;
 
@@ -933,6 +939,7 @@ static int next_record(struct packed_ref_iterator *iter)
 	if (iter->pos == iter->eof)
 		return ITER_DONE;
 
+	iter->record_start = iter->pos;
 	iter->base.ref.flags = REF_ISPACKED;
 	p = iter->pos;
 
@@ -1218,17 +1225,27 @@ static struct ref_iterator *packed_ref_iterator_begin(
 
 /*
  * Write an entry to the packed-refs file for the specified refname.
- * If peeled is non-NULL, write it as the entry's peeled value. On
- * error, return a nonzero value and leave errno set at the value left
- * by the failing call to `fprintf()`.
+ *
+ * If the raw data is available, skip the formatting and directly write to
+ * the file using `fwrite()`. e.g. when deleting references and remaining
+ * refs need to be written verbatim. Otherwise, use `fprintf()`.
+ *
+ * If peeled is non-NULL, write it as the entry's peeled value.
+ *
+ * On error, return a nonzero value and leave errno set at the value left
+ * by the failing call to `fwrite()` or `fprintf()`.
  */
-static int write_packed_entry(FILE *fh, const char *refname,
-			      const struct object_id *oid,
+static int write_packed_entry(FILE *fh, const char *raw, size_t raw_len,
+			      const char *refname, const struct object_id *oid,
 			      const struct object_id *peeled)
 {
-	if (fprintf(fh, "%s %s\n", oid_to_hex(oid), refname) < 0 ||
-	    (peeled && fprintf(fh, "^%s\n", oid_to_hex(peeled)) < 0))
+	if (raw) {
+		if (fwrite(raw, raw_len, 1, fh) != 1)
+			return -1;
+	} else if (fprintf(fh, "%s %s\n", oid_to_hex(oid), refname) < 0 ||
+		   (peeled && fprintf(fh, "^%s\n", oid_to_hex(peeled)) < 0)) {
 		return -1;
+	}
 
 	return 0;
 }
@@ -1530,9 +1547,13 @@ static enum ref_transaction_error write_with_updates(struct packed_ref_store *re
 		}
 
 		if (cmp < 0) {
-			/* Pass the old reference through. */
-			if (write_packed_entry(out, iter->ref.name,
-					       iter->ref.oid, iter->ref.peeled_oid))
+			const struct packed_ref_iterator *packed_iter =
+				(const struct packed_ref_iterator *)iter;
+			size_t len = packed_iter->pos - packed_iter->record_start;
+
+			if (write_packed_entry(out, packed_iter->record_start,
+					       len, iter->ref.name, iter->ref.oid,
+					       iter->ref.peeled_oid))
 				goto write_error;
 
 			if ((ok = ref_iterator_advance(iter)) != ITER_OK) {
@@ -1551,7 +1572,7 @@ static enum ref_transaction_error write_with_updates(struct packed_ref_store *re
 		} else {
 			bool peeled = update->flags & REF_HAVE_PEELED;
 
-			if (write_packed_entry(out, update->refname,
+			if (write_packed_entry(out, NULL, 0, update->refname,
 					       &update->new_oid,
 					       peeled ? &update->peeled : NULL))
 				goto write_error;

---
base-commit: a018953688f1b10bddf91bff8747068f5f4746a4
change-id: 20260930-kn-speedup-packed-refs-9868f5d0abe9


Thanks
- Karthik
Toon Claes· Oct 1, 2026, 20:30 UTC · re: Karthik Nayak · lore

Re: [PATCH] packed-refs: use `fwrite()` when passing refs verbatim

Karthik Nayak <karthik.188@gmail.com> writes:
> The `write_with_updates()` function uses a `struct ref_iterator` to
> iterate over all refs to write to the temporary packfile. It receives

I would say in general "packfile" is only used for objects, but it seems there is one mention of "packfile" in this file, so I'm not sure.

Show 8 quoted lines
> the iterator from `packed_ref_iterator_begin()` which takes a snapshot
> of the 'packed-refs' file.
>
> While writing to the new packfile, writes are routed via
> `write_packed_entry()` which uses `fprintf()`. Even for references which
> haven't changed, we use the same mechanism. Instead, let's track the
> position of unchanged references in the snapshot iterator and directly
> use `fwrite()`.
There are a few sanitizations that get lost by using fwrite():
* oid_to_hex() is no longer called, so if an OID would be written in the
  old packed-ref as uppercase, it would be copied as such
* next_record() uses isspace(3) to split the oid from the refname. If
  the record would be separated by a tab instead of a space, that would
  be copied over.
* next_record() calls check_refname_format(), if it ain't good, oidclr()
  is called. Before this patch, that means the zero OID will be written
  for such ref. That changes with this patch, and the original OID is
  written if refname_is_safe() passes.
  This won't make a difference in Git though, because upon reading back
  from the packed-refs, the zero OID is read into memory anyway, so
  there is no observable difference.
* similar to the item above, when the ref is peelable, and REF_ISBROKEN
  (around line 985), the `peeled_oid` is not filled in. This means for a
  refname not matching the format will not contain a peeled OID for that
  ref.
  For example, in the source file:
    1937b04ead6e53d949a4e2eb97ed743b42692fde refs/tags/a-bad~tag
    ^67c1263aae5c1750e9d9211995e6d1bb38314fca
  is converted to:
    0000000000000000000000000000000000000000 refs/tags/a-bad~tag
  After this patch, the ref and it's peeled commit OID is copied
  verbatim.

I'm not saying any of this is bad. These are just some side-effects of your fwrite() approach. And some might be worht mentioning in the commit message.

Show 16 quoted lines
>
> This removes the unnecessary formatting operation involved. We can see a
> consistent ~20% performance improvement when deleting from packed
> references.
>
> Benchmark 1: update-ref: delete ref (refcount = 100000, revision = master)
>   Time (mean ± σ):      28.7 ms ±   1.7 ms    [User: 22.5 ms, System: 5.9 ms]
>   Range (min … max):    26.7 ms …  33.3 ms    46 runs
>
> Benchmark 2: update-ref: delete ref (refcount = 100000, revision = b4/kn-speedup-packed-refs)
>   Time (mean ± σ):      23.8 ms ±   1.2 ms    [User: 17.5 ms, System: 6.0 ms]
>   Range (min … max):    22.1 ms …  27.7 ms    56 runs
>
> Summary
>   update-ref: delete ref (refcount = 100000, revision = b4/kn-speedup-packed-refs) ran
>     1.21 ± 0.09 times faster than update-ref: delete ref (refformat = files, refcount = 100000, revision = master)
Not bad.
Show 17 quoted lines
>
> Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
> ---
>  refs/packed-backend.c | 43 ++++++++++++++++++++++++++++++++-----------
>  1 file changed, 32 insertions(+), 11 deletions(-)
>
> diff --git a/refs/packed-backend.c b/refs/packed-backend.c
> index a73fc6aca7..ef952cdba6 100644
> --- a/refs/packed-backend.c
> +++ b/refs/packed-backend.c
> @@ -879,6 +879,12 @@ struct packed_ref_iterator {
>  	/* The current position in the snapshot's buffer: */
>  	const char *pos;
>  
> +	/*
> +	 * Start of the current record, set when advancing `pos`. Used to
> +	 * pass records verbatim to `fwrite()`.

I assume there was a version you've been working on that was calling fwrite() directly from ref_transaction_error write_with_updates()? With that being wrapped by a helper function, I think it's better to name that function here.

Show 36 quoted lines
> +	 */
> +	const char *record_start;
> +
>  	/* The end of the part of the buffer that will be iterated over: */
>  	const char *eof;
>  
> @@ -933,6 +939,7 @@ static int next_record(struct packed_ref_iterator *iter)
>  	if (iter->pos == iter->eof)
>  		return ITER_DONE;
>  
> +	iter->record_start = iter->pos;
>  	iter->base.ref.flags = REF_ISPACKED;
>  	p = iter->pos;
>  
> @@ -1218,17 +1225,27 @@ static struct ref_iterator *packed_ref_iterator_begin(
>  
>  /*
>   * Write an entry to the packed-refs file for the specified refname.
> - * If peeled is non-NULL, write it as the entry's peeled value. On
> - * error, return a nonzero value and leave errno set at the value left
> - * by the failing call to `fprintf()`.
> + *
> + * If the raw data is available, skip the formatting and directly write to
> + * the file using `fwrite()`. e.g. when deleting references and remaining
> + * refs need to be written verbatim. Otherwise, use `fprintf()`.
> + *
> + * If peeled is non-NULL, write it as the entry's peeled value.
> + *
> + * On error, return a nonzero value and leave errno set at the value left
> + * by the failing call to `fwrite()` or `fprintf()`.
>   */
> -static int write_packed_entry(FILE *fh, const char *refname,
> -			      const struct object_id *oid,
> +static int write_packed_entry(FILE *fh, const char *raw, size_t raw_len,
> +			      const char *refname, const struct object_id *oid,
>  			      const struct object_id *peeled)

I'm not convinced it's worth to have both ways of writing in a single function.

I rather keep this function as-is and add a function:
    static int write_packed_entry_preformatted(FILE *fh,
                                               const char *line,
                                               size_t line_len)
    {
    	if (fwrite(raw, raw_len, 1, fh) != 1)
    			return -1;
    	return 0;
    }
Or maybe even:
    static int write_packed_entry_from_iter(FILE *fh,
                                            struct packed_ref_iterator *packed_iter)
    {
    	size_t ret = fwrite(packed_iter->record_start,
                            packed_iter->pos - packed_iter->record_start,
                            1, fh);
        if (ret != 1)
    			return -1;
    	return 0;
    }
Show 5 quoted lines
>  {
> -	if (fprintf(fh, "%s %s\n", oid_to_hex(oid), refname) < 0 ||
> -	    (peeled && fprintf(fh, "^%s\n", oid_to_hex(peeled)) < 0))
> +	if (raw) {
> +		if (fwrite(raw, raw_len, 1, fh) != 1)

I see in some places, for example in fwrite_or_die(), `len` and `1` are swapped and the return value is compared against `len`. This is useful when the length can be 0. But that cannot the case here, so no need to change that.

> +			return -1;
> +	} else if (fprintf(fh, "%s %s\n", oid_to_hex(oid), refname) < 0 ||
> +		   (peeled && fprintf(fh, "^%s\n", oid_to_hex(peeled)) < 0)) {

For what's it's worth, I think it's really ugly to do this in a single if() statement, but that just could be me.

Show 15 quoted lines
>  		return -1;
> +	}
>  
>  	return 0;
>  }
> @@ -1530,9 +1547,13 @@ static enum ref_transaction_error write_with_updates(struct packed_ref_store *re
>  		}
>  
>  		if (cmp < 0) {
> -			/* Pass the old reference through. */
> -			if (write_packed_entry(out, iter->ref.name,
> -					       iter->ref.oid, iter->ref.peeled_oid))
> +			const struct packed_ref_iterator *packed_iter =
> +				(const struct packed_ref_iterator *)iter;
> +			size_t len = packed_iter->pos - packed_iter->record_start;

This is nice. Because packed_iter->pos points at the next record, the `len` will include the peeled OID line as well. Which is fwrite(3)'n in one go. That's a nice win.

Show 25 quoted lines
> +
> +			if (write_packed_entry(out, packed_iter->record_start,
> +					       len, iter->ref.name, iter->ref.oid,
> +					       iter->ref.peeled_oid))
>  				goto write_error;
>  
>  			if ((ok = ref_iterator_advance(iter)) != ITER_OK) {
> @@ -1551,7 +1572,7 @@ static enum ref_transaction_error write_with_updates(struct packed_ref_store *re
>  		} else {
>  			bool peeled = update->flags & REF_HAVE_PEELED;
>  
> -			if (write_packed_entry(out, update->refname,
> +			if (write_packed_entry(out, NULL, 0, update->refname,
>  					       &update->new_oid,
>  					       peeled ? &update->peeled : NULL))
>  				goto write_error;
>
> ---
> base-commit: a018953688f1b10bddf91bff8747068f5f4746a4
> change-id: 20260930-kn-speedup-packed-refs-9868f5d0abe9
>
>
> Thanks
> - Karthik
>
-- 
Laters,
Toon
Karthik Nayak· Oct 2, 2026, 00:16 UTC · re: Toon Claes · lore

Re: [PATCH] packed-refs: use `fwrite()` when passing refs verbatim

Toon Claes <toon@iotcl.com> writes:
Show 8 quoted lines
> Karthik Nayak <karthik.188@gmail.com> writes:
>
>> The `write_with_updates()` function uses a `struct ref_iterator` to
>> iterate over all refs to write to the temporary packfile. It receives
>
> I would say in general "packfile" is only used for objects, but it seems
> there is one mention of "packfile" in this file, so I'm not sure.
>
You're right, I will change both of those to say 'packed-refs'.
Show 46 quoted lines
>> the iterator from `packed_ref_iterator_begin()` which takes a snapshot
>> of the 'packed-refs' file.
>>
>> While writing to the new packfile, writes are routed via
>> `write_packed_entry()` which uses `fprintf()`. Even for references which
>> haven't changed, we use the same mechanism. Instead, let's track the
>> position of unchanged references in the snapshot iterator and directly
>> use `fwrite()`.
>
> There are a few sanitizations that get lost by using fwrite():
>
> * oid_to_hex() is no longer called, so if an OID would be written in the
>   old packed-ref as uppercase, it would be copied as such
>
> * next_record() uses isspace(3) to split the oid from the refname. If
>   the record would be separated by a tab instead of a space, that would
>   be copied over.
>
> * next_record() calls check_refname_format(), if it ain't good, oidclr()
>   is called. Before this patch, that means the zero OID will be written
>   for such ref. That changes with this patch, and the original OID is
>   written if refname_is_safe() passes.
>   This won't make a difference in Git though, because upon reading back
>   from the packed-refs, the zero OID is read into memory anyway, so
>   there is no observable difference.
>
> * similar to the item above, when the ref is peelable, and REF_ISBROKEN
>   (around line 985), the `peeled_oid` is not filled in. This means for a
>   refname not matching the format will not contain a peeled OID for that
>   ref.
>   For example, in the source file:
>
>     1937b04ead6e53d949a4e2eb97ed743b42692fde refs/tags/a-bad~tag
>     ^67c1263aae5c1750e9d9211995e6d1bb38314fca
>
>   is converted to:
>
>     0000000000000000000000000000000000000000 refs/tags/a-bad~tag
>
>   After this patch, the ref and it's peeled commit OID is copied
>   verbatim.
>
> I'm not saying any of this is bad. These are just some side-effects of
> your fwrite() approach. And some might be worht mentioning in the commit
> message.
>

While you're right, sanitation is not part of this flow, here we're simply deleting a reference from the packed-refs file, the fact that sanitation was even happening was a side-effect.

But I will mention it in the commit message, I think that makes sense.
[snip]
Show 23 quoted lines
>>
>> Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
>> ---
>>  refs/packed-backend.c | 43 ++++++++++++++++++++++++++++++++-----------
>>  1 file changed, 32 insertions(+), 11 deletions(-)
>>
>> diff --git a/refs/packed-backend.c b/refs/packed-backend.c
>> index a73fc6aca7..ef952cdba6 100644
>> --- a/refs/packed-backend.c
>> +++ b/refs/packed-backend.c
>> @@ -879,6 +879,12 @@ struct packed_ref_iterator {
>>  	/* The current position in the snapshot's buffer: */
>>  	const char *pos;
>>
>> +	/*
>> +	 * Start of the current record, set when advancing `pos`. Used to
>> +	 * pass records verbatim to `fwrite()`.
>
> I assume there was a version you've been working on that was calling
> fwrite() directly from ref_transaction_error write_with_updates()? With
> that being wrapped by a helper function, I think it's better to name
> that function here.
>
I'm not sure I follow what you mean here.
Show 64 quoted lines
>> +	 */
>> +	const char *record_start;
>> +
>>  	/* The end of the part of the buffer that will be iterated over: */
>>  	const char *eof;
>>
>> @@ -933,6 +939,7 @@ static int next_record(struct packed_ref_iterator *iter)
>>  	if (iter->pos == iter->eof)
>>  		return ITER_DONE;
>>
>> +	iter->record_start = iter->pos;
>>  	iter->base.ref.flags = REF_ISPACKED;
>>  	p = iter->pos;
>>
>> @@ -1218,17 +1225,27 @@ static struct ref_iterator *packed_ref_iterator_begin(
>>
>>  /*
>>   * Write an entry to the packed-refs file for the specified refname.
>> - * If peeled is non-NULL, write it as the entry's peeled value. On
>> - * error, return a nonzero value and leave errno set at the value left
>> - * by the failing call to `fprintf()`.
>> + *
>> + * If the raw data is available, skip the formatting and directly write to
>> + * the file using `fwrite()`. e.g. when deleting references and remaining
>> + * refs need to be written verbatim. Otherwise, use `fprintf()`.
>> + *
>> + * If peeled is non-NULL, write it as the entry's peeled value.
>> + *
>> + * On error, return a nonzero value and leave errno set at the value left
>> + * by the failing call to `fwrite()` or `fprintf()`.
>>   */
>> -static int write_packed_entry(FILE *fh, const char *refname,
>> -			      const struct object_id *oid,
>> +static int write_packed_entry(FILE *fh, const char *raw, size_t raw_len,
>> +			      const char *refname, const struct object_id *oid,
>>  			      const struct object_id *peeled)
>
> I'm not convinced it's worth to have both ways of writing in a single
> function.
>
> I rather keep this function as-is and add a function:
>
>     static int write_packed_entry_preformatted(FILE *fh,
>                                                const char *line,
>                                                size_t line_len)
>     {
>     	if (fwrite(raw, raw_len, 1, fh) != 1)
>     			return -1;
>     	return 0;
>     }
>
> Or maybe even:
>
>     static int write_packed_entry_from_iter(FILE *fh,
>                                             struct packed_ref_iterator *packed_iter)
>     {
>     	size_t ret = fwrite(packed_iter->record_start,
>                             packed_iter->pos - packed_iter->record_start,
>                             1, fh);
>         if (ret != 1)
>     			return -1;
>     	return 0;
>     }
>

I did it cause it was small enough, but I don't have strong opinions. So let's add another function, perhaps:

static int write_packed_entry_raw(FILE *fh, const char *entry, size_t len)
{
	if (fwrite(entry, len, 1, fh) != 1)
		return -1;
	return 0;
}
[snip]
Show 6 quoted lines
>> +			return -1;
>> +	} else if (fprintf(fh, "%s %s\n", oid_to_hex(oid), refname) < 0 ||
>> +		   (peeled && fprintf(fh, "^%s\n", oid_to_hex(peeled)) < 0)) {
>
> For what's it's worth, I think it's really ugly to do this in a single
> if() statement, but that just could be me.

I agree, but I don't want to change existing code in this patch as that would simply be a distraction from the purpose.

[snip]
>
> --
> Laters,
> Toon
Thanks for the review.
Karthik Nayak· Oct 2, 2026, 13:03 UTC · re: Karthik Nayak · lore

[PATCH v2] packed-refs: use `fwrite()` when passing refs verbatim

The `write_with_updates()` function uses a `struct ref_iterator` to iterate over all refs to write to the temporary packed-refs file. It receives the iterator from `packed_ref_iterator_begin()` which takes a snapshot of the 'packed-refs' file.

While writing to the new packed-refs file, writes are routed via `write_packed_entry()` which uses `fprintf()`. Even for references which haven't changed, we use the same mechanism. Instead, let's track the position of unchanged references in the snapshot iterator and directly use `fwrite()`.

With this, any sanitation which was happening as a side of reformatting is now lost. But that was never the job of this section of the code, since the main intention is to simply rewrite the remaining refs post deletion of the selective few.

This removes the unnecessary formatting operation involved. We can see a consistent ~20% performance improvement when deleting from packed references.

Benchmark 1: update-ref: delete ref (refcount = 100000, revision = master)
  Time (mean ± σ):      28.7 ms ±   1.7 ms    [User: 22.5 ms, System: 5.9 ms]
  Range (min … max):    26.7 ms …  33.3 ms    46 runs
Benchmark 2: update-ref: delete ref (refcount = 100000, revision = b4/kn-speedup-packed-refs)
  Time (mean ± σ):      23.8 ms ±   1.2 ms    [User: 17.5 ms, System: 6.0 ms]
  Range (min … max):    22.1 ms …  27.7 ms    56 runs
Summary
  update-ref: delete ref (refcount = 100000, revision = b4/kn-speedup-packed-refs) ran
    1.21 ± 0.09 times faster than update-ref: delete ref (refformat = files, refcount = 100000, revision = master)
Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
---
Changes in v2:
- Instead of using the existing function, introduce a new
  `write_packed_entry_raw()`.
- Modify the commit to also note that we lose sanitization.
- Link to v1: https://patch.msgid.link/20260930-kn-speedup-packed-refs-v1-1-111cd03d9b0e@gmail.com
---
 refs/packed-backend.c | 30 +++++++++++++++++++++++++++---
 1 file changed, 27 insertions(+), 3 deletions(-)
Show changes to refs/packed-backend.c +27 −4
diff --git a/refs/packed-backend.c b/refs/packed-backend.c
index a73fc6aca7..43ad674cf4 100644
--- a/refs/packed-backend.c
+++ b/refs/packed-backend.c
@@ -879,6 +879,12 @@ struct packed_ref_iterator {
 	/* The current position in the snapshot's buffer: */
 	const char *pos;
 
+	/*
+	 * Start of the current record, set when advancing `pos`. Used to
+	 * pass records verbatim to `fwrite()`.
+	 */
+	const char *record_start;
+
 	/* The end of the part of the buffer that will be iterated over: */
 	const char *eof;
 
@@ -933,6 +939,7 @@ static int next_record(struct packed_ref_iterator *iter)
 	if (iter->pos == iter->eof)
 		return ITER_DONE;
 
+	iter->record_start = iter->pos;
 	iter->base.ref.flags = REF_ISPACKED;
 	p = iter->pos;
 
@@ -1233,6 +1240,19 @@ static int write_packed_entry(FILE *fh, const char *refname,
 	return 0;
 }
 
+/*
+ * Write an entry to the packed-refs file skip any formatting and directly
+ * write to  the file using `fwrite()`. e.g. when deleting references and
+ * remaining refs need to be written verbatim.
+ */
+static int write_packed_entry_raw(FILE *fh, const char *entry, size_t len)
+{
+	if (fwrite(entry, len, 1, fh) != 1)
+		return -1;
+
+	return 0;
+}
+
 int packed_refs_lock(struct ref_store *ref_store, int flags, struct strbuf *err)
 {
 	struct packed_ref_store *refs =
@@ -1530,9 +1550,13 @@ static enum ref_transaction_error write_with_updates(struct packed_ref_store *re
 		}
 
 		if (cmp < 0) {
-			/* Pass the old reference through. */
-			if (write_packed_entry(out, iter->ref.name,
-					       iter->ref.oid, iter->ref.peeled_oid))
+			const struct packed_ref_iterator *packed_iter =
+				(const struct packed_ref_iterator *)iter;
+			size_t len = packed_iter->pos - packed_iter->record_start;
+
+			if (write_packed_entry_raw(out,
+						   packed_iter->record_start,
+						   len))
 				goto write_error;
 
 			if ((ok = ref_iterator_advance(iter)) != ITER_OK) {

---
base-commit: a018953688f1b10bddf91bff8747068f5f4746a4
change-id: 20260930-kn-speedup-packed-refs-9868f5d0abe9


Thanks
- Karthik
Toon Claes· Oct 5, 2026, 18:34 UTC · re: Karthik Nayak · lore

Re: [PATCH v2] packed-refs: use `fwrite()` when passing refs verbatim

Karthik Nayak <karthik.188@gmail.com> writes:
Show 12 quoted lines
> The `write_with_updates()` function uses a `struct ref_iterator` to
> iterate over all refs to write to the temporary packed-refs file. It
> receives the iterator from `packed_ref_iterator_begin()` which takes a
> snapshot of the 'packed-refs' file.
>
> While writing to the new packed-refs file, writes are routed via
> `write_packed_entry()` which uses `fprintf()`. Even for references which
> haven't changed, we use the same mechanism. Instead, let's track the
> position of unchanged references in the snapshot iterator and directly
> use `fwrite()`.
>
> With this, any sanitation which was happening as a side of reformatting
s/side/side effect/ ?
> is now lost. But that was never the job of this section of the code,
> since the main intention is to simply rewrite the remaining refs post
> deletion of the selective few.
Agreed.
Show 23 quoted lines
>
> This removes the unnecessary formatting operation involved. We can see a
> consistent ~20% performance improvement when deleting from packed
> references.
>
> Benchmark 1: update-ref: delete ref (refcount = 100000, revision = master)
>   Time (mean ± σ):      28.7 ms ±   1.7 ms    [User: 22.5 ms, System: 5.9 ms]
>   Range (min … max):    26.7 ms …  33.3 ms    46 runs
>
> Benchmark 2: update-ref: delete ref (refcount = 100000, revision = b4/kn-speedup-packed-refs)
>   Time (mean ± σ):      23.8 ms ±   1.2 ms    [User: 17.5 ms, System: 6.0 ms]
>   Range (min … max):    22.1 ms …  27.7 ms    56 runs
>
> Summary
>   update-ref: delete ref (refcount = 100000, revision = b4/kn-speedup-packed-refs) ran
>     1.21 ± 0.09 times faster than update-ref: delete ref (refformat = files, refcount = 100000, revision = master)
>
> Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
> ---
> Changes in v2:
> - Instead of using the existing function, introduce a new
>   `write_packed_entry_raw()`.
> - Modify the commit to also note that we lose sanitization.
s/commit/commit message/
> - Link to v1: https://patch.msgid.link/20260930-kn-speedup-packed-refs-v1-1-111cd03d9b0e@gmail.com
Okay, I'm okay with this version.
-- 
Laters,
Toon
Karthik Nayak· Oct 6, 2026, 09:36 UTC · re: Toon Claes · lore

Re: [PATCH v2] packed-refs: use `fwrite()` when passing refs verbatim

Toon Claes <toon@iotcl.com> writes:
Show 17 quoted lines
> Karthik Nayak <karthik.188@gmail.com> writes:
>
>> The `write_with_updates()` function uses a `struct ref_iterator` to
>> iterate over all refs to write to the temporary packed-refs file. It
>> receives the iterator from `packed_ref_iterator_begin()` which takes a
>> snapshot of the 'packed-refs' file.
>>
>> While writing to the new packed-refs file, writes are routed via
>> `write_packed_entry()` which uses `fprintf()`. Even for references which
>> haven't changed, we use the same mechanism. Instead, let's track the
>> position of unchanged references in the snapshot iterator and directly
>> use `fwrite()`.
>>
>> With this, any sanitation which was happening as a side of reformatting
>
> s/side/side effect/ ?
>
That's probably better.
Show 32 quoted lines
>> is now lost. But that was never the job of this section of the code,
>> since the main intention is to simply rewrite the remaining refs post
>> deletion of the selective few.
>
> Agreed.
>
>>
>> This removes the unnecessary formatting operation involved. We can see a
>> consistent ~20% performance improvement when deleting from packed
>> references.
>>
>> Benchmark 1: update-ref: delete ref (refcount = 100000, revision = master)
>>   Time (mean ± σ):      28.7 ms ±   1.7 ms    [User: 22.5 ms, System: 5.9 ms]
>>   Range (min … max):    26.7 ms …  33.3 ms    46 runs
>>
>> Benchmark 2: update-ref: delete ref (refcount = 100000, revision = b4/kn-speedup-packed-refs)
>>   Time (mean ± σ):      23.8 ms ±   1.2 ms    [User: 17.5 ms, System: 6.0 ms]
>>   Range (min … max):    22.1 ms …  27.7 ms    56 runs
>>
>> Summary
>>   update-ref: delete ref (refcount = 100000, revision = b4/kn-speedup-packed-refs) ran
>>     1.21 ± 0.09 times faster than update-ref: delete ref (refformat = files, refcount = 100000, revision = master)
>>
>> Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
>> ---
>> Changes in v2:
>> - Instead of using the existing function, introduce a new
>>   `write_packed_entry_raw()`.
>> - Modify the commit to also note that we lose sanitization.
>
> s/commit/commit message/
>
This doesn't go into the commit itself, so I'll leave it as is.
Show 7 quoted lines
>> - Link to v1: https://patch.msgid.link/20260930-kn-speedup-packed-refs-v1-1-111cd03d9b0e@gmail.com
>
> Okay, I'm okay with this version.
>
> --
> Laters,
> Toon
Thanks for the review.
Karthik Nayak· Oct 6, 2026, 09:18 UTC · re: Karthik Nayak · lore

[PATCH v3] packed-refs: use `fwrite()` when passing refs verbatim

The `write_with_updates()` function uses a `struct ref_iterator` to iterate over all refs to write to the temporary packed-refs file. It receives the iterator from `packed_ref_iterator_begin()` which takes a snapshot of the 'packed-refs' file.

While writing to the new packed-refs file, writes are routed via `write_packed_entry()` which uses `fprintf()`. Even for references which haven't changed, we use the same mechanism. Instead, let's track the position of unchanged references in the snapshot iterator and directly use `fwrite()`.

With this, any sanitation which was happening as a side effect of reformatting is now lost. But that was never the job of this section of the code, since the main intention is to simply rewrite the remaining refs post deletion of the selective few.

This removes the unnecessary formatting operation involved. We can see a consistent ~20% performance improvement when deleting from packed references.

Benchmark 1: update-ref: delete ref (refcount = 100000, revision = master)
  Time (mean ± σ):      28.7 ms ±   1.7 ms    [User: 22.5 ms, System: 5.9 ms]
  Range (min … max):    26.7 ms …  33.3 ms    46 runs
Benchmark 2: update-ref: delete ref (refcount = 100000, revision = b4/kn-speedup-packed-refs)
  Time (mean ± σ):      23.8 ms ±   1.2 ms    [User: 17.5 ms, System: 6.0 ms]
  Range (min … max):    22.1 ms …  27.7 ms    56 runs
Summary
  update-ref: delete ref (refcount = 100000, revision = b4/kn-speedup-packed-refs) ran
    1.21 ± 0.09 times faster than update-ref: delete ref (refformat = files, refcount = 100000, revision = master)
Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
---
Changes in v3:
- Fixed a typo in the commit message.
- Link to v2: https://patch.msgid.link/20261002-kn-speedup-packed-refs-v2-1-2ae75772ebc1@gmail.com
Changes in v2:
- Instead of using the existing function, introduce a new
  `write_packed_entry_raw()`.
- Modify the commit to also note that we lose sanitization.
- Link to v1: https://patch.msgid.link/20260930-kn-speedup-packed-refs-v1-1-111cd03d9b0e@gmail.com
---
 refs/packed-backend.c | 30 +++++++++++++++++++++++++++---
 1 file changed, 27 insertions(+), 3 deletions(-)
Show changes to refs/packed-backend.c +27 −4
diff --git a/refs/packed-backend.c b/refs/packed-backend.c
index a73fc6aca7..43ad674cf4 100644
--- a/refs/packed-backend.c
+++ b/refs/packed-backend.c
@@ -879,6 +879,12 @@ struct packed_ref_iterator {
 	/* The current position in the snapshot's buffer: */
 	const char *pos;
 
+	/*
+	 * Start of the current record, set when advancing `pos`. Used to
+	 * pass records verbatim to `fwrite()`.
+	 */
+	const char *record_start;
+
 	/* The end of the part of the buffer that will be iterated over: */
 	const char *eof;
 
@@ -933,6 +939,7 @@ static int next_record(struct packed_ref_iterator *iter)
 	if (iter->pos == iter->eof)
 		return ITER_DONE;
 
+	iter->record_start = iter->pos;
 	iter->base.ref.flags = REF_ISPACKED;
 	p = iter->pos;
 
@@ -1233,6 +1240,19 @@ static int write_packed_entry(FILE *fh, const char *refname,
 	return 0;
 }
 
+/*
+ * Write an entry to the packed-refs file skip any formatting and directly
+ * write to  the file using `fwrite()`. e.g. when deleting references and
+ * remaining refs need to be written verbatim.
+ */
+static int write_packed_entry_raw(FILE *fh, const char *entry, size_t len)
+{
+	if (fwrite(entry, len, 1, fh) != 1)
+		return -1;
+
+	return 0;
+}
+
 int packed_refs_lock(struct ref_store *ref_store, int flags, struct strbuf *err)
 {
 	struct packed_ref_store *refs =
@@ -1530,9 +1550,13 @@ static enum ref_transaction_error write_with_updates(struct packed_ref_store *re
 		}
 
 		if (cmp < 0) {
-			/* Pass the old reference through. */
-			if (write_packed_entry(out, iter->ref.name,
-					       iter->ref.oid, iter->ref.peeled_oid))
+			const struct packed_ref_iterator *packed_iter =
+				(const struct packed_ref_iterator *)iter;
+			size_t len = packed_iter->pos - packed_iter->record_start;
+
+			if (write_packed_entry_raw(out,
+						   packed_iter->record_start,
+						   len))
 				goto write_error;
 
 			if ((ok = ref_iterator_advance(iter)) != ITER_OK) {

---
base-commit: a018953688f1b10bddf91bff8747068f5f4746a4
change-id: 20260930-kn-speedup-packed-refs-9868f5d0abe9


Thanks
- Karthik
Patrick Steinhardt· Oct 6, 2026, 12:31 UTC · re: Karthik Nayak · lore

Re: [PATCH v3] packed-refs: use `fwrite()` when passing refs verbatim

On Tue, Oct 06, 2026 at 11:18:40AM +0200, Karthik Nayak wrote:
Show 10 quoted lines
> The `write_with_updates()` function uses a `struct ref_iterator` to
> iterate over all refs to write to the temporary packed-refs file. It
> receives the iterator from `packed_ref_iterator_begin()` which takes a
> snapshot of the 'packed-refs' file.
> 
> While writing to the new packed-refs file, writes are routed via
> `write_packed_entry()` which uses `fprintf()`. Even for references which
> haven't changed, we use the same mechanism. Instead, let's track the
> position of unchanged references in the snapshot iterator and directly
> use `fwrite()`.

Nit, not worth a reroll on its own: you state the status quo and then jump to the solution right away without stating what the problem is with the status quo.

Show 13 quoted lines
> diff --git a/refs/packed-backend.c b/refs/packed-backend.c
> index a73fc6aca7..43ad674cf4 100644
> --- a/refs/packed-backend.c
> +++ b/refs/packed-backend.c
> @@ -879,6 +879,12 @@ struct packed_ref_iterator {
>  	/* The current position in the snapshot's buffer: */
>  	const char *pos;
>  
> +	/*
> +	 * Start of the current record, set when advancing `pos`. Used to
> +	 * pass records verbatim to `fwrite()`.
> +	 */
> +	const char *record_start;

The way this is written makes you think that `pos == record_start`, and thus one wonders why we even need this separate variable in the first place. So I assume that we modify `pos` in some cases without modifying the new variable at the same point in time. But if so, the above comment is not true anymore.

Show 16 quoted lines
> @@ -1233,6 +1240,19 @@ static int write_packed_entry(FILE *fh, const char *refname,
>  	return 0;
>  }
>  
> +/*
> + * Write an entry to the packed-refs file skip any formatting and directly
> + * write to  the file using `fwrite()`. e.g. when deleting references and
> + * remaining refs need to be written verbatim.
> + */
> +static int write_packed_entry_raw(FILE *fh, const char *entry, size_t len)
> +{
> +	if (fwrite(entry, len, 1, fh) != 1)
> +		return -1;
> +
> +	return 0;
> +}
Nit: this function is somewhat ponitless as it's a trivial wrapper
around fwrite(3).
Show 10 quoted lines
> @@ -1530,9 +1550,13 @@ static enum ref_transaction_error write_with_updates(struct packed_ref_store *re
>  		}
>  
>  		if (cmp < 0) {
> -			/* Pass the old reference through. */
> -			if (write_packed_entry(out, iter->ref.name,
> -					       iter->ref.oid, iter->ref.peeled_oid))
> +			const struct packed_ref_iterator *packed_iter =
> +				(const struct packed_ref_iterator *)iter;
> +			size_t len = packed_iter->pos - packed_iter->record_start;

Alright, so `pos` and `record_start` do get advanced independent from one another. So the comment that you have for `record_start` is inaccurate indeed.

> +			if (write_packed_entry_raw(out,
> +						   packed_iter->record_start,
> +						   len))
>  				goto write_error;

Other than those nits though I'm quite happy about this change. A 20% win is nothing to scoff at, doubly so because reference deletions are extremely expensive once your repository reaches a certain number of refs.

Thanks!
Patrick
Karthik Nayak· Oct 6, 2026, 21:11 UTC · re: Patrick Steinhardt · lore

Re: [PATCH v3] packed-refs: use `fwrite()` when passing refs verbatim

Patrick Steinhardt <ps@pks.im> writes:
Show 16 quoted lines
> On Tue, Oct 06, 2026 at 11:18:40AM +0200, Karthik Nayak wrote:
>> The `write_with_updates()` function uses a `struct ref_iterator` to
>> iterate over all refs to write to the temporary packed-refs file. It
>> receives the iterator from `packed_ref_iterator_begin()` which takes a
>> snapshot of the 'packed-refs' file.
>>
>> While writing to the new packed-refs file, writes are routed via
>> `write_packed_entry()` which uses `fprintf()`. Even for references which
>> haven't changed, we use the same mechanism. Instead, let's track the
>> position of unchanged references in the snapshot iterator and directly
>> use `fwrite()`.
>
> Nit, not worth a reroll on its own: you state the status quo and then
> jump to the solution right away without stating what the problem is with
> the status quo.
>
Fair, let me amend.
Show 20 quoted lines
>> diff --git a/refs/packed-backend.c b/refs/packed-backend.c
>> index a73fc6aca7..43ad674cf4 100644
>> --- a/refs/packed-backend.c
>> +++ b/refs/packed-backend.c
>> @@ -879,6 +879,12 @@ struct packed_ref_iterator {
>>  	/* The current position in the snapshot's buffer: */
>>  	const char *pos;
>>
>> +	/*
>> +	 * Start of the current record, set when advancing `pos`. Used to
>> +	 * pass records verbatim to `fwrite()`.
>> +	 */
>> +	const char *record_start;
>
> The way this is written makes you think that `pos == record_start`, and
> thus one wonders why we even need this separate variable in the first
> place. So I assume that we modify `pos` in some cases without modifying
> the new variable at the same point in time. But if so, the above comment
> is not true anymore.
>

Hmm. I only state that this is 'set _when_ advancing `pos`', Why do you think that this would mean `pos == record_start`?

Show 19 quoted lines
>> @@ -1233,6 +1240,19 @@ static int write_packed_entry(FILE *fh, const char *refname,
>>  	return 0;
>>  }
>>
>> +/*
>> + * Write an entry to the packed-refs file skip any formatting and directly
>> + * write to  the file using `fwrite()`. e.g. when deleting references and
>> + * remaining refs need to be written verbatim.
>> + */
>> +static int write_packed_entry_raw(FILE *fh, const char *entry, size_t len)
>> +{
>> +	if (fwrite(entry, len, 1, fh) != 1)
>> +		return -1;
>> +
>> +	return 0;
>> +}
>
> Nit: this function is somewhat ponitless as it's a trivial wrapper
> around fwrite(3).
Yeah fair, we can simply inline this.
Show 28 quoted lines
>> @@ -1530,9 +1550,13 @@ static enum ref_transaction_error write_with_updates(struct packed_ref_store *re
>>  		}
>>
>>  		if (cmp < 0) {
>> -			/* Pass the old reference through. */
>> -			if (write_packed_entry(out, iter->ref.name,
>> -					       iter->ref.oid, iter->ref.peeled_oid))
>> +			const struct packed_ref_iterator *packed_iter =
>> +				(const struct packed_ref_iterator *)iter;
>> +			size_t len = packed_iter->pos - packed_iter->record_start;
>
> Alright, so `pos` and `record_start` do get advanced independent from
> one another. So the comment that you have for `record_start` is
> inaccurate indeed.
>
>> +			if (write_packed_entry_raw(out,
>> +						   packed_iter->record_start,
>> +						   len))
>>  				goto write_error;
>
> Other than those nits though I'm quite happy about this change. A 20%
> win is nothing to scoff at, doubly so because reference deletions are
> extremely expensive once your repository reaches a certain number of
> refs.
>
> Thanks!
>
> Patrick
Yeah I agree, I'm quite happy to see the 20% drop too :)

Thanks for the review. Will send in a new version once we sort out the comment on the field.

Patrick Steinhardt· Oct 7, 2026, 05:12 UTC · re: Karthik Nayak · lore

Re: [PATCH v3] packed-refs: use `fwrite()` when passing refs verbatim

On Tue, Oct 06, 2026 at 05:11:57PM -0400, Karthik Nayak wrote:
Show 25 quoted lines
> Patrick Steinhardt <ps@pks.im> writes:
> > On Tue, Oct 06, 2026 at 11:18:40AM +0200, Karthik Nayak wrote:
> >> diff --git a/refs/packed-backend.c b/refs/packed-backend.c
> >> index a73fc6aca7..43ad674cf4 100644
> >> --- a/refs/packed-backend.c
> >> +++ b/refs/packed-backend.c
> >> @@ -879,6 +879,12 @@ struct packed_ref_iterator {
> >>  	/* The current position in the snapshot's buffer: */
> >>  	const char *pos;
> >>
> >> +	/*
> >> +	 * Start of the current record, set when advancing `pos`. Used to
> >> +	 * pass records verbatim to `fwrite()`.
> >> +	 */
> >> +	const char *record_start;
> >
> > The way this is written makes you think that `pos == record_start`, and
> > thus one wonders why we even need this separate variable in the first
> > place. So I assume that we modify `pos` in some cases without modifying
> > the new variable at the same point in time. But if so, the above comment
> > is not true anymore.
> >
> 
> Hmm. I only state that this is 'set _when_ advancing `pos`', Why do you
> think that this would mean `pos == record_start`?

To me it reads as "whenever we advance `pos`, then we set `record_start`". Which is not the case, we also sometimes advance `pos` without setting it.

Patrick
Karthik Nayak· Oct 7, 2026, 09:25 UTC · re: Patrick Steinhardt · lore

Re: [PATCH v3] packed-refs: use `fwrite()` when passing refs verbatim

Patrick Steinhardt <ps@pks.im> writes:
Show 32 quoted lines
> On Tue, Oct 06, 2026 at 05:11:57PM -0400, Karthik Nayak wrote:
>> Patrick Steinhardt <ps@pks.im> writes:
>> > On Tue, Oct 06, 2026 at 11:18:40AM +0200, Karthik Nayak wrote:
>> >> diff --git a/refs/packed-backend.c b/refs/packed-backend.c
>> >> index a73fc6aca7..43ad674cf4 100644
>> >> --- a/refs/packed-backend.c
>> >> +++ b/refs/packed-backend.c
>> >> @@ -879,6 +879,12 @@ struct packed_ref_iterator {
>> >>  	/* The current position in the snapshot's buffer: */
>> >>  	const char *pos;
>> >>
>> >> +	/*
>> >> +	 * Start of the current record, set when advancing `pos`. Used to
>> >> +	 * pass records verbatim to `fwrite()`.
>> >> +	 */
>> >> +	const char *record_start;
>> >
>> > The way this is written makes you think that `pos == record_start`, and
>> > thus one wonders why we even need this separate variable in the first
>> > place. So I assume that we modify `pos` in some cases without modifying
>> > the new variable at the same point in time. But if so, the above comment
>> > is not true anymore.
>> >
>>
>> Hmm. I only state that this is 'set _when_ advancing `pos`', Why do you
>> think that this would mean `pos == record_start`?
>
> To me it reads as "whenever we advance `pos`, then we set
> `record_start`". Which is not the case, we also sometimes advance `pos`
> without setting it.
>
> Patrick

Okay, let me simplify it and simply state its purpose, its implementation will anyways be traceable via the code. Thanks

Junio C Hamano· Oct 6, 2026, 16:19 UTC · re: Karthik Nayak · lore

Re: [PATCH v3] packed-refs: use `fwrite()` when passing refs verbatim

Karthik Nayak <karthik.188@gmail.com> writes:
> With this, any sanitation which was happening as a side effect of
> reformatting is now lost. But that was never the job of this section of
> the code, since the main intention is to simply rewrite the remaining
> refs post deletion of the selective few.
Miniscule nit, but isn't that sanitization, not sanitation?
Karthik Nayak· Oct 6, 2026, 20:59 UTC · re: Junio C Hamano · lore

Re: [PATCH v3] packed-refs: use `fwrite()` when passing refs verbatim

Junio C Hamano <gitster@pobox.com> writes:
Show 8 quoted lines
> Karthik Nayak <karthik.188@gmail.com> writes:
>
>> With this, any sanitation which was happening as a side effect of
>> reformatting is now lost. But that was never the job of this section of
>> the code, since the main intention is to simply rewrite the remaining
>> refs post deletion of the selective few.
>
> Miniscule nit, but isn't that sanitization, not sanitation?
Yikes. Yes. Embarrassing
Karthik Nayak· Oct 7, 2026, 11:20 UTC · re: Karthik Nayak · lore

[PATCH v4] packed-refs: use `fwrite()` when passing refs verbatim

The `write_with_updates()` function uses a `struct ref_iterator` to iterate over all refs to write to the temporary packed-refs file. It receives the iterator from `packed_ref_iterator_begin()` which takes a snapshot of the 'packed-refs' file.

While writing to the new packed-refs file, writes are routed via `write_packed_entry()` which uses `fprintf()`. Even for references which haven't changed, we use the same mechanism. This is slow since we need to do string parsing and formatting. Instead, let's track the position of unchanged references in the snapshot iterator and directly write verbatim via `fwrite()`.

With this, any sanitization which was happening as a side effect of reformatting is now lost. But that was never the job of this section of the code, since the main intention is to simply rewrite the remaining refs post deletion of the selective few.

This removes the unnecessary formatting operation involved. We can see a consistent ~20% performance improvement when deleting from packed references.

Benchmark 1: update-ref: delete ref (refcount = 100000, revision = master)
  Time (mean ± σ):      28.7 ms ±   1.7 ms    [User: 22.5 ms, System: 5.9 ms]
  Range (min … max):    26.7 ms …  33.3 ms    46 runs
Benchmark 2: update-ref: delete ref (refcount = 100000, revision = b4/kn-speedup-packed-refs)
  Time (mean ± σ):      23.8 ms ±   1.2 ms    [User: 17.5 ms, System: 6.0 ms]
  Range (min … max):    22.1 ms …  27.7 ms    56 runs
Summary
  update-ref: delete ref (refcount = 100000, revision = b4/kn-speedup-packed-refs) ran
    1.21 ± 0.09 times faster than update-ref: delete ref (refcount = 100000, revision = master)
Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
---
Changes in v4:
- Modify the commit message to state the issue with the previous
  approach.
- Modify the comment for `record_start` to remove ambiguity around its
  setting.
- Remove `write_packed_entry_raw()` and inline the call to `fwrite()`.
- Link to v3: https://patch.msgid.link/20261006-kn-speedup-packed-refs-v3-1-a1c76b1df9e0@gmail.com
Changes in v3:
- Fixed a typo in the commit message.
- Link to v2: https://patch.msgid.link/20261002-kn-speedup-packed-refs-v2-1-2ae75772ebc1@gmail.com
Changes in v2:
- Instead of using the existing function, introduce a new
  `write_packed_entry_raw()`.
- Modify the commit to also note that we lose sanitization.
- Link to v1: https://patch.msgid.link/20260930-kn-speedup-packed-refs-v1-1-111cd03d9b0e@gmail.com
---
 refs/packed-backend.c | 16 +++++++++++++---
 1 file changed, 13 insertions(+), 3 deletions(-)
Show changes to refs/packed-backend.c +13 −4
diff --git a/refs/packed-backend.c b/refs/packed-backend.c
index a73fc6aca7..53511f2d91 100644
--- a/refs/packed-backend.c
+++ b/refs/packed-backend.c
@@ -879,6 +879,9 @@ struct packed_ref_iterator {
 	/* The current position in the snapshot's buffer: */
 	const char *pos;
 
+	/* The starting position of the current ref record. */
+	const char *record_start;
+
 	/* The end of the part of the buffer that will be iterated over: */
 	const char *eof;
 
@@ -933,6 +936,7 @@ static int next_record(struct packed_ref_iterator *iter)
 	if (iter->pos == iter->eof)
 		return ITER_DONE;
 
+	iter->record_start = iter->pos;
 	iter->base.ref.flags = REF_ISPACKED;
 	p = iter->pos;
 
@@ -1530,9 +1534,15 @@ static enum ref_transaction_error write_with_updates(struct packed_ref_store *re
 		}
 
 		if (cmp < 0) {
-			/* Pass the old reference through. */
-			if (write_packed_entry(out, iter->ref.name,
-					       iter->ref.oid, iter->ref.peeled_oid))
+			const struct packed_ref_iterator *packed_iter =
+				(const struct packed_ref_iterator *)iter;
+			size_t len = packed_iter->pos - packed_iter->record_start;
+
+			/*
+			 * Skip any formatting and directly write to the packed-refs file
+			 * when deleting references and writing the remaining refs verbatim.
+			 */
+			if (fwrite(packed_iter->record_start, len, 1, out) != 1)
 				goto write_error;
 
 			if ((ok = ref_iterator_advance(iter)) != ITER_OK) {

---
base-commit: a018953688f1b10bddf91bff8747068f5f4746a4
change-id: 20260930-kn-speedup-packed-refs-9868f5d0abe9


Thanks
- Karthik
Patrick Steinhardt· Oct 7, 2026, 11:52 UTC · re: Karthik Nayak · lore

Re: [PATCH v4] packed-refs: use `fwrite()` when passing refs verbatim

On Wed, Oct 07, 2026 at 01:20:16PM +0200, Karthik Nayak wrote:
Show 7 quoted lines
> Changes in v4:
> - Modify the commit message to state the issue with the previous
>   approach.
> - Modify the comment for `record_start` to remove ambiguity around its
>   setting.
> - Remove `write_packed_entry_raw()` and inline the call to `fwrite()`.
> - Link to v3: https://patch.msgid.link/20261006-kn-speedup-packed-refs-v3-1-a1c76b1df9e0@gmail.com
Thanks, I'm happy with this version.
Patrick
Toon Claes· Oct 8, 2026, 11:41 UTC · re: Patrick Steinhardt · lore

Re: [PATCH v4] packed-refs: use `fwrite()` when passing refs verbatim

Patrick Steinhardt <ps@pks.im> writes:
Show 10 quoted lines
> On Wed, Oct 07, 2026 at 01:20:16PM +0200, Karthik Nayak wrote:
>> Changes in v4:
>> - Modify the commit message to state the issue with the previous
>>   approach.
>> - Modify the comment for `record_start` to remove ambiguity around its
>>   setting.
>> - Remove `write_packed_entry_raw()` and inline the call to `fwrite()`.
>> - Link to v3: https://patch.msgid.link/20261006-kn-speedup-packed-refs-v3-1-a1c76b1df9e0@gmail.com
>
> Thanks, I'm happy with this version.
Also no further comments from me.
-- 
Laters,
Toon

← back to recent threads