git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH v2 06/10] wrapper: introduce writev(3p) wrappers

From
Patrick Steinhardt <ps@pks.im>
Date
Mar 3, 2026, 15:00 UTC
Message-ID
<20260303-pks-upload-pack-write-contention-v2-6-7321830f08fe@pks.im>
In-Reply-To
<20260303-pks-upload-pack-write-contention-v2-0-7321830f08fe@pks.im>

In the preceding commit we have added a compatibility wrapper for the writev(3p) syscall. Introduce some generic wrappers for this function that we nowadays take for granted in the Git codebase.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 wrapper.c      | 41 +++++++++++++++++++++++++++++++++++++++++
 wrapper.h      |  9 +++++++++
 write-or-die.c |  8 ++++++++
 write-or-die.h |  1 +
 4 files changed, 59 insertions(+)
diff --git a/wrapper.c b/wrapper.c
index 16f5a63fbb..be8fa575e6 100644
--- a/wrapper.c
+++ b/wrapper.c
@@ -323,6 +323,47 @@ ssize_t write_in_full(int fd, const void *buf, size_t count)
 	return total;
 }
 
+ssize_t writev_in_full(int fd, struct iovec *iov, int iovcnt)
+{
+	ssize_t total_written = 0;
+
+	while (iovcnt) {
+		ssize_t bytes_written = writev(fd, iov, iovcnt);
+		if (bytes_written < 0) {
+			if (errno == EINTR || errno == EAGAIN)
+				continue;
+			return -1;
+		}
+		if (!bytes_written) {
+			errno = ENOSPC;
+			return -1;
+		}
+
+		total_written += bytes_written;
+
+		/*
+		 * We first need to discard any iovec entities that have been
+		 * fully written.
+		 */
+		while (iovcnt && (size_t)bytes_written >= iov->iov_len) {
+			bytes_written -= iov->iov_len;
+			iov++;
+			iovcnt--;
+		}
+
+		/*
+		 * Finally, we need to adjust the last iovec in case we have
+		 * performed a partial write.
+		 */
+		if (iovcnt && bytes_written) {
+			iov->iov_base = (char *) iov->iov_base + bytes_written;
+			iov->iov_len -= bytes_written;
+		}
+	}
+
+	return total_written;
+}
+
 ssize_t pread_in_full(int fd, void *buf, size_t count, off_t offset)
 {
 	char *p = buf;
diff --git a/wrapper.h b/wrapper.h
index 15ac3bab6e..27519b32d1 100644
--- a/wrapper.h
+++ b/wrapper.h
@@ -47,6 +47,15 @@ ssize_t read_in_full(int fd, void *buf, size_t count);
 ssize_t write_in_full(int fd, const void *buf, size_t count);
 ssize_t pread_in_full(int fd, void *buf, size_t count, off_t offset);
 
+/*
+ * Try to write all iovecs. Returns -1 in case an error occurred with a proper
+ * errno set, the number of bytes written otherwise.
+ *
+ * Note that the iovec will be modified as a result of this call to adjust for
+ * partial writes!
+ */
+ssize_t writev_in_full(int fd, struct iovec *iov, int iovcnt);
+
 static inline ssize_t write_str_in_full(int fd, const char *str)
 {
 	return write_in_full(fd, str, strlen(str));
diff --git a/write-or-die.c b/write-or-die.c
index 01a9a51fa2..5f522fb728 100644
--- a/write-or-die.c
+++ b/write-or-die.c
@@ -96,6 +96,14 @@ void write_or_die(int fd, const void *buf, size_t count)
 	}
 }
 
+void writev_or_die(int fd, struct iovec *iov, int iovlen)
+{
+	if (writev_in_full(fd, iov, iovlen) < 0) {
+		check_pipe(errno);
+		die_errno("writev error");
+	}
+}
+
 void fwrite_or_die(FILE *f, const void *buf, size_t count)
 {
 	if (fwrite(buf, 1, count, f) != count)
diff --git a/write-or-die.h b/write-or-die.h
index 65a5c42a47..ae3d7d88b8 100644
--- a/write-or-die.h
+++ b/write-or-die.h
@@ -7,6 +7,7 @@ void fprintf_or_die(FILE *, const char *fmt, ...);
 void fwrite_or_die(FILE *f, const void *buf, size_t count);
 void fflush_or_die(FILE *f);
 void write_or_die(int fd, const void *buf, size_t count);
+void writev_or_die(int fd, struct iovec *iov, int iovlen);
 
 /*
  * These values are used to help identify parts of a repository to fsync.
-- 
2.53.0.697.g625c4fb2da.dirty
Previous: Patrick SteinhardtNext: Patrick Steinhardt
Message 29 of 53 in “upload-pack: reduce lock contention when writing packfile data”
  1. 0/2 upload-pack: reduce lock contention when writing packfile dataPatrick Steinhardt, Feb 27, 2026
  2. 1/2 upload-pack: fix debug statement when flushing packfile dataPatrick Steinhardt, Feb 27, 2026
  3. 2/2 upload-pack: reduce lock contention when writing packfile dataPatrick Steinhardt, Feb 27, 2026
  4. brian m. carlsonFeb 27, 2026
  5. Patrick SteinhardtFeb 27, 2026
  6. Junio C HamanoFeb 27, 2026
  7. Jeff KingFeb 27, 2026
  8. Patrick SteinhardtMar 2, 2026
  9. Jeff KingMar 2, 2026
  10. Patrick SteinhardtMar 3, 2026
  11. Jeff KingMar 3, 2026
  12. Patrick SteinhardtMar 3, 2026
  13. 00/10 upload-pack: reduce lock contention when writing packfile dataPatrick Steinhardt, Mar 3, 2026
  14. 01/10 upload-pack: fix debug statement when flushing packfile dataPatrick Steinhardt, Mar 3, 2026
  15. 02/10 upload-pack: adapt keepalives based on bufferingPatrick Steinhardt, Mar 3, 2026
  16. Jeff KingMar 5, 2026
  17. Patrick SteinhardtMar 10, 2026
  18. 03/10 upload-pack: reduce lock contention when writing packfile dataPatrick Steinhardt, Mar 3, 2026
  19. Jeff KingMar 5, 2026
  20. Patrick SteinhardtMar 10, 2026
  21. 04/10 git-compat-util: introduce `cast_size_t_to_ssize_t()`Patrick Steinhardt, Mar 3, 2026
  22. 05/10 compat/posix: introduce writev(3p) wrapperPatrick Steinhardt, Mar 3, 2026
  23. Junio C HamanoMar 4, 2026
  24. Jeff KingMar 5, 2026
  25. brian m. carlsonMar 5, 2026
  26. Johannes SixtMar 5, 2026
  27. brian m. carlsonMar 5, 2026
  28. Patrick SteinhardtMar 10, 2026
  29. 06/10 wrapper: introduce writev(3p) wrappersPatrick Steinhardt, Mar 3, 2026
  30. 07/10 sideband: use writev(3p) to send pktlinesPatrick Steinhardt, Mar 3, 2026
  31. Junio C HamanoMar 4, 2026
  32. 08/10 csum-file: introduce `hashfd_ext()`Patrick Steinhardt, Mar 3, 2026
  33. Junio C HamanoMar 4, 2026
  34. Patrick SteinhardtMar 10, 2026
  35. 09/10 csum-file: drop `hashfd_throughput()`Patrick Steinhardt, Mar 3, 2026
  36. 10/10 builtin/pack-objects: reduce lock contention when writing packfile dataPatrick Steinhardt, Mar 3, 2026
  37. 00/10 upload-pack: reduce lock contention when writing packfile dataPatrick Steinhardt, Mar 10, 2026
  38. 01/10 upload-pack: fix debug statement when flushing packfile dataPatrick Steinhardt, Mar 10, 2026
  39. 02/10 upload-pack: adapt keepalives based on bufferingPatrick Steinhardt, Mar 10, 2026
  40. 03/10 upload-pack: prefer flushing data over sending keepalivePatrick Steinhardt, Mar 10, 2026
  41. Junio C HamanoMar 10, 2026
  42. Patrick SteinhardtMar 10, 2026
  43. 04/10 upload-pack: reduce lock contention when writing packfile dataPatrick Steinhardt, Mar 10, 2026
  44. 05/10 compat/posix: introduce writev(3p) wrapperPatrick Steinhardt, Mar 10, 2026
  45. Junio C HamanoMar 10, 2026
  46. 06/10 wrapper: introduce writev(3p) wrappersPatrick Steinhardt, Mar 10, 2026
  47. 07/10 sideband: use writev(3p) to send pktlinesPatrick Steinhardt, Mar 10, 2026
  48. 08/10 csum-file: introduce `hashfd_ext()`Patrick Steinhardt, Mar 10, 2026
  49. 09/10 csum-file: drop `hashfd_throughput()`Patrick Steinhardt, Mar 10, 2026
  50. 10/10 builtin/pack-objects: reduce lock contention when writing packfile dataPatrick Steinhardt, Mar 10, 2026
  51. Junio C HamanoMar 10, 2026
  52. Johannes SixtMar 10, 2026
  53. Patrick SteinhardtMar 11, 2026

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.