git/list[1] front-page[2] threads[3] people[4] search[5] about
wed 2026-10-07 17:26 UTC

[PATCH v4 06/10] wrapper: introduce writev(3p) wrappers

From
Patrick Steinhardt <ps@pks.im>
Date
Mar 13, 2026, 06:45 UTC
Message-ID
<20260313-pks-upload-pack-write-contention-v4-6-7a9668061f7f@pks.im>
In-Reply-To
<20260313-pks-upload-pack-write-contention-v4-0-7a9668061f7f@pks.im>

In the preceding commit we have added a compatibility wrapper for the writev(3p) syscall. Introduce some generic wrappers for this function that we nowadays take for granted in the Git codebase.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 wrapper.c      | 41 +++++++++++++++++++++++++++++++++++++++++
 wrapper.h      |  9 +++++++++
 write-or-die.c |  8 ++++++++
 write-or-die.h |  1 +
 4 files changed, 59 insertions(+)
diff --git a/wrapper.c b/wrapper.c
index 16f5a63fbb..be8fa575e6 100644
--- a/wrapper.c
+++ b/wrapper.c
@@ -323,6 +323,47 @@ ssize_t write_in_full(int fd, const void *buf, size_t count)
 	return total;
 }
 
+ssize_t writev_in_full(int fd, struct iovec *iov, int iovcnt)
+{
+	ssize_t total_written = 0;
+
+	while (iovcnt) {
+		ssize_t bytes_written = writev(fd, iov, iovcnt);
+		if (bytes_written < 0) {
+			if (errno == EINTR || errno == EAGAIN)
+				continue;
+			return -1;
+		}
+		if (!bytes_written) {
+			errno = ENOSPC;
+			return -1;
+		}
+
+		total_written += bytes_written;
+
+		/*
+		 * We first need to discard any iovec entities that have been
+		 * fully written.
+		 */
+		while (iovcnt && (size_t)bytes_written >= iov->iov_len) {
+			bytes_written -= iov->iov_len;
+			iov++;
+			iovcnt--;
+		}
+
+		/*
+		 * Finally, we need to adjust the last iovec in case we have
+		 * performed a partial write.
+		 */
+		if (iovcnt && bytes_written) {
+			iov->iov_base = (char *) iov->iov_base + bytes_written;
+			iov->iov_len -= bytes_written;
+		}
+	}
+
+	return total_written;
+}
+
 ssize_t pread_in_full(int fd, void *buf, size_t count, off_t offset)
 {
 	char *p = buf;
diff --git a/wrapper.h b/wrapper.h
index 15ac3bab6e..27519b32d1 100644
--- a/wrapper.h
+++ b/wrapper.h
@@ -47,6 +47,15 @@ ssize_t read_in_full(int fd, void *buf, size_t count);
 ssize_t write_in_full(int fd, const void *buf, size_t count);
 ssize_t pread_in_full(int fd, void *buf, size_t count, off_t offset);
 
+/*
+ * Try to write all iovecs. Returns -1 in case an error occurred with a proper
+ * errno set, the number of bytes written otherwise.
+ *
+ * Note that the iovec will be modified as a result of this call to adjust for
+ * partial writes!
+ */
+ssize_t writev_in_full(int fd, struct iovec *iov, int iovcnt);
+
 static inline ssize_t write_str_in_full(int fd, const char *str)
 {
 	return write_in_full(fd, str, strlen(str));
diff --git a/write-or-die.c b/write-or-die.c
index 01a9a51fa2..5f522fb728 100644
--- a/write-or-die.c
+++ b/write-or-die.c
@@ -96,6 +96,14 @@ void write_or_die(int fd, const void *buf, size_t count)
 	}
 }
 
+void writev_or_die(int fd, struct iovec *iov, int iovlen)
+{
+	if (writev_in_full(fd, iov, iovlen) < 0) {
+		check_pipe(errno);
+		die_errno("writev error");
+	}
+}
+
 void fwrite_or_die(FILE *f, const void *buf, size_t count)
 {
 	if (fwrite(buf, 1, count, f) != count)
diff --git a/write-or-die.h b/write-or-die.h
index 65a5c42a47..ae3d7d88b8 100644
--- a/write-or-die.h
+++ b/write-or-die.h
@@ -7,6 +7,7 @@ void fprintf_or_die(FILE *, const char *fmt, ...);
 void fwrite_or_die(FILE *f, const void *buf, size_t count);
 void fflush_or_die(FILE *f);
 void write_or_die(int fd, const void *buf, size_t count);
+void writev_or_die(int fd, struct iovec *iov, int iovlen);
 
 /*
  * These values are used to help identify parts of a repository to fsync.
-- 
2.53.0.904.g2727be2e99.dirty
Previous: Patrick SteinhardtNext: Patrick Steinhardt
Message 7 of 11 in “upload-pack: reduce lock contention when writing packfile data”
  1. 00/10 upload-pack: reduce lock contention when writing packfile dataPatrick Steinhardt, Mar 13, 2026
  2. 01/10 upload-pack: fix debug statement when flushing packfile dataPatrick Steinhardt, Mar 13, 2026
  3. 02/10 upload-pack: adapt keepalives based on bufferingPatrick Steinhardt, Mar 13, 2026
  4. 03/10 upload-pack: prefer flushing data over sending keepalivePatrick Steinhardt, Mar 13, 2026
  5. 04/10 upload-pack: reduce lock contention when writing packfile dataPatrick Steinhardt, Mar 13, 2026
  6. 05/10 compat/posix: introduce writev(3p) wrapperPatrick Steinhardt, Mar 13, 2026
  7. 06/10 wrapper: introduce writev(3p) wrappersPatrick Steinhardt, Mar 13, 2026
  8. 07/10 sideband: use writev(3p) to send pktlinesPatrick Steinhardt, Mar 13, 2026
  9. 08/10 csum-file: introduce `hashfd_ext()`Patrick Steinhardt, Mar 13, 2026
  10. 09/10 csum-file: drop `hashfd_throughput()`Patrick Steinhardt, Mar 13, 2026
  11. 10/10 builtin/pack-objects: reduce lock contention when writing packfile dataPatrick Steinhardt, Mar 13, 2026

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.