git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH v7 06/10] pkt-line: add functions to read/write flush terminated packet streams

From
larsxschneider@gmail.com <larsxschneider@gmail.com>
Date
Sep 8, 2016, 18:21 UTC
Message-ID
<20160908182132.50788-7-larsxschneider@gmail.com>
In-Reply-To
<20160908182132.50788-1-larsxschneider@gmail.com>
From: Lars Schneider <larsxschneider@gmail.com>

write_packetized_from_fd() and write_packetized_from_buf() write a stream of packets. All content packets use the maximal packet size except for the last one. After the last content packet a `flush` control packet is written.

read_packetized_to_buf() reads arbitrary sized packets until it detects a `flush` packet.

Signed-off-by: Lars Schneider <larsxschneider@gmail.com>
---
 pkt-line.c | 68 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 pkt-line.h |  7 +++++++
 2 files changed, 75 insertions(+)
diff --git a/pkt-line.c b/pkt-line.c
index 1d3d725..5001a07 100644
--- a/pkt-line.c
+++ b/pkt-line.c
@@ -209,6 +209,47 @@ void packet_buf_write(struct strbuf *buf, const char *fmt, ...)
 	va_end(args);
 }
 
+int write_packetized_from_fd(int fd_in, int fd_out)
+{
+	static char buf[PKTLINE_DATA_MAXLEN];
+	int err = 0;
+	ssize_t bytes_to_write;
+
+	while (!err) {
+		bytes_to_write = xread(fd_in, buf, sizeof(buf));
+		if (bytes_to_write < 0)
+			return COPY_READ_ERROR;
+		if (bytes_to_write == 0)
+			break;
+		err = packet_write_gently(fd_out, buf, bytes_to_write);
+	}
+	if (!err)
+		err = packet_flush_gently(fd_out);
+	return err;
+}
+
+int write_packetized_from_buf(const char *src_in, size_t len, int fd_out)
+{
+	static char buf[PKTLINE_DATA_MAXLEN];
+	int err = 0;
+	size_t bytes_written = 0;
+	size_t bytes_to_write;
+
+	while (!err) {
+		if ((len - bytes_written) > sizeof(buf))
+			bytes_to_write = sizeof(buf);
+		else
+			bytes_to_write = len - bytes_written;
+		if (bytes_to_write == 0)
+			break;
+		err = packet_write_gently(fd_out, src_in + bytes_written, bytes_to_write);
+		bytes_written += bytes_to_write;
+	}
+	if (!err)
+		err = packet_flush_gently(fd_out);
+	return err;
+}
+
 static int get_packet_data(int fd, char **src_buf, size_t *src_size,
 			   void *dst, unsigned size, int options)
 {
@@ -318,3 +359,30 @@ char *packet_read_line_buf(char **src, size_t *src_len, int *dst_len)
 {
 	return packet_read_line_generic(-1, src, src_len, dst_len);
 }
+
+ssize_t read_packetized_to_buf(int fd_in, struct strbuf *sb_out)
+{
+	int paket_len;
+	int options = PACKET_READ_GENTLE_ON_EOF;
+
+	size_t oldlen = sb_out->len;
+	size_t oldalloc = sb_out->alloc;
+
+	for (;;) {
+		strbuf_grow(sb_out, PKTLINE_DATA_MAXLEN+1);
+		paket_len = packet_read(fd_in, NULL, NULL,
+			sb_out->buf + sb_out->len, PKTLINE_DATA_MAXLEN+1, options);
+		if (paket_len <= 0)
+			break;
+		sb_out->len += paket_len;
+	}
+
+	if (paket_len < 0) {
+		if (oldalloc == 0)
+			strbuf_release(sb_out);
+		else
+			strbuf_setlen(sb_out, oldlen);
+		return paket_len;
+	}
+	return sb_out->len - oldlen;
+}
diff --git a/pkt-line.h b/pkt-line.h
index 3fa0899..6df8449 100644
--- a/pkt-line.h
+++ b/pkt-line.h
@@ -25,6 +25,8 @@ void packet_buf_flush(struct strbuf *buf);
 void packet_buf_write(struct strbuf *buf, const char *fmt, ...) __attribute__((format (printf, 2, 3)));
 int packet_flush_gently(int fd);
 int packet_write_fmt_gently(int fd, const char *fmt, ...) __attribute__((format (printf, 2, 3)));
+int write_packetized_from_fd(int fd_in, int fd_out);
+int write_packetized_from_buf(const char *src_in, size_t len, int fd_out);
 
 /*
  * Read a packetized line into the buffer, which must be at least size bytes
@@ -77,6 +79,11 @@ char *packet_read_line(int fd, int *size);
  */
 char *packet_read_line_buf(char **src_buf, size_t *src_len, int *size);
 
+/*
+ * Reads a stream of variable sized packets until a flush packet is detected.
+ */
+ssize_t read_packetized_to_buf(int fd_in, struct strbuf *sb_out);
+
 #define DEFAULT_PACKET_MAX 1000
 #define LARGE_PACKET_MAX 65520
 extern char packet_buffer[LARGE_PACKET_MAX];
-- 
2.10.0
Previous: Junio C HamanoNext: Stefan Beller
Message 21 of 45 in “Git filter protocol”
  1. 00/10 Git filter protocollarsxschneider@gmail.com, Sep 8, 2016
  2. 01/10 pkt-line: rename packet_write() to packet_write_fmt()larsxschneider@gmail.com, Sep 8, 2016
  3. 02/10 pkt-line: extract set_packet_header()larsxschneider@gmail.com, Sep 8, 2016
  4. 03/10 pkt-line: add packet_write_fmt_gently()larsxschneider@gmail.com, Sep 8, 2016
  5. Stefan BellerSep 8, 2016
  6. Lars SchneiderSep 11, 2016
  7. Stefan BellerSep 11, 2016
  8. Lars SchneiderSep 12, 2016
  9. 04/10 pkt-line: add packet_flush_gently()larsxschneider@gmail.com, Sep 8, 2016
  10. Junio C HamanoSep 12, 2016
  11. Lars SchneiderSep 13, 2016
  12. Junio C HamanoSep 13, 2016
  13. Lars SchneiderSep 15, 2016
  14. Jeff KingSep 15, 2016
  15. Lars SchneiderSep 15, 2016
  16. Junio C HamanoSep 15, 2016
  17. 05/10 pkt-line: add packet_write_gently()larsxschneider@gmail.com, Sep 8, 2016
  18. Stefan BellerSep 8, 2016
  19. Lars SchneiderSep 11, 2016
  20. Junio C HamanoSep 12, 2016
  21. 06/10 pkt-line: add functions to read/write flush terminated packet streamslarsxschneider@gmail.com, Sep 8, 2016
  22. Stefan BellerSep 8, 2016
  23. Lars SchneiderSep 11, 2016
  24. Stefan BellerSep 11, 2016
  25. Junio C HamanoSep 11, 2016
  26. 09/10 convert: make apply_filter() adhere to standard Git error handlinglarsxschneider@gmail.com, Sep 8, 2016
  27. 08/10 convert: modernize testslarsxschneider@gmail.com, Sep 8, 2016
  28. Stefan BellerSep 8, 2016
  29. Lars SchneiderSep 11, 2016
  30. 07/10 convert: quote filter names in error messageslarsxschneider@gmail.com, Sep 8, 2016
  31. 10/10 convert: add filter.<driver>.process optionlarsxschneider@gmail.com, Sep 8, 2016
  32. Torsten BögershausenSep 10, 2016
  33. Lars SchneiderSep 12, 2016
  34. Torsten BögershausenSep 13, 2016
  35. Junio C HamanoSep 13, 2016
  36. Lars SchneiderSep 15, 2016
  37. Junio C HamanoSep 15, 2016
  38. Torsten BögershausenSep 29, 2016
  39. Jakub NarębskiSep 29, 2016
  40. Torsten BögershausenSep 10, 2016
  41. Lars SchneiderSep 13, 2016
  42. Junio C HamanoSep 13, 2016
  43. Lars SchneiderSep 15, 2016
  44. Junio C HamanoSep 15, 2016
  45. Junio C HamanoSep 13, 2016

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.