git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH 09/16] vcs-svn: read inline data from deltas

From
David Barr <david.barr@cordelta.com>
Date
Mar 19, 2011, 07:20 UTC
Message-ID
<1300519254-20201-10-git-send-email-david.barr@cordelta.com>
In-Reply-To
<1300519254-20201-1-git-send-email-david.barr@cordelta.com>
From: Jonathan Nieder <jrnieder@gmail.com>

Each window of an svndiff0-format delta includes a section for new data that will be copied into the preimage (in the order it appears in the window, possibly interspersed with other data).

Read this data when encountering it. It is not actually necessary to do so --- it would be just as easy to copy straight from the delta to output when interpreting the relevant instructions --- but this way, the code that interprets svndiff0 instructions can proceed more quickly because it does not require any I/O.

Subversion's implementation rejects deltas that do not consume all the auxiliary data that is available. Do not check that for now, because it would make it impossible to test the function of this patch until the instructions to consume data are implemented.

Do check for truncated data sections. Since Subversion's applier rejects deltas that end before the new-data section is declared to end, it should be safe for this applier to reject such deltas, too.

Improved-by: Ramkumar Ramachandra <artagnon@gmail.com>
Improved-by: David Barr <david.barr@cordelta.com>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
Acked-by: Ramkumar Ramachandra <artagnon@gmail.com>
Signed-off-by: David Barr <david.barr@cordelta.com>
---
 t/t9011-svn-da.sh |   12 ++++++++++++
 vcs-svn/svndiff.c |   46 +++++++++++++++++++++++++++++++++++-----------
 2 files changed, 47 insertions(+), 11 deletions(-)
diff --git a/t/t9011-svn-da.sh b/t/t9011-svn-da.sh
index 986dce6..4919fae 100755
--- a/t/t9011-svn-da.sh
+++ b/t/t9011-svn-da.sh
@@ -123,4 +123,16 @@ test_expect_success 'preimage view: reject truncated preimage' '
 	test_must_fail test-svn-fe -d preimage clear.longread 9
 '
 
+test_expect_success 'inline data' '
+	printf "SVNQ%b%s%b%s" "QQQQ\003" "bar" "QQQQ\001" "x" |
+		q_to_nul >inline.clear &&
+	test-svn-fe -d preimage inline.clear 18 >actual &&
+	test_cmp empty actual
+'
+
+test_expect_success 'reject truncated inline data' '
+	printf "SVNQ%b%s" "QQQQ\003" "b" | q_to_nul >inline.trunc &&
+	test_must_fail test-svn-fe -d preimage inline.trunc 10
+'
+
 test_done
diff --git a/vcs-svn/svndiff.c b/vcs-svn/svndiff.c
index 1020275..74e0401 100644
--- a/vcs-svn/svndiff.c
+++ b/vcs-svn/svndiff.c
@@ -24,6 +24,17 @@
 #define VLI_DIGIT_MASK	0x7f
 #define VLI_BITS_PER_DIGIT 7
 
+struct window {
+	struct strbuf data;
+};
+
+#define WINDOW_INIT	{ STRBUF_INIT }
+
+static void window_release(struct window *ctx)
+{
+	strbuf_release(&ctx->data);
+}
+
 static int error_short_read(struct line_buffer *input)
 {
 	if (buffer_ferror(input))
@@ -31,24 +42,30 @@ static int error_short_read(struct line_buffer *input)
 	return error("invalid delta: unexpected end of file");
 }
 
+static int read_chunk(struct line_buffer *delta, off_t *delta_len,
+		      struct strbuf *buf, size_t len)
+{
+	strbuf_reset(buf);
+	if (len > *delta_len ||
+	    buffer_read_binary(delta, buf, len) != len)
+		return error_short_read(delta);
+	*delta_len -= buf->len;
+	return 0;
+}
+
 static int read_magic(struct line_buffer *in, off_t *len)
 {
 	static const char magic[] = {'S', 'V', 'N', '\0'};
 	struct strbuf sb = STRBUF_INIT;
 
-	if (*len < sizeof(magic) ||
-	    buffer_read_binary(in, &sb, sizeof(magic)) != sizeof(magic)) {
-		error_short_read(in);
+	if (read_chunk(in, len, &sb, sizeof(magic))) {
 		strbuf_release(&sb);
 		return -1;
 	}
-
 	if (memcmp(sb.buf, magic, sizeof(magic))) {
 		strbuf_release(&sb);
 		return error("invalid delta: unrecognized file type");
 	}
-
-	*len -= sizeof(magic);
 	strbuf_release(&sb);
 	return 0;
 }
@@ -98,6 +115,7 @@ static int read_length(struct line_buffer *in, size_t *result, off_t *len)
 
 static int apply_one_window(struct line_buffer *delta, off_t *delta_len)
 {
+	struct window ctx = WINDOW_INIT;
 	size_t out_len;
 	size_t instructions_len;
 	size_t data_len;
@@ -107,12 +125,18 @@ static int apply_one_window(struct line_buffer *delta, off_t *delta_len)
 	if (read_length(delta, &out_len, delta_len) ||
 	    read_length(delta, &instructions_len, delta_len) ||
 	    read_length(delta, &data_len, delta_len))
-		return -1;
-	if (instructions_len)
-		return error("What do you think I am?  A delta applier?");
-	if (data_len)
-		return error("No support for inline data yet");
+		goto error_out;
+	if (instructions_len) {
+		error("What do you think I am?  A delta applier?");
+		goto error_out;
+	}
+	if (read_chunk(delta, delta_len, &ctx.data, data_len))
+		goto error_out;
+	window_release(&ctx);
 	return 0;
+error_out:
+	window_release(&ctx);
+	return -1;
 }
 
 int svndiff0_apply(struct line_buffer *delta, off_t delta_len,
-- 
1.7.3.2.846.gf4b062
Previous: David BarrNext: David Barr
Message 31 of 72 in “vcs-svn: purge obsolete data structures and code”
  1. David BarrMar 19, 2011
  2. 1/9 vcs-svn: pass paths through to fast-importDavid Barr, Mar 19, 2011
  3. Jonathan NiederMar 19, 2011
  4. 2/9 vcs-svn: avoid using ls command twiceDavid Barr, Mar 19, 2011
  5. Jonathan NiederMar 19, 2011
  6. 3/9 vcs-svn: implement perfect hash for node-prop keysDavid Barr, Mar 19, 2011
  7. Jonathan NiederMar 19, 2011
  8. 1/3 vcs-svn: implement perfect hash for node-prop keysDavid Barr, Mar 21, 2011
  9. 2/3 vcs-svn: implement perfect hash for top-level keysDavid Barr, Mar 21, 2011
  10. 3/3 vcs-svn: use switch rather than cascading ifsDavid Barr, Mar 21, 2011
  11. [PATCHv2] vcs-svn: use switch rather than cascading ifsDavid Barr, Mar 21, 2011
  12. 4/9 vcs-svn: implement perfect hash for top-level keysDavid Barr, Mar 19, 2011
  13. Jonathan NiederMar 19, 2011
  14. 5/9 vcs-svn: factor out usage of string_poolDavid Barr, Mar 19, 2011
  15. Jonathan NiederMar 19, 2011
  16. 6/9 vcs-svn: drop string_poolDavid Barr, Mar 19, 2011
  17. 7/9 vcs-svn: drop trp.hDavid Barr, Mar 19, 2011
  18. 8/9 vcs-svn: drop obj_pool.hDavid Barr, Mar 19, 2011
  19. 9/9 vcs-svn: use strchr to find RFC822 delimiterDavid Barr, Mar 19, 2011
  20. Jonathan NiederMar 19, 2011
  21. vcs-svn: integrate support for text deltasDavid Barr, Mar 19, 2011
  22. 01/16 vcs-svn: improve support for reading large filesDavid Barr, Mar 19, 2011
  23. 02/16 vcs-svn: make buffer_skip_bytes return length readDavid Barr, Mar 19, 2011
  24. 03/16 vcs-svn: make buffer_copy_bytes return length readDavid Barr, Mar 19, 2011
  25. 04/16 vcs-svn: improve reporting of input errorsDavid Barr, Mar 19, 2011
  26. 05/16 vcs-svn: learn to maintain a sliding view of a fileDavid Barr, Mar 19, 2011
  27. 06/16 vcs-svn: skeleton of an svn delta parserDavid Barr, Mar 19, 2011
  28. Jonathan NiederMar 28, 2011
  29. 07/16 vcs-svn: parse svndiff0 window headerDavid Barr, Mar 19, 2011
  30. 08/16 vcs-svn: read the preimage when applying deltasDavid Barr, Mar 19, 2011
  31. 09/16 vcs-svn: read inline data from deltasDavid Barr, Mar 19, 2011
  32. 10/16 vcs-svn: read instructions from deltasDavid Barr, Mar 19, 2011
  33. 11/16 vcs-svn: implement copyfrom_data delta instructionDavid Barr, Mar 19, 2011
  34. 12/16 vcs-svn: verify that deltas consume all inline dataDavid Barr, Mar 19, 2011
  35. 13/16 vcs-svn: let deltas use data from postimageDavid Barr, Mar 19, 2011
  36. 14/16 vcs-svn: let deltas use data from preimageDavid Barr, Mar 19, 2011
  37. 15/16 vcs-svn: microcleanup in svndiff0 window-reading codeDavid Barr, Mar 19, 2011
  38. 16/16 vcs-svn: implement text-delta handlingDavid Barr, Mar 19, 2011
  39. Jonathan NiederMar 28, 2011
  40. David BarrMar 28, 2011
  41. 00/11 vcs-svn: purge obsolete data structures and codeDavid Barr, Mar 21, 2011
  42. 01/11 vcs-svn: use strbuf for revision logDavid Barr, Mar 21, 2011
  43. 02/11 vcs-svn: pass paths through to fast-importDavid Barr, Mar 21, 2011
  44. 03/11 vcs-svn: avoid using ls command twiceDavid Barr, Mar 21, 2011
  45. 04/11 vcs-svn: implement perfect hash for node-prop keysDavid Barr, Mar 21, 2011
  46. 05/11 vcs-svn: implement perfect hash for top-level keysDavid Barr, Mar 21, 2011
  47. 06/11 vcs-svn: use switch rather than cascading ifsDavid Barr, Mar 21, 2011
  48. 07/11 vcs-svn: factor out usage of string_poolDavid Barr, Mar 21, 2011
  49. 08/11 vcs-svn: drop string_poolDavid Barr, Mar 21, 2011
  50. =??q?=5BPATCH=2009/11=5D=20vcs-svn=3A=20drop=20trp=2Eh?=David Barr, Mar 21, 2011
  51. 10/11 vcs-svn: drop obj_pool.hDavid Barr, Mar 21, 2011
  52. 11/11 vcs-svn: use strchr to find RFC822 delimiterDavid Barr, Mar 21, 2011
  53. [PULL svn-fe] vcs-svn: simplifications, error handling improvementsJonathan Nieder, Mar 23, 2011
  54. Junio C HamanoMar 23, 2011
  55. Junio C HamanoMar 23, 2011
  56. Jonathan NiederMar 26, 2011
  57. t0081-line-buffer.sh hangs (Re: [PULL svn-fe] vcs-svn: simplifications, error handling improvements)Jonathan Nieder, Mar 26, 2011
  58. David BarrMar 23, 2011
  59. fixup! vcs-svn: improve reporting of input errorsDavid Barr, Mar 24, 2011
  60. Jonathan NiederMar 25, 2011
  61. 0/4 vcs-svn: null bytes in propertiesJonathan Nieder, Mar 25, 2011
  62. 1/4 vcs-svn: make reading of properties binary-safeJonathan Nieder, Mar 25, 2011
  63. tbMar 28, 2011
  64. Jonathan NiederMar 28, 2011
  65. Torsten BögershausenMar 28, 2011
  66. Jonathan NiederMar 28, 2011
  67. 2/4 vcs-svn: remove buffer_read_stringJonathan Nieder, Mar 25, 2011
  68. 3/4 vcs-svn: avoid unnecessary copying of log message and authorJonathan Nieder, Mar 25, 2011
  69. 4/4 vcs-svn: handle log message with embedded null bytesJonathan Nieder, Mar 25, 2011
  70. Jonathan NiederMar 26, 2011
  71. Junio C HamanoMar 26, 2011
  72. vcs-svn: add missing cast to printf argumentJonathan Nieder, Mar 28, 2011

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.