git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH 15/15] vcs-svn: Implement Prop-delta handling

From
Jonathan Nieder <jrnieder@gmail.com>
Date
Nov 20, 2010, 00:57 UTC
Message-ID
<20101120005746.GP17445@burratino>
In-Reply-To
<20101120004525.GA17445@burratino>
From: David Barr <david.barr@cordelta.com>

The rules for what file is used as delta source for each file are not documented in dump-load-format.txt. Luckily, the Apache Software Foundation repository has rich enough examples to figure out most of the rules:

Node-action: replace implies the empty property set and empty text as
preimage for deltas.  Otherwise, if a copyfrom source is given, that
node is the preimage for deltas.  Lastly, if none of the above applies
and the node path exists in the current revision, then that version
forms the basis.
[jn: refactored, with tests]
Signed-off-by: David Barr <david.barr@cordelta.com>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
---
That's the end of this chapter.  Thanks for reading.

In the next chapter, the line_buffer lib will gain the ability to read from multiple files (based on some patches seen on the list before) and use this ability to manage the cat-blob-fd and some temporary files for applying deltas.

Reading from the cat-blob-fd with stdio functions carries a great risk of deadlock, so we need to find a way to do this without fdopen() (some routines for doing this are already written and usable, but they haven't been made into a nice library yet).

 t/t9010-svn-fe.sh |  102 +++++++++++++++++++++++++++++++++++++++++++++++++++-
 vcs-svn/svndump.c |   54 +++++++++++++++++++++++-----
 2 files changed, 144 insertions(+), 12 deletions(-)
diff --git a/t/t9010-svn-fe.sh b/t/t9010-svn-fe.sh
index f1e8799..7dc0670 100755
--- a/t/t9010-svn-fe.sh
+++ b/t/t9010-svn-fe.sh
@@ -514,7 +514,12 @@ test_expect_success 'deltas not supported' '
 	test_must_fail test-svn-fe delta.dump
 '
 
-test_expect_success 'property deltas not supported' '
+test_expect_success 'property deltas supported' '
+	reinit_git &&
+	cat >expect <<-\EOF &&
+	OBJID
+	:100755 100644 OBJID OBJID M	script.sh
+	EOF
 	{
 		properties \
 			svn:author author@example.com \
@@ -565,7 +570,100 @@ test_expect_success 'property deltas not supported' '
 		PROPS-END
 		EOF
 	} >propdelta.dump &&
-	test_must_fail test-svn-fe propdelta.dump
+	test-svn-fe propdelta.dump >stream &&
+	git fast-import <stream &&
+	{
+		git rev-list HEAD |
+		git diff-tree --stdin |
+		sed "s/$_x40/OBJID/g"
+	} >actual &&
+	test_cmp expect actual
+'
+
+test_expect_success 'deltas for typechange' '
+	reinit_git &&
+	cat >expect <<-\EOF &&
+	OBJID
+	:120000 100644 OBJID OBJID T	test-file
+	OBJID
+	:100755 120000 OBJID OBJID T	test-file
+	OBJID
+	:000000 100755 OBJID OBJID A	test-file
+	EOF
+	cat >deleteprop.dump <<-\EOF &&
+	SVN-fs-dump-format-version: 3
+
+	Revision-number: 1
+	Prop-content-length: 10
+	Content-length: 10
+
+	PROPS-END
+
+	Node-path: test-file
+	Node-kind: file
+	Node-action: add
+	Prop-delta: true
+	Prop-content-length: 35
+	Text-content-length: 17
+	Content-length: 52
+
+	K 14
+	svn:executable
+	V 0
+
+	PROPS-END
+	link testing 123
+
+	Revision-number: 2
+	Prop-content-length: 10
+	Content-length: 10
+
+	PROPS-END
+
+	Node-path: test-file
+	Node-kind: file
+	Node-action: change
+	Prop-delta: true
+	Prop-content-length: 53
+	Text-content-length: 17
+	Content-length: 70
+
+	K 11
+	svn:special
+	V 1
+	*
+	D 14
+	svn:executable
+	PROPS-END
+	link testing 231
+
+	Revision-number: 3
+	Prop-content-length: 10
+	Content-length: 10
+
+	PROPS-END
+
+	Node-path: test-file
+	Node-kind: file
+	Node-action: change
+	Prop-delta: true
+	Prop-content-length: 27
+	Text-content-length: 17
+	Content-length: 44
+
+	D 11
+	svn:special
+	PROPS-END
+	link testing 321
+	EOF
+	test-svn-fe deleteprop.dump >stream &&
+	git fast-import <stream &&
+	{
+		git rev-list HEAD |
+		git diff-tree --root --stdin |
+		sed "s/$_x40/OBJID/g"
+	} >actual &&
+	test_cmp expect actual
 '
 
 test_expect_success 't9135/svn.dump' '
diff --git a/vcs-svn/svndump.c b/vcs-svn/svndump.c
index c3e1e32..8a224ac 100644
--- a/vcs-svn/svndump.c
+++ b/vcs-svn/svndump.c
@@ -115,20 +115,35 @@ static void init_keys(void)
 	keys.prop_delta = pool_intern("Prop-delta");
 }
 
-static void handle_property(uint32_t key, const char *val, uint32_t len)
+static void handle_property(uint32_t key, const char *val, uint32_t len,
+				uint32_t *type_set)
 {
 	if (key == keys.svn_log) {
+		if (!val)
+			die("invalid dump: unsets svn:log");
 		/* Value length excludes terminating nul. */
 		rev_ctx.log = log_copy(len + 1, val);
 	} else if (key == keys.svn_author) {
 		rev_ctx.author = pool_intern(val);
 	} else if (key == keys.svn_date) {
+		if (!val)
+			die("invalid dump: unsets svn:date");
 		if (parse_date_basic(val, &rev_ctx.timestamp, NULL))
-			fprintf(stderr, "Invalid timestamp: %s\n", val);
-	} else if (key == keys.svn_executable) {
-		node_ctx.type = REPO_MODE_EXE;
-	} else if (key == keys.svn_special) {
-		node_ctx.type = REPO_MODE_LNK;
+			warning("invalid timestamp: %s", val);
+	} else if (key == keys.svn_executable || key == keys.svn_special) {
+		if (*type_set) {
+			if (!val)
+				return;
+			die("invalid dump: sets type twice");
+		}
+		if (!val) {
+			node_ctx.type = REPO_MODE_BLB;
+			return;
+		}
+		*type_set = 1;
+		node_ctx.type = key == keys.svn_executable ?
+				REPO_MODE_EXE :
+				REPO_MODE_LNK;
 	}
 }
 
@@ -136,6 +151,19 @@ static void read_props(void)
 {
 	uint32_t key = ~0;
 	const char *t;
+	/*
+	 * NEEDSWORK: to support simple mode changes like
+	 *	K 11
+	 *	svn:special
+	 *	V 1
+	 *	*
+	 *	D 14
+	 *	svn:executable
+	 * we keep track of whether a mode has been set and reset to
+	 * plain file only if not.  We should be keeping track of the
+	 * symlink and executable bits separately instead.
+	 */
+	uint32_t type_set = 0;
 	while ((t = buffer_read_line()) && strcmp(t, "PROPS-END")) {
 		uint32_t len;
 		const char *val;
@@ -151,8 +179,13 @@ static void read_props(void)
 		case 'K':
 			key = pool_intern(val);
 			continue;
+		case 'D':
+			key = pool_intern(val);
+			val = NULL;
+			len = 0;
+			/* fall through */
 		case 'V':
-			handle_property(key, val, len);
+			handle_property(key, val, len, &type_set);
 			key = ~0;
 			continue;
 		default:
@@ -167,8 +200,8 @@ static void handle_node(void)
 	const uint32_t type = node_ctx.type;
 	const int have_props = node_ctx.propLength != LENGTH_UNKNOWN;
 
-	if (node_ctx.text_delta || node_ctx.prop_delta)
-		die("text and property deltas not supported");
+	if (node_ctx.text_delta)
+		die("text deltas not supported");
 	if (node_ctx.textLength != LENGTH_UNKNOWN)
 		mark = next_blob_mark();
 	if (node_ctx.action == NODEACT_DELETE) {
@@ -206,7 +239,8 @@ static void handle_node(void)
 	}
 	if (have_props) {
 		const uint32_t old_mode = node_ctx.type;
-		node_ctx.type = type;
+		if (!node_ctx.prop_delta)
+			node_ctx.type = type;
 		if (node_ctx.propLength)
 			read_props();
 		if (node_ctx.type != old_mode)

-- 
1.7.2.3
Previous: Jonathan NiederNext: Jonathan Nieder
Message 25 of 40 in “svn-fe: recognize v3 dumps”
  1. 0/2 svn-fe: recognize v3 dumpsJonathan Nieder, Nov 18, 2010
  2. 1/2 vcs-svn: Error out for v3 dumpsJonathan Nieder, Nov 18, 2010
  3. 2/2 vcs-svn: Allow simple v3 dumps (no deltas yet)Jonathan Nieder, Nov 18, 2010
  4. 0/15 svn-fe: support for property deltas (but not text deltas yet)Jonathan Nieder, Nov 20, 2010
  5. 01/15 vcs-svn: Check for errors from open()Jonathan Nieder, Nov 20, 2010
  6. 02/15 vcs-svn: Eliminate node_ctx.srcRev globalJonathan Nieder, Nov 20, 2010
  7. 03/15 vcs-svn: Eliminate node_ctx.mark globalJonathan Nieder, Nov 20, 2010
  8. 04/15 vcs-svn: Unclutter handle_node by introducing have_props varJonathan Nieder, Nov 20, 2010
  9. 05/15 vcs-svn: Use mark to indicate nodes with included textJonathan Nieder, Nov 20, 2010
  10. 06/15 vcs-svn: handle_node: Handle deletion case earlyJonathan Nieder, Nov 20, 2010
  11. 07/15 vcs-svn: Replace = Delete + AddJonathan Nieder, Nov 20, 2010
  12. 08/15 vcs-svn: Combine repo_replace and repo_modify functionsJonathan Nieder, Nov 20, 2010
  13. 09/15 vcs-svn: Delay read of per-path propertiesJonathan Nieder, Nov 20, 2010
  14. 10/15 vcs-svn: Reject path nodes without Node-actionJonathan Nieder, Nov 20, 2010
  15. Jonathan NiederNov 20, 2010
  16. 11/15 vcs-svn: More dump format sanity checksJonathan Nieder, Nov 20, 2010
  17. Jonathan NiederNov 30, 2010
  18. 0/2 intermittent repo_tree corruptionJonathan Nieder, Dec 5, 2010
  19. 1/2 treap: make treap_insert return inserted nodeJonathan Nieder, Dec 5, 2010
  20. vcs-svn: Allow change nodes for root of tree (/)Jonathan Nieder, Dec 6, 2010
  21. Jonathan NiederDec 6, 2010
  22. 12/15 vcs-svn: Make source easier to read on small screensJonathan Nieder, Nov 20, 2010
  23. 13/15 vcs-svn: Split off function for handling of individual propertiesJonathan Nieder, Nov 20, 2010
  24. 14/15 vcs-svn: Sharpen parsing of property linesJonathan Nieder, Nov 20, 2010
  25. 15/15 vcs-svn: Implement Prop-delta handlingJonathan Nieder, Nov 20, 2010
  26. 0/8 svn-fe: support for text deltasJonathan Nieder, Nov 20, 2010
  27. 1/8 svn-fe: Prepare for strbuf useJonathan Nieder, Nov 20, 2010
  28. 2/8 vcs-svn: Internal fast_export_save_blob helperJonathan Nieder, Nov 20, 2010
  29. 3/8 vcs-svn: Introduce repo_read_path to check the content at a pathJonathan Nieder, Nov 20, 2010
  30. Jonathan NiederMar 6, 2011
  31. 4/8 vcs-svn: Introduce fd_buffer routinesJonathan Nieder, Nov 20, 2010
  32. 5/8 vcs-svn: Read delta preimage from file descriptorJonathan Nieder, Nov 20, 2010
  33. 6/8 vcs-svn: Let caller set up sliding window for delta preimageJonathan Nieder, Nov 20, 2010
  34. Jonathan NiederNov 20, 2010
  35. 7/8 vcs-svn: Teach line_buffer about temporary filesJonathan Nieder, Nov 20, 2010
  36. 8/8 vcs-svn: Implement text-delta handlingJonathan Nieder, Nov 20, 2010
  37. 10/8 vcs-svn: Consume whole preimage when applying deltasJonathan Nieder, Dec 4, 2010
  38. 9/8 svn-fe: Test script for handling of dumps with --deltasJonathan Nieder, Nov 20, 2010
  39. Jonathan NiederDec 4, 2010
  40. 2/2 vcs-svn: fix intermittent repo_tree corruptionJonathan Nieder, Dec 5, 2010

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.