git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH v2 11/20] fast-import: introduce 'done' command

From
Sverre Rabbelier <srabbelier@gmail.com>
Date
Jun 19, 2011, 15:18 UTC
Message-ID
<1308496725-22329-12-git-send-email-srabbelier@gmail.com>
In-Reply-To
<1308496725-22329-1-git-send-email-srabbelier@gmail.com>

Add a 'done' command that causes fast-import to stop reading from the stream and exit.

If the new --done command line flag was passed on the command line (or a "feature done" declaration included at the start of the stream), make the 'done' command mandatory. So "git fast-import --done"'s input format will be prefix-free, making errors easier to detect when they show up as early termination at some convenient time of the upstream of a pipe writing to fast-import.

Another possible application of the 'done' command would to be allow a fast-import stream that is only a small part of a larger encapsulating stream to be easily parsed, leaving the file offset after the "done\n" so the other application can pick up from there. This patch does not teach fast-import to do that --- fast-import still uses buffered input (stdio).

Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Sverre Rabbelier <srabbelier@gmail.com>
---
  Unchanged.
 Documentation/git-fast-import.txt |   25 ++++++++++++++++++++++
 fast-import.c                     |    8 +++++++
 t/t9300-fast-import.sh            |   42 +++++++++++++++++++++++++++++++++++++
 3 files changed, 75 insertions(+), 0 deletions(-)
diff --git a/Documentation/git-fast-import.txt b/Documentation/git-fast-import.txt
index 249249a..0fc68a9 100644
--- a/Documentation/git-fast-import.txt
+++ b/Documentation/git-fast-import.txt
@@ -101,6 +101,12 @@ OPTIONS
 	when the `cat-blob` command is encountered in the stream.
 	The default behaviour is to write to `stdout`.
 
+--done::
+	Require a `done` command at the end of the stream.
+	This option might be useful for detecting errors that
+	cause the frontend to terminate before it has started to
+	write a stream.
+
 --export-pack-edges=<file>::
 	After creating a packfile, print a line of data to
 	<file> listing the filename of the packfile and the last
@@ -330,6 +336,11 @@ and control the current import process.  More detailed discussion
 	standard output.  This command is optional and is not needed
 	to perform an import.
 
+`done`::
+	Marks the end of the stream. This command is optional
+	unless the `done` feature was requested using the
+	`--done` command line option or `feature done` command.
+
 `cat-blob`::
 	Causes fast-import to print a blob in 'cat-file --batch'
 	format to the file descriptor set with `--cat-blob-fd` or
@@ -1015,6 +1026,11 @@ notes::
 	Versions of fast-import not supporting notes will exit
 	with a message indicating so.
 
+done::
+	Error out if the stream ends without a 'done' command.
+	Without this feature, errors causing the frontend to end
+	abruptly at a convenient point in the stream can go
+	undetected.
 
 `option`
 ~~~~~~~~
@@ -1044,6 +1060,15 @@ not be passed as option:
 * cat-blob-fd
 * force
 
+`done`
+~~~~~~
+If the `done` feature is not in use, treated as if EOF was read.
+This can be used to tell fast-import to finish early.
+
+If the `--done` command line option or `feature done` command is
+in use, the `done` command is mandatory and marks the end of the
+stream.
+
 Crash Reports
 -------------
 If fast-import is supplied invalid input it will terminate with a
diff --git a/fast-import.c b/fast-import.c
index 78d9786..8a8a915 100644
--- a/fast-import.c
+++ b/fast-import.c
@@ -354,6 +354,7 @@ static unsigned int cmd_save = 100;
 static uintmax_t next_mark;
 static struct strbuf new_data = STRBUF_INIT;
 static int seen_data_command;
+static int require_explicit_termination;
 
 /* Signal handling */
 static volatile sig_atomic_t checkpoint_requested;
@@ -3139,6 +3140,8 @@ static int parse_one_feature(const char *feature, int from_stream)
 		relative_marks_paths = 1;
 	} else if (!strcmp(feature, "no-relative-marks")) {
 		relative_marks_paths = 0;
+	} else if (!strcmp(feature, "done")) {
+		require_explicit_termination = 1;
 	} else if (!strcmp(feature, "force")) {
 		force_update = 1;
 	} else if (!strcmp(feature, "notes") || !strcmp(feature, "ls")) {
@@ -3288,6 +3291,8 @@ int main(int argc, const char **argv)
 			parse_reset_branch();
 		else if (!strcmp("checkpoint", command_buf.buf))
 			parse_checkpoint();
+		else if (!strcmp("done", command_buf.buf))
+			break;
 		else if (!prefixcmp(command_buf.buf, "progress "))
 			parse_progress();
 		else if (!prefixcmp(command_buf.buf, "feature "))
@@ -3307,6 +3312,9 @@ int main(int argc, const char **argv)
 	if (!seen_data_command)
 		parse_argv();
 
+	if (require_explicit_termination && feof(stdin))
+		die("stream ends early");
+
 	end_packfile();
 
 	dump_branches();
diff --git a/t/t9300-fast-import.sh b/t/t9300-fast-import.sh
index 6b1ba6c..526231b 100755
--- a/t/t9300-fast-import.sh
+++ b/t/t9300-fast-import.sh
@@ -2197,6 +2197,48 @@ test_expect_success 'R: quiet option results in no stats being output' '
     test_cmp empty output
 '
 
+test_expect_success 'R: feature done means terminating "done" is mandatory' '
+	echo feature done | test_must_fail git fast-import &&
+	test_must_fail git fast-import --done </dev/null
+'
+
+test_expect_success 'R: terminating "done" with trailing gibberish is ok' '
+	git fast-import <<-\EOF &&
+	feature done
+	done
+	trailing gibberish
+	EOF
+	git fast-import <<-\EOF
+	done
+	more trailing gibberish
+	EOF
+'
+
+test_expect_success 'R: terminating "done" within commit' '
+	cat >expect <<-\EOF &&
+	OBJID
+	:000000 100644 OBJID OBJID A	hello.c
+	:000000 100644 OBJID OBJID A	hello2.c
+	EOF
+	git fast-import <<-EOF &&
+	commit refs/heads/done-ends
+	committer $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> $GIT_COMMITTER_DATE
+	data <<EOT
+	Commit terminated by "done" command
+	EOT
+	M 100644 inline hello.c
+	data <<EOT
+	Hello, world.
+	EOT
+	C hello.c hello2.c
+	done
+	EOF
+	git rev-list done-ends |
+	git diff-tree -r --stdin --root --always |
+	sed -e "s/$_x40/OBJID/g" >actual &&
+	test_cmp expect actual
+'
+
 cat >input <<EOF
 option git non-existing-option
 EOF

-- 
1.7.5.1.292.g728120
Previous: Sverre RabbelierNext: Sverre Rabbelier
Message 22 of 44 in “remote-helper improvements”
  1. 00/20 remote-helper improvementsSverre Rabbelier, Jun 19, 2011
  2. 01/20 transport-helper: fix minor leak in push_refs_with_exportSverre Rabbelier, Jun 19, 2011
  3. 02/20 t5800: factor out some ref testsSverre Rabbelier, Jun 19, 2011
  4. 03/20 t5800: use skip_all instead of prereqSverre Rabbelier, Jun 19, 2011
  5. 04/20 t5800: document some non-functional parts of remote helpersSverre Rabbelier, Jun 19, 2011
  6. Jonathan NiederJun 19, 2011
  7. Sverre RabbelierJul 4, 2011
  8. 05/20 teach remote-testgit to import non-HEAD refsSverre Rabbelier, Jun 19, 2011
  9. 06/20 transport-helper: don't feed bogus refs to export pushSverre Rabbelier, Jun 19, 2011
  10. 07/20 git_remote_helpers: push all refs during a non-local exportSverre Rabbelier, Jun 19, 2011
  11. 08/20 remote-curl: accept empty line as terminatorSverre Rabbelier, Jun 19, 2011
  12. Jonathan NiederJun 19, 2011
  13. Sverre RabbelierJul 4, 2011
  14. 08/20 remote-curl: accept empty line as terminatorDmitry Ivankov, Jun 20, 2011
  15. Jonathan NiederJun 20, 2011
  16. Junio C HamanoJun 20, 2011
  17. 09/20 git-remote-testgit: only push for non-local repositoriesSverre Rabbelier, Jun 19, 2011
  18. 10/20 git-remote-testgit: fix error handlingSverre Rabbelier, Jun 19, 2011
  19. Jonathan NiederJun 19, 2011
  20. Brandon CaseyJun 20, 2011
  21. Sverre RabbelierJun 20, 2011
  22. 11/20 fast-import: introduce 'done' commandSverre Rabbelier, Jun 19, 2011
  23. 12/20 fast-export: support done featureSverre Rabbelier, Jun 19, 2011
  24. 13/20 transport-helper: factor out push_update_refs_statusSverre Rabbelier, Jun 19, 2011
  25. 14/20 transport-helper: check status code of finish_commandSverre Rabbelier, Jun 19, 2011
  26. 15/20 transport-helper: use the new done feature where possibleSverre Rabbelier, Jun 19, 2011
  27. Jonathan NiederJun 20, 2011
  28. Junio C HamanoJun 20, 2011
  29. Sverre RabbelierJul 4, 2011
  30. 16/20 transport-helper: update ref status after push with exportSverre Rabbelier, Jun 19, 2011
  31. Jonathan NiederJun 19, 2011
  32. Junio C HamanoJun 21, 2011
  33. Sverre RabbelierJun 21, 2011
  34. 17/20 transport-helper: change import semanticsSverre Rabbelier, Jun 19, 2011
  35. Jonathan NiederJun 19, 2011
  36. Sverre RabbelierJul 4, 2011
  37. Jonathan NiederJul 4, 2011
  38. Sverre RabbelierJul 4, 2011
  39. Jonathan NiederJul 4, 2011
  40. 18/20 transport-helper: export is no longer always the last commandSverre Rabbelier, Jun 19, 2011
  41. 19/20 transport-helper: Use capname for gitdir capability tooSverre Rabbelier, Jun 19, 2011
  42. 20/20 transport-helper: implement marks location as capabilitySverre Rabbelier, Jun 19, 2011
  43. Jonathan NiederJun 20, 2011
  44. Sverre RabbelierJul 4, 2011

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.