git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH 10/19] fast-import: introduce 'done' command

From
Sverre Rabbelier <srabbelier@gmail.com>
Date
Jun 8, 2011, 18:48 UTC
Message-ID
<1307558930-16074-11-git-send-email-srabbelier@gmail.com>
In-Reply-To
<1307558930-16074-1-git-send-email-srabbelier@gmail.com>

Add a 'done' command that causes fast-import to stop reading from the stream and exit.

If the new --done command line flag was passed on the command line (or a "feature done" declaration included at the start of the stream), make the 'done' command mandatory. So "git fast-import --done"'s input format will be prefix-free, making errors easier to detect when they show up as early termination at some convenient time of the upstream of a pipe writing to fast-import.

Another possible application of the 'done' command would to be allow a fast-import stream that is only a small part of a larger encapsulating stream to be easily parsed, leaving the file offset after the "done\n" so the other application can pick up from there. This patch does not teach fast-import to do that --- fast-import still uses buffered input (stdio).

Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Sverre Rabbelier <srabbelier@gmail.com>
---
  This is actually Jonathan's version that makes done required if
  feature done was specified. See also:
	http://mid.gmane.org/20110213094212.GA25435@elie
 Documentation/git-fast-import.txt |   25 ++++++++++++++++++++++
 fast-import.c                     |    8 +++++++
 t/t9300-fast-import.sh            |   42 +++++++++++++++++++++++++++++++++++++
 3 files changed, 75 insertions(+), 0 deletions(-)
diff --git a/Documentation/git-fast-import.txt b/Documentation/git-fast-import.txt
index 249249a..0fc68a9 100644
--- a/Documentation/git-fast-import.txt
+++ b/Documentation/git-fast-import.txt
@@ -101,6 +101,12 @@ OPTIONS
 	when the `cat-blob` command is encountered in the stream.
 	The default behaviour is to write to `stdout`.
 
+--done::
+	Require a `done` command at the end of the stream.
+	This option might be useful for detecting errors that
+	cause the frontend to terminate before it has started to
+	write a stream.
+
 --export-pack-edges=<file>::
 	After creating a packfile, print a line of data to
 	<file> listing the filename of the packfile and the last
@@ -330,6 +336,11 @@ and control the current import process.  More detailed discussion
 	standard output.  This command is optional and is not needed
 	to perform an import.
 
+`done`::
+	Marks the end of the stream. This command is optional
+	unless the `done` feature was requested using the
+	`--done` command line option or `feature done` command.
+
 `cat-blob`::
 	Causes fast-import to print a blob in 'cat-file --batch'
 	format to the file descriptor set with `--cat-blob-fd` or
@@ -1015,6 +1026,11 @@ notes::
 	Versions of fast-import not supporting notes will exit
 	with a message indicating so.
 
+done::
+	Error out if the stream ends without a 'done' command.
+	Without this feature, errors causing the frontend to end
+	abruptly at a convenient point in the stream can go
+	undetected.
 
 `option`
 ~~~~~~~~
@@ -1044,6 +1060,15 @@ not be passed as option:
 * cat-blob-fd
 * force
 
+`done`
+~~~~~~
+If the `done` feature is not in use, treated as if EOF was read.
+This can be used to tell fast-import to finish early.
+
+If the `--done` command line option or `feature done` command is
+in use, the `done` command is mandatory and marks the end of the
+stream.
+
 Crash Reports
 -------------
 If fast-import is supplied invalid input it will terminate with a
diff --git a/fast-import.c b/fast-import.c
index 78d9786..8a8a915 100644
--- a/fast-import.c
+++ b/fast-import.c
@@ -354,6 +354,7 @@ static unsigned int cmd_save = 100;
 static uintmax_t next_mark;
 static struct strbuf new_data = STRBUF_INIT;
 static int seen_data_command;
+static int require_explicit_termination;
 
 /* Signal handling */
 static volatile sig_atomic_t checkpoint_requested;
@@ -3139,6 +3140,8 @@ static int parse_one_feature(const char *feature, int from_stream)
 		relative_marks_paths = 1;
 	} else if (!strcmp(feature, "no-relative-marks")) {
 		relative_marks_paths = 0;
+	} else if (!strcmp(feature, "done")) {
+		require_explicit_termination = 1;
 	} else if (!strcmp(feature, "force")) {
 		force_update = 1;
 	} else if (!strcmp(feature, "notes") || !strcmp(feature, "ls")) {
@@ -3288,6 +3291,8 @@ int main(int argc, const char **argv)
 			parse_reset_branch();
 		else if (!strcmp("checkpoint", command_buf.buf))
 			parse_checkpoint();
+		else if (!strcmp("done", command_buf.buf))
+			break;
 		else if (!prefixcmp(command_buf.buf, "progress "))
 			parse_progress();
 		else if (!prefixcmp(command_buf.buf, "feature "))
@@ -3307,6 +3312,9 @@ int main(int argc, const char **argv)
 	if (!seen_data_command)
 		parse_argv();
 
+	if (require_explicit_termination && feof(stdin))
+		die("stream ends early");
+
 	end_packfile();
 
 	dump_branches();
diff --git a/t/t9300-fast-import.sh b/t/t9300-fast-import.sh
index 6b1ba6c..526231b 100755
--- a/t/t9300-fast-import.sh
+++ b/t/t9300-fast-import.sh
@@ -2197,6 +2197,48 @@ test_expect_success 'R: quiet option results in no stats being output' '
     test_cmp empty output
 '
 
+test_expect_success 'R: feature done means terminating "done" is mandatory' '
+	echo feature done | test_must_fail git fast-import &&
+	test_must_fail git fast-import --done </dev/null
+'
+
+test_expect_success 'R: terminating "done" with trailing gibberish is ok' '
+	git fast-import <<-\EOF &&
+	feature done
+	done
+	trailing gibberish
+	EOF
+	git fast-import <<-\EOF
+	done
+	more trailing gibberish
+	EOF
+'
+
+test_expect_success 'R: terminating "done" within commit' '
+	cat >expect <<-\EOF &&
+	OBJID
+	:000000 100644 OBJID OBJID A	hello.c
+	:000000 100644 OBJID OBJID A	hello2.c
+	EOF
+	git fast-import <<-EOF &&
+	commit refs/heads/done-ends
+	committer $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> $GIT_COMMITTER_DATE
+	data <<EOT
+	Commit terminated by "done" command
+	EOT
+	M 100644 inline hello.c
+	data <<EOT
+	Hello, world.
+	EOT
+	C hello.c hello2.c
+	done
+	EOF
+	git rev-list done-ends |
+	git diff-tree -r --stdin --root --always |
+	sed -e "s/$_x40/OBJID/g" >actual &&
+	test_cmp expect actual
+'
+
 cat >input <<EOF
 option git non-existing-option
 EOF

-- 
1.7.5.1.292.g728120
Previous: Sverre RabbelierNext: Jonathan Nieder
Message 32 of 54 in “remote-helper improvements”
  1. 00/19 remote-helper improvementsSverre Rabbelier, Jun 8, 2011
  2. 01/19 transport-helper: fix minor leak in push_refs_with_exportSverre Rabbelier, Jun 8, 2011
  3. Jeff KingJun 8, 2011
  4. Sverre RabbelierJun 8, 2011
  5. 02/19 t5800: factor out some ref testsSverre Rabbelier, Jun 8, 2011
  6. 03/19 t5800: document some non-functional parts of remote helpersSverre Rabbelier, Jun 8, 2011
  7. Jonathan NiederJun 8, 2011
  8. Jonathan NiederJun 8, 2011
  9. Sverre RabbelierJun 8, 2011
  10. Sverre RabbelierJun 8, 2011
  11. Sverre RabbelierJun 9, 2011
  12. Jeff KingJun 10, 2011
  13. 04/19 teach remote-testgit to import non-HEAD refsSverre Rabbelier, Jun 8, 2011
  14. Jonathan NiederJun 8, 2011
  15. Sverre RabbelierJun 8, 2011
  16. Jeff KingJun 8, 2011
  17. Junio C HamanoJun 8, 2011
  18. Sverre RabbelierJun 9, 2011
  19. 05/19 transport-helper: don't feed bogus refs to export pushSverre Rabbelier, Jun 8, 2011
  20. 06/19 git_remote_helpers: push all refs during a non-local exportSverre Rabbelier, Jun 8, 2011
  21. Jonathan NiederJun 8, 2011
  22. Jeff KingJun 8, 2011
  23. Sverre RabbelierJun 8, 2011
  24. Jonathan NiederJun 9, 2011
  25. Sverre RabbelierJun 9, 2011
  26. Jonathan NiederJun 9, 2011
  27. Sverre RabbelierJun 9, 2011
  28. Jeff KingJun 10, 2011
  29. 07/19 remote-curl: accept empty line as terminatorSverre Rabbelier, Jun 8, 2011
  30. 08/19 git-remote-testgit: only push for non-local repositoriesSverre Rabbelier, Jun 8, 2011
  31. 09/19 git-remote-testgit: fix error handlingSverre Rabbelier, Jun 8, 2011
  32. 10/19 fast-import: introduce 'done' commandSverre Rabbelier, Jun 8, 2011
  33. Jonathan NiederJun 8, 2011
  34. Sverre RabbelierJun 8, 2011
  35. 11/19 fast-export: support done featureSverre Rabbelier, Jun 8, 2011
  36. 12/19 transport-helper: factor out push_update_refs_statusSverre Rabbelier, Jun 8, 2011
  37. 13/19 transport-helper: check status code of finish_commandSverre Rabbelier, Jun 8, 2011
  38. 14/19 transport-helper: use the new done feature where possibleSverre Rabbelier, Jun 8, 2011
  39. 15/19 transport-helper: update ref status after push with exportSverre Rabbelier, Jun 8, 2011
  40. Jonathan NiederJun 9, 2011
  41. Sverre RabbelierJun 9, 2011
  42. 16/19 transport-helper: change import semanticsSverre Rabbelier, Jun 8, 2011
  43. Jonathan NiederJun 8, 2011
  44. Sverre RabbelierJun 8, 2011
  45. 17/19 transport-helper: export is no longer always the last commandSverre Rabbelier, Jun 8, 2011
  46. Junio C HamanoJun 9, 2011
  47. Sverre RabbelierJun 9, 2011
  48. Jonathan NiederJun 9, 2011
  49. Sverre RabbelierJun 9, 2011
  50. Junio C HamanoJun 13, 2011
  51. 18/19 transport-helper: Use capname for gitdir capability tooSverre Rabbelier, Jun 8, 2011
  52. Jonathan NiederJun 8, 2011
  53. Sverre RabbelierJun 8, 2011
  54. 19/19 transport-helper: implement marks location as capabilitySverre Rabbelier, Jun 8, 2011

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.