git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH v10 14/14] contrib/long-running-filter: add long running filter example

From
larsxschneider@gmail.com <larsxschneider@gmail.com>
Date
Oct 8, 2016, 11:25 UTC
Message-ID
<20161008112530.15506-15-larsxschneider@gmail.com>
In-Reply-To
<20161008112530.15506-1-larsxschneider@gmail.com>
From: Lars Schneider <larsxschneider@gmail.com>

Add a simple pass-thru filter as example implementation for the Git filter protocol version 2. See Documentation/gitattributes.txt, section "Filter Protocol" for more info.

Signed-off-by: Lars Schneider <larsxschneider@gmail.com>
---
 Documentation/gitattributes.txt        |   4 +-
 contrib/long-running-filter/example.pl | 127 +++++++++++++++++++++++++++++++++
 2 files changed, 130 insertions(+), 1 deletion(-)
 create mode 100755 contrib/long-running-filter/example.pl
diff --git a/Documentation/gitattributes.txt b/Documentation/gitattributes.txt
index 5868f00..a182ef2 100644
--- a/Documentation/gitattributes.txt
+++ b/Documentation/gitattributes.txt
@@ -514,7 +514,9 @@ the next "key=value" list containing a command. Git will close
 the command pipe on exit. The filter is expected to detect EOF
 and exit gracefully on its own.
 
-If you develop your own long running filter
+A long running filter demo implementation can be found in
+`contrib/long-running-filter/example.pl` located in the Git
+core repository. If you develop your own long running filter
 process then the `GIT_TRACE_PACKET` environment variables can be
 very helpful for debugging (see linkgit:git[1]).
 
diff --git a/contrib/long-running-filter/example.pl b/contrib/long-running-filter/example.pl
new file mode 100755
index 0000000..f4102d2
--- /dev/null
+++ b/contrib/long-running-filter/example.pl
@@ -0,0 +1,127 @@
+#!/usr/bin/perl
+#
+# Example implementation for the Git filter protocol version 2
+# See Documentation/gitattributes.txt, section "Filter Protocol"
+#
+# Please note, this pass-thru filter is a minimal skeleton. No proper
+# error handling was implemented.
+#
+
+use strict;
+use warnings;
+
+my $MAX_PACKET_CONTENT_SIZE = 65516;
+
+sub packet_bin_read {
+	my $buffer;
+	my $bytes_read = read STDIN, $buffer, 4;
+	if ( $bytes_read == 0 ) {
+
+		# EOF - Git stopped talking to us!
+		exit();
+	}
+	elsif ( $bytes_read != 4 ) {
+		die "invalid packet: '$buffer'";
+	}
+	my $pkt_size = hex($buffer);
+	if ( $pkt_size == 0 ) {
+		return ( 1, "" );
+	}
+	elsif ( $pkt_size > 4 ) {
+		my $content_size = $pkt_size - 4;
+		$bytes_read = read STDIN, $buffer, $content_size;
+		if ( $bytes_read != $content_size ) {
+			die "invalid packet ($content_size bytes expected; $bytes_read bytes read)";
+		}
+		return ( 0, $buffer );
+	}
+	else {
+		die "invalid packet size: $pkt_size";
+	}
+}
+
+sub packet_txt_read {
+	my ( $res, $buf ) = packet_bin_read();
+	unless ( $buf =~ s/\n$// ) {
+		die "A non-binary line MUST be terminated by an LF.";
+	}
+	return ( $res, $buf );
+}
+
+sub packet_bin_write {
+	my $buf = shift;
+	print STDOUT sprintf( "%04x", length($buf) + 4 );
+	print STDOUT $buf;
+	STDOUT->flush();
+}
+
+sub packet_txt_write {
+	packet_bin_write( $_[0] . "\n" );
+}
+
+sub packet_flush {
+	print STDOUT sprintf( "%04x", 0 );
+	STDOUT->flush();
+}
+
+( packet_txt_read() eq ( 0, "git-filter-client" ) ) || die "bad initialize";
+( packet_txt_read() eq ( 0, "version=2" ) )         || die "bad version";
+( packet_bin_read() eq ( 1, "" ) )                  || die "bad version end";
+
+packet_txt_write("git-filter-server");
+packet_txt_write("version=2");
+
+( packet_txt_read() eq ( 0, "clean=true" ) )  || die "bad capability";
+( packet_txt_read() eq ( 0, "smudge=true" ) ) || die "bad capability";
+( packet_bin_read() eq ( 1, "" ) )            || die "bad capability end";
+
+packet_txt_write("clean=true");
+packet_txt_write("smudge=true");
+packet_flush();
+
+while (1) {
+	my ($command)  = packet_txt_read() =~ /^command=([^=]+)$/;
+	my ($pathname) = packet_txt_read() =~ /^pathname=([^=]+)$/;
+
+	packet_bin_read();
+
+	my $input = "";
+	{
+		binmode(STDIN);
+		my $buffer;
+		my $done = 0;
+		while ( !$done ) {
+			( $done, $buffer ) = packet_bin_read();
+			$input .= $buffer;
+		}
+	}
+
+	my $output;
+	if ( $command eq "clean" ) {
+		### Perform clean here ###
+		$output = $input;
+	}
+	elsif ( $command eq "smudge" ) {
+		### Perform smudge here ###
+		$output = $input;
+	}
+	else {
+		die "bad command '$command'";
+	}
+
+	packet_txt_write("status=success");
+	packet_flush();
+	while ( length($output) > 0 ) {
+		my $packet = substr( $output, 0, $MAX_PACKET_CONTENT_SIZE );
+		packet_bin_write($packet);
+		if ( length($output) > $MAX_PACKET_CONTENT_SIZE ) {
+			$output = substr( $output, $MAX_PACKET_CONTENT_SIZE );
+		}
+		else {
+			$output = "";
+		}
+	}
+	packet_flush();    # flush content!
+	packet_flush();    # empty list, keep "status=success" unchanged!
+
+}
-- 
2.10.0
Previous: larsxschneider@gmail.comNext: Torsten Bögershausen
Message 32 of 34 in “Git filter protocol”
  1. 00/14 Git filter protocollarsxschneider@gmail.com, Oct 8, 2016
  2. 01/14 convert: quote filter names in error messageslarsxschneider@gmail.com, Oct 8, 2016
  3. 02/14 convert: modernize testslarsxschneider@gmail.com, Oct 8, 2016
  4. 03/14 run-command: move check_pipe() from write_or_die to run_commandlarsxschneider@gmail.com, Oct 8, 2016
  5. 04/14 run-command: add clean_on_exit_handlerlarsxschneider@gmail.com, Oct 8, 2016
  6. Johannes SchindelinOct 11, 2016
  7. Lars SchneiderOct 15, 2016
  8. Johannes SchindelinOct 16, 2016
  9. Lars SchneiderOct 16, 2016
  10. 05/14 pkt-line: rename packet_write() to packet_write_fmt()larsxschneider@gmail.com, Oct 8, 2016
  11. 07/14 pkt-line: add packet_write_fmt_gently()larsxschneider@gmail.com, Oct 8, 2016
  12. 08/14 pkt-line: add packet_flush_gently()larsxschneider@gmail.com, Oct 8, 2016
  13. 09/14 pkt-line: add packet_write_gently()larsxschneider@gmail.com, Oct 8, 2016
  14. 11/14 convert: make apply_filter() adhere to standard Git error handlinglarsxschneider@gmail.com, Oct 8, 2016
  15. 06/14 pkt-line: extract set_packet_header()larsxschneider@gmail.com, Oct 8, 2016
  16. 12/14 convert: prepare filter.<driver>.process optionlarsxschneider@gmail.com, Oct 8, 2016
  17. 13/14 convert: add filter.<driver>.process optionlarsxschneider@gmail.com, Oct 8, 2016
  18. Jakub NarębskiOct 8, 2016
  19. Torsten BögershausenOct 9, 2016
  20. Lars SchneiderOct 11, 2016
  21. Lars SchneiderOct 11, 2016
  22. Jakub NarębskiOct 12, 2016
  23. Lars SchneiderOct 15, 2016
  24. Jeff KingOct 15, 2016
  25. Jakub NarębskiOct 15, 2016
  26. Junio C HamanoOct 10, 2016
  27. Lars SchneiderOct 11, 2016
  28. Torsten BögershausenOct 11, 2016
  29. Lars SchneiderOct 16, 2016
  30. Junio C HamanoOct 17, 2016
  31. 10/14 pkt-line: add functions to read/write flush terminated packet streamslarsxschneider@gmail.com, Oct 8, 2016
  32. 14/14 contrib/long-running-filter: add long running filter examplelarsxschneider@gmail.com, Oct 8, 2016
  33. Torsten BögershausenOct 9, 2016
  34. Lars SchneiderOct 15, 2016

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.