git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH v11 14/14] contrib/long-running-filter: add long running filter example

From
larsxschneider@gmail.com <larsxschneider@gmail.com>
Date
Oct 16, 2016, 23:20 UTC
Message-ID
<20161016232038.84951-15-larsxschneider@gmail.com>
In-Reply-To
<20161016232038.84951-1-larsxschneider@gmail.com>
From: Lars Schneider <larsxschneider@gmail.com>

Add a simple pass-thru filter as example implementation for the Git filter protocol version 2. See Documentation/gitattributes.txt, section "Filter Protocol" for more info.

Signed-off-by: Lars Schneider <larsxschneider@gmail.com>
---
 Documentation/gitattributes.txt        |   4 +-
 contrib/long-running-filter/example.pl | 128 +++++++++++++++++++++++++++++++++
 2 files changed, 131 insertions(+), 1 deletion(-)
 create mode 100755 contrib/long-running-filter/example.pl
diff --git a/Documentation/gitattributes.txt b/Documentation/gitattributes.txt
index 3f4d1ed..976243a 100644
--- a/Documentation/gitattributes.txt
+++ b/Documentation/gitattributes.txt
@@ -516,7 +516,9 @@ the command pipe on exit. The filter is expected to detect EOF
 and exit gracefully on its own. Git will wait until the filter
 process has stopped.
 
-If you develop your own long running filter
+A long running filter demo implementation can be found in
+`contrib/long-running-filter/example.pl` located in the Git
+core repository. If you develop your own long running filter
 process then the `GIT_TRACE_PACKET` environment variables can be
 very helpful for debugging (see linkgit:git[1]).
 
diff --git a/contrib/long-running-filter/example.pl b/contrib/long-running-filter/example.pl
new file mode 100755
index 0000000..3945705
--- /dev/null
+++ b/contrib/long-running-filter/example.pl
@@ -0,0 +1,128 @@
+#!/usr/bin/perl
+#
+# Example implementation for the Git filter protocol version 2
+# See Documentation/gitattributes.txt, section "Filter Protocol"
+#
+# Please note, this pass-thru filter is a minimal skeleton. No proper
+# error handling was implemented.
+#
+
+use strict;
+use warnings;
+
+my $MAX_PACKET_CONTENT_SIZE = 65516;
+
+sub packet_bin_read {
+	my $buffer;
+	my $bytes_read = read STDIN, $buffer, 4;
+	if ( $bytes_read == 0 ) {
+
+		# EOF - Git stopped talking to us!
+		exit();
+	}
+	elsif ( $bytes_read != 4 ) {
+		die "invalid packet: '$buffer'";
+	}
+	my $pkt_size = hex($buffer);
+	if ( $pkt_size == 0 ) {
+		return ( 1, "" );
+	}
+	elsif ( $pkt_size > 4 ) {
+		my $content_size = $pkt_size - 4;
+		$bytes_read = read STDIN, $buffer, $content_size;
+		if ( $bytes_read != $content_size ) {
+			die "invalid packet ($content_size bytes expected; $bytes_read bytes read)";
+		}
+		return ( 0, $buffer );
+	}
+	else {
+		die "invalid packet size: $pkt_size";
+	}
+}
+
+sub packet_txt_read {
+	my ( $res, $buf ) = packet_bin_read();
+	unless ( $buf =~ s/\n$// ) {
+		die "A non-binary line MUST be terminated by an LF.";
+	}
+	return ( $res, $buf );
+}
+
+sub packet_bin_write {
+	my $buf = shift;
+	print STDOUT sprintf( "%04x", length($buf) + 4 );
+	print STDOUT $buf;
+	STDOUT->flush();
+}
+
+sub packet_txt_write {
+	packet_bin_write( $_[0] . "\n" );
+}
+
+sub packet_flush {
+	print STDOUT sprintf( "%04x", 0 );
+	STDOUT->flush();
+}
+
+( packet_txt_read() eq ( 0, "git-filter-client" ) ) || die "bad initialize";
+( packet_txt_read() eq ( 0, "version=2" ) )         || die "bad version";
+( packet_bin_read() eq ( 1, "" ) )                  || die "bad version end";
+
+packet_txt_write("git-filter-server");
+packet_txt_write("version=2");
+packet_flush();
+
+( packet_txt_read() eq ( 0, "capability=clean" ) )  || die "bad capability";
+( packet_txt_read() eq ( 0, "capability=smudge" ) ) || die "bad capability";
+( packet_bin_read() eq ( 1, "" ) )                  || die "bad capability end";
+
+packet_txt_write("capability=clean");
+packet_txt_write("capability=smudge");
+packet_flush();
+
+while (1) {
+	my ($command)  = packet_txt_read() =~ /^command=([^=]+)$/;
+	my ($pathname) = packet_txt_read() =~ /^pathname=([^=]+)$/;
+
+	packet_bin_read();
+
+	my $input = "";
+	{
+		binmode(STDIN);
+		my $buffer;
+		my $done = 0;
+		while ( !$done ) {
+			( $done, $buffer ) = packet_bin_read();
+			$input .= $buffer;
+		}
+	}
+
+	my $output;
+	if ( $command eq "clean" ) {
+		### Perform clean here ###
+		$output = $input;
+	}
+	elsif ( $command eq "smudge" ) {
+		### Perform smudge here ###
+		$output = $input;
+	}
+	else {
+		die "bad command '$command'";
+	}
+
+	packet_txt_write("status=success");
+	packet_flush();
+	while ( length($output) > 0 ) {
+		my $packet = substr( $output, 0, $MAX_PACKET_CONTENT_SIZE );
+		packet_bin_write($packet);
+		if ( length($output) > $MAX_PACKET_CONTENT_SIZE ) {
+			$output = substr( $output, $MAX_PACKET_CONTENT_SIZE );
+		}
+		else {
+			$output = "";
+		}
+	}
+	packet_flush();    # flush content!
+	packet_flush();    # empty list, keep "status=success" unchanged!
+
+}
-- 
2.10.0
Previous: Johannes SixtNext: Junio C Hamano
Message 24 of 25 in “Git filter protocol”
  1. 00/14 Git filter protocollarsxschneider@gmail.com, Oct 16, 2016
  2. 01/14 convert: quote filter names in error messageslarsxschneider@gmail.com, Oct 16, 2016
  3. 02/14 convert: modernize testslarsxschneider@gmail.com, Oct 16, 2016
  4. 03/14 run-command: move check_pipe() from write_or_die to run_commandlarsxschneider@gmail.com, Oct 16, 2016
  5. 04/14 run-command: add clean_on_exit_handlerlarsxschneider@gmail.com, Oct 16, 2016
  6. 05/14 pkt-line: rename packet_write() to packet_write_fmt()larsxschneider@gmail.com, Oct 16, 2016
  7. 08/14 pkt-line: add packet_flush_gently()larsxschneider@gmail.com, Oct 16, 2016
  8. 07/14 pkt-line: add packet_write_fmt_gently()larsxschneider@gmail.com, Oct 16, 2016
  9. 06/14 pkt-line: extract set_packet_header()larsxschneider@gmail.com, Oct 16, 2016
  10. 09/14 pkt-line: add packet_write_gently()larsxschneider@gmail.com, Oct 16, 2016
  11. 10/14 pkt-line: add functions to read/write flush terminated packet streamslarsxschneider@gmail.com, Oct 16, 2016
  12. 11/14 convert: make apply_filter() adhere to standard Git error handlinglarsxschneider@gmail.com, Oct 16, 2016
  13. 12/14 convert: prepare filter.<driver>.process optionlarsxschneider@gmail.com, Oct 16, 2016
  14. 13/14 convert: add filter.<driver>.process optionlarsxschneider@gmail.com, Oct 16, 2016
  15. Johannes SixtNov 2, 2016
  16. Lars SchneiderNov 3, 2016
  17. t0021: expect more variations in the output of uniq -cJohannes Sixt, Nov 3, 2016
  18. t0021: use arithmetic expansion to trim whitespace from wc -c outputJohannes Sixt, Nov 3, 2016
  19. Lars SchneiderNov 6, 2016
  20. Johannes SixtNov 6, 2016
  21. Lars SchneiderNov 6, 2016
  22. Lars SchneiderNov 6, 2016
  23. Johannes SixtNov 6, 2016
  24. 14/14 contrib/long-running-filter: add long running filter examplelarsxschneider@gmail.com, Oct 16, 2016
  25. Junio C HamanoOct 17, 2016

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.