threads / patch / 32770

patchgit-send-email: add ~/.authinfo parsing

Subject: [PATCH] git-send-email: add ~/.authinfo parsing

## tl;dr

78 messages between Jan 29, 2013 and Feb 8, 2013. Diffs are folded; open one to read it.

replies: 77people: 8as markdown or json

Michal Nazarewicz· Jan 29, 2013, 19:13 UTC · lore
From: Michal Nazarewicz <mina86@mina86.com>

Make git-send-email read password from a ~/.authinfo file instead of requiring it to be stored in git configuration, passed as command line argument or typed in.

There are various other applications that use this file for authentication information so letting users use it for git-send-email is convinient. Furthermore, some users store their ~/.gitconfig file in a public repository and having to store password there makes it easy to publish the password.

Not to introduce any new dependencies, ~/.authinfo file is parsed only if Text::CSV Perl module is installed. If it's not, a notification is printed and the file is ignored.

Signed-off-by: Michal Nazarewicz <mina86@mina86.com>
---
 Documentation/git-send-email.txt | 15 +++++++--
 git-send-email.perl              | 69 +++++++++++++++++++++++++++++++++-------
 2 files changed, 70 insertions(+), 14 deletions(-)
Show changes to 2 files +70 −14

Documentation/git-send-email.txt, git-send-email.perl

diff --git a/Documentation/git-send-email.txt b/Documentation/git-send-email.txt
index eeb561c..b83576e 100644
--- a/Documentation/git-send-email.txt
+++ b/Documentation/git-send-email.txt
@@ -158,14 +158,25 @@ Sending
 --smtp-pass[=<password>]::
 	Password for SMTP-AUTH. The argument is optional: If no
 	argument is specified, then the empty string is used as
-	the password. Default is the value of 'sendemail.smtppass',
-	however '--smtp-pass' always overrides this value.
+	the password. Default is the value of 'sendemail.smtppass'
+	or value read from '~/.authinfo' file, however '--smtp-pass'
+	always overrides this value.
 +
 Furthermore, passwords need not be specified in configuration files
 or on the command line. If a username has been specified (with
 '--smtp-user' or a 'sendemail.smtpuser'), but no password has been
 specified (with '--smtp-pass' or 'sendemail.smtppass'), then the
 user is prompted for a password while the input is masked for privacy.
++
+The '~/.authinfo' file is read if Text::CSV Perl module is installed
+on the system; if it's missing, a notification message will be printed
+and the file ignored altogether.  The file should contain a line with
+the following format:
++
+  machine <domain> port <port> login <user> password <pass>
++
+Contrary to other tools, 'git-send-email' does not support symbolic
+port names like 'imap' thus `<port>` must be a number.
 
 --smtp-server=<host>::
 	If set, specifies the outgoing SMTP server to use (e.g.
diff --git a/git-send-email.perl b/git-send-email.perl
index be809e5..d824098 100755
--- a/git-send-email.perl
+++ b/git-send-email.perl
@@ -1045,6 +1045,62 @@ sub maildomain {
 	return maildomain_net() || maildomain_mta() || 'localhost.localdomain';
 }
 
+
+sub read_password_from_stdin {
+	my $line;
+
+	system "stty -echo";
+
+	do {
+		print "Password: ";
+		$line = <STDIN>;
+		print "\n";
+	} while (!defined $line);
+
+	system "stty echo";
+
+	chomp $line;
+	return $line;
+}
+
+sub read_password_from_authinfo {
+	my $fd;
+	if (!open $fd, '<', $ENV{'HOME'} . '/.authinfo') {
+		return;
+	}
+
+	if (!eval { require Text::CSV; 1 }) {
+		print STDERR "Text::CSV missing, won't read ~/.authinfo\n";
+		close $fd;
+		return;
+	}
+
+	my $csv = Text::CSV->new( { sep_char => ' ' } );
+	my $password;
+	while (my $line = <$fd>) {
+		chomp $line;
+		$csv->parse($line);
+		my %row = $csv->fields();
+		if (defined $row{'machine'} &&
+		    defined $row{'login'} &&
+		    defined $row{'port'} &&
+		    defined $row{'password'} &&
+		    $row{'machine'} eq $smtp_server &&
+		    $row{'login'} eq $smtp_authuser &&
+		    $row{'port'} eq $smtp_server_port) {
+			$password = $row{'password'};
+			last;
+		}
+	}
+
+	close $fd;
+	return $password;
+}
+
+sub read_password {
+	return read_password_from_authinfo || read_password_from_stdin;
+}
+
 # Returns 1 if the message was sent, and 0 otherwise.
 # In actuality, the whole program dies when there
 # is an error sending a message.
@@ -1194,18 +1250,7 @@ X-Mailer: git-send-email $gitversion
 			};
 
 			if (!defined $smtp_authpass) {
-
-				system "stty -echo";
-
-				do {
-					print "Password: ";
-					$_ = <STDIN>;
-					print "\n";
-				} while (!defined $_);
-
-				chomp($smtp_authpass = $_);
-
-				system "stty echo";
+				$smtp_authpass = read_password
 			}
 
 			$auth ||= $smtp->auth( $smtp_authuser, $smtp_authpass ) or die $smtp->message;
-- 
1.8.1
Junio C Hamano· Jan 29, 2013, 19:53 UTC · re: Michal Nazarewicz · lore

Re: [PATCH] git-send-email: add ~/.authinfo parsing

Michal Nazarewicz <mpn@google.com> writes:
Show 5 quoted lines
> From: Michal Nazarewicz <mina86@mina86.com>
>
> Make git-send-email read password from a ~/.authinfo file instead of
> requiring it to be stored in git configuration, passed as command line
> argument or typed in.
Makes one wonder why .authinfo and not .netrc; 
http://www.gnu.org/software/emacs/manual/html_node/auth/Help-for-users.html
phrases it amusingly:
        “Netrc” files are usually called .authinfo or .netr
        nowadays .authinfo seems to be more popular and the
        auth-source library encourages this confusion by accepting
        both

Either way it still encourages a plaintext password to be on disk, which may not be what we want, even though it may be slight if not really much of an improvement. Again the Help-for-users has this amusing bit:

	You could just say (but we don't recommend it, we're just
	showing that it's possible)
	     password mypassword
	to use the same password everywhere. Again, DO NOT DO THIS
	or you will be pwned as the kids say.
Show 6 quoted lines
> +The '~/.authinfo' file is read if Text::CSV Perl module is installed
> +on the system; if it's missing, a notification message will be printed
> +and the file ignored altogether.  The file should contain a line with
> +the following format:
> ++
> +  machine <domain> port <port> login <user> password <pass>

It is rather strange to require a comma-separated-values parser to read a file format this simple, isn't it?

> ++
> +Contrary to other tools, 'git-send-email' does not support symbolic
> +port names like 'imap' thus `<port>` must be a number.

Perhaps you can convert at least some popular ones yourself? After all, the user may be using an _existing_ .authinfo/.netrc that she has been using with other programs that do understand symbolic port names. Rather than forcing all such users to update their files, the patch can work a bit harder for them and the world will be a better place, no?

Michal Nazarewicz· Jan 29, 2013, 21:08 UTC · re: Junio C Hamano · lore

[PATCHv2] git-send-email: add ~/.authinfo parsing

From: Michal Nazarewicz <mina86@mina86.com>

Make git-send-email read password from a ~/.authinfo or a ~/.netrc file instead of requiring it to be stored in git configuration, passed as command line argument or typed in.

There are various other applications that use this file for authentication information so letting users use it for git-send-email is convinient. Furthermore, some users store their ~/.gitconfig file in a public repository and having to store password there makes it easy to publish the password.

Signed-off-by: Michal Nazarewicz <mina86@mina86.com>
---
 Documentation/git-send-email.txt |  34 +++++++++--
 git-send-email.perl              | 124 +++++++++++++++++++++++++++++++++++----
 2 files changed, 140 insertions(+), 18 deletions(-)
On Tue, Jan 29 2013, Junio C Hamano wrote:
> Makes one wonder why .authinfo and not .netrc; 
Fine… Let's parse both. ;)
> Either way it still encourages a plaintext password to be on disk,
> which may not be what we want, even though it may be slight if not
> really much of an improvement.

Well… Users store passwords on disks in a lot of places. I wager that most have mail clients configured not to ask for password but instead store it on hard drive. I don't see that changing any time soon, so at least we can try and minimise number of places where a password is stored.

> It is rather strange to require a comma-separated-values parser to
> read a file format this simple, isn't it?

I was worried about spaces in password. CVS should handle such case nicely, whereas simple split won't. Nonetheless, I guess that in the end this is not likely enough to add the dependency.

Show 6 quoted lines
> Perhaps you can convert at least some popular ones yourself?  After
> all, the user may be using an _existing_ .authinfo/.netrc that she
> has been using with other programs that do understand symbolic port
> names.  Rather than forcing all such users to update their files,
> the patch can work a bit harder for them and the world will be a
> better place, no?
Parsing /etc/services added.
Show changes to 2 files +140 −18

Documentation/git-send-email.txt, git-send-email.perl

diff --git a/Documentation/git-send-email.txt b/Documentation/git-send-email.txt
index eeb561c..ee20714 100644
--- a/Documentation/git-send-email.txt
+++ b/Documentation/git-send-email.txt
@@ -158,14 +158,36 @@ Sending
 --smtp-pass[=<password>]::
 	Password for SMTP-AUTH. The argument is optional: If no
 	argument is specified, then the empty string is used as
-	the password. Default is the value of 'sendemail.smtppass',
-	however '--smtp-pass' always overrides this value.
+	the password. Default is the value of 'sendemail.smtppass'
+	or value read from ~/.authinfo file, however '--smtp-pass'
+	always overrides this value.
 +
-Furthermore, passwords need not be specified in configuration files
-or on the command line. If a username has been specified (with
+Furthermore, passwords need not be specified in configuration files or
+on the command line. If a username has been specified (with
 '--smtp-user' or a 'sendemail.smtpuser'), but no password has been
-specified (with '--smtp-pass' or 'sendemail.smtppass'), then the
-user is prompted for a password while the input is masked for privacy.
+specified (with '--smtp-pass', 'sendemail.smtppass' or via
+~/.authinfo file), then the user is prompted for a password while
+the input is masked for privacy.
++
+The ~/.authinfo file should contain a line with the following
+format:
++
+  machine <domain> port <port> login <user> password <pass>
++
+Each pair (expect for `password <pass>`) can be omitted which will
+skip matching of the given value.  Lines are interpreted in order and
+password from the first line that matches will be used.  `<port>` can
+be either an integer or a symbolic name.  In the latter case, it is
+looked up in `/etc/services` file (if it exists).  For instance, you
+can put
++
+  machine example.com login testuser port ssmtp password smtppassword
+  machine example.com login testuser            password testpassword
++
+if you want to use `smtppassword` for authenticating to a service at
+port 465 (SSMTP) and `testpassword` for all other services.  As shown
+in the example, `<port>` can use   If ~/.authinfo file is
+missing, 'git-send-email' will also try ~/.netrc file.
 
 --smtp-server=<host>::
 	If set, specifies the outgoing SMTP server to use (e.g.
diff --git a/git-send-email.perl b/git-send-email.perl
index be809e5..2d8fd1b 100755
--- a/git-send-email.perl
+++ b/git-send-email.perl
@@ -1045,6 +1045,117 @@ sub maildomain {
 	return maildomain_net() || maildomain_mta() || 'localhost.localdomain';
 }
 
+
+sub read_password_from_stdin {
+	my $line;
+
+	system "stty -echo";
+
+	do {
+		print "Password: ";
+		$line = <STDIN>;
+		print "\n";
+	} while (!defined $line);
+
+	system "stty echo";
+
+	chomp $line;
+	return $line;
+}
+
+sub read_etc_services {
+	my $fd;
+	if (!open $fd, '<', '/etc/services') {
+		return {};
+	}
+
+	my $ret = {};
+	while (my $line = <$fd>) {
+		$line =~ s/^\s+|\s*(?:#.*)?$//g;
+		my @line = split /\s+/, $line;
+		if (@line < 2 || $line[1] !~ m~^(\d+)/tcp$~) {
+			next;
+		}
+
+		my $num = int $1;
+		undef $line[1];
+		for my $service (@line) {
+			if (defined $service && !defined $ret->{$service}) {
+				$ret->{$service} = $num;
+			}
+		}
+	}
+
+	close $fd;
+	return $ret;
+}
+
+my $authinfo_parse_port;
+
+sub authinfo_is_eq_port {
+	my ($from_file, $value, $filename) = @_;
+
+	if (!defined $from_file) {
+		return 1;
+	} elsif ($from_file =~ /^\d+$/) {
+		return $from_file == $value;
+	}
+
+	if (!defined $authinfo_parse_port) {
+		$authinfo_parse_port = read_etc_services;
+	}
+
+	my $port = $authinfo_parse_port->{$from_file};
+	if (!defined $port) {
+		print STDERR "$filename: invalid port name: $from_file\n";
+		return;
+	}
+
+	return $port == $value;
+}
+
+sub authinfo_is_eq {
+	my ($from_file, $value) = @_;
+	return defined $from_file || $from_file eq $value;
+}
+
+sub read_password_from_authinfo {
+	my $filename = join '/', $ENV{'HOME'}, $_[0] // '.authinfo';
+	my $fd;
+	if (!open $fd, '<', $filename) {
+		return;
+	}
+
+	my $password;
+	while (my $line = <$fd>) {
+		$line =~ s/^\s+|\s+$//g;
+		my @line = split /\s+/, $line;
+		if (@line % 2) {
+			next;
+		}
+
+		my %line = @line;
+		if (defined $line{'password'} &&
+		    authinfo_is_eq $line{'machine'}, $smtp_server &&
+		    authinfo_is_eq $line{'login'}, $smtp_authuser &&
+		    authinfo_is_eq_port $line{'port'}, $smtp_server_port, $filename) {
+			$password = $line{'password'};
+			last;
+		}
+	}
+
+	close $fd;
+	return $password;
+}
+
+sub read_password {
+	return
+	  read_password_from_authinfo '.authinfo' ||
+	  read_password_from_authinfo '.netrc' ||
+	  read_password_from_stdin;
+}
+
+
 # Returns 1 if the message was sent, and 0 otherwise.
 # In actuality, the whole program dies when there
 # is an error sending a message.
@@ -1194,18 +1305,7 @@ X-Mailer: git-send-email $gitversion
 			};
 
 			if (!defined $smtp_authpass) {
-
-				system "stty -echo";
-
-				do {
-					print "Password: ";
-					$_ = <STDIN>;
-					print "\n";
-				} while (!defined $_);
-
-				chomp($smtp_authpass = $_);
-
-				system "stty echo";
+				$smtp_authpass = read_password
 			}
 
 			$auth ||= $smtp->auth( $smtp_authuser, $smtp_authpass ) or die $smtp->message;
-- 
1.8.1
Junio C Hamano· Jan 29, 2013, 22:38 UTC · re: Michal Nazarewicz · lore

Re: [PATCHv2] git-send-email: add ~/.authinfo parsing

Michal Nazarewicz <mpn@google.com> writes:
Show 6 quoted lines
>> It is rather strange to require a comma-separated-values parser to
>> read a file format this simple, isn't it?
>
> I was worried about spaces in password.  CVS should handle such case
> nicely, whereas simple split won't.  Nonetheless, I guess that in the
> end this is not likely enough to add the dependency.

But .netrc/.authinfo format separates its entries with SP, HT, or LF. An entry begins with "machine <remote-hostname>" token pair.

split(/\s+/) will not work for an entry that span multiple lines but CSV will not help, either.

Is it bad to use Net::Netrc instead? This looks like exactly the use case that module was written for, no?

Show 8 quoted lines
>> Perhaps you can convert at least some popular ones yourself?  After
>> all, the user may be using an _existing_ .authinfo/.netrc that she
>> has been using with other programs that do understand symbolic port
>> names.  Rather than forcing all such users to update their files,
>> the patch can work a bit harder for them and the world will be a
>> better place, no?
>
> Parsing /etc/services added.
Hmph.  I would have expected to see getservbyname.
Michal Nazarewicz· Jan 30, 2013, 00:03 UTC · re: Junio C Hamano · lore

[PATCHv3] git-send-email: add ~/.authinfo parsing

From: Michal Nazarewicz <mina86@mina86.com>

Make git-send-email read password from a ~/.authinfo or ~/.netrc file instead of requiring it to be stored in git configuration, passed as command line argument or typed in.

There are various other applications that use this file for authentication information so letting users use it for git-send-email is convinient. Furthermore, some users store their ~/.gitconfig file in a public repository and having to store password there makes it easy to publish the password.

Signed-off-by: Michal Nazarewicz <mina86@mina86.com>
---
 Documentation/git-send-email.txt | 47 +++++++++++++++++---
 git-send-email.perl              | 93 ++++++++++++++++++++++++++++++++++------
 2 files changed, 122 insertions(+), 18 deletions(-)
On Tue, Jan 29 2013, Junio C Hamano wrote:
Show 8 quoted lines
> But .netrc/.authinfo format separates its entries with SP, HT, or
> LF.  An entry begins with "machine <remote-hostname>" token pair.
>
> split(/\s+/) will not work for an entry that span multiple lines but
> CSV will not help, either.
>
> Is it bad to use Net::Netrc instead?  This looks like exactly the
> use case that module was written for, no?

I don't think that's the case. For one, Net::Netrc does not seem to process port number.

There is a Text::Authinfo module but it just uses Text::CSV.

I can change the code to use Net::Netrc, but I dunno if that's really the best option, since I feel people would expect parsing to be somehow compatible with <http://www.gnu.org/software/emacs/manual/html_node/gnus/NNTP.html> rather than the original .netrc file format.

> Hmph.  I would have expected to see getservbyname.
Ha!  Even better. :]
Show changes to 2 files +122 −18

Documentation/git-send-email.txt, git-send-email.perl

diff --git a/Documentation/git-send-email.txt b/Documentation/git-send-email.txt
index eeb561c..ac020d1 100644
--- a/Documentation/git-send-email.txt
+++ b/Documentation/git-send-email.txt
@@ -158,14 +158,49 @@ Sending
 --smtp-pass[=<password>]::
 	Password for SMTP-AUTH. The argument is optional: If no
 	argument is specified, then the empty string is used as
-	the password. Default is the value of 'sendemail.smtppass',
-	however '--smtp-pass' always overrides this value.
+	the password. Default is the value of 'sendemail.smtppass'
+	or value read from ~/.authinfo file, however '--smtp-pass'
+	always overrides this value.
 +
-Furthermore, passwords need not be specified in configuration files
-or on the command line. If a username has been specified (with
+Furthermore, passwords need not be specified in configuration files or
+on the command line. If a username has been specified (with
 '--smtp-user' or a 'sendemail.smtpuser'), but no password has been
-specified (with '--smtp-pass' or 'sendemail.smtppass'), then the
-user is prompted for a password while the input is masked for privacy.
+specified (with '--smtp-pass', 'sendemail.smtppass' or via
+~/.authinfo file), then the user is prompted for a password while
+the input is masked for privacy.
++
+The ~/.authinfo file should contain a line with the following
+format:
++
+  machine <domain> port <port> login <user> password <pass>
++
+Instead of `machine <domain>` pair a `default` token can be used
+instead in which case all domains will match.  Similarly, `port
+<port>` and `login <user>` pairs can be omitted in which case matching
+of the given value will be skipped.  `<port>` can be either an integer
+or a symbolic name.  Lines are interpreted in order and password from
+the first line that matches will be used.  For instance, one may end
+up with:
++
+  machine example.com login jane port ssmtp password smtppassword
+  machine example.com login jane            password janepassword
+  default             login janedoe         password doepassword
++
+if she wants to use `smtppassword` for authenticating as `jane` to
+a service at example.com:465 (SSMTP), `janepassword` for all other
+services at example.com; and `doepassword` when authonticating as
+`janedoe` to any service.  If ~/.authinfo file is missing,
+'git-send-email' will also try ~/.netrc file (even though parsing is
+not fully compatible with ftp's .netrc file format).
++
+Note that you should never make ~/.authinfo file world-readable.  To
+help guarantee that, you might want to create the file with the
+following command:
++
+  ( umask 077; cat >~/.authinfo <<EOF
+  ... file contents ...
+  EOF
+  )
 
 --smtp-server=<host>::
 	If set, specifies the outgoing SMTP server to use (e.g.
diff --git a/git-send-email.perl b/git-send-email.perl
index be809e5..a62dfa4 100755
--- a/git-send-email.perl
+++ b/git-send-email.perl
@@ -1045,6 +1045,86 @@ sub maildomain {
 	return maildomain_net() || maildomain_mta() || 'localhost.localdomain';
 }
 
+
+sub read_password_from_stdin {
+	my $line;
+
+	system "stty -echo";
+
+	do {
+		print "Password: ";
+		$line = <STDIN>;
+		print "\n";
+	} while (!defined $line);
+
+	system "stty echo";
+
+	chomp $line;
+	return $line;
+}
+
+sub authinfo_is_port_eq {
+	my ($from_file, $value, $filename) = @_;
+
+	if (!defined $from_file) {
+		return 1;
+	} elsif ($from_file =~ /^\d+$/) {
+		return $from_file == $value;
+	}
+
+	my $port = getservbyname $from_file, 'tcp';
+	if (!defined $port) {
+		print STDERR "$filename: invalid port name: $from_file\n";
+		return;
+	}
+
+	return $port == $value;
+}
+
+sub read_password_from_authinfo {
+	my $filename = join '/', $ENV{'HOME'}, $_[0] // '.authinfo';
+	my $fd;
+	if (!open $fd, '<', $filename) {
+		return;
+	}
+
+	my $password;
+	while (my $line = <$fd>) {
+		$line =~ s/^\s+|\s+$//g;
+		my @line = split /\s+/, $line;
+		my %line;
+		while (@line) {
+			my $token = shift @line;
+			if ($token eq 'default') {
+				$line{'machine'} = $smtp_server;
+			} elsif (@line) {
+				$line{$token} = shift @line;
+			}
+		}
+
+		if (defined $line{'password'} &&
+		    defined $line{'machine'} &&
+		    $line{'machine'} eq $smtp_server &&
+		    (!defined $line{'login'} ||
+		     $line{'login'} eq $smtp_authuser) &&
+		    authinfo_is_port_eq($line{'port'}, $smtp_server_port, $filename)) {
+			$password = $line{'password'};
+			last;
+		}
+	}
+
+	close $fd;
+	return $password;
+}
+
+sub read_password {
+	return
+	  read_password_from_authinfo '.authinfo' ||
+	  read_password_from_authinfo '.netrc' ||
+	  read_password_from_stdin;
+}
+
+
 # Returns 1 if the message was sent, and 0 otherwise.
 # In actuality, the whole program dies when there
 # is an error sending a message.
@@ -1194,18 +1274,7 @@ X-Mailer: git-send-email $gitversion
 			};
 
 			if (!defined $smtp_authpass) {
-
-				system "stty -echo";
-
-				do {
-					print "Password: ";
-					$_ = <STDIN>;
-					print "\n";
-				} while (!defined $_);
-
-				chomp($smtp_authpass = $_);
-
-				system "stty echo";
+				$smtp_authpass = read_password
 			}
 
 			$auth ||= $smtp->auth( $smtp_authuser, $smtp_authpass ) or die $smtp->message;
-- 
1.8.1
Junio C Hamano· Jan 30, 2013, 00:34 UTC · re: Michal Nazarewicz · lore

Re: [PATCHv3] git-send-email: add ~/.authinfo parsing

Michal Nazarewicz <mpn@google.com> writes:
Show 13 quoted lines
>> Is it bad to use Net::Netrc instead?  This looks like exactly the
>> use case that module was written for, no?
>
> I don't think that's the case.  For one, Net::Netrc does not seem to
> process port number.
>
> There is a Text::Authinfo module but it just uses Text::CSV.
>
> I can change the code to use Net::Netrc, but I dunno if that's really
> the best option, since I feel people would expect parsing to be
> somehow compatible with
> <http://www.gnu.org/software/emacs/manual/html_node/gnus/NNTP.html>
> rather than the original .netrc file format.

Thanks for pushing back (I wish more contributors did so when I suggest nonsense ;-)); you are right that both canned modules are lacking.

Will queue.  Thanks.
Jeff King· Jan 30, 2013, 07:43 UTC · re: Junio C Hamano · lore

Re: [PATCH] git-send-email: add ~/.authinfo parsing

On Tue, Jan 29, 2013 at 11:53:19AM -0800, Junio C Hamano wrote:
> Either way it still encourages a plaintext password to be on disk,
> which may not be what we want, even though it may be slight if not
> really much of an improvement.  Again the Help-for-users has this
> amusing bit:

I do not mind a .netrc or .authinfo parser, because while those formats do have security problems, they are standard files that may already be in use. So as long as we are not encouraging their use, I do not see a problem in supporting them (and we already do the same with curl's netrc support).

But it would probably make sense for send-email to support the existing git-credential subsystem, so that it can take advantage of secure system-specific storage. And that is where we should be pointing new users. I think contrib/mw-to-git even has credential support written in perl, so it would just need to be factored out to Git.pm.

-Peff
Junio C Hamano· Jan 30, 2013, 15:57 UTC · re: Jeff King · lore

Re: [PATCH] git-send-email: add ~/.authinfo parsing

Jeff King <peff@peff.net> writes:
Show 5 quoted lines
> But it would probably make sense for send-email to support the existing
> git-credential subsystem, so that it can take advantage of secure
> system-specific storage. And that is where we should be pointing new
> users. I think contrib/mw-to-git even has credential support written in
> perl, so it would just need to be factored out to Git.pm.
Yeah, that sounds like a neat idea.
Ted Zlatanov· Jan 31, 2013, 15:23 UTC · re: Junio C Hamano · lore

Re: [PATCH] git-send-email: add ~/.authinfo parsing

On Wed, 30 Jan 2013 07:57:29 -0800 Junio C Hamano <gitster@pobox.com> wrote: 
JCH> Jeff King <peff@peff.net> writes:
Show 5 quoted lines
>> But it would probably make sense for send-email to support the existing
>> git-credential subsystem, so that it can take advantage of secure
>> system-specific storage. And that is where we should be pointing new
>> users. I think contrib/mw-to-git even has credential support written in
>> perl, so it would just need to be factored out to Git.pm.
JCH> Yeah, that sounds like a neat idea.

Jeff, is there a way for git-credential to currently support authinfo/netrc parsing? I assume that's the right way, instead of using Michal's proposal to parse internally?

I'd like to add that, plus support for the 'string' and "string" formats, and authinfo.gpg decoding through GPG. I'd write it in Perl, if there's a choice.

Ted
Jeff King· Jan 31, 2013, 19:38 UTC · re: Ted Zlatanov · lore

Re: [PATCH] git-send-email: add ~/.authinfo parsing

On Thu, Jan 31, 2013 at 10:23:51AM -0500, Ted Zlatanov wrote:
Show 7 quoted lines
> Jeff, is there a way for git-credential to currently support
> authinfo/netrc parsing?  I assume that's the right way, instead of using
> Michal's proposal to parse internally?
> 
> I'd like to add that, plus support for the 'string' and "string"
> formats, and authinfo.gpg decoding through GPG.  I'd write it in Perl,
> if there's a choice.

Yes, you could write a credential helper that understands netrc and friends; git talks to the helpers over a socket, so there is no problem with writing it in Perl. See Documentation/technical/api-credentials.txt for an overview, or the sample implementation in credential-store.c for a simple example.

-Peff
Ted Zlatanov· Feb 2, 2013, 11:57 UTC · re: Jeff King · lore

Re: [PATCH] git-send-email: add ~/.authinfo parsing

On Thu, 31 Jan 2013 14:38:45 -0500 Jeff King <peff@peff.net> wrote: 
JK> On Thu, Jan 31, 2013 at 10:23:51AM -0500, Ted Zlatanov wrote:
Show 7 quoted lines
>> Jeff, is there a way for git-credential to currently support
>> authinfo/netrc parsing?  I assume that's the right way, instead of using
>> Michal's proposal to parse internally?
>> 
>> I'd like to add that, plus support for the 'string' and "string"
>> formats, and authinfo.gpg decoding through GPG.  I'd write it in Perl,
>> if there's a choice.

JK> Yes, you could write a credential helper that understands netrc and JK> friends; git talks to the helpers over a socket, so there is no problem JK> with writing it in Perl. See Documentation/technical/api-credentials.txt JK> for an overview, or the sample implementation in credential-store.c for a JK> simple example.

I wrote a Perl credential helper for netrc parsing which is pretty robust, has built-in docs with -h, and doesn't depend on external modules. The netrc parser regex was stolen from Net::Netrc.

It will by default use ~/.authinfo.gpg, ~/.netrc.gpg, ~/.authinfo, and ~/.netrc (whichever is found first) and this can be overridden with -f.

If the file name ends with ".gpg", it will run "gpg --decrypt FILE" and use the output. So non-interactively, that could hang if GPG was waiting for input. Does Git handle that, or should I check for a TTY?

Take a look at the proposed patch and let me know if it's usable, if you need a formal copyright assignment, etc.

Thanks Ted

commit 3d28bc2a610ebcc988eba5443d82d0ded92c24bc
Author: Ted Zlatanov <tzz@lifelogs.com>
Date:   Sat Feb 2 06:42:13 2013 -0500
    Add contrib/credentials/netrc with GPG support
Show changes to contrib/credential/netrc/git-credential-netrc +242 −0
diff --git a/contrib/credential/netrc/git-credential-netrc b/contrib/credential/netrc/git-credential-netrc
new file mode 100755
index 0000000..92fc306
--- /dev/null
+++ b/contrib/credential/netrc/git-credential-netrc
@@ -0,0 +1,242 @@
+#!/usr/bin/perl
+
+use strict;
+use warnings;
+
+use Data::Dumper;
+
+use Getopt::Long;
+use File::Basename;
+
+my $VERSION = "0.1";
+
+my %options = (
+               help => 0,
+               debug => 0,
+
+               # identical token maps, e.g. host -> host, will be inserted later
+               tmap => {
+                        port => 'protocol',
+                        machine => 'host',
+                        path => 'path',
+                        login => 'username',
+                        user => 'username',
+                        password => 'password',
+                       }
+              );
+
+foreach my $v (values %{$options{tmap}})
+{
+ $options{tmap}->{$v} = $v;
+}
+
+foreach my $suffix ('.gpg', '')
+{
+ foreach my $base (qw/authinfo netrc/)
+ {
+  my $file = glob("~/.$base$suffix");
+  next unless (defined $file && -f $file);
+  $options{file} = $file ;
+ }
+}
+
+Getopt::Long::Configure("bundling");
+
+# TODO: maybe allow the token map $options{tmap} to be configurable.
+GetOptions(\%options,
+           "help|h",
+           "debug|d",
+           "file|f=s",
+          );
+
+if ($options{help})
+{
+ my $shortname = basename($0);
+ $shortname =~ s/git-credential-//;
+
+ print <<EOHIPPUS;
+
+$0 [-f AUTHFILE] [-d] get
+
+Version $VERSION by tzz\@lifelogs.com.  License: any use is OK.
+
+Options:
+  -f AUTHFILE: specify a netrc-style file
+  -d: turn on debugging
+
+To enable (note that Git will prepend "git-credential-" to the helper
+name and look for it in the path):
+
+  git config credential.helper '$shortname -f AUTHFILE'
+
+And if you want lots of debugging info:
+
+  git config credential.helper '$shortname -f AUTHFILE -d'
+
+Only "get" mode is supported by this credential helper.  It opens
+AUTHFILE and looks for entries that match the requested search
+criteria:
+
+ 'port|protocol':
+   The protocol that will be used (e.g., https). (protocol=X)
+
+ 'machine|host':
+   The remote hostname for a network credential. (host=X)
+
+ 'path':
+   The path with which the credential will be used. (path=X)
+
+ 'login|user|username':
+   The credential’s username, if we already have one. (username=X)
+
+Thus, when we get "protocol=https\nusername=tzz", this credential
+helper will look for lines in AUTHFILE that match
+
+port https login tzz
+
+OR
+
+protocol https login tzz
+
+OR... etc. acceptable tokens as listed above.  Any unknown tokens are
+simply ignored.
+
+Then, the helper will print out whatever tokens it got from the line,
+including "password" tokens, mapping e.g. "port" back to "protocol".
+
+The first matching line is used.  Tokens can be quoted as 'STRING' or
+"STRING".
+
+No caching is performed by this credential helper.
+
+EOHIPPUS
+
+ exit;
+}
+
+my $mode = shift @ARGV;
+
+# credentials may get 'get', 'store', or 'erase' as parameters but
+# only acknowledge 'get'
+die "Syntax: $0 [-f AUTHFILE] [-d] get" unless defined $mode;
+
+# only support 'get' mode
+exit unless $mode eq 'get';
+
+my $debug = $options{debug};
+my $file = $options{file};
+
+die "Sorry, you need to specify an existing netrc file (with or without a .gpg extension) with -f AUTHFILE"
+ unless defined $file;
+
+die "Sorry, the specified netrc $file is not accessible"
+ unless -f $file;
+
+if ($file =~ m/\.gpg$/)
+{
+ $file = "gpg --decrypt $file|";
+}
+
+my @data = load($file);
+chomp @data;
+
+die "Sorry, we could not load data from [$file]"
+ unless (scalar @data);
+
+# the query
+my %q;
+
+foreach my $v (values %{$options{tmap}})
+{
+ undef $q{$v};
+}
+
+while (<STDIN>)
+{
+ next unless m/([a-z]+)=(.+)/;
+
+ my ($token, $value) = ($1, $2);
+ die "Unknown search token $1" unless exists $q{$token};
+ $q{$token} = $value;
+}
+
+# build reverse token map
+my %rmap;
+foreach my $k (keys %{$options{tmap}})
+{
+ push @{$rmap{$options{tmap}->{$k}}}, $k;
+}
+
+# there are CPAN modules to do this better, but we want to avoid
+# dependencies and generally, complex netrc-style files are rare
+
+if ($debug)
+{
+ foreach (sort keys %q)
+ {
+  printf STDERR "searching for %s = %s\n",
+   $_, $q{$_} || '(any value)';
+ }
+}
+
+LINE: foreach my $line (@data)
+{
+
+ print STDERR "line [$line]\n" if $debug;
+ my @tok;
+ # gratefully stolen from Net::Netrc
+ while (length $line &&
+        $line =~ s/^("((?:[^"]+|\\.)*)"|((?:[^\\\s]+|\\.)*))\s*//)
+ {
+  (my $tok = $+) =~ s/\\(.)/$1/g;
+  push(@tok, $tok);
+ }
+
+ my %tokens;
+ while (@tok)
+ {
+  my ($k, $v) = (shift @tok, shift @tok);
+  next unless defined $v;
+  next unless exists $options{tmap}->{$k};
+  $tokens{$options{tmap}->{$k}} = $v;
+ }
+
+ foreach my $check (sort keys %q)
+ {
+  if (exists $tokens{$check} && defined $q{$check})
+  {
+   print STDERR "comparing [$tokens{$check}] to [$q{$check}] in line [$line]\n" if $debug;
+   next LINE unless $tokens{$check} eq $q{$check};
+  }
+  else
+  {
+   print STDERR "we could not find [$check] but it's OK\n" if $debug;
+  }
+ }
+
+ print STDERR "line has passed all the search checks\n" if $debug;
+ foreach my $token (sort keys %rmap)
+ {
+  print STDERR "looking for useful token $token\n" if $debug;
+  next unless exists $tokens{$token}; # did we match?
+
+  foreach my $rctoken (@{$rmap{$token}})
+  {
+   next if defined $q{$rctoken};           # don't re-print given tokens
+  }
+
+  print STDERR "FOUND: $token=$tokens{$token}\n" if $debug;
+  printf "%s=%s\n", $token, $tokens{$token};
+ }
+
+ last;
+}
+
+sub load
+{
+ my $file = shift;
+ # this supports pipes too
+ my $io = new IO::File($file) or die "Could not open $file: $!\n";
+
+ return <$io>;                          # whole file
+}
Jeff King· Feb 3, 2013, 19:41 UTC · re: Ted Zlatanov · lore

Re: [PATCH] git-send-email: add ~/.authinfo parsing

On Sat, Feb 02, 2013 at 06:57:29AM -0500, Ted Zlatanov wrote:
Show 6 quoted lines
> I wrote a Perl credential helper for netrc parsing which is pretty
> robust, has built-in docs with -h, and doesn't depend on external
> modules.  The netrc parser regex was stolen from Net::Netrc.
>
> It will by default use ~/.authinfo.gpg, ~/.netrc.gpg, ~/.authinfo, and
> ~/.netrc (whichever is found first) and this can be overridden with -f.
Cool, thanks for working on this.
> If the file name ends with ".gpg", it will run "gpg --decrypt FILE" and
> use the output.  So non-interactively, that could hang if GPG was
> waiting for input.  Does Git handle that, or should I check for a TTY?

No, git does not do anything special with respect to credential helpers and ttys (nor should it, since one use of helpers is to get credentials when there is no tty). I think it is GPG's problem to deal with, though. We will invoke it, and it is up to it to decide whether it can acquire the passphrase or not (either through the tty, or possibly from gpg-agent). So it would be wrong to do the tty check yourself.

I haven't tested GPG, but I assume it properly tries to read from /dev/tty and not stdin. Your helper's stdio is connected to git and speaking the credential-helper protocol, so GPG reading from stdin would either steal your input (if run before you read it), or just get EOF (if you have read all of the pipe content already). If GPG isn't well behaved, it may be worth redirecting its stdin from /dev/null as a safety measure.

> Take a look at the proposed patch and let me know if it's usable, if you
> need a formal copyright assignment, etc.

Overall looks sane to me, though my knowledge of .netrc is not especially good. Usually we try to send patches inline in the email (i.e., as generated by git-format-patch), and include a "Signed-off-by" line indicating that content is released to the project; see Documentation/SubmittingPatches.

> +use Data::Dumper;
I don't see it used here. Leftover from debugging?
> + print <<EOHIPPUS;
Cute, I haven't seen that one before.
> +$0 [-f AUTHFILE] [-d] get
> +
> +Version $VERSION by tzz\@lifelogs.com.  License: any use is OK.

I don't know if we have a particular policy for items in contrib/, but this license may be too vague. In particular, it does not explicitly allow redistribution, which would make Junio shipping a release with it a copyright violation.

Any objection to just putting it under some well-known simple license (GPL, BSD, or whatever)?

> +if ($file =~ m/\.gpg$/)
> +{
> + $file = "gpg --decrypt $file|";
> +}

Does this need to quote $file, since the result will get passed to the shell? It might be easier to just use the list form of open(), like:

  my @data = $file =~ /\.gpg$/ ?
             load('-|', qw(gpg --decrypt), $file) :
             load('<', $file);
(and then obviously update load to just dump all of @_ to open()).
> +die "Sorry, we could not load data from [$file]"
> + unless (scalar @data);

Probably not that interesting a corner case, but this means we die on an empty .netrc, whereas it might be more sensible for it to behave as "no match".

For the same reason, it might be worth silently exiting when we don't find a .netrc (or any of its variants). That lets people who share their dot-files across machines configure git globally, even if they don't necessarily have a netrc on every machine.

Show 7 quoted lines
> +# the query
> +my %q;
> +
> +foreach my $v (values %{$options{tmap}})
> +{
> + undef $q{$v};
> +}

Just my personal style, but I find the intent more obvious with "map" (I know some people find it unreadable, though):

  my %q = map { $_ => undef } values(%{$options{tmap}});
> +while (<STDIN>)
> +{
> + next unless m/([a-z]+)=(.+)/;

We don't currently have any exotic tokens that this would not match, nor do I plan to add them, but the credential documentation defines a valid line as /^([^=]+)=(.+)/.

It's also possible for the value to be empty, but I do not think off-hand that current git will ever send such an empty value.

> [...]

The rest of it looks fine to me. I don't think any of my comments are show-stoppers. Tests would be nice, but integrating contrib/ stuff with the test harness is kind of a pain.

-Peff
Ted Zlatanov· Feb 4, 2013, 16:40 UTC · re: Jeff King · lore

Re: [PATCH] git-send-email: add ~/.authinfo parsing

On Sun, 3 Feb 2013 14:41:49 -0500 Jeff King <peff@peff.net> wrote: 
JK> On Sat, Feb 02, 2013 at 06:57:29AM -0500, Ted Zlatanov wrote:
>> If the file name ends with ".gpg", it will run "gpg --decrypt FILE" and
>> use the output.  So non-interactively, that could hang if GPG was
>> waiting for input.  Does Git handle that, or should I check for a TTY?

JK> No, git does not do anything special with respect to credential helpers JK> and ttys (nor should it, since one use of helpers is to get credentials JK> when there is no tty). I think it is GPG's problem to deal with, though. JK> We will invoke it, and it is up to it to decide whether it can acquire JK> the passphrase or not (either through the tty, or possibly from JK> gpg-agent). So it would be wrong to do the tty check yourself.

JK> I haven't tested GPG, but I assume it properly tries to read from JK> /dev/tty and not stdin. Your helper's stdio is connected to git and JK> speaking the credential-helper protocol, so GPG reading from stdin would JK> either steal your input (if run before you read it), or just get EOF (if JK> you have read all of the pipe content already). If GPG isn't well JK> behaved, it may be worth redirecting its stdin from /dev/null as a JK> safety measure.

In my testing GPG did the right thing, so I think this is OK.
>> Take a look at the proposed patch and let me know if it's usable, if you
>> need a formal copyright assignment, etc.

JK> Overall looks sane to me, though my knowledge of .netrc is not JK> especially good. Usually we try to send patches inline in the email JK> (i.e., as generated by git-format-patch), and include a "Signed-off-by" JK> line indicating that content is released to the project; see JK> Documentation/SubmittingPatches.

OK, thanks.  I will fire that off.
>> +use Data::Dumper;
JK> I don't see it used here. Leftover from debugging?
It's part of my Perl new script skeleton, sorry.
>> + print <<EOHIPPUS;
JK> Cute, I haven't seen that one before.

Heh heh. I've had to explain that one in code review many times. "See, it's the precursor to the modern horse..."

>> +$0 [-f AUTHFILE] [-d] get
>> +
>> +Version $VERSION by tzz\@lifelogs.com.  License: any use is OK.

JK> I don't know if we have a particular policy for items in contrib/, but JK> this license may be too vague. In particular, it does not explicitly JK> allow redistribution, which would make Junio shipping a release with it JK> a copyright violation.

JK> Any objection to just putting it under some well-known simple license JK> (GPL, BSD, or whatever)?

No, I didn't know what Git requires, and I'd like it to be the least restrictive, so BSD is OK. Stated in -h now.

>> +if ($file =~ m/\.gpg$/)
>> +{
>> + $file = "gpg --decrypt $file|";
>> +}

JK> Does this need to quote $file, since the result will get passed to the JK> shell? It might be easier to just use the list form of open(), like:

JK> my @data = $file =~ /\.gpg$/ ? JK> load('-|', qw(gpg --decrypt), $file) : JK> load('<', $file);

JK> (and then obviously update load to just dump all of @_ to open()).
Yes, thanks.  Done.
>> +die "Sorry, we could not load data from [$file]"
>> + unless (scalar @data);

JK> Probably not that interesting a corner case, but this means we die on an JK> empty .netrc, whereas it might be more sensible for it to behave as "no JK> match".

JK> For the same reason, it might be worth silently exiting when we don't JK> find a .netrc (or any of its variants). That lets people who share their JK> dot-files across machines configure git globally, even if they don't JK> necessarily have a netrc on every machine.

OK; done.
Show 7 quoted lines
>> +# the query
>> +my %q;
>> +
>> +foreach my $v (values %{$options{tmap}})
>> +{
>> + undef $q{$v};
>> +}

JK> Just my personal style, but I find the intent more obvious with "map" (I JK> know some people find it unreadable, though):

JK>   my %q = map { $_ => undef } values(%{$options{tmap}});
Yes, changed.
>> +while (<STDIN>)
>> +{
>> + next unless m/([a-z]+)=(.+)/;

JK> We don't currently have any exotic tokens that this would not match, nor JK> do I plan to add them, but the credential documentation defines a valid JK> line as /^([^=]+)=(.+)/.

JK> It's also possible for the value to be empty, but I do not think JK> off-hand that current git will ever send such an empty value.

Yes, changed.

JK> The rest of it looks fine to me. I don't think any of my comments are JK> show-stoppers. Tests would be nice, but integrating contrib/ stuff with JK> the test harness is kind of a pain.

"I tested it on AIX, it works great!" :)

It's pretty easy to write a local Makefile with a test target, if you think it worthwhile.

Ted
Ted Zlatanov· Feb 4, 2013, 16:42 UTC · re: Jeff King · lore

[PATCH 1/3] Add contrib/credentials/netrc with GPG support

Signed-off-by: Ted Zlatanov <tzz@lifelogs.com>
---
 contrib/credential/netrc/git-credential-netrc |  242 +++++++++++++++++++++++++
 1 files changed, 242 insertions(+), 0 deletions(-)
 create mode 100755 contrib/credential/netrc/git-credential-netrc
Show changes to contrib/credential/netrc/git-credential-netrc +242 −0
diff --git a/contrib/credential/netrc/git-credential-netrc b/contrib/credential/netrc/git-credential-netrc
new file mode 100755
index 0000000..92fc306
--- /dev/null
+++ b/contrib/credential/netrc/git-credential-netrc
@@ -0,0 +1,242 @@
+#!/usr/bin/perl
+
+use strict;
+use warnings;
+
+use Data::Dumper;
+
+use Getopt::Long;
+use File::Basename;
+
+my $VERSION = "0.1";
+
+my %options = (
+               help => 0,
+               debug => 0,
+
+               # identical token maps, e.g. host -> host, will be inserted later
+               tmap => {
+                        port => 'protocol',
+                        machine => 'host',
+                        path => 'path',
+                        login => 'username',
+                        user => 'username',
+                        password => 'password',
+                       }
+              );
+
+foreach my $v (values %{$options{tmap}})
+{
+ $options{tmap}->{$v} = $v;
+}
+
+foreach my $suffix ('.gpg', '')
+{
+ foreach my $base (qw/authinfo netrc/)
+ {
+  my $file = glob("~/.$base$suffix");
+  next unless (defined $file && -f $file);
+  $options{file} = $file ;
+ }
+}
+
+Getopt::Long::Configure("bundling");
+
+# TODO: maybe allow the token map $options{tmap} to be configurable.
+GetOptions(\%options,
+           "help|h",
+           "debug|d",
+           "file|f=s",
+          );
+
+if ($options{help})
+{
+ my $shortname = basename($0);
+ $shortname =~ s/git-credential-//;
+
+ print <<EOHIPPUS;
+
+$0 [-f AUTHFILE] [-d] get
+
+Version $VERSION by tzz\@lifelogs.com.  License: any use is OK.
+
+Options:
+  -f AUTHFILE: specify a netrc-style file
+  -d: turn on debugging
+
+To enable (note that Git will prepend "git-credential-" to the helper
+name and look for it in the path):
+
+  git config credential.helper '$shortname -f AUTHFILE'
+
+And if you want lots of debugging info:
+
+  git config credential.helper '$shortname -f AUTHFILE -d'
+
+Only "get" mode is supported by this credential helper.  It opens
+AUTHFILE and looks for entries that match the requested search
+criteria:
+
+ 'port|protocol':
+   The protocol that will be used (e.g., https). (protocol=X)
+
+ 'machine|host':
+   The remote hostname for a network credential. (host=X)
+
+ 'path':
+   The path with which the credential will be used. (path=X)
+
+ 'login|user|username':
+   The credential’s username, if we already have one. (username=X)
+
+Thus, when we get "protocol=https\nusername=tzz", this credential
+helper will look for lines in AUTHFILE that match
+
+port https login tzz
+
+OR
+
+protocol https login tzz
+
+OR... etc. acceptable tokens as listed above.  Any unknown tokens are
+simply ignored.
+
+Then, the helper will print out whatever tokens it got from the line,
+including "password" tokens, mapping e.g. "port" back to "protocol".
+
+The first matching line is used.  Tokens can be quoted as 'STRING' or
+"STRING".
+
+No caching is performed by this credential helper.
+
+EOHIPPUS
+
+ exit;
+}
+
+my $mode = shift @ARGV;
+
+# credentials may get 'get', 'store', or 'erase' as parameters but
+# only acknowledge 'get'
+die "Syntax: $0 [-f AUTHFILE] [-d] get" unless defined $mode;
+
+# only support 'get' mode
+exit unless $mode eq 'get';
+
+my $debug = $options{debug};
+my $file = $options{file};
+
+die "Sorry, you need to specify an existing netrc file (with or without a .gpg extension) with -f AUTHFILE"
+ unless defined $file;
+
+die "Sorry, the specified netrc $file is not accessible"
+ unless -f $file;
+
+if ($file =~ m/\.gpg$/)
+{
+ $file = "gpg --decrypt $file|";
+}
+
+my @data = load($file);
+chomp @data;
+
+die "Sorry, we could not load data from [$file]"
+ unless (scalar @data);
+
+# the query
+my %q;
+
+foreach my $v (values %{$options{tmap}})
+{
+ undef $q{$v};
+}
+
+while (<STDIN>)
+{
+ next unless m/([a-z]+)=(.+)/;
+
+ my ($token, $value) = ($1, $2);
+ die "Unknown search token $1" unless exists $q{$token};
+ $q{$token} = $value;
+}
+
+# build reverse token map
+my %rmap;
+foreach my $k (keys %{$options{tmap}})
+{
+ push @{$rmap{$options{tmap}->{$k}}}, $k;
+}
+
+# there are CPAN modules to do this better, but we want to avoid
+# dependencies and generally, complex netrc-style files are rare
+
+if ($debug)
+{
+ foreach (sort keys %q)
+ {
+  printf STDERR "searching for %s = %s\n",
+   $_, $q{$_} || '(any value)';
+ }
+}
+
+LINE: foreach my $line (@data)
+{
+
+ print STDERR "line [$line]\n" if $debug;
+ my @tok;
+ # gratefully stolen from Net::Netrc
+ while (length $line &&
+        $line =~ s/^("((?:[^"]+|\\.)*)"|((?:[^\\\s]+|\\.)*))\s*//)
+ {
+  (my $tok = $+) =~ s/\\(.)/$1/g;
+  push(@tok, $tok);
+ }
+
+ my %tokens;
+ while (@tok)
+ {
+  my ($k, $v) = (shift @tok, shift @tok);
+  next unless defined $v;
+  next unless exists $options{tmap}->{$k};
+  $tokens{$options{tmap}->{$k}} = $v;
+ }
+
+ foreach my $check (sort keys %q)
+ {
+  if (exists $tokens{$check} && defined $q{$check})
+  {
+   print STDERR "comparing [$tokens{$check}] to [$q{$check}] in line [$line]\n" if $debug;
+   next LINE unless $tokens{$check} eq $q{$check};
+  }
+  else
+  {
+   print STDERR "we could not find [$check] but it's OK\n" if $debug;
+  }
+ }
+
+ print STDERR "line has passed all the search checks\n" if $debug;
+ foreach my $token (sort keys %rmap)
+ {
+  print STDERR "looking for useful token $token\n" if $debug;
+  next unless exists $tokens{$token}; # did we match?
+
+  foreach my $rctoken (@{$rmap{$token}})
+  {
+   next if defined $q{$rctoken};           # don't re-print given tokens
+  }
+
+  print STDERR "FOUND: $token=$tokens{$token}\n" if $debug;
+  printf "%s=%s\n", $token, $tokens{$token};
+ }
+
+ last;
+}
+
+sub load
+{
+ my $file = shift;
+ # this supports pipes too
+ my $io = new IO::File($file) or die "Could not open $file: $!\n";
+
+ return <$io>;                          # whole file
+}
-- 
1.7.9.rc2
Junio C Hamano· Feb 4, 2013, 17:24 UTC · re: Ted Zlatanov · lore

Re: [PATCH 1/3] Add contrib/credentials/netrc with GPG support

[administrivia: I would really wish you didn't put "Mail-copies-to: never" above].

Ted Zlatanov <tzz@lifelogs.com> writes:
> +foreach my $v (values %{$options{tmap}})
> +{
> + $options{tmap}->{$v} = $v;
> +}

Please follow the styles of existing Perl scripts, e.g. indent with tab, etc. Style requests are not optional; it is a prerequisite to make the patch readable and reviewable.

> + print <<EOHIPPUS;
> + ...
> +EOHIPPUS

Do we really need to refer readers to Wikipedia or something to learn about extinct equid ungulates ;-)?

Ted Zlatanov· Feb 4, 2013, 18:33 UTC · re: Junio C Hamano · lore

Re: [PATCH 1/3] Add contrib/credentials/netrc with GPG support

On Mon, 04 Feb 2013 09:24:03 -0800 Junio C Hamano <gitster@pobox.com> wrote: 

JCH> [administrivia: I would really wish you didn't put "Mail-copies-to: JCH> never" above].

I normally post through GMane and don't need the extra CC on any list I read. I'll make an effort to remove that header here, and apologize for the inconvenience.

JCH> Ted Zlatanov <tzz@lifelogs.com> writes:
>> +foreach my $v (values %{$options{tmap}})
>> +{
>> + $options{tmap}->{$v} = $v;
>> +}

JCH> Please follow the styles of existing Perl scripts, e.g. indent with JCH> tab, etc. Style requests are not optional; it is a prerequisite to JCH> make the patch readable and reviewable.

Sorry, I didn't realize contrib/ stuff was under the same rules. I will attempt to make my contributions fit the project's requirements.

It would help if the requirements were codified as the fairly standard Emacs file-local variables, so I can just put them in the Perl code or in .dir-locals.el in the source tree. At least for Perl I'd like that, and it could be nice for the Emacs users who write C too.

Would you like me to propose that as a patch?

Either way, I guessed that these settings are what you want as far as tabs and indentation (I use cperl-mode but perl-mode is the same):

# -*- mode: cperl; tab-width: 8; cperl-indent-level: 4; indent-tabs-mode: t; -*-
...plus hanging braces and avoiding one-line blocks.  I hope that's right.
>> + print <<EOHIPPUS;
>> + ...
>> +EOHIPPUS

JCH> Do we really need to refer readers to Wikipedia or something to JCH> learn about extinct equid ungulates ;-)?

I think the marker's name is irrelevant, and hope you are OK with leaving it.

Since the change is a pretty big reformatting, should I squash my 3 commits plus the reformatting commit into one patch, or keep them as a series?

I am appending the script in its current form so you can review it and tell me if there's anything else I should add or change in the formatting.

Thanks Ted

#!/usr/bin/perl # -*- mode: cperl; tab-width: 8; cperl-indent-level: 4; indent-tabs-mode: t; -*-

use strict; use warnings;

use Getopt::Long; use File::Basename;

my $VERSION = "0.1";
my %options = (
               help => 0,
               debug => 0,
               # identical token maps, e.g. host -> host, will be inserted later
               tmap => {
                        port => 'protocol',
                        machine => 'host',
                        path => 'path',
                        login => 'username',
                        user => 'username',
                        password => 'password',
                       }
              );

# map each credential protocol token to itself on the netrc side $options{tmap}->{$_} = $_ foreach my $v (values %{$options{tmap}});

foreach my $suffix ('.gpg', '') {
    foreach my $base (qw/authinfo netrc/) {
	my $file = glob("~/.$base$suffix");
	next unless (defined $file && -f $file);
	$options{file} = $file ;
    }
}
Getopt::Long::Configure("bundling");
# TODO: maybe allow the token map $options{tmap} to be configurable.
GetOptions(\%options,
           "help|h",
           "debug|d",
           "file|f=s",
          );
if ($options{help}) {
    my $shortname = basename($0);
    $shortname =~ s/git-credential-//;
    print <<EOHIPPUS;
$0 [-f AUTHFILE] [-d] get
Version $VERSION by tzz\@lifelogs.com.  License: BSD.
Options:
  -f AUTHFILE: specify a netrc-style file
  -d: turn on debugging

To enable (note that Git will prepend "git-credential-" to the helper name and look for it in the path):

  git config credential.helper '$shortname -f AUTHFILE'
And if you want lots of debugging info:
  git config credential.helper '$shortname -f AUTHFILE -d'

Only "get" mode is supported by this credential helper. It opens AUTHFILE and looks for entries that match the requested search criteria:

 'port|protocol':
   The protocol that will be used (e.g., https). (protocol=X)
 'machine|host':
   The remote hostname for a network credential. (host=X)
 'path':
   The path with which the credential will be used. (path=X)
 'login|user|username':
   The credential’s username, if we already have one. (username=X)

Thus, when we get "protocol=https\nusername=tzz", this credential helper will look for lines in AUTHFILE that match

port https login tzz
OR
protocol https login tzz

OR... etc. acceptable tokens as listed above. Any unknown tokens are simply ignored.

Then, the helper will print out whatever tokens it got from the line, including "password" tokens, mapping e.g. "port" back to "protocol".

The first matching line is used. Tokens can be quoted as 'STRING' or "STRING".

No caching is performed by this credential helper.
EOHIPPUS
    exit;
}
my $mode = shift @ARGV;

# credentials may get 'get', 'store', or 'erase' as parameters but # only acknowledge 'get' die "Syntax: $0 [-f AUTHFILE] [-d] get" unless defined $mode;

# only support 'get' mode exit unless $mode eq 'get';

my $debug = $options{debug}; my $file = $options{file};

die "Sorry, you need to specify an existing netrc file (with or without a .gpg extension) with -f AUTHFILE"
 unless defined $file;
unless (-f $file) {
    print STDERR "Sorry, the specified netrc $file is not accessible\n" if $debug;
    exit 0;
}
my @data;
if ($file =~ m/\.gpg$/) {
    @data = load('-|', qw(gpg --decrypt), $file)
}
else {
    @data = load('<', $file);
}
chomp @data;
unless (scalar @data) {
    print STDERR "Sorry, we could not load data from [$file]\n" if $debug;
    exit;
}

# the query: start with every token with no value my %q = map { $_ => undef } values(%{$options{tmap}});

while (<STDIN>) {
    next unless m/([^=]+)=(.+)/;
    my ($token, $value) = ($1, $2);
    die "Unknown search token $1" unless exists $q{$token};
    $q{$token} = $value;
}
# build reverse token map
my %rmap;
foreach my $k (keys %{$options{tmap}}) {
    push @{$rmap{$options{tmap}->{$k}}}, $k;
}

# there are CPAN modules to do this better, but we want to avoid # dependencies and generally, complex netrc-style files are rare

if ($debug) {
    printf STDERR "searching for %s = %s\n", $_, $q{$_} || '(any value)'
     foreach sort keys %q;
}
LINE: foreach my $line (@data) {
    print STDERR "line [$line]\n" if $debug;
    my @tok;
    # gratefully stolen from Net::Netrc
    while (length $line &&
	   $line =~ s/^("((?:[^"]+|\\.)*)"|((?:[^\\\s]+|\\.)*))\s*//) {
	(my $tok = $+) =~ s/\\(.)/$1/g;
	push(@tok, $tok);
    }
    # skip blank lines, comments, etc.
    next LINE unless scalar @tok;
    my %tokens;
    while (@tok) {
	my ($k, $v) = (shift @tok, shift @tok);
	next unless defined $v;
	next unless exists $options{tmap}->{$k};
	$tokens{$options{tmap}->{$k}} = $v;
    }
    foreach my $check (sort keys %q) {
	if (exists $tokens{$check} && defined $q{$check}) {
	    print STDERR "comparing [$tokens{$check}] to [$q{$check}] in line [$line]\n" if $debug;
	    next LINE unless $tokens{$check} eq $q{$check};
	}
	else {
	    print STDERR "we could not find [$check] but it's OK\n" if $debug;
	}
    }
    print STDERR "line has passed all the search checks\n" if $debug;
 TOKEN:
    foreach my $token (sort keys %rmap) {
	print STDERR "looking for useful token $token\n" if $debug;
	next unless exists $tokens{$token}; # did we match?
	foreach my $rctoken (@{$rmap{$token}}) {
	    next TOKEN if defined $q{$rctoken};           # don't re-print given tokens
	}
	print STDERR "FOUND: $token=$tokens{$token}\n" if $debug;
	printf "%s=%s\n", $token, $tokens{$token};
    }
    last;
}
sub load {
    # this supports pipes too
    my $io = new IO::File(@_) or die "Could not open [@_]: $!\n";
    return <$io>;                          # whole file
}
Junio C Hamano· Feb 4, 2013, 19:06 UTC · re: Ted Zlatanov · lore

Re: [PATCH 1/3] Add contrib/credentials/netrc with GPG support

Ted Zlatanov <tzz@lifelogs.com> writes:
> Sorry, I didn't realize contrib/ stuff was under the same rules.

I had a feeling that this may start out from contrib/ but will soon prove to be fairly important to be part of the Git proper.

Show 6 quoted lines
> It would help if the requirements were codified as the fairly standard
> Emacs file-local variables, so I can just put them in the Perl code or
> in .dir-locals.el in the source tree.  At least for Perl I'd like that,
> and it could be nice for the Emacs users who write C too.
>
> Would you like me to propose that as a patch?

I thought that we tend to avoid Emacs/Vim formatting cruft left in the file. Do we have any in existing file outside contrib/?

> Either way, I guessed that these settings are what you want as far as
> tabs and indentation (I use cperl-mode but perl-mode is the same):
>
> # -*- mode: cperl; tab-width: 8; cperl-indent-level: 4; indent-tabs-mode: t; -*-

Indent is done with a Tab and indent level is 8 places (check add--interactive.perl and imitate it, perhaps?).

Thanks.
Ted Zlatanov· Feb 4, 2013, 19:40 UTC · re: Junio C Hamano · lore

Re: [PATCH 1/3] Add contrib/credentials/netrc with GPG support

On Mon, 04 Feb 2013 11:06:16 -0800 Junio C Hamano <gitster@pobox.com> wrote: 
JCH> Ted Zlatanov <tzz@lifelogs.com> writes:
>> Sorry, I didn't realize contrib/ stuff was under the same rules.

JCH> I had a feeling that this may start out from contrib/ but will soon JCH> prove to be fairly important to be part of the Git proper.

Cool!
Show 6 quoted lines
>> It would help if the requirements were codified as the fairly standard
>> Emacs file-local variables, so I can just put them in the Perl code or
>> in .dir-locals.el in the source tree.  At least for Perl I'd like that,
>> and it could be nice for the Emacs users who write C too.
>> 
>> Would you like me to propose that as a patch?

JCH> I thought that we tend to avoid Emacs/Vim formatting cruft left in JCH> the file. Do we have any in existing file outside contrib/?

No, but it's a nice way to express the settings so no one is guessing what the project prefers. At least for me it's not an issue anymore, since I understand your criteria better now, so let me know if you want me to express it in the CodingGuidelines, in a dir-locals.el file, or somewhere else.

>> Either way, I guessed that these settings are what you want as far as
>> tabs and indentation (I use cperl-mode but perl-mode is the same):
>> 
>> # -*- mode: cperl; tab-width: 8; cperl-indent-level: 4; indent-tabs-mode: t; -*-

JCH> Indent is done with a Tab and indent level is 8 places (check add--interactive.perl JCH> and imitate it, perhaps?).

Yup, got it.  My mistake on the size-4 indents.

I found this helpful, at least while I was indenting, for anyone else who might want to indent Perl appropriately to imitate existing Perl code:

# -*- mode: cperl; tab-width: 8; cperl-indent-level: 8; indent-tabs-mode: t; -*-
I'll resubmit now.

Thanks Ted

Junio C Hamano· Feb 4, 2013, 23:10 UTC · re: Ted Zlatanov · lore

Re: [PATCH 1/3] Add contrib/credentials/netrc with GPG support

Ted Zlatanov <tzz@lifelogs.com> writes:
Show 8 quoted lines
> JCH> I thought that we tend to avoid Emacs/Vim formatting cruft left in
> JCH> the file.  Do we have any in existing file outside contrib/?
>
> No, but it's a nice way to express the settings so no one is guessing
> what the project prefers.  At least for me it's not an issue anymore,
> since I understand your criteria better now, so let me know if you want
> me to express it in the CodingGuidelines, in a dir-locals.el file, or
> somewhere else.

Historically we treated this from CodingGuidelines a sufficient clue:

    As for more concrete guidelines, just imitate the existing code
    (this is a good guideline, no matter which project you are
    contributing to). It is always preferable to match the _local_
    convention. New code added to git suite is expected to match
    the overall style of existing code. Modifications to existing
    code is expected to match the style the surrounding code already
    uses (even if it doesn't match the overall style of existing code).

but over time people wanted more specific guidelines and added language specific style guides there. We have sections that cover C, shell and Python, and I do not think adding Perl would not hurt.

Ted Zlatanov· Feb 6, 2013, 15:10 UTC · re: Junio C Hamano · lore

CodingGuidelines Perl amendment (was: [PATCH 1/3] Add contrib/credentials/netrc with GPG support)

On Mon, 04 Feb 2013 15:10:45 -0800 Junio C Hamano <gitster@pobox.com> wrote: 

JCH> Ted Zlatanov <tzz@lifelogs.com> writes: JCH> I thought that we tend to avoid Emacs/Vim formatting cruft left in JCH> the file. Do we have any in existing file outside contrib/?

Show 6 quoted lines
>> 
>> No, but it's a nice way to express the settings so no one is guessing
>> what the project prefers.  At least for me it's not an issue anymore,
>> since I understand your criteria better now, so let me know if you want
>> me to express it in the CodingGuidelines, in a dir-locals.el file, or
>> somewhere else.

JCH> Historically we treated this from CodingGuidelines a sufficient JCH> clue:

JCH> As for more concrete guidelines, just imitate the existing code JCH> (this is a good guideline, no matter which project you are JCH> contributing to). It is always preferable to match the _local_ JCH> convention. New code added to git suite is expected to match JCH> the overall style of existing code. Modifications to existing JCH> code is expected to match the style the surrounding code already JCH> uses (even if it doesn't match the overall style of existing code).

JCH> but over time people wanted more specific guidelines and added JCH> language specific style guides there. We have sections that cover JCH> C, shell and Python, and I do not think adding Perl would not hurt.

The following is how I have interpreted the Perl guidelines. I hope it's OK to include Emacs-specific settings; they make it much easier to reindent code to be acceptable.

I will submit as a patch if you think this is reasonable at all.
The org-mode markers around the code are just a suggestion.
For Perl 5 programs:
 - Most of the C guidelines above apply.
 - We try to support Perl 5.8 and later ("use Perl 5.008").
 - use strict and use warnings are strongly preferred.
 - As in C (see above), we avoid using braces unnecessarily (but Perl
   forces braces around if/unless/else/foreach blocks, so this is not
   always possible).
 - Don't abuse statement modifiers (unless $youmust).
 - We try to avoid assignments inside if().
 - Learn and use Git.pm if you need that functionality.
 - For Emacs, it's useful to put the following in
   GIT_CHECKOUT/.dir-locals.el, assuming you use cperl-mode:
#+begin_src lisp
((nil . ((indent-tabs-mode . t)
              (tab-width . 8)
              (fill-column . 80)))
 (cperl-mode . ((cperl-indent-level . 8)
                (cperl-extra-newline-before-brace . nil)
                (cperl-merge-trailing-else . t))))
#+end_src
Junio C Hamano· Feb 6, 2013, 16:29 UTC · re: Ted Zlatanov · lore

Re: CodingGuidelines Perl amendment

Ted Zlatanov <tzz@lifelogs.com> writes:
>  - As in C (see above), we avoid using braces unnecessarily (but Perl
>    forces braces around if/unless/else/foreach blocks, so this is not
>    always possible).
Is it ever (as opposed to "not always") possible to omit braces?

It sounds as if we encourage the use of statement modifiers, which certainly is not what I want to see.

You probably would want to mention that opening braces for "if/else/elsif" do not sit on their own line, and closing braces for them will be followed the next "else/elseif" on the same line instead, but that is part of "most of the C guidelines above apply" so it may be redundant.

>  - Don't abuse statement modifiers (unless $youmust).

It does not make a useful guidance to leave $youmust part unspecified.

Incidentally, your sentence is a good example of where use of statement modifiers is appropriate: $youmust is rarely true.

In general:
	... do something ...
	do_this() unless (condition);
        ... do something else ...
is easier to follow the flow of the logic than
	... do something ...
	unless (condition) {
		do_this();
	}
        ... do something else ...

*only* when condition is extremely rare, iow, when do_this() is expected to be almost always called.

demerphq· Feb 6, 2013, 17:45 UTC · re: Junio C Hamano · lore

Re: CodingGuidelines Perl amendment

On 6 February 2013 17:29, Junio C Hamano <gitster@pobox.com> wrote:
Show 7 quoted lines
> Ted Zlatanov <tzz@lifelogs.com> writes:
>
>>  - As in C (see above), we avoid using braces unnecessarily (but Perl
>>    forces braces around if/unless/else/foreach blocks, so this is not
>>    always possible).
>
> Is it ever (as opposed to "not always") possible to omit braces?
Only in a statement modifier.
> It sounds as if we encourage the use of statement modifiers, which
> certainly is not what I want to see.
As you mention below statement modifiers have their place. For instance
  next if $whatever;
Is considered preferable to
if ($whatever) {
  next;
}
Similarly
open my $fh, ">", $filename
   or die "Failed to open '$filename': $!";
Is considered preferable by most Perl programmers to:
my $fh;
if ( not open $fh, ">", $filename ) {
  die "Failed to open '$filename': $!";
}
Show 14 quoted lines
> You probably would want to mention that opening braces for
> "if/else/elsif" do not sit on their own line,
> and closing braces for
> them will be followed the next "else/elseif" on the same line
> instead, but that is part of "most of the C guidelines above apply"
> so it may be redundant.
>
>>  - Don't abuse statement modifiers (unless $youmust).
>
> It does not make a useful guidance to leave $youmust part
> unspecified.
>
> Incidentally, your sentence is a good example of where use of
> statement modifiers is appropriate: $youmust is rarely true.

"unless" often leads to maintenance errors as the expression gets more complicated over time, more branches need to be added to the statement, etc. Basically people are bad at doing De Morgans law in their head.

Show 16 quoted lines
> In general:
>
>         ... do something ...
>         do_this() unless (condition);
>         ... do something else ...
>
> is easier to follow the flow of the logic than
>
>         ... do something ...
>         unless (condition) {
>                 do_this();
>         }
>         ... do something else ...
>
> *only* when condition is extremely rare, iow, when do_this() is
> expected to be almost always called.
if (not $condition) {
  do_this();
}
Is much less error prone in terms of maintenance than
unless ($condition) {
  do_this();
}
Similarly
do_this() if not $condition;
leads to less maintenance errors than
do_this() unless $condition;
So if you objective is maintainability I would just ban "unless" outright.

Cheers, Yves

-- 
perl -Mre=debug -e "/just|another|perl|hacker/"
Ted Zlatanov· Feb 6, 2013, 18:08 UTC · re: demerphq · lore

Re: CodingGuidelines Perl amendment

On Wed, 6 Feb 2013 18:45:56 +0100 demerphq <demerphq@gmail.com> wrote: 
d> So if you objective is maintainability I would just ban "unless" outright.
Please consider me opposed to such a ban.
Ted
Junio C Hamano· Feb 6, 2013, 18:14 UTC · re: demerphq · lore

Re: CodingGuidelines Perl amendment

demerphq <demerphq@gmail.com> writes:
Show 21 quoted lines
> As you mention below statement modifiers have their place. For instance
>
>   next if $whatever;
>
> Is considered preferable to
>
> if ($whatever) {
>   next;
> }
>
> Similarly
>
> open my $fh, ">", $filename
>    or die "Failed to open '$filename': $!";
>
> Is considered preferable by most Perl programmers to:
>
> my $fh;
> if ( not open $fh, ">", $filename ) {
>   die "Failed to open '$filename': $!";
> }

Yeah, and that is for the same reason. When you are trying to get a birds-eye view of the codeflow, the former makes it clear that "we do something, and then we open, and then we ...", without letting the error handling (which also is rare case) distract us.

> "unless" often leads to maintenance errors as the expression gets more
> complicated over time,...

That might also be true, but my comment was not an endorsement for (or suggestion against) use of unless. I was commenting on statement modifiers, which some people tend to overuse (or abuse) and make the resulting code harder to follow.

demerphq· Feb 6, 2013, 18:18 UTC · re: Junio C Hamano · lore

Re: CodingGuidelines Perl amendment

On 6 February 2013 19:14, Junio C Hamano <gitster@pobox.com> wrote:
Show 28 quoted lines
> demerphq <demerphq@gmail.com> writes:
>
>> As you mention below statement modifiers have their place. For instance
>>
>>   next if $whatever;
>>
>> Is considered preferable to
>>
>> if ($whatever) {
>>   next;
>> }
>>
>> Similarly
>>
>> open my $fh, ">", $filename
>>    or die "Failed to open '$filename': $!";
>>
>> Is considered preferable by most Perl programmers to:
>>
>> my $fh;
>> if ( not open $fh, ">", $filename ) {
>>   die "Failed to open '$filename': $!";
>> }
>
> Yeah, and that is for the same reason.  When you are trying to get a
> birds-eye view of the codeflow, the former makes it clear that "we
> do something, and then we open, and then we ...", without letting
> the error handling (which also is rare case) distract us.

perldoc perlstyle has language which explains this well if you want to crib a description from somewhere.

Show 7 quoted lines
>> "unless" often leads to maintenance errors as the expression gets more
>> complicated over time,...
>
> That might also be true, but my comment was not an endorsement for
> (or suggestion against) use of unless.  I was commenting on
> statement modifiers, which some people tend to overuse (or abuse)
> and make the resulting code harder to follow.

That's also my point about unless. They tend to get abused and then lead to maint devs making errors, and people misunderstanding the code. The only time that unless IMO is "ok" (ish) is when it really is a very simple statement. As soon as it mentions more than one var it should be converted to an if. This applies even more so to the modifier form.

Yves
-- 
perl -Mre=debug -e "/just|another|perl|hacker/"
Ted Zlatanov· Feb 6, 2013, 17:55 UTC · re: Junio C Hamano · lore

[PATCH] Update CodingGuidelines for Perl 5

Update the coding guidelines for Perl 5.
Signed-off-by: Ted Zlatanov <tzz@lifelogs.com>
---
 Documentation/CodingGuidelines |   44 ++++++++++++++++++++++++++++++++++++++++
 1 files changed, 44 insertions(+), 0 deletions(-)
Show changes to Documentation/CodingGuidelines +44 −0
diff --git a/Documentation/CodingGuidelines b/Documentation/CodingGuidelines
index 1d7de5f..951d74c 100644
--- a/Documentation/CodingGuidelines
+++ b/Documentation/CodingGuidelines
@@ -179,6 +179,50 @@ For C programs:
  - Use Git's gettext wrappers to make the user interface
    translatable. See "Marking strings for translation" in po/README.
 
+For Perl 5 programs:
+
+ - Most of the C guidelines above apply.
+
+ - We try to support Perl 5.8 and later ("use Perl 5.008").
+
+ - use strict and use warnings are strongly preferred.
+
+ - As in C (see above), we avoid using braces unnecessarily (but Perl forces
+   braces around if/unless/else/foreach blocks, so this is not always possible).
+   At least make sure braces do not sit on their own line, like with C.
+
+ - Don't abuse statement modifiers--they are discouraged.  But in general:
+
+	... do something ...
+	do_this() unless (condition);
+        ... do something else ...
+
+   should be used instead of
+
+	... do something ...
+	unless (condition) {
+		do_this();
+	}
+        ... do something else ...
+
+   *only* when when the condition is so rare that do_this() will be called
+   almost always.
+
+ - We try to avoid assignments inside if().
+
+ - Learn and use Git.pm if you need that functionality.
+
+ - For Emacs, it's useful to put the following in
+   GIT_CHECKOUT/.dir-locals.el, assuming you use cperl-mode:
+
+    ;; note the first part is useful for C editing, too
+    ((nil . ((indent-tabs-mode . t)
+                  (tab-width . 8)
+                  (fill-column . 80)))
+     (cperl-mode . ((cperl-indent-level . 8)
+                    (cperl-extra-newline-before-brace . nil)
+                    (cperl-merge-trailing-else . t))))
+
 Writing Documentation:
 
  Every user-visible change should be reflected in the documentation.
-- 
1.7.9.rc2
Ted Zlatanov· Feb 6, 2013, 18:05 UTC · re: Junio C Hamano · lore

Re: CodingGuidelines Perl amendment

On Wed, 06 Feb 2013 08:29:30 -0800 Junio C Hamano <gitster@pobox.com> wrote: 
JCH> Is it ever (as opposed to "not always") possible to omit braces?

Oh yes! Not that I recommend it, and I'm not even going to touch on Perl Golf :)

JCH> It sounds as if we encourage the use of statement modifiers, which JCH> certainly is not what I want to see.

Yup. I think I captured that in the patch, but please feel free to revise it after applying or throw it back to me.

JCH> You probably would want to mention that opening braces for JCH> "if/else/elsif" do not sit on their own line, and closing braces for JCH> them will be followed the next "else/elseif" on the same line JCH> instead, but that is part of "most of the C guidelines above apply" JCH> so it may be redundant.

OK; done.
>> - Don't abuse statement modifiers (unless $youmust).

JCH> It does not make a useful guidance to leave $youmust part JCH> unspecified.

JCH> Incidentally, your sentence is a good example of where use of JCH> statement modifiers is appropriate: $youmust is rarely true.

I was trying to be funny, honestly.  But OK; reworded.
Ted
Junio C Hamano· Feb 6, 2013, 18:16 UTC · re: Ted Zlatanov · lore

Re: CodingGuidelines Perl amendment

Ted Zlatanov <tzz@lifelogs.com> writes:
Show 12 quoted lines
> On Wed, 06 Feb 2013 08:29:30 -0800 Junio C Hamano <gitster@pobox.com> wrote: 
>
> JCH> Is it ever (as opposed to "not always") possible to omit braces?
>
> Oh yes!  Not that I recommend it, and I'm not even going to touch on
> Perl Golf :)
>
> JCH> It sounds as if we encourage the use of statement modifiers, which
> JCH> certainly is not what I want to see.
>
> Yup.  I think I captured that in the patch, but please feel free to
> revise it after applying or throw it back to me.
I'd suggest to just drop that "try to write without braces" entirely.
> JCH> Incidentally, your sentence is a good example of where use of
> JCH> statement modifiers is appropriate: $youmust is rarely true.
>
> I was trying to be funny, honestly.  But OK; reworded.
It wasn't a useful guidance, but it _was_ funny.  
Ted Zlatanov· Feb 6, 2013, 18:27 UTC · re: Junio C Hamano · lore

Re: CodingGuidelines Perl amendment

On Wed, 06 Feb 2013 10:16:21 -0800 Junio C Hamano <gitster@pobox.com> wrote: 
JCH> I'd suggest to just drop that "try to write without braces" entirely.
OK, I'll do it on the reroll, or you can just make the change directly.
I agree it was not going anywhere :)
Ted
Show changes to Documentation/CodingGuidelines +0 −4
diff --git a/Documentation/CodingGuidelines b/Documentation/CodingGuidelines
index 951d74c..857f4e2 100644
--- a/Documentation/CodingGuidelines
+++ b/Documentation/CodingGuidelines
@@ -187,10 +187,6 @@ For Perl 5 programs:
 
  - use strict and use warnings are strongly preferred.
 
- - As in C (see above), we avoid using braces unnecessarily (but Perl forces
-   braces around if/unless/else/foreach blocks, so this is not always possible).
-   At least make sure braces do not sit on their own line, like with C.
-
  - Don't abuse statement modifiers--they are discouraged.  But in general:
 
        ... do something ...
demerphq· Feb 6, 2013, 18:25 UTC · re: Ted Zlatanov · lore

Re: CodingGuidelines Perl amendment

On 6 February 2013 19:05, Ted Zlatanov <tzz@lifelogs.com> wrote:
Show 6 quoted lines
> On Wed, 06 Feb 2013 08:29:30 -0800 Junio C Hamano <gitster@pobox.com> wrote:
>
> JCH> Is it ever (as opposed to "not always") possible to omit braces?
>
> Oh yes!  Not that I recommend it, and I'm not even going to touch on
> Perl Golf :)
I think you are wrong. Can you provide an example?

Larry specifically wanted to avoid the "dangling else" problem that C suffers from, and made it so that blocks are mandatory. The only exception is statement modifiers, which are not only allowed to omit the braces but also the parens on the condition.

Yves
-- 
perl -Mre=debug -e "/just|another|perl|hacker/"
Ted Zlatanov· Feb 6, 2013, 18:35 UTC · re: demerphq · lore

Re: CodingGuidelines Perl amendment

On Wed, 6 Feb 2013 19:25:43 +0100 demerphq <demerphq@gmail.com> wrote: 
d> On 6 February 2013 19:05, Ted Zlatanov <tzz@lifelogs.com> wrote:
>> On Wed, 06 Feb 2013 08:29:30 -0800 Junio C Hamano <gitster@pobox.com> wrote:
>> 
JCH> Is it ever (as opposed to "not always") possible to omit braces?
>> 
>> Oh yes!  Not that I recommend it, and I'm not even going to touch on
>> Perl Golf :)
d> I think you are wrong. Can you provide an example?

d> Larry specifically wanted to avoid the "dangling else" problem that C d> suffers from, and made it so that blocks are mandatory. The only d> exception is statement modifiers, which are not only allowed to omit d> the braces but also the parens on the condition.

Oh, perhaps I didn't state it correctly. You can avoid braces, but not if you want to use if/elsif/else/unless/etc. which require them:

condition && do_this(); condition || do_this(); condition ? do_this() : do_that();

(and others I can't recall right now)

But my point was only that it's always possible to get around these artificial restrictions; it's more important to ask for legible sensible code. Sorry if that was unclear!

Ted
demerphq· Feb 6, 2013, 18:44 UTC · re: Ted Zlatanov · lore

Re: CodingGuidelines Perl amendment

On 6 February 2013 19:35, Ted Zlatanov <tzz@lifelogs.com> wrote:
Show 29 quoted lines
> On Wed, 6 Feb 2013 19:25:43 +0100 demerphq <demerphq@gmail.com> wrote:
>
> d> On 6 February 2013 19:05, Ted Zlatanov <tzz@lifelogs.com> wrote:
>>> On Wed, 06 Feb 2013 08:29:30 -0800 Junio C Hamano <gitster@pobox.com> wrote:
>>>
> JCH> Is it ever (as opposed to "not always") possible to omit braces?
>>>
>>> Oh yes!  Not that I recommend it, and I'm not even going to touch on
>>> Perl Golf :)
>
> d> I think you are wrong. Can you provide an example?
>
> d> Larry specifically wanted to avoid the "dangling else" problem that C
> d> suffers from, and made it so that blocks are mandatory. The only
> d> exception is statement modifiers, which are not only allowed to omit
> d> the braces but also the parens on the condition.
>
> Oh, perhaps I didn't state it correctly.  You can avoid braces, but not
> if you want to use if/elsif/else/unless/etc. which require them:
>
> condition && do_this();
> condition || do_this();
> condition ? do_this() : do_that();
>
> (and others I can't recall right now)
>
> But my point was only that it's always possible to get around these
> artificial restrictions; it's more important to ask for legible sensible
> code.  Sorry if that was unclear!
Ah ok. Right, at a low level:
if (condition) { do_this() }
is identical to
condition && do_this();
IOW, Perl allows logical operators to act as control flow statements.

I hope your document include something that says that using logical operators as control flow statements should be used sparingly, and generally should be restricted to low precedence operators and should never involve more than one operator.

Yves
-- 
perl -Mre=debug -e "/just|another|perl|hacker/"
Ted Zlatanov· Feb 6, 2013, 18:54 UTC · re: demerphq · lore

Re: CodingGuidelines Perl amendment

On Wed, 6 Feb 2013 19:44:16 +0100 demerphq <demerphq@gmail.com> wrote: 
d> Ah ok. Right, at a low level:
d> if (condition) { do_this() }
d> is identical to
d> condition && do_this();
d> IOW, Perl allows logical operators to act as control flow statements.

d> I hope your document include something that says that using logical d> operators as control flow statements should be used sparingly, and d> generally should be restricted to low precedence operators and should d> never involve more than one operator.

I'd stay away from wording it so tightly, but instead just say
"Make your code readable and sensible, and don't try to be clever."

But this is good C and shell advice too, so I'd put it under "General Guidelines" and leave it for Junio to decide if it's appropriate.

Ted
Junio C Hamano· Feb 6, 2013, 19:37 UTC · re: Ted Zlatanov · lore

Re: CodingGuidelines Perl amendment

Ted Zlatanov <tzz@lifelogs.com> writes:
> "Make your code readable and sensible, and don't try to be clever."
>
> But this is good C and shell advice too,...
Sounds sensible.
Ted Zlatanov· Feb 6, 2013, 19:49 UTC · re: Junio C Hamano · lore

[PATCH v2] Update CodingGuidelines for Perl 5

Update the coding guidelines for Perl 5.
Signed-off-by: Ted Zlatanov <tzz@lifelogs.com>
---
Changes since PATCHv1:
- removed brace guidelines
- add "don't try to be clever" at beginning
 Documentation/CodingGuidelines |   42 ++++++++++++++++++++++++++++++++++++++++
 1 files changed, 42 insertions(+), 0 deletions(-)
Show changes to Documentation/CodingGuidelines +42 −0
diff --git a/Documentation/CodingGuidelines b/Documentation/CodingGuidelines
index 1d7de5f..166c141 100644
--- a/Documentation/CodingGuidelines
+++ b/Documentation/CodingGuidelines
@@ -18,6 +18,8 @@ code.  For Git in general, three rough rules are:
    judgement call, the decision based more on real world
    constraints people face than what the paper standard says.
 
+For any programming language below, make your code readable and sensible, and
+don't try to be clever.
 
 As for more concrete guidelines, just imitate the existing code
 (this is a good guideline, no matter which project you are
@@ -179,6 +181,46 @@ For C programs:
  - Use Git's gettext wrappers to make the user interface
    translatable. See "Marking strings for translation" in po/README.
 
+For Perl 5 programs:
+
+ - Most of the C guidelines above apply.
+
+ - We try to support Perl 5.8 and later ("use Perl 5.008").
+
+ - use strict and use warnings are strongly preferred.
+
+ - Don't abuse statement modifiers--they are discouraged.  But in general:
+
+	... do something ...
+	do_this() unless (condition);
+        ... do something else ...
+
+   should be used instead of
+
+	... do something ...
+	unless (condition) {
+		do_this();
+	}
+        ... do something else ...
+
+   *only* when when the condition is so rare that do_this() will be called
+   almost always.
+
+ - We try to avoid assignments inside if().
+
+ - Learn and use Git.pm if you need that functionality.
+
+ - For Emacs, it's useful to put the following in
+   GIT_CHECKOUT/.dir-locals.el, assuming you use cperl-mode:
+
+    ;; note the first part is useful for C editing, too
+    ((nil . ((indent-tabs-mode . t)
+                  (tab-width . 8)
+                  (fill-column . 80)))
+     (cperl-mode . ((cperl-indent-level . 8)
+                    (cperl-extra-newline-before-brace . nil)
+                    (cperl-merge-trailing-else . t))))
+
 Writing Documentation:
 
  Every user-visible change should be reflected in the documentation.
-- 
1.7.9.rc2
Ted Zlatanov· Feb 4, 2013, 16:42 UTC · re: Jeff King · lore

[PATCH 2/3] Skip blank and commented lines in contrib/credentials/netrc

Signed-off-by: Ted Zlatanov <tzz@lifelogs.com>
---
 contrib/credential/netrc/git-credential-netrc |    3 +++
 1 files changed, 3 insertions(+), 0 deletions(-)
Show changes to contrib/credential/netrc/git-credential-netrc +3 −0
diff --git a/contrib/credential/netrc/git-credential-netrc b/contrib/credential/netrc/git-credential-netrc
index 92fc306..a47a223 100755
--- a/contrib/credential/netrc/git-credential-netrc
+++ b/contrib/credential/netrc/git-credential-netrc
@@ -192,6 +192,9 @@ LINE: foreach my $line (@data)
   push(@tok, $tok);
  }
 
+ # skip blank lines, comments, etc.
+ next LINE unless scalar @tok;
+
  my %tokens;
  while (@tok)
  {
-- 
1.7.9.rc2
Ted Zlatanov· Feb 4, 2013, 16:43 UTC · re: Jeff King · lore

[PATCH 3/3] Fix contrib/credentials/netrc minor issues: exit quietly; use 3-parameter open; etc.

Signed-off-by: Ted Zlatanov <tzz@lifelogs.com>
---
 contrib/credential/netrc/git-credential-netrc |   38 +++++++++++++------------
 1 files changed, 20 insertions(+), 18 deletions(-)
Show changes to contrib/credential/netrc/git-credential-netrc +20 −18
diff --git a/contrib/credential/netrc/git-credential-netrc b/contrib/credential/netrc/git-credential-netrc
index a47a223..0e35918 100755
--- a/contrib/credential/netrc/git-credential-netrc
+++ b/contrib/credential/netrc/git-credential-netrc
@@ -3,8 +3,6 @@
 use strict;
 use warnings;
 
-use Data::Dumper;
-
 use Getopt::Long;
 use File::Basename;
 
@@ -58,7 +56,7 @@ if ($options{help})
 
 $0 [-f AUTHFILE] [-d] get
 
-Version $VERSION by tzz\@lifelogs.com.  License: any use is OK.
+Version $VERSION by tzz\@lifelogs.com.  License: BSD.
 
 Options:
   -f AUTHFILE: specify a netrc-style file
@@ -129,31 +127,36 @@ my $file = $options{file};
 die "Sorry, you need to specify an existing netrc file (with or without a .gpg extension) with -f AUTHFILE"
  unless defined $file;
 
-die "Sorry, the specified netrc $file is not accessible"
- unless -f $file;
+unless (-f $file)
+{
+ print STDERR "Sorry, the specified netrc $file is not accessible\n" if $debug;
+ exit 0;
+}
 
+my @data;
 if ($file =~ m/\.gpg$/)
 {
- $file = "gpg --decrypt $file|";
+ @data = load('-|', qw(gpg --decrypt), $file)
+}
+else
+{
+ @data = load('<', $file);
 }
 
-my @data = load($file);
 chomp @data;
 
-die "Sorry, we could not load data from [$file]"
- unless (scalar @data);
-
-# the query
-my %q;
-
-foreach my $v (values %{$options{tmap}})
+unless (scalar @data)
 {
- undef $q{$v};
+ print STDERR "Sorry, we could not load data from [$file]\n" if $debug;
+ exit;
 }
 
+# the query: start with every token with no value
+my %q = map { $_ => undef } values(%{$options{tmap}});
+
 while (<STDIN>)
 {
- next unless m/([a-z]+)=(.+)/;
+ next unless m/([^=]+)=(.+)/;
 
  my ($token, $value) = ($1, $2);
  die "Unknown search token $1" unless exists $q{$token};
@@ -237,9 +240,8 @@ LINE: foreach my $line (@data)
 
 sub load
 {
- my $file = shift;
  # this supports pipes too
- my $io = new IO::File($file) or die "Could not open $file: $!\n";
+ my $io = new IO::File(@_) or die "Could not open [@_]: $!\n";
 
  return <$io>;                          # whole file
 }
-- 
1.7.9.rc2
Junio C Hamano· Feb 4, 2013, 17:27 UTC · re: Ted Zlatanov · lore

Re: [PATCH 3/3] Fix contrib/credentials/netrc minor issues: exit quietly; use 3-parameter open; etc.

Ted Zlatanov <tzz@lifelogs.com> writes:
> Signed-off-by: Ted Zlatanov <tzz@lifelogs.com>
> ---
>  contrib/credential/netrc/git-credential-netrc |   38 +++++++++++++------------
>  1 files changed, 20 insertions(+), 18 deletions(-)

Especially because this is an initial submission, please equash three patches into one, instead of sending three "here is my first attempt with many problems I know I do not want to be there", "one small improvement", "another one to fix remaining issues".

Otherwise you will waste reviewers' time, getting distracted by undesirable details they find in an earlier patch while reviewing, without realizing that some of them are fixed in a later one.

Thanks.
Ted Zlatanov· Feb 4, 2013, 18:41 UTC · re: Junio C Hamano · lore

Re: [PATCH 3/3] Fix contrib/credentials/netrc minor issues: exit quietly; use 3-parameter open; etc.

On Mon, 04 Feb 2013 09:27:46 -0800 Junio C Hamano <gitster@pobox.com> wrote: 
JCH> Ted Zlatanov <tzz@lifelogs.com> writes:
>> Signed-off-by: Ted Zlatanov <tzz@lifelogs.com>
>> ---
>> contrib/credential/netrc/git-credential-netrc |   38 +++++++++++++------------
>> 1 files changed, 20 insertions(+), 18 deletions(-)

JCH> Especially because this is an initial submission, please equash JCH> three patches into one, instead of sending three "here is my first JCH> attempt with many problems I know I do not want to be there", "one JCH> small improvement", "another one to fix remaining issues".

JCH> Otherwise you will waste reviewers' time, getting distracted by JCH> undesirable details they find in an earlier patch while reviewing, JCH> without realizing that some of them are fixed in a later one.

OK, thanks. I wasn't sure, since Jeff already reviewed it, if it was better to squash or not. Ignore this same question in my other reply to you, and thanks for your patience.

Ted
Michal Nazarewicz· Feb 4, 2013, 16:33 UTC · re: Jeff King · lore

Re: [PATCH] git-send-email: add ~/.authinfo parsing

On Wed, Jan 30 2013, Jeff King wrote:
Show 11 quoted lines
> I do not mind a .netrc or .authinfo parser, because while those formats
> do have security problems, they are standard files that may already be
> in use. So as long as we are not encouraging their use, I do not see a
> problem in supporting them (and we already do the same with curl's netrc
> support).
>
> But it would probably make sense for send-email to support the existing
> git-credential subsystem, so that it can take advantage of secure
> system-specific storage. And that is where we should be pointing new
> users. I think contrib/mw-to-git even has credential support written in
> perl, so it would just need to be factored out to Git.pm.

As far as I understand, there could be a git-credential helper that reads ~/.authinfo and than git-send-email would just call “git credential fill”, right?

I've noticed though, that git-credential does not support port argument, which makes it slightly incompatible with ~/.authinfo.

-- 
Best regards,                                         _     _
.o. | Liege of Serenely Enlightened Majesty of      o' \,=./ `o
..o | Computer Science,  Michał “mina86” Nazarewicz    (o o)
ooo +----<email/xmpp: mpn@google.com>--------------ooO--(_)--Ooo--
Ted Zlatanov· Feb 4, 2013, 17:00 UTC · re: Michal Nazarewicz · lore

Re: [PATCH] git-send-email: add ~/.authinfo parsing

On Mon, 04 Feb 2013 17:33:58 +0100 Michal Nazarewicz <mina86@mina86.com> wrote: 

MN> As far as I understand, there could be a git-credential helper that MN> reads ~/.authinfo and than git-send-email would just call “git MN> credential fill”, right?

MN> I've noticed though, that git-credential does not support port argument, MN> which makes it slightly incompatible with ~/.authinfo.

My proposed netrc credential helper does this :)
The token mapping I use:

port, protocol => protocol machine, host => host path => path login, username, user => username password => password

I think that's sensible.
Ted
Jeff King· Feb 4, 2013, 20:10 UTC · re: Ted Zlatanov · lore

Re: [PATCH] git-send-email: add ~/.authinfo parsing

On Mon, Feb 04, 2013 at 12:00:34PM -0500, Ted Zlatanov wrote:
Show 20 quoted lines
> On Mon, 04 Feb 2013 17:33:58 +0100 Michal Nazarewicz <mina86@mina86.com> wrote: 
> 
> MN> As far as I understand, there could be a git-credential helper that
> MN> reads ~/.authinfo and than git-send-email would just call “git
> MN> credential fill”, right?
> 
> MN> I've noticed though, that git-credential does not support port argument,
> MN> which makes it slightly incompatible with ~/.authinfo.
> 
> My proposed netrc credential helper does this :)
> 
> The token mapping I use:
> 
> port, protocol        => protocol
> machine, host         => host
> path                  => path
> login, username, user => username
> password              => password
> 
> I think that's sensible.
Technically you can speak a particular protocol on an alternate port:
  https://example.com:31337/repo.git
In this case, git will send you the host as:
  example.com:31337

You might want to map this to "port" in .autoinfo separately if it's available.

-Peff
Ted Zlatanov· Feb 4, 2013, 20:28 UTC · re: Jeff King · lore

Re: [PATCH] git-send-email: add ~/.authinfo parsing

On Mon, 4 Feb 2013 15:10:40 -0500 Jeff King <peff@peff.net> wrote: 
JK> Technically you can speak a particular protocol on an alternate port:
JK>   https://example.com:31337/repo.git
JK> In this case, git will send you the host as:
JK>   example.com:31337

JK> You might want to map this to "port" in .autoinfo separately if it's JK> available.

That would create the following possibilities:
* host example.com:31337, protocol https
* host example.com:31337, protocol unspecified
* host example.com, protocol https
* host example.com, protocol unspecified

How would you like each one to be handled? My preference would be to make the user say "host example.com:31337" in the netrc file (the current situation); that's what we do in Emacs and it lets applications request credentials for a logical service no matter what the port is.

It means that example.com credentials won't be used for example.com:31337. In practice, that has not been a problem for us.

Ted
Jeff King· Feb 4, 2013, 20:59 UTC · re: Ted Zlatanov · lore

Re: [PATCH] git-send-email: add ~/.authinfo parsing

On Mon, Feb 04, 2013 at 03:28:52PM -0500, Ted Zlatanov wrote:
Show 9 quoted lines
> JK> You might want to map this to "port" in .autoinfo separately if it's
> JK> available.
> 
> That would create the following possibilities:
> 
> * host example.com:31337, protocol https
> * host example.com:31337, protocol unspecified
> * host example.com, protocol https
> * host example.com, protocol unspecified

Possibilities for .netrc, or for git? Git will always specify the protocol.

Show 7 quoted lines
> How would you like each one to be handled?  My preference would be to
> make the user say "host example.com:31337" in the netrc file (the
> current situation); that's what we do in Emacs and it lets applications
> request credentials for a logical service no matter what the port is.
> 
> It means that example.com credentials won't be used for
> example.com:31337.  In practice, that has not been a problem for us.

Yeah, I think that is a good thing. The credentials used for example.com:31337 are not necessarily the same as for the main site. It's less convenient, but a more secure default.

What I was more wondering (and I know very little about .netrc, so this might not be a possibility at all) is a line like:

  host example.com port 5001 protocol https username foo password bar

To match git's representation on a token-by-token basis, you would have to either split out git's "host:port" pair, or combine the .netrc's representation to "example.com:5001".

-Peff
Ted Zlatanov· Feb 4, 2013, 21:08 UTC · re: Jeff King · lore

Re: [PATCH] git-send-email: add ~/.authinfo parsing

On Mon, 4 Feb 2013 15:59:11 -0500 Jeff King <peff@peff.net> wrote: 

JK> On Mon, Feb 04, 2013 at 03:28:52PM -0500, Ted Zlatanov wrote: JK> You might want to map this to "port" in .autoinfo separately if it's JK> available.

Show 7 quoted lines
>> 
>> That would create the following possibilities:
>> 
>> * host example.com:31337, protocol https
>> * host example.com:31337, protocol unspecified
>> * host example.com, protocol https
>> * host example.com, protocol unspecified

JK> Possibilities for .netrc, or for git? Git will always specify the JK> protocol.

Possibilities for the netrc data. How clever do we want to be with taking 31337 and mapping it to the "protocol"? My preference is to be very simple here.

JK> What I was more wondering (and I know very little about .netrc, so this JK> might not be a possibility at all) is a line like:

JK>   host example.com port 5001 protocol https username foo password bar

JK> To match git's representation on a token-by-token basis, you would have JK> to either split out git's "host:port" pair, or combine the .netrc's JK> representation to "example.com:5001".

Currently, we map both the "port" and "protocol" netrc tokens to the credential helper protocol's "protocol". So this will have undefined results. To do what you specify could be pretty simple: we could do a preliminary scan of the tokens, looking for "host X port Y" where Y is an integer, and rewriting the host to be "X:Y". That would be clean and simple, unless the user breaks it with "host x:23 port 22". Let me know if you agree and I'll do.

Ted
Jeff King· Feb 4, 2013, 21:22 UTC · re: Ted Zlatanov · lore

Re: [PATCH] git-send-email: add ~/.authinfo parsing

On Mon, Feb 04, 2013 at 04:08:52PM -0500, Ted Zlatanov wrote:
Show 13 quoted lines
> >> That would create the following possibilities:
> >> 
> >> * host example.com:31337, protocol https
> >> * host example.com:31337, protocol unspecified
> >> * host example.com, protocol https
> >> * host example.com, protocol unspecified
> 
> JK> Possibilities for .netrc, or for git? Git will always specify the
> JK> protocol.
> 
> Possibilities for the netrc data.  How clever do we want to be with
> taking 31337 and mapping it to the "protocol"?  My preference is to be
> very simple here.

I think simple is OK, as we can iterate on it as specific use-cases come up. The important thing is to make sure we err on the side of "does not match" and not "oops, we accidentally sent your plaintext credentials to the wrong server".

Show 7 quoted lines
> Currently, we map both the "port" and "protocol" netrc tokens to the
> credential helper protocol's "protocol".  So this will have undefined
> results.  To do what you specify could be pretty simple: we could do a
> preliminary scan of the tokens, looking for "host X port Y" where Y is
> an integer, and rewriting the host to be "X:Y".  That would be clean and
> simple, unless the user breaks it with "host x:23 port 22".  Let me know
> if you agree and I'll do.

Yeah, I think that is simple and obvious. If the user is saying "host x:23 port 22", that is nonsensical.

-Peff
Ted Zlatanov· Feb 4, 2013, 21:41 UTC · re: Jeff King · lore

Re: [PATCH] git-send-email: add ~/.authinfo parsing

On Mon, 4 Feb 2013 16:22:03 -0500 Jeff King <peff@peff.net> wrote: 
Show 7 quoted lines
>> Currently, we map both the "port" and "protocol" netrc tokens to the
>> credential helper protocol's "protocol".  So this will have undefined
>> results.  To do what you specify could be pretty simple: we could do a
>> preliminary scan of the tokens, looking for "host X port Y" where Y is
>> an integer, and rewriting the host to be "X:Y".  That would be clean and
>> simple, unless the user breaks it with "host x:23 port 22".  Let me know
>> if you agree and I'll do.

JK> Yeah, I think that is simple and obvious. If the user is saying "host JK> x:23 port 22", that is nonsensical.

OK; added.
Ted
Junio C Hamano· Feb 5, 2013, 23:10 UTC · re: Jeff King · lore

Re: [PATCH] git-send-email: add ~/.authinfo parsing

Jeff King <peff@peff.net> writes:
Show 18 quoted lines
> On Tue, Jan 29, 2013 at 11:53:19AM -0800, Junio C Hamano wrote:
>
>> Either way it still encourages a plaintext password to be on disk,
>> which may not be what we want, even though it may be slight if not
>> really much of an improvement.  Again the Help-for-users has this
>> amusing bit:
>
> I do not mind a .netrc or .authinfo parser, because while those formats
> do have security problems, they are standard files that may already be
> in use. So as long as we are not encouraging their use, I do not see a
> problem in supporting them (and we already do the same with curl's netrc
> support).
>
> But it would probably make sense for send-email to support the existing
> git-credential subsystem, so that it can take advantage of secure
> system-specific storage. And that is where we should be pointing new
> users. I think contrib/mw-to-git even has credential support written in
> perl, so it would just need to be factored out to Git.pm.

I see a lot of rerolls on the credential helper front, but is there anybody working on hooking send-email to the credential framework?

Matthieu Moy· Feb 6, 2013, 08:11 UTC · re: Junio C Hamano · lore

Re: [PATCH] git-send-email: add ~/.authinfo parsing

Junio C Hamano <gitster@pobox.com> writes:
> I see a lot of rerolls on the credential helper front, but is there
> anybody working on hooking send-email to the credential framework?

Not answering the question, but git-remote-mediawiki supports the credential framework. It is written in perl, and the credential support is rather cleanly written and doesn't have dependencies on the wiki part, so the way to go for send-email is probably to libify the credential support in git-remote-mediawiki, and to use it in send-email.

-- 
Matthieu Moy
http://www-verimag.imag.fr/~moy/
Ted Zlatanov· Feb 6, 2013, 14:53 UTC · re: Matthieu Moy · lore

Re: [PATCH] git-send-email: add ~/.authinfo parsing

On Wed, 06 Feb 2013 09:11:17 +0100 Matthieu Moy <Matthieu.Moy@grenoble-inp.fr> wrote: 
MM> Junio C Hamano <gitster@pobox.com> writes:
>> I see a lot of rerolls on the credential helper front, but is there
>> anybody working on hooking send-email to the credential framework?

MM> Not answering the question, but git-remote-mediawiki supports the MM> credential framework. It is written in perl, and the credential support MM> is rather cleanly written and doesn't have dependencies on the wiki MM> part, so the way to go for send-email is probably to libify the MM> credential support in git-remote-mediawiki, and to use it in send-email.

I looked and that's indeed very useful. If it's put in a library, I'd use credential_read() and credential_write() in my netrc credential helper. But I would formalize it a little more about the token names and output, and I wouldn't necessarily die() on error. Maybe this can be merged with the netrc credential helper's read_credential_data_from_stdin() and print_credential_data()?

Let me know if you'd like me to libify this... I'm happy to leave it to Matthieu or Michal, or anyone else interested.

Ted
Matthieu Moy· Feb 6, 2013, 15:10 UTC · re: Ted Zlatanov · lore

Re: [PATCH] git-send-email: add ~/.authinfo parsing

Ted Zlatanov <tzz@lifelogs.com> writes:
Show 7 quoted lines
> MM> [...] so the way to go for send-email is probably to libify the
> MM> credential support in git-remote-mediawiki, and to use it in send-email.
>
> I looked and that's indeed very useful.  If it's put in a library, I'd
> use credential_read() and credential_write() in my netrc credential
> helper.  But I would formalize it a little more about the token names
> and output,

Can you elaborate on this? The idea of the Perl code was to mimick a call to the C API, keeping essentially the same names.

> and I wouldn't necessarily die() on error. 
Sure, die()ing in a library is bad.
> Maybe this can be merged with the netrc credential helper's
> read_credential_data_from_stdin() and print_credential_data()?

I don't know about the netrc credential helper, but I guess that's another layer. The git-remote-mediawiki code is the code to call the credential C API, that in turn may (or may not) call a credential helper.

> Let me know if you'd like me to libify this...  I'm happy to leave it to
> Matthieu or Michal, or anyone else interested.

I'd happily let you do the job, but I can help if needed. One thing to be careful about: git-remote-mediawiki is currently a standalone script, so it can be installed with a plain "cp git-remote-mediawiki $somewhere/". One consequence of libification is that it adds a dependency on the library (e.g. Git.pm). We should be carefull to keep it easy for the user to install it (e.g. some kind of "make install", or update the doc).

-- 
Matthieu Moy
http://www-verimag.imag.fr/~moy/
Ted Zlatanov· Feb 6, 2013, 15:58 UTC · re: Matthieu Moy · lore

Re: [PATCH] git-send-email: add ~/.authinfo parsing

On Wed, 06 Feb 2013 16:10:12 +0100 Matthieu Moy <Matthieu.Moy@grenoble-inp.fr> wrote: 

MM> Ted Zlatanov <tzz@lifelogs.com> writes: MM> [...] so the way to go for send-email is probably to libify the MM> credential support in git-remote-mediawiki, and to use it in send-email.

Show 5 quoted lines
>> 
>> I looked and that's indeed very useful.  If it's put in a library, I'd
>> use credential_read() and credential_write() in my netrc credential
>> helper.  But I would formalize it a little more about the token names
>> and output,

MM> Can you elaborate on this? The idea of the Perl code was to mimick a MM> call to the C API, keeping essentially the same names.

None of these are a big deal, and Michal said he's working on libifying this anyhow:

- making 'fill' a special operation is weird
- anchor the key regex to beginning of line (not strictly necessary)
- sort the output tokens (after 'url' is extracted) so the output is consistent and testable
>> and I wouldn't necessarily die() on error. 
MM> Sure, die()ing in a library is bad.
>> Maybe this can be merged with the netrc credential helper's
>> read_credential_data_from_stdin() and print_credential_data()?

MM> I don't know about the netrc credential helper, but I guess that's MM> another layer. The git-remote-mediawiki code is the code to call the MM> credential C API, that in turn may (or may not) call a credential MM> helper.

Yup. But what you call "read" and "write" are, to the credential helper, "write" and "read" but it's the same protocol :) So maybe the names should be changed to reflect that, e.g. "query" and "response."

MM> One thing to be careful about: git-remote-mediawiki is currently a MM> standalone script, so it can be installed with a plain "cp MM> git-remote-mediawiki $somewhere/". One consequence of libification MM> is that it adds a dependency on the library (e.g. Git.pm). We should MM> be carefull to keep it easy for the user to install it (e.g. some MM> kind of "make install", or update the doc).

I don't know--it's up to the `git-remote-mediawiki' maintainers... But I think anywhere you have Git, you also have Git.pm, right? Maybe? But then you also have to look at whether Git.pm has the functionality you need... so I better go quiet :)

Ted
Matthieu Moy· Feb 6, 2013, 16:41 UTC · re: Ted Zlatanov · lore

Re: [PATCH] git-send-email: add ~/.authinfo parsing

Ted Zlatanov <tzz@lifelogs.com> writes:
> None of these are a big deal, and Michal said he's working on libifying
> this anyhow:
>
> - making 'fill' a special operation is weird

Well, 'fill' is the only operation that mutates the credential structure (i.e. the only one for which "git credential" emits an output to be parsed), so you don't have much choice.

> - anchor the key regex to beginning of line (not strictly necessary)

Right. The greedyness of * ensures correction, but I like explicit anchors ^...$ too.

> - sort the output tokens (after 'url' is extracted) so the output is consistent and testable

Why not, if you want to use the output of credential_write in tests. But credential_write is essentially used to talk to "git credential", so the important information is the content of the hash before credential_write and after credential_read. They are unordered, but consistent and testable.

Show 11 quoted lines
>>> Maybe this can be merged with the netrc credential helper's
>>> read_credential_data_from_stdin() and print_credential_data()?
>
> MM> I don't know about the netrc credential helper, but I guess that's
> MM> another layer. The git-remote-mediawiki code is the code to call the
> MM> credential C API, that in turn may (or may not) call a credential
> MM> helper.
>
> Yup.  But what you call "read" and "write" are, to the credential
> helper, "write" and "read" but it's the same protocol :)  So maybe the
> names should be changed to reflect that, e.g. "query" and "response."

I don't think that would be a better naming. Maybe "serialize" and "parse" would be better, but "query" would sound like it establishes the connection and possibly reads the response to me.

Show 8 quoted lines
> MM> One thing to be careful about: git-remote-mediawiki is currently a
> MM> standalone script, so it can be installed with a plain "cp
> MM> git-remote-mediawiki $somewhere/".  One consequence of libification
> MM> is that it adds a dependency on the library (e.g. Git.pm). We should
> MM> be carefull to keep it easy for the user to install it (e.g. some
> MM> kind of "make install", or update the doc).
>
> I don't know--it's up to the `git-remote-mediawiki' maintainers...
That is, me ;-).
> But I think anywhere you have Git, you also have Git.pm, right?

Yes, but you have to find out where it is installed. Git's Makefile hardcodes the path to Git.pm at build time, inserting one line in the perl script:

use lib (split(/:/, $ENV{GITPERLLIB} || "$INSTLIBDIR"));

The same needs to be done for git-remote-mediawiki. As much as possible, I'd rather avoid copy-pasting from Git's Makefile, so this means extracting the perl part of Git's Makefile and make it available in contrib/.

I'll try a patch in this direction.
> Maybe? But then you also have to look at whether Git.pm has the
> functionality you need...

If git-remote-mediawiki is installed from Git's source, I think it's OK to assume that Git.pm will be up to date, but that would be even better if we can issue a clean error message when the functions to be called do not exist.

-- 
Matthieu Moy
http://www-verimag.imag.fr/~moy/
Ted Zlatanov· Feb 6, 2013, 17:40 UTC · re: Matthieu Moy · lore

Re: [PATCH] git-send-email: add ~/.authinfo parsing

On Wed, 06 Feb 2013 17:41:01 +0100 Matthieu Moy <Matthieu.Moy@grenoble-inp.fr> wrote: 
MM> Ted Zlatanov <tzz@lifelogs.com> writes:
>> - sort the output tokens (after 'url' is extracted) so the output is consistent and testable

MM> Why not, if you want to use the output of credential_write in tests. But MM> credential_write is essentially used to talk to "git credential", so the MM> important information is the content of the hash before credential_write MM> and after credential_read. They are unordered, but consistent and MM> testable.

I like testing output (especially when it's part of an API), so we should make the externally observable output consistent and testable.

The change is tiny, just sort the keys instead of calling each(), so I hope it makes it in the final version.

>> Yup.  But what you call "read" and "write" are, to the credential
>> helper, "write" and "read" but it's the same protocol :)  So maybe the
>> names should be changed to reflect that, e.g. "query" and "response."

MM> I don't think that would be a better naming. Maybe "serialize" and MM> "parse" would be better, but "query" would sound like it establishes the MM> connection and possibly reads the response to me.

I'm OK with anything unambiguous.

Thanks! Ted

Matthieu Moy· Feb 6, 2013, 18:11 UTC · re: Matthieu Moy · lore

[PATCH 0/4] Allow contrib/ to use Git's Makefile for perl code

The very final goal is to be able to move painlessly (credential) code from git-remote-mediawiki to Git.pm, but then it's nice for the user to be able to say just "cd contrib/mw-to-git && make install" and let the Makefile set perl's library path just like other Git commands written in perl.

This series does this while trying to minimize code duplication, and to make it easy for future other tools in contrib to do the same.

Matthieu Moy (4):
  Makefile: extract perl-related rules to make them available from other
    dirs
  perl.mak: introduce $(GIT_ROOT_DIR) to allow inclusion from other
    directories
  Makefile: factor common configuration in git-default-config.mak
  git-remote-mediawiki: use Git's Makefile to build the script
 Makefile                                           | 108 +--------------------
 contrib/mw-to-git/.gitignore                       |   1 +
 contrib/mw-to-git/Makefile                         |  45 ++++++---
 ...-remote-mediawiki => git-remote-mediawiki.perl} |   0
 default-config.mak                                 |  61 ++++++++++++
 perl.mak                                           |  52 ++++++++++
 6 files changed, 145 insertions(+), 122 deletions(-)
 create mode 100644 contrib/mw-to-git/.gitignore
 rename contrib/mw-to-git/{git-remote-mediawiki => git-remote-mediawiki.perl} (100%)
 create mode 100644 default-config.mak
 create mode 100644 perl.mak
-- 
1.8.1.2.526.gf51a757
Matthieu Moy· Feb 6, 2013, 18:11 UTC · re: Matthieu Moy · lore

[PATCH 1/4] Makefile: extract perl-related rules to make them available from other dirs

The final goal is to make it easy to write Git commands in perl in the contrib/ directory. It is currently possible to do so, but without the benefits of Git's Makefile: adapt first line with $(PERL_PATH), hardcode the path to Git.pm, ...

We make the perl-related part of the Makefile available from directories other than the toplevel so that:

* Developers can include it, to avoid code duplication
* Users can get a consistent behavior of "make install"
Signed-off-by: Matthieu Moy <Matthieu.Moy@imag.fr>
---
 Makefile | 46 +---------------------------------------------
 perl.mak | 49 +++++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 50 insertions(+), 45 deletions(-)
 create mode 100644 perl.mak
Show changes to 2 files +50 −45

Makefile, perl.mak

diff --git a/Makefile b/Makefile
index 731b6a8..f39d4a9 100644
--- a/Makefile
+++ b/Makefile
@@ -573,14 +573,10 @@ BINDIR_PROGRAMS_NO_X += git-cvsserver
 ifndef SHELL_PATH
 	SHELL_PATH = /bin/sh
 endif
-ifndef PERL_PATH
-	PERL_PATH = /usr/bin/perl
-endif
 ifndef PYTHON_PATH
 	PYTHON_PATH = /usr/bin/python
 endif
 
-export PERL_PATH
 export PYTHON_PATH
 
 LIB_FILE = libgit.a
@@ -1441,10 +1437,6 @@ ifeq ($(TCLTK_PATH),)
 NO_TCLTK = NoThanks
 endif
 
-ifeq ($(PERL_PATH),)
-NO_PERL = NoThanks
-endif
-
 ifeq ($(PYTHON_PATH),)
 NO_PYTHON = NoThanks
 endif
@@ -1522,7 +1514,6 @@ prefix_SQ = $(subst ','\'',$(prefix))
 gitwebdir_SQ = $(subst ','\'',$(gitwebdir))
 
 SHELL_PATH_SQ = $(subst ','\'',$(SHELL_PATH))
-PERL_PATH_SQ = $(subst ','\'',$(PERL_PATH))
 PYTHON_PATH_SQ = $(subst ','\'',$(PYTHON_PATH))
 TCLTK_PATH_SQ = $(subst ','\'',$(TCLTK_PATH))
 DIFF_SQ = $(subst ','\'',$(DIFF))
@@ -1715,9 +1706,6 @@ $(SCRIPT_LIB) : % : %.sh GIT-SCRIPT-DEFINES
 	$(QUIET_GEN)$(cmd_munge_script) && \
 	mv $@+ $@
 
-ifndef NO_PERL
-$(patsubst %.perl,%,$(SCRIPT_PERL)): perl/perl.mak
-
 perl/perl.mak: perl/PM.stamp
 
 perl/PM.stamp: FORCE
@@ -1728,39 +1716,7 @@ perl/PM.stamp: FORCE
 perl/perl.mak: GIT-CFLAGS GIT-PREFIX perl/Makefile perl/Makefile.PL
 	$(QUIET_SUBDIR0)perl $(QUIET_SUBDIR1) PERL_PATH='$(PERL_PATH_SQ)' prefix='$(prefix_SQ)' $(@F)
 
-$(patsubst %.perl,%,$(SCRIPT_PERL)): % : %.perl GIT-VERSION-FILE
-	$(QUIET_GEN)$(RM) $@ $@+ && \
-	INSTLIBDIR=`MAKEFLAGS= $(MAKE) -C perl -s --no-print-directory instlibdir` && \
-	sed -e '1{' \
-	    -e '	s|#!.*perl|#!$(PERL_PATH_SQ)|' \
-	    -e '	h' \
-	    -e '	s=.*=use lib (split(/$(pathsep)/, $$ENV{GITPERLLIB} || "'"$$INSTLIBDIR"'"));=' \
-	    -e '	H' \
-	    -e '	x' \
-	    -e '}' \
-	    -e 's/@@GIT_VERSION@@/$(GIT_VERSION)/g' \
-	    $@.perl >$@+ && \
-	chmod +x $@+ && \
-	mv $@+ $@
-
-
-.PHONY: gitweb
-gitweb:
-	$(QUIET_SUBDIR0)gitweb $(QUIET_SUBDIR1) all
-
-git-instaweb: git-instaweb.sh gitweb GIT-SCRIPT-DEFINES
-	$(QUIET_GEN)$(cmd_munge_script) && \
-	chmod +x $@+ && \
-	mv $@+ $@
-else # NO_PERL
-$(patsubst %.perl,%,$(SCRIPT_PERL)) git-instaweb: % : unimplemented.sh
-	$(QUIET_GEN)$(RM) $@ $@+ && \
-	sed -e '1s|#!.*/sh|#!$(SHELL_PATH_SQ)|' \
-	    -e 's|@@REASON@@|NO_PERL=$(NO_PERL)|g' \
-	    unimplemented.sh >$@+ && \
-	chmod +x $@+ && \
-	mv $@+ $@
-endif # NO_PERL
+include perl.mak
 
 ifndef NO_PYTHON
 $(patsubst %.py,%,$(SCRIPT_PYTHON)): GIT-CFLAGS GIT-PREFIX GIT-PYTHON-VARS
diff --git a/perl.mak b/perl.mak
new file mode 100644
index 0000000..8bbeef3
--- /dev/null
+++ b/perl.mak
@@ -0,0 +1,49 @@
+# Rules to build Git commands written in perl
+
+ifndef PERL_PATH
+	PERL_PATH = /usr/bin/perl
+endif
+export PERL_PATH
+PERL_PATH_SQ = $(subst ','\'',$(PERL_PATH))
+
+ifeq ($(PERL_PATH),)
+NO_PERL = NoThanks
+endif
+
+ifndef NO_PERL
+$(patsubst %.perl,%,$(SCRIPT_PERL)): perl/perl.mak
+
+
+$(patsubst %.perl,%,$(SCRIPT_PERL)): % : %.perl GIT-VERSION-FILE
+	$(QUIET_GEN)$(RM) $@ $@+ && \
+	INSTLIBDIR=`MAKEFLAGS= $(MAKE) -C perl -s --no-print-directory instlibdir` && \
+	sed -e '1{' \
+	    -e '	s|#!.*perl|#!$(PERL_PATH_SQ)|' \
+	    -e '	h' \
+	    -e '	s=.*=use lib (split(/$(pathsep)/, $$ENV{GITPERLLIB} || "'"$$INSTLIBDIR"'"));=' \
+	    -e '	H' \
+	    -e '	x' \
+	    -e '}' \
+	    -e 's/@@GIT_VERSION@@/$(GIT_VERSION)/g' \
+	    $@.perl >$@+ && \
+	chmod +x $@+ && \
+	mv $@+ $@
+
+
+.PHONY: gitweb
+gitweb:
+	$(QUIET_SUBDIR0)gitweb $(QUIET_SUBDIR1) all
+
+git-instaweb: git-instaweb.sh gitweb GIT-SCRIPT-DEFINES
+	$(QUIET_GEN)$(cmd_munge_script) && \
+	chmod +x $@+ && \
+	mv $@+ $@
+else # NO_PERL
+$(patsubst %.perl,%,$(SCRIPT_PERL)) git-instaweb: % : unimplemented.sh
+	$(QUIET_GEN)$(RM) $@ $@+ && \
+	sed -e '1s|#!.*/sh|#!$(SHELL_PATH_SQ)|' \
+	    -e 's|@@REASON@@|NO_PERL=$(NO_PERL)|g' \
+	    unimplemented.sh >$@+ && \
+	chmod +x $@+ && \
+	mv $@+ $@
+endif # NO_PERL
-- 
1.8.1.2.526.gf51a757
Junio C Hamano· Feb 7, 2013, 19:16 UTC · re: Matthieu Moy · lore

Re: [PATCH 1/4] Makefile: extract perl-related rules to make them available from other dirs

Matthieu Moy <Matthieu.Moy@imag.fr> writes:
Show 13 quoted lines
> The final goal is to make it easy to write Git commands in perl in the
> contrib/ directory. It is currently possible to do so, but without the
> benefits of Git's Makefile: adapt first line with $(PERL_PATH),
> hardcode the path to Git.pm, ...
>
> We make the perl-related part of the Makefile available from directories
> other than the toplevel so that:
>
> * Developers can include it, to avoid code duplication
>
> * Users can get a consistent behavior of "make install"
>
> Signed-off-by: Matthieu Moy <Matthieu.Moy@imag.fr>
The goal may be worthy, but the split does not look quite right.

What business do contrib/ scripts have knowing how gitweb and git-instaweb are built and what they depend on, for example?

Matthieu Moy· Feb 6, 2013, 18:11 UTC · re: Matthieu Moy · lore

[PATCH 2/4] perl.mak: introduce $(GIT_ROOT_DIR) to allow inclusion from other directories

perl.mak uses relative path, which is OK when called from the toplevel, but won't be anymore if one includes it from elsewhere. It is now possible to include the file using:

GIT_ROOT_DIR=<whatever> include $(GIT_ROOT_DIR)/perl.mak

Signed-off-by: Matthieu Moy <Matthieu.Moy@imag.fr>
---
 perl.mak | 11 +++++++----
 1 file changed, 7 insertions(+), 4 deletions(-)
Show changes to perl.mak +7 −4
diff --git a/perl.mak b/perl.mak
index 8bbeef3..a2b8717 100644
--- a/perl.mak
+++ b/perl.mak
@@ -1,5 +1,9 @@
 # Rules to build Git commands written in perl
 
+ifndef GIT_ROOT_DIR
+	GIT_ROOT_DIR = .
+endif
+
 ifndef PERL_PATH
 	PERL_PATH = /usr/bin/perl
 endif
@@ -11,12 +15,11 @@ NO_PERL = NoThanks
 endif
 
 ifndef NO_PERL
-$(patsubst %.perl,%,$(SCRIPT_PERL)): perl/perl.mak
-
+$(patsubst %.perl,%,$(SCRIPT_PERL)): $(GIT_ROOT_DIR)/perl/perl.mak
 
-$(patsubst %.perl,%,$(SCRIPT_PERL)): % : %.perl GIT-VERSION-FILE
+$(patsubst %.perl,%,$(SCRIPT_PERL)): % : %.perl $(GIT_ROOT_DIR)/GIT-VERSION-FILE
 	$(QUIET_GEN)$(RM) $@ $@+ && \
-	INSTLIBDIR=`MAKEFLAGS= $(MAKE) -C perl -s --no-print-directory instlibdir` && \
+	INSTLIBDIR=`MAKEFLAGS= $(MAKE) -C $(GIT_ROOT_DIR)/perl -s --no-print-directory instlibdir` && \
 	sed -e '1{' \
 	    -e '	s|#!.*perl|#!$(PERL_PATH_SQ)|' \
 	    -e '	h' \
-- 
1.8.1.2.526.gf51a757
Matthieu Moy· Feb 6, 2013, 18:11 UTC · re: Matthieu Moy · lore

[PATCH 3/4] Makefile: factor common configuration in git-default-config.mak

Similarly to the extraction of perl-related code in perl.mak, we extract general default configuration from the Makefile to make it available from directories other than the toplevel.

This is required to make perl.mak usable because it requires $(pathsep) to be set.

Signed-off-by: Matthieu Moy <Matthieu.Moy@imag.fr>
---
 Makefile           | 62 +-----------------------------------------------------
 default-config.mak | 61 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 62 insertions(+), 61 deletions(-)
 create mode 100644 default-config.mak
Show changes to 2 files +62 −61

Makefile, default-config.mak

diff --git a/Makefile b/Makefile
index f39d4a9..9649a41 100644
--- a/Makefile
+++ b/Makefile
@@ -346,67 +346,7 @@ GIT-VERSION-FILE: FORCE
 	@$(SHELL_PATH) ./GIT-VERSION-GEN
 -include GIT-VERSION-FILE
 
-# CFLAGS and LDFLAGS are for the users to override from the command line.
-
-CFLAGS = -g -O2 -Wall
-LDFLAGS =
-ALL_CFLAGS = $(CPPFLAGS) $(CFLAGS)
-ALL_LDFLAGS = $(LDFLAGS)
-STRIP ?= strip
-
-# Among the variables below, these:
-#   gitexecdir
-#   template_dir
-#   mandir
-#   infodir
-#   htmldir
-#   sysconfdir
-# can be specified as a relative path some/where/else;
-# this is interpreted as relative to $(prefix) and "git" at
-# runtime figures out where they are based on the path to the executable.
-# This can help installing the suite in a relocatable way.
-
-prefix = $(HOME)
-bindir_relative = bin
-bindir = $(prefix)/$(bindir_relative)
-mandir = share/man
-infodir = share/info
-gitexecdir = libexec/git-core
-mergetoolsdir = $(gitexecdir)/mergetools
-sharedir = $(prefix)/share
-gitwebdir = $(sharedir)/gitweb
-localedir = $(sharedir)/locale
-template_dir = share/git-core/templates
-htmldir = share/doc/git-doc
-ETC_GITCONFIG = $(sysconfdir)/gitconfig
-ETC_GITATTRIBUTES = $(sysconfdir)/gitattributes
-lib = lib
-# DESTDIR =
-pathsep = :
-
-export prefix bindir sharedir sysconfdir gitwebdir localedir
-
-CC = cc
-AR = ar
-RM = rm -f
-DIFF = diff
-TAR = tar
-FIND = find
-INSTALL = install
-RPMBUILD = rpmbuild
-TCL_PATH = tclsh
-TCLTK_PATH = wish
-XGETTEXT = xgettext
-MSGFMT = msgfmt
-PTHREAD_LIBS = -lpthread
-PTHREAD_CFLAGS =
-GCOV = gcov
-
-export TCL_PATH TCLTK_PATH
-
-SPARSE_FLAGS =
-
-
+include default-config.mak
 
 ### --- END CONFIGURATION SECTION ---
 
diff --git a/default-config.mak b/default-config.mak
new file mode 100644
index 0000000..b2aab3d
--- /dev/null
+++ b/default-config.mak
@@ -0,0 +1,61 @@
+# CFLAGS and LDFLAGS are for the users to override from the command line.
+
+CFLAGS = -g -O2 -Wall
+LDFLAGS =
+ALL_CFLAGS = $(CPPFLAGS) $(CFLAGS)
+ALL_LDFLAGS = $(LDFLAGS)
+STRIP ?= strip
+
+# Among the variables below, these:
+#   gitexecdir
+#   template_dir
+#   mandir
+#   infodir
+#   htmldir
+#   sysconfdir
+# can be specified as a relative path some/where/else;
+# this is interpreted as relative to $(prefix) and "git" at
+# runtime figures out where they are based on the path to the executable.
+# This can help installing the suite in a relocatable way.
+
+prefix = $(HOME)
+bindir_relative = bin
+bindir = $(prefix)/$(bindir_relative)
+mandir = share/man
+infodir = share/info
+gitexecdir = libexec/git-core
+mergetoolsdir = $(gitexecdir)/mergetools
+sharedir = $(prefix)/share
+gitwebdir = $(sharedir)/gitweb
+localedir = $(sharedir)/locale
+template_dir = share/git-core/templates
+htmldir = share/doc/git-doc
+ETC_GITCONFIG = $(sysconfdir)/gitconfig
+ETC_GITATTRIBUTES = $(sysconfdir)/gitattributes
+lib = lib
+# DESTDIR =
+pathsep = :
+
+export prefix bindir sharedir sysconfdir gitwebdir localedir
+
+CC = cc
+AR = ar
+RM = rm -f
+DIFF = diff
+TAR = tar
+FIND = find
+INSTALL = install
+RPMBUILD = rpmbuild
+TCL_PATH = tclsh
+TCLTK_PATH = wish
+XGETTEXT = xgettext
+MSGFMT = msgfmt
+PTHREAD_LIBS = -lpthread
+PTHREAD_CFLAGS =
+GCOV = gcov
+
+export TCL_PATH TCLTK_PATH
+
+SPARSE_FLAGS =
+
+
-- 
1.8.1.2.526.gf51a757
Junio C Hamano· Feb 7, 2013, 19:28 UTC · re: Matthieu Moy · lore

Re: [PATCH 3/4] Makefile: factor common configuration in git-default-config.mak

Matthieu Moy <Matthieu.Moy@imag.fr> writes:
Show 9 quoted lines
> Similarly to the extraction of perl-related code in perl.mak, we extract
> general default configuration from the Makefile to make it available from
> directories other than the toplevel.
>
> This is required to make perl.mak usable because it requires $(pathsep)
> to be set.
>
> Signed-off-by: Matthieu Moy <Matthieu.Moy@imag.fr>
> ---

I really think this is going in a wrong direction. Whatever you happen to have chosen in this patch will be available to others, while whatever are left out will not be. When adding new things, people need to ask if it needs to be sharable or not, and the right answer to that question will even change over time.

Matthieu Moy· Feb 8, 2013, 17:05 UTC · re: Junio C Hamano · lore

Re: [PATCH 3/4] Makefile: factor common configuration in git-default-config.mak

Junio C Hamano <gitster@pobox.com> writes:
Show 5 quoted lines
> I really think this is going in a wrong direction.  Whatever you
> happen to have chosen in this patch will be available to others,
> while whatever are left out will not be.  When adding new things,
> people need to ask if it needs to be sharable or not, and the right
> answer to that question will even change over time.

My feeling is that Git's toplevel Makefile has become too large to remain completely monolithic, and splitting is good to organize it (and yes, splitting code into several files imply that future added code will have to be added in the right file, but that's not very different from splitting C code into several .c files to me). But that is another matter, and ...

Junio C Hamano <gitster@pobox.com> writes:
Show 6 quoted lines
> Then do something like
>
> 	all::
> 		$(MAKE) -C ../.. \
> 			PERL_SCRIPT=contrib/mw-to-git/git-remote-mediawiki.perl \
>                         build-perl-script

This ended up being very simple to implement (essentially, the Makefile already knows how to do this, so this just means adding convenience build-perl-script target to the toplevel), so 2 new patches follow doing this, with a ridiculously small new version of mw-to-git/Makefile.

-- 
Matthieu Moy
http://www-verimag.imag.fr/~moy/
Matthieu Moy· Feb 8, 2013, 17:31 UTC · re: Matthieu Moy · lore

[PATCH 1/2] Makefile: make script-related rules usable from subdirectories

Git's Makefile provides a few nice features for script build and installation (substitute the first line with the right path, hardcode the path to Git library, ...).

The Makefile already knows how to process files outside the toplevel directory with e.g.

  make SCRIPT_PERL=path/to/file.perl path/to/file

but we can make it simpler for callers by exposing build, install and clean rules as .PHONY targets.

Signed-off-by: Matthieu Moy <Matthieu.Moy@imag.fr>
---
The goal of this series is to use perl, but it is as easy to do it
with sh and python too, so I did it for them too. I tested a manual
"make -C ../../" in contrib/subtree and contrib/hg-to-git/ to check that
it actually works.
 Makefile | 35 ++++++++++++++++++++++++++++++++---
 1 file changed, 32 insertions(+), 3 deletions(-)
Show changes to Makefile +32 −3
diff --git a/Makefile b/Makefile
index 5a2e02d..b4af30d 100644
--- a/Makefile
+++ b/Makefile
@@ -480,9 +480,38 @@ SCRIPT_PERL += git-svn.perl
 SCRIPT_PYTHON += git-remote-testpy.py
 SCRIPT_PYTHON += git-p4.py
 
-SCRIPTS = $(patsubst %.sh,%,$(SCRIPT_SH)) \
-	  $(patsubst %.perl,%,$(SCRIPT_PERL)) \
-	  $(patsubst %.py,%,$(SCRIPT_PYTHON)) \
+# Generated files for scripts
+SCRIPT_SH_GEN = $(patsubst %.sh,%,$(SCRIPT_SH))
+SCRIPT_PERL_GEN = $(patsubst %.perl,%,$(SCRIPT_PERL))
+SCRIPT_PYTHON_GEN = $(patsubst %.py,%,$(SCRIPT_PYTHON))
+
+# Individual rules to allow e.g.
+# "make -C ../.. SCRIPT_PERL=contrib/foo/bar.perl build-perl-script"
+# from subdirectories like contrib/*/
+.PHONY: build-perl-script build-sh-script build-python-script
+build-perl-script: $(SCRIPT_PERL_GEN)
+build-sh-script: $(SCRIPT_SH_GEN)
+build-python-script: $(SCRIPT_PYTHON_GEN)
+
+.PHONY: install-perl-script install-sh-script install-python-script
+install-sh-script: $(SCRIPT_SH_GEN)
+	$(INSTALL) $(SCRIPT_SH_GEN) '$(DESTDIR_SQ)$(gitexec_instdir_SQ)'
+install-perl-script: $(SCRIPT_PERL_GEN)
+	$(INSTALL) $(SCRIPT_PERL_GEN) '$(DESTDIR_SQ)$(gitexec_instdir_SQ)'
+install-python-script: $(SCRIPT_PYTHON_GEN)
+	$(INSTALL) $(SCRIPT_PYTHON_GEN) '$(DESTDIR_SQ)$(gitexec_instdir_SQ)'
+
+.PHONY: clean-perl-script clean-sh-script clean-python-script
+clean-sh-script:
+	$(RM) $(SCRIPT_SH_GEN)
+clean-perl-script:
+	$(RM) $(SCRIPT_PERL_GEN)
+clean-python-script:
+	$(RM) $(SCRIPT_PYTHON_GEN)
+
+SCRIPTS = $(SCRIPT_SH_GEN) \
+	  $(SCRIPT_PERL_GEN) \
+	  $(SCRIPT_PYTHON_GEN) \
 	  git-instaweb
 
 ETAGS_TARGET = TAGS
-- 
1.8.1.2.530.g3cc16e4.dirty
Matthieu Moy· Feb 8, 2013, 17:31 UTC · re: Matthieu Moy · lore

[PATCH 2/2] git-remote-mediawiki: use toplevel's Makefile

This makes the Makefile simpler, while providing more features, and more consistency (the exact same rules with the exact same configuration as Git official commands are applied with the new version).

Signed-off-by: Matthieu Moy <Matthieu.Moy@imag.fr>
---
 contrib/mw-to-git/.gitignore                       |  1 +
 contrib/mw-to-git/Makefile                         | 64 ++++++----------------
 ...-remote-mediawiki => git-remote-mediawiki.perl} |  0
 3 files changed, 18 insertions(+), 47 deletions(-)
 create mode 100644 contrib/mw-to-git/.gitignore
 rewrite contrib/mw-to-git/Makefile (96%)
 rename contrib/mw-to-git/{git-remote-mediawiki => git-remote-mediawiki.perl} (100%)
Show changes to 3 files +18 −47

contrib/mw-to-git/.gitignore, contrib/mw-to-git/Makefile, contrib/mw-to-git/git-remote-mediawiki

diff --git a/contrib/mw-to-git/.gitignore b/contrib/mw-to-git/.gitignore
new file mode 100644
index 0000000..b919655
--- /dev/null
+++ b/contrib/mw-to-git/.gitignore
@@ -0,0 +1 @@
+git-remote-mediawiki
diff --git a/contrib/mw-to-git/Makefile b/contrib/mw-to-git/Makefile
dissimilarity index 96%
index 3ed728b..f149719 100644
--- a/contrib/mw-to-git/Makefile
+++ b/contrib/mw-to-git/Makefile
@@ -1,47 +1,17 @@
-#
-# Copyright (C) 2012
-#     Charles Roussel <charles.roussel@ensimag.imag.fr>
-#     Simon Cathebras <simon.cathebras@ensimag.imag.fr>
-#     Julien Khayat <julien.khayat@ensimag.imag.fr>
-#     Guillaume Sasdy <guillaume.sasdy@ensimag.imag.fr>
-#     Simon Perrat <simon.perrat@ensimag.imag.fr>
-#
-## Build git-remote-mediawiki
-
--include ../../config.mak.autogen
--include ../../config.mak
-
-ifndef PERL_PATH
-	PERL_PATH = /usr/bin/perl
-endif
-ifndef gitexecdir
-	gitexecdir = $(shell git --exec-path)
-endif
-
-PERL_PATH_SQ = $(subst ','\'',$(PERL_PATH))
-gitexecdir_SQ = $(subst ','\'',$(gitexecdir))
-SCRIPT = git-remote-mediawiki
-
-.PHONY: install help doc test clean
-
-help:
-	@echo 'This is the help target of the Makefile. Current configuration:'
-	@echo '  gitexecdir = $(gitexecdir_SQ)'
-	@echo '  PERL_PATH = $(PERL_PATH_SQ)'
-	@echo 'Run "$(MAKE) install" to install $(SCRIPT) in gitexecdir'
-	@echo 'Run "$(MAKE) test" to run the testsuite'
-
-install:
-	sed -e '1s|#!.*/perl|#!$(PERL_PATH_SQ)|' $(SCRIPT) \
-		> '$(gitexecdir_SQ)/$(SCRIPT)'
-	chmod +x '$(gitexecdir)/$(SCRIPT)'
-
-doc:
-	@echo 'Sorry, "make doc" is not implemented yet for $(SCRIPT)'
-
-test:
-	$(MAKE) -C t/ test
-
-clean:
-	$(RM) '$(gitexecdir)/$(SCRIPT)'
-	$(MAKE) -C t/ clean
+#
+# Copyright (C) 2013
+#     Matthieu Moy <Matthieu.Moy@imag.fr>
+#
+## Build git-remote-mediawiki
+
+SCRIPT_PERL=git-remote-mediawiki.perl
+GIT_ROOT_DIR=../..
+HERE=contrib/mw-to-git/
+
+SCRIPT_PERL_FULL=$(patsubst %,$(HERE)/%,$(SCRIPT_PERL))
+
+all: build
+
+build install clean:
+	$(MAKE) -C $(GIT_ROOT_DIR) SCRIPT_PERL=$(SCRIPT_PERL_FULL) \
+                $@-perl-script
diff --git a/contrib/mw-to-git/git-remote-mediawiki b/contrib/mw-to-git/git-remote-mediawiki.perl
similarity index 100%
rename from contrib/mw-to-git/git-remote-mediawiki
rename to contrib/mw-to-git/git-remote-mediawiki.perl
-- 
1.8.1.2.530.g3cc16e4.dirty
Matthieu Moy· Feb 6, 2013, 18:11 UTC · re: Matthieu Moy · lore

[PATCH 4/4] git-remote-mediawiki: use Git's Makefile to build the script

The configuration of the install directory is not reused from the toplevel Makefile: we assume Git is already built, hence just call "git --exec-path". This avoids too much surgery in the toplevel Makefile.

git-remote-mediawiki.perl can now "use Git;".
Signed-off-by: Matthieu Moy <Matthieu.Moy@imag.fr>
---
 contrib/mw-to-git/.gitignore                       |  1 +
 contrib/mw-to-git/Makefile                         | 45 ++++++++++++++--------
 ...-remote-mediawiki => git-remote-mediawiki.perl} |  0
 3 files changed, 30 insertions(+), 16 deletions(-)
 create mode 100644 contrib/mw-to-git/.gitignore
 rename contrib/mw-to-git/{git-remote-mediawiki => git-remote-mediawiki.perl} (100%)
Show changes to 3 files +30 −16

contrib/mw-to-git/.gitignore, contrib/mw-to-git/Makefile, contrib/mw-to-git/git-remote-mediawiki

diff --git a/contrib/mw-to-git/.gitignore b/contrib/mw-to-git/.gitignore
new file mode 100644
index 0000000..b919655
--- /dev/null
+++ b/contrib/mw-to-git/.gitignore
@@ -0,0 +1 @@
+git-remote-mediawiki
diff --git a/contrib/mw-to-git/Makefile b/contrib/mw-to-git/Makefile
index 3ed728b..ed8073b 100644
--- a/contrib/mw-to-git/Makefile
+++ b/contrib/mw-to-git/Makefile
@@ -8,40 +8,53 @@
 #
 ## Build git-remote-mediawiki
 
--include ../../config.mak.autogen
--include ../../config.mak
+all:
+
+GIT_ROOT_DIR=../../
+include $(GIT_ROOT_DIR)/default-config.mak
+-include $(GIT_ROOT_DIR)/config.mak.autogen
+-include $(GIT_ROOT_DIR)/config.mak
+-include $(GIT_ROOT_DIR)/GIT-VERSION-FILE
+
+
+SCRIPT_PERL = git-remote-mediawiki.perl
+ALL_PROGRAMS = $(patsubst %.perl,%,$(SCRIPT_PERL))
+
+include $(GIT_ROOT_DIR)/perl.mak
 
-ifndef PERL_PATH
-	PERL_PATH = /usr/bin/perl
-endif
 ifndef gitexecdir
 	gitexecdir = $(shell git --exec-path)
 endif
 
-PERL_PATH_SQ = $(subst ','\'',$(PERL_PATH))
-gitexecdir_SQ = $(subst ','\'',$(gitexecdir))
-SCRIPT = git-remote-mediawiki
+ifneq ($(filter /%,$(firstword $(gitexecdir))),)
+gitexec_instdir = $(gitexecdir)
+else
+gitexec_instdir = $(prefix)/$(gitexecdir)
+endif
+gitexec_instdir_SQ = $(subst ','\'',$(gitexec_instdir))
 
 .PHONY: install help doc test clean
 
 help:
 	@echo 'This is the help target of the Makefile. Current configuration:'
-	@echo '  gitexecdir = $(gitexecdir_SQ)'
+	@echo '  gitexec_instdir = $(gitexec_instdir_SQ)'
 	@echo '  PERL_PATH = $(PERL_PATH_SQ)'
-	@echo 'Run "$(MAKE) install" to install $(SCRIPT) in gitexecdir'
+	@echo 'Run "$(MAKE) all" to build the script'
+	@echo 'Run "$(MAKE) install" to install $(ALL_PROGRAMS) in gitexec_instdir'
 	@echo 'Run "$(MAKE) test" to run the testsuite'
 
-install:
-	sed -e '1s|#!.*/perl|#!$(PERL_PATH_SQ)|' $(SCRIPT) \
-		> '$(gitexecdir_SQ)/$(SCRIPT)'
-	chmod +x '$(gitexecdir)/$(SCRIPT)'
+all: $(ALL_PROGRAMS)
+
+install: $(ALL_PROGRAMS)
+	$(INSTALL) $(ALL_PROGRAMS) '$(DESTDIR_SQ)$(gitexec_instdir_SQ)'
 
 doc:
-	@echo 'Sorry, "make doc" is not implemented yet for $(SCRIPT)'
+	@echo 'Sorry, "make doc" is not implemented yet for $(ALL_PROGRAMS)'
 
 test:
 	$(MAKE) -C t/ test
 
 clean:
-	$(RM) '$(gitexecdir)/$(SCRIPT)'
+	$(RM) $(ALL_PROGRAMS)
+	$(RM) $(patsubst %,$(gitexec_instdir)/%,/$(ALL_PROGRAMS))
 	$(MAKE) -C t/ clean
diff --git a/contrib/mw-to-git/git-remote-mediawiki b/contrib/mw-to-git/git-remote-mediawiki.perl
similarity index 100%
rename from contrib/mw-to-git/git-remote-mediawiki
rename to contrib/mw-to-git/git-remote-mediawiki.perl
-- 
1.8.1.2.526.gf51a757
Junio C Hamano· Feb 7, 2013, 19:28 UTC · re: Matthieu Moy · lore

Re: [PATCH 4/4] git-remote-mediawiki: use Git's Makefile to build the script

Matthieu Moy <Matthieu.Moy@imag.fr> writes:
Show 8 quoted lines
> The configuration of the install directory is not reused from the
> toplevel Makefile: we assume Git is already built, hence just call
> "git --exec-path". This avoids too much surgery in the toplevel Makefile.
>
> git-remote-mediawiki.perl can now "use Git;".
>
> Signed-off-by: Matthieu Moy <Matthieu.Moy@imag.fr>
> ---

Continuing to the comment on 3/4, I wonder if it would be a lot simpler and more maintainable if you replaced 1/4 to 3/4 with a smaller patch to the top-level Makefile to teach it to munge arbitrary path/to/foo.perl to path/to/foo the same way as we do to other path/tool.perl that are known to the top-level Makefile (similarly, another target to install the resulting path/to/foo at an arbitrary place). Then do something like

	all::
		$(MAKE) -C ../.. \
			PERL_SCRIPT=contrib/mw-to-git/git-remote-mediawiki.perl \
                        build-perl-script
	install::
		$(MAKE) -C ../.. \
			PERL_SCRIPT=contrib/mw-to-git/git-remote-mediawiki.perl \
                        install-perl-script
in this step.
Jeff King· Feb 8, 2013, 04:28 UTC · re: Junio C Hamano · lore

Re: [PATCH 4/4] git-remote-mediawiki: use Git's Makefile to build the script

On Thu, Feb 07, 2013 at 11:28:31AM -0800, Junio C Hamano wrote:
Show 29 quoted lines
> Matthieu Moy <Matthieu.Moy@imag.fr> writes:
> 
> > The configuration of the install directory is not reused from the
> > toplevel Makefile: we assume Git is already built, hence just call
> > "git --exec-path". This avoids too much surgery in the toplevel Makefile.
> >
> > git-remote-mediawiki.perl can now "use Git;".
> >
> > Signed-off-by: Matthieu Moy <Matthieu.Moy@imag.fr>
> > ---
> 
> Continuing to the comment on 3/4, I wonder if it would be a lot
> simpler and more maintainable if you replaced 1/4 to 3/4 with a
> smaller patch to the top-level Makefile to teach it to munge
> arbitrary path/to/foo.perl to path/to/foo the same way as we do to
> other path/tool.perl that are known to the top-level Makefile
> (similarly, another target to install the resulting path/to/foo at
> an arbitrary place).  Then do something like
> 
> 	all::
> 		$(MAKE) -C ../.. \
> 			PERL_SCRIPT=contrib/mw-to-git/git-remote-mediawiki.perl \
>                         build-perl-script
> 	install::
> 		$(MAKE) -C ../.. \
> 			PERL_SCRIPT=contrib/mw-to-git/git-remote-mediawiki.perl \
>                         install-perl-script
> 
> in this step.

That seems much cleaner to me. If done right, it could also let people put:

  CONTRIB_PERL += contrib/mw-to-git/git-remote-mediawiki

or similar into their config.mak, and just get specific contrib bits built and installed along with the rest of git.

-Peff
Matthieu Moy· Feb 8, 2013, 17:34 UTC · re: Jeff King · lore

Re: [PATCH 4/4] git-remote-mediawiki: use Git's Makefile to build the script

Jeff King <peff@peff.net> writes:
> That seems much cleaner to me. If done right, it could also let people
> put:
>
>   CONTRIB_PERL += contrib/mw-to-git/git-remote-mediawiki
Actually, you can already do this:
  SCRIPT_PERL += contrib/mw-to-git/git-remote-mediawiki.perl
probably not by design, but it works!
-- 
Matthieu Moy
http://www-verimag.imag.fr/~moy/
Jeff King· Feb 8, 2013, 17:43 UTC · re: Matthieu Moy · lore

Re: [PATCH 4/4] git-remote-mediawiki: use Git's Makefile to build the script

On Fri, Feb 08, 2013 at 06:34:37PM +0100, Matthieu Moy wrote:
Show 12 quoted lines
> Jeff King <peff@peff.net> writes:
> 
> > That seems much cleaner to me. If done right, it could also let people
> > put:
> >
> >   CONTRIB_PERL += contrib/mw-to-git/git-remote-mediawiki
> 
> Actually, you can already do this:
> 
>   SCRIPT_PERL += contrib/mw-to-git/git-remote-mediawiki.perl
> 
> probably not by design, but it works!
So putting:
  ROOT=contrib/mw-to-git
  git-remote-mediawiki: FORCE
          @make -C ../.. SCRIPT_PERL=$(ROOT)/$@.perl $(ROOT)/$@
in contrib/mw-to-git/Makefile would already work? Neat.
-Peff
Junio C Hamano· Feb 8, 2013, 18:13 UTC · re: Jeff King · lore

Re: [PATCH 4/4] git-remote-mediawiki: use Git's Makefile to build the script

Jeff King <peff@peff.net> writes:
Show 22 quoted lines
> On Fri, Feb 08, 2013 at 06:34:37PM +0100, Matthieu Moy wrote:
>
>> Jeff King <peff@peff.net> writes:
>> 
>> > That seems much cleaner to me. If done right, it could also let people
>> > put:
>> >
>> >   CONTRIB_PERL += contrib/mw-to-git/git-remote-mediawiki
>> 
>> Actually, you can already do this:
>> 
>>   SCRIPT_PERL += contrib/mw-to-git/git-remote-mediawiki.perl
>> 
>> probably not by design, but it works!
>
> So putting:
>
>   ROOT=contrib/mw-to-git
>   git-remote-mediawiki: FORCE
>           @make -C ../.. SCRIPT_PERL=$(ROOT)/$@.perl $(ROOT)/$@
>
> in contrib/mw-to-git/Makefile would already work? Neat.
That essentially is what [v2 2/2] does, no?
Jeff King· Feb 8, 2013, 18:15 UTC · re: Junio C Hamano · lore

Re: [PATCH 4/4] git-remote-mediawiki: use Git's Makefile to build the script

On Fri, Feb 08, 2013 at 10:13:23AM -0800, Junio C Hamano wrote:
Show 26 quoted lines
> Jeff King <peff@peff.net> writes:
> 
> > On Fri, Feb 08, 2013 at 06:34:37PM +0100, Matthieu Moy wrote:
> >
> >> Jeff King <peff@peff.net> writes:
> >> 
> >> > That seems much cleaner to me. If done right, it could also let people
> >> > put:
> >> >
> >> >   CONTRIB_PERL += contrib/mw-to-git/git-remote-mediawiki
> >> 
> >> Actually, you can already do this:
> >> 
> >>   SCRIPT_PERL += contrib/mw-to-git/git-remote-mediawiki.perl
> >> 
> >> probably not by design, but it works!
> >
> > So putting:
> >
> >   ROOT=contrib/mw-to-git
> >   git-remote-mediawiki: FORCE
> >           @make -C ../.. SCRIPT_PERL=$(ROOT)/$@.perl $(ROOT)/$@
> >
> > in contrib/mw-to-git/Makefile would already work? Neat.
> 
> That essentially is what [v2 2/2] does, no?

Yes (this one was cc'd to me, but the others were not, so I read it in isolation). I think Matthieu's series is nicer than just that, though, because it handles the single-file case installation, too, which requires more support from the parent Makefile.

-Peff
Jeff King· Feb 6, 2013, 21:57 UTC · re: Ted Zlatanov · lore

Re: [PATCH] git-send-email: add ~/.authinfo parsing

On Wed, Feb 06, 2013 at 10:58:13AM -0500, Ted Zlatanov wrote:
Show 8 quoted lines
> MM> I don't know about the netrc credential helper, but I guess that's
> MM> another layer. The git-remote-mediawiki code is the code to call the
> MM> credential C API, that in turn may (or may not) call a credential
> MM> helper.
> 
> Yup.  But what you call "read" and "write" are, to the credential
> helper, "write" and "read" but it's the same protocol :)  So maybe the
> names should be changed to reflect that, e.g. "query" and "response."

Is that true? As a user of the credential system, git-remote-mediawiki would want to "write" to git-credential, then "read" the response. As a helper, git-credential-netrc would want to "read" the query then "write" the response. The order is different, but the operations should be the same in both cases.

The big difference is that mediawiki would want an additional function to open a pipe to "git credential" and operate on that, whereas the helper will be reading/writing stdio.

-Peff
Ted Zlatanov· Feb 6, 2013, 23:12 UTC · re: Jeff King · lore

Re: [PATCH] git-send-email: add ~/.authinfo parsing

On Wed, 6 Feb 2013 16:57:24 -0500 Jeff King <peff@peff.net> wrote: 

JK> On Wed, Feb 06, 2013 at 10:58:13AM -0500, Ted Zlatanov wrote: MM> I don't know about the netrc credential helper, but I guess that's MM> another layer. The git-remote-mediawiki code is the code to call the MM> credential C API, that in turn may (or may not) call a credential MM> helper.

>> 
>> Yup.  But what you call "read" and "write" are, to the credential
>> helper, "write" and "read" but it's the same protocol :)  So maybe the
>> names should be changed to reflect that, e.g. "query" and "response."

JK> Is that true? As a user of the credential system, git-remote-mediawiki JK> would want to "write" to git-credential, then "read" the response. As a JK> helper, git-credential-netrc would want to "read" the query then JK> "write" the response. The order is different, but the operations should JK> be the same in both cases.

Logically they are different steps (query and response), even though the data protocol is the same. But it's really not a big deal, I know what it means either way.

JK> The big difference is that mediawiki would want an additional function JK> to open a pipe to "git credential" and operate on that, whereas the JK> helper will be reading/writing stdio.

Yup.
Ted
Matthieu Moy· Feb 7, 2013, 07:08 UTC · re: Ted Zlatanov · lore

Re: [PATCH] git-send-email: add ~/.authinfo parsing

Ted Zlatanov <tzz@lifelogs.com> writes:
> Logically they are different steps (query and response), even though the
> data protocol is the same.  But it's really not a big deal, I know what
> it means either way.

Yes, but if you rename write() to query(), then on the helper side, you'll have to call query() to send the response, and response() to read the query. Much worse than keeping read/write.

Plus, read/write has already been used for a while in the C API, so I'd rather keep the same names for the Perl equivalent.

-- 
Matthieu Moy
http://www-verimag.imag.fr/~moy/
Ted Zlatanov· Feb 7, 2013, 14:30 UTC · re: Matthieu Moy · lore

Re: [PATCH] git-send-email: add ~/.authinfo parsing

On Thu, 07 Feb 2013 08:08:59 +0100 Matthieu Moy <Matthieu.Moy@grenoble-inp.fr> wrote: 

MM> Plus, read/write has already been used for a while in the C API, so I'd MM> rather keep the same names for the Perl equivalent.

That makes perfect sense.
Ted
Michal Nazarewicz· Feb 6, 2013, 13:26 UTC · re: Junio C Hamano · lore

Re: [PATCH] git-send-email: add ~/.authinfo parsing

On Wed, Feb 06 2013, Junio C Hamano <gitster@pobox.com> wrote:
> I see a lot of rerolls on the credential helper front, but is there
> anybody working on hooking send-email to the credential framework?

I assumed someone had, but if not I can take a stab at it. I'm not sure however how should I map server, server-port, and user to credential key-value pairs. I'm leaning towards

	protocol=smtp
	host=<smtp-server>:<smtp-port>
	user=<user>

and than netrc/authinfo helper splitting host to host name and port number, unless port is not in host in which case protocol is assumed as port.

-- 
Best regards,                                         _     _
.o. | Liege of Serenely Enlightened Majesty of      o' \,=./ `o
..o | Computer Science,  Michał “mina86” Nazarewicz    (o o)
ooo +----<email/xmpp: mpn@google.com>--------------ooO--(_)--Ooo--
Ted Zlatanov· Feb 6, 2013, 14:47 UTC · re: Michal Nazarewicz · lore

Re: [PATCH] git-send-email: add ~/.authinfo parsing

On Wed, 06 Feb 2013 14:26:46 +0100 Michal Nazarewicz <mina86@mina86.com> wrote: 
MN> On Wed, Feb 06 2013, Junio C Hamano <gitster@pobox.com> wrote:
>> I see a lot of rerolls on the credential helper front, but is there
>> anybody working on hooking send-email to the credential framework?

MN> I assumed someone had, but if not I can take a stab at it. I'm not sure MN> however how should I map server, server-port, and user to credential MN> key-value pairs. I'm leaning towards

MN> protocol=smtp MN> host=<smtp-server>:<smtp-port> MN> user=<user>

MN> and than netrc/authinfo helper splitting host to host name and port MN> number, unless port is not in host in which case protocol is assumed as MN> port.

That would work (with my PATCHv6 of the netrc credential helper) as follows:

1) just host
host=H
maps to
machine H login Y password Z
2) host + protocol smtp

host=H protocol=smtp

maps to any of:

machine H port smtp login Y password Z machine H protocol smtp login Y password Z

3) host:port + protocol smtp

host=H:25 protocol=smtp

maps to any of:

machine H port 25 protocol smtp login Y password Z machine H:25 port smtp login Y password Z machine H:25 protocol smtp login Y password Z

That's my understanding of what we discussed with Peff and Junio about token mapping. Note we don't split the input host, but instead say "if token 'port' is numeric, append it to the host token" on the netrc side.

Does that sound reasonable? If yes, I can add it to the testing Makefile for the netrc credential helper, to make sure it's clearly stated and tested.

Ted
Ted Zlatanov· Jan 30, 2013, 15:03 UTC · re: Junio C Hamano · lore

Re: [PATCH] git-send-email: add ~/.authinfo parsing

On Tue, 29 Jan 2013 11:53:19 -0800 Junio C Hamano <gitster@pobox.com> wrote: 
JCH> Makes one wonder why .authinfo and not .netrc; 
JCH> http://www.gnu.org/software/emacs/manual/html_node/auth/Help-for-users.html
JCH> phrases it amusingly:

JCH> “Netrc” files are usually called .authinfo or .netr JCH> nowadays .authinfo seems to be more popular and the JCH> auth-source library encourages this confusion by accepting JCH> both

I wrote this and the auth-source.el library in Emacs (I'm glad it was amusing :). The confusion is further perpetuated by our (in Emacs) encouragement to use a .authinfo.gpg file, which is then decrypted on the fly by Emacs through GPG. The format is the same; by the time auth-source.el sees the contents, they are plain text since the decoding happens at the file handler level.

I think it makes sense to write the code to support both `git-send-email' and credentials. I have had it in my TODO list for almost 2 years now to work on credential support, and to support the ~/.authinfo.gpg decoding specifically. Ideally this would also support the other formats... Michal, would you be interested in that feature? I promise to get off my rear and help out.

Show 6 quoted lines
>> +The '~/.authinfo' file is read if Text::CSV Perl module is installed
>> +on the system; if it's missing, a notification message will be printed
>> +and the file ignored altogether.  The file should contain a line with
>> +the following format:
>> ++
>> +  machine <domain> port <port> login <user> password <pass>

JCH> It is rather strange to require a comma-separated-values parser to JCH> read a file format this simple, isn't it?

I'd recommend a hand-crafted parser. Among other things, you should accept both "strings" and 'strings' if possible (I've seen both formats in the wild), and the format is simple enough to avoid the module dependency.

>> ++
>> +Contrary to other tools, 'git-send-email' does not support symbolic
>> +port names like 'imap' thus `<port>` must be a number.

JCH> Perhaps you can convert at least some popular ones yourself? After JCH> all, the user may be using an _existing_ .authinfo/.netrc that she JCH> has been using with other programs that do understand symbolic port JCH> names. Rather than forcing all such users to update their files, JCH> the patch can work a bit harder for them and the world will be a JCH> better place, no?

I agree, "port imap" is a nice self-documenting token. Maybe it can be interpreted by the program that requests the token with a services lookup, where supported.

Ted

← back to recent threads