{"thread":{"id":"47064","subject":"[PATCH] remote-mediawiki: limit filenames to legal","startedAt":"2017-10-29T16:37:27Z","lastAt":"2017-10-30T12:31:15Z","messageCount":6,"participants":["Antoine Beaupré","Matthieu Moy"],"isPatch":true,"patchVersion":1,"patchTotal":null},"messages":[{"id":"331279","messageId":"20171029163714.4818-1-anarcat@debian.org","threadId":"47064","inReplyTo":null,"subject":"[PATCH] remote-mediawiki: limit filenames to legal","fromName":"Antoine Beaupré","fromEmail":"anarcat@debian.org","sentAt":"2017-10-29T16:37:14Z","receivedAt":"2017-10-29T16:37:27Z","isPatch":true,"sender":{"key":"anarcat@debian.org","avatar":"https://avatars.githubusercontent.com/u/796623?v=4"},"body":"mediawiki pages can have names longer than NAME_MAX (generally 255)\ncharacters, which will fail on checkout. we simply strip out extra\ncharacters, which may mean one page's content will overwrite another\n(the last editing winning).\n\nideally, we would do a more clever system to find unique names, but\nthat would be more difficult and error prone for a situation that\nshould rarely happen in the first place.\n---\n contrib/mw-to-git/git-remote-mediawiki.perl | 3 ++-\n 1 file changed, 2 insertions(+), 1 deletion(-)\n\ndiff --git a/contrib/mw-to-git/git-remote-mediawiki.perl b/contrib/mw-to-git/git-remote-mediawiki.perl\nindex e7f857c1a..58870d197 100755\n--- a/contrib/mw-to-git/git-remote-mediawiki.perl\n+++ b/contrib/mw-to-git/git-remote-mediawiki.perl\n@@ -18,6 +18,7 @@ use Git::Mediawiki qw(clean_filename smudge_filename connect_maybe\n \t\t\t\t\tEMPTY HTTP_CODE_OK);\n use DateTime::Format::ISO8601;\n use warnings;\n+use POSIX;\n \n # By default, use UTF-8 to communicate with Git and the user\n binmode STDERR, ':encoding(UTF-8)';\n@@ -703,7 +704,7 @@ sub import_file_revision {\n \t\t%mediafile = %{$mediafile};\n \t}\n \n-\tmy $title = $commit{title};\n+\tmy $title = substr($commit{title}, 0, NAME_MAX);\n \tmy $comment = $commit{comment};\n \tmy $content = $commit{content};\n \tmy $author = $commit{author};\n-- \n2.11.0\n\n"},{"id":"331285","messageId":"20171029181034.27707-1-anarcat@debian.org","threadId":"47064","inReplyTo":"20171029163714.4818-1-anarcat@debian.org","subject":"[PATCH v2] remote-mediawiki: limit filenames to legal","fromName":"Antoine Beaupré","fromEmail":"anarcat@debian.org","sentAt":"2017-10-29T18:10:34Z","receivedAt":"2017-10-29T18:13:55Z","isPatch":true,"sender":{"key":"anarcat@debian.org","avatar":"https://avatars.githubusercontent.com/u/796623?v=4"},"body":"mediawiki pages can have names longer than NAME_MAX (generally 255)\ncharacters, which will fail on checkout. we simply strip out extra\ncharacters, which may mean one page's content will overwrite another\n(the last editing winning).\n\nideally, we would do a more clever system to find unique names, but\nthat would be more difficult and error prone for a situation that\nshould rarely happen in the first place.\n\nSigned-off-by: Antoine Beaupré \n---\n contrib/mw-to-git/Git/Mediawiki.pm | 3 ++-\n 1 file changed, 2 insertions(+), 1 deletion(-)\n\ndiff --git a/contrib/mw-to-git/Git/Mediawiki.pm b/contrib/mw-to-git/Git/Mediawiki.pm\nindex d13c4dfa7..c9f22680a 100644\n--- a/contrib/mw-to-git/Git/Mediawiki.pm\n+++ b/contrib/mw-to-git/Git/Mediawiki.pm\n@@ -2,6 +2,7 @@ package Git::Mediawiki;\n \n use 5.008;\n use strict;\n+use POSIX;\n use Git;\n \n BEGIN {\n@@ -52,7 +53,7 @@ sub smudge_filename {\n \t$filename =~ s/ /_/g;\n \t# Decode forbidden characters encoded in clean_filename\n \t$filename =~ s/_%_([0-9a-fA-F][0-9a-fA-F])/sprintf('%c', hex($1))/ge;\n-\treturn $filename;\n+\treturn substr($filename, 0, NAME_MAX-3);\n }\n \n sub connect_maybe {\n-- \n2.11.0\n\n"},{"id":"331286","messageId":"20171029181511.29148-1-anarcat@debian.org","threadId":"47064","inReplyTo":"20171029163714.4818-1-anarcat@debian.org","subject":"","fromName":"Antoine Beaupré","fromEmail":"anarcat@debian.org","sentAt":"2017-10-29T18:15:10Z","receivedAt":"2017-10-29T18:15:24Z","isPatch":false,"sender":{"key":"anarcat@debian.org","avatar":"https://avatars.githubusercontent.com/u/796623?v=4"},"body":"\nsorry for the noise here, but the original patch didn't fix the length\nin the right place. v2 fixed it in the library properly, but i forgot\nto also include the length of the suffix. this should be good to go...\n"},{"id":"331287","messageId":"20171029181511.29148-2-anarcat@debian.org","threadId":"47064","inReplyTo":"20171029181511.29148-1-anarcat@debian.org","subject":"[PATCH v3] remote-mediawiki: limit filenames to legal","fromName":"Antoine Beaupré","fromEmail":"anarcat@debian.org","sentAt":"2017-10-29T18:15:11Z","receivedAt":"2017-10-29T18:15:27Z","isPatch":true,"sender":{"key":"anarcat@debian.org","avatar":"https://avatars.githubusercontent.com/u/796623?v=4"},"body":"mediawiki pages can have names longer than NAME_MAX (generally 255)\ncharacters, which will fail on checkout. we simply strip out extra\ncharacters, which may mean one page's content will overwrite another\n(the last editing winning).\n\nideally, we would do a more clever system to find unique names, but\nthat would be more difficult and error prone for a situation that\nshould rarely happen in the first place.\n\nSigned-off-by: Antoine Beaupré \n---\n contrib/mw-to-git/Git/Mediawiki.pm | 3 ++-\n 1 file changed, 2 insertions(+), 1 deletion(-)\n\ndiff --git a/contrib/mw-to-git/Git/Mediawiki.pm b/contrib/mw-to-git/Git/Mediawiki.pm\nindex d13c4dfa7..917d9e2d3 100644\n--- a/contrib/mw-to-git/Git/Mediawiki.pm\n+++ b/contrib/mw-to-git/Git/Mediawiki.pm\n@@ -2,6 +2,7 @@ package Git::Mediawiki;\n \n use 5.008;\n use strict;\n+use POSIX;\n use Git;\n \n BEGIN {\n@@ -52,7 +53,7 @@ sub smudge_filename {\n \t$filename =~ s/ /_/g;\n \t# Decode forbidden characters encoded in clean_filename\n \t$filename =~ s/_%_([0-9a-fA-F][0-9a-fA-F])/sprintf('%c', hex($1))/ge;\n-\treturn $filename;\n+\treturn substr($filename, 0, NAME_MAX-length('.mw'));\n }\n \n sub connect_maybe {\n-- \n2.11.0\n\n"},{"id":"331338","messageId":"q7h9fua0zzz0.fsf@orange.lip.ens-lyon.fr","threadId":"47064","inReplyTo":"20171029181034.27707-1-anarcat@debian.org","subject":"Re: [PATCH v2] remote-mediawiki: limit filenames to legal","fromName":"Matthieu Moy","fromEmail":"matthieu.moy@univ-lyon1.fr","sentAt":"2017-10-30T10:34:11Z","receivedAt":"2017-10-30T10:34:21Z","isPatch":true,"sender":{"key":"matthieu.moy@univ-lyon1.fr","avatar":"https://gravatar.com/avatar/8ab83b763226bd297b59ddd1463a8bfc852924577191233f73a953b86888fe0c?d=mp&s=160"},"body":"Antoine Beaupré writes:\n\n> @@ -52,7 +53,7 @@ sub smudge_filename {\n> \t$filename =~ s/ /_/g;\n> \t# Decode forbidden characters encoded in clean_filename\n> \t$filename =~ s/_%_([0-9a-fA-F][0-9a-fA-F])/sprintf('%c', hex($1))/ge;\n> -\treturn $filename;\n> +\treturn substr($filename, 0, NAME_MAX-3);\n\nThere's a request to allow a configurable extension (.mediawiki would\nhelp importing in some wikis, see\nhttps://github.com/Git-Mediawiki/Git-Mediawiki/issues/42). You should at\nleast make this stg like length(\".mw\") so that the next search&replace\nfor \".mw\" finds this.\n\nAlso, note that your solution works for using Git-Mediawiki in a\nread-only way, but if you start modifying and pushing such files, you'll\nget into trouble. It probably makes sense to issue a warnign in such\ncase.\n\nRegards,\n\n-- \nMatthieu Moy\nhttps://matthieu-moy.fr/\n"},{"id":"331345","messageId":"87h8ug7r75.fsf@curie.anarc.at","threadId":"47064","inReplyTo":"q7h9fua0zzz0.fsf@orange.lip.ens-lyon.fr","subject":"Re: [PATCH v2] remote-mediawiki: limit filenames to legal","fromName":"Antoine Beaupré","fromEmail":"anarcat@debian.org","sentAt":"2017-10-30T12:31:10Z","receivedAt":"2017-10-30T12:31:15Z","isPatch":true,"sender":{"key":"anarcat@debian.org","avatar":"https://avatars.githubusercontent.com/u/796623?v=4"},"body":"On 2017-10-30 11:34:11, Matthieu Moy wrote:\n> Antoine Beaupré writes:\n>\n>> @@ -52,7 +53,7 @@ sub smudge_filename {\n>> \t$filename =~ s/ /_/g;\n>> \t# Decode forbidden characters encoded in clean_filename\n>> \t$filename =~ s/_%_([0-9a-fA-F][0-9a-fA-F])/sprintf('%c', hex($1))/ge;\n>> -\treturn $filename;\n>> +\treturn substr($filename, 0, NAME_MAX-3);\n>\n> There's a request to allow a configurable extension (.mediawiki would\n> help importing in some wikis, see\n> https://github.com/Git-Mediawiki/Git-Mediawiki/issues/42). You should at\n> least make this stg like length(\".mw\") so that the next search&replace\n> for \".mw\" finds this.\n\nI believe I did that in v3.\n\n> Also, note that your solution works for using Git-Mediawiki in a\n> read-only way, but if you start modifying and pushing such files, you'll\n> get into trouble. It probably makes sense to issue a warnign in such\n> case.\n\nTrue. I didn't consider that, but then again the patch is not a\nregression: you couldn't have pushed those repos in the first place\nanyways...\n\nA.\n\n-- \nThe history of any one part of the earth, like the life of a soldier,\nconsists of long periods of boredom and short periods of terror.\n - British geologist Derek V. Ager\n"}]}