{"thread":{"id":"30954","subject":"[PATCH 00/12] git-remote-mediawiki: tests and optimizations","startedAt":"2012-07-05T07:36:00Z","lastAt":"2012-07-06T10:03:15Z","messageCount":30,"participants":["Matthieu Moy","Junio C Hamano"],"isPatch":true,"patchVersion":1,"patchTotal":12},"messages":[{"id":"194634","messageId":"1341473772-28023-1-git-send-email-Matthieu.Moy@imag.fr","threadId":"30954","inReplyTo":null,"subject":"[PATCH 00/12] git-remote-mediawiki: tests and optimizations","fromName":"Matthieu Moy","fromEmail":"matthieu.moy@imag.fr","sentAt":"2012-07-05T07:36:00Z","receivedAt":"2012-07-05T07:36:00Z","isPatch":true,"sender":{"key":"git@matthieu-moy.fr","avatar":"https://avatars.githubusercontent.com/u/14709?v=4"},"body":"This patch serie starts with the implementation of an automated\ntestsuite for git-remote-mediawiki. It was already discussed here:\n\n http://thread.gmane.org/gmane.comp.version-control.git/199718/focus=199905\n\nbut has been reworked a bit (no huge change in behavior, but a lot of\nstyle fixes). The first two patches implement the infrastructure, the\nnext few ones provide a set of testcases for existing features.\n\nOn top of that, a new fetch strategy is implemented, that is much\nfaster than the existing on wikis with little activity and many pages.\nThe existing tests for push and pull are reused to be tested with this\nnew strategy. This part can be split in a separate patch serie if\nneeded.\n\nGuillaume Sasdy (4):\n git-remote-mediawiki: scripts to install, delete and clear a\n MediaWiki\n git-remote-mediawiki: test environment of git-remote-mediawiki\n git-remote-mediawiki (t9360): test git-remote-mediawiki clone\n git-remote-mediawiki (t9361): test git-remote-mediawiki pull and push\n\nMatthieu Moy (6):\n git-remote-mediawiki: support for uploading file in test environment\n git-remote-mediawiki: change return type of get_mw_pages\n git-remote-mediawiki: refactor loop over revision ids\n git-remote-mediawiki: extract revision-importing loop to a function\n git-remote-mediawiki: more efficient 'pull' in the best case\n git-remote-mediawiki: be more defensive when requests fail\n\nNGUYEN Kim Thuat (1):\n git-remote-mediawiki (t9363): test 'File:' import and export\n\nSimon Cathebras (1):\n git-remote-mediawiki (t9362): test git-remote-mediawiki with UTF8\n characters\n\n contrib/mw-to-git/Makefile | 48 +++\n contrib/mw-to-git/git-remote-mediawiki | 151 +++++--\n contrib/mw-to-git/t/.gitignore | 4 +\n contrib/mw-to-git/t/Makefile | 32 ++\n contrib/mw-to-git/t/README | 124 ++++++\n contrib/mw-to-git/t/install-wiki.sh | 45 +++\n contrib/mw-to-git/t/install-wiki/.gitignore | 1 +\n contrib/mw-to-git/t/install-wiki/LocalSettings.php | 129 ++++++\n contrib/mw-to-git/t/install-wiki/db_install.php | 120 ++++++\n contrib/mw-to-git/t/push-pull-tests.sh | 144 +++++++\n contrib/mw-to-git/t/t9360-mw-to-git-clone.sh | 257 ++++++++++++\n contrib/mw-to-git/t/t9361-mw-to-git-push-pull.sh | 24 ++\n contrib/mw-to-git/t/t9362-mw-to-git-utf8.sh | 301 ++++++++++++++\n .../mw-to-git/t/t9363-mw-to-git-export-import.sh | 198 ++++++++++\n contrib/mw-to-git/t/t9364-pull-by-rev.sh | 17 +\n contrib/mw-to-git/t/test-gitmw-lib.sh | 435 +++++++++++++++++++++\n contrib/mw-to-git/t/test-gitmw.pl | 225 +++++++++++\n contrib/mw-to-git/t/test.config | 35 ++\n 18 files changed, 2263 insertions(+), 27 deletions(-)\n create mode 100644 contrib/mw-to-git/Makefile\n create mode 100644 contrib/mw-to-git/t/.gitignore\n create mode 100644 contrib/mw-to-git/t/Makefile\n create mode 100644 contrib/mw-to-git/t/README\n create mode 100755 contrib/mw-to-git/t/install-wiki.sh\n create mode 100644 contrib/mw-to-git/t/install-wiki/.gitignore\n create mode 100644 contrib/mw-to-git/t/install-wiki/LocalSettings.php\n create mode 100644 contrib/mw-to-git/t/install-wiki/db_install.php\n create mode 100644 contrib/mw-to-git/t/push-pull-tests.sh\n create mode 100755 contrib/mw-to-git/t/t9360-mw-to-git-clone.sh\n create mode 100755 contrib/mw-to-git/t/t9361-mw-to-git-push-pull.sh\n create mode 100755 contrib/mw-to-git/t/t9362-mw-to-git-utf8.sh\n create mode 100755 contrib/mw-to-git/t/t9363-mw-to-git-export-import.sh\n create mode 100755 contrib/mw-to-git/t/t9364-pull-by-rev.sh\n create mode 100755 contrib/mw-to-git/t/test-gitmw-lib.sh\n create mode 100755 contrib/mw-to-git/t/test-gitmw.pl\n create mode 100644 contrib/mw-to-git/t/test.config\n\n-- \n1.7.11.1.147.g47a574d\n"},{"id":"194635","messageId":"1341473772-28023-2-git-send-email-Matthieu.Moy@imag.fr","threadId":"30954","inReplyTo":"1341473772-28023-1-git-send-email-Matthieu.Moy@imag.fr","subject":"[PATCH 01/12] git-remote-mediawiki: scripts to install, delete and clear a MediaWiki","fromName":"Matthieu Moy","fromEmail":"matthieu.moy@imag.fr","sentAt":"2012-07-05T07:36:01Z","receivedAt":"2012-07-05T07:36:01Z","isPatch":true,"sender":{"key":"git@matthieu-moy.fr","avatar":"https://avatars.githubusercontent.com/u/14709?v=4"},"body":"From: Guillaume Sasdy \n\ninstall_wiki.sh allows the user to install a MediaWiki instance in a\nsingle shell command. Like \"git instaweb\", it configures and launches\nlighttpd without requiring root priviledges. To simplify database\nmanagement, it uses SQLite, which doesn't require a running daemon, and\nallows reseting the database by simply replacing a single file. This\nallows install_wiki to also defines a function wiki_reset which clear all\ncontent of the previously created wiki, which will be very useful to run\nseveral indepenant tests on the same wiki.\n\nNote those functionnalities are made to be used from the user command\nline in the directory git/contrib/mw-to-git/t/\n\nSigned-off-by: Simon CATHEBRAS \nSigned-off-by: Julien KHAYAT \nSigned-off-by: Simon Perrat \nSigned-off-by: Charles ROUSSEL \nSigned-off-by: Guillaume SASDY \nSigned-off-by: Matthieu Moy \n---\n contrib/mw-to-git/t/.gitignore | 2 +\n contrib/mw-to-git/t/install-wiki.sh | 45 ++++\n contrib/mw-to-git/t/install-wiki/.gitignore | 1 +\n contrib/mw-to-git/t/install-wiki/LocalSettings.php | 128 +++++++++\n contrib/mw-to-git/t/install-wiki/db_install.php | 120 +++++++++\n contrib/mw-to-git/t/test-gitmw-lib.sh | 289 +++++++++++++++++++++\n contrib/mw-to-git/t/test.config | 35 +++\n 7 files changed, 620 insertions(+)\n create mode 100644 contrib/mw-to-git/t/.gitignore\n create mode 100755 contrib/mw-to-git/t/install-wiki.sh\n create mode 100644 contrib/mw-to-git/t/install-wiki/.gitignore\n create mode 100644 contrib/mw-to-git/t/install-wiki/LocalSettings.php\n create mode 100644 contrib/mw-to-git/t/install-wiki/db_install.php\n create mode 100755 contrib/mw-to-git/t/test-gitmw-lib.sh\n create mode 100644 contrib/mw-to-git/t/test.config\n\ndiff --git a/contrib/mw-to-git/t/.gitignore b/contrib/mw-to-git/t/.gitignore\nnew file mode 100644\nindex 0000000..a060414\n--- /dev/null\n+++ b/contrib/mw-to-git/t/.gitignore\n@@ -0,0 +1,2 @@\n+WEB/\n+wiki/\ndiff --git a/contrib/mw-to-git/t/install-wiki.sh b/contrib/mw-to-git/t/install-wiki.sh\nnew file mode 100755\nindex 0000000..c6d6fa3\n--- /dev/null\n+++ b/contrib/mw-to-git/t/install-wiki.sh\n@@ -0,0 +1,45 @@\n+#!/bin/sh\n+\n+# This script installs or deletes a MediaWiki on your computer.\n+# It requires a web server with PHP and SQLite running. In addition, if you\n+# do not have MediaWiki sources on your computer, the option 'install'\n+# downloads them for you.\n+# Please set the CONFIGURATION VARIABLES in ./test-gitmw-lib.sh\n+\n+WIKI_TEST_DIR=$(cd \"$(dirname \"$0\")\" && pwd)\n+\n+if test -z \"$WIKI_TEST_DIR\"\n+then\n+\tWIKI_TEST_DIR=.\n+fi\n+\n+. \"$WIKI_TEST_DIR\"/test-gitmw-lib.sh\n+usage () {\n+\techo \"Usage: \"\n+\techo \"\t./install-wiki.sh \"\n+\techo \"\t\tinstall | -i :\tInstall a wiki on your computer.\"\n+\techo \"\t\tdelete | -d : Delete the wiki and all its pages and \"\n+\techo \"\t\t\tcontent.\"\n+}\n+\n+\n+# Argument: install, delete, --help | -h\n+case \"$1\" in\n+\t\"install\" | \"-i\")\n+\t\twiki_install\n+\t\texit 0\n+\t\t;;\n+\t\"delete\" | \"-d\")\n+\t\twiki_delete\n+\t\texit 0\n+\t\t;;\n+\t\"--help\" | \"-h\")\n+\t\tusage\n+\t\texit 0\n+\t\t;;\n+\t*)\n+\t\techo \"Invalid argument: $1\"\n+\t\tusage\n+\t\texit 1\n+\t\t;;\n+esac\ndiff --git a/contrib/mw-to-git/t/install-wiki/.gitignore b/contrib/mw-to-git/t/install-wiki/.gitignore\nnew file mode 100644\nindex 0000000..b5a2a44\n--- /dev/null\n+++ b/contrib/mw-to-git/t/install-wiki/.gitignore\n@@ -0,0 +1 @@\n+wikidb.sqlite\ndiff --git a/contrib/mw-to-git/t/install-wiki/LocalSettings.php b/contrib/mw-to-git/t/install-wiki/LocalSettings.php\nnew file mode 100644\nindex 0000000..aa72518\n--- /dev/null\n+++ b/contrib/mw-to-git/t/install-wiki/LocalSettings.php\n@@ -0,0 +1,128 @@\n+ and \n+ *\n+ */\n+$argc = $_SERVER['argc'];\n+$argv = $_SERVER['argv'];\n+\n+$login = $argv[2];\n+$pass = $argv[3];\n+$tmp = $argv[4];\n+$port = $argv[5];\n+\n+$url = 'http://localhost:'.$port.'/wiki/mw-config/index.php';\n+$db_dir = urlencode($tmp);\n+$tmp_cookie = tempnam($tmp, \"COOKIE_\");\n+/*\n+ * Fetchs a page with cURL.\n+ */\n+function get($page_name = \"\") {\n+\t$curl = curl_init();\n+\t$page_name_add = \"\";\n+\tif ($page_name != \"\") {\n+\t\t$page_name_add = '?page='.$page_name;\n+\t}\n+\t$url = $GLOBALS['url'].$page_name_add;\n+\t$tmp_cookie = $GLOBALS['tmp_cookie'];\n+\tcurl_setopt($curl, CURLOPT_COOKIEJAR, $tmp_cookie);\n+\tcurl_setopt($curl, CURLOPT_RETURNTRANSFER, true);\n+\tcurl_setopt($curl, CURLOPT_FOLLOWLOCATION, true);\n+\tcurl_setopt($curl, CURLOPT_COOKIEFILE, $tmp_cookie);\n+\tcurl_setopt($curl, CURLOPT_HEADER, true);\n+\tcurl_setopt($curl, CURLOPT_URL, $url);\n+\n+\t$page = curl_exec($curl);\n+\tif (!$page) {\n+\t\tdie(\"Could not get page: $url\\n\");\n+\t}\n+\tcurl_close($curl);\n+\treturn $page;\n+}\n+\n+/*\n+ * Submits a form with cURL.\n+ */\n+function submit($page_name, $option = \"\") {\n+\t$curl = curl_init();\n+\t$datapost = 'submit-continue=Continue+%E2%86%92';\n+\tif ($option != \"\") {\n+\t\t$datapost = $option.'&'.$datapost;\n+\t}\n+\t$url = $GLOBALS['url'].'?page='.$page_name;\n+\t$tmp_cookie = $GLOBALS['tmp_cookie'];\n+\tcurl_setopt($curl, CURLOPT_URL, $url);\n+\tcurl_setopt($curl, CURLOPT_POST, true);\n+\tcurl_setopt($curl, CURLOPT_FOLLOWLOCATION, true);\n+\tcurl_setopt($curl, CURLOPT_POSTFIELDS, $datapost);\n+\tcurl_setopt($curl, CURLOPT_RETURNTRANSFER, true);\n+\tcurl_setopt($curl, CURLOPT_COOKIEJAR, $tmp_cookie);\n+\tcurl_setopt($curl, CURLOPT_COOKIEFILE, $tmp_cookie);\n+\n+\t$page = curl_exec($curl);\n+\tif (!$page) {\n+\t\tdie(\"Could not get page: $url\\n\");\n+\t}\n+\tcurl_close($curl);\n+\treturn \"$page\";\n+}\n+\n+/*\n+ * Here starts this script: simulates the behavior of the user\n+ * submitting forms to generates the database file.\n+ * Note this simulation was made for the MediaWiki version 1.19.0,\n+ * we can't assume it works with other versions.\n+ *\n+ */\n+\n+$page = get();\n+if (!preg_match('/input type=\"hidden\" value=\"([0-9]+)\" name=\"LanguageRequestTime\"/',\n+\t\t$page, $matches)) {\n+\techo \"Unexpected content for page downloaded:\\n\";\n+\techo \"$page\";\n+\tdie;\n+};\n+$timestamp = $matches[1];\n+$language = \"LanguageRequestTime=$timestamp&uselang=en&ContLang=en\";\n+$page = submit('Language', $language);\n+\n+submit('Welcome');\n+\n+$db_config = 'DBType=sqlite';\n+$db_config = $db_config.'&sqlite_wgSQLiteDataDir='.$db_dir;\n+$db_config = $db_config.'&sqlite_wgDBname='.$argv[1];\n+submit('DBConnect', $db_config);\n+\n+$wiki_config = 'config_wgSitename=TEST';\n+$wiki_config = $wiki_config.'&config__NamespaceType=site-name';\n+$wiki_config = $wiki_config.'&config_wgMetaNamespace=MyWiki';\n+$wiki_config = $wiki_config.'&config__AdminName='.$login;\n+\n+$wiki_config = $wiki_config.'&config__AdminPassword='.$pass;\n+$wiki_config = $wiki_config.'&config__AdminPassword2='.$pass;\n+\n+$wiki_config = $wiki_config.'&wiki__configEmail=email%40email.org';\n+$wiki_config = $wiki_config.'&config__SkipOptional=skip';\n+submit('Name', $wiki_config);\n+submit('Install');\n+submit('Install');\n+\n+unlink($tmp_cookie);\n+?>\ndiff --git a/contrib/mw-to-git/t/test-gitmw-lib.sh b/contrib/mw-to-git/t/test-gitmw-lib.sh\nnew file mode 100755\nindex 0000000..47341f7\n--- /dev/null\n+++ b/contrib/mw-to-git/t/test-gitmw-lib.sh\n@@ -0,0 +1,289 @@\n+# Copyright (C) 2012\n+# Charles Roussel \n+# Simon Cathebras \n+# Julien Khayat \n+# Guillaume Sasdy \n+# Simon Perrat \n+# License: GPL v2 or later\n+\n+#\n+# CONFIGURATION VARIABLES\n+# You might want to change these ones\n+#\n+\n+. ./test.config\n+\n+CURR_DIR=$(pwd)\n+TEST_OUTPUT_DIRECTORY=$(pwd)\n+TEST_DIRECTORY=\"$CURR_DIR\"/../../../t\n+\n+export TEST_OUTPUT_DIRECTORY TEST_DIRECTORY CURR_DIR\n+\n+if test \"$LIGHTTPD\" = \"false\" ; then\n+\tPORT=80\n+else\n+\tWIKI_DIR_INST=\"$CURR_DIR/$WEB_WWW\"\n+fi\n+\n+# ================= Install part =================\n+\n+error () {\n+\techo \"$@\" >&2\n+\texit 1\n+}\n+\n+# config_lighttpd\n+#\n+# Create the configuration files and the folders necessary to start lighttpd.\n+# Overwrite any existing file.\n+config_lighttpd () {\n+\tmkdir -p $WEB\n+\tmkdir -p $WEB_TMP\n+\tmkdir -p $WEB_WWW\n+\tcat > $WEB/lighttpd.conf <\t\"application/pdf\",\n+\t\".sig\"\t\t=>\t\"application/pgp-signature\",\n+\t\".spl\"\t\t=>\t\"application/futuresplash\",\n+\t\".class\"\t=>\t\"application/octet-stream\",\n+\t\".ps\"\t\t=>\t\"application/postscript\",\n+\t\".torrent\"\t=>\t\"application/x-bittorrent\",\n+\t\".dvi\"\t\t=>\t\"application/x-dvi\",\n+\t\".gz\"\t\t=>\t\"application/x-gzip\",\n+\t\".pac\"\t\t=>\t\"application/x-ns-proxy-autoconfig\",\n+\t\".swf\"\t\t=>\t\"application/x-shockwave-flash\",\n+\t\".tar.gz\"\t=>\t\"application/x-tgz\",\n+\t\".tgz\"\t\t=>\t\"application/x-tgz\",\n+\t\".tar\"\t\t=>\t\"application/x-tar\",\n+\t\".zip\"\t\t=>\t\"application/zip\",\n+\t\".mp3\"\t\t=>\t\"audio/mpeg\",\n+\t\".m3u\"\t\t=>\t\"audio/x-mpegurl\",\n+\t\".wma\"\t\t=>\t\"audio/x-ms-wma\",\n+\t\".wax\"\t\t=>\t\"audio/x-ms-wax\",\n+\t\".ogg\"\t\t=>\t\"application/ogg\",\n+\t\".wav\"\t\t=>\t\"audio/x-wav\",\n+\t\".gif\"\t\t=>\t\"image/gif\",\n+\t\".jpg\"\t\t=>\t\"image/jpeg\",\n+\t\".jpeg\"\t\t=>\t\"image/jpeg\",\n+\t\".png\"\t\t=>\t\"image/png\",\n+\t\".xbm\"\t\t=>\t\"image/x-xbitmap\",\n+\t\".xpm\"\t\t=>\t\"image/x-xpixmap\",\n+\t\".xwd\"\t\t=>\t\"image/x-xwindowdump\",\n+\t\".css\"\t\t=>\t\"text/css\",\n+\t\".html\"\t\t=>\t\"text/html\",\n+\t\".htm\"\t\t=>\t\"text/html\",\n+\t\".js\"\t\t=>\t\"text/javascript\",\n+\t\".asc\"\t\t=>\t\"text/plain\",\n+\t\".c\"\t\t=>\t\"text/plain\",\n+\t\".cpp\"\t\t=>\t\"text/plain\",\n+\t\".log\"\t\t=>\t\"text/plain\",\n+\t\".conf\"\t\t=>\t\"text/plain\",\n+\t\".text\"\t\t=>\t\"text/plain\",\n+\t\".txt\"\t\t=>\t\"text/plain\",\n+\t\".dtd\"\t\t=>\t\"text/xml\",\n+\t\".xml\"\t\t=>\t\"text/xml\",\n+\t\".mpeg\"\t\t=>\t\"video/mpeg\",\n+\t\".mpg\"\t\t=>\t\"video/mpeg\",\n+\t\".mov\"\t\t=>\t\"video/quicktime\",\n+\t\".qt\"\t\t=>\t\"video/quicktime\",\n+\t\".avi\"\t\t=>\t\"video/x-msvideo\",\n+\t\".asf\"\t\t=>\t\"video/x-ms-asf\",\n+\t\".asx\"\t\t=>\t\"video/x-ms-asf\",\n+\t\".wmv\"\t\t=>\t\"video/x-ms-wmv\",\n+\t\".bz2\"\t\t=>\t\"application/x-bzip\",\n+\t\".tbz\"\t\t=>\t\"application/x-bzip-compressed-tar\",\n+\t\".tar.bz2\"\t=>\t\"application/x-bzip-compressed-tar\",\n+\t\"\"\t\t=>\t\"text/plain\"\n+\t)\n+\n+\tfastcgi.server = ( \".php\" =>\n+\t(\"localhost\" =>\n+\t( \"socket\" => \"$CURR_DIR/$WEB_TMP/php.socket\",\n+\t\"bin-path\" => \"$PHP_DIR/php-cgi -c $CURR_DIR/$WEB/php.ini\"\n+\n+\t)\n+\t)\n+\t)\n+EOF\n+\n+\tcat > $WEB/php.ini < \"$file_swap\"\n+\tmv \"$file_swap\" \"$FILES_FOLDER/LocalSettings-tmp.php\"\n+\tsed \"s,@WG_SERVER@,http://$SERVER_ADDR,\" \\\n+\t\t\"$FILES_FOLDER/LocalSettings-tmp.php\" > \"$file_swap\"\n+\tmv \"$file_swap\" \"$FILES_FOLDER/LocalSettings-tmp.php\"\n+\tsed \"s,@WG_SQLITE_DATADIR@,$TMP,\" \\\n+\t\t\"$FILES_FOLDER/LocalSettings-tmp.php\" > \"$file_swap\"\n+\tmv \"$file_swap\" \"$FILES_FOLDER/LocalSettings-tmp.php\"\n+\tsed \"s,@WG_SQLITE_DATAFILE@,$( basename $DB_FILE .sqlite),\" \\\n+\t\t\"$FILES_FOLDER/LocalSettings-tmp.php\" > \"$file_swap\"\n+\tmv \"$file_swap\" \"$FILES_FOLDER/LocalSettings-tmp.php\"\n+\n+\tmv \"$FILES_FOLDER/LocalSettings-tmp.php\" \\\n+\t\t\"$WIKI_DIR_INST/$WIKI_DIR_NAME/LocalSettings.php\" ||\n+\t\terror \"Unable to move $FILES_FOLDER/LocalSettings-tmp.php\" \\\n+\t\t\"in $WIKI_DIR_INST/$WIKI_DIR_NAME\"\n+\techo \"File $FILES_FOLDER/LocalSettings.php is set in\" \\\n+\t\t\" $WIKI_DIR_INST/$WIKI_DIR_NAME\"\n+\n+\techo \"Your wiki has been installed. You can check it at\n+\t\thttp://$SERVER_ADDR/$WIKI_DIR_NAME\"\n+}\n+\n+# Reset the database of the wiki and the password of the admin\n+#\n+# Warning: This function must be called only in a subdirectory of t/ directory\n+wiki_reset () {\n+\t# Copy initial database of the wiki\n+\tif [ ! -f \"../$FILES_FOLDER/$DB_FILE\" ] ; then\n+\t\terror \"Can't find ../$FILES_FOLDER/$DB_FILE in the current folder.\"\n+\tfi\n+\tcp \"../$FILES_FOLDER/$DB_FILE\" \"$TMP\" ||\n+\t\terror \"Can't copy ../$FILES_FOLDER/$DB_FILE in $TMP\"\n+\techo \"File $FILES_FOLDER/$DB_FILE is set in $TMP\"\n+}\n+\n+# Delete the wiki created in the web server's directory and all its content\n+# saved in the database.\n+wiki_delete () {\n+\tif test $LIGHTTPD = \"true\"; then\n+\t\tstop_lighttpd\n+\telse\n+\t\t# Delete the wiki's directory.\n+\t\trm -rf \"$WIKI_DIR_INST/$WIKI_DIR_NAME\" ||\n+\t\t\terror \"Wiki's directory $WIKI_DIR_INST/\" \\\n+\t\t\t\"$WIKI_DIR_NAME could not be deleted\"\n+\t\t# Delete the wiki's SQLite database.\n+\t\trm -f \"$TMP/$DB_FILE\" ||\n+\t\t\terror \"Database $TMP/$DB_FILE could not be deleted.\"\n+\tfi\n+\n+\t# Delete the wiki's SQLite database\n+\trm -f \"$TMP/$DB_FILE\" || error \"Database $TMP/$DB_FILE could not be deleted.\"\n+\trm -f \"$FILES_FOLDER/$DB_FILE\"\n+\trm -rf \"$TMP/$MW_VERSION\"\n+}\ndiff --git a/contrib/mw-to-git/t/test.config b/contrib/mw-to-git/t/test.config\nnew file mode 100644\nindex 0000000..958b37b\n--- /dev/null\n+++ b/contrib/mw-to-git/t/test.config\n@@ -0,0 +1,35 @@\n+# Name of the web server's directory dedicated to the wiki is WIKI_DIR_NAME\n+WIKI_DIR_NAME=wiki\n+\n+# Login and password of the wiki's admin\n+WIKI_ADMIN=WikiAdmin\n+WIKI_PASSW=AdminPass\n+\n+# Address of the web server\n+SERVER_ADDR=localhost\n+\n+# SQLite database of the wiki, named DB_FILE, is located in TMP\n+TMP=/tmp\n+DB_FILE=wikidb.sqlite\n+\n+# If LIGHTTPD is not set to true, the script will use the defaut\n+# web server running in WIKI_DIR_INST.\n+WIKI_DIR_INST=/var/www\n+\n+# If LIGHTTPD is set to true, the script will use Lighttpd to run\n+# the wiki.\n+LIGHTTPD=true\n+\n+# The variables below are useful only if LIGHTTPD is set to true.\n+PORT=1234\n+PHP_DIR=/usr/bin\n+LIGHTTPD_DIR=/usr/sbin\n+WEB=WEB\n+WEB_TMP=$WEB/tmp\n+WEB_WWW=$WEB/www\n+\n+# The variables below are used by the script to install a wiki.\n+# You should not modify these unless you are modifying the script itself.\n+MW_VERSION=mediawiki-1.19.0\n+FILES_FOLDER=install-wiki\n+DB_INSTALL_SCRIPT=db_install.php\n-- \n1.7.11.1.147.g47a574d\n"},{"id":"194645","messageId":"1341473772-28023-3-git-send-email-Matthieu.Moy@imag.fr","threadId":"30954","inReplyTo":"1341473772-28023-1-git-send-email-Matthieu.Moy@imag.fr","subject":"[PATCH 02/12] git-remote-mediawiki: test environment of git-remote-mediawiki","fromName":"Matthieu Moy","fromEmail":"matthieu.moy@imag.fr","sentAt":"2012-07-05T07:36:02Z","receivedAt":"2012-07-05T07:36:02Z","isPatch":true,"sender":{"key":"git@matthieu-moy.fr","avatar":"https://avatars.githubusercontent.com/u/14709?v=4"},"body":"From: Guillaume Sasdy \n\nIn order to test git-remote-mediawiki, a set of functions is needed to\nmanage a MediaWiki: edit a page, remove a page, fetch a page, fetch all\npages on a given wiki.\n\nWe also provide a few helper function to check the content of\ndirectories.\n\nIn addition, this patch provides Makefiles to execute tests.\nSee the README file for more details.\n\nSigned-off-by: Simon CATHEBRAS \nSigned-off-by: Julien KHAYAT \nSigned-off-by: Simon Perrat \nSigned-off-by: Charles ROUSSEL \nSigned-off-by: Guillaume SASDY \nSigned-off-by: Matthieu Moy \n---\n contrib/mw-to-git/Makefile | 48 ++++++++\n contrib/mw-to-git/t/.gitignore | 2 +\n contrib/mw-to-git/t/Makefile | 32 ++++++\n contrib/mw-to-git/t/README | 124 ++++++++++++++++++++\n contrib/mw-to-git/t/test-gitmw-lib.sh | 143 +++++++++++++++++++++++\n contrib/mw-to-git/t/test-gitmw.pl | 209 ++++++++++++++++++++++++++++++++++\n 6 files changed, 558 insertions(+)\n create mode 100644 contrib/mw-to-git/Makefile\n create mode 100644 contrib/mw-to-git/t/Makefile\n create mode 100644 contrib/mw-to-git/t/README\n create mode 100755 contrib/mw-to-git/t/test-gitmw.pl\n\ndiff --git a/contrib/mw-to-git/Makefile b/contrib/mw-to-git/Makefile\nnew file mode 100644\nindex 0000000..c9b1795\n--- /dev/null\n+++ b/contrib/mw-to-git/Makefile\n@@ -0,0 +1,48 @@\n+#\n+# Copyright (C) 2012\n+# Charles Roussel \n+# Simon Cathebras \n+# Julien Khayat \n+# Guillaume Sasdy \n+# Simon Perrat \n+#\n+## Build git-remote-mediawiki\n+\n+-include ../../config.mak.autogen\n+-include ../../config.mak\n+\n+ifndef PERL_PATH\n+\tPERL_PATH = /usr/bin/perl\n+endif\n+ifndef gitexecdir\n+\tgitexecdir = $(shell git --exec-path)\n+endif\n+\n+PERL_PATH_SQ = $(subst ','\\'',$(PERL_PATH))\n+gitexecdir_SQ = $(subst ','\\'',$(gitexecdir))\n+SCRIPT = git-remote-mediawiki\n+\n+.PHONY: install help doc test clean\n+\n+help:\n+\t@echo 'This is the help target of the Makefile. Current configuration:'\n+\t@echo ' gitexecdir = $(gitexecdir_SQ)'\n+\t@echo ' PERL_PATH = $(PERL_PATH_SQ)'\n+\t@echo 'Run \"$(MAKE) install\" to install $(SCRIPT) in gitexecdir'\n+\t@echo 'Run \"$(MAKE) test\" to run the testsuite'\n+\n+install:\n+\tsed -e '1s|#!.*/perl|#!$(PERL_PATH_SQ)|' $(SCRIPT) \\\n+ > '$(gitexecdir_SQ)/$(SCRIPT)'\n+\tchmod +x '$(gitexecdir)/$(SCRIPT)'\n+\n+doc:\n+\t@echo 'Sorry, \"make doc\" is not implemented yet for $(SCRIPT)'\n+\n+test:\n+\t$(MAKE) -C t/ test\n+\n+clean:\n+\t$(RM) '$(gitexecdir)/$(SCRIPT)'\n+\t$(MAKE) -C t/ clean\n+\ndiff --git a/contrib/mw-to-git/t/.gitignore b/contrib/mw-to-git/t/.gitignore\nindex a060414..a7a40b4 100644\n--- a/contrib/mw-to-git/t/.gitignore\n+++ b/contrib/mw-to-git/t/.gitignore\n@@ -1,2 +1,4 @@\n WEB/\n wiki/\n+trash directory.t*/\n+test-results/\ndiff --git a/contrib/mw-to-git/t/Makefile b/contrib/mw-to-git/t/Makefile\nnew file mode 100644\nindex 0000000..1ebe1c7\n--- /dev/null\n+++ b/contrib/mw-to-git/t/Makefile\n@@ -0,0 +1,32 @@\n+#\n+# Copyright (C) 2012\n+# Charles Roussel \n+# Simon Cathebras \n+# Julien Khayat \n+# Guillaume Sasdy \n+# Simon Perrat \n+#\n+## Test git-remote-mediawiki\n+\n+all: test\n+\n+-include ../../../config.mak.autogen\n+-include ../../../config.mak\n+\n+T = $(wildcard t[0-9][0-9][0-9][0-9]-*.sh)\n+\n+.PHONY: help test clean all\n+\n+help:\n+\t@echo 'Run \"$(MAKE) test\" to launch test scripts'\n+\t@echo 'Run \"$(MAKE) clean\" to remove trash folders'\n+\n+test:\n+\t@for t in $(T); do \\\n+\t\techo \"$$t\"; \\\n+\t\t\"./$$t\" || exit 1; \\\n+\tdone\n+\n+clean:\n+\t$(RM) -r 'trash directory'.*\n+\ndiff --git a/contrib/mw-to-git/t/README b/contrib/mw-to-git/t/README\nnew file mode 100644\nindex 0000000..96e9739\n--- /dev/null\n+++ b/contrib/mw-to-git/t/README\n@@ -0,0 +1,124 @@\n+Tests for Mediawiki-to-Git\n+==========================\n+\n+Introduction\n+------------\n+This manual describes how to install the git-remote-mediawiki test\n+environment on a machine with git installed on it.\n+\n+Prerequisite\n+------------\n+\n+In order to run this test environment correctly, you will need to\n+install the following packages (Debian/Ubuntu names, may need to be\n+adapted for another distribution):\n+\n+* lighttpd\n+* php5\n+* php5-cgi\n+* php5-cli\n+* php5-curl\n+* php5-sqlite\n+\n+Principles and Technical Choices\n+--------------------------------\n+\n+The test environment makes it easy to install and manipulate one or\n+several MediaWiki instances. To allow developers to run the testsuite\n+easily, the environment does not require root priviledge (except to\n+install the required packages if needed). It starts a webserver\n+instance on the user's account (using lighttpd greatly helps for\n+that), and does not need a separate database daemon (thanks to the use\n+of sqlite).\n+\n+Run the test environment\n+------------------------\n+\n+Install a new wiki\n+~~~~~~~~~~~~~~~~~~\n+\n+Once you have all the prerequisite, you need to install a MediaWiki\n+instance on your machine. If you already have one, it is still\n+strongly recommended to install one with the script provided. Here's\n+how to work it:\n+\n+a. change directory to contrib/mw-to-git/t/\n+b. if needed, edit test.config to choose your installation parameters\n+c. run `./install-wiki.sh install`\n+d. check on your favourite web browser if your wiki is correctly\n+ installed.\n+\n+Remove an existing wiki\n+~~~~~~~~~~~~~~~~~~~~~~~\n+\n+Edit the file test.config to fit the wiki you want to delete, and then\n+execute the command `./install-wiki.sh delete` from the\n+contrib/mw-to-git/t directory.\n+\n+Run the existing tests\n+~~~~~~~~~~~~~~~~~~~~~~\n+\n+The provided tests are currently in the `contrib/mw-to-git/t` directory.\n+The files are all the t936[0-9]-*.sh shell scripts.\n+\n+a. Run all tests:\n+To do so, run \"make test\" from the contrib/mw-to-git/ directory.\n+\n+b. Run a specific test:\n+To run a given test , run ./ from the\n+contrib/mw-to-git/t directory.\n+\n+How to create new tests\n+-----------------------\n+\n+Available functions\n+~~~~~~~~~~~~~~~~~~~\n+\n+The test environment of git-remote-mediawiki provides some functions\n+useful to test its behaviour. for more details about the functions'\n+parameters, please refer to the `test-gitmw-lib.sh` and\n+`test-gitmw.pl` files.\n+\n+** `test_check_wiki_precond`:\n+Check if the tests must be skipped or not. Please use this function\n+at the beggining of each new test file.\n+\n+** `wiki_getpage`:\n+Fetch a given page from the wiki and puts its content in the\n+directory in parameter.\n+\n+** `wiki_delete_page`:\n+Delete a given page from the wiki.\n+\n+** `wiki_edit_page`:\n+Create or modify a given page in the wiki. You can specify several\n+parameters like a summary for the page edition, or add the page to a\n+given category.\n+See test-gitmw.pl for more details.\n+\n+** `wiki_getallpage`:\n+Fetch all pages from the wiki into a given directory. The directory\n+is created if it does not exists.\n+\n+** `test_diff_directories`:\n+Compare the content of two directories. The content must be the same.\n+Use this function to compare the content of a git directory and a wiki\n+one created by wiki_getallpage.\n+\n+** `test_contains_N_files`:\n+Check if the given directory contains a given number of file.\n+\n+** `wiki_page_exists`:\n+Tests if a given page exists on the wiki.\n+\n+** `wiki_reset`:\n+Reset the wiki, i.e. flush the database. Use this function at the\n+begining of each new test, except if the test re-uses the same wiki\n+(and history) as the previous test.\n+\n+How to write a new test\n+~~~~~~~~~~~~~~~~~~~~~~~\n+\n+Please, follow the standards given by git. See git/t/README.\n+New file should be named as t936[0-9]-*.sh.\n+Be sure to reset your wiki regulary with the function `wiki_reset`.\ndiff --git a/contrib/mw-to-git/t/test-gitmw-lib.sh b/contrib/mw-to-git/t/test-gitmw-lib.sh\nindex 47341f7..14f6577 100755\n--- a/contrib/mw-to-git/t/test-gitmw-lib.sh\n+++ b/contrib/mw-to-git/t/test-gitmw-lib.sh\n@@ -13,6 +13,7 @@\n \n . ./test.config\n \n+WIKI_URL=http://\"$SERVER_ADDR:$PORT/$WIKI_DIR_NAME\"\n CURR_DIR=$(pwd)\n TEST_OUTPUT_DIRECTORY=$(pwd)\n TEST_DIRECTORY=\"$CURR_DIR\"/../../../t\n@@ -25,6 +26,148 @@ else\n \tWIKI_DIR_INST=\"$CURR_DIR/$WEB_WWW\"\n fi\n \n+\n+wiki_getpage () {\n+\t\"$CURR_DIR\"/test-gitmw.pl get_page \"$@\"\n+}\n+\n+wiki_delete_page () {\n+\t\"$CURR_DIR\"/test-gitmw.pl delete_page \"$@\"\n+}\n+\n+wiki_editpage () {\n+\t\"$CURR_DIR\"/test-gitmw.pl edit_page \"$@\"\n+}\n+\n+die () {\n+\tdie_with_status 1 \"$@\"\n+}\n+\n+die_with_status () {\n+\tstatus=$1\n+\tshift\n+\techo >&2 \"$*\"\n+\texit \"$status\"\n+}\n+\n+\n+# Check the preconditions to run git-remote-mediawiki's tests\n+test_check_precond () {\n+\tif ! test_have_prereq PERL\n+\tthen\n+\t\tskip_all='skipping gateway git-mw tests, perl not available'\n+\t\ttest_done\n+\tfi\n+\n+\tif [ ! -f \"$GIT_BUILD_DIR\"/git-remote-mediawiki ];\n+\tthen\n+\t\techo \"No remote mediawiki for git found. Copying it in git\"\n+\t\techo \"cp $GIT_BUILD_DIR/contrib/mw-to-git/git-remote-mediawiki $GIT_BUILD_DIR/\"\n+\t\tln -s \"$GIT_BUILD_DIR\"/contrib/mw-to-git/git-remote-mediawiki \"$GIT_BUILD_DIR\"\n+\tfi\n+\n+\tif [ ! -d \"$WIKI_DIR_INST/$WIKI_DIR_NAME\" ];\n+\tthen\n+\t\tskip_all='skipping gateway git-mw tests, no mediawiki found'\n+\t\ttest_done\n+\tfi\n+}\n+\n+# test_diff_directories \n+#\n+# Compare the contents of directories and with diff\n+# and errors if they do not match. The program will\n+# not look into .git in the process.\n+# Warning: the first argument MUST be the directory containing the git data\n+test_diff_directories () {\n+\trm -rf \"$1_tmp\"\n+\tmkdir -p \"$1_tmp\"\n+\tcp \"$1\"/*.mw \"$1_tmp\"\n+\tdiff -r -b \"$1_tmp\" \"$2\"\n+}\n+\n+# $1=\n+# $2=\n+#\n+# Check that contains exactly files\n+test_contains_N_files () {\n+\tif test `ls -- \"$1\" | wc -l` -ne \"$2\"; then\n+\t\techo \"directory $1 sould contain $2 files\"\n+\t\techo \"it contains these files:\"\n+\t\tls \"$1\"\n+\t\tfalse\n+\tfi\n+}\n+\n+\n+# wiki_check_content \n+#\n+# Compares the contents of the file and the wiki page\n+# and exits with error 1 if they do not match.\n+wiki_check_content () {\n+\tmkdir -p wiki_tmp\n+\twiki_getpage \"$2\" wiki_tmp\n+\t# replacement of forbidden character in file name\n+\tpage_name=$(printf \"%s\\n\" \"$2\" | sed -e \"s/\\//%2F/g\")\n+\n+\tdiff -b \"$1\" wiki_tmp/\"$page_name\".mw\n+\tif test $? -ne 0\n+\tthen\n+\t\trm -rf wiki_tmp\n+\t\terror \"ERROR: file $2 not found on wiki\"\n+\tfi\n+\trm -rf wiki_tmp\n+}\n+\n+# wiki_page_exist \n+#\n+# Check the existence of the page on the wiki and exits\n+# with error if it is absent from it.\n+wiki_page_exist () {\n+\tmkdir -p wiki_tmp\n+\twiki_getpage \"$1\" wiki_tmp\n+\tpage_name=$(printf \"%s\\n\" \"$1\" | sed \"s/\\//%2F/g\")\n+\tif test -f wiki_tmp/\"$page_name\".mw ; then\n+\t\trm -rf wiki_tmp\n+\telse\n+\t\trm -rf wiki_tmp\n+\t\terror \"test failed: file $1 not found on wiki\"\n+\tfi\n+}\n+\n+# wiki_getallpagename\n+#\n+# Fetch the name of each page on the wiki.\n+wiki_getallpagename () {\n+\t\"$CURR_DIR\"/test-gitmw.pl getallpagename\n+}\n+\n+# wiki_getallpagecategory \n+#\n+# Fetch the name of each page belonging to on the wiki.\n+wiki_getallpagecategory () {\n+\t\"$CURR_DIR\"/test-gitmw.pl getallpagename \"$@\"\n+}\n+\n+# wiki_getallpage []\n+#\n+# Fetch all the pages from the wiki and place them in the directory\n+# .\n+# If is define, then wiki_getallpage fetch the pages included\n+# in .\n+wiki_getallpage () {\n+\tif test -z \"$2\";\n+\tthen\n+\t\twiki_getallpagename\n+\telse\n+\t\twiki_getallpagecategory \"$2\"\n+\tfi\n+\tmkdir -p \"$1\"\n+\twhile read -r line; do\n+\t\twiki_getpage \"$line\" $1;\n+\tdone < all.txt\n+}\n+\n # ================= Install part =================\n \n error () {\ndiff --git a/contrib/mw-to-git/t/test-gitmw.pl b/contrib/mw-to-git/t/test-gitmw.pl\nnew file mode 100755\nindex 0000000..2611991\n--- /dev/null\n+++ b/contrib/mw-to-git/t/test-gitmw.pl\n@@ -0,0 +1,209 @@\n+#!/usr/bin/perl -w -s\n+# Copyright (C) 2012\n+# Charles Roussel \n+# Simon Cathebras \n+# Julien Khayat \n+# Guillaume Sasdy \n+# Simon Perrat \n+# License: GPL v2 or later\n+\n+# Usage:\n+# ./test-gitmw.pl [argument]*\n+# Execute in terminal using the name of the function to call as first\n+# parameter, and the function's arguments as following parameters\n+#\n+# Example:\n+# ./test-gitmw.pl \"get_page\" foo .\n+# will call with arguments and <.>\n+#\n+# Available functions are:\n+# \"get_page\"\n+# \"delete_page\"\n+# \"edit_page\"\n+# \"getallpagename\"\n+\n+use MediaWiki::API;\n+use Getopt::Long;\n+use encoding 'utf8';\n+use DateTime::Format::ISO8601;\n+use open ':encoding(utf8)';\n+use constant SLASH_REPLACEMENT => \"%2F\";\n+\n+#Parsing of the config file\n+\n+my $configfile = \"$ENV{'CURR_DIR'}/test.config\";\n+my %config;\n+open my $CONFIG, \"<\", $configfile or die \"can't open $configfile: $!\";\n+while (<$CONFIG>)\n+{\n+ chomp;\n+ s/#.*//;\n+ s/^\\s+//;\n+ s/\\s+$//;\n+ next unless length;\n+ my ($key, $value) = split (/\\s*=\\s*/,$_, 2);\n+ $config{$key} = $value;\n+\tlast if ($key eq 'LIGHTTPD' and $value eq 'false');\n+\tlast if ($key eq 'PORT');\n+}\n+close $CONFIG or die \"can't close $configfile: $!\";\n+\n+my $wiki_address = \"http://$config{'SERVER_ADDR'}\".\":\".\"$config{'PORT'}\";\n+my $wiki_url = \"$wiki_address/$config{'WIKI_DIR_NAME'}/api.php\";\n+my $wiki_admin = \"$config{'WIKI_ADMIN'}\";\n+my $wiki_admin_pass = \"$config{'WIKI_PASSW'}\";\n+my $mw = MediaWiki::API->new;\n+$mw->{config}->{api_url} = $wiki_url;\n+\n+\n+# wiki_login \n+#\n+# Logs the user with and in the global variable\n+# of the mediawiki $mw\n+sub wiki_login {\n+\t$mw->login( { lgname => \"$_[0]\",lgpassword => \"$_[1]\" } )\n+\t|| die \"getpage: login failed\";\n+}\n+\n+# wiki_getpage \n+#\n+# fetch a page from the wiki referenced in the global variable\n+# $mw and copies its content in directory dest_path\n+sub wiki_getpage {\n+\tmy $pagename = $_[0];\n+\tmy $destdir = $_[1];\n+\n+\tmy $page = $mw->get_page( { title => $pagename } );\n+\tif (!defined($page)) {\n+\t\tdie \"getpage: wiki does not exist\";\n+\t}\n+\n+\tmy $content = $page->{'*'};\n+\tif (!defined($content)) {\n+\t\tdie \"getpage: page does not exist\";\n+\t}\n+\n+\t$pagename=$page->{'title'};\n+\t# Replace spaces by underscore in the page name\n+\t$pagename =~ s/ /_/g;\n+\t$pagename =~ s/\\//%2F/g;\n+\topen(my $file, \">$destdir/$pagename.mw\");\n+\tprint $file \"$content\";\n+\tclose ($file);\n+\n+}\n+\n+# wiki_delete_page \n+#\n+# delete the page with name from the wiki referenced\n+# in the global variable $mw\n+sub wiki_delete_page {\n+\tmy $pagename = $_[0];\n+\n+\tmy $exist=$mw->get_page({title => $pagename});\n+\n+\tif (defined($exist->{'*'})){\n+\t\t$mw->edit({ action => 'delete',\n+\t\t\t\ttitle => $pagename})\n+\t\t|| die $mw->{error}->{code} . \": \" . $mw->{error}->{details};\n+\t} else {\n+\t\tdie \"no page with such name found: $pagename\\n\";\n+\t}\n+}\n+\n+# wiki_editpage [-c=] [-s=]\n+#\n+# Edit a page named with content on the wiki\n+# referenced with the global variable $mw\n+# If == true : append at the end of the actual\n+# content of the page \n+# If doesn't exist, that page is created with the \n+sub wiki_editpage {\n+\tmy $wiki_page = $_[0];\n+\tmy $wiki_content = $_[1];\n+\tmy $wiki_append = $_[2];\n+\tmy $summary = \"\";\n+\tmy ($summ, $cat) = ();\n+\tGetOptions('s=s' => \\$summ, 'c=s' => \\$cat);\n+\n+\tmy $append = 0;\n+\tif (defined($wiki_append) && $wiki_append eq 'true') {\n+\t\t$append=1;\n+\t}\n+\n+\tmy $previous_text =\"\";\n+\n+\tif ($append) {\n+\t\tmy $ref = $mw->get_page( { title => $wiki_page } );\n+\t\t$previous_text = $ref->{'*'};\n+\t}\n+\n+\tmy $text = $wiki_content;\n+\tif (defined($previous_text)) {\n+\t\t$text=\"$previous_text$text\";\n+\t}\n+\n+\t# Eventually, add this page to a category.\n+\tif (defined($cat)) {\n+\t\tmy $category_name=\"[[Category:$cat]]\";\n+\t\t$text=\"$text\\n $category_name\";\n+\t}\n+\tif(defined($summ)){\n+\t\t$summary=$summ;\n+\t}\n+\n+\t$mw->edit( { action => 'edit', title => $wiki_page, summary => $summary, text => \"$text\"} );\n+}\n+\n+# wiki_getallpagename []\n+#\n+# Fetch all pages of the wiki referenced by the global variable $mw\n+# and print the names of each one in the file all.txt with a new line\n+# (\"\\n\") between these.\n+# If the argument is defined, then this function get only the pages\n+# belonging to .\n+sub wiki_getallpagename {\n+\t# fetch the pages of the wiki\n+\tif (defined($_[0])) {\n+\t\tmy $mw_pages = $mw->list ( { action => 'query',\n+\t\t\t\tlist => 'categorymembers',\n+\t\t\t\tcmtitle => \"Category:$_[0]\",\n+\t\t\t\tcmnamespace => 0,\n+\t\t\t\tcmlimit => 500 },\n+\t\t)\n+\t\t|| die $mw->{error}->{code}.\": \".$mw->{error}->{details};\n+\t\topen(my $file, \">all.txt\");\n+\t\tforeach my $page (@{$mw_pages}) {\n+\t\t\tprint $file \"$page->{title}\\n\";\n+\t\t}\n+\t\tclose ($file);\n+\n+\t} else {\n+\t\tmy $mw_pages = $mw->list({\n+\t\t\t\taction => 'query',\n+\t\t\t\tlist => 'allpages',\n+\t\t\t\taplimit => 500,\n+\t\t\t})\n+\t\t|| die $mw->{error}->{code}.\": \".$mw->{error}->{details};\n+\t\topen(my $file, \">all.txt\");\n+\t\tforeach my $page (@{$mw_pages}) {\n+\t\t\tprint $file \"$page->{title}\\n\";\n+\t\t}\n+\t\tclose ($file);\n+\t}\n+}\n+\n+# Main part of this script: parse the command line arguments\n+# and select which function to execute\n+my $fct_to_call = shift;\n+\n+wiki_login($wiki_admin, $wiki_admin_pass);\n+\n+my %functions_to_call = qw(\n+ get_page wiki_getpage\n+ delete_page wiki_delete_page\n+ edit_page wiki_editpage\n+ getallpagename wiki_getallpagename\n+);\n+die \"$0 ERROR: wrong argument\" unless exists $functions_to_call{$fct_to_call};\n+&{$functions_to_call{$fct_to_call}}(@ARGV);\n-- \n1.7.11.1.147.g47a574d\n"},{"id":"194637","messageId":"1341473772-28023-4-git-send-email-Matthieu.Moy@imag.fr","threadId":"30954","inReplyTo":"1341473772-28023-1-git-send-email-Matthieu.Moy@imag.fr","subject":"[PATCH 03/12] git-remote-mediawiki (t9360): test git-remote-mediawiki clone","fromName":"Matthieu Moy","fromEmail":"matthieu.moy@imag.fr","sentAt":"2012-07-05T07:36:03Z","receivedAt":"2012-07-05T07:36:03Z","isPatch":true,"sender":{"key":"git@matthieu-moy.fr","avatar":"https://avatars.githubusercontent.com/u/14709?v=4"},"body":"From: Guillaume Sasdy \n\nSigned-off-by: Simon CATHEBRAS \nSigned-off-by: Julien KHAYAT \nSigned-off-by: Simon Perrat \nSigned-off-by: Charles ROUSSEL \nSigned-off-by: Guillaume SASDY \nSigned-off-by: Matthieu Moy \n---\n contrib/mw-to-git/t/t9360-mw-to-git-clone.sh | 257 +++++++++++++++++++++++++++\n 1 file changed, 257 insertions(+)\n create mode 100755 contrib/mw-to-git/t/t9360-mw-to-git-clone.sh\n\ndiff --git a/contrib/mw-to-git/t/t9360-mw-to-git-clone.sh b/contrib/mw-to-git/t/t9360-mw-to-git-clone.sh\nnew file mode 100755\nindex 0000000..811a90c\n--- /dev/null\n+++ b/contrib/mw-to-git/t/t9360-mw-to-git-clone.sh\n@@ -0,0 +1,257 @@\n+#!/bin/sh\n+#\n+# Copyright (C) 2012\n+# Charles Roussel \n+# Simon Cathebras \n+# Julien Khayat \n+# Guillaume Sasdy \n+# Simon Perrat \n+#\n+# License: GPL v2 or later\n+\n+\n+test_description='Test the Git Mediawiki remote helper: git clone'\n+\n+. ./test-gitmw-lib.sh\n+. $TEST_DIRECTORY/test-lib.sh\n+\n+\n+test_check_precond\n+\n+\n+test_expect_success 'Git clone creates the expected git log with one file' '\n+\twiki_reset &&\n+\twiki_editpage foo \"this is not important\" false -c cat -s \"this must be the same\" &&\n+\tgit clone mediawiki::'\"$WIKI_URL\"' mw_dir_1 &&\n+\t(\n+\t\tcd mw_dir_1 &&\n+\t\tgit log --format=%s HEAD^..HEAD >log.tmp\n+\t) &&\n+\techo \"this must be the same\" >msg.tmp &&\n+\tdiff -b mw_dir_1/log.tmp msg.tmp\n+'\n+\n+\n+test_expect_success 'Git clone creates the expected git log with multiple files' '\n+\twiki_reset &&\n+\twiki_editpage daddy \"this is not important\" false -s=\"this must be the same\" &&\n+\twiki_editpage daddy \"neither is this\" true -s=\"this must also be the same\" &&\n+\twiki_editpage daddy \"neither is this\" true -s=\"same same same\" &&\n+\twiki_editpage dj \"dont care\" false -s=\"identical\" &&\n+\twiki_editpage dj \"dont care either\" true -s=\"identical too\" &&\n+\tgit clone mediawiki::'\"$WIKI_URL\"' mw_dir_2 &&\n+\t(\n+\t\tcd mw_dir_2 &&\n+\t\tgit log --format=%s Daddy.mw >logDaddy.tmp &&\n+\t\tgit log --format=%s Dj.mw >logDj.tmp\n+\t) &&\n+\techo \"same same same\" >msgDaddy.tmp &&\n+\techo \"this must also be the same\" >>msgDaddy.tmp &&\n+\techo \"this must be the same\" >>msgDaddy.tmp &&\n+\techo \"identical too\" >msgDj.tmp &&\n+\techo \"identical\" >>msgDj.tmp &&\n+\tdiff -b mw_dir_2/logDaddy.tmp msgDaddy.tmp &&\n+\tdiff -b mw_dir_2/logDj.tmp msgDj.tmp\n+'\n+\n+\n+test_expect_success 'Git clone creates only Main_Page.mw with an empty wiki' '\n+\twiki_reset &&\n+\tgit clone mediawiki::'\"$WIKI_URL\"' mw_dir_3 &&\n+\ttest_contains_N_files mw_dir_3 1 &&\n+\ttest_path_is_file mw_dir_3/Main_Page.mw\n+'\n+\n+test_expect_success 'Git clone does not fetch a deleted page' '\n+\twiki_reset &&\n+\twiki_editpage foo \"this page must be deleted before the clone\" false &&\n+\twiki_delete_page foo &&\n+\tgit clone mediawiki::'\"$WIKI_URL\"' mw_dir_4 &&\n+\ttest_contains_N_files mw_dir_4 1 &&\n+\ttest_path_is_file mw_dir_4/Main_Page.mw &&\n+\ttest_path_is_missing mw_dir_4/Foo.mw\n+'\n+\n+test_expect_success 'Git clone works with page added' '\n+\twiki_reset &&\n+\twiki_editpage foo \" I will be cloned\" false &&\n+\twiki_editpage bar \"I will be cloned\" false &&\n+\tgit clone mediawiki::'\"$WIKI_URL\"' mw_dir_5 &&\n+\twiki_getallpage ref_page_5 &&\n+\ttest_diff_directories mw_dir_5 ref_page_5 &&\n+\twiki_delete_page foo &&\n+\twiki_delete_page bar\n+'\n+\n+test_expect_success 'Git clone works with an edited page ' '\n+\twiki_reset &&\n+\twiki_editpage foo \"this page will be edited\" \\\n+\t\tfalse -s \"first edition of page foo\"&&\n+\twiki_editpage foo \"this page has been edited and must be on the clone \" true &&\n+\tgit clone mediawiki::'\"$WIKI_URL\"' mw_dir_6 &&\n+\ttest_path_is_file mw_dir_6/Foo.mw &&\n+\ttest_path_is_file mw_dir_6/Main_Page.mw &&\n+\twiki_getallpage mw_dir_6/page_ref_6 &&\n+\ttest_diff_directories mw_dir_6 mw_dir_6/page_ref_6 &&\n+\t(\n+\t\tcd mw_dir_6 &&\n+\t\tgit log --format=%s HEAD^ Foo.mw > ../Foo.log\n+\t) &&\n+\techo \"first edition of page foo\" > FooExpect.log &&\n+\tdiff FooExpect.log Foo.log\n+'\n+\n+\n+test_expect_success 'Git clone works with several pages and some deleted ' '\n+\twiki_reset &&\n+\twiki_editpage foo \"this page will not be deleted\" false &&\n+\twiki_editpage bar \"I must not be erased\" false &&\n+\twiki_editpage namnam \"I will not be there at the end\" false &&\n+\twiki_editpage nyancat \"nyan nyan nyan delete me\" false &&\n+\twiki_delete_page namnam &&\n+\twiki_delete_page nyancat &&\n+\tgit clone mediawiki::'\"$WIKI_URL\"' mw_dir_7 &&\n+\ttest_path_is_file mw_dir_7/Foo.mw &&\n+\ttest_path_is_file mw_dir_7/Bar.mw &&\n+\ttest_path_is_missing mw_dir_7/Namnam.mw &&\n+\ttest_path_is_missing mw_dir_7/Nyancat.mw &&\n+\twiki_getallpage mw_dir_7/page_ref_7 &&\n+\ttest_diff_directories mw_dir_7 mw_dir_7/page_ref_7\n+'\n+\n+\n+test_expect_success 'Git clone works with one specific page cloned ' '\n+\twiki_reset &&\n+\twiki_editpage foo \"I will not be cloned\" false &&\n+\twiki_editpage bar \"Do not clone me\" false &&\n+\twiki_editpage namnam \"I will be cloned :)\" false -s=\"this log must stay\" &&\n+\twiki_editpage nyancat \"nyan nyan nyan you cant clone me\" false &&\n+\tgit clone -c remote.origin.pages=namnam \\\n+\t\tmediawiki::'\"$WIKI_URL\"' mw_dir_8 &&\n+\ttest_contains_N_files mw_dir_8 1 &&\n+\ttest_path_is_file mw_dir_8/Namnam.mw &&\n+\ttest_path_is_missing mw_dir_8/Main_Page.mw &&\n+\t(\n+\t\tcd mw_dir_8 &&\n+\t\techo \"this log must stay\" >msg.tmp &&\n+\t\tgit log --format=%s >log.tmp &&\n+\t\tdiff -b msg.tmp log.tmp\n+\t) &&\n+\twiki_check_content mw_dir_8/Namnam.mw Namnam\n+'\n+\n+test_expect_success 'Git clone works with multiple specific page cloned ' '\n+\twiki_reset &&\n+\twiki_editpage foo \"I will be there\" false &&\n+\twiki_editpage bar \"I will not disapear\" false &&\n+\twiki_editpage namnam \"I be erased\" false &&\n+\twiki_editpage nyancat \"nyan nyan nyan you will not erase me\" false &&\n+\twiki_delete_page namnam &&\n+\tgit clone -c remote.origin.pages=\"foo bar nyancat namnam\" \\\n+\t\tmediawiki::'\"$WIKI_URL\"' mw_dir_9 &&\n+\ttest_contains_N_files mw_dir_9 3 &&\n+\ttest_path_is_missing mw_dir_9/Namnam.mw &&\n+\ttest_path_is_file mw_dir_9/Foo.mw &&\n+\ttest_path_is_file mw_dir_9/Nyancat.mw &&\n+\ttest_path_is_file mw_dir_9/Bar.mw &&\n+\twiki_check_content mw_dir_9/Foo.mw Foo &&\n+\twiki_check_content mw_dir_9/Bar.mw Bar &&\n+\twiki_check_content mw_dir_9/Nyancat.mw Nyancat\n+'\n+\n+test_expect_success 'Mediawiki-clone of several specific pages on wiki' '\n+\twiki_reset &&\n+\twiki_editpage foo \"foo 1\" false &&\n+\twiki_editpage bar \"bar 1\" false &&\n+\twiki_editpage dummy \"dummy 1\" false &&\n+\twiki_editpage cloned_1 \"cloned_1 1\" false &&\n+\twiki_editpage cloned_2 \"cloned_2 2\" false &&\n+\twiki_editpage cloned_3 \"cloned_3 3\" false &&\n+\tmkdir -p ref_page_10 &&\n+\twiki_getpage cloned_1 ref_page_10 &&\n+\twiki_getpage cloned_2 ref_page_10 &&\n+\twiki_getpage cloned_3 ref_page_10 &&\n+\tgit clone -c remote.origin.pages=\"cloned_1 cloned_2 cloned_3\" \\\n+\t\tmediawiki::'\"$WIKI_URL\"' mw_dir_10 &&\n+\ttest_diff_directories mw_dir_10 ref_page_10\n+'\n+\n+test_expect_success 'Git clone works with the shallow option' '\n+\twiki_reset &&\n+\twiki_editpage foo \"1st revision, should be cloned\" false &&\n+\twiki_editpage bar \"1st revision, should be cloned\" false &&\n+\twiki_editpage nyan \"1st revision, should not be cloned\" false &&\n+\twiki_editpage nyan \"2nd revision, should be cloned\" false &&\n+\tgit -c remote.origin.shallow=true clone \\\n+\t\tmediawiki::'\"$WIKI_URL\"' mw_dir_11 &&\n+\ttest_contains_N_files mw_dir_11 4 &&\n+\ttest_path_is_file mw_dir_11/Nyan.mw &&\n+\ttest_path_is_file mw_dir_11/Foo.mw &&\n+\ttest_path_is_file mw_dir_11/Bar.mw &&\n+\ttest_path_is_file mw_dir_11/Main_Page.mw &&\n+\t(\n+\t\tcd mw_dir_11 &&\n+\t\ttest `git log --oneline Nyan.mw | wc -l` -eq 1 &&\n+\t\ttest `git log --oneline Foo.mw | wc -l` -eq 1 &&\n+\t\ttest `git log --oneline Bar.mw | wc -l` -eq 1 &&\n+\t\ttest `git log --oneline Main_Page.mw | wc -l ` -eq 1\n+\t) &&\n+\twiki_check_content mw_dir_11/Nyan.mw Nyan &&\n+\twiki_check_content mw_dir_11/Foo.mw Foo &&\n+\twiki_check_content mw_dir_11/Bar.mw Bar &&\n+\twiki_check_content mw_dir_11/Main_Page.mw Main_Page\n+'\n+\n+test_expect_success 'Git clone works with the shallow option with a delete page' '\n+\twiki_reset &&\n+\twiki_editpage foo \"1st revision, will be deleted\" false &&\n+\twiki_editpage bar \"1st revision, should be cloned\" false &&\n+\twiki_editpage nyan \"1st revision, should not be cloned\" false &&\n+\twiki_editpage nyan \"2nd revision, should be cloned\" false &&\n+\twiki_delete_page foo &&\n+\tgit -c remote.origin.shallow=true clone \\\n+\t\tmediawiki::'\"$WIKI_URL\"' mw_dir_12 &&\n+\ttest_contains_N_files mw_dir_12 3 &&\n+\ttest_path_is_file mw_dir_12/Nyan.mw &&\n+\ttest_path_is_missing mw_dir_12/Foo.mw &&\n+\ttest_path_is_file mw_dir_12/Bar.mw &&\n+\ttest_path_is_file mw_dir_12/Main_Page.mw &&\n+\t(\n+\t\tcd mw_dir_12 &&\n+\t\ttest `git log --oneline Nyan.mw | wc -l` -eq 1 &&\n+\t\ttest `git log --oneline Bar.mw | wc -l` -eq 1 &&\n+\t\ttest `git log --oneline Main_Page.mw | wc -l ` -eq 1\n+\t) &&\n+\twiki_check_content mw_dir_12/Nyan.mw Nyan &&\n+\twiki_check_content mw_dir_12/Bar.mw Bar &&\n+\twiki_check_content mw_dir_12/Main_Page.mw Main_Page\n+'\n+\n+test_expect_success 'Test of fetching a category' '\n+\twiki_reset &&\n+\twiki_editpage Foo \"I will be cloned\" false -c=Category &&\n+\twiki_editpage Bar \"Meet me on the repository\" false -c=Category &&\n+\twiki_editpage Dummy \"I will not come\" false &&\n+\twiki_editpage BarWrong \"I will stay online only\" false -c=NotCategory &&\n+\tgit clone -c remote.origin.categories=\"Category\" \\\n+\t\tmediawiki::'\"$WIKI_URL\"' mw_dir_13 &&\n+\twiki_getallpage ref_page_13 Category &&\n+\ttest_diff_directories mw_dir_13 ref_page_13\n+'\n+\n+test_expect_success 'Test of resistance to modification of category on wiki for clone' '\n+\twiki_reset &&\n+\twiki_editpage Tobedeleted \"this page will be deleted\" false -c=Catone &&\n+\twiki_editpage Tobeedited \"this page will be modified\" false -c=Catone &&\n+\twiki_editpage Normalone \"this page wont be modified and will be on git\" false -c=Catone &&\n+\twiki_editpage Notconsidered \"this page will not appear on local\" false &&\n+\twiki_editpage Othercategory \"this page will not appear on local\" false -c=Cattwo &&\n+\twiki_editpage Tobeedited \"this page have been modified\" true -c=Catone &&\n+\twiki_delete_page Tobedeleted\n+\tgit clone -c remote.origin.categories=\"Catone\" \\\n+\t\tmediawiki::'\"$WIKI_URL\"' mw_dir_14 &&\n+\twiki_getallpage ref_page_14 Catone &&\n+\ttest_diff_directories mw_dir_14 ref_page_14\n+'\n+\n+test_done\n-- \n1.7.11.1.147.g47a574d\n"},{"id":"194636","messageId":"1341473772-28023-5-git-send-email-Matthieu.Moy@imag.fr","threadId":"30954","inReplyTo":"1341473772-28023-1-git-send-email-Matthieu.Moy@imag.fr","subject":"[PATCH 04/12] git-remote-mediawiki (t9361): test git-remote-mediawiki pull and push","fromName":"Matthieu Moy","fromEmail":"matthieu.moy@imag.fr","sentAt":"2012-07-05T07:36:04Z","receivedAt":"2012-07-05T07:36:04Z","isPatch":true,"sender":{"key":"git@matthieu-moy.fr","avatar":"https://avatars.githubusercontent.com/u/14709?v=4"},"body":"From: Guillaume Sasdy \n\nThis patch provides a set of tests for the pull and push fonctionnality\nof git-remote-mediawiki. The actual tests are kept in a separate function\nto allow further tests to re-run the same set of commands with different\npush and pull strategies.\n\nSigned-off-by: Simon CATHEBRAS \nSigned-off-by: Julien KHAYAT \nSigned-off-by: Simon Perrat \nSigned-off-by: Charles ROUSSEL \nSigned-off-by: Guillaume SASDY \nSigned-off-by: Matthieu Moy \n---\n contrib/mw-to-git/t/push-pull-tests.sh | 144 +++++++++++++++++++++++\n contrib/mw-to-git/t/t9361-mw-to-git-push-pull.sh | 24 ++++\n 2 files changed, 168 insertions(+)\n create mode 100644 contrib/mw-to-git/t/push-pull-tests.sh\n create mode 100755 contrib/mw-to-git/t/t9361-mw-to-git-push-pull.sh\n\ndiff --git a/contrib/mw-to-git/t/push-pull-tests.sh b/contrib/mw-to-git/t/push-pull-tests.sh\nnew file mode 100644\nindex 0000000..b9e38dc\n--- /dev/null\n+++ b/contrib/mw-to-git/t/push-pull-tests.sh\n@@ -0,0 +1,144 @@\n+test_push_pull () {\n+\n+\ttest_expect_success 'Git pull works after adding a new wiki page' '\n+\t\twiki_reset &&\n+\t\n+\t\tgit clone mediawiki::'\"$WIKI_URL\"' mw_dir_1 &&\n+\t\twiki_editpage Foo \"page created after the git clone\" false &&\n+\t\n+\t\t(\n+\t\t\tcd mw_dir_1 &&\n+\t\t\tgit pull\n+\t\t) &&\n+\t\n+\t\twiki_getallpage ref_page_1 &&\n+\t\ttest_diff_directories mw_dir_1 ref_page_1\n+\t'\n+\n+\ttest_expect_success 'Git pull works after editing a wiki page' '\n+\t\twiki_reset &&\n+\t\n+\t\twiki_editpage Foo \"page created before the git clone\" false &&\n+\t\tgit clone mediawiki::'\"$WIKI_URL\"' mw_dir_2 &&\n+\t\twiki_editpage Foo \"new line added on the wiki\" true &&\n+\t\n+\t\t(\n+\t\t\tcd mw_dir_2 &&\n+\t\t\tgit pull\n+\t\t) &&\n+\t\n+\t\twiki_getallpage ref_page_2 &&\n+\t\ttest_diff_directories mw_dir_2 ref_page_2\n+\t'\n+\n+\ttest_expect_success 'git pull works on conflict handled by auto-merge' '\n+\t\twiki_reset &&\n+\t\n+\t\twiki_editpage Foo \"1 init\n+3\n+5\n+\t\" false &&\n+\t\tgit clone mediawiki::'\"$WIKI_URL\"' mw_dir_3 &&\n+\t\n+\t\twiki_editpage Foo \"1 init\n+2 content added on wiki after clone\n+3\n+5\n+\t\" false &&\n+\t\n+\t\t(\n+\t\t\tcd mw_dir_3 &&\n+\t\techo \"1 init\n+3\n+4 content added on git after clone\n+5\n+\" >Foo.mw &&\n+\t\t\tgit commit -am \"conflicting change on foo\" &&\n+\t\t\tgit pull &&\n+\t\t\tgit push\n+\t\t)\n+\t'\n+\n+\ttest_expect_success 'Git push works after adding a file .mw' '\n+\t\twiki_reset &&\n+\t\tgit clone mediawiki::'\"$WIKI_URL\"' mw_dir_4 &&\n+\t\twiki_getallpage ref_page_4 &&\n+\t\t(\n+\t\t\tcd mw_dir_4 &&\n+\t\t\ttest ! -f Foo.mw &&\n+\t\t\ttouch Foo.mw &&\n+\t\t\techo \"hello world\" >>Foo.mw &&\n+\t\t\tgit add Foo.mw &&\n+\t\t\tgit commit -m \"Foo\" &&\n+\t\t\tgit push\n+\t\t) &&\n+\t\twiki_getallpage ref_page_4 &&\n+\t\ttest_diff_directories mw_dir_4 ref_page_4\n+\t'\n+\n+\ttest_expect_success 'Git push works after editing a file .mw' '\n+\t\twiki_reset &&\n+\t\twiki_editpage \"Foo\" \"page created before the git clone\" false &&\n+\t\tgit clone mediawiki::'\"$WIKI_URL\"' mw_dir_5 &&\n+\t\n+\t\t(\n+\t\t\tcd mw_dir_5 &&\n+\t\t\techo \"new line added in the file Foo.mw\" >>Foo.mw &&\n+\t\t\tgit commit -am \"edit file Foo.mw\" &&\n+\t\t\tgit push\n+\t\t) &&\n+\t\n+\t\twiki_getallpage ref_page_5 &&\n+\t\ttest_diff_directories mw_dir_5 ref_page_5\n+\t'\n+\n+\ttest_expect_failure 'Git push works after deleting a file' '\n+\t\twiki_reset &&\n+\t\twiki_editpage Foo \"wiki page added before git clone\" false &&\n+\t\tgit clone mediawiki::'\"$WIKI_URL\"' mw_dir_6 &&\n+\t\n+\t\t(\n+\t\t\tcd mw_dir_6 &&\n+\t\t\tgit rm Foo.mw &&\n+\t\t\tgit commit -am \"page Foo.mw deleted\" &&\n+\t\t\tgit push\n+\t\t) &&\n+\t\n+\t\ttest ! wiki_page_exist Foo\n+\t'\n+\n+\ttest_expect_success 'Merge conflict expected and solving it' '\n+\t\twiki_reset &&\n+\t\n+\t\tgit clone mediawiki::'\"$WIKI_URL\"' mw_dir_7 &&\n+\t\twiki_editpage Foo \"1 conflict\n+3 wiki\n+4\" false &&\n+\t\n+\t\t(\n+\t\t\tcd mw_dir_7 &&\n+\t\techo \"1 conflict\n+2 git\n+4\" >Foo.mw &&\n+\t\t\tgit add Foo.mw &&\n+\t\t\tgit commit -m \"conflict created\" &&\n+\t\t\ttest_must_fail git pull &&\n+\t\t\tperl -pi -e \"s/[<=>].*//g\" Foo.mw &&\n+\t\t\tgit commit -am \"merge conflict solved\" &&\n+\t\t\tgit push\n+\t\t)\n+\t'\n+\n+\ttest_expect_failure 'git pull works after deleting a wiki page' '\n+\t\twiki_reset &&\n+\t\twiki_editpage Foo \"wiki page added before the git clone\" false &&\n+\t\tgit clone mediawiki::'\"$WIKI_URL\"' mw_dir_8 &&\n+\t\n+\t\twiki_delete_page Foo &&\n+\t\t(\n+\t\t\tcd mw_dir_8 &&\n+\t\t\tgit pull &&\n+\t\t\ttest ! -f Foo.mw\n+\t\t)\n+\t'\n+}\ndiff --git a/contrib/mw-to-git/t/t9361-mw-to-git-push-pull.sh b/contrib/mw-to-git/t/t9361-mw-to-git-push-pull.sh\nnew file mode 100755\nindex 0000000..9ea2014\n--- /dev/null\n+++ b/contrib/mw-to-git/t/t9361-mw-to-git-push-pull.sh\n@@ -0,0 +1,24 @@\n+#!/bin/sh\n+#\n+# Copyright (C) 2012\n+# Charles Roussel \n+# Simon Cathebras \n+# Julien Khayat \n+# Guillaume Sasdy \n+# Simon Perrat \n+#\n+# License: GPL v2 or later\n+\n+# tests for git-remote-mediawiki\n+\n+test_description='Test the Git Mediawiki remote helper: git push and git pull simple test cases'\n+\n+. ./test-gitmw-lib.sh\n+. ./push-pull-tests.sh\n+. $TEST_DIRECTORY/test-lib.sh\n+\n+test_check_precond\n+\n+test_push_pull\n+\n+test_done\n-- \n1.7.11.1.147.g47a574d\n"},{"id":"194646","messageId":"1341473772-28023-6-git-send-email-Matthieu.Moy@imag.fr","threadId":"30954","inReplyTo":"1341473772-28023-1-git-send-email-Matthieu.Moy@imag.fr","subject":"[PATCH 05/12] git-remote-mediawiki (t9362): test git-remote-mediawiki with UTF8 characters","fromName":"Matthieu Moy","fromEmail":"matthieu.moy@imag.fr","sentAt":"2012-07-05T07:36:05Z","receivedAt":"2012-07-05T07:36:05Z","isPatch":true,"sender":{"key":"git@matthieu-moy.fr","avatar":"https://avatars.githubusercontent.com/u/14709?v=4"},"body":"From: Simon Cathebras \n\nNon-ascii encoding create many particular cases when used in page\ncontent, name, and edit/commit message. Test these cases.\n\nSigned-off-by: Simon CATHEBRAS \nSigned-off-by: Julien KHAYAT \nSigned-off-by: Simon Perrat \nSigned-off-by: Charles ROUSSEL \nSigned-off-by: Guillaume SASDY \nSigned-off-by: Matthieu Moy \n---\n contrib/mw-to-git/t/t9362-mw-to-git-utf8.sh | 301 ++++++++++++++++++++++++++++\n 1 file changed, 301 insertions(+)\n create mode 100755 contrib/mw-to-git/t/t9362-mw-to-git-utf8.sh\n\ndiff --git a/contrib/mw-to-git/t/t9362-mw-to-git-utf8.sh b/contrib/mw-to-git/t/t9362-mw-to-git-utf8.sh\nnew file mode 100755\nindex 0000000..8635878\n--- /dev/null\n+++ b/contrib/mw-to-git/t/t9362-mw-to-git-utf8.sh\n@@ -0,0 +1,301 @@\n+#!/bin/sh\n+#\n+# Copyright (C) 2012\n+# Charles Roussel \n+# Simon Cathebras \n+# Julien Khayat \n+# Guillaume Sasdy \n+# Simon Perrat \n+#\n+# License: GPL v2 or later\n+\n+# tests for git-remote-mediawiki\n+\n+test_description='Test git-mediawiki with special characters in filenames'\n+\n+. ./test-gitmw-lib.sh\n+. $TEST_DIRECTORY/test-lib.sh\n+\n+\n+test_check_precond\n+\n+\n+test_expect_success 'Git clone works for a wiki with accents in the page names' '\n+\twiki_reset &&\n+\twiki_editpage féé \"This page must be délétéd before clone\" false &&\n+\twiki_editpage kèè \"This page must be deleted before clone\" false &&\n+\twiki_editpage hàà \"This page must be deleted before clone\" false &&\n+\twiki_editpage kîî \"This page must be deleted before clone\" false &&\n+\twiki_editpage foo \"This page must be deleted before clone\" false &&\n+\tgit clone mediawiki::'\"$WIKI_URL\"' mw_dir_1 &&\n+\twiki_getallpage ref_page_1 &&\n+\ttest_diff_directories mw_dir_1 ref_page_1\n+'\n+\n+\n+test_expect_success 'Git pull works with a wiki with accents in the pages names' '\n+\twiki_reset &&\n+\twiki_editpage kîî \"this page must be cloned\" false &&\n+\twiki_editpage foo \"this page must be cloned\" false &&\n+\tgit clone mediawiki::'\"$WIKI_URL\"' mw_dir_2 &&\n+\twiki_editpage éàîôû \"This page must be pulled\" false &&\n+\t(\n+\t\tcd mw_dir_2 &&\n+\t\tgit pull\n+\t) &&\n+\twiki_getallpage ref_page_2 &&\n+\ttest_diff_directories mw_dir_2 ref_page_2\n+'\n+\n+\n+test_expect_success 'Cloning a chosen page works with accents' '\n+\twiki_reset &&\n+\twiki_editpage kîî \"this page must be cloned\" false &&\n+\tgit clone -c remote.origin.pages=kîî \\\n+\t\tmediawiki::'\"$WIKI_URL\"' mw_dir_3 &&\n+\twiki_check_content mw_dir_3/Kîî.mw Kîî &&\n+\ttest_path_is_file mw_dir_3/Kîî.mw &&\n+\trm -rf mw_dir_3\n+'\n+\n+\n+test_expect_success 'The shallow option works with accents' '\n+\twiki_reset &&\n+\twiki_editpage néoà \"1st revision, should not be cloned\" false &&\n+\twiki_editpage néoà \"2nd revision, should be cloned\" false &&\n+\tgit -c remote.origin.shallow=true clone \\\n+\t\tmediawiki::'\"$WIKI_URL\"' mw_dir_4 &&\n+\ttest_contains_N_files mw_dir_4 2 &&\n+\ttest_path_is_file mw_dir_4/Néoà.mw &&\n+\ttest_path_is_file mw_dir_4/Main_Page.mw &&\n+\t(\n+\t\tcd mw_dir_4 &&\n+\t\ttest `git log --oneline Néoà.mw | wc -l` -eq 1 &&\n+\t\ttest `git log --oneline Main_Page.mw | wc -l ` -eq 1\n+\t) &&\n+\twiki_check_content mw_dir_4/Néoà.mw Néoà &&\n+\twiki_check_content mw_dir_4/Main_Page.mw Main_Page\n+'\n+\n+\n+test_expect_success 'Cloning works when page name first letter has an accent' '\n+\twiki_reset &&\n+\twiki_editpage îî \"this page must be cloned\" false &&\n+\tgit clone -c remote.origin.pages=îî \\\n+\t\tmediawiki::'\"$WIKI_URL\"' mw_dir_5 &&\n+\ttest_path_is_file mw_dir_5/Îî.mw &&\n+\twiki_check_content mw_dir_5/Îî.mw Îî\n+'\n+\n+\n+test_expect_success 'Git push works with a wiki with accents' '\n+\twiki_reset &&\n+\twiki_editpage féé \"lots of accents : éèàÖ\" false &&\n+\twiki_editpage foo \"this page must be cloned\" false &&\n+\tgit clone mediawiki::'\"$WIKI_URL\"' mw_dir_6 &&\n+\t(\n+\t\tcd mw_dir_6 &&\n+\t\techo \"A wild Pîkächû appears on the wiki\" >Pîkächû.mw &&\n+\t\tgit add Pîkächû.mw &&\n+\t\tgit commit -m \"A new page appears\" &&\n+\t\tgit push\n+\t) &&\n+\twiki_getallpage ref_page_6 &&\n+\ttest_diff_directories mw_dir_6 ref_page_6\n+'\n+\n+test_expect_success 'Git clone works with accentsand spaces' '\n+\twiki_reset &&\n+\twiki_editpage \"é à î\" \"this page must be délété before the clone\" false &&\n+\tgit clone mediawiki::'\"$WIKI_URL\"' mw_dir_7 &&\n+\twiki_getallpage ref_page_7 &&\n+\ttest_diff_directories mw_dir_7 ref_page_7\n+'\n+\n+test_expect_success 'character $ in page name (mw -> git)' '\n+\twiki_reset &&\n+\twiki_editpage file_\\$_foo \"expect to be called file_$_foo\" false &&\n+\tgit clone mediawiki::'\"$WIKI_URL\"' mw_dir_8 &&\n+\ttest_path_is_file mw_dir_8/File_\\$_foo.mw &&\n+\twiki_getallpage ref_page_8 &&\n+\ttest_diff_directories mw_dir_8 ref_page_8\n+'\n+\n+\n+\n+test_expect_success 'character $ in file name (git -> mw) ' '\n+\twiki_reset &&\n+\tgit clone mediawiki::'\"$WIKI_URL\"' mw_dir_9 &&\n+\t(\n+\t\tcd mw_dir_9 &&\n+\t\techo \"this file is called File_\\$_foo.mw\" >File_\\$_foo.mw &&\n+\t\tgit add . &&\n+\t\tgit commit -am \"file File_\\$_foo.mw\" &&\n+\t\tgit pull &&\n+\t\tgit push\n+\t) &&\n+\twiki_getallpage ref_page_9 &&\n+\ttest_diff_directories mw_dir_9 ref_page_9\n+'\n+\n+\n+test_expect_failure 'capital at the begining of file names' '\n+\twiki_reset &&\n+\tgit clone mediawiki::'\"$WIKI_URL\"' mw_dir_10 &&\n+\t(\n+\t\tcd mw_dir_10 &&\n+\t\techo \"my new file foo\" >foo.mw &&\n+\t\techo \"my new file Foo... Finger crossed\" >Foo.mw &&\n+\t\tgit add . &&\n+\t\tgit commit -am \"file foo.mw\" &&\n+\t\tgit pull &&\n+\t\tgit push\n+\t) &&\n+\twiki_getallpage ref_page_10 &&\n+\ttest_diff_directories mw_dir_10 ref_page_10\n+'\n+\n+\n+test_expect_failure 'special character at the begining of file name from mw to git' '\n+\twiki_reset &&\n+\tgit clone mediawiki::'\"$WIKI_URL\"' mw_dir_11 &&\n+\twiki_editpage {char_1 \"expect to be renamed {char_1\" false &&\n+\twiki_editpage [char_2 \"expect to be renamed [char_2\" false &&\n+\t(\n+\t\tcd mw_dir_11 &&\n+\t\tgit pull\n+\t) &&\n+\ttest_path_is_file mw_dir_11/{char_1 &&\n+\ttest_path_is_file mw_dir_11/[char_2\n+'\n+\n+test_expect_success 'test of correct formating for file name from mw to git' '\n+\twiki_reset &&\n+\tgit clone mediawiki::'\"$WIKI_URL\"' mw_dir_12 &&\n+\twiki_editpage char_%_7b_1 \"expect to be renamed char{_1\" false &&\n+\twiki_editpage char_%_5b_2 \"expect to be renamed char{_2\" false &&\n+\t(\n+\t\tcd mw_dir_12 &&\n+\t\tgit pull\n+\t) &&\n+\ttest_path_is_file mw_dir_12/Char\\{_1.mw &&\n+\ttest_path_is_file mw_dir_12/Char\\[_2.mw &&\n+\twiki_getallpage ref_page_12 &&\n+\tmv ref_page_12/Char_%_7b_1.mw ref_page_12/Char\\{_1.mw &&\n+\tmv ref_page_12/Char_%_5b_2.mw ref_page_12/Char\\[_2.mw &&\n+\ttest_diff_directories mw_dir_12 ref_page_12\n+'\n+\n+\n+test_expect_failure 'test of correct formating for file name begining with special character' '\n+\twiki_reset &&\n+\tgit clone mediawiki::'\"$WIKI_URL\"' mw_dir_13 &&\n+\t(\n+\t\tcd mw_dir_13 &&\n+\t\techo \"my new file {char_1\" >\\{char_1.mw &&\n+\t\techo \"my new file [char_2\" >\\[char_2.mw &&\n+\t\tgit add . &&\n+\t\tgit commit -am \"commiting some exotic file name...\" &&\n+\t\tgit push &&\n+\t\tgit pull\n+\t) &&\n+\twiki_getallpage ref_page_13 &&\n+\ttest_path_is_file ref_page_13/{char_1.mw &&\n+\ttest_path_is_file ref_page_13/[char_2.mw &&\n+\ttest_diff_directories mw_dir_13 ref_page_13\n+'\n+\n+\n+test_expect_success 'test of correct formating for file name from git to mw' '\n+\twiki_reset &&\n+\tgit clone mediawiki::'\"$WIKI_URL\"' mw_dir_14 &&\n+\t(\n+\t\tcd mw_dir_14 &&\n+\t\techo \"my new file char{_1\" >Char\\{_1.mw &&\n+\t\techo \"my new file char[_2\" >Char\\[_2.mw &&\n+\t\tgit add . &&\n+\t\tgit commit -m \"commiting some exotic file name...\" &&\n+\t\tgit push\n+\t) &&\n+\twiki_getallpage ref_page_14 &&\n+\tmv mw_dir_14/Char\\{_1.mw mw_dir_14/Char_%_7b_1.mw &&\n+\tmv mw_dir_14/Char\\[_2.mw mw_dir_14/Char_%_5b_2.mw &&\n+\ttest_diff_directories mw_dir_14 ref_page_14\n+'\n+\n+\n+test_expect_success 'git clone with /' '\n+\twiki_reset &&\n+\twiki_editpage \\/fo\\/o \"this is not important\" false -c=Deleted &&\n+\tgit clone mediawiki::'\"$WIKI_URL\"' mw_dir_15 &&\n+\ttest_path_is_file mw_dir_15/%2Ffo%2Fo.mw &&\n+\twiki_check_content mw_dir_15/%2Ffo%2Fo.mw \\/fo\\/o\n+'\n+\n+\n+test_expect_success 'git push with /' '\n+\twiki_reset &&\n+\tgit clone mediawiki::'\"$WIKI_URL\"' mw_dir_16 &&\n+\techo \"I will be on the wiki\" >mw_dir_16/%2Ffo%2Fo.mw &&\n+\t(\n+\t\tcd mw_dir_16 &&\n+\t\tgit add %2Ffo%2Fo.mw &&\n+\t\tgit commit -m \" %2Ffo%2Fo added\" &&\n+\t\tgit push\n+\t) &&\n+\twiki_page_exist \\/fo\\/o &&\n+\twiki_check_content mw_dir_16/%2Ffo%2Fo.mw \\/fo\\/o\n+\n+'\n+\n+\n+test_expect_success 'git clone with \\' '\n+\twiki_reset &&\n+\twiki_editpage \\\\ko\\\\o \"this is not important\" false -c=Deleted &&\n+\tgit clone mediawiki::'\"$WIKI_URL\"' mw_dir_17 &&\n+\ttest_path_is_file mw_dir_17/\\\\ko\\\\o.mw &&\n+\twiki_check_content mw_dir_17/\\\\ko\\\\o.mw \\\\ko\\\\o\n+'\n+\n+\n+test_expect_success 'git push with \\' '\n+\twiki_reset &&\n+\tgit clone mediawiki::'\"$WIKI_URL\"' mw_dir_18 &&\n+\techo \"I will be on the wiki\" >mw_dir_18/\\\\ko\\\\o.mw &&\n+\t(\n+\t\tcd mw_dir_18 &&\n+\t\tgit add \\\\ko\\\\o.mw &&\n+\t\tgit commit -m \" \\\\ko\\\\o added\" &&\n+\t\tgit push\n+\t)&&\n+\twiki_page_exist \\\\ko\\\\o &&\n+\twiki_check_content mw_dir_18/\\\\ko\\\\o.mw \\\\ko\\\\o\n+\n+'\n+\n+test_expect_success 'git clone with \\ in format control' '\n+\twiki_reset &&\n+\twiki_editpage \\\\no\\\\o \"this is not important\" false &&\n+\tgit clone mediawiki::'\"$WIKI_URL\"' mw_dir_19 &&\n+\ttest_path_is_file mw_dir_19/\\\\no\\\\o.mw &&\n+\twiki_check_content mw_dir_19/\\\\no\\\\o.mw \\\\no\\\\o\n+'\n+\n+\n+test_expect_success 'git push with \\ in format control' '\n+\twiki_reset &&\n+\tgit clone mediawiki::'\"$WIKI_URL\"' mw_dir_20 &&\n+\techo \"I will be on the wiki\" >mw_dir_20/\\\\fo\\\\o.mw &&\n+\t(\n+\t\tcd mw_dir_20 &&\n+\t\tgit add \\\\fo\\\\o.mw &&\n+\t\tgit commit -m \" \\\\fo\\\\o added\" &&\n+\t\tgit push\n+\t)&&\n+\twiki_page_exist \\\\fo\\\\o &&\n+\twiki_check_content mw_dir_20/\\\\fo\\\\o.mw \\\\fo\\\\o\n+\n+'\n+\n+\n+test_done\n-- \n1.7.11.1.147.g47a574d\n"},{"id":"194638","messageId":"1341473772-28023-7-git-send-email-Matthieu.Moy@imag.fr","threadId":"30954","inReplyTo":"1341473772-28023-1-git-send-email-Matthieu.Moy@imag.fr","subject":"[PATCH 06/12] git-remote-mediawiki: support for uploading file in test environment","fromName":"Matthieu Moy","fromEmail":"matthieu.moy@imag.fr","sentAt":"2012-07-05T07:36:06Z","receivedAt":"2012-07-05T07:36:06Z","isPatch":true,"sender":{"key":"git@matthieu-moy.fr","avatar":"https://avatars.githubusercontent.com/u/14709?v=4"},"body":"This will be used for testing git-remote-mediawiki's import feature on a\nwiki containing media files.\n\nSigned-off-by: Simon CATHEBRAS \nSigned-off-by: Julien KHAYAT \nSigned-off-by: Simon Perrat \nSigned-off-by: Charles ROUSSEL \nSigned-off-by: Guillaume SASDY \nSigned-off-by: Matthieu Moy \n---\n contrib/mw-to-git/t/install-wiki/LocalSettings.php | 3 ++-\n contrib/mw-to-git/t/test-gitmw-lib.sh | 3 +++\n contrib/mw-to-git/t/test-gitmw.pl | 16 ++++++++++++++++\n 3 files changed, 21 insertions(+), 1 deletion(-)\n\ndiff --git a/contrib/mw-to-git/t/install-wiki/LocalSettings.php b/contrib/mw-to-git/t/install-wiki/LocalSettings.php\nindex aa72518..29f1251 100644\n--- a/contrib/mw-to-git/t/install-wiki/LocalSettings.php\n+++ b/contrib/mw-to-git/t/install-wiki/LocalSettings.php\n@@ -67,9 +67,10 @@ $wgMemCachedServers = array();\n \n ## To enable image uploads, make sure the 'images' directory\n ## is writable, then set this to true:\n-$wgEnableUploads = false;\n+$wgEnableUploads = true;\n $wgUseImageMagick = true;\n $wgImageMagickConvertCommand =\"@CONVERT@\";\n+$wgFileExtensions[] = 'txt';\n \n # InstantCommons allows wiki to use images from http://commons.wikimedia.org\n $wgUseInstantCommons = false;\ndiff --git a/contrib/mw-to-git/t/test-gitmw-lib.sh b/contrib/mw-to-git/t/test-gitmw-lib.sh\nindex 14f6577..3b2cfac 100755\n--- a/contrib/mw-to-git/t/test-gitmw-lib.sh\n+++ b/contrib/mw-to-git/t/test-gitmw-lib.sh\n@@ -26,6 +26,9 @@ else\n \tWIKI_DIR_INST=\"$CURR_DIR/$WEB_WWW\"\n fi\n \n+wiki_upload_file () {\n+\t\"$CURR_DIR\"/test-gitmw.pl upload_file \"$@\"\n+}\n \n wiki_getpage () {\n \t\"$CURR_DIR\"/test-gitmw.pl get_page \"$@\"\ndiff --git a/contrib/mw-to-git/t/test-gitmw.pl b/contrib/mw-to-git/t/test-gitmw.pl\nindex 2611991..783dd82 100755\n--- a/contrib/mw-to-git/t/test-gitmw.pl\n+++ b/contrib/mw-to-git/t/test-gitmw.pl\n@@ -193,6 +193,21 @@ sub wiki_getallpagename {\n \t}\n }\n \n+sub wiki_upload_file {\n+\tmy $file_name = $_[0];\n+\tmy $resultat = $mw->edit ( {\n+\t\taction => 'upload',\n+\t\tfilename => $file_name,\n+\t\tcomment => 'upload a file',\n+\t\tfile => [ $file_name ],\n+\t\tignorewarnings=>1,\n+\t}, {\n+\t\tskip_encoding => 1\n+\t} ) || die $mw->{error}->{code} . ' : ' . $mw->{error}->{details};\n+}\n+\n+\n+\n # Main part of this script: parse the command line arguments\n # and select which function to execute\n my $fct_to_call = shift;\n@@ -200,6 +215,7 @@ my $fct_to_call = shift;\n wiki_login($wiki_admin, $wiki_admin_pass);\n \n my %functions_to_call = qw(\n+ upload_file wiki_upload_file\n get_page wiki_getpage\n delete_page wiki_delete_page\n edit_page wiki_editpage\n-- \n1.7.11.1.147.g47a574d\n"},{"id":"194641","messageId":"1341473772-28023-8-git-send-email-Matthieu.Moy@imag.fr","threadId":"30954","inReplyTo":"1341473772-28023-1-git-send-email-Matthieu.Moy@imag.fr","subject":"[PATCH 07/12] git-remote-mediawiki (t9363): test 'File:' import and export","fromName":"Matthieu Moy","fromEmail":"matthieu.moy@imag.fr","sentAt":"2012-07-05T07:36:07Z","receivedAt":"2012-07-05T07:36:07Z","isPatch":true,"sender":{"key":"git@matthieu-moy.fr","avatar":"https://avatars.githubusercontent.com/u/14709?v=4"},"body":"From: NGUYEN Kim Thuat \n\nSigned-off-by: Pavel Volek \nSigned-off-by: NGUYEN Kim Thuat \nSigned-off-by: ROUCHER IGLESIAS Javier \nSigned-off-by: Matthieu Moy \n---\n .../mw-to-git/t/t9363-mw-to-git-export-import.sh | 198 +++++++++++++++++++++\n 1 file changed, 198 insertions(+)\n create mode 100755 contrib/mw-to-git/t/t9363-mw-to-git-export-import.sh\n\ndiff --git a/contrib/mw-to-git/t/t9363-mw-to-git-export-import.sh b/contrib/mw-to-git/t/t9363-mw-to-git-export-import.sh\nnew file mode 100755\nindex 0000000..3a4dce1\n--- /dev/null\n+++ b/contrib/mw-to-git/t/t9363-mw-to-git-export-import.sh\n@@ -0,0 +1,198 @@\n+#!/bin/sh\n+#\n+# Copyright (C) 2012\n+# Charles Roussel \n+# Simon Cathebras \n+# Julien Khayat \n+# Guillaume Sasdy \n+# Simon Perrat \n+#\n+# License: GPL v2 or later\n+\n+# tests for git-remote-mediawiki\n+\n+test_description='Test the Git Mediawiki remote helper: git push and git pull simple test cases'\n+\n+. ./test-gitmw-lib.sh\n+. $TEST_DIRECTORY/test-lib.sh\n+\n+\n+test_check_precond\n+\n+\n+test_git_reimport () {\n+\tgit -c remote.origin.dumbPush=true push &&\n+\tgit -c remote.origin.mediaImport=true pull --rebase\n+}\n+\n+# Don't bother with permissions, be administrator by default\n+test_expect_success 'setup config' '\n+\tgit config --global remote.origin.mwLogin WikiAdmin &&\n+\tgit config --global remote.origin.mwPassword AdminPass &&\n+\ttest_might_fail git config --global --unset remote.origin.mediaImport\n+'\n+\n+test_expect_success 'git push can upload media (File:) files' '\n+\twiki_reset &&\n+\tgit clone mediawiki::'\"$WIKI_URL\"' mw_dir &&\n+\t(\n+\t\tcd mw_dir &&\n+\t\techo \"hello world\" >Foo.txt &&\n+\t\tgit add Foo.txt &&\n+\t\tgit commit -m \"add a text file\" &&\n+\t\tgit push &&\n+\t\tperl -e \"print STDOUT \\\"binary content: \\\".chr(255);\" >Foo.txt &&\n+\t\tgit add Foo.txt &&\n+\t\tgit commit -m \"add a text file with binary content\" &&\n+\t\tgit push\n+\t)\n+'\n+\n+test_expect_success 'git clone works on previously created wiki with media files' '\n+\ttest_when_finished \"rm -rf mw_dir mw_dir_clone\" &&\n+\tgit clone -c remote.origin.mediaimport=true \\\n+\t\tmediawiki::'\"$WIKI_URL\"' mw_dir_clone &&\n+\ttest_cmp mw_dir_clone/Foo.txt mw_dir/Foo.txt &&\n+\t(cd mw_dir_clone && git checkout HEAD^) &&\n+\t(cd mw_dir && git checkout HEAD^) &&\n+\ttest_cmp mw_dir_clone/Foo.txt mw_dir/Foo.txt\n+'\n+\n+test_expect_success 'git push & pull work with locally renamed media files' '\n+\twiki_reset &&\n+\tgit clone mediawiki::'\"$WIKI_URL\"' mw_dir &&\n+\ttest_when_finished \"rm -fr mw_dir\" &&\n+\t(\n+\t\tcd mw_dir &&\n+\t\techo \"A File\" >Foo.txt &&\n+\t\tgit add Foo.txt &&\n+\t\tgit commit -m \"add a file\" &&\n+\t\tgit mv Foo.txt Bar.txt &&\n+\t\tgit commit -m \"Rename a file\" &&\n+\t\ttest_git_reimport &&\n+\t\techo \"A File\" >expect &&\n+\t\ttest_cmp expect Bar.txt &&\n+\t\ttest_path_is_missing Foo.txt\n+\t)\n+'\n+\n+test_expect_success 'git push can propagate local page deletion' '\n+\twiki_reset &&\n+\tgit clone mediawiki::'\"$WIKI_URL\"' mw_dir &&\n+\ttest_when_finished \"rm -fr mw_dir\" &&\n+\t(\n+\t\tcd mw_dir &&\n+\t\ttest_path_is_missing Foo.mw &&\n+\t\techo \"hello world\" >Foo.mw &&\n+\t\tgit add Foo.mw &&\n+\t\tgit commit -m \"Add the page Foo\" &&\n+\t\tgit push &&\n+\t\trm -f Foo.mw &&\n+\t\tgit commit -am \"Delete the page Foo\" &&\n+\t\ttest_git_reimport &&\n+\t\ttest_path_is_missing Foo.mw\n+\t)\n+'\n+\n+test_expect_success 'git push can propagate local media file deletion' '\n+\twiki_reset &&\n+\tgit clone mediawiki::'\"$WIKI_URL\"' mw_dir &&\n+\ttest_when_finished \"rm -fr mw_dir\" &&\n+\t(\n+\t\tcd mw_dir &&\n+\t\techo \"hello world\" >Foo.txt &&\n+\t\tgit add Foo.txt &&\n+\t\tgit commit -m \"Add the text file Foo\" &&\n+\t\tgit rm Foo.txt &&\n+\t\tgit commit -m \"Delete the file Foo\" &&\n+\t\ttest_git_reimport &&\n+\t\ttest_path_is_missing Foo.txt\n+\t)\n+'\n+\n+# test failure: the file is correctly uploaded, and then deleted but\n+# as no page link to it, the import (which looks at page revisions)\n+# doesn't notice the file deletion on the wiki. We fetch the list of\n+# files from the wiki, but as the file is deleted, it doesn't appear.\n+test_expect_failure 'git pull correctly imports media file deletion when no page link to it' '\n+\twiki_reset &&\n+\tgit clone mediawiki::'\"$WIKI_URL\"' mw_dir &&\n+\ttest_when_finished \"rm -fr mw_dir\" &&\n+\t(\n+\t\tcd mw_dir &&\n+\t\techo \"hello world\" >Foo.txt &&\n+\t\tgit add Foo.txt &&\n+\t\tgit commit -m \"Add the text file Foo\" &&\n+\t\tgit push &&\n+\t\tgit rm Foo.txt &&\n+\t\tgit commit -m \"Delete the file Foo\" &&\n+\t\ttest_git_reimport &&\n+\t\ttest_path_is_missing Foo.txt\n+\t)\n+'\n+\n+test_expect_success 'git push properly warns about insufficient permissions' '\n+\twiki_reset &&\n+\tgit clone mediawiki::'\"$WIKI_URL\"' mw_dir &&\n+\ttest_when_finished \"rm -fr mw_dir\" &&\n+\t(\n+\t\tcd mw_dir &&\n+\t\techo \"A File\" >foo.forbidden &&\n+\t\tgit add foo.forbidden &&\n+\t\tgit commit -m \"add a file\" &&\n+\t\tgit push 2>actual &&\n+\t\ttest_i18ngrep \"foo.forbidden is not a permitted file\" actual\n+\t)\n+'\n+\n+test_expect_success 'setup a repository with media files' '\n+\twiki_reset &&\n+\twiki_editpage testpage \"I am linking a file [[File:File.txt]]\" false &&\n+\techo \"File content\" >File.txt &&\n+\twiki_upload_file File.txt &&\n+\techo \"Another file content\" >AnotherFile.txt &&\n+\twiki_upload_file AnotherFile.txt\n+'\n+\n+test_expect_success 'git clone works with one specific page cloned and mediaimport=true' '\n+\tgit clone -c remote.origin.pages=testpage \\\n+\t\t -c remote.origin.mediaimport=true \\\n+\t\t\tmediawiki::'\"$WIKI_URL\"' mw_dir_15 &&\n+\ttest_when_finished \"rm -rf mw_dir_15\" &&\n+\ttest_contains_N_files mw_dir_15 3 &&\n+\ttest_path_is_file mw_dir_15/Testpage.mw &&\n+\ttest_path_is_file mw_dir_15/File:File.txt.mw &&\n+\ttest_path_is_file mw_dir_15/File.txt &&\n+\ttest_path_is_missing mw_dir_15/Main_Page.mw &&\n+\ttest_path_is_missing mw_dir_15/File:AnotherFile.txt.mw &&\n+\ttest_path_is_missing mw_dir_15/AnothetFile.txt &&\n+\twiki_check_content mw_dir_15/Testpage.mw Testpage &&\n+\ttest_cmp mw_dir_15/File.txt File.txt\n+'\n+\n+test_expect_success 'git clone works with one specific page cloned and mediaimport=false' '\n+\ttest_when_finished \"rm -rf mw_dir_16\" &&\n+\tgit clone -c remote.origin.pages=testpage \\\n+\t\t\tmediawiki::'\"$WIKI_URL\"' mw_dir_16 &&\n+\ttest_contains_N_files mw_dir_16 1 &&\n+\ttest_path_is_file mw_dir_16/Testpage.mw &&\n+\ttest_path_is_missing mw_dir_16/File:File.txt.mw &&\n+\ttest_path_is_missing mw_dir_16/File.txt &&\n+\ttest_path_is_missing mw_dir_16/Main_Page.mw &&\n+\twiki_check_content mw_dir_16/Testpage.mw Testpage\n+'\n+\n+# should behave like mediaimport=false\n+test_expect_success 'git clone works with one specific page cloned and mediaimport unset' '\n+\ttest_when_finished \"rm -fr mw_dir_17\" &&\n+\tgit clone -c remote.origin.pages=testpage \\\n+\t\tmediawiki::'\"$WIKI_URL\"' mw_dir_17 &&\n+\ttest_contains_N_files mw_dir_17 1 &&\n+\ttest_path_is_file mw_dir_17/Testpage.mw &&\n+\ttest_path_is_missing mw_dir_17/File:File.txt.mw &&\n+\ttest_path_is_missing mw_dir_17/File.txt &&\n+\ttest_path_is_missing mw_dir_17/Main_Page.mw &&\n+\twiki_check_content mw_dir_17/Testpage.mw Testpage\n+'\n+\n+test_done\n-- \n1.7.11.1.147.g47a574d\n"},{"id":"194639","messageId":"1341473772-28023-9-git-send-email-Matthieu.Moy@imag.fr","threadId":"30954","inReplyTo":"1341473772-28023-1-git-send-email-Matthieu.Moy@imag.fr","subject":"[PATCH 08/12] git-remote-mediawiki: change return type of get_mw_pages","fromName":"Matthieu Moy","fromEmail":"matthieu.moy@imag.fr","sentAt":"2012-07-05T07:36:08Z","receivedAt":"2012-07-05T07:36:08Z","isPatch":true,"sender":{"key":"git@matthieu-moy.fr","avatar":"https://avatars.githubusercontent.com/u/14709?v=4"},"body":"The previous version was returning the list of pages to be fetched, but\nwe are going to need an efficient membership test (i.e. is the page\n$title tracked), hence exposing a hash will be more convenient.\n\nSigned-off-by: Matthieu Moy \n---\n contrib/mw-to-git/git-remote-mediawiki | 8 +++++---\n 1 file changed, 5 insertions(+), 3 deletions(-)\n\ndiff --git a/contrib/mw-to-git/git-remote-mediawiki b/contrib/mw-to-git/git-remote-mediawiki\nindex 76b78bc..3f57598 100755\n--- a/contrib/mw-to-git/git-remote-mediawiki\n+++ b/contrib/mw-to-git/git-remote-mediawiki\n@@ -377,7 +377,7 @@ sub get_mw_pages {\n \t\t\tget_all_mediafiles(\\%pages);\n \t\t}\n \t}\n-\treturn values(%pages);\n+\treturn %pages;\n }\n \n # usage: $out = run_git(\"command args\");\n@@ -563,7 +563,8 @@ my %basetimestamps;\n sub get_last_remote_revision {\n \tmw_connect_maybe();\n \n-\tmy @pages = get_mw_pages();\n+\tmy %pages_hash = get_mw_pages();\n+\tmy @pages = values(%pages_hash);\n \n \tmy $max_rev_num = 0;\n \n@@ -829,7 +830,8 @@ sub mw_import_ref {\n \n \tmw_connect_maybe();\n \n-\tmy @pages = get_mw_pages();\n+\tmy %pages_hash = get_mw_pages();\n+\tmy @pages = values(%pages_hash);\n \n \tprint STDERR \"Searching revisions...\\n\";\n \tmy $last_local = get_last_local_revision();\n-- \n1.7.11.1.147.g47a574d\n"},{"id":"194642","messageId":"1341473772-28023-10-git-send-email-Matthieu.Moy@imag.fr","threadId":"30954","inReplyTo":"1341473772-28023-1-git-send-email-Matthieu.Moy@imag.fr","subject":"[PATCH 09/12] git-remote-mediawiki: refactor loop over revision ids","fromName":"Matthieu Moy","fromEmail":"matthieu.moy@imag.fr","sentAt":"2012-07-05T07:36:09Z","receivedAt":"2012-07-05T07:36:09Z","isPatch":true,"sender":{"key":"git@matthieu-moy.fr","avatar":"https://avatars.githubusercontent.com/u/14709?v=4"},"body":"Without changing the behavior, we turn the foreach loop on an array of\nrevisions into a loop on an array of integer. It will be easier to\nimplement other strategies as they will only need to produce an array of\ninteger instead of a more complex data-structure.\n\nSigned-off-by: Matthieu Moy \n---\n contrib/mw-to-git/git-remote-mediawiki | 17 +++++++++++------\n 1 file changed, 11 insertions(+), 6 deletions(-)\n\ndiff --git a/contrib/mw-to-git/git-remote-mediawiki b/contrib/mw-to-git/git-remote-mediawiki\nindex 3f57598..e19402c 100755\n--- a/contrib/mw-to-git/git-remote-mediawiki\n+++ b/contrib/mw-to-git/git-remote-mediawiki\n@@ -846,30 +846,35 @@ sub mw_import_ref {\n \t# Creation of the fast-import stream\n \tprint STDERR \"Fetching & writing export data...\\n\";\n \n+\t@revisions = sort {$a->{revid} <=> $b->{revid}} @revisions;\n+\tmy @revision_ids = map $_->{revid}, @revisions;\n+\n \t$n = 0;\n \tmy $last_timestamp = 0; # Placeholer in case $rev->timestamp is undefined\n \n-\tforeach my $pagerevid (sort {$a->{revid} <=> $b->{revid}} @revisions) {\n+\tforeach my $pagerevid (@revision_ids) {\n \t\t# fetch the content of the pages\n \t\tmy $query = {\n \t\t\taction => 'query',\n \t\t\tprop => 'revisions',\n \t\t\trvprop => 'content|timestamp|comment|user|ids',\n-\t\t\trevids => $pagerevid->{revid},\n+\t\t\trevids => $pagerevid,\n \t\t};\n \n \t\tmy $result = $mediawiki->api($query);\n \n-\t\tmy $rev = pop(@{$result->{query}->{pages}->{$pagerevid->{pageid}}->{revisions}});\n+\t\tmy @result_pages = values(%{$result->{query}->{pages}});\n+\t\tmy $result_page = $result_pages[0];\n+\t\tmy $rev = $result_pages[0]->{revisions}->[0];\n \n \t\t$n++;\n \n-\t\tmy $page_title = $result->{query}->{pages}->{$pagerevid->{pageid}}->{title};\n+\t\tmy $page_title = $result_page->{title};\n \t\tmy %commit;\n \t\t$commit{author} = $rev->{user} || 'Anonymous';\n \t\t$commit{comment} = $rev->{comment} || '*Empty MediaWiki Message*';\n \t\t$commit{title} = mediawiki_smudge_filename($page_title);\n-\t\t$commit{mw_revision} = $pagerevid->{revid};\n+\t\t$commit{mw_revision} = $rev->{revid};\n \t\t$commit{content} = mediawiki_smudge($rev->{'*'});\n \n \t\tif (!defined($rev->{timestamp})) {\n@@ -891,7 +896,7 @@ sub mw_import_ref {\n \t\t# If this is a revision of the media page for new version\n \t\t# of a file do one common commit for both file and media page.\n \t\t# Else do commit only for that page.\n-\t\tprint STDERR \"$n/\", scalar(@revisions), \": Revision #$pagerevid->{revid} of $commit{title}\\n\";\n+\t\tprint STDERR \"$n/\", scalar(@revision_ids), \": Revision #$rev->{revid} of $commit{title}\\n\";\n \t\tif (%mediafile) {\n \t\t\tprint STDERR \"\\tDownloading file $mediafile{title}, version $mediafile{timestamp}\\n\";\n \t\t\timport_file_revision(\\%commit, ($fetch_from == 1), $n, \\%mediafile);\n-- \n1.7.11.1.147.g47a574d\n"},{"id":"194640","messageId":"1341473772-28023-11-git-send-email-Matthieu.Moy@imag.fr","threadId":"30954","inReplyTo":"1341473772-28023-1-git-send-email-Matthieu.Moy@imag.fr","subject":"[PATCH 10/12] git-remote-mediawiki: extract revision-importing loop to a function","fromName":"Matthieu Moy","fromEmail":"matthieu.moy@imag.fr","sentAt":"2012-07-05T07:36:10Z","receivedAt":"2012-07-05T07:36:10Z","isPatch":true,"sender":{"key":"git@matthieu-moy.fr","avatar":"https://avatars.githubusercontent.com/u/14709?v=4"},"body":"\nSigned-off-by: Matthieu Moy \n---\n contrib/mw-to-git/git-remote-mediawiki | 16 +++++++++++++---\n 1 file changed, 13 insertions(+), 3 deletions(-)\n\ndiff --git a/contrib/mw-to-git/git-remote-mediawiki b/contrib/mw-to-git/git-remote-mediawiki\nindex e19402c..beb4566 100755\n--- a/contrib/mw-to-git/git-remote-mediawiki\n+++ b/contrib/mw-to-git/git-remote-mediawiki\n@@ -849,10 +849,18 @@ sub mw_import_ref {\n \t@revisions = sort {$a->{revid} <=> $b->{revid}} @revisions;\n \tmy @revision_ids = map $_->{revid}, @revisions;\n \n-\t$n = 0;\n+\treturn mw_import_revids($fetch_from, \\@revision_ids, \\%pages_hash);\n+}\n+\n+sub mw_import_revids {\n+\tmy $fetch_from = shift;\n+\tmy $revision_ids = shift;\n+\tmy $pages = shift;\n+\n+\tmy $n = 0;\n \tmy $last_timestamp = 0; # Placeholer in case $rev->timestamp is undefined\n \n-\tforeach my $pagerevid (@revision_ids) {\n+\tforeach my $pagerevid (@$revision_ids) {\n \t\t# fetch the content of the pages\n \t\tmy $query = {\n \t\t\taction => 'query',\n@@ -896,7 +904,7 @@ sub mw_import_ref {\n \t\t# If this is a revision of the media page for new version\n \t\t# of a file do one common commit for both file and media page.\n \t\t# Else do commit only for that page.\n-\t\tprint STDERR \"$n/\", scalar(@revision_ids), \": Revision #$rev->{revid} of $commit{title}\\n\";\n+\t\tprint STDERR \"$n/\", scalar(@$revision_ids), \": Revision #$rev->{revid} of $commit{title}\\n\";\n \t\tif (%mediafile) {\n \t\t\tprint STDERR \"\\tDownloading file $mediafile{title}, version $mediafile{timestamp}\\n\";\n \t\t\timport_file_revision(\\%commit, ($fetch_from == 1), $n, \\%mediafile);\n@@ -911,6 +919,8 @@ sub mw_import_ref {\n \t\t# thrown saying that HEAD is refering to unknown object 0000000000000000000\n \t\t# and the clone fails.\n \t}\n+\n+\treturn $n;\n }\n \n sub error_non_fast_forward {\n-- \n1.7.11.1.147.g47a574d\n"},{"id":"194644","messageId":"1341473772-28023-12-git-send-email-Matthieu.Moy@imag.fr","threadId":"30954","inReplyTo":"1341473772-28023-1-git-send-email-Matthieu.Moy@imag.fr","subject":"[PATCH 11/12] git-remote-mediawiki: more efficient 'pull' in the best case","fromName":"Matthieu Moy","fromEmail":"matthieu.moy@imag.fr","sentAt":"2012-07-05T07:36:11Z","receivedAt":"2012-07-05T07:36:11Z","isPatch":true,"sender":{"key":"git@matthieu-moy.fr","avatar":"https://avatars.githubusercontent.com/u/14709?v=4"},"body":"The only way to fetch new revisions from a wiki before this patch was to\nquery each page for new revisions. This is good when tracking a small set\nof pages on a large wiki, but very inefficient when tracking many pages\non a wiki with little activity.\n\nImplement a new strategy that queries the wiki for its last global\nrevision, queries each new revision, and filter out pages that are not\ntracked.\n\nSigned-off-by: Simon Perrat \nSigned-off-by: Simon CATHEBRAS \nSigned-off-by: Julien KHAYAT \nSigned-off-by: Charles ROUSSEL \nSigned-off-by: Guillaume SASDY \nSigned-off-by: Matthieu Moy \n---\n contrib/mw-to-git/git-remote-mediawiki | 116 +++++++++++++++++++++++++------\n contrib/mw-to-git/t/t9364-pull-by-rev.sh | 17 +++++\n 2 files changed, 111 insertions(+), 22 deletions(-)\n create mode 100755 contrib/mw-to-git/t/t9364-pull-by-rev.sh\n\ndiff --git a/contrib/mw-to-git/git-remote-mediawiki b/contrib/mw-to-git/git-remote-mediawiki\nindex beb4566..dd18142 100755\n--- a/contrib/mw-to-git/git-remote-mediawiki\n+++ b/contrib/mw-to-git/git-remote-mediawiki\n@@ -13,12 +13,9 @@\n #\n # Known limitations:\n #\n-# - Poor performance in the best case: it takes forever to check\n-# whether we're up-to-date (on fetch or push) or to fetch a few\n-# revisions from a large wiki, because we use exclusively a\n-# page-based synchronization. We could switch to a wiki-wide\n-# synchronization when the synchronization involves few revisions\n-# but the wiki is large.\n+# - Several strategies are provided to fetch modifications from the\n+# wiki, but no automatic heuristics is provided, the user has\n+# to understand and chose which strategy is appropriate for him.\n #\n # - Git renames could be turned into MediaWiki renames (see TODO\n # below)\n@@ -93,6 +90,21 @@ $shallow_import = ($shallow_import eq \"true\");\n # Cache for MediaWiki namespace ids.\n my %namespace_id;\n \n+# Fetch (clone and pull) by revisions instead of by pages. This behavior\n+# is more efficient when we have a wiki with lots of pages and we fetch\n+# the revisions quite often so that they concern only few pages.\n+# Possible values:\n+# - by_rev: perform one query per new revision on the remote wiki\n+# - by_page: query each tracked page for new revision\n+my $fetch_strategy = run_git(\"config --get remote.$remotename.fetchStrategy\");\n+unless ($fetch_strategy) {\n+\t$fetch_strategy = run_git(\"config --get mediawiki.fetchStrategy\");\n+}\n+chomp($fetch_strategy);\n+unless ($fetch_strategy) {\n+\t$fetch_strategy = \"by_page\";\n+}\n+\n # Dumb push: don't update notes and mediawiki ref to reflect the last push.\n #\n # Configurable with mediawiki.dumbPush, or per-remote with\n@@ -560,6 +572,26 @@ sub get_last_local_revision {\n # Remember the timestamp corresponding to a revision id.\n my %basetimestamps;\n \n+# Get the last remote revision without taking in account which pages are\n+# tracked or not. This function makes a single request to the wiki thus\n+# avoid a loop onto all tracked pages. This is useful for the fetch-by-rev\n+# option.\n+sub get_last_global_remote_rev {\n+\tmw_connect_maybe();\n+\n+\tmy $query = {\n+\t\taction => 'query',\n+\t\tlist => 'recentchanges',\n+\t\tprop => 'revisions',\n+\t\trclimit => '1',\n+\t\trcdir => 'older',\n+\t};\n+\tmy $result = $mediawiki->api($query);\n+\treturn $result->{query}->{recentchanges}[0]->{revid};\n+}\n+\n+# Get the last remote revision concerning the tracked pages and the tracked\n+# categories.\n sub get_last_remote_revision {\n \tmw_connect_maybe();\n \n@@ -830,9 +862,6 @@ sub mw_import_ref {\n \n \tmw_connect_maybe();\n \n-\tmy %pages_hash = get_mw_pages();\n-\tmy @pages = values(%pages_hash);\n-\n \tprint STDERR \"Searching revisions...\\n\";\n \tmy $last_local = get_last_local_revision();\n \tmy $fetch_from = $last_local + 1;\n@@ -841,10 +870,35 @@ sub mw_import_ref {\n \t} else {\n \t\tprint STDERR \", fetching from here.\\n\";\n \t}\n-\tmy ($n, @revisions) = fetch_mw_revisions(\\@pages, $fetch_from);\n \n-\t# Creation of the fast-import stream\n-\tprint STDERR \"Fetching & writing export data...\\n\";\n+\tmy $n = 0;\n+\tif ($fetch_strategy eq \"by_rev\") {\n+\t\tprint STDERR \"Fetching & writing export data by revs...\\n\";\n+\t\t$n = mw_import_ref_by_revs($fetch_from);\n+\t} elsif ($fetch_strategy eq \"by_page\") {\n+\t\tprint STDERR \"Fetching & writing export data by pages...\\n\";\n+\t\t$n = mw_import_ref_by_pages($fetch_from);\n+\t} else {\n+\t\tprint STDERR \"fatal: invalid fetch strategy \\\"$fetch_strategy\\\".\\n\";\n+\t\tprint STDERR \"Check your configuration variables remote.$remotename.fetchStrategy and mediawiki.fetchStrategy\\n\";\n+\t\texit 1;\n+\t}\n+\n+\tif ($fetch_from == 1 && $n == 0) {\n+\t\tprint STDERR \"You appear to have cloned an empty MediaWiki.\\n\";\n+\t\t# Something has to be done remote-helper side. If nothing is done, an error is\n+\t\t# thrown saying that HEAD is refering to unknown object 0000000000000000000\n+\t\t# and the clone fails.\n+\t}\n+}\n+\n+sub mw_import_ref_by_pages {\n+\n+\tmy $fetch_from = shift;\n+\tmy %pages_hash = get_mw_pages();\n+\tmy @pages = values(%pages_hash);\n+\n+\tmy ($n, @revisions) = fetch_mw_revisions(\\@pages, $fetch_from);\n \n \t@revisions = sort {$a->{revid} <=> $b->{revid}} @revisions;\n \tmy @revision_ids = map $_->{revid}, @revisions;\n@@ -852,12 +906,26 @@ sub mw_import_ref {\n \treturn mw_import_revids($fetch_from, \\@revision_ids, \\%pages_hash);\n }\n \n+sub mw_import_ref_by_revs {\n+\n+\tmy $fetch_from = shift;\n+\tmy %pages_hash = get_mw_pages();\n+\n+\tmy $last_remote = get_last_global_remote_rev();\n+\tmy @revision_ids = $fetch_from..$last_remote;\n+\treturn mw_import_revids($fetch_from, \\@revision_ids, \\%pages_hash);\n+}\n+\n+# Import revisions given in second argument (array of integers).\n+# Only pages appearing in the third argument (hash indexed by page titles)\n+# will be imported.\n sub mw_import_revids {\n \tmy $fetch_from = shift;\n \tmy $revision_ids = shift;\n \tmy $pages = shift;\n \n \tmy $n = 0;\n+\tmy $n_actual = 0;\n \tmy $last_timestamp = 0; # Placeholer in case $rev->timestamp is undefined\n \n \tforeach my $pagerevid (@$revision_ids) {\n@@ -875,9 +943,20 @@ sub mw_import_revids {\n \t\tmy $result_page = $result_pages[0];\n \t\tmy $rev = $result_pages[0]->{revisions}->[0];\n \n+\t # Count page even if we skip it, since we display\n+\t\t# $n/$total and $total includes skipped pages.\n \t\t$n++;\n \n \t\tmy $page_title = $result_page->{title};\n+\n+\t\tif (!exists($pages->{$page_title})) {\n+\t\t\tprint STDERR \"$n/\", scalar(@$revision_ids),\n+\t\t\t\t\": Skipping revision #$rev->{revid} of $page_title\\n\";\n+\t\t\tnext;\n+\t\t}\n+\n+\t\t$n_actual++;\n+\n \t\tmy %commit;\n \t\t$commit{author} = $rev->{user} || 'Anonymous';\n \t\t$commit{comment} = $rev->{comment} || '*Empty MediaWiki Message*';\n@@ -907,20 +986,13 @@ sub mw_import_revids {\n \t\tprint STDERR \"$n/\", scalar(@$revision_ids), \": Revision #$rev->{revid} of $commit{title}\\n\";\n \t\tif (%mediafile) {\n \t\t\tprint STDERR \"\\tDownloading file $mediafile{title}, version $mediafile{timestamp}\\n\";\n-\t\t\timport_file_revision(\\%commit, ($fetch_from == 1), $n, \\%mediafile);\n+\t\t\timport_file_revision(\\%commit, ($fetch_from == 1), $n_actual, \\%mediafile);\n \t\t} else {\n-\t\t\timport_file_revision(\\%commit, ($fetch_from == 1), $n);\n+\t\t\timport_file_revision(\\%commit, ($fetch_from == 1), $n_actual);\n \t\t}\n \t}\n \n-\tif ($fetch_from == 1 && $n == 0) {\n-\t\tprint STDERR \"You appear to have cloned an empty MediaWiki.\\n\";\n-\t\t# Something has to be done remote-helper side. If nothing is done, an error is\n-\t\t# thrown saying that HEAD is refering to unknown object 0000000000000000000\n-\t\t# and the clone fails.\n-\t}\n-\n-\treturn $n;\n+\treturn $n_actual;\n }\n \n sub error_non_fast_forward {\ndiff --git a/contrib/mw-to-git/t/t9364-pull-by-rev.sh b/contrib/mw-to-git/t/t9364-pull-by-rev.sh\nnew file mode 100755\nindex 0000000..5c22457\n--- /dev/null\n+++ b/contrib/mw-to-git/t/t9364-pull-by-rev.sh\n@@ -0,0 +1,17 @@\n+#!/bin/sh\n+\n+test_description='Test the Git Mediawiki remote helper: git pull by revision'\n+\n+. ./test-gitmw-lib.sh\n+. ./push-pull-tests.sh\n+. $TEST_DIRECTORY/test-lib.sh\n+\n+test_check_precond\n+\n+test_expect_success 'configuration' '\n+\tgit config --global mediawiki.fetchStrategy by_rev\n+'\n+\n+test_push_pull\n+\n+test_done\n-- \n1.7.11.1.147.g47a574d\n"},{"id":"194643","messageId":"1341473772-28023-13-git-send-email-Matthieu.Moy@imag.fr","threadId":"30954","inReplyTo":"1341473772-28023-1-git-send-email-Matthieu.Moy@imag.fr","subject":"[PATCH 12/12] git-remote-mediawiki: be more defensive when requests fail","fromName":"Matthieu Moy","fromEmail":"matthieu.moy@imag.fr","sentAt":"2012-07-05T07:36:12Z","receivedAt":"2012-07-05T07:36:12Z","isPatch":true,"sender":{"key":"git@matthieu-moy.fr","avatar":"https://avatars.githubusercontent.com/u/14709?v=4"},"body":"\nSigned-off-by: Matthieu Moy \n---\n contrib/mw-to-git/git-remote-mediawiki | 8 ++++++++\n 1 file changed, 8 insertions(+)\n\ndiff --git a/contrib/mw-to-git/git-remote-mediawiki b/contrib/mw-to-git/git-remote-mediawiki\nindex dd18142..63f02b8 100755\n--- a/contrib/mw-to-git/git-remote-mediawiki\n+++ b/contrib/mw-to-git/git-remote-mediawiki\n@@ -939,6 +939,14 @@ sub mw_import_revids {\n \n \t\tmy $result = $mediawiki->api($query);\n \n+\t\tif (!$result) {\n+\t\t\tdie \"Failed to retrieve modified page for revision $pagerevid\";\n+\t\t}\n+\n+\t\tif (!defined($result->{query}->{pages})) {\n+\t\t\tdie \"Invalid revision $pagerevid.\";\n+\t\t}\n+\n \t\tmy @result_pages = values(%{$result->{query}->{pages}});\n \t\tmy $result_page = $result_pages[0];\n \t\tmy $rev = $result_pages[0]->{revisions}->[0];\n-- \n1.7.11.1.147.g47a574d\n"},{"id":"194684","messageId":"7vmx3dpzsn.fsf@alter.siamese.dyndns.org","threadId":"30954","inReplyTo":"1341473772-28023-3-git-send-email-Matthieu.Moy@imag.fr","subject":"Re: [PATCH 02/12] git-remote-mediawiki: test environment of git-remote-mediawiki","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2012-07-05T23:13:12Z","receivedAt":"2012-07-05T23:13:12Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Matthieu Moy writes:\n\n> From: Guillaume Sasdy \n>\n> In order to test git-remote-mediawiki, a set of functions is needed to\n> manage a MediaWiki: edit a page, remove a page, fetch a page, fetch all\n> pages on a given wiki.\n>\n> We also provide a few helper function to check the content of\n> directories.\n>\n> In addition, this patch provides Makefiles to execute tests.\n> See the README file for more details.\n>\n> Signed-off-by: Simon CATHEBRAS \n> Signed-off-by: Julien KHAYAT \n> Signed-off-by: Simon Perrat \n> Signed-off-by: Charles ROUSSEL \n> Signed-off-by: Guillaume SASDY \n> Signed-off-by: Matthieu Moy \n> ---\n> contrib/mw-to-git/Makefile | 48 ++++++++\n> contrib/mw-to-git/t/.gitignore | 2 +\n> contrib/mw-to-git/t/Makefile | 32 ++++++\n> contrib/mw-to-git/t/README | 124 ++++++++++++++++++++\n> contrib/mw-to-git/t/test-gitmw-lib.sh | 143 +++++++++++++++++++++++\n> contrib/mw-to-git/t/test-gitmw.pl | 209 ++++++++++++++++++++++++++++++++++\n> 6 files changed, 558 insertions(+)\n> create mode 100644 contrib/mw-to-git/Makefile\n> create mode 100644 contrib/mw-to-git/t/Makefile\n> create mode 100644 contrib/mw-to-git/t/README\n> create mode 100755 contrib/mw-to-git/t/test-gitmw.pl\n>\n> diff --git a/contrib/mw-to-git/Makefile b/contrib/mw-to-git/Makefile\n> new file mode 100644\n> index 0000000..c9b1795\n> --- /dev/null\n> +++ b/contrib/mw-to-git/Makefile\n> @@ -0,0 +1,48 @@\n> +#\n> +# Copyright (C) 2012\n> +# Charles Roussel \n> +# Simon Cathebras \n> +# Julien Khayat \n> +# Guillaume Sasdy \n> +# Simon Perrat \n> +#\n> +## Build git-remote-mediawiki\n> +\n> +-include ../../config.mak.autogen\n> +-include ../../config.mak\n> +\n> +ifndef PERL_PATH\n> +\tPERL_PATH = /usr/bin/perl\n> +endif\n> +ifndef gitexecdir\n> +\tgitexecdir = $(shell git --exec-path)\n> +endif\n> +\n> +PERL_PATH_SQ = $(subst ','\\'',$(PERL_PATH))\n> +gitexecdir_SQ = $(subst ','\\'',$(gitexecdir))\n> +SCRIPT = git-remote-mediawiki\n> +\n> +.PHONY: install help doc test clean\n> +\n> +help:\n> +\t@echo 'This is the help target of the Makefile. Current configuration:'\n> +\t@echo ' gitexecdir = $(gitexecdir_SQ)'\n> +\t@echo ' PERL_PATH = $(PERL_PATH_SQ)'\n> +\t@echo 'Run \"$(MAKE) install\" to install $(SCRIPT) in gitexecdir'\n> +\t@echo 'Run \"$(MAKE) test\" to run the testsuite'\n> +\n> +install:\n> +\tsed -e '1s|#!.*/perl|#!$(PERL_PATH_SQ)|' $(SCRIPT) \\\n> + > '$(gitexecdir_SQ)/$(SCRIPT)'\n> +\tchmod +x '$(gitexecdir)/$(SCRIPT)'\n> +\n> +doc:\n> +\t@echo 'Sorry, \"make doc\" is not implemented yet for $(SCRIPT)'\n> +\n> +test:\n> +\t$(MAKE) -C t/ test\n> +\n> +clean:\n> +\t$(RM) '$(gitexecdir)/$(SCRIPT)'\n> +\t$(MAKE) -C t/ clean\n> +\n\nNew blank link at EOF.\n\n> diff --git a/contrib/mw-to-git/t/.gitignore b/contrib/mw-to-git/t/.gitignore\n> index a060414..a7a40b4 100644\n> --- a/contrib/mw-to-git/t/.gitignore\n> +++ b/contrib/mw-to-git/t/.gitignore\n> @@ -1,2 +1,4 @@\n> WEB/\n> wiki/\n> +trash directory.t*/\n> +test-results/\n> diff --git a/contrib/mw-to-git/t/Makefile b/contrib/mw-to-git/t/Makefile\n> new file mode 100644\n> index 0000000..1ebe1c7\n> --- /dev/null\n> +++ b/contrib/mw-to-git/t/Makefile\n> @@ -0,0 +1,32 @@\n> +#\n> +# Copyright (C) 2012\n> +# Charles Roussel \n> +# Simon Cathebras \n> +# Julien Khayat \n> +# Guillaume Sasdy \n> +# Simon Perrat \n> +#\n> +## Test git-remote-mediawiki\n> +\n> +all: test\n> +\n> +-include ../../../config.mak.autogen\n> +-include ../../../config.mak\n> +\n> +T = $(wildcard t[0-9][0-9][0-9][0-9]-*.sh)\n> +\n> +.PHONY: help test clean all\n> +\n> +help:\n> +\t@echo 'Run \"$(MAKE) test\" to launch test scripts'\n> +\t@echo 'Run \"$(MAKE) clean\" to remove trash folders'\n> +\n> +test:\n> +\t@for t in $(T); do \\\n> +\t\techo \"$$t\"; \\\n> +\t\t\"./$$t\" || exit 1; \\\n> +\tdone\n> +\n> +clean:\n> +\t$(RM) -r 'trash directory'.*\n> +\n\nNew blank line at EOF.\n\n> diff --git a/contrib/mw-to-git/t/README b/contrib/mw-to-git/t/README\n> new file mode 100644\n> index 0000000..96e9739\n> --- /dev/null\n> +++ b/contrib/mw-to-git/t/README\n> @@ -0,0 +1,124 @@\n> +Tests for Mediawiki-to-Git\n> +==========================\n> +\n> +Introduction\n> +------------\n> +This manual describes how to install the git-remote-mediawiki test\n> +environment on a machine with git installed on it.\n> +\n> +Prerequisite\n> +------------\n> +\n> +In order to run this test environment correctly, you will need to\n> +install the following packages (Debian/Ubuntu names, may need to be\n> +adapted for another distribution):\n> +\n> +* lighttpd\n> +* php5\n> +* php5-cgi\n> +* php5-cli\n> +* php5-curl\n> +* php5-sqlite\n> +\n> +Principles and Technical Choices\n> +--------------------------------\n> +\n> +The test environment makes it easy to install and manipulate one or\n> +several MediaWiki instances. To allow developers to run the testsuite\n> +easily, the environment does not require root priviledge (except to\n> +install the required packages if needed). It starts a webserver\n> +instance on the user's account (using lighttpd greatly helps for\n> +that), and does not need a separate database daemon (thanks to the use\n> +of sqlite).\n> +\n> +Run the test environment\n> +------------------------\n> +\n> +Install a new wiki\n> +~~~~~~~~~~~~~~~~~~\n> +\n> +Once you have all the prerequisite, you need to install a MediaWiki\n> +instance on your machine. If you already have one, it is still\n> +strongly recommended to install one with the script provided. Here's\n> +how to work it:\n> +\n> +a. change directory to contrib/mw-to-git/t/\n> +b. if needed, edit test.config to choose your installation parameters\n> +c. run `./install-wiki.sh install`\n> +d. check on your favourite web browser if your wiki is correctly\n> + installed.\n> +\n> +Remove an existing wiki\n> +~~~~~~~~~~~~~~~~~~~~~~~\n> +\n> +Edit the file test.config to fit the wiki you want to delete, and then\n> +execute the command `./install-wiki.sh delete` from the\n> +contrib/mw-to-git/t directory.\n> +\n> +Run the existing tests\n> +~~~~~~~~~~~~~~~~~~~~~~\n> +\n> +The provided tests are currently in the `contrib/mw-to-git/t` directory.\n> +The files are all the t936[0-9]-*.sh shell scripts.\n> +\n> +a. Run all tests:\n> +To do so, run \"make test\" from the contrib/mw-to-git/ directory.\n> +\n> +b. Run a specific test:\n> +To run a given test , run ./ from the\n> +contrib/mw-to-git/t directory.\n> +\n> +How to create new tests\n> +-----------------------\n> +\n> +Available functions\n> +~~~~~~~~~~~~~~~~~~~\n> +\n> +The test environment of git-remote-mediawiki provides some functions\n> +useful to test its behaviour. for more details about the functions'\n> +parameters, please refer to the `test-gitmw-lib.sh` and\n> +`test-gitmw.pl` files.\n> +\n> +** `test_check_wiki_precond`:\n> +Check if the tests must be skipped or not. Please use this function\n> +at the beggining of each new test file.\n> +\n> +** `wiki_getpage`:\n> +Fetch a given page from the wiki and puts its content in the\n> +directory in parameter.\n> +\n> +** `wiki_delete_page`:\n> +Delete a given page from the wiki.\n> +\n> +** `wiki_edit_page`:\n> +Create or modify a given page in the wiki. You can specify several\n> +parameters like a summary for the page edition, or add the page to a\n> +given category.\n> +See test-gitmw.pl for more details.\n> +\n> +** `wiki_getallpage`:\n> +Fetch all pages from the wiki into a given directory. The directory\n> +is created if it does not exists.\n> +\n> +** `test_diff_directories`:\n> +Compare the content of two directories. The content must be the same.\n> +Use this function to compare the content of a git directory and a wiki\n> +one created by wiki_getallpage.\n> +\n> +** `test_contains_N_files`:\n> +Check if the given directory contains a given number of file.\n> +\n> +** `wiki_page_exists`:\n> +Tests if a given page exists on the wiki.\n> +\n> +** `wiki_reset`:\n> +Reset the wiki, i.e. flush the database. Use this function at the\n> +begining of each new test, except if the test re-uses the same wiki\n> +(and history) as the previous test.\n> +\n> +How to write a new test\n> +~~~~~~~~~~~~~~~~~~~~~~~\n> +\n> +Please, follow the standards given by git. See git/t/README.\n> +New file should be named as t936[0-9]-*.sh.\n> +Be sure to reset your wiki regulary with the function `wiki_reset`.\n> diff --git a/contrib/mw-to-git/t/test-gitmw-lib.sh b/contrib/mw-to-git/t/test-gitmw-lib.sh\n> index 47341f7..14f6577 100755\n> --- a/contrib/mw-to-git/t/test-gitmw-lib.sh\n> +++ b/contrib/mw-to-git/t/test-gitmw-lib.sh\n> @@ -13,6 +13,7 @@\n> \n> . ./test.config\n> \n> +WIKI_URL=http://\"$SERVER_ADDR:$PORT/$WIKI_DIR_NAME\"\n> CURR_DIR=$(pwd)\n> TEST_OUTPUT_DIRECTORY=$(pwd)\n> TEST_DIRECTORY=\"$CURR_DIR\"/../../../t\n> @@ -25,6 +26,148 @@ else\n> \tWIKI_DIR_INST=\"$CURR_DIR/$WEB_WWW\"\n> fi\n> \n> +\n> +wiki_getpage () {\n> +\t\"$CURR_DIR\"/test-gitmw.pl get_page \"$@\"\n> +}\n> +\n> +wiki_delete_page () {\n> +\t\"$CURR_DIR\"/test-gitmw.pl delete_page \"$@\"\n> +}\n> +\n> +wiki_editpage () {\n> +\t\"$CURR_DIR\"/test-gitmw.pl edit_page \"$@\"\n> +}\n> +\n> +die () {\n> +\tdie_with_status 1 \"$@\"\n> +}\n> +\n> +die_with_status () {\n> +\tstatus=$1\n> +\tshift\n> +\techo >&2 \"$*\"\n> +\texit \"$status\"\n> +}\n> +\n> +\n> +# Check the preconditions to run git-remote-mediawiki's tests\n> +test_check_precond () {\n> +\tif ! test_have_prereq PERL\n> +\tthen\n> +\t\tskip_all='skipping gateway git-mw tests, perl not available'\n> +\t\ttest_done\n> +\tfi\n> +\n> +\tif [ ! -f \"$GIT_BUILD_DIR\"/git-remote-mediawiki ];\n> +\tthen\n> +\t\techo \"No remote mediawiki for git found. Copying it in git\"\n> +\t\techo \"cp $GIT_BUILD_DIR/contrib/mw-to-git/git-remote-mediawiki $GIT_BUILD_DIR/\"\n> +\t\tln -s \"$GIT_BUILD_DIR\"/contrib/mw-to-git/git-remote-mediawiki \"$GIT_BUILD_DIR\"\n> +\tfi\n> +\n> +\tif [ ! -d \"$WIKI_DIR_INST/$WIKI_DIR_NAME\" ];\n> +\tthen\n> +\t\tskip_all='skipping gateway git-mw tests, no mediawiki found'\n> +\t\ttest_done\n> +\tfi\n> +}\n> +\n> +# test_diff_directories \n> +#\n> +# Compare the contents of directories and with diff\n> +# and errors if they do not match. The program will\n> +# not look into .git in the process.\n> +# Warning: the first argument MUST be the directory containing the git data\n> +test_diff_directories () {\n> +\trm -rf \"$1_tmp\"\n> +\tmkdir -p \"$1_tmp\"\n> +\tcp \"$1\"/*.mw \"$1_tmp\"\n> +\tdiff -r -b \"$1_tmp\" \"$2\"\n> +}\n> +\n> +# $1=\n> +# $2=\n> +#\n> +# Check that contains exactly files\n> +test_contains_N_files () {\n> +\tif test `ls -- \"$1\" | wc -l` -ne \"$2\"; then\n> +\t\techo \"directory $1 sould contain $2 files\"\n> +\t\techo \"it contains these files:\"\n> +\t\tls \"$1\"\n> +\t\tfalse\n> +\tfi\n> +}\n> +\n> +\n> +# wiki_check_content \n> +#\n> +# Compares the contents of the file and the wiki page\n> +# and exits with error 1 if they do not match.\n> +wiki_check_content () {\n> +\tmkdir -p wiki_tmp\n> +\twiki_getpage \"$2\" wiki_tmp\n> +\t# replacement of forbidden character in file name\n> +\tpage_name=$(printf \"%s\\n\" \"$2\" | sed -e \"s/\\//%2F/g\")\n> +\n> +\tdiff -b \"$1\" wiki_tmp/\"$page_name\".mw\n> +\tif test $? -ne 0\n> +\tthen\n> +\t\trm -rf wiki_tmp\n> +\t\terror \"ERROR: file $2 not found on wiki\"\n> +\tfi\n> +\trm -rf wiki_tmp\n> +}\n> +\n> +# wiki_page_exist \n> +#\n> +# Check the existence of the page on the wiki and exits\n> +# with error if it is absent from it.\n> +wiki_page_exist () {\n> +\tmkdir -p wiki_tmp\n> +\twiki_getpage \"$1\" wiki_tmp\n> +\tpage_name=$(printf \"%s\\n\" \"$1\" | sed \"s/\\//%2F/g\")\n> +\tif test -f wiki_tmp/\"$page_name\".mw ; then\n> +\t\trm -rf wiki_tmp\n> +\telse\n> +\t\trm -rf wiki_tmp\n> +\t\terror \"test failed: file $1 not found on wiki\"\n> +\tfi\n> +}\n> +\n> +# wiki_getallpagename\n> +#\n> +# Fetch the name of each page on the wiki.\n> +wiki_getallpagename () {\n> +\t\"$CURR_DIR\"/test-gitmw.pl getallpagename\n> +}\n> +\n> +# wiki_getallpagecategory \n> +#\n> +# Fetch the name of each page belonging to on the wiki.\n> +wiki_getallpagecategory () {\n> +\t\"$CURR_DIR\"/test-gitmw.pl getallpagename \"$@\"\n> +}\n> +\n> +# wiki_getallpage []\n> +#\n> +# Fetch all the pages from the wiki and place them in the directory\n> +# .\n> +# If is define, then wiki_getallpage fetch the pages included\n> +# in .\n> +wiki_getallpage () {\n> +\tif test -z \"$2\";\n> +\tthen\n> +\t\twiki_getallpagename\n> +\telse\n> +\t\twiki_getallpagecategory \"$2\"\n> +\tfi\n> +\tmkdir -p \"$1\"\n> +\twhile read -r line; do\n> +\t\twiki_getpage \"$line\" $1;\n> +\tdone < all.txt\n> +}\n> +\n> # ================= Install part =================\n> \n> error () {\n> diff --git a/contrib/mw-to-git/t/test-gitmw.pl b/contrib/mw-to-git/t/test-gitmw.pl\n> new file mode 100755\n> index 0000000..2611991\n> --- /dev/null\n> +++ b/contrib/mw-to-git/t/test-gitmw.pl\n> @@ -0,0 +1,209 @@\n> +#!/usr/bin/perl -w -s\n> +# Copyright (C) 2012\n> +# Charles Roussel \n> +# Simon Cathebras \n> +# Julien Khayat \n> +# Guillaume Sasdy \n> +# Simon Perrat \n> +# License: GPL v2 or later\n> +\n> +# Usage:\n> +# ./test-gitmw.pl [argument]*\n> +# Execute in terminal using the name of the function to call as first\n> +# parameter, and the function's arguments as following parameters\n> +#\n> +# Example:\n> +# ./test-gitmw.pl \"get_page\" foo .\n> +# will call with arguments and <.>\n> +#\n> +# Available functions are:\n> +# \"get_page\"\n> +# \"delete_page\"\n> +# \"edit_page\"\n> +# \"getallpagename\"\n> +\n> +use MediaWiki::API;\n> +use Getopt::Long;\n> +use encoding 'utf8';\n> +use DateTime::Format::ISO8601;\n> +use open ':encoding(utf8)';\n> +use constant SLASH_REPLACEMENT => \"%2F\";\n> +\n> +#Parsing of the config file\n> +\n> +my $configfile = \"$ENV{'CURR_DIR'}/test.config\";\n> +my %config;\n> +open my $CONFIG, \"<\", $configfile or die \"can't open $configfile: $!\";\n> +while (<$CONFIG>)\n> +{\n> + chomp;\n> + s/#.*//;\n> + s/^\\s+//;\n> + s/\\s+$//;\n> + next unless length;\n> + my ($key, $value) = split (/\\s*=\\s*/,$_, 2);\n> + $config{$key} = $value;\n> +\tlast if ($key eq 'LIGHTTPD' and $value eq 'false');\n> +\tlast if ($key eq 'PORT');\n> +}\n> +close $CONFIG or die \"can't close $configfile: $!\";\n> +\n> +my $wiki_address = \"http://$config{'SERVER_ADDR'}\".\":\".\"$config{'PORT'}\";\n> +my $wiki_url = \"$wiki_address/$config{'WIKI_DIR_NAME'}/api.php\";\n> +my $wiki_admin = \"$config{'WIKI_ADMIN'}\";\n> +my $wiki_admin_pass = \"$config{'WIKI_PASSW'}\";\n> +my $mw = MediaWiki::API->new;\n> +$mw->{config}->{api_url} = $wiki_url;\n> +\n> +\n> +# wiki_login \n> +#\n> +# Logs the user with and in the global variable\n> +# of the mediawiki $mw\n> +sub wiki_login {\n> +\t$mw->login( { lgname => \"$_[0]\",lgpassword => \"$_[1]\" } )\n> +\t|| die \"getpage: login failed\";\n> +}\n> +\n> +# wiki_getpage \n> +#\n> +# fetch a page from the wiki referenced in the global variable\n> +# $mw and copies its content in directory dest_path\n> +sub wiki_getpage {\n> +\tmy $pagename = $_[0];\n> +\tmy $destdir = $_[1];\n> +\n> +\tmy $page = $mw->get_page( { title => $pagename } );\n> +\tif (!defined($page)) {\n> +\t\tdie \"getpage: wiki does not exist\";\n> +\t}\n> +\n> +\tmy $content = $page->{'*'};\n> +\tif (!defined($content)) {\n> +\t\tdie \"getpage: page does not exist\";\n> +\t}\n> +\n> +\t$pagename=$page->{'title'};\n> +\t# Replace spaces by underscore in the page name\n> +\t$pagename =~ s/ /_/g;\n> +\t$pagename =~ s/\\//%2F/g;\n> +\topen(my $file, \">$destdir/$pagename.mw\");\n> +\tprint $file \"$content\";\n> +\tclose ($file);\n> +\n> +}\n> +\n> +# wiki_delete_page \n> +#\n> +# delete the page with name from the wiki referenced\n> +# in the global variable $mw\n> +sub wiki_delete_page {\n> +\tmy $pagename = $_[0];\n> +\n> +\tmy $exist=$mw->get_page({title => $pagename});\n> +\n> +\tif (defined($exist->{'*'})){\n> +\t\t$mw->edit({ action => 'delete',\n> +\t\t\t\ttitle => $pagename})\n> +\t\t|| die $mw->{error}->{code} . \": \" . $mw->{error}->{details};\n> +\t} else {\n> +\t\tdie \"no page with such name found: $pagename\\n\";\n> +\t}\n> +}\n> +\n> +# wiki_editpage [-c=] [-s=]\n> +#\n> +# Edit a page named with content on the wiki\n> +# referenced with the global variable $mw\n> +# If == true : append at the end of the actual\n> +# content of the page \n> +# If doesn't exist, that page is created with the \n> +sub wiki_editpage {\n> +\tmy $wiki_page = $_[0];\n> +\tmy $wiki_content = $_[1];\n> +\tmy $wiki_append = $_[2];\n> +\tmy $summary = \"\";\n> +\tmy ($summ, $cat) = ();\n> +\tGetOptions('s=s' => \\$summ, 'c=s' => \\$cat);\n> +\n> +\tmy $append = 0;\n> +\tif (defined($wiki_append) && $wiki_append eq 'true') {\n> +\t\t$append=1;\n> +\t}\n> +\n> +\tmy $previous_text =\"\";\n> +\n> +\tif ($append) {\n> +\t\tmy $ref = $mw->get_page( { title => $wiki_page } );\n> +\t\t$previous_text = $ref->{'*'};\n> +\t}\n> +\n> +\tmy $text = $wiki_content;\n> +\tif (defined($previous_text)) {\n> +\t\t$text=\"$previous_text$text\";\n> +\t}\n> +\n> +\t# Eventually, add this page to a category.\n> +\tif (defined($cat)) {\n> +\t\tmy $category_name=\"[[Category:$cat]]\";\n> +\t\t$text=\"$text\\n $category_name\";\n> +\t}\n> +\tif(defined($summ)){\n> +\t\t$summary=$summ;\n> +\t}\n> +\n> +\t$mw->edit( { action => 'edit', title => $wiki_page, summary => $summary, text => \"$text\"} );\n> +}\n> +\n> +# wiki_getallpagename []\n> +#\n> +# Fetch all pages of the wiki referenced by the global variable $mw\n> +# and print the names of each one in the file all.txt with a new line\n> +# (\"\\n\") between these.\n> +# If the argument is defined, then this function get only the pages\n> +# belonging to .\n> +sub wiki_getallpagename {\n> +\t# fetch the pages of the wiki\n> +\tif (defined($_[0])) {\n> +\t\tmy $mw_pages = $mw->list ( { action => 'query',\n> +\t\t\t\tlist => 'categorymembers',\n> +\t\t\t\tcmtitle => \"Category:$_[0]\",\n> +\t\t\t\tcmnamespace => 0,\n> +\t\t\t\tcmlimit => 500 },\n> +\t\t)\n> +\t\t|| die $mw->{error}->{code}.\": \".$mw->{error}->{details};\n> +\t\topen(my $file, \">all.txt\");\n> +\t\tforeach my $page (@{$mw_pages}) {\n> +\t\t\tprint $file \"$page->{title}\\n\";\n> +\t\t}\n> +\t\tclose ($file);\n> +\n> +\t} else {\n> +\t\tmy $mw_pages = $mw->list({\n> +\t\t\t\taction => 'query',\n> +\t\t\t\tlist => 'allpages',\n> +\t\t\t\taplimit => 500,\n> +\t\t\t})\n> +\t\t|| die $mw->{error}->{code}.\": \".$mw->{error}->{details};\n> +\t\topen(my $file, \">all.txt\");\n> +\t\tforeach my $page (@{$mw_pages}) {\n> +\t\t\tprint $file \"$page->{title}\\n\";\n> +\t\t}\n> +\t\tclose ($file);\n> +\t}\n> +}\n> +\n> +# Main part of this script: parse the command line arguments\n> +# and select which function to execute\n> +my $fct_to_call = shift;\n> +\n> +wiki_login($wiki_admin, $wiki_admin_pass);\n> +\n> +my %functions_to_call = qw(\n> + get_page wiki_getpage\n> + delete_page wiki_delete_page\n> + edit_page wiki_editpage\n> + getallpagename wiki_getallpagename\n> +);\n> +die \"$0 ERROR: wrong argument\" unless exists $functions_to_call{$fct_to_call};\n> +&{$functions_to_call{$fct_to_call}}(@ARGV);\n"},{"id":"194685","messageId":"7vipe1pzr2.fsf@alter.siamese.dyndns.org","threadId":"30954","inReplyTo":"1341473772-28023-5-git-send-email-Matthieu.Moy@imag.fr","subject":"Re: [PATCH 04/12] git-remote-mediawiki (t9361): test git-remote-mediawiki pull and push","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2012-07-05T23:14:09Z","receivedAt":"2012-07-05T23:14:09Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Matthieu Moy writes:\n\n> From: Guillaume Sasdy \n>\n> This patch provides a set of tests for the pull and push fonctionnality\n> of git-remote-mediawiki. The actual tests are kept in a separate function\n> to allow further tests to re-run the same set of commands with different\n> push and pull strategies.\n>\n> Signed-off-by: Simon CATHEBRAS \n> Signed-off-by: Julien KHAYAT \n> Signed-off-by: Simon Perrat \n> Signed-off-by: Charles ROUSSEL \n> Signed-off-by: Guillaume SASDY \n> Signed-off-by: Matthieu Moy \n> ---\n> contrib/mw-to-git/t/push-pull-tests.sh | 144 +++++++++++++++++++++++\n> contrib/mw-to-git/t/t9361-mw-to-git-push-pull.sh | 24 ++++\n> 2 files changed, 168 insertions(+)\n> create mode 100644 contrib/mw-to-git/t/push-pull-tests.sh\n> create mode 100755 contrib/mw-to-git/t/t9361-mw-to-git-push-pull.sh\n>\n> diff --git a/contrib/mw-to-git/t/push-pull-tests.sh b/contrib/mw-to-git/t/push-pull-tests.sh\n> new file mode 100644\n> index 0000000..b9e38dc\n> --- /dev/null\n> +++ b/contrib/mw-to-git/t/push-pull-tests.sh\n> @@ -0,0 +1,144 @@\n> +test_push_pull () {\n> +\n> +\ttest_expect_success 'Git pull works after adding a new wiki page' '\n> +\t\twiki_reset &&\n> +\t\n\nTrailing whitespace\n\n> +\t\tgit clone mediawiki::'\"$WIKI_URL\"' mw_dir_1 &&\n> +\t\twiki_editpage Foo \"page created after the git clone\" false &&\n> +\t\n\nTrailing whitespace\n\nThere are many others, 16 in total, I think.\n"},{"id":"194686","messageId":"7vehoppzjl.fsf@alter.siamese.dyndns.org","threadId":"30954","inReplyTo":"1341473772-28023-9-git-send-email-Matthieu.Moy@imag.fr","subject":"Re: [PATCH 08/12] git-remote-mediawiki: change return type of get_mw_pages","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2012-07-05T23:18:38Z","receivedAt":"2012-07-05T23:18:38Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Matthieu Moy writes:\n\n> The previous version was returning the list of pages to be fetched, but\n> we are going to need an efficient membership test (i.e. is the page\n> $title tracked), hence exposing a hash will be more convenient.\n>\n> Signed-off-by: Matthieu Moy \n> ---\n> contrib/mw-to-git/git-remote-mediawiki | 8 +++++---\n> 1 file changed, 5 insertions(+), 3 deletions(-)\n>\n> diff --git a/contrib/mw-to-git/git-remote-mediawiki b/contrib/mw-to-git/git-remote-mediawiki\n> index 76b78bc..3f57598 100755\n> --- a/contrib/mw-to-git/git-remote-mediawiki\n> +++ b/contrib/mw-to-git/git-remote-mediawiki\n> @@ -377,7 +377,7 @@ sub get_mw_pages {\n> \t\t\tget_all_mediafiles(\\%pages);\n> \t\t}\n> \t}\n> -\treturn values(%pages);\n> +\treturn %pages;\n> }\n> \n> # usage: $out = run_git(\"command args\");\n\nThis suggests the series is not based on 'master', which is not\nfatal per-se (\"am -3\" guesses that came from b3d98595), but then\napplying the series on mm/mediawiki-file-attachments topic will show\nthat the index line recorded in [PATCH 09/12] is useless.\n\nOn what commit was this meant to be applied???\n"},{"id":"194702","messageId":"vpqhatl9t4a.fsf@bauges.imag.fr","threadId":"30954","inReplyTo":"7vmx3dpzsn.fsf@alter.siamese.dyndns.org","subject":"Re: [PATCH 02/12] git-remote-mediawiki: test environment of git-remote-mediawiki","fromName":"Matthieu Moy","fromEmail":"matthieu.moy@grenoble-inp.fr","sentAt":"2012-07-06T08:44:05Z","receivedAt":"2012-07-06T08:44:05Z","isPatch":true,"sender":{"key":"matthieu.moy@grenoble-inp.fr","avatar":"https://gravatar.com/avatar/72c8a2705971a25dfaff23cece15130d405685845d911aedd5667ace277f3fc5?d=mp&s=160"},"body":"Junio C Hamano writes:\n\n>> +clean:\n>> +\t$(RM) '$(gitexecdir)/$(SCRIPT)'\n>> +\t$(MAKE) -C t/ clean\n>> +\n>\n> New blank link at EOF.\n\n[...]\n\n>> +\t\tgit clone mediawiki::'\"$WIKI_URL\"' mw_dir_1 &&\n>> +\t\twiki_editpage Foo \"page created after the git clone\" false &&\n>> +\t\n>\n> Trailing whitespace\n>\n> There are many others, 16 in total, I think.\n\nOops, sorry, I should have noticed that. I \"rebase --whitespace=fix\"ed\nthe serie (there were still some indent-with-space too).\n\nJunio C Hamano writes:\n\n> This suggests the series is not based on 'master', which is not\n> fatal per-se (\"am -3\" guesses that came from b3d98595), but then\n> applying the series on mm/mediawiki-file-attachments topic will show\n> that the index line recorded in [PATCH 09/12] is useless.\n>\n> On what commit was this meant to be applied???\n\nSorry, this was based on an intermediate commit, that were rewritten in\nthe meantime.\n\nI'll resend a proper serie soon.\n\n-- \nMatthieu Moy\nhttp://www-verimag.imag.fr/~moy/\n"},{"id":"194703","messageId":"1341568995-12467-1-git-send-email-Matthieu.Moy@imag.fr","threadId":"30954","inReplyTo":"vpqhatl9t4a.fsf@bauges.imag.fr","subject":"[PATCH 00/12] git-remote-mediawiki: tests and optimizations","fromName":"Matthieu Moy","fromEmail":"matthieu.moy@imag.fr","sentAt":"2012-07-06T10:03:03Z","receivedAt":"2012-07-06T10:03:03Z","isPatch":true,"sender":{"key":"git@matthieu-moy.fr","avatar":"https://avatars.githubusercontent.com/u/14709?v=4"},"body":"This should fix the whitespaces issues reported by Junio.\n\nThis is now based on 6df7e0d (git-remote-mediawiki: improve support\nfor non-English Wikis), in pu.\n\nGuillaume Sasdy (4):\n git-remote-mediawiki: scripts to install, delete and clear a\n MediaWiki\n git-remote-mediawiki: test environment of git-remote-mediawiki\n git-remote-mediawiki (t9360): test git-remote-mediawiki clone\n git-remote-mediawiki (t9361): test git-remote-mediawiki pull and push\n\nMatthieu Moy (6):\n git-remote-mediawiki: support for uploading file in test environment\n git-remote-mediawiki: change return type of get_mw_pages\n git-remote-mediawiki: refactor loop over revision ids\n git-remote-mediawiki: extract revision-importing loop to a function\n git-remote-mediawiki: more efficient 'pull' in the best case\n git-remote-mediawiki: be more defensive when requests fail\n\nNGUYEN Kim Thuat (1):\n git-remote-mediawiki (t9363): test 'File:' import and export\n\nSimon Cathebras (1):\n git-remote-mediawiki (t9362): test git-remote-mediawiki with UTF8\n characters\n\n contrib/mw-to-git/Makefile | 47 +++\n contrib/mw-to-git/git-remote-mediawiki | 149 +++++--\n contrib/mw-to-git/t/.gitignore | 4 +\n contrib/mw-to-git/t/Makefile | 31 ++\n contrib/mw-to-git/t/README | 124 ++++++\n contrib/mw-to-git/t/install-wiki.sh | 45 +++\n contrib/mw-to-git/t/install-wiki/.gitignore | 1 +\n contrib/mw-to-git/t/install-wiki/LocalSettings.php | 129 ++++++\n contrib/mw-to-git/t/install-wiki/db_install.php | 120 ++++++\n contrib/mw-to-git/t/push-pull-tests.sh | 144 +++++++\n contrib/mw-to-git/t/t9360-mw-to-git-clone.sh | 257 ++++++++++++\n contrib/mw-to-git/t/t9361-mw-to-git-push-pull.sh | 24 ++\n contrib/mw-to-git/t/t9362-mw-to-git-utf8.sh | 301 ++++++++++++++\n .../mw-to-git/t/t9363-mw-to-git-export-import.sh | 198 ++++++++++\n contrib/mw-to-git/t/t9364-pull-by-rev.sh | 17 +\n contrib/mw-to-git/t/test-gitmw-lib.sh | 435 +++++++++++++++++++++\n contrib/mw-to-git/t/test-gitmw.pl | 225 +++++++++++\n contrib/mw-to-git/t/test.config | 35 ++\n 18 files changed, 2260 insertions(+), 26 deletions(-)\n create mode 100644 contrib/mw-to-git/Makefile\n create mode 100644 contrib/mw-to-git/t/.gitignore\n create mode 100644 contrib/mw-to-git/t/Makefile\n create mode 100644 contrib/mw-to-git/t/README\n create mode 100755 contrib/mw-to-git/t/install-wiki.sh\n create mode 100644 contrib/mw-to-git/t/install-wiki/.gitignore\n create mode 100644 contrib/mw-to-git/t/install-wiki/LocalSettings.php\n create mode 100644 contrib/mw-to-git/t/install-wiki/db_install.php\n create mode 100644 contrib/mw-to-git/t/push-pull-tests.sh\n create mode 100755 contrib/mw-to-git/t/t9360-mw-to-git-clone.sh\n create mode 100755 contrib/mw-to-git/t/t9361-mw-to-git-push-pull.sh\n create mode 100755 contrib/mw-to-git/t/t9362-mw-to-git-utf8.sh\n create mode 100755 contrib/mw-to-git/t/t9363-mw-to-git-export-import.sh\n create mode 100755 contrib/mw-to-git/t/t9364-pull-by-rev.sh\n create mode 100755 contrib/mw-to-git/t/test-gitmw-lib.sh\n create mode 100755 contrib/mw-to-git/t/test-gitmw.pl\n create mode 100644 contrib/mw-to-git/t/test.config\n\n-- \n1.7.11.1.147.g47a574d\n"},{"id":"194706","messageId":"1341568995-12467-2-git-send-email-Matthieu.Moy@imag.fr","threadId":"30954","inReplyTo":"1341568995-12467-1-git-send-email-Matthieu.Moy@imag.fr","subject":"[PATCH 01/12] git-remote-mediawiki: scripts to install, delete and clear a MediaWiki","fromName":"Matthieu Moy","fromEmail":"matthieu.moy@imag.fr","sentAt":"2012-07-06T10:03:04Z","receivedAt":"2012-07-06T10:03:04Z","isPatch":true,"sender":{"key":"git@matthieu-moy.fr","avatar":"https://avatars.githubusercontent.com/u/14709?v=4"},"body":"From: Guillaume Sasdy \n\ninstall_wiki.sh allows the user to install a MediaWiki instance in a\nsingle shell command. Like \"git instaweb\", it configures and launches\nlighttpd without requiring root priviledges. To simplify database\nmanagement, it uses SQLite, which doesn't require a running daemon, and\nallows reseting the database by simply replacing a single file. This\nallows install_wiki to also defines a function wiki_reset which clear all\ncontent of the previously created wiki, which will be very useful to run\nseveral indepenant tests on the same wiki.\n\nNote those functionnalities are made to be used from the user command\nline in the directory git/contrib/mw-to-git/t/\n\nSigned-off-by: Simon CATHEBRAS \nSigned-off-by: Julien KHAYAT \nSigned-off-by: Simon Perrat \nSigned-off-by: Charles ROUSSEL \nSigned-off-by: Guillaume SASDY \nSigned-off-by: Matthieu Moy \n---\n contrib/mw-to-git/t/.gitignore | 2 +\n contrib/mw-to-git/t/install-wiki.sh | 45 ++++\n contrib/mw-to-git/t/install-wiki/.gitignore | 1 +\n contrib/mw-to-git/t/install-wiki/LocalSettings.php | 128 +++++++++\n contrib/mw-to-git/t/install-wiki/db_install.php | 120 +++++++++\n contrib/mw-to-git/t/test-gitmw-lib.sh | 289 +++++++++++++++++++++\n contrib/mw-to-git/t/test.config | 35 +++\n 7 files changed, 620 insertions(+)\n create mode 100644 contrib/mw-to-git/t/.gitignore\n create mode 100755 contrib/mw-to-git/t/install-wiki.sh\n create mode 100644 contrib/mw-to-git/t/install-wiki/.gitignore\n create mode 100644 contrib/mw-to-git/t/install-wiki/LocalSettings.php\n create mode 100644 contrib/mw-to-git/t/install-wiki/db_install.php\n create mode 100755 contrib/mw-to-git/t/test-gitmw-lib.sh\n create mode 100644 contrib/mw-to-git/t/test.config\n\ndiff --git a/contrib/mw-to-git/t/.gitignore b/contrib/mw-to-git/t/.gitignore\nnew file mode 100644\nindex 0000000..a060414\n--- /dev/null\n+++ b/contrib/mw-to-git/t/.gitignore\n@@ -0,0 +1,2 @@\n+WEB/\n+wiki/\ndiff --git a/contrib/mw-to-git/t/install-wiki.sh b/contrib/mw-to-git/t/install-wiki.sh\nnew file mode 100755\nindex 0000000..c6d6fa3\n--- /dev/null\n+++ b/contrib/mw-to-git/t/install-wiki.sh\n@@ -0,0 +1,45 @@\n+#!/bin/sh\n+\n+# This script installs or deletes a MediaWiki on your computer.\n+# It requires a web server with PHP and SQLite running. In addition, if you\n+# do not have MediaWiki sources on your computer, the option 'install'\n+# downloads them for you.\n+# Please set the CONFIGURATION VARIABLES in ./test-gitmw-lib.sh\n+\n+WIKI_TEST_DIR=$(cd \"$(dirname \"$0\")\" && pwd)\n+\n+if test -z \"$WIKI_TEST_DIR\"\n+then\n+\tWIKI_TEST_DIR=.\n+fi\n+\n+. \"$WIKI_TEST_DIR\"/test-gitmw-lib.sh\n+usage () {\n+\techo \"Usage: \"\n+\techo \"\t./install-wiki.sh \"\n+\techo \"\t\tinstall | -i :\tInstall a wiki on your computer.\"\n+\techo \"\t\tdelete | -d : Delete the wiki and all its pages and \"\n+\techo \"\t\t\tcontent.\"\n+}\n+\n+\n+# Argument: install, delete, --help | -h\n+case \"$1\" in\n+\t\"install\" | \"-i\")\n+\t\twiki_install\n+\t\texit 0\n+\t\t;;\n+\t\"delete\" | \"-d\")\n+\t\twiki_delete\n+\t\texit 0\n+\t\t;;\n+\t\"--help\" | \"-h\")\n+\t\tusage\n+\t\texit 0\n+\t\t;;\n+\t*)\n+\t\techo \"Invalid argument: $1\"\n+\t\tusage\n+\t\texit 1\n+\t\t;;\n+esac\ndiff --git a/contrib/mw-to-git/t/install-wiki/.gitignore b/contrib/mw-to-git/t/install-wiki/.gitignore\nnew file mode 100644\nindex 0000000..b5a2a44\n--- /dev/null\n+++ b/contrib/mw-to-git/t/install-wiki/.gitignore\n@@ -0,0 +1 @@\n+wikidb.sqlite\ndiff --git a/contrib/mw-to-git/t/install-wiki/LocalSettings.php b/contrib/mw-to-git/t/install-wiki/LocalSettings.php\nnew file mode 100644\nindex 0000000..aa72518\n--- /dev/null\n+++ b/contrib/mw-to-git/t/install-wiki/LocalSettings.php\n@@ -0,0 +1,128 @@\n+ and \n+ *\n+ */\n+$argc = $_SERVER['argc'];\n+$argv = $_SERVER['argv'];\n+\n+$login = $argv[2];\n+$pass = $argv[3];\n+$tmp = $argv[4];\n+$port = $argv[5];\n+\n+$url = 'http://localhost:'.$port.'/wiki/mw-config/index.php';\n+$db_dir = urlencode($tmp);\n+$tmp_cookie = tempnam($tmp, \"COOKIE_\");\n+/*\n+ * Fetchs a page with cURL.\n+ */\n+function get($page_name = \"\") {\n+\t$curl = curl_init();\n+\t$page_name_add = \"\";\n+\tif ($page_name != \"\") {\n+\t\t$page_name_add = '?page='.$page_name;\n+\t}\n+\t$url = $GLOBALS['url'].$page_name_add;\n+\t$tmp_cookie = $GLOBALS['tmp_cookie'];\n+\tcurl_setopt($curl, CURLOPT_COOKIEJAR, $tmp_cookie);\n+\tcurl_setopt($curl, CURLOPT_RETURNTRANSFER, true);\n+\tcurl_setopt($curl, CURLOPT_FOLLOWLOCATION, true);\n+\tcurl_setopt($curl, CURLOPT_COOKIEFILE, $tmp_cookie);\n+\tcurl_setopt($curl, CURLOPT_HEADER, true);\n+\tcurl_setopt($curl, CURLOPT_URL, $url);\n+\n+\t$page = curl_exec($curl);\n+\tif (!$page) {\n+\t\tdie(\"Could not get page: $url\\n\");\n+\t}\n+\tcurl_close($curl);\n+\treturn $page;\n+}\n+\n+/*\n+ * Submits a form with cURL.\n+ */\n+function submit($page_name, $option = \"\") {\n+\t$curl = curl_init();\n+\t$datapost = 'submit-continue=Continue+%E2%86%92';\n+\tif ($option != \"\") {\n+\t\t$datapost = $option.'&'.$datapost;\n+\t}\n+\t$url = $GLOBALS['url'].'?page='.$page_name;\n+\t$tmp_cookie = $GLOBALS['tmp_cookie'];\n+\tcurl_setopt($curl, CURLOPT_URL, $url);\n+\tcurl_setopt($curl, CURLOPT_POST, true);\n+\tcurl_setopt($curl, CURLOPT_FOLLOWLOCATION, true);\n+\tcurl_setopt($curl, CURLOPT_POSTFIELDS, $datapost);\n+\tcurl_setopt($curl, CURLOPT_RETURNTRANSFER, true);\n+\tcurl_setopt($curl, CURLOPT_COOKIEJAR, $tmp_cookie);\n+\tcurl_setopt($curl, CURLOPT_COOKIEFILE, $tmp_cookie);\n+\n+\t$page = curl_exec($curl);\n+\tif (!$page) {\n+\t\tdie(\"Could not get page: $url\\n\");\n+\t}\n+\tcurl_close($curl);\n+\treturn \"$page\";\n+}\n+\n+/*\n+ * Here starts this script: simulates the behavior of the user\n+ * submitting forms to generates the database file.\n+ * Note this simulation was made for the MediaWiki version 1.19.0,\n+ * we can't assume it works with other versions.\n+ *\n+ */\n+\n+$page = get();\n+if (!preg_match('/input type=\"hidden\" value=\"([0-9]+)\" name=\"LanguageRequestTime\"/',\n+\t\t$page, $matches)) {\n+\techo \"Unexpected content for page downloaded:\\n\";\n+\techo \"$page\";\n+\tdie;\n+};\n+$timestamp = $matches[1];\n+$language = \"LanguageRequestTime=$timestamp&uselang=en&ContLang=en\";\n+$page = submit('Language', $language);\n+\n+submit('Welcome');\n+\n+$db_config = 'DBType=sqlite';\n+$db_config = $db_config.'&sqlite_wgSQLiteDataDir='.$db_dir;\n+$db_config = $db_config.'&sqlite_wgDBname='.$argv[1];\n+submit('DBConnect', $db_config);\n+\n+$wiki_config = 'config_wgSitename=TEST';\n+$wiki_config = $wiki_config.'&config__NamespaceType=site-name';\n+$wiki_config = $wiki_config.'&config_wgMetaNamespace=MyWiki';\n+$wiki_config = $wiki_config.'&config__AdminName='.$login;\n+\n+$wiki_config = $wiki_config.'&config__AdminPassword='.$pass;\n+$wiki_config = $wiki_config.'&config__AdminPassword2='.$pass;\n+\n+$wiki_config = $wiki_config.'&wiki__configEmail=email%40email.org';\n+$wiki_config = $wiki_config.'&config__SkipOptional=skip';\n+submit('Name', $wiki_config);\n+submit('Install');\n+submit('Install');\n+\n+unlink($tmp_cookie);\n+?>\ndiff --git a/contrib/mw-to-git/t/test-gitmw-lib.sh b/contrib/mw-to-git/t/test-gitmw-lib.sh\nnew file mode 100755\nindex 0000000..47341f7\n--- /dev/null\n+++ b/contrib/mw-to-git/t/test-gitmw-lib.sh\n@@ -0,0 +1,289 @@\n+# Copyright (C) 2012\n+# Charles Roussel \n+# Simon Cathebras \n+# Julien Khayat \n+# Guillaume Sasdy \n+# Simon Perrat \n+# License: GPL v2 or later\n+\n+#\n+# CONFIGURATION VARIABLES\n+# You might want to change these ones\n+#\n+\n+. ./test.config\n+\n+CURR_DIR=$(pwd)\n+TEST_OUTPUT_DIRECTORY=$(pwd)\n+TEST_DIRECTORY=\"$CURR_DIR\"/../../../t\n+\n+export TEST_OUTPUT_DIRECTORY TEST_DIRECTORY CURR_DIR\n+\n+if test \"$LIGHTTPD\" = \"false\" ; then\n+\tPORT=80\n+else\n+\tWIKI_DIR_INST=\"$CURR_DIR/$WEB_WWW\"\n+fi\n+\n+# ================= Install part =================\n+\n+error () {\n+\techo \"$@\" >&2\n+\texit 1\n+}\n+\n+# config_lighttpd\n+#\n+# Create the configuration files and the folders necessary to start lighttpd.\n+# Overwrite any existing file.\n+config_lighttpd () {\n+\tmkdir -p $WEB\n+\tmkdir -p $WEB_TMP\n+\tmkdir -p $WEB_WWW\n+\tcat > $WEB/lighttpd.conf <\t\"application/pdf\",\n+\t\".sig\"\t\t=>\t\"application/pgp-signature\",\n+\t\".spl\"\t\t=>\t\"application/futuresplash\",\n+\t\".class\"\t=>\t\"application/octet-stream\",\n+\t\".ps\"\t\t=>\t\"application/postscript\",\n+\t\".torrent\"\t=>\t\"application/x-bittorrent\",\n+\t\".dvi\"\t\t=>\t\"application/x-dvi\",\n+\t\".gz\"\t\t=>\t\"application/x-gzip\",\n+\t\".pac\"\t\t=>\t\"application/x-ns-proxy-autoconfig\",\n+\t\".swf\"\t\t=>\t\"application/x-shockwave-flash\",\n+\t\".tar.gz\"\t=>\t\"application/x-tgz\",\n+\t\".tgz\"\t\t=>\t\"application/x-tgz\",\n+\t\".tar\"\t\t=>\t\"application/x-tar\",\n+\t\".zip\"\t\t=>\t\"application/zip\",\n+\t\".mp3\"\t\t=>\t\"audio/mpeg\",\n+\t\".m3u\"\t\t=>\t\"audio/x-mpegurl\",\n+\t\".wma\"\t\t=>\t\"audio/x-ms-wma\",\n+\t\".wax\"\t\t=>\t\"audio/x-ms-wax\",\n+\t\".ogg\"\t\t=>\t\"application/ogg\",\n+\t\".wav\"\t\t=>\t\"audio/x-wav\",\n+\t\".gif\"\t\t=>\t\"image/gif\",\n+\t\".jpg\"\t\t=>\t\"image/jpeg\",\n+\t\".jpeg\"\t\t=>\t\"image/jpeg\",\n+\t\".png\"\t\t=>\t\"image/png\",\n+\t\".xbm\"\t\t=>\t\"image/x-xbitmap\",\n+\t\".xpm\"\t\t=>\t\"image/x-xpixmap\",\n+\t\".xwd\"\t\t=>\t\"image/x-xwindowdump\",\n+\t\".css\"\t\t=>\t\"text/css\",\n+\t\".html\"\t\t=>\t\"text/html\",\n+\t\".htm\"\t\t=>\t\"text/html\",\n+\t\".js\"\t\t=>\t\"text/javascript\",\n+\t\".asc\"\t\t=>\t\"text/plain\",\n+\t\".c\"\t\t=>\t\"text/plain\",\n+\t\".cpp\"\t\t=>\t\"text/plain\",\n+\t\".log\"\t\t=>\t\"text/plain\",\n+\t\".conf\"\t\t=>\t\"text/plain\",\n+\t\".text\"\t\t=>\t\"text/plain\",\n+\t\".txt\"\t\t=>\t\"text/plain\",\n+\t\".dtd\"\t\t=>\t\"text/xml\",\n+\t\".xml\"\t\t=>\t\"text/xml\",\n+\t\".mpeg\"\t\t=>\t\"video/mpeg\",\n+\t\".mpg\"\t\t=>\t\"video/mpeg\",\n+\t\".mov\"\t\t=>\t\"video/quicktime\",\n+\t\".qt\"\t\t=>\t\"video/quicktime\",\n+\t\".avi\"\t\t=>\t\"video/x-msvideo\",\n+\t\".asf\"\t\t=>\t\"video/x-ms-asf\",\n+\t\".asx\"\t\t=>\t\"video/x-ms-asf\",\n+\t\".wmv\"\t\t=>\t\"video/x-ms-wmv\",\n+\t\".bz2\"\t\t=>\t\"application/x-bzip\",\n+\t\".tbz\"\t\t=>\t\"application/x-bzip-compressed-tar\",\n+\t\".tar.bz2\"\t=>\t\"application/x-bzip-compressed-tar\",\n+\t\"\"\t\t=>\t\"text/plain\"\n+\t)\n+\n+\tfastcgi.server = ( \".php\" =>\n+\t(\"localhost\" =>\n+\t( \"socket\" => \"$CURR_DIR/$WEB_TMP/php.socket\",\n+\t\"bin-path\" => \"$PHP_DIR/php-cgi -c $CURR_DIR/$WEB/php.ini\"\n+\n+\t)\n+\t)\n+\t)\n+EOF\n+\n+\tcat > $WEB/php.ini < \"$file_swap\"\n+\tmv \"$file_swap\" \"$FILES_FOLDER/LocalSettings-tmp.php\"\n+\tsed \"s,@WG_SERVER@,http://$SERVER_ADDR,\" \\\n+\t\t\"$FILES_FOLDER/LocalSettings-tmp.php\" > \"$file_swap\"\n+\tmv \"$file_swap\" \"$FILES_FOLDER/LocalSettings-tmp.php\"\n+\tsed \"s,@WG_SQLITE_DATADIR@,$TMP,\" \\\n+\t\t\"$FILES_FOLDER/LocalSettings-tmp.php\" > \"$file_swap\"\n+\tmv \"$file_swap\" \"$FILES_FOLDER/LocalSettings-tmp.php\"\n+\tsed \"s,@WG_SQLITE_DATAFILE@,$( basename $DB_FILE .sqlite),\" \\\n+\t\t\"$FILES_FOLDER/LocalSettings-tmp.php\" > \"$file_swap\"\n+\tmv \"$file_swap\" \"$FILES_FOLDER/LocalSettings-tmp.php\"\n+\n+\tmv \"$FILES_FOLDER/LocalSettings-tmp.php\" \\\n+\t\t\"$WIKI_DIR_INST/$WIKI_DIR_NAME/LocalSettings.php\" ||\n+\t\terror \"Unable to move $FILES_FOLDER/LocalSettings-tmp.php\" \\\n+\t\t\"in $WIKI_DIR_INST/$WIKI_DIR_NAME\"\n+\techo \"File $FILES_FOLDER/LocalSettings.php is set in\" \\\n+\t\t\" $WIKI_DIR_INST/$WIKI_DIR_NAME\"\n+\n+\techo \"Your wiki has been installed. You can check it at\n+\t\thttp://$SERVER_ADDR/$WIKI_DIR_NAME\"\n+}\n+\n+# Reset the database of the wiki and the password of the admin\n+#\n+# Warning: This function must be called only in a subdirectory of t/ directory\n+wiki_reset () {\n+\t# Copy initial database of the wiki\n+\tif [ ! -f \"../$FILES_FOLDER/$DB_FILE\" ] ; then\n+\t\terror \"Can't find ../$FILES_FOLDER/$DB_FILE in the current folder.\"\n+\tfi\n+\tcp \"../$FILES_FOLDER/$DB_FILE\" \"$TMP\" ||\n+\t\terror \"Can't copy ../$FILES_FOLDER/$DB_FILE in $TMP\"\n+\techo \"File $FILES_FOLDER/$DB_FILE is set in $TMP\"\n+}\n+\n+# Delete the wiki created in the web server's directory and all its content\n+# saved in the database.\n+wiki_delete () {\n+\tif test $LIGHTTPD = \"true\"; then\n+\t\tstop_lighttpd\n+\telse\n+\t\t# Delete the wiki's directory.\n+\t\trm -rf \"$WIKI_DIR_INST/$WIKI_DIR_NAME\" ||\n+\t\t\terror \"Wiki's directory $WIKI_DIR_INST/\" \\\n+\t\t\t\"$WIKI_DIR_NAME could not be deleted\"\n+\t\t# Delete the wiki's SQLite database.\n+\t\trm -f \"$TMP/$DB_FILE\" ||\n+\t\t\terror \"Database $TMP/$DB_FILE could not be deleted.\"\n+\tfi\n+\n+\t# Delete the wiki's SQLite database\n+\trm -f \"$TMP/$DB_FILE\" || error \"Database $TMP/$DB_FILE could not be deleted.\"\n+\trm -f \"$FILES_FOLDER/$DB_FILE\"\n+\trm -rf \"$TMP/$MW_VERSION\"\n+}\ndiff --git a/contrib/mw-to-git/t/test.config b/contrib/mw-to-git/t/test.config\nnew file mode 100644\nindex 0000000..958b37b\n--- /dev/null\n+++ b/contrib/mw-to-git/t/test.config\n@@ -0,0 +1,35 @@\n+# Name of the web server's directory dedicated to the wiki is WIKI_DIR_NAME\n+WIKI_DIR_NAME=wiki\n+\n+# Login and password of the wiki's admin\n+WIKI_ADMIN=WikiAdmin\n+WIKI_PASSW=AdminPass\n+\n+# Address of the web server\n+SERVER_ADDR=localhost\n+\n+# SQLite database of the wiki, named DB_FILE, is located in TMP\n+TMP=/tmp\n+DB_FILE=wikidb.sqlite\n+\n+# If LIGHTTPD is not set to true, the script will use the defaut\n+# web server running in WIKI_DIR_INST.\n+WIKI_DIR_INST=/var/www\n+\n+# If LIGHTTPD is set to true, the script will use Lighttpd to run\n+# the wiki.\n+LIGHTTPD=true\n+\n+# The variables below are useful only if LIGHTTPD is set to true.\n+PORT=1234\n+PHP_DIR=/usr/bin\n+LIGHTTPD_DIR=/usr/sbin\n+WEB=WEB\n+WEB_TMP=$WEB/tmp\n+WEB_WWW=$WEB/www\n+\n+# The variables below are used by the script to install a wiki.\n+# You should not modify these unless you are modifying the script itself.\n+MW_VERSION=mediawiki-1.19.0\n+FILES_FOLDER=install-wiki\n+DB_INSTALL_SCRIPT=db_install.php\n-- \n1.7.11.1.147.g47a574d\n"},{"id":"194705","messageId":"1341568995-12467-3-git-send-email-Matthieu.Moy@imag.fr","threadId":"30954","inReplyTo":"1341568995-12467-1-git-send-email-Matthieu.Moy@imag.fr","subject":"[PATCH 02/12] git-remote-mediawiki: test environment of git-remote-mediawiki","fromName":"Matthieu Moy","fromEmail":"matthieu.moy@imag.fr","sentAt":"2012-07-06T10:03:05Z","receivedAt":"2012-07-06T10:03:05Z","isPatch":true,"sender":{"key":"git@matthieu-moy.fr","avatar":"https://avatars.githubusercontent.com/u/14709?v=4"},"body":"From: Guillaume Sasdy \n\nIn order to test git-remote-mediawiki, a set of functions is needed to\nmanage a MediaWiki: edit a page, remove a page, fetch a page, fetch all\npages on a given wiki.\n\nA few helper function are also provided to check the content of\ndirectories.\n\nIn addition, this patch provides Makefiles to execute tests.\nSee the README file for more details.\n\nSigned-off-by: Simon CATHEBRAS \nSigned-off-by: Julien KHAYAT \nSigned-off-by: Simon Perrat \nSigned-off-by: Charles ROUSSEL \nSigned-off-by: Guillaume SASDY \nSigned-off-by: Matthieu Moy \n---\n contrib/mw-to-git/Makefile | 47 ++++++++\n contrib/mw-to-git/t/.gitignore | 2 +\n contrib/mw-to-git/t/Makefile | 31 +++++\n contrib/mw-to-git/t/README | 124 ++++++++++++++++++++\n contrib/mw-to-git/t/test-gitmw-lib.sh | 143 +++++++++++++++++++++++\n contrib/mw-to-git/t/test-gitmw.pl | 209 ++++++++++++++++++++++++++++++++++\n 6 files changed, 556 insertions(+)\n create mode 100644 contrib/mw-to-git/Makefile\n create mode 100644 contrib/mw-to-git/t/Makefile\n create mode 100644 contrib/mw-to-git/t/README\n create mode 100755 contrib/mw-to-git/t/test-gitmw.pl\n\ndiff --git a/contrib/mw-to-git/Makefile b/contrib/mw-to-git/Makefile\nnew file mode 100644\nindex 0000000..3ed728b\n--- /dev/null\n+++ b/contrib/mw-to-git/Makefile\n@@ -0,0 +1,47 @@\n+#\n+# Copyright (C) 2012\n+# Charles Roussel \n+# Simon Cathebras \n+# Julien Khayat \n+# Guillaume Sasdy \n+# Simon Perrat \n+#\n+## Build git-remote-mediawiki\n+\n+-include ../../config.mak.autogen\n+-include ../../config.mak\n+\n+ifndef PERL_PATH\n+\tPERL_PATH = /usr/bin/perl\n+endif\n+ifndef gitexecdir\n+\tgitexecdir = $(shell git --exec-path)\n+endif\n+\n+PERL_PATH_SQ = $(subst ','\\'',$(PERL_PATH))\n+gitexecdir_SQ = $(subst ','\\'',$(gitexecdir))\n+SCRIPT = git-remote-mediawiki\n+\n+.PHONY: install help doc test clean\n+\n+help:\n+\t@echo 'This is the help target of the Makefile. Current configuration:'\n+\t@echo ' gitexecdir = $(gitexecdir_SQ)'\n+\t@echo ' PERL_PATH = $(PERL_PATH_SQ)'\n+\t@echo 'Run \"$(MAKE) install\" to install $(SCRIPT) in gitexecdir'\n+\t@echo 'Run \"$(MAKE) test\" to run the testsuite'\n+\n+install:\n+\tsed -e '1s|#!.*/perl|#!$(PERL_PATH_SQ)|' $(SCRIPT) \\\n+\t\t> '$(gitexecdir_SQ)/$(SCRIPT)'\n+\tchmod +x '$(gitexecdir)/$(SCRIPT)'\n+\n+doc:\n+\t@echo 'Sorry, \"make doc\" is not implemented yet for $(SCRIPT)'\n+\n+test:\n+\t$(MAKE) -C t/ test\n+\n+clean:\n+\t$(RM) '$(gitexecdir)/$(SCRIPT)'\n+\t$(MAKE) -C t/ clean\ndiff --git a/contrib/mw-to-git/t/.gitignore b/contrib/mw-to-git/t/.gitignore\nindex a060414..a7a40b4 100644\n--- a/contrib/mw-to-git/t/.gitignore\n+++ b/contrib/mw-to-git/t/.gitignore\n@@ -1,2 +1,4 @@\n WEB/\n wiki/\n+trash directory.t*/\n+test-results/\ndiff --git a/contrib/mw-to-git/t/Makefile b/contrib/mw-to-git/t/Makefile\nnew file mode 100644\nindex 0000000..f422203\n--- /dev/null\n+++ b/contrib/mw-to-git/t/Makefile\n@@ -0,0 +1,31 @@\n+#\n+# Copyright (C) 2012\n+# Charles Roussel \n+# Simon Cathebras \n+# Julien Khayat \n+# Guillaume Sasdy \n+# Simon Perrat \n+#\n+## Test git-remote-mediawiki\n+\n+all: test\n+\n+-include ../../../config.mak.autogen\n+-include ../../../config.mak\n+\n+T = $(wildcard t[0-9][0-9][0-9][0-9]-*.sh)\n+\n+.PHONY: help test clean all\n+\n+help:\n+\t@echo 'Run \"$(MAKE) test\" to launch test scripts'\n+\t@echo 'Run \"$(MAKE) clean\" to remove trash folders'\n+\n+test:\n+\t@for t in $(T); do \\\n+\t\techo \"$$t\"; \\\n+\t\t\"./$$t\" || exit 1; \\\n+\tdone\n+\n+clean:\n+\t$(RM) -r 'trash directory'.*\ndiff --git a/contrib/mw-to-git/t/README b/contrib/mw-to-git/t/README\nnew file mode 100644\nindex 0000000..96e9739\n--- /dev/null\n+++ b/contrib/mw-to-git/t/README\n@@ -0,0 +1,124 @@\n+Tests for Mediawiki-to-Git\n+==========================\n+\n+Introduction\n+------------\n+This manual describes how to install the git-remote-mediawiki test\n+environment on a machine with git installed on it.\n+\n+Prerequisite\n+------------\n+\n+In order to run this test environment correctly, you will need to\n+install the following packages (Debian/Ubuntu names, may need to be\n+adapted for another distribution):\n+\n+* lighttpd\n+* php5\n+* php5-cgi\n+* php5-cli\n+* php5-curl\n+* php5-sqlite\n+\n+Principles and Technical Choices\n+--------------------------------\n+\n+The test environment makes it easy to install and manipulate one or\n+several MediaWiki instances. To allow developers to run the testsuite\n+easily, the environment does not require root priviledge (except to\n+install the required packages if needed). It starts a webserver\n+instance on the user's account (using lighttpd greatly helps for\n+that), and does not need a separate database daemon (thanks to the use\n+of sqlite).\n+\n+Run the test environment\n+------------------------\n+\n+Install a new wiki\n+~~~~~~~~~~~~~~~~~~\n+\n+Once you have all the prerequisite, you need to install a MediaWiki\n+instance on your machine. If you already have one, it is still\n+strongly recommended to install one with the script provided. Here's\n+how to work it:\n+\n+a. change directory to contrib/mw-to-git/t/\n+b. if needed, edit test.config to choose your installation parameters\n+c. run `./install-wiki.sh install`\n+d. check on your favourite web browser if your wiki is correctly\n+ installed.\n+\n+Remove an existing wiki\n+~~~~~~~~~~~~~~~~~~~~~~~\n+\n+Edit the file test.config to fit the wiki you want to delete, and then\n+execute the command `./install-wiki.sh delete` from the\n+contrib/mw-to-git/t directory.\n+\n+Run the existing tests\n+~~~~~~~~~~~~~~~~~~~~~~\n+\n+The provided tests are currently in the `contrib/mw-to-git/t` directory.\n+The files are all the t936[0-9]-*.sh shell scripts.\n+\n+a. Run all tests:\n+To do so, run \"make test\" from the contrib/mw-to-git/ directory.\n+\n+b. Run a specific test:\n+To run a given test , run ./ from the\n+contrib/mw-to-git/t directory.\n+\n+How to create new tests\n+-----------------------\n+\n+Available functions\n+~~~~~~~~~~~~~~~~~~~\n+\n+The test environment of git-remote-mediawiki provides some functions\n+useful to test its behaviour. for more details about the functions'\n+parameters, please refer to the `test-gitmw-lib.sh` and\n+`test-gitmw.pl` files.\n+\n+** `test_check_wiki_precond`:\n+Check if the tests must be skipped or not. Please use this function\n+at the beggining of each new test file.\n+\n+** `wiki_getpage`:\n+Fetch a given page from the wiki and puts its content in the\n+directory in parameter.\n+\n+** `wiki_delete_page`:\n+Delete a given page from the wiki.\n+\n+** `wiki_edit_page`:\n+Create or modify a given page in the wiki. You can specify several\n+parameters like a summary for the page edition, or add the page to a\n+given category.\n+See test-gitmw.pl for more details.\n+\n+** `wiki_getallpage`:\n+Fetch all pages from the wiki into a given directory. The directory\n+is created if it does not exists.\n+\n+** `test_diff_directories`:\n+Compare the content of two directories. The content must be the same.\n+Use this function to compare the content of a git directory and a wiki\n+one created by wiki_getallpage.\n+\n+** `test_contains_N_files`:\n+Check if the given directory contains a given number of file.\n+\n+** `wiki_page_exists`:\n+Tests if a given page exists on the wiki.\n+\n+** `wiki_reset`:\n+Reset the wiki, i.e. flush the database. Use this function at the\n+begining of each new test, except if the test re-uses the same wiki\n+(and history) as the previous test.\n+\n+How to write a new test\n+~~~~~~~~~~~~~~~~~~~~~~~\n+\n+Please, follow the standards given by git. See git/t/README.\n+New file should be named as t936[0-9]-*.sh.\n+Be sure to reset your wiki regulary with the function `wiki_reset`.\ndiff --git a/contrib/mw-to-git/t/test-gitmw-lib.sh b/contrib/mw-to-git/t/test-gitmw-lib.sh\nindex 47341f7..14f6577 100755\n--- a/contrib/mw-to-git/t/test-gitmw-lib.sh\n+++ b/contrib/mw-to-git/t/test-gitmw-lib.sh\n@@ -13,6 +13,7 @@\n \n . ./test.config\n \n+WIKI_URL=http://\"$SERVER_ADDR:$PORT/$WIKI_DIR_NAME\"\n CURR_DIR=$(pwd)\n TEST_OUTPUT_DIRECTORY=$(pwd)\n TEST_DIRECTORY=\"$CURR_DIR\"/../../../t\n@@ -25,6 +26,148 @@ else\n \tWIKI_DIR_INST=\"$CURR_DIR/$WEB_WWW\"\n fi\n \n+\n+wiki_getpage () {\n+\t\"$CURR_DIR\"/test-gitmw.pl get_page \"$@\"\n+}\n+\n+wiki_delete_page () {\n+\t\"$CURR_DIR\"/test-gitmw.pl delete_page \"$@\"\n+}\n+\n+wiki_editpage () {\n+\t\"$CURR_DIR\"/test-gitmw.pl edit_page \"$@\"\n+}\n+\n+die () {\n+\tdie_with_status 1 \"$@\"\n+}\n+\n+die_with_status () {\n+\tstatus=$1\n+\tshift\n+\techo >&2 \"$*\"\n+\texit \"$status\"\n+}\n+\n+\n+# Check the preconditions to run git-remote-mediawiki's tests\n+test_check_precond () {\n+\tif ! test_have_prereq PERL\n+\tthen\n+\t\tskip_all='skipping gateway git-mw tests, perl not available'\n+\t\ttest_done\n+\tfi\n+\n+\tif [ ! -f \"$GIT_BUILD_DIR\"/git-remote-mediawiki ];\n+\tthen\n+\t\techo \"No remote mediawiki for git found. Copying it in git\"\n+\t\techo \"cp $GIT_BUILD_DIR/contrib/mw-to-git/git-remote-mediawiki $GIT_BUILD_DIR/\"\n+\t\tln -s \"$GIT_BUILD_DIR\"/contrib/mw-to-git/git-remote-mediawiki \"$GIT_BUILD_DIR\"\n+\tfi\n+\n+\tif [ ! -d \"$WIKI_DIR_INST/$WIKI_DIR_NAME\" ];\n+\tthen\n+\t\tskip_all='skipping gateway git-mw tests, no mediawiki found'\n+\t\ttest_done\n+\tfi\n+}\n+\n+# test_diff_directories \n+#\n+# Compare the contents of directories and with diff\n+# and errors if they do not match. The program will\n+# not look into .git in the process.\n+# Warning: the first argument MUST be the directory containing the git data\n+test_diff_directories () {\n+\trm -rf \"$1_tmp\"\n+\tmkdir -p \"$1_tmp\"\n+\tcp \"$1\"/*.mw \"$1_tmp\"\n+\tdiff -r -b \"$1_tmp\" \"$2\"\n+}\n+\n+# $1=\n+# $2=\n+#\n+# Check that contains exactly files\n+test_contains_N_files () {\n+\tif test `ls -- \"$1\" | wc -l` -ne \"$2\"; then\n+\t\techo \"directory $1 sould contain $2 files\"\n+\t\techo \"it contains these files:\"\n+\t\tls \"$1\"\n+\t\tfalse\n+\tfi\n+}\n+\n+\n+# wiki_check_content \n+#\n+# Compares the contents of the file and the wiki page\n+# and exits with error 1 if they do not match.\n+wiki_check_content () {\n+\tmkdir -p wiki_tmp\n+\twiki_getpage \"$2\" wiki_tmp\n+\t# replacement of forbidden character in file name\n+\tpage_name=$(printf \"%s\\n\" \"$2\" | sed -e \"s/\\//%2F/g\")\n+\n+\tdiff -b \"$1\" wiki_tmp/\"$page_name\".mw\n+\tif test $? -ne 0\n+\tthen\n+\t\trm -rf wiki_tmp\n+\t\terror \"ERROR: file $2 not found on wiki\"\n+\tfi\n+\trm -rf wiki_tmp\n+}\n+\n+# wiki_page_exist \n+#\n+# Check the existence of the page on the wiki and exits\n+# with error if it is absent from it.\n+wiki_page_exist () {\n+\tmkdir -p wiki_tmp\n+\twiki_getpage \"$1\" wiki_tmp\n+\tpage_name=$(printf \"%s\\n\" \"$1\" | sed \"s/\\//%2F/g\")\n+\tif test -f wiki_tmp/\"$page_name\".mw ; then\n+\t\trm -rf wiki_tmp\n+\telse\n+\t\trm -rf wiki_tmp\n+\t\terror \"test failed: file $1 not found on wiki\"\n+\tfi\n+}\n+\n+# wiki_getallpagename\n+#\n+# Fetch the name of each page on the wiki.\n+wiki_getallpagename () {\n+\t\"$CURR_DIR\"/test-gitmw.pl getallpagename\n+}\n+\n+# wiki_getallpagecategory \n+#\n+# Fetch the name of each page belonging to on the wiki.\n+wiki_getallpagecategory () {\n+\t\"$CURR_DIR\"/test-gitmw.pl getallpagename \"$@\"\n+}\n+\n+# wiki_getallpage []\n+#\n+# Fetch all the pages from the wiki and place them in the directory\n+# .\n+# If is define, then wiki_getallpage fetch the pages included\n+# in .\n+wiki_getallpage () {\n+\tif test -z \"$2\";\n+\tthen\n+\t\twiki_getallpagename\n+\telse\n+\t\twiki_getallpagecategory \"$2\"\n+\tfi\n+\tmkdir -p \"$1\"\n+\twhile read -r line; do\n+\t\twiki_getpage \"$line\" $1;\n+\tdone < all.txt\n+}\n+\n # ================= Install part =================\n \n error () {\ndiff --git a/contrib/mw-to-git/t/test-gitmw.pl b/contrib/mw-to-git/t/test-gitmw.pl\nnew file mode 100755\nindex 0000000..b0117e2\n--- /dev/null\n+++ b/contrib/mw-to-git/t/test-gitmw.pl\n@@ -0,0 +1,209 @@\n+#!/usr/bin/perl -w -s\n+# Copyright (C) 2012\n+# Charles Roussel \n+# Simon Cathebras \n+# Julien Khayat \n+# Guillaume Sasdy \n+# Simon Perrat \n+# License: GPL v2 or later\n+\n+# Usage:\n+# ./test-gitmw.pl [argument]*\n+# Execute in terminal using the name of the function to call as first\n+# parameter, and the function's arguments as following parameters\n+#\n+# Example:\n+# ./test-gitmw.pl \"get_page\" foo .\n+# will call with arguments and <.>\n+#\n+# Available functions are:\n+# \"get_page\"\n+# \"delete_page\"\n+# \"edit_page\"\n+# \"getallpagename\"\n+\n+use MediaWiki::API;\n+use Getopt::Long;\n+use encoding 'utf8';\n+use DateTime::Format::ISO8601;\n+use open ':encoding(utf8)';\n+use constant SLASH_REPLACEMENT => \"%2F\";\n+\n+#Parsing of the config file\n+\n+my $configfile = \"$ENV{'CURR_DIR'}/test.config\";\n+my %config;\n+open my $CONFIG, \"<\", $configfile or die \"can't open $configfile: $!\";\n+while (<$CONFIG>)\n+{\n+\tchomp;\n+\ts/#.*//;\n+\ts/^\\s+//;\n+\ts/\\s+$//;\n+\tnext unless length;\n+\tmy ($key, $value) = split (/\\s*=\\s*/,$_, 2);\n+\t$config{$key} = $value;\n+\tlast if ($key eq 'LIGHTTPD' and $value eq 'false');\n+\tlast if ($key eq 'PORT');\n+}\n+close $CONFIG or die \"can't close $configfile: $!\";\n+\n+my $wiki_address = \"http://$config{'SERVER_ADDR'}\".\":\".\"$config{'PORT'}\";\n+my $wiki_url = \"$wiki_address/$config{'WIKI_DIR_NAME'}/api.php\";\n+my $wiki_admin = \"$config{'WIKI_ADMIN'}\";\n+my $wiki_admin_pass = \"$config{'WIKI_PASSW'}\";\n+my $mw = MediaWiki::API->new;\n+$mw->{config}->{api_url} = $wiki_url;\n+\n+\n+# wiki_login \n+#\n+# Logs the user with and in the global variable\n+# of the mediawiki $mw\n+sub wiki_login {\n+\t$mw->login( { lgname => \"$_[0]\",lgpassword => \"$_[1]\" } )\n+\t|| die \"getpage: login failed\";\n+}\n+\n+# wiki_getpage \n+#\n+# fetch a page from the wiki referenced in the global variable\n+# $mw and copies its content in directory dest_path\n+sub wiki_getpage {\n+\tmy $pagename = $_[0];\n+\tmy $destdir = $_[1];\n+\n+\tmy $page = $mw->get_page( { title => $pagename } );\n+\tif (!defined($page)) {\n+\t\tdie \"getpage: wiki does not exist\";\n+\t}\n+\n+\tmy $content = $page->{'*'};\n+\tif (!defined($content)) {\n+\t\tdie \"getpage: page does not exist\";\n+\t}\n+\n+\t$pagename=$page->{'title'};\n+\t# Replace spaces by underscore in the page name\n+\t$pagename =~ s/ /_/g;\n+\t$pagename =~ s/\\//%2F/g;\n+\topen(my $file, \">$destdir/$pagename.mw\");\n+\tprint $file \"$content\";\n+\tclose ($file);\n+\n+}\n+\n+# wiki_delete_page \n+#\n+# delete the page with name from the wiki referenced\n+# in the global variable $mw\n+sub wiki_delete_page {\n+\tmy $pagename = $_[0];\n+\n+\tmy $exist=$mw->get_page({title => $pagename});\n+\n+\tif (defined($exist->{'*'})){\n+\t\t$mw->edit({ action => 'delete',\n+\t\t\t\ttitle => $pagename})\n+\t\t|| die $mw->{error}->{code} . \": \" . $mw->{error}->{details};\n+\t} else {\n+\t\tdie \"no page with such name found: $pagename\\n\";\n+\t}\n+}\n+\n+# wiki_editpage [-c=] [-s=]\n+#\n+# Edit a page named with content on the wiki\n+# referenced with the global variable $mw\n+# If == true : append at the end of the actual\n+# content of the page \n+# If doesn't exist, that page is created with the \n+sub wiki_editpage {\n+\tmy $wiki_page = $_[0];\n+\tmy $wiki_content = $_[1];\n+\tmy $wiki_append = $_[2];\n+\tmy $summary = \"\";\n+\tmy ($summ, $cat) = ();\n+\tGetOptions('s=s' => \\$summ, 'c=s' => \\$cat);\n+\n+\tmy $append = 0;\n+\tif (defined($wiki_append) && $wiki_append eq 'true') {\n+\t\t$append=1;\n+\t}\n+\n+\tmy $previous_text =\"\";\n+\n+\tif ($append) {\n+\t\tmy $ref = $mw->get_page( { title => $wiki_page } );\n+\t\t$previous_text = $ref->{'*'};\n+\t}\n+\n+\tmy $text = $wiki_content;\n+\tif (defined($previous_text)) {\n+\t\t$text=\"$previous_text$text\";\n+\t}\n+\n+\t# Eventually, add this page to a category.\n+\tif (defined($cat)) {\n+\t\tmy $category_name=\"[[Category:$cat]]\";\n+\t\t$text=\"$text\\n $category_name\";\n+\t}\n+\tif(defined($summ)){\n+\t\t$summary=$summ;\n+\t}\n+\n+\t$mw->edit( { action => 'edit', title => $wiki_page, summary => $summary, text => \"$text\"} );\n+}\n+\n+# wiki_getallpagename []\n+#\n+# Fetch all pages of the wiki referenced by the global variable $mw\n+# and print the names of each one in the file all.txt with a new line\n+# (\"\\n\") between these.\n+# If the argument is defined, then this function get only the pages\n+# belonging to .\n+sub wiki_getallpagename {\n+\t# fetch the pages of the wiki\n+\tif (defined($_[0])) {\n+\t\tmy $mw_pages = $mw->list ( { action => 'query',\n+\t\t\t\tlist => 'categorymembers',\n+\t\t\t\tcmtitle => \"Category:$_[0]\",\n+\t\t\t\tcmnamespace => 0,\n+\t\t\t\tcmlimit => 500 },\n+\t\t)\n+\t\t|| die $mw->{error}->{code}.\": \".$mw->{error}->{details};\n+\t\topen(my $file, \">all.txt\");\n+\t\tforeach my $page (@{$mw_pages}) {\n+\t\t\tprint $file \"$page->{title}\\n\";\n+\t\t}\n+\t\tclose ($file);\n+\n+\t} else {\n+\t\tmy $mw_pages = $mw->list({\n+\t\t\t\taction => 'query',\n+\t\t\t\tlist => 'allpages',\n+\t\t\t\taplimit => 500,\n+\t\t\t})\n+\t\t|| die $mw->{error}->{code}.\": \".$mw->{error}->{details};\n+\t\topen(my $file, \">all.txt\");\n+\t\tforeach my $page (@{$mw_pages}) {\n+\t\t\tprint $file \"$page->{title}\\n\";\n+\t\t}\n+\t\tclose ($file);\n+\t}\n+}\n+\n+# Main part of this script: parse the command line arguments\n+# and select which function to execute\n+my $fct_to_call = shift;\n+\n+wiki_login($wiki_admin, $wiki_admin_pass);\n+\n+my %functions_to_call = qw(\n+\tget_page wiki_getpage\n+\tdelete_page wiki_delete_page\n+\tedit_page wiki_editpage\n+\tgetallpagename wiki_getallpagename\n+);\n+die \"$0 ERROR: wrong argument\" unless exists $functions_to_call{$fct_to_call};\n+&{$functions_to_call{$fct_to_call}}(@ARGV);\n-- \n1.7.11.1.147.g47a574d\n"},{"id":"194704","messageId":"1341568995-12467-4-git-send-email-Matthieu.Moy@imag.fr","threadId":"30954","inReplyTo":"1341568995-12467-1-git-send-email-Matthieu.Moy@imag.fr","subject":"[PATCH 03/12] git-remote-mediawiki (t9360): test git-remote-mediawiki clone","fromName":"Matthieu Moy","fromEmail":"matthieu.moy@imag.fr","sentAt":"2012-07-06T10:03:06Z","receivedAt":"2012-07-06T10:03:06Z","isPatch":true,"sender":{"key":"git@matthieu-moy.fr","avatar":"https://avatars.githubusercontent.com/u/14709?v=4"},"body":"From: Guillaume Sasdy \n\nSigned-off-by: Simon CATHEBRAS \nSigned-off-by: Julien KHAYAT \nSigned-off-by: Simon Perrat \nSigned-off-by: Charles ROUSSEL \nSigned-off-by: Guillaume SASDY \nSigned-off-by: Matthieu Moy \n---\n contrib/mw-to-git/t/t9360-mw-to-git-clone.sh | 257 +++++++++++++++++++++++++++\n 1 file changed, 257 insertions(+)\n create mode 100755 contrib/mw-to-git/t/t9360-mw-to-git-clone.sh\n\ndiff --git a/contrib/mw-to-git/t/t9360-mw-to-git-clone.sh b/contrib/mw-to-git/t/t9360-mw-to-git-clone.sh\nnew file mode 100755\nindex 0000000..811a90c\n--- /dev/null\n+++ b/contrib/mw-to-git/t/t9360-mw-to-git-clone.sh\n@@ -0,0 +1,257 @@\n+#!/bin/sh\n+#\n+# Copyright (C) 2012\n+# Charles Roussel \n+# Simon Cathebras \n+# Julien Khayat \n+# Guillaume Sasdy \n+# Simon Perrat \n+#\n+# License: GPL v2 or later\n+\n+\n+test_description='Test the Git Mediawiki remote helper: git clone'\n+\n+. ./test-gitmw-lib.sh\n+. $TEST_DIRECTORY/test-lib.sh\n+\n+\n+test_check_precond\n+\n+\n+test_expect_success 'Git clone creates the expected git log with one file' '\n+\twiki_reset &&\n+\twiki_editpage foo \"this is not important\" false -c cat -s \"this must be the same\" &&\n+\tgit clone mediawiki::'\"$WIKI_URL\"' mw_dir_1 &&\n+\t(\n+\t\tcd mw_dir_1 &&\n+\t\tgit log --format=%s HEAD^..HEAD >log.tmp\n+\t) &&\n+\techo \"this must be the same\" >msg.tmp &&\n+\tdiff -b mw_dir_1/log.tmp msg.tmp\n+'\n+\n+\n+test_expect_success 'Git clone creates the expected git log with multiple files' '\n+\twiki_reset &&\n+\twiki_editpage daddy \"this is not important\" false -s=\"this must be the same\" &&\n+\twiki_editpage daddy \"neither is this\" true -s=\"this must also be the same\" &&\n+\twiki_editpage daddy \"neither is this\" true -s=\"same same same\" &&\n+\twiki_editpage dj \"dont care\" false -s=\"identical\" &&\n+\twiki_editpage dj \"dont care either\" true -s=\"identical too\" &&\n+\tgit clone mediawiki::'\"$WIKI_URL\"' mw_dir_2 &&\n+\t(\n+\t\tcd mw_dir_2 &&\n+\t\tgit log --format=%s Daddy.mw >logDaddy.tmp &&\n+\t\tgit log --format=%s Dj.mw >logDj.tmp\n+\t) &&\n+\techo \"same same same\" >msgDaddy.tmp &&\n+\techo \"this must also be the same\" >>msgDaddy.tmp &&\n+\techo \"this must be the same\" >>msgDaddy.tmp &&\n+\techo \"identical too\" >msgDj.tmp &&\n+\techo \"identical\" >>msgDj.tmp &&\n+\tdiff -b mw_dir_2/logDaddy.tmp msgDaddy.tmp &&\n+\tdiff -b mw_dir_2/logDj.tmp msgDj.tmp\n+'\n+\n+\n+test_expect_success 'Git clone creates only Main_Page.mw with an empty wiki' '\n+\twiki_reset &&\n+\tgit clone mediawiki::'\"$WIKI_URL\"' mw_dir_3 &&\n+\ttest_contains_N_files mw_dir_3 1 &&\n+\ttest_path_is_file mw_dir_3/Main_Page.mw\n+'\n+\n+test_expect_success 'Git clone does not fetch a deleted page' '\n+\twiki_reset &&\n+\twiki_editpage foo \"this page must be deleted before the clone\" false &&\n+\twiki_delete_page foo &&\n+\tgit clone mediawiki::'\"$WIKI_URL\"' mw_dir_4 &&\n+\ttest_contains_N_files mw_dir_4 1 &&\n+\ttest_path_is_file mw_dir_4/Main_Page.mw &&\n+\ttest_path_is_missing mw_dir_4/Foo.mw\n+'\n+\n+test_expect_success 'Git clone works with page added' '\n+\twiki_reset &&\n+\twiki_editpage foo \" I will be cloned\" false &&\n+\twiki_editpage bar \"I will be cloned\" false &&\n+\tgit clone mediawiki::'\"$WIKI_URL\"' mw_dir_5 &&\n+\twiki_getallpage ref_page_5 &&\n+\ttest_diff_directories mw_dir_5 ref_page_5 &&\n+\twiki_delete_page foo &&\n+\twiki_delete_page bar\n+'\n+\n+test_expect_success 'Git clone works with an edited page ' '\n+\twiki_reset &&\n+\twiki_editpage foo \"this page will be edited\" \\\n+\t\tfalse -s \"first edition of page foo\"&&\n+\twiki_editpage foo \"this page has been edited and must be on the clone \" true &&\n+\tgit clone mediawiki::'\"$WIKI_URL\"' mw_dir_6 &&\n+\ttest_path_is_file mw_dir_6/Foo.mw &&\n+\ttest_path_is_file mw_dir_6/Main_Page.mw &&\n+\twiki_getallpage mw_dir_6/page_ref_6 &&\n+\ttest_diff_directories mw_dir_6 mw_dir_6/page_ref_6 &&\n+\t(\n+\t\tcd mw_dir_6 &&\n+\t\tgit log --format=%s HEAD^ Foo.mw > ../Foo.log\n+\t) &&\n+\techo \"first edition of page foo\" > FooExpect.log &&\n+\tdiff FooExpect.log Foo.log\n+'\n+\n+\n+test_expect_success 'Git clone works with several pages and some deleted ' '\n+\twiki_reset &&\n+\twiki_editpage foo \"this page will not be deleted\" false &&\n+\twiki_editpage bar \"I must not be erased\" false &&\n+\twiki_editpage namnam \"I will not be there at the end\" false &&\n+\twiki_editpage nyancat \"nyan nyan nyan delete me\" false &&\n+\twiki_delete_page namnam &&\n+\twiki_delete_page nyancat &&\n+\tgit clone mediawiki::'\"$WIKI_URL\"' mw_dir_7 &&\n+\ttest_path_is_file mw_dir_7/Foo.mw &&\n+\ttest_path_is_file mw_dir_7/Bar.mw &&\n+\ttest_path_is_missing mw_dir_7/Namnam.mw &&\n+\ttest_path_is_missing mw_dir_7/Nyancat.mw &&\n+\twiki_getallpage mw_dir_7/page_ref_7 &&\n+\ttest_diff_directories mw_dir_7 mw_dir_7/page_ref_7\n+'\n+\n+\n+test_expect_success 'Git clone works with one specific page cloned ' '\n+\twiki_reset &&\n+\twiki_editpage foo \"I will not be cloned\" false &&\n+\twiki_editpage bar \"Do not clone me\" false &&\n+\twiki_editpage namnam \"I will be cloned :)\" false -s=\"this log must stay\" &&\n+\twiki_editpage nyancat \"nyan nyan nyan you cant clone me\" false &&\n+\tgit clone -c remote.origin.pages=namnam \\\n+\t\tmediawiki::'\"$WIKI_URL\"' mw_dir_8 &&\n+\ttest_contains_N_files mw_dir_8 1 &&\n+\ttest_path_is_file mw_dir_8/Namnam.mw &&\n+\ttest_path_is_missing mw_dir_8/Main_Page.mw &&\n+\t(\n+\t\tcd mw_dir_8 &&\n+\t\techo \"this log must stay\" >msg.tmp &&\n+\t\tgit log --format=%s >log.tmp &&\n+\t\tdiff -b msg.tmp log.tmp\n+\t) &&\n+\twiki_check_content mw_dir_8/Namnam.mw Namnam\n+'\n+\n+test_expect_success 'Git clone works with multiple specific page cloned ' '\n+\twiki_reset &&\n+\twiki_editpage foo \"I will be there\" false &&\n+\twiki_editpage bar \"I will not disapear\" false &&\n+\twiki_editpage namnam \"I be erased\" false &&\n+\twiki_editpage nyancat \"nyan nyan nyan you will not erase me\" false &&\n+\twiki_delete_page namnam &&\n+\tgit clone -c remote.origin.pages=\"foo bar nyancat namnam\" \\\n+\t\tmediawiki::'\"$WIKI_URL\"' mw_dir_9 &&\n+\ttest_contains_N_files mw_dir_9 3 &&\n+\ttest_path_is_missing mw_dir_9/Namnam.mw &&\n+\ttest_path_is_file mw_dir_9/Foo.mw &&\n+\ttest_path_is_file mw_dir_9/Nyancat.mw &&\n+\ttest_path_is_file mw_dir_9/Bar.mw &&\n+\twiki_check_content mw_dir_9/Foo.mw Foo &&\n+\twiki_check_content mw_dir_9/Bar.mw Bar &&\n+\twiki_check_content mw_dir_9/Nyancat.mw Nyancat\n+'\n+\n+test_expect_success 'Mediawiki-clone of several specific pages on wiki' '\n+\twiki_reset &&\n+\twiki_editpage foo \"foo 1\" false &&\n+\twiki_editpage bar \"bar 1\" false &&\n+\twiki_editpage dummy \"dummy 1\" false &&\n+\twiki_editpage cloned_1 \"cloned_1 1\" false &&\n+\twiki_editpage cloned_2 \"cloned_2 2\" false &&\n+\twiki_editpage cloned_3 \"cloned_3 3\" false &&\n+\tmkdir -p ref_page_10 &&\n+\twiki_getpage cloned_1 ref_page_10 &&\n+\twiki_getpage cloned_2 ref_page_10 &&\n+\twiki_getpage cloned_3 ref_page_10 &&\n+\tgit clone -c remote.origin.pages=\"cloned_1 cloned_2 cloned_3\" \\\n+\t\tmediawiki::'\"$WIKI_URL\"' mw_dir_10 &&\n+\ttest_diff_directories mw_dir_10 ref_page_10\n+'\n+\n+test_expect_success 'Git clone works with the shallow option' '\n+\twiki_reset &&\n+\twiki_editpage foo \"1st revision, should be cloned\" false &&\n+\twiki_editpage bar \"1st revision, should be cloned\" false &&\n+\twiki_editpage nyan \"1st revision, should not be cloned\" false &&\n+\twiki_editpage nyan \"2nd revision, should be cloned\" false &&\n+\tgit -c remote.origin.shallow=true clone \\\n+\t\tmediawiki::'\"$WIKI_URL\"' mw_dir_11 &&\n+\ttest_contains_N_files mw_dir_11 4 &&\n+\ttest_path_is_file mw_dir_11/Nyan.mw &&\n+\ttest_path_is_file mw_dir_11/Foo.mw &&\n+\ttest_path_is_file mw_dir_11/Bar.mw &&\n+\ttest_path_is_file mw_dir_11/Main_Page.mw &&\n+\t(\n+\t\tcd mw_dir_11 &&\n+\t\ttest `git log --oneline Nyan.mw | wc -l` -eq 1 &&\n+\t\ttest `git log --oneline Foo.mw | wc -l` -eq 1 &&\n+\t\ttest `git log --oneline Bar.mw | wc -l` -eq 1 &&\n+\t\ttest `git log --oneline Main_Page.mw | wc -l ` -eq 1\n+\t) &&\n+\twiki_check_content mw_dir_11/Nyan.mw Nyan &&\n+\twiki_check_content mw_dir_11/Foo.mw Foo &&\n+\twiki_check_content mw_dir_11/Bar.mw Bar &&\n+\twiki_check_content mw_dir_11/Main_Page.mw Main_Page\n+'\n+\n+test_expect_success 'Git clone works with the shallow option with a delete page' '\n+\twiki_reset &&\n+\twiki_editpage foo \"1st revision, will be deleted\" false &&\n+\twiki_editpage bar \"1st revision, should be cloned\" false &&\n+\twiki_editpage nyan \"1st revision, should not be cloned\" false &&\n+\twiki_editpage nyan \"2nd revision, should be cloned\" false &&\n+\twiki_delete_page foo &&\n+\tgit -c remote.origin.shallow=true clone \\\n+\t\tmediawiki::'\"$WIKI_URL\"' mw_dir_12 &&\n+\ttest_contains_N_files mw_dir_12 3 &&\n+\ttest_path_is_file mw_dir_12/Nyan.mw &&\n+\ttest_path_is_missing mw_dir_12/Foo.mw &&\n+\ttest_path_is_file mw_dir_12/Bar.mw &&\n+\ttest_path_is_file mw_dir_12/Main_Page.mw &&\n+\t(\n+\t\tcd mw_dir_12 &&\n+\t\ttest `git log --oneline Nyan.mw | wc -l` -eq 1 &&\n+\t\ttest `git log --oneline Bar.mw | wc -l` -eq 1 &&\n+\t\ttest `git log --oneline Main_Page.mw | wc -l ` -eq 1\n+\t) &&\n+\twiki_check_content mw_dir_12/Nyan.mw Nyan &&\n+\twiki_check_content mw_dir_12/Bar.mw Bar &&\n+\twiki_check_content mw_dir_12/Main_Page.mw Main_Page\n+'\n+\n+test_expect_success 'Test of fetching a category' '\n+\twiki_reset &&\n+\twiki_editpage Foo \"I will be cloned\" false -c=Category &&\n+\twiki_editpage Bar \"Meet me on the repository\" false -c=Category &&\n+\twiki_editpage Dummy \"I will not come\" false &&\n+\twiki_editpage BarWrong \"I will stay online only\" false -c=NotCategory &&\n+\tgit clone -c remote.origin.categories=\"Category\" \\\n+\t\tmediawiki::'\"$WIKI_URL\"' mw_dir_13 &&\n+\twiki_getallpage ref_page_13 Category &&\n+\ttest_diff_directories mw_dir_13 ref_page_13\n+'\n+\n+test_expect_success 'Test of resistance to modification of category on wiki for clone' '\n+\twiki_reset &&\n+\twiki_editpage Tobedeleted \"this page will be deleted\" false -c=Catone &&\n+\twiki_editpage Tobeedited \"this page will be modified\" false -c=Catone &&\n+\twiki_editpage Normalone \"this page wont be modified and will be on git\" false -c=Catone &&\n+\twiki_editpage Notconsidered \"this page will not appear on local\" false &&\n+\twiki_editpage Othercategory \"this page will not appear on local\" false -c=Cattwo &&\n+\twiki_editpage Tobeedited \"this page have been modified\" true -c=Catone &&\n+\twiki_delete_page Tobedeleted\n+\tgit clone -c remote.origin.categories=\"Catone\" \\\n+\t\tmediawiki::'\"$WIKI_URL\"' mw_dir_14 &&\n+\twiki_getallpage ref_page_14 Catone &&\n+\ttest_diff_directories mw_dir_14 ref_page_14\n+'\n+\n+test_done\n-- \n1.7.11.1.147.g47a574d\n"},{"id":"194707","messageId":"1341568995-12467-5-git-send-email-Matthieu.Moy@imag.fr","threadId":"30954","inReplyTo":"1341568995-12467-1-git-send-email-Matthieu.Moy@imag.fr","subject":"[PATCH 04/12] git-remote-mediawiki (t9361): test git-remote-mediawiki pull and push","fromName":"Matthieu Moy","fromEmail":"matthieu.moy@imag.fr","sentAt":"2012-07-06T10:03:07Z","receivedAt":"2012-07-06T10:03:07Z","isPatch":true,"sender":{"key":"git@matthieu-moy.fr","avatar":"https://avatars.githubusercontent.com/u/14709?v=4"},"body":"From: Guillaume Sasdy \n\nThis patch provides a set of tests for the pull and push fonctionnality\nof git-remote-mediawiki. The actual tests are kept in a separate function\nto allow further tests to re-run the same set of commands with different\npush and pull strategies.\n\nSigned-off-by: Simon CATHEBRAS \nSigned-off-by: Julien KHAYAT \nSigned-off-by: Simon Perrat \nSigned-off-by: Charles ROUSSEL \nSigned-off-by: Guillaume SASDY \nSigned-off-by: Matthieu Moy \n---\n contrib/mw-to-git/t/push-pull-tests.sh | 144 +++++++++++++++++++++++\n contrib/mw-to-git/t/t9361-mw-to-git-push-pull.sh | 24 ++++\n 2 files changed, 168 insertions(+)\n create mode 100644 contrib/mw-to-git/t/push-pull-tests.sh\n create mode 100755 contrib/mw-to-git/t/t9361-mw-to-git-push-pull.sh\n\ndiff --git a/contrib/mw-to-git/t/push-pull-tests.sh b/contrib/mw-to-git/t/push-pull-tests.sh\nnew file mode 100644\nindex 0000000..6692a0f\n--- /dev/null\n+++ b/contrib/mw-to-git/t/push-pull-tests.sh\n@@ -0,0 +1,144 @@\n+test_push_pull () {\n+\n+\ttest_expect_success 'Git pull works after adding a new wiki page' '\n+\t\twiki_reset &&\n+\n+\t\tgit clone mediawiki::'\"$WIKI_URL\"' mw_dir_1 &&\n+\t\twiki_editpage Foo \"page created after the git clone\" false &&\n+\n+\t\t(\n+\t\t\tcd mw_dir_1 &&\n+\t\t\tgit pull\n+\t\t) &&\n+\n+\t\twiki_getallpage ref_page_1 &&\n+\t\ttest_diff_directories mw_dir_1 ref_page_1\n+\t'\n+\n+\ttest_expect_success 'Git pull works after editing a wiki page' '\n+\t\twiki_reset &&\n+\n+\t\twiki_editpage Foo \"page created before the git clone\" false &&\n+\t\tgit clone mediawiki::'\"$WIKI_URL\"' mw_dir_2 &&\n+\t\twiki_editpage Foo \"new line added on the wiki\" true &&\n+\n+\t\t(\n+\t\t\tcd mw_dir_2 &&\n+\t\t\tgit pull\n+\t\t) &&\n+\n+\t\twiki_getallpage ref_page_2 &&\n+\t\ttest_diff_directories mw_dir_2 ref_page_2\n+\t'\n+\n+\ttest_expect_success 'git pull works on conflict handled by auto-merge' '\n+\t\twiki_reset &&\n+\n+\t\twiki_editpage Foo \"1 init\n+3\n+5\n+\t\" false &&\n+\t\tgit clone mediawiki::'\"$WIKI_URL\"' mw_dir_3 &&\n+\n+\t\twiki_editpage Foo \"1 init\n+2 content added on wiki after clone\n+3\n+5\n+\t\" false &&\n+\n+\t\t(\n+\t\t\tcd mw_dir_3 &&\n+\t\techo \"1 init\n+3\n+4 content added on git after clone\n+5\n+\" >Foo.mw &&\n+\t\t\tgit commit -am \"conflicting change on foo\" &&\n+\t\t\tgit pull &&\n+\t\t\tgit push\n+\t\t)\n+\t'\n+\n+\ttest_expect_success 'Git push works after adding a file .mw' '\n+\t\twiki_reset &&\n+\t\tgit clone mediawiki::'\"$WIKI_URL\"' mw_dir_4 &&\n+\t\twiki_getallpage ref_page_4 &&\n+\t\t(\n+\t\t\tcd mw_dir_4 &&\n+\t\t\ttest_path_is_missing Foo.mw &&\n+\t\t\ttouch Foo.mw &&\n+\t\t\techo \"hello world\" >>Foo.mw &&\n+\t\t\tgit add Foo.mw &&\n+\t\t\tgit commit -m \"Foo\" &&\n+\t\t\tgit push\n+\t\t) &&\n+\t\twiki_getallpage ref_page_4 &&\n+\t\ttest_diff_directories mw_dir_4 ref_page_4\n+\t'\n+\n+\ttest_expect_success 'Git push works after editing a file .mw' '\n+\t\twiki_reset &&\n+\t\twiki_editpage \"Foo\" \"page created before the git clone\" false &&\n+\t\tgit clone mediawiki::'\"$WIKI_URL\"' mw_dir_5 &&\n+\n+\t\t(\n+\t\t\tcd mw_dir_5 &&\n+\t\t\techo \"new line added in the file Foo.mw\" >>Foo.mw &&\n+\t\t\tgit commit -am \"edit file Foo.mw\" &&\n+\t\t\tgit push\n+\t\t) &&\n+\n+\t\twiki_getallpage ref_page_5 &&\n+\t\ttest_diff_directories mw_dir_5 ref_page_5\n+\t'\n+\n+\ttest_expect_failure 'Git push works after deleting a file' '\n+\t\twiki_reset &&\n+\t\twiki_editpage Foo \"wiki page added before git clone\" false &&\n+\t\tgit clone mediawiki::'\"$WIKI_URL\"' mw_dir_6 &&\n+\n+\t\t(\n+\t\t\tcd mw_dir_6 &&\n+\t\t\tgit rm Foo.mw &&\n+\t\t\tgit commit -am \"page Foo.mw deleted\" &&\n+\t\t\tgit push\n+\t\t) &&\n+\n+\t\ttest ! wiki_page_exist Foo\n+\t'\n+\n+\ttest_expect_success 'Merge conflict expected and solving it' '\n+\t\twiki_reset &&\n+\n+\t\tgit clone mediawiki::'\"$WIKI_URL\"' mw_dir_7 &&\n+\t\twiki_editpage Foo \"1 conflict\n+3 wiki\n+4\" false &&\n+\n+\t\t(\n+\t\t\tcd mw_dir_7 &&\n+\t\techo \"1 conflict\n+2 git\n+4\" >Foo.mw &&\n+\t\t\tgit add Foo.mw &&\n+\t\t\tgit commit -m \"conflict created\" &&\n+\t\t\ttest_must_fail git pull &&\n+\t\t\t\"$PERL_PATH\" -pi -e \"s/[<=>].*//g\" Foo.mw &&\n+\t\t\tgit commit -am \"merge conflict solved\" &&\n+\t\t\tgit push\n+\t\t)\n+\t'\n+\n+\ttest_expect_failure 'git pull works after deleting a wiki page' '\n+\t\twiki_reset &&\n+\t\twiki_editpage Foo \"wiki page added before the git clone\" false &&\n+\t\tgit clone mediawiki::'\"$WIKI_URL\"' mw_dir_8 &&\n+\n+\t\twiki_delete_page Foo &&\n+\t\t(\n+\t\t\tcd mw_dir_8 &&\n+\t\t\tgit pull &&\n+\t\t\ttest_path_is_missing Foo.mw\n+\t\t)\n+\t'\n+}\ndiff --git a/contrib/mw-to-git/t/t9361-mw-to-git-push-pull.sh b/contrib/mw-to-git/t/t9361-mw-to-git-push-pull.sh\nnew file mode 100755\nindex 0000000..9ea2014\n--- /dev/null\n+++ b/contrib/mw-to-git/t/t9361-mw-to-git-push-pull.sh\n@@ -0,0 +1,24 @@\n+#!/bin/sh\n+#\n+# Copyright (C) 2012\n+# Charles Roussel \n+# Simon Cathebras \n+# Julien Khayat \n+# Guillaume Sasdy \n+# Simon Perrat \n+#\n+# License: GPL v2 or later\n+\n+# tests for git-remote-mediawiki\n+\n+test_description='Test the Git Mediawiki remote helper: git push and git pull simple test cases'\n+\n+. ./test-gitmw-lib.sh\n+. ./push-pull-tests.sh\n+. $TEST_DIRECTORY/test-lib.sh\n+\n+test_check_precond\n+\n+test_push_pull\n+\n+test_done\n-- \n1.7.11.1.147.g47a574d\n"},{"id":"194708","messageId":"1341568995-12467-6-git-send-email-Matthieu.Moy@imag.fr","threadId":"30954","inReplyTo":"1341568995-12467-1-git-send-email-Matthieu.Moy@imag.fr","subject":"[PATCH 05/12] git-remote-mediawiki (t9362): test git-remote-mediawiki with UTF8 characters","fromName":"Matthieu Moy","fromEmail":"matthieu.moy@imag.fr","sentAt":"2012-07-06T10:03:08Z","receivedAt":"2012-07-06T10:03:08Z","isPatch":true,"sender":{"key":"git@matthieu-moy.fr","avatar":"https://avatars.githubusercontent.com/u/14709?v=4"},"body":"From: Simon Cathebras \n\nNon-ascii encoding create many particular cases when used in page\ncontent, name, and edit/commit message. Test these cases.\n\nSigned-off-by: Simon CATHEBRAS \nSigned-off-by: Julien KHAYAT \nSigned-off-by: Simon Perrat \nSigned-off-by: Charles ROUSSEL \nSigned-off-by: Guillaume SASDY \nSigned-off-by: Matthieu Moy \n---\n contrib/mw-to-git/t/t9362-mw-to-git-utf8.sh | 301 ++++++++++++++++++++++++++++\n 1 file changed, 301 insertions(+)\n create mode 100755 contrib/mw-to-git/t/t9362-mw-to-git-utf8.sh\n\ndiff --git a/contrib/mw-to-git/t/t9362-mw-to-git-utf8.sh b/contrib/mw-to-git/t/t9362-mw-to-git-utf8.sh\nnew file mode 100755\nindex 0000000..8635878\n--- /dev/null\n+++ b/contrib/mw-to-git/t/t9362-mw-to-git-utf8.sh\n@@ -0,0 +1,301 @@\n+#!/bin/sh\n+#\n+# Copyright (C) 2012\n+# Charles Roussel \n+# Simon Cathebras \n+# Julien Khayat \n+# Guillaume Sasdy \n+# Simon Perrat \n+#\n+# License: GPL v2 or later\n+\n+# tests for git-remote-mediawiki\n+\n+test_description='Test git-mediawiki with special characters in filenames'\n+\n+. ./test-gitmw-lib.sh\n+. $TEST_DIRECTORY/test-lib.sh\n+\n+\n+test_check_precond\n+\n+\n+test_expect_success 'Git clone works for a wiki with accents in the page names' '\n+\twiki_reset &&\n+\twiki_editpage féé \"This page must be délétéd before clone\" false &&\n+\twiki_editpage kèè \"This page must be deleted before clone\" false &&\n+\twiki_editpage hàà \"This page must be deleted before clone\" false &&\n+\twiki_editpage kîî \"This page must be deleted before clone\" false &&\n+\twiki_editpage foo \"This page must be deleted before clone\" false &&\n+\tgit clone mediawiki::'\"$WIKI_URL\"' mw_dir_1 &&\n+\twiki_getallpage ref_page_1 &&\n+\ttest_diff_directories mw_dir_1 ref_page_1\n+'\n+\n+\n+test_expect_success 'Git pull works with a wiki with accents in the pages names' '\n+\twiki_reset &&\n+\twiki_editpage kîî \"this page must be cloned\" false &&\n+\twiki_editpage foo \"this page must be cloned\" false &&\n+\tgit clone mediawiki::'\"$WIKI_URL\"' mw_dir_2 &&\n+\twiki_editpage éàîôû \"This page must be pulled\" false &&\n+\t(\n+\t\tcd mw_dir_2 &&\n+\t\tgit pull\n+\t) &&\n+\twiki_getallpage ref_page_2 &&\n+\ttest_diff_directories mw_dir_2 ref_page_2\n+'\n+\n+\n+test_expect_success 'Cloning a chosen page works with accents' '\n+\twiki_reset &&\n+\twiki_editpage kîî \"this page must be cloned\" false &&\n+\tgit clone -c remote.origin.pages=kîî \\\n+\t\tmediawiki::'\"$WIKI_URL\"' mw_dir_3 &&\n+\twiki_check_content mw_dir_3/Kîî.mw Kîî &&\n+\ttest_path_is_file mw_dir_3/Kîî.mw &&\n+\trm -rf mw_dir_3\n+'\n+\n+\n+test_expect_success 'The shallow option works with accents' '\n+\twiki_reset &&\n+\twiki_editpage néoà \"1st revision, should not be cloned\" false &&\n+\twiki_editpage néoà \"2nd revision, should be cloned\" false &&\n+\tgit -c remote.origin.shallow=true clone \\\n+\t\tmediawiki::'\"$WIKI_URL\"' mw_dir_4 &&\n+\ttest_contains_N_files mw_dir_4 2 &&\n+\ttest_path_is_file mw_dir_4/Néoà.mw &&\n+\ttest_path_is_file mw_dir_4/Main_Page.mw &&\n+\t(\n+\t\tcd mw_dir_4 &&\n+\t\ttest `git log --oneline Néoà.mw | wc -l` -eq 1 &&\n+\t\ttest `git log --oneline Main_Page.mw | wc -l ` -eq 1\n+\t) &&\n+\twiki_check_content mw_dir_4/Néoà.mw Néoà &&\n+\twiki_check_content mw_dir_4/Main_Page.mw Main_Page\n+'\n+\n+\n+test_expect_success 'Cloning works when page name first letter has an accent' '\n+\twiki_reset &&\n+\twiki_editpage îî \"this page must be cloned\" false &&\n+\tgit clone -c remote.origin.pages=îî \\\n+\t\tmediawiki::'\"$WIKI_URL\"' mw_dir_5 &&\n+\ttest_path_is_file mw_dir_5/Îî.mw &&\n+\twiki_check_content mw_dir_5/Îî.mw Îî\n+'\n+\n+\n+test_expect_success 'Git push works with a wiki with accents' '\n+\twiki_reset &&\n+\twiki_editpage féé \"lots of accents : éèàÖ\" false &&\n+\twiki_editpage foo \"this page must be cloned\" false &&\n+\tgit clone mediawiki::'\"$WIKI_URL\"' mw_dir_6 &&\n+\t(\n+\t\tcd mw_dir_6 &&\n+\t\techo \"A wild Pîkächû appears on the wiki\" >Pîkächû.mw &&\n+\t\tgit add Pîkächû.mw &&\n+\t\tgit commit -m \"A new page appears\" &&\n+\t\tgit push\n+\t) &&\n+\twiki_getallpage ref_page_6 &&\n+\ttest_diff_directories mw_dir_6 ref_page_6\n+'\n+\n+test_expect_success 'Git clone works with accentsand spaces' '\n+\twiki_reset &&\n+\twiki_editpage \"é à î\" \"this page must be délété before the clone\" false &&\n+\tgit clone mediawiki::'\"$WIKI_URL\"' mw_dir_7 &&\n+\twiki_getallpage ref_page_7 &&\n+\ttest_diff_directories mw_dir_7 ref_page_7\n+'\n+\n+test_expect_success 'character $ in page name (mw -> git)' '\n+\twiki_reset &&\n+\twiki_editpage file_\\$_foo \"expect to be called file_$_foo\" false &&\n+\tgit clone mediawiki::'\"$WIKI_URL\"' mw_dir_8 &&\n+\ttest_path_is_file mw_dir_8/File_\\$_foo.mw &&\n+\twiki_getallpage ref_page_8 &&\n+\ttest_diff_directories mw_dir_8 ref_page_8\n+'\n+\n+\n+\n+test_expect_success 'character $ in file name (git -> mw) ' '\n+\twiki_reset &&\n+\tgit clone mediawiki::'\"$WIKI_URL\"' mw_dir_9 &&\n+\t(\n+\t\tcd mw_dir_9 &&\n+\t\techo \"this file is called File_\\$_foo.mw\" >File_\\$_foo.mw &&\n+\t\tgit add . &&\n+\t\tgit commit -am \"file File_\\$_foo.mw\" &&\n+\t\tgit pull &&\n+\t\tgit push\n+\t) &&\n+\twiki_getallpage ref_page_9 &&\n+\ttest_diff_directories mw_dir_9 ref_page_9\n+'\n+\n+\n+test_expect_failure 'capital at the begining of file names' '\n+\twiki_reset &&\n+\tgit clone mediawiki::'\"$WIKI_URL\"' mw_dir_10 &&\n+\t(\n+\t\tcd mw_dir_10 &&\n+\t\techo \"my new file foo\" >foo.mw &&\n+\t\techo \"my new file Foo... Finger crossed\" >Foo.mw &&\n+\t\tgit add . &&\n+\t\tgit commit -am \"file foo.mw\" &&\n+\t\tgit pull &&\n+\t\tgit push\n+\t) &&\n+\twiki_getallpage ref_page_10 &&\n+\ttest_diff_directories mw_dir_10 ref_page_10\n+'\n+\n+\n+test_expect_failure 'special character at the begining of file name from mw to git' '\n+\twiki_reset &&\n+\tgit clone mediawiki::'\"$WIKI_URL\"' mw_dir_11 &&\n+\twiki_editpage {char_1 \"expect to be renamed {char_1\" false &&\n+\twiki_editpage [char_2 \"expect to be renamed [char_2\" false &&\n+\t(\n+\t\tcd mw_dir_11 &&\n+\t\tgit pull\n+\t) &&\n+\ttest_path_is_file mw_dir_11/{char_1 &&\n+\ttest_path_is_file mw_dir_11/[char_2\n+'\n+\n+test_expect_success 'test of correct formating for file name from mw to git' '\n+\twiki_reset &&\n+\tgit clone mediawiki::'\"$WIKI_URL\"' mw_dir_12 &&\n+\twiki_editpage char_%_7b_1 \"expect to be renamed char{_1\" false &&\n+\twiki_editpage char_%_5b_2 \"expect to be renamed char{_2\" false &&\n+\t(\n+\t\tcd mw_dir_12 &&\n+\t\tgit pull\n+\t) &&\n+\ttest_path_is_file mw_dir_12/Char\\{_1.mw &&\n+\ttest_path_is_file mw_dir_12/Char\\[_2.mw &&\n+\twiki_getallpage ref_page_12 &&\n+\tmv ref_page_12/Char_%_7b_1.mw ref_page_12/Char\\{_1.mw &&\n+\tmv ref_page_12/Char_%_5b_2.mw ref_page_12/Char\\[_2.mw &&\n+\ttest_diff_directories mw_dir_12 ref_page_12\n+'\n+\n+\n+test_expect_failure 'test of correct formating for file name begining with special character' '\n+\twiki_reset &&\n+\tgit clone mediawiki::'\"$WIKI_URL\"' mw_dir_13 &&\n+\t(\n+\t\tcd mw_dir_13 &&\n+\t\techo \"my new file {char_1\" >\\{char_1.mw &&\n+\t\techo \"my new file [char_2\" >\\[char_2.mw &&\n+\t\tgit add . &&\n+\t\tgit commit -am \"commiting some exotic file name...\" &&\n+\t\tgit push &&\n+\t\tgit pull\n+\t) &&\n+\twiki_getallpage ref_page_13 &&\n+\ttest_path_is_file ref_page_13/{char_1.mw &&\n+\ttest_path_is_file ref_page_13/[char_2.mw &&\n+\ttest_diff_directories mw_dir_13 ref_page_13\n+'\n+\n+\n+test_expect_success 'test of correct formating for file name from git to mw' '\n+\twiki_reset &&\n+\tgit clone mediawiki::'\"$WIKI_URL\"' mw_dir_14 &&\n+\t(\n+\t\tcd mw_dir_14 &&\n+\t\techo \"my new file char{_1\" >Char\\{_1.mw &&\n+\t\techo \"my new file char[_2\" >Char\\[_2.mw &&\n+\t\tgit add . &&\n+\t\tgit commit -m \"commiting some exotic file name...\" &&\n+\t\tgit push\n+\t) &&\n+\twiki_getallpage ref_page_14 &&\n+\tmv mw_dir_14/Char\\{_1.mw mw_dir_14/Char_%_7b_1.mw &&\n+\tmv mw_dir_14/Char\\[_2.mw mw_dir_14/Char_%_5b_2.mw &&\n+\ttest_diff_directories mw_dir_14 ref_page_14\n+'\n+\n+\n+test_expect_success 'git clone with /' '\n+\twiki_reset &&\n+\twiki_editpage \\/fo\\/o \"this is not important\" false -c=Deleted &&\n+\tgit clone mediawiki::'\"$WIKI_URL\"' mw_dir_15 &&\n+\ttest_path_is_file mw_dir_15/%2Ffo%2Fo.mw &&\n+\twiki_check_content mw_dir_15/%2Ffo%2Fo.mw \\/fo\\/o\n+'\n+\n+\n+test_expect_success 'git push with /' '\n+\twiki_reset &&\n+\tgit clone mediawiki::'\"$WIKI_URL\"' mw_dir_16 &&\n+\techo \"I will be on the wiki\" >mw_dir_16/%2Ffo%2Fo.mw &&\n+\t(\n+\t\tcd mw_dir_16 &&\n+\t\tgit add %2Ffo%2Fo.mw &&\n+\t\tgit commit -m \" %2Ffo%2Fo added\" &&\n+\t\tgit push\n+\t) &&\n+\twiki_page_exist \\/fo\\/o &&\n+\twiki_check_content mw_dir_16/%2Ffo%2Fo.mw \\/fo\\/o\n+\n+'\n+\n+\n+test_expect_success 'git clone with \\' '\n+\twiki_reset &&\n+\twiki_editpage \\\\ko\\\\o \"this is not important\" false -c=Deleted &&\n+\tgit clone mediawiki::'\"$WIKI_URL\"' mw_dir_17 &&\n+\ttest_path_is_file mw_dir_17/\\\\ko\\\\o.mw &&\n+\twiki_check_content mw_dir_17/\\\\ko\\\\o.mw \\\\ko\\\\o\n+'\n+\n+\n+test_expect_success 'git push with \\' '\n+\twiki_reset &&\n+\tgit clone mediawiki::'\"$WIKI_URL\"' mw_dir_18 &&\n+\techo \"I will be on the wiki\" >mw_dir_18/\\\\ko\\\\o.mw &&\n+\t(\n+\t\tcd mw_dir_18 &&\n+\t\tgit add \\\\ko\\\\o.mw &&\n+\t\tgit commit -m \" \\\\ko\\\\o added\" &&\n+\t\tgit push\n+\t)&&\n+\twiki_page_exist \\\\ko\\\\o &&\n+\twiki_check_content mw_dir_18/\\\\ko\\\\o.mw \\\\ko\\\\o\n+\n+'\n+\n+test_expect_success 'git clone with \\ in format control' '\n+\twiki_reset &&\n+\twiki_editpage \\\\no\\\\o \"this is not important\" false &&\n+\tgit clone mediawiki::'\"$WIKI_URL\"' mw_dir_19 &&\n+\ttest_path_is_file mw_dir_19/\\\\no\\\\o.mw &&\n+\twiki_check_content mw_dir_19/\\\\no\\\\o.mw \\\\no\\\\o\n+'\n+\n+\n+test_expect_success 'git push with \\ in format control' '\n+\twiki_reset &&\n+\tgit clone mediawiki::'\"$WIKI_URL\"' mw_dir_20 &&\n+\techo \"I will be on the wiki\" >mw_dir_20/\\\\fo\\\\o.mw &&\n+\t(\n+\t\tcd mw_dir_20 &&\n+\t\tgit add \\\\fo\\\\o.mw &&\n+\t\tgit commit -m \" \\\\fo\\\\o added\" &&\n+\t\tgit push\n+\t)&&\n+\twiki_page_exist \\\\fo\\\\o &&\n+\twiki_check_content mw_dir_20/\\\\fo\\\\o.mw \\\\fo\\\\o\n+\n+'\n+\n+\n+test_done\n-- \n1.7.11.1.147.g47a574d\n"},{"id":"194715","messageId":"1341568995-12467-7-git-send-email-Matthieu.Moy@imag.fr","threadId":"30954","inReplyTo":"1341568995-12467-1-git-send-email-Matthieu.Moy@imag.fr","subject":"[PATCH 06/12] git-remote-mediawiki: support for uploading file in test environment","fromName":"Matthieu Moy","fromEmail":"matthieu.moy@imag.fr","sentAt":"2012-07-06T10:03:09Z","receivedAt":"2012-07-06T10:03:09Z","isPatch":true,"sender":{"key":"git@matthieu-moy.fr","avatar":"https://avatars.githubusercontent.com/u/14709?v=4"},"body":"This will be used for testing git-remote-mediawiki's import feature on a\nwiki containing media files.\n\nSigned-off-by: Simon CATHEBRAS \nSigned-off-by: Julien KHAYAT \nSigned-off-by: Simon Perrat \nSigned-off-by: Charles ROUSSEL \nSigned-off-by: Guillaume SASDY \nSigned-off-by: Matthieu Moy \n---\n contrib/mw-to-git/t/install-wiki/LocalSettings.php | 3 ++-\n contrib/mw-to-git/t/test-gitmw-lib.sh | 3 +++\n contrib/mw-to-git/t/test-gitmw.pl | 16 ++++++++++++++++\n 3 files changed, 21 insertions(+), 1 deletion(-)\n\ndiff --git a/contrib/mw-to-git/t/install-wiki/LocalSettings.php b/contrib/mw-to-git/t/install-wiki/LocalSettings.php\nindex aa72518..29f1251 100644\n--- a/contrib/mw-to-git/t/install-wiki/LocalSettings.php\n+++ b/contrib/mw-to-git/t/install-wiki/LocalSettings.php\n@@ -67,9 +67,10 @@ $wgMemCachedServers = array();\n \n ## To enable image uploads, make sure the 'images' directory\n ## is writable, then set this to true:\n-$wgEnableUploads = false;\n+$wgEnableUploads = true;\n $wgUseImageMagick = true;\n $wgImageMagickConvertCommand =\"@CONVERT@\";\n+$wgFileExtensions[] = 'txt';\n \n # InstantCommons allows wiki to use images from http://commons.wikimedia.org\n $wgUseInstantCommons = false;\ndiff --git a/contrib/mw-to-git/t/test-gitmw-lib.sh b/contrib/mw-to-git/t/test-gitmw-lib.sh\nindex 14f6577..3b2cfac 100755\n--- a/contrib/mw-to-git/t/test-gitmw-lib.sh\n+++ b/contrib/mw-to-git/t/test-gitmw-lib.sh\n@@ -26,6 +26,9 @@ else\n \tWIKI_DIR_INST=\"$CURR_DIR/$WEB_WWW\"\n fi\n \n+wiki_upload_file () {\n+\t\"$CURR_DIR\"/test-gitmw.pl upload_file \"$@\"\n+}\n \n wiki_getpage () {\n \t\"$CURR_DIR\"/test-gitmw.pl get_page \"$@\"\ndiff --git a/contrib/mw-to-git/t/test-gitmw.pl b/contrib/mw-to-git/t/test-gitmw.pl\nindex b0117e2..0ff7625 100755\n--- a/contrib/mw-to-git/t/test-gitmw.pl\n+++ b/contrib/mw-to-git/t/test-gitmw.pl\n@@ -193,6 +193,21 @@ sub wiki_getallpagename {\n \t}\n }\n \n+sub wiki_upload_file {\n+\tmy $file_name = $_[0];\n+\tmy $resultat = $mw->edit ( {\n+\t\taction => 'upload',\n+\t\tfilename => $file_name,\n+\t\tcomment => 'upload a file',\n+\t\tfile => [ $file_name ],\n+\t\tignorewarnings=>1,\n+\t}, {\n+\t\tskip_encoding => 1\n+\t} ) || die $mw->{error}->{code} . ' : ' . $mw->{error}->{details};\n+}\n+\n+\n+\n # Main part of this script: parse the command line arguments\n # and select which function to execute\n my $fct_to_call = shift;\n@@ -200,6 +215,7 @@ my $fct_to_call = shift;\n wiki_login($wiki_admin, $wiki_admin_pass);\n \n my %functions_to_call = qw(\n+\tupload_file wiki_upload_file\n \tget_page wiki_getpage\n \tdelete_page wiki_delete_page\n \tedit_page wiki_editpage\n-- \n1.7.11.1.147.g47a574d\n"},{"id":"194711","messageId":"1341568995-12467-8-git-send-email-Matthieu.Moy@imag.fr","threadId":"30954","inReplyTo":"1341568995-12467-1-git-send-email-Matthieu.Moy@imag.fr","subject":"[PATCH 07/12] git-remote-mediawiki (t9363): test 'File:' import and export","fromName":"Matthieu Moy","fromEmail":"matthieu.moy@imag.fr","sentAt":"2012-07-06T10:03:10Z","receivedAt":"2012-07-06T10:03:10Z","isPatch":true,"sender":{"key":"git@matthieu-moy.fr","avatar":"https://avatars.githubusercontent.com/u/14709?v=4"},"body":"From: NGUYEN Kim Thuat \n\nSigned-off-by: Pavel Volek \nSigned-off-by: NGUYEN Kim Thuat \nSigned-off-by: ROUCHER IGLESIAS Javier \nSigned-off-by: Matthieu Moy \n---\n .../mw-to-git/t/t9363-mw-to-git-export-import.sh | 198 +++++++++++++++++++++\n 1 file changed, 198 insertions(+)\n create mode 100755 contrib/mw-to-git/t/t9363-mw-to-git-export-import.sh\n\ndiff --git a/contrib/mw-to-git/t/t9363-mw-to-git-export-import.sh b/contrib/mw-to-git/t/t9363-mw-to-git-export-import.sh\nnew file mode 100755\nindex 0000000..5a03739\n--- /dev/null\n+++ b/contrib/mw-to-git/t/t9363-mw-to-git-export-import.sh\n@@ -0,0 +1,198 @@\n+#!/bin/sh\n+#\n+# Copyright (C) 2012\n+# Charles Roussel \n+# Simon Cathebras \n+# Julien Khayat \n+# Guillaume Sasdy \n+# Simon Perrat \n+#\n+# License: GPL v2 or later\n+\n+# tests for git-remote-mediawiki\n+\n+test_description='Test the Git Mediawiki remote helper: git push and git pull simple test cases'\n+\n+. ./test-gitmw-lib.sh\n+. $TEST_DIRECTORY/test-lib.sh\n+\n+\n+test_check_precond\n+\n+\n+test_git_reimport () {\n+\tgit -c remote.origin.dumbPush=true push &&\n+\tgit -c remote.origin.mediaImport=true pull --rebase\n+}\n+\n+# Don't bother with permissions, be administrator by default\n+test_expect_success 'setup config' '\n+\tgit config --global remote.origin.mwLogin WikiAdmin &&\n+\tgit config --global remote.origin.mwPassword AdminPass &&\n+\ttest_might_fail git config --global --unset remote.origin.mediaImport\n+'\n+\n+test_expect_success 'git push can upload media (File:) files' '\n+\twiki_reset &&\n+\tgit clone mediawiki::'\"$WIKI_URL\"' mw_dir &&\n+\t(\n+\t\tcd mw_dir &&\n+\t\techo \"hello world\" >Foo.txt &&\n+\t\tgit add Foo.txt &&\n+\t\tgit commit -m \"add a text file\" &&\n+\t\tgit push &&\n+\t\t\"$PERL_PATH\" -e \"print STDOUT \\\"binary content: \\\".chr(255);\" >Foo.txt &&\n+\t\tgit add Foo.txt &&\n+\t\tgit commit -m \"add a text file with binary content\" &&\n+\t\tgit push\n+\t)\n+'\n+\n+test_expect_success 'git clone works on previously created wiki with media files' '\n+\ttest_when_finished \"rm -rf mw_dir mw_dir_clone\" &&\n+\tgit clone -c remote.origin.mediaimport=true \\\n+\t\tmediawiki::'\"$WIKI_URL\"' mw_dir_clone &&\n+\ttest_cmp mw_dir_clone/Foo.txt mw_dir/Foo.txt &&\n+\t(cd mw_dir_clone && git checkout HEAD^) &&\n+\t(cd mw_dir && git checkout HEAD^) &&\n+\ttest_cmp mw_dir_clone/Foo.txt mw_dir/Foo.txt\n+'\n+\n+test_expect_success 'git push & pull work with locally renamed media files' '\n+\twiki_reset &&\n+\tgit clone mediawiki::'\"$WIKI_URL\"' mw_dir &&\n+\ttest_when_finished \"rm -fr mw_dir\" &&\n+\t(\n+\t\tcd mw_dir &&\n+\t\techo \"A File\" >Foo.txt &&\n+\t\tgit add Foo.txt &&\n+\t\tgit commit -m \"add a file\" &&\n+\t\tgit mv Foo.txt Bar.txt &&\n+\t\tgit commit -m \"Rename a file\" &&\n+\t\ttest_git_reimport &&\n+\t\techo \"A File\" >expect &&\n+\t\ttest_cmp expect Bar.txt &&\n+\t\ttest_path_is_missing Foo.txt\n+\t)\n+'\n+\n+test_expect_success 'git push can propagate local page deletion' '\n+\twiki_reset &&\n+\tgit clone mediawiki::'\"$WIKI_URL\"' mw_dir &&\n+\ttest_when_finished \"rm -fr mw_dir\" &&\n+\t(\n+\t\tcd mw_dir &&\n+\t\ttest_path_is_missing Foo.mw &&\n+\t\techo \"hello world\" >Foo.mw &&\n+\t\tgit add Foo.mw &&\n+\t\tgit commit -m \"Add the page Foo\" &&\n+\t\tgit push &&\n+\t\trm -f Foo.mw &&\n+\t\tgit commit -am \"Delete the page Foo\" &&\n+\t\ttest_git_reimport &&\n+\t\ttest_path_is_missing Foo.mw\n+\t)\n+'\n+\n+test_expect_success 'git push can propagate local media file deletion' '\n+\twiki_reset &&\n+\tgit clone mediawiki::'\"$WIKI_URL\"' mw_dir &&\n+\ttest_when_finished \"rm -fr mw_dir\" &&\n+\t(\n+\t\tcd mw_dir &&\n+\t\techo \"hello world\" >Foo.txt &&\n+\t\tgit add Foo.txt &&\n+\t\tgit commit -m \"Add the text file Foo\" &&\n+\t\tgit rm Foo.txt &&\n+\t\tgit commit -m \"Delete the file Foo\" &&\n+\t\ttest_git_reimport &&\n+\t\ttest_path_is_missing Foo.txt\n+\t)\n+'\n+\n+# test failure: the file is correctly uploaded, and then deleted but\n+# as no page link to it, the import (which looks at page revisions)\n+# doesn't notice the file deletion on the wiki. We fetch the list of\n+# files from the wiki, but as the file is deleted, it doesn't appear.\n+test_expect_failure 'git pull correctly imports media file deletion when no page link to it' '\n+\twiki_reset &&\n+\tgit clone mediawiki::'\"$WIKI_URL\"' mw_dir &&\n+\ttest_when_finished \"rm -fr mw_dir\" &&\n+\t(\n+\t\tcd mw_dir &&\n+\t\techo \"hello world\" >Foo.txt &&\n+\t\tgit add Foo.txt &&\n+\t\tgit commit -m \"Add the text file Foo\" &&\n+\t\tgit push &&\n+\t\tgit rm Foo.txt &&\n+\t\tgit commit -m \"Delete the file Foo\" &&\n+\t\ttest_git_reimport &&\n+\t\ttest_path_is_missing Foo.txt\n+\t)\n+'\n+\n+test_expect_success 'git push properly warns about insufficient permissions' '\n+\twiki_reset &&\n+\tgit clone mediawiki::'\"$WIKI_URL\"' mw_dir &&\n+\ttest_when_finished \"rm -fr mw_dir\" &&\n+\t(\n+\t\tcd mw_dir &&\n+\t\techo \"A File\" >foo.forbidden &&\n+\t\tgit add foo.forbidden &&\n+\t\tgit commit -m \"add a file\" &&\n+\t\tgit push 2>actual &&\n+\t\ttest_i18ngrep \"foo.forbidden is not a permitted file\" actual\n+\t)\n+'\n+\n+test_expect_success 'setup a repository with media files' '\n+\twiki_reset &&\n+\twiki_editpage testpage \"I am linking a file [[File:File.txt]]\" false &&\n+\techo \"File content\" >File.txt &&\n+\twiki_upload_file File.txt &&\n+\techo \"Another file content\" >AnotherFile.txt &&\n+\twiki_upload_file AnotherFile.txt\n+'\n+\n+test_expect_success 'git clone works with one specific page cloned and mediaimport=true' '\n+\tgit clone -c remote.origin.pages=testpage \\\n+\t\t -c remote.origin.mediaimport=true \\\n+\t\t\tmediawiki::'\"$WIKI_URL\"' mw_dir_15 &&\n+\ttest_when_finished \"rm -rf mw_dir_15\" &&\n+\ttest_contains_N_files mw_dir_15 3 &&\n+\ttest_path_is_file mw_dir_15/Testpage.mw &&\n+\ttest_path_is_file mw_dir_15/File:File.txt.mw &&\n+\ttest_path_is_file mw_dir_15/File.txt &&\n+\ttest_path_is_missing mw_dir_15/Main_Page.mw &&\n+\ttest_path_is_missing mw_dir_15/File:AnotherFile.txt.mw &&\n+\ttest_path_is_missing mw_dir_15/AnothetFile.txt &&\n+\twiki_check_content mw_dir_15/Testpage.mw Testpage &&\n+\ttest_cmp mw_dir_15/File.txt File.txt\n+'\n+\n+test_expect_success 'git clone works with one specific page cloned and mediaimport=false' '\n+\ttest_when_finished \"rm -rf mw_dir_16\" &&\n+\tgit clone -c remote.origin.pages=testpage \\\n+\t\t\tmediawiki::'\"$WIKI_URL\"' mw_dir_16 &&\n+\ttest_contains_N_files mw_dir_16 1 &&\n+\ttest_path_is_file mw_dir_16/Testpage.mw &&\n+\ttest_path_is_missing mw_dir_16/File:File.txt.mw &&\n+\ttest_path_is_missing mw_dir_16/File.txt &&\n+\ttest_path_is_missing mw_dir_16/Main_Page.mw &&\n+\twiki_check_content mw_dir_16/Testpage.mw Testpage\n+'\n+\n+# should behave like mediaimport=false\n+test_expect_success 'git clone works with one specific page cloned and mediaimport unset' '\n+\ttest_when_finished \"rm -fr mw_dir_17\" &&\n+\tgit clone -c remote.origin.pages=testpage \\\n+\t\tmediawiki::'\"$WIKI_URL\"' mw_dir_17 &&\n+\ttest_contains_N_files mw_dir_17 1 &&\n+\ttest_path_is_file mw_dir_17/Testpage.mw &&\n+\ttest_path_is_missing mw_dir_17/File:File.txt.mw &&\n+\ttest_path_is_missing mw_dir_17/File.txt &&\n+\ttest_path_is_missing mw_dir_17/Main_Page.mw &&\n+\twiki_check_content mw_dir_17/Testpage.mw Testpage\n+'\n+\n+test_done\n-- \n1.7.11.1.147.g47a574d\n"},{"id":"194709","messageId":"1341568995-12467-9-git-send-email-Matthieu.Moy@imag.fr","threadId":"30954","inReplyTo":"1341568995-12467-1-git-send-email-Matthieu.Moy@imag.fr","subject":"[PATCH 08/12] git-remote-mediawiki: change return type of get_mw_pages","fromName":"Matthieu Moy","fromEmail":"matthieu.moy@imag.fr","sentAt":"2012-07-06T10:03:11Z","receivedAt":"2012-07-06T10:03:11Z","isPatch":true,"sender":{"key":"git@matthieu-moy.fr","avatar":"https://avatars.githubusercontent.com/u/14709?v=4"},"body":"The previous version was returning the list of pages to be fetched, but\nwe are going to need an efficient membership test (i.e. is the page\n$title tracked), hence exposing a hash will be more convenient.\n\nSigned-off-by: Matthieu Moy \n---\n contrib/mw-to-git/git-remote-mediawiki | 8 +++++---\n 1 file changed, 5 insertions(+), 3 deletions(-)\n\ndiff --git a/contrib/mw-to-git/git-remote-mediawiki b/contrib/mw-to-git/git-remote-mediawiki\nindex 063a978..4bf990b 100755\n--- a/contrib/mw-to-git/git-remote-mediawiki\n+++ b/contrib/mw-to-git/git-remote-mediawiki\n@@ -379,7 +379,7 @@ sub get_mw_pages {\n \t\t\tget_all_mediafiles(\\%pages);\n \t\t}\n \t}\n-\treturn values(%pages);\n+\treturn %pages;\n }\n \n # usage: $out = run_git(\"command args\");\n@@ -536,7 +536,8 @@ my %basetimestamps;\n sub get_last_remote_revision {\n \tmw_connect_maybe();\n \n-\tmy @pages = get_mw_pages();\n+\tmy %pages_hash = get_mw_pages();\n+\tmy @pages = values(%pages_hash);\n \n \tmy $max_rev_num = 0;\n \n@@ -802,7 +803,8 @@ sub mw_import_ref {\n \n \tmw_connect_maybe();\n \n-\tmy @pages = get_mw_pages();\n+\tmy %pages_hash = get_mw_pages();\n+\tmy @pages = values(%pages_hash);\n \n \tprint STDERR \"Searching revisions...\\n\";\n \tmy $last_local = get_last_local_revision();\n-- \n1.7.11.1.147.g47a574d\n"},{"id":"194710","messageId":"1341568995-12467-10-git-send-email-Matthieu.Moy@imag.fr","threadId":"30954","inReplyTo":"1341568995-12467-1-git-send-email-Matthieu.Moy@imag.fr","subject":"[PATCH 09/12] git-remote-mediawiki: refactor loop over revision ids","fromName":"Matthieu Moy","fromEmail":"matthieu.moy@imag.fr","sentAt":"2012-07-06T10:03:12Z","receivedAt":"2012-07-06T10:03:12Z","isPatch":true,"sender":{"key":"git@matthieu-moy.fr","avatar":"https://avatars.githubusercontent.com/u/14709?v=4"},"body":"Without changing the behavior, we turn the foreach loop on an array of\nrevisions into a loop on an array of integer. It will be easier to\nimplement other strategies as they will only need to produce an array of\ninteger instead of a more complex data-structure.\n\nSigned-off-by: Matthieu Moy \n---\n contrib/mw-to-git/git-remote-mediawiki | 17 +++++++++++------\n 1 file changed, 11 insertions(+), 6 deletions(-)\n\ndiff --git a/contrib/mw-to-git/git-remote-mediawiki b/contrib/mw-to-git/git-remote-mediawiki\nindex 4bf990b..47dd574 100755\n--- a/contrib/mw-to-git/git-remote-mediawiki\n+++ b/contrib/mw-to-git/git-remote-mediawiki\n@@ -819,30 +819,35 @@ sub mw_import_ref {\n \t# Creation of the fast-import stream\n \tprint STDERR \"Fetching & writing export data...\\n\";\n \n+\t@revisions = sort {$a->{revid} <=> $b->{revid}} @revisions;\n+\tmy @revision_ids = map $_->{revid}, @revisions;\n+\n \t$n = 0;\n \tmy $last_timestamp = 0; # Placeholer in case $rev->timestamp is undefined\n \n-\tforeach my $pagerevid (sort {$a->{revid} <=> $b->{revid}} @revisions) {\n+\tforeach my $pagerevid (@revision_ids) {\n \t\t# fetch the content of the pages\n \t\tmy $query = {\n \t\t\taction => 'query',\n \t\t\tprop => 'revisions',\n \t\t\trvprop => 'content|timestamp|comment|user|ids',\n-\t\t\trevids => $pagerevid->{revid},\n+\t\t\trevids => $pagerevid,\n \t\t};\n \n \t\tmy $result = $mediawiki->api($query);\n \n-\t\tmy $rev = pop(@{$result->{query}->{pages}->{$pagerevid->{pageid}}->{revisions}});\n+\t\tmy @result_pages = values(%{$result->{query}->{pages}});\n+\t\tmy $result_page = $result_pages[0];\n+\t\tmy $rev = $result_pages[0]->{revisions}->[0];\n \n \t\t$n++;\n \n-\t\tmy $page_title = $result->{query}->{pages}->{$pagerevid->{pageid}}->{title};\n+\t\tmy $page_title = $result_page->{title};\n \t\tmy %commit;\n \t\t$commit{author} = $rev->{user} || 'Anonymous';\n \t\t$commit{comment} = $rev->{comment} || '*Empty MediaWiki Message*';\n \t\t$commit{title} = mediawiki_smudge_filename($page_title);\n-\t\t$commit{mw_revision} = $pagerevid->{revid};\n+\t\t$commit{mw_revision} = $rev->{revid};\n \t\t$commit{content} = mediawiki_smudge($rev->{'*'});\n \n \t\tif (!defined($rev->{timestamp})) {\n@@ -861,7 +866,7 @@ sub mw_import_ref {\n \t\t# If this is a revision of the media page for new version\n \t\t# of a file do one common commit for both file and media page.\n \t\t# Else do commit only for that page.\n-\t\tprint STDERR \"$n/\", scalar(@revisions), \": Revision #$pagerevid->{revid} of $commit{title}\\n\";\n+\t\tprint STDERR \"$n/\", scalar(@revision_ids), \": Revision #$rev->{revid} of $commit{title}\\n\";\n \t\timport_file_revision(\\%commit, ($fetch_from == 1), $n, \\%mediafile);\n \t}\n \n-- \n1.7.11.1.147.g47a574d\n"},{"id":"194712","messageId":"1341568995-12467-11-git-send-email-Matthieu.Moy@imag.fr","threadId":"30954","inReplyTo":"1341568995-12467-1-git-send-email-Matthieu.Moy@imag.fr","subject":"[PATCH 10/12] git-remote-mediawiki: extract revision-importing loop to a function","fromName":"Matthieu Moy","fromEmail":"matthieu.moy@imag.fr","sentAt":"2012-07-06T10:03:13Z","receivedAt":"2012-07-06T10:03:13Z","isPatch":true,"sender":{"key":"git@matthieu-moy.fr","avatar":"https://avatars.githubusercontent.com/u/14709?v=4"},"body":"\nSigned-off-by: Matthieu Moy \n---\n contrib/mw-to-git/git-remote-mediawiki | 16 +++++++++++++---\n 1 file changed, 13 insertions(+), 3 deletions(-)\n\ndiff --git a/contrib/mw-to-git/git-remote-mediawiki b/contrib/mw-to-git/git-remote-mediawiki\nindex 47dd574..9305204 100755\n--- a/contrib/mw-to-git/git-remote-mediawiki\n+++ b/contrib/mw-to-git/git-remote-mediawiki\n@@ -822,10 +822,18 @@ sub mw_import_ref {\n \t@revisions = sort {$a->{revid} <=> $b->{revid}} @revisions;\n \tmy @revision_ids = map $_->{revid}, @revisions;\n \n-\t$n = 0;\n+\treturn mw_import_revids($fetch_from, \\@revision_ids, \\%pages_hash);\n+}\n+\n+sub mw_import_revids {\n+\tmy $fetch_from = shift;\n+\tmy $revision_ids = shift;\n+\tmy $pages = shift;\n+\n+\tmy $n = 0;\n \tmy $last_timestamp = 0; # Placeholer in case $rev->timestamp is undefined\n \n-\tforeach my $pagerevid (@revision_ids) {\n+\tforeach my $pagerevid (@$revision_ids) {\n \t\t# fetch the content of the pages\n \t\tmy $query = {\n \t\t\taction => 'query',\n@@ -866,7 +874,7 @@ sub mw_import_ref {\n \t\t# If this is a revision of the media page for new version\n \t\t# of a file do one common commit for both file and media page.\n \t\t# Else do commit only for that page.\n-\t\tprint STDERR \"$n/\", scalar(@revision_ids), \": Revision #$rev->{revid} of $commit{title}\\n\";\n+\t\tprint STDERR \"$n/\", scalar(@$revision_ids), \": Revision #$rev->{revid} of $commit{title}\\n\";\n \t\timport_file_revision(\\%commit, ($fetch_from == 1), $n, \\%mediafile);\n \t}\n \n@@ -876,6 +884,8 @@ sub mw_import_ref {\n \t\t# thrown saying that HEAD is refering to unknown object 0000000000000000000\n \t\t# and the clone fails.\n \t}\n+\n+\treturn $n;\n }\n \n sub error_non_fast_forward {\n-- \n1.7.11.1.147.g47a574d\n"},{"id":"194714","messageId":"1341568995-12467-12-git-send-email-Matthieu.Moy@imag.fr","threadId":"30954","inReplyTo":"1341568995-12467-1-git-send-email-Matthieu.Moy@imag.fr","subject":"[PATCH 11/12] git-remote-mediawiki: more efficient 'pull' in the best case","fromName":"Matthieu Moy","fromEmail":"matthieu.moy@imag.fr","sentAt":"2012-07-06T10:03:14Z","receivedAt":"2012-07-06T10:03:14Z","isPatch":true,"sender":{"key":"git@matthieu-moy.fr","avatar":"https://avatars.githubusercontent.com/u/14709?v=4"},"body":"The only way to fetch new revisions from a wiki before this patch was to\nquery each page for new revisions. This is good when tracking a small set\nof pages on a large wiki, but very inefficient when tracking many pages\non a wiki with little activity.\n\nImplement a new strategy that queries the wiki for its last global\nrevision, queries each new revision, and filter out pages that are not\ntracked.\n\nSigned-off-by: Simon Perrat \nSigned-off-by: Simon CATHEBRAS \nSigned-off-by: Julien KHAYAT \nSigned-off-by: Charles ROUSSEL \nSigned-off-by: Guillaume SASDY \nSigned-off-by: Matthieu Moy \n---\n contrib/mw-to-git/git-remote-mediawiki | 114 +++++++++++++++++++++++++------\n contrib/mw-to-git/t/t9364-pull-by-rev.sh | 17 +++++\n 2 files changed, 110 insertions(+), 21 deletions(-)\n create mode 100755 contrib/mw-to-git/t/t9364-pull-by-rev.sh\n\ndiff --git a/contrib/mw-to-git/git-remote-mediawiki b/contrib/mw-to-git/git-remote-mediawiki\nindex 9305204..6bf1d1a 100755\n--- a/contrib/mw-to-git/git-remote-mediawiki\n+++ b/contrib/mw-to-git/git-remote-mediawiki\n@@ -13,12 +13,9 @@\n #\n # Known limitations:\n #\n-# - Poor performance in the best case: it takes forever to check\n-# whether we're up-to-date (on fetch or push) or to fetch a few\n-# revisions from a large wiki, because we use exclusively a\n-# page-based synchronization. We could switch to a wiki-wide\n-# synchronization when the synchronization involves few revisions\n-# but the wiki is large.\n+# - Several strategies are provided to fetch modifications from the\n+# wiki, but no automatic heuristics is provided, the user has\n+# to understand and chose which strategy is appropriate for him.\n #\n # - Git renames could be turned into MediaWiki renames (see TODO\n # below)\n@@ -89,6 +86,21 @@ my $shallow_import = run_git(\"config --get --bool remote.\". $remotename .\".shall\n chomp($shallow_import);\n $shallow_import = ($shallow_import eq \"true\");\n \n+# Fetch (clone and pull) by revisions instead of by pages. This behavior\n+# is more efficient when we have a wiki with lots of pages and we fetch\n+# the revisions quite often so that they concern only few pages.\n+# Possible values:\n+# - by_rev: perform one query per new revision on the remote wiki\n+# - by_page: query each tracked page for new revision\n+my $fetch_strategy = run_git(\"config --get remote.$remotename.fetchStrategy\");\n+unless ($fetch_strategy) {\n+\t$fetch_strategy = run_git(\"config --get mediawiki.fetchStrategy\");\n+}\n+chomp($fetch_strategy);\n+unless ($fetch_strategy) {\n+\t$fetch_strategy = \"by_page\";\n+}\n+\n # Dumb push: don't update notes and mediawiki ref to reflect the last push.\n #\n # Configurable with mediawiki.dumbPush, or per-remote with\n@@ -533,6 +545,26 @@ sub get_last_local_revision {\n # Remember the timestamp corresponding to a revision id.\n my %basetimestamps;\n \n+# Get the last remote revision without taking in account which pages are\n+# tracked or not. This function makes a single request to the wiki thus\n+# avoid a loop onto all tracked pages. This is useful for the fetch-by-rev\n+# option.\n+sub get_last_global_remote_rev {\n+\tmw_connect_maybe();\n+\n+\tmy $query = {\n+\t\taction => 'query',\n+\t\tlist => 'recentchanges',\n+\t\tprop => 'revisions',\n+\t\trclimit => '1',\n+\t\trcdir => 'older',\n+\t};\n+\tmy $result = $mediawiki->api($query);\n+\treturn $result->{query}->{recentchanges}[0]->{revid};\n+}\n+\n+# Get the last remote revision concerning the tracked pages and the tracked\n+# categories.\n sub get_last_remote_revision {\n \tmw_connect_maybe();\n \n@@ -803,9 +835,6 @@ sub mw_import_ref {\n \n \tmw_connect_maybe();\n \n-\tmy %pages_hash = get_mw_pages();\n-\tmy @pages = values(%pages_hash);\n-\n \tprint STDERR \"Searching revisions...\\n\";\n \tmy $last_local = get_last_local_revision();\n \tmy $fetch_from = $last_local + 1;\n@@ -814,10 +843,35 @@ sub mw_import_ref {\n \t} else {\n \t\tprint STDERR \", fetching from here.\\n\";\n \t}\n-\tmy ($n, @revisions) = fetch_mw_revisions(\\@pages, $fetch_from);\n \n-\t# Creation of the fast-import stream\n-\tprint STDERR \"Fetching & writing export data...\\n\";\n+\tmy $n = 0;\n+\tif ($fetch_strategy eq \"by_rev\") {\n+\t\tprint STDERR \"Fetching & writing export data by revs...\\n\";\n+\t\t$n = mw_import_ref_by_revs($fetch_from);\n+\t} elsif ($fetch_strategy eq \"by_page\") {\n+\t\tprint STDERR \"Fetching & writing export data by pages...\\n\";\n+\t\t$n = mw_import_ref_by_pages($fetch_from);\n+\t} else {\n+\t\tprint STDERR \"fatal: invalid fetch strategy \\\"$fetch_strategy\\\".\\n\";\n+\t\tprint STDERR \"Check your configuration variables remote.$remotename.fetchStrategy and mediawiki.fetchStrategy\\n\";\n+\t\texit 1;\n+\t}\n+\n+\tif ($fetch_from == 1 && $n == 0) {\n+\t\tprint STDERR \"You appear to have cloned an empty MediaWiki.\\n\";\n+\t\t# Something has to be done remote-helper side. If nothing is done, an error is\n+\t\t# thrown saying that HEAD is refering to unknown object 0000000000000000000\n+\t\t# and the clone fails.\n+\t}\n+}\n+\n+sub mw_import_ref_by_pages {\n+\n+\tmy $fetch_from = shift;\n+\tmy %pages_hash = get_mw_pages();\n+\tmy @pages = values(%pages_hash);\n+\n+\tmy ($n, @revisions) = fetch_mw_revisions(\\@pages, $fetch_from);\n \n \t@revisions = sort {$a->{revid} <=> $b->{revid}} @revisions;\n \tmy @revision_ids = map $_->{revid}, @revisions;\n@@ -825,12 +879,26 @@ sub mw_import_ref {\n \treturn mw_import_revids($fetch_from, \\@revision_ids, \\%pages_hash);\n }\n \n+sub mw_import_ref_by_revs {\n+\n+\tmy $fetch_from = shift;\n+\tmy %pages_hash = get_mw_pages();\n+\n+\tmy $last_remote = get_last_global_remote_rev();\n+\tmy @revision_ids = $fetch_from..$last_remote;\n+\treturn mw_import_revids($fetch_from, \\@revision_ids, \\%pages_hash);\n+}\n+\n+# Import revisions given in second argument (array of integers).\n+# Only pages appearing in the third argument (hash indexed by page titles)\n+# will be imported.\n sub mw_import_revids {\n \tmy $fetch_from = shift;\n \tmy $revision_ids = shift;\n \tmy $pages = shift;\n \n \tmy $n = 0;\n+\tmy $n_actual = 0;\n \tmy $last_timestamp = 0; # Placeholer in case $rev->timestamp is undefined\n \n \tforeach my $pagerevid (@$revision_ids) {\n@@ -848,9 +916,20 @@ sub mw_import_revids {\n \t\tmy $result_page = $result_pages[0];\n \t\tmy $rev = $result_pages[0]->{revisions}->[0];\n \n+\t # Count page even if we skip it, since we display\n+\t\t# $n/$total and $total includes skipped pages.\n \t\t$n++;\n \n \t\tmy $page_title = $result_page->{title};\n+\n+\t\tif (!exists($pages->{$page_title})) {\n+\t\t\tprint STDERR \"$n/\", scalar(@$revision_ids),\n+\t\t\t\t\": Skipping revision #$rev->{revid} of $page_title\\n\";\n+\t\t\tnext;\n+\t\t}\n+\n+\t\t$n_actual++;\n+\n \t\tmy %commit;\n \t\t$commit{author} = $rev->{user} || 'Anonymous';\n \t\t$commit{comment} = $rev->{comment} || '*Empty MediaWiki Message*';\n@@ -875,17 +954,10 @@ sub mw_import_revids {\n \t\t# of a file do one common commit for both file and media page.\n \t\t# Else do commit only for that page.\n \t\tprint STDERR \"$n/\", scalar(@$revision_ids), \": Revision #$rev->{revid} of $commit{title}\\n\";\n-\t\timport_file_revision(\\%commit, ($fetch_from == 1), $n, \\%mediafile);\n-\t}\n-\n-\tif ($fetch_from == 1 && $n == 0) {\n-\t\tprint STDERR \"You appear to have cloned an empty MediaWiki.\\n\";\n-\t\t# Something has to be done remote-helper side. If nothing is done, an error is\n-\t\t# thrown saying that HEAD is refering to unknown object 0000000000000000000\n-\t\t# and the clone fails.\n+\t\timport_file_revision(\\%commit, ($fetch_from == 1), $n_actual, \\%mediafile);\n \t}\n \n-\treturn $n;\n+\treturn $n_actual;\n }\n \n sub error_non_fast_forward {\ndiff --git a/contrib/mw-to-git/t/t9364-pull-by-rev.sh b/contrib/mw-to-git/t/t9364-pull-by-rev.sh\nnew file mode 100755\nindex 0000000..5c22457\n--- /dev/null\n+++ b/contrib/mw-to-git/t/t9364-pull-by-rev.sh\n@@ -0,0 +1,17 @@\n+#!/bin/sh\n+\n+test_description='Test the Git Mediawiki remote helper: git pull by revision'\n+\n+. ./test-gitmw-lib.sh\n+. ./push-pull-tests.sh\n+. $TEST_DIRECTORY/test-lib.sh\n+\n+test_check_precond\n+\n+test_expect_success 'configuration' '\n+\tgit config --global mediawiki.fetchStrategy by_rev\n+'\n+\n+test_push_pull\n+\n+test_done\n-- \n1.7.11.1.147.g47a574d\n"},{"id":"194713","messageId":"1341568995-12467-13-git-send-email-Matthieu.Moy@imag.fr","threadId":"30954","inReplyTo":"1341568995-12467-1-git-send-email-Matthieu.Moy@imag.fr","subject":"[PATCH 12/12] git-remote-mediawiki: be more defensive when requests fail","fromName":"Matthieu Moy","fromEmail":"matthieu.moy@imag.fr","sentAt":"2012-07-06T10:03:15Z","receivedAt":"2012-07-06T10:03:15Z","isPatch":true,"sender":{"key":"git@matthieu-moy.fr","avatar":"https://avatars.githubusercontent.com/u/14709?v=4"},"body":"\nSigned-off-by: Matthieu Moy \n---\n contrib/mw-to-git/git-remote-mediawiki | 8 ++++++++\n 1 file changed, 8 insertions(+)\n\ndiff --git a/contrib/mw-to-git/git-remote-mediawiki b/contrib/mw-to-git/git-remote-mediawiki\nindex 6bf1d1a..ff9384e 100755\n--- a/contrib/mw-to-git/git-remote-mediawiki\n+++ b/contrib/mw-to-git/git-remote-mediawiki\n@@ -912,6 +912,14 @@ sub mw_import_revids {\n \n \t\tmy $result = $mediawiki->api($query);\n \n+\t\tif (!$result) {\n+\t\t\tdie \"Failed to retrieve modified page for revision $pagerevid\";\n+\t\t}\n+\n+\t\tif (!defined($result->{query}->{pages})) {\n+\t\t\tdie \"Invalid revision $pagerevid.\";\n+\t\t}\n+\n \t\tmy @result_pages = values(%{$result->{query}->{pages}});\n \t\tmy $result_page = $result_pages[0];\n \t\tmy $rev = $result_pages[0]->{revisions}->[0];\n-- \n1.7.11.1.147.g47a574d\n"}]}