{"thread":{"id":"615","subject":"[PATCH 2/2] Test framework documentation.","startedAt":"2005-05-14T07:25:06Z","lastAt":"2005-05-14T07:25:06Z","messageCount":1,"participants":["Junio C Hamano"],"isPatch":true,"patchVersion":1,"patchTotal":2},"messages":[{"id":"3287","messageId":"7vwtq246ct.fsf@assigned-by-dhcp.cox.net","threadId":"615","inReplyTo":null,"subject":"[PATCH 2/2] Test framework documentation.","fromName":"Junio C Hamano","fromEmail":"junkio@cox.net","sentAt":"2005-05-14T07:25:06Z","receivedAt":"2005-05-14T07:25:06Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"This adds instruction for running tests, and writing new tests.\n\nSigned-off-by: Junio C Hamano <junkio@cox.net>\n---\n\nt/README |  174 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++\n1 files changed, 174 insertions(+)\nt/README (. --> 100644)\n\n--- a/t/README\n+++ b/t/README\n@@ -0,0 +1,174 @@\n+Core GIT Tests\n+==============\n+\n+This directory holds many test scripts for core GIT tools.  The\n+first part of this short document describes how to run the tests\n+and read their output.\n+\n+When fixing the tools or adding enhancements, you are strongly\n+encouraged to add tests in this directory to cover what you are\n+trying to fix or enhance.  The later part of this short document\n+describes how your test scripts should be organized.\n+\n+\n+Running Tests\n+-------------\n+\n+The easiest way to run tests is to say \"make\".  This runs all\n+the tests.\n+\n+    *** t0000-basic.sh ***\n+    * ok #1: .git/objects should be empty after git-init-db in an empty repo.\n+    * ok #2: .git/objects should have 256 subdirectories.\n+    * ok #3: git-update-cache without --add should fail adding.\n+    ...\n+    * ok #23: no diff after checkout and git-update-cache --refresh.\n+    * passed all 23 test(s)\n+    *** t0100-environment-names.sh ***\n+    * ok #1: using old names should issue warnings.\n+    * ok #2: using old names but having new names should not issue warnings.\n+    ...\n+\n+Or you can run each test individually from command line, like\n+this:\n+\n+    $ sh ./t0500-ls-files.sh\n+    * ok #1: git-update-cache --add to add various paths.\n+    * ok #2: git-ls-files -k to show killed files.\n+    * ok #3: validate git-ls-files -k output.\n+    * passed all 3 test(s)\n+\n+You can pass --verbose (or -v), --debug (or -d), and --immediate\n+(or -i) command line argument to the test.\n+\n+--verbose::\n+\tThis makes the test more verbose.  Specifically, the\n+\tcommand being run and their output if any are also\n+\toutput.\n+\n+--debug::\n+\tThis may help the person who is developing a new test.\n+\tIt causes the command defined with test_debug to run.\n+\n+--immediate::\n+\tThis causes the test to immediately exit upon the first\n+\tfailed test.\n+\n+\n+Writing Tests\n+-------------\n+\n+The test script is written as a shell script.  It should start\n+with the standard \"#!/bin/sh\" with copyright notices, and an\n+assignment to variable 'test_description', like this:\n+\n+\t#!/bin/sh\n+\t#\n+\t# Copyright (c) 2005 Junio C Hamano\n+\t#\n+\n+\ttest_description='xxx test (new option --frotz)\n+\n+\tThis test registers the following structure in the cache\n+\tand tries to run git-ls-files with new option --frotz.'\n+\n+Source 'test-lib.sh'\n+--------------------\n+\n+After assigning test_description, the test script should source\n+test-lib.sh like this:\n+\n+\t. ./test-lib.sh\n+\n+This test harness library does the following things:\n+\n+ - If the script is invoked with command line argument --help\n+   (or -h), it shows the test_description and exits.\n+\n+ - Creates an empty test directory with an empty .git/objects\n+   database and chdir(2) into it.  This directory is 't/trash'\n+   if you must know, but I do not think you care.\n+\n+ - Defines standard test helper functions for your scripts to\n+   use.  These functions are designed to make all scripts behave\n+   consistently when command line arguments --verbose (or -v),\n+   --debug (or -d), and --immediate (or -i) is given.\n+\n+\n+End with test_done\n+------------------\n+\n+Your script will be a sequence of tests, using helper functions\n+from the test harness library.  At the end of the script, call\n+'test_done'.\n+\n+\n+Test harness library\n+--------------------\n+\n+There are a handful helper functions defined in the test harness\n+library for your script to use.\n+\n+ - test_expect_success <message> <script>\n+\n+   This takes two strings as parameter, and evaluates the\n+   <script>.  If it yields success, test is considered\n+   successful.  <message> should state what it is testing.\n+\n+   Example:\n+\n+\ttest_expect_success \\\n+\t    'git-write-tree should be able to write an empty tree.' \\\n+\t    'tree=$(git-write-tree)'\n+\n+ - test_expect_failure <message> <script>\n+\n+   This is the opposite of test_expect_success.  If <script>\n+   yields success, test is considered a failure.\n+\n+   Example:\n+\n+\ttest_expect_failure \\\n+\t    'git-update-cache without --add should fail adding.' \\\n+\t    'git-update-cache should-be-empty'\n+\n+ - test_debug <script>\n+\n+   This takes a single argument, <script>, and evaluates it only\n+   when the test script is started with --debug command line\n+   argument.  This is primarily meant for use during the\n+   development of a new test script.\n+\n+ - test_done\n+\n+   Your test script must have test_done at the end.  Its purpose\n+   is to summarize successes and failures in the test script and\n+   exit with an appropriate error code.\n+\n+\n+Tips for Writing Tests\n+----------------------\n+\n+As with any programming projects, existing programs are the best\n+source of the information.  However, do _not_ emulate\n+t0000-basic.sh when writing your tests.  The test is special in\n+that it tries to validate the very core of GIT.  For example, it\n+knows that there will be 256 subdirectories under .git/objects/,\n+and it knows that the object ID of an empty tree is a certain\n+40-byte string.  This is deliberately done so in t0000-basic.sh\n+because the things the very basic core test tries to achieve is\n+to serve as a basis for people who are changing the GIT internal\n+drastically.  For these people, after making certain changes,\n+not seeing failures from the basic test _is_ a failure.  And\n+such drastic changes to the core GIT that even changes these\n+otherwise supposedly stable object IDs should be accompanied by\n+an update to t0000-basic.sh.\n+\n+However, other tests that simply rely on basic parts of the core\n+GIT working properly should not have that level of intimate\n+knowledge of the core GIT internals.  If all the test scripts\n+hardcoded the object IDs like t0000-basic.sh does, that defeats\n+the purpose of t0000-basic.sh, which is to isolate that level of\n+validation in one place.  Your test also ends up needing\n+updating when such a change to the internal happens, so do _not_\n+do it and leave the low level of validation to t0000-basic.sh.\n------------------------------------------------\n\n"}]}