ok 3 - plain bare
Since the tests all output TAP (see http://testanything.org) they can
-be run with any TAP harness. Here's an example of paralell testing
+be run with any TAP harness. Here's an example of parallel testing
powered by a recent version of prove(1):
$ prove --timer --jobs 15 ./t[0-9]*.sh
consistently when command line arguments --verbose (or -v),
--debug (or -d), and --immediate (or -i) is given.
+Do's, don'ts & things to keep in mind
+-------------------------------------
+
+Here are a few examples of things you probably should and shouldn't do
+when writing tests.
+
+Do:
+
+ - Put all code inside test_expect_success and other assertions.
+
+ Even code that isn't a test per se, but merely some setup code
+ should be inside a test assertion.
+
+ - Chain your test assertions
+
+ Write test code like this:
+
+ git merge foo &&
+ git push bar &&
+ test ...
+
+ Instead of:
+
+ git merge hla
+ git push gh
+ test ...
+
+ That way all of the commands in your tests will succeed or fail. If
+ you must ignore the return value of something (e.g., the return
+ after unsetting a variable that was already unset is unportable) it's
+ best to indicate so explicitly with a semicolon:
+
+ unset HLAGH;
+ git merge hla &&
+ git push gh &&
+ test ...
+
+Don't:
+
+ - exit() within a <script> part.
+
+ The harness will catch this as a programming error of the test.
+ Use test_done instead if you need to stop the tests early (see
+ "Skipping tests" below).
+
+ - Break the TAP output
+
+ The raw output from your test may be interpreted by a TAP harness. TAP
+ harnesses will ignore everything they don't know about, but don't step
+ on their toes in these areas:
+
+ - Don't print lines like "$x..$y" where $x and $y are integers.
+
+ - Don't print lines that begin with "ok" or "not ok".
+
+ TAP harnesses expect a line that begins with either "ok" and "not
+ ok" to signal a test passed or failed (and our harness already
+ produces such lines), so your script shouldn't emit such lines to
+ their output.
+
+ You can glean some further possible issues from the TAP grammar
+ (see http://search.cpan.org/perldoc?TAP::Parser::Grammar#TAP_Grammar)
+ but the best indication is to just run the tests with prove(1),
+ it'll complain if anything is amiss.
+
+Keep in mind:
+
+ - Inside <script> part, the standard output and standard error
+ streams are discarded, and the test harness only reports "ok" or
+ "not ok" to the end user running the tests. Under --verbose, they
+ are shown to help debugging the tests.
+
+
+Skipping tests
+--------------
+
+If you need to skip all the remaining tests you should set skip_all
+and immediately call test_done. The string you give to skip_all will
+be used as an explanation for why the test was skipped. for instance:
+
+ if ! test_have_prereq PERL
+ then
+ skip_all='skipping perl interface tests, perl not available'
+ test_done
+ fi
End with test_done
------------------
There are a handful helper functions defined in the test harness
library for your script to use.
- - test_expect_success <message> <script>
+ - test_expect_success [<prereq>] <message> <script>
- This takes two strings as parameter, and evaluates the
+ Usually takes two strings as parameter, and evaluates the
<script>. If it yields success, test is considered
successful. <message> should state what it is testing.
'git-write-tree should be able to write an empty tree.' \
'tree=$(git-write-tree)'
- - test_expect_failure <message> <script>
+ If you supply three parameters the first will be taken to be a
+ prerequisite, see the test_set_prereq and test_have_prereq
+ documentation below:
+
+ test_expect_success TTY 'git --paginate rev-list uses a pager' \
+ ' ... '
+
+ - test_expect_failure [<prereq>] <message> <script>
This is NOT the opposite of test_expect_success, but is used
to mark a test that demonstrates a known breakage. Unlike
success and "still broken" on failure. Failures from these
tests won't cause -i (immediate) to stop.
+ Like test_expect_success this function can optionally use a three
+ argument invocation with a prerequisite as the first argument.
+
+ - test_expect_code [<prereq>] <code> <message> <script>
+
+ Analogous to test_expect_success, but pass the test if it exits
+ with a given exit <code>
+
+ test_expect_code 1 'Merge with d/f conflicts' 'git merge "merge msg" B master'
+
- test_debug <script>
This takes a single argument, <script>, and evaluates it only
Merges the given rev using the given message. Like test_commit,
creates a tag and calls test_tick before committing.
+ - test_set_prereq SOME_PREREQ
+
+ Set a test prerequisite to be used later with test_have_prereq. The
+ test-lib will set some prerequisites for you, e.g. PERL and PYTHON
+ which are derived from ./GIT-BUILD-OPTIONS (grep test_set_prereq
+ test-lib.sh for more). Others you can set yourself and use later
+ with either test_have_prereq directly, or the three argument
+ invocation of test_expect_success and test_expect_failure.
+
+ - test_have_prereq SOME PREREQ
+
+ Check if we have a prerequisite previously set with
+ test_set_prereq. The most common use of this directly is to skip
+ all the tests if we don't have some essential prerequisite:
+
+ if ! test_have_prereq PERL
+ then
+ skip_all='skipping perl interface tests, perl not available'
+ test_done
+ fi
+
+ - test_external [<prereq>] <message> <external> <script>
+
+ Execute a <script> with an <external> interpreter (like perl). This
+ was added for tests like t9700-perl-git.sh which do most of their
+ work in an external test script.
+
+ test_external \
+ 'GitwebCache::*FileCache*' \
+ "$PERL_PATH" "$TEST_DIRECTORY"/t9503/test_cache_interface.pl
+
+ If the test is outputting its own TAP you should set the
+ test_external_has_tap variable somewhere before calling the first
+ test_external* function. See t9700-perl-git.sh for an example.
+
+ # The external test will outputs its own plan
+ test_external_has_tap=1
+
+ - test_external_without_stderr [<prereq>] <message> <external> <script>
+
+ Like test_external but fail if there's any output on stderr,
+ instead of checking the exit code.
+
+ test_external_without_stderr \
+ 'Perl API' \
+ "$PERL_PATH" "$TEST_DIRECTORY"/t9700/test.pl
+
+ - test_must_fail <git-command>
+
+ Run a git command and ensure it fails in a controlled way. Use
+ this instead of "! <git-command>". When git-command dies due to a
+ segfault, test_must_fail diagnoses it as an error; "! <git-command>"
+ treats it as just another expected failure, which would let such a
+ bug go unnoticed.
+
+ - test_might_fail <git-command>
+
+ Similar to test_must_fail, but tolerate success, too. Use this
+ instead of "<git-command> || :" to catch failures due to segv.
+
+ - test_cmp <expected> <actual>
+
+ Check whether the content of the <actual> file matches the
+ <expected> file. This behaves like "cmp" but produces more
+ helpful output when the test is run with "-v" option.
+
+ - test_when_finished <script>
+
+ Prepend <script> to a list of commands to run to clean up
+ at the end of the current test. If some clean-up command
+ fails, the test will not pass.
+
+ Example:
+
+ test_expect_success 'branch pointing to non-commit' '
+ git rev-parse HEAD^{tree} >.git/refs/heads/invalid &&
+ test_when_finished "git update-ref -d refs/heads/invalid" &&
+ ...
+ '
+
+
Tips for Writing Tests
----------------------
projects / git.git / blobdiff