Tests: Difference between revisions
Jump to navigation
Jump to search
→Shared functions: "between" is only for two
Carandraug (talk | contribs) (much expansion of text, added a lot more examples, no text left from previous) |
(→Shared functions: "between" is only for two) |
||
(5 intermediate revisions by 3 users not shown) | |||
Line 1: | Line 1: | ||
Having a thorough [ | Having a thorough [https://en.wikipedia.org/wiki/Test_suite test suite] is something very important which is usually overlooked. It is an incredible help in preventing regression bugs and quickly assess the status of old code. For example, many packages in Octave Forge become deprecated after losing their maintainer simply because they have no test suite. | ||
something very important which is usually overlooked. It is an incredible | |||
help in preventing regression bugs and quickly assess the status of old code. | |||
For example, many packages in Octave Forge become deprecated after losing | |||
their maintainer simply because they have no test suite. | |||
GNU Octave has multiple tools that help in creating a comprehensive test | GNU Octave has multiple tools that help in creating a comprehensive test suite, accessible to both developers and end-users, as detailed on the [http://www.gnu.org/software/octave/doc/interpreter/Test-Functions.html Octave manual]. Basically, test blocks are {{codeline|%!test}} comment blocks, typically at the end of a source file, which are ignored by the Octave interpreter and only read by the {{manual|test}} function. | ||
suite, accessible to both developers and end-users, as detailed on the | |||
[http://www.gnu.org/software/octave/doc/interpreter/Test-Functions.html Octave manual]. | |||
Basically, test blocks are {{codeline|%!test}} comment blocks, typically at the | |||
end of a source file, which are ignored by the Octave interpreter and only | |||
read by the {{ | |||
== Running tests == | == Running tests == | ||
To run all the tests of a specific function, simply use the {{ | To run all the tests of a specific function, simply use the {{manual|test}} command at the Octave prompt. For example, to run the tests of the Octave function {{manual|mean}} type: | ||
command at the Octave prompt. For example, to run the tests | |||
{{ | |||
>> test mean | |||
PASSES 17 out of 17 tests | PASSES 17 out of 17 tests | ||
These tests are written in the Octave language | These tests are written in the Octave language [http://hg.savannah.gnu.org/hgweb/octave/file/6443693a176f/scripts/statistics/base/mean.m#l130 at the bottom of <code>mean.m</code>] which defines the {{manual|mean}} function. It is important that these tests are also available for the end users so they can test the status of their installation. The whole Octave test suite can be run with: | ||
[http://hg.savannah.gnu.org/hgweb/octave/file/6443693a176f/scripts/statistics/base/mean.m#l130 m | |||
which defines {{ | |||
also available for the end users so they can test the status of their | |||
installation. The whole Octave test suite can be | |||
>> __run_test_suite__ | |||
Integrated test scripts: | Integrated test scripts: | ||
Line 42: | Line 27: | ||
See the file test/fntests.log for additional details. | See the file test/fntests.log for additional details. | ||
To run tests in a specific file, one can simply specify the path instead of | To run tests in a specific file, one can simply specify the path instead of a function name: | ||
a function name: | |||
test /full/path/to/file. | test /full/path/to/file.m | ||
== Writing tests == | |||
Tests appear as <code>%!</code> blocks at the bottom of the source file, together with <code>%!demo</code> blocks. A typical m function file, will have the following structure: | |||
<syntaxhighlight lang="Octave"> | |||
## Copyright | |||
## | |||
## A block with the copyright notice | |||
= | ## -*- texinfo -*- | ||
## | |||
## A block with the help text | |||
function [x, y, z] = foo (bar) | |||
## some amazing code | |||
endfunction | |||
%!assert (foo (1)) | |||
%!assert (foo (1:10)) | |||
%!assert (foo ("on"), "off") | |||
%!error <must be positive integer> foo (-1) | |||
%!error <must be positive integer> foo (1.5) | |||
%!demo | |||
%! ## see how cool foo() is: | |||
%! foo([1:100]) | |||
</syntaxhighlight> | |||
Tests can be added to oct functions in the C++ sources just as easily, see | Tests can be added to oct functions in the C++ sources just as easily, see | ||
Line 83: | Line 66: | ||
following structure: | following structure: | ||
<syntaxhighlight lang="c++"> | |||
// Copyright | |||
// | |||
// A block with the copyright notice | |||
DEFUN_DLD (foo, args, , | |||
"-*- texinfo -*-\n\ | |||
A block with the help text") | |||
{ | |||
// some amazing code | |||
} | |||
/* | |||
%!assert (foo (1)) | |||
%!assert (foo (1:10)) | |||
%!assert (foo ("on"), "off") | |||
%!error <must be positive integer> foo (-1) | |||
%!error <must be positive integer> foo (1.5) | |||
*/ | |||
</syntaxhighlight> | |||
=== Assert === | === Assert === | ||
{{codeline|%!assert}} lines are simplest tests to write and also the most | {{codeline|%!assert}} lines are the simplest tests to write and also the most | ||
common: | common: | ||
<syntaxhighlight lang="Octave"> | |||
%!assert (foo (bar)) # test fails if "foo (bar)" returns false | |||
%!assert (foo (bar), qux) # test fails if "foo (bar)" is different from "qux" | |||
</syntaxhighlight> | |||
These are actually a shorthand version of | These are actually a shorthand version of | ||
{{codeline|%!test assert (foo (bar))}}, and {{codeline|assert}} is simply | {{codeline|%!test assert (foo (bar))}}, and {{codeline|assert}} is simply | ||
an Octave function that throws | an Octave function that throws an error when two arguments fail to compare. | ||
=== Test === | === Test === | ||
While single | While single {{codeline|%!assert}} lines are the most common used tests, {{codeline|%!test}} blocks offer more features and flexibility. The code within {{codeline|%!test}} blocks is simply processed through the Octave interpreter. If the code generates an error, the test is said to fail. Often {{codeline|%!test}} blocks end with a call to {{codeline|assert}}: | ||
{{codeline|%!test}} blocks | |||
The code within | |||
<syntaxhighlight lang="Octave"> | |||
%!test | |||
%! a = [0 1 0 0 3 0 0 5 0 2 1]; | |||
%! b = [2 5 8 10 11]; | |||
%! for i = 1:5 | |||
%! assert (find (a, i), b(1:i)) | |||
%! endfor | |||
</syntaxhighlight> | |||
==== Test for no failure ==== | ==== Test for no failure ==== | ||
Line 139: | Line 120: | ||
simply with: | simply with: | ||
<syntaxhighlight lang="Octave"> | |||
%!test foo (bar) | |||
</syntaxhighlight> | |||
=== Error === | === Error / Warning === | ||
It is also important to test that a function performs its checks correctly | It is also important to test that a function performs its checks correctly | ||
and throws errors when it receives garbage. This can be done with | and throws errors (or warnings) when it receives garbage. This can be done with | ||
{{codeline|error}} blocks: | {{codeline|error}} (or {{codeline|warning}}) blocks: | ||
<syntaxhighlight lang="Octave"> | |||
%!error foo () # test that causes any error | |||
%!error <BAR must be a positive integer> foo (-1.5) # test that throws specific error message | |||
%!error id=Octave:invalid-fun-call foo () # test that throws specific error id | |||
%!warning foo () # test that causes any warning | |||
%!warning <negative values might give inaccurate results> foo (-1.5) # test that triggers a specific warning message | |||
%!warning id=BAR:possibly-inaccurate-result foo (-1.5) # test that triggers a specific warning id | |||
</syntaxhighlight> | |||
=== Shared functions === | === Shared functions === | ||
It is often useful to share a function | It is often useful to share a function among multiple tests. Sometimes | ||
these are only small helper functions, but more often these are just simpler | these are only small helper functions, but more often these are just simpler | ||
low performance implementations of the function being tested. These are | low performance implementations of the function being tested. These are | ||
created in {{codeline|%!function}} blocks: | created in {{codeline|%!function}} blocks: | ||
<syntaxhighlight lang="Octave"> | |||
%!function x = slow_foo (bar) | |||
%! ## a simple implementation of foo, definitely correct, but | |||
%! ## unfortunately too slow for anything other than tests. | |||
%!endfunction | |||
%!assert (foo (bar), slow_foo (bar)) | |||
== | %!test | ||
%! for i = -100:100 | |||
%! bar = qux (i); | |||
%! assert (foo (bar), slow_foo (bar)) | |||
%! endfor | |||
</syntaxhighlight> | |||
[[Category:Testing]] | |||
[[Category:Development]] | [[Category:Development]] |