Editing Tests

Jump to navigation Jump to search

Warning: You are not logged in. Your IP address will be publicly visible if you make any edits. If you log in or create an account, your edits will be attributed to your username, along with other benefits.

The edit can be undone. Please check the comparison below to verify that this is what you want to do, and then save the changes below to finish undoing the edit.

Latest revision Your text
Line 1: Line 1:
Having a thorough [https://en.wikipedia.org/wiki/Test_suite test suite] is something very important which is usually overlooked.  It is an incredible help in preventing regression bugs and quickly assess the status of old code. For example, many packages in Octave Forge become deprecated after losing their maintainer simply because they have no test suite.
+
Having a thorough [http://en.wikipedia.org/wiki/Test_suite test suite] is
 +
something very important which is usually overlooked.  It is an incredible
 +
help in preventing regression bugs and quickly assess the status of old code.
 +
For example, many packages in Octave Forge become deprecated after losing
 +
their maintainer simply because they have no test suite.
  
GNU Octave has multiple tools that help in creating a comprehensive test suite, accessible to both developers and end-users, as detailed on the [http://www.gnu.org/software/octave/doc/interpreter/Test-Functions.html Octave manual]. Basically, test blocks are {{codeline|%!test}} comment blocks, typically at the end of a source file, which are ignored by the Octave interpreter and only read by the {{manual|test}} function.
+
GNU Octave has multiple tools that help in creating a comprehensive test
 +
suite, accessible to both developers and end-users, as detailed on the
 +
[http://www.gnu.org/software/octave/doc/interpreter/Test-Functions.html Octave manual].
 +
Basically, test blocks are {{codeline|%!test}} comment blocks, typically at the
 +
end of a source file, which are ignored by the Octave interpreter and only
 +
read by the {{codeline|test}} function.
  
 
== Running tests ==
 
== Running tests ==
  
To run all the tests of a specific function, simply use the {{manual|test}} command at the Octave prompt.  For example, to run the tests of the Octave function {{manual|mean}} type:
+
To run all the tests of a specific function, simply use the {{codeline|test}}
 +
command at the Octave prompt.  For example, to run the tests in
 +
{{codeline|mean()}}:
  
  >> test mean
+
  octave-cli-3.8.2> test mean
 
  PASSES 17 out of 17 tests
 
  PASSES 17 out of 17 tests
  
These tests are written in the Octave language [http://hg.savannah.gnu.org/hgweb/octave/file/6443693a176f/scripts/statistics/base/mean.m#l130 at the bottom of <code>mean.m</code>] which defines the {{manual|mean}} function.  It is important that these tests are also available for the end users so they can test the status of their installation.  The whole Octave test suite can be run with:
+
These tests are written in the Octave language at the bottom of the
 +
[http://hg.savannah.gnu.org/hgweb/octave/file/6443693a176f/scripts/statistics/base/mean.m#l130 m file]
 +
which defines {{codeline|mean()}}.  It is important that these tests are
 +
also available for the end users so they can test the status of their
 +
installation.  The whole Octave test suite can be ran with:
  
  >> __run_test_suite__
+
  octave-cli-3.8.2> __run_test_suite__
 
   
 
   
 
  Integrated test scripts:
 
  Integrated test scripts:
Line 27: Line 42:
 
  See the file test/fntests.log for additional details.
 
  See the file test/fntests.log for additional details.
  
To run tests in a specific file, one can simply specify the path instead of a function name:
+
To run tests in a specific file, one can simply specify the path instead of
 +
a function name:
 +
 
 +
  test /full/path/to/file.cc
  
  test /full/path/to/file.m
 
  
 
== Writing tests ==
 
== Writing tests ==
  
Tests appear as <code>%!</code> blocks at the bottom of the source file, together with <code>%!demo</code> blocks.  A typical m function file, will have the following structure:
+
Tests appear as {{codeline|%!}} blocks at the bottom of the source file,
 +
together with {{codeline|%!demo}} blocks.  A typical m function file, will
 +
have the following structure:
  
<syntaxhighlight lang="Octave">
+
## Copyright
## Copyright
+
##
##
+
## A block with the copyright notice
## A block with the copyright notice
+
 
+
## -*- texinfo -*-
## -*- texinfo -*-
+
##
##
+
## A block with the help text
## A block with the help text
+
 
+
function [x, y, z] = foo (bar)
function [x, y, z] = foo (bar)
+
  ## here's some amazing code
  ## some amazing code
+
endfunction
endfunction
+
 
+
%!assert (foo (1))
%!assert (foo (1))
+
%!assert (foo (1:10))
%!assert (foo (1:10))
+
%!assert (foo ("on"), "off")
%!assert (foo ("on"), "off")
+
%!error <must be positive integer> foo (-1)
%!error <must be positive integer> foo (-1)
+
%!error <must be positive integer> foo (1.5)
%!error <must be positive integer> foo (1.5)
+
 
+
%!demo
%!demo
+
%! ## see how cool foo() is:
%! ## see how cool foo() is:
+
%! foo([1:100])
%! foo([1:100])
 
</syntaxhighlight>
 
  
 
Tests can be added to oct functions in the C++ sources just as easily, see
 
Tests can be added to oct functions in the C++ sources just as easily, see
Line 66: Line 83:
 
following structure:
 
following structure:
  
<syntaxhighlight lang="c++">
+
// Copyright
// Copyright
+
//
//
+
// A block with the copyright notice
// A block with the copyright notice
 
 
   
 
   
DEFUN_DLD (foo, args, ,
+
DEFUN_DLD (foo, args, ,
"-*- texinfo -*-\n\
+
"-*- texinfo -*-\n\
A block with the help text")
+
A block with the help text")
{
+
{
   // some amazing code
+
   \\ here's some amazing code
}
+
}
 +
function [x, y, z] = foo (bar)
 +
  ## here's some amazing code
 +
endfunction
 
   
 
   
/*
+
/*
%!assert (foo (1))
+
%!assert (foo (1))
%!assert (foo (1:10))
+
%!assert (foo (1:10))
%!assert (foo ("on"), "off")
+
%!assert (foo ("on"), "off")
%!error <must be positive integer> foo (-1)
+
%!error <must be positive integer> foo (-1)
%!error <must be positive integer> foo (1.5)
+
%!error <must be positive integer> foo (1.5)
*/
+
*/
</syntaxhighlight>
+
 
  
 
=== Assert ===
 
=== Assert ===
  
{{codeline|%!assert}} lines are the simplest tests to write and also the most
+
{{codeline|%!assert}} lines are simplest tests to write and also the most
 
common:
 
common:
  
<syntaxhighlight lang="Octave">
+
%!assert (foo (bar))      # test fails if "foo (bar)" returns false
%!assert (foo (bar))      # test fails if "foo (bar)" returns false
+
%!assert (foo (bar), qux) # test fails if "foo (bar)" is different from "qux
%!assert (foo (bar), qux) # test fails if "foo (bar)" is different from "qux"
 
</syntaxhighlight>
 
  
 
These are actually a shorthand version of
 
These are actually a shorthand version of
 
{{codeline|%!test assert (foo (bar))}}, and {{codeline|assert}} is simply
 
{{codeline|%!test assert (foo (bar))}}, and {{codeline|assert}} is simply
an Octave function that throws an error when two arguments fail to compare.
+
an Octave function that throws errors when two arguments fail to compare.
  
 
=== Test ===
 
=== Test ===
  
While single {{codeline|%!assert}} lines are the most common used tests, {{codeline|%!test}} blocks offer more features and flexibility. The code within {{codeline|%!test}} blocks is simply processed through the Octave interpreter.  If the code generates an error, the test is said to fail.  Often {{codeline|%!test}} blocks end with a call to {{codeline|assert}}:
+
While single line {{codeline|%!assert}}s are the most common test used,
 +
{{codeline|%!test}} blocks are the ultimate, most useful, and flexible.
 +
The code within such block is simply processed through the Octave interpreter
 +
and if the code generates an error, then the test is said to fail.  These
 +
often end with a call to {{codeline|assert}}:
  
<syntaxhighlight lang="Octave">
+
%!test
%!test
+
%! a = [0 1 0 0 3 0 0 5 0 2 1];
%! a = [0 1 0 0 3 0 0 5 0 2 1];
+
%! b = [2 5 8 10 11];
%! b = [2 5 8 10 11];
+
%! for i = 1:5
%! for i = 1:5
+
%!  assert (find (a, i), b(1:i))
%!  assert (find (a, i), b(1:i))
+
%! endfor
%! endfor
 
</syntaxhighlight>
 
  
 
==== Test for no failure ====
 
==== Test for no failure ====
Line 120: Line 139:
 
simply with:
 
simply with:
  
<syntaxhighlight lang="Octave">
+
%!test foo (bar)
%!test foo (bar)
 
</syntaxhighlight>
 
  
 
=== Error / Warning ===
 
=== Error / Warning ===
Line 130: Line 147:
 
{{codeline|error}} (or {{codeline|warning}}) blocks:
 
{{codeline|error}} (or {{codeline|warning}}) blocks:
  
<syntaxhighlight lang="Octave">
+
%!error foo ()  # test that causes any error
%!error foo ()  # test that causes any error
+
%!error <BAR must be a positive integer> foo (-1.5)  # test that throws specific error message
%!error <BAR must be a positive integer> foo (-1.5)  # test that throws specific error message
+
%!error id=Octave:invalid-fun-call foo ()  # test that throws specific error id
%!error id=Octave:invalid-fun-call foo ()  # test that throws specific error id
 
  
%!warning foo ()  # test that causes any warning
+
%!warning foo ()  # test that causes any warning
%!warning <negative values might give inaccurate results> foo (-1.5)  # test that triggers a specific warning message
+
%!warning <negative values might give inaccurate results> foo (-1.5)  # test that triggers a specific warning message
%!warning id=BAR:possibly-inaccurate-result foo (-1.5)  # test that triggers a specific warning id
+
%!warning id=BAR:possibly-inaccurate-result foo (-1.5)  # test that triggers a specific warning id
</syntaxhighlight>
 
  
 
=== Shared functions ===
 
=== Shared functions ===
  
It is often useful to share a function among multiple tests.  Sometimes
+
It is often useful to share a function between multiple test.  Sometimes
 
these are only small helper functions, but more often these are just simpler
 
these are only small helper functions, but more often these are just simpler
 
low performance implementations of the function being tested.  These are
 
low performance implementations of the function being tested.  These are
 
created in {{codeline|%!function}} blocks:
 
created in {{codeline|%!function}} blocks:
  
<syntaxhighlight lang="Octave">
 
%!function x = slow_foo (bar)
 
%!  ## a simple implementation of foo, definitely correct, but
 
%!  ## unfortunately too slow for anything other than tests.
 
%!endfunction
 
  
%!assert (foo (bar), slow_foo (bar))
+
%!function x = slow_foo (bar)
 +
%!  ## a simple implementation of foo, definitely correct, but
 +
%!  ## unfortunately too slow for anything other than tests.
 +
%!endfunction
 +
 +
%!assert (foo (bar), slow_foo (bar))
 +
 +
%!test
 +
%! for i = -100:100
 +
%!  bar = qux (i);
 +
%!  assert (foo (bar), slow_foo (bar))
 +
%! endfor
 +
 
  
%!test
+
== Code coverage ==
%! for i = -100:100
 
%!  bar = qux (i);
 
%!  assert (foo (bar), slow_foo (bar))
 
%! endfor
 
</syntaxhighlight>
 
  
[[Category:Testing]]
 
 
[[Category:Development]]
 
[[Category:Development]]

Please note that all contributions to Octave may be edited, altered, or removed by other contributors. If you do not want your writing to be edited mercilessly, then do not submit it here.
You are also promising us that you wrote this yourself, or copied it from a public domain or similar free resource (see Octave:Copyrights for details). Do not submit copyrighted work without permission!

To edit this page, please answer the question that appears below (more info):

Cancel Editing help (opens in new window)

Templates used on this page: