Finding Memory Leaks

Overview

Memory leaks are a fact of life for programs which use dynamic memory. As the complexity of the program grows, the probability of accidentally introducing some sort of memory bug (out-of-range access, misaligned access, use after free, etc.) grows. To combat this, programmers use a variety of tools to identify memory issues. Previously the Octave developers had relied on Valgrind, but this is very slow. The current strategy is to use an Address Sanitizer which can be compiled in with a program by modern versions of gcc or clang.

Building Octave

Creating a version of Octave which works with an Address Sanitizer is quite simple. After obtaining the source code for Octave, use the configure script with whatever options you normally build with and add the special option --enable-address-sanitizer-flags. In addition, it is often very very useful for the backtraces if debugging information is included during the build. As an example of configuring Octave for finding memory leaks

setenv CFLAGS "-ggdb3 -O0 -pipe"
setenv CXXFLAGS "${CFLAGS}"
setenv FFLAGS "${CFLAGS}"
./configure --prefix=/home/user/local --enable-address-sanitizer-flags -C

GUI leaks

If there is a memory problem, the Address Sanitizer will dump information about the leak, including a backtrace, to stderr. However, if you are using the GUI it will have redirected stderr so that it no longer shows up in the terminal window from which Octave was launched. In order to debug the GUI, temporarily undo this redirection by commenting out lines 117-119 in libgui/qterminal/libqterminal/unix/QUnixTerminalImpl.cpp. The patch to do so is shown below. Copy this to a file and then execute patch -p1 < dup.patch.

File: dup.patch

diff -r f50ef29d874e libgui/qterminal/libqterminal/unix/QUnixTerminalImpl.cpp
--- a/libgui/qterminal/libqterminal/unix/QUnixTerminalImpl.cpp	Thu Sep 14 09:02:07 2017 -0700
+++ b/libgui/qterminal/libqterminal/unix/QUnixTerminalImpl.cpp	Thu Sep 14 09:40:07 2017 -0700
@@ -114,9 +114,11 @@ void QUnixTerminalImpl::connectToPty()
 
     int fds = m_kpty->slaveFd();
 
+    /*
     dup2 (fds, STDIN_FILENO);
     dup2 (fds, STDOUT_FILENO);
     dup2 (fds, STDERR_FILENO);
+    */
 
     if(!isatty(STDIN_FILENO)) {
         qDebug("Error: stdin is not a tty.");

After patching the code, build as before with the correct configure options and then proceed normally. The GUI will run, but the Command Window will now appear back in the terminal window from which you launched Octave.

Configuring Address Sanitizer

The Address Sanitizer is now built in to the Octave executable. The behavior of the Sanitizer can be controlled with the environment variable ASAN_OPTIONS. For example, this is a useful value of ASAN_OPTIONS:

setenv ASAN_OPTIONS "leak_check_at_exit=0:verbosity=1"

This stops reporting on any leaks at the end of the program, as those will be cleaned up by the operating system anyways when the process itself is destroyed.

Finding Leaks

Now that Octave has been built with the Address Sanitizer, and the Sanitizer itself has been configured by its environment variable, the next step is to launch Octave (typically ./run-octave), and execute code. This might mean running the default set of tests that come with Octave, __run_test_suite__. It could mean a particular function, "test ls". Or it could be simply running your own scripts, commands, etc. to see if a leak is triggered. If a memory error is detected the Sanitizer will dump a summary and backtrace to stderr. Make a new bug report at http://bugs.octave.org and include the backtrace in the report.