OctConf 2018 Notes

We will be taking notes as OctConf is going on. Keep an eye on this page.

Monday

Talking

• jwe presented state of Octave development
• Michele Ginesi presented his Octave work on special functions

list topics for discussion and work for this OctConf

this very page

frequency of Octave releases

a new release from stable this OctConf. A new release from default in 4 weeks time. From now on, new releases every year in January with freeze before the Christmas holidays.

implementing String class

Matlab added double quote strings, which instantiate objects of their new String class. This behaves different from our double quote strings because it does not escape special characters. Being Matlab compatible will mean being backwards incompatible. Decided to first implement the String class and make the Octave internals handle it. Then we will worry about having the double quotes create them and decide about the escaping of characters.

scope of functions in script files

Matlab scripts can now include function definitions. However, these behave different from Octave scripts in two ways:

1. functions defined in script files are locally scoped and will not be available after running the script
2. functions in the script file can de defined in the script after the statements that use them while Octave requires them to be defined first.

jwe says that this behaviour should be simple to implement but will be backwards incompatible. Solution is to have the run function change and behave like Matlab does. Currently, the function source is the same as run. The function source will keep the old behaviour and can be used to source a file to import functions definitions into the global scope, as if the sourced file had been ran in the interpreter.

Doing

email maintainers mailing list with conclusions

done by carandraug

Tuesday

Talking

future GSoC students

we have questioned the value of this. Most students don't stay after GSoC and most projects do not get merged into core and the packages. Even if it costs us no money, it costs us a lot of time since most students need a lot of hand holding, and it's the whole community that is actually mentoring. Ignoring the students is also not nice to them. Nir is the person responsible and he's not attending this year so we are not deciding anything. However, we need to ensure that students know much, and do a better job at filtering out clueless students.

refactoring (threading) / developer docs / symbol naming / liboctave interface stability and usability

These are all very related issues. We have some issues threading and those are mainly because we don't know how to do things properly. Some of us have learned how since but we don't have that knowledge shared and so it's a bit of lack of documentation for ourselves. We decided that changes to Octave internals, the C++ stuff, will need to include doxygen docs for the things that it changes. We have notes on how to write doxygen docs on our C++ style guide.

We also have builds of doxygen docs online, including development version http://octave.org/doxygen/ but they are not up to date. jwe will set his build bots to update the doxygen docs daily.

We also discussed making available builds of the default branch for users. This would be nice because we get users reporting bugs earlier. However, we are afraid that people will not understand that the latest version is a work in progress, not the latest usable version. So we decided against making such builds available.

test coverage

We can already get the line coverage of our test suite if we run make coverage but that only includes the C++ code. jwe says that it would be simple to add some fields to the interpreter that would allow us to also get the line coverage. This would only be done if Octave was built with specific flags. Oliver pointed out that because of vectorized options, line coverage in m file is not as useful as in other languages since the conditions that in other languages is done with if blocks, are instead done by using indexing with an empty array of an array of false elements.

measurements of performance

the test suite is not the best measure of performance. We are estimating that most of its time is spent on running assert which is irrelevant for real Octave programs. It is also mainly cover corner cases and a lot of tests are if it fails when input checking is correct. We should instead have real Octave programs that we used for benchmarking. There wasn't much discussion of this topic beyond noting the problem exists.

Octave Forge community

Currently, there is a high barrier to publish new packages for Octave on Octave-Forge. This mainly comes from the maintenance burden that the Octave Forge Community has with the packages. In particular:

• The package has to be actively maintained in order to stay compatible with updates to Octave core
• There is no known good solution to have a Matlab-compatible package, which respects the requirements of an OF package
• The list of initial requirements (legal and technical) is quite high for outside contributers, especially for domain experts (which we want to address) who are no software engineers

This creates problems for some external package authors, who basically want to release on their own. Also, this is not resolved by the *two package groups* that we have introduced during the last year.

We see potential for more Octave package publications, which come from “one-off” publications (e. g. with a book / thesis), existing Matlab packages (which also offer a Octave variant), and simple (experimental) libraries of m-files. To increase the pool of high-level users we need to reduce even more the burden to install and distribute packages independently.

The problem that we see today is that Octave-Forge is two-fold: (1) a (collaborative) software development platform and (2) a (re-)distributor of software. So, these new use-cases would not be part of Octave-Forge. In the context of an increasing number of independent “Github developers” this is not needed so much.

To do this we bring back the idea of allowing `pkg` to download and install directly from a url to a `.zip` file or `.tar.gz` file, e.g. `pkg install 'https://myrepo.org/mypackage.zip'`

• There will be no checks (nor quality standard) on these packages and the user should be clearly warned:
  • Security issue: With no https the download can be redirected. The user can be tricked into entering this command, which downloads and installs (=runs) arbitrary software.
  • Feedback/QA issue: If that package doesn't work, users are probably asking for help.
• Should `pkg update` also work for these packages?
  • Maybe no. Just re-run the install command.
  • If needed, there could be an update url in the DESCRIPTION file
• We need to provide guidelines, recommendations and tools for people trying to distribute their packages. We are doing something in this direction, but we need a little boost here: https://octave.sourceforge.io/templates/Makefile, https://sourceforge.net/p/octave/example-package/ci/default/tree/
• If a package can be installed directly from the repository, it would be supported out-of-the-box by Github, Gitlab, …

Also, we discussed a “package database” like it is done in Julia (see [1] for a very lightweight solution, which only contains package names, URLs and version), but don't see advantages from that right now.

packages into Octave core

Decided that Octave releases will include some core packages. These will be developed along Octave core, in the same repository, and will be released at the same time. For all purposes they will be a single software project. However, they will still need to be loaded like any other packages.

Initially, these packages will cover signal processing, statistics, image processing, optimisation, communications, and control. Their aim is to be compatible with their Matlab counterparts and they will be partly imported from the existing Octave Forge packages.

We won't import all packages in one go. We will start with statistics because we have recently moved a bunch of core functions into it and so they will be easier to import back. signal will probably follow but Pantxo pointed out that this would require a lot of work on its documentation. optim and communications will probably go last since communications is abandoned and optim does not actually have many Matlab compatible functions.

The current forge packages include extra functions that are not in the Matlab counterparts and their inclusion in the core packages will be decided on a case by case basis. Not sure what will happen with them most likely they would go into a separate package like statistics-legacy.

Doing

merging of previous GSoC projects

Rik merged Ginesi's special functions and Carlo's worked remotely on merging Cristiano Dorigo's improvement of iterative solvers.

polygon reconstruction algorithm

Rik and Pantxo worked on this

new release

jwe been preparing for wednesday's release

dinner in town

That was a very nice time with much discussion about future of Octave and Octave Forge, packages, and makefiles.

email maintainers mailing list with conclusions

done

Wednesday

Talking

SWAN presentation

CERN presented their customized Jupyter instance named SWAN which integrates with the other CERN tools for data and data processing.

make {{Codeline|--no-gui} default

Mike http://lists.gnu.org/archive/html/octave-maintainers/2018-03/msg00044.html suggested on the mailing to make this change for the upcoming 4.4 release. We discussed it all together, and everyone agrees. This should make things simpler for everyone. There will be only one flag which will be --gui to start the gui and will be used by launchers. This will simplify the octave code too since we will no longer have to guess whether to start it or not based on whether we have persist or scripts to eval.

review our review of code contributions

build system issues

Doing

new Octave 4.2.2 release

new release of image package

more tests to increase code coverage

CERN visit

email maintainers mailing list with conclusions

Thursday

Talking

OctConf 2019

pending discussions and work

Doing

email maintainers mailing list with conclusions