OctConf 2018 Notes

We will be taking notes as OctConf is going on. Keep an eye on this page.

Monday

Talking

jwe presented state of Octave development
Michele Ginesi presented his Octave work on special functions

list topics for discussion and work for this OctConf: this very page

frequency of Octave releases: a new release from stable this OctConf. A new release from default in 4 weeks time. From now on, new releases every year in January with freeze before the Christmas holidays.

implementing String class: Matlab added double quote strings, which instantiate objects of their new String class. This behaves different from our double quote strings because it does not escape special characters. Being Matlab compatible will mean being backwards incompatible. Decided to first implement the String class and make the Octave internals handle it. Then we will worry about having the double quotes create them and decide about the escaping of characters.

scope of functions in script files

Matlab scripts can now include function definitions. However, these behave different from Octave scripts in two ways:

functions defined in script files are locally scoped and will not be available after running the script
functions in the script file can de defined in the script after the statements that use them while Octave requires them to be defined first.

jwe says that this behaviour should be simple to implement but will be backwards incompatible. Solution is to have the run function change and behave like Matlab does. Currently, the function source is the same as run. The function source will keep the old behaviour and can be used to source a file to import functions definitions into the global scope, as if the sourced file had been ran in the interpreter.

Doing

email maintainers mailing list with conclusions: done by carandraug

Tuesday

Talking

future GSoC students: we have questioned the value of this. Most students don't stay after GSoC and most projects do not get merged into core and the packages. Even if it costs us no money, it costs us a lot of time since most students need a lot of hand holding, and it's the whole community that is actually mentoring. Ignoring the students is also not nice to them. Nir is the person responsible and he's not attending this year so we are not deciding anything. However, we need to ensure that students know much, and do a better job at filtering out clueless students.

refactoring (threading) / developer docs / symbol naming / liboctave interface stability and usability: These are all very related issues. We have some issues threading and those are mainly because we don't know how to do things properly. Some of us have learned how since but we don't have that knowledge shared and so it's a bit of lack of documentation for ourselves. We decided that changes to Octave internals, the C++ stuff, will need to include doxygen docs for the things that it changes. We have notes on how to write doxygen docs on our C++ style guide.; We also have builds of doxygen docs online, including development version http://octave.org/doxygen/ but they are not up to date. jwe will set his build bots to update the doxygen docs daily.; We also discussed making available builds of the default branch for users. This would be nice because we get users reporting bugs earlier. However, we are afraid that people will not understand that the latest version is a work in progress, not the latest usable version. So we decided against making such builds available.

test coverage: We can already get the line coverage of our test suite if we run make coverage but that only includes the C++ code. jwe says that it would be simple to add some fields to the interpreter that would allow us to also get the line coverage. This would only be done if Octave was built with specific flags. Oliver pointed out that because of vectorized options, line coverage in m file is not as useful as in other languages since the conditions that in other languages is done with if blocks, are instead done by using indexing with an empty array of an array of false elements.

measurements of performance: the test suite is not the best measure of performance. We are estimating that most of its time is spent on running assert which is irrelevant for real Octave programs. It is also mainly cover corner cases and a lot of tests are if it fails when input checking is correct. We should instead have real Octave programs that we used for benchmarking. There wasn't much discussion of this topic beyond noting the problem exists.

Octave Forge community

Currently, there is a high barrier to publish new packages for Octave on Octave-Forge. This mainly comes from the maintenance burden that the Octave Forge Community has with the packages. In particular:

The package has to be actively maintained in order to stay compatible with updates to Octave core
There is no known good solution to have a Matlab-compatible package, which respects the requirements of an OF package
The list of initial requirements (legal and technical) is quite high for outside contributers, especially for domain experts (which we want to address) who are no software engineers

This creates problems for some external package authors, who basically want to release on their own. Also, this is not resolved by the *two package groups* that we have introduced during the last year.

We see potential for more Octave package publications, which come from “one-off” publications (e. g. with a book / thesis), existing Matlab packages (which also offer a Octave variant), and simple (experimental) libraries of m-files. To increase the pool of high-level users we need to reduce even more the burden to install and distribute packages independently.

The problem that we see today is that Octave-Forge is two-fold: (1) a (collaborative) software development platform and (2) a (re-)distributor of software. So, these new use-cases would not be part of Octave-Forge. In the context of an increasing number of independent “Github developers” this is not needed so much.

To do this we bring back the idea of allowing `pkg` to download and install directly from a url to a `.zip` file or `.tar.gz` file, e.g. `pkg install 'https://myrepo.org/mypackage.zip'`

There will be no checks (nor quality standard) on these packages and the user should be clearly warned:
- Security issue: With no https the download can be redirected. The user can be tricked into entering this command, which downloads and installs (=runs) arbitrary software.
- Feedback/QA issue: If that package doesn't work, users are probably asking for help.
Should `pkg update` also work for these packages?
- Maybe no. Just re-run the install command.
- If needed, there could be an update url in the DESCRIPTION file
We need to provide guidelines, recommendations and tools for people trying to distribute their packages. We are doing something in this direction, but we need a little boost here: https://octave.sourceforge.io/templates/Makefile, https://sourceforge.net/p/octave/example-package/ci/default/tree/
If a package can be installed directly from the repository, it would be supported out-of-the-box by Github, Gitlab, …

Also, we discussed a “package database” like it is done in Julia (see [1] for a very lightweight solution, which only contains package names, URLs and version), but don't see advantages from that right now.

packages into Octave core: Decided that Octave releases will include some core packages. These will be developed along Octave core, in the same repository, and will be released at the same time. For all purposes they will be a single software project. However, they will still need to be loaded like any other packages.; Initially, these packages will cover signal processing, statistics, image processing, optimisation, communications, and control. Their aim is to be compatible with their Matlab counterparts and they will be partly imported from the existing Octave Forge packages.; We won't import all packages in one go. We will start with statistics because we have recently moved a bunch of core functions into it and so they will be easier to import back. signal will probably follow but Pantxo pointed out that this would require a lot of work on its documentation. optim and communications will probably go last since communications is abandoned and optim does not actually have many Matlab compatible functions.; The current forge packages include extra functions that are not in the Matlab counterparts and their inclusion in the core packages will be decided on a case by case basis. Not sure what will happen with them most likely they would go into a separate package like statistics-legacy.

Doing

merging of previous GSoC projects: Rik merged Ginesi's special functions and Carlo's worked remotely on merging Cristiano Dorigo's improvement of iterative solvers.

polygon reconstruction algorithm: Rik and Pantxo worked on this

new release: jwe been preparing for wednesday's release

dinner in town: That was a very nice time with much discussion about future of Octave and Octave Forge, packages, and makefiles.

email maintainers mailing list with conclusions: done

Wednesday

Talking

SWAN presentation: CERN presented their customized Jupyter instance named SWAN which integrates with the other CERN tools for data and data processing.

make --no-gui default: Mike http://lists.gnu.org/archive/html/octave-maintainers/2018-03/msg00044.html suggested on the mailing to make this change for the upcoming 4.4 release. We discussed it all together, and everyone agrees. This should make things simpler for everyone. There will be only one flag which will be --gui to start the gui and will be used by launchers. This will simplify the octave code too since we will no longer have to guess whether to start it or not based on whether we have persist or scripts to eval.; We then coded that change.

review our review of code contributions: a quick rehash of previous discussions. We do a lousy job at review contributions and would need better tools to make the reviewing easier. However, no is available to have such a tool which actually means replacing savannah with our own instance.

the complicated code in pkg: Juan showed that the code in pkg is a bit overcomplicated. Some changes were done to simplify it some 3-4 years ago but never got merged. Those changes no longer apply. Juan will try to make those changes again but first will be adding tests for pkg.

warning users if installing packages with url: discussed adding a warning that a package was not community maintained if user tried to use a url, and required user to answer yes or no. Decided against such warning. We may instead consider add a warning in the future if user tries to install something with an insecure connection (http:// instead of https:// or file://)

Doing

new Octave 4.2.2 release: It was done. In the process of merging default to stable, we may do that tomorrow instead.

new release of image package: carandraug not even had time to get to it, too busy with changes to core and more discussions.

more tests to increase code coverage

CERN visit

email maintainers mailing list with conclusions

Thursday

Talking

OctConf 2019: We discussed the place, date, fee, and objective of OctConf 2019.; ----; For place, CERN offered to host it again next year. However, it was also suggested that we should try other places to meet new people and make it easier for people from far away, namely west coast USA. However, this also means more work since it's new logistics and other places have different costs. Seems like most contributions are European and flying them all to USA would be more expensive than flying the American developers to Europe. CERN is also quite cheap. Flying to Geneva from Europe is cheap, the rooms are free, and the CERN hotels are very affordable. It was also suggested to change place every two years or to come to CERN every two years with a new place each other year. We decided to have a call for an organizer, with a deadline of 1st of May. If no suitable option appears, then the place will be CERN. Rik will send out the call for organizer.; ----; The date for the conference in 2019 must happen during the second half of March or the first half of April, according to the organizers availability. If it happens at CERN, it must be after 17th of March to avoid the motor exposition. As for the duration, next time it will be 3 full days instead of 2 days and two half days. The morning of the first day will be planning for the other days and the last evening to discuss OctConf 2020.; ----; Because of free registration, many registered people do not show. This makes organization difficult, we don't know how much coffee to order, how many people for the restaurants, or the size of the room. We will introduce a registration fee of ~ 20€ which then can be used to cover some costs of the social event. Other possibilities is a much higher fee which would include the accommodation since if attendees are coming, they would have to pay anyway. However, it was decided against since they will only see the high cost and decide against coming.; ----; This years conference was pretty much a developers conference. There was a total of 3 talks with all other time spent on discussing developer topics and coding. It was discussed the issue of Octave visibility and that a user space could exist during OctConf, which would be one day for users to present their work to each other and have some workshops. This could even happen at the same time as developers would meet on other rooms. It was decided against, as there was not enough interest among the developers. This makes organization simpler since we also only need a smaller room with a whiteboard. There will be john's yearly talk about the state of Octave and Rik will make a more outreach talk, more sales pitch, showing what can be done with Octave and where it is being used.

Doing

email maintainers mailing list with conclusions: done by carandraug