OctConf 2018 Notes: Difference between revisions

From Octave
Jump to navigation Jump to search
(→‎Talking: Octave Forge community)
Line 45: Line 45:


; Octave Forge community
; Octave Forge community
: To increase the pool of high-level users we need to reduce even more the burden to install and distribute packages independently. To do this we bring back the idea of allowing `pkg` to download and install directly from a url to a `.zip` file or `.tar.gz` file, e.g. `pkg install 'https://myrepo.org/mypackage.zip'`
Currently, there is a high barrier to publish new packages for Octave on Octave-Forge. This mainly comes from the maintenance burden that the Octave Forge Community has with the packages. In particular:
: There will be no checks (nor quality standard) on these packages and the user should be clearly warned about the insecurity (y/n answer?).
* The package has to be actively maintained in order to stay compatible with updates to Octave core
:: * Should `pkg update` also work for these packages? Maybe no. Just re-run the install command.
* There is no known good solution to have a Matlab-compatible package, which respects the requirements of an OF package
:: * We need to provide guidelines, recommendations and tools for people trying to distribute their packages. We are doing something in this direction, but we need a little boost here: https://octave.sourceforge.io/templates/Makefile, https://sourceforge.net/p/octave/example-package/ci/default/tree/
* The list of initial requirements (legal and technical) is quite high for outside contributers, especially for domain experts (which we want to address) who are no software engineers
This creates problems for some external package authors, who basically want to release on their own. Also, this is not resolved by the *two package groups* that we have introduced during the last year.
 
We see potential for more Octave package publications, which come from “one-off” publications (e. g. with a book / thesis), existing Matlab packages (which also offer a Octave variant), and simple (experimental) libraries of m-files. To increase the pool of high-level users we need to reduce even more the burden to install and distribute packages independently.
 
The problem that we see today is that Octave-Forge is two-fold: (1) a (collaborative) software development platform and (2) a (re-)distributor of software. So, these new use-cases would not be part of Octave-Forge. In the context of an increasing number of independent “Github developers” this is not needed so much.
 
To do this we bring back the idea of allowing `pkg` to download and install directly from a url to a `.zip` file or `.tar.gz` file, e.g. `pkg install 'https://myrepo.org/mypackage.zip'`
* There will be no checks (nor quality standard) on these packages and the user should be clearly warned:
** Security issue: With no https the download can be redirected. The user can be tricked into entering this command, which downloads and installs (=runs) arbitrary software.
** Feedback/QA issue: If that package doesn't work, users are probably asking for help.
* Should `pkg update` also work for these packages?
** Maybe no. Just re-run the install command.
** If needed, there could be an update url in the DESCRIPTION file
* We need to provide guidelines, recommendations and tools for people trying to distribute their packages. We are doing something in this direction, but we need a little boost here: https://octave.sourceforge.io/templates/Makefile, https://sourceforge.net/p/octave/example-package/ci/default/tree/
* If a package can be installed directly from the repository, it would be supported out-of-the-box by Github, Gitlab, …
 
Also, we discussed a “package database” like it is done in Julia (see [https://github.com/JuliaLang/METADATA.jl] for a very lightweight solution, which only contains package names, URLs and version), but don't see advantages from that right now.


; packages into Octave core
; packages into Octave core

Revision as of 17:31, 13 March 2018

We will be taking notes as OctConf is going on. Keep an eye on this page.

Monday

Talking

  • jwe presented state of Octave development
  • Michele Ginesi presented his Octave work on special functions
list topics for discussion and work for this OctConf
this very page
frequency of Octave releases
a new release from stable this OctConf. A new release from default in 4 weeks time. From now on, new releases every year in January with freeze before the Christmas holidays.
implementing String class
Matlab added double quote strings, which instantiate objects of their new String class. This behaves different from our double quote strings because it does not escape special characters. Being Matlab compatible will mean being backwards incompatible. Decided to first implement the String class and make the Octave internals handle it. Then we will worry about having the double quotes create them and decide about the escaping of characters.
scope of functions in script files
Matlab scripts can now include function definitions. However, these behave different from Octave scripts in two ways:
  1. functions defined in script files are locally scoped and will not be available after running the script
  2. functions in the script file can de defined in the script after the statements that use them while Octave requires them to be defined first.
jwe says that this behaviour should be simple to implement but will be backwards incompatible. Solution is to have the run function change and behave like Matlab does. Currently, the function source is the same as run. The function source will keep the old behaviour and can be used to source a file to import functions definitions into the global scope, as if the sourced file had been ran in the interpreter.

Doing

email maintainers mailing list with conclusions
done by carandraug


Tuesday

Talking

future GSoC students
we have questioned the value of this. Most students don't stay after GSoC and most projects do not get merged into core and the packages. Even if it costs us no money, it costs us a lot of time since most students need a lot of hand holding, and it's the whole community that is actually mentoring. Ignoring the students is also not nice to them. Nir is the person responsible and he's not attending this year so we are not deciding anything. However, we need to ensure that students know much, and do a better job at filtering out clueless students.
refactoring (threading) / developer docs / symbol naming / liboctave interface stability and usability
These are all very related issues. We have some issues threading and those are mainly because we don't know how to do things properly. Some of us have learned how since but we don't have that knowledge shared and so it's a bit of lack of documentation for ourselves. We decided that changes to Octave internals, the C++ stuff, will need to include doxygen docs for the things that it changes. We have notes on how to write doxygen docs on our C++ style guide.
We also have builds of doxygen docs online, including development version http://octave.org/doxygen/ but they are not up to date. jwe will set his build bots to update the doxygen docs daily.
We also discussed making available builds of the default branch for users. This would be nice because we get users reporting bugs earlier. However, we are afraid that people will not understand that the latest version is a work in progress, not the latest usable version. So we decided against making such builds available.
test coverage
measurements of performance
Octave Forge community

Currently, there is a high barrier to publish new packages for Octave on Octave-Forge. This mainly comes from the maintenance burden that the Octave Forge Community has with the packages. In particular:

  • The package has to be actively maintained in order to stay compatible with updates to Octave core
  • There is no known good solution to have a Matlab-compatible package, which respects the requirements of an OF package
  • The list of initial requirements (legal and technical) is quite high for outside contributers, especially for domain experts (which we want to address) who are no software engineers

This creates problems for some external package authors, who basically want to release on their own. Also, this is not resolved by the *two package groups* that we have introduced during the last year.

We see potential for more Octave package publications, which come from “one-off” publications (e. g. with a book / thesis), existing Matlab packages (which also offer a Octave variant), and simple (experimental) libraries of m-files. To increase the pool of high-level users we need to reduce even more the burden to install and distribute packages independently.

The problem that we see today is that Octave-Forge is two-fold: (1) a (collaborative) software development platform and (2) a (re-)distributor of software. So, these new use-cases would not be part of Octave-Forge. In the context of an increasing number of independent “Github developers” this is not needed so much.

To do this we bring back the idea of allowing `pkg` to download and install directly from a url to a `.zip` file or `.tar.gz` file, e.g. `pkg install 'https://myrepo.org/mypackage.zip'`

  • There will be no checks (nor quality standard) on these packages and the user should be clearly warned:
    • Security issue: With no https the download can be redirected. The user can be tricked into entering this command, which downloads and installs (=runs) arbitrary software.
    • Feedback/QA issue: If that package doesn't work, users are probably asking for help.
  • Should `pkg update` also work for these packages?
    • Maybe no. Just re-run the install command.
    • If needed, there could be an update url in the DESCRIPTION file
  • We need to provide guidelines, recommendations and tools for people trying to distribute their packages. We are doing something in this direction, but we need a little boost here: https://octave.sourceforge.io/templates/Makefile, https://sourceforge.net/p/octave/example-package/ci/default/tree/
  • If a package can be installed directly from the repository, it would be supported out-of-the-box by Github, Gitlab, …

Also, we discussed a “package database” like it is done in Julia (see [1] for a very lightweight solution, which only contains package names, URLs and version), but don't see advantages from that right now.

packages into Octave core

Doing

merging of previous GSoC projects
Rik merged Ginesi's special functions and Carlo's worked on merging ode45
polygon reconstruction algorithm
Rik and Pantxo worked on this
new release
jwe been preparing for wednesday's release
dinner in town
email maintainers mailing list with conclusions

Wednesday

Talking

review our review of code contributions
build system issues

Doing

new Octave 4.2.2 release
new release of image package
more tests to increase code coverage
CERN visit
email maintainers mailing list with conclusions


Thursday

Talking

OctConf 2019
pending discussions and work

Doing

email maintainers mailing list with conclusions