Online Developer Meeting (2020-11-10): Difference between revisions

From Octave
Jump to navigation Jump to search
(Announce next meeting.)
 
(9 intermediate revisions by 2 users not shown)
Line 8: Line 8:
=== The future of Octave Packages ===
=== The future of Octave Packages ===
: ''See also [https://octave.discourse.group/t/online-developer-meeting-2020-11-10-the-future-of-octave-packages/349 Discourse].''
: ''See also [https://octave.discourse.group/t/online-developer-meeting-2020-11-10-the-future-of-octave-packages/349 Discourse].''
==== Key points to discuss (Kai) ====


  [[Octave Forge]] packages ⊂ [https://octave.org/doc/v5.2.0/Creating-Packages.html Octave] packages
  [[Octave Forge]] packages ⊂ [https://octave.org/doc/v5.2.0/Creating-Packages.html Octave] packages
* Creating, maintaining, installing Octave packages must become more simple/fun.
* Octave Forge needs more automation ([[#pkg-index]]) or manpower.
* Guide new package contributors (remove old, stale, irritating complex documentation).


==== The Octave Forge legacy ====
==== The Octave Forge legacy ====
Line 33: Line 39:
* Website looks great.
* Website looks great.
* No major updates (necessary) since 2018 http://hg.code.sf.net/p/octave/project-web
* No major updates (necessary) since 2018 http://hg.code.sf.net/p/octave/project-web
* Useful [https://octave.sourceforge.io/docs.php Function List]. '''How to update this {{bug|53860}}?'''
* Useful [https://octave.sourceforge.io/docs.php Function List]. '''How to update this {{bug|53860}}?''' Maybe "generate_html" package.
* Original developer (Oliver) no longer active.
* Original developer (Oliver) no longer active.
* Adding and updating packages here is expensive/dangerous
* Adding and updating packages here is expensive/dangerous
Line 68: Line 74:
When reporting an Octave (core) bug on https://savannah.gnu.org/bugs/?group=octave you are "greeted" with '''1523 open bug reports''' of which [https://savannah.gnu.org/bugs/?group=octave&func=browse&set=custom&msort=0&status_id=1&resolution_id=0&submitted_by=0&assigned_to=0&category_id=108&bug_group_id=0&severity=0&priority=0&summary=&details=&advsrch=0&msort=0&chunksz=100&spamscore=5&report_id=101&sumORdet=0&morder=bug_id%3C&sumOrdet=0&offset=0#results '''330 reports (22%)'''] belong to Octave Forge packages and '''240 (15%) are older than a year'''.
When reporting an Octave (core) bug on https://savannah.gnu.org/bugs/?group=octave you are "greeted" with '''1523 open bug reports''' of which [https://savannah.gnu.org/bugs/?group=octave&func=browse&set=custom&msort=0&status_id=1&resolution_id=0&submitted_by=0&assigned_to=0&category_id=108&bug_group_id=0&severity=0&priority=0&summary=&details=&advsrch=0&msort=0&chunksz=100&spamscore=5&report_id=101&sumORdet=0&morder=bug_id%3C&sumOrdet=0&offset=0#results '''330 reports (22%)'''] belong to Octave Forge packages and '''240 (15%) are older than a year'''.


* They are not likely to be closed or worked on and leave a maintenance burden and bad impression (user requests for unmaintained packages are ignored there) back to Octave (core).
* They are not likely to be closed or worked on and leave a maintenance burden and bad impression (user requests for unmaintained packages are ignored there) back to Octave (core).  A recent example of such user frustration {{bug|60048}}.


Ideas:
<strike>Ideas:


* Ignore and collect requests to infinity?
* Ignore and collect requests to infinity?
* Close bugs of inactive packages older than X years?  (Drop the illusion, that someone ever will fix it.)
* Close bugs of inactive packages older than X years?  (Drop the illusion, that someone ever will fix it.)
* Octave Forge suggests to [https://octave.sourceforge.io/support-help.php use the Savannah Bug tracker] for bug reports.  Drop this announcement?
* Octave Forge suggests to [https://octave.sourceforge.io/support-help.php use the Savannah Bug tracker] for bug reports.  Drop this announcement?
* Reopen the [https://sourceforge.net/p/octave/bugs/ old Octave Forge bug tracker]?
* Reopen the [https://sourceforge.net/p/octave/bugs/ old Octave Forge bug tracker]?</strike> (see below)


===== MXE-Octave =====
===== MXE-Octave =====


* The MS Windows installer [[Octave for Microsoft Windows#Packages|bundles 47]] Octave packages.
* The MS Windows installer [[Octave for Microsoft Windows#Packages|bundles 47]] Octave packages.
* Are there criteria for including packages in the Windows installer? '''No.'''
* Forced updates to the packages? '''No.'''
* Make criterion to drop package from installer if it no longer compiles "normally" (document somewhere)? Depends on decision of mxe-octave.
* Document how to request package inclusion in the MS Windows installer (open bug report, Discourse, manual)? wiki
* About 26 patches necessary to achieve this.
* About 26 patches necessary to achieve this.


Line 108: Line 118:
  of-tisean-1-fixes.patch
  of-tisean-1-fixes.patch
  of-video-1-fixes.patch
  of-video-1-fixes.patch
* Forced updates to the packages?
* Make criterion to drop package from installer if it no longer compiles "normally" (document somewhere)?
* Document how to request package inclusion in the MS Windows installer (open bug report, Discourse, manual)?


==== New directions ====
==== New directions ====
Line 122: Line 128:
* Super set of Octave Forge (all packages in latest version included)
* Super set of Octave Forge (all packages in latest version included)
* Easy to add, update and delete entries (just one file of meta data)
* Easy to add, update and delete entries (just one file of meta data)
* Advertise pkg-index as (only) major platform for Octave packages?
* Advertise pkg-index as (only) major platform for Octave packages? '''Not exclusive, but stronger highlight approach.'''


* Future development:
* Future development:
Line 133: Line 139:
* Alternatives exist [https://github.com/apjanke/octave-packajoozle octave-packajoozle (pkj)]
* Alternatives exist [https://github.com/apjanke/octave-packajoozle octave-packajoozle (pkj)]


* General design: '''Making pkg an Octave package?'''
* General design: '''Making pkg an Octave package?''' (see below)
** Changes to pkg or the package format can be applied to old Octave versions.
** Changes to pkg or the package format can be applied to old Octave versions.
** Multiple pkg tools can be developed.
** Multiple pkg tools can be developed.
Line 141: Line 147:
* Documented [https://octave.org/doc/v5.2.0/Creating-Packages.html in the Octave manual]
* Documented [https://octave.org/doc/v5.2.0/Creating-Packages.html in the Octave manual]
* In general good Octave programming project organization.
* In general good Octave programming project organization.
* Permitting "DESCRIPTION.md" or "DESCRIPTION.txt" extensions? (Nice highlighting in many modern source code hosting platforms.)
* Permitting "DESCRIPTION.md" or "DESCRIPTION.txt" extensions? (Nice highlighting in many modern source code hosting platforms.) '''Not decided.'''


=== General questions, open talk ===
=== General questions, open talk ===
Line 152: Line 158:
** Grace period after public email on the mailing-list?
** Grace period after public email on the mailing-list?
** Forbidden at all, only package forking?
** Forbidden at all, only package forking?
* Where to host package documentation?
** doc folder, wiki, in the repo itself (pkg-example README.md)?
=== Some outcomes of the meeting ===
* The responsibility for a package is with the maintainer, not Octave (core).
** Okay to continue the use of Savannah for current Octave Forge packages, but not for pkg-index.
* pkg-index
** How to treat malicious code/packages?  User reports, package removal.  Depending on the actual case.
* pkg tool as package
** If synchronized with Octave core, no usage of "if version ... else ..." code. (edit: I can't remember if we decided that this was a no-go. Just something that might be worth considering... -- [[User:Mmuetzel|Mmuetzel]] ([[User talk:Mmuetzel|talk]]))
** Watch out for problems with encodings (treated differently in older Octave versions) (edit: I'm no longer sure this is the case. I might have been thinking of `test` instead of `pkg`. I'm sorry for any confusion that might have caused. -- [[User:Mmuetzel|Mmuetzel]] ([[User talk:Mmuetzel|talk]]))
** Treatment of <code>pkg install -forge</code> if package moved away from Octave Forge
* documentation
** mxe-octave: necessary steps to include a package
*** only necessary if depending on external library, otherwise should install properly


== Ideas for next meeting ==
== Ideas for next meeting ==
Line 165: Line 190:
== See also ==
== See also ==


* Next meeting: TBA
* Next meeting: [[Online Developer Meeting (2021-03-23)]]
* Last meeting: [[Online Developer Meeting (2020-10-27)]]
* Last meeting: [[Online Developer Meeting (2020-10-27)]]


[[Category:2020]]
[[Category:2020]]
[[Category:Meetings]]
[[Category:Meetings]]

Latest revision as of 08:54, 23 March 2021

Todays topics[edit]

  • Meet and greet 5 minutes before meeting (audio testing).

The future of Octave Packages[edit]

See also Discourse.

Key points to discuss (Kai)[edit]

Octave Forge packages ⊂ Octave packages
  • Creating, maintaining, installing Octave packages must become more simple/fun.
  • Octave Forge needs more automation (#pkg-index) or manpower.
  • Guide new package contributors (remove old, stale, irritating complex documentation).

The Octave Forge legacy[edit]

Facts and figures[edit]
  • https://octave.sourceforge.io/packages.php
    • 50 community packages
    • 21 external packages
    • several unmaintained / renamed packages
    • 25 new package releases (40 in total) for 2020.
    • about 5 do not work with Octave 5.2 and about 12 will not work in Octave 6.1 (see link)
  • 5 request for new packages in the last two years (2 approved).
    • Octave Forge is not attractive, why?
https://octave.sourceforge.io/[edit]
  • Website looks great.
  • No major updates (necessary) since 2018 http://hg.code.sf.net/p/octave/project-web
  • Useful Function List. How to update this #53860? Maybe "generate_html" package.
  • Original developer (Oliver) no longer active.
  • Adding and updating packages here is expensive/dangerous
    • manual SFTP uploads
    • no version control for the individual package documentation
    • approx. 5-10 minutes per package release.
https://sourceforge.net/projects/octave/[edit]
  • FLOSS software based on Apache Allura
  • One of the last hosting services offering Mercurial (hg) support
  • Slow and many commercials, buggy UI.
Clumsy package development / releasing[edit]
  • The procedure is documented in the wiki: Reviewing Octave Forge packages
    • Experience of Kai after a few releases:
      • An "easy release" with basic installation/functionality checking/uploading takes me about 20-30 minutes.
      • If something is wrong with the tarballs another 20-30 minutes.
      • Version numbers are tracked in too many locations (DESCRIPTION, configure.ac, ...)
      • Octave Forge claims "high quality"
        • No OF admin can enforce it within 20-30 minutes (e.g. lack of package domain knowledge).
        • Users find many bugs despite this "high quality" release check procedures.
        • Why not just automatically release packages (pkg-index)?
Savannah[edit]

When reporting an Octave (core) bug on https://savannah.gnu.org/bugs/?group=octave you are "greeted" with 1523 open bug reports of which 330 reports (22%) belong to Octave Forge packages and 240 (15%) are older than a year.

  • They are not likely to be closed or worked on and leave a maintenance burden and bad impression (user requests for unmaintained packages are ignored there) back to Octave (core). A recent example of such user frustration #60048.

Ideas:

  • Ignore and collect requests to infinity?
  • Close bugs of inactive packages older than X years? (Drop the illusion, that someone ever will fix it.)
  • Octave Forge suggests to use the Savannah Bug tracker for bug reports. Drop this announcement?
  • Reopen the old Octave Forge bug tracker? (see below)
MXE-Octave[edit]
  • The MS Windows installer bundles 47 Octave packages.
  • Are there criteria for including packages in the Windows installer? No.
  • Forced updates to the packages? No.
  • Make criterion to drop package from installer if it no longer compiles "normally" (document somewhere)? Depends on decision of mxe-octave.
  • Document how to request package inclusion in the MS Windows installer (open bug report, Discourse, manual)? wiki
  • About 26 patches necessary to achieve this.
of-communications-1-catop.patch
of-fits-1-cross-fixes.patch
of-fits-2-fixes.patch
of-fl-core-1-fixes.patch
of-gsl-1-cross-fixes.patch
of-interval-1-cross-fixes.patch
of-ltfat-1-cross-fixes.patch
of-nurbs-1-fixes.patch
of-nurbs-2-dev-fixes.patch
of-ocs-1-cross-fixes.patch
of-ocs-2-dev-fixes.patch
of-ocs-3-break-fixes.patch
of-ocs-4-pkgadd-fixes.patch
of-ocs-5-no-odepkg.patch
of-odepkg-1-fixes.patch
of-odepkg-2-fixes.patch
of-odepkg-3-deprecated.patch
of-quaternion-1-cross-fixes.patch
of-quaternion-2-dev-fixes.patch
of-sockets-1-cross-fixes.patch
of-sockets-2-deprecated.patch
of-specfun-1-deprecated.patch
of-statistics-1-cross.patch
of-strings-1-fixes.patch
of-tisean-1-fixes.patch
of-video-1-fixes.patch

New directions[edit]

pkg-index[edit]
  • Super set of Octave Forge (all packages in latest version included)
  • Easy to add, update and delete entries (just one file of meta data)
  • Advertise pkg-index as (only) major platform for Octave packages? Not exclusive, but stronger highlight approach.
  • Future development:
    • Automated quality checks (dead URLs, pkg installation problems)
pkg tool[edit]
  • Current pkg very interleaved with / limited by Octave Forge -forge
    • Abstracting package registry / index?
  • Alternatives exist octave-packajoozle (pkj)
  • General design: Making pkg an Octave package? (see below)
    • Changes to pkg or the package format can be applied to old Octave versions.
    • Multiple pkg tools can be developed.
Octave package format[edit]
  • Documented in the Octave manual
  • In general good Octave programming project organization.
  • Permitting "DESCRIPTION.md" or "DESCRIPTION.txt" extensions? (Nice highlighting in many modern source code hosting platforms.) Not decided.

General questions, open talk[edit]

  • How to teach people that creating an Octave package is as simple as pkg-example?
  • Packages must become smaller?
    • Split large packages into different functionalities?
  • Octave Forge package maintenance overtaking (inactive packages)?
    • Authorship can never be "overtaken".
    • Grace period after public email on the mailing-list?
    • Forbidden at all, only package forking?
  • Where to host package documentation?
    • doc folder, wiki, in the repo itself (pkg-example README.md)?

Some outcomes of the meeting[edit]

  • The responsibility for a package is with the maintainer, not Octave (core).
    • Okay to continue the use of Savannah for current Octave Forge packages, but not for pkg-index.
  • pkg-index
    • How to treat malicious code/packages? User reports, package removal. Depending on the actual case.
  • pkg tool as package
    • If synchronized with Octave core, no usage of "if version ... else ..." code. (edit: I can't remember if we decided that this was a no-go. Just something that might be worth considering... -- Mmuetzel (talk))
    • Watch out for problems with encodings (treated differently in older Octave versions) (edit: I'm no longer sure this is the case. I might have been thinking of `test` instead of `pkg`. I'm sorry for any confusion that might have caused. -- Mmuetzel (talk))
    • Treatment of pkg install -forge if package moved away from Octave Forge
  • documentation
    • mxe-octave: necessary steps to include a package
      • only necessary if depending on external library, otherwise should install properly

Ideas for next meeting[edit]

Topic suggestions[edit]

Octave 7[edit]

  • Not only providing major releases that "fix Matlab incompatibilities".
  • Great new features for a great new release.
  • Code sprints.

See also[edit]