JWE Project Ideas: Difference between revisions

From Octave
Jump to navigation Jump to search
 
(27 intermediate revisions by 2 users not shown)
Line 1: Line 1:
=== Improve interface for communication between GUI and interpreter ===
<!-- This file should be edited at https://wiki.octave.org/JWE_Project_Ideas -->


Currently, communication between the GUI and the interpreter
The following are projects that I would like to work on, roughly prioritized by my level of interestI intend to provide expanded explanations for each of these items.
mostly happens when the interpreter is otherwise idle and waiting
for user input at the command prompt and the implementation is
somewhat complicatedWe need to determine whether this is the
best we can do, or if there is some other implementation that
would be more flexible and reliable.


=== GUI command window ===
== Comment parsing ==


The implementation of the GUI command window for Unix-like systems
Refactor comment handling in lexer and parser.
is a completely separate implementations from the one used on
Windows systems.  There should be only one, and the GUI should be
completely in charge of user input and output.  This will probably
require implementing some kind of simple output pager internally
instead of using an external program, but overall user interaction
could be improved.


=== Interrupt handling in the GUI ===
* Gather and attach all comments to tokens in the lexer, never in the parser.  This change will allow us to simplify the grammar by eliminating the stash_comment nonterminal symbol in the parser.
* Store tokens (with location info and comments attached) in the parse tree instead of imprecise data like "leading_comment" or "trailing_comment".  This change will allow better location info for error reporting and easier and more accurate access to comments so that we can more easily find documentation strings or test and demo blocks (some of that work is already done).


This issue is related to the GUI command window.  Interrupt
== MException object ==
signals (typically generated by typing Control-C at the command
prompt) cause some trouble with the GUI and when multiple threads
are active, particularly inside of library code like the BLAS.
There are a number of bug reports for this problem.  We need to
find a way to reliably interrupt the interpreter.


=== Generating publication-quality figures ===
* Provide a mostly Matlab-compatible `MException` object.
* Make the `MException` constructor a built-in function (this job may require some changes to the classdef implementation).
* Fix the Octave interpreter to create an `MException` object when it throws an error and make that object available in `catch` blocks.


Generating EPS or PDF versions of figures needs improvement.
== FTLK graphics widget ==


=== OpenGL graphics system issues ===
Eliminate fltk graphics widget or move to external package.


* Scaling plot data values/ranges to fit in single-precision OpenGL values
== gnuplot graphics widget ==
* Performance issues
 
* Lack of WYSIWYG
Eliminate gnuplot graphics widget or move to external package.
* Duplication of effort with FLTK and Qt widgets. With the rest
 
of the GUI using Qt widgets, we should eliminte the FLTK plotting
== Function objects ==
widget.  To do that, we will need to make the Qt plotting widget
 
work when Octave is started with --no-gui and ensure that all
Refactor function objects.
features in the FLTK widget are also present in the Qt widget.
 
== arguments blocks ==
 
Finish implementation of Matlab-compatible argument validation blocks.
 
== local functions ==
 
Implement Matlab-compatible local functions in script files.
 
== string object ==
 
== load_path class ==


=== Improvements to classdef (the Matlab object-oriented programming framework) ===
Refactor (or rewrite) load_path class.


* Resolve remaining Matlab compatibility issues.
== Broadcasting ==
* Make it possible to load and save classdef objects.
* Improve and simplify the implementation.  Although the basic
  features that are implemented now appear to mostly work, the
  implementation seems overly complicated, making it difficult to
  debug and modify.  There seems to be quite a bit of room for
  improvement here.


=== String class ===
* Refactor broadcasting.
* Make broadcasting work for sparse matrices.


* Matlab now uses "" to create string objects that behave
== GUI command widget ==
  differently from Octave double-quoted strings.


=== Handle UTF-8 (or whatever) characters properly ===
Make common command widget for Windows and Unixy systems work well enough to become the default command line interface for the GUI.


* Try to do this in a Matlab-compatible way.
== OpenGL graphics ==


=== Handle single and integer values for ranges ===
* Modernize our use of OpenGL graphics to use shader programs instead of the legacy OpenGL API.
* Scaling plot data values/ranges to fit in single-precision OpenGL values


=== Local functions ===
== classdef ==


The semantics for local functions in scripts is different from the
Refactor (or rewrite) classdef implementation.
way Octave currently handles functions that are defined in script
files.


=== Allow large files to be loaded and saved ===
== Qt graphics widget ==


Make the load and save commands compatible with Matlab's
Refactor (or rewrite) Qt graphics widget.
HDF5-based file format.  Matlab users expect this and we need
something like this to support large arrays anyway.


=== Matlab packages (+DIR directories in the loadpath; related to classdef) ===
== graphics properties ==


Octave already searches for files in package directories and
Refactor graphics properties classes.
understands the PKG.fcn syntax and functionality.  The big missing
piece is implementation of the "import" functionality and handling
it efficiently and in a way that is compatible with Matlab.


=== Toolboxes ===
== graphics threading issues ==


Move some core toolboxes (communications, control systems, image
Fix handling of graphics properties to be properly thread safe.
processing, optimization, signal processing, and statistics), to
core Octave so development is managed along with Octave.  Core
Octave developers are already responsible for these packages
anyway, and users don't seem to understand why they need to
install them separately.  Core parts of the ordinary differential
equations package have already been moved to Octave.


=== General code quality improvements ===
== HDF5 load and save ==


* Use C++11 features where possible.
Implement Matlab compatible HDF5-based load and save functions.
* Better and more complete use of C++ namespaces.
 
* Better use of C++ featuresEspecially standard library features
* Make the load and save commands compatible with Matlab's HDF5-based file format.  Matlab users expect this and we need something like this to support large arrays anyway.  As much as possible, the initial implementation should be written in Octave's scripting language and the proposed Low-level interface to HDF5 functions so that it can easily be updated and patched as needed while we are still working out the details.  Only later should we consider translating performance-critical parts to C++, and then, only if really necessary.
  as their implementation becomes more widely availableFor
* Phase out Octave's own text and binary formats.  Too much effort is required to maintain the code to support all the various formats.
  example, we might be able to simplify some things in Octave by
* Low-level interface to HDF5 functionsCreate a thin wrapper for the HDF5 library.  As much as possible, make it compatible with the [https://www.mathworks.com/help/matlab/low-level-functions.html Matlab interface to HDF5]However, we may support newer functions (as of 2020/10/30, the list of Matlab functions appears to correspond to an older version of the library than is presently available in the HDF5 library itself) and support for legacy functions has a low priority.
  using the C++17 filesystem and special functions libraries, if
  they provide results that are at least as good what we are using
  now.


=== Eliminate C preprocessor macros where possible ===
== External editors ==


=== GUI code editor ===
Make it possible to use external editors such as Emacs, vim, or others with the GUI in addition to Octave's built-in code editor.


Make it possible to use external editors such as Emacs, vim, or
== who -file ==
others with the GUI in addition to Octave's built-in code editor


=== Documentation ===
Fix who -file to just read file and list info, not create dummy scope.


* Continue to improve Doxygen documentation for Octave internals to
== import ==
  make it easier for new contributors to understand the Octave code
  base.


=== JIT compiler ===
Make "import" work in a matlab-compatible way ==


A proof-of-concept implementation was done several years ago by a
== Code quality ==
Google Summer of Code student.  It was never complete and little
work has been done since.  It also depends on an old version of
LLVM.  In addition to LLVM, we should consider the JIT library
features of GCC.


This is probably the most difficult item (at least for me) since it
* Better and more complete use of C++ namespaces.
will require fairly advanced knowledge of compiler infrastructure
* Better use of C++ features. Especially standard library features as their implementation becomes more widely available. For example, we might be able to simplify some things in Octave by using the C++17 filesystem and special functions libraries, if they provide results that are at least as good what we are using now.
and Octave internals.
* Eliminate C preprocessor macros where possible.
* Use const in more parse tree functions.
* remove, replace, or at least rename the "added static" concept in the symbol_record class.
* Should not expose symbol_record in call_stack functions if possible.
* Remove unused symbol_table/scope/record functions.
* Do recursive functions work properly with load/save now?
* Use enums for options internally (typically to replace bool values).
* Audit global variables and eliminate them where possible.
* Audit use of panic_* functions and replace with calls to error where possible.
* Fix symbol visibility so we are mostly tagging namespace decls, not individual functions.
* Complete use of dispatch types for functions (search for "classes:" to find the few current examples).
* Tag for built-in functions to specify maxiumum number of inputs.


=== Windows distribution ===
== Windows distribution ==


Eliminate the following msys packages.  Some might be removed
Eliminate the following msys packages.  Some might be removed
Line 140: Line 121:
but the msys version of less depends on the msys termcap library.
but the msys version of less depends on the msys termcap library.


  bash       less       perl
* bash
  coreutils  libcrypt   regex
* coreutils
  diffutils  libiconv   sed
* diffutils
  dos2unix  libintl     tar
* dos2unix
  file      libmagic   termcap
* file
  findutils  libopenssl unzip
* findutils
  gawk      make       zip
* gawk
  grep      msys-core   wget
* grep
  gzip      patch      zlib
* gzip
* less
* libcrypt
* libiconv
* libintl
* libmagic
* libopenssl
* make
* msys-core
* patch
* perl
* regex
* sed
* tar
* termcap
* unzip
* wget
* zip
* zlib

Latest revision as of 18:56, 27 March 2024


The following are projects that I would like to work on, roughly prioritized by my level of interest. I intend to provide expanded explanations for each of these items.

Comment parsing

Refactor comment handling in lexer and parser.

  • Gather and attach all comments to tokens in the lexer, never in the parser. This change will allow us to simplify the grammar by eliminating the stash_comment nonterminal symbol in the parser.
  • Store tokens (with location info and comments attached) in the parse tree instead of imprecise data like "leading_comment" or "trailing_comment". This change will allow better location info for error reporting and easier and more accurate access to comments so that we can more easily find documentation strings or test and demo blocks (some of that work is already done).

MException object

  • Provide a mostly Matlab-compatible `MException` object.
  • Make the `MException` constructor a built-in function (this job may require some changes to the classdef implementation).
  • Fix the Octave interpreter to create an `MException` object when it throws an error and make that object available in `catch` blocks.

FTLK graphics widget

Eliminate fltk graphics widget or move to external package.

gnuplot graphics widget

Eliminate gnuplot graphics widget or move to external package.

Function objects

Refactor function objects.

arguments blocks

Finish implementation of Matlab-compatible argument validation blocks.

local functions

Implement Matlab-compatible local functions in script files.

string object

load_path class

Refactor (or rewrite) load_path class.

Broadcasting

  • Refactor broadcasting.
  • Make broadcasting work for sparse matrices.

GUI command widget

Make common command widget for Windows and Unixy systems work well enough to become the default command line interface for the GUI.

OpenGL graphics

  • Modernize our use of OpenGL graphics to use shader programs instead of the legacy OpenGL API.
  • Scaling plot data values/ranges to fit in single-precision OpenGL values

classdef

Refactor (or rewrite) classdef implementation.

Qt graphics widget

Refactor (or rewrite) Qt graphics widget.

graphics properties

Refactor graphics properties classes.

graphics threading issues

Fix handling of graphics properties to be properly thread safe.

HDF5 load and save

Implement Matlab compatible HDF5-based load and save functions.

  • Make the load and save commands compatible with Matlab's HDF5-based file format. Matlab users expect this and we need something like this to support large arrays anyway. As much as possible, the initial implementation should be written in Octave's scripting language and the proposed Low-level interface to HDF5 functions so that it can easily be updated and patched as needed while we are still working out the details. Only later should we consider translating performance-critical parts to C++, and then, only if really necessary.
  • Phase out Octave's own text and binary formats. Too much effort is required to maintain the code to support all the various formats.
  • Low-level interface to HDF5 functions. Create a thin wrapper for the HDF5 library. As much as possible, make it compatible with the Matlab interface to HDF5. However, we may support newer functions (as of 2020/10/30, the list of Matlab functions appears to correspond to an older version of the library than is presently available in the HDF5 library itself) and support for legacy functions has a low priority.

External editors

Make it possible to use external editors such as Emacs, vim, or others with the GUI in addition to Octave's built-in code editor.

who -file

Fix who -file to just read file and list info, not create dummy scope.

import

Make "import" work in a matlab-compatible way ==

Code quality

  • Better and more complete use of C++ namespaces.
  • Better use of C++ features. Especially standard library features as their implementation becomes more widely available. For example, we might be able to simplify some things in Octave by using the C++17 filesystem and special functions libraries, if they provide results that are at least as good what we are using now.
  • Eliminate C preprocessor macros where possible.
  • Use const in more parse tree functions.
  • remove, replace, or at least rename the "added static" concept in the symbol_record class.
  • Should not expose symbol_record in call_stack functions if possible.
  • Remove unused symbol_table/scope/record functions.
  • Do recursive functions work properly with load/save now?
  • Use enums for options internally (typically to replace bool values).
  • Audit global variables and eliminate them where possible.
  • Audit use of panic_* functions and replace with calls to error where possible.
  • Fix symbol visibility so we are mostly tagging namespace decls, not individual functions.
  • Complete use of dispatch types for functions (search for "classes:" to find the few current examples).
  • Tag for built-in functions to specify maxiumum number of inputs.

Windows distribution

Eliminate the following msys packages. Some might be removed entirely if they are unnecessary for running Octave or building Octave Forge packages. Otherwise, we should be building them from source as we do all other tools and libraries that are distributed with Octave. The difficulty is that although the msys packges are typically based on old versions of these packages, they sometimes have fixes that are needed to allow them to run properly on Windows systems. Note also that we distribute a termcap library, but the msys version of less depends on the msys termcap library.

  • bash
  • coreutils
  • diffutils
  • dos2unix
  • file
  • findutils
  • gawk
  • grep
  • gzip
  • less
  • libcrypt
  • libiconv
  • libintl
  • libmagic
  • libopenssl
  • make
  • msys-core
  • patch
  • perl
  • regex
  • sed
  • tar
  • termcap
  • unzip
  • wget
  • zip
  • zlib