JWE Project Ideas: Difference between revisions

Jump to navigation Jump to search
no edit summary
No edit summary
No edit summary
(5 intermediate revisions by the same user not shown)
Line 1: Line 1:
<!-- This file should be edited at https://wiki.octave.org/JWE_Project_Ideas -->
== Language and functions ==
=== classdef issues ===
==== Compatibility issues ====
Make a list here, pointing to individual bug reports?
==== Load/save for classdef ====
See also general load/save issues.
==== Improve / simplify implementation ====
Although the basic features that are implemented now appear to mostly work, the implementation seems overly complicated, making it difficult to debug and modify.  There seems to be quite a bit of room for improvement here.
=== Syntax, semantics, and data types ===
==== Function handle refactoring ====
* Load/save for all types of function handles and all data formats (ascii, binary, hdf5, mat5)
* Use std::shared_ptr for function objects instead of bare pointer to octave_function.
==== String class ====
Matlab now uses "" to create string objects that behave differently from Octave double-quoted strings.  We could start by creating a compatible string class, then hooking it up to the "" syntax.  No matter what, the transition will be difficult because Matlab's "" strings still treat "\n" as two characters (backslash and n) rather than a single character (newline).
==== Other new data types ====
Andrew Janke has implementations of these classes (FIX: link to repos here)
* table
* datetime, duration, calendarDuration
* categorical
* timetable
* timeseries
==== single / integer valued ranges ====
This is a compatibility issue.
==== Refactor load-path ====
* Directories are not properly removed from load path (FIX: link to bug report here)
* Should we really have ADD_PKG and DEL_PKG files?  If so, how can we make them safe?
==== Eliminate special matrix types ====
Although the special range, diagonal matrix, and permutation matrix data types in Octave require less memory than storing full matrices, they tend to cause trouble when people expect full compatibility or exactly the same results when performing arithmetic on Ranges vs. Matrices.  Now that we have broadcasting operators, the need for diagonal matrices is not as great.
==== Special case FOR loop limits ====
Currently, "for i = 1:N ..." uses a Range object for the "1:N" loop bounds.  If we eliminate Ranges as a special space-saving type, then we should handle this syntax as a special case.  Even if we don't eliminate Ranges, that might be a good idea, as we could handle "for i = 1:Inf ..." easily without having to worry about how to deal with that in an ordinary Range object vs. FOR loop bounds.
==== Local functions ====
The semantics for local functions in scripts is different from the
way Octave currently handles functions that are defined in script
files.
==== Matlab packages ====
+DIR directories in the loadpath; related to classdef
Octave already searches for files in package directories and
understands the PKG.fcn syntax and functionality.  The big missing
piece is implementation of the "import" functionality and handling
it efficiently and in a way that is compatible with Matlab.
==== Refactor broadcasting ====
Are there better ways to use templates to handle function calls rather than using macros to define a set of functions for array/array, array/scalar, and scalar/array ops as in DEFMXBINOP in mx-inlines.cc?
==== Sparse matrix issues ====
==== Broadcasting ====
Broadcasting does not work for sparse matrices.  This seems like a big missing feature.
==== Structural zeros ====
Octave currently skips structural zeros for most (all?) sparse matrix operations.  Matlab returns a sparse matrix filled with NaNs for something like "sprand (5, 5, 0.1) .^ NaN".  Should we go for full compatibility?  Mathematical correctness?  Traditional behavior of sparse matrix libraries?  It seems no one really agrees on what is correct or best.  Maybe compatibility should win?
==== Indexed assignment ====
In an assignment like Sparse_object(idx) = GrB_object(idx), Octave does not attempt to apply a conversion operator to transform the RHS type to the LHS type.  Is this also a problem for assignments of objects with conversion operators to full matrix objects?
==== graph and digraph ====
Would it be difficult to provide these objects?
== GUI ==
== GUI ==


=== Improve interface for communication between GUI and interpreter ===
=== Communication with interpreter ===


Currently, communication between the GUI and the interpreter
Currently, communication between the GUI and the interpreter
Line 27: Line 120:
== Graphics ==
== Graphics ==


=== Generating publication-quality figures ===
=== Publication-quality figures ===


Generating EPS or PDF versions of figures needs improvement.
Generating EPS or PDF versions of figures needs improvement.


=== OpenGL graphics system issues ===
=== OpenGL graphics ===


* Scaling plot data values/ranges to fit in single-precision OpenGL values
* Scaling plot data values/ranges to fit in single-precision OpenGL values
* Performance issues
* Performance issues
* Lack of WYSIWYG
* Lack of WYSIWYG
* Duplication of effort with FLTK and Qt widgets.  With the rest of
* With the GUI using Qt widgets, we should eliminate the FLTK plotting widget.  It duplicates functionality and requires additional effort to maintain.  Maybe we no longer need the octave-cli binary (the one that is not linked with Qt libraries)?


=== Threading Issues for Qt Graphics Toolkit ===
=== FLTK widget ===
 
With the rest of the GUI using Qt widgets, we should eliminate the FLTK plotting widget.  It duplicates functionality and requires additional effort to maintain.  Maybe we no longer need the octave-cli binary (the one that is not linked with Qt libraries)?
 
=== Qt toolkit threading ===


It seems likely that the locking of the gh_manager object is insufficient or even incorrect in some cases.
It seems likely that the locking of the gh_manager object is insufficient or even incorrect in some cases.


=== Use classdef for Graphics Objects ===
=== classdef graphics objects ===


This is a large project, but one that will likely have to be tackled at some point.
This is a large project, but one that will likely have to be tackled at some point.


== Language ==
=== Miscellaneous ===


=== Improvements to classdef (the Matlab object-oriented programming framework) ===
==== Handle UTF-8 ====


* Resolve remaining Matlab compatibility issues.
We need to handle UTF-8 (or whatever) characters properly in all parts of OctaveTry to do this in a Matlab-compatible way.
* Make it possible to load and save classdef objects.
* Improve and simplify the implementation.  Although the basic features that are implemented now appear to mostly work, the implementation seems overly complicated, making it difficult to debug and modifyThere seems to be quite a bit of room for improvement here.


=== String class ===
==== Load / save for large files ====


Matlab now uses "" to create string objects that behave differently from Octave double-quoted stringsWe could start by creating a compatible string class, then hooking it up to the "" syntax. No matter what, the transition will be difficult because Matlab's "" strings still treat "\n" as two characters (backslash and n) rather than a single character (newline).
* Make the load and save commands compatible with Matlab's HDF5-based file formatMatlab users expect this and we need something like this to support large arrays anyway.
* Phase out Octave's own text and binary formats.  Too much effort is required to maintain the code to support all the various formats.


=== Handle UTF-8 (or whatever) characters properly ===
==== RNG issues ====


Try to do this in a Matlab-compatible way.
RandStream and Other RNG issues


=== Handle Single and Integer Values for Ranges ===
This is likely a large project, but it would be nice to have updated, compatible interfaces.


This is a compatibility issue.
==== MEX Interface ====


=== Eliminate Special Range, Diagonal Matrix, and Permutation Matrix Data Types ===
Implement mxMakeReal and mxMakeComplex functions.


Although these data types in Octave require less memory than storing full matrices, they tend to cause trouble when people expect full compatibility or exactly the same results when performing arithmetic on Ranges vs. Matrices.  Now that we have broadcasting operators, the need for diagonal matrices is not as great.
==== JIT compiler ====


=== Use Special Case Instead of Range for FOR loops ===
A proof-of-concept implementation was done several years ago by a
Google Summer of Code student.  It was never complete and little
work has been done since.  It also depends on an old version of
LLVM.  In addition to LLVM, we should consider the JIT library
features of GCC.


Currently, "for i = 1:N ..." uses a Range object for the "1:N" loop bounds.  If we eliminate Ranges as a special space-saving type, then we should handle this syntax as a special case.  Even if we don't eliminate Ranges, that might be a good idea, as we could handle "for i = 1:Inf ..." easily without having to worry about how to deal with that in an ordinary Range object vs. FOR loop bounds.
This is probably the most difficult item (at least for me) since it
will require fairly advanced knowledge of compiler infrastructure
and Octave internals.


=== Local functions ===
==== loadlibrary ====


The semantics for local functions in scripts is different from the
This feature might be nice to have but it has a low priority.
way Octave currently handles functions that are defined in script
files.
 
=== Allow large files to be loaded and saved ===


* Make the load and save commands compatible with Matlab's HDF5-based file format.  Matlab users expect this and we need something like this to support large arrays anyway.
==== Complex integers ====
* Phase out Octave's own text and binary formats.  Too much effort is required to maintain the code to support all the various formats.


=== Matlab packages (+DIR directories in the loadpath; related to classdef) ===
Should we support this feature?  Should we refactor the implementation of array objects to make this job easier?


Octave already searches for files in package directories and
==== who -file option ====
understands the PKG.fcn syntax and functionality.  The big missing
piece is implementation of the "import" functionality and handling
it efficiently and in a way that is compatible with Matlab.


=== Broadcasting for Sparse Matrices ===
Should just read file and list info, not create dummy scope.  Likewise for whos function.


This seems like a big missing feature.
=== Maintenance and packaging ===


=== Refactor Broadcasting ===
==== General code quality ====


Are there better ways to use templates to handle function calls rather than using macros to define a set of functions for array/array, array/scalar, and scalar/array ops as in DEFMXBINOP in mx-inlines.cc?
* Use C++11 features where possible.
* Better and more complete use of C++ namespaces.
* Better use of C++ features.  Especially standard library features as their implementation becomes more widely available.  For example, we might be able to simplify some things in Octave by using the C++17 filesystem and special functions libraries, if they provide results that are at least as good what we are using now.
* Eliminate C preprocessor macros where possible
* added_static must go! (not sure about this now)
* Should not expose symbol_record in call_stack functions if possible
* Remove unused symbol_table/scope/record functions
* Use const in more parse tree functions
* Do recursive functions work properly with load/save now?
* Use enums for options internally (typically to replace bool values)
* Audit global variables and eliminate them where possible


=== Compatibility of .^ for Sparse Matrices ===
==== Symbol visibility ====


Octave currently skips structural zeros for most (all?) sparse matrix operations.  Matlab returns a sparse matrix filled with NaNs for something like "sprand (5, 5, 0.1) .^ NaN".
We really should be tagging the functions that we wish to export from shared libraries.


=== etree for Sparse Logical ===
==== Dispatch types for functions ====


Matlab's etree function appears to handle sparse logical arrays.
Search for "classes:" in sources to find the few current examples.


=== Type Conversion for Indexed Sparse Assignment ===
==== min/max nargin values ====


In an assignment like Sparse_object(idx) = GrB_object(idx), Octave does not attempt to apply a conversion operator to transform the RHS type to the LHS type.  Is this also a problem for assignments of objects with conversion operators to full matrix objects?
Should we do this, and allow the interpreter to automatically error when a function is given too few/many arguments?


=== 5th and 6th Outputs for dmperm ===
==== Toolboxes ====
 
Octave doesn't support those.
 
=== graph and digraph Functions ===
 
Would it be difficult to provide these?
 
=== RandStream and Other RNG issues ===
 
This is likely a large project, but it would be nice to have updated, compatible interfaces.
 
== Miscellaneous ==
 
=== MEX Interface Changes ===
 
Implement mxMakeReal and mxMakeComplex functions.
 
=== Toolboxes ===


Move some core toolboxes (communications, control systems, image
Move some core toolboxes (communications, control systems, image
Line 141: Line 225:
equations package have already been moved to Octave.
equations package have already been moved to Octave.


=== General code quality improvements ===
==== Documentation ====
 
* Use C++11 features where possible.
* Better and more complete use of C++ namespaces.
* Better use of C++ features.  Especially standard library features as their implementation becomes more widely available.  For example, we might be able to simplify some things in Octave by using the C++17 filesystem and special functions libraries, if they provide results that are at least as good what we are using now.
* Eliminate C preprocessor macros where possible
 
=== Documentation ===
 
* Continue to improve Doxygen documentation for Octave internals to make it easier for new contributors to understand the Octave code base.
 
=== JIT compiler ===
 
A proof-of-concept implementation was done several years ago by a
Google Summer of Code student.  It was never complete and little
work has been done since.  It also depends on an old version of
LLVM.  In addition to LLVM, we should consider the JIT library
features of GCC.


This is probably the most difficult item (at least for me) since it
* Docs for call stack with examples and illustrations
will require fairly advanced knowledge of compiler infrastructure
* Docs for lexer and parser with examples and illustrations
and Octave internals.
* Docs for fcn_info object
* Docs for load_path object
* Docs for classdef internals
* Docs for Qt graphics toolkit internals
* Docs for Qt GUI and communication with interpreter
* Improve other Doxygen docs for internals to make it easier for new contributors to understand the Octave code base.


=== Windows distribution ===
==== Windows distribution ====


Eliminate the following msys packages.  Some might be removed
Eliminate the following msys packages.  Some might be removed
Line 176: Line 248:
but the msys version of less depends on the msys termcap library.
but the msys version of less depends on the msys termcap library.


  bash       less       perl
* bash
  coreutils  libcrypt   regex
* coreutils
  diffutils  libiconv   sed
* diffutils
  dos2unix  libintl     tar
* dos2unix
  file      libmagic   termcap
* file
  findutils  libopenssl unzip
* findutils
  gawk      make       zip
* gawk
  grep      msys-core   wget
* grep
  gzip      patch      zlib
* gzip
* less
* libcrypt
* libiconv
* libintl
* libmagic
* libopenssl
* make
* msys-core
* patch
* perl
* regex
* sed
* tar
* termcap
* unzip
* wget
* zip
* zlib

Navigation menu