Difference between revisions of "Code sprint: pkg.m"

From Octave
Jump to navigation Jump to search
 
(21 intermediate revisions by 2 users not shown)
Line 6: Line 6:
 
* [[User:KaKiLa|KaKiLa]] 07:34, 2 October 2012 (PDT)
 
* [[User:KaKiLa|KaKiLa]] 07:34, 2 October 2012 (PDT)
 
* [[User:Carandraug|carandraug]] 06:47, 3 October 2012 (PDT)
 
* [[User:Carandraug|carandraug]] 06:47, 3 October 2012 (PDT)
 +
* [[User:JordiGH|JordiGH]]
 +
* [[User:Carlo|cdf]]
  
= Schedule =
+
== Clone Carnë's repository ==
 +
https://bitbucket.org/carandraug/octave/changesets/tip/..bookmark%28%22pkg%22%29
 +
 
 +
= Parsing arguments passed to pkg =
 +
One of the most intimidating parts of pkg.m is the monstrous way the parsing of arguments is done. The current mayhem is probably due to the natural evolution of the code.
 +
the method use to parse the arguments is based on a flag system.
 +
 
 +
We started cleaning up the parsing of options to give some structure and make pkg.m easy to extend in the future. For this we focus on encapsulation.
 +
 
 +
== Definitions of actions, targets and modifiers ==
 +
A typical call to pkg looks like
 +
 
 +
  pkg install -forge pkg1 pkg2 pkg3 -j4
 +
 
 +
We can identify three types of input arguments
 +
* Action: install
 +
* Modifiers: -forge, -j4
 +
* Targets: pkg1, pkg2, pk3
 +
 
 +
So far we have defined them as follows
 +
* Action: The first input argument and can't start with a hyphen.
 +
* Modifier: Must start with a hyphen.
 +
* Target: Anything that is not an action nor a modifier
 +
 
 +
== Dispatching ==
 +
 
 +
Each action has its own set of valid modifiers and behaves accordingly, that is the function ''pkg'' does not parse targets nor modifiers. Therefore the rôle of the main function ''pkg'' is to call the right action once general sanity checks have been performed.
 +
 
 +
The particular implementation of this dispatch behavior was source of some discussion.
 +
* Using structure
 +
: One way of implementing the dispatch is using structures with fieldnames equal to the action and containing a function handle. [http://agora.octave.org/snippet/5UUz/ Here is an example].
 +
: Some people do not like this because they consider it a weird use of a structure.
 +
* Using ''feval''
 +
: Another way of doing it (and the implemented as of revision 2c16a852bae9 in carandraug's repository) is building a string of the function to call and use ''feval''. [http://agora.octave.org/snippet/ozym/ Here the example]
 +
* Using objects
 +
: Where we have an ''action'' class with fields valid_modifiers and method parse_modfiers. All ''_pkg'' actions inherit the general valid_modifiers and extend them with their own, as well as overloading the parser_modifier method. This gives the maximum flexibility, but since is too much of a change we haven't code it.
 +
 
 +
=== Action functions ===
 +
 
 +
The action functions have the suffix ''_pkg'' in their names. These functions should contain a cell of valid modifiers if they accept any. The function ''parse_target_modifier'' is called to check the validity of the modifiers and to separate them from targets. The cell of valid modifiers can contain regular expressions. This is required since we want to accept modifiers compatible with ''make'' flags (such as -jX for parallel compilation).
 +
 
 +
=== No globals ===
 +
We are trying to avoid global variables at all cost. Everytime a read-only value is needed in several functions, we create a helper function that returns the value.
 +
 
 +
We have to see how we are going to solve the issue with actions like 'global_list' (deprecated, now called 'global_db') that can change the pointer to the file containing the package database.
 +
 
 +
= Desired features =
 +
 
 +
== Reverse autoload function ==
 +
Though is not related with pkg.m, the function autoload can simplify considerably the PKG_* scripts. The idea is to give generate a function that reverses autoload.
 +
 
 +
* Change set [http://octave.1599824.n4.nabble.com/reverse-of-autoload-tt4646746.html committed to mailing list]. [[User:KaKiLa|KaKiLa]] 13:44, 17 November 2012 (PST)
  
= Objectives =
 
 
== Option update should behave as option install ==
 
== Option update should behave as option install ==
* It should parse a list of packages and maybe the same flags as install (-auto, -noauto, etc)
+
* It should parse a list of packages and maybe the same flags as install  
 +
* It should not stop updating if a package is not found (i.e. user local package, not in Forge).
 +
 
 +
=== Coders ===
 +
* [[User:KaKiLa|KaKiLa]] 04:42, 1 November 2012 (PDT)
 +
 
 +
== Integrate generate_html into Octave-core ==
 +
* The system used to generate pakage releases should be more closely integrated with the package manager
 +
* Maybe a "pkg release" command possibly with "pkg release -binary" option would be a good interface
 +
=== Coders ===
 +
* cdf
 +
 
 +
== Keep sources in installed packages ==
 +
* "pkg rebuild" should re-generate any .oct files, this would be useful when an Octave update breaks the API
 
=== Coders ===
 
=== Coders ===
  
 
== Remove -global and -local flags ==
 
== Remove -global and -local flags ==
* Those flags are currently ignored internally anyways so they should disappear
+
* Those flags are have shaky functionality so they should disappear
 
=== Coders ===
 
=== Coders ===
 
* Carnë has already a changeset for this
 
* Carnë has already a changeset for this
 +
 
== Allow packages to deprecate functions ==
 
== Allow packages to deprecate functions ==
 
* Create a system that allows packages to deprecate functions as in core. Possibilities are:
 
* Create a system that allows packages to deprecate functions as in core. Possibilities are:
Line 42: Line 108:
 
* Accept -url <url> to install files and packages directly from the web
 
* Accept -url <url> to install files and packages directly from the web
 
=== Coders ===
 
=== Coders ===
== Add helper functions ==
+
== Add where is functionality ==
 +
<code>pkg describe pkg_name</code> gives useful information. But we should allow for more direct queries like
 +
* <pre>pkg whereis function_name</pre> list the installed packages (loaded or not) that have a function matching the name function_name.
 +
* <pre>pkg whereis -forge function_name</pre> list all forge packages that have a function matching the name function_name.
 +
=== Coders ===
 +
* KaKiLa

Latest revision as of 13:16, 20 November 2012

This page is meant to organize and report the efforts to improve pkg.m

Participants[edit]

If you want to participate add your signature, your IRC nick or your sourceforge username.

Clone Carnë's repository[edit]

https://bitbucket.org/carandraug/octave/changesets/tip/..bookmark%28%22pkg%22%29

Parsing arguments passed to pkg[edit]

One of the most intimidating parts of pkg.m is the monstrous way the parsing of arguments is done. The current mayhem is probably due to the natural evolution of the code. the method use to parse the arguments is based on a flag system.

We started cleaning up the parsing of options to give some structure and make pkg.m easy to extend in the future. For this we focus on encapsulation.

Definitions of actions, targets and modifiers[edit]

A typical call to pkg looks like

 pkg install -forge pkg1 pkg2 pkg3 -j4

We can identify three types of input arguments

  • Action: install
  • Modifiers: -forge, -j4
  • Targets: pkg1, pkg2, pk3

So far we have defined them as follows

  • Action: The first input argument and can't start with a hyphen.
  • Modifier: Must start with a hyphen.
  • Target: Anything that is not an action nor a modifier

Dispatching[edit]

Each action has its own set of valid modifiers and behaves accordingly, that is the function pkg does not parse targets nor modifiers. Therefore the rôle of the main function pkg is to call the right action once general sanity checks have been performed.

The particular implementation of this dispatch behavior was source of some discussion.

  • Using structure
One way of implementing the dispatch is using structures with fieldnames equal to the action and containing a function handle. Here is an example.
Some people do not like this because they consider it a weird use of a structure.
  • Using feval
Another way of doing it (and the implemented as of revision 2c16a852bae9 in carandraug's repository) is building a string of the function to call and use feval. Here the example
  • Using objects
Where we have an action class with fields valid_modifiers and method parse_modfiers. All _pkg actions inherit the general valid_modifiers and extend them with their own, as well as overloading the parser_modifier method. This gives the maximum flexibility, but since is too much of a change we haven't code it.

Action functions[edit]

The action functions have the suffix _pkg in their names. These functions should contain a cell of valid modifiers if they accept any. The function parse_target_modifier is called to check the validity of the modifiers and to separate them from targets. The cell of valid modifiers can contain regular expressions. This is required since we want to accept modifiers compatible with make flags (such as -jX for parallel compilation).

No globals[edit]

We are trying to avoid global variables at all cost. Everytime a read-only value is needed in several functions, we create a helper function that returns the value.

We have to see how we are going to solve the issue with actions like 'global_list' (deprecated, now called 'global_db') that can change the pointer to the file containing the package database.

Desired features[edit]

Reverse autoload function[edit]

Though is not related with pkg.m, the function autoload can simplify considerably the PKG_* scripts. The idea is to give generate a function that reverses autoload.

Option update should behave as option install[edit]

  • It should parse a list of packages and maybe the same flags as install
  • It should not stop updating if a package is not found (i.e. user local package, not in Forge).

Coders[edit]

  • KaKiLa 04:42, 1 November 2012 (PDT)

Integrate generate_html into Octave-core[edit]

  • The system used to generate pakage releases should be more closely integrated with the package manager
  • Maybe a "pkg release" command possibly with "pkg release -binary" option would be a good interface

Coders[edit]

  • cdf

Keep sources in installed packages[edit]

  • "pkg rebuild" should re-generate any .oct files, this would be useful when an Octave update breaks the API

Coders[edit]

Remove -global and -local flags[edit]

  • Those flags are have shaky functionality so they should disappear

Coders[edit]

  • Carnë has already a changeset for this

Allow packages to deprecate functions[edit]

  • Create a system that allows packages to deprecate functions as in core. Possibilities are:
    • Get pkg to accept a deprecated directory inside the package and add it to the search path. Functions in those directories would have to be treated the same as the ones inside the core deprecated
    • PKG_ADD can be used to hack this. Package developers would still have to actually write the warnings on the function code but this would allow to have the functions in a separate directory so they don't foget to remove them on the next release
    • The package developer can also use something like Make to create a normal package from something that actually had a more complex structure, inclusive deprecated directories

Coders[edit]

Resolve packages dependencies automatically. Download and install[edit]

  • Get pkg to resolve dependencies automatically by downloading and installing them too.

Coders[edit]

Allow download and install of multiple versions of the same package[edit]

Coders[edit]

Make pkg.m more verbose by default[edit]

  • make the package just a bit more verbose by default

Coders[edit]

Support for more than one src directory(?)[edit]

Coders[edit]

Accept make flags[edit]

  • make pkg able to supply extra configure and make flags, useful for distributions, including -j for make

Coders[edit]

Remove auto option[edit]

Coders[edit]

Allow installation from given url[edit]

  • Accept -url <url> to install files and packages directly from the web

Coders[edit]

Add where is functionality[edit]

pkg describe pkg_name gives useful information. But we should allow for more direct queries like

  • pkg whereis function_name
    list the installed packages (loaded or not) that have a function matching the name function_name.
  • pkg whereis -forge function_name
    list all forge packages that have a function matching the name function_name.

Coders[edit]

  • KaKiLa