Open main menu

Python interface

Revision as of 09:26, 29 May 2016 by Mtmiller (talk | contribs) (reorganize content)

There is a small project in development to bring a Python calling interface to Octave. The broad goal of this project is to add functions and types to Octave to allow calling Python functions directly from Octave.

Contents

Features

Features and capabilities of Octave's Python interface may include:

  • Import and call Python modules and functions from the Octave interpreter
  • Automatically convert basic Octave and Python types seamlessly between the two environments
  • Be able to handle arbitrary unknown Python objects (print their repr, store in a variable, pass back in to a Python function)
  • Store references to Python functions (and other "callables") and be able to call them as if they were function handles

Development

Project development is ongoing among a small group of developers. Communication takes place on the Octave maintainers mailing list. The official Mercurial repository is at http://hg.octave.org/pytave, but there is also a Bitbucket clone and a network of forks, for those who prefer that model of development, at https://bitbucket.org/mtmiller/pytave.

Documentation

The current development needs to be documented. We are using doxygen for the documentation.

Roadmap / Ideas

Python from Octave

The conversion of Python's dict is not unique. For that we have decided to load a Python's dict as a structure. This works only when all the keys fo the dict are strings. When the keys are something else there is the option to use `repr` to create the fields of the Octave's struct, e.g.

Code: Conversion of Python's dict to structure
> x = pyeval ("{1:'one',2:'two'}");
> x.("1")
ans = one

> x.("2")
ans = two

> x = pyeval ("{(1,1):'one',2.5:'two'}");
> x.("(1,1)")
ans = one

> x.("2.5")
ans = two

This would be the default behavior. We will extend pyeval to receive an optional argument specifying the Octave type that should be used as the output of the conversion, e.g. when the dict uses continuos numbers as keys one could do

Code: pyeval optional argument
> x = pyeval ("{1:'one',2:'two'}",@cell);
> x{1}
ans = one
> x{2}
ans = two

> x = pyeval ("{1:'one',2:'two'}",@char);
> x

x = 

one
two
 
> whos x
Variables in the current scope:

   Attr Name        Size                     Bytes  Class
   ==== ====        ====                     =====  ===== 
        x           2x3                          6  char

Total is 6 elements using 6 bytes

> x = pyeval ("[i for i in xrange(3)]",@double)
x =
0 1 2

The optional argument could be the constructor of an Octave class.

Octave view of Python

Currently we can do things like

Code: Objects in python exists across calls to pyexec and pyeval
pyexec ("import networkx as nx")
pyexec ("G=nx.complete_graph(10)")
x = pyeval ("G.nodes()")

But we cannot get a representation of G within Octave.

The idea is to have an object that maintains a pointer to an Python object and that has all the methods to convert that object to Octave.

The python object can be manipulated by python calls like pycall and pyexec without needing to do an explicit conversion. For example

Code: Octave object that points to a python object
1 pyexec ("import networkx as nx")
2 pyexec ("G=nx.complete_graph(10)")
3 G      = pyobj ("G") ## G is an object without Octave representation but with methods to do the conversion if required
4 nodes  = pycall ("nx.nodes",G) # Python nx.nodes(G)
5 nodes2 = pyeval ("G.nodes()")  # Equivalent

Comment: It seems that pyexec and pycall do not share workspace, so the code above wont work because nx doesn't exist when we call pycall at line 4.

The conversion methods should provide different output objects to Octave. The idea is to be able to call something like

Code: Octave view of Python object with conversion methods
G_cell   = cell(G);   # G converted to a cell
G_struct = struct(G); # G converted to a struct


Python Objects in Octave

The @pyobj classdef class is intended to wrap arbitrary Python objects so they can be accessed an manipulated from within Octave.

Code: avoiding garbage collection
1 pyexec('d = dict(one=1, two=2)')    # create object in Python
2 x = pyobj('d')                      # create pyobj wrapper for that object
3 pyexec('d = []')                    # careful, don't lose the object to the GC
4 x.keys()                            # list the keys of the dict
5 clear x                             # now the object could be GCed

One proposed way to do this:

1. `x` stores the pointer to `d`. The `@pyobj` ctor creates a dummy reference to `d`, this prevents GC

2. on deletion of x (`clear x`) we delete the dummy reference in Python.

Notes:

  • Seems like the relevant "pointer" is id(). Haven't seen yet how to access an object from its id, except that its a bad idea...
  • My plan to create a dict in Python, indexed by hex(id(x)), maybe called __InOct__. Then pass the id of the object to the @pyobj/pyobj constructor.
  • Rejected idea: store the `repr` as a string in `x`. But this makes a copy of the object rather than a reference to the original object.
Interface design
  • pyeval should be modified to return a @pyobj for things that it cannot convert to Octave-native types. See the `networkx` example above: `G` could be returned by `pyeval`.
  • @pyobj/pyobj constructor would not normally be called by users.

Pytave

This project is currently derived from an earlier project called Pytave, which was developed to work in the opposite direction, to allow Python to call Octave functions on an embedded Octave interpreter. The bulk of the project is in the code to convert between Octave and Python data types, so most of that is reusable and serves both purposes. As a side goal, we may continue to maintain the Python wrapper around Octave and incorporate that into Octave as well, so that Octave can provide its own native Python module.