Jump to navigation Jump to search
9,946 bytes added ,  08:06, 11 June 2019
Warn about old, maybe outdated, content. File contains some dead links.
{{Warning|This page has not been revisited since 2014. Please refer to the GNU Octave manual for information about {{manual|pkg}}.}}
== Abstract ==
This OEP refers to Octave's design of the pkg system. The purpose of this system
This document attempts to design a solution for this.
The main idea is to have multiple databases with information from installed packagesin different locations in the filesystem. While this is similar to the current implementation,we plan to design solutions for when package installations clash. This proposal also suggests to keep the source of the solution packages. This will allow foreasy reinstall of packages (after an Octave upgrade) and test of .oct files frompackages (since their tests are in the .cc sources). == Rationale ==This design is meant to make allow the following:* keep database multiple versions of the same package installed side-by-side* keep multiple versions of Octave in a system using the same installed packages* deal with dependencies correctly when multiple Octave and packages co-exist* allow use of packages that may have been installed anywhere* reinstall a package* test installed packages See the user cases section below for several examples. The definition of a package manager according to wikipedia: * Verifying file checksums to ensure correct and complete packages;* Verifying digital signatures to authenticate the origin of packages;* Applying file archivers to manage encapsulated files ;* Upgrading software with each latest versions, typically from a software repository;* Grouping of packages by function to reduce user confusion;* Managing dependencies to ensure a package locationis installed with all packages it requires. This resolved the problem known as Dependency Hell. == Available vs Loaded ==To avoid problems reading this document, the distinction between available and dependenciesloadedpackage should be done early. An available package is a package that is currently available to pkg for loading,unloading or reinstall. It is already installed but not necessarily loaded. A loaded package is an installed package whose functions have been added to Octave'sfunction search path. == Types of package installs ==This design supports 3 types of package installations: global (relative to theOctave installation), local (user specific) and allow external (in any other place). ;global install: available from startup to everyone.;local install: available from startup only for the user that installed it.;external install: needs to be made available first. Octave install has no information about it. Note that Octave itself can be installed in some different ways. It might be a system-wideinstallation (located somewhere in {{Path|/usr/local/}} for example), a local installationof a normal user ({{Path|/home/user/anywhere}}), or installed in the homedirectory of a system user (anywhere really). === Global installs ===Packages installed globally will be available to everyone from startup. This is the merge type of such package installation that a system administrator would most likely do. Themeaning of global here is relative to the Octave installation though. If an Octaveinstallation is local (installed by a user in {{Path|~/my-builds}}), a global installationof a package will still place its filesin the home directory of the user (in{{Path|~/my-builds}}). A global installation is performed automatically if the user installing the packagehas write permissions to those directories (''localfcnfiledir'' and ''localapioctfiledir'').In case it has no permissions, a local package installation is performed instead.
To allow reinstallation of the === Local installs ===Local packagesare specific to a user. They are located in that user home directoryinto an {{Path|.octave}} directory. As with global package installations, they are availablefrom startup. Unlike global, we propose they are user specific, only available to keep the source of the packageuser thatinstalled it.A local install for a user can be an external install for some otherThis would also make it easier to run the tests of packagesuser.
This are the type of package installation done by users that want to have the latestpackage version before is available in their system repository, but are not going tobuild Octave themselves. Also to be used by those who run Octave in a system that theydo not maintain where Octave is installed but not packages. ===External installs = Rationale and examples ==This design These are like local packages but in a non-standard location. Octave does not knowabout this installations at startup even though they might have been installing thesame Octave that is meant to allow running at the following:moment. These can be packages installed in afilesystem that is not always mounted, local packages installs from another user * keeping multiple versions of in the same system, or anything else really. An external package was still installed and load with pkg, the difference being that therecord is not kept by Octave after it. An external package install will have a specific onedb * keep packages installed associated file just like the db files for multiple versions of Octavethe local installs. To load an externalpackage, specially in the case path for the db file needs to be passed to pkg and the db named (becausethere may be more than one db. These are most like the less used type ofpackages and will require a bit more .oct files which knowledge (they will need to point pkg to a .db file, that is all). They will be rebuilt mostly used for each octave functionplaces that develop their own packages and people who don't want to * reinstall install the package themselves, instead simply using a local install of others asan external package from . === Playing nice with downstream packagers ===The recommended method for installing Octave and its cache after installing packages is to use their OS packagingsystem. Downstream packagers should have the packaging systems make global installs of thepackages. If a user wants to install a new octave versionof a package that is not yet availableon its system repository, it should make a local package install (default since has a normal * run user he won't have write permissions to the tests from packages (find tests in Octave directory).cc sources) * clean If the user decides to make a global package cacheinstall (install the package using pkg whilerunning Octave with sudo), then he's trying to act as system administrator and should know * usage what he's doing. If he breaks it, its his own fault. Installation of alternate database filessystem-wide software * usage of packages in remote directories which may is meant to be handled by the system packaging tool. It is just not be available at possible to make pkgcover all timesof them.
== Package names ==
For the parsing of the commands and files, some limitations on package names are required. This willlimit what pkg commands can do. For example, if a package name is allowed to use scorea hyphen, then
commands such as "pkg load image-2.0.0" can no longer be used to load a specific package version.
Something such as "pkg load image::2.0.0" would have to be used. Using this alternative syntax
Also, supporting multiple packages versions means that the word "all" to refer to all
packages has new limitations. Should we load only the latest version of each package?
And if there's multiple packages with the same version on varios various db, which one should
be loaded? I'd propose the default to be:
- load the latest version availaleavailable
- load the local install of the package
- load the global install of the package
For package names, the proposal is to limit package names to the same as variable
names (makes it even easier to check validaity validity with isvarname). So package name
must start with a letter, and otherwise be comprised of alphanumeric and underscores
characters. Unlike variable names, package names will not be case sensitive since
it would create problems when installing packages in filesystems that are not case
sensitive (creating directories named Image and image would not be possible in FAT systems). == Version numbers == === specifying version ===Actions dependent on a package version can be specified with a -version modifier for thataction. It is however necessary to define the default order. Comparison operatorsshould be used to specify versions. If no comparison is use then greater than orequal is assumed. So that the following: ;pkg load image: loads latest version of the image package. If package is not installed, give error;pkg load -version 1.0.5 image: load the latest version greater than or equal to 1.0.5. If no such version found, give error;pkg load -version >=1.0.5 image: same as not specifying comparison;pkg load -version >1.0.5 image: load anything above that version (does it make sense supporting this? It's not a lot of trouble...);pkg load -version =1.0.5 image: load image package only if the same version (should we use == instead? Why not only =? Should not support both syntax);pkg load -version !1.0.5 image: load any image package available except 1.0.5 (because regressions do exist) For the other 2 remaining comparisons (< and <=), the question used for > and >=is the same. Does it make sense to support both? For ''greater than'', the onlything that makes sense is ''greater than or equal'' and for ''lesser than'', theonly think that makes sense is ''only lesser than'' since people will mark themas the first release that implemented, or the first release that no longer had,a specific feature. Whatever code is used on this section should also be used for solving packagedependencies. Should versions take precedence over the database for loading order? For example,if there is a global installation of image 1.0.5 and a 2.0.0 version on an externaldatabase named labdev, what version should be loaded? ;pkg load image: load version 1.0.5 from global (database takes precedence over version);pkg load -version >1.0.0 image: load version 1.0.5 from global (database takes precedence over version);pkg load -version >2.0.0 image: load version 2.0.0 from labdev (only version that meets the requirements);pkg load -version >1.0.0 -db labdev image: load version 2.0.0 from labdev (while database takes precedence, labdev was specified so we load the latest) Should the -db modifier make pkg ignore completely version? If a system has signalversion 1.0.0 on an external named labdev, and 1.2.0 on a global, what should be loaded? ;pkg load signal: load version 1.2.0 from global;pkg load -db labdev image: load latest version from global or from labdev? === version definition ===The current implementation only accepts versions on the format x.y.z. This doesnot allow for dev versions, beta or release candidates releases such x.y.z-rc0, x.y.z+, etc We have compare_versions in core to check for version numbers, whatever is decidedshould be used with compare_version (or HFS systemscompare_version should be made to support it).
== Types of package installs ==
== User cases ==
=== Case User case #1 : global, local and external ===Jenny is using Octave on the department cluster. She is not the administrator butthere's already a system-wide installation of Octave with the general andsignal image installed. She starts Octave and has these 2 packages available toher. These are globally installed packages, available to everyone that startsOctave. But Jenny also requires the image package and she installs it with "pkg install -forge image". Shedoes not have permissions to administer the system so the image package is installedlocally in her home directory. When she starts Octave, she now has 3 packages available,general and signal package which are global (available to everyone that starts Octave), andthe image package which is local (available only to her). Jenny's supervisor is working on a new package (img_analysis) that he makes availablefor all his students and wants Jenny to use it. Rather than sending them the packages,he wants them to use the package he has installed on his own home directory and tellsthem to load it as an external package. Jenny uses"pkg load-db boss /home/supervisor/.octave/octave_packages.db" to make his supervisorpackages available to her. She now has 4 available packages, the new one (img_analysis)being an external package. However, relative to her supervisor, the same package is alocal installation. The next time she starts Octave, there is no trace of the external packages, pkg stillonly have 3 available packages so she adds the "pkg load-db" command to her {{Path|.octaverc}}file. In this case however, her supervisor would do better in installing his img_analysis packagein some other place to avoid clash with his own local packages. For example, he couldhave installed it at {{Path|/home/supervisor/group/octave}}. Or he could have a filesystemon the network that his students could mount whenever they needed it. === User case #2: keeping tarball ===
Denise installs Octave 3.4.3 and installs the latest version of the financial (1.0.4) and
image (2.0.0) package with "pkg install -forge financial image". After installing the packages,
tests in the package (using the cached package to run the tests in the .cc files).
==== different package versions ====
Later, Denise installs Octave 3.6.2 but keeps the previous version of Octave on the
system since some of her old code no longer runs correctly. Loading the financial
While using Octave 3.6.2, Denise installs the new version of the package
"pkg install -forge financial". The files for the previous version of the package
are kept altough although "pkg load financial" will only load the latest version. However, when
Denise is using Octave 3.4.3, as financial 1.2.0 requires Octave 3.6.0, pkg load
will only load financial 1.0.4.
==== comments ==== shouldn't `rebuild` be used instead of `reinstall` ? === Case 2 User case #3: installing and loading different package versions ===
Owen is stuck using the financial package 1.0.4 because some of his code no
longer works in the latest versions. However the latest version of financial
while "pkg load financial" always loads the latest version of the package.
=== Case 3 User case #4: Local installation of packages and Octave ===
Lisa is using Octave in a remote machine on the biochemistry department. The
system administrator installed Octave 3.6.2, signal package 1.2.0, and
general 1.0.0. Lisas Lisa uses all of them but she also requires the image package.
However, the system administrator does not have time to access security issues
with the package and tells her to install that package locally. She runs "pkg
When Octave 3.6.3 is released, Lisa wants to use the new version since it fixes
one bug that has been aanoying annoying her for a long time but the system administrator
does not want to make the update and tells her to build it herself locally
=== Case 4 User case #5: users (no sudo) sharing Octave installation with local & global packages ===
Diana is a student that wants to run her code in the departmental cluster. However,
the system does not have an installation of Octave and she needs to install it on
to her own list of available packages. which she can load.
==== comments ==== User case #6:Why not store the "packages.db" together with the packages? instead of loading the a packages database file. Then, Diana could just say {{Codeline|pkg addpath ~Ligia/octave}}:: Because she might want to use some of her packages, not all. This adding the .db file to her instance of Octave will not load the package, she still needs to load it. And she may want to load only some of them. === Case 5 Automatic dependency tracking ===
John is a professor of biomechanics and uses Octave on his classes. Most of the
exercises he gives to the class require the use of multiple packages in Octave
metapackage for his student listing all required packages. The students install
it with "pkg install -url path-to-his-metapackage". The metapackage has no file
it simply lists a bunch of package has as dependencies. Since pkg solves this
dependencies automatically, a message showing which packages will be installed
is displayed before doing it.
=== User case #7: Package testing ==="pkg test" command that would run all tests for a given package. == Where to install things ===
These should not be hardcoded and taken from octave_config_info. There's many paths there whose purpose is explained on octave sources [ buil-aux/] (see the ''Where To Install Things'' and ''Octave-specific directories'' sections on that file.)

Navigation menu