Python: module boopak.collect

boopak.collect | index boopak/collect.py

`collect: The PackageCollection class.
This module contains PackageCollection, which is responsible for installing
and updating the Boodler packages in your collection. It is a subclass
of PackageLoader.`

Modules
`` |  |
boopak.fetch os | boopak.pload boopak.version | zipfile |

Classes
`` |  | boopak.pload.PackageLoader PackageCollection
class PackageCollection(boopak.pload.PackageLoader)
`` | `PackageCollection: manages a package collection. This is a subclass of PackageLoader; it adds the ability to install and delete packages. This class is intended to be used by a Boodler UI. It should not be used by a sound-playing process. Most of the methods defined here (installing, deleting, even examining packages) invoke the clear_cache() method defined in PackageLoader. This reflects the fact that any package change can change the dependency tree, which is part of what PackageLoader caches. Clearing the cache entirely is somewhat crude, but it's simple and it works. PackageCollection(basedir=None, coldir=None, dldir=None, boodler_api_vers=None, importing_ok=False) -- constructor Create a PackageCollection. It requires two directory names: coldir is the collection of packages, and dldir is a temporary workspace for storing downloaded and unzipped files. Or you can just provide a basedir, in which case these will be basedir/Collection and basedir/Download (respectively). The coldir and dldir will be created, if necessary. When the PackageCollection shuts down, it will delete the files it has created in the dldir. If a version is provided (either as a VersionNumber or a string which converts to one), then this is noted as the Boodler version which packages must be compatible with. (As noted in the boodler.api_required metadata field.) (Note that the boodler.api_required field is not checked until the package is actually loaded, which is after version selection occurs. Therefore, it is best if the collection only contains packages which are compatible with the installed Boodler version. None of this is likely to be important, since the Boodler API version is not expected to change much after 2.0.) If the importing_ok flag is false, this loader will refuse to load (that is, execute) Python code in packages. This is the default. It should only be true if the create-package command is to be used. Publicly readable fields: collecdir -- the directory containing the package collection downloaddir -- the directory containing temporary files and directories currently_creating -- during a create operation, the package which is being created (at the moment) Public methods: find_source() -- load a package from the collection, a .boop file, or the Internet install_source() -- install a package into the collection from a file or the Internet fetch_source() -- prepare to download a package from the Internet delete_package() -- delete a package from the collection delete_group() -- delete all versions of a package from the collection delete_whole_collection() -- delete all packages in the entire collection start_import_recording() -- begin noting all bimport() calls stop_import_recording() -- stop noting bimport() calls, and return them record_import() -- note a bimport() call by a package being imported create_temp_dir() -- create a new, empty directory in the temporary dir create_temp_file() -- create the pathname of a new temporary file clean_temp() -- clean up the temporary workspace shut_down() -- shut down the PackageCollection Internal methods: rewrite_versions_file() -- write (or overwrite) a new Versions file`
| Methods defined here: __init__(self, basedir=None, coldir=None, dldir=None, boodler_api_vers=None, importing_ok=False) clean_temp(self) `clean_temp() -> None Clean up the temporary download workspace. Delete all files, and clear the internal caches that referred to them.` create_temp_dir(self, label='tmp') `create_temp_dir(label='tmp') -> str Return the pathname of a new, empty directory in the temporary workspace. This directory will be deleted by the next clean_temp() call. The optional label argument will be used in the directory name.` create_temp_file(self, label='tmp') `create_temp_file(label='tmp') -> str Return the pathname of a new file in the temporary workspace. The file will not exist, but can be opened for writing. This file will be deleted by the next clean_temp() call. The optional label argument will be used in the file name.` delete_group(self, pkgname) `delete_group(pkgname) -> None Delete all versions of a package from the collection. This method does not generate an error if the package group is malformed, or even missing.` delete_package(self, pkgname, vers=None) `delete_package(pkgname, vers=None) -> None Delete a package from the collection. If no second argument is given, the most recent available version of the package is deleted. If the argument is a VersionNumber, that version will be deleted. If it is a VersionSpec, the most recent version that matches the spec will be deleted. A string value will be converted to a VersionSpec (not a VersionNumber). If the last version of a package is deleted, the entire group directory is deleted too.` delete_whole_collection(self) `delete_whole_collection() -> None Delete all packages in the entire collection.` fetch_source(self, srctype, loc) `fetch_source(srctype, loc) -> None or Fetcher Make sure that a package of the form (Source_URL, url) is downloaded and ready to be passed to find_source() or install_source(). If it already is (or if srctype is Source_FILE or Source_PACKAGE), then this function returns None, and the caller may proceed. If the URL is *not* downloaded, this returns a Fetcher object. This should be called in some wise like this: while (not fetcher.is_done()): fetcher.work() Once fetcher.is_done() returns true, the package is downloaded, and the caller may proceed to find_source() or install_source().` find_source(self, srctype, loc) `find_source(srctype, loc) -> PackageInfo Load a package given (srctype, loc) in one of the following forms: (Source_PACKAGE, (pkgname, vers)) (Source_FILE, filename) (Source_URL, url) Source_PACKAGE will be found and loaded from the collection. (The vers part may be a VersionNumber, VersionSpec, or None.) Source_FILE will be unzipped (in the temporary work directory) and then loaded. (However, the temporary directory will not be kept in the loader's external package list. So you cannot reload the package without doing another find_source(). To install the file permanently, use install_source().) Source_URL will be unzipped and loaded, but only if it has already been fetched with fetch_source(). This is a blivet in the API; it would be nice if find_source() could fetch the URL too, but that is potentially a slow operation, and find_source() is not set up for slow operations.` install_source(self, srctype, loc) `install_source(srctype, loc) -> PackageInfo Install a package given (srctype, loc) in one of the following forms: (Source_PACKAGE, (pkgname, vers)) (Source_FILE, filename) (Source_URL, url) Source_PACKAGE is not a valid argument for this function, since it refers to a package which is already installed. Source_FILE will be unzipped (in the temporary work directory) and then moved to the collection directory. Source_URL will be unzipped and installed, but only if it has already been fetched with fetch_source(). This is a blivet in the API; it would be nice if install_source() could fetch the URL too, but that is potentially a slow operation, and install_source() is not set up for slow operations.` record_import(self, pkg, name, spec=None) `record_import(pkg, name, spec=None) -> None Note a bimport() call by a package being imported. This should only be called by bimport().` rewrite_versions_file(self, dirname, versionlist, pkgname='<unknown>') `rewrite_versions_file(dirname, versionlist, pkgname='<unknown>') -> None Write (or overwrite) a new Versions file, in the given directory, with the given list of VersionNumber objects. The pkgname is stored in a comment line. The versionlist may be altered (sorted) by this function, so the caller should not pass a list object which will be used again.` shut_down(self) `shut_down() -> None Shut down the PackageCollection. It may not be used again. This cleans up the temporary download workspace.` start_import_recording(self) `start_import_recording() -> None Set the collection to noting all bimport() calls in packages being imported. (This is used only during package creation, to figure out dependencies.)` stop_import_recording(self) `stop_import_recording() -> dic Stop noting bimport() calls, and return a list of all such calls noted. The return dict maps (name,version) pairs to lists of import requests by that package. Each request in such a list looks like (name,spec), or (name,version) for an exact request, for (name,None) for an any-version request.` Data and other attributes defined here: instance_count = 0 Methods inherited from boopak.pload.PackageLoader: add_external_package(self, dirname, metadata=None, resources=None) `add_external_package(dirname, metadata=None, resources=None) -> (str, VersionNumber) Add an external directory to load a package from. The argument must be a complete (unpacked) Boodler directory, with a Metadata file. That package will then be loadable (possibly hiding a package in the collection directory). The metadata and resources fields, if supplied, override the Metadata and Resources files in the directory. (The files do not even have to exist.) If the metadata and resources fields are None, the directory's files are checked as usual. (This override feature is used by the package creation tool. It should not be used for any other purpose.) Returns the name and version of the package that was found. Since this changes the set of what packages are available, it implicitly invokes clear_cache(). The external package facility is not intended to be how most packages are loaded. Most packages should be in the collection directory. You might add an external package in your development workspace (while developing a soundscape), or in a temporary download directory (while deciding whether to install a newly- downloaded package).` attrify_filename(self, pkg, mod, wholekey, res, filename) `attrify_filename(pkg, mod, wholekey, res, filename) -> None Given a filename, create a File representing it, and store the File in the module at a location defined by wholekey. Submodules are created as necessary. The res argument is the Resource object associated with wholekey. The filename must be in universal format: relative to the package root, and written with forward slashes, not backslashes. (If the filename is invalid or unsafe, ValueError is raised. However, this does not check whether the file exists.)` clear_cache(self) `clear_cache() -> None Erase the PackageLoader's entire knowledge of what groups and packages are available. This should be called whenever packages are added to, modified in, or deleted from the collection directory.` clear_external_packages(self) `clear_external_packages() -> None Remove all external directories. Since this changes the set of what packages are available, it implicitly invokes clear_cache().` discover_all_groups(self) `discover_all_groups() -> None Search through the collection directory, and load all the PackageGroups that are available. This only searches the on-disk directory the first time you call it. To force it to re-scan the directory, call clear_cache() first.` find_all_dependencies(self) `find_all_dependencies() -> (dict, dict, dict) Go through the entire collection, and determine exactly which packages depend on which other packages. This returns a triple (forward, backward, bad). In the forward dict, each package key (pkgname,vers) maps to an array of package keys which the package depends on. In the backward dict, each package key maps to an array of packages which depend *on* it. In the bad dict, each package key maps to an array of (pkgname,spec) of dependencies which could not be loaded. (In the bad dict tuples, the second element may be None, a VersionSpec, or a VersionNumber.) This only searches the on-disk directory the first time you call it. To force it to re-scan the directory, call clear_cache() first.` find_item_resources(self, obj) `find_item_resources(obj) -> (PackageInfo, Resource) Given an object in a package module, try to find the package, and the Resource that represents the object. If it has no metadata defined, this returns a blank Resource object. This tries to use object attributes such as __module__ and __name__ to identify it. This will work on classes (but not instances) defined in a module; this covers the normal case of Agent classes. It will also work on File objects defined in a module. Beyond that, results are questionable.` generate_package_path(self, pkgname, vers=None) `generate_package_path(pkgname, vers=None) -> str Return the pathname in the collection directory which contains the given package group (if vers is None), or the given package (otherwise). This does not create the directory or check for its existence. This is guaranteed to create a distinct pathname for every pkgname or (pkgname, vers) pair. This is true even on case- insensitive filesystems, which is a wrinkle, since version strings are case-sensitive. We work around this by putting a "^" character before each capital letter.` import_package_content(self, pkg) `import_package_content(pkg) -> None Import the package's content, if it hasn't already been imported. Warning: this method imports Python source code from the package directory, which means it *executes* Python source code from the package directory. Do not call this on untrusted packages. A sound-player will have to call this, but a package manager should not.` list_all_current_packages(self) `list_all_current_packages() -> list of (str, VersionNumber) Search through the collection directory, and return a list of all the available packages. If multiple versions of a package are available, only the most recent will be listed. Returns a list of (packagename, version) tuples (in no particular order). This only searches the on-disk directory the first time you call it. To force it to re-scan the directory, call clear_cache() first.` list_all_packages(self) `list_all_packages() -> list of (str, list of VersionNumber) Search through the collection directory, and return a list of all the available packages. Returns a list of (packagename, list of version) tuples. The top list is in no particular order, but the version list will be sorted newest-to-oldest. This only searches the on-disk directory the first time you call it. To force it to re-scan the directory, call clear_cache() first.` load(self, pkgname, versionspec=None) `load(pkgname, versionspec=None) -> PackageInfo Load a package, given its name and a version spec. If no second argument is given, the most recent available version of the package is loaded. If the argument is a VersionNumber, that version will be loaded. If it is a VersionSpec, the most recent version that matches the spec will be loaded. A string value will be converted to a VersionSpec (not a VersionNumber). This generates a PackageNotFoundError if no matching package is available. It generates PackageLoadError if the package was malformed in some way which prevented loading.` load_group(self, pkgname) `load_group(self, pkgname) -> PackageGroup Load a PackageGroup, given its package name. This is not tremendously useful for outside users, although you can call it if you want. A PackageGroup represents all the available versions of a particular package.` load_item_by_name(self, name, package=None) `load_item_by_name(name, package=None) -> value Given a string that names a resource -- for example, 'com.eblong.example/reptile.Hiss' -- import the module and return the resource object. If the string begins with a slash ('/boodle.builtin.NullAgent') then the regular Python modules are searched. No importing is done in this case; it is really intended only for the contents of boodle.agent. If the string ends with a slash ('com.eblong.example/'), then the module itself is returned. If the package argument is supplied (a PackageInfo object), it becomes the package to look in for unqualified resource names ('reptile.Hiss'). If no package argument is supplied, then an unqualified resource name raises ValueError.` load_package_dependencies(self, pkg) `load_package_dependencies(pkg) -> (set, dict, int) Attempt to load all the packages which the given package depends on. The argument should be a PackageInfo object. This returns a triple (good, bad, count): - good is a set containing (packagename, version) pairs for every package that was loaded successfully. (This will include the original package.) - bad is a dict. The keys are packagenames which did not load successfully. Each maps to a (nonempty) list of version requests for that package, which could not be fulfilled. (The list contains None, VersionSpecs, and VersionNumbers. Values may occur more than once.) - count is an int, representing how many actual errors occurred. This describes package format problems and read errors. It does not include packages that were simply not available. (The bad dict includes both errors and not-availables; so len(bad) >= count.) If bad is empty, then all dependencies are available. Note that the good list may include more than one version of a package.` load_specific(self, pkgname, vers) `load_specific(pkgname, vers) -> PackageInfo Load a package, given its name and a specific version number. (The version number may be a VersionNumber or a string that can be converted to one.) This is an internal call; external callers should use load(). (load_specific() is intended to be called by a package function which has already consulted package_groups and a PackageGroup object. If you bypass those, you might load a package which is not part of any PackageGroup. That would leave the cache in a confusing state.)` remove_external_package(self, dirname) `remove_external_package(dirname) -> None Remove an external directory. The argument should be one that was previously passed to add_external_package(). (If it is not a current external directory, this silently does nothing.) Since this changes the set of what packages are available, it implicitly invokes clear_cache().` Data and other attributes inherited from boopak.pload.PackageLoader: global_loader = None import_recorder = None

Functions
`` |  | locate_package_directory(dirname) `locate_package_directory(dirname) -> str Given a directory name, locate the (sole) subdirectory which is a package directory. (This may be the same as the directory that was passed in.) This is necessary because some Zip archives unpack directly into the destination directory; others create a subdirectory and unpack there. We locate the "real" package directory by looking for a Metadata file. If there is none, but there is a single subdirectory, we search there. (Files and directories beginning with "." are ignored in this process; that lets us cope with Mac directories and their ".DS_Store" annoyances.) Raises ValueError if no package directory is found.` remove_recursively(name) `remove_recursively(path) -> None Delete a file or directory (including all contents, for a directory). This deletes symlinks but does not follow them.` unpack_zip_file(filename, dirname) `unpack_zip_file(filename, dirname) -> None Unpack the Zip archive at filename, creating directory dirname. If there was a directory dirname already, it is deleted before the unpacking begins. This raises zipfile.BadZipfile if filename is not a valid Zip file.`

Data
`` |  | Filename_Collection = 'Collection' Filename_Download = 'Download' REPOSITORY_URL = 'http://boodler.org/lib/' Source_FILE = 2 Source_PACKAGE = 1 Source_URL = 3 Suffix_PackageArchive = '.boop'