API Docs¶
CondaEnv¶
- class conda_pack.CondaEnv(prefix, files, excluded_files=None)¶
A Conda environment for packaging.
Use
CondaEnv.from_prefix()
,CondaEnv.from_name()
, orCondaEnv.from_default()
instead of the default constructor.Examples
Package the environment
foo
into a zip archive:>>> (CondaEnv.from_name("foo") ... .pack(output="foo.zip")) "/full/path/to/foo.zip"
Package the environment
foo
into a parcel:>>> (CondaEnv.from_prefix("/path/to/envs/foo") ... .pack(format="parcel", parcel_version="2020.09.01")) "/full/path/to/foo-2020.09.01.parcel"
Package the current environment into a
tar.gz
archive:>>> (CondaEnv.from_default() ... .pack(output="output.tar.gz")) "/full/path/to/output.tar.gz"
Create a CondaEnv object from the current environment, excluding all
*.pyx
files, except those fromcytoolz
.>>> env = (CondaEnv.from_default() ... .exclude("*.pyx") ... .include("lib/python3.7/site-packages/cytoolz/*.pyx")) CondaEnv<'~/miniconda/envs/example', 1234 files>
- Attributes:
- prefixstr
The path to the conda environment.
- fileslist of File
A list of
File
objects representing all files in conda environment.
- property name¶
The name of the environment
- classmethod from_name(name, **kwargs)¶
Create a
CondaEnv
from a named environment.- Parameters:
- namestr
The name of the conda environment.
- Returns:
- envCondaEnv
- classmethod from_prefix(prefix, **kwargs)¶
Create a
CondaEnv
from a given prefix.- Parameters:
- prefixstr
The path to the conda environment.
- Returns:
- envCondaEnv
- classmethod from_default(**kwargs)¶
Create a
CondaEnv
from the current environment.- Returns:
- envCondaEnv
- exclude(pattern)¶
Exclude all files that match
pattern
from being packaged.This can be useful to remove functionality that isn’t needed in the archive but is part of the original conda package.
- Parameters:
- patternstr
A file pattern. May include shell-style wildcards a-la
glob
.
- Returns:
- envCondaEnv
A new env with any matching files excluded.
- include(pattern)¶
Re-add all excluded files that match
pattern
- Parameters:
- patternstr
A file pattern. May include shell-style wildcards a-la
glob
.
- Returns:
- envCondaEnv
A new env with any matching files that were previously excluded re-included.
- pack(output=None, format='infer', arcroot='', dest_prefix=None, parcel_root=None, parcel_name=None, parcel_version=None, parcel_distro=None, verbose=False, force=False, compress_level=4, n_threads=1, zip_symlinks=False, zip_64=True)¶
Package the conda environment into an archive file.
- Parameters:
- outputstr, optional
The path of the output file. The basename of the output file defaults to the basename of the
dest_prefix
value, if supplied; otherwise to the basename of the environment. The suffix will be determined by the output format (e.g.my_env.tar.gz
).- format{‘infer’, ‘zip’, ‘tar.gz’, ‘tgz’, ‘tar.bz2’, ‘tbz2’, ‘tar’, ‘parcel’, ‘squashfs’,
‘no-archive’} The archival format to use. By default this is inferred from the output file extension, and defaults to
tar.gz
if this is not supplied.- arcrootstr, optional
The relative path in the archive to the conda environment. Defaults to ‘’.
- dest_prefixstr, optional
If present, prefixes will be rewritten to this path before packaging. In this case the
conda-unpack
script will not be generated.- parcel_root, parcel_name, parcel_version, parcel_distrostr, optional
(Parcels only) the root directory, name, version, and target distribution of the parcel. The name and version will be embedded into parcel metadata. The default values are:
parcel_root
:/opt/cloudera/parcels
parcel_name
: the base name of the environment directoryparcel_version
: the current date inYYYY.MM.DD
format.parcel_distro
:el7
It is important that
parcel_root
match the directory into which all parcels are unpacked on your cluster. Neitherparcel_name
norparcel_version
may contain dashes. The final destination of the parcel is assumed to beparcel_root/parcel_name-parcel_version
, and botharcroot
anddest_prefix
are set accordingly. The default filename will beparcel_name-parcel_version-parcel_distro.parcel
.- verbosebool, optional
If True, progress is reported to stdout. Default is False.
- forcebool, optional
Whether to overwrite any existing archive at the output path if present, or create the output directory structure if it’s missing. Default is False.
- compress_levelint, optional
The compression level to use, from 0 to 9. If ZSTD is used, compression is supported up to 19. Higher numbers decrease output file size at the expense of compression time. Ignored for
format='zip'
. Default is 4.- n_threadsint, optional
The number of threads to use. Set to -1 to use the number of cpus on this machine. If a file format doesn’t support threaded packaging, this option will be ignored. Default is 1.
- zip_symlinksbool, optional
(
zip
format only) Symbolic links aren’t supported by the Zip standard, but are supported by many common Zip implementations. IfTrue
, symbolic links will be stored in the archive. IfFalse
, a copy of the linked file will be included instead. ChoosingTrue
can avoid storing multiple copies of the same file, but the archive may silently fail on decompression if theunzip
implementation does not support symbolic links. For that reason, the default isFalse
.- zip_64bool, optional
(
zip
format only) Whether to enable ZIP64 extensions. Default is True.
- Returns:
- out_pathstr
The path to the archived environment.
File¶
- class conda_pack.File(source, target, is_conda=True, file_mode=None, prefix_placeholder=None)¶
A single archive record.
- Parameters:
- sourcestr
Absolute path to the source.
- targetstr
Relative path from the target prefix (e.g.
lib/foo/bar.py
).- is_condabool, optional
Whether the file was installed by conda, or comes from somewhere else.
- file_mode{None, ‘text’, ‘binary’, ‘unknown’}, optional
The type of record.
- prefix_placeholderNone or str, optional
The prefix placeholder in the file (if any)
pack¶
- conda_pack.pack(name=None, prefix=None, output=None, format='infer', arcroot='', dest_prefix=None, parcel_root=None, parcel_name=None, parcel_version=None, parcel_distro=None, verbose=False, force=False, compress_level=4, n_threads=1, zip_symlinks=False, zip_64=True, filters=None, ignore_editable_packages=False, ignore_missing_files=False)¶
Package an existing conda environment into an archive file.
- Parameters:
- namestr, optional
The name of the conda environment to pack.
- prefixstr, optional
A path to a conda environment to pack. Only one of
name
andprefix
should be supplied.- outputstr, optional
The path of the output file. Defaults to the environment name with a suffix determined by the format; e.g.
my_env.tar.gz
.- format{‘infer’, ‘zip’, ‘tar.gz’, ‘tgz’, ‘tar.bz2’, ‘tbz2’, ‘tar’, ‘parcel’,
‘no-archive’}, optional The archival format to use. By default, this is inferred from the output file extension, and defaults to
tar.gz
ifoutput
is not supplied.- arcrootstr, optional
The relative path in the archive to the conda environment. Defaults to ‘’.
- dest_prefixstr, optional
If present, prefixes will be rewritten to this path before packaging. In this case the
conda-unpack
script will not be generated.- parcel_root, parcel_name, parcel_version, parcel_distrostr, optional
(Parcels only) the root directory, name, version, and target distribution of the parcel. The name and version will be embedded into parcel metadata. The default values are:
parcel_root
:/opt/cloudera/parcels
parcel_name
: the base name of the environment directoryparcel_version
: the current date inYYYY.MM.DD
format.parcel_distro
:el7
It is important that
parcel_root
match the directory into which all parcels are unpacked on your cluster. Neitherparcel_name
norparcel_version
may contain dashes. The final destination of the parcel is assumed to beparcel_root/parcel_name-parcel_version
, and botharcroot
anddest_prefix
are set accordingly. The default filename will beparcel_name-parcel_version-parcel_distro.parcel
.- verbosebool, optional
If True, progress is reported to stdout. Default is False.
- forcebool, optional
Whether to overwrite any existing archive at the output path if present, or create the output directory structure if it’s missing. Default is False.
- compress_levelint, optional
The compression level to use, from 0 to 9. If ZSTD is used, compression is supported up to 19. Higher numbers decrease output file size at the expense of compression time. Ignored for
format='zip'
. Default is 4.- zip_symlinksbool, optional
(
zip
format only) Symbolic links aren’t supported by the Zip standard, but are supported by many common Zip implementations. IfTrue
, symbolic links will be stored in the archive. IfFalse
, a copy of the linked file will be included instead. ChoosingTrue
can avoid storing multiple copies of the same file, but the archive may silently fail on decompression if theunzip
implementation does not support symbolic links. For that reason, the default isFalse
.- n_threadsint, optional
The number of threads to use. Set to -1 to use the number of cpus on this machine. If a file format doesn’t support threaded packaging, this option will be ignored. Default is 1.
- zip_64bool, optional
(
zip
format only) Whether to enable ZIP64 extensions. Default is True.- filterslist, optional
A list of filters to apply to the files. Each filter is a tuple of
(kind, pattern)
, wherekind
is either'exclude'
or'include'
andpattern
is a file pattern. Filters are applied in the order specified.- ignore_editable_packagesbool, optional
By default conda-pack will error in the presence of editable packages. Set to True to skip these checks.
- ignore_missing_filesbool, optional
Ignore that files are missing that should be present in the conda environment as specified by the conda metadata.
- Returns:
- out_pathstr
The path to the archived environment.