conda_recipe_manager.fetcher.artifact_fetcher

conda_recipe_manager.fetcher.artifact_fetcher#

Description:

Module that provides general Artifact Fetching utilities and factory methods.

Functions

from_recipe(recipe[, ignore_unsupported])

Parses and constructs a list of artifact-fetching objects based on the contents of a recipe.

from_recipe_fetch(recipe_reader[, ...])

Starts a threadpool that pulls-down all source artifacts for a recipe file, with a built-in retry mechanism.

from_recipe_fetch_corrected(recipe_reader[, ...])

Starts a threadpool that pulls-down all source artifacts for a recipe file, with a built-in retry mechanism AND attempts to find corrected PyPI source URLs.
conda_recipe_manager.fetcher.artifact_fetcher.from_recipe(recipe: RecipeReader, ignore_unsupported: bool = False) → Generator[FetcherTable][source]#

Parses and constructs a list of artifact-fetching objects based on the contents of a recipe.

NOTE: To keep this function fast, this function does not invoke fetch() on any artifacts found. It is up to the caller to manage artifact retrieval.

Currently supported sources (per recipe schema):

  • HTTP/HTTPS with tar or zip artifacts (V0 and V1)

  • git (unauthenticated) (V0 and V1)

Parameters:

  • recipe -- Parser instance for the target recipe

  • ignore_unsupported -- (Optional) If set to True, ignore currently unsupported artifacts found in the source section and return the list of supported sources. Otherwise, throw an exception.

Raises:
Returns:

A context-managed-generator that yields a map containing one path and Artifact Fetcher instance pair per source found in the recipe file.

conda_recipe_manager.fetcher.artifact_fetcher.from_recipe_fetch(recipe_reader: RecipeReader, ignore_unsupported: bool = False, retry_interval: float = 10, retries: int = 5) → Generator[FetcherFuturesTable][source]#

Starts a threadpool that pulls-down all source artifacts for a recipe file, with a built-in retry mechanism.

Parameters:

  • ignore_unsupported -- (Optional) If set to True, ignore currently unsupported artifacts found in the source section and return the list of supported sources. Otherwise, throw an exception.

  • recipe_reader -- READ-ONLY Parser instance for the target recipe. Ensuring this is a read-only parsing class provides some thread safety through abusing a type checker (like mypy).

  • retry_interval -- (Optional) Base quantity of time (in seconds) to wait between fetch attempts. Defaults to the DEFAULT_RETRY_INTERVAL constant.

  • retries -- (Optional) Number of retries to attempt. Defaults to the DEFAULT_RETRY_LIMIT constant.

Raises:
Returns:

A generator containing a table that maps futures to the source artifact path in the recipe file and the fetcher instance itself.

conda_recipe_manager.fetcher.artifact_fetcher.from_recipe_fetch_corrected(recipe_reader: RecipeReader, ignore_unsupported: bool = False, retry_interval: float = 10, retries: int = 5) → Generator[FetcherFuturesTable][source]#

Starts a threadpool that pulls-down all source artifacts for a recipe file, with a built-in retry mechanism AND attempts to find corrected PyPI source URLs.

Parameters:

  • recipe_reader -- READ-ONLY Parser instance for the target recipe. Ensuring this is a read-only parsing class provides some thread safety through abusing a type checker (like mypy).

  • ignore_unsupported -- (Optional) If set to True, ignore currently unsupported artifacts found in the source section and return the list of supported sources. Otherwise, throw an exception.

  • retry_interval -- (Optional) Base quantity of time (in seconds) to wait between fetch attempts.

  • retries -- (Optional) Number of retries to attempt. Defaults to _RETRY_LIMIT constant.

Raises:
Returns:

A generator containing a table that maps futures to the source artifact path in the recipe file and the fetcher instance itself.