Skip to content

Gateway#

Gateway #

The gateway manages all the quircks and complex bits of efficiently acquiring repodata. It implements all the necessary logic to fetch the repodata from a remote server, cache it locally and convert it into python objects.

The gateway can also easily be used concurrently, as it is designed to be thread-safe. When two threads are querying the same channel at the same time, their requests are coalesced into a single request. This is done to reduce the number of requests made to the remote server and reduce the overall memory usage.

The gateway caches the repodata internally, so if the same channel is queried multiple times the records will only be fetched once. However, the conversion of the records to a python object is done every time the query method is called. Therefor, instead of requesting records directly, its more efficient to pass the gateway itself to methods that accepts it.

__init__(cache_dir=None, default_config=None, per_channel_config=None, max_concurrent_requests=100) #

Parameters:

Name Type Description Default
cache_dir Optional[PathLike[str]]

The directory where the repodata should be cached. If not specified the default cache directory is used.

 None
default_config Optional[SourceConfig]

The default configuration for channels.

 None
per_channel_config Optional[dict[str, SourceConfig]]

Source configuration on a per-URL basis. This URL is used as a prefix, so any channel that starts with the URL uses the configuration. The configuration with the longest matching prefix is used.

 None
max_concurrent_requests int

The maximum number of concurrent requests that can be made.

 100
Examples#
>>> Gateway()
Gateway()
>>>

__repr__() #

Returns a representation of the Gateway.

Examples#
>>> Gateway()
Gateway()
>>>

clear_repodata_cache(channel, subdirs=None) #

Clears any in-memory cache for the given channel.

Any subsequent query will re-fetch any required data from the source.

This method does not clear any on-disk cache.

Parameters:

Name Type Description Default
channel Channel | str

The channel to clear the cache for.

 required
subdirs Optional[List[Platform | PlatformLiteral]]

A selection of subdirectories to clear, if None is specified all subdirectories of the channel are cleared.

 None
Examples#
>>> gateway = Gateway()
>>> gateway.clear_repodata_cache("conda-forge", ["linux-64"])
>>> gateway.clear_repodata_cache("robostack")
>>>

names(channels, platforms) async #

Queries all the names of packages in a channel.

Parameters:

Name Type Description Default
channels List[Channel | str]

The channels to query.

 required
platforms List[Platform | PlatformLiteral]

The platforms to query.

 required

Returns:

Type Description
List[PackageName]

A list of package names that are present in the given subdirectories.
Examples#
>>> import asyncio
>>> gateway = Gateway()
>>> records = asyncio.run(gateway.names(["conda-forge"], ["linux-64"]))
>>> PackageName("python") in records
True
>>>

query(channels, platforms, specs, recursive=True) async #

Queries the gateway for repodata.

If recursive is True the gateway will recursively fetch the dependencies of the encountered records. If recursive is False only the records with the package names specified in specs are returned.

The specs can either be a MatchSpec, PackageName or a string. If a string or a PackageName is provided it will be converted into a MatchSpec that matches any record with the given name. If a MatchSpec is provided all records that match the name specified in the spec will be returned, but only the dependencies of the records that match the entire spec are recursively fetched.

The gateway caches the records internally, so if the same channel is queried multiple times the records will only be fetched once. However, the conversion of the records to a python object is done every time the query method is called.

Parameters:

Name Type Description Default
channels List[Channel | str]

The channels to query.

 required
platforms List[Platform | PlatformLiteral]

The platforms to query.

 required
specs List[MatchSpec | PackageName | str]

The specs to query.

 required
recursive bool

Whether recursively fetch dependencies or not.

 True

Returns:

Type Description
List[List[RepoDataRecord]]

A list of lists of RepoDataRecords. The outer list contains the results for each
List[List[RepoDataRecord]]

channel in the same order they are provided in the channels argument.
Examples#
>>> import asyncio
>>> gateway = Gateway()
>>> records = asyncio.run(gateway.query(["conda-forge"], ["linux-aarch64"], ["python"]))
>>> assert len(records) == 1
>>>

SourceConfig dataclass #

Describes properties about a channel.

This can be used to configure the Gateway to handle channels in a certain way.

bz2_enabled: bool = True class-attribute instance-attribute #

Whether the BZ2 compression is enabled or not.

cache_action: CacheAction = 'cache-or-fetch' class-attribute instance-attribute #

How to interact with the cache.

  • 'cache-or-fetch' (default): Use the cache if its up to date or fetch from the URL if there is no valid cached value.
  • 'use-cache-only': Only use the cache, but error out if the cache is not up to date
  • 'force-cache-only': Only use the cache, ignore whether or not it is up to date.
  • 'no-cache': Do not use the cache even if there is an up to date entry

jlap_enabled: bool = True class-attribute instance-attribute #

Whether the JLAP compression is enabled or not.

sharded_enabled: bool = False class-attribute instance-attribute #

Whether sharded repodata is enabled or not.

zstd_enabled: bool = True class-attribute instance-attribute #

Whether the ZSTD compression is enabled or not.