Metadata-Version: 2.4
Name: ecmwf-datastores-client
Version: 0.5.3
Summary: ECMWF Data Stores Service (DSS) API Python client
License-Expression: Apache-2.0
Classifier: Development Status :: 5 - Production/Stable
Classifier: Intended Audience :: Science/Research
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Topic :: Scientific/Engineering
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: attrs>=23.1.0
Requires-Dist: multiurl>=0.3.7
Requires-Dist: requests>=2.5.0
Requires-Dist: typing-extensions>=4.13.0
Provides-Extra: legacy
Requires-Dist: cdsapi>=0.7.6; extra == "legacy"
Dynamic: license-file

<p align="center">
  <a href="https://github.com/ecmwf/codex/raw/refs/heads/main/ESEE">
    <img src="https://github.com/ecmwf/codex/raw/refs/heads/main/ESEE/data_provision_badge.svg" alt="ECMWF Software EnginE"></a>
  <a href="https://github.com/ecmwf/codex/raw/refs/heads/main/Project Maturity">
    <img src="https://github.com/ecmwf/codex/raw/refs/heads/main/Project Maturity/graduated_badge.svg" alt="Maturity Level"></a>
  <!-- <a href="https://codecov.io/gh/ecmwf/ecmwf-datastores-client">
    <img src="https://codecov.io/gh/ecmwf/ecmwf-datastores-client/branch/main/graph/badge.svg" alt="Code Coverage"></a> -->
  <a href="https://opensource.org/licenses/apache-2-0">
    <img src="https://img.shields.io/badge/Licence-Apache 2.0-blue.svg" alt="Licence"></a>
  <a href="https://github.com/ecmwf/ecmwf-datastores-client/releases">
    <img src="https://img.shields.io/github/v/release/ecmwf/ecmwf-datastores-client?color=purple&label=Release" alt="Latest Release"></a>
</p>

<p align="center">
  <a href="#quick-start">Quick Start</a>
  •
  <a href="#installation">Installation</a>
  •
  <a href="https://ecmwf.github.io/ecmwf-datastores-client/">Documentation</a>
</p>

# ecmwf-datastores-client

ECMWF Data Stores Service (DSS) API Python client.

> [!IMPORTANT]
> This software is **Graduated** and subject to ECMWF's guidelines on [Software Maturity](https://github.com/ecmwf/codex/raw/refs/heads/main/Project%20Maturity).

Technical documentation: https://ecmwf.github.io/ecmwf-datastores-client/

## Installation

Install with conda:

```
$ conda install -c conda-forge ecmwf-datastores-client
```

Install with pip:

```
$ pip install ecmwf-datastores-client
```

## Configuration

The `Client` requires the `url` to the API root and a valid API `key`. These can be provided in three ways, in order of precedence:

1. As keyword arguments when instantiating the `Client`.
1. Via the `ECMWF_DATASTORES_URL` and `ECMWF_DATASTORES_KEY` environment variables.
1. From a configuration file, which must be located at `~/.ecmwfdatastoresrc` or at the path specified by the `ECMWF_DATASTORES_RC_FILE` environment variable.

```
$ cat $HOME/.ecmwfdatastoresrc
url: https://cds.climate.copernicus.eu/api
key: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
```

## Quick Start

Configure the logging level to display INFO messages:

```python
>>> import logging
>>> logging.basicConfig(level="INFO")

```

> [!NOTE]
> All Python code examples in this README are automatically tested as part of the unit test suite.

Instantiate the API client and optionally verify authentication:

```python
>>> from ecmwf.datastores import Client
>>> client = Client()
>>> client.check_authentication()  # optional check
{...}

```

Retrieve data:

```python
>>> collection_id = "reanalysis-era5-pressure-levels"
>>> request = {
...     "product_type": ["reanalysis"],
...     "variable": ["temperature"],
...     "year": ["2022"],
...     "month": ["01"],
...     "day": ["01"],
...     "time": ["00:00"],
...     "pressure_level": ["1000"],
...     "data_format": "grib",
...     "download_format": "unarchived",
... }

>>> client.retrieve(collection_id, request, target="target_1.grib")  # blocks
'target_1.grib'

```

Alternative methods to retrieve data:

```python
>>> remote = client.submit(collection_id, request)  # doesn't block
>>> remote
Remote(...)
>>> remote.download("target_2.grib")  # blocks
'target_2.grib'

>>> results = client.submit_and_wait_on_results(collection_id, request)  # blocks
>>> results
Results(...)
>>> results.download("target_3.grib")
'target_3.grib'

>>> client.download_results(remote.request_id, "target_4.grib")  # blocks
'target_4.grib'

```

List all collection IDs sorted by last update:

```python
>>> collections = client.get_collections(sortby="update")

>>> collection_ids = []
>>> while collections is not None:  # Loop over pages
...     collection_ids.extend(collections.collection_ids)
...     collections = collections.next  # Move to the next page
...

>>> collection_ids
[...]
>>> collection_id in collection_ids
True

```

Explore a collection:

```python
>>> collection = client.get_collection(collection_id)

>>> collection.id == collection_id
True
>>> collection.title
'...'
>>> collection.description
'...'

>>> collection.published_at
datetime.datetime(...)
>>> collection.updated_at
datetime.datetime(...)

>>> collection.begin_datetime
datetime.datetime(...)
>>> collection.end_datetime
datetime.datetime(...)
>>> collection.bbox
(...)

>>> collection.submit(request)
Remote(...)

>>> collection.apply_constraints(request)
{...}

```

Interact with results:

```python
>>> results = client.get_results(remote.request_id)

>>> results.content_length > 0
True
>>> results.content_type
'application/x-grib'
>>> results.location
'...'

>>> results.download("target_5.grib")
'target_5.grib'

```

List all successful jobs, sorted by newest first:

```python
>>> jobs = client.get_jobs(sortby="-created", status="successful")

>>> request_ids = []
>>> while jobs is not None:  # Loop over pages
...     request_ids.extend(jobs.request_ids)
...     jobs = jobs.next  # Move to the next page
...

>>> request_ids
[...]
>>> remote.request_id in request_ids
True

```

Interact with a previously submitted job:

```python
>>> remote = client.get_remote(remote.request_id)

>>> remote.collection_id == collection_id
True
>>> remote.request
{...}

>>> remote.status
'successful'
>>> remote.results_ready
True

>>> remote.created_at  # See also started_at, finished_at, updated_at
datetime.datetime(...)

>>> remote.download("target_6.grib")
'target_6.grib'

>>> remote.get_receipt()
{...}

>>> remote.get_results()
Results(...)

>>> remote.delete()
{...}

```

Apply constraints and find the number of available days in a given month:

```python
>>> month = {"year": "2000", "month": "02"}
>>> constrained_request = client.apply_constraints(collection_id, month)

>>> len(constrained_request["day"])
29

```

## Developer Workflow

### 1. Set Up the Environment

Configure your virtual environment and pre-commit hooks:

```bash
make install
```

> [!NOTE]
> This project uses uv for dependency management. Commit the generated uv.lock file to version control.

### 2. Run Quality Assurance

Check formatting, linting, and lockfile consistency:

```bash
make qa
```

### 3. Commit and Push

Save and push any automatically formatted changes:

```bash
git add .
git commit -m "format codebase and sync lockfile"
git push origin HEAD
```

The CI/CD pipeline triggers on pull requests, merges to main/upstream, and new releases.

## Using the Makefile

All development tasks are exposed as self-documenting targets in the `Makefile`. To see a complete list of available targets and their descriptions, run:

```bash
make help
```

This displays all available utility commands, including:

- Environment setup: `make install`
- Quality assurance: `make qa`
- Unit tests: `make unit-tests`
- Type checking: `make type-check`

To run the full set of quality checks, tests, and build steps in a single command, use:

```bash
make all
```

## License

```
Copyright 2022, European Union.

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
```
