Apolo docsConsoleGitHub

README

Apolo Extras

A set of tools and commands to extend the functionality of Apolo platform CLI client.

Usage

Check-out apolo-extras CLI reference here for main commands syntax and use-cases.

Dowloading and extracting archive from bucket

To platform storage: or disk:

  • From Google bucket storage:

    • apolo-extras data cp -x -t -v secret:gcp-creds:/gcp-creds.txt -e GOOGLE_APPLICATION_CREDENTIALS=/gcp-creds.txt gs://BUCKET_NAME/dataset.tar.gz storage:/project/dataset

    • secret:gcp-creds is a secret, containing authentication credentials file used by config gsutil

  • From AWS-compatible object storage:

    • apolo-extras data cp -x -t -v secret:s3-creds:/s3-creds.txt -e AWS_SHARED_CREDENTIALS_FILE=/s3-creds.txt s3://BUCKET_NAME/dataset.tar.gz disk:disk-name-or-id:/project/dataset

    • secret:s3-creds is a secret, containing auth data file for aws utility.

  • From Azure blob object storage:

    • apolo-extras data cp -x -t -e AZURE_SAS_TOKEN=secret:azure-sas-token azure+https://BUCKET_NAME/dataset.tar.gz storage:/project/dataset

    • secret:azure-sas-token is a secret, containing SAS token for accessing needed blob.

  • From HTTP/HTTPS server:

    • apolo-extras data cp -x -t https://example.org/dataset.tar.gz disk:disk-name-or-id:/project/dataset

To local machine

  • From GCP bucket storage:

    • apolo-extras data cp -x gs://BUCKET_NAME/dataset.tar.gz /project/dataset

    • gsutil utility should be installed on local machine and authenticated to read needed bucket

    • Supported Python verions are 3 (3.5 to 3.8, 3.7 recommended) and 2 (2.7.9 or higher)

  • From AWS-compatible object storage:

    • apolo-extras data cp -x s3://BUCKET_NAME/dataset.tar.gz /project/dataset

    • aws utility should be installed on local machine and authenticated to read needed bucket

    • If needed, install it with pipx install awscli not to conflict with apolo-cli

  • From Azure blob object storage:

    • AZURE_SAS_TOKEN=$TOKEN apolo-extras data cp -x azure+https://BUCKET_NAME/dataset.tar.gz storage:/project/dataset

    • rclone should be installed on the local machine

  • From HTTP/HTTPS server:

    • apolo-extras data cp -x -t https://example.org/dataset.tar.gz /project/dataset

    • rclone should be installed on the local machine

NextCLI Reference

Last updated