Creating a Cluster

Introduction lets you create a cluster in any of the three major cloud providers - Amazon Web Services, Google Cloud Platform, and Azure. After you sign up with a cloud provider, you have to share your service account information and the configuration you need with our team. We will then set up a cluster on your behalf, and install on your cloud for you.

To set up a cluster, you have to do the following:

  1. Set up a cloud environment:

  2. Prepare a cluster configuration YAML file.

  3. Share the configuration YAML file with us. Once we receive the YAML file, we will set up and run the cluster (usually completed within one business day). You can start adding your users to the cluster while it is being set up.

Apart from the process mentioned above, there are other methods of setup:

  • Set up on an existing cluster provided by AWS, GCP, or Azure. This requires the configuration of the node pool.

  • Set up on an existing cluster provided by other cloud service providers.

  • Set up on-premise (or “bare metal”).

For any of these other methods of setup, please contact our team.

Cluster configuration YAML

You must create a project/VPC/resource group and a service account with all required permissions before you can start preparing a cluster configuration YAML file. The YAML file is used by the team to set up and run the cluster. You should use the neuro admin generate-cluster-config command to generate the YAML file. It is an interactive tool that generates a valid configuration file with the default node configuration based on your responses.

The command prompts for the following information in order to generate the configuration file:



Cluster Type

Enter cloud service provider - AWS, GCP, Azure

GCP Project Name or AWS VPC ID or Azure subscription ID

  • For GCP, enter the name of the project.

  • For AWS, enter the VPC ID you created in the previous step.

  • For Azure, enter the Azure subscription ID you created in the previous step.

Service Account Key File (.json)


AWS Profile Name

Or Azure information

Below is a sample command for GCP:

> neuro admin generate-cluster-config
Select cluster type (aws, gcp, azure): gcp
GCP project name: My Project
Service Account Key File (.json): GCP_User.json
Cluster config cluster.yml is generated.

The command creates the cluster.yml file that includes the information required by the team to set up your cluster. Here is a sample cluster.yml generated for a new GCP cluster:

type: gcp
location_type: multi_zonal
region: us-central1
- us-central1-a
- us-central1-c
project: My Project
 client_id: '105087309181394151560'
 private_key: ...
 private_key_id: ...
 project_id: intricate-web-236410
 type: service_account
- id: n1_highmem_8
 min_size: 1
 max_size: 4
- id: n1_highmem_8_1x_nvidia_tesla_k80
 min_size: 1
 max_size: 4
- id: n1_highmem_8_1x_nvidia_tesla_v100
 min_size: 0
 max_size: 1
 id: standard
 capacity_tb: 4

The file contains a default nodes pools configuration that is used as a starting point:

  • 1 non-preemptive node with K80

  • 1 non-preemptive node with V100

  • 4 non-preemptive non-GPU nodes

  • and 4 Tb of standard storage

The file will create a cluster with the following presets:

  • cpu-small,

  • cpu-large,

  • gpu-small (a node with K80), and

  • gpu-large (a node with V100).

To get information about available options for each of the cloud providers, run:

neuro admin show-cluster-options .

You can further update the cluster.yml file as per your requirements before you send it to us. If you have any issues updating the file, then contact us. Once you are done updating the configuration file, you should send the cluster.yml file to the team for the cluster setup using the command:

neuro admin add-cluster <path/to/config>

Once we receive the YAML file, we will set up and run the cluster (usually completed within one business day). After the command is run, you become the admin of the cluster. You can start adding users to the cluster as soon as you run the command above.

Last updated