Open source · for Google Cloud Cluster Toolkit

HPC clusters on Google Cloud, from a web browser

The Open Front End is a web interface for the Cluster Toolkit, formerly the HPC Toolkit. Administrators create Slurm clusters, storage and applications in a few forms. Users submit jobs and open remote desktops. Nobody writes Terraform.

Deploy it in Cloud Shell Read the Admin Guide GitHub
The Open Front End sits between your browser and the Cluster Toolkit, which builds Slurm clusters on Google Cloud Your browser Open Front End Django web app on one service VM Cluster Toolkit + Terraform generated blueprint, applied for you Slurm cluster on Google Cloud controller login compute compute
What it does

Everything a small HPC team needs, in one place

One Django application on a single VM. It drives the Cluster Toolkit for you and keeps track of what it created.

Clusters and storage

  • Slurm clusters with any partitions and machine types, created and destroyed from a form
  • VPC networks and Filestore filesystems, created or imported, shared across clusters
  • Deploy in one project and manage resources in others with separate credentials

Applications and jobs

  • Install software from Spack recipes, custom build scripts, or register what is already there
  • Submit jobs with input from HTTP or gs://, output and logs back to a bucket
  • Group runs into benchmarks to compare instance types and build options

Desktops, notebooks and users

  • Per-user Linux remote desktops in the browser, on the login node or a GPU visualisation node
  • Vertex AI Workbench notebooks with cluster storage mounted
  • Google sign-in for users, with per-user spend quotas set by administrators
How it works

Four steps from an empty project to a running cluster

The first step happens once, from a terminal. Everything after it happens in the web interface.

  1. Deploy the service machine

    Run the deploy script. Terraform creates one VM, a bucket and a Pub/Sub topic, then installs the Front End and the Cluster Toolkit. About 15 minutes, plus up to 15 for the software stack.

  2. Register a credential

    Paste a service account key into the Credentials form. Create or import a VPC and a Filestore filesystem. Both outlive any cluster.

  3. Create a cluster

    Name it, pick a subnet, zone and partitions. The Front End writes the blueprint and runs the toolkit. Status moves from creating to ready in 15 to 20 minutes.

  4. Install software, run jobs

    Install applications with Spack. Users submit jobs from an application page, or open a remote desktop on the cluster, straight from the browser.

Deploy it

Try it in about 40 minutes

You need a Google Cloud project with billing and an account that owns it. The guided tutorial takes care of the rest.

Guided, in Cloud Shell

Cloud Shell already has git, Terraform and the Google Cloud CLI, and is signed in as you. The tutorial walks through every prompt, waits for the service machine, and ends with a ready cluster and a clean teardown.

Open in Cloud Shell Read the tutorial first

Or from your own machine

Linux, WSL or macOS with git, Terraform and the Google Cloud CLI installed and authenticated.

# The Front End lives inside the Cluster Toolkit repository
git clone https://github.com/GoogleCloudPlatform/cluster-toolkit.git
cd cluster-toolkit/community/front-end/ofe
./deploy.sh
Full deployment guide
  • Project. Billing enabled. The script offers to enable the APIs it needs.
  • Account. Project Owner, or the smaller role set in the Admin Guide.
  • Cost. A small VM and a bucket while the Front End exists; clusters bill only while they exist, compute nodes only while jobs run.
Example

A typical first session

From a freshly deployed Front End to a finished GROMACS job, with links to the matching documentation.

Admin · 10 min

Network and storage

Paste the generated key under Credentials. Create a VPC with one subnet under Networks and apply cloud changes. Create a Filestore instance under Filesystems.

Network and filesystem management
Admin · 20 min

A cluster with two partitions

Create a cluster, keep the default c2-standard-60 partition and add a GPU one. Authorise your users. Click Create and watch the status reach ready.

Cluster management
Admin, then user

Install GROMACS, run a job

Add a Spack application with spec gromacs and install it on a compute partition. A user opens it, clicks New Job, points the run script at a gs:// input set and submits, then checks the result on a remote desktop.

Spack installs
Documentation

Pick your guide

Written by role. Adapted from the Cluster Toolkit repository and kept in step with it.

Tutorial

Deploy with Cloud Shell

Guided, step by step, from an empty project to a ready cluster.

 Users

User Guide

Log in, submit jobs, use cloud storage, benchmarks, remote desktops and notebooks.

 Administrators

Admin Guide

Deployment, credentials, networks, filesystems, clusters, users and teardown. Plus the advanced, application and workbench guides.

 Developers

Developer Guide

Architecture, service machine layout, run-time data and how to extend the code.
FAQ

Frequently asked questions

What is the Open Front End?

A web application, built with Django, that uses the Google Cloud Cluster Toolkit to create and manage Slurm HPC clusters on Google Cloud. Administrators manage clusters, storage, applications and users from a browser; users submit jobs, open remote desktops and notebooks. The upstream documentation also calls it the Cluster Toolkit FrontEnd, or TKFE.

Is it free to use?

Yes. It is open source under the Apache 2.0 licence and lives in the Cluster Toolkit repository. You pay only for the Google Cloud resources it creates: the service VM, cluster nodes, Filestore and storage buckets.

Do I need to know Terraform or Cluster Toolkit blueprints?

No. The Front End generates the blueprint and runs the toolkit and Terraform for you. Administrators need basic Google Cloud knowledge: projects, IAM roles and billing. Terraform is only used by the one-off deploy script, and Cloud Shell already has it installed.

How long does deployment take?

The deploy script finishes in about 15 minutes, then the service machine takes up to 15 more to install its software stack. A first cluster typically reaches the ready state 15 to 20 minutes after you click Create.

Can users get a graphical desktop on the cluster?

Yes. Each cluster can offer per-user remote desktops that open in the browser over noVNC: a desktop on the login node for general interactive work, and optionally a visualisation desktop on a dedicated single-node Slurm partition that can carry a GPU and uses TurboVNC with VirtualGL for accelerated OpenGL. Each session runs under the user's own OS Login account.

Who maintains it and where do I get help?

It is developed in the GoogleCloudPlatform/cluster-toolkit repository on GitHub, with contributions from NAG. Report bugs as GitHub issues. For help deploying or customising it for your organisation, contact NAG using the link on this page.

Need help deploying or customising it?

NAG's HPC engineers contribute to the Open Front End and help organisations deploy, customise and support it on Google Cloud. Tell us about your project.

Contact NAG