This post is part of a series on the Thermo.io Cloud platform. Check it out here for a rundown on the technology we use and a list of related posts.

Thermo.io Cloud Lab: Passthrough GPUs

We’ve had Hopper and the Thermo Physistis hard at work in the lab on the next major feature of the Thermo.io Cloud platform. It was time to drag Hopper out of the lab to get some fresh air and let us know what the team has been working on.

This first edition of Cloud Lab previews our passthrough GPU feature. This will be a reoccurring format as a roadmap teaser for upcoming features to the Thermo.io Cloud platform.

Why GPUs?

The use of GPUs across workloads is growing rapidly. There are a growing number of cases in which CPUs simply do not cut it as the go-to processing unit. As GPU-aware libraries and frameworks continue to mature, we are going to see even more workloads move to GPUs.

A good example of this fast evolving landscape is GPU-backed databases, such as MapD and Kinetica. These emerging solutions map traditional SQL-style queries through run-time compiler frameworks optimized to execute on GPUs. Then, result sets take advantage of the GPU’s fast memory, allowing for very large queries or processing of streaming data at blazingly fast speeds and unmatched levels of parallelism.

We also see traditionally CPU-based parallel execution frameworks such as OpenMPI shifting to GPU-awareness. This opens an entirely uncharted area to GPU acceleration. In time and with experimentation, we’re going to see some exciting results.

What exactly is a passthrough device?

In a passthrough setup, a device (in this case, a GPU) is dedicated to your virtual machine (VM) alone in order to prove bare-metal performance. Your VM sees the GPU as a fully qualified graphics card, and uses the same drivers to enable that GPU as it would on bare metal.

In addition, vCPU cores assigned to your VM on all GPU flavors are also dedicated. This is done by pinning your vCore to a physical CPU core and assigning it only to your VM. We also validate that the core is not a logical hyper-threaded core, but instead a physical core.

Our current working limit on passthrough GPU dies to a VM is six. This is subject to change as we continue to finalize the offering.

What types of GPUs will be offered?

We don’t have a firm commitment baked into our models yet, but we’re looking at a mix of both Nvidia Tesla and AMD Radeon WX series GPUs. This is a complex offering, so we can’t yet specify exact models or the maximum amount of GPU dies per-vendor, per-compute node.

What we can say is each GPU will be paired with a set of flavor (package) sizes relative to cores and RAM, which will optimize them for their respective GPU resources. As with all existing flavors, GPU flavors will also have high-speed SSD storage.

What type of libraries and frameworks will be offered?

The choice is yours. We provide the VMs that are optimized for GPU operations and the passthrough GPU devices, and you select the operating system and set up your dependencies and workloads.

We are looking at providing image templates that include a common set of libraries and frameworks such as TensorFlow, Caffe, and Keras. These would be paired with flavor sizes and default configurations that optimize for specific use cases. This is something we are exploring, so we’re unable to provide more details yet.

You can find an excellent resource for workload specific GPU libraries on the Nvidia Accelerated Computing portal. A similarly deep, but less intuitive resource can be found at the AMD Developer Central portal.

Can I mine cryptocurrency?

Yes! You can mine Bitcoin, Ethereum, or any other cryptocurrency. The Thermo.io Cloud platform will be optimized for GPU workloads that provides hash rates to satisfy any miner.

We recognize the software available for mining is a fragmented ecosystem, littered with ambiguous information, suboptimal mining software, and even malware. For these reasons, as with GPU workload libraries, we may provide image templates that include common configurations for mining cryptocurrencies. We have no firm commitment on these at this time.

Alright, when can we get GPUs?

We have not nailed down a launch date yet. As soon as we have more information on the timeline, we will post an update.