CONTENTS

GPUs vs. CPUs

Here’s the story of how VSSC built its supercomputer on GPUs from Nvidia and what it means for Indian supercomputing. By Prashant L Rao

The world of supercomputing is shaken and stirred. The disruption of late is largely on account of the advent of GPUs as a more than capable replacement for CPUs and this can be seen in Top 500 rankings wherein three of the top five supercomputers in the world are powered by Nvidia GPUs. The chip in question is the Nvidia Tesla, a GPU that offers better than 10x the performance of a top-of-the-line CPU. All of which translates into space savings, power savings and relatively affordable supercomputing. Obviously, this is all very nice but Research Directors aren’t likely to fork over their scarce funds for a technology without adequate references and that’s where Vikram Sarabhai Space Center (VSSC) comes into the picture as a litmus test of sorts for GPU usage in Indian supercomputing.

VSSC’s tryst with GPUs

"We are struggling to be in the top 50. Whether it is for the knowledge network, research or life sciences, India needs to scale up." Vishal Dhupar Mng. Director - South Asia, Nvidia Graphics Pvt. Ltd.

Vishal Dhupar, Managing Director - South Asia, Nvidia Graphics Pvt. Ltd., narrated, “I went to Vikram Sarabhai Space Center (VSSC) in May. Dr. Radhakrishna was inaugurating that center. They had equipment in a single room delivering 220 teraflops and this was built at a cost of Rs. 14 crores including the civil work.”

VSSC does a lot of work on CFD especially with regard to satellites. To this end the scientists at VSSC have developed a homegrown application called PARAS or Parallel Aerodynamic Simulation that they have been working on for over a decade. PARAS ran on the Intel architecture earlier.

“To get close to 200 teraflops, they would have needed 5,800 CPUs. The code had been written on the x86 architecture. We offered them the same architecture, use the same room and offer a quantum jump in performance with a hybrid architecture of CPUs and GPUs. By adding 400 GPUs to the existing 400 CPUs, they got to 220 teraflops,” commented Dhupar.

In comparison, Tata CRL has a 170 teraflops system with 3600 CPUs built at a cost of $30 mn. VSSC achieved 220 teraflops with an investment of $3-3.5 mn. “Only the code that was more parallelized had to be tweaked and this gave them a 40x performance boost on one account and a 60x boost on the other. The chairman made a statement that we should be looking for petaflop level performance,” said Dhupar.

What’s next

Nvidia is targeting the Indian R&D and educational segments. “There are scientists, engineers and researchers in the country—whether they are at CSIR Labs, DRDO Labs, educational institutes etc. Our goal is to provide them with 2-8 teraflops on a personal supercomputer. They can form clusters or grids of these supercomputers and achieve a new level of performance in the data center,” said Dhupar.

With 2 teraflops available for $10,000 it changes the equation. “We want every scientist/researcher to have this,” said Dhupar.

One challenge is to make it easy for researchers to reuse existing code. Compute Unified Device Architecture (CUDA) from Nvidia helps them find which part of their code can scale.

“These guys want to use more datasets and have a bigger sample size and try out more combinations,” said Dhupar outlining the problem faced by the Indian scientific community.

Cost being a perennial problem, Nvidia hopes to convince scientists that they should move their data centers onto GPUs. At the same time, it wants to boost the acceptance of CUDA. “They have been looking at Message Passing Interface (MPI) for parallel computing. MPI is a subset of the CUDA framework. So, there’s no relearning. The framework has SDKs, debuggers, libraries, compilers etc. Whether you use Fortran, C or C++, it’s all supported,” claimed Dhupar.

About a hundred people are teaching CUDA in various Indian institutions including the IITs and IISc.

Server side story

On the HPC side, Nvidia is counting on the server OEMs to lead the way. “HP and IBM dominate the scene and we work closely with both of them. We also work with Fujitsu and Dell. In addition we have a lot of channel partners—particularly those who have built up expertise around HPC like Super Micro. We are also working with Wipro, Netweb and Locus,” said Dhupar outlining his strategy for the HPC part of the market.

Back in the day, when sanctions were in existence, people went with the industry standard x86 architecture and developed home-grown applications on that platform. “Now that the sanctions are no longer in force, we have an opportunity to help them scale up their performance. VSSC is a classic example where you haven’t changed the architecture but by adding four hundred GPUs you have taken the performance to a different level,” argued Dhupar.

“All the IITs have it. IISc has it,” he added.

So far Nvidia has done a couple of deals with Fujitsu, a few with HP and one with Dell. The deals vary from 16-32 nodes to bigger clusters. In many cases, it involves a department—it could be physics, chemistry or CFD—that goes in for this technology. “In CSIR Labs, the CFD guys who are doing simulation went in for it. Either they will look at software that can take advantage of the GPU or they have their own application that they want to scale,” said Dhupar.

Today, China has come from nowhere to dominate the supercomputing lists. It’s two petaflop supercomputer makes India’s top setups look paltry. “We are struggling to be in the top 50. Whether it is for the knowledge network, research or life sciences, India needs to scale up. Even the government has earmarked some investment for HPC in the next five year plan,” said Dhupar.

He gave examples of commercial setups using his company’s equipment including Tata Motors that uses Tesla for the CFD part. “Typically, people wait for the graphics to finish and then do the compute. We are offering Quadro with Tesla so that the same workstation can do analysis and compute at the same time,” he said.

Talking of the market opportunity, he said, “As per IDC, the Indian HPC market is worth $200 mn and it is growing at 10%. The market’s been stuck in a rut as people have been using MPI and they can only do so much with that.”

The proof of the pudding’s in the eating. For many decades Cray used to be the gold standard in supercomputing with its own vector computing technology. Today, even a Cray has adopted Tesla.

There’s a substantial energy efficiency advantage from using GPUs. VSSC consumes 150 kWh for generating 220 teraflops. Tata CRL, on the other hand, is using 2.5 mWh for 170 teraflops. “A GPU’s power consumption is extremely low,” concluded Dhupar.

prashant.rao@expressindia.com