Topics

Featured in Development

Peter Alvaro talks about the reasons one should engage in language design and why many of us would (or should) do something so perverse as to design a language that no one will ever use. He shares some of the extreme and sometimes obnoxious opinions that guided his design process.

Featured in AI, ML & Data Engineering

Today on The InfoQ Podcast, Wes talks with Katharine Jarmul about privacy and fairness in machine learning algorithms. Jarul discusses what’s meant by Ethical Machine Learning and some things to consider when working towards achieving fairness. Jarmul is the co-founder at KIProtect a machine learning security and privacy firm based in Germany and is one of the three keynote speakers at QCon.ai.

Featured in Culture & Methods

Organizations struggle to scale their agility. While every organization is different, common patterns explain the major challenges that most organizations face: organizational design, trying to copy others, “one-size-fits-all” scaling, scaling in siloes, and neglecting engineering practices. This article explains why, what to do about it, and how the three leading scaling frameworks compare.

Microsoft Pushes New Azure Offerings into the High-Performance Computing Market

Microsoft is entering the high-performance computing (HPC) market with their announcement of the general availability of Azure CycleCloud, a tool for creating, managing, operating, and optimizing HPC clusters of any scale in Azure. It is suitable for IT organizations of Azure customers, enabling them to create secure and flexible cloud HPC and Big Compute environments for their end users. Furthermore, Microsoft announced it would support NVIDIA GPU Cloud (NGC).

In August 2017 Microsoft acquired CycleCloud from the founders Jason Stowe, Rachel Christensen, Rob Futrick and Doug Clayton. Their company, Cycle Computing, was started in 2005 helped companies to do high-performance computing using Condor as an open-source scheduler. By doing big computing fast in the cloud, the company won several awards.

The company was also observing cloud vendors that could provide a robust infrastructure for their solution. James Stow said in a Bio.IT World article on the GA of CycleCloud:

Since we started CycleCloud over 10 years ago, the amount of compute power in a server has continued to exponentially increase. This is in part thanks to FPGAs and GPUs, and Azure has the broadest fleet of these accelerators.

IT administrators can quickly deploy high-performance clusters of computing, storage, filesystem, and application capability in Azure. According to the announcement, CycleCloud’s role-based policies and governance features make it easy for "customer organizations to deliver the hybrid compute power where needed, while avoiding runaway costs". Subsequently, end-users can rely on Azure CycleCloud to orchestrate their job and data workflows across these clusters. Customers can use start using CycleCloud by downloading the tool or through an ARM template.

The support for the NVIDIA GPU Cloud (NGC) in Azure helps customers to accelerate their AI, and HPC workflows on a variety of virtual machines enabled with NVIDIA GPUs. NVIDIA provides a library of 35 GPU-accelerated containers for deep learning software, HPC applications, HPC visualization tools and a variety of partner applications from the NGC container registry. Customers can run these containers on the following Microsoft Azure instance types with NVIDIA GPUs, according to NVIDIA blog about the support of NVIDIA GPU Cloud on Azure:

Microsoft is not the only public cloud provider supporting HPC. AWS has GPU-powered EC2 instances available since October last year, which can be powered by up to eight NVIDIA Tesla V100 GPUs. These instances were designed to handle compute-intensive workloads ranging from machine learning to genomics. Besides Microsoft and Amazon supporting NVIDIA GPU’s, Google has their custom chips supporting customers to run machine learning workloads written for its TensorFlow framework. Each major public cloud provider appears to be pushing their cloud offerings towards the high-performance computing market.