What makes life sciences different from other HPC industries?
Life sciences center relies on HPC to accelerate the time to science to further our knowledge of the world and to save lives. The compute needs are often distinct from the requirements of more general HPC centers or other industry verticals. For example, the biologists and bioinformaticians at Bielefeld University’s Center for Biotechnology (CeBiTec) use a blend of industry tools for sequencing data evaluation, assembly, gene prediction, analysis, and more, with homegrown applications and scripts using Python, C++, Perl, or Java. Since many life sciences applications use the DRMAA2 API to submit workloads to an HPC cluster, it is important for them to have a workload manager that supports the DRMAA2 API.
This heady mix of applications is powered and supported by CeBiTec’s HPC cluster comprised of 100 servers, 3,000 CPU cores, and approximately 15.5 TB of RAM. The tech team at Bielefeld University was faced with the mounting challenges of massive volumes of data and constantly increasing user demand that their open-source workload scheduler couldn’t handle. Genomic data has been growing rapidly over the last 20 years and often runs into petabytes of storage. The life sciences industry is also seeing a growth in real-time workloads that process data streams from sequencers or medical scanners or workloads that need to process data to meet a diagnostic deadline.
Due to the higher demand placed on the high-performance computing environment, some important projects at CeBiTec weren’t accessing the resources they needed. They needed an enterprise-grade workload management solution that would deliver amplified performance improvements to address inefficient resource usage and bottlenecks.
CeBiTec’s team chose Altair® Grid Engine® because it has 100% DRMAA2 support for integration with their life sciences applications such as QuPE, MeltDB, and Fusion, which rely on the DRMAA2 interface for workload submission. Grid Engine is also able to handle the throughput and performance that their workloads demand. Users saw an immediate improvement over the open-source solution and system administrators saved time by using a commercially supported product.
“Altair Grid Engine enabled highly efficient usage of our compute resources with a very small footprint,” explains Dr. Stefan Albaum, leader of the Bioinformatics Resource Facility at CeBiTec. “We like the fact that users who do not have experience can quickly submit jobs to the cluster.”
Read the full customer story here to see how Altair Grid Engine is empowering CeBiTec to move biotechnology forward.