Introduction to HPC with MPI for Data Science

This textbook provides a comprehensive introduction to High Performance Computing (HPC) techniques specifically tailored for Data Science applications. It is organized into two main parts: the first part covers the fundamentals of HPC, while the second part focuses on Data Science basics and demonstrates how to write distributed programs for basic sequential algorithms to handle large-scale datasets.

The book distinguishes between two primary approaches to parallel algorithm design: parallelizing on single multi-core shared memory machines using multi-threading, and parallelizing on clusters of machines with distributed memory. It emphasizes the use of MPI (Message Passing Interface) as the key tool for implementing distributed memory parallelism.

Readers will learn how to scale up resources dynamically according to dataset size, leveraging the flexibility of cluster computing with heterogeneous nodes and various interconnection topologies. The text includes numerous examples and problems, making it suitable for self-study or as a course textbook for undergraduates in computer science.

Key topics covered include parallel programming models, MPI programming, distributed algorithms for data science tasks, and practical strategies for handling large datasets. The book is part of the Undergraduate Topics in Computer Science series, known for its high-quality instructional content.