Parallel computing in Julia



 Table of contents


February 1st (Part 1) and February 8th (Part 2)
Both days 10:00am - noon Pacific Time

Julia is a high-level programming language well suited for scientific computing and data science. Just-in-time compilation, among other things, makes Julia really fast yet interactive. For heavy computations, Julia supports multi-threaded and multi-process parallelism, both natively and via a number of external packages. It also supports memory arrays distributed across multiple processes either on the same or different nodes. In this hands-on workshop, we will start with Julia’s multi-threading features and then focus on Distributed multi-processing standard library and its large array of tools. We will demo parallelization using two problems: a slowly converging series and a Julia set. We will run examples on a multi-core laptop and an HPC cluster.

Instructor: Alex Razoumov (SFU)

Prerequisites: Ideally, some familiarity with the Alliance’s HPC cluster environment, in particular, with the Slurm scheduler. Having some previous serial Julia programming experience would help, but we will start slowly so you will be able to follow up even if you are new to Julia.

Software: There are a couple of options:

  1. You can work on our remote training cluster through an SSH client. You will need a remote secure shell (SSH) client installed on your computer in order to participate in course exercises. On Windows we recommend the free Home Edition of MobaXterm. On Mac and Linux computers SSH is usually pre-installed (try typing ssh in a terminal to make sure it is there). We will provide guest accounts on our training cluster. No need to install Julia on your computer.

  2. You can use Julia on your own computer (if already installed).


Part 1
Introduction to Julia language
Intro to parallelism
Multi-threading with Base.Threads (slow series)
Multi-threading with ThreadsX (slow series)
Parallelizing the Julia set with Base.Threads
Parallelizing the Julia set with ThreadsX


Part 2
Distributed.jl: basics
Distributed.jl: three scalable versions of the slow series
DistributedArrays.jl
Parallelizing the Julia set with DistributedArrays
SharedArrays.jl
Parallelizing the N-body problem (supplemental material)
Parallelizing the additive Schwarz method (supplemental material)
Distributed linear algebra in Julia (supplemental material)