Parallel computing in Julia



 Table of contents


March 20th (Part 1) and 27th (Part 2)
10:00am - noon Pacific Time

Julia is a high-level programming language well suited for scientific computing and data science. Just-in-time compilation, among other things, makes Julia really fast yet interactive. For heavy computations, Julia supports multi-threaded and multi-process parallelism, both natively and via a number of external packages. It also supports memory arrays distributed across multiple processes either on the same or different nodes. In this hands-on workshop, we will start with Julia’s multi-threading features and then focus on Distributed multi-processing standard library and its large array of tools. We will demo parallelization using two problems: a slowly converging series and a Julia set. We will run examples on a multi-core laptop and an HPC cluster.

Instructor: Alex Razoumov (SFU)

Prerequisites: Ideally, some familiarity with the Alliance’s HPC cluster environment, in particular, with the Slurm scheduler. Having some previous serial Julia programming experience would help, but we will start slowly so you will be able to follow up even if you are new to Julia.

Software: There are a couple of options:

  1. You can run Julia on our training cluster, in which case you will need a remote secure shell (SSH) client installed on your computer. On Mac and Linux computers, SSH is usually pre-installed – try typing ssh in a terminal to make sure it is there. Many versions of Windows also provide an OpenSSH client by default –- try opening PowerShell and typing ssh to see if it is available. If not, then we recommend installing the free Home Edition of MobaXterm from https://mobaxterm.mobatek.net/download.html. We will provide guest accounts on our training cluster, and you would not need to install Julia on your computer in this setup.

  2. You can run Julia on your own computer, in which case you can install it from https://julialang.org/downloads – this may take a while so please do this before the class.


Part 1
Introduction to Julia language
Intro to parallelism
Multi-threading with Base.Threads (slow series)
Multi-threading with ThreadsX (slow series)
Parallelizing the Julia set with Base.Threads
Parallelizing the Julia set with ThreadsX


Part 2
Distributed.jl: basics
Distributed.jl: three scalable versions of the slow series
Persistent storage on workers   (new)
DistributedArrays.jl
Parallelizing the Julia set with DistributedArrays
SharedArrays.jl
Parallelizing the N-body problem   (supplemental material)
Parallelizing the additive Schwarz method   (supplemental material)
Distributed linear algebra in Julia   (supplemental material)

Our Julia webinars

Since 2020, we’ve been teaching occasional webinars on parallel programming in Julia – watch the recordings here .