Managing Workloads with OpenMP Directives in Fortran

Parallel computing has become an essential tool for optimizing the performance of computationally intensive tasks. In Fortran, OpenMP (Open Multi-Processing) is a widely used API for parallel programming, providing an easy way to parallelize loops, sections of code, and more. The ability to manage workloads and distribute them efficiently across multiple processors or cores can drastically improve the performance of your application.

OpenMP directives allow programmers to manage workload distribution, control the number of threads, and ensure proper synchronization between threads. This post will walk you through the various OpenMP directives used for workload management, including common constructs like parallel do, for, and single, and show you how to optimize parallel execution using the num_threads clause and the OMP_NUM_THREADS environment variable.

What is OpenMP?

OpenMP is an API that provides a set of compiler directives, library routines, and environment variables for parallel programming in shared-memory architectures. It allows you to parallelize code with minimal changes, making it easier to exploit the full potential of multi-core processors.

Fortran provides several OpenMP directives that can be used to parallelize loops, sections, and regions of code. Some key features of OpenMP include:

• Parallelization of loops: Use the !$omp parallel do or !$omp do directives to parallelize loop iterations.
• Control over thread management: Specify the number of threads and manage how the work is split between threads.
• Synchronization: Manage dependencies between threads using critical, barrier, atomic, and other synchronization constructs.
• Environment variables: Use environment variables like OMP_NUM_THREADS to control the number of threads at runtime.

---

OpenMP Work-Share Constructs

OpenMP provides several directives that control how the work is shared between threads. These include work-sharing constructs that allow different threads to execute parts of a program concurrently.

1. Parallel Loop: !$omp parallel do

The !$omp parallel do directive allows you to parallelize a do loop by splitting its iterations among multiple threads. This is one of the most commonly used directives in OpenMP to achieve parallelism in Fortran.

Example: Parallelizing a Loop

program parallel_loop_example
  integer :: i
  integer, dimension(1000) :: a, b, c

  ! Initialize arrays
  do i = 1, 1000
b(i) = i
c(i) = 2 * i
  end do

  !$omp parallel do num_threads(4)
  do i = 1, 1000
a(i) = b(i) + c(i)
  end do

  print *, "First few results: ", a(1), a(2), a(3)
end program parallel_loop_example

Explanation:

• !$omp parallel do num_threads(4): This directive tells the compiler to parallelize the following loop and use 4 threads to process the loop iterations. The num_threads(4) clause specifies that 4 threads will be used.
• The loop will be split across the 4 threads, with each thread processing a chunk of the loop iterations concurrently. This can lead to a significant speedup when working with large arrays or computationally expensive tasks.

2. Parallel Region: !$omp parallel

The !$omp parallel directive defines a parallel region where multiple threads will execute the code in parallel. It is typically used for general parallelism and can contain multiple blocks of code to be executed concurrently.

Example: Parallel Region

program parallel_region_example
  integer :: i
  integer, dimension(1000) :: a, b, c

  !$omp parallel
  do i = 1, 1000
a(i) = b(i) + c(i)
  end do

  !$omp end parallel
end program parallel_region_example

Explanation:

• !$omp parallel: This directive marks the beginning of a parallel region. All threads in the parallel region will execute the code inside concurrently. In this case, all threads will work on the do loop.
• !$omp end parallel: Marks the end of the parallel region.

While the !$omp parallel do directive is specifically used for parallelizing loops, the !$omp parallel directive can be used when you want to execute multiple code blocks concurrently, not just loops.

3. Single Thread Execution: !$omp single

The !$omp single directive is used to specify that only a single thread should execute a particular block of code, even though the code is within a parallel region. This is useful for cases where certain tasks should only be done once, such as initialization or printing results.

Example: Single Thread Execution

program single_thread_example
  integer :: i
  integer, dimension(1000) :: a, b, c

  !$omp parallel
  do i = 1, 1000
a(i) = b(i) + c(i)
  end do

  !$omp single
  print *, "Parallel processing complete."

  !$omp end parallel
end program single_thread_example

Explanation:

• The !$omp single directive ensures that only one thread will print “Parallel processing complete.” Even though the code is within a parallel region, the task of printing the message is not parallelized.

---

Controlling the Number of Threads

OpenMP provides two ways to control the number of threads used in a parallel region: using the num_threads clause and the OMP_NUM_THREADS environment variable.

1. num_threads Clause

The num_threads clause is used directly within a parallel region to specify the number of threads to be used for that particular region or loop.

Example: Specifying the Number of Threads

program num_threads_example
  integer :: i
  integer, dimension(1000) :: a, b, c

  !$omp parallel do num_threads(4)
  do i = 1, 1000
a(i) = b(i) + c(i)
  end do
end program num_threads_example

Explanation:

• The num_threads(4) clause specifies that the parallelized do loop will use exactly 4 threads. You can adjust this number based on the number of available CPU cores or the problem size to optimize performance.

2. OMP_NUM_THREADS Environment Variable

The OMP_NUM_THREADS environment variable is a system-level variable that determines the number of threads used by OpenMP parallel regions. If you do not specify a number of threads in the code using the num_threads clause, OpenMP will fall back on the value of OMP_NUM_THREADS.

Example: Setting the OMP_NUM_THREADS Environment Variable

In a shell (before running your program), you can set the OMP_NUM_THREADS environment variable:

export OMP_NUM_THREADS=8

This command sets the number of threads to 8 for any OpenMP parallel regions in the program, unless overridden by the num_threads clause in the code.

Explanation:

• By setting OMP_NUM_THREADS=8, you are telling OpenMP to use 8 threads in any parallel region. This is a convenient way to control the number of threads globally without modifying the code.

---

Optimizing Parallel Execution

While OpenMP allows you to parallelize your code easily, there are several strategies for optimizing the performance of parallel execution. Here are some tips:

1. Workload Distribution

Ensure that the workload is evenly distributed among the threads. If one thread performs much more work than others, it can become a bottleneck, leading to poor performance. The default behavior of OpenMP usually ensures even distribution, but this can be customized using schedule clauses to control how iterations are divided.

Example: Customizing Work Distribution

!$omp parallel do schedule(static, 10)
do i = 1, 1000
  a(i) = b(i) + c(i)
end do

2. Avoiding False Sharing

False sharing occurs when multiple threads write to different variables that are located close to each other in memory. This can result in unnecessary cache coherence traffic, reducing performance. To avoid false sharing, ensure that variables updated by different threads are sufficiently spaced in memory.

3. Using the nowait Clause

If a loop does not require synchronization after execution, you can use the nowait clause to allow threads to continue without waiting for others.

!$omp parallel do nowait
do i = 1, 1000
  a(i) = b(i) + c(i)
end do

4. Profiling and Tuning

Use profiling tools to measure the performance of your parallel code and identify bottlenecks. You can adjust the number of threads, the scheduling strategy, and the workload distribution based on the results of the profiling.