License

Archer is distributed under the terms of the Apache License.

Please see LICENSE.txt for usage terms.

LLNL-CODE-773957

Introduction

Archer is an OMPT tool which annotates OpenMP synchronization semantics for data race detection. This avoids false alerts in data race detection. Archer is automatically loaded for OpenMP applications which are compiled with ThreadSanitizer option.

Build Archer within Clang/LLVM

This distribution of Archer is automatically built with the OpenMP runtime and automatically loaded by the OpenMP runtime.

Usage

How to compile

To use archer, compile the application with the extra flag -fsanitize=thread:

clang -O3 -g -fopenmp -fsanitize=thread app.c
clang++ -O3 -g -fopenmp -fsanitize=thread app.cpp

To compile Fortran applications, compile with gfortran, link with clang:

gfortran -g -c -fopenmp -fsanitize=thread app.f
clang -fopenmp -fsanitize=thread app.o -lgfortran

Runtime Flags

TSan runtime flags are passed via TSAN_OPTIONS environment variable, we highly recommend the following option to aviod false alerts for the OpenMP or MPI runtime implementation:

export TSAN_OPTIONS="ignore_noninstrumented_modules=1"

Runtime flags are passed via ARCHER_OPTIONS environment variable, different flags are separated by spaces, e.g.:

ARCHER_OPTIONS="flush_shadow=1" ./myprogram

Flag Name	Default value	Description
flush_shadow	0	Flush shadow memory at the end of an outer OpenMP parallel region. Our experiments show that this can reduce memory overhead by ~30% and runtime overhead by ~10%. This flag is useful for large OpenMP applications that typically require large amounts of memory, causing out-of-memory exceptions when checked by Archer.
print_ompt_counters	0	Print the number of triggered OMPT events at the end of the execution.
print_max_rss	0	Print the RSS memory peak at the end of the execution.
verbose	0	Print startup information.
enable	1	Use Archer runtime library during execution.

Example

Let us take the program below and follow the steps to compile and check the program for data races.

Suppose our program is called myprogram.c:

 1  #include <stdio.h>
 2
 3  #define N 1000
 4
 5  int main (int argc, char **argv)
 6  {
 7    int a[N];
 8
 9  #pragma omp parallel for
10    for (int i = 0; i < N - 1; i++) {
11      a[i] = a[i + 1];
12    }
13  }

We compile the program as follow:

clang -fsanitize=thread -fopenmp -g myprogram.c -o myprogram

Now we can run the program with the following commands:

export OMP_NUM_THREADS=2
./myprogram

Archer will output a report in case it finds data races. In our case the report will look as follow:

==================
WARNING: ThreadSanitizer: data race (pid=13641)
  Read of size 4 at 0x7fff79a01170 by main thread:
    #0 .omp_outlined. myprogram.c:11:12 (myprogram+0x00000049b5a2)
    #1 __kmp_invoke_microtask <null> (libomp.so+0x000000077842)
    #2 __libc_start_main /build/glibc-t3gR2i/glibc-2.23/csu/../csu/libc-start.c:291 (libc.so.6+0x00000002082f)

  Previous write of size 4 at 0x7fff79a01170 by thread T1:
    #0 .omp_outlined. myprogram.c:11:10 (myprogram+0x00000049b5d6)
    #1 __kmp_invoke_microtask <null> (libomp.so+0x000000077842)

  Location is stack of main thread.

  Thread T1 (tid=13643, running) created by main thread at:
    #0 pthread_create tsan_interceptors.cc:902:3 (myprogram+0x00000043db75)
    #1 __kmp_create_worker <null> (libomp.so+0x00000006c364)
    #2 __libc_start_main /build/glibc-t3gR2i/glibc-2.23/csu/../csu/libc-start.c:291 (libc.so.6+0x00000002082f)

SUMMARY: ThreadSanitizer: data race myprogram.c:11:12 in .omp_outlined.
==================
ThreadSanitizer: reported 1 warnings

Contacts and Support

• Google group

• Slack Channel

  • For an invitation please write an email to Simone Atzeni with a reason why you want to be part of the PRUNERS Slack Team.

• E-Mail Contacts:

  • Simone Atzeni
  • Joachim Protze