Using Coarrays

Coarrays are not supported on OS X* systems.

Coarrays, a data sharing concept standardized in Fortran 2008, enable parallel processing using multiple copies of a single program. Each copy, called an image, has ordinary local variables and also shared variables called coarrays or covariables. A covariable, which can be either an array or a scalar, is a variable whose storage spans all the images in the program. In this Partitioned Global Address Space (PGAS) model, each image can access its own piece of a covariable as a local variable and can access those pieces that live on other images using coindices, which are enclosed in square brackets.

Intel® Fortran supports coarray programs that run using shared memory on a multicore or multiprocessor system. In some products (see the Feature Requirements section), coarray programs can also be built to run using distributed memory across a Linux* or Windows* cluster. They can also run on Linux* systems using Intel® Many Integrated Core Architecture (Intel® MIC Architecture).

Please refer to the product system requirements in the Release Notes for further details.

For more information on how to write programs using coarrays, see books on the Fortran 2008 language or the ISO Fortran 2008 standard.

Using Coarray Program Syntax

The additional syntax required by coarrays includes:

• CODIMENSION attribute and "[cobounds]" to declare an object a coarray (covariable)

• [coindices] notation to reference covariables on other images

• SYNC ALL, SYNC IMAGES, and SYNC MEMORY statements to provide points where images must communicate to synchronize shared data

• CRITICAL and END CRITICAL statements to form a block of code executed by one image at a time

• LOCK and UNLOCK statements to control objects called locks, used to synchronize actions on specific images

• ERROR STOP statement to end all images

• ALLOCATE and DEALLOCATE statements may specify coarrays

• Intrinsic procedures IMAGE_INDEX, LCOBOUND, NUM_IMAGES, THIS_IMAGE, and UCOBOUND

• ATOMIC_DEFINE and ATOMIC_REF for defining and referencing an atomic variable

Using the Coarray Compiler Options

You must use the -coarray (Linux) or /Qcoarray (Windows) compiler option (hereafter referred to as [Q]coarray) to enable the compiler to recognize coarray syntax. If you do not use this compiler option, a program that uses coarray syntax or features produces a compile-time error.

In the list that follows, only one option is valid on the command line; if multiple coarray compiler options are specified, the last one specified is used. An exception to this rule is the [Q]coarray compiler option using keyword single; if specified, this option takes precedence regardless of where it appears on the command line.

• Using [Q]coarray with no keyword is equivalent to running on one node (shared memory).

• Using [Q]coarray with keyword shared causes the underlying Intel® Message Passing Interface (MPI) parallelization to run on one node with multiple cores or processors with shared memory.

• Using [Q]coarray with keyword distributed requires a special license to be installed (see the Feature Requirements section) and causes the underlying Intel® MPI Library parallelization to run in a multi-node environment (multiple CPUs with distributed memory).

• Using -coarray=coprocessor (Linux*) specifies a configuration where the first images runs on the host and other images run on the host or the target. A configuration file specifies your exact configuration and determines where the images will be run. Use the -coarray-config-file=filename compiler option to specify a configuration file.

• Using [Q]coarray with keyword single creates an executable that will not be replicated, resulting in a single running image. (This is in contrast to the self-replicating behavior that occurs when any other coarray keyword is specified.) This option is useful for debugging purposes.

No special procedure is necessary to run a program that uses coarrays; you simply run the executable file. The underlying parallelization implementation uses the Intel® MPI Library. Installation of the compiler automatically installs the necessary Intel® MPI run-time libraries to run on shared memory. Products supporting clusters will also install the necessary Intel® MPI Library run-time libraries to run on distributed memory. Use of coarray applications with any other MPI implementation, or with OpenMP*, is not supported.

By default, the number of images created is equal to the number of execution units on the current system. You can override this by specifying a number using the [Q]coarray-num-images compiler option on the ifort command line that compiles the main program. You can also specify the number of images at execution time in the environment variable FOR_COARRAY_NUM_IMAGES.

Using a Configuration File

Use of the config-file option is appropriate only in a limited number of cases:

• As mentioned previously, you need to specify a configuration file when using the -coarray=coprocessor (Linux*) compiler option syntax.

• You can take advantage of Intel® MPI Library features in the coarray environment. To do so, specify the command line segments used by "mpiexec -config filename" in a file named filename and pass that file name to the Intel® MPI Library using the /[Q]coarray-config-file compiler option. If the [Q]coarray-num-images compiler option also appears on the command line, it will be overridden by what is in the configuration file. Rules for using an MPI configuration files are as follows:

  • The format of a configuration file is described in the Intel® MPI Library documentation; you will need to add the MPI option "-genv FOR_ICAF_STATUS launched" in the configuration file in order for coarrays to work on multi-node (distributed memory) systems.

  • You can also set the environment variable FOR_COARRAY_CONFIG_FILE to be the filename and path of the Intel® MPI Library configuration file you want to use at execution time.

Examples on Windows*:

• /Qcoarray:shared /Qcoarray-num-images:8 runs a coarray program on shared memory using 8 images.

• /Qcoarray:shared /Qcoarray-config-file:filename runs a coarray program on shared memory using the MPI configuration detailed in filename.

• /Qcoarray:distributed /Qcoarray-config-file:filename runs a coarray program on distributed memory using the Intel® MPI Library configuration detailed in filename.

Examples on Linux*:

• -coarray=shared -coarray-num-images=8 runs a coarray program on shared memory using 8 images.

• -coarray=distributed -coarray-num-images=8 runs a coarray program on distributed memory across 8 images.

• -coarray=coprocessor -coarray-config-file=filename runs a coarray program on a configuration where the first images runs on the host processor and other images run on some combination of host processors or coprocessors based on the Intel® Many Integrated Core Architecture (Intel® MIC Architecture). This command line specifies use of the Intel® MPI Library configuration detailed in filename.

-n 4 -host MYCPU -genv FOR_ICAF_STATUS=true ./myProg : \
-n 2 -host CARD1 -genv FOR_ICAF_STATUS=true /home/mydir/myProgMIC :\
-n 2 -host CARD2 -genv FOR_ICAF_STATUS=true /home/mydir/myProgMIC :\
-n 2 -host CARD3 -genv FOR_ICAF_STATUS=true /home/mydir/myProgMIC :\
-n 2 -host CARD4 -genv FOR_ICAF_STATUS=true /home/mydir/myProgMIC

!DIR$ OFFLOAD BEGIN
 <user code>
 !DIR$ END OFFLOAD