DKRZ projects

The major point of this release is to improve test coverage, fix edge cases and add corresponding regression tests, and complete missing parts in the API.

Minor feature additions:

• Adds convenience wrappers for arrays of single- and double-precision complex type to the Fortran xt_redist interfaces.
• Adds the xt_redist_collection_static_custom_new and xt_redist_collection_custom_new interfaces to the Fortran module. This is primarily for trivial translations from C code to Fortran: all constructors in these interfaces have already been available through the xt_redist_collection_static_new and xt_redist_collection_custom_new interfaces as well.
• Rounds out the aggregation of redist collections into redist collections.
• Make more of the sorting routines available to Fortran.
• When available and the default MPI error handler is overridden by the user, MPI errors also result in a stack trace.

This release also fixes a number of issues for the 0.12 release branch:

• Prevent spurious recompilation of identically generated sources.
• Amend the incomplete argument adjustment of xt_exchanger_copy from C to Fortran.
• Fix the Fortran interface for xt_xmap_intersection_new when one of the argument arrays has size 0.
• Fix copying of a block in an int array with an incorrect size for redists constructed from position extents and correspondingly fix the growth of a temporary buffer.
• Fix incorrect handling of datatype caching for nested redist collections.
• Prevents double-free of cached MPI datatypes.
• Fix handling of inequal offsets for the receiving and sending side of a transfer with the neighbour communicator exchanger.
• Fix index position queries for xt_idxempty to assign -1 to the position out parameter.
• Add deprecation notice for xt_idxlist_get_positions_of_indices_off and xt_idxlist_get_position_of_index_off
• Add configure-time detection of a bug in the nvfortran 26.1 to 26.7 compilers that prevents correct resolution of some generic interfaces.
• Fix unexpected coupling of configure tests for Fortran support for abstract interfaces and interface blocks for functions accepting bind(c) functions which return a struct as argument. This resulted in build failures for NEC nfort.
• Fix libtool handling of some Intel compiler command-line arguments which are available in a single argument variant since version 15 (e.g. -diag-disable=10101 instead of -diag-disable 10101).
• Fix a bug in the intersection size computed for a mapping with a negative iteration direction position extent.
• Fix a bug in the appending to stripes of length 1 resulting in an incorrect index sequence.
• Fix an edge case when an idxsection has alternating dimension iteration directions.
• Fix position queries for indices just beyond the bounds for index lists of type idxsection.
• Fix datatype queries for the redist collection type when queried for ranks for which no communication occurs.
• Fix behaviour of zero-stride stripes in intersection computation.
• Fix edge cases in the sorting query of index list collections when a list containing a single index value is part of the collection.
• Use a conversion to unsigned to enforce 2-complement arithmetic and prevent undefined behaviour from abs(INT_MIN).
• Document the changed caching behaviour introduced in the 0.12 release series.
• Fix an incorrect count when not all indices can be mapped to stripes.
• Fix an index duplication issue when large index values are used for intersections and single_match_only is requested.
• Minor memory leak when the bucket computation is changed more than once for a custom configuration object.
• New query xt_redist_get_bsend_buf_size for the size requirements imposed by a redist on the buffer attached with
  MPI_Buf_attach for buffered send operations.

This release fixes the following bugs for the 0.12.x series of releases:

• The xt_redist_get_send_MPI_Datatype and xt_redist_get_recv_MPI_Datatype queries are supposed to return MPI_DATATYPE_NULL for ranks with which no communication in the respective direction occurs. In release 0.12.0 such calls produced a fatal run-time error instead. 0.12.1 restores the specified behavior.
• The nvfortran 26.x Fortran compiler injects new exported data symbols into the library, these are now
  explicitly ignored by the test suite check for exported symbols.
• There's a bug in nvfortran 26.1, 26.3 and 26.5, at least, in which temporary array pointers to different slices of a encompassing array, passed to MPI_GET_ADDRESS, yield incorrect address results.
• In a slight extension of unspecified behavior: Redist collections of a single redist now actually return the datatypes for the first (and only component) instead of producing a fatal error.
• This leads to a minor extension of the test suite for redists.

This release adds two work-arounds for the nvfortran 26.x compilers. Users of other compilers have no immediate need to upgrade from 0.11.5.2.

This release

• adds count and offset to each exchanger message, simplifying the MPI datatypes in turn,
• adds an interface to iterate over the extended messages
• improves multi-threaded operations for exchangers
• improve code-sharing between exchangers by inserting an actual base class
• experimental: exposes the xt_exchanger family of classes (Fortran: USE xt_exchanger_abstract, C: #include <xt/xt_exchanger.h>)
• adds a configurable mode to put exchangers into synchronous operation mode for improved fault isolation
• adds documentation for the now public exchanger hierarchy
• make the alternate datatype generator previously only available with compile-time macro definition COMPACT_DT a run-time customizable part
• add a limit for the size of the MPI datatype cache, needed large, sparse indexed access into huge local arrays
• various improvements to the test suite for new bugs in new compilers (nvfortran mostly) and old bugs in old compilers (ifort mostly)
• improve the build tools for robustness
• refines and extends the configurable bucket generator interface
• permits suppressing the use of pthreads locking (user then needs to ensure creation of index lists or redists is non-concurrent via other means)
• various minor optimizations in handling of MPI datatypes

This release fixes a minor bug in the build system affecting build certain configurations. In these, a pre-processor macro that influences the conditional expansion of Fortran or C header module dependencies is defined by a command-line flag (usually using a -D switch) with a non-empty value. Such builds will fail with an error message.

If building with 0.11.5 succeeds, no upgrade to this patch release is necessary since only build tooling is different comparing 0.11.5.1 to 0.11.5.

This release

• contains two fixes relevant for intersections using the optimized paths introduced in 0.11,
• fixes potentially problematic use of realloc now exhibit canonical behaviour: whenever an allocation of size 0 is created, NULL is returned and realloc with size 0 uses free instead,
• brings some improvements of the documentation,
• introduces multiple changes that affect the build system (mostly in terms of ergonomics),
• properly detects two failures in the nvfortran compiler and uses work-arounds regardless of compiler macro definitions,
• changes handling of failure cases to print a backtrace before terminating the program,
• improves compatibility with macOS.

It's recommended for all users of older 0.11 releases to upgrade to this version.

This release updates fixes for the build system for newer compilers but remains functionally identical to the 0.11.3 release.

It became necessary to publish an intermediate release for ICON, since the next proper release with improvements for exchangers, derived data type creation and general improvements is not ready yet.

This release adds some minor improvements to the build and installation system, the most significant results are

• fewer rebuilds when used as external by better handling of time stamps and effectively unchanged files
• support of the flang-new line of LLVM Fortran compilers
• introduction of headers for C++ (yaxt.hpp)

Download: https://swprojects.dkrz.de/redmine/projects/yaxt/wiki/Downloads

Apart from bug fixes and compatibility improvements, this release expands on the custom bucket construction facility introduced in 0.11.0 and adds a corresponding performance test program.

This release addresses minor bugs in the internal derived data type packing/unpacking where

• MPI_FLOAT_INT might lead to a SIGBUS on some architectures.
• Reference tests for 0-size datatypes incorrectly use a buffer of size 0, but the MPI spec requires a buffer of the size provided by MPI_Pack_size.

Also, it includes some improvements for xt_idxsection and xt_idxstripes and a few refactorings that were release ready.

The internal derived data type packing/unpacking can now be disabled with the configure switch --disable-xt-ddt-exchanger.