Release Notes for CPE 25.09 on SLES 15 sp7
Abnormal Termination Processing (ATP) 3.15.7
atp 3.15.7
================================================================================
Product Description
---------------------------
- Abnormal Termination Processing (ATP) is a system that
monitors user applications, and should an application
take a system trap, ATP preforms analysis on the dying
application. All of the stack backtraces of the
application processes are gathered into a merged
stack backtrace tree and written to disk as the file
"atpMergedBT.dot". The stack backtrace for the first process
to die is sent to stderr as is the number of the signal that
caused the death. If the core file size limit (RLIMIT_CORE)
is non-zero, a heuristically selected set of processes dump
their core.
atpMergedBT.dot can be viewed with 'statview', a component
of the STAT package (module load cray-stat). The merged stack
backtrace tree provides a concise, yet comprehensive, view
of what the application was doing at the time of its death.
Changelog
---------------------------
=======================================================================
## [unreleased]
=======================================================================
### Features
* Remove dependence on cdst-support package
=======================================================================
## [3.15.5]
=======================================================================
### Bug Fixes
* CPE-6064: Don't reset ZMQNet install directory
### Features
* CPE-7888: Use roccoremerge to merge CPU and GPU corefiles
=======================================================================
## [3.15.6]
=======================================================================
### Bug Fixes
* CPE-6064: Don't reset ZMQNet install directory
* PE-54139: Fix Fortran user API with AOCC Fortran compiler
### Features
* Build with g++14 on rhel 9.5 and sles15sp6
* Add rhel 9.5, remove rhel 8.9 and sles15sp4
* CPE-4541: Allow coredump using ROCgdb
* Support rhel 8.10
* Add builds for rhel 9.4 and sles15sp6
OS Dependencies
---------------------------
- HPE CSM systems with SLES 15 SP5 - SP6
- HPE HPCM systems with SLES 15 SP5 - SP6 or RHEL 8.10 - 9.4
- HPE Apollo systems with RHEL 8.10
Product Dependencies
---------------------------
- cray-mrnet >= 5.1, < 6 installed
- cray-cti >= 2.20, < 3 installed
Documentation
---------------------------
- Type `man atp` with the atp module loaded to read the atp(1) man page.
Installation Instructions
---------------------------
- rpm -ivh atp-3.15.7-20250827190612_2a9e01f471f3-190.sles15sp7.x86_64.rpm
To make atp 3.15.7 the default version of atp, execute:
/opt/cray/pe/admin-pe/set_default_files/set_default_atp_3.15.7
Legal Notices
---------------------------
Certain components, files or programs contained within this package or product are Copyright 2010-2025 Hewlett Packard Enterprise Development LP.
Attribution notices for open source licensed software for this package are detailed in the file:
/opt/cray/pe/atp/3.15.7/ATTRIBUTIONS_atp.txt
CDEUX driver (craype) 2.7.35
CDEUX driver (craype) 2.7.35 - 10
==============
Purpose:
--------
The Cray Development Environment User eXperience driver (CDEUX) (craype)
provides a common set of drivers for all CPE programming environments,
notably the C/C++ (cc/CC) and Fortran (ftn) languages.
Key Changes and Bugs Closed:
----------------------------
- Updated NVIDIA target support for current release
- Updated AOCC target support for current release
- Updated Intel Fortran frontend (ifx) for current release
- Updated target support adding Intel Emerald Rapids
- Fixed --help behavior to use compiler help
- Support CCE Cray LibSci pattern matching option -flibsci-libs
- Make build symbol support consistent between RHEL/SUSE builds of driver
- Fixed CAST-36714 segfault with conflicting compiler arguments
- Fixed numerous accelerator offload target behaviors with AMD
- Fixed CAST-30275 Support CRAY_ROCM_PATH
Product and OS Dependencies:
----------------------------
The CDEUX driver (craype) 2.7.35 release is supported on the following
HPE systems:
* HPE Cray EX systems
* HPE Apollo XD systems as part of the Cray Programming Environment
Documentation:
---------------
See manpages for cc, CC, ftn, and intro_craype-api.
See the pkg-config manpage or
http://www.freedesktop.org/wiki/Software/pkg-config for a pkg-config
introduction.
Product description:
--------------------
CrayPE contains drivers, cc, CC, and ftn to compile for all CPE
Programming Environments.
Certain components, files or programs contained within this package or product
are Copyright 2013-2025 Hewlett Packard Enterprise Development LP.
License:
--------
Except for the third party components and software licensed by HPE
through proprietary agreements, components, files or programs contained
within this package or product are Copyright 2025 Hewlett Packard
Enterprise Development LP.
CM PE Integration
CM-PE-INTEGRATION CHANGELOG
===========================
25.09 (2025-06-30)
* Add /etc/cray-pe.d/pe_overlay.conf for configuring CPE image overlay
* Allow setting the default CPE image using /etc/cray-pe.d/pe_overlay.conf
* Allow disabling CPE image mounts from overwriting existing images using
/etc/cray-pe.d/pe_overlay.conf
* Allow setting additional overlay paths using /etc/cray-pe.d/pe_overlay.conf
* Allow /etc/cray-pe.d/cpe_images to override the expected CPE image name
* Allow the OS version in the CPE image to use . - or sp to separate
the major and minor version numbers
* Fix issue where legacy images were mounted in the wrong order
* Fix PELOCAL_PRGENV for zsh
* Fix LMOD_SYSTEM_DEFAULT_MODULES trailing whitespace in zsh
25.03 (2025-02-05)
* Update deploy script to support locally built HPCM images
* numactl will detect and filter out nodes that have memory_failures
24.07 (2023-06-05)
* Remove rpm queries from 77-deploy-pe.sh for improved performance
* Remove find commands from 77-deploy-pe.sh for improved performance
* Replace the systemd file with a new pe.service that depends on pe.mount
* CAST-35461 fix setting MODULEPATH in csh.cshrc.local
* Replace find commands with for loops to allow using symlink paths in gen_modulefiles.sh
24.03 (2023-02-16)
* Fix regex in gen_modulefiles for Intel OneAPI
* Add a umask 022 to ensure CPE access is not limited
* Add nfs-utils as an rpm dependencies
23.12 (2023-11-27)
* Determine the craype-target architecture during shell initialization
* Add support for compute nodes running on aarch64 hardware
* Update gen_modulefiles.sh to better support new compiler versions
* Add better logging information to `77-deploy-pe`
23.05 (2023-05-10)
* Allow sites to override the lmod install path by setting `lmod_prefix_path`
in `cray-pe-configuration.sh` and `cray-pe-configuration.csh`. Sites can
now use newer versions of lmod provided by the OS distribution.
* Update `76-add-pe-to-fstab` so that `/pe` is not added to `/etc/fstab` more
then once.
23.03 (2023-03-03)
* Spread the squashfs image mounts across NUMA domains to balance memory
usage across NUMA nodes.
* Add RPM dependency on `numactl`
23.02 (2023-01-30)
* Update shell profile scripts to initialize lmod using HPE provided package.
* Remove leading and trailing `:` in `LMOD_SYSTEM_DEFAULT_MODULES` that is no
longer an acceptable format in new versions of lmod.
22.12 (2022-12-12)
* Add a `site_module_list` to `cray-pe-configuration.sh` and
`cray-pe-configuration.csh` to allow sites to add modules loaded at login
without the need to modify HPE defaults `init_module_list`.
22.11 (2022-11-10)
* Update `gen_modulefiles.sh` to recreating symlinks to ensure idempotence.
22.10 (2022-10-03)
* Changes for HPCM 1.8 to ensure that `/pe` is mounted in `76-add-pe-to-fstab`
before running `77-deploy-pe`
22.09 (2022-08-23)
* Update `pe_overlay.sh` to only remove workdirs when unmounts are successful.
* Update `gen_modulefiles.sh` to run `one_off_set_defaults` after generating
all modulefiles.
22.08 (2022-07-22)
* Enable older PE images to deploy on system with newer minor base OS version.
* Update `gen_modulefiles.sh` to use the latest available version of
`craypkg-gen`.
22.05 (2022-04-26)
* Remove `/opt/modulefiles` from `MODULEPATH`.
22.04 (2022-03-28)
* Add a temporary workaround in `pe_overlay.sh` that copies lmod GPU
modulefiles into the PE overlay's upper layer so they are visible after
`/opt/cray/pe` is mounted.
CPE Descriptive Manifest 1.3
cpe-descriptive-manifest 25.09
=================================
Release Date: Sep 2025
Product description:
--------------------
The cpe-descriptive-manifest packages contain the JSON manifest files that
are to map CPE to a format that Spack can consume.
Schema Changelog:
-----------------
v1.4
----
* Add 'wrappers' as a boolean parameter
v1.3
----
* Change '_meta' opject's 'version' property to 'schema-version'.
* Add 'cpe-version' property '_meta' object.
v1.2
----
* Add 'system-type' property to the '_meta' object.
v1.1
----
* Add '_meta' object to define file type and schema version.
v1.0
----
* Initial schema.
Cray Common Tool Interface (CTI) 2.20.0
cray-cti 2.20.0
================================================================================
Product Description
---------------------------
- Cray Common Tools Interface (CTI) is a library that abstracts debugger tool support
to a common interface regardless of the underlying work load manager. It facilitates
the staging of files and the launch of tool daemons on the compute nodes associated
with a parallel job. Options and interfaces can be found in the cti(1) and cti(3)
manpages.
Known Issues
---------------------------
- Race condition workaround for PALS 1.2.3 in CTI 2.17.0 (22.11). In PALS 1.2.3, there
is a race condition between the tool launcher releasing a job from the startup barrier
and the job actually getting to the startup barrier. This can result in the job receiving
the startup barrier release signal before it actually arrives there, resulting in the job
getting stuck in the barrier. As a workaround, the environment variable
CTI_PALS_BARRIER_RELEASE_DELAY can be set to add a delay between job startup and barrier
release. If set to a positve integer n, CTI will wait n seconds between starting a
job and releasing it from the barrier on PALS. A delay as small as one second works in
most cases.
Notes
---------------------------
- CTI 2.17.1 (PE 22.12) contains a new feature, file deduplication, for Slurm and PALS
workload managers. If enabled, CTI will run a check on compute nodes to determine if files
can be excluded from the shipped package. This can significantly improve application startup
time. To enable, set the environment variable CTI_DEDUPLICATE_FILES=1.
Changelog
---------------------------
=======================================================================
## [2.20.0]
=======================================================================
### Bug Fixes
* CAST-38116: Check Flux binary list return
* CAST-37841: Support batch/extern wrapper steps
* CPE-9311: Use hetjob leader ID for daemon launches
* CPE-9079: Use MPIR shim for daemon launch when Slurm using wrapper
### Features
* Remove dependence on cray-cdst-support package
=======================================================================
## [2.19.1]
=======================================================================
### Features
* Build with g++14 on rhel 9.5 and sles15sp6
=======================================================================
## [2.18.4]
=======================================================================
### Bug Fixes
* PE-54265: Don't add quotes to LD_PRELOAD
* PE-54079: Fix unreliable Flux application output
OS Dependencies
---------------------------
- HPE CSM systems with SLES 15 SP5 - SP6
- HPE HPCM systems with SLES 15 SP5 - SP6 or RHEL 8.10 - 9.4
- HPE Apollo systems with RHEL 8.10
Product Dependencies
---------------------------
- cray-cdst-support <cdst_version_range> installed
- cray-dyninst 12.3.0 or later installed
Documentation
---------------------------
- Currently the only documentation available is found in common_tools_fe.h, common_tools_be.h,
and the tests in the 'tests' directory.
Installation Instructions
---------------------------
- rpm -ivh cray-cti-2.20.0-20250606220817_cad2695e4bf2-222.sles15sp7.x86_64.rpm
To make cray-cti 2.20.0 the default version of cray-cti, execute:
/opt/cray/admin-pe/set_default_files/set_default_cray-cti-2.20.0
Legal Notices
---------------------------
Certain components, files or programs contained within this package or product are Copyright 2010-2025 Hewlett Packard Enterprise Development LP. SPDX-License-Identifier: Linux-OpenIB
Attribution notices for open source licensed software for this package are detailed in the file:
/opt/cray/pe/cti/2.20.0/ATTRIBUTIONS_cti.txt
Cray Comparative Debugger (CCDB) 5.0.7
cray-ccdb 5.0.7
================================================================================
Product Description
---------------------------
- CCDB, Cray's next generation debugging tool extends the comparative
debugging capabilities of gdb4hpc with a Graphical User Interface (GUI)
enabling programmers to compare corresponding data structures
between two executing applications. Comparative debugging assists users with
locating sections of code containing data deviations introduced by algorithm
changes, compiler differences, and porting to new architectures/libraries.
Changelog
---------------------------
=======================================================================
## [unreleased]
=======================================================================
### Features
* We'd like to thank anyone with interest in ccdb and browser based tooling. The central features of ccdb are now available via the gdb4hpc extension for VSCode now available in the marketplace.
=======================================================================
## [5.0.6]
=======================================================================
### Bug Fixes
* CPE-6964: Specify Flask version, disable Werkzeug server
* Missing yaml file for apollo install
### Features
* Add rhel 9.5, remove rhel 8.9 and sles15sp4
=======================================================================
## [5.0.5]
=======================================================================
### Bug Fixes
* PE-54348: Fix manpage path
* PE-50334: Fix missing console messages
### Features
* Support rhel 8.10
* Add builds for rhel 9.4 and sles15sp6
* Remove old gtk based ccdb-x
* PE-51960: Added support for rhel89/93 & gcc13
OS Dependencies
---------------------------
- HPE CSM systems with SLES 15 SP5 - SP6
- HPE HPCM systems with SLES 15 SP5 - SP6 or RHEL 8.10 - 9.4
- HPE Apollo systems with RHEL 8.10
Product Dependencies
---------------------------
- gdb4hpc/4.14.x or later installed
Documentation
---------------------------
- Type `man ccdb` with the cray-ccdb module loaded to read the ccdb(1) man page.
- Documentation is available through CCDB from the 'Help' menu.
Installation Instructions
---------------------------
- rpm -ivh --oldpackage cray-ccdb-5.0.7-20250902163938_bf97da4d615b-77.sles15sp7.x86_64.rpm
To make cray-ccdb 5.0.7 the default version of cray-ccdb, execute:
/opt/cray/pe/admin-pe/set_default_files/set_default_cray-ccdb_5.0.7
Legal Notices
---------------------------
Certain components, files or programs contained within this package or product are Copyright 2007-2025 Hewlett Packard Enterprise Development LP.
Attribution notices for open source licensed software for this package are detailed in the file:
/opt/cray/pe/ccdb/5.0.7/ATTRIBUTIONS_ccdb.txt
Cray DSMML 0.3.1
Cray DSMML 0.3.1:
==========================
Release Date:
-------------
March, 2025
Product Information:
--------------------
DSMML is a stand-alone memory management library for maintaining distributed
shared symmetric memory heaps for top level PGAS languages and libraries like
Coarray Fortran, UPC, and OpenSHMEM. DSMML allows user libraries to create
multiple symmetric heaps and share information with other libraries. Through
DSMML, we could extract interoperability between PGAS programming models.
Announcements, release informations, supported environments, and backward
compatibility informations about this product can be viewed in the following
location: https://pe-cray.github.io/cray-dsmml/
Purpose:
--------
Cray DSMML 0.3.1 supports the following features:
- Create symmetric heap segments for being used by PGAS languages and
libraries
- Allows options to allocate, deallocate memory buffers from previously
created symmetric heap segments
- Enables sharing information about the symmetric heap segments between
different programming models
- Added support for shared symmetric heap (SSHEAP) that can be used to
provide enhanced SMP data transport in user-libraries like SHMEM, CAF, UPC
The following features are fixed in Cray DSMML 0.3.1 when compared
to Cray DSMML 0.3.0:
- Add support for differnt build env (RHEL, SLES 15 SP6)
Refer intro_dsmml(3) manpage for more information on the supported features
and syntax for different routines
Product and OS Dependencies:
----------------------------
The Cray DSMML 0.3.1 release is supported on the following
Cray systems:
* HPE Cray EX systems with CLE
Product and OS Dependencies by network type:
--------------------------------------------------+
| Shasta |
----------------------------+---------------------+
craype | >= 2.7.3 |
----------------------------+---------------------+
One or more compilers:
* AOCC 2.2 or later
* CCE 9.1 or later
* GNU 9.1 or later
* Intel 19.0 or later
* Nvidia 20.7 or later
Documentation:
--------------
Use the Cray DSMML man pages for more information on the library
functions and use intro_dsmml(3) man page for general information.
Use https://pe-cray.github.io/whitepapers/ for access to different
Cray DSMML specific whitepapers
Announcements, release informations, supported environments, and backward
compatibility informations about this product can be viewed in the following
location: https://pe-cray.github.io/cray-dsmml/
Modulefile:
-----------
module load cray-dsmml/0.3.1
Certain components, files or programs contained within this package or
product have the following Copyright:
Copyright 2018-2021,2024-2025 Hewlett Packard Enterprise Development LP.
Cray Dyninst 12.3.6
cray-dyninst 12.3.6
================================================================================
Product Description
---------------------------
- cray-dyninst provides tools for binary instrumentation, analysis, and modification.
Changelog
---------------------------
=======================================================================
## [unreleased]
=======================================================================
### Features
* Remove dependence on cray-cdst-support package
=======================================================================
## [12.3.5]
=======================================================================
### Features
* Add build for rhel 9.5; remove build for rhel 8.9 and sles15sp4
=======================================================================
## [12.3.4]
=======================================================================
### Features
* Support rhel 8.10
* Build on sles15sp6 and rhel9.4
OS Dependencies
---------------------------
- HPE CSM systems with SLES 15 SP5 - SP6
- HPE HPCM systems with SLES 15 SP5 - SP6 or RHEL 8.10 - 9.4
- HPE Apollo systems with RHEL 8.10
Documentation
---------------------------
- https://github.com/dyninst/dyninst
Installation Instructions
---------------------------
- rpm -ivh cray-dyninst-12.3.6-20250604222258_2e98b0e19aff-99.sles15sp7.x86_64.rpm
To make this the default version of cray-dyninst, execute:
/opt/cray/pe/admin-pe/set_default_files/set_default_cray-dyninst_12.3.6
Legal Notices
---------------------------
Certain components, files or programs contained within this package or product are Copyright 2007-2025 Hewlett Packard Enterprise Development LP.
Attribution notices for open source licensed software for this package are detailed in the file:
/opt/cray/pe/dyninst/12.3.6/ATTRIBUTIONS_dyninst.txt
Cray LibSci 25.09.0
Cray LibSci 25.09.0
=====================
Purpose:
--------
Cray LibSci 25.09.0 provides scientific libraries for Cray
HPC systems. Cray LibSci is supported on the host CPU but
not on the accelerator of these systems.
The Cray LibSci 25.09.0 release provides updated support for
the following:
* SLES 15 SP7 (x86_64 and aarch64)
* RHEL 9.6 (x86_64 and aarch64)
* AMD ROCm 6.4
* Intel oneAPI 2025.1
* AMD AOCC 5.0
* Nvidia HPC SDK 25.5
* GCC 14
Cray LibSci 25.09.0 includes the following versions of publicly
available libraries:
* LAPACK 3.12.1 - For further information, see
http://www.netlib.org/lapack
* ScaLAPACK 2.2.0 - (Scalable LAPACK) For further information, see
http://www.netlib.org/scalapack.
* QDWH 2.0.0 KSVD 1.0.0 - Polar decomposition and SVD packages, see
https://github.com/ecrc/qdwh
https://github.com/ecrc/ksvd
Key Changes and Bugs Closed:
----------------------------
Changes in Cray LibSci 25.09.0
* Update to LAPACK 3.12.1
Changes in Cray LibSci 25.03.0
* Support for Intel Emerald Rapids
* CAST-36293 - Issue calling CGEMM with multiple threads
Changes in Cray LibSci 24.11.0
* Support for AMD MI300A and Turin
* Miscellaneous bug fixes
Changes in Cray LibSci 24.07.0
* Miscellaneous bug fixes
* Enabled module to set LD_LIBRARY_PATH when PE_LD_LIBRARY_PATH=system
* Removed Iterative Refinement Toolkit (IRT) support
Changes in Cray LibSci 24.03.0
* CAST-34956 - Lua module environment variable issue
Changed in Cray LibSci 23.12.5
* CAST-33777 - LibSci mutex contention issue
* Removed support for Intel KNL
Product and OS Dependencies:
----------------------------
The Cray LibSci 25.09.0 release requires the following:
* SLES 15 SP6/SP7 or RHEL 8.10/9.5/9.6
* SLES 15 SP6 (x86_64 or aarch64)
* SLES 15 SP7 (x86_64 or aarch64)
* RHEL 8.10 (x86_64)
* RHEL 9.5 (x86_64 or aarch64)
* RHEL 9.6 (x86_64 or aarch64)
* cray-mpich/8.1.0 or later
Supported on the following HPE systems:
* HPE Cray EX systems with CLE
* HPE Cray XD systems with CLE
One or more of the following compiler major versions:
* CCE 20.x or later
* GCC 12.3 or later (SLES)
* gcc-toolset-10.3 to 14.2 (RHEL 8.10)
* gcc-toolset-12.2 to 14.2 (RHEL 9.5)
* gcc-toolset-12.2 to 14.2 (RHEL 9.6)
* AOCC 4.1.0 or later
* AMD ROCm 6.0.0 or later
* Intel 2025.0.0 or later
* Nvidia 23.11 or later (aarch64) and 23.3 or later (x86_64)
Notes and Limitations:
----------------------
OMP threaded versions:
The CrayPE 2.1.2 and later releases add support for link line generation
for the multi-threaded versions of the libsci library based on the OpenMP
flags the user specifies for each compiler:
* CCE by default links to the OpenMP LibSci library. CrayPE will link in
the serial version of LibSci when the CCE flag -hnoomp is used.
* GNU by default links serial LibSci library. CrayPE will link in the
OpenMP version of LibSci when the GNU flag -fopenmp is used.
* INTEL by default links serial LibSci library. CrayPE will link in the
OpenMP version of LibSci when the INTEL flag -qopenmp is used.
Stack size limit:
The 'ulimit -s unlimited' system stack size setting is required for
cray-libsci on all supported Cray platforms. This is typically set as a
system default by admin but may otherwise need to be set at runtime.
Non-default dynamic linking:
When using a non-default version of cray-libsci with dynamic linking
users should set at runtime and at linktime after loading the desired
cray-libsci module--or set the equivalent for their linux shell:
export LD_LIBRARY_PATH=$CRAY_LD_LIBRARY_PATH:$LD_LIBRARY_PATH
Documentation:
--------------
See the intro_libsci man page for additional information.
Modulefile:
-----------
module load cray-libsci
Installation instructions:
--------------------------
LibSci is now packaged into separate compiler specific RPMs to allow
rpmbuild to correctly include compiler dependencies.
License:
--------
Except for the third party components and software licensed by HPE
through proprietary agreements, components, files or programs
contained within this package or product are Copyright 2001-2025
Hewlett Packard Enterprise Development LP.
Attribution notices for open source licensed software contained in this
package are detailed in the file:
/opt/cray/libsci/25.09.0/ATTRIBUTIONS
Cray LibSci for Accelerators 25.09.2
Cray LibSci_ACC 25.09.2
=======================
Purpose:
--------
The Cray LibSci_ACC 25.09.2 release provides accelerated versions of
scientific libraries for HPE Cray systems with accelerators.
Product and OS Dependencies:
----------------------------
Cray LibSci_ACC 25.09.2 requires the following platform:
HPE Cray EX system with AMD MI250X & MI300A
HPE Cray EX system with Nvidia A100 & H100
Cray LibSci_ACC 25.09.2 supports the following OS:
RHEL 8.10 x86_64
RHEL 9.5 x86_64 & aarch64
RHEL 9.6 x86_64 & aarch64
SLES 15SP6 x86_64 & aarch64
Notes and Limitations:
----------------------
LAPACK workspace calculations from other implementations or hard-coded
values may not be compatible with cray-libsci_acc. Workspaces reported from
a query to the corresponding cray-libsci_acc function should be used.
The current supported usecase is a one-rank to one-gpu mapping for apps.
Performance improvements can be achieved in programs calling cray-libsci_acc
subroutines by using pinned memory. See the intro_libsci_acc man page for
details.
Use of the aprun option "-cc none" or the srun option "--cpu_bind=none"
to disable core affinity is strongly suggested to maintain performance.
When using non-default cray-libsci_acc versions users should set equivalent
at runtime and linktime after loading the desired cray-libsci_acc module:
export LD_LIBRARY_PATH=$CRAY_LD_LIBRARY_PATH:$LD_LIBRARY_PATH
cray-libsci_acc may generate wrong numerical results on AMD GPUs with
ROCm 6.4.0 and 6.4.1. To resolve this, users can continue using ROCm
6.4.0 or 6.4.1 at compile time and downgrade only the rocBLAS library to
an earlier ROCm release at application runtime. For example, if ROCm 6.3.3
is installed at "/opt/rocm-6.3.3", users can still load the rocm/6.4.0
module during compilation and prefix the run command by
"LD_PRELOAD=/opt/rocm-6.3.3/lib/librocblas.so.4" so the executable can
pickup rocBLAS from the ROCm 6.3.3 release and continue using other ROCm
components from the ROCm 6.4.0 release.
Documentation:
--------------
See the intro_libsci_acc man page for additional information.
Modulefile:
-----------
module load cray-libsci_acc
Sample Installation instructions:
---------------------------------
RPM install command:
rpm -ivh cray-libsci-acc-25.09.2-*.##ARCH##.rpm
The "*" in the install command represents compiler version combinations.
To make this the default version:
##PREFIX##/admin-pe/set_default_files/set_default_libsci_acc_25.09.2
License:
--------
Except for the third party modules and software licensed by HPE
through proprietary agreements, components, files or programs
contained within this package or product are Copyright -2025 Hewlett
Packard Enterprise Development LP.
Attribution notices for open source licensed software contained in
this package are detailed in the file:
##PREFIX##/libsci_acc/25.09.2/ATTRIBUTIONS_libsci_acc_25.09.2.txt
Cray Lmod 8.7.60
Cray Lmod 8.7.60
================================================================================
Release Date:
--------------------------------------------------------------------------------
September 2025
Purpose:
--------------------------------------------------------------------------------
Provide a Lmod 8.7.60 package for CPE customers
Bugs fixed in this release:
--------------------------------------------------------------------------------
N/A
Documentation:
--------------------------------------------------------------------------------
See CPE's Installation and User Guides for documentation.
Also see Lmod's official website: https://lmod.readthedocs.io
Dependencies:
--------------------------------------------------------------------------------
The Lmod Scripts 8.7.60 release is dependent on the following
software:
bc
sed
lua >= 5.3
lua-luaposix
lua-luafilesystem
Installation instructions:
--------------------------------------------------------------------------------
rpm -Uvh cray-lmod-8.7.60-1.sles15sp7.x86_64.rpm
To make this the default version, update the symbolic "lmod" link
to the desired Lmod version directory.
Cray Lmod Scripts 3.2.1.0
Lmod Scripts 3.2.1.0
================================================================================
Release Date:
--------------------------------------------------------------------------------
December, 2023
Purpose:
--------------------------------------------------------------------------------
Updates package license.
Bugs fixed in this release:
--------------------------------------------------------------------------------
Documentation:
--------------------------------------------------------------------------------
See system Installation and User Guides for documentation
Dependencies:
--------------------------------------------------------------------------------
The Lmod Scripts 3.2.1.0 release is dependent on the following
software:
set_default_3
Lua
Lmod
Installation instructions:
--------------------------------------------------------------------------------
rpm -Uvh lmod_scripts-3.2.1.0-202506282057.5db31ff7c9241-0.noarch.rpm
To make this the default version, execute:
/opt/cray/pe/admin-pe/set_default_files/set_default_lmod_scripts_3.2.1.0
Product description:
--------------------------------------------------------------------------------
Contains Lua scripts which support the Cray implementation of Lmod
Copyright 2020-2023 Hewlett Packard Enterprise Development LP
Cray MRNet 5.1.6
cray-mrnet 5.1.6
================================================================================
Product Description
---------------------------
- MRNet is a customizable, high-performance software infrastructure for
building scalable tools and applications. It supports efficient
multicast and data aggregation functionality using a tree of processes
between the tool's front-end and back-ends. MRNet-based tools may use
these internal processes to distribute many important tool activities,
for example to reduce data analysis time and keep tool front-end loads
manageable.
MRNet-based tool components communicate across logical channels called
streams. At MRNet internal processes, filters are bound to these streams to
synchronize and aggregate dataflows. Using filters, MRNet can efficiently
compute averages, sums, and other more complex aggregations and analyses
on tool data. MRNet also supports facilities that allow tool developers
dynamically load new tool-specific filters into the system.
Changelog
---------------------------
=======================================================================
## [unreleased]
=======================================================================
### Features
* Remove dependence on cdst-support
=======================================================================
## [5.1.6]
=======================================================================
### Features
* Remove dependence on cdst-support package
=======================================================================
## [5.1.5]
=======================================================================
### Features
* Add rhel 9.5, remove rhel 8.9 and sles15sp4
OS Dependencies
---------------------------
- HPE CSM systems with SLES 15 SP5 - SP6
- HPE HPCM systems with SLES 15 SP5 - SP6 or RHEL 8.10 - 9.4
- HPE Apollo systems with RHEL 8.10
Product Dependencies
---------------------------
- cray-cti >= 2.20, < 3 installed
Installation Instructions
---------------------------
- rpm -ivh cray-mrnet-5.1.6-20250605181848_b81ea2c8d0f9-75.sles15sp7.x86_64.rpm
To make cray-mrnet 5.1.6 the default version of cray-mrnet, execute:
/opt/cray/pe/admin-pe/set_default_files/set_default_cray-mrnet_5.1.6
Legal Notices
---------------------------
Certain components, files or programs contained within this package or product are Copyright 2022-2025 Hewlett Packard Enterprise Development LP.
Attribution notices for open source licensed software for this package are detailed in the file:
/opt/cray/pe/mrnet/5.1.6/ATTRIBUTIONS_mrnet.txt
Cray OpenSHMEMX 11.8.0
Cray OpenSHMEMX 11.8.0:
===============================
Release Date:
-------------
September, 2025
Purpose:
--------
OpenSHMEM is a Partitioned Global Address Space (PGAS) library interface
specification, which is the culmination of a standardization effort among
many implementers and users of SHMEM programming model. SHMEM has a long
history as a parallel programming model on HPE Cray supercomputer systems.
For the past two decades SHMEM library implementation in HPE Cray systems
evolved through different generations. Cray OpenSHMEMX is a proprietary,
OpenSHMEM specification compliant SHMEM implementation for HPE Cray
product line.
Key Changes:
------------
Major differences in Cray OpenSHMEMX 11.8.0 from
Cray OpenSHMEMX version 11.7.4 includes the following:
- Add support for true non-blocking AMOs
- Add support for AMD Turin processors
- Enable default support on Non-Cray OS platforms (SLES, RHEL, and Rocky)
- Enable default support with Libfabric/2.0 builds
Bugs Closed:
------------
The following bugs are fixed as part of the Cray OpenSHMEMX 11.8.0
release:
- Fix shmem_test_any completion issue for repeated calls
- Fix team-split 2D creation and internal resource management
Product and OS Dependencies:
----------------------------
The Cray OpenSHMEMX 11.8.0 release is supported on the following
Cray systems:
* HPE Cray EX systems with CLE
Product and OS Dependencies by network type:
--------------------------------------------------+
| HPE Slingshot |
----------------------------+---------------------+
craype | >= 2.7.3 |
----------------------------+---------------------+
cray-pals | >= 0.3.5 |
----------------------------+---------------------+
cray-pmi | >= 6.0.1 |
----------------------------+---------------------+
libfabric | >= 1.9.0 |
----------------------------+---------------------+
cray-dsmml | >= 0.2.2 |
----------------------------+---------------------+
One or more compilers:
* AOCC 2.2 or later
* CCE 9.1 or later
* GNU 9.1 or later
* Intel 19.0 or later
* Nvidia 20.7 or later
Notes and Limitations:
----------------------
Announcements, release information, supported environments, limitations,
usage models, and backward compatibility informations about this product
can be viewed in the following location:
https://pe-cray.github.io/cray-openshmemx/
Documentation:
--------------
Use the Cray OpenSHMEMX man pages for more information on the library
functions and use intro_shmem(3) man page for general information.
Initial support for Cray OpenSHMEMX man pages are derived from OpenSHMEM
standards specification document
Use https://pe-cray.github.io/whitepapers/ for access to different
Cray OpenSHMEMX specific whitepapers
Modulefile:
-----------
module load cray-openshmemx/11.8.0
License:
--------
Components, files, and programs contained within this package or
product are Copyright Hewlett Packard Enterprise Development LP
Cray PAPI 7.2.0.2
cray-papi
=================
Release date:
-------------
September 2025
Package:
--------
cray-papi-7.2.0.2-202508292113.6556598f5ed8f-4 x86_64
Purpose:
--------
New version of papi 7.2.0.2 release 4
Documentation:
--------------
Overview: https://github.com/icl-utk-edu/papi
Web Site: https://icl.utk.edu/papi
Product description:
--------------------
PAPI aims to provide the tool designer and application engineer with a
consistent interface and methodology for use of the performance counter
hardware found in most major microprocessors. PAPI enables software
engineers to see, in near real time, the relation between software
performance and processor events.
Dependencies:
-------------
For a list of software used when validating this version of
PAPI on Cray and HPE systems, see the HPE Cray Programming
Environment release announcements.
Copyright 2015-2017,2019-2025 Hewlett Packard Enterprise Development LP
Cray PE Targets 1.16.0
CrayPE Targets 1.16.0
================================================================================
Release Date:
--------------------------------------------------------------------------------
September, 2025
Purpose:
--------------------------------------------------------------------------------
Updates support for CPE hugepages support. Removes craype-broadwell and craype-network-infiniband module-files.
Bugs fixed in this release:
--------------------------------------------------------------------------------
None
Dependencies:
--------------------------------------------------------------------------------
The CrayPE Targets 1.16.0 release is dependent on the following
software products:
CrayPE
set_default_3
lmod_scripts
Documentation:
--------------------------------------------------------------------------------
See manpages for cc, CC, ftn, intro_craype-api, intro_hugepages and pkg-config
See section 2.6 Using Targeting Modules of the Cray Programming Environment
User's Guide (S-2529-116)
See http://www.freedesktop.org/wiki/Software/pkg-config for a pkg-config
introduction.
Installation instructions:
--------------------------------------------------------------------------------
rpm -Uvh craype-targets-ex-1.16.0-202506280056.0c5f6d12a83bf-1.ex.noarch.rpm
To make this the default version, execute:
/opt/cray/pe/admin-pe/set_default_files/set_default_craype-targets_1.16.0-tcl
/opt/cray/pe/admin-pe/set_default_files/set_default_craype-targets_1.16.0-lua
Product description:
--------------------------------------------------------------------------------
CrayPE Targets contains the modules for CrayPE targets.
TCL Modules - craype-accel-host craype-accel-amd-gfx90a craype-accel-amd-gfx908 craype-accel-amd-gfx940 craype-accel-amd-gfx942 craype-accel-nvidia70 craype-accel-nvidia80 craype-accel-nvidia90 craype-hugepages1G craype-hugepages2M craype-hugepages32M craype-hugepages512M craype-arm-grace craype-x86-milan craype-x86-milan-x craype-x86-rome craype-x86-spr craype-x86-spr-hbm craype-x86-trento craype-x86-turin craype-x86-genoa craype-network-none craype-network-ofi craype-network-ucx
Lmod Modules - craype-accel-host craype-accel-amd-gfx90a craype-accel-amd-gfx908 craype-accel-amd-gfx940 craype-accel-amd-gfx942 craype-accel-nvidia70 craype-accel-nvidia80 craype-accel-nvidia90 craype-hugepages1G craype-hugepages2M craype-hugepages32M craype-hugepages512M craype-arm-grace craype-x86-milan craype-x86-milan-x craype-x86-rome craype-x86-spr craype-x86-spr-hbm craype-x86-trento craype-x86-turin craype-x86-genoa craype-network-none craype-network-ofi craype-network-ucx
Copyright 2013-2023 Hewlett Packard Enterprise Development LP
Cray PMI 6.1.16
Cray PMI 6.1.16
==============================================================================
Release Date:
--------------
June 28, 2025
Purpose:
--------
The HPE Cray Process Manager Interface library provides the interface
between the application launcher and other communication libraries
such as MPI and OpenSHMEM.
HPE Cray PMI 6.1.16 is optimized for the HPE EX product line.
Key Changes and Bugs Closed:
----------------------------
Bug fixes new to Cray PMI 6.1.16 include:
- CPE-6890 - Add Spawn support for the Flux launcher
- CPE-8144 - Clean up PMI socket creation in PMI_Init
- CPE-8333 - Add PMI_MAX_PORT_NAME to pmi.h
- CPE-9423 - Add man page documentation for PMI_CONNECT_RETRIES,
PMI_CONNECT_TIMEOUT, and PMI_MMAP_SYNC_WAIT_TIME
Product and OS Dependencies:
----------------------------
The Cray PMI 6.1.16 release is supported on the following systems:
* HPE Cray EX systems with CLE
* HPE Apollo systems as part of the Cray Programming Environment
Product Dependencies:
----------------------------------------+
cray-pals | >= 1.0.6 |
-----------------------+----------------+
slurm | >= 20.02 |
-----------------------+----------------+
Notes and Limitations:
----------------------
Spawning MPMD applications when using the Flux application launcher
is not officially supported. The Flux maintainers have developed a
mprun.py script that may make it work in some situations. Sites are
expected to install the script themselves and ensure its location is
in PATH. It can be retrieved from Flux GitHub issue 2414 at
https://github.com/flux-framework/flux-core/issues/2414 .
Documentation:
--------------
For more information see the intro_pmi man page.
Modulefile:
-----------
module load cray-pmi/6.1.16
License:
--------
Except for the third party components and software licensed by HPE
through proprietary agreements, components, files or programs contained
within this package or product are Copyright 2020-2021 Hewlett Packard
Enterprise Development LP.
Attribution notices for open source licensed software for this
package are detailed in the file:
/opt/cray/pe/pmi/6.1.16/ATTRIBUTIONS
Cray Python 3.11.7
cray-python 3.11.7
====================
Release Date
------------
July 2024
Purpose
-------
Cray Python is an implementation of the Python programming language for the Cray
Programming Environment. The numpy and scipy modules are configured to call Cray
Libsci routines. The mpi4py module is configured to call Cray MPICH routines.
Cray Python is designed to run Python codes on the compute nodes of an HPE
supercomputer. HPE does not make changes to the Python source or any of its
libraries nor does it plan to make changes in future releases.
The cray-python 3.11.7 release contains
- python-3.11.7
- numpy-1.24.4
- scipy-1.10.1
- mpi4py-3.1.4
- dask-2023.6.1
Product and OS Dependencies
---------------------------
The cray-python 3.11.7 release is supported on
- Cray EX systems running SLES 15 or RHEL 8
Documentation
-------------
https://www.python.org/doc
Changelog
---------
https://docs.python.org/release/3.11.7/whatsnew/changelog.html
Modulefile
----------
module load cray-python/3.11.7
Installation
------------
rpm -ihv cray-python-3.11.7-20240617153007.0687944-1.sles15sp4.x86_64.rpm
To make this the default version, execute
[PREFIX]/admin-pe/set_default_files/set_default_python_3.11.7
Certain components, files, or programs contained within this package are
© Copyright 2021-2024 Hewlett Packard Enterprise Development LP
Cray R 4.4.0
cray-R 4.4.0
============
Release Date
------------
March 2024
Purpose
-------
The cray-R 4.4.0 release.
cray-R is configured to call cray-libsci routines.
Product and OS Dependencies
---------------------------
The cray-R 4.4.0 release is supported on
- Cray EX systems running SLES 15 or RHEL 8
Documentation
-------------
https://cran.r-project.org/manuals.html
Modulefile
----------
module load cray-R/4.4.0
Installation
------------
rpm -ihv cray-R-4.4.0-20240617153119.a134495-1.sles15sp4.x86_64.rpm
To make this the default version, execute:
[PREFIX]/admin-pe/set_default_files/set_default_R_4.4.0
Certain components, files or programs contained within this package or product are
Copyright 2018-2024 Hewlett Packard Enterprise Development LP
Cray Stat 4.12.6
cray-stat 4.12.6
================================================================================
Product Description
---------------------------
- The Stack Trace Analysis Tool (STAT) is a highly scalable, lightweight
tool that gathers and merges stack traces from all of the processes of
a parallel application to form call graph prefix trees.
Notes
---------------------------
- To enable gathering stack traces on an Nvidia GPU, the environment variable
STAT_GDB must be set to either cuda-gdb, which will require cuda-gdb to be
available in PATH. Or it can be set a specific debugger. We suggest the
version of cuda-gdb packaged with gdb4hpc, as in:
export STAT_GDB=/opt/cray/pe/gdb4hpc/$GDB4HPC_VERSION/libexec/cuda110-gdb-guard
After which stat-cl would be called using the options: -w -G -i
Changelog
---------------------------
=======================================================================
## [4.12.6]
=======================================================================
### Features
* Remove dependence on cdst-support
=======================================================================
## [4.12.5]
=======================================================================
### Bug Fixes
* CPE-5994: Use Flux attach, diagnostics when Flux cannot initialize
### Features
* Build with g++14 on rhel 9.5 and sles15sp6
* Add rhel 9.5, remove rhel 8.9 and sles15sp4
=======================================================================
## [4.12.4]
=======================================================================
### Features
* Support rhel 8.10
* Build on rhel 9.4 and sles15sp6
* PE-53556: STAT dotfiles compatible with other dotfile viewers
OS Dependencies
---------------------------
- HPE CSM systems with SLES 15 SP5 - SP6
- HPE HPCM systems with SLES 15 SP5 - SP6 or RHEL 8.10 - 9.4
- HPE Apollo systems with RHEL 8.10
Product Dependencies
---------------------------
- cray-mrnet >= 5.1, < 6 installed
- cray-cti 2.20, < 3 installed
Documentation
---------------------------
- https://computing.llnl.gov/code/STAT/
Installation Instructions
---------------------------
- rpm -ivh --oldpackage cray-stat-4.12.6-20250827185605_25d2d93078e6-102.sles15sp7.x86_64.rpm
To make cray-stat 4.12.6 the default version of cray-stat, execute:
/opt/cray/pe/admin-pe/set_default_files/set_default_cray-stat_4.12.6
Legal Notices
---------------------------
Certain components, files or programs contained within this package or product are Copyright 2007-2025 Hewlett Packard Enterprise Development LP.
Attribution notices for open source licensed software for this package are detailed in the file:
/opt/cray/pe/stat/4.12.6/ATTRIBUTIONS_stat.txt
Cray zmqnet 1.3.2
cray-zmqnet 1.3.2
================================================================================
Product Description
---------------------------
- zmqnet is a library for creating a network tree between applications
and HPC jobs using zmq.
Changelog
---------------------------
=======================================================================
## [1.3.2]
=======================================================================
### Bug Fixes
* CPE-8343: Zero-initialize state struct before passing to `initstate_r`
### Features
* Remove dependence on cdst-support package
=======================================================================
## [1.3.1]
=======================================================================
### Bug Fixes
* CPE-8040: Use random_r instead of crashing rand in ZMQ
=======================================================================
## [1.0.0]
=======================================================================
### Bug Fixes
* CPE-7182: Remove TimeoutAfterLast from variant, create non-variant interface
* Missing apollo install yamls PE-55041
* PE-54822: Throw exception when fail to detect hostname
* PE-53720: Include capnp tools in support directory
* PE-53603: Show diagnostic when libsodium cannot be loaded
* PE-51694: Make pings thread safe
* PE-51165: Assign rank and PID in packed mode
* PE-49776: Check TMPDIR before using it
* PE-49611: Detect spurious timeouts and fix backend timeout setting
* PE-48636: Remove message traversal limit
* Fix install path exported by lmod modulefile on relocated install
* PE-45208: Don't remove auth keyfile until setup completed
* Fix rarely dropped messages during startup
* Ensure nodes in newly created communicators are fully connected before returning new communicator to user
### Features
* Build with g++ 14 on rhel 9.5 and sles15sp6
* Add rhel 9.5, remove rhel 8.9 and sles15sp4
* CPE-6082: Configurable backend timeout for flushing messages before exit
* PE-5880: Add Timeout After Last synchronization option
* Build rpms for rhel 8.10
* Build on sles sp6 and rhel 9.4
* PE-53605: Build and package libsodium dependency
* PE-51960: Added support for rhel89/93 & gcc13
* PE-44112: Add optional heartbeat monitor thread to AsyncFrontend
* PE-50464: Implement configurable port base
* Move to gcc-toolset-12 on redhat
* PE-48567: ZMQNet documentation
* Introduce forceDisconnect function to AsyncFrontend
* PE-41205: Add option to disable encryption at build and run time
=======================================================================
## [1.0.0] - 2022-9-26 (22.10)
=======================================================================
* Initial release
OS Dependencies
---------------------------
- HPE CSM systems with SLES 15 SP5 - SP6
- HPE HPCM systems with SLES 15 SP5 - SP6 or RHEL 8.10 - 9.4
- HPE Apollo systems with RHEL 8.10
Product Dependencies
---------------------------
- cray-cti 2.20, < 3 installed
Installation Instructions
---------------------------
- rpm -ivh cray-zmqnet-1.3.2-20250605043925_9fca93f25f27-120.sles15sp7.x86_64.rpm
To make cray-zmqnet 1.3.2 the default version of cray-zmqnet, execute:
/opt/cray/pe/admin-pe/set_default_files/set_default_cray-zmqnet_1.3.2
Legal Notices
---------------------------
Certain components, files or programs contained within this package or product are Copyright 2020-2025 Hewlett Packard Enterprise Development LP.
Attribution notices for open source licensed software for this package are detailed in the file:
/opt/cray/pe/zmqnet/1.3.2/PLACEHOLDER_ATTRIBUTIONS_zmqnet.txt
Craypkg-gen 1.3.36
Craypkg-gen 1.3.36
==============
Release date
------------
June 2025
Purpose
-------
Bug fix release.
Bugs fixed in this release
--------------------------
- Update intel modulefile to use ifx instead of ifort for 2025.0 and later
Dependencies
------------
The craypkg-gen 1.3.36 release is supported on the following HPE
Cray systems:
- Cray EX systems SLE 15.0 or later and RHEL 8.0 or later.
Driver support for integrating Third Party C, C++, and Fortran libraries
through .pc files using pkg-config is used in CrayPE 2.x and later.
Limitation
----------
- Library dependencies for static libraries are not added to the .pc files
for keywords Requires.private and Libs.private. A warning is issued by
the craypkg-gen tool to advise users to add this information to the
libraries .pc files.
- RPM limits packages to 4GB
Documentation
-------------
Man pages for craypkg-gen are found by executing `module load craypkg-gen`
and then `man craypkg-gen`. See
http://www.freedesktop.org/wiki/Software/pkg-config for a pkg-config
introduction.
Examples for creating modulefiles for Intel, PGI and Python are included in
the craypkg-gen ‘doc’ directory:
/opt/cray/craypkg-gen/1.3.36/doc/intel_example.txt
/opt/cray/craypkg-gen/1.3.36/doc/pgi_example.txt
/opt/cray/craypkg-gen/1.3.36/doc/python_example.txt
Example:
- As an example, the Intel 16.0.3.210 compiler was recently released.
After installing the compiler the administrator creates a modulefile for
this release by executing the following commands:
# module load craypkg-gen
# craypkg-gen -m /opt/intel/compilers_and_libraries_2016.3.210
This version of the Intel compiler is made default by executing the
command:
# /opt/admin-pe/set_default_craypkg/set_default_intel_16.0.3.210
Installation instructions
-------------------------
rpm -ivh craypkg-gen-1.3.36-1.3.36.10.g47d3ef7-1.sles15sp7.noarch.rpm
To change the product version to default after installation:
/opt/cray/pe/admin-pe/set_default_files/set_default_craypkg-gen_1.3.36
Product description
-------------------
The craypkg-gen 1.3.36 utility provides the system administrator
a tool to integrate third party software with the Cray software stack.
Craypkg-gen assists with integration by creating .pc files for C, C++, and
Fortran libraries, pkg-config enabled modulefiles and RPMs.
The workflow for using craypkg-gen is
1) Build the open source software
2) Create .pc files for libraries
3) Create pkg-config enabled modulefiles
4) Customize pkg-config and modulefiles if needed
Certain components, files or programs contained within this package or product
are Copyright 2013-2024 Hewlett Packard Enterprise Development LP.
FFTW 3.3.10.11
FFTW 3.3.10.11
============
Purpose:
--------
This Cray FFTW 3.3.10.11 release is supported on HPE/Cray EX and
HPE/Cray XD/Apollo systems with x86-64 CPUs. Cray FFTW is supported
on the host CPU but not on the accelerator of HPE/Cray systems.
The Cray FFTW 3.3.10.11 release provides the following:
- Compiler and dependency updates
See the Product and OS Dependencies section for details.
Product and OS Dependencies:
----------------------------
This Cray FFTW 3.3.10.11 release is supported on these HPE/Cray systems:
- HPE/Cray EX Systems with SLES 15 SP6 or later, RHEL 8.10, or RHEL 9.5 or later based OS
- HPE/Cray XD/Apollo Systems with RHEL 8.10 or RHEL 9.5 or later based OS
This Cray FFTW 3.3.10.11 release requires the following software products:
craype 2.7.35 or later
cray-mpich 9.0 or later
One or more of the following major compiler versions:
CCE 20
GCC 14
AOCC 5.0
AMD (ROCm) 6.4
Nvidia 25.5 (except SLES 15.7)
Intel 2025
Notes and Limitations:
----------------------
Starting with cray-fftw/3.3.6.1 the fftw module has been renamed from fftw
to cray-fftw.
Documentation:
--------------
http://www.fftw.org/index.html#documentation
See the intro_fftw3 manual page for additional information.
Modulefile:
-----------
module load cray-fftw
Installation
------------
To install the rpm, execute:
rpm -ivh cray-fftw-3.3.10.11-202508271831.ddaaa72f140ce-1.sles15sp7.x86_64.rpm
To make this the default version, execute:
/opt/cray/pe/admin-pe/set_default_files/set_default_fftw_3.3.10.11
Certain components, files or programs contained within this package or
product are Copyright 2011-2025 Hewlett Packard Enterprise Development LP
gdb4hpc 4.16.5
gdb4hpc 4.16.5
================================================================================
Product Description
---------------------------
- gdb4hpc is a GDB-based parallel debugger used to debug applications compiled with
CCE, AOCC, GNU, and Intel Fortran, C and C++ compilers. It allows programmers to
either launch an application or attach to an already running application that
was launched on the system. Additionally, it provides comparative debugging
technology that enables programmers to compare data structures between two
executing applications. Comparative debugging should be used in conjunction
with the CCDB GUI tool accessed by loading the cray-ccdb module.
- Some features of gdb include:
* Command line parallel debugger allows for launching/attaching applications.
* Utilizes process sets to operate on a subset of application ranks.
* GDB-like feel, also implements a gdbmode to enable a true parallel gdb.
* GPU debugging is supported for OpenMP 4.0+ GPU directives; OpenMP 5.0 on AMD
and Nvidia GPU's.
* Workload manager support via Common Tools Interface (cray-cti).
Changelog
---------------------------
=======================================================================
## [4.16.5]
=======================================================================
### Bug Fixes
* Ignore quotes around the --env argument in launch
* CPE-9018: Survive segv in message queue debugging shared libraries
* CPE-9350: Fix incorrect pending breakpoint response in mi mode
### Features
* Remove dependence on cdst-support package
* CPE-9081: Add info mpi pending command
* Add support for float16 and bfloat
* CPE-8683: Add --gdbinit launch option to enable reading .gdbinit files in backend
=======================================================================
## [4.16.3]
=======================================================================
### Bug Fixes
* CPE-5960: Fix failure to launch application when using certain acceleration runtimes
* CPE-7944: Work around gdb bug when finishing during startup sequence on aarch/python
* CPE-8161: print an error message when trying to print a dereferenced null pointer
* CPE-7930: Don't print parse error to stderr
### Features
* CPE-8868: Remove MRNet network protocol
* CPE-8194: Better node management when using the run command to relaunch jobs
=======================================================================
## [4.16.4]
=======================================================================
### Bug Fixes
* CPE-6196: Allow CUDA breakpoints on kernel launch
* Print NaN and infinity values
* Add RUNPATH for unit tests to libexec/dbgsrv
* CPE-5982: Fix cuda command failures in some cases
* CPE-5960: Fix failure to launch application when using certain acceleration runtimes
* No not generate missing MPI breakpoint if a program use mpi_init_thread
* PE-52432: Load MRNet filter library from complete path
* PE-54147: Use main function from debug information
* Fix printing std::unique_ptr with newer stdc++
* Fix print of character arrays shorted by set print elements
* Fix backend gdb crash on vector<vector<string>>
### Features
* CPE-7286: Support basic list command in python mode
* CPE-7395: Support 'catch throw' and 'catch raise' commands in python mode
* CPE-6731: Apply pager setting to results of `info threads` command
* CPE-5688: In python mode, stop the application on uncaught exceptions
* CPE-635: Verify python breakpoint locations
* CPE-6345: Add --pending option to break
* CPE-5686: Add frame and up/down commands to python mode
* CPE-6994: Handle unregistered breakpoints and watchpoints
* CPE-6094: Support setting breakpoints in multithreaded python apps
* Add rhel 9.5, remove rhel 8.9 and sles15sp4
* CPE-6664: Support debugging python 3.5, python 3.7, python 3.13
* CPE-6561: Improve CUDA API error reporting
* CPE-6513: Show cuda-gdb cuda API warnings
* CPE-5690: Change command prompt when in python mode
* CPE-6037: Add GPU debugging tutorial
* CPE-615: Add python mode documentation
* CPE-2316: Add "what's new in 4.16.3" document
* CPE-633: support finish command in python mode
* CPE-2350: Autocomplete struct members for `print`
* PE-53102: Set python breakpoints and step/next in python
* Support rhel 8.10
* Add builds for rhel 9.4 and sles15sp6
* PE-53101: Evaluate python statements with the print command
* PE-55035: Python mode and python backtraces
* PE-53105: Launch applications with the same syntax as an srun command
* PE-54824: Support non-MPI PALS launches
* PE-54073: Print job output on failed launch
* PE-53771: Pass arguments to launched app on the end of the launch command
OS Dependencies
---------------------------
- HPE CSM systems with SLES 15 SP5 - SP6
- HPE HPCM systems with SLES 15 SP5 - SP6 or RHEL 8.10 - 9.4
- HPE Apollo systems with RHEL 8.10
Product Dependencies
---------------------------
- cray-cti 2.20, < 3 installed
Documentation
---------------------------
- Type `man gdb4hpc` with the gdb4hpc module loaded to read the gdb4hpc(1) man page.
Simple usage examples are provided at the end.
Also, type `help` at the gdb4hpc command line for more information on commands.
Installation Instructions
---------------------------
- rpm -ivh gdb4hpc-4.16.5-20250827191052_24ac56f569b3-573.sles15sp7.x86_64.rpm
To make gdb4hpc 4.16.5 the default version of gdb4hpc, execute:
/opt/cray/pe/admin-pe/set_default_files/set_default_gdb4hpc_4.16.5
Legal Notices
---------------------------
Certain components, files or programs contained within this package or product are Copyright 2007-2025 Hewlett Packard Enterprise Development LP.
Attribution notices for open source licensed software for this package are detailed in the file:
/opt/cray/pe/gdb4hpc/4.16.5/ATTRIBUTIONS_gdb4hpc.txt
HDF5 1.14.3.7
# HDF5 1.14.3.7
## Purpose:
This HDF5 release contains the following:
* SLES 15 SP6 (x86_64, aarch64)
* SLES 15 SP7 (x86_64, aarch64)
* RHEL 9.5 (x86_64, aarch64)
* RHEL 9.6 (x86_64, aarch64)
* RHEL 8.10 (x86_64)
* Apollo RHEL 8.10 (x86_64)
* CCE 20.0.0
* AMD ROCm 6.4
* AOCC 5.0
* Intel OneAPI 2025.1
* Nvidia HPC SDK 25.5
* GCC 14
## Product and OS Dependencies:
The HDF5 1.14.3.7 release is supported on the following systems, requiring the following software products:
* CrayPE 2.1.2 or later
* SLES 15 SP6/SP7 (x86_64, aarch64)
* RHEL 8.10 (x86_64)
* Apollo RHEL 8.10 (x86_64)
* RHEL 9.5 (x86_64, aarch64)
* RHEL 9.6 (x86_64, aarch64)
* cray-mpich/8.1.0 or later
Supported on the following HPE systems:
* HPE Cray EX systems with CLE
* HPE Cray XD systems with CLE
One or more of the following compiler major versions:
* CCE 20.x or later (SLES, RHEL)
* GCC 12.3 or later (SLES)
* gcc-toolset-10.3 to 14.2 (RHEL 8.10)
* gcc-toolset-12.2 to 14.2 (RHEL 9.5)
* gcc-toolset-12.2 to 14.2 (RHEL 9.6)
* AOCC 4.2.0 or later
* AMD ROCm 6.2.1 or later (SLES and RHEL) or AMD ROCm 6.2 or later (Apollo RHEL 8.10)
* Intel 2024.2.0 or later
* Nvidia 23.11 or later (aarch64) and 23.3 or later (x86_64)
## Notes and Limitations:
Users are encouraged to use the Cray compiler scripts (cc, ftn, CC) to include the HDF5 header files and link in the HDF5 libraries, rather than the HDF5 compiler scripts (h5cc, h5fc, h5c++) included in the release.
## Documentation:
http://www.hdfgroup.org/HDF5/doc/index.html
## Modulefile:
`module load cray-hdf5`
OR
`module load cray-hdf5-parallel`
## Product description:
HDF5 is a data model, library, and file format for storing and managing data. It supports an unlimited variety of datatypes, and is designed for flexible and efficient I/O and for high volume and complex data. HDF5 is portable and is extensible, allowing applications to evolve in their use of HDF5. The HDF5 Technology suite includes tools and applications for managing, manipulating, viewing, and analyzing data in the HDF5 format.
## Installation:
`rpm -ivh cray-hdf5-1.14.3.7-*-202508271616.c03ae81422329.@ARCH@.rpm`
The "`*`" in the install command represents compiler version combinations.
To make this the default version, execute:
`/opt/cray/pe/admin-pe/set_default_files/set_default_hdf5_1.14.3.7`
Certain components, files or programs contained within this package or product are Copyright 2019-2025 Hewlett Packard Enterprise Development LP.
HPE Cray Compiling Environment (CCE) 20.0.0
HPE Cray Compiling Environment (CCE) 20.0.0
===========================================
Purpose:
--------
CCE 20.0.0 provides Fortran, C, and C++ compilers for
HPE Cray supercomputer and HPE Apollo 2000 Gen10Plus systems.
Key Changes and Support with CCE 20.0.0:
----------------------------------------
CCE 20.0.0:
-----------
- LLVM 20 base (merges up to May 27, 2025 - LLVM version 20.1.6)
- C++ virtual and indirect function calls are supported in C/C++ OpenMP offload regions when compiling with either -fopenmp-offload-translate-at-map or -fopenmp-offload-translate-at-call
- Added or extended support for the following OpenMP features:
[6.0]
- free-agent threads
- reductions on private variables
- self mapping for accelerators
- Fortran: misc. extensions to `default` and `defaultmap` clauses
[5.2]
- `masked` construct with `filter` clause
- `OMP_PLACES` extended syntax
- Fortran: `allocators` construct for `ALLOCATE` statement
- misc. new syntax allowed by specification (e.g. `declare target`, `ordered`, metadirective)
[5.1]
- Fortran: `thread_limit` clause on `target`
- Fortran: `tile` and `unroll` constructs
- Fortran: allow `BLOCK`...`ENDBLOCK` as OpenMP structured block
- 'strict' extension for `num_tasks` and `grain_size` on `taskloop`
- `masked` construct and related combined constructs
- `primary` thread affinity policy
- `_DEV` suffix for device-specific environment variables
- `omp_target_is_accessible` routine
- allow `scan` directive to apply to array sections
- This release of HPE CCE has support for the language features specified
by the Fortran 2023 standard, with the exception of .NIL., as noted below.
- New Fortran 2023 Intrinsics
- `TOKENIZE`
- `SPLIT`
- `PREVIOUS`
- `NEXT`
- Updated `KIND` data types within `ISO_FORTRAN_ENV`
- `LOGICAL8`
- `LOGICAL16`
- `LOGICAL32`
- `LOGICAL64`
- `REAL16`
- Automatic allocation of lengths of character variables
- The `at` edit descriptor
- Control over leading zeros in output of real values
- Allow an object of a type with a coarray ultimate component to be an array or allocatable
- Full support for the scaled-by-Pi trig functions (`sinpi`, `tanpi`, etc)
- Update to the `SYSTEM_CLOCK` intrinsic argument `COUNT_RATE`
- `ISO_C_BINDING` updated to support the new `F_C_STRING` and `C_F_STRPOINTER` subroutines
- Conditional Expressions
- Conditional Arguments (exclusive of `.NIL.`)
- `IEEE_ARITHMETIC` routines are now `PURE` and `SIMPLE`
- Updated line and statement length maximums, as well as the number of continuation lines
- Put with notify
- Subscripts and section subscripts support the '@' identifier and can be specified using integer arrays.
- Integer arrays can be used to specify bounds in an array specification.
- AMD Turin Support (500W)
- SLES15 SP7 x86_64 support
- SLES15 SP7 aarch64 support
- RHEL 9.6 x86_64 support
- RHEL 9.6 aarch64 support
- ROCm 6.4
- NVIDIA SDK 25.05
- Enhanced DWARF Support
User and Application Impact:
----------------------------
CCE 20.0.0:
-----------
CCE-20.0.0
- CCE 20.0.0 is not compatible with Fortran modules produced by prior versions
of CCE. Compiling source code that uses a Fortran module produced by a prior
version of CCE results in an ftn-1774 error message at compile time. If you
encounter this error message, you must rebuild your Fortran modules with CCE
20.0.0.
Issues or Bugs Resolved:
------------------------
CCE 20.0.0:
-----------
CAST-38682 Thornado linking error with CCE/20.0.0
CAST-38098 BerkeleyGW unrecoverable lib-4425 Fortran error when -homp
CAST-38096 use_device_ptr Fortran clause not correctly handled
CAST-37908 ftn sanitize=address has problems in cce 18 and cce 19
CAST-37842 Memory leak with CCE Fortran with lots of open/close calls
CAST-37681 Fortran does not support use_device_addr with omp requires unified_shared_memory
CAST-37645 OpenACC error when renaming module variable
CAST-37556 CCE 19.0.0 RC -g causes some OpenMP GPU code to crash at runtime
CAST-37555 CCE 18.0.1 Fortran ftn-1426 recursive reshape error
CAST-37554 Fortran ICE when compiling submodules with subroutines that contain procedures
CAST-37534 FTN internal compiler error with subroutine calls within submodule function block constructs
CAST-37455 Compilation performance issue with cce17.0.0/cce18.0.0
CAST-37351 Issue compiling coarray Fortran
CAST-37191 Fortran OpenMP runtime error when using map(alloc:) on MI300A (Minikin)
CAST-37086 CCE/17.0.0 linker bug
CAST-36485 derived datatypes not thread safe
CAST-36453 target openmp array reduction poor performance
CAST-36272 Fortran derived type data transfer not working with CCE (openmp target offload)
CAST-35535 crayftn runtime error with user defined operator on associate name
CAST-35034 OpenACC reductions with array variables are not supported
CAST-35032 Bad performance with OpenACC "kernels" when used with assumed shape arrays
CAST-34762 Fortran + OpenAcc on AMD GPU
CAST-34739 Wrong data attribute for scalar in OpenACC data
CAST-33011 OpenACC compile time error with Fortran strings
CAST-33010 OpenACC error with zero-sized Fortran array
CAST-30651 Unable to use runtime variable in openmp thread_limit(var) clause
Product and OS Dependencies:
----------------------------
This CCE release is supported on
- HPE Cray supercomputer systems running CSM with COS 2.5.X (SLES15 SP4) or COS 23.11 (SLES15 SP5)
- HPE Cray supercomputer systems running HPCM with COS 2.5.X (SLES15 SP4) or COS 23.11 (SLES15 SP5)
- HPE Cray supercomputer systems running HPCM with RHEL 8.10
- HPE Apollo 2000 Gen10Plus systems running RHEL 8.9
- AMD GPU offloading support requires at least ROCm 6.0 and is tested up to ROCm 6.4
Notes and Limitations:
----------------------
- LLVM backends for unsupported targets are deprecated and will be removed in a future release. LLVM backends for all supported CPU and GPU targets will be retained (X86, AArch64, NVPTX, and AMDGPU).
- Support for the AMD GPU gfx940 target is deprecated and will be removed in a future release.
- AMD GPU code object version 4 is deprecated and will be removed in a future release. The default code object version is 5, so this change only affects explicit use of the flag '-mcode-object-version=4'.
Documentation:
--------------
- S-5212 Cray Compiling Environment Release Overview (20.0)
- S-3901 Cray Fortran Reference Manual
- S-2179 Cray C and C++ Quick Reference
- Basic man pages: crayftn(1), craycc(1), crayCC(1), intro_openmp(7), intro_openacc(7)
- Please see https://clang.llvm.org/docs/UsersManual.html or use the -help command line option for
more information on using Clang
Modulefile:
-----------
The following will load the modules necessary to use CCE:
module load PrgEnv-cray
The following will switch to x.y.z version of CCE:
module swap cce cce/x.y.z
Installation instructions:
--------------------------
rpm -ivh cce-20.0.0-202509231735.95f64a80f3ee7-0.sles15sp4.x86_64.rpm
The following script will set CCE version 20.0.0 default:
/opt/cray/pe/admin-pe/set_default_files/set_default_cce_20.0.0
License:
--------
Except for the third party components and software licensed by HPE through
proprietary agreements, components, files or programs contained within this
package or product are Copyright -2025 Hewlett Packard Enterprise Development LP.
Attribution notices for open source licensed software for this
package are detailed in the file:
/opt/cray/pe/cce/20.0.0/ATTRIBUTIONS_20.0.txt
HPE Cray MPI 8.1.33
Cray MPICH 8.1.33:
=======================================
Release Date:
-------------
July 16, 2025
Purpose:
--------
HPE Cray MPI 9.0.1 is now the recommended production release of HPE Cray
MPI. This version (Cray MPICH <VERSION) is provided as a fallback version
in case there are problems with the new HPE Cray MPI 9.0.1 version. This
release will be the last Cray MPICH 8.x release.
Cray MPICH 8.1.33 is based upon ANL MPICH 3.4a2 with support for libfabric
and is optimized for the Cray Programming Environment. Cray MPICH 8.1.33
is built to work with libfabric ABI version 1.6 or later (libfabric API version
1.14 or later).
Major Differences Cray MPICH 8.1.33 from the XC Cray MPICH include:
- Uses the new ANL MPICH CH4 code path and libfabric for network
support.
- Does not support -default64 mode for Fortran
- Does not support C++ language bindings
New Cray MPICH features for HPE Cray EX and Apollo systems:
- Starting from the 8.1.26 release, Cray MPICH supports the Intel Sapphire Rapids CPU HBM
processor architecture.
- On systems with AMD GPUs, Cray MPICH 8.1.26 supports all ROCm
versions starting from ROCm 5.0, including the latest ROCm 5.5.0
release.
The Cray MPICH 8.1.25 release and prior versions of
Cray MPICH are only compatible with ROCm versions up to (and
including) the ROCm 5.4.0 release.
- On Slinghot-11 systems, Cray MPICH uses the libfabric "cxi" provider
by default. On Slingshot-10 and Apollo systems, Cray MPICH uses the
libfabric "verbs;ofi_rxm" provider by default.
- Cray MPICH offers support for multiple NICs per node. Starting with
version 8.0.8, by default Cray MPICH will use all available NICs on
a node. Several rank-to-NIC assignment policies are supported. For
details on choosing a policy for assigning ranks to NICS, or for
selecting a subset of available NICs, please see the following
environment variables documented in the mpi man page.
MPICH_OFI_NIC_VERBOSE
MPICH_OFI_NIC_POLICY
MPICH_OFI_NIC_MAPPING
MPICH_OFI_NUM_NICS
- Enhancements to the MPICH_OFI_NIC_POLICY NUMA mode have been added.
Starting with version 8.0.14, if the user selects the NUMA policy,
the NIC closest to the rank is selected. A NIC no longer needs to
reside in the same numa node as the rank. If multiple NICs are
assigned to the same numa node, the local ranks will round-robin
between them. Numa distances are analyzed to select the closest NIC.
- On Slingshot-10 and Apollo systems, Cray MPICH supports runtime switching
to the UCX netmod starting with version 8.0.14. To do this load the
craype-network-ucx module and module swap between Cray-MPICH and
Cray-MPICH-UCX modules. For more information including relevant
environment variables reference the intro_mpi man page with the
Cray-MPICH-UCX module loaded.
- Lmod support for HPE Cray EX starting with Cray MPICH 8.0.16.
Key Changes and Bugs Closed:
----------------------------
Changes in Cray MPICH 8.1.33
Features and Bug Fixes
MPICH:
- CAST-37440 - Add additional GTL symlinks to MPICH lib directories
- CAST-37637 - Add optional coalescing-eviction to XPMEM cache
- CAST-37980 - Fix datatype bug with nesting of contig and resized types
- CAST-38066 - Add debug libraries for Intel and Nvidia compilers
- CPE-1563 - Update manpage documentation with memory-related variables
- CPE-1730 - Disable GPU_MANAGED_MEMORY_SUPPORT_ENABLED on non-NVIDIA GPUs
- CPE-1781 - Remove special handling of FI_ENOMEM with SlingShot RMA
- CPE-2405 - Handle additional errnos from statfs
- CPE-3421 - Fix a ROMIO GPFS memory error
- CPE-7202 - Fix definition mismatch for gtl_request in MPICH and GTL
- CPE-7737 - Fix in MPI_Reduce_scatter payload size check
- CPE-8160 - Improve manpage documentation for MPICH compiler wrappers
- CPE-8502 - Lower the MPICH_SMP_SINGLE_COPY_SIZE default on x86_64
- CPE-8609 - Change mpifort wrapper for intel to use ifx instead of ifort
- CPE-9013 - Document that rank reordering does not affect placement
- CPE-9035 - Improve thresholds for different send paths
- CPE-9167 - Enable shutting down XPMEM/CMA without affecting GPU IPC
- CPE-9204 - Update manpage documentation for MPICH_REDUCE_INTRA_ALGORITHM
- CPE-9296 - Add support for CCE 20
- CPE-9307 - Use a different GPU IPC threshold if intra-GPU
- CPE-9441 - Update manpage documentation for FI_CXI_DEFAULT_TX_SIZE
- CPE-9584 - Update MPI man page with Grace Hopper advice
- CPE-10296 - Change MPICH_REDUCE_INTRA_ALGORITHM to reduce_scatter_gather
- CPE-10337 - Call the GPU handle closer when done with transfers
- CPE-10503 - Add lrank parameter to MPIR_gpu_get_mem_handle() calls
- CPE-10588 - Update manpage documentation with FI_MR_CACHE_MONITOR
- CPE-10750 - MPI-IO improvement for collective IO calls
- CPE-10830 - Fix issue in MPIR_gpu_can_use_reduce_kernel
- CPE-11148 - MPI-IO bugfix if Lustre returns zero stripe_count
GTL:
- CAST-37068 - Fix OOM issue by using HSA IPC signals
- CAST-37703 - Fix reduction kernels for non-multiple of 512
- CPE-9345 - Add an API to get the GPU model
- CPE-9490 - Print GTL_VERSION_DISPLAY to stderr
- CPE-9896 - Fix in gtlt_cuda_pointer_type() initialization
- CPE-10316 - Fix in UDREG data structures
- CPE-10334 - Enable reuse of HSA signal info objects
- CPE-10337 - Handle overflow of the HSA signal table
- CPE-10489 - Extend gtl_ipc_desc_t changes to all GPU types
- CPE-10648 - Turn off the HSA signal cache by default
- CPE-10879 - Add debug log to report IPC cache usage
- CPE-10885 - Change defaults of HSA cache sizes
Product and OS Dependencies:
----------------------------
The Cray MPICH 8.1.33 release is supported on the following HPE systems:
* HPE Cray EX systems
* HPE XD systems
Product and OS Dependencies by network type:
----------------------------+---------------------+
craype | >= 2.7.6 |
----------------------------+---------------------+
cray-pals | >= 1.0.6 |
----------------------------+---------------------+
cray-pmi | >= 6.0.1 |
----------------------------+---------------------+
libfabric | >= 1.14.0 |
----------------------------+---------------------+
One or more compilers:
* AMD ROCM 6.0 or later
* AOCC 4.1 or later
* CCE 20.0 or later
* GNU 12.3 or later
* Intel 2022.1 or later
* Nvidia 23.3 or later
Notes and Limitations:
----------------------
Limitations in Cray MPICH 8.1.33:
- Cray MPICH 8.1.33 can support up to ~4090 simultaneous MPI
communicators.
- Support of SS-10 and Apollo systems/InfiniBand has been dropped.
- On systems with Slingshot-11 hardware, some MPI programs using
MPI_PROBE (or it's variants) experience an application failure,
from libfabric, with the error message
libfabric:766248:1740758749::cxi:core:cxip_claim_onload_cb():2581<warn>
RXC (0x173:0) PtlTE 16:[Fatal] Unexpected event SEARCH, rc = NO_MATCH
This problem is being tracked by NETCASSINI-7129 and a solution is
expected in a future release of Slingshot Host Software (SHS).
In the interim, this problem may be avoided by setting
FI_CXI_RX_MATCH_MODE=software
in the software environment where the application is run.
intro_mpi(1) has information about FI_CXI_RX_MATCH_MODE.
Other Notes
- This version of HPE Cray MPI (8.1.33) supports CCE 20. CCE 20 is
incompatible with Fortran modules compiled by previous CCE versions.
Please see the CCE 20 release notes for further information.
To prevent use of CCE 20 with other CPE products with incompatible
Fortran modules, the Lmod compatibility index has been changed for
CCE 20. This change makes CCE 20 incompatible with any any HPE
Cray MPI versions prior to this version (8.1.33). One cannot load
a prior HPE Cray MPI module with CCE 20. In addition, this version
of HPE Cray MPI cannot be module loaded with any previous version of
CCE.
- Some versions of NVIDIA compilers, in particular NVIDIA 25.3 and 25.5
compilers, add "/usr/lib64" to the rpath of any program linked with
the NVIDIA compiler drivers. This rpath blocks the use of other
parts of Cray PE. In particular it prevents Cray PMI from being used.
The most common run-time error one sees if using Slurm PMI with HPE
Cray MPI when Cray PMI is expected is:
xxxx: ../src/mpid/common/cray/cray_pmi_utils.c:108: _mpidi_cray_pmi_send: Assertion `rc == PMI2_SUCCESS' failed
One way to avoid this behavior is to add the NVIDA compiler option
"-Mnorpath" when linking programs with NVIDIA compilers.
Documentation:
--------------
For more information see the intro_mpi man page.
Modulefile:
-----------
module load cray-mpich/8.1.33
License:
--------
Except for the third party components and software licensed by HPE
through proprietary agreements, components, files or programs contained
within this package or product are Copyright -2021 Hewlett Packard
Enterprise Development LP.
Attribution notices for open source licensed software for this
package are detailed in the file:
/opt/cray/pe/mpich/8.1.33/ATTRIBUTIONS
Copyright -2025 Hewlett Packard Enterprise Development LP
HPE Cray MPI 9.0.1
HPE Cray MPI 9.0.1:
=======================================
Release Date:
-------------
July 16, 2025
Purpose:
--------
HPE Cray MPI 9.x is now the recommended production version of HPE Cray
MPI beginning with CPE 25.09 and this release of HPE Cray MPI(9.0.1).
One final HPE Cray MPI 8.x release has been released with CPE 25.09 as
a fallback MPI version in case of problems with HPE Cray MPI 9.0.1.
This 8.x fallback version in CPE 25.09 will be the last release of
Cray MPI 8.x
HPE Cray MPI 9.0.1 is a version of HPE Cray MPI that supports the
MPI 4.0 standard. Cray MPICH 9.0.1 is based upon ANL MPICH 4.1.2
with support for libfabric and is optimized for the Cray Programming
Environment. Cray MPICH 9.0.1 is built to work with libfabric
ABI version 1.6 or later (libfabric API version 1.14 or later).
Major Differences HPE Cray MPI 8.x and HPE Cray MPI 9.x:
- HPE Cray MPI 9.x supports the MPI 4.0 standard. The 8.x series
supports the MPI 3.1 standard.
- HPE Cray MPI 9.x is based on ANL MPICH 4.1.2. The 8.x series is
based on ANL MPICH 3.4a2.
New HPE CRAY MPI features for HPE Cray EX systems:
The 9.x series is based on the 8.x series and thus inherits all
of 8.x's features, improvements, etc.
- Starting from the 8.1.26 release, HPE Cray MPI supports the Intel
Sapphire Rapids CPU HBM processor architecture.
- On systems with AMD GPUs, HPE Cray MPI 9.0.1 and 8.1.33 support
all ROCm versions starting from ROCm 6.0, including the latest ROCm
6.4.0 release. Advanced support for limiting the growth of memory used
for sharing GPU buffers requires at least ROCm 6.4.0.
- On Slinghot-11 systems, HPE Cray MPI uses the libfabric "cxi" provider
by default.
- HPE Cray MPI offers support for multiple NICs per node. Starting with
version 8.0.8, by default HPE Cray MPI will use all available NICs on
a node. Several rank-to-NIC assignment policies are supported. For
details on choosing a policy for assigning ranks to NICS, or for
selecting a subset of available NICs, please see the following
environment variables documented in the mpi man page.
MPICH_OFI_NIC_VERBOSE
MPICH_OFI_NIC_POLICY
MPICH_OFI_NIC_MAPPING
MPICH_OFI_NUM_NICS
- Enhancements to the MPICH_OFI_NIC_POLICY NUMA mode have been added.
Starting with version 8.0.14, if the user selects the NUMA policy,
the NIC closest to the rank is selected. A NIC no longer needs to
reside in the same numa node as the rank. If multiple NICs are
assigned to the same numa node, the local ranks will round-robin
between them. Numa distances are analyzed to select the closest NIC.
- Lmod support for HPE Cray EX starting with HPE Cray MPI 8.0.16.
Key Changes and Bugs Closed:
----------------------------
Changes in HPE Cray MPI 9.0.1
Features and Bug Fixes:
MPICH:
- Support for MPI 4.0 standard
- 9.x is now performant. Previous 9.x were functional with some
performance issues. Performance should be close to or better than
8.x performance.
- F90 type checking. ANL MPICH 4.1.2 (what HPE Cray MPI 9.x is
based on) now provides full F90 interfaces and thus does type
checking. Incorrect F90 Fortran codes will now hit compile
errors that are not encountered with 8.x.
- CAST-35201 - Adjust default cross-over points for MPICH_ALLTOALL_SHORT_MSG
- CAST-37440 - Add additional GTL symlinks to MPICH lib directories
- CAST-37637 - Add optional coalescing-eviction to XPMEM cache
- CAST-37980 - Fix datatype bug with nesting of contig and resized types
- CAST-38066 - Add debug libraries for Intel and Nvidia compilers
- CPE-1563 - Update manpage documentation with memory-related variables
- CPE-1730 - Disable GPU_MANAGED_MEMORY_SUPPORT_ENABLED on non-NVIDIA GPUs
- CPE-1781 - Remove special handling of FI_ENOMEM with SlingShot RMA
- CPE-2405 - Handle additional errnos from statfs
- CPE-3421 - Fix a ROMIO GPFS memory error
- CPE-7202 - Fix definition mismatch for gtl_request in MPICH and GTL
- CPE-7737 - Fix in MPI_Reduce_scatter payload size check
- CPE-8160 - Improve manpage documentation for MPICH compiler wrappers
- CPE-8502 - Lower the MPICH_SMP_SINGLE_COPY_SIZE default on x86_64
- CPE-8609 - Change mpifort wrapper for intel to use ifx instead of ifort
- CPE-9013 - Document that rank reordering does not affect placement
- CPE-9035 - Improve thresholds for different send paths
- CPE-9167 - Enable shutting down XPMEM/CMA without affecting GPU IPC
- CPE-9204 - Update manpage documentation for MPICH_REDUCE_INTRA_ALGORITHM
- CPE-9296 - Add support for CCE 20
- CPE-9307 - Use a different GPU IPC threshold if intra-GPU
- CPE-9441 - Update manpage documentation for FI_CXI_DEFAULT_TX_SIZE
- CPE-9584 - Update MPI man page with Grace Hopper advice
- CPE-10296 - Change MPICH_REDUCE_INTRA_ALGORITHM to reduce_scatter_gather
- CPE-10337 - Call the GPU handle closer when done with transfers
- CPE-10503 - Add lrank parameter to MPIR_gpu_get_mem_handle() calls
- CPE-10588 - Update manpage documentation with FI_MR_CACHE_MONITOR
- CPE-10750 - MPI-IO improvement for collective IO calls
- CPE-10830 - Fix issue in MPIR_gpu_can_use_reduce_kernel
- CPE-11148 - MPI-IO bugfix if Lustre returns zero stripe_count
GTL:
- CAST-37068 - Fix OOM issue by using HSA IPC signals
- CAST-37703 - Fix reduction kernels for non-multiple of 512
- CPE-9345 - Add an API to get the GPU model
- CPE-9490 - Print GTL_VERSION_DISPLAY to stderr
- CPE-9896 - Fix in gtlt_cuda_pointer_type() initialization
- CPE-10316 - Fix in UDREG data structures
- CPE-10334 - Enable reuse of HSA signal info objects
- CPE-10337 - Handle overflow of the HSA signal table
- CPE-10489 - Extend gtl_ipc_desc_t changes to all GPU types
- CPE-10648 - Turn off the HSA signal cache by default
- CPE-10879 - Add debug log to report IPC cache usage
- CPE-10885 - Change defaults of HSA cache sizes
Product and OS Dependencies:
----------------------------
The HPE Cray MPI 9.0.1 release is supported on the following HPE systems:
* HPE Cray EX systems with CLE
Product and OS Dependencies by network type:
--------------------------------------------------+
| Shasta |
----------------------------+---------------------+
craype | >= 2.7.6 |
----------------------------+---------------------+
cray-pals | >= 1.0.6 |
----------------------------+---------------------+
cray-pmi | >= 6.0.1 |
----------------------------+---------------------+
libfabric | >= 1.9.0 |
----------------------------+---------------------+
One or more compilers:
* AMD ROCM 6.0 or later
* AOCC 4.1 or later
* CCE 20.0 or later
* GNU 12.3 or later
* Intel 2022.1 or later
* Nvidia 23.3 or later
Notes and Limitations:
----------------------
Limitations in HPE Cray MPI 9.0.1:
- HPE Cray MPI 9.0.1 does not support C++ language bindings. Note
that HPE Cray MPI 8.x releases also does not support C++ language
bindings. There is no plan to add C++ language bindings at this time.
- GPU Async Communication - Stream Triggered and Kernel Triggered
communication - is currently unsupported. Both features are planned
to be added in a future release.
- Support of SS-10 and InfiniBand has been dropped.
- On systems with Slingshot-11 hardware, some MPI programs using
MPI_PROBE (or it's variants) experience an application failure,
from libfabric, with the error message
libfabric:766248:1740758749::cxi:core:cxip_claim_onload_cb():2581<warn>
RXC (0x173:0) PtlTE 16:[Fatal] Unexpected event SEARCH, rc = NO_MATCH
This problem is being tracked by NETCASSINI-7129 and a solution is
expected in a future release of Slingshot Host Software (SHS).
In the interim, this problem may be avoided by setting
FI_CXI_RX_MATCH_MODE=software
in the software environment where the application is run. intro_mpi(1)
has information about FI_CXI_RX_MATCH_MODE.
- GPU KERNEL reductions are not available when using the new MPI 4.0
functions MPI_Allreduce_c, MPI_Reduce_c and MPI_Reduce_scatter_block_c
with 'count' values of 2GB or larger. GPU KERNEL reductions are still
available for count values less than 2GB.
Other Notes
- This version of HPE Cray MPI (9.0.1) supports CCE 20. CCE 20 is
incompatible with Fortran modules compiled by previous CCE versions.
Please see the CCE 20 release notes for further information.
To prevent use of CCE 20 with other CPE products with incompatible
Fortran modules, the Lmod compatibility index has been changed for
CCE 20. This change makes CCE 20 incompatible with any any HPE
Cray MPI versions prior to this version (9.0.1). One cannot load
a prior HPE Cray MPI module with CCE 20. In addition, this version
of HPE Cray MPI cannot be module loaded with any previous version of
CCE.
- Some versions of NVIDIA compilers, in particular NVIDIA 25.3 and 25.5
compilers, add "/usr/lib64" to the rpath of any program linked with
the NVIDIA compiler drivers. This rpath blocks the use of other
parts of Cray PE. In particular it prevents Cray PMI from being used.
The most common run-time error one sees if using Slurm PMI with HPE
Cray MPI when Cray PMI is expected is:
xxxx: ../src/mpid/common/cray/cray_pmi_utils.c:108: _mpidi_cray_pmi_send: Assertion `rc == PMI2_SUCCESS' failed
One way to avoid this behavior is to add the NVIDA compiler option
"-Mnorpath" when linking programs with NVIDIA compilers.
Documentation:
--------------
For more information see the intro_mpi man page.
Modulefile:
-----------
module load cray-mpich/9.0.1
License:
--------
Except for the third party components and software licensed by HPE
through proprietary agreements, components, files or programs contained
within this package or product are Copyright -2021 Hewlett Packard
Enterprise Development LP.
Attribution notices for open source licensed software for this
package are detailed in the file:
/opt/cray/pe/mpich/9.0.1/ATTRIBUTIONS
Copyright -2025 Hewlett Packard Enterprise Development LP
HPE Cray MPIxlate 1.0.9
Copyright 2022-2025 Hewlett Packard Enterprise Development LP
Product Release Information
================================================================================
Product identification:
-----------------------
HPE Cray MPIxlate: version 1.0.9 (built on 2025-09-03T18:34:15Z from 4307e89)
Purpose:
--------
HPE Cray MPIxlate enables applications compiled using an MPI library that
is not binary compatible with HPE Cray MPI, to be run without recompilation
on supported HPE platforms.
Warnings & Limitations:
-----------------------
- See NOTES in mpixlate(1)
- Starting with CPE-25.09, HPE Cray MPICH has dropped support for SS-10 and
InfiniBand. Since MPIxlate depends on HPE Cray MPICH, the same limitation
will apply to MPIxlate.
Changes from previous release:
------------------------------
- New features:
* None
- Defects Fixed:
* None
Product dependencies:
---------------------
cray-mpich >= 8.1.33
Documentation:
--------------
mpixlate(1)
Environment Modules
-------------------
module load cray-mpixlate/1.0.9
================================================================================
modules 3.2.11.7
modules 3.2.11.7
==================
Documentation:
---------------
See man-pages for module and modulefile.
NetCDF 4.9.2.1
# NetCDF 4.9.2.1
## Purpose:
This NetCDF release contains the following:
* SLES 15 SP6 (x86_64, aarch64)
* SLES 15 SP7 (x86_64, aarch64)
* RHEL 9.5 (x86_64, aarch64)
* RHEL 9.6 (x86_64, aarch64)
* RHEL 8.10 (x86_64)
* Apollo RHEL 8.10 (x86_64)
* CCE 20.0.0
* AMD ROCm 6.4
* AOCC 5.0
* Intel OneAPI 2025.1
* Nvidia HPC SDK 25.5
* GCC 14
## Product and OS Dependencies:
The NetCDF 4.9.2.1 release is supported on the following systems, requiring the following software products:
* Cray HDF5 1.14.3.7
* CrayPE 2.1.2 or later
* SLES 15 SP6 (x86_64, aarch64)
* SLES 15 SP7 (x86_64, aarch64)
* RHEL 8.10 (x86_64)
* Apollo RHEL 8.10 (x86_64)
* RHEL 9.5 (x86_64, aarch64)
* RHEL 9.6 (x86_64, aarch64)
* cray-mpich/8.1.0 or later
Supported on the following HPE systems:
* HPE Cray EX systems with CLE
* HPE Cray XD systems with CLE
One or more of the following compiler major versions:
* CCE 20.x or later (SLES, RHEL)
* GCC 12.3 or later (SLES)
* gcc-toolset-10.3 to 14.2 (RHEL 8.10)
* gcc-toolset-12.2 to 14.2 (RHEL 9.5)
* gcc-toolset-12.2 to 14.2 (RHEL 9.6)
* AOCC 4.2.0 or later
* AMD ROCm 6.2.1 or later (SLES and RHEL) or AMD ROCm 6.2 or later (Apollo RHEL 8.10)
* Intel 2024.2.0 or later
* Nvidia 23.11 or later (aarch64) and 23.3 or later (x86_64)
## Notes and Limitations:
For certain F77 source files, when compiling with `gcc/10+`, it may be necessary to configure/compile with:
`export FCFLAGS="-fallow-argument-mismatch"`
`export FFLAGS="-fallow-argument-mismatch"`
This will turn mismatch errors between actual and dummy argument lists to warnings. See:
https://github.com/Unidata/netcdf-fortran/issues/212
https://gcc.gnu.org/gcc-10/changes.html
Unidata now packages Netcdf-4 and legacy Netcdf-3 separately. HPE has decided not to continue supplying the legacy Netcdf-3 package. Due to CCE changes a version of netcdf built with "-sreal64" is neither needed nor provided.
NetCDF is supported on the host CPU but not on the accelerator.
## Documentation:
http://www.unidata.ucar.edu/software/netcdf/docs
## Modulefile:
`module load cray-netcdf`
OR
`module load cray-netcdf-hdf5parallel`
## Product Description:
NetCDF (network Common Data Form) is a set of interfaces for array-oriented data access and a freely-distributed collection of data access libraries for C, Fortran, C++, Java, and other languages. The netCDF libraries support a machine-independent format for representing scientific data. Together, the interfaces, libraries, and format support the creation, access, and sharing of scientific data.
## Installation:
`rpm -ivh cray-netcdf-4.9.2.1-*4.9.2.1.@ARCH@.rpm`
The "*" in the install command represents compiler version combinations.
To make this the default version, execute:
`/opt/cray/pe/admin-pe/set_default_files/set_default_netcdf_4.9.2.1`
Certain components, files or programs contained within this package or product are Copyright 2019-2025 Hewlett Packard Enterprise Development LP.
Parallel NetCDF 1.12.3.19
# Parallel NetCDF 1.12.3.19
## Purpose:
This Parallel NetCDF release contains the following:
* SLES 15 SP6 (x86_64, aarch64)
* SLES 15 SP7 (x86_64, aarch64)
* RHEL 9.5 (x86_64, aarch64)
* RHEL 9.6 (x86_64, aarch64)
* RHEL 8.10 (x86_64)
* Apollo RHEL 8.10 (x86_64)
* CCE 20.0.0
* AMD ROCm 6.4
* AOCC 5.0
* Intel OneAPI 2025.1
* Nvidia HPC SDK 25.5
* GCC 14
See the Parallel NetCDF documentation for details and usage information.
## Product and OS Dependencies:
This Parallel NetCDF 1.12.3.19 release is supported on on the following systems, requiring the following software products:
* CrayPE 2.1.2 or later
* SLES 15 SP6 (x86_64, aarch64)
* SLES 15 SP7 (x86_64, aarch64)
* RHEL 8.10 (x86_64)
* Apollo RHEL 8.10 (x86_64)
* RHEL 9.5 (x86_64, aarch64)
* RHEL 9.6 (x86_64, aarch64)
* cray-mpich/8.1.0 or later
Supported on the following HPE systems:
* HPE Cray EX systems with CLE
* HPE Cray XD systems with CLE
One or more of the following compiler major versions:
* CCE 20.x or later (SLES, RHEL)
* GCC 12.3 or later (SLES)
* gcc-toolset-10.3 to 14.2 (RHEL 8.10)
* gcc-toolset-12.2 to 14.2 (RHEL 9.5)
* gcc-toolset-12.2 to 14.2 (RHEL 9.6)
* AOCC 4.2.0 or later
* AMD ROCm 6.2.1 or later (SLES and RHEL) or AMD ROCm 6.2 or later (Apollo RHEL 8.10)
* Intel 2024.2.0 or later
* Nvidia 23.11 or later (aarch64) and 23.3 or later (x86_64)
## Notes and Limitations:
For certain F77 source files, when compiling with `gcc/10+`, it may be necessary to configure/compile with:
`export FCFLAGS="-fallow-argument-mismatch"`
`export FFLAGS="-fallow-argument-mismatch"`
This will turn mismatch errors between actual and dummy argument lists to warnings. See:
https://github.com/Unidata/netcdf-fortran/issues/212
https://gcc.gnu.org/gcc-10/changes.html
## Documentation:
https://parallel-netcdf.github.io/wiki/Documentation.html
## Modulefile:
`module load cray-parallel-netcdf`
## Product Description:
Parallel NetCDF is a library providing high-performance I/O while maintaining file-format compatibility with Unidata's NetCDF.
## Installation:
`rpm -ivh cray-parallel-netcdf-1.12.3.19-*202508271620.a2b367d8e8dbf.@ARCH@.rpm`
The "`*`" in the install command represents compiler version combinations.
To make this the default version, execute:
`/opt/cray/pe/admin-pe/set_default_files/set_default_parallel-netcdf_1.12.3.19`
Certain components, files or programs contained within this package or product are Copyright 2019-2025 Hewlett Packard Enterprise Development LP.
Perftools 25.09.0
perftools %{pkgversion}
================================================================================
Product Description
---------------------------
- perftools is a suite of tools for sampling and tracing HPC program to identify
areas of poor performance.
Known Issues
---------------------------
- Use of ROCm 6.3.0 may result in logs to stdout of the following form':'
"E20250612 13:37:18.297655 140234151356288 agent.cpp:969] size of rocprofiler agent
struct used by caller is ABI-incompatible with rocprofiler_agent_v0_t in rocprofiler"
These are benign and can be ignored.
- In a profile or other report table, if some function or kernel names are empty
or are difficult to interpret, try using either or both of these
pat_report options: -P and -s prune_name=''
both with the original table and with the option: -O callers
- We have observed some application crashes when running with perftools-lite-gpu,
running with pat_run -m lite-gpu can be used as a work around.
Notes
---------------------------
- This is a feature and bugfix release for the following systems
- HPE Cray EX and HPE Cray Supercomputer Systems with HPCM
- HPE Cray EX and HPE Cray Supercomputer Systems with CSM
- Supported components
- ROCm 6.4.0 and 6.3.0
- CUDA 12.8
- PAPI 7.2.0.2
- DWARF 2.0.0
- Supported OS
- HPE CSM systems with SLES 15 SP6 - SP7
- PE HPCM systems with SLES 15 SP6 - SP7 or RHEL 8.10, 9.5, 9.6
- Deprecations and removals
- Apprentice2 is removed from this release
- The experimental high bandwidth memory (HBM) profiling is
deprecated and will be removed from future releases.
- Additional dependencies
- For a list of software used when validating this version of
Perftools on Cray and HPE systems, see the HPE Cray Programming Environment
release announcements.
Changelog
---------------------------
=======================================================================
## [25.09.0]
=======================================================================
### Bug Fixes
* Fix crash from memory overwrite
* Unintalling perftools leaves left over image files
* CPE-8326: __libc_start_call_main shows in calltree before main
* Recognize ignoring counter group files correctly
### Features
* Add configuration dialog to enable openssh based connection in Apprentice 3
* Cpe-9148: support collecting of Cassini retry handler counters
* Cpe-2086: produce supporting files for cassini sensor counters
* CPE-5645: Add ability to connect to data on a remote host via OpenSSH
* Update the Cassini counter groups and native avail files
* Automate creation of the changelog
* CPE-7993: Support MPI RMA in mosaic view
* Add apprentice 3 documentation for Profile and Counter panels
* Add getting started guide
=======================================================================
## [25.03.0]
=======================================================================
### Bug Fixes
* Fix crash in otf2 export CPE-7830
* Update default python pruning for 3.12
* Ensure install dir exists and error out if unwritable
* Fix multiple Mac installers in develop branch rpms.
* Fix crash initializing the call graph
* Missing character when a program has operator%
* Remove OmpTool group warning
* Update CUDA PM
* Get cortland builds going
* Fix mac buildbot builds
* Apprentice2: Two-Pie Panel Help is Wrong.
### Features
* Add getting started guide
* Build with gcc-14 on rhel 9.5 and sles15sp6
* Add Apprentice 3 documentation to web site
* Expand HIP PMs
* Shift buildbot build command to repo
Documentation
---------------------------
- See `https':'//cpe.ext.hpe.com/docs/latest/performance-tools/index.html`
- Start with `man intro_craypat`
- See these man pages':'
- app3, grid_order, intro_craypat, pat_build, pat_help, pat_info, pat_opts,
pat_report, pat_run, pat_view
- intro_papi
- perftools-base, perftools-lite, perftools-preload
- accpc, cray_cassini, cray_pm, hwpc, nwpc, papi_counters, uncore
- PAPIlicnotices
Installation Instructions
---------------------------
- rpm -ivh %{cray_name}-%{pkgversion}-202508271723.ec0552af6b997-8.sles15sp7.x86_64.rpm
- To make %{cray_name} %{pkgversion} the default version of %{cray_name}, execute':'
%{cray_prefix}/admin-pe/set_default_files/set_default_%{cray_name}_%{pkgversion}
Legal Notices
---------------------------
%{copyright}
Attribution notices for open source licensed software for this package are detailed in the file:
$CRAYPAT_ROOT/ATTRIBUTIONS_perftools.txt
- Except for the third party modules discussed below and software licensed
by HPE through proprietary agreements, components, files or programs
contained within this package or product are Copyright 2001-2023
Hewlett Packard Enterprise Development LP.
Attribution notices for open source licensed software contained in this
package are detailed in the file:
$CRAYPAT_ROOT/ATTRIBUTIONS_perftools.txt
sanitizers4hpc 1.1.6
sanitizers4hpc 1.1.6
================================================================================
Product Description
---------------------------
- Sanitizers4hpc is a debugging tool to aid in the detection of memory leaks
and errors in parallel applications. Using LLVM's AddressSanitizer or
LeakSanitizer as a data source, Sanitizers4hpc aggregates any duplicate
messages across ranks to help provide an understandable picture of
program behavior.
Notes
---------------------------
- Shared library exposure in previous versions of sanitizers4hpc.
Previous versions of sanitizers4hpc incorrectly exported internal sanitizers4hpc
libraries into /opt/cray/pe/lib64, causing other programs that had that path in
their runpath or LD_LIBRARY_PATH to load libraries they didn't expect.
This has been fixed in the 22.08 release of sanitizers4hpc. However, links will
be present in /opt/cray/pe/lib64 pointing to older versions of sanitizers4hpc,
if an older version is installed because old installations will still modify
/opt/cray/pe/lib64 when ldconfig is run.
To clean up libraries left by previous versions of sanitizers4hpc:
Option 1:
1. Uninstall all previously installed versions of sanitizers4hpc, or remove
the .cray_dynamic_file_list file from all previously installed versions
of sanitizers4hpc, typically installed in /opt/cray/pe/sanitizers4hpc.
2. Remove any links in /opt/cray/pe/lib64 pointing into a sanitizers4hpc installation.
3. run ldconfig
Option 2:
1. Uninstall all previous versions of santizers4hpc.
Changelog
---------------------------
=======================================================================
## [1.1.4]
=======================================================================
### Bug Fixes
* CPE-6064: Don't reset ZMQNet install directory
=======================================================================
## [1.1.3]
=======================================================================
### Bug Fixes
* PE-54265: Automatically preload libfabric if linked
### Features
* CPE-8108: Improve diagnostic when debug info stripped
* CPE-7776: Add check for HIP ASan instrumentation
=======================================================================
## [1.1.5]
=======================================================================
### Bug Fixes
* CPE-6849: Suppress AMD libhsa-runtime false positives #99
* CPE-6064: Don't reset ZMQNet install directory
* PE-54417: Strip trailing BuildId if present
* PE-54265: Automatically preload libfabric if linked
* PE-53866: Improve ThreadSanitizers and Nvidia Compute Sanitizers aggregation
* PE-53603: Update ZMQNet submodule for libsodium support
* PE-51193: Activate PALS startup barrier manually through libpals
* PE-49611: Update ZMQNet
### Features
* Build with g++14 on rhel 9.5 and sles15sp6
* Add rhel 9.5, remove rhel 8.9 and sles15sp4
* CPE-6125: Support Clang MemorySanitizers
* CPE-6249: Aggregate GPU workgroup IDs
* CPE-1039: Automatically detect and add instrumented HIP runtime to LD_LIBRARY_PATH
* CPE-5880: Automatically build ASan MPI wrappers
* Support rhel 8.10
* Add builds for rhel 9.4 and sles15sp6
* Add builds for rhel 9.4 and sles15sp6
* PE-51960: Added support for rhel89/93 & gcc13
* Use gcc toolset 12 on redhat
OS Dependencies
---------------------------
- HPE CSM systems with SLES 15 SP5 - SP6
- HPE HPCM systems with SLES 15 SP5 - SP6 or RHEL 8.10 - 9.4
- HPE Apollo systems with RHEL 8.10
Product Dependencies
---------------------------
- cray-cti >= 2.20, < 3 installed
Installation Instructions
---------------------------
- rpm -ivh sanitizers4hpc-1.1.6-20250827184436_695ef5c0eb4c-122.sles15sp7.x86_64.rpm
To make sanitizers4hpc 1.1.6 the default version of sanitizers4hpc, execute:
/opt/cray/pe/admin-pe/set_default_files/set_default_sanitizers4hpc_1.1.6
Legal Notices
---------------------------
Certain components, files or programs contained within this package or product are Copyright 2022-2025 Hewlett Packard Enterprise Development LP.
Attribution notices for open source licensed software for this package are detailed in the file:
/opt/cray/pe/sanitizers4hpc/1.1.6/ATTRIBUTIONS_sanitizers4hpc.txt
valgrind4hpc 2.13.6
valgrind4hpc 2.13.6
================================================================================
Product Description
---------------------------
- Valgrind4hpc is a Valgrind-based debugging tool to aid in the detection of memory
leaks and errors in parallel applications. Valgrind4hpc aggregates any duplicate
messages across ranks to help provide an understandable picture of
program behavior. Valgrind4hpc manages starting and redirecting output from many
copies of Valgrind, as well as deduplicating and filtering Valgrind messages.
If your program can be debugged with Valgrind, it can be debugged with Valgrind4hpc.
Changelog
---------------------------
=======================================================================
## [2.13.6]
=======================================================================
### Bug Fixes
* CPE-9080: Fix output file path
### Features
* Remove dependence on cdst-support
=======================================================================
## [2.13.5]
=======================================================================
### Features
* CPE-7743: Update Valgrind to 3.24
* Build with g++14 on rhel 9.5 and sles15sp6
* Add rhel 9.5, remove rhel 8.9 and sles15sp4
* CPE-6037: Add source code to user guide
=======================================================================
## [2.13.4]
=======================================================================
### Bug Fixes
* PE-54291: Finish analysis early if job exited unexpectedly
### Features
* Support rhel 8.10
* Support rhel 8.10
* Add builds for rhel 9.4 and sles15sp6
* PE-54471: Add suppressions for aarch64 libcxi
* PE-55037: Remove deprecated exp-sgcheck tool support
OS Dependencies
---------------------------
- HPE CSM systems with SLES 15 SP5 - SP6
- HPE HPCM systems with SLES 15 SP5 - SP6 or RHEL 8.10 - 9.4
- HPE Apollo systems with RHEL 8.10
Product Dependencies
---------------------------
- cray-mrnet >= 5.1, < 6 installed
- cray-cti >= 2.20, < 3 installed
Documentation
---------------------------
- See man valgrind4hpc.
Installation Instructions
---------------------------
- rpm -ivh valgrind4hpc-2.13.6-20250827185037_cd609cb5c0f2-444.sles15sp7.x86_64.rpm
To make valgrind4hpc 2.13.6 the default version of valgrind4hpc, execute:
/opt/cray/pe/admin-pe/set_default_files/set_default_valgrind4hpc_2.13.6
Legal Notices
---------------------------
Certain components, files or programs contained within this package or product are Copyright 2010-2025 Hewlett Packard Enterprise Development LP.
Attribution notices for open source licensed software for this package are detailed in the file:
/opt/cray/pe/valgrind4hpc/2.13.6/ATTRIBUTIONS_valgrind4hpc.txt