Release Notes
CPE Release Notes For RHEL 8.10
HPE Cray Programming Environment 24.07 Full Release Notes for RHEL 8.10
=========================================================
atp 3.15.4
================================================================================
*Released: June 2024*
Product Description
---------------------------
- Abnormal Termination Processing (ATP) is a system that
monitors user applications, and should an application
take a system trap, ATP preforms analysis on the dying
application. All of the stack backtraces of the
application processes are gathered into a merged
stack backtrace tree and written to disk as the file
"atpMergedBT.dot". The stack backtrace for the first process
to die is sent to stderr as is the number of the signal that
caused the death. If the core file size limit (RLIMIT_CORE)
is non-zero, a heuristically selected set of processes dump
their core.
atpMergedBT.dot can be viewed with 'statview', a component
of the STAT package (module load cray-stat). The merged stack
backtrace tree provides a concise, yet comprehensive, view
of what the application was doing at the time of its death.
Changelog
---------------------------
=======================================================================
## [3.15.4] - 2024-05-2 (24.06)
=======================================================================
### Bug Fixes
* PE-53603: ZMQNet libsodium support
* PE-52560: Pass all ATP environment variables to Flux plugin
* PE-51520: Remove disable message from Flux plugin
* PE-52423: Disable PALS local launch in modulefile
### Features
* PE-53605: Build and package libsodium
* PE-51960: Added support for rhel89/93 & gcc13
* PE-51520: Automatic plugin loading for Flux
=======================================================================
## [3.15.2] - 2023-10-4 (23.11)
=======================================================================
### Bug Fixes
* PE-50510: Set CCE 16 Fortran line number workaround as optional
* PE-49776: Fall back to /tmp if TMPDIR or HOME is invalid
* PE-49611: Update ZMQNet submodule
### Features
* Move to gcc toolset 12 on redhat
* PE-49434: Add function and file to stack trace summary
=======================================================================
## [3.15.1] - 2023-07-19 (23.08)
=======================================================================
### Bug Fixes
* PE-47129: Work around CCE 16 Fortran source file info bug
* PE-47436: Reset Lmod module load variable after use
### Features
* PE-48781: Automatic MRNet load not required due to rpath update
* Use native compilers on sles
* PE-47183: Use RTD style for HTML docs
* PE-47182: Provide web documentation at `share/doc`
OS Dependencies
---------------------------
- HPE CSM systems with SLES 15 SP4
- HPE HPCM systems with SLES 15 SP5 and RHEL 8.8
- HPE Apollo systems with RHEL 8.8
Product Dependencies
---------------------------
- cray-mrnet >= 5.1, < 6 installed
- cray-cti >= 2.18, < 3 installed
- cdst-support >= 2.14, < 3 installed
Documentation
---------------------------
- Type `man atp` with the atp module loaded to read the atp(1) man page.
Installation Instructions
---------------------------
- rpm -ivh atp-3.15.4-20240508181140_4766b5970bed-1.el8.x86_64.rpm
To make atp 3.15.4 the default version of atp, execute:
/opt/cray/pe/admin-pe/set_default_files/set_default_atp_3.15.4
Legal Notices
---------------------------
Certain components, files or programs contained within this package or product are Copyright 2010-2024 Hewlett Packard Enterprise Development LP.
Attribution notices for open source licensed software for this package are detailed in the file:
/opt/cray/pe/atp/3.15.4/ATTRIBUTIONS_atp.txt
cray-ccdb 5.0.4
================================================================================
*Released: May 2024*
Product Description
---------------------------
- CCDB, Cray's next generation debugging tool extends the comparative
debugging capabilities of gdb4hpc with a Graphical User Interface (GUI)
enabling programmers to compare corresponding data structures
between two executing applications. Comparative debugging assists users with
locating sections of code containing data deviations introduced by algorithm
changes, compiler differences, and porting to new architectures/libraries.
Changelog
---------------------------
=======================================================================
## [5.0.4] - 2024-04-22 (24.05)
=======================================================================
### Bug Fixes
* PE-50334: Fix missing console messages
### Features
* PE-51960: Added support for rhel89/93 & gcc13
=======================================================================
## [5.0.3] - 2024-02-7 (24.03)
=======================================================================
### Bug Fixes
* PE-51115: Make socket directory, add PBS allocation instructions
=======================================================================
## [5.0.2] - 2023-11-1 (23.12)
=======================================================================
### Features
* Use gcc toolset 12 on redhat
* Use gcc toolset 12 on redhat
OS Dependencies
---------------------------
- HPE CSM systems with SLES 15 SP4
- HPE HPCM systems with SLES 15 SP5 and RHEL 8.8
- HPE Apollo systems with RHEL 8.8
Product Dependencies
---------------------------
- gdb4hpc/4.14.x or later installed
Documentation
---------------------------
- Type `man ccdb` with the cray-ccdb module loaded to read the ccdb(1) man page.
- Documentation is available through CCDB from the 'Help' menu.
Installation Instructions
---------------------------
- rpm -ivh --oldpackage cray-ccdb-5.0.4-20240513224431_723809ac94e4-3.el8.x86_64.rpm
To make cray-ccdb 5.0.4 the default version of cray-ccdb, execute:
/opt/cray/pe/admin-pe/set_default_files/set_default_cray-ccdb_5.0.4
Legal Notices
---------------------------
Certain components, files or programs contained within this package or product are Copyright 2007-2024 Hewlett Packard Enterprise Development LP.
Attribution notices for open source licensed software for this package are detailed in the file:
/opt/cray/pe/ccdb/5.0.4/ATTRIBUTIONS_ccdb.txt
HPE Cray Compiling Environment (CCE) 18.0.0
===========================================
Release Date:
-------------
July 2024
Purpose:
--------
CCE 18.0.0 provides Fortran, C, and C++ compilers for
HPE Cray supercomputer and HPE Apollo 2000 Gen10Plus systems.
Key Changes and Support with CCE 18.0.0:
----------------------------------------
CCE 18.0.0:
-----------
- LLVM 18 base (Merges up to May 18th 2024. LLVM version 18.1.6)
- Split CCE into subpackages
- Updated OMP_PLACE CPU affinity mapping policy to use absolute indices instead of relative
- New "-fopenmp-force-usm" compiler flag (C/C++/Fortran) enables OpenMP unified shared memory (USM) by treating each source file
as though it contains an explicit "omp requires unified_shared_memory" directive
- Added new "compiler_interop" man page documenting CCE compiler interoperability
- Added the following OpenMP 5.1/5.2 Features:
- interop construct and API routines
- compare and fail clause for atomic construct
- present modifier and defaultmap(present)
- nowait clause for taskwait directive
- default(private) and default(firstprivate) (C/C++)
- omp_target_memcpy_async and omp_target_memcpy_rect_async
- nothing directive (C/C++)
- Added the following OpenACC features:
- OpenACC 2.5
- Automatic allocate/deallocate for device_resident/create allocatables/pointers
- Support acc_memcpy_device routine
- Support num_gangs, num_workers and vector_length on kernels directive
- Support async copy routines
- Handle scoping for lexically visible data regions
- OpenACC 2.7
- Snap subarray/subscript expressions for use at the end of data regions
- Reduction on a compute construct implies a copy clause
- OpenACC 3.1
- Implement OpenACC serial
- Added the following Fortran 2023 features
- Control over leading zeros in output of real values: LEADING_ZERO= specifier
- Auto-allocation of character variables in more places - statements
- TYPEOF and CLASSOF declarations
- More use of BOZ constants
- Partial implementation of conditional expressions and arguments
- ENUMERATION TYPE
- SIMPLE attribute for procedures
- RHEL 8.10 x86_64 support
- RHEL 9.4 x86_64 support
- RHEL 9.4 aarch64 support
- OpenMP floating-point atomic optimization hints for AMD MI250X GPUs (Fortran)
- Advanced AMD GPU DWARF support
- Fortran scalar variables and fixed-size arrays in offload regions
- Fortran common blocks
- Optimized NVIDIA Grace CPU Superchip support
User and Application Impact:
----------------------------
CCE 18.0.0:
-----------
Issues or Bugs Resolved:
------------------------
CCE 18.0.0:
-----------
CAST-24911 Incorrect result form ASSOCIATED
CAST-26951 RFE: Need CCE OpenMP OMP_PLACES affinity to be able to map to taskset -c
CAST-30200 ftn compiler error module pointer omp target teams distribute parallel do
CAST-30667 [ELCAP-121] ftn compiler error module pointer omp target teams distribute parallel do
CAST-32453 Cray Fortran compiler is generating wrong assembly with OpenACC
CAST-32454 CCE runtime issue related to explicit type conversion of pointer variables with OpenACC.
CAST-32456 (ELCAP-318) Fortran compiler error in omp declare target subroutine - Elevation of case 5372036627
CAST-32588 Cray Compiler Bug : Issue with polymorphic and character type.
CAST-33337 XC50-CLE-Cray Fortran Compiler- Assignment (=) is not overridden
CAST-34158 crayftn Overloaded Constructor With Polymorphic Argument in Array Constructor
CAST-34265 (ELCAP-466) paradyn target offload built with -O0 gets runtime ACC error
CAST-34571 Internal compiler error with OpenACC declare
CAST-34664 (ELCAP-493) Fortran compiler hangs/crashes on !$omp continuation statements.
CAST-34817 (ELCAP-506) ftn omp target offload runtime error with -O0
CAST-34819 HPECOE-34 CCE Fortran OpenMP offload Bug with Procedure as Argument in Declare Target is unsupported
CAST-34871 (ELCAP-510) CCE17 Fortran crashes
CAST-34910 Incorrect code with floating point optimizations in Cray Fortran
CAST-35003 (ELCAP-522) Minikin - ftn unified memory wrong result for module save variable and declare target
CAST-35004 (ELCAP-527) Minikin runtime error with CCE 17 ftn
CAST-35054 (ELCAP-513) Fortran OMP declare target in module causes runtime error with ROCm-5.7.0+
CAST-35193 Cray Compiler error 'ftn-7060' with debug build of OpenACC code
CAST-35240 Cray EX - several compilation errors compiling vasp with cce17.0.0 [5379756819 Elevation]
CAST-35257 (ELCAP-533) Fortran seg fault with nested module subroutines
CAST-35401 Cray Openmp implementation and buggy thread affinity
CAST-35512 [LUMI] ftn -hfunc_trace crashes the compiler when END statement is not named for modules
CAST-35571 Bug with compiler cce/17.0.0
CAST-35625 No source line number displayed when run with MemorySanitizer in PrgEnv-cray
CAST-35726 ftn-7206 warning in cce/17.0
CAST-35776 crayftn ICE when trying to build dftd4
CAST-35903 (ELCAP-596) Fortran compiler segfault with OpenMP and requires statement
CAST-35946 (ELCAP-610) unified memory runtime error using ftn allocatable array from:module
CAST-35963 (ELCAP-615) Paradyn ftn openmp problem: bad size of allocated pointer with omp requires unified_shared_memory
Product and OS Dependencies:
----------------------------
This CCE release is supported on
- HPE Cray supercomputer systems running CSM with COS 2.5.X (SLES15 SP4) or COS 23.11 (SLES15 SP5)
- HPE Cray supercomputer systems running HPCM with COS 2.5.X (SLES15 SP4) or COS 23.11 (SLES15 SP5)
- HPE Cray supercomputer systems running HPCM with RHEL 8.10
- HPE Apollo 2000 Gen10Plus systems running RHEL 8.9
- AMD GPU offloading support requires at least ROCm 6.0 and is tested up to ROCm 6.1
Notes and Limitations:
----------------------
Documentation:
--------------
- S-5212 Cray Compiling Environment Release Overview (18.0)
- S-3901 Cray Fortran Reference Manual
- S-2179 Cray C and C++ Quick Reference
- Basic man pages: crayftn(1), craycc(1), crayCC(1), intro_openmp(7), intro_openacc(7)
- Please see https://clang.llvm.org/docs/UsersManual.html or use the -help command line option for
more information on using Clang
Modulefile:
-----------
The following will load the modules necessary to use CCE:
module load PrgEnv-cray
The following will switch to x.y.z version of CCE:
module swap cce cce/x.y.z
Installation instructions:
--------------------------
rpm -ivh cce-18.0.0-202406031331.98280ea519a0c-5.el8.x86_64.rpm.x86_64.rpm
The following script will set CCE version 18.0.0 default:
/opt/cray/pe/admin-pe/set_default_files/set_default_cce_18.0.0
License:
--------
Except for the third party components and software licensed by HPE through
proprietary agreements, components, files or programs contained within this
package or product are Copyright -2024 Hewlett Packard Enterprise Development LP.
Attribution notices for open source licensed software for this
package are detailed in the file:
/opt/cray/pe/cce/18.0.0/ATTRIBUTIONS_18.0.txt
cpe-descriptive-manifest 24.07
=================================
Release Date: Jul 2024
Product description:
--------------------
The cpe-descriptive-manifest packages contain the JSON manifest files that
are to map CPE to a format that Spack can consume.
Schema Changelog:
-----------------
v1.4
----
* Add 'wrappers' as a boolean parameter
v1.3
----
* Change '_meta' opject's 'version' property to 'schema-version'.
* Add 'cpe-version' property '_meta' object.
v1.2
----
* Add 'system-type' property to the '_meta' object.
v1.1
----
* Add '_meta' object to define file type and schema version.
v1.0
----
* Initial schema.
Cray PE DL Plugin with Resiliency Support 22.06.1.2:
Release Date:
-------------
June 2022
Purpose:
--------
The following changes have been made since Cray PE DL Plugin 21.04.1:
o Updated DL framework support
o Introduction of resiliency support
o BFloat16 data type support
Supported Configurations:
-------------------------
Shasta/EX: CPU and AMD GPU support with PrgEnv-gnu
The following Deep Learning frameworks and versions are supported:
* TensorFlow v2.6
* PyTorch v1.10.2
Documentation:
--------------
For more information see the intro_dl_plugin man pages.
For info regarding use of the new resiliency feature, see the examples
and README.md in ${CRAYPE_ML_PLUGIN_BASEDIR}/examples/resiliency
Sample Installation instructions:
---------------------------------
Note: Package must be installed or sym linked in the default /opt/cray/pe path.,
ie: use of the --prefix flag for rpm is not supported.
rpm -ivh craype-dl-plugin-ftr-22.06.1.2*.x86_64.rpm
The "*" in the install command represents date and hash specific information.
Modulefile:
-----------
Shasta/EX:
module load craype-dl-plugin-ftr/22.06.1.2
Certain components, files or programs contained within this package or
product are Copyright 2017-2022 Hewlett Packard Enterprise Development LP.
Cray PE DL Plugin 21.02.1.3:
Release Date:
-------------
February 2021
Purpose:
--------
The following changes have been made since Cray PE DL Plugin 20.10.1:
o Bug fixes for CUDA 11 support and pip install of include source dist
o Initial CUDA 11.1 support
o Updated DL framework support
Supported Configurations:
-------------------------
CS: OpenMPI 4.0.2 and cray-mvapich 2.3.2 or newer with CPU and GPU support
XC CLE 7: CPU and GPU support
Shasta: CPU-only support
Apollo: CPU-only support
The following Deep Learning frameworks and versions are supported:
* TensorFlow v2.4
* PyTorch v1.7.1
Documentation:
--------------
For more information see the intro_dl_plugin man pages.
Known Issues:
-------------
* Using Tensorflow 1.14 binaries distributed by Google and Intel require portions of the DL
Plugin to be compiled with gcc 4.8. If using the included source distribution of the
DL Plugin to install the Python packagesinto a given Python installation, gcc 4.8
will need to be used to install said packages, assuming Google and Intel distributed
Tensorflow 1.14 binaries are used. For TensorFlow 1.15 and greater, use of gcc 7.x
is required instead.
Sample Installation instructions:
rpm -ivh craype-dl-plugin-py3-21.02.1.3*.x86_64.rpm
The "*" in the install command represents date and hash specific information.
Modulefile:
-----------
XC/Shasta:
module load craype-dl-plugin-py3/21.02.1.3
CS:
module load craype-dl-plugin-py3/openmpi/21.02.1.3
module load craype-dl-plugin-py3/mvapich/21.02.1.3
Certain components, files or programs contained within this package or
product are Copyright 2017-2021 Hewlett Packard Enterprise Development LP.
Cray PE DL Plugin 22.09.1:
Release Date:
-------------
September 2022
Purpose:
--------
The following changes have been made since Cray PE DL Plugin 22.08.1:
o Keras callbacks bug fixes
Supported Configurations:
-------------------------
CS: OpenMPI 4.0.2 and cray-mvapich 2.3.2 or newer with CPU and GPU support
XC: CLE 7 CPU and GPU support
EX: CPU and GPU support
Apollo: CPU-only support
The following Deep Learning frameworks and versions are supported:
* TensorFlow v2.6
* PyTorch v1.10
Documentation:
--------------
For more information see the intro_dl_plugin man pages.
Known Issues:
-------------
* Using Tensorflow 1.14 binaries distributed by Google and Intel require portions of the DL
Plugin to be compiled with gcc 4.8. If using the included source distribution of the
DL Plugin to install the Python packagesinto a given Python installation, gcc 4.8
will need to be used to install said packages, assuming Google and Intel distributed
Tensorflow 1.14 binaries are used. For TensorFlow 1.15 and greater, use of gcc 7.x
is required instead.
Sample Installation instructions:
rpm -ivh craype-dl-plugin-py3-22.09.1*.x86_64.rpm
The "*" in the install command represents date and hash specific information.
Modulefile:
-----------
XC/EX:
module load craype-dl-plugin-py3/22.09.1
CS:
module load craype-dl-plugin-py3/openmpi/21.04.1
module load craype-dl-plugin-py3/mvapich/21.04.1
Certain components, files or programs contained within this package or
product are Copyright 2017-2021 Hewlett Packard Enterprise Development LP.
Cray PE DL Plugin 22.12.1:
Release Date:
-------------
December 2022
Purpose:
--------
The following changes have been made since Cray PE DL Plugin 22.09.1:
o TF v2.9 Support
Supported Configurations:
-------------------------
CS: OpenMPI 4.0.2 and cray-mvapich 2.3.2 or newer with CPU and GPU support
XC: CLE 7 CPU and GPU support
EX: CPU and GPU support
Apollo: RHEL8.6 CPU and GPU support
The following Deep Learning frameworks and versions are supported:
* TensorFlow v2.9
* PyTorch v1.10
Documentation:
--------------
For more information see the intro_dl_plugin man pages.
Known Issues:
-------------
* Using Tensorflow 1.14 binaries distributed by Google and Intel require portions of the DL
Plugin to be compiled with gcc 4.8. If using the included source distribution of the
DL Plugin to install the Python packagesinto a given Python installation, gcc 4.8
will need to be used to install said packages, assuming Google and Intel distributed
Tensorflow 1.14 binaries are used. For TensorFlow 1.15 and greater, use of gcc 7.x
is required instead.
Sample Installation instructions:
rpm -ivh craype-dl-plugin-py3-22.12.1*.x86_64.rpm
The "*" in the install command represents date and hash specific information.
Modulefile:
-----------
XC/EX:
module load craype-dl-plugin-py3/22.12.1
CS:
module load craype-dl-plugin-py3/openmpi/21.04.1
module load craype-dl-plugin-py3/mvapich/21.04.1
Certain components, files or programs contained within this package or
product are Copyright 2017-2021 Hewlett Packard Enterprise Development LP.
CrayPE Targets 1.13.2
================================================================================
Release Date:
--------------------------------------------------------------------------------
July, 2024
Purpose:
--------------------------------------------------------------------------------
Adds modulefiles for targets - craype-accel-amd-gfx942
Bugs fixed in this release:
--------------------------------------------------------------------------------
Updates dependencies for NVIDIA accelerators
Dependencies:
--------------------------------------------------------------------------------
The CrayPE Targets 1.13.2 release is dependent on the following
software products:
CrayPE
set_default_3
lmod_scripts
Documentation:
--------------------------------------------------------------------------------
See manpages for cc, CC, ftn, intro_craype-api and pkg-config
See section 2.6 Using Targeting Modules of the Cray Programming Environment
User's Guide (S-2529-116)
See http://www.freedesktop.org/wiki/Software/pkg-config for a pkg-config
introduction.
Installation instructions:
--------------------------------------------------------------------------------
rpm -Uvh craype-targets-ex-hpcm-1.13.2-202406062350.172a8f0148d89-0.ex.hpcm.noarch.rpm
To make this the default version, execute:
/opt/cray/pe/admin-pe/set_default_files/set_default_craype-targets_1.13.2-tcl
/opt/cray/pe/admin-pe/set_default_files/set_default_craype-targets_1.13.2-lua
Product description:
--------------------------------------------------------------------------------
CrayPE Targets contains the modules for CrayPE targets.
TCL Modules - craype-accel-host craype-accel-amd-gfx90a craype-accel-amd-gfx908 craype-accel-amd-gfx940 craype-accel-amd-gfx942 craype-accel-intel-max craype-accel-nvidia70 craype-accel-nvidia80 craype-accel-nvidia90 craype-hugepages2M craype-hugepages4M craype-hugepages8M craype-hugepages16M craype-hugepages32M craype-hugepages64M craype-hugepages128M craype-hugepages256M craype-hugepages512M craype-hugepages1G craype-hugepages2G craype-arm-grace craype-x86-milan craype-x86-milan-x craype-x86-rome craype-x86-spr craype-x86-spr-hbm craype-x86-trento craype-x86-genoa craype-network-none craype-network-ofi craype-network-ucx
Lmod Modules - craype-accel-host craype-accel-amd-gfx90a craype-accel-amd-gfx908 craype-accel-amd-gfx940 craype-accel-amd-gfx942 craype-accel-intel-max craype-accel-nvidia70 craype-accel-nvidia80 craype-accel-nvidia90 craype-hugepages2M craype-hugepages4M craype-hugepages8M craype-hugepages16M craype-hugepages32M craype-hugepages64M craype-hugepages128M craype-hugepages256M craype-hugepages512M craype-hugepages1G craype-hugepages2G craype-arm-grace craype-x86-milan craype-x86-milan-x craype-x86-rome craype-x86-spr craype-x86-spr-hbm craype-x86-trento craype-x86-genoa craype-network-none craype-network-ofi craype-network-ucx
Copyright 2013-2023 Hewlett Packard Enterprise Development LP
CDEUX driver (craype) 2.7.32 - 3
==============
Release Date:
--------------
July 2024
Purpose:
--------
The Cray Development Environment User eXperience driver (CDEUX) (craype)
provides a common set of drivers for all CPE programming environments,
notably the C/C++ (cc/CC) and Fortran (ftn) languages.
New CDEUX driver (craype) features for HPE Cray EX and Apollo systems:
- Support for newer Red Hat Enterprise Linux versions
Key Changes and Bugs Closed:
----------------------------
- Fix loading a perftools module breaks "direct/moduleless" craycc
- Fix cc fails to create shared library in PrgEnv-gnu-amd
- Add missing documentation about RPATHs
Product and OS Dependencies:
----------------------------
The CDEUX driver (craype) 2.7.32 release is supported on the following
HPE systems:
* HPE Cray EX systems with CLE
* HPE Apollo systems as part of the Cray Programming Environment
Documentation:
---------------
See manpages for cc, CC, ftn, and intro_craype-api.
See the pkg-config manpage or
http://www.freedesktop.org/wiki/Software/pkg-config for a pkg-config
introduction.
Product description:
--------------------
CrayPE contains drivers, cc, CC, and ftn to compile for all CPE
Programming Environments.
Certain components, files or programs contained within this package or product
are Copyright 2013-2023 Hewlett Packard Enterprise Development LP.
License:
--------
Except for the third party components and software licensed by HPE
through proprietary agreements, components, files or programs contained
within this package or product are Copyright 2023 Hewlett Packard
Enterprise Development LP.
Craypkg-gen 1.3.33
==============
Release date
------------
June 2024
Purpose
-------
Bug fix release.
Bugs fixed in this release
--------------------------
- Add CUDA modulefiles and pkg-config file support
- Update the NVIDIA HPC SDK modulefiles to more closely resemble the AMD toolkit modules
Dependencies
------------
The craypkg-gen 1.3.33 release is supported on the following HPE
Cray systems:
- Cray EX systems SLE 15.0 or later and RHEL 8.0 or later.
Driver support for integrating Third Party C, C++, and Fortran libraries
through .pc files using pkg-config is used in CrayPE 2.x and later.
Limitation
----------
- Library dependencies for static libraries are not added to the .pc files
for keywords Requires.private and Libs.private. A warning is issued by
the craypkg-gen tool to advise users to add this information to the
libraries .pc files.
- RPM limits packages to 4GB
Documentation
-------------
Man pages for craypkg-gen are found by executing `module load craypkg-gen`
and then `man craypkg-gen`. See
http://www.freedesktop.org/wiki/Software/pkg-config for a pkg-config
introduction.
Examples for creating modulefiles for Intel, PGI and Python are included in
the craypkg-gen ‘doc’ directory:
/opt/cray/craypkg-gen/1.3.33/doc/intel_example.txt
/opt/cray/craypkg-gen/1.3.33/doc/pgi_example.txt
/opt/cray/craypkg-gen/1.3.33/doc/python_example.txt
Example:
- As an example, the Intel 16.0.3.210 compiler was recently released.
After installing the compiler the administrator creates a modulefile for
this release by executing the following commands:
# module load craypkg-gen
# craypkg-gen -m /opt/intel/compilers_and_libraries_2016.3.210
This version of the Intel compiler is made default by executing the
command:
# /opt/admin-pe/set_default_craypkg/set_default_intel_16.0.3.210
Installation instructions
-------------------------
rpm -ivh craypkg-gen-1.3.33-1.3.33-1.el8.noarch.rpm
To change the product version to default after installation:
/opt/cray/pe/admin-pe/set_default_files/set_default_craypkg-gen_1.3.33
Product description
-------------------
The craypkg-gen 1.3.33 utility provides the system administrator
a tool to integrate third party software with the Cray software stack.
Craypkg-gen assists with integration by creating .pc files for C, C++, and
Fortran libraries, pkg-config enabled modulefiles and RPMs.
The workflow for using craypkg-gen is
1) Build the open source software
2) Create .pc files for libraries
3) Create pkg-config enabled modulefiles
4) Customize pkg-config and modulefiles if needed
Certain components, files or programs contained within this package or product
are Copyright 2013-2024 Hewlett Packard Enterprise Development LP.
cray-cti 2.18.4
================================================================================
*Released: June 2024*
Product Description
---------------------------
- Cray Common Tools Interface (CTI) is a library that abstracts debugger tool support
to a common interface regardless of the underlying work load manager. It facilitates
the staging of files and the launch of tool daemons on the compute nodes associated
with a parallel job. Options and interfaces can be found in the cti(1) and cti(3)
manpages.
Known Issues
---------------------------
- Race condition workaround for PALS 1.2.3 in CTI 2.17.0 (22.11). In PALS 1.2.3, there
is a race condition between the tool launcher releasing a job from the startup barrier
and the job actually getting to the startup barrier. This can result in the job receiving
the startup barrier release signal before it actually arrives there, resulting in the job
getting stuck in the barrier. As a workaround, the environment variable
CTI_PALS_BARRIER_RELEASE_DELAY can be set to add a delay between job startup and barrier
release. If set to a positve integer n, CTI will wait n seconds between starting a
job and releasing it from the barrier on PALS. A delay as small as one second works in
most cases.
Notes
---------------------------
- CTI 2.17.1 (PE 22.12) contains a new feature, file deduplication, for Slurm and PALS
workload managers. If enabled, CTI will run a check on compute nodes to determine if files
can be excluded from the shipped package. This can significantly improve application startup
time. To enable, set the environment variable CTI_DEDUPLICATE_FILES=1.
Changelog
---------------------------
=======================================================================
## [unreleased]
=======================================================================
### Bug Fixes
* PE-54265: Don't add quotes to LD_PRELOAD
* PE-54079: Fix unreliable Flux application output
=======================================================================
## [2.18.4] - 2024-05-3 (24.06)
=======================================================================
### Bug Fixes
* PE-53554: Allow launch if debug log directory doesn't exist
### Features
* Add lock to C interface, basic multithread test
* PE-53555: Parse multiple Flux job ID formats
* PE-52995: Support Flux 0.60.0
* PE-51960: Added support for rhel89/93 & gcc13
=======================================================================
## [2.18.3] - 2024-02-15 (24.03)
=======================================================================
### Bug Fixes
* PE-52423: Fix deduplication check for PALS
* PE-52423: Disable local launch for PALS
* PE-52540: Always include rank 0 in Flux launch
* PE-51694: Flux: fix job layout case and file deduplication
* PE-51955: Only rebuild checksum if input files changed
* PE-51495: Fix Slurm isRunning for stopped srun and backgrounded sattach
* PE-51694: Fix Flux implementation for 0.57.0
* PE-51193: Support rank remapping in PALS backend
### Features
* PE-48498: Accept PALS attach in form pbs_job_id:pals_apid
OS Dependencies
---------------------------
- HPE CSM systems with SLES 15 SP4
- HPE HPCM systems with SLES 15 SP5 and RHEL 8.8
- HPE Apollo systems with RHEL 8.8
Product Dependencies
---------------------------
- cray-cdst-support >= 2.14, < 3 installed
- cray-dyninst 12.3.0 or later installed
Documentation
---------------------------
- Currently the only documentation available is found in common_tools_fe.h, common_tools_be.h,
and the tests in the 'tests' directory.
Installation Instructions
---------------------------
- rpm -ivh cray-cti-2.18.4-20240523151326_b5630478cfc3-4.el8.x86_64.rpm
To make cray-cti 2.18.4 the default version of cray-cti, execute:
/opt/cray/admin-pe/set_default_files/set_default_cray-cti-2.18.4
Legal Notices
---------------------------
Certain components, files or programs contained within this package or product are Copyright 2010-2024 Hewlett Packard Enterprise Development LP. SPDX-License-Identifier: Linux-OpenIB
Attribution notices for open source licensed software for this package are detailed in the file:
/opt/cray/pe/cti/2.18.4/ATTRIBUTIONS_cti.txt
Cray DSMML 0.3.0:
==========================
Release Date:
-------------
March, 2024
Product Information:
--------------------
DSMML is a stand-alone memory management library for maintaining distributed
shared symmetric memory heaps for top level PGAS languages and libraries like
Coarray Fortran, UPC, and OpenSHMEM. DSMML allows user libraries to create
multiple symmetric heaps and share information with other libraries. Through
DSMML, we could extract interoperability between PGAS programming models.
Announcements, release informations, supported environments, and backward
compatibility informations about this product can be viewed in the following
location: https://pe-cray.github.io/cray-dsmml/
Purpose:
--------
Cray DSMML 0.3.0 supports the following features:
- Create symmetric heap segments for being used by PGAS languages and
libraries
- Allows options to allocate, deallocate memory buffers from previously
created symmetric heap segments
- Enables sharing information about the symmetric heap segments between
different programming models
- Added support for shared symmetric heap (SSHEAP) that can be used to
provide enhanced SMP data transport in user-libraries like SHMEM, CAF, UPC
The following features are fixed in Cray DSMML 0.3.0 when compared
to Cray DSMML 0.2.2:
- Add support for the library for AARCH64 processor based platforms
- Internal bug fixes involving hugepage cleanups
Refer intro_dsmml(3) manpage for more information on the supported features
and syntax for different routines
Product and OS Dependencies:
----------------------------
The Cray DSMML 0.3.0 release is supported on the following
Cray systems:
* HPE Cray EX systems with CLE
Product and OS Dependencies by network type:
--------------------------------------------------+
| Shasta |
----------------------------+---------------------+
craype | >= 2.7.3 |
----------------------------+---------------------+
One or more compilers:
* AOCC 2.2 or later
* CCE 9.1 or later
* GNU 9.1 or later
* Intel 19.0 or later
* Nvidia 20.7 or later
Documentation:
--------------
Use the Cray DSMML man pages for more information on the library
functions and use intro_dsmml(3) man page for general information.
Use https://pe-cray.github.io/whitepapers/ for access to different
Cray DSMML specific whitepapers
Announcements, release informations, supported environments, and backward
compatibility informations about this product can be viewed in the following
location: https://pe-cray.github.io/cray-dsmml/
Modulefile:
-----------
module load cray-dsmml/0.3.0
Certain components, files or programs contained within this package or
product have the following Copyright:
Copyright 2018-2021 Hewlett Packard Enterprise Development LP.
cray-dyninst 12.3.2
================================================================================
*Released: May 2024*
Product Description
---------------------------
- cray-dyninst provides tools for binary instrumentation, analysis, and modification.
Changelog
---------------------------
=======================================================================
## [12.3.2] - 2024-04-22 (24.05)
=======================================================================
### Features
* PE-51960: Added support for rhel89/93 & gcc13
=======================================================================
## [12.3.1] - 2023-09-25 (23.10)
=======================================================================
### Features
* Use gcc toolset 12 on redhat
=======================================================================
## [12.3.0] - 2023-06-27 (23.07)
=======================================================================
### Features
* PE-48354: Add LD_LIBRARY_PATH to Modulefiles
* PE-48354: Update Dyninst to 12.3.0
* Use native compilers on sles
OS Dependencies
---------------------------
- HPE CSM systems with SLES 15 SP4
- HPE HPCM systems with SLES 15 SP5 and RHEL 8.8
- HPE Apollo systems with RHEL 8.8
Product Dependencies
---------------------------
- cdst-support >= 2.14, < 3 installed
Documentation
---------------------------
- https://github.com/dyninst/dyninst
Installation Instructions
---------------------------
- rpm -ivh cray-dyninst-12.3.2-20240503231748_8668bb4a546f-1.el8.x86_64.rpm
To make this the default version of cray-dyninst, execute:
/opt/cray/pe/admin-pe/set_default_files/set_default_cray-dyninst_12.3.2
Legal Notices
---------------------------
Certain components, files or programs contained within this package or product are Copyright 2007-2024 Hewlett Packard Enterprise Development LP.
Attribution notices for open source licensed software for this package are detailed in the file:
/opt/cray/pe/dyninst/12.3.2/ATTRIBUTIONS_dyninst.txt
FFTW 3.3.10.8
============
Release Date:
-------------
July 2024
Purpose:
--------
This Cray FFTW 3.3.10.8 release is supported on HPE/Cray EX and
HPE/Cray XD/Apollo systems. Cray FFTW is supported on the host CPU
but not on the accelerator of HPE/Cray systems.
The Cray FFTW 3.3.10.8 release provides the following:
- Compiler and dependency updates
See the Product and OS Dependencies section for details.
Product and OS Dependencies:
----------------------------
This Cray FFTW 3.3.10.8 release is supported on these HPE/Cray systems:
- HPE/Cray EX Systems with SLES 15 SP5, RHEL 8.10, or RHEL 9.4 based OS
- HPE/Cray XD/Apollo Systems with RHEL 8.10 or RHEL 9.4 based OS
This Cray FFTW 3.3.10.8 release requires the following software products:
craype 2.7.32 or later
cray-mpich 8.1.30 or later
One or more of the following major compiler versions:
CCE 18
GCC 13
AOCC 4
Nvidia 24.3 or later
AMD (ROCm) 6
Intel 2024 or later
Notes and Limitations:
----------------------
Starting with cray-fftw/3.3.6.1 the fftw module has been renamed from fftw
to cray-fftw.
Documentation:
--------------
http://www.fftw.org/index.html#documentation
See the intro_fftw3 manual page for additional information.
Modulefile:
-----------
module load cray-fftw
Installation
------------
To install the rpm, execute:
rpm -ivh cray-fftw-3.3.10.8-202406131706.ee3ebe23b54c9-1.el8.x86_64.rpm
To make this the default version, execute:
/opt/cray/pe/admin-pe/set_default_files/set_default_fftw_3.3.10.8
Certain components, files or programs contained within this package or
product are Copyright 2011-2024 Hewlett Packard Enterprise Development LP
gdb4hpc 4.16.2
================================================================================
*Released: June 2024*
Product Description
---------------------------
- gdb4hpc is a GDB-based parallel debugger used to debug applications compiled with
CCE, AOCC, GNU, and Intel Fortran, C and C++ compilers. It allows programmers to
either launch an application or attach to an already running application that
was launched on the system. Additionally, it provides comparative debugging
technology that enables programmers to compare data structures between two
executing applications. Comparative debugging should be used in conjunction
with the CCDB GUI tool accessed by loading the cray-ccdb module.
- Some features of gdb include:
* Command line parallel debugger allows for launching/attaching applications.
* Utilizes process sets to operate on a subset of application ranks.
* GDB-like feel, also implements a gdbmode to enable a true parallel gdb.
* GPU debugging is supported for OpenMP 4.0+ GPU directives; OpenMP 5.0 on AMD
and Nvidia GPU's.
* Workload manager support via Common Tools Interface (cray-cti).
Changelog
---------------------------
=======================================================================
## [4.16.2] - 2024-05-2 (24.06)
=======================================================================
### Bug Fixes
* PE-52737: Fix backend crash when debugging Intel OneAPI GPUs
### Features
* PE-47910: Run job cleanup in background
* PE-53716: Added user guide for using the Python gdb extension
* PE-51960: Added support for rhel89/93 & gcc13
* PE-53828: Implement Flux attach
* Automatically load python extensions when debugger starts
* PE-53100: Support for aggregated gdb python extension commands py-print and py-locals if underlying gdb has python extensions loaded
* PE-53100: Support for aggregated gdb python extension commands py-bt, py-list, py-up, and py-down if underlying gdb has python extensions loaded
* Handle "~" in the --debug launch argument
* PE-52589: Tweak default assertion script behavior - now assertion scripts stop iff an assertion fails default
* PE-51936: Added tab completion for the print command
=======================================================================
## [4.16.1] - 2024-02-7 (24.03)
=======================================================================
### Bug Fixes
* PE-52387: Fix segfault from list command
* PE-51694: Fix occasional crash when operating on a subset of ranks
* PE-44112: Track job status while running for quicker timeout and exit
* Fix recursive source command behavior
* Properly format /csv output with data over 1 million items
* PE-51165: Query ZMQNet for correct rank and target PID
### Features
* PE-51935: Tab completion with file names when setting breakpoints
* PE-51935: Tab completion with function names when setting breakpoints
* PE-50319: Improve feedback when switching command modes
* PE-51749: Tab completion for help, set, show commands
* PE-51752: Moved vscode plugin to gdb4hpc and packaged executable
* PE-51598: Add save all command
* PE-51254: Save and restore assertion scripts and decompositions
* PE-51253: Save and restore breakpoints
* Update included hpcc_demo
* Add set print value-caching
* Save decomposition values as convenience variables
=======================================================================
## [4.16.0] - 2023-11-7 (23.12)
=======================================================================
### Bug Fixes
* PE-50810: Support storing chunked values in client side variables
* Fix failure with p 1+1+1
* Fix formatting when printing very large arrays and vectors
* PE-50510: Set CCE 16 Fortran line number workaround as optional
* Crash where gdb cannot list a line
* PE-49611: Update ZMQNet submodule
### Features
* PE-50840: Update ZMQNet submodule
* PE-50503: Add set variable command
* PE-50619: Save values on the gdb4hpc client as convenience variables with $foo = my_variable syntax
* PE-50671: Support scoped assignment in print statements
* Support --debug=<dir> in launch and attach
* Use dots for progress when connecting
* PE-49938: Support Slurm MPMD launch
* PE-46238: Send signals other than SIGKILL to application
* Move to gcc toolset 12 on redhat
OS Dependencies
---------------------------
- HPE CSM systems with SLES 15 SP4
- HPE HPCM systems with SLES 15 SP5 and RHEL 8.8
- HPE Apollo systems with RHEL 8.8
Product Dependencies
---------------------------
- cray-mrnet >= 5.1, < 6 installed
- cray-cti 2.18, < 3 installed
- cdst-support >= 2.14, < 3 installed
Documentation
---------------------------
- Type `man gdb4hpc` with the gdb4hpc module loaded to read the gdb4hpc(1) man page.
Simple usage examples are provided at the end.
Also, type `help` at the gdb4hpc command line for more information on commands.
Installation Instructions
---------------------------
- rpm -ivh gdb4hpc-4.16.2-20240509040630_70d6b1455a0e-1.el8.x86_64.rpm
To make gdb4hpc 4.16.2 the default version of gdb4hpc, execute:
/opt/cray/pe/admin-pe/set_default_files/set_default_gdb4hpc_4.16.2
Legal Notices
---------------------------
Certain components, files or programs contained within this package or product are Copyright 2007-2024 Hewlett Packard Enterprise Development LP.
Attribution notices for open source licensed software for this package are detailed in the file:
/opt/cray/pe/gdb4hpc/4.16.2/ATTRIBUTIONS_gdb4hpc.txt
# HDF5 1.14.3.1
## Release Date:
* July 2024
## Purpose:
This HDF5 release contains the following:
* SLES 15 SP5 (x86_64 and aarch64)
* RHEL 9.4 (x86_64 and aarch64)
* RHEL 8.10 (x86_64)
* Nvidia HPC SDK 24.03
* AMD ROCm 6.1
* AOCC 4.2
* Intel OneAPI 2024.0.1
* RHEL gcc-toolset-13
## Product and OS Dependencies:
The HDF5 1.14.3.1 release is supported on the following systems, requiring the following software products:
* CrayPE 2.1.2 or later
* SLES 15 SP5 (x86_64 and aarch64)
* RHEL 8.10 (x86_64)
* RHEL 9.4 (x86_64 and aarch64)
* cray-mpich/8.1.0 or later
Supported on the following HPE systems:
* HPE Cray EX systems with CLE
* HPE Cray XD systems with CLE
One or more of the following compiler major versions:
* CCE 17.x or later (SLES)
* CCE 18.x or later (RHEL)
* GCC 12.3 or later (SLES)
* gcc-toolset-10.3/11.2/12.2 (RHEL 8.10)
* gcc-toolset-11.2/12.2/13.1 (RHEL 9.4)
* AOCC 4.1.0 or later
* AMD ROCm 6.0.0 or later
* Intel 2023.2.0 or later
* Nvidia 23.11 or later (aarch64) and 23.3 or later (x86_64)
## Notes and Limitations:
Users are encouraged to use the Cray compiler scripts (cc, ftn, CC) to include the HDF5 header files and link in the HDF5 libraries, rather than the HDF5 compiler scripts (h5cc, h5fc, h5c++) included in the release.
## Documentation:
http://www.hdfgroup.org/HDF5/doc/index.html
## Modulefile:
`module load cray-hdf5`
OR
`module load cray-hdf5-parallel`
## Product description:
HDF5 is a data model, library, and file format for storing and managing data. It supports an unlimited variety of datatypes, and is designed for flexible and efficient I/O and for high volume and complex data. HDF5 is portable and is extensible, allowing applications to evolve in their use of HDF5. The HDF5 Technology suite includes tools and applications for managing, manipulating, viewing, and analyzing data in the HDF5 format.
## Installation:
`rpm -ivh cray-hdf5-1.14.3.1-*-202406100539.1b08b37223c27.@ARCH@.rpm`
The "`*`" in the install command represents compiler version combinations.
To make this the default version, execute:
`/opt/cray/pe/admin-pe/set_default_files/set_default_hdf5_1.14.3.1`
Certain components, files or programs contained within this package or product are Copyright 2019-2024 Hewlett Packard Enterprise Development LP.
Cray LibSci 24.07.0
=====================
Release Date:
-------------
July 2024
Purpose:
--------
Cray LibSci 24.07.0 provides scientific libraries for Cray
HPC systems. Cray LibSci is supported on the host CPU but
not on the accelerator of these systems.
The Cray LibSci 24.07.0 release provides support for the following:
* RHEL 9.4 (x86_64 and aarch64)
* RHEL 8.10 (x86_64)
* Nvidia HPC SDK 24.03
* AMD ROCm 6.1
* AOCC 4.2
* Intel oneAPI 2024.0.1
* RHEL gcc-toolset-13
Cray LibSci 24.07.0 includes the following versions of publicly
available libraries:
* LAPACK 3.10.1 - For further information, see
http://www.netlib.org/lapack
* ScaLAPACK 2.2.0 - (Scalable LAPACK) For further information, see
http://www.netlib.org/scalapack.
* QDWH 2.0.0 KSVD 1.0.0 - Polar decomposition and SVD packages, see
https://github.com/ecrc/qdwh
https://github.com/ecrc/ksvd
Key Changes and Bugs Closed:
----------------------------
Changes in Cray LibSci 24.07.0
* Miscellaneous bug fixes
* Enabled module to set LD_LIBRARY_PATH when PE_LD_LIBRARY_PATH=system
* Removed Iterative Refinement Toolkit (IRT) support
Changes in Cray LibSci 24.03.0
* CAST-34956 - Lua module environment variable issue
Changed in Cray LibSci 23.12.5
* CAST-33777 - LibSci mutex contention issue
* Removed support for Intel KNL
Product and OS Dependencies:
----------------------------
The Cray LibSci 24.07.0 release requires the following:
* SLES 15 SP5 or RHEL 8.10/9.4
* SLES 15 SP5 (x86_64 and aarch64)
* RHEL 8.10 (x86_64)
* RHEL 9.4 (x86_64 and aarch64)
* cray-mpich/8.1.0 or later
Supported on the following HPE systems:
* HPE Cray EX systems with CLE
* HPE Cray XD systems with CLE
One or more of the following compiler major versions:
* CCE 17.x or later (SLES)
* CCE 18.x or later (RHEL)
* GCC 12.3 or later (SLES)
* gcc-toolset-10.3/11.2/12.2 (RHEL 8.10)
* gcc-toolset-11.2/12.2/13.1 (RHEL 9.4)
* AOCC 4.1.0 or later
* AMD ROCm 6.0.0 or later
* Intel 2023.2.0 or later
* Nvidia 23.11 or later (aarch64) and 23.3 or later (x86_64)
Notes and Limitations:
----------------------
OMP threaded versions:
The CrayPE 2.1.2 and later releases add support for link line generation
for the multi-threaded versions of the libsci library based on the OpenMP
flags the user specifies for each compiler:
* CCE by default links to the OpenMP LibSci library. CrayPE will link in
the serial version of LibSci when the CCE flag -hnoomp is used.
* GNU by default links serial LibSci library. CrayPE will link in the
OpenMP version of LibSci when the GNU flag -fopenmp is used.
* INTEL by default links serial LibSci library. CrayPE will link in the
OpenMP version of LibSci when the INTEL flag -qopenmp is used.
Stack size limit:
The 'ulimit -s unlimited' system stack size setting is required for
cray-libsci on all supported Cray platforms. This is typically set as a
system default by admin but may otherwise need to be set at runtime.
Non-default dynamic linking:
When using a non-default version of cray-libsci with dynamic linking
users should set at runtime and at linktime after loading the desired
cray-libsci module--or set the equivalent for their linux shell:
export LD_LIBRARY_PATH=$CRAY_LD_LIBRARY_PATH:$LD_LIBRARY_PATH
Documentation:
--------------
See the intro_libsci man page for additional information.
See the csmlversion man page for information to display version information
on the currently loaded scientific libraries.
Modulefile:
-----------
module load cray-libsci
Installation instructions:
--------------------------
LibSci is now packaged into separate compiler specific RPMs to allow
rpmbuild to correctly include compiler dependencies.
License:
--------
Except for the third party components and software licensed by HPE
through proprietary agreements, components, files or programs
contained within this package or product are Copyright 2001-2023
Hewlett Packard Enterprise Development LP.
Attribution notices for open source licensed software contained in this
package are detailed in the file:
/opt/cray/libsci/24.07.0/ATTRIBUTIONS
Cray LibSci_ACC 24.07.0
=======================
Release Date:
-------------
July 2024
Purpose:
--------
The Cray LibSci_ACC 24.07.0 release provides accelerated versions of
scientific libraries for HPE Cray systems with accelerators.
Product and OS Dependencies:
----------------------------
Cray LibSci_ACC 24.07.0 requires the following platform:
HPE Cray EX system with AMD MI250X
HPE Cray EX system with AMD MI300A
HPE Cray EX system with Nvidia Hopper
Cray LibSci_ACC 24.07.0 supports the following OS:
RHEL 8.10 x86_64
RHEL 9.4 x86_64 & aarch64
SLES 15 SP5 x86_64 & aarch64
Notes and Limitations:
----------------------
LAPACK workspace calculations from other implementations or hard-coded
values may not be compatible with libsci_acc. Workspaces reported from
a query to the corresponding libsci_acc function should be used.
The current supported usecase is a one-rank to one-gpu mapping for apps.
Performance improvements can be achieved in programs calling libsci_acc
subroutines by using pinned memory. See the intro_libsci_acc man page for
details.
Use of the aprun option "-cc none" or the srun option "--cpu_bind=none"
to disable core affinity is strongly suggested to maintain performance.
When using non-default cray-libsci_acc versions users should set equivalent
at runtime and linktime after loading the desired cray-libsci_acc module:
export LD_LIBRARY_PATH=$CRAY_LD_LIBRARY_PATH:$LD_LIBRARY_PATH
Documentation:
--------------
See the intro_libsci_acc man page for additional information.
Modulefile:
-----------
module load cray-libsci_acc
Sample Installation instructions:
---------------------------------
RPM install command:
rpm -ivh cray-libsci-acc-24.07.0-*.##ARCH##.rpm
The "*" in the install command represents compiler version combinations.
To make this the default version:
##PREFIX##/admin-pe/set_default_files/set_default_libsci_acc_24.07.0
License:
--------
Except for the third party modules and software licensed by HPE
through proprietary agreements, components, files or programs
contained within this package or product are Copyright -2023 Hewlett
Packard Enterprise Development LP.
Attribution notices for open source licensed software contained in
this package are detailed in the file:
##PREFIX##/libsci_acc/24.07.0/ATTRIBUTIONS_libsci_acc_24.07.0.txt
Cray Lmod 8.7.37
================================================================================
Release Date:
--------------------------------------------------------------------------------
March, 2024
Purpose:
--------------------------------------------------------------------------------
Provide a Lmod 8.7.37 package for CPE customers
Bugs fixed in this release:
--------------------------------------------------------------------------------
N/A
Documentation:
--------------------------------------------------------------------------------
See CPE's Installation and User Guides for documentation.
Also see Lmod's official website: https://lmod.readthedocs.io
Dependencies:
--------------------------------------------------------------------------------
The Lmod Scripts 8.7.37 release is dependent on the following
software:
bc
sed
lua >= 5.3
lua-luaposix
lua-luafilesystem
Installation instructions:
--------------------------------------------------------------------------------
rpm -Uvh cray-lmod-8.7.37-1.el8.x86_64.rpm
To make this the default version, update the symbolic "lmod" link
to the desired Lmod version directory.
Lmod Scripts 3.2.1
================================================================================
Release Date:
--------------------------------------------------------------------------------
December, 2023
Purpose:
--------------------------------------------------------------------------------
Updates package license.
Bugs fixed in this release:
--------------------------------------------------------------------------------
Documentation:
--------------------------------------------------------------------------------
See system Installation and User Guides for documentation
Dependencies:
--------------------------------------------------------------------------------
The Lmod Scripts 3.2.1 release is dependent on the following
software:
set_default_3
Lua
Lmod
Installation instructions:
--------------------------------------------------------------------------------
rpm -Uvh lmod_scripts-3.2.1-202405031504.c3177a9cde245-0.noarch.rpm
To make this the default version, execute:
/opt/cray/pe/admin-pe/set_default_files/set_default_lmod_scripts_3.2.1
Product description:
--------------------------------------------------------------------------------
Contains Lua scripts which support the Cray implementation of Lmod
Copyright 2020-2023 Hewlett Packard Enterprise Development LP
modules 3.2.11.7
==================
Release Date:
-------------
Dec, 2023
Purpose:
--------
- Package dependency update
Documentation:
---------------
See man-pages for module and modulefile.
Cray MPICH 8.1.30:
=======================================
Release Date:
-------------
June 1, 2024
Purpose:
--------
Cray MPICH 8.1.30 is based upon ANL MPICH 3.4a2 with support for libfabric
and is optimized for the Cray Programming Environment.
Major Differences Cray MPICH 8.1.30 from the XC Cray MPICH include:
- Uses the new ANL MPICH CH4 code path and libfabric for network
support.
- Does not support -default64 mode for Fortran
- Does not support C++ language bindings
New Cray MPICH features for HPE Cray EX and Apollo systems:
- Starting from the 8.1.26 release, Cray MPICH supports the Intel Sapphire Rapids CPU HBM
processor architecture.
- On systems with AMD GPUs, Cray MPICH 8.1.26 supports all ROCm
versions starting from ROCm 5.0, including the latest ROCm 5.5.0
release.
The Cray MPICH 8.1.25 release and prior versions of
Cray MPICH are only compatible with ROCm versions up to (and
including) the ROCm 5.4.0 release.
- Cray MPICH uses the libfabric "verbs;ofi_rxm" provider by default.
This is the supported and optimized OFI libfabric provider for
Slingshot-10 and Apollo systems.
- Cray MPICH offers support for multiple NICs per node. Starting with
version 8.0.8, by default Cray MPICH will use all available NICs on
a node. Several rank-to-NIC assignment policies are supported. For
details on choosing a policy for assigning ranks to NICS, or for
selecting a subset of available NICs, please see the following
environment variables documented in the mpi man page.
MPICH_OFI_NIC_VERBOSE
MPICH_OFI_NIC_POLICY
MPICH_OFI_NIC_MAPPING
MPICH_OFI_NUM_NICS
- Enhancements to the MPICH_OFI_NIC_POLICY NUMA mode have been added.
Starting with version 8.0.14, if the user selects the NUMA policy,
the NIC closest to the rank is selected. A NIC no longer needs to
reside in the same numa node as the rank. If multiple NICs are
assigned to the same numa node, the local ranks will round-robin
between them. Numa distances are analyzed to select the closest NIC.
- Cray MPICH supports creating a full connection grid during MPI_Init.
By default, OFI connections between ranks are set up on demand. This
allows for optimal performance while minimizing memory requirements.
However, for jobs requiring an all-to-all communication pattern, it
may be beneficial to create all OFI connections in a coordinated
manner at startup. See the MPICH_OFI_STARTUP_CONNECT description in
the mpi man page.
- Cray MPICH supports runtime switching to the UCX netmod starting
with version 8.0.14. To do this load the craype-network-ucx module
and module swap between Cray-MPICH and Cray-MPICH-UCX modules. For
more information including relevant environment variables reference
the intro_mpi man page with the Cray-MPICH-UCX module loaded.
- Lmod support for HPE Cray EX starting with Cray MPICH 8.0.16.
Key Changes and Bugs Closed:
----------------------------
Changes in Cray MPICH 8.1.30
New Features:
- Cray MPICH 8.1.30 for aarch64 is in early stage of support and
has some issues. Setting these variables will help circumvent most
known issues.
export MPICH_SMP_SINGLE_COPY_MODE=CMA
export MPICH_MALLOC_FALLBACK=1
Features and Bug Fixes:
- CAST-34323 - Implement MPICH CQ stall feature for performance improvements on Sapphire Rapids
- CAST-34363 - Enable GTL to use multiple CUDA streams per process
- CAST-35020 - Improve error handling when running out of space in the UDREG cache
- CAST-35124 - Bugfix on aarch64 for MPI_THREAD_MULTIPLE
- CAST-35420 - Bugfix for MPI-IO aggregator selection
- CAST-35543 - Enable GPU kernel support for MPI_Reduce
- CAST-35648 - RMA Windows - Plug memory leaks by freeing shm_cookies and target_info
- PE-52424 - Enable module to set LD_LIBRARY_PATH when PE_LD_LIBRARY_PATH=system
- PE-52488 - Turn off aarch64 XPMEM when GPU support is requested
- PE-52582 - Fix MPI_LONG_DOUBLE datatype for CCE on aarch64
- PE-52604 - Bugfix for memory leak in GTL HSA IPC cache
- PE-53506 - Bugfix for a race condition with some CUDA use cases
- PE-54032 - Enable Allreduce ops to use GPU staging buffers larger than 1GB
- PE-54192 - Document aarch64 CUDA smp behavior
- PE-54326 - Document new MPICH CQ Stall variables
- PE-54303 - Bugfix for HSA seg_info cache
Product and OS Dependencies:
----------------------------
The Cray MPICH 8.1.30 release is supported on the following HPE systems:
* HPE Cray EX systems with CLE
* HPE Apollo systems as part of the Cray Programming Environment
Product and OS Dependencies by network type:
--------------------------------------------------+
| Shasta |
----------------------------+---------------------+
craype | >= 2.7.6 |
----------------------------+---------------------+
cray-pals | >= 1.0.6 |
----------------------------+---------------------+
cray-pmi | >= 6.0.1 |
----------------------------+---------------------+
libfabric | >= 1.9.0 |
----------------------------+---------------------+
One or more compilers:
* AMD ROCM 6.0 or later
* AOCC 4.1 or later
* CCE 18.0 or later
* GNU 11.2 or later
* Intel 2022.1 or later
* Nvidia 23.3 or later
Notes and Limitations:
----------------------
Limitations in Cray MPICH 8.1.30:
- Cray MPICH 8.1.30 can support only ~2040 simultaneous MPI
communicators. This limit is less the XC Cray MPICH limit of
~4090 simultaneous communicators. Cray intends to raise the
limit in a future release of Cray MPICH for Shasta to at least
the XC limit.
Documentation:
--------------
For more information see the intro_mpi man page.
Modulefile:
-----------
module load cray-mpich/8.1.30
License:
--------
Except for the third party components and software licensed by HPE
through proprietary agreements, components, files or programs contained
within this package or product are Copyright -2021 Hewlett Packard
Enterprise Development LP.
Attribution notices for open source licensed software for this
package are detailed in the file:
/opt/cray/pe/mpich/8.1.30/ATTRIBUTIONS
Copyright -2023 Hewlett Packard Enterprise Development LP
Copyright 2022-2024 Hewlett Packard Enterprise Development LP
Product Release Information
================================================================================
Product identification:
-----------------------
HPE Cray MPIxlate: version 1.0.5 (built on 2024-06-03T22:40:39Z from 8085bef)
Purpose:
--------
HPE Cray MPIxlate enables applications compiled using an MPI library that
is not binary compatible with HPE Cray MPI, to be run without recompilation
on supported HPE platforms.
Warnings & Limitations:
-----------------------
- See NOTES in mpixlate(1)
Changes from previous release:
------------------------------
- New features:
* None
- Defects Fixed:
* PE-53958: Invalid library path for PrgEnv-cray
Platform, Operating System and Product dependencies:
----------------------------------------------------
Supported HPE platforms:
- HPE Cray EX systems
- HPE Apollo systems with Cray Programming Environment
Supported Operating Systems:
- SLES 15 SP4 and COS-2.5
- SLES 15 SP5 and COS-3.0
- RHEL 8.10
- RHEL 9.4
Product dependencies:
-----------------+----------
cray-mpich[-ucx] | >= 8.1.30
-----------------+----------
Documentation:
--------------
mpixlate(1)
Environment Modules
-------------------
module load cray-mpixlate/1.0.5
================================================================================
cray-mrnet 5.1.3
================================================================================
*Released: May 2024*
Product Description
---------------------------
- MRNet is a customizable, high-performance software infrastructure for
building scalable tools and applications. It supports efficient
multicast and data aggregation functionality using a tree of processes
between the tool's front-end and back-ends. MRNet-based tools may use
these internal processes to distribute many important tool activities,
for example to reduce data analysis time and keep tool front-end loads
manageable.
MRNet-based tool components communicate across logical channels called
streams. At MRNet internal processes, filters are bound to these streams to
synchronize and aggregate dataflows. Using filters, MRNet can efficiently
compute averages, sums, and other more complex aggregations and analyses
on tool data. MRNet also supports facilities that allow tool developers
dynamically load new tool-specific filters into the system.
Changelog
---------------------------
=======================================================================
## [5.1.3] - 2024-04-22 (24.05)
=======================================================================
### Features
* PE-51960: Added support for rhel89/93 & gcc13
=======================================================================
## [5.1.2] - 2023-09-25 (23.10)
=======================================================================
### Features
* Use gcc toolset 12 on redhat
=======================================================================
## [5.1.1] - 2023-06-14 (23.07)
=======================================================================
### Features
* Use native sles compilers
OS Dependencies
---------------------------
- HPE CSM systems with SLES 15 SP4
- HPE HPCM systems with SLES 15 SP5 and RHEL 8.8
- HPE Apollo systems with RHEL 8.8
Product Dependencies
---------------------------
- cray-cti >= 2.18, < 3 installed
- cdst-support >= 2.14, < 3 installed
Installation Instructions
---------------------------
- rpm -ivh cray-mrnet-5.1.3-20240506181137_4fa0df65d0e3-3.el8.x86_64.rpm
To make cray-mrnet 5.1.3 the default version of cray-mrnet, execute:
/opt/cray/pe/admin-pe/set_default_files/set_default_cray-mrnet_5.1.3
Legal Notices
---------------------------
Certain components, files or programs contained within this package or product are Copyright 2022-2024 Hewlett Packard Enterprise Development LP.
Attribution notices for open source licensed software for this package are detailed in the file:
/opt/cray/pe/mrnet/5.1.3/ATTRIBUTIONS_mrnet.txt
# NetCDF 4.9.0.13
## Release Date:
* July 2024
## Purpose:
This NetCDF release contains the following:
* SLES 15 SP5 (x86_64 and aarch64)
* RHEL 9.4 (x86_64 and aarch64)
* RHEL 8.10 (x86_64)
* Nvidia HPC SDK 24.03
* AMD ROCm 6.1
* AOCC 4.2
* Intel OneAPI 2024.0.1
* RHEL gcc-toolset-13
## Product and OS Dependencies:
The NetCDF 4.9.0.13 release is supported on the following systems, requiring the following software products:
* Cray HDF5 1.14.3.*
* CrayPE 2.1.2 or later
* Cray MPT 8.1 or later
* SLES 15 SP5 (x86_64 and aarch64)
* RHEL 8.10 (x86_64)
* RHEL 9.4 (x86_64 and aarch64)
Supported on the following HPE systems:
* HPE Cray EX systems with CLE
* HPE Cray XD systems with CLE
One or more of the following compiler major versions:
* CCE 17.x or later (SLES)
* CCE 18.x or later (RHEL)
* GCC 12.3 or later (SLES)
* gcc-toolset-10.3/11.2/12.2 (RHEL 8.10)
* gcc-toolset-11.2/12.2/13.1 (RHEL 9.4)
* AOCC 4.1.0 or later
* AMD ROCm 6.0.0 or later
* Intel 2023.2.0 or later
* Nvidia 23.11 or later (aarch64) and 23.3 or later (x86_64)
## Notes and Limitations:
For certain F77 source files, when compiling with `gcc/10+`, it may be necessary to configure/compile with:
`export FCFLAGS="-fallow-argument-mismatch"`
`export FFLAGS="-fallow-argument-mismatch"`
This will turn mismatch errors between actual and dummy argument lists to warnings. See:
https://github.com/Unidata/netcdf-fortran/issues/212
https://gcc.gnu.org/gcc-10/changes.html
Unidata now packages Netcdf-4 and legacy Netcdf-3 separately. HPE has decided not to continue supplying the legacy Netcdf-3 package. Due to CCE changes a version of netcdf built with "-sreal64" is neither needed nor provided.
NetCDF is supported on the host CPU but not on the accelerator.
## Documentation:
http://www.unidata.ucar.edu/software/netcdf/docs
## Modulefile:
`module load cray-netcdf`
OR
`module load cray-netcdf-hdf5parallel`
## Product Description:
NetCDF (network Common Data Form) is a set of interfaces for array-oriented data access and a freely-distributed collection of data access libraries for C, Fortran, C++, Java, and other languages. The netCDF libraries support a machine-independent format for representing scientific data. Together, the interfaces, libraries, and format support the creation, access, and sharing of scientific data.
## Installation:
`rpm -ivh cray-netcdf-4.9.0.13-*202406081619.32645f8ced41e-5.el8.x86_64.rpm`
The "*" in the install command represents compiler version combinations.
To make this the default version, execute:
`/opt/cray/pe/admin-pe/set_default_files/set_default_netcdf_4.9.0.13`
Certain components, files or programs contained within this package or product are Copyright 2019-2024 Hewlett Packard Enterprise Development LP.
Cray PALS 1.3.2:
================
Release Date:
-------------
Nov 2023
Purpose:
--------
Cray Parallel Application Launch Service (PALS) is an application launcher for
Cray PE applications.
Key Changes:
------------
Add host2xname tool
Allow configuring the PMI port range
Add more examples in manpages
Add SLES 15 SP5 build
Add SLES 15 SP4 and SP5 aarch64 build
Add PMI and PMI2 as --pmi options for mpiexec
Periodically ping applications
Add the ability to allocate additional VNIs for special services like DAOS
Add mpiexec --disable-rdzv-get option to disable rendezvous gets in the NICs
Enable --cpu-bind and --gpu-bind=verbose to work with --rankfile
Add PALSD_FANOUT env var for palsd to set fanout tree width
Add ability for simultaneous mpiexecs to divide up PBS job slots
Add RHEL 8.8 build
Add support for AMD & NVIDIA GPU visibility
Change the wire format for PMIx data to make PMIx_Fence more efficient
Increase core file size limit on RHEL systems
Add retries to SPIRE auth token call
Install aprun and mpirun as symlinks
Use position-independent executables
Change F_CXI_COLL_* -> FI_CXI_COLL_* environment variables
Configure Instant On URL on Shasta
Move binaries and libraries to /opt/cray
Change pmi_example to print NIC distances for all ranks
Make PALS command's nodelists consistent
Set ATOM_SOCKET by default
Improve vnid request error reporting
Launch with PMIx support automatically fetch NIC information
Reduce default fanout to 32 nodes
Require PMIx version 4.2.4 or newer
Send the job-step name to vnid when creating a job-step VNI
Include pid in palsd log messages
Raise maximum number of ZMQ sockets to 4096
Munge auth no longer used for some internal RPCs with trusted connections
Increase default send timeout to 8s
Fix errors found with clang scan-build
Fix nohup mpiexec behavior
Improve cleanup when launch RPC fails
Implement the verbose option for GPU binding
Fix support for the cycle keyword with the mpiexec --rankfile option
Fix error checking for GPU binding options
Better handle mpiexec disconnects
Fix segfault on PMIx RPC for missing app
Fix spawn-related memory leaks
Fix misleading warning in line buffer mode
Remove SLES 15 SP3 build
Deprecate PALS_FANOUT and APRUN_FANOUT environment variables
Install cray-pals RPM
Documentation:
--------------
For more information see aprun, mpiexec, palscmd, palsctrl, palsig, and palstat man page.
Modulefile:
-----------
module load cray-pals
Installation instructions:
--------------------------
rpm -ihv cray-pals-<version>-<date>.el8.x86_64.rpm
rpm -ihv cray-palsd-<version>-<date>.el8.x86_64.rpm
Copyright 2022-2023 Hewlett Packard Enterprise Development LP
cray-papi
=================
Release date:
-------------
June 2024
Package:
--------
cray-papi-7.1.0.2-202405031627.5e39f84427b3c-3 x86_64
Purpose:
--------
New version of papi 7.1.0.2 release 3
Documentation:
--------------
Overview: https://github.com/icl-utk-edu/papi
Web Site: https://icl.utk.edu/papi
Product description:
--------------------
PAPI aims to provide the tool designer and application engineer with a
consistent interface and methodology for use of the performance counter
hardware found in most major microprocessors. PAPI enables software
engineers to see, in near real time, the relation between software
performance and processor events.
Dependencies:
-------------
For a list of software used when validating this version of
PAPI on Cray and HPE systems, see the HPE Cray Programming
Environment release announcements.
Copyright 2015-2017,2019-2024 Hewlett Packard Enterprise Development LP
# Parallel NetCDF 1.12.3.13
## Release Date:
* July 2024
## Purpose:
This Parallel NetCDF release contains the following:
* SLES 15 SP5 (x86_64 and aarch64)
* RHEL 9.4 (x86_64 and aarch64)
* RHEL 8.10 (x86_64)
* Nvidia HPC SDK 24.03
* AMD ROCm 6.1
* AOCC 4.2
* Intel OneAPI 2024.0.1
* RHEL gcc-toolset-13
See the Parallel NetCDF documentation for details and usage information.
## Product and OS Dependencies:
This Parallel NetCDF 1.12.3.13 release is supported on on the following systems, requiring the following software products:
* CrayPE 2.1.2 or later
* Cray MPT 8.1 or later
* SLES 15 SP5 (x86_64 and aarch64)
* RHEL 8.10 (x86_64)
* RHEL 9.4 (x86_64 and aarch64)
Supported on the following HPE systems:
* HPE Cray EX systems with CLE
* HPE Cray XD systems with CLE
One or more of the following compiler major versions:
* CCE 17.x or later (SLES)
* CCE 18.x or later (RHEL)
* GCC 12.3 or later (SLES)
* gcc-toolset-10.3/11.2/12.2 (RHEL 8.10)
* gcc-toolset-11.2/12.2/13.1 (RHEL 9.4)
* AOCC 4.1.0 or later
* AMD ROCm 6.0.0 or later
* Intel 2023.2.0 or later
* Nvidia 23.11 or later (aarch64) and 23.3 or later (x86_64)
## Notes and Limitations:
For certain F77 source files, when compiling with `gcc/10+`, it may be necessary to configure/compile with:
`export FCFLAGS="-fallow-argument-mismatch"`
`export FFLAGS="-fallow-argument-mismatch"`
This will turn mismatch errors between actual and dummy argument lists to warnings. See:
https://github.com/Unidata/netcdf-fortran/issues/212
https://gcc.gnu.org/gcc-10/changes.html
## Documentation:
https://parallel-netcdf.github.io/wiki/Documentation.html
## Modulefile:
`module load cray-parallel-netcdf`
## Product Description:
Parallel NetCDF is a library providing high-performance I/O while maintaining file-format compatibility with Unidata's NetCDF.
## Installation:
`rpm -ivh cray-parallel-netcdf-1.12.3.13-*202406080630.dd5a30e54e2bc-6.el8.x86_64.rpm`
The "`*`" in the install command represents compiler version combinations.
To make this the default version, execute:
`/opt/cray/pe/admin-pe/set_default_files/set_default_parallel-netcdf_1.12.3.13`
Certain components, files or programs contained within this package or product are Copyright 2019-2024 Hewlett Packard Enterprise Development LP.
Perftools 24.07.0
=================
Release Date: July, 2024
Purpose:
========
This is a feature and bugfix release for the following systems:
- HPE Cray EX and HPE Cray Supercomputer Systems with HPCM
- HPE Cray EX and HPE Cray Supercomputer Systems with CSM
- HPE Apollo 2000 Gen 10 Plus Systems (x86)
Key enhancements or changes from the previous release:
======================================================
o Add support for ROCm 6.1.0
o Add support for CUDA 12.4
o Upgrade to PAPI 7.1.0.2
o Upgrade to DWARF 0.9.2
o Upgrade to support MPICH v8
o First GA release for Apprentice 3
o Accelerator-to-accelerator and host-to-host transfers from roctracer are recorded
o Histograms of message sizes are recorded by transfer type. See environment variables
PAT_RT_ACC_*_BINS in "man pat" for details.
o Extended support for tracing python code
o Record python class names when applicable in calltree leafs
o Add tracing groups cuda_math, hip_math, and rocm_math
o Add tracing groups for PyTorch
o Add tracing support for NCCL/RCCL
o Update trace group 'cuda' to CUDA 12.4
o Update trace group 'hip' and 'hsa' to support ROCm 6.1.0
o Update trace group 'fabric' to support libfabric 1.20.1
o Update trace group 'mpi' to support MPICH v8.1
Bugs Fixed:
===========
o Fix ssh access from Apprentice 2/3
o Fix crash on Apprentice 2 windows client
o Support applications where multiple threads trigger roctracer callback
o Account for missing 'product_name' file and check 'device' file in ROCm installs
o Suppress runtime error for lite-hbm in AMD cpus
o Fix missing 0 count events
o Fix potential loading problems by disabling trace for dlopen by default
o Support floating point counter events for AMD processors
o Support performance counters for AMD MI300A processor
o Various Apprentice2 displays
o Various improvements in tracing Python
o Properly support multiple accelerator threads
o No longer trace dynamic library functions by default
Notes:
======
o This release of Perftools only supports ROCm 6.1.0
o This release of Perftools only supports CUDA 12.0 and later
Known Issues:
=============
o MI300 and MI300a GPU performance counter collection is not supported
o For apps using PyTorch+ROCm installs, pat_run will use ROCm shared libraries found in $ROCM_PATH rather than those installed with PyTorch
o Apprentice 3.0 known issues
o Mac install packages are not signed in the release. To install you must "open" the .dmg file
and agree to the warning dialog.
o The windows desktop can fail to access remote experiments in some installs.
o The mac desktop can show up with inverted or wrong graphics if an older version of gtk is
installed on the machine.
o Potential crash when cutting and pasting into the windows client.
o Searching for function name in the time line view does not work.
o The timeline view can rest to an interval of [NaN,NaN]; reload the experiment to continue.
o Missing tool tips, help, etc.
Deprecations and Removals:
==========================
o Remove custom cray-libelf packaging; use system libelf
o Reveal is no longer supported
Dependencies:
=============
o A PDF reader (such as evince, acroread or okular) is required to use the
app2 online help.
o For a list of software used when validating this version of
Perftools on Cray and HPE systems, see the HPE Cray Programming Environment
release announcements.
Installation instructions:
==========================
For Apollo 80 systems, please see the HPE Cray Programming Environments
Installation Guide: Apollo 80 (ARM) System (S-8013).
For Apollo 2000 systems, please see the HPE Cray Programming Environments
Installation Guide: Apollo 2000 Gen10 Plus (x86) System (S-8012).
For HPE Cray EX (Shasta Architecture) systems, refer to the Cray Asynchronous
Installer Guide (S-8003).
Installation of app2 remote client (+ server) on Mac systems:
-------------------------------------------------------------
Apprentice2Installer-24.07.0.dmg
Apprentice3Installer-24.07.0.dmg
The Cray Apprentice2 and Apprentice3 installers for Mac are included in the
perftools-clients rpm, and placed in
$CRAYPAT_ROOT/share/desktop_installers/
on a Cray Shasta User Access Node (UAN) or on a Cray login node when the
perftools software is installed.
Download the Cray Apprentice2 or Apprentice3 installer onto a desktop or laptop
running Mac OS El Capitan through Mojave. Double click on installer
to begin installation. The installer will walk you through the
process for your system.
Installation of app2 remote client (+ server) on Windows 10 systems:
--------------------------------------------------------------------
Apprentice2Installer-24.07.0.exe
Apprentice3Installer-24.07.0.exe
The Cray Apprentice2 and Apprentice3 installers for Windows are included in the
perftools-clients rpm, and placed in
$CRAYPAT_ROOT/share/desktop_installers/
on a Cray Shasta User Access Node (UAN) or on a Cray login node when the
perftools software is installed.
Download the Cray Apprentice2 installer onto a desktop or laptop
running Windows 10. Double click on installer to begin installation.
The installer will walk you through the process for your system.
Documentation:
==============
See the following man pages:
app2, grid_order, intro_craypat, pat_build, pat_help, pat_info, pat_opts,
pat_report, pat_run, pat_view
intro_papi
perftools-base, perftools-lite, perftools-preload
accpc, cray_cassini, cray_pm, hwpc, nwpc, papi_counters, uncore
PAPIlicnotices
Search for perftools on https://support.hpe.com to access documentation
on Cray Performance Measurement and Analysis Tools
License:
========
Except for the third party modules discussed below and software licensed
by HPE through proprietary agreements, components, files or programs
contained within this package or product are Copyright 2001-2024
Hewlett Packard Enterprise Development LP.
Attribution notices for open source licensed software contained in this
package are detailed in the file:
$CRAYPAT_ROOT/ATTRIBUTIONS_perftools.txt
Cray PMI 6.1.15.6
==============================================================================
Release Date:
--------------
June 26, 2024
Purpose:
--------
The Cray Process Manager Interface Library provides the interface between the
application launcher and other communication libraries such as MPICH and SHMEM.
Cray PMI 6.1.15.6 is the initial package with support for libfabric and
optimized for the Cray Shasta product line.
Major features new to Cray PMI 6.1.15.6 include:
Bug fixes new to Cray PMI 6.1.15.6 include:
* PE-29641 PMI_VERSION_DISPLAY=1 broken
Known limitations include:
Certain components, files or programs contained within this package or product
are Copyright 2004-2023 Hewlett Packard Enterprise Development LP
cray-python 3.11.7
====================
Release Date
------------
July 2024
Purpose
-------
Cray Python is an implementation of the Python programming language for the Cray
Programming Environment. The numpy and scipy modules are configured to call Cray
Libsci routines. The mpi4py module is configured to call Cray MPICH routines.
Cray Python is designed to run Python codes on the compute nodes of an HPE
supercomputer. HPE does not make changes to the Python source or any of its
libraries nor does it plan to make changes in future releases.
The cray-python 3.11.7 release contains
- python-3.11.7
- numpy-1.24.4
- scipy-1.10.1
- mpi4py-3.1.4
- dask-2023.6.1
Product and OS Dependencies
---------------------------
The cray-python 3.11.7 release is supported on
- Cray EX systems running SLES 15 or RHEL 8
Documentation
-------------
https://www.python.org/doc
Changelog
---------
https://docs.python.org/release/3.11.7/whatsnew/changelog.html
Modulefile
----------
module load cray-python/3.11.7
Installation
------------
rpm -ihv cray-python-3.11.7-20240617153006.0687944-1.el8.x86_64.rpm
To make this the default version, execute
[PREFIX]/admin-pe/set_default_files/set_default_python_3.11.7
Certain components, files, or programs contained within this package are
© Copyright 2021-2024 Hewlett Packard Enterprise Development LP
cray-R 4.4.0
============
Release Date
------------
March 2024
Purpose
-------
The cray-R 4.4.0 release.
cray-R is configured to call cray-libsci routines.
Product and OS Dependencies
---------------------------
The cray-R 4.4.0 release is supported on
- Cray EX systems running SLES 15 or RHEL 8
Documentation
-------------
https://cran.r-project.org/manuals.html
Modulefile
----------
module load cray-R/4.4.0
Installation
------------
rpm -ihv cray-R-4.4.0-20240617153123.a134495-1.el8.x86_64.rpm
To make this the default version, execute:
[PREFIX]/admin-pe/set_default_files/set_default_R_4.4.0
Certain components, files or programs contained within this package or product are
Copyright 2018-2024 Hewlett Packard Enterprise Development LP
sanitizers4hpc 1.1.3
================================================================================
*Released: June 2024*
Product Description
---------------------------
- Sanitizers4hpc is a debugging tool to aid in the detection of memory leaks
and errors in parallel applications. Using LLVM's AddressSanitizer or
LeakSanitizer as a data source, Sanitizers4hpc aggregates any duplicate
messages across ranks to help provide an understandable picture of
program behavior.
Notes
---------------------------
- Shared library exposure in previous versions of sanitizers4hpc.
Previous versions of sanitizers4hpc incorrectly exported internal sanitizers4hpc
libraries into /opt/cray/pe/lib64, causing other programs that had that path in
their runpath or LD_LIBRARY_PATH to load libraries they didn't expect.
This has been fixed in the 22.08 release of sanitizers4hpc. However, links will
be present in /opt/cray/pe/lib64 pointing to older versions of sanitizers4hpc,
if an older version is installed because old installations will still modify
/opt/cray/pe/lib64 when ldconfig is run.
To clean up libraries left by previous versions of sanitizers4hpc:
Option 1:
1. Uninstall all previously installed versions of sanitizers4hpc, or remove
the .cray_dynamic_file_list file from all previously installed versions
of sanitizers4hpc, typically installed in /opt/cray/pe/sanitizers4hpc.
2. Remove any links in /opt/cray/pe/lib64 pointing into a sanitizers4hpc installation.
3. run ldconfig
Option 2:
1. Uninstall all previous versions of santizers4hpc.
Changelog
---------------------------
=======================================================================
## [unreleased]
=======================================================================
### Bug Fixes
* PE-54265: Automatically preload libfabric if linked
=======================================================================
## [1.1.3] - 2024-05-2 (24.06)
=======================================================================
### Bug Fixes
* PE-53866: Improve ThreadSanitizers and Nvidia Compute Sanitizers aggregation
* PE-53603: Update ZMQNet submodule for libsodium support
* PE-51193: Activate PALS startup barrier manually through libpals
* PE-49611: Update ZMQNet
### Features
* PE-51960: Added support for rhel89/93 & gcc13
* Use gcc toolset 12 on redhat
=======================================================================
## [1.1.1] - 2023-07-7 (23.08)
=======================================================================
### Features
* PE-47186: Add Sanitizers4hpc User Guide
* PE-48278: Supply CUDA Sanitizer options via frontend argument
* Use native compilers on sles
* PE-48186: Support SHMEM non-shared heap checking, HTML documentation in share/doc
OS Dependencies
---------------------------
- HPE CSM systems with SLES 15 SP4
- HPE HPCM systems with SLES 15 SP5 and RHEL 8.8
- HPE Apollo systems with RHEL 8.8
Product Dependencies
---------------------------
- cray-cti >= 2.18, < 3 installed
- cdst-support >= 2.14, < 3 installed
Installation Instructions
---------------------------
- rpm -ivh sanitizers4hpc-1.1.3-20240523190500_2688b90db106-2.el8.x86_64.rpm
To make sanitizers4hpc 1.1.3 the default version of sanitizers4hpc, execute:
/opt/cray/pe/admin-pe/set_default_files/set_default_sanitizers4hpc_1.1.3
Legal Notices
---------------------------
Certain components, files or programs contained within this package or product are Copyright 2022-2024 Hewlett Packard Enterprise Development LP.
Attribution notices for open source licensed software for this package are detailed in the file:
/opt/cray/pe/sanitizers4hpc/1.1.3/ATTRIBUTIONS_sanitizers4hpc.txt
Cray OpenSHMEMX 11.7.2:
===============================
Release Date:
-------------
July, 2024
Purpose:
--------
OpenSHMEM is a Partitioned Global Address Space (PGAS) library interface
specification, which is the culmination of a standardization effort among
many implementers and users of SHMEM programming model. SHMEM has a long
history as a parallel programming model on HPE Cray supercomputer systems.
For the past two decades SHMEM library implementation in HPE Cray systems
evolved through different generations. Cray OpenSHMEMX is a proprietary,
OpenSHMEM specification compliant SHMEM implementation for HPE Cray
product line.
Key Changes:
------------
Major differences in Cray OpenSHMEMX 11.7.2 from
Cray OpenSHMEMX version 11.7.1 includes the following:
- Add support for using 4/8 byte puts for application-level signaling
Bugs Closed:
------------
The following bugs are fixed as part of the Cray OpenSHMEMX 11.7.2
release:
- Fix several shmem_quiet flushing issues for default and non-default
settings
- Fix the implementation of signaling sets to match OSM-1.5 semantics
Product and OS Dependencies:
----------------------------
The Cray OpenSHMEMX 11.7.2 release is supported on the following
Cray systems:
* HPE Cray EX systems with CLE
Product and OS Dependencies by network type:
--------------------------------------------------+
| HPE Slingshot |
----------------------------+---------------------+
craype | >= 2.7.3 |
----------------------------+---------------------+
cray-pals | >= 0.3.5 |
----------------------------+---------------------+
cray-pmi | >= 6.0.1 |
----------------------------+---------------------+
libfabric | >= 1.9.0 |
----------------------------+---------------------+
cray-dsmml | >= 0.2.2 |
----------------------------+---------------------+
One or more compilers:
* AOCC 2.2 or later
* CCE 9.1 or later
* GNU 9.1 or later
* Intel 19.0 or later
* Nvidia 20.7 or later
Notes and Limitations:
----------------------
Announcements, release information, supported environments, limitations,
usage models, and backward compatibility informations about this product
can be viewed in the following location:
https://pe-cray.github.io/cray-openshmemx/
Documentation:
--------------
Use the Cray OpenSHMEMX man pages for more information on the library
functions and use intro_shmem(3) man page for general information.
Initial support for Cray OpenSHMEMX man pages are derived from OpenSHMEM
standards specification document
Use https://pe-cray.github.io/whitepapers/ for access to different
Cray OpenSHMEMX specific whitepapers
Modulefile:
-----------
module load cray-openshmemx/11.7.2
License:
--------
Components, files, and programs contained within this package or
product are Copyright Hewlett Packard Enterprise Development LP
cray-stat 4.12.3
================================================================================
*Released: May 2024*
Product Description
---------------------------
- The Stack Trace Analysis Tool (STAT) is a highly scalable, lightweight
tool that gathers and merges stack traces from all of the processes of
a parallel application to form call graph prefix trees.
Notes
---------------------------
- To enable gathering stack traces on an Nvidia GPU, the environment variable
STAT_GDB must be set to either cuda-gdb, which will require cuda-gdb to be
available in PATH. Or it can be set a specific debugger. We suggest the
version of cuda-gdb packaged with gdb4hpc, as in:
export STAT_GDB=/opt/cray/pe/gdb4hpc/$GDB4HPC_VERSION/libexec/cuda110-gdb-guard
After which stat-cl would be called using the options: -w -G -i
Changelog
---------------------------
=======================================================================
## [4.12.3] - 2024-04-23 (24.05)
=======================================================================
### Features
* PE-51960: Added support for rhel89/93 & gcc13
* Update to gcc toolset 12 on redhat
* Update to gcc toolset 12 on redhat
=======================================================================
## [4.12.1] - 2023-07-11 (23.08)
=======================================================================
### Features
* Use native compilers on sles
* Use native sles compilers
=======================================================================
## [4.12.0] - 2023-04-11 (23.05)
=======================================================================
### Features
* Build with gcc-toolset-10 on redhat
OS Dependencies
---------------------------
- HPE CSM systems with SLES 15 SP4
- HPE HPCM systems with SLES 15 SP5 and RHEL 8.8
- HPE Apollo systems with RHEL 8.8
Product Dependencies
---------------------------
- cray-mrnet >= 5.1, < 6 installed
- cray-cti 2.18, < 3 installed
- cdst-support >= 2.14, < 3 installed
Documentation
---------------------------
- https://computing.llnl.gov/code/STAT/
Installation Instructions
---------------------------
- yum --assumeyes --nogpgcheck install cray-stat-4.12.3-20240605164513_d896288ae0cd-3.el8.x86_64.rpm
To make cray-stat 4.12.3 the default version of cray-stat, execute:
/opt/cray/pe/admin-pe/set_default_files/set_default_cray-stat_4.12.3
Legal Notices
---------------------------
Certain components, files or programs contained within this package or product are Copyright 2007-2024 Hewlett Packard Enterprise Development LP.
Attribution notices for open source licensed software for this package are detailed in the file:
/opt/cray/pe/stat/4.12.3/ATTRIBUTIONS_stat.txt
totalviewsup 2024.1.21
================================================================================
*Released: March 2024*
Product Description
---------------------------
- Provides the HPE Totalview support package which contains modulefiles (tcl & lua),
set_default files, and content projection files for Totalview 2024.1.21.
- The only supported install directory for HPE totalviewsup is /opt/cray/pe.
- The only supported install directory for Perforce Totalview is /opt/totalview.
- Drop the license file into: /opt/totalview/2024.1.21/
Changelog
---------------------------
=======================================================================
## [2023.4.16] - 2024-02-26 (24.03)
=======================================================================
### Features
* PE-49958: Implemented the auto-gen release notes/changelog tool
- Updated supported version to 2023.4.16
OS Dependencies
---------------------------
- HPE CSM systems with SLES 15 SP4
- HPE HPCM systems with SLES 15 SP5 and RHEL 8.8
- HPE Apollo systems with RHEL 8.8
Product Dependencies
---------------------------
- This totalviewsup 2024.1.21 package has a dependency on Totalview 2024.1.21.
Documentation
---------------------------
- Totalview docs can be found in /opt/toolworks/totalview.2024.1.21/doc/
- Totalview user guide:
/opt/toolworks/totalview.2024.1.21/doc/TotalView_User_Guide.pdf
- Totalview installation guide:
/opt/toolworks/totalview.2024.1.21/doc/TotalView_Installation_Guide.pdf
- The Totalview release notes:
/opt/toolworks/totalview.2024.1.21/doc/TotalView_Change_Log.pdf
- The Totalviewsup support package release notes can be found in:
/opt/cray/pe/totalviewsup/2024.1.21/release_notes.md
Installation Instructions
---------------------------
- rpm -ivh totalviewsup-2024.1.21-20240513190627_eed2518e7091-1.el8.x86_64.rpm
To make Totalview 2024.1.21 the default version, execute:
/opt/cray/pe/admin-pe/set_default_files/set_default_totalview_2024.1.21
Legal Notices
---------------------------
Copyright 2007-2024 Hewlett Packard Enterprise Development LP.
CM-PE-INTEGRATION CHANGELOG
===========================
24.07 (2023-06-05)
* Remove rpm queries from 77-deploy-pe.sh for improved performance
* Remove find commands from 77-deploy-pe.sh for improved performance
* Replace the systemd file with a new pe.service that depends on pe.mount
* CAST-35461 fix setting MODULEPATH in csh.cshrc.local
* Replace find commands with for loops to allow using symlink paths in gen_modulefiles.sh
24.03 (2023-02-16)
* Fix regex in gen_modulefiles for Intel OneAPI
* Add a umask 022 to ensure CPE access is not limited
* Add nfs-utils as an rpm dependencies
23.12 (2023-11-27)
* Determine the craype-target architecture during shell initialization
* Add support for compute nodes running on aarch64 hardware
* Update gen_modulefiles.sh to better support new compiler versions
* Add better logging information to `77-deploy-pe`
23.05 (2023-05-10)
* Allow sites to override the lmod install path by setting `lmod_prefix_path`
in `cray-pe-configuration.sh` and `cray-pe-configuration.csh`. Sites can
now use newer versions of lmod provided by the OS distribution.
* Update `76-add-pe-to-fstab` so that `/pe` is not added to `/etc/fstab` more
then once.
23.03 (2023-03-03)
* Spread the squashfs image mounts across NUMA domains to balance memory
usage across NUMA nodes.
* Add RPM dependency on `numactl`
23.02 (2023-01-30)
* Update shell profile scripts to initialize lmod using HPE provided package.
* Remove leading and trailing `:` in `LMOD_SYSTEM_DEFAULT_MODULES` that is no
longer an acceptable format in new versions of lmod.
22.12 (2022-12-12)
* Add a `site_module_list` to `cray-pe-configuration.sh` and
`cray-pe-configuration.csh` to allow sites to add modules loaded at login
without the need to modify HPE defaults `init_module_list`.
22.11 (2022-11-10)
* Update `gen_modulefiles.sh` to recreating symlinks to ensure idempotence.
22.10 (2022-10-03)
* Changes for HPCM 1.8 to ensure that `/pe` is mounted in `76-add-pe-to-fstab`
before running `77-deploy-pe`
22.09 (2022-08-23)
* Update `pe_overlay.sh` to only remove workdirs when unmounts are successful.
* Update `gen_modulefiles.sh` to run `one_off_set_defaults` after generating
all modulefiles.
22.08 (2022-07-22)
* Enable older PE images to deploy on system with newer minor base OS version.
* Update `gen_modulefiles.sh` to use the latest available version of
`craypkg-gen`.
22.05 (2022-04-26)
* Remove `/opt/modulefiles` from `MODULEPATH`.
22.04 (2022-03-28)
* Add a temporary workaround in `pe_overlay.sh` that copies lmod GPU
modulefiles into the PE overlay's upper layer so they are visible after
`/opt/cray/pe` is mounted.
valgrind4hpc 2.13.3
================================================================================
*Released: May 2024*
Product Description
---------------------------
- Valgrind4hpc is a Valgrind-based debugging tool to aid in the detection of memory
leaks and errors in parallel applications. Valgrind4hpc aggregates any duplicate
messages across ranks to help provide an understandable picture of
program behavior. Valgrind4hpc manages starting and redirecting output from many
copies of Valgrind, as well as deduplicating and filtering Valgrind messages.
If your program can be debugged with Valgrind, it can be debugged with Valgrind4hpc.
Changelog
---------------------------
=======================================================================
## [2.13.3] - 2024-04-23 (24.05)
=======================================================================
### Bug Fixes
* PE-51571: Parse arguments before starting network
### Features
* PE-51960: Added support for rhel89/93 & gcc13
* Update Valgrind to 3.22, support syscall userfaultfd
* Build with gcc toolset 12 on redhat
=======================================================================
## [2.13.1] - 2023-07-7 (23.08)
=======================================================================
### Features
* PE-47184: Add valgrind4hpc user guide
* Use native compilers on sles
=======================================================================
## [2.13.0] - 2023-04-11 (23.05)
=======================================================================
### Bug Fixes
* PE-45205: Fix --from-ranks filtering and suppression generation
* Fix install path exported by lmod modulefile on relocated install
### Features
* Build with gcc-toolset-10 on redhat
* PE-45055: Add GDB4hpc connection functionality for VGDB mode
OS Dependencies
---------------------------
- HPE CSM systems with SLES 15 SP4
- HPE HPCM systems with SLES 15 SP5 and RHEL 8.8
- HPE Apollo systems with RHEL 8.8
Product Dependencies
---------------------------
- cray-mrnet >= 5.1, < 6 installed
- cray-cti >= 2.18, < 3 installed
- cdst-support >= 2.14, < 3 installed
Documentation
---------------------------
- See man valgrind4hpc.
Installation Instructions
---------------------------
- rpm -ivh valgrind4hpc-2.13.3-20240508222518_ddb177766849-1.el8.x86_64.rpm
To make valgrind4hpc 2.13.3 the default version of valgrind4hpc, execute:
/opt/cray/pe/admin-pe/set_default_files/set_default_valgrind4hpc_2.13.3
Legal Notices
---------------------------
Certain components, files or programs contained within this package or product are Copyright 2010-2024 Hewlett Packard Enterprise Development LP.
Attribution notices for open source licensed software for this package are detailed in the file:
/opt/cray/pe/valgrind4hpc/2.13.3/ATTRIBUTIONS_valgrind4hpc.txt
CPE Release Notes For RHEL 9.4
HPE Cray Programming Environment 24.07 Full Release Notes for RHEL 9.4
=========================================================
atp 3.15.4
================================================================================
*Released: June 2024*
Product Description
---------------------------
- Abnormal Termination Processing (ATP) is a system that
monitors user applications, and should an application
take a system trap, ATP preforms analysis on the dying
application. All of the stack backtraces of the
application processes are gathered into a merged
stack backtrace tree and written to disk as the file
"atpMergedBT.dot". The stack backtrace for the first process
to die is sent to stderr as is the number of the signal that
caused the death. If the core file size limit (RLIMIT_CORE)
is non-zero, a heuristically selected set of processes dump
their core.
atpMergedBT.dot can be viewed with 'statview', a component
of the STAT package (module load cray-stat). The merged stack
backtrace tree provides a concise, yet comprehensive, view
of what the application was doing at the time of its death.
Changelog
---------------------------
=======================================================================
## [3.15.4] - 2024-05-2 (24.06)
=======================================================================
### Bug Fixes
* PE-53603: ZMQNet libsodium support
* PE-52560: Pass all ATP environment variables to Flux plugin
* PE-51520: Remove disable message from Flux plugin
* PE-52423: Disable PALS local launch in modulefile
### Features
* PE-53605: Build and package libsodium
* PE-51960: Added support for rhel89/93 & gcc13
* PE-51520: Automatic plugin loading for Flux
=======================================================================
## [3.15.2] - 2023-10-4 (23.11)
=======================================================================
### Bug Fixes
* PE-50510: Set CCE 16 Fortran line number workaround as optional
* PE-49776: Fall back to /tmp if TMPDIR or HOME is invalid
* PE-49611: Update ZMQNet submodule
### Features
* Move to gcc toolset 12 on redhat
* PE-49434: Add function and file to stack trace summary
=======================================================================
## [3.15.1] - 2023-07-19 (23.08)
=======================================================================
### Bug Fixes
* PE-47129: Work around CCE 16 Fortran source file info bug
* PE-47436: Reset Lmod module load variable after use
### Features
* PE-48781: Automatic MRNet load not required due to rpath update
* Use native compilers on sles
* PE-47183: Use RTD style for HTML docs
* PE-47182: Provide web documentation at `share/doc`
OS Dependencies
---------------------------
- HPE CSM systems with SLES 15 SP4
- HPE HPCM systems with SLES 15 SP5 and RHEL 8.8
- HPE Apollo systems with RHEL 8.8
Product Dependencies
---------------------------
- cray-mrnet >= 5.1, < 6 installed
- cray-cti >= 2.18, < 3 installed
- cdst-support >= 2.14, < 3 installed
Documentation
---------------------------
- Type `man atp` with the atp module loaded to read the atp(1) man page.
Installation Instructions
---------------------------
- rpm -ivh atp-3.15.4-20240508172747_4766b5970bed-1.el9.x86_64.rpm
To make atp 3.15.4 the default version of atp, execute:
/opt/cray/pe/admin-pe/set_default_files/set_default_atp_3.15.4
Legal Notices
---------------------------
Certain components, files or programs contained within this package or product are Copyright 2010-2024 Hewlett Packard Enterprise Development LP.
Attribution notices for open source licensed software for this package are detailed in the file:
/opt/cray/pe/atp/3.15.4/ATTRIBUTIONS_atp.txt
cray-ccdb 5.0.4
================================================================================
*Released: May 2024*
Product Description
---------------------------
- CCDB, Cray's next generation debugging tool extends the comparative
debugging capabilities of gdb4hpc with a Graphical User Interface (GUI)
enabling programmers to compare corresponding data structures
between two executing applications. Comparative debugging assists users with
locating sections of code containing data deviations introduced by algorithm
changes, compiler differences, and porting to new architectures/libraries.
Changelog
---------------------------
=======================================================================
## [5.0.4] - 2024-04-22 (24.05)
=======================================================================
### Bug Fixes
* PE-50334: Fix missing console messages
### Features
* PE-51960: Added support for rhel89/93 & gcc13
=======================================================================
## [5.0.3] - 2024-02-7 (24.03)
=======================================================================
### Bug Fixes
* PE-51115: Make socket directory, add PBS allocation instructions
=======================================================================
## [5.0.2] - 2023-11-1 (23.12)
=======================================================================
### Features
* Use gcc toolset 12 on redhat
* Use gcc toolset 12 on redhat
OS Dependencies
---------------------------
- HPE CSM systems with SLES 15 SP4
- HPE HPCM systems with SLES 15 SP5 and RHEL 8.8
- HPE Apollo systems with RHEL 8.8
Product Dependencies
---------------------------
- gdb4hpc/4.14.x or later installed
Documentation
---------------------------
- Type `man ccdb` with the cray-ccdb module loaded to read the ccdb(1) man page.
- Documentation is available through CCDB from the 'Help' menu.
Installation Instructions
---------------------------
- rpm -ivh --oldpackage cray-ccdb-5.0.4-20240513224859_723809ac94e4-2.el9.x86_64.rpm
To make cray-ccdb 5.0.4 the default version of cray-ccdb, execute:
/opt/cray/pe/admin-pe/set_default_files/set_default_cray-ccdb_5.0.4
Legal Notices
---------------------------
Certain components, files or programs contained within this package or product are Copyright 2007-2024 Hewlett Packard Enterprise Development LP.
Attribution notices for open source licensed software for this package are detailed in the file:
/opt/cray/pe/ccdb/5.0.4/ATTRIBUTIONS_ccdb.txt
HPE Cray Compiling Environment (CCE) 18.0.0
===========================================
Release Date:
-------------
July 2024
Purpose:
--------
CCE 18.0.0 provides Fortran, C, and C++ compilers for
HPE Cray supercomputer and HPE Apollo 2000 Gen10Plus systems.
Key Changes and Support with CCE 18.0.0:
----------------------------------------
CCE 18.0.0:
-----------
- LLVM 18 base (Merges up to May 18th 2024. LLVM version 18.1.6)
- Split CCE into subpackages
- Updated OMP_PLACE CPU affinity mapping policy to use absolute indices instead of relative
- New "-fopenmp-force-usm" compiler flag (C/C++/Fortran) enables OpenMP unified shared memory (USM) by treating each source file
as though it contains an explicit "omp requires unified_shared_memory" directive
- Added new "compiler_interop" man page documenting CCE compiler interoperability
- Added the following OpenMP 5.1/5.2 Features:
- interop construct and API routines
- compare and fail clause for atomic construct
- present modifier and defaultmap(present)
- nowait clause for taskwait directive
- default(private) and default(firstprivate) (C/C++)
- omp_target_memcpy_async and omp_target_memcpy_rect_async
- nothing directive (C/C++)
- Added the following OpenACC features:
- OpenACC 2.5
- Automatic allocate/deallocate for device_resident/create allocatables/pointers
- Support acc_memcpy_device routine
- Support num_gangs, num_workers and vector_length on kernels directive
- Support async copy routines
- Handle scoping for lexically visible data regions
- OpenACC 2.7
- Snap subarray/subscript expressions for use at the end of data regions
- Reduction on a compute construct implies a copy clause
- OpenACC 3.1
- Implement OpenACC serial
- Added the following Fortran 2023 features
- Control over leading zeros in output of real values: LEADING_ZERO= specifier
- Auto-allocation of character variables in more places - statements
- TYPEOF and CLASSOF declarations
- More use of BOZ constants
- Partial implementation of conditional expressions and arguments
- ENUMERATION TYPE
- SIMPLE attribute for procedures
- RHEL 8.10 x86_64 support
- RHEL 9.4 x86_64 support
- RHEL 9.4 aarch64 support
- OpenMP floating-point atomic optimization hints for AMD MI250X GPUs (Fortran)
- Advanced AMD GPU DWARF support
- Fortran scalar variables and fixed-size arrays in offload regions
- Fortran common blocks
- Optimized NVIDIA Grace CPU Superchip support
User and Application Impact:
----------------------------
CCE 18.0.0:
-----------
Issues or Bugs Resolved:
------------------------
CCE 18.0.0:
-----------
CAST-24911 Incorrect result form ASSOCIATED
CAST-26951 RFE: Need CCE OpenMP OMP_PLACES affinity to be able to map to taskset -c
CAST-30200 ftn compiler error module pointer omp target teams distribute parallel do
CAST-30667 [ELCAP-121] ftn compiler error module pointer omp target teams distribute parallel do
CAST-32453 Cray Fortran compiler is generating wrong assembly with OpenACC
CAST-32454 CCE runtime issue related to explicit type conversion of pointer variables with OpenACC.
CAST-32456 (ELCAP-318) Fortran compiler error in omp declare target subroutine - Elevation of case 5372036627
CAST-32588 Cray Compiler Bug : Issue with polymorphic and character type.
CAST-33337 XC50-CLE-Cray Fortran Compiler- Assignment (=) is not overridden
CAST-34158 crayftn Overloaded Constructor With Polymorphic Argument in Array Constructor
CAST-34265 (ELCAP-466) paradyn target offload built with -O0 gets runtime ACC error
CAST-34571 Internal compiler error with OpenACC declare
CAST-34664 (ELCAP-493) Fortran compiler hangs/crashes on !$omp continuation statements.
CAST-34817 (ELCAP-506) ftn omp target offload runtime error with -O0
CAST-34819 HPECOE-34 CCE Fortran OpenMP offload Bug with Procedure as Argument in Declare Target is unsupported
CAST-34871 (ELCAP-510) CCE17 Fortran crashes
CAST-34910 Incorrect code with floating point optimizations in Cray Fortran
CAST-35003 (ELCAP-522) Minikin - ftn unified memory wrong result for module save variable and declare target
CAST-35004 (ELCAP-527) Minikin runtime error with CCE 17 ftn
CAST-35054 (ELCAP-513) Fortran OMP declare target in module causes runtime error with ROCm-5.7.0+
CAST-35193 Cray Compiler error 'ftn-7060' with debug build of OpenACC code
CAST-35240 Cray EX - several compilation errors compiling vasp with cce17.0.0 [5379756819 Elevation]
CAST-35257 (ELCAP-533) Fortran seg fault with nested module subroutines
CAST-35401 Cray Openmp implementation and buggy thread affinity
CAST-35512 [LUMI] ftn -hfunc_trace crashes the compiler when END statement is not named for modules
CAST-35571 Bug with compiler cce/17.0.0
CAST-35625 No source line number displayed when run with MemorySanitizer in PrgEnv-cray
CAST-35726 ftn-7206 warning in cce/17.0
CAST-35776 crayftn ICE when trying to build dftd4
CAST-35903 (ELCAP-596) Fortran compiler segfault with OpenMP and requires statement
CAST-35946 (ELCAP-610) unified memory runtime error using ftn allocatable array from:module
CAST-35963 (ELCAP-615) Paradyn ftn openmp problem: bad size of allocated pointer with omp requires unified_shared_memory
Product and OS Dependencies:
----------------------------
This CCE release is supported on
- HPE Cray supercomputer systems running CSM with COS 2.5.X (SLES15 SP4) or COS 23.11 (SLES15 SP5)
- HPE Cray supercomputer systems running HPCM with COS 2.5.X (SLES15 SP4) or COS 23.11 (SLES15 SP5)
- HPE Cray supercomputer systems running HPCM with RHEL 8.10
- HPE Apollo 2000 Gen10Plus systems running RHEL 8.9
- AMD GPU offloading support requires at least ROCm 6.0 and is tested up to ROCm 6.1
Notes and Limitations:
----------------------
Documentation:
--------------
- S-5212 Cray Compiling Environment Release Overview (18.0)
- S-3901 Cray Fortran Reference Manual
- S-2179 Cray C and C++ Quick Reference
- Basic man pages: crayftn(1), craycc(1), crayCC(1), intro_openmp(7), intro_openacc(7)
- Please see https://clang.llvm.org/docs/UsersManual.html or use the -help command line option for
more information on using Clang
Modulefile:
-----------
The following will load the modules necessary to use CCE:
module load PrgEnv-cray
The following will switch to x.y.z version of CCE:
module swap cce cce/x.y.z
Installation instructions:
--------------------------
rpm -ivh cce-18.0.0-202406031331.98280ea519a0c-4.el9.x86_64.rpm.x86_64.rpm
The following script will set CCE version 18.0.0 default:
/opt/cray/pe/admin-pe/set_default_files/set_default_cce_18.0.0
License:
--------
Except for the third party components and software licensed by HPE through
proprietary agreements, components, files or programs contained within this
package or product are Copyright -2024 Hewlett Packard Enterprise Development LP.
Attribution notices for open source licensed software for this
package are detailed in the file:
/opt/cray/pe/cce/18.0.0/ATTRIBUTIONS_18.0.txt
cpe-descriptive-manifest 24.07
=================================
Release Date: Jul 2024
Product description:
--------------------
The cpe-descriptive-manifest packages contain the JSON manifest files that
are to map CPE to a format that Spack can consume.
Schema Changelog:
-----------------
v1.4
----
* Add 'wrappers' as a boolean parameter
v1.3
----
* Change '_meta' opject's 'version' property to 'schema-version'.
* Add 'cpe-version' property '_meta' object.
v1.2
----
* Add 'system-type' property to the '_meta' object.
v1.1
----
* Add '_meta' object to define file type and schema version.
v1.0
----
* Initial schema.
CrayPE Targets 1.13.2
================================================================================
Release Date:
--------------------------------------------------------------------------------
July, 2024
Purpose:
--------------------------------------------------------------------------------
Adds modulefiles for targets - craype-accel-amd-gfx942
Bugs fixed in this release:
--------------------------------------------------------------------------------
Updates dependencies for NVIDIA accelerators
Dependencies:
--------------------------------------------------------------------------------
The CrayPE Targets 1.13.2 release is dependent on the following
software products:
CrayPE
set_default_3
lmod_scripts
Documentation:
--------------------------------------------------------------------------------
See manpages for cc, CC, ftn, intro_craype-api and pkg-config
See section 2.6 Using Targeting Modules of the Cray Programming Environment
User's Guide (S-2529-116)
See http://www.freedesktop.org/wiki/Software/pkg-config for a pkg-config
introduction.
Installation instructions:
--------------------------------------------------------------------------------
rpm -Uvh craype-targets-ex-hpcm-1.13.2-202406062350.172a8f0148d89-0.ex.hpcm.noarch.rpm
To make this the default version, execute:
/opt/cray/pe/admin-pe/set_default_files/set_default_craype-targets_1.13.2-tcl
/opt/cray/pe/admin-pe/set_default_files/set_default_craype-targets_1.13.2-lua
Product description:
--------------------------------------------------------------------------------
CrayPE Targets contains the modules for CrayPE targets.
TCL Modules - craype-accel-host craype-accel-amd-gfx90a craype-accel-amd-gfx908 craype-accel-amd-gfx940 craype-accel-amd-gfx942 craype-accel-intel-max craype-accel-nvidia70 craype-accel-nvidia80 craype-accel-nvidia90 craype-hugepages2M craype-hugepages4M craype-hugepages8M craype-hugepages16M craype-hugepages32M craype-hugepages64M craype-hugepages128M craype-hugepages256M craype-hugepages512M craype-hugepages1G craype-hugepages2G craype-arm-grace craype-x86-milan craype-x86-milan-x craype-x86-rome craype-x86-spr craype-x86-spr-hbm craype-x86-trento craype-x86-genoa craype-network-none craype-network-ofi craype-network-ucx
Lmod Modules - craype-accel-host craype-accel-amd-gfx90a craype-accel-amd-gfx908 craype-accel-amd-gfx940 craype-accel-amd-gfx942 craype-accel-intel-max craype-accel-nvidia70 craype-accel-nvidia80 craype-accel-nvidia90 craype-hugepages2M craype-hugepages4M craype-hugepages8M craype-hugepages16M craype-hugepages32M craype-hugepages64M craype-hugepages128M craype-hugepages256M craype-hugepages512M craype-hugepages1G craype-hugepages2G craype-arm-grace craype-x86-milan craype-x86-milan-x craype-x86-rome craype-x86-spr craype-x86-spr-hbm craype-x86-trento craype-x86-genoa craype-network-none craype-network-ofi craype-network-ucx
Copyright 2013-2023 Hewlett Packard Enterprise Development LP
CDEUX driver (craype) 2.7.32 - 3
==============
Release Date:
--------------
July 2024
Purpose:
--------
The Cray Development Environment User eXperience driver (CDEUX) (craype)
provides a common set of drivers for all CPE programming environments,
notably the C/C++ (cc/CC) and Fortran (ftn) languages.
New CDEUX driver (craype) features for HPE Cray EX and Apollo systems:
- Support for newer Red Hat Enterprise Linux versions
Key Changes and Bugs Closed:
----------------------------
- Fix loading a perftools module breaks "direct/moduleless" craycc
- Fix cc fails to create shared library in PrgEnv-gnu-amd
- Add missing documentation about RPATHs
Product and OS Dependencies:
----------------------------
The CDEUX driver (craype) 2.7.32 release is supported on the following
HPE systems:
* HPE Cray EX systems with CLE
* HPE Apollo systems as part of the Cray Programming Environment
Documentation:
---------------
See manpages for cc, CC, ftn, and intro_craype-api.
See the pkg-config manpage or
http://www.freedesktop.org/wiki/Software/pkg-config for a pkg-config
introduction.
Product description:
--------------------
CrayPE contains drivers, cc, CC, and ftn to compile for all CPE
Programming Environments.
Certain components, files or programs contained within this package or product
are Copyright 2013-2023 Hewlett Packard Enterprise Development LP.
License:
--------
Except for the third party components and software licensed by HPE
through proprietary agreements, components, files or programs contained
within this package or product are Copyright 2023 Hewlett Packard
Enterprise Development LP.
Craypkg-gen 1.3.33
==============
Release date
------------
June 2024
Purpose
-------
Bug fix release.
Bugs fixed in this release
--------------------------
- Add CUDA modulefiles and pkg-config file support
- Update the NVIDIA HPC SDK modulefiles to more closely resemble the AMD toolkit modules
Dependencies
------------
The craypkg-gen 1.3.33 release is supported on the following HPE
Cray systems:
- Cray EX systems SLE 15.0 or later and RHEL 8.0 or later.
Driver support for integrating Third Party C, C++, and Fortran libraries
through .pc files using pkg-config is used in CrayPE 2.x and later.
Limitation
----------
- Library dependencies for static libraries are not added to the .pc files
for keywords Requires.private and Libs.private. A warning is issued by
the craypkg-gen tool to advise users to add this information to the
libraries .pc files.
- RPM limits packages to 4GB
Documentation
-------------
Man pages for craypkg-gen are found by executing `module load craypkg-gen`
and then `man craypkg-gen`. See
http://www.freedesktop.org/wiki/Software/pkg-config for a pkg-config
introduction.
Examples for creating modulefiles for Intel, PGI and Python are included in
the craypkg-gen ‘doc’ directory:
/opt/cray/craypkg-gen/1.3.33/doc/intel_example.txt
/opt/cray/craypkg-gen/1.3.33/doc/pgi_example.txt
/opt/cray/craypkg-gen/1.3.33/doc/python_example.txt
Example:
- As an example, the Intel 16.0.3.210 compiler was recently released.
After installing the compiler the administrator creates a modulefile for
this release by executing the following commands:
# module load craypkg-gen
# craypkg-gen -m /opt/intel/compilers_and_libraries_2016.3.210
This version of the Intel compiler is made default by executing the
command:
# /opt/admin-pe/set_default_craypkg/set_default_intel_16.0.3.210
Installation instructions
-------------------------
rpm -ivh craypkg-gen-1.3.33-1.3.33-1.el9.noarch.rpm
To change the product version to default after installation:
/opt/cray/pe/admin-pe/set_default_files/set_default_craypkg-gen_1.3.33
Product description
-------------------
The craypkg-gen 1.3.33 utility provides the system administrator
a tool to integrate third party software with the Cray software stack.
Craypkg-gen assists with integration by creating .pc files for C, C++, and
Fortran libraries, pkg-config enabled modulefiles and RPMs.
The workflow for using craypkg-gen is
1) Build the open source software
2) Create .pc files for libraries
3) Create pkg-config enabled modulefiles
4) Customize pkg-config and modulefiles if needed
Certain components, files or programs contained within this package or product
are Copyright 2013-2024 Hewlett Packard Enterprise Development LP.
cray-cti 2.18.4
================================================================================
*Released: June 2024*
Product Description
---------------------------
- Cray Common Tools Interface (CTI) is a library that abstracts debugger tool support
to a common interface regardless of the underlying work load manager. It facilitates
the staging of files and the launch of tool daemons on the compute nodes associated
with a parallel job. Options and interfaces can be found in the cti(1) and cti(3)
manpages.
Known Issues
---------------------------
- Race condition workaround for PALS 1.2.3 in CTI 2.17.0 (22.11). In PALS 1.2.3, there
is a race condition between the tool launcher releasing a job from the startup barrier
and the job actually getting to the startup barrier. This can result in the job receiving
the startup barrier release signal before it actually arrives there, resulting in the job
getting stuck in the barrier. As a workaround, the environment variable
CTI_PALS_BARRIER_RELEASE_DELAY can be set to add a delay between job startup and barrier
release. If set to a positve integer n, CTI will wait n seconds between starting a
job and releasing it from the barrier on PALS. A delay as small as one second works in
most cases.
Notes
---------------------------
- CTI 2.17.1 (PE 22.12) contains a new feature, file deduplication, for Slurm and PALS
workload managers. If enabled, CTI will run a check on compute nodes to determine if files
can be excluded from the shipped package. This can significantly improve application startup
time. To enable, set the environment variable CTI_DEDUPLICATE_FILES=1.
Changelog
---------------------------
=======================================================================
## [unreleased]
=======================================================================
### Bug Fixes
* PE-54265: Don't add quotes to LD_PRELOAD
* PE-54079: Fix unreliable Flux application output
=======================================================================
## [2.18.4] - 2024-05-3 (24.06)
=======================================================================
### Bug Fixes
* PE-53554: Allow launch if debug log directory doesn't exist
### Features
* Add lock to C interface, basic multithread test
* PE-53555: Parse multiple Flux job ID formats
* PE-52995: Support Flux 0.60.0
* PE-51960: Added support for rhel89/93 & gcc13
=======================================================================
## [2.18.3] - 2024-02-15 (24.03)
=======================================================================
### Bug Fixes
* PE-52423: Fix deduplication check for PALS
* PE-52423: Disable local launch for PALS
* PE-52540: Always include rank 0 in Flux launch
* PE-51694: Flux: fix job layout case and file deduplication
* PE-51955: Only rebuild checksum if input files changed
* PE-51495: Fix Slurm isRunning for stopped srun and backgrounded sattach
* PE-51694: Fix Flux implementation for 0.57.0
* PE-51193: Support rank remapping in PALS backend
### Features
* PE-48498: Accept PALS attach in form pbs_job_id:pals_apid
OS Dependencies
---------------------------
- HPE CSM systems with SLES 15 SP4
- HPE HPCM systems with SLES 15 SP5 and RHEL 8.8
- HPE Apollo systems with RHEL 8.8
Product Dependencies
---------------------------
- cray-cdst-support >= 2.14, < 3 installed
- cray-dyninst 12.3.0 or later installed
Documentation
---------------------------
- Currently the only documentation available is found in common_tools_fe.h, common_tools_be.h,
and the tests in the 'tests' directory.
Installation Instructions
---------------------------
- rpm -ivh cray-cti-2.18.4-20240523151413_b5630478cfc3-5.el9.x86_64.rpm
To make cray-cti 2.18.4 the default version of cray-cti, execute:
/opt/cray/admin-pe/set_default_files/set_default_cray-cti-2.18.4
Legal Notices
---------------------------
Certain components, files or programs contained within this package or product are Copyright 2010-2024 Hewlett Packard Enterprise Development LP. SPDX-License-Identifier: Linux-OpenIB
Attribution notices for open source licensed software for this package are detailed in the file:
/opt/cray/pe/cti/2.18.4/ATTRIBUTIONS_cti.txt
Cray DSMML 0.3.0:
==========================
Release Date:
-------------
March, 2024
Product Information:
--------------------
DSMML is a stand-alone memory management library for maintaining distributed
shared symmetric memory heaps for top level PGAS languages and libraries like
Coarray Fortran, UPC, and OpenSHMEM. DSMML allows user libraries to create
multiple symmetric heaps and share information with other libraries. Through
DSMML, we could extract interoperability between PGAS programming models.
Announcements, release informations, supported environments, and backward
compatibility informations about this product can be viewed in the following
location: https://pe-cray.github.io/cray-dsmml/
Purpose:
--------
Cray DSMML 0.3.0 supports the following features:
- Create symmetric heap segments for being used by PGAS languages and
libraries
- Allows options to allocate, deallocate memory buffers from previously
created symmetric heap segments
- Enables sharing information about the symmetric heap segments between
different programming models
- Added support for shared symmetric heap (SSHEAP) that can be used to
provide enhanced SMP data transport in user-libraries like SHMEM, CAF, UPC
The following features are fixed in Cray DSMML 0.3.0 when compared
to Cray DSMML 0.2.2:
- Add support for the library for AARCH64 processor based platforms
- Internal bug fixes involving hugepage cleanups
Refer intro_dsmml(3) manpage for more information on the supported features
and syntax for different routines
Product and OS Dependencies:
----------------------------
The Cray DSMML 0.3.0 release is supported on the following
Cray systems:
* HPE Cray EX systems with CLE
Product and OS Dependencies by network type:
--------------------------------------------------+
| Shasta |
----------------------------+---------------------+
craype | >= 2.7.3 |
----------------------------+---------------------+
One or more compilers:
* AOCC 2.2 or later
* CCE 9.1 or later
* GNU 9.1 or later
* Intel 19.0 or later
* Nvidia 20.7 or later
Documentation:
--------------
Use the Cray DSMML man pages for more information on the library
functions and use intro_dsmml(3) man page for general information.
Use https://pe-cray.github.io/whitepapers/ for access to different
Cray DSMML specific whitepapers
Announcements, release informations, supported environments, and backward
compatibility informations about this product can be viewed in the following
location: https://pe-cray.github.io/cray-dsmml/
Modulefile:
-----------
module load cray-dsmml/0.3.0
Certain components, files or programs contained within this package or
product have the following Copyright:
Copyright 2018-2021 Hewlett Packard Enterprise Development LP.
cray-dyninst 12.3.2
================================================================================
*Released: May 2024*
Product Description
---------------------------
- cray-dyninst provides tools for binary instrumentation, analysis, and modification.
Changelog
---------------------------
=======================================================================
## [12.3.2] - 2024-04-22 (24.05)
=======================================================================
### Features
* PE-51960: Added support for rhel89/93 & gcc13
=======================================================================
## [12.3.1] - 2023-09-25 (23.10)
=======================================================================
### Features
* Use gcc toolset 12 on redhat
=======================================================================
## [12.3.0] - 2023-06-27 (23.07)
=======================================================================
### Features
* PE-48354: Add LD_LIBRARY_PATH to Modulefiles
* PE-48354: Update Dyninst to 12.3.0
* Use native compilers on sles
OS Dependencies
---------------------------
- HPE CSM systems with SLES 15 SP4
- HPE HPCM systems with SLES 15 SP5 and RHEL 8.8
- HPE Apollo systems with RHEL 8.8
Product Dependencies
---------------------------
- cdst-support >= 2.14, < 3 installed
Documentation
---------------------------
- https://github.com/dyninst/dyninst
Installation Instructions
---------------------------
- rpm -ivh cray-dyninst-12.3.2-20240503230421_8668bb4a546f-1.el9.x86_64.rpm
To make this the default version of cray-dyninst, execute:
/opt/cray/pe/admin-pe/set_default_files/set_default_cray-dyninst_12.3.2
Legal Notices
---------------------------
Certain components, files or programs contained within this package or product are Copyright 2007-2024 Hewlett Packard Enterprise Development LP.
Attribution notices for open source licensed software for this package are detailed in the file:
/opt/cray/pe/dyninst/12.3.2/ATTRIBUTIONS_dyninst.txt
FFTW 3.3.10.8
============
Release Date:
-------------
July 2024
Purpose:
--------
This Cray FFTW 3.3.10.8 release is supported on HPE/Cray EX and
HPE/Cray XD/Apollo systems. Cray FFTW is supported on the host CPU
but not on the accelerator of HPE/Cray systems.
The Cray FFTW 3.3.10.8 release provides the following:
- Compiler and dependency updates
See the Product and OS Dependencies section for details.
Product and OS Dependencies:
----------------------------
This Cray FFTW 3.3.10.8 release is supported on these HPE/Cray systems:
- HPE/Cray EX Systems with SLES 15 SP5, RHEL 8.10, or RHEL 9.4 based OS
- HPE/Cray XD/Apollo Systems with RHEL 8.10 or RHEL 9.4 based OS
This Cray FFTW 3.3.10.8 release requires the following software products:
craype 2.7.32 or later
cray-mpich 8.1.30 or later
One or more of the following major compiler versions:
CCE 18
GCC 13
AOCC 4
Nvidia 24.3 or later
AMD (ROCm) 6
Intel 2024 or later
Notes and Limitations:
----------------------
Starting with cray-fftw/3.3.6.1 the fftw module has been renamed from fftw
to cray-fftw.
Documentation:
--------------
http://www.fftw.org/index.html#documentation
See the intro_fftw3 manual page for additional information.
Modulefile:
-----------
module load cray-fftw
Installation
------------
To install the rpm, execute:
rpm -ivh cray-fftw-3.3.10.8-202406131706.ee3ebe23b54c9-1.el9.x86_64.rpm
To make this the default version, execute:
/opt/cray/pe/admin-pe/set_default_files/set_default_fftw_3.3.10.8
Certain components, files or programs contained within this package or
product are Copyright 2011-2024 Hewlett Packard Enterprise Development LP
gdb4hpc 4.16.2
================================================================================
*Released: June 2024*
Product Description
---------------------------
- gdb4hpc is a GDB-based parallel debugger used to debug applications compiled with
CCE, AOCC, GNU, and Intel Fortran, C and C++ compilers. It allows programmers to
either launch an application or attach to an already running application that
was launched on the system. Additionally, it provides comparative debugging
technology that enables programmers to compare data structures between two
executing applications. Comparative debugging should be used in conjunction
with the CCDB GUI tool accessed by loading the cray-ccdb module.
- Some features of gdb include:
* Command line parallel debugger allows for launching/attaching applications.
* Utilizes process sets to operate on a subset of application ranks.
* GDB-like feel, also implements a gdbmode to enable a true parallel gdb.
* GPU debugging is supported for OpenMP 4.0+ GPU directives; OpenMP 5.0 on AMD
and Nvidia GPU's.
* Workload manager support via Common Tools Interface (cray-cti).
Changelog
---------------------------
=======================================================================
## [4.16.2] - 2024-05-2 (24.06)
=======================================================================
### Bug Fixes
* PE-52737: Fix backend crash when debugging Intel OneAPI GPUs
### Features
* PE-47910: Run job cleanup in background
* PE-53716: Added user guide for using the Python gdb extension
* PE-51960: Added support for rhel89/93 & gcc13
* PE-53828: Implement Flux attach
* Automatically load python extensions when debugger starts
* PE-53100: Support for aggregated gdb python extension commands py-print and py-locals if underlying gdb has python extensions loaded
* PE-53100: Support for aggregated gdb python extension commands py-bt, py-list, py-up, and py-down if underlying gdb has python extensions loaded
* Handle "~" in the --debug launch argument
* PE-52589: Tweak default assertion script behavior - now assertion scripts stop iff an assertion fails default
* PE-51936: Added tab completion for the print command
=======================================================================
## [4.16.1] - 2024-02-7 (24.03)
=======================================================================
### Bug Fixes
* PE-52387: Fix segfault from list command
* PE-51694: Fix occasional crash when operating on a subset of ranks
* PE-44112: Track job status while running for quicker timeout and exit
* Fix recursive source command behavior
* Properly format /csv output with data over 1 million items
* PE-51165: Query ZMQNet for correct rank and target PID
### Features
* PE-51935: Tab completion with file names when setting breakpoints
* PE-51935: Tab completion with function names when setting breakpoints
* PE-50319: Improve feedback when switching command modes
* PE-51749: Tab completion for help, set, show commands
* PE-51752: Moved vscode plugin to gdb4hpc and packaged executable
* PE-51598: Add save all command
* PE-51254: Save and restore assertion scripts and decompositions
* PE-51253: Save and restore breakpoints
* Update included hpcc_demo
* Add set print value-caching
* Save decomposition values as convenience variables
=======================================================================
## [4.16.0] - 2023-11-7 (23.12)
=======================================================================
### Bug Fixes
* PE-50810: Support storing chunked values in client side variables
* Fix failure with p 1+1+1
* Fix formatting when printing very large arrays and vectors
* PE-50510: Set CCE 16 Fortran line number workaround as optional
* Crash where gdb cannot list a line
* PE-49611: Update ZMQNet submodule
### Features
* PE-50840: Update ZMQNet submodule
* PE-50503: Add set variable command
* PE-50619: Save values on the gdb4hpc client as convenience variables with $foo = my_variable syntax
* PE-50671: Support scoped assignment in print statements
* Support --debug=<dir> in launch and attach
* Use dots for progress when connecting
* PE-49938: Support Slurm MPMD launch
* PE-46238: Send signals other than SIGKILL to application
* Move to gcc toolset 12 on redhat
OS Dependencies
---------------------------
- HPE CSM systems with SLES 15 SP4
- HPE HPCM systems with SLES 15 SP5 and RHEL 8.8
- HPE Apollo systems with RHEL 8.8
Product Dependencies
---------------------------
- cray-mrnet >= 5.1, < 6 installed
- cray-cti 2.18, < 3 installed
- cdst-support >= 2.14, < 3 installed
Documentation
---------------------------
- Type `man gdb4hpc` with the gdb4hpc module loaded to read the gdb4hpc(1) man page.
Simple usage examples are provided at the end.
Also, type `help` at the gdb4hpc command line for more information on commands.
Installation Instructions
---------------------------
- rpm -ivh gdb4hpc-4.16.2-20240509035636_70d6b1455a0e-1.el9.x86_64.rpm
To make gdb4hpc 4.16.2 the default version of gdb4hpc, execute:
/opt/cray/pe/admin-pe/set_default_files/set_default_gdb4hpc_4.16.2
Legal Notices
---------------------------
Certain components, files or programs contained within this package or product are Copyright 2007-2024 Hewlett Packard Enterprise Development LP.
Attribution notices for open source licensed software for this package are detailed in the file:
/opt/cray/pe/gdb4hpc/4.16.2/ATTRIBUTIONS_gdb4hpc.txt
# HDF5 1.14.3.1
## Release Date:
* July 2024
## Purpose:
This HDF5 release contains the following:
* SLES 15 SP5 (x86_64 and aarch64)
* RHEL 9.4 (x86_64 and aarch64)
* RHEL 8.10 (x86_64)
* Nvidia HPC SDK 24.03
* AMD ROCm 6.1
* AOCC 4.2
* Intel OneAPI 2024.0.1
* RHEL gcc-toolset-13
## Product and OS Dependencies:
The HDF5 1.14.3.1 release is supported on the following systems, requiring the following software products:
* CrayPE 2.1.2 or later
* SLES 15 SP5 (x86_64 and aarch64)
* RHEL 8.10 (x86_64)
* RHEL 9.4 (x86_64 and aarch64)
* cray-mpich/8.1.0 or later
Supported on the following HPE systems:
* HPE Cray EX systems with CLE
* HPE Cray XD systems with CLE
One or more of the following compiler major versions:
* CCE 17.x or later (SLES)
* CCE 18.x or later (RHEL)
* GCC 12.3 or later (SLES)
* gcc-toolset-10.3/11.2/12.2 (RHEL 8.10)
* gcc-toolset-11.2/12.2/13.1 (RHEL 9.4)
* AOCC 4.1.0 or later
* AMD ROCm 6.0.0 or later
* Intel 2023.2.0 or later
* Nvidia 23.11 or later (aarch64) and 23.3 or later (x86_64)
## Notes and Limitations:
Users are encouraged to use the Cray compiler scripts (cc, ftn, CC) to include the HDF5 header files and link in the HDF5 libraries, rather than the HDF5 compiler scripts (h5cc, h5fc, h5c++) included in the release.
## Documentation:
http://www.hdfgroup.org/HDF5/doc/index.html
## Modulefile:
`module load cray-hdf5`
OR
`module load cray-hdf5-parallel`
## Product description:
HDF5 is a data model, library, and file format for storing and managing data. It supports an unlimited variety of datatypes, and is designed for flexible and efficient I/O and for high volume and complex data. HDF5 is portable and is extensible, allowing applications to evolve in their use of HDF5. The HDF5 Technology suite includes tools and applications for managing, manipulating, viewing, and analyzing data in the HDF5 format.
## Installation:
`rpm -ivh cray-hdf5-1.14.3.1-*-202406100539.1b08b37223c27.@ARCH@.rpm`
The "`*`" in the install command represents compiler version combinations.
To make this the default version, execute:
`/opt/cray/pe/admin-pe/set_default_files/set_default_hdf5_1.14.3.1`
Certain components, files or programs contained within this package or product are Copyright 2019-2024 Hewlett Packard Enterprise Development LP.
Cray LibSci 24.07.0
=====================
Release Date:
-------------
July 2024
Purpose:
--------
Cray LibSci 24.07.0 provides scientific libraries for Cray
HPC systems. Cray LibSci is supported on the host CPU but
not on the accelerator of these systems.
The Cray LibSci 24.07.0 release provides support for the following:
* RHEL 9.4 (x86_64 and aarch64)
* RHEL 8.10 (x86_64)
* Nvidia HPC SDK 24.03
* AMD ROCm 6.1
* AOCC 4.2
* Intel oneAPI 2024.0.1
* RHEL gcc-toolset-13
Cray LibSci 24.07.0 includes the following versions of publicly
available libraries:
* LAPACK 3.10.1 - For further information, see
http://www.netlib.org/lapack
* ScaLAPACK 2.2.0 - (Scalable LAPACK) For further information, see
http://www.netlib.org/scalapack.
* QDWH 2.0.0 KSVD 1.0.0 - Polar decomposition and SVD packages, see
https://github.com/ecrc/qdwh
https://github.com/ecrc/ksvd
Key Changes and Bugs Closed:
----------------------------
Changes in Cray LibSci 24.07.0
* Miscellaneous bug fixes
* Enabled module to set LD_LIBRARY_PATH when PE_LD_LIBRARY_PATH=system
* Removed Iterative Refinement Toolkit (IRT) support
Changes in Cray LibSci 24.03.0
* CAST-34956 - Lua module environment variable issue
Changed in Cray LibSci 23.12.5
* CAST-33777 - LibSci mutex contention issue
* Removed support for Intel KNL
Product and OS Dependencies:
----------------------------
The Cray LibSci 24.07.0 release requires the following:
* SLES 15 SP5 or RHEL 8.10/9.4
* SLES 15 SP5 (x86_64 and aarch64)
* RHEL 8.10 (x86_64)
* RHEL 9.4 (x86_64 and aarch64)
* cray-mpich/8.1.0 or later
Supported on the following HPE systems:
* HPE Cray EX systems with CLE
* HPE Cray XD systems with CLE
One or more of the following compiler major versions:
* CCE 17.x or later (SLES)
* CCE 18.x or later (RHEL)
* GCC 12.3 or later (SLES)
* gcc-toolset-10.3/11.2/12.2 (RHEL 8.10)
* gcc-toolset-11.2/12.2/13.1 (RHEL 9.4)
* AOCC 4.1.0 or later
* AMD ROCm 6.0.0 or later
* Intel 2023.2.0 or later
* Nvidia 23.11 or later (aarch64) and 23.3 or later (x86_64)
Notes and Limitations:
----------------------
OMP threaded versions:
The CrayPE 2.1.2 and later releases add support for link line generation
for the multi-threaded versions of the libsci library based on the OpenMP
flags the user specifies for each compiler:
* CCE by default links to the OpenMP LibSci library. CrayPE will link in
the serial version of LibSci when the CCE flag -hnoomp is used.
* GNU by default links serial LibSci library. CrayPE will link in the
OpenMP version of LibSci when the GNU flag -fopenmp is used.
* INTEL by default links serial LibSci library. CrayPE will link in the
OpenMP version of LibSci when the INTEL flag -qopenmp is used.
Stack size limit:
The 'ulimit -s unlimited' system stack size setting is required for
cray-libsci on all supported Cray platforms. This is typically set as a
system default by admin but may otherwise need to be set at runtime.
Non-default dynamic linking:
When using a non-default version of cray-libsci with dynamic linking
users should set at runtime and at linktime after loading the desired
cray-libsci module--or set the equivalent for their linux shell:
export LD_LIBRARY_PATH=$CRAY_LD_LIBRARY_PATH:$LD_LIBRARY_PATH
Documentation:
--------------
See the intro_libsci man page for additional information.
See the csmlversion man page for information to display version information
on the currently loaded scientific libraries.
Modulefile:
-----------
module load cray-libsci
Installation instructions:
--------------------------
LibSci is now packaged into separate compiler specific RPMs to allow
rpmbuild to correctly include compiler dependencies.
License:
--------
Except for the third party components and software licensed by HPE
through proprietary agreements, components, files or programs
contained within this package or product are Copyright 2001-2023
Hewlett Packard Enterprise Development LP.
Attribution notices for open source licensed software contained in this
package are detailed in the file:
/opt/cray/libsci/24.07.0/ATTRIBUTIONS
Cray LibSci_ACC 24.07.0
=======================
Release Date:
-------------
July 2024
Purpose:
--------
The Cray LibSci_ACC 24.07.0 release provides accelerated versions of
scientific libraries for HPE Cray systems with accelerators.
Product and OS Dependencies:
----------------------------
Cray LibSci_ACC 24.07.0 requires the following platform:
HPE Cray EX system with AMD MI250X
HPE Cray EX system with AMD MI300A
HPE Cray EX system with Nvidia Hopper
Cray LibSci_ACC 24.07.0 supports the following OS:
RHEL 8.10 x86_64
RHEL 9.4 x86_64 & aarch64
SLES 15 SP5 x86_64 & aarch64
Notes and Limitations:
----------------------
LAPACK workspace calculations from other implementations or hard-coded
values may not be compatible with libsci_acc. Workspaces reported from
a query to the corresponding libsci_acc function should be used.
The current supported usecase is a one-rank to one-gpu mapping for apps.
Performance improvements can be achieved in programs calling libsci_acc
subroutines by using pinned memory. See the intro_libsci_acc man page for
details.
Use of the aprun option "-cc none" or the srun option "--cpu_bind=none"
to disable core affinity is strongly suggested to maintain performance.
When using non-default cray-libsci_acc versions users should set equivalent
at runtime and linktime after loading the desired cray-libsci_acc module:
export LD_LIBRARY_PATH=$CRAY_LD_LIBRARY_PATH:$LD_LIBRARY_PATH
Documentation:
--------------
See the intro_libsci_acc man page for additional information.
Modulefile:
-----------
module load cray-libsci_acc
Sample Installation instructions:
---------------------------------
RPM install command:
rpm -ivh cray-libsci-acc-24.07.0-*.##ARCH##.rpm
The "*" in the install command represents compiler version combinations.
To make this the default version:
##PREFIX##/admin-pe/set_default_files/set_default_libsci_acc_24.07.0
License:
--------
Except for the third party modules and software licensed by HPE
through proprietary agreements, components, files or programs
contained within this package or product are Copyright -2023 Hewlett
Packard Enterprise Development LP.
Attribution notices for open source licensed software contained in
this package are detailed in the file:
##PREFIX##/libsci_acc/24.07.0/ATTRIBUTIONS_libsci_acc_24.07.0.txt
Cray Lmod 8.7.37
================================================================================
Release Date:
--------------------------------------------------------------------------------
March, 2024
Purpose:
--------------------------------------------------------------------------------
Provide a Lmod 8.7.37 package for CPE customers
Bugs fixed in this release:
--------------------------------------------------------------------------------
N/A
Documentation:
--------------------------------------------------------------------------------
See CPE's Installation and User Guides for documentation.
Also see Lmod's official website: https://lmod.readthedocs.io
Dependencies:
--------------------------------------------------------------------------------
The Lmod Scripts 8.7.37 release is dependent on the following
software:
bc
sed
lua >= 5.3
lua-luaposix
lua-luafilesystem
Installation instructions:
--------------------------------------------------------------------------------
rpm -Uvh cray-lmod-8.7.37-1.el9.x86_64.rpm
To make this the default version, update the symbolic "lmod" link
to the desired Lmod version directory.
Lmod Scripts 3.2.1
================================================================================
Release Date:
--------------------------------------------------------------------------------
December, 2023
Purpose:
--------------------------------------------------------------------------------
Updates package license.
Bugs fixed in this release:
--------------------------------------------------------------------------------
Documentation:
--------------------------------------------------------------------------------
See system Installation and User Guides for documentation
Dependencies:
--------------------------------------------------------------------------------
The Lmod Scripts 3.2.1 release is dependent on the following
software:
set_default_3
Lua
Lmod
Installation instructions:
--------------------------------------------------------------------------------
rpm -Uvh lmod_scripts-3.2.1-202405031504.c3177a9cde245-0.noarch.rpm
To make this the default version, execute:
/opt/cray/pe/admin-pe/set_default_files/set_default_lmod_scripts_3.2.1
Product description:
--------------------------------------------------------------------------------
Contains Lua scripts which support the Cray implementation of Lmod
Copyright 2020-2023 Hewlett Packard Enterprise Development LP
modules 3.2.11.7
==================
Release Date:
-------------
Dec, 2023
Purpose:
--------
- Package dependency update
Documentation:
---------------
See man-pages for module and modulefile.
Cray MPICH 8.1.30:
=======================================
Release Date:
-------------
June 1, 2024
Purpose:
--------
Cray MPICH 8.1.30 is based upon ANL MPICH 3.4a2 with support for libfabric
and is optimized for the Cray Programming Environment.
Major Differences Cray MPICH 8.1.30 from the XC Cray MPICH include:
- Uses the new ANL MPICH CH4 code path and libfabric for network
support.
- Does not support -default64 mode for Fortran
- Does not support C++ language bindings
New Cray MPICH features for HPE Cray EX and Apollo systems:
- Starting from the 8.1.26 release, Cray MPICH supports the Intel Sapphire Rapids CPU HBM
processor architecture.
- On systems with AMD GPUs, Cray MPICH 8.1.26 supports all ROCm
versions starting from ROCm 5.0, including the latest ROCm 5.5.0
release.
The Cray MPICH 8.1.25 release and prior versions of
Cray MPICH are only compatible with ROCm versions up to (and
including) the ROCm 5.4.0 release.
- Cray MPICH uses the libfabric "verbs;ofi_rxm" provider by default.
This is the supported and optimized OFI libfabric provider for
Slingshot-10 and Apollo systems.
- Cray MPICH offers support for multiple NICs per node. Starting with
version 8.0.8, by default Cray MPICH will use all available NICs on
a node. Several rank-to-NIC assignment policies are supported. For
details on choosing a policy for assigning ranks to NICS, or for
selecting a subset of available NICs, please see the following
environment variables documented in the mpi man page.
MPICH_OFI_NIC_VERBOSE
MPICH_OFI_NIC_POLICY
MPICH_OFI_NIC_MAPPING
MPICH_OFI_NUM_NICS
- Enhancements to the MPICH_OFI_NIC_POLICY NUMA mode have been added.
Starting with version 8.0.14, if the user selects the NUMA policy,
the NIC closest to the rank is selected. A NIC no longer needs to
reside in the same numa node as the rank. If multiple NICs are
assigned to the same numa node, the local ranks will round-robin
between them. Numa distances are analyzed to select the closest NIC.
- Cray MPICH supports creating a full connection grid during MPI_Init.
By default, OFI connections between ranks are set up on demand. This
allows for optimal performance while minimizing memory requirements.
However, for jobs requiring an all-to-all communication pattern, it
may be beneficial to create all OFI connections in a coordinated
manner at startup. See the MPICH_OFI_STARTUP_CONNECT description in
the mpi man page.
- Cray MPICH supports runtime switching to the UCX netmod starting
with version 8.0.14. To do this load the craype-network-ucx module
and module swap between Cray-MPICH and Cray-MPICH-UCX modules. For
more information including relevant environment variables reference
the intro_mpi man page with the Cray-MPICH-UCX module loaded.
- Lmod support for HPE Cray EX starting with Cray MPICH 8.0.16.
Key Changes and Bugs Closed:
----------------------------
Changes in Cray MPICH 8.1.30
New Features:
- Cray MPICH 8.1.30 for aarch64 is in early stage of support and
has some issues. Setting these variables will help circumvent most
known issues.
export MPICH_SMP_SINGLE_COPY_MODE=CMA
export MPICH_MALLOC_FALLBACK=1
Features and Bug Fixes:
- CAST-34323 - Implement MPICH CQ stall feature for performance improvements on Sapphire Rapids
- CAST-34363 - Enable GTL to use multiple CUDA streams per process
- CAST-35020 - Improve error handling when running out of space in the UDREG cache
- CAST-35124 - Bugfix on aarch64 for MPI_THREAD_MULTIPLE
- CAST-35420 - Bugfix for MPI-IO aggregator selection
- CAST-35543 - Enable GPU kernel support for MPI_Reduce
- CAST-35648 - RMA Windows - Plug memory leaks by freeing shm_cookies and target_info
- PE-52424 - Enable module to set LD_LIBRARY_PATH when PE_LD_LIBRARY_PATH=system
- PE-52488 - Turn off aarch64 XPMEM when GPU support is requested
- PE-52582 - Fix MPI_LONG_DOUBLE datatype for CCE on aarch64
- PE-52604 - Bugfix for memory leak in GTL HSA IPC cache
- PE-53506 - Bugfix for a race condition with some CUDA use cases
- PE-54032 - Enable Allreduce ops to use GPU staging buffers larger than 1GB
- PE-54192 - Document aarch64 CUDA smp behavior
- PE-54326 - Document new MPICH CQ Stall variables
- PE-54303 - Bugfix for HSA seg_info cache
Product and OS Dependencies:
----------------------------
The Cray MPICH 8.1.30 release is supported on the following HPE systems:
* HPE Cray EX systems with CLE
* HPE Apollo systems as part of the Cray Programming Environment
Product and OS Dependencies by network type:
--------------------------------------------------+
| Shasta |
----------------------------+---------------------+
craype | >= 2.7.6 |
----------------------------+---------------------+
cray-pals | >= 1.0.6 |
----------------------------+---------------------+
cray-pmi | >= 6.0.1 |
----------------------------+---------------------+
libfabric | >= 1.9.0 |
----------------------------+---------------------+
One or more compilers:
* AMD ROCM 6.0 or later
* AOCC 4.1 or later
* CCE 18.0 or later
* GNU 11.2 or later
* Intel 2022.1 or later
* Nvidia 23.3 or later
Notes and Limitations:
----------------------
Limitations in Cray MPICH 8.1.30:
- Cray MPICH 8.1.30 can support only ~2040 simultaneous MPI
communicators. This limit is less the XC Cray MPICH limit of
~4090 simultaneous communicators. Cray intends to raise the
limit in a future release of Cray MPICH for Shasta to at least
the XC limit.
Documentation:
--------------
For more information see the intro_mpi man page.
Modulefile:
-----------
module load cray-mpich/8.1.30
License:
--------
Except for the third party components and software licensed by HPE
through proprietary agreements, components, files or programs contained
within this package or product are Copyright -2021 Hewlett Packard
Enterprise Development LP.
Attribution notices for open source licensed software for this
package are detailed in the file:
/opt/cray/pe/mpich/8.1.30/ATTRIBUTIONS
Copyright -2023 Hewlett Packard Enterprise Development LP
Copyright 2022-2024 Hewlett Packard Enterprise Development LP
Product Release Information
================================================================================
Product identification:
-----------------------
HPE Cray MPIxlate: version 1.0.5 (built on 2024-06-03T22:39:16Z from 8085bef)
Purpose:
--------
HPE Cray MPIxlate enables applications compiled using an MPI library that
is not binary compatible with HPE Cray MPI, to be run without recompilation
on supported HPE platforms.
Warnings & Limitations:
-----------------------
- See NOTES in mpixlate(1)
Changes from previous release:
------------------------------
- New features:
* None
- Defects Fixed:
* PE-53958: Invalid library path for PrgEnv-cray
Platform, Operating System and Product dependencies:
----------------------------------------------------
Supported HPE platforms:
- HPE Cray EX systems
- HPE Apollo systems with Cray Programming Environment
Supported Operating Systems:
- SLES 15 SP4 and COS-2.5
- SLES 15 SP5 and COS-3.0
- RHEL 8.10
- RHEL 9.4
Product dependencies:
-----------------+----------
cray-mpich[-ucx] | >= 8.1.30
-----------------+----------
Documentation:
--------------
mpixlate(1)
Environment Modules
-------------------
module load cray-mpixlate/1.0.5
================================================================================
cray-mrnet 5.1.3
================================================================================
*Released: May 2024*
Product Description
---------------------------
- MRNet is a customizable, high-performance software infrastructure for
building scalable tools and applications. It supports efficient
multicast and data aggregation functionality using a tree of processes
between the tool's front-end and back-ends. MRNet-based tools may use
these internal processes to distribute many important tool activities,
for example to reduce data analysis time and keep tool front-end loads
manageable.
MRNet-based tool components communicate across logical channels called
streams. At MRNet internal processes, filters are bound to these streams to
synchronize and aggregate dataflows. Using filters, MRNet can efficiently
compute averages, sums, and other more complex aggregations and analyses
on tool data. MRNet also supports facilities that allow tool developers
dynamically load new tool-specific filters into the system.
Changelog
---------------------------
=======================================================================
## [5.1.3] - 2024-04-22 (24.05)
=======================================================================
### Features
* PE-51960: Added support for rhel89/93 & gcc13
=======================================================================
## [5.1.2] - 2023-09-25 (23.10)
=======================================================================
### Features
* Use gcc toolset 12 on redhat
=======================================================================
## [5.1.1] - 2023-06-14 (23.07)
=======================================================================
### Features
* Use native sles compilers
OS Dependencies
---------------------------
- HPE CSM systems with SLES 15 SP4
- HPE HPCM systems with SLES 15 SP5 and RHEL 8.8
- HPE Apollo systems with RHEL 8.8
Product Dependencies
---------------------------
- cray-cti >= 2.18, < 3 installed
- cdst-support >= 2.14, < 3 installed
Installation Instructions
---------------------------
- rpm -ivh cray-mrnet-5.1.3-20240506181321_4fa0df65d0e3-3.el9.x86_64.rpm
To make cray-mrnet 5.1.3 the default version of cray-mrnet, execute:
/opt/cray/pe/admin-pe/set_default_files/set_default_cray-mrnet_5.1.3
Legal Notices
---------------------------
Certain components, files or programs contained within this package or product are Copyright 2022-2024 Hewlett Packard Enterprise Development LP.
Attribution notices for open source licensed software for this package are detailed in the file:
/opt/cray/pe/mrnet/5.1.3/ATTRIBUTIONS_mrnet.txt
# NetCDF 4.9.0.13
## Release Date:
* July 2024
## Purpose:
This NetCDF release contains the following:
* SLES 15 SP5 (x86_64 and aarch64)
* RHEL 9.4 (x86_64 and aarch64)
* RHEL 8.10 (x86_64)
* Nvidia HPC SDK 24.03
* AMD ROCm 6.1
* AOCC 4.2
* Intel OneAPI 2024.0.1
* RHEL gcc-toolset-13
## Product and OS Dependencies:
The NetCDF 4.9.0.13 release is supported on the following systems, requiring the following software products:
* Cray HDF5 1.14.3.*
* CrayPE 2.1.2 or later
* Cray MPT 8.1 or later
* SLES 15 SP5 (x86_64 and aarch64)
* RHEL 8.10 (x86_64)
* RHEL 9.4 (x86_64 and aarch64)
Supported on the following HPE systems:
* HPE Cray EX systems with CLE
* HPE Cray XD systems with CLE
One or more of the following compiler major versions:
* CCE 17.x or later (SLES)
* CCE 18.x or later (RHEL)
* GCC 12.3 or later (SLES)
* gcc-toolset-10.3/11.2/12.2 (RHEL 8.10)
* gcc-toolset-11.2/12.2/13.1 (RHEL 9.4)
* AOCC 4.1.0 or later
* AMD ROCm 6.0.0 or later
* Intel 2023.2.0 or later
* Nvidia 23.11 or later (aarch64) and 23.3 or later (x86_64)
## Notes and Limitations:
For certain F77 source files, when compiling with `gcc/10+`, it may be necessary to configure/compile with:
`export FCFLAGS="-fallow-argument-mismatch"`
`export FFLAGS="-fallow-argument-mismatch"`
This will turn mismatch errors between actual and dummy argument lists to warnings. See:
https://github.com/Unidata/netcdf-fortran/issues/212
https://gcc.gnu.org/gcc-10/changes.html
Unidata now packages Netcdf-4 and legacy Netcdf-3 separately. HPE has decided not to continue supplying the legacy Netcdf-3 package. Due to CCE changes a version of netcdf built with "-sreal64" is neither needed nor provided.
NetCDF is supported on the host CPU but not on the accelerator.
## Documentation:
http://www.unidata.ucar.edu/software/netcdf/docs
## Modulefile:
`module load cray-netcdf`
OR
`module load cray-netcdf-hdf5parallel`
## Product Description:
NetCDF (network Common Data Form) is a set of interfaces for array-oriented data access and a freely-distributed collection of data access libraries for C, Fortran, C++, Java, and other languages. The netCDF libraries support a machine-independent format for representing scientific data. Together, the interfaces, libraries, and format support the creation, access, and sharing of scientific data.
## Installation:
`rpm -ivh cray-netcdf-4.9.0.13-*202406081619.32645f8ced41e-5.el9.x86_64.rpm`
The "*" in the install command represents compiler version combinations.
To make this the default version, execute:
`/opt/cray/pe/admin-pe/set_default_files/set_default_netcdf_4.9.0.13`
Certain components, files or programs contained within this package or product are Copyright 2019-2024 Hewlett Packard Enterprise Development LP.
cray-papi
=================
Release date:
-------------
June 2024
Package:
--------
cray-papi-7.1.0.2-202405031627.5e39f84427b3c-3 x86_64
Purpose:
--------
New version of papi 7.1.0.2 release 3
Documentation:
--------------
Overview: https://github.com/icl-utk-edu/papi
Web Site: https://icl.utk.edu/papi
Product description:
--------------------
PAPI aims to provide the tool designer and application engineer with a
consistent interface and methodology for use of the performance counter
hardware found in most major microprocessors. PAPI enables software
engineers to see, in near real time, the relation between software
performance and processor events.
Dependencies:
-------------
For a list of software used when validating this version of
PAPI on Cray and HPE systems, see the HPE Cray Programming
Environment release announcements.
Copyright 2015-2017,2019-2024 Hewlett Packard Enterprise Development LP
# Parallel NetCDF 1.12.3.13
## Release Date:
* July 2024
## Purpose:
This Parallel NetCDF release contains the following:
* SLES 15 SP5 (x86_64 and aarch64)
* RHEL 9.4 (x86_64 and aarch64)
* RHEL 8.10 (x86_64)
* Nvidia HPC SDK 24.03
* AMD ROCm 6.1
* AOCC 4.2
* Intel OneAPI 2024.0.1
* RHEL gcc-toolset-13
See the Parallel NetCDF documentation for details and usage information.
## Product and OS Dependencies:
This Parallel NetCDF 1.12.3.13 release is supported on on the following systems, requiring the following software products:
* CrayPE 2.1.2 or later
* Cray MPT 8.1 or later
* SLES 15 SP5 (x86_64 and aarch64)
* RHEL 8.10 (x86_64)
* RHEL 9.4 (x86_64 and aarch64)
Supported on the following HPE systems:
* HPE Cray EX systems with CLE
* HPE Cray XD systems with CLE
One or more of the following compiler major versions:
* CCE 17.x or later (SLES)
* CCE 18.x or later (RHEL)
* GCC 12.3 or later (SLES)
* gcc-toolset-10.3/11.2/12.2 (RHEL 8.10)
* gcc-toolset-11.2/12.2/13.1 (RHEL 9.4)
* AOCC 4.1.0 or later
* AMD ROCm 6.0.0 or later
* Intel 2023.2.0 or later
* Nvidia 23.11 or later (aarch64) and 23.3 or later (x86_64)
## Notes and Limitations:
For certain F77 source files, when compiling with `gcc/10+`, it may be necessary to configure/compile with:
`export FCFLAGS="-fallow-argument-mismatch"`
`export FFLAGS="-fallow-argument-mismatch"`
This will turn mismatch errors between actual and dummy argument lists to warnings. See:
https://github.com/Unidata/netcdf-fortran/issues/212
https://gcc.gnu.org/gcc-10/changes.html
## Documentation:
https://parallel-netcdf.github.io/wiki/Documentation.html
## Modulefile:
`module load cray-parallel-netcdf`
## Product Description:
Parallel NetCDF is a library providing high-performance I/O while maintaining file-format compatibility with Unidata's NetCDF.
## Installation:
`rpm -ivh cray-parallel-netcdf-1.12.3.13-*202406080630.dd5a30e54e2bc-5.el9.x86_64.rpm`
The "`*`" in the install command represents compiler version combinations.
To make this the default version, execute:
`/opt/cray/pe/admin-pe/set_default_files/set_default_parallel-netcdf_1.12.3.13`
Certain components, files or programs contained within this package or product are Copyright 2019-2024 Hewlett Packard Enterprise Development LP.
Perftools 24.07.0
=================
Release Date: July, 2024
Purpose:
========
This is a feature and bugfix release for the following systems:
- HPE Cray EX and HPE Cray Supercomputer Systems with HPCM
- HPE Cray EX and HPE Cray Supercomputer Systems with CSM
- HPE Apollo 2000 Gen 10 Plus Systems (x86)
Key enhancements or changes from the previous release:
======================================================
o Add support for ROCm 6.1.0
o Add support for CUDA 12.4
o Upgrade to PAPI 7.1.0.2
o Upgrade to DWARF 0.9.2
o Upgrade to support MPICH v8
o First GA release for Apprentice 3
o Accelerator-to-accelerator and host-to-host transfers from roctracer are recorded
o Histograms of message sizes are recorded by transfer type. See environment variables
PAT_RT_ACC_*_BINS in "man pat" for details.
o Extended support for tracing python code
o Record python class names when applicable in calltree leafs
o Add tracing groups cuda_math, hip_math, and rocm_math
o Add tracing groups for PyTorch
o Add tracing support for NCCL/RCCL
o Update trace group 'cuda' to CUDA 12.4
o Update trace group 'hip' and 'hsa' to support ROCm 6.1.0
o Update trace group 'fabric' to support libfabric 1.20.1
o Update trace group 'mpi' to support MPICH v8.1
Bugs Fixed:
===========
o Fix ssh access from Apprentice 2/3
o Fix crash on Apprentice 2 windows client
o Support applications where multiple threads trigger roctracer callback
o Account for missing 'product_name' file and check 'device' file in ROCm installs
o Suppress runtime error for lite-hbm in AMD cpus
o Fix missing 0 count events
o Fix potential loading problems by disabling trace for dlopen by default
o Support floating point counter events for AMD processors
o Support performance counters for AMD MI300A processor
o Various Apprentice2 displays
o Various improvements in tracing Python
o Properly support multiple accelerator threads
o No longer trace dynamic library functions by default
Notes:
======
o This release of Perftools only supports ROCm 6.1.0
o This release of Perftools only supports CUDA 12.0 and later
Known Issues:
=============
o MI300 and MI300a GPU performance counter collection is not supported
o For apps using PyTorch+ROCm installs, pat_run will use ROCm shared libraries found in $ROCM_PATH rather than those installed with PyTorch
o Apprentice 3.0 known issues
o Mac install packages are not signed in the release. To install you must "open" the .dmg file
and agree to the warning dialog.
o The windows desktop can fail to access remote experiments in some installs.
o The mac desktop can show up with inverted or wrong graphics if an older version of gtk is
installed on the machine.
o Potential crash when cutting and pasting into the windows client.
o Searching for function name in the time line view does not work.
o The timeline view can rest to an interval of [NaN,NaN]; reload the experiment to continue.
o Missing tool tips, help, etc.
Deprecations and Removals:
==========================
o Remove custom cray-libelf packaging; use system libelf
o Reveal is no longer supported
Dependencies:
=============
o A PDF reader (such as evince, acroread or okular) is required to use the
app2 online help.
o For a list of software used when validating this version of
Perftools on Cray and HPE systems, see the HPE Cray Programming Environment
release announcements.
Installation instructions:
==========================
For Apollo 80 systems, please see the HPE Cray Programming Environments
Installation Guide: Apollo 80 (ARM) System (S-8013).
For Apollo 2000 systems, please see the HPE Cray Programming Environments
Installation Guide: Apollo 2000 Gen10 Plus (x86) System (S-8012).
For HPE Cray EX (Shasta Architecture) systems, refer to the Cray Asynchronous
Installer Guide (S-8003).
Installation of app2 remote client (+ server) on Mac systems:
-------------------------------------------------------------
Apprentice2Installer-24.07.0.dmg
Apprentice3Installer-24.07.0.dmg
The Cray Apprentice2 and Apprentice3 installers for Mac are included in the
perftools-clients rpm, and placed in
$CRAYPAT_ROOT/share/desktop_installers/
on a Cray Shasta User Access Node (UAN) or on a Cray login node when the
perftools software is installed.
Download the Cray Apprentice2 or Apprentice3 installer onto a desktop or laptop
running Mac OS El Capitan through Mojave. Double click on installer
to begin installation. The installer will walk you through the
process for your system.
Installation of app2 remote client (+ server) on Windows 10 systems:
--------------------------------------------------------------------
Apprentice2Installer-24.07.0.exe
Apprentice3Installer-24.07.0.exe
The Cray Apprentice2 and Apprentice3 installers for Windows are included in the
perftools-clients rpm, and placed in
$CRAYPAT_ROOT/share/desktop_installers/
on a Cray Shasta User Access Node (UAN) or on a Cray login node when the
perftools software is installed.
Download the Cray Apprentice2 installer onto a desktop or laptop
running Windows 10. Double click on installer to begin installation.
The installer will walk you through the process for your system.
Documentation:
==============
See the following man pages:
app2, grid_order, intro_craypat, pat_build, pat_help, pat_info, pat_opts,
pat_report, pat_run, pat_view
intro_papi
perftools-base, perftools-lite, perftools-preload
accpc, cray_cassini, cray_pm, hwpc, nwpc, papi_counters, uncore
PAPIlicnotices
Search for perftools on https://support.hpe.com to access documentation
on Cray Performance Measurement and Analysis Tools
License:
========
Except for the third party modules discussed below and software licensed
by HPE through proprietary agreements, components, files or programs
contained within this package or product are Copyright 2001-2024
Hewlett Packard Enterprise Development LP.
Attribution notices for open source licensed software contained in this
package are detailed in the file:
$CRAYPAT_ROOT/ATTRIBUTIONS_perftools.txt
Cray PMI 6.1.15.4
==============================================================================
Release Date:
--------------
June 26, 2024
Purpose:
--------
The Cray Process Manager Interface Library provides the interface between the
application launcher and other communication libraries such as MPICH and SHMEM.
Cray PMI 6.1.15.4 is the initial package with support for libfabric and
optimized for the Cray Shasta product line.
Major features new to Cray PMI 6.1.15.4 include:
Bug fixes new to Cray PMI 6.1.15.4 include:
* PE-29641 PMI_VERSION_DISPLAY=1 broken
Known limitations include:
Certain components, files or programs contained within this package or product
are Copyright 2004-2023 Hewlett Packard Enterprise Development LP
cray-python 3.11.7
====================
Release Date
------------
July 2024
Purpose
-------
Cray Python is an implementation of the Python programming language for the Cray
Programming Environment. The numpy and scipy modules are configured to call Cray
Libsci routines. The mpi4py module is configured to call Cray MPICH routines.
Cray Python is designed to run Python codes on the compute nodes of an HPE
supercomputer. HPE does not make changes to the Python source or any of its
libraries nor does it plan to make changes in future releases.
The cray-python 3.11.7 release contains
- python-3.11.7
- numpy-1.24.4
- scipy-1.10.1
- mpi4py-3.1.4
- dask-2023.6.1
Product and OS Dependencies
---------------------------
The cray-python 3.11.7 release is supported on
- Cray EX systems running SLES 15 or RHEL 8
Documentation
-------------
https://www.python.org/doc
Changelog
---------
https://docs.python.org/release/3.11.7/whatsnew/changelog.html
Modulefile
----------
module load cray-python/3.11.7
Installation
------------
rpm -ihv cray-python-3.11.7-20240617153005.0687944-1.el9.x86_64.rpm
To make this the default version, execute
[PREFIX]/admin-pe/set_default_files/set_default_python_3.11.7
Certain components, files, or programs contained within this package are
© Copyright 2021-2024 Hewlett Packard Enterprise Development LP
cray-R 4.4.0
============
Release Date
------------
March 2024
Purpose
-------
The cray-R 4.4.0 release.
cray-R is configured to call cray-libsci routines.
Product and OS Dependencies
---------------------------
The cray-R 4.4.0 release is supported on
- Cray EX systems running SLES 15 or RHEL 8
Documentation
-------------
https://cran.r-project.org/manuals.html
Modulefile
----------
module load cray-R/4.4.0
Installation
------------
rpm -ihv cray-R-4.4.0-20240617153123.a134495-1.el9.x86_64.rpm
To make this the default version, execute:
[PREFIX]/admin-pe/set_default_files/set_default_R_4.4.0
Certain components, files or programs contained within this package or product are
Copyright 2018-2024 Hewlett Packard Enterprise Development LP
sanitizers4hpc 1.1.3
================================================================================
*Released: June 2024*
Product Description
---------------------------
- Sanitizers4hpc is a debugging tool to aid in the detection of memory leaks
and errors in parallel applications. Using LLVM's AddressSanitizer or
LeakSanitizer as a data source, Sanitizers4hpc aggregates any duplicate
messages across ranks to help provide an understandable picture of
program behavior.
Notes
---------------------------
- Shared library exposure in previous versions of sanitizers4hpc.
Previous versions of sanitizers4hpc incorrectly exported internal sanitizers4hpc
libraries into /opt/cray/pe/lib64, causing other programs that had that path in
their runpath or LD_LIBRARY_PATH to load libraries they didn't expect.
This has been fixed in the 22.08 release of sanitizers4hpc. However, links will
be present in /opt/cray/pe/lib64 pointing to older versions of sanitizers4hpc,
if an older version is installed because old installations will still modify
/opt/cray/pe/lib64 when ldconfig is run.
To clean up libraries left by previous versions of sanitizers4hpc:
Option 1:
1. Uninstall all previously installed versions of sanitizers4hpc, or remove
the .cray_dynamic_file_list file from all previously installed versions
of sanitizers4hpc, typically installed in /opt/cray/pe/sanitizers4hpc.
2. Remove any links in /opt/cray/pe/lib64 pointing into a sanitizers4hpc installation.
3. run ldconfig
Option 2:
1. Uninstall all previous versions of santizers4hpc.
Changelog
---------------------------
=======================================================================
## [unreleased]
=======================================================================
### Bug Fixes
* PE-54265: Automatically preload libfabric if linked
=======================================================================
## [1.1.3] - 2024-05-2 (24.06)
=======================================================================
### Bug Fixes
* PE-53866: Improve ThreadSanitizers and Nvidia Compute Sanitizers aggregation
* PE-53603: Update ZMQNet submodule for libsodium support
* PE-51193: Activate PALS startup barrier manually through libpals
* PE-49611: Update ZMQNet
### Features
* PE-51960: Added support for rhel89/93 & gcc13
* Use gcc toolset 12 on redhat
=======================================================================
## [1.1.1] - 2023-07-7 (23.08)
=======================================================================
### Features
* PE-47186: Add Sanitizers4hpc User Guide
* PE-48278: Supply CUDA Sanitizer options via frontend argument
* Use native compilers on sles
* PE-48186: Support SHMEM non-shared heap checking, HTML documentation in share/doc
OS Dependencies
---------------------------
- HPE CSM systems with SLES 15 SP4
- HPE HPCM systems with SLES 15 SP5 and RHEL 8.8
- HPE Apollo systems with RHEL 8.8
Product Dependencies
---------------------------
- cray-cti >= 2.18, < 3 installed
- cdst-support >= 2.14, < 3 installed
Installation Instructions
---------------------------
- rpm -ivh sanitizers4hpc-1.1.3-20240523190230_2688b90db106-2.el9.x86_64.rpm
To make sanitizers4hpc 1.1.3 the default version of sanitizers4hpc, execute:
/opt/cray/pe/admin-pe/set_default_files/set_default_sanitizers4hpc_1.1.3
Legal Notices
---------------------------
Certain components, files or programs contained within this package or product are Copyright 2022-2024 Hewlett Packard Enterprise Development LP.
Attribution notices for open source licensed software for this package are detailed in the file:
/opt/cray/pe/sanitizers4hpc/1.1.3/ATTRIBUTIONS_sanitizers4hpc.txt
Cray OpenSHMEMX 11.7.2:
===============================
Release Date:
-------------
July, 2024
Purpose:
--------
OpenSHMEM is a Partitioned Global Address Space (PGAS) library interface
specification, which is the culmination of a standardization effort among
many implementers and users of SHMEM programming model. SHMEM has a long
history as a parallel programming model on HPE Cray supercomputer systems.
For the past two decades SHMEM library implementation in HPE Cray systems
evolved through different generations. Cray OpenSHMEMX is a proprietary,
OpenSHMEM specification compliant SHMEM implementation for HPE Cray
product line.
Key Changes:
------------
Major differences in Cray OpenSHMEMX 11.7.2 from
Cray OpenSHMEMX version 11.7.1 includes the following:
- Add support for using 4/8 byte puts for application-level signaling
Bugs Closed:
------------
The following bugs are fixed as part of the Cray OpenSHMEMX 11.7.2
release:
- Fix several shmem_quiet flushing issues for default and non-default
settings
- Fix the implementation of signaling sets to match OSM-1.5 semantics
Product and OS Dependencies:
----------------------------
The Cray OpenSHMEMX 11.7.2 release is supported on the following
Cray systems:
* HPE Cray EX systems with CLE
Product and OS Dependencies by network type:
--------------------------------------------------+
| HPE Slingshot |
----------------------------+---------------------+
craype | >= 2.7.3 |
----------------------------+---------------------+
cray-pals | >= 0.3.5 |
----------------------------+---------------------+
cray-pmi | >= 6.0.1 |
----------------------------+---------------------+
libfabric | >= 1.9.0 |
----------------------------+---------------------+
cray-dsmml | >= 0.2.2 |
----------------------------+---------------------+
One or more compilers:
* AOCC 2.2 or later
* CCE 9.1 or later
* GNU 9.1 or later
* Intel 19.0 or later
* Nvidia 20.7 or later
Notes and Limitations:
----------------------
Announcements, release information, supported environments, limitations,
usage models, and backward compatibility informations about this product
can be viewed in the following location:
https://pe-cray.github.io/cray-openshmemx/
Documentation:
--------------
Use the Cray OpenSHMEMX man pages for more information on the library
functions and use intro_shmem(3) man page for general information.
Initial support for Cray OpenSHMEMX man pages are derived from OpenSHMEM
standards specification document
Use https://pe-cray.github.io/whitepapers/ for access to different
Cray OpenSHMEMX specific whitepapers
Modulefile:
-----------
module load cray-openshmemx/11.7.2
License:
--------
Components, files, and programs contained within this package or
product are Copyright Hewlett Packard Enterprise Development LP
cray-stat 4.12.3
================================================================================
*Released: May 2024*
Product Description
---------------------------
- The Stack Trace Analysis Tool (STAT) is a highly scalable, lightweight
tool that gathers and merges stack traces from all of the processes of
a parallel application to form call graph prefix trees.
Notes
---------------------------
- To enable gathering stack traces on an Nvidia GPU, the environment variable
STAT_GDB must be set to either cuda-gdb, which will require cuda-gdb to be
available in PATH. Or it can be set a specific debugger. We suggest the
version of cuda-gdb packaged with gdb4hpc, as in:
export STAT_GDB=/opt/cray/pe/gdb4hpc/$GDB4HPC_VERSION/libexec/cuda110-gdb-guard
After which stat-cl would be called using the options: -w -G -i
Changelog
---------------------------
=======================================================================
## [4.12.3] - 2024-04-23 (24.05)
=======================================================================
### Features
* PE-51960: Added support for rhel89/93 & gcc13
* Update to gcc toolset 12 on redhat
* Update to gcc toolset 12 on redhat
=======================================================================
## [4.12.1] - 2023-07-11 (23.08)
=======================================================================
### Features
* Use native compilers on sles
* Use native sles compilers
=======================================================================
## [4.12.0] - 2023-04-11 (23.05)
=======================================================================
### Features
* Build with gcc-toolset-10 on redhat
OS Dependencies
---------------------------
- HPE CSM systems with SLES 15 SP4
- HPE HPCM systems with SLES 15 SP5 and RHEL 8.8
- HPE Apollo systems with RHEL 8.8
Product Dependencies
---------------------------
- cray-mrnet >= 5.1, < 6 installed
- cray-cti 2.18, < 3 installed
- cdst-support >= 2.14, < 3 installed
Documentation
---------------------------
- https://computing.llnl.gov/code/STAT/
Installation Instructions
---------------------------
- rpm -ivh --oldpackage cray-stat-4.12.3-20240605164752_d896288ae0cd-3.el9.x86_64.rpm
To make cray-stat 4.12.3 the default version of cray-stat, execute:
/opt/cray/pe/admin-pe/set_default_files/set_default_cray-stat_4.12.3
Legal Notices
---------------------------
Certain components, files or programs contained within this package or product are Copyright 2007-2024 Hewlett Packard Enterprise Development LP.
Attribution notices for open source licensed software for this package are detailed in the file:
/opt/cray/pe/stat/4.12.3/ATTRIBUTIONS_stat.txt
totalviewsup 2024.1.21
================================================================================
*Released: March 2024*
Product Description
---------------------------
- Provides the HPE Totalview support package which contains modulefiles (tcl & lua),
set_default files, and content projection files for Totalview 2024.1.21.
- The only supported install directory for HPE totalviewsup is /opt/cray/pe.
- The only supported install directory for Perforce Totalview is /opt/totalview.
- Drop the license file into: /opt/totalview/2024.1.21/
Changelog
---------------------------
=======================================================================
## [2023.4.16] - 2024-02-26 (24.03)
=======================================================================
### Features
* PE-49958: Implemented the auto-gen release notes/changelog tool
- Updated supported version to 2023.4.16
OS Dependencies
---------------------------
- HPE CSM systems with SLES 15 SP4
- HPE HPCM systems with SLES 15 SP5 and RHEL 8.8
- HPE Apollo systems with RHEL 8.8
Product Dependencies
---------------------------
- This totalviewsup 2024.1.21 package has a dependency on Totalview 2024.1.21.
Documentation
---------------------------
- Totalview docs can be found in /opt/toolworks/totalview.2024.1.21/doc/
- Totalview user guide:
/opt/toolworks/totalview.2024.1.21/doc/TotalView_User_Guide.pdf
- Totalview installation guide:
/opt/toolworks/totalview.2024.1.21/doc/TotalView_Installation_Guide.pdf
- The Totalview release notes:
/opt/toolworks/totalview.2024.1.21/doc/TotalView_Change_Log.pdf
- The Totalviewsup support package release notes can be found in:
/opt/cray/pe/totalviewsup/2024.1.21/release_notes.md
Installation Instructions
---------------------------
- rpm -ivh totalviewsup-2024.1.21-20240513190631_eed2518e7091-1.el9.x86_64.rpm
To make Totalview 2024.1.21 the default version, execute:
/opt/cray/pe/admin-pe/set_default_files/set_default_totalview_2024.1.21
Legal Notices
---------------------------
Copyright 2007-2024 Hewlett Packard Enterprise Development LP.
CM-PE-INTEGRATION CHANGELOG
===========================
24.07 (2023-06-05)
* Remove rpm queries from 77-deploy-pe.sh for improved performance
* Remove find commands from 77-deploy-pe.sh for improved performance
* Replace the systemd file with a new pe.service that depends on pe.mount
* CAST-35461 fix setting MODULEPATH in csh.cshrc.local
* Replace find commands with for loops to allow using symlink paths in gen_modulefiles.sh
24.03 (2023-02-16)
* Fix regex in gen_modulefiles for Intel OneAPI
* Add a umask 022 to ensure CPE access is not limited
* Add nfs-utils as an rpm dependencies
23.12 (2023-11-27)
* Determine the craype-target architecture during shell initialization
* Add support for compute nodes running on aarch64 hardware
* Update gen_modulefiles.sh to better support new compiler versions
* Add better logging information to `77-deploy-pe`
23.05 (2023-05-10)
* Allow sites to override the lmod install path by setting `lmod_prefix_path`
in `cray-pe-configuration.sh` and `cray-pe-configuration.csh`. Sites can
now use newer versions of lmod provided by the OS distribution.
* Update `76-add-pe-to-fstab` so that `/pe` is not added to `/etc/fstab` more
then once.
23.03 (2023-03-03)
* Spread the squashfs image mounts across NUMA domains to balance memory
usage across NUMA nodes.
* Add RPM dependency on `numactl`
23.02 (2023-01-30)
* Update shell profile scripts to initialize lmod using HPE provided package.
* Remove leading and trailing `:` in `LMOD_SYSTEM_DEFAULT_MODULES` that is no
longer an acceptable format in new versions of lmod.
22.12 (2022-12-12)
* Add a `site_module_list` to `cray-pe-configuration.sh` and
`cray-pe-configuration.csh` to allow sites to add modules loaded at login
without the need to modify HPE defaults `init_module_list`.
22.11 (2022-11-10)
* Update `gen_modulefiles.sh` to recreating symlinks to ensure idempotence.
22.10 (2022-10-03)
* Changes for HPCM 1.8 to ensure that `/pe` is mounted in `76-add-pe-to-fstab`
before running `77-deploy-pe`
22.09 (2022-08-23)
* Update `pe_overlay.sh` to only remove workdirs when unmounts are successful.
* Update `gen_modulefiles.sh` to run `one_off_set_defaults` after generating
all modulefiles.
22.08 (2022-07-22)
* Enable older PE images to deploy on system with newer minor base OS version.
* Update `gen_modulefiles.sh` to use the latest available version of
`craypkg-gen`.
22.05 (2022-04-26)
* Remove `/opt/modulefiles` from `MODULEPATH`.
22.04 (2022-03-28)
* Add a temporary workaround in `pe_overlay.sh` that copies lmod GPU
modulefiles into the PE overlay's upper layer so they are visible after
`/opt/cray/pe` is mounted.
valgrind4hpc 2.13.3
================================================================================
*Released: May 2024*
Product Description
---------------------------
- Valgrind4hpc is a Valgrind-based debugging tool to aid in the detection of memory
leaks and errors in parallel applications. Valgrind4hpc aggregates any duplicate
messages across ranks to help provide an understandable picture of
program behavior. Valgrind4hpc manages starting and redirecting output from many
copies of Valgrind, as well as deduplicating and filtering Valgrind messages.
If your program can be debugged with Valgrind, it can be debugged with Valgrind4hpc.
Changelog
---------------------------
=======================================================================
## [2.13.3] - 2024-04-23 (24.05)
=======================================================================
### Bug Fixes
* PE-51571: Parse arguments before starting network
### Features
* PE-51960: Added support for rhel89/93 & gcc13
* Update Valgrind to 3.22, support syscall userfaultfd
* Build with gcc toolset 12 on redhat
=======================================================================
## [2.13.1] - 2023-07-7 (23.08)
=======================================================================
### Features
* PE-47184: Add valgrind4hpc user guide
* Use native compilers on sles
=======================================================================
## [2.13.0] - 2023-04-11 (23.05)
=======================================================================
### Bug Fixes
* PE-45205: Fix --from-ranks filtering and suppression generation
* Fix install path exported by lmod modulefile on relocated install
### Features
* Build with gcc-toolset-10 on redhat
* PE-45055: Add GDB4hpc connection functionality for VGDB mode
OS Dependencies
---------------------------
- HPE CSM systems with SLES 15 SP4
- HPE HPCM systems with SLES 15 SP5 and RHEL 8.8
- HPE Apollo systems with RHEL 8.8
Product Dependencies
---------------------------
- cray-mrnet >= 5.1, < 6 installed
- cray-cti >= 2.18, < 3 installed
- cdst-support >= 2.14, < 3 installed
Documentation
---------------------------
- See man valgrind4hpc.
Installation Instructions
---------------------------
- rpm -ivh valgrind4hpc-2.13.3-20240508223821_ddb177766849-1.el9.x86_64.rpm
To make valgrind4hpc 2.13.3 the default version of valgrind4hpc, execute:
/opt/cray/pe/admin-pe/set_default_files/set_default_valgrind4hpc_2.13.3
Legal Notices
---------------------------
Certain components, files or programs contained within this package or product are Copyright 2010-2024 Hewlett Packard Enterprise Development LP.
Attribution notices for open source licensed software for this package are detailed in the file:
/opt/cray/pe/valgrind4hpc/2.13.3/ATTRIBUTIONS_valgrind4hpc.txt
CPE Release Notes For SLES 15 SP5
HPE Cray Programming Environment 24.07 Full Release Notes for SLES15 SP5
=========================================================
atp 3.15.4
================================================================================
*Released: June 2024*
Product Description
---------------------------
- Abnormal Termination Processing (ATP) is a system that
monitors user applications, and should an application
take a system trap, ATP preforms analysis on the dying
application. All of the stack backtraces of the
application processes are gathered into a merged
stack backtrace tree and written to disk as the file
"atpMergedBT.dot". The stack backtrace for the first process
to die is sent to stderr as is the number of the signal that
caused the death. If the core file size limit (RLIMIT_CORE)
is non-zero, a heuristically selected set of processes dump
their core.
atpMergedBT.dot can be viewed with 'statview', a component
of the STAT package (module load cray-stat). The merged stack
backtrace tree provides a concise, yet comprehensive, view
of what the application was doing at the time of its death.
Changelog
---------------------------
=======================================================================
## [3.15.4] - 2024-05-2 (24.06)
=======================================================================
### Bug Fixes
* PE-53603: ZMQNet libsodium support
* PE-52560: Pass all ATP environment variables to Flux plugin
* PE-51520: Remove disable message from Flux plugin
* PE-52423: Disable PALS local launch in modulefile
### Features
* PE-53605: Build and package libsodium
* PE-51960: Added support for rhel89/93 & gcc13
* PE-51520: Automatic plugin loading for Flux
=======================================================================
## [3.15.2] - 2023-10-4 (23.11)
=======================================================================
### Bug Fixes
* PE-50510: Set CCE 16 Fortran line number workaround as optional
* PE-49776: Fall back to /tmp if TMPDIR or HOME is invalid
* PE-49611: Update ZMQNet submodule
### Features
* Move to gcc toolset 12 on redhat
* PE-49434: Add function and file to stack trace summary
=======================================================================
## [3.15.1] - 2023-07-19 (23.08)
=======================================================================
### Bug Fixes
* PE-47129: Work around CCE 16 Fortran source file info bug
* PE-47436: Reset Lmod module load variable after use
### Features
* PE-48781: Automatic MRNet load not required due to rpath update
* Use native compilers on sles
* PE-47183: Use RTD style for HTML docs
* PE-47182: Provide web documentation at `share/doc`
=======================================================================
## [3.15.0] - 2023-04-11 (23.05)
=======================================================================
### Bug Fixes
* PE-46283: Only dump requesting processes in user API
* PE-38038: Get more crashing thread info from GDB, correct hidden thread count
* PE-46490: Fix ZMQNet core dump in rank mode
### Features
* PE-47065: Copy signal handler to cross-mounted HOME if running in container
* Build with gcc-toolset-10 on redhat
OS Dependencies
---------------------------
- HPE CSM systems with SLES 15 SP4
- HPE HPCM systems with SLES 15 SP5 and RHEL 8.8
- HPE Apollo systems with RHEL 8.8
Product Dependencies
---------------------------
- cray-mrnet >= 5.1, < 6 installed
- cray-cti >= 2.18, < 3 installed
- cdst-support >= 2.14, < 3 installed
Documentation
---------------------------
- Type `man atp` with the atp module loaded to read the atp(1) man page.
Installation Instructions
---------------------------
- rpm -ivh atp-3.15.4-20240509144552_4766b5970bed-3.sles15sp5.x86_64.rpm
To make atp 3.15.4 the default version of atp, execute:
/opt/cray/pe/admin-pe/set_default_files/set_default_atp_3.15.4
Legal Notices
---------------------------
Certain components, files or programs contained within this package or product are Copyright 2010-2024 Hewlett Packard Enterprise Development LP.
Attribution notices for open source licensed software for this package are detailed in the file:
/opt/cray/pe/atp/3.15.4/ATTRIBUTIONS_atp.txt
cray-ccdb 5.0.4
================================================================================
*Released: May 2024*
Product Description
---------------------------
- CCDB, Cray's next generation debugging tool extends the comparative
debugging capabilities of gdb4hpc with a Graphical User Interface (GUI)
enabling programmers to compare corresponding data structures
between two executing applications. Comparative debugging assists users with
locating sections of code containing data deviations introduced by algorithm
changes, compiler differences, and porting to new architectures/libraries.
Changelog
---------------------------
=======================================================================
## [5.0.4] - 2024-04-22 (24.05)
=======================================================================
### Bug Fixes
* PE-50334: Fix missing console messages
### Features
* PE-51960: Added support for rhel89/93 & gcc13
=======================================================================
## [5.0.3] - 2024-02-7 (24.03)
=======================================================================
### Bug Fixes
* PE-51115: Make socket directory, add PBS allocation instructions
=======================================================================
## [5.0.2] - 2023-11-1 (23.12)
=======================================================================
### Features
* Use gcc toolset 12 on redhat
* Use gcc toolset 12 on redhat
=======================================================================
## [5.0.1] - 2023-07-26 (23.08)
=======================================================================
### Features
* PE-48655: Add option `--unix-socket` for listening on Unix socket
* Use native compiles on sles
* Use native compilers on sles
OS Dependencies
---------------------------
- HPE CSM systems with SLES 15 SP4
- HPE HPCM systems with SLES 15 SP5 and RHEL 8.8
- HPE Apollo systems with RHEL 8.8
Product Dependencies
---------------------------
- gdb4hpc/4.14.x or later installed
Documentation
---------------------------
- Type `man ccdb` with the cray-ccdb module loaded to read the ccdb(1) man page.
- Documentation is available through CCDB from the 'Help' menu.
Installation Instructions
---------------------------
- rpm -ivh --oldpackage cray-ccdb-5.0.4-20240514053816_723809ac94e4-3.sles15sp5.x86_64.rpm
To make cray-ccdb 5.0.4 the default version of cray-ccdb, execute:
/opt/cray/pe/admin-pe/set_default_files/set_default_cray-ccdb_5.0.4
Legal Notices
---------------------------
Certain components, files or programs contained within this package or product are Copyright 2007-2024 Hewlett Packard Enterprise Development LP.
Attribution notices for open source licensed software for this package are detailed in the file:
/opt/cray/pe/ccdb/5.0.4/ATTRIBUTIONS_ccdb.txt
HPE Cray Compiling Environment (CCE) 18.0.0
===========================================
Release Date:
-------------
July 2024
Purpose:
--------
CCE 18.0.0 provides Fortran, C, and C++ compilers for
HPE Cray supercomputer and HPE Apollo 2000 Gen10Plus systems.
Key Changes and Support with CCE 18.0.0:
----------------------------------------
CCE 18.0.0:
-----------
- LLVM 18 base (Merges up to May 18th 2024. LLVM version 18.1.6)
- Split CCE into subpackages
- Updated OMP_PLACE CPU affinity mapping policy to use absolute indices instead of relative
- New "-fopenmp-force-usm" compiler flag (C/C++/Fortran) enables OpenMP unified shared memory (USM) by treating each source file
as though it contains an explicit "omp requires unified_shared_memory" directive
- Added new "compiler_interop" man page documenting CCE compiler interoperability
- Added the following OpenMP 5.1/5.2 Features:
- interop construct and API routines
- compare and fail clause for atomic construct
- present modifier and defaultmap(present)
- nowait clause for taskwait directive
- default(private) and default(firstprivate) (C/C++)
- omp_target_memcpy_async and omp_target_memcpy_rect_async
- nothing directive (C/C++)
- Added the following OpenACC features:
- OpenACC 2.5
- Automatic allocate/deallocate for device_resident/create allocatables/pointers
- Support acc_memcpy_device routine
- Support num_gangs, num_workers and vector_length on kernels directive
- Support async copy routines
- Handle scoping for lexically visible data regions
- OpenACC 2.7
- Snap subarray/subscript expressions for use at the end of data regions
- Reduction on a compute construct implies a copy clause
- OpenACC 3.1
- Implement OpenACC serial
- Added the following Fortran 2023 features
- Control over leading zeros in output of real values: LEADING_ZERO= specifier
- Auto-allocation of character variables in more places - statements
- TYPEOF and CLASSOF declarations
- More use of BOZ constants
- Partial implementation of conditional expressions and arguments
- ENUMERATION TYPE
- SIMPLE attribute for procedures
- RHEL 8.10 x86_64 support
- RHEL 9.4 x86_64 support
- RHEL 9.4 aarch64 support
- OpenMP floating-point atomic optimization hints for AMD MI250X GPUs (Fortran)
- Advanced AMD GPU DWARF support
- Fortran scalar variables and fixed-size arrays in offload regions
- Fortran common blocks
- Optimized NVIDIA Grace CPU Superchip support
User and Application Impact:
----------------------------
CCE 18.0.0:
-----------
Issues or Bugs Resolved:
------------------------
CCE 18.0.0:
-----------
CAST-24911 Incorrect result form ASSOCIATED
CAST-26951 RFE: Need CCE OpenMP OMP_PLACES affinity to be able to map to taskset -c
CAST-30200 ftn compiler error module pointer omp target teams distribute parallel do
CAST-30667 [ELCAP-121] ftn compiler error module pointer omp target teams distribute parallel do
CAST-32453 Cray Fortran compiler is generating wrong assembly with OpenACC
CAST-32454 CCE runtime issue related to explicit type conversion of pointer variables with OpenACC.
CAST-32456 (ELCAP-318) Fortran compiler error in omp declare target subroutine - Elevation of case 5372036627
CAST-32588 Cray Compiler Bug : Issue with polymorphic and character type.
CAST-33337 XC50-CLE-Cray Fortran Compiler- Assignment (=) is not overridden
CAST-34158 crayftn Overloaded Constructor With Polymorphic Argument in Array Constructor
CAST-34265 (ELCAP-466) paradyn target offload built with -O0 gets runtime ACC error
CAST-34571 Internal compiler error with OpenACC declare
CAST-34664 (ELCAP-493) Fortran compiler hangs/crashes on !$omp continuation statements.
CAST-34817 (ELCAP-506) ftn omp target offload runtime error with -O0
CAST-34819 HPECOE-34 CCE Fortran OpenMP offload Bug with Procedure as Argument in Declare Target is unsupported
CAST-34871 (ELCAP-510) CCE17 Fortran crashes
CAST-34910 Incorrect code with floating point optimizations in Cray Fortran
CAST-35003 (ELCAP-522) Minikin - ftn unified memory wrong result for module save variable and declare target
CAST-35004 (ELCAP-527) Minikin runtime error with CCE 17 ftn
CAST-35054 (ELCAP-513) Fortran OMP declare target in module causes runtime error with ROCm-5.7.0+
CAST-35193 Cray Compiler error 'ftn-7060' with debug build of OpenACC code
CAST-35240 Cray EX - several compilation errors compiling vasp with cce17.0.0 [5379756819 Elevation]
CAST-35257 (ELCAP-533) Fortran seg fault with nested module subroutines
CAST-35401 Cray Openmp implementation and buggy thread affinity
CAST-35512 [LUMI] ftn -hfunc_trace crashes the compiler when END statement is not named for modules
CAST-35571 Bug with compiler cce/17.0.0
CAST-35625 No source line number displayed when run with MemorySanitizer in PrgEnv-cray
CAST-35726 ftn-7206 warning in cce/17.0
CAST-35776 crayftn ICE when trying to build dftd4
CAST-35903 (ELCAP-596) Fortran compiler segfault with OpenMP and requires statement
CAST-35946 (ELCAP-610) unified memory runtime error using ftn allocatable array from:module
CAST-35963 (ELCAP-615) Paradyn ftn openmp problem: bad size of allocated pointer with omp requires unified_shared_memory
Product and OS Dependencies:
----------------------------
This CCE release is supported on
- HPE Cray supercomputer systems running CSM with COS 2.5.X (SLES15 SP4) or COS 23.11 (SLES15 SP5)
- HPE Cray supercomputer systems running HPCM with COS 2.5.X (SLES15 SP4) or COS 23.11 (SLES15 SP5)
- HPE Cray supercomputer systems running HPCM with RHEL 8.10
- HPE Apollo 2000 Gen10Plus systems running RHEL 8.9
- AMD GPU offloading support requires at least ROCm 6.0 and is tested up to ROCm 6.1
Notes and Limitations:
----------------------
Documentation:
--------------
- S-5212 Cray Compiling Environment Release Overview (18.0)
- S-3901 Cray Fortran Reference Manual
- S-2179 Cray C and C++ Quick Reference
- Basic man pages: crayftn(1), craycc(1), crayCC(1), intro_openmp(7), intro_openacc(7)
- Please see https://clang.llvm.org/docs/UsersManual.html or use the -help command line option for
more information on using Clang
Modulefile:
-----------
The following will load the modules necessary to use CCE:
module load PrgEnv-cray
The following will switch to x.y.z version of CCE:
module swap cce cce/x.y.z
Installation instructions:
--------------------------
rpm -ivh cce-18.0.0-202406031331.98280ea519a0c-4.sles15sp4.x86_64.rpm.x86_64.rpm
The following script will set CCE version 18.0.0 default:
/opt/cray/pe/admin-pe/set_default_files/set_default_cce_18.0.0
License:
--------
Except for the third party components and software licensed by HPE through
proprietary agreements, components, files or programs contained within this
package or product are Copyright -2024 Hewlett Packard Enterprise Development LP.
Attribution notices for open source licensed software for this
package are detailed in the file:
/opt/cray/pe/cce/18.0.0/ATTRIBUTIONS_18.0.txt
cpe-descriptive-manifest 24.07
=================================
Release Date: Jul 2024
Product description:
--------------------
The cpe-descriptive-manifest packages contain the JSON manifest files that
are to map CPE to a format that Spack can consume.
Schema Changelog:
-----------------
v1.4
----
* Add 'wrappers' as a boolean parameter
v1.3
----
* Change '_meta' opject's 'version' property to 'schema-version'.
* Add 'cpe-version' property '_meta' object.
v1.2
----
* Add 'system-type' property to the '_meta' object.
v1.1
----
* Add '_meta' object to define file type and schema version.
v1.0
----
* Initial schema.
Cray PE DL Plugin with Resiliency Support 22.06.1.2:
Release Date:
-------------
June 2022
Purpose:
--------
The following changes have been made since Cray PE DL Plugin 21.04.1:
o Updated DL framework support
o Introduction of resiliency support
o BFloat16 data type support
Supported Configurations:
-------------------------
Shasta/EX: CPU and AMD GPU support with PrgEnv-gnu
The following Deep Learning frameworks and versions are supported:
* TensorFlow v2.6
* PyTorch v1.10.2
Documentation:
--------------
For more information see the intro_dl_plugin man pages.
For info regarding use of the new resiliency feature, see the examples
and README.md in ${CRAYPE_ML_PLUGIN_BASEDIR}/examples/resiliency
Sample Installation instructions:
---------------------------------
Note: Package must be installed or sym linked in the default /opt/cray/pe path.,
ie: use of the --prefix flag for rpm is not supported.
rpm -ivh craype-dl-plugin-ftr-22.06.1.2*.x86_64.rpm
The "*" in the install command represents date and hash specific information.
Modulefile:
-----------
Shasta/EX:
module load craype-dl-plugin-ftr/22.06.1.2
Certain components, files or programs contained within this package or
product are Copyright 2017-2022 Hewlett Packard Enterprise Development LP.
Cray PE DL Plugin 21.02.1.3:
Release Date:
-------------
February 2021
Purpose:
--------
The following changes have been made since Cray PE DL Plugin 20.10.1:
o Bug fixes for CUDA 11 support and pip install of include source dist
o Initial CUDA 11.1 support
o Updated DL framework support
Supported Configurations:
-------------------------
CS: OpenMPI 4.0.2 and cray-mvapich 2.3.2 or newer with CPU and GPU support
XC CLE 7: CPU and GPU support
Shasta: CPU-only support
Apollo: CPU-only support
The following Deep Learning frameworks and versions are supported:
* TensorFlow v2.4
* PyTorch v1.7.1
Documentation:
--------------
For more information see the intro_dl_plugin man pages.
Known Issues:
-------------
* Using Tensorflow 1.14 binaries distributed by Google and Intel require portions of the DL
Plugin to be compiled with gcc 4.8. If using the included source distribution of the
DL Plugin to install the Python packagesinto a given Python installation, gcc 4.8
will need to be used to install said packages, assuming Google and Intel distributed
Tensorflow 1.14 binaries are used. For TensorFlow 1.15 and greater, use of gcc 7.x
is required instead.
Sample Installation instructions:
rpm -ivh craype-dl-plugin-py3-21.02.1.3*.x86_64.rpm
The "*" in the install command represents date and hash specific information.
Modulefile:
-----------
XC/Shasta:
module load craype-dl-plugin-py3/21.02.1.3
CS:
module load craype-dl-plugin-py3/openmpi/21.02.1.3
module load craype-dl-plugin-py3/mvapich/21.02.1.3
Certain components, files or programs contained within this package or
product are Copyright 2017-2021 Hewlett Packard Enterprise Development LP.
Cray PE DL Plugin 22.06.1.2:
Release Date:
-------------
June 2022
Purpose:
--------
The following changes have been made since Cray PE DL Plugin 22.04.1.1:
o Updated DL framework support
o BFloat16 data type support
o ROCm 5.x support
Supported Configurations:
-------------------------
CS: OpenMPI 4.0.2 and cray-mvapich 2.3.2 or newer with CPU and GPU support
XC CLE 7: CPU and GPU support
Shasta: CPU-only support
Apollo: CPU-only support
The following Deep Learning frameworks and versions are supported:
* TensorFlow v2.6
* PyTorch v1.10
Documentation:
--------------
For more information see the intro_dl_plugin man pages.
Known Issues:
-------------
* Using Tensorflow 1.14 binaries distributed by Google and Intel require portions of the DL
Plugin to be compiled with gcc 4.8. If using the included source distribution of the
DL Plugin to install the Python packagesinto a given Python installation, gcc 4.8
will need to be used to install said packages, assuming Google and Intel distributed
Tensorflow 1.14 binaries are used. For TensorFlow 1.15 and greater, use of gcc 7.x
is required instead.
Sample Installation instructions:
rpm -ivh craype-dl-plugin-py3-22.06.1.2*.x86_64.rpm
The "*" in the install command represents date and hash specific information.
Modulefile:
-----------
XC/Shasta:
module load craype-dl-plugin-py3/22.06.1.2
CS:
module load craype-dl-plugin-py3/openmpi/21.04.1
module load craype-dl-plugin-py3/mvapich/21.04.1
Certain components, files or programs contained within this package or
product are Copyright 2017-2021 Hewlett Packard Enterprise Development LP.
Cray PE DL Plugin 22.08.1:
Release Date:
-------------
August 2022
Purpose:
--------
The following changes have been made since Cray PE DL Plugin 22.06.1.2:
o PyTorch Apex Amp Optimization Support
Supported Configurations:
-------------------------
CS: OpenMPI 4.0.2 and cray-mvapich 2.3.2 or newer with CPU and GPU support
XC: CLE 7 CPU and GPU support
EX: CPU and GPU support
Apollo: CPU-only support
The following Deep Learning frameworks and versions are supported:
* TensorFlow v2.6
* PyTorch v1.10
Documentation:
--------------
For more information see the intro_dl_plugin man pages.
Known Issues:
-------------
* Using Tensorflow 1.14 binaries distributed by Google and Intel require portions of the DL
Plugin to be compiled with gcc 4.8. If using the included source distribution of the
DL Plugin to install the Python packagesinto a given Python installation, gcc 4.8
will need to be used to install said packages, assuming Google and Intel distributed
Tensorflow 1.14 binaries are used. For TensorFlow 1.15 and greater, use of gcc 7.x
is required instead.
Sample Installation instructions:
rpm -ivh craype-dl-plugin-py3-22.08.1*.x86_64.rpm
The "*" in the install command represents date and hash specific information.
Modulefile:
-----------
XC/EX:
module load craype-dl-plugin-py3/22.08.1
CS:
module load craype-dl-plugin-py3/openmpi/21.04.1
module load craype-dl-plugin-py3/mvapich/21.04.1
Certain components, files or programs contained within this package or
product are Copyright 2017-2021 Hewlett Packard Enterprise Development LP.
Cray PE DL Plugin 22.09.1:
Release Date:
-------------
September 2022
Purpose:
--------
The following changes have been made since Cray PE DL Plugin 22.08.1:
o Keras callbacks bug fixes
Supported Configurations:
-------------------------
CS: OpenMPI 4.0.2 and cray-mvapich 2.3.2 or newer with CPU and GPU support
XC: CLE 7 CPU and GPU support
EX: CPU and GPU support
Apollo: CPU-only support
The following Deep Learning frameworks and versions are supported:
* TensorFlow v2.6
* PyTorch v1.10
Documentation:
--------------
For more information see the intro_dl_plugin man pages.
Known Issues:
-------------
* Using Tensorflow 1.14 binaries distributed by Google and Intel require portions of the DL
Plugin to be compiled with gcc 4.8. If using the included source distribution of the
DL Plugin to install the Python packagesinto a given Python installation, gcc 4.8
will need to be used to install said packages, assuming Google and Intel distributed
Tensorflow 1.14 binaries are used. For TensorFlow 1.15 and greater, use of gcc 7.x
is required instead.
Sample Installation instructions:
rpm -ivh craype-dl-plugin-py3-22.09.1*.x86_64.rpm
The "*" in the install command represents date and hash specific information.
Modulefile:
-----------
XC/EX:
module load craype-dl-plugin-py3/22.09.1
CS:
module load craype-dl-plugin-py3/openmpi/21.04.1
module load craype-dl-plugin-py3/mvapich/21.04.1
Certain components, files or programs contained within this package or
product are Copyright 2017-2021 Hewlett Packard Enterprise Development LP.
Cray PE DL Plugin 22.12.1:
Release Date:
-------------
December 2022
Purpose:
--------
The following changes have been made since Cray PE DL Plugin 22.09.1:
o TF v2.9 Support
Supported Configurations:
-------------------------
CS: OpenMPI 4.0.2 and cray-mvapich 2.3.2 or newer with CPU and GPU support
XC: CLE 7 CPU and GPU support
EX: CPU and GPU support
Apollo: RHEL8.6 CPU and GPU support
The following Deep Learning frameworks and versions are supported:
* TensorFlow v2.9
* PyTorch v1.10
Documentation:
--------------
For more information see the intro_dl_plugin man pages.
Known Issues:
-------------
* Using Tensorflow 1.14 binaries distributed by Google and Intel require portions of the DL
Plugin to be compiled with gcc 4.8. If using the included source distribution of the
DL Plugin to install the Python packagesinto a given Python installation, gcc 4.8
will need to be used to install said packages, assuming Google and Intel distributed
Tensorflow 1.14 binaries are used. For TensorFlow 1.15 and greater, use of gcc 7.x
is required instead.
Sample Installation instructions:
rpm -ivh craype-dl-plugin-py3-22.12.1*.x86_64.rpm
The "*" in the install command represents date and hash specific information.
Modulefile:
-----------
XC/EX:
module load craype-dl-plugin-py3/22.12.1
CS:
module load craype-dl-plugin-py3/openmpi/21.04.1
module load craype-dl-plugin-py3/mvapich/21.04.1
Certain components, files or programs contained within this package or
product are Copyright 2017-2021 Hewlett Packard Enterprise Development LP.
Cray PE DL Plugin 23.09.1:
Release Date:
-------------
September 2023
Purpose:
--------
The following changes have been made since Cray PE DL Plugin 22.12.1:
o ROCm 5.3.x support
o TF v2.11 Support
o PT v1.12 Support
Supported Configurations:
-------------------------
CS: OpenMPI 4.0.2 and cray-mvapich 2.3.2 or newer with CPU and GPU support
XC: CLE 7 CPU and GPU support
EX: CPU and GPU support for both AMD and NVIDIA
Apollo: RHEL8.6 CPU and GPU support
The following Deep Learning frameworks and versions are supported:
* TensorFlow v2.11
* PyTorch v1.12
Documentation:
--------------
For more information see the intro_dl_plugin man pages.
Known Issues:
-------------
* Using TensorFlow, if you see an exception error related to apply_gradients() then
downgrade your keras installation version in your python environment.
pip install keras==2.10.0
Sample Installation instructions:
rpm -ivh craype-dl-plugin-py3-23.09.1*.x86_64.rpm
The "*" in the install command represents date and hash specific information.
Modulefile:
-----------
XC/EX:
module load craype-dl-plugin-py3/23.09.1
CS:
module load craype-dl-plugin-py3/openmpi/21.04.1
module load craype-dl-plugin-py3/mvapich/21.04.1
Certain components, files or programs contained within this package or
product are Copyright 2017-2021 Hewlett Packard Enterprise Development LP.
HPE Cray DL Plugin 24.03.1:
Release Date:
-------------
March 2024
Purpose:
--------
The following changes have been made since HPE Cray DL Plugin 23.09.1:
o AMD MI300A APU support
o TensorFlow v2.15 support
o PyTorch v2.0.1 support
o ROCm 6.0.x support
o Cray-mpich 8.1.27 support
o GCC 12.2.0 support
Supported Configurations:
-------------------------
CS: OpenMPI 4.0.2 and cray-mvapich 2.3.2 or newer with CPU and GPU support
XC: CLE 7 CPU and GPU support
EX: CPU and GPU support for both AMD and NVIDIA
Apollo: RHEL8.6 CPU and GPU support
The following Deep Learning frameworks and versions are supported:
* TensorFlow v2.15
* PyTorch v2.0.1
Documentation:
--------------
For more information see the intro_dl_plugin man pages.
Known Issues:
-------------
* Using TensorFlow, if you see an exception error related to apply_gradients() then
downgrade your keras installation version in your python environment.
pip install keras==2.10.0
Sample Installation instructions:
rpm -ivh craype-dl-plugin-py3-24.03.1*.x86_64.rpm
The "*" in the install command represents date and hash specific information.
Modulefile:
-----------
XC/EX:
module load craype-dl-plugin-py3/24.03.1
CS:
module load craype-dl-plugin-py3/openmpi/21.04.1
module load craype-dl-plugin-py3/mvapich/21.04.1
Certain components, files or programs contained within this package or
product are Copyright 2017-2021 Hewlett Packard Enterprise Development LP.
CrayPE Targets 1.13.2
================================================================================
Release Date:
--------------------------------------------------------------------------------
July, 2024
Purpose:
--------------------------------------------------------------------------------
Adds modulefiles for targets - craype-accel-amd-gfx942
Bugs fixed in this release:
--------------------------------------------------------------------------------
Updates dependencies for NVIDIA accelerators
Dependencies:
--------------------------------------------------------------------------------
The CrayPE Targets 1.13.2 release is dependent on the following
software products:
CrayPE
set_default_3
lmod_scripts
Documentation:
--------------------------------------------------------------------------------
See manpages for cc, CC, ftn, intro_craype-api, intro_hugepages and pkg-config
See section 2.6 Using Targeting Modules of the Cray Programming Environment
User's Guide (S-2529-116)
See http://www.freedesktop.org/wiki/Software/pkg-config for a pkg-config
introduction.
Installation instructions:
--------------------------------------------------------------------------------
rpm -Uvh craype-targets-ex-1.13.2-202406062350.172a8f0148d89-0.ex.noarch.rpm
To make this the default version, execute:
/opt/cray/pe/admin-pe/set_default_files/set_default_craype-targets_1.13.2-tcl
/opt/cray/pe/admin-pe/set_default_files/set_default_craype-targets_1.13.2-lua
Product description:
--------------------------------------------------------------------------------
CrayPE Targets contains the modules for CrayPE targets.
TCL Modules - craype-accel-host craype-accel-amd-gfx90a craype-accel-amd-gfx908 craype-accel-amd-gfx940 craype-accel-amd-gfx942 craype-accel-nvidia70 craype-accel-nvidia80 craype-accel-nvidia90 craype-hugepages2M craype-hugepages4M craype-hugepages8M craype-hugepages16M craype-hugepages32M craype-hugepages64M craype-hugepages128M craype-hugepages256M craype-hugepages512M craype-hugepages1G craype-hugepages2G craype-arm-grace craype-x86-milan craype-x86-milan-x craype-x86-rome craype-x86-spr craype-x86-spr-hbm craype-x86-trento craype-x86-genoa craype-network-none craype-network-ofi craype-network-ucx
Lmod Modules - craype-accel-host craype-accel-amd-gfx90a craype-accel-amd-gfx908 craype-accel-amd-gfx940 craype-accel-amd-gfx942 craype-accel-nvidia70 craype-accel-nvidia80 craype-accel-nvidia90 craype-hugepages2M craype-hugepages4M craype-hugepages8M craype-hugepages16M craype-hugepages32M craype-hugepages64M craype-hugepages128M craype-hugepages256M craype-hugepages512M craype-hugepages1G craype-hugepages2G craype-arm-grace craype-x86-milan craype-x86-milan-x craype-x86-rome craype-x86-spr craype-x86-spr-hbm craype-x86-trento craype-x86-genoa craype-network-none craype-network-ofi craype-network-ucx
Copyright 2013-2023 Hewlett Packard Enterprise Development LP
CDEUX driver (craype) 2.7.32 - 3
==============
Release Date:
--------------
July 2024
Purpose:
--------
The Cray Development Environment User eXperience driver (CDEUX) (craype)
provides a common set of drivers for all CPE programming environments,
notably the C/C++ (cc/CC) and Fortran (ftn) languages.
New CDEUX driver (craype) features for HPE Cray EX and Apollo systems:
- Support for newer Red Hat Enterprise Linux versions
Key Changes and Bugs Closed:
----------------------------
- Fix loading a perftools module breaks "direct/moduleless" craycc
- Fix cc fails to create shared library in PrgEnv-gnu-amd
- Add missing documentation about RPATHs
Product and OS Dependencies:
----------------------------
The CDEUX driver (craype) 2.7.32 release is supported on the following
HPE systems:
* HPE Cray EX systems with CLE
* HPE Apollo systems as part of the Cray Programming Environment
Documentation:
---------------
See manpages for cc, CC, ftn, and intro_craype-api.
See the pkg-config manpage or
http://www.freedesktop.org/wiki/Software/pkg-config for a pkg-config
introduction.
Product description:
--------------------
CrayPE contains drivers, cc, CC, and ftn to compile for all CPE
Programming Environments.
Certain components, files or programs contained within this package or product
are Copyright 2013-2023 Hewlett Packard Enterprise Development LP.
License:
--------
Except for the third party components and software licensed by HPE
through proprietary agreements, components, files or programs contained
within this package or product are Copyright 2023 Hewlett Packard
Enterprise Development LP.
Craypkg-gen 1.3.33
==============
Release date
------------
June 2024
Purpose
-------
Bug fix release.
Bugs fixed in this release
--------------------------
- Add CUDA modulefiles and pkg-config file support
- Update the NVIDIA HPC SDK modulefiles to more closely resemble the AMD toolkit modules
Dependencies
------------
The craypkg-gen 1.3.33 release is supported on the following HPE
Cray systems:
- Cray EX systems SLE 15.0 or later and RHEL 8.0 or later.
Driver support for integrating Third Party C, C++, and Fortran libraries
through .pc files using pkg-config is used in CrayPE 2.x and later.
Limitation
----------
- Library dependencies for static libraries are not added to the .pc files
for keywords Requires.private and Libs.private. A warning is issued by
the craypkg-gen tool to advise users to add this information to the
libraries .pc files.
- RPM limits packages to 4GB
Documentation
-------------
Man pages for craypkg-gen are found by executing `module load craypkg-gen`
and then `man craypkg-gen`. See
http://www.freedesktop.org/wiki/Software/pkg-config for a pkg-config
introduction.
Examples for creating modulefiles for Intel, PGI and Python are included in
the craypkg-gen ‘doc’ directory:
/opt/cray/craypkg-gen/1.3.33/doc/intel_example.txt
/opt/cray/craypkg-gen/1.3.33/doc/pgi_example.txt
/opt/cray/craypkg-gen/1.3.33/doc/python_example.txt
Example:
- As an example, the Intel 16.0.3.210 compiler was recently released.
After installing the compiler the administrator creates a modulefile for
this release by executing the following commands:
# module load craypkg-gen
# craypkg-gen -m /opt/intel/compilers_and_libraries_2016.3.210
This version of the Intel compiler is made default by executing the
command:
# /opt/admin-pe/set_default_craypkg/set_default_intel_16.0.3.210
Installation instructions
-------------------------
rpm -ivh craypkg-gen-1.3.33-1.3.33-1.sles15sp5.noarch.rpm
To change the product version to default after installation:
/opt/cray/pe/admin-pe/set_default_files/set_default_craypkg-gen_1.3.33
Product description
-------------------
The craypkg-gen 1.3.33 utility provides the system administrator
a tool to integrate third party software with the Cray software stack.
Craypkg-gen assists with integration by creating .pc files for C, C++, and
Fortran libraries, pkg-config enabled modulefiles and RPMs.
The workflow for using craypkg-gen is
1) Build the open source software
2) Create .pc files for libraries
3) Create pkg-config enabled modulefiles
4) Customize pkg-config and modulefiles if needed
Certain components, files or programs contained within this package or product
are Copyright 2013-2024 Hewlett Packard Enterprise Development LP.
cray-cti 2.18.4
================================================================================
*Released: June 2024*
Product Description
---------------------------
- Cray Common Tools Interface (CTI) is a library that abstracts debugger tool support
to a common interface regardless of the underlying work load manager. It facilitates
the staging of files and the launch of tool daemons on the compute nodes associated
with a parallel job. Options and interfaces can be found in the cti(1) and cti(3)
manpages.
Known Issues
---------------------------
- Race condition workaround for PALS 1.2.3 in CTI 2.17.0 (22.11). In PALS 1.2.3, there
is a race condition between the tool launcher releasing a job from the startup barrier
and the job actually getting to the startup barrier. This can result in the job receiving
the startup barrier release signal before it actually arrives there, resulting in the job
getting stuck in the barrier. As a workaround, the environment variable
CTI_PALS_BARRIER_RELEASE_DELAY can be set to add a delay between job startup and barrier
release. If set to a positve integer n, CTI will wait n seconds between starting a
job and releasing it from the barrier on PALS. A delay as small as one second works in
most cases.
Notes
---------------------------
- CTI 2.17.1 (PE 22.12) contains a new feature, file deduplication, for Slurm and PALS
workload managers. If enabled, CTI will run a check on compute nodes to determine if files
can be excluded from the shipped package. This can significantly improve application startup
time. To enable, set the environment variable CTI_DEDUPLICATE_FILES=1.
Changelog
---------------------------
=======================================================================
## [unreleased]
=======================================================================
### Bug Fixes
* PE-54265: Don't add quotes to LD_PRELOAD
* PE-54079: Fix unreliable Flux application output
=======================================================================
## [2.18.4] - 2024-05-3 (24.06)
=======================================================================
### Bug Fixes
* PE-53554: Allow launch if debug log directory doesn't exist
### Features
* Add lock to C interface, basic multithread test
* PE-53555: Parse multiple Flux job ID formats
* PE-52995: Support Flux 0.60.0
* PE-51960: Added support for rhel89/93 & gcc13
=======================================================================
## [2.18.3] - 2024-02-15 (24.03)
=======================================================================
### Bug Fixes
* PE-52423: Fix deduplication check for PALS
* PE-52423: Disable local launch for PALS
* PE-52540: Always include rank 0 in Flux launch
* PE-51694: Flux: fix job layout case and file deduplication
* PE-51955: Only rebuild checksum if input files changed
* PE-51495: Fix Slurm isRunning for stopped srun and backgrounded sattach
* PE-51694: Fix Flux implementation for 0.57.0
* PE-51193: Support rank remapping in PALS backend
### Features
* PE-48498: Accept PALS attach in form pbs_job_id:pals_apid
=======================================================================
## [2.18.2] - 2023-11-3 (23.12)
=======================================================================
### Bug Fixes
* Copy rank list in cti_getAppBinaryList
* PE-49914: Verify cminfo address on HPCM systems
* PE-49857: Narrow Slurm multi-cluster check
* PE-49079: Support attach for non-MPI PALS applications
* PE-49585: Detect HSN address on HPCM Slurm and use node name provided by Slurmd
### Features
* PE-50581: Support Eproxy/Slurm systems
* PE-50356: Automatically set Slurm daemon GPU GRES
* PE-50201: Add CTI_SSH_PORT option for SSH implementation
* PE-49299: Slurm MPMD launch and attach support
* Move to gcc toolset 12
OS Dependencies
---------------------------
- HPE CSM systems with SLES 15 SP4
- HPE HPCM systems with SLES 15 SP5 and RHEL 8.8
- HPE Apollo systems with RHEL 8.8
Product Dependencies
---------------------------
- cray-cdst-support >= 2.14, < 3 installed
- cray-dyninst 12.3.0 or later installed
Documentation
---------------------------
- Currently the only documentation available is found in common_tools_fe.h, common_tools_be.h,
and the tests in the 'tests' directory.
Installation Instructions
---------------------------
- rpm -ivh cray-cti-2.18.4-20240523151820_b5630478cfc3-9.sles15sp5.x86_64.rpm
To make cray-cti 2.18.4 the default version of cray-cti, execute:
/opt/cray/admin-pe/set_default_files/set_default_cray-cti-2.18.4
Legal Notices
---------------------------
Certain components, files or programs contained within this package or product are Copyright 2010-2024 Hewlett Packard Enterprise Development LP. SPDX-License-Identifier: Linux-OpenIB
Attribution notices for open source licensed software for this package are detailed in the file:
/opt/cray/pe/cti/2.18.4/ATTRIBUTIONS_cti.txt
Cray DSMML 0.3.0:
==========================
Release Date:
-------------
March, 2024
Product Information:
--------------------
DSMML is a stand-alone memory management library for maintaining distributed
shared symmetric memory heaps for top level PGAS languages and libraries like
Coarray Fortran, UPC, and OpenSHMEM. DSMML allows user libraries to create
multiple symmetric heaps and share information with other libraries. Through
DSMML, we could extract interoperability between PGAS programming models.
Announcements, release informations, supported environments, and backward
compatibility informations about this product can be viewed in the following
location: https://pe-cray.github.io/cray-dsmml/
Purpose:
--------
Cray DSMML 0.3.0 supports the following features:
- Create symmetric heap segments for being used by PGAS languages and
libraries
- Allows options to allocate, deallocate memory buffers from previously
created symmetric heap segments
- Enables sharing information about the symmetric heap segments between
different programming models
- Added support for shared symmetric heap (SSHEAP) that can be used to
provide enhanced SMP data transport in user-libraries like SHMEM, CAF, UPC
The following features are fixed in Cray DSMML 0.3.0 when compared
to Cray DSMML 0.2.2:
- Add support for the library for AARCH64 processor based platforms
- Internal bug fixes involving hugepage cleanups
Refer intro_dsmml(3) manpage for more information on the supported features
and syntax for different routines
Product and OS Dependencies:
----------------------------
The Cray DSMML 0.3.0 release is supported on the following
Cray systems:
* HPE Cray EX systems with CLE
Product and OS Dependencies by network type:
--------------------------------------------------+
| Shasta |
----------------------------+---------------------+
craype | >= 2.7.3 |
----------------------------+---------------------+
One or more compilers:
* AOCC 2.2 or later
* CCE 9.1 or later
* GNU 9.1 or later
* Intel 19.0 or later
* Nvidia 20.7 or later
Documentation:
--------------
Use the Cray DSMML man pages for more information on the library
functions and use intro_dsmml(3) man page for general information.
Use https://pe-cray.github.io/whitepapers/ for access to different
Cray DSMML specific whitepapers
Announcements, release informations, supported environments, and backward
compatibility informations about this product can be viewed in the following
location: https://pe-cray.github.io/cray-dsmml/
Modulefile:
-----------
module load cray-dsmml/0.3.0
Certain components, files or programs contained within this package or
product have the following Copyright:
Copyright 2018-2021 Hewlett Packard Enterprise Development LP.
cray-dyninst 12.3.2
================================================================================
*Released: May 2024*
Product Description
---------------------------
- cray-dyninst provides tools for binary instrumentation, analysis, and modification.
Changelog
---------------------------
=======================================================================
## [12.3.2] - 2024-04-22 (24.05)
=======================================================================
### Features
* PE-51960: Added support for rhel89/93 & gcc13
=======================================================================
## [12.3.1] - 2023-09-25 (23.10)
=======================================================================
### Features
* Use gcc toolset 12 on redhat
=======================================================================
## [12.3.0] - 2023-06-27 (23.07)
=======================================================================
### Features
* PE-48354: Add LD_LIBRARY_PATH to Modulefiles
* PE-48354: Update Dyninst to 12.3.0
* Use native compilers on sles
=======================================================================
## [12.2.0] - 2023-04-10 (23.05)
=======================================================================
### Features
* Build with gcc-toolset-10 on redhat
OS Dependencies
---------------------------
- HPE CSM systems with SLES 15 SP4
- HPE HPCM systems with SLES 15 SP5 and RHEL 8.8
- HPE Apollo systems with RHEL 8.8
Product Dependencies
---------------------------
- cdst-support >= 2.14, < 3 installed
Documentation
---------------------------
- https://github.com/dyninst/dyninst
Installation Instructions
---------------------------
- rpm -ivh cray-dyninst-12.3.2-20240506150030_8668bb4a546f-3.sles15sp5.x86_64.rpm
To make this the default version of cray-dyninst, execute:
/opt/cray/pe/admin-pe/set_default_files/set_default_cray-dyninst_12.3.2
Legal Notices
---------------------------
Certain components, files or programs contained within this package or product are Copyright 2007-2024 Hewlett Packard Enterprise Development LP.
Attribution notices for open source licensed software for this package are detailed in the file:
/opt/cray/pe/dyninst/12.3.2/ATTRIBUTIONS_dyninst.txt
FFTW 3.3.10.8
============
Release Date:
-------------
July 2024
Purpose:
--------
This Cray FFTW 3.3.10.8 release is supported on HPE/Cray EX and
HPE/Cray XD/Apollo systems. Cray FFTW is supported on the host CPU
but not on the accelerator of HPE/Cray systems.
The Cray FFTW 3.3.10.8 release provides the following:
- Compiler and dependency updates
See the Product and OS Dependencies section for details.
Product and OS Dependencies:
----------------------------
This Cray FFTW 3.3.10.8 release is supported on these HPE/Cray systems:
- HPE/Cray EX Systems with SLES 15 SP5, RHEL 8.10, or RHEL 9.4 based OS
- HPE/Cray XD/Apollo Systems with RHEL 8.10 or RHEL 9.4 based OS
This Cray FFTW 3.3.10.8 release requires the following software products:
craype 2.7.32 or later
cray-mpich 8.1.30 or later
One or more of the following major compiler versions:
CCE 18
GCC 13
AOCC 4
Nvidia 24.3 or later
AMD (ROCm) 6
Intel 2024 or later
Notes and Limitations:
----------------------
Starting with cray-fftw/3.3.6.1 the fftw module has been renamed from fftw
to cray-fftw.
Documentation:
--------------
http://www.fftw.org/index.html#documentation
See the intro_fftw3 manual page for additional information.
Modulefile:
-----------
module load cray-fftw
Installation
------------
To install the rpm, execute:
rpm -ivh cray-fftw-3.3.10.8-202406131706.ee3ebe23b54c9-1.sles15sp5.x86_64.rpm
To make this the default version, execute:
/opt/cray/pe/admin-pe/set_default_files/set_default_fftw_3.3.10.8
Certain components, files or programs contained within this package or
product are Copyright 2011-2024 Hewlett Packard Enterprise Development LP
forgesup 23.1.2
================================================================================
*Released: May 2024*
Product Description
---------------------------
- Provides the HPE Forge support package which contains modulefiles (tcl & lua),
set_default files, and content projection files for Forge 23.1.2.
- The only supported install directory for forgesup is /opt/cray/pe.
- The only supported install directory for Lenaro forge is /opt/forge.
- Drop the license file into: /opt/forge/23.1.2/
- Currently, forge-23.1.2 by default drops logfiles into the $HOME directory
which is not projected on a EX system. In order for forge-23.1.2 to successfully
launch an application on EX perform the following configuration:
export ALLINEA_SESSION_PATH=ProjectedDirectory
e.g:
export ALLINEA_SESSION_PATH=/lus/USERNAME
Changelog
---------------------------
=======================================================================
## [23.1.2] - 2024-04-26 (24.05)
=======================================================================
### Features
* PE-53213: Updated Forgesup to support Forge 23.1.2
=======================================================================
## [23.1] - 2024-02-29 (24.03)
=======================================================================
### Features
* PE-49958: Implemented Auto-gen release notes/changelog tool
* Updated supported version to 23.1
OS Dependencies
---------------------------
- HPE CSM systems with SLES 15 SP4
- HPE HPCM systems with SLES 15 SP5 and RHEL 8.8
- HPE Apollo systems with RHEL 8.8
Product Dependencies
---------------------------
- This forgesup 23.1.2 package has a dependency on Forge 23.1.2.
Documentation
---------------------------
- The Forge user guide can be found in /opt/forge/23.1.2/doc/userguide-forge.pdf
- The Forge release notes can be found in /opt/forge/23.1.2/doc/RELEASE-NOTES
- The Forgesup release notes can be found in /opt/cray/pe/forgesup/23.1.2/
Installation Instructions
---------------------------
- rpm -ivh forgesup-23.1.2-20240513190506_dd371b3b512d-1.sles15sp5.x86_64.rpm
To make forge 23.1.2 the default version, execute:
/opt/cray/pe/admin-pe/set_default_files/set_default_forge_23.1.2
Legal Notices
---------------------------
Copyright 2007-2024 Hewlett Packard Enterprise Development LP.
gdb4hpc 4.16.2
================================================================================
*Released: June 2024*
Product Description
---------------------------
- gdb4hpc is a GDB-based parallel debugger used to debug applications compiled with
CCE, AOCC, GNU, and Intel Fortran, C and C++ compilers. It allows programmers to
either launch an application or attach to an already running application that
was launched on the system. Additionally, it provides comparative debugging
technology that enables programmers to compare data structures between two
executing applications. Comparative debugging should be used in conjunction
with the CCDB GUI tool accessed by loading the cray-ccdb module.
- Some features of gdb include:
* Command line parallel debugger allows for launching/attaching applications.
* Utilizes process sets to operate on a subset of application ranks.
* GDB-like feel, also implements a gdbmode to enable a true parallel gdb.
* GPU debugging is supported for OpenMP 4.0+ GPU directives; OpenMP 5.0 on AMD
and Nvidia GPU's.
* Workload manager support via Common Tools Interface (cray-cti).
Changelog
---------------------------
=======================================================================
## [4.16.2] - 2024-05-2 (24.06)
=======================================================================
### Bug Fixes
* PE-52737: Fix backend crash when debugging Intel OneAPI GPUs
### Features
* PE-47910: Run job cleanup in background
* PE-53716: Added user guide for using the Python gdb extension
* PE-51960: Added support for rhel89/93 & gcc13
* PE-53828: Implement Flux attach
* Automatically load python extensions when debugger starts
* PE-53100: Support for aggregated gdb python extension commands py-print and py-locals if underlying gdb has python extensions loaded
* PE-53100: Support for aggregated gdb python extension commands py-bt, py-list, py-up, and py-down if underlying gdb has python extensions loaded
* Handle "~" in the --debug launch argument
* PE-52589: Tweak default assertion script behavior - now assertion scripts stop iff an assertion fails default
* PE-51936: Added tab completion for the print command
=======================================================================
## [4.16.1] - 2024-02-7 (24.03)
=======================================================================
### Bug Fixes
* PE-52387: Fix segfault from list command
* PE-51694: Fix occasional crash when operating on a subset of ranks
* PE-44112: Track job status while running for quicker timeout and exit
* Fix recursive source command behavior
* Properly format /csv output with data over 1 million items
* PE-51165: Query ZMQNet for correct rank and target PID
### Features
* PE-51935: Tab completion with file names when setting breakpoints
* PE-51935: Tab completion with function names when setting breakpoints
* PE-50319: Improve feedback when switching command modes
* PE-51749: Tab completion for help, set, show commands
* PE-51752: Moved vscode plugin to gdb4hpc and packaged executable
* PE-51598: Add save all command
* PE-51254: Save and restore assertion scripts and decompositions
* PE-51253: Save and restore breakpoints
* Update included hpcc_demo
* Add set print value-caching
* Save decomposition values as convenience variables
=======================================================================
## [4.16.0] - 2023-11-7 (23.12)
=======================================================================
### Bug Fixes
* PE-50810: Support storing chunked values in client side variables
* Fix failure with p 1+1+1
* Fix formatting when printing very large arrays and vectors
* PE-50510: Set CCE 16 Fortran line number workaround as optional
* Crash where gdb cannot list a line
* PE-49611: Update ZMQNet submodule
### Features
* PE-50840: Update ZMQNet submodule
* PE-50503: Add set variable command
* PE-50619: Save values on the gdb4hpc client as convenience variables with $foo = my_variable syntax
* PE-50671: Support scoped assignment in print statements
* Support --debug=<dir> in launch and attach
* Use dots for progress when connecting
* PE-49938: Support Slurm MPMD launch
* PE-46238: Send signals other than SIGKILL to application
* Move to gcc toolset 12 on redhat
=======================================================================
## [4.15.1] - 2023-07-21 (23.08)
=======================================================================
### Bug Fixes
* Remove error message when calling a void function
* Fix failure to launch at 50K ranks.
* Handle -gdb= options where the debugger relies on LD_LIBRARY_PATH
* PE-48404 - Fix misprinting of types like int *
* Remove load of cray-mrnet module
* PE-48366 new nvidia driver version was not recognized by --gpu on launch and attach
* Catch evaluation exceptions in more cases
* PE-48481: Update attach help for PALS
* Correctly print references to arrays
* Fix printing multi-d arrays truncated by set print elements
* Printing std::string with embedded quotes in PrgEnv-gcc
* Set print elements interacts poorly with gdbmode
### Features
* PE-47564: Update RAJA Documentation
* Pretty printing for std::list and std::tuple
* Use native g++ on sles
* Handle std::pair
* Print derived class types inside C++ smart pointers
* Configurable client-side paging for large data
* Support decomposing RAJA views without requiring an explicit slice
* Add directory and set substitute-path commands to find application source code
* Support --tty command line option
* Allow open ranges for array slicing: minIdx.., ..maxIdx, and ..
* Add support std::map, std::multimap, std::unordered_map, and std::unordered_multimap
* Use C style array ordering for kokkos::Views
* Add --gdb-data-dir option to launch and attach
* Support decomposing RAJA::Views
* Add tutorial for decompositions
* Support RAJA reductions
* More support for RAJA::Views
* Support RAJA::Views of structs
* One line decompostion definition
OS Dependencies
---------------------------
- HPE CSM systems with SLES 15 SP4
- HPE HPCM systems with SLES 15 SP5 and RHEL 8.8
- HPE Apollo systems with RHEL 8.8
Product Dependencies
---------------------------
- cray-mrnet >= 5.1, < 6 installed
- cray-cti 2.18, < 3 installed
- cdst-support >= 2.14, < 3 installed
Documentation
---------------------------
- Type `man gdb4hpc` with the gdb4hpc module loaded to read the gdb4hpc(1) man page.
Simple usage examples are provided at the end.
Also, type `help` at the gdb4hpc command line for more information on commands.
Installation Instructions
---------------------------
- rpm -ivh gdb4hpc-4.16.2-20240509144719_70d6b1455a0e-2.sles15sp5.x86_64.rpm
To make gdb4hpc 4.16.2 the default version of gdb4hpc, execute:
/opt/cray/pe/admin-pe/set_default_files/set_default_gdb4hpc_4.16.2
Legal Notices
---------------------------
Certain components, files or programs contained within this package or product are Copyright 2007-2024 Hewlett Packard Enterprise Development LP.
Attribution notices for open source licensed software for this package are detailed in the file:
/opt/cray/pe/gdb4hpc/4.16.2/ATTRIBUTIONS_gdb4hpc.txt
# HDF5 1.14.3.1
## Release Date:
* July 2024
## Purpose:
This HDF5 release contains the following:
* SLES 15 SP5 (x86_64 and aarch64)
* RHEL 9.4 (x86_64 and aarch64)
* RHEL 8.10 (x86_64)
* Nvidia HPC SDK 24.03
* AMD ROCm 6.1
* AOCC 4.2
* Intel OneAPI 2024.0.1
* RHEL gcc-toolset-13
## Product and OS Dependencies:
The HDF5 1.14.3.1 release is supported on the following systems, requiring the following software products:
* CrayPE 2.1.2 or later
* SLES 15 SP5 (x86_64 and aarch64)
* RHEL 8.10 (x86_64)
* RHEL 9.4 (x86_64 and aarch64)
* cray-mpich/8.1.0 or later
Supported on the following HPE systems:
* HPE Cray EX systems with CLE
* HPE Cray XD systems with CLE
One or more of the following compiler major versions:
* CCE 17.x or later (SLES)
* CCE 18.x or later (RHEL)
* GCC 12.3 or later (SLES)
* gcc-toolset-10.3/11.2/12.2 (RHEL 8.10)
* gcc-toolset-11.2/12.2/13.1 (RHEL 9.4)
* AOCC 4.1.0 or later
* AMD ROCm 6.0.0 or later
* Intel 2023.2.0 or later
* Nvidia 23.11 or later (aarch64) and 23.3 or later (x86_64)
## Notes and Limitations:
Users are encouraged to use the Cray compiler scripts (cc, ftn, CC) to include the HDF5 header files and link in the HDF5 libraries, rather than the HDF5 compiler scripts (h5cc, h5fc, h5c++) included in the release.
## Documentation:
http://www.hdfgroup.org/HDF5/doc/index.html
## Modulefile:
`module load cray-hdf5`
OR
`module load cray-hdf5-parallel`
## Product description:
HDF5 is a data model, library, and file format for storing and managing data. It supports an unlimited variety of datatypes, and is designed for flexible and efficient I/O and for high volume and complex data. HDF5 is portable and is extensible, allowing applications to evolve in their use of HDF5. The HDF5 Technology suite includes tools and applications for managing, manipulating, viewing, and analyzing data in the HDF5 format.
## Installation:
`rpm -ivh cray-hdf5-1.14.3.1-*-202406100539.1b08b37223c27.@ARCH@.rpm`
The "`*`" in the install command represents compiler version combinations.
To make this the default version, execute:
`/opt/cray/pe/admin-pe/set_default_files/set_default_hdf5_1.14.3.1`
Certain components, files or programs contained within this package or product are Copyright 2019-2024 Hewlett Packard Enterprise Development LP.
Cray LibSci 24.07.0
=====================
Release Date:
-------------
July 2024
Purpose:
--------
Cray LibSci 24.07.0 provides scientific libraries for Cray
HPC systems. Cray LibSci is supported on the host CPU but
not on the accelerator of these systems.
The Cray LibSci 24.07.0 release provides support for the following:
* RHEL 9.4 (x86_64 and aarch64)
* RHEL 8.10 (x86_64)
* Nvidia HPC SDK 24.03
* AMD ROCm 6.1
* AOCC 4.2
* Intel oneAPI 2024.0.1
* RHEL gcc-toolset-13
Cray LibSci 24.07.0 includes the following versions of publicly
available libraries:
* LAPACK 3.10.1 - For further information, see
http://www.netlib.org/lapack
* ScaLAPACK 2.2.0 - (Scalable LAPACK) For further information, see
http://www.netlib.org/scalapack.
* QDWH 2.0.0 KSVD 1.0.0 - Polar decomposition and SVD packages, see
https://github.com/ecrc/qdwh
https://github.com/ecrc/ksvd
Key Changes and Bugs Closed:
----------------------------
Changes in Cray LibSci 24.07.0
* Miscellaneous bug fixes
* Enabled module to set LD_LIBRARY_PATH when PE_LD_LIBRARY_PATH=system
* Removed Iterative Refinement Toolkit (IRT) support
Changes in Cray LibSci 24.03.0
* CAST-34956 - Lua module environment variable issue
Changed in Cray LibSci 23.12.5
* CAST-33777 - LibSci mutex contention issue
* Removed support for Intel KNL
Product and OS Dependencies:
----------------------------
The Cray LibSci 24.07.0 release requires the following:
* SLES 15 SP5 or RHEL 8.10/9.4
* SLES 15 SP5 (x86_64 and aarch64)
* RHEL 8.10 (x86_64)
* RHEL 9.4 (x86_64 and aarch64)
* cray-mpich/8.1.0 or later
Supported on the following HPE systems:
* HPE Cray EX systems with CLE
* HPE Cray XD systems with CLE
One or more of the following compiler major versions:
* CCE 17.x or later (SLES)
* CCE 18.x or later (RHEL)
* GCC 12.3 or later (SLES)
* gcc-toolset-10.3/11.2/12.2 (RHEL 8.10)
* gcc-toolset-11.2/12.2/13.1 (RHEL 9.4)
* AOCC 4.1.0 or later
* AMD ROCm 6.0.0 or later
* Intel 2023.2.0 or later
* Nvidia 23.11 or later (aarch64) and 23.3 or later (x86_64)
Notes and Limitations:
----------------------
OMP threaded versions:
The CrayPE 2.1.2 and later releases add support for link line generation
for the multi-threaded versions of the libsci library based on the OpenMP
flags the user specifies for each compiler:
* CCE by default links to the OpenMP LibSci library. CrayPE will link in
the serial version of LibSci when the CCE flag -hnoomp is used.
* GNU by default links serial LibSci library. CrayPE will link in the
OpenMP version of LibSci when the GNU flag -fopenmp is used.
* INTEL by default links serial LibSci library. CrayPE will link in the
OpenMP version of LibSci when the INTEL flag -qopenmp is used.
Stack size limit:
The 'ulimit -s unlimited' system stack size setting is required for
cray-libsci on all supported Cray platforms. This is typically set as a
system default by admin but may otherwise need to be set at runtime.
Non-default dynamic linking:
When using a non-default version of cray-libsci with dynamic linking
users should set at runtime and at linktime after loading the desired
cray-libsci module--or set the equivalent for their linux shell:
export LD_LIBRARY_PATH=$CRAY_LD_LIBRARY_PATH:$LD_LIBRARY_PATH
Documentation:
--------------
See the intro_libsci man page for additional information.
See the csmlversion man page for information to display version information
on the currently loaded scientific libraries.
Modulefile:
-----------
module load cray-libsci
Installation instructions:
--------------------------
LibSci is now packaged into separate compiler specific RPMs to allow
rpmbuild to correctly include compiler dependencies.
License:
--------
Except for the third party components and software licensed by HPE
through proprietary agreements, components, files or programs
contained within this package or product are Copyright 2001-2023
Hewlett Packard Enterprise Development LP.
Attribution notices for open source licensed software contained in this
package are detailed in the file:
/opt/cray/libsci/24.07.0/ATTRIBUTIONS
Cray LibSci_ACC 24.07.0
=======================
Release Date:
-------------
July 2024
Purpose:
--------
The Cray LibSci_ACC 24.07.0 release provides accelerated versions of
scientific libraries for HPE Cray systems with accelerators.
Product and OS Dependencies:
----------------------------
Cray LibSci_ACC 24.07.0 requires the following platform:
HPE Cray EX system with AMD MI250X
HPE Cray EX system with AMD MI300A
HPE Cray EX system with Nvidia Hopper
Cray LibSci_ACC 24.07.0 supports the following OS:
RHEL 8.10 x86_64
RHEL 9.4 x86_64 & aarch64
SLES 15 SP5 x86_64 & aarch64
Notes and Limitations:
----------------------
LAPACK workspace calculations from other implementations or hard-coded
values may not be compatible with libsci_acc. Workspaces reported from
a query to the corresponding libsci_acc function should be used.
The current supported usecase is a one-rank to one-gpu mapping for apps.
Performance improvements can be achieved in programs calling libsci_acc
subroutines by using pinned memory. See the intro_libsci_acc man page for
details.
Use of the aprun option "-cc none" or the srun option "--cpu_bind=none"
to disable core affinity is strongly suggested to maintain performance.
When using non-default cray-libsci_acc versions users should set equivalent
at runtime and linktime after loading the desired cray-libsci_acc module:
export LD_LIBRARY_PATH=$CRAY_LD_LIBRARY_PATH:$LD_LIBRARY_PATH
Documentation:
--------------
See the intro_libsci_acc man page for additional information.
Modulefile:
-----------
module load cray-libsci_acc
Sample Installation instructions:
---------------------------------
RPM install command:
rpm -ivh cray-libsci-acc-24.07.0-*.##ARCH##.rpm
The "*" in the install command represents compiler version combinations.
To make this the default version:
##PREFIX##/admin-pe/set_default_files/set_default_libsci_acc_24.07.0
License:
--------
Except for the third party modules and software licensed by HPE
through proprietary agreements, components, files or programs
contained within this package or product are Copyright -2023 Hewlett
Packard Enterprise Development LP.
Attribution notices for open source licensed software contained in
this package are detailed in the file:
##PREFIX##/libsci_acc/24.07.0/ATTRIBUTIONS_libsci_acc_24.07.0.txt
Cray Lmod 8.7.37
================================================================================
Release Date:
--------------------------------------------------------------------------------
March, 2024
Purpose:
--------------------------------------------------------------------------------
Provide a Lmod 8.7.37 package for CPE customers
Bugs fixed in this release:
--------------------------------------------------------------------------------
N/A
Documentation:
--------------------------------------------------------------------------------
See CPE's Installation and User Guides for documentation.
Also see Lmod's official website: https://lmod.readthedocs.io
Dependencies:
--------------------------------------------------------------------------------
The Lmod Scripts 8.7.37 release is dependent on the following
software:
bc
sed
lua >= 5.3
lua-luaposix
lua-luafilesystem
Installation instructions:
--------------------------------------------------------------------------------
rpm -Uvh cray-lmod-8.7.37-1.sles15sp5.x86_64.rpm
To make this the default version, update the symbolic "lmod" link
to the desired Lmod version directory.
Lmod Scripts 3.2.1
================================================================================
Release Date:
--------------------------------------------------------------------------------
December, 2023
Purpose:
--------------------------------------------------------------------------------
Updates package license.
Bugs fixed in this release:
--------------------------------------------------------------------------------
Documentation:
--------------------------------------------------------------------------------
See system Installation and User Guides for documentation
Dependencies:
--------------------------------------------------------------------------------
The Lmod Scripts 3.2.1 release is dependent on the following
software:
set_default_3
Lua
Lmod
Installation instructions:
--------------------------------------------------------------------------------
rpm -Uvh lmod_scripts-3.2.1-202405031504.c3177a9cde245-0.noarch.rpm
To make this the default version, execute:
/opt/cray/pe/admin-pe/set_default_files/set_default_lmod_scripts_3.2.1
Product description:
--------------------------------------------------------------------------------
Contains Lua scripts which support the Cray implementation of Lmod
Copyright 2020-2023 Hewlett Packard Enterprise Development LP
modules 3.2.11.7
==================
Release Date:
-------------
Dec, 2023
Purpose:
--------
- Package dependency update
Documentation:
---------------
See man-pages for module and modulefile.
Cray MPICH 8.1.30:
=======================================
Release Date:
-------------
June 1, 2024
Purpose:
--------
Cray MPICH 8.1.30 is based upon ANL MPICH 3.4a2 with support for libfabric
and is optimized for the Cray Programming Environment.
Major Differences Cray MPICH 8.1.30 from the XC Cray MPICH include:
- Uses the new ANL MPICH CH4 code path and libfabric for network
support.
- Does not support -default64 mode for Fortran
- Does not support C++ language bindings
New Cray MPICH features for HPE Cray EX and Apollo systems:
- Starting from the 8.1.26 release, Cray MPICH supports the Intel Sapphire Rapids CPU HBM
processor architecture.
- On systems with AMD GPUs, Cray MPICH 8.1.26 supports all ROCm
versions starting from ROCm 5.0, including the latest ROCm 5.5.0
release.
The Cray MPICH 8.1.25 release and prior versions of
Cray MPICH are only compatible with ROCm versions up to (and
including) the ROCm 5.4.0 release.
- Cray MPICH uses the libfabric "verbs;ofi_rxm" provider by default.
This is the supported and optimized OFI libfabric provider for
Slingshot-10 and Apollo systems.
- Cray MPICH offers support for multiple NICs per node. Starting with
version 8.0.8, by default Cray MPICH will use all available NICs on
a node. Several rank-to-NIC assignment policies are supported. For
details on choosing a policy for assigning ranks to NICS, or for
selecting a subset of available NICs, please see the following
environment variables documented in the mpi man page.
MPICH_OFI_NIC_VERBOSE
MPICH_OFI_NIC_POLICY
MPICH_OFI_NIC_MAPPING
MPICH_OFI_NUM_NICS
- Enhancements to the MPICH_OFI_NIC_POLICY NUMA mode have been added.
Starting with version 8.0.14, if the user selects the NUMA policy,
the NIC closest to the rank is selected. A NIC no longer needs to
reside in the same numa node as the rank. If multiple NICs are
assigned to the same numa node, the local ranks will round-robin
between them. Numa distances are analyzed to select the closest NIC.
- Cray MPICH supports creating a full connection grid during MPI_Init.
By default, OFI connections between ranks are set up on demand. This
allows for optimal performance while minimizing memory requirements.
However, for jobs requiring an all-to-all communication pattern, it
may be beneficial to create all OFI connections in a coordinated
manner at startup. See the MPICH_OFI_STARTUP_CONNECT description in
the mpi man page.
- Cray MPICH supports runtime switching to the UCX netmod starting
with version 8.0.14. To do this load the craype-network-ucx module
and module swap between Cray-MPICH and Cray-MPICH-UCX modules. For
more information including relevant environment variables reference
the intro_mpi man page with the Cray-MPICH-UCX module loaded.
- Lmod support for HPE Cray EX starting with Cray MPICH 8.0.16.
Key Changes and Bugs Closed:
----------------------------
Changes in Cray MPICH 8.1.30
New Features:
- Cray MPICH 8.1.30 for aarch64 is in early stage of support and
has some issues. Setting these variables will help circumvent most
known issues.
export MPICH_SMP_SINGLE_COPY_MODE=CMA
export MPICH_MALLOC_FALLBACK=1
Features and Bug Fixes:
- CAST-34323 - Implement MPICH CQ stall feature for performance improvements on Sapphire Rapids
- CAST-34363 - Enable GTL to use multiple CUDA streams per process
- CAST-35020 - Improve error handling when running out of space in the UDREG cache
- CAST-35124 - Bugfix on aarch64 for MPI_THREAD_MULTIPLE
- CAST-35420 - Bugfix for MPI-IO aggregator selection
- CAST-35543 - Enable GPU kernel support for MPI_Reduce
- CAST-35648 - RMA Windows - Plug memory leaks by freeing shm_cookies and target_info
- PE-52424 - Enable module to set LD_LIBRARY_PATH when PE_LD_LIBRARY_PATH=system
- PE-52488 - Turn off aarch64 XPMEM when GPU support is requested
- PE-52582 - Fix MPI_LONG_DOUBLE datatype for CCE on aarch64
- PE-52604 - Bugfix for memory leak in GTL HSA IPC cache
- PE-53506 - Bugfix for a race condition with some CUDA use cases
- PE-54032 - Enable Allreduce ops to use GPU staging buffers larger than 1GB
- PE-54192 - Document aarch64 CUDA smp behavior
- PE-54326 - Document new MPICH CQ Stall variables
- PE-54303 - Bugfix for HSA seg_info cache
Product and OS Dependencies:
----------------------------
The Cray MPICH 8.1.30 release is supported on the following HPE systems:
* HPE Cray EX systems with CLE
* HPE Apollo systems as part of the Cray Programming Environment
Product and OS Dependencies by network type:
--------------------------------------------------+
| Shasta |
----------------------------+---------------------+
craype | >= 2.7.6 |
----------------------------+---------------------+
cray-pals | >= 1.0.6 |
----------------------------+---------------------+
cray-pmi | >= 6.0.1 |
----------------------------+---------------------+
libfabric | >= 1.9.0 |
----------------------------+---------------------+
One or more compilers:
* AMD ROCM 6.0 or later
* AOCC 4.1 or later
* CCE 17.0 or later
* GNU 12.3 or later
* Intel 2022.1 or later
* Nvidia 23.3 or later
Notes and Limitations:
----------------------
Limitations in Cray MPICH 8.1.30:
- Cray MPICH 8.1.30 can support only ~2040 simultaneous MPI
communicators. This limit is less the XC Cray MPICH limit of
~4090 simultaneous communicators. Cray intends to raise the
limit in a future release of Cray MPICH for Shasta to at least
the XC limit.
Documentation:
--------------
For more information see the intro_mpi man page.
Modulefile:
-----------
module load cray-mpich/8.1.30
License:
--------
Except for the third party components and software licensed by HPE
through proprietary agreements, components, files or programs contained
within this package or product are Copyright -2021 Hewlett Packard
Enterprise Development LP.
Attribution notices for open source licensed software for this
package are detailed in the file:
/opt/cray/pe/mpich/8.1.30/ATTRIBUTIONS
Copyright -2023 Hewlett Packard Enterprise Development LP
Copyright 2022-2024 Hewlett Packard Enterprise Development LP
Product Release Information
================================================================================
Product identification:
-----------------------
HPE Cray MPIxlate: version 1.0.5 (built on 2024-06-03T22:40:56Z from 8085bef)
Purpose:
--------
HPE Cray MPIxlate enables applications compiled using an MPI library that
is not binary compatible with HPE Cray MPI, to be run without recompilation
on supported HPE platforms.
Warnings & Limitations:
-----------------------
- See NOTES in mpixlate(1)
Changes from previous release:
------------------------------
- New features:
* None
- Defects Fixed:
* PE-53958: Invalid library path for PrgEnv-cray
Platform, Operating System and Product dependencies:
----------------------------------------------------
Supported HPE platforms:
- HPE Cray EX systems
- HPE Apollo systems with Cray Programming Environment
Supported Operating Systems:
- SLES 15 SP4 and COS-2.5
- SLES 15 SP5 and COS-3.0
- RHEL 8.10
- RHEL 9.4
Product dependencies:
-----------------+----------
cray-mpich[-ucx] | >= 8.1.30
-----------------+----------
Documentation:
--------------
mpixlate(1)
Environment Modules
-------------------
module load cray-mpixlate/1.0.5
================================================================================
cray-mrnet 5.1.3
================================================================================
*Released: May 2024*
Product Description
---------------------------
- MRNet is a customizable, high-performance software infrastructure for
building scalable tools and applications. It supports efficient
multicast and data aggregation functionality using a tree of processes
between the tool's front-end and back-ends. MRNet-based tools may use
these internal processes to distribute many important tool activities,
for example to reduce data analysis time and keep tool front-end loads
manageable.
MRNet-based tool components communicate across logical channels called
streams. At MRNet internal processes, filters are bound to these streams to
synchronize and aggregate dataflows. Using filters, MRNet can efficiently
compute averages, sums, and other more complex aggregations and analyses
on tool data. MRNet also supports facilities that allow tool developers
dynamically load new tool-specific filters into the system.
Changelog
---------------------------
=======================================================================
## [5.1.3] - 2024-04-22 (24.05)
=======================================================================
### Features
* PE-51960: Added support for rhel89/93 & gcc13
=======================================================================
## [5.1.2] - 2023-09-25 (23.10)
=======================================================================
### Features
* Use gcc toolset 12 on redhat
=======================================================================
## [5.1.1] - 2023-06-14 (23.07)
=======================================================================
### Features
* Use native sles compilers
=======================================================================
## [5.1.0] - 2023-04-10 (23.05)
=======================================================================
### Bug Fixes
* Resolved ImportError: /lib64/libstdc++.so.6: version `GLIBCXX_3.4.26' not found - PE-42742
### Features
* Build with gcc-toolset-10 on redhat
* PE-43547 - Added jenkinsfiles for sp3/4 aarch64
OS Dependencies
---------------------------
- HPE CSM systems with SLES 15 SP4
- HPE HPCM systems with SLES 15 SP5 and RHEL 8.8
- HPE Apollo systems with RHEL 8.8
Product Dependencies
---------------------------
- cray-cti >= 2.18, < 3 installed
- cdst-support >= 2.14, < 3 installed
Installation Instructions
---------------------------
- rpm -ivh cray-mrnet-5.1.3-20240508200844_4fa0df65d0e3-5.sles15sp5.x86_64.rpm
To make cray-mrnet 5.1.3 the default version of cray-mrnet, execute:
/opt/cray/pe/admin-pe/set_default_files/set_default_cray-mrnet_5.1.3
Legal Notices
---------------------------
Certain components, files or programs contained within this package or product are Copyright 2022-2024 Hewlett Packard Enterprise Development LP.
Attribution notices for open source licensed software for this package are detailed in the file:
/opt/cray/pe/mrnet/5.1.3/ATTRIBUTIONS_mrnet.txt
# NetCDF 4.9.0.13
## Release Date:
* July 2024
## Purpose:
This NetCDF release contains the following:
* SLES 15 SP5 (x86_64 and aarch64)
* RHEL 9.4 (x86_64 and aarch64)
* RHEL 8.10 (x86_64)
* Nvidia HPC SDK 24.03
* AMD ROCm 6.1
* AOCC 4.2
* Intel OneAPI 2024.0.1
* RHEL gcc-toolset-13
## Product and OS Dependencies:
The NetCDF 4.9.0.13 release is supported on the following systems, requiring the following software products:
* Cray HDF5 1.14.3.*
* CrayPE 2.1.2 or later
* Cray MPT 8.1 or later
* SLES 15 SP5 (x86_64 and aarch64)
* RHEL 8.10 (x86_64)
* RHEL 9.4 (x86_64 and aarch64)
Supported on the following HPE systems:
* HPE Cray EX systems with CLE
* HPE Cray XD systems with CLE
One or more of the following compiler major versions:
* CCE 17.x or later (SLES)
* CCE 18.x or later (RHEL)
* GCC 12.3 or later (SLES)
* gcc-toolset-10.3/11.2/12.2 (RHEL 8.10)
* gcc-toolset-11.2/12.2/13.1 (RHEL 9.4)
* AOCC 4.1.0 or later
* AMD ROCm 6.0.0 or later
* Intel 2023.2.0 or later
* Nvidia 23.11 or later (aarch64) and 23.3 or later (x86_64)
## Notes and Limitations:
For certain F77 source files, when compiling with `gcc/10+`, it may be necessary to configure/compile with:
`export FCFLAGS="-fallow-argument-mismatch"`
`export FFLAGS="-fallow-argument-mismatch"`
This will turn mismatch errors between actual and dummy argument lists to warnings. See:
https://github.com/Unidata/netcdf-fortran/issues/212
https://gcc.gnu.org/gcc-10/changes.html
Unidata now packages Netcdf-4 and legacy Netcdf-3 separately. HPE has decided not to continue supplying the legacy Netcdf-3 package. Due to CCE changes a version of netcdf built with "-sreal64" is neither needed nor provided.
NetCDF is supported on the host CPU but not on the accelerator.
## Documentation:
http://www.unidata.ucar.edu/software/netcdf/docs
## Modulefile:
`module load cray-netcdf`
OR
`module load cray-netcdf-hdf5parallel`
## Product Description:
NetCDF (network Common Data Form) is a set of interfaces for array-oriented data access and a freely-distributed collection of data access libraries for C, Fortran, C++, Java, and other languages. The netCDF libraries support a machine-independent format for representing scientific data. Together, the interfaces, libraries, and format support the creation, access, and sharing of scientific data.
## Installation:
`rpm -ivh cray-netcdf-4.9.0.13-*202406081619.32645f8ced41e-6.sle15.x86_64.rpm`
The "*" in the install command represents compiler version combinations.
To make this the default version, execute:
`/opt/cray/pe/admin-pe/set_default_files/set_default_netcdf_4.9.0.13`
Certain components, files or programs contained within this package or product are Copyright 2019-2024 Hewlett Packard Enterprise Development LP.
Cray PALS 1.3.2:
================
Release Date:
-------------
Nov 2023
Purpose:
--------
Cray Parallel Application Launch Service (PALS) is an application launcher for
Cray PE applications.
Key Changes:
------------
Add host2xname tool
Allow configuring the PMI port range
Add more examples in manpages
Add SLES 15 SP5 build
Add SLES 15 SP4 and SP5 aarch64 build
Add PMI and PMI2 as --pmi options for mpiexec
Periodically ping applications
Add the ability to allocate additional VNIs for special services like DAOS
Add mpiexec --disable-rdzv-get option to disable rendezvous gets in the NICs
Enable --cpu-bind and --gpu-bind=verbose to work with --rankfile
Add PALSD_FANOUT env var for palsd to set fanout tree width
Add ability for simultaneous mpiexecs to divide up PBS job slots
Add RHEL 8.8 build
Add support for AMD & NVIDIA GPU visibility
Change the wire format for PMIx data to make PMIx_Fence more efficient
Increase core file size limit on RHEL systems
Add retries to SPIRE auth token call
Install aprun and mpirun as symlinks
Use position-independent executables
Change F_CXI_COLL_* -> FI_CXI_COLL_* environment variables
Configure Instant On URL on Shasta
Move binaries and libraries to /opt/cray
Change pmi_example to print NIC distances for all ranks
Make PALS command's nodelists consistent
Set ATOM_SOCKET by default
Improve vnid request error reporting
Launch with PMIx support automatically fetch NIC information
Reduce default fanout to 32 nodes
Require PMIx version 4.2.4 or newer
Send the job-step name to vnid when creating a job-step VNI
Include pid in palsd log messages
Raise maximum number of ZMQ sockets to 4096
Munge auth no longer used for some internal RPCs with trusted connections
Increase default send timeout to 8s
Fix errors found with clang scan-build
Fix nohup mpiexec behavior
Improve cleanup when launch RPC fails
Implement the verbose option for GPU binding
Fix support for the cycle keyword with the mpiexec --rankfile option
Fix error checking for GPU binding options
Better handle mpiexec disconnects
Fix segfault on PMIx RPC for missing app
Fix spawn-related memory leaks
Fix misleading warning in line buffer mode
Remove SLES 15 SP3 build
Deprecate PALS_FANOUT and APRUN_FANOUT environment variables
Install cray-pals RPM
Documentation:
--------------
For more information see aprun, mpiexec, palscmd, palsctrl, palsig, and palstat man page.
Modulefile:
-----------
module load cray-pals
Installation instructions:
--------------------------
rpm -ihv cray-pals-<version>-<date>.el8.x86_64.rpm
rpm -ihv cray-palsd-<version>-<date>.el8.x86_64.rpm
Copyright 2022-2023 Hewlett Packard Enterprise Development LP
cray-papi
=================
Release date:
-------------
June 2024
Package:
--------
cray-papi-7.1.0.2-202405031627.5e39f84427b3c-3 x86_64
Purpose:
--------
New version of papi 7.1.0.2 release 3
Documentation:
--------------
Overview: https://github.com/icl-utk-edu/papi
Web Site: https://icl.utk.edu/papi
Product description:
--------------------
PAPI aims to provide the tool designer and application engineer with a
consistent interface and methodology for use of the performance counter
hardware found in most major microprocessors. PAPI enables software
engineers to see, in near real time, the relation between software
performance and processor events.
Dependencies:
-------------
For a list of software used when validating this version of
PAPI on Cray and HPE systems, see the HPE Cray Programming
Environment release announcements.
Copyright 2015-2017,2019-2024 Hewlett Packard Enterprise Development LP
# Parallel NetCDF 1.12.3.13
## Release Date:
* July 2024
## Purpose:
This Parallel NetCDF release contains the following:
* SLES 15 SP5 (x86_64 and aarch64)
* RHEL 9.4 (x86_64 and aarch64)
* RHEL 8.10 (x86_64)
* Nvidia HPC SDK 24.03
* AMD ROCm 6.1
* AOCC 4.2
* Intel OneAPI 2024.0.1
* RHEL gcc-toolset-13
See the Parallel NetCDF documentation for details and usage information.
## Product and OS Dependencies:
This Parallel NetCDF 1.12.3.13 release is supported on on the following systems, requiring the following software products:
* CrayPE 2.1.2 or later
* Cray MPT 8.1 or later
* SLES 15 SP5 (x86_64 and aarch64)
* RHEL 8.10 (x86_64)
* RHEL 9.4 (x86_64 and aarch64)
Supported on the following HPE systems:
* HPE Cray EX systems with CLE
* HPE Cray XD systems with CLE
One or more of the following compiler major versions:
* CCE 17.x or later (SLES)
* CCE 18.x or later (RHEL)
* GCC 12.3 or later (SLES)
* gcc-toolset-10.3/11.2/12.2 (RHEL 8.10)
* gcc-toolset-11.2/12.2/13.1 (RHEL 9.4)
* AOCC 4.1.0 or later
* AMD ROCm 6.0.0 or later
* Intel 2023.2.0 or later
* Nvidia 23.11 or later (aarch64) and 23.3 or later (x86_64)
## Notes and Limitations:
For certain F77 source files, when compiling with `gcc/10+`, it may be necessary to configure/compile with:
`export FCFLAGS="-fallow-argument-mismatch"`
`export FFLAGS="-fallow-argument-mismatch"`
This will turn mismatch errors between actual and dummy argument lists to warnings. See:
https://github.com/Unidata/netcdf-fortran/issues/212
https://gcc.gnu.org/gcc-10/changes.html
## Documentation:
https://parallel-netcdf.github.io/wiki/Documentation.html
## Modulefile:
`module load cray-parallel-netcdf`
## Product Description:
Parallel NetCDF is a library providing high-performance I/O while maintaining file-format compatibility with Unidata's NetCDF.
## Installation:
`rpm -ivh cray-parallel-netcdf-1.12.3.13-*202406080630.dd5a30e54e2bc-5.sle15.x86_64.rpm`
The "`*`" in the install command represents compiler version combinations.
To make this the default version, execute:
`/opt/cray/pe/admin-pe/set_default_files/set_default_parallel-netcdf_1.12.3.13`
Certain components, files or programs contained within this package or product are Copyright 2019-2024 Hewlett Packard Enterprise Development LP.
Perftools 24.07.0
=================
Release Date: July, 2024
Purpose:
========
This is a feature and bugfix release for the following systems:
- HPE Cray EX and HPE Cray Supercomputer Systems with HPCM
- HPE Cray EX and HPE Cray Supercomputer Systems with CSM
- HPE Apollo 2000 Gen 10 Plus Systems (x86)
Key enhancements or changes from the previous release:
======================================================
o Add support for ROCm 6.1.0
o Add support for CUDA 12.4
o Upgrade to PAPI 7.1.0.2
o Upgrade to DWARF 0.9.2
o Upgrade to support MPICH v8
o First GA release for Apprentice 3
o Accelerator-to-accelerator and host-to-host transfers from roctracer are recorded
o Histograms of message sizes are recorded by transfer type. See environment variables
PAT_RT_ACC_*_BINS in "man pat" for details.
o Extended support for tracing python code
o Record python class names when applicable in calltree leafs
o Add tracing groups cuda_math, hip_math, and rocm_math
o Add tracing groups for PyTorch
o Add tracing support for NCCL/RCCL
o Update trace group 'cuda' to CUDA 12.4
o Update trace group 'hip' and 'hsa' to support ROCm 6.1.0
o Update trace group 'fabric' to support libfabric 1.20.1
o Update trace group 'mpi' to support MPICH v8.1
Bugs Fixed:
===========
o Fix ssh access from Apprentice 2/3
o Fix crash on Apprentice 2 windows client
o Support applications where multiple threads trigger roctracer callback
o Account for missing 'product_name' file and check 'device' file in ROCm installs
o Suppress runtime error for lite-hbm in AMD cpus
o Fix missing 0 count events
o Fix potential loading problems by disabling trace for dlopen by default
o Support floating point counter events for AMD processors
o Support performance counters for AMD MI300A processor
o Various Apprentice2 displays
o Various improvements in tracing Python
o Properly support multiple accelerator threads
o No longer trace dynamic library functions by default
Notes:
======
o This release of Perftools only supports ROCm 6.1.0
o This release of Perftools only supports CUDA 12.0 and later
Known Issues:
=============
o MI300 and MI300a GPU performance counter collection is not supported
o For apps using PyTorch+ROCm installs, pat_run will use ROCm shared libraries found in $ROCM_PATH rather than those installed with PyTorch
o Apprentice 3.0 known issues
o Mac install packages are not signed in the release. To install you must "open" the .dmg file
and agree to the warning dialog.
o The windows desktop can fail to access remote experiments in some installs.
o The mac desktop can show up with inverted or wrong graphics if an older version of gtk is
installed on the machine.
o Potential crash when cutting and pasting into the windows client.
o Searching for function name in the time line view does not work.
o The timeline view can rest to an interval of [NaN,NaN]; reload the experiment to continue.
o Missing tool tips, help, etc.
Deprecations and Removals:
==========================
o Remove custom cray-libelf packaging; use system libelf
o Reveal is no longer supported
Dependencies:
=============
o A PDF reader (such as evince, acroread or okular) is required to use the
app2 online help.
o For a list of software used when validating this version of
Perftools on Cray and HPE systems, see the HPE Cray Programming Environment
release announcements.
Installation instructions:
==========================
For Apollo 80 systems, please see the HPE Cray Programming Environments
Installation Guide: Apollo 80 (ARM) System (S-8013).
For Apollo 2000 systems, please see the HPE Cray Programming Environments
Installation Guide: Apollo 2000 Gen10 Plus (x86) System (S-8012).
For HPE Cray EX (Shasta Architecture) systems, refer to the Cray Asynchronous
Installer Guide (S-8003).
Installation of app2 remote client (+ server) on Mac systems:
-------------------------------------------------------------
Apprentice2Installer-24.07.0.dmg
Apprentice3Installer-24.07.0.dmg
The Cray Apprentice2 and Apprentice3 installers for Mac are included in the
perftools-clients rpm, and placed in
$CRAYPAT_ROOT/share/desktop_installers/
on a Cray Shasta User Access Node (UAN) or on a Cray login node when the
perftools software is installed.
Download the Cray Apprentice2 or Apprentice3 installer onto a desktop or laptop
running Mac OS El Capitan through Mojave. Double click on installer
to begin installation. The installer will walk you through the
process for your system.
Installation of app2 remote client (+ server) on Windows 10 systems:
--------------------------------------------------------------------
Apprentice2Installer-24.07.0.exe
Apprentice3Installer-24.07.0.exe
The Cray Apprentice2 and Apprentice3 installers for Windows are included in the
perftools-clients rpm, and placed in
$CRAYPAT_ROOT/share/desktop_installers/
on a Cray Shasta User Access Node (UAN) or on a Cray login node when the
perftools software is installed.
Download the Cray Apprentice2 installer onto a desktop or laptop
running Windows 10. Double click on installer to begin installation.
The installer will walk you through the process for your system.
Documentation:
==============
See the following man pages:
app2, grid_order, intro_craypat, pat_build, pat_help, pat_info, pat_opts,
pat_report, pat_run, pat_view
intro_papi
perftools-base, perftools-lite, perftools-preload
accpc, cray_cassini, cray_pm, hwpc, nwpc, papi_counters, uncore
PAPIlicnotices
Search for perftools on https://support.hpe.com to access documentation
on Cray Performance Measurement and Analysis Tools
License:
========
Except for the third party modules discussed below and software licensed
by HPE through proprietary agreements, components, files or programs
contained within this package or product are Copyright 2001-2024
Hewlett Packard Enterprise Development LP.
Attribution notices for open source licensed software contained in this
package are detailed in the file:
$CRAYPAT_ROOT/ATTRIBUTIONS_perftools.txt
Cray PMI 6.1.15.21
==============================================================================
Release Date:
--------------
June 26, 2024
Purpose:
--------
The Cray Process Manager Interface Library provides the interface between the
application launcher and other communication libraries such as MPICH and SHMEM.
Cray PMI 6.1.15.21 is the initial package with support for libfabric and
optimized for the Cray Shasta product line.
Major features new to Cray PMI 6.1.15.21 include:
Bug fixes new to Cray PMI 6.1.15.21 include:
* PE-29641 PMI_VERSION_DISPLAY=1 broken
Known limitations include:
Certain components, files or programs contained within this package or product
are Copyright 2004-2023 Hewlett Packard Enterprise Development LP
cray-python 3.11.7
====================
Release Date
------------
July 2024
Purpose
-------
Cray Python is an implementation of the Python programming language for the Cray
Programming Environment. The numpy and scipy modules are configured to call Cray
Libsci routines. The mpi4py module is configured to call Cray MPICH routines.
Cray Python is designed to run Python codes on the compute nodes of an HPE
supercomputer. HPE does not make changes to the Python source or any of its
libraries nor does it plan to make changes in future releases.
The cray-python 3.11.7 release contains
- python-3.11.7
- numpy-1.24.4
- scipy-1.10.1
- mpi4py-3.1.4
- dask-2023.6.1
Product and OS Dependencies
---------------------------
The cray-python 3.11.7 release is supported on
- Cray EX systems running SLES 15 or RHEL 8
Documentation
-------------
https://www.python.org/doc
Changelog
---------
https://docs.python.org/release/3.11.7/whatsnew/changelog.html
Modulefile
----------
module load cray-python/3.11.7
Installation
------------
rpm -ihv cray-python-3.11.7-20240617153007.0687944-1.sles15sp4.x86_64.rpm
To make this the default version, execute
[PREFIX]/admin-pe/set_default_files/set_default_python_3.11.7
Certain components, files, or programs contained within this package are
© Copyright 2021-2024 Hewlett Packard Enterprise Development LP
cray-R 4.4.0
============
Release Date
------------
March 2024
Purpose
-------
The cray-R 4.4.0 release.
cray-R is configured to call cray-libsci routines.
Product and OS Dependencies
---------------------------
The cray-R 4.4.0 release is supported on
- Cray EX systems running SLES 15 or RHEL 8
Documentation
-------------
https://cran.r-project.org/manuals.html
Modulefile
----------
module load cray-R/4.4.0
Installation
------------
rpm -ihv cray-R-4.4.0-20240617153119.a134495-1.sles15sp4.x86_64.rpm
To make this the default version, execute:
[PREFIX]/admin-pe/set_default_files/set_default_R_4.4.0
Certain components, files or programs contained within this package or product are
Copyright 2018-2024 Hewlett Packard Enterprise Development LP
sanitizers4hpc 1.1.3
================================================================================
*Released: June 2024*
Product Description
---------------------------
- Sanitizers4hpc is a debugging tool to aid in the detection of memory leaks
and errors in parallel applications. Using LLVM's AddressSanitizer or
LeakSanitizer as a data source, Sanitizers4hpc aggregates any duplicate
messages across ranks to help provide an understandable picture of
program behavior.
Notes
---------------------------
- Shared library exposure in previous versions of sanitizers4hpc.
Previous versions of sanitizers4hpc incorrectly exported internal sanitizers4hpc
libraries into /opt/cray/pe/lib64, causing other programs that had that path in
their runpath or LD_LIBRARY_PATH to load libraries they didn't expect.
This has been fixed in the 22.08 release of sanitizers4hpc. However, links will
be present in /opt/cray/pe/lib64 pointing to older versions of sanitizers4hpc,
if an older version is installed because old installations will still modify
/opt/cray/pe/lib64 when ldconfig is run.
To clean up libraries left by previous versions of sanitizers4hpc:
Option 1:
1. Uninstall all previously installed versions of sanitizers4hpc, or remove
the .cray_dynamic_file_list file from all previously installed versions
of sanitizers4hpc, typically installed in /opt/cray/pe/sanitizers4hpc.
2. Remove any links in /opt/cray/pe/lib64 pointing into a sanitizers4hpc installation.
3. run ldconfig
Option 2:
1. Uninstall all previous versions of santizers4hpc.
Changelog
---------------------------
=======================================================================
## [unreleased]
=======================================================================
### Bug Fixes
* PE-54265: Automatically preload libfabric if linked
=======================================================================
## [1.1.3] - 2024-05-2 (24.06)
=======================================================================
### Bug Fixes
* PE-53866: Improve ThreadSanitizers and Nvidia Compute Sanitizers aggregation
* PE-53603: Update ZMQNet submodule for libsodium support
* PE-51193: Activate PALS startup barrier manually through libpals
* PE-49611: Update ZMQNet
### Features
* PE-51960: Added support for rhel89/93 & gcc13
* Use gcc toolset 12 on redhat
=======================================================================
## [1.1.1] - 2023-07-7 (23.08)
=======================================================================
### Features
* PE-47186: Add Sanitizers4hpc User Guide
* PE-48278: Supply CUDA Sanitizer options via frontend argument
* Use native compilers on sles
* PE-48186: Support SHMEM non-shared heap checking, HTML documentation in share/doc
=======================================================================
## [1.1.0] - 2023-04-11 (23.05)
=======================================================================
### Bug Fixes
* PE-45171: s4hStarter was not exiting properly on ALPS
* PE-45185: Use Clang path to add LLVM utilities to PATH
* PE-45170: Fix log directory creation with CCE 15 Sanitizers
### Features
* PE-46016: Add option to filter output not matching files / functions
* Update to build with gcc-toolset-10 on redhat
* PE-45209: Update CUDA-Memcheck to Nvidia Compute Sanitizer
OS Dependencies
---------------------------
- HPE CSM systems with SLES 15 SP4
- HPE HPCM systems with SLES 15 SP5 and RHEL 8.8
- HPE Apollo systems with RHEL 8.8
Product Dependencies
---------------------------
- cray-cti >= 2.18, < 3 installed
- cdst-support >= 2.14, < 3 installed
Installation Instructions
---------------------------
- rpm -ivh sanitizers4hpc-1.1.3-20240523191358_2688b90db106-2.sles15sp5.x86_64.rpm
To make sanitizers4hpc 1.1.3 the default version of sanitizers4hpc, execute:
/opt/cray/pe/admin-pe/set_default_files/set_default_sanitizers4hpc_1.1.3
Legal Notices
---------------------------
Certain components, files or programs contained within this package or product are Copyright 2022-2024 Hewlett Packard Enterprise Development LP.
Attribution notices for open source licensed software for this package are detailed in the file:
/opt/cray/pe/sanitizers4hpc/1.1.3/ATTRIBUTIONS_sanitizers4hpc.txt
Cray OpenSHMEMX 11.7.2:
===============================
Release Date:
-------------
July, 2024
Purpose:
--------
OpenSHMEM is a Partitioned Global Address Space (PGAS) library interface
specification, which is the culmination of a standardization effort among
many implementers and users of SHMEM programming model. SHMEM has a long
history as a parallel programming model on HPE Cray supercomputer systems.
For the past two decades SHMEM library implementation in HPE Cray systems
evolved through different generations. Cray OpenSHMEMX is a proprietary,
OpenSHMEM specification compliant SHMEM implementation for HPE Cray
product line.
Key Changes:
------------
Major differences in Cray OpenSHMEMX 11.7.2 from
Cray OpenSHMEMX version 11.7.1 includes the following:
- Add support for using 4/8 byte puts for application-level signaling
Bugs Closed:
------------
The following bugs are fixed as part of the Cray OpenSHMEMX 11.7.2
release:
- Fix several shmem_quiet flushing issues for default and non-default
settings
- Fix the implementation of signaling sets to match OSM-1.5 semantics
Product and OS Dependencies:
----------------------------
The Cray OpenSHMEMX 11.7.2 release is supported on the following
Cray systems:
* HPE Cray EX systems with CLE
Product and OS Dependencies by network type:
--------------------------------------------------+
| HPE Slingshot |
----------------------------+---------------------+
craype | >= 2.7.3 |
----------------------------+---------------------+
cray-pals | >= 0.3.5 |
----------------------------+---------------------+
cray-pmi | >= 6.0.1 |
----------------------------+---------------------+
libfabric | >= 1.9.0 |
----------------------------+---------------------+
cray-dsmml | >= 0.2.2 |
----------------------------+---------------------+
One or more compilers:
* AOCC 2.2 or later
* CCE 9.1 or later
* GNU 9.1 or later
* Intel 19.0 or later
* Nvidia 20.7 or later
Notes and Limitations:
----------------------
Announcements, release information, supported environments, limitations,
usage models, and backward compatibility informations about this product
can be viewed in the following location:
https://pe-cray.github.io/cray-openshmemx/
Documentation:
--------------
Use the Cray OpenSHMEMX man pages for more information on the library
functions and use intro_shmem(3) man page for general information.
Initial support for Cray OpenSHMEMX man pages are derived from OpenSHMEM
standards specification document
Use https://pe-cray.github.io/whitepapers/ for access to different
Cray OpenSHMEMX specific whitepapers
Modulefile:
-----------
module load cray-openshmemx/11.7.2
License:
--------
Components, files, and programs contained within this package or
product are Copyright Hewlett Packard Enterprise Development LP
cray-stat 4.12.3
================================================================================
*Released: May 2024*
Product Description
---------------------------
- The Stack Trace Analysis Tool (STAT) is a highly scalable, lightweight
tool that gathers and merges stack traces from all of the processes of
a parallel application to form call graph prefix trees.
Notes
---------------------------
- To enable gathering stack traces on an Nvidia GPU, the environment variable
STAT_GDB must be set to either cuda-gdb, which will require cuda-gdb to be
available in PATH. Or it can be set a specific debugger. We suggest the
version of cuda-gdb packaged with gdb4hpc, as in:
export STAT_GDB=/opt/cray/pe/gdb4hpc/$GDB4HPC_VERSION/libexec/cuda110-gdb-guard
After which stat-cl would be called using the options: -w -G -i
Changelog
---------------------------
=======================================================================
## [4.12.3] - 2024-04-23 (24.05)
=======================================================================
### Features
* PE-51960: Added support for rhel89/93 & gcc13
* Update to gcc toolset 12 on redhat
* Update to gcc toolset 12 on redhat
=======================================================================
## [4.12.1] - 2023-07-11 (23.08)
=======================================================================
### Features
* Use native compilers on sles
* Use native sles compilers
=======================================================================
## [4.12.0] - 2023-04-11 (23.05)
=======================================================================
### Features
* Build with gcc-toolset-10 on redhat
=======================================================================
## [4.11.13] - 2022-09-6 (22.10)
=======================================================================
### Bug Fixes
* Resolved ImportError: /lib64/libstdc++.so.6: version `GLIBCXX_3.4.26' not found - Bugfix/PE-42092
OS Dependencies
---------------------------
- HPE CSM systems with SLES 15 SP4
- HPE HPCM systems with SLES 15 SP5 and RHEL 8.8
- HPE Apollo systems with RHEL 8.8
Product Dependencies
---------------------------
- cray-mrnet >= 5.1, < 6 installed
- cray-cti 2.18, < 3 installed
- cdst-support >= 2.14, < 3 installed
Documentation
---------------------------
- https://computing.llnl.gov/code/STAT/
Installation Instructions
---------------------------
- rpm -ivh --oldpackage cray-stat-4.12.3-20240509022838_14bc8325fab8-3.sles15sp5.x86_64.rpm
To make cray-stat 4.12.3 the default version of cray-stat, execute:
/opt/cray/pe/admin-pe/set_default_files/set_default_cray-stat_4.12.3
Legal Notices
---------------------------
Certain components, files or programs contained within this package or product are Copyright 2007-2024 Hewlett Packard Enterprise Development LP.
Attribution notices for open source licensed software for this package are detailed in the file:
/opt/cray/pe/stat/4.12.3/ATTRIBUTIONS_stat.txt
totalviewsup 2024.1.21
================================================================================
*Released: March 2024*
Product Description
---------------------------
- Provides the HPE Totalview support package which contains modulefiles (tcl & lua),
set_default files, and content projection files for Totalview 2024.1.21.
- The only supported install directory for HPE totalviewsup is /opt/cray/pe.
- The only supported install directory for Perforce Totalview is /opt/totalview.
- Drop the license file into: /opt/totalview/2024.1.21/
Changelog
---------------------------
=======================================================================
## [2023.4.16] - 2024-02-26 (24.03)
=======================================================================
### Features
* PE-49958: Implemented the auto-gen release notes/changelog tool
- Updated supported version to 2023.4.16
OS Dependencies
---------------------------
- HPE CSM systems with SLES 15 SP4
- HPE HPCM systems with SLES 15 SP5 and RHEL 8.8
- HPE Apollo systems with RHEL 8.8
Product Dependencies
---------------------------
- This totalviewsup 2024.1.21 package has a dependency on Totalview 2024.1.21.
Documentation
---------------------------
- Totalview docs can be found in /opt/toolworks/totalview.2024.1.21/doc/
- Totalview user guide:
/opt/toolworks/totalview.2024.1.21/doc/TotalView_User_Guide.pdf
- Totalview installation guide:
/opt/toolworks/totalview.2024.1.21/doc/TotalView_Installation_Guide.pdf
- The Totalview release notes:
/opt/toolworks/totalview.2024.1.21/doc/TotalView_Change_Log.pdf
- The Totalviewsup support package release notes can be found in:
/opt/cray/pe/totalviewsup/2024.1.21/release_notes.md
Installation Instructions
---------------------------
- rpm -ivh totalviewsup-2024.1.21-20240513190717_eed2518e7091-1.sles15sp5.x86_64.rpm
To make Totalview 2024.1.21 the default version, execute:
/opt/cray/pe/admin-pe/set_default_files/set_default_totalview_2024.1.21
Legal Notices
---------------------------
Copyright 2007-2024 Hewlett Packard Enterprise Development LP.
CM-PE-INTEGRATION CHANGELOG
===========================
24.07 (2023-06-05)
* Remove rpm queries from 77-deploy-pe.sh for improved performance
* Remove find commands from 77-deploy-pe.sh for improved performance
* Replace the systemd file with a new pe.service that depends on pe.mount
* CAST-35461 fix setting MODULEPATH in csh.cshrc.local
* Replace find commands with for loops to allow using symlink paths in gen_modulefiles.sh
24.03 (2023-02-16)
* Fix regex in gen_modulefiles for Intel OneAPI
* Add a umask 022 to ensure CPE access is not limited
* Add nfs-utils as an rpm dependencies
23.12 (2023-11-27)
* Determine the craype-target architecture during shell initialization
* Add support for compute nodes running on aarch64 hardware
* Update gen_modulefiles.sh to better support new compiler versions
* Add better logging information to `77-deploy-pe`
23.05 (2023-05-10)
* Allow sites to override the lmod install path by setting `lmod_prefix_path`
in `cray-pe-configuration.sh` and `cray-pe-configuration.csh`. Sites can
now use newer versions of lmod provided by the OS distribution.
* Update `76-add-pe-to-fstab` so that `/pe` is not added to `/etc/fstab` more
then once.
23.03 (2023-03-03)
* Spread the squashfs image mounts across NUMA domains to balance memory
usage across NUMA nodes.
* Add RPM dependency on `numactl`
23.02 (2023-01-30)
* Update shell profile scripts to initialize lmod using HPE provided package.
* Remove leading and trailing `:` in `LMOD_SYSTEM_DEFAULT_MODULES` that is no
longer an acceptable format in new versions of lmod.
22.12 (2022-12-12)
* Add a `site_module_list` to `cray-pe-configuration.sh` and
`cray-pe-configuration.csh` to allow sites to add modules loaded at login
without the need to modify HPE defaults `init_module_list`.
22.11 (2022-11-10)
* Update `gen_modulefiles.sh` to recreating symlinks to ensure idempotence.
22.10 (2022-10-03)
* Changes for HPCM 1.8 to ensure that `/pe` is mounted in `76-add-pe-to-fstab`
before running `77-deploy-pe`
22.09 (2022-08-23)
* Update `pe_overlay.sh` to only remove workdirs when unmounts are successful.
* Update `gen_modulefiles.sh` to run `one_off_set_defaults` after generating
all modulefiles.
22.08 (2022-07-22)
* Enable older PE images to deploy on system with newer minor base OS version.
* Update `gen_modulefiles.sh` to use the latest available version of
`craypkg-gen`.
22.05 (2022-04-26)
* Remove `/opt/modulefiles` from `MODULEPATH`.
22.04 (2022-03-28)
* Add a temporary workaround in `pe_overlay.sh` that copies lmod GPU
modulefiles into the PE overlay's upper layer so they are visible after
`/opt/cray/pe` is mounted.
valgrind4hpc 2.13.3
================================================================================
*Released: May 2024*
Product Description
---------------------------
- Valgrind4hpc is a Valgrind-based debugging tool to aid in the detection of memory
leaks and errors in parallel applications. Valgrind4hpc aggregates any duplicate
messages across ranks to help provide an understandable picture of
program behavior. Valgrind4hpc manages starting and redirecting output from many
copies of Valgrind, as well as deduplicating and filtering Valgrind messages.
If your program can be debugged with Valgrind, it can be debugged with Valgrind4hpc.
Changelog
---------------------------
=======================================================================
## [2.13.3] - 2024-04-23 (24.05)
=======================================================================
### Bug Fixes
* PE-51571: Parse arguments before starting network
### Features
* PE-51960: Added support for rhel89/93 & gcc13
* Update Valgrind to 3.22, support syscall userfaultfd
* Build with gcc toolset 12 on redhat
=======================================================================
## [2.13.1] - 2023-07-7 (23.08)
=======================================================================
### Features
* PE-47184: Add valgrind4hpc user guide
* Use native compilers on sles
=======================================================================
## [2.13.0] - 2023-04-11 (23.05)
=======================================================================
### Bug Fixes
* PE-45205: Fix --from-ranks filtering and suppression generation
* Fix install path exported by lmod modulefile on relocated install
### Features
* Build with gcc-toolset-10 on redhat
* PE-45055: Add GDB4hpc connection functionality for VGDB mode
=======================================================================
## [2.12.11] - 2023-01-9 (23.02)
=======================================================================
### Features
* PE-44991: Update Valgrind to 3.20 and add VGDB option support
OS Dependencies
---------------------------
- HPE CSM systems with SLES 15 SP4
- HPE HPCM systems with SLES 15 SP5 and RHEL 8.8
- HPE Apollo systems with RHEL 8.8
Product Dependencies
---------------------------
- cray-mrnet >= 5.1, < 6 installed
- cray-cti >= 2.18, < 3 installed
- cdst-support >= 2.14, < 3 installed
Documentation
---------------------------
- See man valgrind4hpc.
Installation Instructions
---------------------------
- rpm -ivh valgrind4hpc-2.13.3-20240509191316_ddb177766849-4.sles15sp5.x86_64.rpm
To make valgrind4hpc 2.13.3 the default version of valgrind4hpc, execute:
/opt/cray/pe/admin-pe/set_default_files/set_default_valgrind4hpc_2.13.3
Legal Notices
---------------------------
Certain components, files or programs contained within this package or product are Copyright 2010-2024 Hewlett Packard Enterprise Development LP.
Attribution notices for open source licensed software for this package are detailed in the file:
/opt/cray/pe/valgrind4hpc/2.13.3/ATTRIBUTIONS_valgrind4hpc.txt