Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
32 changes: 0 additions & 32 deletions amd/comgr/src/hotswap/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -52,35 +52,3 @@ target_link_libraries(hotswap-transpiler
LLVMSupport
LLVMTargetParser
)

# HSA_TOOLS_LIB hotswap tool (libamd_comgr_hotswap_tool.so): ROCR loads it via
# HSA_TOOLS_LIB and it rewrites code objects through the comgr hotswap API.
# Off by default - it is an opt-in artifact most comgr consumers do not need.
# When enabled it requires the HSA headers (inc/hsa.h, from
# HOTSWAP_TOOL_HSA_INCLUDE_ROOT); configuration errors if they are not found.
option(HOTSWAP_BUILD_TOOL "Build the HSA_TOOLS_LIB hotswap tool" OFF)
if(HOTSWAP_BUILD_TOOL)
if(NOT CMAKE_SYSTEM_NAME STREQUAL "Linux")
message(FATAL_ERROR
"HOTSWAP_BUILD_TOOL is only supported on Linux (it requires the HSA "
"runtime); set HOTSWAP_BUILD_TOOL=OFF on this platform.")
endif()
find_path(HOTSWAP_TOOL_HSA_INC inc/hsa.h HINTS "${HOTSWAP_TOOL_HSA_INCLUDE_ROOT}")
if(NOT HOTSWAP_TOOL_HSA_INC)
message(FATAL_ERROR
"HOTSWAP_BUILD_TOOL is ON but inc/hsa.h was not found. Set "
"HOTSWAP_TOOL_HSA_INCLUDE_ROOT to the rocr-runtime hsa-runtime directory, "
"or set HOTSWAP_BUILD_TOOL=OFF.")
endif()
# amd/comgr/src/hotswap binary dir -> amd/comgr binary dir (generated amd_comgr.h).
get_filename_component(_comgr_bin "${CMAKE_CURRENT_BINARY_DIR}/../.." ABSOLUTE)
add_library(amd_comgr_hotswap_tool SHARED comgr-hotswap-tool.cpp)
set_target_properties(amd_comgr_hotswap_tool PROPERTIES
CXX_STANDARD 17 CXX_STANDARD_REQUIRED ON POSITION_INDEPENDENT_CODE ON)
target_include_directories(amd_comgr_hotswap_tool PRIVATE
"${HOTSWAP_TOOL_HSA_INC}"
"${_comgr_bin}/include") # generated amd_comgr.h
target_link_libraries(amd_comgr_hotswap_tool PRIVATE amd_comgr)
install(TARGETS amd_comgr_hotswap_tool
LIBRARY DESTINATION "${CMAKE_INSTALL_LIBDIR}" COMPONENT amd-comgr)
endif()
82 changes: 9 additions & 73 deletions amd/comgr/src/hotswap/README.md
Original file line number Diff line number Diff line change
@@ -1,78 +1,14 @@
# HotSwap

HotSwap rewrites AMDGPU code objects at load time so that a binary built for one
gfx1250 stepping runs correctly on another. The `amd_comgr_hotswap_rewrite` API
applies a small set of stepping-specific patches in place, without recompiling
the code object.

This directory ships two things:

- The comgr hotswap tool (`libamd_comgr_hotswap_tool.so`), loaded via the
`HSA_TOOLS_LIB` env var, which applies the rewrite automatically at runtime.
- The transpiler, a raiser-based path for the heavier cross-ISA case. It is
documented at the bottom of this file.

The tool is Linux only: it requires the HSA runtime, so the build errors on
other platforms.

## Running with the tool

Point `HSA_TOOLS_LIB` at the tool and run any HIP or HSA application unchanged:

```bash
HSA_TOOLS_LIB=/opt/rocm/lib/libamd_comgr_hotswap_tool.so ./my_app
```

`HSA_TOOLS_LIB` tells the `libhsa-runtime` what tool to hand each code object it encounters to before dispatch. When the tool detects a gfx1250 A0 board, it rewrites every gfx1250 code object in place via `amd_comgr_hotswap_rewrite`. Everything else passes through untouched.

If a rewrite fails, the tool logs the failure and forwards the original code
object. The application still runs, just without the rewrite applied.

## Supported architectures

| Architecture | Status |
| ------------------- | ----------- |
| gfx1250, ASIC rev A0 | Rewrite armed |
| gfx950 | Coming soon |
| gfx942 | Coming soon |

HotSwap currently requires a homogeneous GPU setup. Running it across multiple
GPUs is not supported.

The tool detects the device on its own. It reads the agent ISA name and
`HSA_AMD_AGENT_INFO_ASIC_REVISION`, then arms the rewrite only when the target
is gfx1250, the revision is 0 (A0), and the revision query succeeded. A failed
query is never treated as A0. On any other device or revision, code objects pass
through unchanged. No environment variable is needed to turn this on.

To confirm the rewrite is active, set `HSA_HOTSWAP_TOOL_VERBOSE=1` and look for
this line in the logs:

```bash
hotswap_tool: device=gfx1250 asic_revision=0 -> A0 (rewrite armed)
```

## Environment variables

| Variable | Effect |
| -------------------------- | ------------------------------------------------------------- |
| `HSA_TOOLS_LIB` | Standard HSA hook. Set it to this `.so` to load the tool. |
| `HSA_HOTSWAP_TOOL_VERBOSE` | Set to `1` for diagnostic logging to stderr, covering device detection and per-code-object rewrite results. Logging only; does not change behavior. Off by default. |

## Building the tool

The tool is off by default, since most comgr consumers do not need it. Enable it
and point the build at the HSA headers:

```bash
cmake -S amd/comgr -B build \
-DHOTSWAP_BUILD_TOOL=ON \
-DHOTSWAP_TOOL_HSA_INCLUDE_ROOT=/path/to/rocr-runtime/runtime/hsa-runtime
ninja -C build amd_comgr_hotswap_tool
```

If `HOTSWAP_BUILD_TOOL` is on but `inc/hsa.h` cannot be found under
`HOTSWAP_TOOL_HSA_INCLUDE_ROOT`, the build fails.
HotSwap is COMGR's AMDGPU code-object rewriting support. The public
`amd_comgr_hotswap_rewrite` API takes an executable code object plus source and
target ISA names, then returns a new executable code object with the applicable
rewrite applied. The input code object is not modified.

This directory contains COMGR's hotswap transpiler scaffolding, the raiser-based
path for heavier cross-ISA transformations. The same-family stepping patches are
implemented in the surrounding COMGR source files and are exposed through
`amd_comgr_hotswap_rewrite`.

## Transpiler (cross-gen)

Expand Down
86 changes: 0 additions & 86 deletions amd/comgr/src/hotswap/comgr-hotswap-tool-detect.h

This file was deleted.

Loading
Loading