Skip to content

Latest commit

 

History

History
267 lines (205 loc) · 12.6 KB

BUILD.md

File metadata and controls

267 lines (205 loc) · 12.6 KB

Contents

Build QAI AppBuilder for WoS device

Build QAI AppBuilder from source with Visual Studio 2022 on WoS device:

pip install wheel==0.45.1 setuptools==80.9.0 pybind11==2.13.6 build==1.4.0
  • Clone this repository to local:
git clone https://github.com/qualcomm/qai-appbuilder.git --recursive
  • If you have cloned it before, you can update the code by the following command:
cd qai-appbuilder
git pull --recurse-submodules
  • Set environment 'QNN_SDK_ROOT' to the Qualcomm® AI Runtime SDK path which you're using. E.g.:
Set QNN_SDK_ROOT=C:\Qualcomm\AIStack\QAIRT\2.48.40.260702\
  • Use the commands below to build and install Python extension(*.whl):
    *Note: Please get the corresponding "Supported Toolchains" and "Hexagon Arch" with your device from Supported Snapdragon devices.
cd qai-appbuilder
*Note: Make sure to build in the regular Windows Command Prompt — not in the 'ARM64 Native Tools Command Prompt for VS 2022' and not in the 'Power Shell' window.* <br>
     set QNN_SDK_ROOT=C:/Qualcomm/AIStack/QAIRT/2.48.40.260702/

     set QAI_TOOLCHAINS=aarch64-windows-msvc (For ARM64 Windows Python) 
     [or] 
     set QAI_TOOLCHAINS=arm64x-windows-msvc (For AMD(X64) Windows on Snapdragon Python)
     [or] 
     set QAI_TOOLCHAINS=x86_64-windows-msvc  (For x86_64 Windows PC)

     python -m build -w

# Install the extension:
pip install --force-reinstall dist\qai_appbuilder-2.48.40-cp312-cp312-win_amd64.whl

Build QAI AppBuilder for Linux

Clone the repository with submodules:

git clone https://github.com/qualcomm/qai-appbuilder.git --recursive
cd qai-appbuilder

Download QNN SDK

Download the Qualcomm® AI Runtime (QAIRT) SDK on your device. This SDK includes the required QNN runtime libraries for AI model execution.

Download Link of QAIRT v2.40.0.251030: https://softwarecenter.qualcomm.com/api/download/software/sdks/Qualcomm_AI_Runtime_Community/All/2.40.0.251030/v2.40.0.251030.zip.

# Download QAIRT SDK package 
wget https://softwarecenter.qualcomm.com/api/download/software/sdks/Qualcomm_AI_Runtime_Community/All/2.40.0.251030/v2.40.0.251030.zip

# Extract the runtime libraries
unzip v2.40.0.251030.zip

# Verify extraction
ls v2.40.0.251030/

Set Environment Variables

Configure the required environment variables on your device. Replace <path_to_v2.40.0.251030> with the actual path to your extracted QNN SDK directory.

Common variables for both platforms:

export QNN_SDK_ROOT=<path_to_v2.40.0.251030>
export QAI_TOOLCHAINS=aarch64-oe-linux-gcc11.2

Build with direct TFLite support (Linux ARM64 QNN or Windows CPU)

TFLite support is disabled by default. Linux ARM64 uses the matching QAIRT QNN delegate; Windows uses the TFLite CPU interpreter without QNN dependencies. Both paths require a redistributable TFLite C API SDK.

export QNN_SDK_ROOT=<path-to-qairt>
export TFLITE_ROOT=<path-to-tflite-sdk>
export QAI_TOOLCHAINS=aarch64-oe-linux-gcc11.2
export APPBUILDER_ENABLE_TFLITE=ON
python -m build -w

TFLITE_ROOT must contain include/tensorflow/lite/c/c_api.h and a C API library under lib/ named libtensorflowlite_c.so or libtflite_c.so. The QNN SDK must contain include/QNN/TFLiteDelegate/QnnTFLiteDelegate.h and lib/aarch64-oe-linux-gcc11.2/libQnnTFLiteDelegate.so. The resulting wheel places these direct runtime libraries under qai_appbuilder/libs/; inspect them with:

unzip -l dist/qai_appbuilder-*.whl | grep -E 'tensorflowlite|tflite|QnnTFLiteDelegate'

The Windows CPU build uses a separate feature gate:

$env:TFLITE_ROOT = "C:/path/to/tflite-sdk"
$env:APPBUILDER_ENABLE_TFLITE_CPU = "ON"
python -m build -w

The Windows SDK must provide include/tensorflow/lite/c/c_api.h, a tensorflowlite_c.lib or tflite_c.lib import library under lib/, and the matching tensorflowlite_c.dll or tflite_c.dll under bin/ or lib/. The wheel contains the CPU runtime DLL under qai_appbuilder/libs/ and does not package libQnnTFLiteDelegate.so. APPBUILDER_ENABLE_TFLITE and APPBUILDER_ENABLE_TFLITE_CPU are mutually exclusive. Do not enable either option unless the runtime artifacts are licensed for redistribution.

Direct TFLite runtime limitations

Direct .tflite contexts expose a single interpreter graph. Pass only graphIndex=0 (or omit it) to QNNContext.Inference() and metadata getters such as getInputShapes(); non-zero graph indices raise an error and multi-graph selection is not supported for .tflite models.

getProfilingEvent() returns 0 for direct .tflite contexts because this path does not expose QNN profiling events. Do not interpret 0 as a measured profiling result.

Build with direct ExecuTorch support (Linux and Android arm64)

ExecuTorch support is disabled by default and consumes a prebuilt ExecuTorch C++ SDK selected with EXECUTORCH_ROOT. The SDK must match the target platform: use a Linux SDK for Linux builds and an Android arm64-v8a SDK for Android NDK builds. Static single-method .pte programs with tensor-only forward inputs and outputs are supported.

export EXECUTORCH_ROOT=<path-to-executorch-sdk>
export APPBUILDER_ENABLE_EXECUTORCH=ON
# Optional Qualcomm backend. The .pte must have been lowered for QNN.
export QNN_SDK_ROOT=<path-to-qairt>
export APPBUILDER_ENABLE_EXECUTORCH_QNN=ON
python -m build -w

EXECUTORCH_ROOT must contain extension/module/module.h, extension/tensor/tensor_ptr.h, the extension_module and executorch runtime libraries. An XNNPACK backend library (executorch_backend_xnnpack or xnnpack_backend) is linked when the SDK provides one; CPU-only builds do not require it. QNN builds additionally require backends/qualcomm/runtime/QnnExecuTorch.h, QNN_SDK_ROOT, and the ExecuTorch Qualcomm backend library. Runtime shared libraries are copied under qai_appbuilder/libs/ when present. Do not redistribute ExecuTorch, XNNPACK, QNN, or QAIRT artifacts unless their licenses permit it.

For Android, pass the Android SDK root to make/Android.mk and keep .pte assets uncompressed when they are memory mapped:

ndk-build EXECUTORCH_ROOT=<android-arm64-executorch-sdk> \
    EXECUTORCH_ENABLE_XNNPACK=1 EXECUTORCH_ENABLE_QNN=1 APP_ABI=arm64-v8a
androidResources {
    noCompress.add("pte")
}

Both CPU/XNNPACK and Qualcomm programs report their provider mode through QNNContext.getProviderMode(). Performance profiles, asynchronous execution, QNN-only device options, dynamic tensor dimensions, and non-forward methods are rejected for .pte models in this initial integration.

Install Python Dependencies

Upgrade build tooling and install required Python packages:

# Install system dependencies
sudo apt update
sudo apt install -y cmake build-essential python3.12-dev

# Install core Python packages
pip install wheel==0.45.1 setuptools==80.9.0 pybind11==2.13.6 build==1.4.0

Build QAI AppBuilder Python and C/C++ Libraries

Build the QAI AppBuilder libraries from the project root directory.

python -m build -w

Note: The build process may take several minutes. The output wheel file will be created in the dist/ directory.

Install QAI AppBuilder Wheel Package

Install the built wheel package:

python -m pip install dist/qai_appbuilder-*.whl

Note: The version number may vary based on your QNN SDK version.

Verify Installation:

python -c "import qai_appbuilder; print('QAI AppBuilder installed successfully')"

Note: QAI AppBuilder provides multiple examples of AI applications developed using Python, covering scenarios such as image super-resolution, object detection, and image classification. Refer to (samples/linux/README.md)

Build QAI AppBuilder for android

Download QAI AppBuilder source codes:

Run below command in Windows terminal:

git clone https://github.com/qualcomm/qai-appbuilder.git --recursive

Set PATH and run make.exe to build QAI AppBuilder

• Download android ndk.
• Run following commands in Windows terminal:

cd qai-appbuilder
Set QNN_SDK_ROOT=C:\Qualcomm\AIStack\QAIRT\{Qualcomm® AI Runtime SDK version}\
Set NDK_ROOT={your ndk root directory}
set PATH=%PATH%;%NDK_ROOT%\toolchains\llvm\prebuilt\windows-x86_64\bin
Set ANDROID_NDK_ROOT=%NDK_ROOT%
 
"%NDK_ROOT%\prebuilt\windows-x86_64\bin\make.exe" android

• Then you will see the generated file qai-appbuilder\libs\arm64-v8a\libappbuilder.so.

Debug issues about AppBuilder

• Sometimes we will meet error which is related with libAppBuilder.so, for example below abnormal info when execute SuperResolution app on Snapdragon® 8 Elite mobile device.

/real_esrgan_x4plus/real_esrgan_x4plus.bin" "/sdcard/AIModels/SuperResolution/real_esrgan_x4plus/input.jpg" "/sdcard/AIModels/SuperResolution/real_esrgan_x4plus/output.jpg"
          <
     0.3ms [ ERROR ] Unable to find a valid interface.
     0.4ms [ ERROR ] Error initializing QNN Function Pointers

• To check above errors info about AppBuilder further, we can 'Compile All Sources' in android studio to generate the bin file of superresolution after modify its CMakeLists.txt file as below,

- add_library(${CMAKE_PROJECT_NAME} SHARED native-lib.cpp)
+ # add_library(${CMAKE_PROJECT_NAME} SHARED native-lib.cpp)

- # add_executable(${CMAKE_PROJECT_NAME} native-lib.cpp) # Build command line executable binary for debugging.
+ add_executable(${CMAKE_PROJECT_NAME} native-lib.cpp) # Build command line executable binary for debugging.

• Copy SuperResolution bin file and other below 9 .so files from SuperResolution\app\build\intermediates\cxx\Debug\63644d4r\obj\arm64-v8a and C:\Qualcomm\AIStack\QAIRT{Qualcomm® AI Runtime SDK version}\lib\ to /data/local/tmp/debug of android device.

libappbuilder.so
libc++_shared.so
libopencv_java4.so
libQnnHtp.so
libQnnHtpNetRunExtensions.so
libQnnHtpPrepare.so
libQnnHtpV79Skel.so
libQnnHtpV79Stub.so
libQnnSystem.so

• Then run below 5 shell commands in android device to debug:

cd  /data/local/tmp/debug
export LD_LIBRARY_PATH=/data/local/tmp/debug
export PATH=$LD_LIBRARY_PATH:$PATH
chmod +x ./superresolution
./superresolution "/data/local/tmp/debug" "/sdcard/AIModels/SuperResolution/real_esrgan_x4plus/real_esrgan_x4plus.bin" "/sdcard/AIModels/SuperResolution/real_esrgan_x4plus/input.jpg" "/sdcard/AIModels/SuperResolution/real_esrgan_x4plus/output.jpg"

• Root cause
Above error info is due to version incompatible between libappbuilder.so and QAIRT sdk version. It is resolved by recompile libappbuilder.so after set correct QNN_SDK_ROOT.