RK3576 LINUX RKNN SDK 测试

news/2024/11/8 4:56:04 标签: linux, 运维, 服务器

安装Conda工具

安装 Miniforge Conda
wget -c https://github.com/conda-forge/miniforge/releases/latest/download/Miniforge3-Linux-x86_64.sh
chmod 777 Miniforge3-Linux-x86_64.sh
bash Miniforge3-Linux-x86_64.sh

source ~/miniforge3/bin/activate # Miniforge 安装的目录


# 创建名称为 toolkit2 的 Python 3.8 环境
conda create -n toolkit2 python=3.8

# 激活toolkit2环境
conda activate toolkit2

工程代码下载

# 新建 Projects 文件夹
mkdir Projects
# 进入该目录
cd Projects
# 下载 RKNN-Toolkit2 仓库
git clone https://github.com/airockchip/rknn-toolkit2.git --depth 1
# 下载 RKNN Model Zoo 仓库
git clone https://github.com/airockchip/rknn_model_zoo.git --depth 1
# 下载 rknn-llm 仓库
git clone https://github.com/airockchip/rknn-llm.git --depth 1

确认板端环境
在这里插入图片描述
在这里插入图片描述

安装依赖环境以及YOLOv5运行

# 安装依赖库和 RKNN-Toolkit2
# 进入 rknn-toolkit2 目录
cd Projects/rknn-toolkit2/rknn-toolkit2
# 请根据不同的 python 版本，选择不同的 requirements 文件
# 例如 python3.8 对应 requirements_cp38.txt
pip install -r packages/requirements_cpxx.txt
# 安装 RKNN-Toolkit2
# 请根据不同的 python 版本及处理器架构，选择不同的 wheel 安装包文件：
# 其中 x.x.x 是 RKNN-Toolkit2 版本号，cpxx 是 python 版本号，请根据实际数值进行替换
pip install packages/rknn_toolkit2--x.x.x-cpxx-cpxx-manylinux_2_17_x86_64.manylinux2014_x86_64.whl

# 验证是否安装成功
# 进入 Python 交互模式
python
# 导入 RKNN 类
from rknn.api import RKNN

# 更新包列表
sudo apt update
# 安装 cmake
sudo apt install cmake


 # 进入 rknpu2 目录
cd Projects/rknn-toolkit2/rknpu2
# 推送 rknn_server 到板端
# 注：在64位Linux系统中，BOARD_ARCH对应a
# adb push runtime/Linux/rknn_server/${BOARD_ARC
adb push  runtime\Linux\rknn_server\aarch64\usr\bin\rknn_server  /usr/bin
adb push  runtime\Linux\rknn_server\aarch64\usr\bin\start_rknn.sh  /usr/bin
adb push  runtime\Linux\rknn_server\aarch64\usr\bin\restart_r knn.sh  /usr/bin
# 推送 librknnrt.so
# adb push runtime/Linux/librknn_api/${BOARD_ARC
adb push runtime\Linux\rlibrknn_api\aarch64\librknnrt.so   / usr/lib
# 进入板端
adb shell
# 赋予可执行权限
chmod +x /usr/bin/rknn_server
chmod +x /usr/bin/start_rknn.sh
chmod +x /usr/bin/restart_rknn.sh
# 重启 rknn_server 服务
restart_rknn.sh



 RKNN C Demo 使用方法
 
准备模型
# 进入 rknn_model_zoo/examples/yolov5/model 目录
cd Projects/rknn_model_zoo/examples/yolov5/model
# 运行 download_model.sh 脚本，下载 yolov5 onnx 模型
# 例如，下载好的 onnx 模型存放路径为 model/yolov5s_relu.onnx
./download_model.sh



模型转换
# 进入 rknn_model_zoo/examples/yolov5/python 目录
cd Projects/rknn_model_zoo/examples/yolov5/python
# 运行 convert.py 脚本，将原始的 ONNX 模型转成 RKNN 模型
# 用法: python convert.py model_path [rk3566|rk3588|rk3562] [i8/fp] [output_path]
python convert.py ../model/yolov5s_relu.onnx rk3576 i8 ../model/yolov5s_relu.rknn    
    
# 添加到 build-linux.sh 脚本的开头位置即可
GCC_COMPILER=Projects/gcc-linaro-6.3.1-2017.05-x86_64_aarch64-linux-gnu/bin/aarch64-linux-gnu

# 用的是sdk自带的gcc-arm-10.3-2021.07-x86_64-aarch64-none-linux-gnu 
# 路径 linux/prebuilts/gcc/linux-x86/aarch64/gcc-arm-10.3-2021.07-x86_64-aarch64-none-linux-gnu
--- a/build-linux.sh
+++ b/build-linux.sh
@@ -1,5 +1,7 @@
 #!/bin/bash

+GCC_COMPILER=Projects/gcc-arm-10.3-2021.07-x86_64-aarch64-none-linux-gnu/bin/aarch64-rockchip1031-linux-gnu
+
 set -e

 echo "$0 $@"
 # 编译之前修改 库文件
   cd 3rdparty/mpp/Linux/aarch64/
   git status  ./
   rm librockchip_mpp.so
   mv librockchip_mpp.so.0  librockchip_mpp.so
   cd ..
   cd rknn_yolov5_demo/

 
 # 编译
 ./build-linux.sh -t rk3576 -a aarch64 -b Release
 (toolkit2) ip3@ip3-MOMENTPLUS-M2:~/work/linux/Projects/rknn-toolkit2/rknpu2/examples/rknn_yolov5_demo$ ./build-linux.sh -t rk3576 -a aarch64 -b Release
./build-linux.sh -t rk3576 -a aarch64 -b Release
/home/ip3/work/linux/Projects/gcc-arm-10.3-2021.07-x86_64-aarch64-none-linux-gnu/bin/aarch64-rockchip1031-linux-gnu
===================================
TARGET_SOC=RK3576
TARGET_ARCH=aarch64
BUILD_TYPE=Release
BUILD_DIR=/home/ip3/work/linux/Projects/rknn-toolkit2/rknpu2/examples/rknn_yolov5_demo/build/build_RK3576_linux_aarch64_Release
CC=/home/ip3/work/linux/Projects/gcc-arm-10.3-2021.07-x86_64-aarch64-none-linux-gnu/bin/aarch64-rockchip1031-linux-gnu-gcc
CXX=/home/ip3/work/linux/Projects/gcc-arm-10.3-2021.07-x86_64-aarch64-none-linux-gnu/bin/aarch64-rockchip1031-linux-gnu-g++
===================================
-- Configuring done
-- Generating done
-- Build files have been written to: /home/ip3/work/linux/Projects/rknn-toolkit2/rknpu2/examples/rknn_yolov5_demo/build/build_RK3576_linux_aarch64_Release
Consolidate compiler generated dependencies of target rknn_yolov5_video_demo
Consolidate compiler generated dependencies of target rknn_yolov5_demo
[ 60%] Built target rknn_yolov5_video_demo
[100%] Built target rknn_yolov5_demo
[ 40%] Built target rknn_yolov5_demo
[100%] Built target rknn_yolov5_video_demo
Install the project...
-- Install configuration: "Release"
-- Up-to-date: /home/ip3/work/linux/Projects/rknn-toolkit2/rknpu2/examples/rknn_yolov5_demo/install/rknn_yolov5_demo_Linux/./rknn_yolov5_demo
-- Up-to-date: /home/ip3/work/linux/Projects/rknn-toolkit2/rknpu2/examples/rknn_yolov5_demo/install/rknn_yolov5_demo_Linux/lib/librknnrt.so
-- Up-to-date: /home/ip3/work/linux/Projects/rknn-toolkit2/rknpu2/examples/rknn_yolov5_demo/install/rknn_yolov5_demo_Linux/lib/librga.so
-- Up-to-date: /home/ip3/work/linux/Projects/rknn-toolkit2/rknpu2/examples/rknn_yolov5_demo/install/rknn_yolov5_demo_Linux/./model/RK3576
-- Up-to-date: /home/ip3/work/linux/Projects/rknn-toolkit2/rknpu2/examples/rknn_yolov5_demo/install/rknn_yolov5_demo_Linux/./model/RK3576/yolov5s-640-640.rknn
-- Up-to-date: /home/ip3/work/linux/Projects/rknn-toolkit2/rknpu2/examples/rknn_yolov5_demo/install/rknn_yolov5_demo_Linux/./model/bus.jpg
-- Up-to-date: /home/ip3/work/linux/Projects/rknn-toolkit2/rknpu2/examples/rknn_yolov5_demo/install/rknn_yolov5_demo_Linux/./model/coco_80_labels_list.txt
-- Up-to-date: /home/ip3/work/linux/Projects/rknn-toolkit2/rknpu2/examples/rknn_yolov5_demo/install/rknn_yolov5_demo_Linux/./rknn_yolov5_video_demo
-- Up-to-date: /home/ip3/work/linux/Projects/rknn-toolkit2/rknpu2/examples/rknn_yolov5_demo/install/rknn_yolov5_demo_Linux/lib/librockchip_mpp.so
-- Up-to-date: /home/ip3/work/linux/Projects/rknn-toolkit2/rknpu2/examples/rknn_yolov5_demo/install/rknn_yolov5_demo_Linux/lib/libmk_api.so


  
 
 # 进入 rknn_model_zoo 目录
cd Projects/rknn_model_zoo
# 运行 build-linux.sh 脚本
# 用法:./build-linux.sh -t <target> -a <arch> -d <build_demo_name> [-b <build_type>] [-m]
# -t : target (rk356x/rk3588) # 平台类型，rk3568/rk3566 都统一为rk356x
# -a : arch (aarch64/armhf) # 板端系统架构
# -d : demo name # 对应 examples 目录下子文件夹的名称，如yolov5、mobilenet
# -b : build_type(Debug/Release)
# -m : enable address sanitizer, build_type need set to Debug
chmod +x build-linux.sh
./build-linux.sh -t rk3576 -a aarch64 -d yolov5

(toolkit2) ip3@ip3-MOMENTPLUS-M2:~/work/linux/Projects/rknn_model_zoo$ ./build-linux.sh -t rk3576 -a aarch64 -d yolov5
./build-linux.sh -t rk3576 -a aarch64 -d yolov5
/home/ip3/work/linux/Projects/gcc-arm-10.3-2021.07-x86_64-aarch64-none-linux-gnu/bin/aarch64-rockchip1031-linux-gnu
===================================
BUILD_DEMO_NAME=yolov5
BUILD_DEMO_PATH=examples/yolov5/cpp
TARGET_SOC=rk3576
TARGET_ARCH=aarch64
BUILD_TYPE=Release
ENABLE_ASAN=OFF
DISABLE_RGA=OFF
INSTALL_DIR=/home/ip3/work/linux/Projects/rknn_model_zoo/install/rk3576_linux_aarch64/rknn_yolov5_demo
BUILD_DIR=/home/ip3/work/linux/Projects/rknn_model_zoo/build/build_rknn_yolov5_demo_rk3576_linux_aarch64_Release
CC=/home/ip3/work/linux/Projects/gcc-arm-10.3-2021.07-x86_64-aarch64-none-linux-gnu/bin/aarch64-rockchip1031-linux-gnu-gcc
CXX=/home/ip3/work/linux/Projects/gcc-arm-10.3-2021.07-x86_64-aarch64-none-linux-gnu/bin/aarch64-rockchip1031-linux-gnu-g++
===================================
-- Configuring done
-- Generating done
-- Build files have been written to: /home/ip3/work/linux/Projects/rknn_model_zoo/build/build_rknn_yolov5_demo_rk3576_linux_aarch64_Release
Consolidate compiler generated dependencies of target imageutils
Consolidate compiler generated dependencies of target fileutils
Consolidate compiler generated dependencies of target imagedrawing
Consolidate compiler generated dependencies of target audioutils
[ 25%] Built target imagedrawing
[ 41%] Built target fileutils
[ 50%] Built target audioutils
[ 66%] Built target imageutils
Consolidate compiler generated dependencies of target rknn_yolov5_demo
[100%] Built target rknn_yolov5_demo
[ 16%] Built target imageutils
[ 33%] Built target fileutils
[ 50%] Built target imagedrawing
[ 83%] Built target rknn_yolov5_demo
[100%] Built target audioutils
Install the project...
-- Install configuration: "Release"
-- Installing: /home/ip3/work/linux/Projects/rknn_model_zoo/install/rk3576_linux_aarch64/rknn_yolov5_demo/./rknn_yolov5_demo
-- Set runtime path of "/home/ip3/work/linux/Projects/rknn_model_zoo/install/rk3576_linux_aarch64/rknn_yolov5_demo/./rknn_yolov5_demo" to "$ORIGIN/lib"
-- Installing: /home/ip3/work/linux/Projects/rknn_model_zoo/install/rk3576_linux_aarch64/rknn_yolov5_demo/./model/bus.jpg
-- Installing: /home/ip3/work/linux/Projects/rknn_model_zoo/install/rk3576_linux_aarch64/rknn_yolov5_demo/./model/coco_80_labels_list.txt
-- Installing: /home/ip3/work/linux/Projects/rknn_model_zoo/install/rk3576_linux_aarch64/rknn_yolov5_demo/model/yolov5s_relu.rknn
-- Installing: /home/ip3/work/linux/Projects/rknn_model_zoo/install/rk3576_linux_aarch64/rknn_yolov5_demo/lib/librknnrt.so
-- Installing: /home/ip3/work/linux/Projects/rknn_model_zoo/install/rk3576_linux_aarch64/rknn_yolov5_demo/lib/librga.so

#编译之后的文件
(toolkit2) ip3@ip3-MOMENTPLUS-M2:~/work/linux/Projects/rknn_model_zoo/install$ tree
.
└── rk3576_linux_aarch64
    └── rknn_yolov5_demo
        ├── lib
        │   ├── librga.so
        │   └── librknnrt.so
        ├── model
        │   ├── bus.jpg
        │   ├── coco_80_labels_list.txt
        │   └── yolov5s_relu.rknn
        └── rknn_yolov5_demo



# 推送文件到板端
# 进入 rknn_model_zoo 目录
cd Projects/rknn_model_zoo
# 推送整个 rknn_yolov5_demo 文件夹到板端
# 注：rknn_yolov5_demo 文件夹下有一个同名的可执行文件 rknn_yolov5_demo
# 注：使用不同的模型和平台时，建议直接在 install 下找对应的路径
adb push install/rk3576_linux_aarch64/rknn_yolov5_demo /data/



# 板端运行 Demo
# 进入板端
adb shell
# 进入 rknn_yolov5_demo 目录
cd /data/rknn_yolov5_demo/
# 设置依赖库环境
export LD_LIBRARY_PATH=./lib
# 运行可执行文件
# 用法: ./rknn_yolov5_demo <model_path> <input_path>
./rknn_yolov5_demo model/yolov5s_relu.rknn  model/bus.jpg


# 查看结果
# 拉取到本地当前目录
adb pull /data/rknn_yolov5_demo/out.png .

在这里插入图片描述

RKLLM部署

# 代码 rknn-llm
# 环境 激活toolkit2环境
conda activate toolkit2

# 切换到rkllm-toolkit目录
cd rknn-llm/rkllm-toolkit/

# 安装rkllm_toolkit(文件请根据具体版本修改)，会自动下载RKLLM-Toolkit工具所需要的相关依赖包。
pip install packages/rkllm_toolkit-1.0.1-cp38-cp38-linux_x86_64.whl

#模型下载
cd Projects 
mkdir modem

Qwen-1_8B-Chat_downloader.py
from modelscope import snapshot_download
#model_dir = snapshot_download('qwen/Qwen-7B-Chat',cache_dir='./modem')
model_dir = snapshot_download('qwen/Qwen-1_8B-Chat',cache_dir='./modem')

运行Qwen-1_8B-Chat_downloader.py
python Qwen-1_8B-Chat_downloader.py



# 模型转换
# 在rknn-llm/rkllm-toolkit/examples/下新建 huggingface目录
# 新建download.py 代码如下:
from rkllm.api import RKLLM
# 模型路径
modelpath = '/home/ip3/work/linux/Projects/modem/qwen/Qwen-1_8B-Chat'
# 初始化RKLLM对象
llm = RKLLM()
# 模型加载
ret = llm.load_huggingface(model = modelpath)
if ret != 0:
    print('Load model failed!')
    exit(ret)
# 模型的量化构建
ret = llm.build(do_quantization=True, optimization_level=1, quantized_dtype='w8a8', target_platform='rk3576',num_npu_core=2)
if ret != 0:
    print('Build model failed!')
    exit(ret)
# 导出rkllm模型
ret = llm.export_rkllm("./qwen.rkllm")
if ret != 0:
    print('Export model failed!')
    exit(ret)



# RKLLM Runtime 编译
cd  Projects/rknn-llm/rkllm-runtime/examples/rkllm_api_demo

#设置编译工具连接

+++ b/rkllm-runtime/examples/rkllm_api_demo/build-linux.sh
@@ -1,10 +1,12 @@
 #!/bin/bash
+
 # Debug / Release / RelWithDebInfo
 if [[ -z ${BUILD_TYPE} ]];then
     BUILD_TYPE=Release
 fi

-GCC_COMPILER_PATH=~/gcc-arm-10.2-2020.11-x86_64-aarch64-none-linux-gnu/bin/aarch64-none-linux-gnu
+GCC_COMPILER_PATH=~/work/linux/Projects/gcc-arm-10.3-2021.07-x86_64-aarch64-none-linux-gnu/bin/aarch64-none-linux-gnu

(toolkit2) ip3@ip3-MOMENTPLUS-M2:~/work/linux/Projects/rknn-llm/rkllm-runtime/examples/rkllm_api_demo$ ./build-linux.sh
-- Configuring done
-- Generating done
-- Build files have been written to: /home/ip3/work/linux/Projects/rknn-llm/rkllm-runtime/examples/rkllm_api_demo/build/build_linux_aarch64_Release
Consolidate compiler generated dependencies of target llm_demo
Consolidate compiler generated dependencies of target multimodel_demo
[ 50%] Built target llm_demo
[100%] Built target multimodel_demo


(toolkit2) ip3@ip3-MOMENTPLUS-M2:~/work/linux/Projects/rknn-llm/rkllm-runtime/examples/rkllm_api_demo/build$ tree  -L 2
.
└── build_linux_aarch64_Release
    ├── CMakeCache.txt
    ├── CMakeFiles
    ├── cmake_install.cmake
    ├── llm_demo
    ├── Makefile
    └── multimodel_demo

2 directories, 5 files


# 推送板端

# 注意：要去到llm_demo 和 librkllmrt.so所在的目录下去执行
adb push llm_demo /data  # 将llm_demo使用ssh传输到开发板的家目录
adb push ./rkllm-runtime/runtime/Linux/librkllm_api/aarch64/librkllmrt.so /data/lib
adb push  Projects/rknn-llm/rkllm-toolkit/examples/huggingface/qwen.rkllm  /data

运行
adb shell  && cd data
cd librkllmrt.so所在的目录
export LD_LIBRARY_PATH=./lib

# 修改最大可打开文件的数量
ulimit -HSn 10240

root@linaro-alip:/data# ./llm_demo qwen.rkllm  128 4080
rkllm init start
I rkllm: rkllm-runtime version: 1.1.2, rknpu driver version: 0.9.8, platform: RK3576

# 或者运行绑定cpu运行测试
# taskset f0 ./llm_demo /path/qwen.rkllm

在这里插入图片描述


http://www.niftyadmin.cn/n/5743242.html

相关文章

[docker]拉取镜像失败

[docker]拉取镜像失败

List item vim /etc/docker/daemon.json在insecure-registries中添加你的socket&#xff08;IP端口号&#xff09; {"bip": "33.33.33.1/24","registry-mirrors": ["https://docker.mirrors.ustc.edu.cn/"],"insecure-registri…
阅读更多...
2024年国家信息安全水平等级考试NISP二级题目卷（二）（答案附后）

2024年国家信息安全水平等级考试NISP二级题目卷（二）（答案附后）

国家信息安全水平等级考试NISP二级卷为100道单选题&#xff0c;每题一分&#xff0c;共100分&#xff0c;70分合格&#xff0c;考试时长为120分钟。 1. 处理报废电脑的流程时&#xff0c;以下哪一个选项对于安全专业人员来说是最重要考虑的内容&#xff1f; A.在扇区这个级别…
阅读更多...
【科普笔记】——什么是环境变量

【科普笔记】——什么是环境变量

定义 环境变量&#xff08;environment variables&#xff09;是操作系统中用来指定操作系统运行环境的一些参数。它们是操作系统中的一个具有特定名字的对象&#xff0c;包含了一个或多个应用程序将使用到的信息。环境变量可以影响程序的运行方式&#xff0c;为系统和应用程序…
阅读更多...
输入两个整数求a~~b整数的和（c基础）

输入两个整数求a~~b整数的和（c基础）

#include<stdio.h> int main() {int a 0;int b 0;int answer 0;scanf_s("%d %d", &a, &b);a < b ? (a a, b b) : (a a b, b a - b, a a - b);for (a a; a < b; a)answer a;printf("%d", answer);return 0; } 介绍一下三目…
阅读更多...
【系统设计】数据库压缩技术详解：从基础到实践（附Redis内存优化实战案例）

【系统设计】数据库压缩技术详解：从基础到实践（附Redis内存优化实战案例）

概述 在现代数据库系统中&#xff0c;压缩技术对于提高存储效率和加速查询性能至关重要。特别是在处理大规模数据时&#xff0c;压缩能够极大地减少存储空间&#xff0c;并优化查询性能。本文将总结几种常见的压缩方式&#xff0c;并通过详细的解释和示例清晰地展示每种压缩方…
阅读更多...
Unreal5从入门到精通之如何在指定的显示器上运行UE程序

Unreal5从入门到精通之如何在指定的显示器上运行UE程序

前言 我们有一个设备，是一个带双显示器的机柜，主显示器是一个小竖屏，可以触屏操作，大显示器是一个普通的横屏显示器。我们用这个机柜的原因就是可以摆脱鼠标和键盘，直接使用触屏操作，又可以在大屏观看，非常适合用于教学。 然后我们为这款机柜做了很多个VR项目，包括Uni…
阅读更多...
qt QDir详解

qt QDir详解

1、概述 QDir是Qt框架中的一个核心类&#xff0c;它提供了对文件系统目录的操作接口。Qt是一个跨平台的应用程序开发框架&#xff0c;广泛用于开发桌面、移动和嵌入式设备上的应用程序。QDir类使得开发者能够方便地在不同操作系统上处理目录和文件&#xff0c;如进行目录遍历、…
阅读更多...
C语言 -- qsort的简单使用

C语言 -- qsort的简单使用

qsort函数 一、介绍二、语法格式三、使用函数从小到大从大到小 四、结语 一、介绍 qsort 函数是 C 标准库中的一个通用排序函数&#xff0c;用于对数组进行快速排序。它定义在 <stdlib.h> 头文件中。这个非常灵活&#xff0c;因为它允许用户指定数组的元素类型、数组的大…
阅读更多...
最新文章

Copyright @ 2022~2023