全志V853 NPU 踩坑记录

阿志小管家

发布于 2024-02-02 17:06:43

1030

发布于 2024-02-02 17:06:43

文章被收录于专栏：全志嵌入式那些事全志嵌入式那些事

首先的步骤，安装模型转换工具下载Verisilicon_Tool_Acuity_Toolkit和Verisilicon_Tool_VivanteIDE，按照文档安装配置

Vivante_IDE居然有Windows版本的，还亏我费劲装了个Ubuntu虚拟机

好像需要License？

去申请了一个license，马上就到货了

先准备下模型，去darknet上下载预先训练好了的 https://pjreddie.com/darknet/

配置一下环境变量，写了个脚本干这个

export ACTU_BASE=$(ls | grep acu*) && \
export ACTU_IDE_BASE=$(ls | grep *IDE*) && \
echo -e "ACUITY_TOOLS_METHOD='$PWD/$ACTU_BASE'\nexport ACUITY_PATH='$PWD/$ACTU_BASE/bin/'\nexport VIV_SDK='$PWD/$ACTU_IDE_BASE/cmdtools'\nexport PATH=$PATH:$PWD/$ACTU_BASE/bin/:$PWD/$ACTU_IDE_BASE/ide/\nexport pegasus=$PWD/$ACTU_BASE/bin/pegasus\nalias pegasus=$PWD/$ACTU_BASE/bin/pegasus" >> ~/.bashrc && \
source ~/.bashrc

按照文档说明导入模型，转换模型，量化，测试，导出小端模型

报错了

Create Neural Network: 208ms or 208405us
E [../vnn_pre_process.c:_decode_jpeg:279]CHECK PTR 279
E [../vnn_pre_process.c:_get_jpeg_data:622]CHECK PTR 622
E [../vnn_pre_process.c:_handle_multiple_inputs:762]CHECK PTR 762
E [../vnn_pre_process.c:vnn_PreProcessYolov3prj:910]CHECK STATUS(-1:A generic error code, used when no other describes the error.)
E [../main.c:main:241]CHECK STATUS(-1:A generic error code, used when no other describes the error.)

顺着调用查了一下

bmpFile = fopen( name, "rb" );

这里报错了

打印一下看看，好像很正常

    printf("%s", name);

    bmpFile = fopen( name, "rb" );
    TEST_CHECK_PTR(bmpFile, final);

把TEST_CHECK注释了看看

//TEST_CHECK_PTR(bmpFile, final);

试试用这个记录下运行时间

#define BILLION                                 1000000000
static uint64_t get_perf_count()
{
#if defined(__linux__) || defined(__ANDROID__) || defined(__QNX__) || defined(__CYGWIN__)
    struct timespec ts;

    clock_gettime(CLOCK_MONOTONIC, &ts);

    return (uint64_t)((uint64_t)ts.tv_nsec + (uint64_t)ts.tv_sec * BILLION);
#elif defined(_WIN32) || defined(UNDER_CE)
    LARGE_INTEGER ln;

    QueryPerformanceCounter(&ln);

    return (uint64_t)ln.QuadPart;
#endif
}

int main(){
    tmsStart = get_perf_count();

    // xxx

    tmsEnd = get_perf_count();
    msVal = (tmsEnd - tmsStart)/1000000;
    usVal = (tmsEnd - tmsStart)/1000;
}

好像还是没啥用，貌似是图片没打开？加一行试试看

    if(bmpFile == NULL){
    	printf("File Open Error");
    }

果然，图片路径设置错误了，打错路径了

input tensor 生成了

补充下模型转换过程：

1）导入模型

pegasus import darknet --model yolov3.cfg --weights yolov3.weights --output-model yolov3.json --output-data yolov3.data

2）创建 YML 文件对网络的输入和输出的超参数进行描述以及配置

pegasus generate inputmeta --model yolov3.json --input-meta-output yolov3_inputmeta.yml
pegasus generate postprocess-file --model yolov3.json --postprocess-file-output yolov3_postprocessmeta.yml

3）量化

pegasus quantize --model yolov3.json --model-data yolov3.data --batch-size 1 --device CPU --with-input-meta yolov3_inputmeta.yml --rebuild --model-quantize yolov3.quantize --quantizer asymmetric_affine --qtype uint8

4）预推理

pegasus inference --model yolov3.json --model-data yolov3.data --batch-size 1 --dtype quantized --model-quantize yolov3.quantize --device CPU --with-input-meta yolov3_inputmeta.yml --postprocessfile yolov3_postprocessmeta.yml

5）导出模型

pegasus export ovxlib --model yolov3.json --model-data yolov3.data --dtype quantized --model-quantize yolov3.quantize --batch-size 1 --save-fused-graph --target-ide-project 'linux64' --with-input-meta yolov3_inputmeta.yml --output-path ovxilb/yolov3/yolov3prj --pack-nbg-unify --postprocess-file yolov3_postprocessmeta.yml --optimize "VIP9000PICO_PID0XEE" --viv-sdk ${VIV_SDK}

编译了vpm_run作为板子的运行器

make -f makefile.linux

成功运行npu，网络yolov3

root@TinaLinux:/mnt/UDISK# ./vpm_run sample.txt
vpm_run sample.txt loop_run_coun[  720.963277] npu[4a6][4a6] vipcore, device ini                                                                                                                         t..
t device_id
    sample.txt: to[  720.971229] set_vip_power_clk ON
 include one ore more network bi[  720.979698] enter aw vip mem alloc size 10485                                                                                                                         76
nary graph (NBG) data file resou[  720.985630] aw_vip_mem_alloc vir 0xe2101000,                                                                                                                          phy 0x48d00000
rce.  See sample.txt for details[  720.994403] npu[4a6][4a6] gckvip_drv_init  ke                                                                                                                         rnel logical phy address=0x48d00000  virtual =0xe2101000
.
loop_run_count: the number of loop run network.
device_id: specify this NBG runs device.
example: ./vpm_run sample.txt 1 1, specify the NBG runs on device 1.
         ./vpm_run sample.txt 1000, run this network 1000 times.

test started.

init vip lite, driver version=0x00010800...
[0x4dd08]vip_init[104],
The version of Viplite is: 1.8.0-0-AW-2022-04-21
vip lite init OK.

cid=0xee, device_count=1
  device[0] core_count=1
init test resources, batch_count: 1 ...
create/prepare networks ...
batch i=0, binary name: ./network_binary.nb
[  721.472066] enter aw vip mem alloc size 38656768
[  721.499090] aw_vip_mem_alloc vir 0xe2202000, phy 0x48e00000
input 0 dim 320 320 3 1, data_fo[  721.574304] enter aw vip mem alloc size 49162                                                                                                                         24
rmat=2, quant_format=2, name=input[0], scale=0.003900, zero_poin[  721.584850] a                                                                                                                         w_vip_mem_alloc vir 0xe46e1000, phy 0x4b300000
t=0
ouput 0 dim 10 10 255 1, data_format=2, name=uid_198_out_0, scale=0.100275, zero                                                                                                                         _point=196
ouput 1 dim 20 20 255 1, data_format=2, name=uid_224_out_0, scale=0.103535, zero                                                                                                                         _point=201
ouput 2 dim 40 40 255 1, data_format=2, name=uid_250_out_0, scale=0.140436, zero                                                                                                                         _point=218
nbg name=./network_binary.nb
create network 3: 102471 us.
memory pool size=4916224byte
input 0 name: ./input_0.dat
prepare network 0: 27407 us.
batch: 0, loop count: 1
start to run network=./network_binary.nb
run time for this network 0: 107790 us.
run network done...
profile inference time=107636us, cycle=55530353
******* nb TOP5 ********
 --- Top5 ---
17545: 5.815971
17546: 5.214319
17555: 5.214319
17544: 5.013768
17554: 4.813218
******* nb TOP5 ********
 --- Top5 ---
12991: 5.176749
13011: 4.969679
2095: 4.348469
12992: 4.348469
46991: 4.348469
******* nb TOP5 ********
 --- Top5 ---[  723.594634] aw_vip_mem_free vir 0xe46e1000, phy 0x4b300000

51983: 5.196128
187983: 5.055[  723.600977] aw_vip_mem_free dma_unmap_sg_atrs
693
188023: 5.055693
51984: 4.[  723.608738] aw_vip_mem_free ion_unmap_kernel
493949
52023: 4.213077
[  723.616315] aw_vip_mem_free ion_free
[  723.622426] aw_vip_mem_free ion_client_destroy
[  723.630841] aw_vip_mem_free vir 0xe2202000, phy 0x48e00000
[  723.637162] aw_vip_mem_free dma_unmap_sg_atrs
[  723.642126] aw_vip_mem_free ion_unmap_kernel
[  723.651382] aw_vip_mem_free ion_free
[  723.655482] aw_vip_mem_free ion_client_destroy
destroy teset resource batch_count=1
[  723.708967] npu[4a6][4a6] gckvip_drv_exit, aw_vip_mem_free
[  723.715276] aw_vip_mem_free vir 0xe2101000, phy 0x48d00000
[  723.721422] aw_vip_mem_free dma_unmap_sg_atrs
[  723.726398] aw_vip_mem_free ion_unmap_kernel
[  723.731221] aw_vip_mem_free ion_free
[  723.735279] aw_vip_mem_free ion_client_destroy
[  723.740762] npu[4a6][4a6] vipcore, device un-init..