鲁班猫4跑监控检测实验的问题

systemchip

hi，各位好

最近买了一个鲁班猫4 4+32版本
在tf卡上装了ubuntu 22.04 server版（百度云上3月的镜像20240314-ubuntu-22.04.4-server-arm64-lubancat-4）
摄像头用的是OV5693 USB摄像头

想跑一下咱们的监控检测的小实验
按照教程安装好了rknn-toolkit-lite2，版本是1.6.0，使用pip3 list命令可以查看到

将rknpu2/runtime/Linux/librknn_api/aarch64下面的librknnrt.so也拷贝到了/usr/lib下面
rknn_toolkit_lite2下面的example也可以正常跑


然后我clone了lubancat-flask-opencv-rknn，修改了main.py中的ip，查看了摄像头id是默认的0

最后执行sudo python3 main.py

报了个错，提示：ModuleNotFoundError: No module named 'rknnlite'

请各位指点一下这个可能是哪里出了问题，应该从哪查起。
感谢您的回复。

systemchip

好吧，帖子才放出来
这个问题我已经解决了

原因是运行例子使用的是sudo python3 main.py
而安装rknn_toolkit_lite2用的命令是pip3 install rknn_toolkit_lite2-1.6.0-cp310-cp310-linux_aarch64.whl

解决方案是使用sudo pip3 install rknn_toolkit_lite2-1.6.0-cp310-cp310-linux_aarch64.whl
再安装一遍

但是我先遇到了一个新问题
E RKNN: [16:12:13.513] failed to check rknpu hardware version: 0x46495245
E RKNN: [16:12:13.513] This rknn model is for RK3568, but current platform is RK3588
E RKNN: [16:12:13.513] Import rknn model failed!
E RKNN: [16:12:13.513] rknn_init, load model failed!

也就是说，lubancat-flask-opencv-rknn/controller/utils下面的yolov5s.rknn是针对rk3568编译的，rk3588用不了
这个问题那位大神能解答下呢？

おおおじ。

https://gitee.com/LubanCat/luban ... es/yolov5_inference

这里有3588的

systemchip

おおおじ。 发表于 2024-4-10 17:53
https://gitee.com/LubanCat/lubancat_ai_manual_code/tree/master/dev_env/rknn_toolkit_lite2/examples/y ...

感谢，我试一下

systemchip

おおおじ。 发表于 2024-4-10 17:53
https://gitee.com/LubanCat/lubancat_ai_manual_code/tree/master/dev_env/rknn_toolkit_lite2/examples/y ...

试了一下，不报模型的错误
页面也能出摄像头的图像，但是一点击检测就画面就不动了，命令行的log如下：
I:werkzeug:172.16.11.32 - - [11/Apr/2024 09:17:04] "GET / HTTP/1.1" 200 -
[ WARN:0] global ./modules/videoio/src/cap_gstreamer.cpp (616) isPipelinePlaying OpenCV | GStreamer warning: GStreamer: pipeline have not been created
I:werkzeug:172.16.11.32 - - [11/Apr/2024 09:17:04] "GET /static/button_process.js HTTP/1.1" 304 -
--> Load RKNN model
--> Init runtime environment
I RKNN: [09:17:04.600] RKNN Runtime Information, librknnrt version: 2.0.0b0 (35a6907d79@2024-03-24T10:31:14)
I RKNN: [09:17:04.601] RKNN Driver Information, version: 0.9.3
I RKNN: [09:17:04.601] RKNN Model Information, version: 4, toolkit version: 1.5.0+1fa95b5c(compiler version: 1.5.0 (e6fe0c678@2023-05-25T16:15:03)), target: RKNPU v2, target platform: rk3588, framework name: ONNX, framework layout: NCHW, model inference type: static_shape
I:werkzeug:172.16.11.32 - - [11/Apr/2024 09:17:04] "GET /video_viewer HTTP/1.1" 200 -
I:werkzeug:172.16.11.32 - - [11/Apr/2024 09:17:14] "POST /process_status HTTP/1.1" 200 -
E Catch exception when setting inputs.
E Traceback (most recent call last):
  File "/usr/local/lib/python3.10/dist-packages/rknnlite/api/rknn_lite.py", line 201, in inference
    self.rknn_runtime.set_inputs(inputs, data_type, data_format, inputs_pass_through=inputs_pass_through)
  File "rknnlite/api/rknn_runtime.py", line 1082, in rknnlite.api.rknn_runtime.RKNNRuntime.set_inputs
Exception: The input[0] need 4dims input, but 3dims input buffer feed.

E:werkzeug:Error on request:
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/werkzeug/serving.py", line 319, in run_wsgi
    execute(self.server.app)
  File "/usr/lib/python3/dist-packages/werkzeug/serving.py", line 310, in execute
    for data in application_iter:
  File "/usr/lib/python3/dist-packages/werkzeug/wsgi.py", line 462, in __next__
    return self._next()
  File "/usr/lib/python3/dist-packages/werkzeug/wrappers/response.py", line 49, in _iter_encoded
    for item in iterable:
  File "/home/cat/lubancat-flask-opencv-rknn/controller/modules/home/views.py", line 29, in video_stream
    frame = video_camera.get_frame()
  File "/home/cat/lubancat-flask-opencv-rknn/controller/utils/camera.py", line 75, in get_frame
    self.frame = process_image(self.image, self.outputs)
  File "/home/cat/lubancat-flask-opencv-rknn/controller/utils/rknn_image.py", line 227, in process_image
    input0_data = outputs[0]
TypeError: 'NoneType' object is not subscriptable

麻烦指点一下，十分感谢

systemchip

systemchip 发表于 2024-4-11 09:39
试了一下，不报模型的错误
页面也能出摄像头的图像，但是一点击检测就画面就不动了，命令行的log如下：
I ...

我感觉是摄像头导致
我还有个鲁班猫0
在鲁班猫0上使用mipi摄像头就没问题，使用usb摄像头也报错

鲁班猫4好像只支持usb摄像头

YiangYiang

Exception: The input[0] need 4dims input, but 3dims input buffer feed.
参数有缺，你试试在py文件的img = cvtcolor xxxxxxxx字句下面加一行参数
img =np.expand_dims(img,0)
再试试看

systemchip

YiangYiang 发表于 2024-4-12 12:00
Exception: The input[0] need 4dims input, but 3dims input buffer feed.
参数有缺，你试试在py文件的img ...

感谢您的指点
我在camera.py中添加了
self.image = np.expand_dims(self.image, 0)

问题依旧，但是报错发生变化了

I:werkzeug:172.16.11.32 - - [12/Apr/2024 16:47:05] "GET /video_viewer HTTP/1.1" 200 -
I:werkzeug:172.16.11.32 - - [12/Apr/2024 16:47:08] "POST /process_status HTTP/1.1" 200 -
E RKNN: [16:47:08.887] rknn_inputs_set, param input size(921600) < model input size(1228800)
E Catch exception when setting inputs.
E Traceback (most recent call last):
  File "/usr/local/lib/python3.10/dist-packages/rknnlite/api/rknn_lite.py", line 201, in inference
    self.rknn_runtime.set_inputs(inputs, data_type, data_format, inputs_pass_through=inputs_pass_through)
  File "rknnlite/api/rknn_runtime.py", line 1164, in rknnlite.api.rknn_runtime.RKNNRuntime.set_inputs
Exception: Set inputs failed. error code: RKNN_ERR_PARAM_INVALID

E:werkzeug:Error on request:
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/werkzeug/serving.py", line 319, in run_wsgi
    execute(self.server.app)
  File "/usr/lib/python3/dist-packages/werkzeug/serving.py", line 310, in execute
    for data in application_iter:
  File "/usr/lib/python3/dist-packages/werkzeug/wsgi.py", line 462, in __next__
    return self._next()
  File "/usr/lib/python3/dist-packages/werkzeug/wrappers/response.py", line 49, in _iter_encoded
    for item in iterable:
  File "/home/cat/lubancat-flask-opencv-rknn/controller/modules/home/views.py", line 29, in video_stream
    frame = video_camera.get_frame()
  File "/home/cat/lubancat-flask-opencv-rknn/controller/utils/camera.py", line 76, in get_frame
    self.frame = process_image(self.image, self.outputs)
  File "/home/cat/lubancat-flask-opencv-rknn/controller/utils/rknn_image.py", line 227, in process_image
    input0_data = outputs[0]
TypeError: 'NoneType' object is not subscriptable
I:werkzeug:172.16.11.32 - - [12/Apr/2024 16:47:26] "POST /process_status HTTP/1.1" 200

能否再帮忙看下呢，感谢

YiangYiang

可能是摄像头没有获取到图片导致的检测失败，你可以在获取图片那里加个判断，看看有没有图片，然后再检测