How we can read OV13850 camera from OpenCV ?

[ISTANBUL]

Hi,

I installed friendly Desktop ubuntu, and cant read the OV13850 MIPI camera from VideoCapture cap(0) ,

gst-camera.sh is working.

How I can read video stream from camera with openCV ?

pls help

[ayaromenok]

1) V4L:

Code: Select all

VideoCapture capture("/dev/video0", CAP_V4L); 

note: video in NV12 format, so need to be decoded to RGB/BGR to use in OpenCV operations

2) GST:

Code: Select all

 VideoCapture capture(" rkisp device=/dev/video0 sensor-id=1 io-mode=4 path-iqf=/etc/cam_iq/rk-ov13850.xml ! video/x-raw,format=NV12,width=640,height=480,framerate=30/1 ! videoconvert ! appsink",CAP_GSTREAMER);

[NoXQS]

Trying to figure out how to convert NV12 to be useful in OpenCV. Seems (python)

Code: Select all

 frame = cv2.cvtColor(frame, cv2.COLOR_YUV2BGR_NV12)

alone is not enough. There are many color types.
Any hint how to convert NV12 to BGR ?

[ayaromenok]

Trying to figure out how to convert NV12 to be useful in OpenCV. Seems (python)

Code: Select all

 frame = cv2.cvtColor(frame, cv2.COLOR_YUV2BGR_NV12)

alone is not enough. There are many color types.
Any hint how to convert NV12 to BGR ?

It is not a good idea - instead of use Rockchip ISP hardware you will use OpenCV CPU/OpenCL to make this conversion.

In my case usage of V4L2 from OpenCV in resolutions, smaller than a full frame (13MPix or 4MPix), make a freeze a board (but maybe it's just a issue with my board only). Performance on full-frame resolution is not perfect too (much less, than a 30fps).

So looks like gstreamer solution (from my post above - which provide a BGR/RGB cv::Mat image as result) is only suitable case

[NoXQS]

Ofcourse best is ISP hardware solution so we keep the CPU free for userspace applications.
I hope/think we will get this but for testing my application I can work at lower FPS (for now).

In another forum thread ( https://www.friendlyarm.com/Forum/viewtopic.php?f=61&t=2133#p5864 ) I saw you are experimenting with the Armbian distro and enabling the ROCKCHIP_ISP in the kernel config.

Any hope of success ? Las time I ran armbian I got exactly the same errors opening the camera by index as on FriendlyDesktop.

[ayaromenok]

Any hope of success ? Las time I ran armbian I got exactly the same errors opening the camera by index as on FriendlyDesktop.

It's always hope Right now Armbian came with kernel, where ISP and both camera drivers disabled. so, simple reconfiguration and kernel recompile give some access to ISP: it's list of supported formats from camera, but can't start gathering image. Looks like some additional configuration required(and I don't get Media Configuration API device right now on my system: only V4L2 ).

[JAGITA]

Trying to figure out how to convert NV12 to be useful in OpenCV. Seems (python)

Code: Select all

 frame = cv2.cvtColor(frame, cv2.COLOR_YUV2BGR_NV12)

alone is not enough. There are many color types.
Any hint how to convert NV12 to BGR ?

It is not a good idea - instead of use Rockchip ISP hardFileZilla Malwarebytes Rufusware you will use OpenCV CPU/OpenCL to make this conversion.

In my case usage of V4L2 from OpenCV in resolutions, smaller than a full frame (13MPix or 4MPix), make a freeze a board (but maybe it's just a issue with my board only). Performance on full-frame resolution is not perfect too (much less, than a 30fps).

So looks like gstreamer solution (from my post above - which provide a BGR/RGB cv::Mat image as result) is only suitable case

I installed friendly Desktop ubuntu, and cant read the OV13850 MIPI camera from VideoCapture cap(0) ,

gst-camera.sh is working.

How I can read video stream from camera with openCV ?

[knoname]

I am attempting to use an M4 + dual OV13850's + OpenCV + Ubuntu. In addition, I need to control the camera manually for focus, exposure, white balance, etc. I have been blocked trying to read even a single camera from OpenCV for several months.

The gst and v4l2 solutions listed by ayaromenok earlier do not seem to work on my system. gst freezes my system. v4l2 works briefly and then freezes just reading the video port after a few reads (no attempt to display). I suspect v4l2 method could work if there was a built-in conversion to bgr and the image size was sensed correctly -- my guess is a buffer overrun of some sort.

Once the image can be read and displayed in OpenCV, the next challenge is there isn't a method to control the camera settings in Ubuntu?

gst-camera.sh works fine. I am using downloaded friendly desktop and downloaded friendly opencv (trying to keep known sources until *something* works).

I hope I am simply missing something stupid on my part. I would appreciate any suggestions.

[ayaromenok]

I installed friendly Desktop ubuntu, and cant read the OV13850 MIPI camera from VideoCapture cap(0) ,
gst-camera.sh is working.

main idea to use camera in opencv: if gst-camera command working, you need to grab command line from it change last parameter: from XXXsink to appsink

maybe you need to change a resolution/check a path to camera file

[ayaromenok]

I am attempting to use an M4 + dual OV13850's + OpenCV + Ubuntu. In addition, I need to control the camera manually for focus, exposure, white balance, etc. I have been blocked trying to read even a single camera from OpenCV for several months.

The gst and v4l2 solutions listed by ayaromenok earlier do not seem to work on my system. gst freezes my system. v4l2 works briefly and then freezes just reading the video port after a few reads (no attempt to display). I suspect v4l2 method could work if there was a built-in conversion to bgr and the image size was sensed correctly -- my guess is a buffer overrun of some sort.

my suggestion - take a raspberry pi compute+stereo module(http://stereopi.com/) or ordinary I/O dev board - it's working almost from the box with stereo(need to update cam config as described here). Please note, that not all clones of RPi cameras support focus,WB, etc

regarding M4/Rk3399 - I didn't manage to make even one camera(I have 4MP cam, but it's similar to 13MP) working for longer than a 20 minutes in OpenCV - it's just freeze after some time. Dual-camera mode not working at all due to 4 line CSI cameras, while Rk ISP support only 6(so second camera don't have enough lines). maybe it can be fixed with switching camera config to 2x2 lines, but I didn't manage to do this.

[Fx7py]

1) V4L:

Code: Select all

VideoCapture capture("/dev/video0", CAP_V4L); 

note: video in NV12 format, so need to be decoded to RGB/BGR to use in OpenCV operations

2) GST:

Code: Select all

 VideoCapture capture(" rkisp device=/dev/video0 sensor-id=1 io-mode=4 path-iqf=/etc/cam_iq/rk-ov13850.xml ! video/x-raw,format=NV12,width=640,height=480,framerate=30/1 ! videoconvert ! appsink",CAP_GSTREAMER);

We just bought the T4 because it says that 2 simul 18350 cameras are supported under Ubuntu.
Can anyone confirm this ?
Can anyone post a short py3 code snippet on how to access even one camera ? Following your line of code, I get an error CAP_V4L is not defined when calling videoCap0 = cv2.VideoCapture("/dev/video0, CAP_V4L)

Thanks a lot

[Fx7py]

1) V4L:

Code: Select all

VideoCapture capture("/dev/video0", CAP_V4L); 

note: video in NV12 format, so need to be decoded to RGB/BGR to use in OpenCV operations

2) GST:

Code: Select all

 VideoCapture capture(" rkisp device=/dev/video0 sensor-id=1 io-mode=4 path-iqf=/etc/cam_iq/rk-ov13850.xml ! video/x-raw,format=NV12,width=640,height=480,framerate=30/1 ! videoconvert ! appsink",CAP_GSTREAMER);

Thanks for this. It helps a lot, but could you post the import statements for both please ? CAP_V4L is unknown to cv when I try cap = VideoCapture("dev/video0", CAP_V4L)

[ayaromenok]

CAP_V4L is unknown to cv when I try cap = VideoCapture("dev/video0", CAP_V4L)

You need to be sure, that you build OpenCV with Video4Linux2 (V4L) support - if you want to use this API.