CN117170896A - Image acquisition method based on QNX Hypervisor - Google Patents

Abstract

The invention discloses a QNX Hypervisor-based image acquisition method in the field of intelligent cabins of automobiles, which comprises the following steps of 1, starting a proxy main process; step 2, the main process creates a bufferq initialization thread; step 3, the main process creates a bufferq handle thread; step 4, the main process creates a signal semaphore processing thread; and 5, initializing the kernel v4l2 loopback equipment by the main process. The problems of low efficiency, large delay and low safety of conventional image transmission are solved, the difficulty of application development of camera at the android client is simplified, the camera image can be directly acquired by using a native android camera2 interface, and the requirements of image data transmission among different domains of an intelligent cabin system are better met.

Description

Image acquisition method based on QNX Hypervisor

Technical Field

The invention relates to a QNX Hypervisor-based image acquisition method in the field of intelligent cabins of automobiles.

Background

The traditional vehicle-mounted operating system is a single operating system, the instrument is one set of system, the video entertainment IVI is another set of system, the hardware and the software of the two sets of systems are independent, and the two sets of systems interact data through a CAN bus. The intelligent cabin system instrument and the entertainment are one set of hardware and one set of software, but a plurality of domains are isolated and used for the instrument and the entertainment respectively, so that the functional safety can be met.

Different domains run different operating systems, such as QNX, android and the like, and image transmission requirements exist among the different systems, such as an instrument transmits a camera image to an IVI for processing and displaying. The conventional method is that the QNX host server sends image data in a virtual local area network VLAN mode, and the Android client receives the data and displays the data in a surface view after processing, but the mode is low in efficiency, large in delay and low in safety, is not suitable for scenes with high practicality requirements, and has the problem that if a plurality of applications need to acquire the image data, each application needs to independently write a receiving code, and the development difficulty is high. In addition, the current QNX hypersor+android GVM system cannot support the native Android camera2 interface to acquire the camera image.

Disclosure of Invention

The invention aims to provide an image acquisition method based on QNX Hypervisor, which solves the problems of low efficiency, large delay and low safety of conventional image transmission, simplifies the difficulty of camera application development at an android client end, can directly acquire a camera image by using a native android camera2 interface, and better meets the requirements of image data transmission among different domains of an intelligent cabin system.

In order to achieve the above object, the present invention provides a QNX Hypervisor-based image acquisition, comprising the following,

step 1, starting a proxy main process;

step 2, the main process creates a bufferq initialization thread;

step 3, the main process creates a bufferq handle thread;

step 4, the main process creates a signal semaphore processing thread;

and 5, initializing the kernel v4l2 loopback equipment by the main process.

As a further improvement of the present invention, the details of step 2 are as follows,

step 2.1, starting a bufferq initialization thread, wherein the thread is pulled up by a proxy main process;

step 2.2, allocating 6 image buffer buffers by using mmap, wherein each buffer has a size of 1920x720x 3= 4147200bytes;

step 2.3, initializing a buffer_share_init () interface to obtain a buffer_share_init () object instance;

and 2.4, obtaining 6 buffer addresses by using a buffer_share_preparation_buffer () interface, wherein the addresses are used for writing subsequent image data.

As a further improvement of the present invention, the details of step 3 are as follows,

step 3.1, checking whether the bufferq is initialized, if so, calling a bufferq_fetch_new_frame () function, if not, indicating that the qnx end sends the service abnormality of the image data, and calling the bufferq_fill_frame_buffer () function;

step 3.2, the image acquisition data sent by the qnx end is acquired by continuously cycling the buffer_share_ dequeue, buffer _share_request through the buffer_fetch_new_frame () function, the acquired image data is in the format of BGRA8888, and then the acquired image data is converted into the UYVY format through the following formula:

then, the YUV data are arranged according to the sequence of UYVY and written into a v4l2 loopback buffer;

and 3.3, when the qnx end cannot normally send the image data, the android can receive the blue screen picture through the buffer_fill_frame_buffer () function, so that a user can conveniently judge the fault cause, wherein the principle of sending the blue screen picture is to write pure blue point UYVY data (0 xef,0x28,0x6d,0x 28) into each pixel of the v4l2 loopback buffer, and the corresponding RGB data is (0 x0,0 xff), namely pure blue.

As a further improvement of the present invention, the details of step 4 are as follows,

step 4.1, using sigwait to wait for the semaphore;

step 4.2, entering an exit flow when the semaphore is the following signal: SIGCHLD, SIGIO, SIGURG, SIGWINCH, SIGSTOP, SIGTSTP, SIGTTIN, SIGTTOU, SIGCONT;

and 4.3, after entering the exit flow, clearing the pipeline handle, otherwise, restarting the process, and then causing the fact that a new handle cannot be applied.

As a further improvement of the present invention, the details of step 5 are as follows,

step 5.1, calling an open function to open a v4l2 loopback node;

step 5.2, calling ioctl (v4l2_params- > v4l2sink, VIDIOC_G_FMT) to obtain the current format;

and 5.3, reconfiguring the format, and configuring the following parameters:

fmt.pix.width, fmt.pix.height, fmt.pix.pixelformat, fmt.pix.sizeimage, then call ioctl (v4l2_params- > v4l2sink, vidioc_s_fmt) for parameter issue;

and 5.4, exiting the thread after completing v4l2 parameter configuration.

Compared with the prior art, the method has the advantages that the method receives image buffer data through the buffer proxy, then simulates standard v4l2 image acquisition equipment through injecting the image buffer data into a v4l2 log back driver of a kernel, then a Camera Provider of a Native layer reads the v4l2 log back node through a HAL library, and transmits the image data to an APP through a Camera Service interface of an Android standard, wherein the buffer proxy is a proxy Service program of an Android end, is responsible for acquiring the Camera image data from qnx and transmitting the Camera image data to the v4l2 node of an Android kernel, so that an Android Native Camera Service can take the image data.

Drawings

Fig. 1 is a flow chart of a main process of the invention.

FIG. 2 is a flowchart of the bufferq initialization thread execution of the present invention.

FIG. 3 is a flow chart of the bufferq handle thread execution of the present invention.

FIG. 4 is a flow chart of the signal handle thread execution of the present invention.

FIG. 5 is a flow chart of the initialization v4l2 device execution of the present invention.

FIG. 6 is a data flow chart of the present invention.

FIG. 7 is a model diagram of a BufferQ proxy module of the present invention.

FIG. 8 is a diagram of a detailed function call relationship according to the present invention.

Fig. 9 is a flowchart of the start-up of an AVM application of the present invention.

FIG. 10 is a graph of the test results of the present invention.

Detailed Description

The invention is further described below with reference to the accompanying drawings:

an image acquisition method based on QNX Hypervisor as shown in FIGS. 1-10, comprises the following steps,

and step 1, starting a proxy main process.

Step 2, the main process creates a bufferq initialization thread; this thread is used to establish communications between qnx and android, i.e., bufferq communications.

Step 2.1, starting a bufferq initialization thread, wherein the thread is pulled up by a proxy main process;

step 2.2, allocating 6 image buffer buffers by using mmap, wherein each buffer has a size of 1920x720x 3= 4147200bytes;

step 2.3, initializing a buffer_share_init () interface to obtain a buffer_share_init () object instance;

step 2.4, obtaining 6 buffer addresses using the buffer_share_prepare_buffer () interface, the addresses to be used

For writing of subsequent image data.

Step 3, the main process creates a bufferq handle thread; the thread is used to handle bufferq data transfers, namely Queue Buff and Dequeue Buff.

Step 3.1, checking whether the bufferq is initialized, if so, calling a bufferq_fetch_new_frame () function, if not, indicating that the qnx end sends the service abnormality of the image data, and calling the bufferq_fill_frame_buffer () function;

step 3.2, the image acquisition data sent by the qnx end is acquired by continuously cycling the buffer_share_ dequeue, buffer _share_request through the buffer_fetch_new_frame () function, the acquired image data is in the format of BGRA8888, and then the acquired image data is converted into the UYVY format through the following formula:

then, the YUV data are arranged according to the sequence of UYVY and written into a v4l2 loopback buffer;

and 3.3, when the qnx end cannot normally send the image data, the android can receive the blue screen picture through the buffer_fill_frame_buffer () function, so that a user can conveniently judge the fault cause, wherein the principle of sending the blue screen picture is to write pure blue point UYVY data (0 xef,0x28,0x6d,0x 28) into each pixel of the v4l2 loopback buffer, and the corresponding RGB data is (0 x0,0 xff), namely pure blue.

Step 4, the main process creates a signal semaphore processing thread; the thread is used for monitoring the running state of the main process, for example, related resources need to be released when the main process is abnormally exited, so that the service can normally run after restarting.

Step 4.1, using sigwait to wait for the semaphore;

step 4.2, entering an exit flow when the semaphore is the following signal: SIGCHLD, SIGIO, SIGURG, SIGWINCH, SIGSTOP, SIGTSTP, SIGTTIN, SIGTTOU, SIGCONT;

and 4.3, after entering the exit flow, clearing the pipeline handle, otherwise, restarting the process, and then causing the fact that a new handle cannot be applied.

Step 5, initializing kernel v4l2 loopback equipment by the main process; this is a virtual device that receives image data from the proxy process and provides a standard interface for v4l2 to read by the android upper layer camera service.

Step 5.1, calling an open function to open a v4l2 loopback node;

step 5.2, calling ioctl (v4l2_params- > v4l2sink, VIDIOC_G_FMT) to obtain the current format;

and 5.3, reconfiguring the format, and configuring the following parameters:

fmt.pix.width, fmt.pix.height, fmt.pix.pixelformat, fmt.pix.sizeimage, then call ioctl (v4l2_params- > v4l2sink, vidioc_s_fmt) for parameter issue;

and 5.4, exiting the thread after completing v4l2 parameter configuration.

The invention is based on QNX standard HAB (Hypervisor Abstraction layer) architecture, receives image buffer data through autonomously developed buffer Q proxy, then simulates standard v4l2 image acquisition equipment through injecting into a kernel v4l2 log back driver, then a Camera Provider of Native layer reads v4l2 log back node through HAL library, and transmits the image data to APP through a Camera Service interface of Android standard, and a data flow chart is shown in figure 6.

The basic model of the BufferQ proxy module is shown in fig. 7, the detailed function call relationship is shown in fig. 8, the starting flow of the AVM application is shown in fig. 9, and after the AVM app is opened, the AVM picture is displayed on the right side of the screen, as shown in fig. 10.

According to the invention, the APP end does not need to care about a specific receiving mode of the image, and only needs to call according to the standard android camera interface, so that the APP software architecture is simplified, and the development efficiency is improved; in addition, the invention greatly improves the efficiency, delay and safety of image transmission.

The present invention is not limited to the above-described embodiments, and based on the technical solutions of the present disclosure, those skilled in the art may make some substitutions and modifications to some technical features thereof without creative efforts, which are all within the scope of the present invention.

Claims (5)

1. An image acquisition method based on QNX Hypervisor is characterized by comprising the following steps: comprises the steps of,

step 1, starting a proxy main process;

step 2, the main process creates a bufferq initialization thread;

step 3, the main process creates a bufferq handle thread;

step 4, the main process creates a signal semaphore processing thread;

and 5, initializing the kernel v4l2 loopback equipment by the main process.

2. The image acquisition method based on QNX hypersor according to claim 1, wherein the method comprises the following steps: the specific content of the step 2 is as follows,

step 2.1, starting a bufferq initialization thread, wherein the thread is pulled up by a proxy main process;

step 2.2, allocating 6 image buffer buffers by using mmap, wherein each buffer has a size of 1920x720x 3= 4147200bytes;

step 2.3, initializing a buffer_share_init () interface to obtain a buffer_share_init () object instance;

step 2.4, obtaining 6 buffer addresses using the buffer_share_prepare_buffer () interface, the addresses to be used

For writing of subsequent image data.

3. The image acquisition method based on QNX hypersor according to claim 2, wherein the method comprises the following steps: the specific content of step 3 is as follows,

step 3.1, checking whether the bufferq is initialized, if so, calling a bufferq_fetch_new_frame () function, if not, indicating that the qnx end sends the service abnormality of the image data, and calling the bufferq_fill_frame_buffer () function;

step 3.2, the image acquisition data sent by the qnx end is acquired by continuously cycling the buffer_share_ dequeue, buffer _share_request through the buffer_fetch_new_frame () function, the acquired image data is in the format of BGRA8888, and then the acquired image data is converted into the UYVY format through the following formula:

then, the YUV data are arranged according to the sequence of UYVY and written into a v4l2 loopback buffer;

and 3.3, when the qnx end cannot normally send the image data, the android can receive the blue screen picture through the buffer_fill_frame_buffer () function, so that a user can conveniently judge the fault cause, wherein the principle of sending the blue screen picture is to write pure blue point UYVY data (0 xef,0x28,0x6d,0x 28) into each pixel of the v4l2 loopback buffer, and the corresponding RGB data is (0 x0,0 xff), namely pure blue.

4. The image acquisition method based on QNX hypersor according to claim 3, wherein the method comprises the following steps: the details of step 4 are as follows,

step 4.1, using sigwait to wait for the semaphore;

step 4.2, entering an exit flow when the semaphore is the following signal: SIGCHLD, SIGIO, SIGURG, SIGWINCH, SIGSTOP, SIGTSTP, SIGTTIN, SIGTTOU, SIGCONT;

and 4.3, after entering the exit flow, clearing the pipeline handle, otherwise, restarting the process, and then causing the fact that a new handle cannot be applied.

5. The image acquisition method based on QNX hypersor according to claim 3, wherein the method comprises the following steps: the details of step 5 are as follows,

step 5.1, calling an open function to open a v4l2 loopback node;

step 5.2, calling ioctl (v4l2_params- > v4l2sink, VIDIOC_G_FMT) to obtain the current format;

and 5.3, reconfiguring the format, and configuring the following parameters:

fmt.pix.width, fmt.pix.height, fmt.pix.pixelformat, fmt.pix.sizeimage, then call ioctl (v4l2_params- > v4l2sink, vidioc_s_fmt) for parameter issue;

and 5.4, exiting the thread after completing v4l2 parameter configuration.