CN118264773A

CN118264773A - Vehicle-mounted video acquisition system, visual field linkage method and wearable electronic equipment

Info

Publication number: CN118264773A
Application number: CN202410352922.2A
Authority: CN
Inventors: 何思远; 刘豪; 薛念明
Original assignee: Guangzhou Haige Communication Group Inc Co
Current assignee: Guangzhou Haige Communication Group Inc Co
Priority date: 2024-03-26
Filing date: 2024-03-26
Publication date: 2024-06-28

Abstract

The invention provides a vehicle-mounted video acquisition system, a visual field linkage method and wearable electronic equipment. Wherein the system comprises: the first camera, the second camera and the third camera are arranged at the front part of the vehicle; the first camera is arranged between the second camera and the third camera in the first direction and the second direction, the visual angles of the first camera and the second camera are overlapped, the visual angles of the first camera and the third camera are overlapped, the first direction is the vehicle length direction, and the second direction is the vehicle width direction. The video images acquired by the first camera, the second camera and the third camera are fused and spliced to be more in line with the habit of human eyes, so that the distortion of pictures is avoided, and the driving safety of a remote control automobile is improved.

Description

Vehicle-mounted video acquisition system, visual field linkage method and wearable electronic equipment

Technical Field

The invention relates to the technical field of image processing, in particular to a vehicle-mounted video acquisition system, a visual field linkage method and wearable electronic equipment.

Background

In the process of remotely controlling the unmanned automobile, an operator needs to rely on the vehicle-mounted camera to transmit the real-time shooting image back to the control end for display, and the operator remotely controls the unmanned automobile according to the image. In the prior art, an ultra-large visual field is acquired through a wide-angle camera, but a picture shot by the wide-angle camera has a large distortion, so that the distance of a target point cannot be effectively judged, and potential safety hazards exist.

Disclosure of Invention

The invention provides a vehicle-mounted video acquisition system, a visual field linkage method and wearable electronic equipment, which are used for solving the defect that a wide-angle camera is adopted to acquire a large visual field and can cause distortion of a picture in the prior art, realizing acquisition of a panoramic image with low distortion degree and improving the safety of remote control driving.

The invention provides a vehicle-mounted video acquisition system, which comprises:

the first camera, the second camera and the third camera are arranged at the front part of the vehicle;

the first camera is arranged between the second camera and the third camera in a first direction and a second direction, the visual angles of the first camera and the second camera are overlapped, the visual angles of the first camera and the third camera are overlapped, the first direction is the vehicle length direction, and the second direction is the vehicle width direction.

The invention provides a vehicle-mounted video acquisition system, which further comprises:

Fourth and fifth cameras disposed at front and rear sides of the vehicle bottom, and an image transmission module;

the image transmission module is used for transmitting video images shot by the fourth camera and/or the fifth camera based on the vehicle running speed.

The invention provides a visual field linkage method, which comprises the following steps:

Acquiring a first image and displaying the first image in a first display area of a display screen of the wearable device, wherein the first image is obtained by splicing video images acquired by the first camera, the second camera and the third camera;

obtaining a visual field change angle, wherein the visual field change angle reflects the visual field change intention of a user;

And when the visual field change angle is within a preset range, rotating the position of the first image in the first display area based on the visual field change angle.

According to the visual field linkage method provided by the invention, the visual field change angle acquisition method comprises the following steps:

determining the view change angle based on sensor data;

the sensor data are acquired based on sensors of the wearable equipment.

According to the visual field linkage method provided by the invention, the method further comprises the following steps:

displaying a second image in a second display area on the display screen, wherein the second image is obtained based on shooting by a camera arranged at the rear part of the vehicle;

Acquiring a user eye image, and determining an eye closure state of a user based on the user eye image, wherein the eye closure state reflects whether the eyes of the user are open or closed;

And switching display contents of the first display area and the second display area based on the eye-closed state.

when the visual field change angle exceeds the preset range, displaying a third image or a fourth image in the first display area;

wherein the third image is an image reflecting the environment above the vehicle, and the fourth image is an image reflecting the environment below the vehicle.

The present invention also provides a field of view linkage comprising:

The image display module is used for acquiring a first image and displaying the first image in a first display area of a display screen of the wearable device, wherein the first image is obtained by splicing video images acquired by the first camera, the second camera and the third camera;

the visual field tracking module is used for acquiring a visual field change angle, and the visual field change angle reflects the visual field change intention of a user;

And the display adjustment module is used for rotating the position of the first image in the first display area based on the visual field change angle when the visual field change angle is in a preset range.

The invention also provides wearable electronic equipment, which comprises a memory, a processor and a computer program stored on the memory and capable of running on the processor, wherein the processor realizes any one of the visual field linkage methods when executing the program.

The present invention also provides a non-transitory computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements a field of view linkage method as described in any of the above.

The invention also provides a computer program product comprising a computer program which when executed by a processor implements a vision field linkage method as described in any one of the above.

According to the vehicle-mounted video acquisition system, the visual field linkage method and the wearable electronic device, the first camera, the second camera and the third camera which are arranged at the front part of the vehicle are used for acquiring images, the first camera is arranged between the second camera and the third camera in the vehicle length direction and the vehicle width direction, and the first camera is overlapped with the second camera and the second camera respectively in visual angles, so that the images acquired by the first camera, the second camera and the third camera are fused and spliced to be more in line with the habit of human eyes, distortion of images is avoided, and the safety of remote control of vehicle driving is improved.

Drawings

In order to more clearly illustrate the invention or the technical solutions of the prior art, the following description will briefly explain the drawings used in the embodiments or the description of the prior art, and it is obvious that the drawings in the following description are some embodiments of the invention, and other drawings can be obtained according to the drawings without inventive effort for a person skilled in the art.

FIG. 1 is a schematic diagram of an in-vehicle video acquisition system provided by the present invention;

FIG. 2 is a schematic diagram of a vehicle-mounted video acquisition system according to the second embodiment of the present invention;

FIG. 3 is a schematic diagram of an image of a vehicle bottom generated in a vehicle-mounted video acquisition system provided by the invention;

FIG. 4 is a schematic flow chart of a visual field linkage method provided by the invention;

FIG. 5 is a schematic view of a first image in a visual field linkage method according to the present invention;

FIG. 6 is a schematic view of a view linkage provided by the present invention;

fig. 7 is a schematic structural diagram of a wearable electronic device provided by the invention.

Detailed Description

For the purpose of making the objects, technical solutions and advantages of the present invention more apparent, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings, and it is apparent that the described embodiments are some embodiments of the present invention, not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.

In the process of remotely controlling the unmanned automobile, an operator needs to rely on the vehicle-mounted camera to transmit the real-time shooting image back to the control end for display, and the operator remotely controls the unmanned automobile according to the image. In the prior art, an ultra-large visual field is acquired through a wide-angle camera, but a picture shot by the wide-angle camera has a large distortion, so that the distance of a target point cannot be effectively judged, and potential safety hazards exist. Aiming at the defects, the invention provides a vehicle-mounted video acquisition system, a visual field linkage method and wearable electronic equipment, which are used for acquiring panoramic images with low distortion degree and improving the safety of remote control driving.

The following describes a vehicle-mounted video acquisition system provided by the invention with reference to the accompanying drawings, as shown in fig. 1, the system comprises: the first camera (A), the second camera (B) and the third camera (C) are arranged at the front part of the vehicle, so that an environment image of the front part of the vehicle can be collected to be used for providing information of the front part of the vehicle for operators, and the operators can drive and control based on the information.

Obviously, the front information of the vehicle is very important for the operator to drive and control, because the visual angle range of a common single camera is limited, and the images acquired by a plurality of cameras are displayed by adopting a plurality of screens, the control end is required to have a large space to accommodate a plurality of displays, the cost is high, in the prior art, a wide-angle camera can be adopted to enable a single image to have a larger visual field, but the wide-angle camera can generate distortion. In the vehicle-mounted video acquisition system provided by the invention, three common cameras are sequentially arranged in the vehicle width direction, the vehicle length direction is the direction of the straight running of the vehicle (such as the y direction in fig. 1), the vehicle width direction is perpendicular to the vehicle length direction (such as the x direction in fig. 1), and the three cameras are overlapped in view angle, so that information can be provided for splicing images acquired by the three cameras, the effect of acquiring a panoramic image can be realized by splicing and fusing the images acquired by the three cameras, and the spliced images have better continuity.

As shown in fig. 1, in the vehicle-mounted video acquisition system provided by the invention, the first camera (a) is located between the second camera (B) and the third camera (C) in the vehicle length direction and the vehicle width direction, that is, except for overlapping of the visual angles of the three cameras, the first camera and the third camera are sequentially behind the second camera in the vehicle length direction, so that the spliced and fused images can avoid distortion, and are more in line with the habit of human eyes.

Further, there is a 30 degree angle of view overlap region between the first camera and the second camera, and a 60 degree angle of view overlap region between the first camera and the third camera. The first camera is responsible for (30 degrees, 150 degrees) viewing angle, the second camera is responsible for (90 degrees, 210 degrees) viewing angle, and the third camera is responsible for (-30 degrees, 90 degrees) viewing angle. And the images respectively acquired based on the first camera, the second camera and the third camera are spliced and fused, so that a single image reflecting the real environment condition can be obtained.

Further, the vehicle-mounted video acquisition system provided by the invention further comprises three cameras arranged at the rear part of the vehicle, and the arrangement modes of the three cameras are consistent with the arrangement modes of the first camera, the second camera and the third camera, so that when an environment image (for example, when a vehicle is in a reverse direction) at the rear part of the vehicle needs to be acquired, panoramic images can be obtained through splicing the three cameras arranged at the rear part of the vehicle.

Further, as shown in fig. 2, the vehicle-mounted video acquisition system provided by the present invention further includes: and image composition modules for transmitting video images photographed by the fourth camera and/or the fifth camera based on a vehicle traveling speed.

In the method provided by the invention, cameras are arranged at the front side and the rear side of the bottom of the vehicle to respectively collect road images at the front and the rear of the vehicle, when the vehicle advances, a map shot by a fourth camera arranged at the front side of the bottom of the vehicle is sent to serve as an image of the bottom of the vehicle, and when the vehicle backs, a video image shot by a fifth camera arranged at the rear side of the bottom of the vehicle is sent to serve as an image of the bottom of the vehicle. When the vehicle moves forward and moves backward, the process of transmitting the vehicle bottom image is consistent, and only the source images are different, and the vehicle moving forward will be described as an example.

Specifically, when the vehicle advances, the image synthesis module acquires an image shot by the fourth camera, and based on the vehicle running speed and the setting position of the fourth camera, the moment when the environment content corresponding to the image shot by the fourth camera appears at the vehicle bottom can be obtained, and the image shot by the fourth camera is transmitted at the moment to serve as the vehicle bottom image. As shown in fig. 3, assuming that the fourth camera captures an image of the P1 feature at time t1, as the vehicle travels, the P1 feature reaches the P3 position (the vehicle bottom), and the image of the P1 feature is transmitted, so that an effect of transmitting the real-time vehicle bottom condition of the vehicle based on the traveling condition of the vehicle can be achieved, and a remote operator can have an immersed driving experience.

Furthermore, the vehicle-mounted video acquisition system provided by the invention further comprises a sixth camera arranged at the top of the vehicle, wherein the sixth camera is used for acquiring an environment image above the vehicle, and because the environment image above the vehicle has very low influence on the running of the vehicle, the sixth camera is a wide-angle camera, even if the acquired image is deformed, the driving safety is not influenced, and the immersive driving is realized under the condition of reducing the cost.

Based on the vehicle-mounted video acquisition system, the invention further provides a visual field linkage method which can be realized through the wearable electronic equipment.

As shown in fig. 4, the method comprises the steps of:

S110, acquiring a first image and displaying the first image in a first display area of a display screen of the wearable device, wherein the first image is obtained by splicing video images acquired by a first camera, a second camera and a third camera;

s120, acquiring a visual field change angle, wherein the visual field change angle reflects the visual field change intention of a user;

And S130, when the view change angle is within a preset range, rotating the position of the first image in the first display area based on the view change angle.

The video images acquired by the first camera, the second camera and the third camera can be spliced by adopting the existing image splicing algorithm, specifically, the characteristic points can be detected at first, then the characteristic points are matched, and then fusion is carried out. Via the visual field fusion and the splicing of the cameras, the obtained pictures are better in continuity, and the viewing experience is improved.

And displaying a first image obtained by splicing the video images acquired by the first camera, the second camera and the third camera in a first display area of a display screen of the wearable device. The first display area may be the entire area of the display screen of the wearable device or may be a partial area thereof. Therefore, the operator can see the image of the environment in front of the vehicle only by wearing the wearable device, and immersed remote control is realized.

The horizontal viewing angle of a single eye of a person can reach 150 degrees at maximum, and the horizontal viewing angle of two eyes can reach 188 degrees at maximum. The overlapping vision of the two eyes of the person is 124 degrees, and the comfortable vision of the single eye is 60 degrees. The overlapping vision of the two eyes of the human body has 124 degrees. That is, only objects within the 124-degree viewing angle have a stereoscopic impression within the range that the human eye views. In other words, only the 124-degree cross section of the object viewed in the overlapping view is more than 180 degrees, so that a stereoscopic effect is formed. The monocular comfort field of view is 60 degrees. The peripheral portion outside 30 degrees from the gaze point is called peripheral field of vision, commonly known as the afterlight of the eye, which is in fact the area where the human eye is not sensitive, i.e. is not visible. The visual field area can be roughly divided into a stereoscopic visual area, a fuzzy early warning area and a blind area according to the characteristics of the visual field area of the human eyes. The human body can quickly realize the quick switching of the stereoscopic visual area and the fuzzy early warning area by dynamically adjusting the positions of eyeballs, and the blind area needs to rotate the head or the body to obtain a wide field of view. Therefore, just as the human eyes need to rotate eyeballs or heads to see the areas outside the stereoscopic vision area when actually watching the front environment of the vehicle, the generated first image has large visual field content, but only a part of the clearly visible part of the human eyes is clear, so that the immersive remote control effect is realized.

Specifically, acquiring the view change angle includes:

Determining a field of view variation angle based on the sensor data;

the sensor data are acquired based on sensors of the wearable device.

As previously described, due to the limitation of the visual area of the human eye, this is typically accomplished by rotating the head or body when the user needs to pay attention to the blind spot. The horizontal view of the horizontal area is obtained by horizontally rotating the head, and the view of the ground or sky is obtained by raising or lowering the head. The wearable device can be provided with a multi-axis magnetic force sensor, a 3D acceleration angular velocity sensor and the like, and sensor data are acquired based on the number of sensors.

When the wearable device is worn, the relative position of the display screen relative to the eyes is unchanged, that is, the stereoscopic visual area of the eyes corresponds to the same area (such as an area P in fig. 5) on the display screen of the wearable device, and when a user rotates the body, the positions of the first images in the first display area are correspondingly adjusted, so that different areas in the first images are displayed in the stereoscopic visual area on the display screen, and the effect that the images follow rotation and continuously change when the user rotates the body is realized.

Further, the first camera, the second camera and the third camera can acquire only the environmental images in front of the vehicle, that is, in the first image, blind areas in the rear, the upper and the bottom of the vehicle still exist. In order to achieve the immersive manipulation effect, the method provided by the invention further comprises the following steps:

displaying a second image in a second display area on the display screen, wherein the second image is photographed based on a camera provided at the rear of the vehicle;

Acquiring a user eye image, and determining an eye closure state of the user based on the user eye image, wherein the eye closure state reflects whether the eyes of the user are open or closed;

The display contents of the first display area and the second display area are switched based on the eye-closed state.

The second display area may be smaller than the first display area, that is, the first display area is a large screen and the second display area is a small screen.

The second image may be obtained by photographing three cameras provided at the rear of the vehicle and then stitching, that is, three cameras provided at the rear of the vehicle in the same arrangement as the front of the vehicle.

When the user needs to mainly view the second image, the switching can be achieved by means of a button. In the method provided by the invention, in order to promote the eyes of the user to lift, the switching between the first display area and the second display area is realized by a human body biological characteristic signal mode, namely by a closed state of two eyes of the human body, for example: the left eye is closed, the right eye is opened, the first display area is switched to display the first image, the second display area is switched to display the second image, the left eye is opened, the right eye is closed, the first display area is switched to display the second image, and the second display area is switched to display the first image. Finally, the full-screen display of the first image can be automatically restored through the physical keys.

Further, the method provided by the invention further comprises the following steps:

When the visual field change angle exceeds a preset range, displaying a third image or a fourth image in the first display area;

The fourth image is a video image shot by a fourth camera or a fifth camera in the vehicle-mounted video acquisition system. The visual field change angle exceeds a preset range and corresponds to the user head-up or head-down exceeding a certain angle, when the user head-up is at a certain angle, a third image is displayed, and when the user head-down is at a certain angle, a fourth image is displayed.

By displaying the first image, the second image, the third image, and the fourth image in the first display area, a remote manipulation can be achieved like a control of sitting on a transparent car body.

The field linkage provided by the invention is described below, and the field linkage described below and the field linkage method described above can be referred to correspondingly. As shown in fig. 6, the field linkage provided by the present invention includes:

the image display module 610 is configured to obtain a first image and display the first image in a first display area of a display screen of the wearable device, where the first image is obtained by stitching video images acquired by the first camera, the second camera, and the third camera;

The view tracking module 620 is configured to obtain a view change angle, where the view change angle reflects a view change intention of the user;

The display adjustment module 630 is configured to rotate, when the view change angle is within a preset range, a position of the first image in the first display area based on the view change angle.

Fig. 7 illustrates a physical structure diagram of a wearable electronic device, as shown in fig. 7, where the wearable electronic device may include: processor 710, communication interface (Communications Interface) 720, memory 730, and communication bus 740, wherein processor 710, communication interface 720, memory 730 communicate with each other via communication bus 740. Processor 710 may invoke logic instructions in memory 730 to perform a view linkage method comprising: acquiring a first image and displaying the first image in a first display area of a display screen of the wearable device, wherein the first image is obtained by splicing video images acquired by a first camera, a second camera and a third camera; acquiring a visual field change angle, wherein the visual field change angle reflects the visual field change intention of a user; and when the visual field change angle is within the preset range, rotating the position of the first image in the first display area based on the visual field change angle.

Further, the logic instructions in the memory 730 described above may be implemented in the form of software functional units and may be stored in a computer readable storage medium when sold or used as a stand alone product. Based on this understanding, the technical solution of the present invention may be embodied essentially or in a part contributing to the prior art or in a part of the technical solution, in the form of a software product stored in a storage medium, comprising several instructions for causing a computer device (which may be a personal computer, a server, a network device, etc.) to perform all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a usb disk, a removable hard disk, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), a magnetic disk, or an optical disk, or other various media capable of storing program codes.

In another aspect, the present invention also provides a computer program product, the computer program product comprising a computer program, the computer program being storable on a non-transitory computer readable storage medium, the computer program, when executed by a processor, being capable of performing the vision linkage method provided by the above methods, the method comprising: acquiring a first image and displaying the first image in a first display area of a display screen of the wearable device, wherein the first image is obtained by splicing video images acquired by a first camera, a second camera and a third camera; acquiring a visual field change angle, wherein the visual field change angle reflects the visual field change intention of a user; and when the visual field change angle is within the preset range, rotating the position of the first image in the first display area based on the visual field change angle.

In yet another aspect, the present invention also provides a non-transitory computer readable storage medium having stored thereon a computer program which, when executed by a processor, is implemented to perform a field of view linkage method provided by the above methods, the method comprising: acquiring a first image and displaying the first image in a first display area of a display screen of the wearable device, wherein the first image is obtained by splicing video images acquired by a first camera, a second camera and a third camera; acquiring a visual field change angle, wherein the visual field change angle reflects the visual field change intention of a user; and when the visual field change angle is within the preset range, rotating the position of the first image in the first display area based on the visual field change angle.

The apparatus embodiments described above are merely illustrative, wherein the elements illustrated as separate elements may or may not be physically separate, and the elements shown as elements may or may not be physical elements, may be located in one place, or may be distributed over a plurality of network elements. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art will understand and implement the present invention without undue burden.

From the above description of the embodiments, it will be apparent to those skilled in the art that the embodiments may be implemented by means of software plus necessary general hardware platforms, or of course may be implemented by means of hardware. Based on this understanding, the foregoing technical solution may be embodied essentially or in a part contributing to the prior art in the form of a software product, which may be stored in a computer readable storage medium, such as ROM/RAM, a magnetic disk, an optical disk, etc., including several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the method described in the respective embodiments or some parts of the embodiments.

Finally, it should be noted that: the above embodiments are only for illustrating the technical solution of the present invention, and are not limiting; although the invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A vehicle-mounted video acquisition system, comprising:

2. The in-vehicle video capture system of claim 1, wherein the system further comprises:

3. A field of view linkage method based on the in-vehicle video acquisition system of any one of claims 1-2, comprising:

4. The view linkage method according to claim 3, wherein the obtaining the view change angle includes:

determining the view change angle based on sensor data;

the sensor data are acquired based on sensors of the wearable equipment.

5. A field of view linkage method according to claim 3, wherein the method further comprises:

6. A field of view linkage method according to claim 3, wherein the method further comprises:

7. A field of view linkage based on the in-vehicle video acquisition system of any one of claims 1-2, comprising:

8. A wearable electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the processor implements the view linkage method of any of claims 3 to 6 when the program is executed by the processor.

9. A non-transitory computer readable storage medium having stored thereon a computer program, wherein the computer program when executed by a processor implements the view linkage method of any of claims 3 to 6.

10. A computer program product comprising a computer program which, when executed by a processor, implements the vision field linkage method of any one of claims 3 to 6.