WO2019192359A1 - Système et procédé d'affichage de vidéo panoramique de véhicule, et dispositif de commande de véhicule - Google Patents

Système et procédé d'affichage de vidéo panoramique de véhicule, et dispositif de commande de véhicule Download PDF

Info

Publication number
WO2019192359A1
WO2019192359A1 PCT/CN2019/079732 CN2019079732W WO2019192359A1 WO 2019192359 A1 WO2019192359 A1 WO 2019192359A1 CN 2019079732 W CN2019079732 W CN 2019079732W WO 2019192359 A1 WO2019192359 A1 WO 2019192359A1
Authority
WO
WIPO (PCT)
Prior art keywords
video
vehicle
panoramic
information
spliced
Prior art date
Application number
PCT/CN2019/079732
Other languages
English (en)
Chinese (zh)
Inventor
王泽文
Original Assignee
杭州海康威视数字技术股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 杭州海康威视数字技术股份有限公司 filed Critical 杭州海康威视数字技术股份有限公司
Publication of WO2019192359A1 publication Critical patent/WO2019192359A1/fr

Links

Images

Classifications

    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R1/00—Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
    • B60R1/20—Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
    • B60R1/22—Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles for viewing an area outside the vehicle, e.g. the exterior of the vehicle
    • B60R1/23—Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles for viewing an area outside the vehicle, e.g. the exterior of the vehicle with a predetermined field of view
    • B60R1/27—Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles for viewing an area outside the vehicle, e.g. the exterior of the vehicle with a predetermined field of view providing all-round vision, e.g. using omnidirectional cameras
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R2300/00—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle
    • B60R2300/10—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the type of camera system used
    • B60R2300/105—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the type of camera system used using multiple cameras
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R2300/00—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle
    • B60R2300/30—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the type of image processing
    • B60R2300/303—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the type of image processing using joined images, e.g. multiple camera images
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R2300/00—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle
    • B60R2300/80—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the intended use of the viewing arrangement
    • B60R2300/8093—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the intended use of the viewing arrangement for obstacle warning

Definitions

  • the present application relates to the field of intelligent assisted driving technology, and in particular, to a vehicle panoramic video display system, method, and vehicle controller.
  • the intelligent assisted driving technology of the vehicle plays an important role in driving safety and efficient driving efficiency.
  • the use of intelligent assisted driving technology can largely avoid traffic accidents caused by driving errors of the driver of the vehicle.
  • the driving recorder is used to capture the road condition ahead of the vehicle, and based on the captured video, it is identified whether there is an obstacle target ahead of the vehicle, and if the obstacle target is identified, an early warning signal is generated, and according to the The warning signal prompts the driver to avoid obstacles.
  • the factors affecting the driving safety of the vehicle include not only the obstacles directly in front of the vehicle, but also the driving state of the driver affecting the driving safety. For example, the driver is in a fatigue driving state, and the probability of a traffic accident is also increased.
  • a camera is added to the driving recorder for collecting the driver's eye information, and by analyzing the driver's eye information, if it is determined that the driver is in a fatigue driving state, or based on the above-mentioned shooting road conditions.
  • the video recognizes that there is an obstacle target in front of the vehicle, and generates an early warning signal, and according to the warning signal, prompts the driver to have a safety hazard.
  • the driving recorder can only capture the road condition directly in front of the vehicle and the driver's video, and it is impossible to photograph the side of the vehicle, and the safety of the side of the vehicle is difficult to be secured; and the driving recorder is performing When the video is displayed, the in-vehicle video or the off-camera video is displayed independently, and the driver cannot observe the overall situation of the vehicle in a macroscopic view, and the display effect is poor.
  • the purpose of the embodiments of the present application is to provide a vehicle panoramic video display system, method, and vehicle controller to improve the safety of driving the vehicle and the display effect of the captured video.
  • the specific technical solutions are as follows:
  • an embodiment of the present application provides a vehicle panoramic video display system, where the system includes an in-vehicle video collector, a vehicle controller, and an in-vehicle display;
  • the vehicle video capture device includes a plurality of cameras, and each camera captures original video in the coverage area, and the total coverage area of all the cameras in the vehicle video collector is greater than or equal to 360 degrees; the original video collected by each camera is sent to The vehicle controller;
  • the vehicle controller is configured to receive the original video collected by each camera sent by the vehicle video capture device; and search for a pre-established panoramic synthesis mapping table, respectively generate a video to be spliced corresponding to each original video and a video to be spliced a splicing relationship
  • the panorama synthesis mapping table includes a correspondence between pixel coordinates of the video to be spliced and pixel coordinates of the original video; and splicing the video to be spliced according to the splicing relationship to generate a panoramic video; Sended to the on-board display;
  • the on-board display is configured to receive the panoramic video sent by the onboard controller; and display the panoramic video by using a preset display policy.
  • an embodiment of the present application provides a vehicle panoramic video display method, which is applied to an in-vehicle controller, and the method includes:
  • the video to be spliced is spliced to generate a panoramic video
  • the panoramic video is sent to the onboard display.
  • an embodiment of the present application provides an onboard controller, where the onboard controller includes a processor and a memory;
  • the memory is configured to store a computer program
  • the processor when executing the program stored on the memory, implements the following steps:
  • the video to be spliced is spliced to generate a panoramic video
  • the panoramic video is sent to the onboard display.
  • the embodiment of the present application provides a computer readable storage medium, where the computer readable storage medium stores a computer program, and when the computer program is executed by the processor, the following steps are implemented:
  • the video to be spliced is spliced to generate a panoramic video
  • the panoramic video is sent to the onboard display.
  • the vehicle panoramic video display system includes an in-vehicle video collector, a vehicle controller, and an in-vehicle display.
  • the vehicle video capture device includes a plurality of cameras, each camera respectively collects original video in the coverage area, and sends the original video collected by each camera to the vehicle controller; the vehicle controller is used to find a pre-established panoramic synthesis mapping table.
  • Panoramic shooting of the surrounding conditions of the vehicle to effectively improve the safety of the vehicle and the vehicle controller synthesizes the panoramic video by splicing the video to be spliced corresponding to the original video collected by each camera, and using the vehicle display to view the panorama
  • the display of the video effectively improves the display effect of the video, and enables the driver to more intuitively observe the surroundings of the vehicle from the display content, thereby improving the driver's interactive experience.
  • FIG. 1 is a schematic structural diagram of a vehicle panoramic video display system according to an embodiment of the present application.
  • FIG. 2 is a schematic flowchart of a method for displaying a vehicle panoramic video according to an embodiment of the present application
  • FIG. 3a is a schematic structural diagram of a corresponding fisheye device including a video processing chip
  • FIG. 3b is a schematic structural diagram of a corresponding fisheye device without a video processing chip
  • FIG. 4 is a schematic flowchart of a working mechanism of an on-board controller according to an embodiment of the present application
  • FIG. 5 is a schematic flowchart of a process for generating a panoramic synthesis mapping table according to an embodiment of the present application
  • FIG. 6 is a schematic structural diagram of a panoramic synthesis mapping table according to an embodiment of the present application.
  • FIG. 7 is a schematic flowchart of a panoramic video generation process according to an embodiment of the present application.
  • FIG. 8 is a schematic diagram of adjusting a panoramic video when a driver's posture changes according to an embodiment of the present application.
  • FIG. 9 is a schematic diagram of adjusting a panoramic video when an object of an obstacle outside the vehicle is recognized according to an embodiment of the present application.
  • FIG. 10 is a schematic structural diagram of a vehicle panoramic video display system according to another embodiment of the present application.
  • FIG. 11 is a schematic diagram of adjustment of panoramic video when a steering wheel of a vehicle rotates according to an embodiment of the present application.
  • FIG. 12 is a schematic diagram showing the display of a panoramic video when the steering wheel angle is ⁇ 0 according to an embodiment of the present application.
  • 12b is a schematic diagram showing the display of a panoramic video when the steering wheel angle is ⁇ 1 according to an embodiment of the present application;
  • FIG. 13 is a schematic structural diagram of an on-board controller according to an embodiment of the present application.
  • the embodiment of the present application provides a vehicle panoramic video display system, method, and vehicle controller, which are respectively described in detail below.
  • FIG. 1 is a schematic structural diagram of a vehicle panoramic video display system, including: a vehicle video capture device 110 , a vehicle controller 120 , and a vehicle Display 130.
  • the vehicle panoramic video display method provided by the embodiment of the present application is introduced from the perspective of the interaction of the devices in the vehicle panoramic video display system as shown in FIG. 1.
  • the vehicle panoramic video display method can be Including the following steps.
  • each camera in the vehicle video capture device separately collects the original video in the coverage area.
  • the car video capture device is mounted on the vehicle and can be located in the front of the vehicle.
  • the vehicle video capture device includes a plurality of cameras, and the total coverage area of all the cameras is greater than or equal to 360 degrees.
  • the in-vehicle video collector can include four cameras, each of which covers a coverage area of 90 degrees, and the total coverage area of the four cameras is a 360-degree coverage area; for example, in a car video collector. It can include 3 cameras, each of which has a coverage area of 120 degrees, and the total coverage area of the three cameras is a 360-degree coverage area.
  • the coverage area of each camera in the vehicle video capture device may not be equal.
  • the vehicle video capture device may include five cameras, the coverage area of two cameras is a coverage area of 90 degrees, and the coverage areas of the other three cameras are A coverage area of 60 degrees; for example, the vehicle video capture device may include five cameras, and the coverage areas of the cameras are 45 degrees, 60 degrees, 65 degrees, 90 degrees, and 100 degrees, respectively.
  • the coverage area of the camera is the area covered by the camera's field of view.
  • the in-vehicle video collector may be a fisheye device.
  • the fisheye device shown in Figure 3a and the fisheye device shown in Figure 3b are both back-to-back fisheye cameras (first fisheye)
  • the camera 301 and the second fisheye camera 302) are composed.
  • the fisheye device shown in Fig. 3a includes a video processing chip 303, and the fisheye device shown in Fig. 3b does not include a video processing chip.
  • the difference between the two types of fisheye equipment is whether it has video processing capability, and the video processing chip can realize functions such as panoramic video synthesis and intelligent detection.
  • the fisheye lens device shown in Fig. 3a includes the video processing chip 303, the size and weight of the fisheye device are significantly heavier than the binocular device shown in Fig.
  • the fisheye device shown in FIG. 3b the fisheye device without the video processing chip, the panoramic video synthesis function, and only the video capture can be used, and the fisheye device can complete the image filtering.
  • Basic image processing functions such as noise and white balance. In this way, the system cost can be effectively controlled, the miniaturization of the fisheye device can be facilitated, and the baseline convergence between the cameras can be reduced.
  • Each camera in the car video capture device may not have consistent key parameters and installation parameters, including but not limited to camera internal parameters, field of view angle, distortion, focal length, etc.; installation parameters include but are not limited to installation position, installation angle Wait. As long as the total coverage area of all cameras satisfies greater than or equal to 360 degrees.
  • the vehicle video collector sends the original video collected by each camera to the vehicle controller.
  • the vehicle controller is a device installed in the central control position of the vehicle, communicating with the vehicle host and the communication bus according to a prescribed protocol, and completing the collection, storage and transmission of information.
  • the vehicle video capture device sends the original video collected by each camera to the vehicle controller, so that the vehicle controller can complete functions such as panoramic video generation and intelligent detection.
  • the original video collected by each camera may be transmitted through the interaction module.
  • the original video captured by each camera can be transmitted between the vehicle video capture device and the vehicle controller through an interactive module.
  • the interaction module includes a communication bus between the vehicle video capture device and the vehicle controller, various data protocols, and a vehicle video capture device. Communication interface between communication interface and vehicle controller.
  • the communication bus may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus.
  • the original video captured by each camera can also be transmitted by wireless transmission between the vehicle video capture device and the vehicle controller, and will not be described here.
  • the in-vehicle controller searches for a pre-established panoramic synthesis mapping table, and respectively generates a stitching relationship between the video to be stitched corresponding to each original video and each video to be stitched.
  • the panoramic composition map includes the correspondence between the pixel coordinates of the video to be stitched and the pixel coordinates of the original video.
  • the working mechanism of the vehicle controller is as shown in FIG. 4, and the vehicle controller stores a panoramic synthesis mapping table and an intelligent detection model file.
  • the function of the on-board controller is to complete, including but not limited to, panoramic video generation, intelligent detection, etc., output panoramic video and detection information, and can perform viewpoint and perspective transformation on the panoramic video according to the control information input to realize panoramic roaming.
  • the panoramic synthesis map is pre-established and stored in the onboard controller, and the panoramic synthesis map is used to synthesize panoramic video.
  • the panoramic synthesis mapping table describes a one-to-one correspondence between the pixels of the video to be spliced and the pixels of the original video.
  • the panoramic synthesis mapping table stores the pixel coordinates of the video and the splicing relationship between the videos to be spliced.
  • the method for establishing the panoramic synthesis mapping table may include the following steps: acquiring sample images collected by each camera in the vehicle video capture device, and participating in each camera of the vehicle video collector calibration Camera external reference; according to each camera internal reference, each sample image is spherically projected to generate a spherical projection image corresponding to each sample image; according to each camera external parameter, the relative positional relationship between each spherical projection image is obtained; according to the relative positional relationship Convert and crop each spherical projection image to obtain a panoramic composition map.
  • the sample image is a pre-acquired image when the vehicle video capture device establishes the panoramic synthesis mapping table, and the sample image may be a pre-photographed image, or may be any video frame in the pre-acquired video sample, or multiple video frames;
  • the sample image may also be any video frame or multiple video frames extracted from the original video before the panoramic video is generated for the original video captured by each camera.
  • the camera internal parameter or the camera external parameter is often pre-calibrated. Therefore, each camera internal parameter or each camera external parameter can be obtained directly through each sample image.
  • a process of generating a panoramic synthesis map is provided.
  • the transformation of the two images may be: using one of the images as a reference, and transforming the other image to achieve seamless mosaic of the panorama.
  • the internal parameters and external parameters of the camera can be obtained by offline calibration, and the relative positional relationship between the images/videos collected by each camera is also fixed, and the coordinate image can be
  • the coordinates of the spherical projection are expanded and transformed, and after cropping, a panoramic composite mapping table can be obtained.
  • the panorama synthesis map is only relevant to the resolution of the input image. With off-line calibration, only one calibration is required for each car video collector, which can quickly generate panoramic video.
  • the process of generating the panoramic synthesis mapping table may also be generated according to an inverse process, establishing an empty panoramic mapping table, and establishing a correspondence between the panoramic composite mapping table and the original image through inverse transformation and pseudo projection expansion.
  • the method for establishing the panoramic synthesis mapping table may include the following steps: acquiring sample images collected by each camera in the vehicle video capture device, and participating in each camera of the vehicle video collector calibration a camera external parameter; establishing a first panoramic mapping table, the content of the first panoramic mapping table is empty; according to each camera external parameter, obtaining a positional relationship between each preset area in the first panoramic mapping table; according to the positional relationship, each The preset area is inversely transformed to obtain an inverse transformation area of each preset area; according to each camera internal parameter, spherical image projection is performed on each sample image to generate a spherical projection image corresponding to each sample image; according to each spherical projection image and each inverse transformation The correspondence between the regions determines the panoramic composition map.
  • the panoramic synthesis map can be generated by the above positive process or the inverse process, and the positive process generates a panoramic synthesis map with less time, and the inverse process generates a panoramic synthesis map with higher accuracy.
  • the external parameters of each camera of the car video capture device are often fixed. After the car video capture device is installed, the external position of the camera installation position and installation angle are fixed, but the camera internal reference and the camera external reference need to be pre-independent.
  • the auxiliary calibration equipment is calibrated. In order to reduce the number of auxiliary calibration equipment, in order to improve the efficiency of internal reference and external reference calibration, the external reference of the camera in the image can be calibrated by the external reference to calculate the internal reference of the camera, that is, only the external parameters of each camera need to be calibrated.
  • the internal parameters of each camera can be obtained by the preset internal parameter detection algorithm, and the internal reference parameters of each camera are not required to be calibrated by a separate internal reference calibration auxiliary device.
  • the manner of establishing the panoramic synthesis mapping table may further include: acquiring an external reference calibration image collected by each camera in the vehicle video capture device, and a calibration camera outside the calibration image of each external reference. Detecting; detecting the outer edge of the effective region of each external reference calibration image; fitting the outer edge of the effective region of each external reference calibration image by curve fitting, obtaining the outer contour of the effective region of each external reference calibration image; calibrating based on each external parameter
  • the outer contour of the effective area of the image determines the camera internal parameters for collecting the calibration images of the external parameters; according to the internal parameters of each camera, the spherical images of the external reference calibration images are respectively spherically generated, and the spherical projection images corresponding to the external reference calibration images are generated; according to the respective cameras
  • the outer parameter obtains a relative positional relationship between the spherical projection images; and according to the relative positional relationship, each spherical projection image is transformed and cropped to obtain a panoramic composite mapping table.
  • the external reference calibration image is an image of the vehicle-mounted video collector that is pre-acquired based on the external parameters of each camera when establishing the panoramic composite mapping table.
  • the external reference calibration image may be a pre-photographed image, or may be a video pre-acquired based on each camera external parameter. Any video frame in the sample, or multiple video frames; the external reference calibration image may also be any video frame or multiple video frames extracted from the original video before the panoramic video is generated for the original video captured by each camera. .
  • the camera external parameter is pre-calibrated in the external reference calibration image. Therefore, the calibration external camera parameters can be obtained directly from each external reference calibration image.
  • the method of detecting the outer edge of the effective area of each external reference calibration image may detect the outer edge of the effective area by recognizing the transition edge of the black and white area of the image, or search for the four farthest points of the outer edge of the effective area by means of a straight line search.
  • the outer edge of the effective area is obtained by connecting the four farthest points.
  • the panoramic synthesis map may also be generated through the inverse process by using the camera external parameters and the calculated camera internal parameters based on the calibration of the external reference calibration image.
  • the manner of establishing the panoramic synthesis mapping table may further include: acquiring an external reference calibration image collected by each camera in the vehicle video capture device, and a calibration camera outside the calibration image of each external reference. Detecting; detecting the outer edge of the effective region of each external reference calibration image; fitting the outer edge of the effective region of each external reference calibration image by curve fitting, obtaining the outer contour of the effective region of each external reference calibration image; calibrating based on each external parameter An outer contour of the effective area of the image, determining a camera internal parameter for collecting the calibration image of each foreign parameter; establishing a first panoramic mapping table, the content of the first panoramic mapping table is empty; according to each camera external parameter, obtaining each of the first panoramic mapping table Positional relationship between preset areas; inversely transforming each preset area according to the positional relationship, and obtaining an inverse transformation area of each preset area; performing spherical projection on each of the external reference calibration images according to each camera internal parameter, generating each a spherical projection
  • the panoramic synthesis mapping table includes respective mapping tables of the cameras in the in-vehicle video collector, and there is an overlapping area between the two mapping tables.
  • the panoramic synthesis mapping table is divided into two parts, a first fisheye camera mapping table 601 and a second fisheye camera mapping table. 602, which respectively represents the mapping relationship between the images acquired by the two fisheye cameras in the plane panorama.
  • the first fisheye camera mapping table 601 and the second fisheye camera mapping table 602 each have an overlapping area representing the overlapping area of the two fisheye cameras on the field of view, and the contents in the two overlapping areas are the same.
  • the splicing relationship between the video to be spliced and the spliced video to be spliced corresponding to the original video may be generated for each original video by using a pre-established panoramic compositing mapping table.
  • S204 The vehicle controller splices the video to be spliced according to the splicing relationship to generate a panoramic video.
  • the panoramic synthesis mapping table needs to be stored in the vehicle controller, and when the vehicle video collector collects the video, the panoramic synthesis mapping table is used. Looking up the table, the video to be spliced is obtained, and the panoramic video can be generated by splicing and merging the videos to be spliced.
  • the splicing relationship is given in the splicing relationship, for example, the first column of pixels of the second video to be spliced overlaps with the twentieth column of the first video to be spliced, and the second video to be spliced is flipped 45 After the degree overlaps with the first video to be stitched and so on.
  • the panoramic synthesis mapping table is generated based on the sample image, when the video is spliced, the video frames may be spliced, and then the spliced video frames are video-combined to obtain a panoramic video.
  • the S203 may include: searching for a pre-established panoramic synthesis mapping table, respectively generating a video frame to be spliced corresponding to each video frame in each original video and a video frame to be spliced Splicing relationship; for each video frame to be spliced generated by the same original video, using a preset video generation technology to generate a video to be spliced.
  • the S204 may include: splicing corresponding video frames to be spliced in each video to be spliced according to a splicing relationship between the video frames to be spliced, to obtain multiple panoramic images; and using preset video generation technology based on each panoramic image , generate a panoramic video.
  • the splicing relationship between the video frames to be spliced and the video frames to be spliced corresponding to each video frame in the original video can be obtained based on the panoramic compositing mapping table. Since the video is generated by the video frame, the preset video generation technology can be used to generate the splicing relationship. For the video to be spliced, the preset video generation technology specifies the order of the video frames in the video and the playback speed when the video is generated. Based on the splicing relationship between the video frames to be spliced, the corresponding video frames to be spliced may be spliced to obtain multiple panoramic images, and the panoramic video may be generated by using a preset video generation technology.
  • the panoramic video can be a two-dimensional video or a three-dimensional video. Since the three-dimensional video can more clearly display the spatial positional relationship between obstacles, vehicles, and drivers in the three-dimensional space, the panoramic video is mainly three-dimensional video.
  • the S204 may be specifically: splicing each video to be spliced according to a splicing relationship to generate a two-dimensional video; and performing three-dimensional projection on the two-dimensional video according to a preset three-dimensional projection manner, Panoramic video.
  • the vehicle video capture device as the fisheye device as an example
  • the video collected by the two fisheye cameras is used to look up the table by using the panoramic synthesis mapping table, and two videos to be stitched are obtained, according to the stitching relationship.
  • the two videos to be stitched are stitched together to generate a two-dimensional video, and a three-dimensional spherical panoramic video can be generated by performing spherical rendering on the two-dimensional video.
  • the video frames collected by the two fisheye cameras can also be extracted, and the panoramic synthesis map is used to perform the table lookup, and two subgraphs to be spliced are obtained, and two are to be spliced.
  • the sub-picture is spliced by the image to obtain a two-dimensional panoramic image.
  • the two-dimensional panoramic video is generated, and the two-dimensional panoramic video is spherically rendered to generate a three-dimensional spherical panoramic video.
  • the 2D panoramic video can generate corresponding 3D spherical panoramic video through cylindrical projection, cube projection and other projection methods.
  • the two-dimensional panoramic image may be spherically projected to obtain a three-dimensional panoramic image, and a three-dimensional spherical panoramic video is generated by a video generation technology, which is not specifically limited herein.
  • the vehicle controller sends the panoramic video to the vehicle display.
  • the in-vehicle display is a display device for displaying panoramic video installed in a centrally controlled position of the vehicle.
  • the panoramic video can be transmitted through the interaction module between the onboard controller and the onboard display.
  • the panoramic video can be transmitted between the vehicle controller and the vehicle display through the interaction module.
  • the interaction module includes a communication bus between the vehicle controller and the vehicle display, various data protocols, and a communication interface of the vehicle controller and a communication interface of the vehicle display.
  • the communication bus may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus.
  • the panoramic video can also be transmitted between the on-board controller and the on-board display by wireless transmission, and will not be described here.
  • the on-board display displays a panoramic video by using a preset display strategy.
  • the preset display strategy may be to clear the content that has been displayed on the in-vehicle display, and then display the panoramic video; for the panoramic video of the three-dimensional spherical surface, the preset display strategy may also be a three-dimensional display mode, which is not described here.
  • the method further includes the following steps: the vehicle controller acquires vehicle state information; the vehicle controller determines the to-be-processed information of the panoramic video according to the vehicle state information; and the vehicle controller is configured according to the vehicle
  • the information to be processed is processed by the panoramic video to obtain a processed panoramic video.
  • S205 may specifically be: the vehicle controller sends the processed panoramic video to the on-board display.
  • the S206 may specifically be: the on-vehicle display displays the processed panoramic video after a preset display strategy.
  • the vehicle state information may include steering wheel rotation information, driver attitude change information, vehicle exterior obstacle target information, and the like, corresponding to the in-vehicle device state, the driver state, and the vehicle exterior state, respectively.
  • the processing of the panoramic video may be an adjustment of parameters such as a viewing angle and a viewpoint.
  • the vehicle status information includes driver attitude change information.
  • the vehicle video capture device can also be used to identify driver attitude change information and transmit driver attitude change information to the vehicle controller.
  • the step of acquiring the vehicle state information by the vehicle controller may be specifically: the vehicle controller acquires driver posture change information sent by the vehicle video collector.
  • the step of determining, by the in-vehicle controller, the information to be processed of the panoramic video according to the vehicle state information may be specifically: the vehicle controller determines the viewpoint conversion parameter of the panoramic video by the viewpoint conversion relationship according to the driver posture change information.
  • the step of the vehicle controller processing the panoramic video according to the to-be-processed information to obtain the processed panoramic video may be: the vehicle controller performs the viewpoint update on the panoramic video according to the viewpoint conversion parameter, and obtains the panoramic video after the viewpoint update.
  • the step of the vehicle controller transmitting the processed panoramic video to the in-vehicle display may be: the vehicle controller sends the updated panoramic video of the viewpoint to the on-board display.
  • the step of displaying the processed panoramic video through the preset display strategy of the on-vehicle display may be: the on-board display passes the preset display strategy, and displays the panoramic video after the viewpoint update.
  • the driver's posture has changed. For example, the driver's body leans back and causes the head to move.
  • the camera in the car video collector that recognizes the inside of the car recognizes the change of the human body posture, and transmits the collected human body posture change video to the vehicle.
  • the controller detects that the driver's head has a displacement of (x m , y m , z m ), and the vehicle controller adjusts the viewpoint of the panoramic video by the parameter, and the viewpoint is (x 0 , y 0 , z 0 )
  • the change is (x 1 , y 1 , z 1 )
  • the viewpoint transformation parameter can be described as:
  • k x , k y and k z are viewpoint control parameters and can be set in advance.
  • the vehicle controller performs motion analysis and tracking, and uses the motion information to the panoramic view.
  • the viewpoint is updated to establish an interaction between the in-vehicle personnel and the vehicle panoramic video display system.
  • the vehicle status information may include: an off-vehicle obstacle target information.
  • the vehicle video collector can also be used to identify the obstacle information of the vehicle outside the vehicle and send the obstacle information of the vehicle to the vehicle controller.
  • the step of acquiring the vehicle state information by the vehicle controller may be specifically: the vehicle controller acquires the vehicle obstacle target information identified by the vehicle video collector.
  • the step of determining, by the vehicle controller, the information to be processed of the panoramic video according to the vehicle status information may specifically be: the vehicle controller determines the category information of the obstacle outside the vehicle according to the target information of the obstacle outside the vehicle, and the obstacle video in the panoramic video Location information in .
  • the step of the vehicle controller processing the panoramic video according to the to-be-processed information to obtain the processed panoramic video may specifically be: the vehicle controller superimposes the category information and the location information on the panoramic video to obtain the panoramic video after the superimposed information.
  • the step of the vehicle controller transmitting the processed panoramic video to the vehicle display may be: the vehicle controller sends the panoramic video after the superimposed information to the on-board display.
  • the step of displaying the processed panoramic video through the preset display strategy of the on-vehicle display may specifically be: the on-vehicle display passes the preset display strategy, and displays the panoramic video after the superimposed information.
  • the camera facing the outside of the car in the car video capture device can capture the obstacle target that affects the vehicle outside the vehicle.
  • the category information of the obstacle outside the vehicle and the position information of the obstacle outside the vehicle in the panoramic video can be obtained.
  • the category information and location information of the external obstacle target are superimposed on the panoramic video and displayed on the on-board display, and the warning purpose of the obstacle target can be achieved, as shown in FIG.
  • the vehicle panoramic video display system includes an in-vehicle video collector, a vehicle controller, and an in-vehicle display.
  • the vehicle video capture device includes a plurality of cameras, each camera respectively collects original video in the coverage area, and sends the original video collected by each camera to the vehicle controller; the vehicle controller is used to find a pre-established panoramic synthesis mapping table. Generating a splicing relationship between the video to be spliced corresponding to each original video and each video to be spliced respectively; according to the splicing relationship, splicing each video to be spliced to generate a panoramic video, and transmitting the generated panoramic video to the vehicle display; Used to display the received panoramic video through a preset display policy.
  • the vehicle controller Since the total coverage area of all the cameras in the vehicle video capture device is greater than or equal to 360 degrees, during the running of the vehicle, not only the road ahead of the vehicle is photographed, but also the side of the vehicle and the interior of the vehicle are photographed, and the 360 degree has no dead angle. Panoramic shooting of the surrounding conditions of the vehicle to effectively improve the safety of the vehicle; and the vehicle controller synthesizes the panoramic video by splicing the video to be spliced corresponding to the original video collected by each camera, and using the vehicle display to view the panorama The display of the video effectively improves the display effect of the video, and enables the driver to more intuitively observe the surroundings of the vehicle from the display content, thereby improving the driver's interactive experience.
  • a closed loop is formed between the vehicle video capture device, the vehicle controller and the vehicle display, which can provide useful support for assisted driving.
  • migrating the video splicing function from the vehicle video capture device to the vehicle controller not only saves the purchase cost of the display processing device, but also contributes to the miniaturization of the vehicle video capture device, reduces the baseline convergence between the cameras, and reduces the parallax. Improve the quality of the stitching.
  • FIG. 10 it is a schematic structural diagram of another vehicle panoramic video display system, which includes the following devices: an in-vehicle video collector 1010 and an on-board controller. 1020. On-board display 1030 and body sensor 1040.
  • the interaction between the in-vehicle video capture device 1010, the in-vehicle controller 1020, and the in-vehicle display 1030 is as shown in FIG. 2, and details are not described herein again.
  • the body sensor 1040 is configured to collect steering wheel rotation information and transmit steering wheel rotation information to the onboard controller 1020.
  • the vehicle controller 1020 can also be configured to receive the steering wheel rotation information sent by the vehicle body sensor 1040; according to the steering wheel rotation information, obtain the rotation angle information of the panoramic video by the rotation angle conversion; rotate the information according to the rotation angle of the viewing angle, and obtain the rotation conversion of the panoramic video.
  • the converted panoramic video; the converted panoramic video is transmitted to the in-vehicle display 1030.
  • the in-vehicle display 1030 is further configured to receive the converted panoramic video sent by the on-board controller 1020; and display the converted panoramic video through a preset display policy.
  • the driver's field of vision changes, and the direction of change is related to the direction of rotation of the steering wheel.
  • the steering wheel rotation information is sent to the vehicle controller, and the vehicle controller recalculates the angle of view.
  • the panoramic video after changing the viewing angle is displayed by the in-vehicle display.
  • the vehicle panoramic video display system further includes a vehicle body sensor for collecting steering wheel rotation information.
  • the vehicle body sensor rotates, the vehicle body sensor transmits the panoramic video after the angle of view rotation to the vehicle display, and the display of the vehicle display is performed. Displaying the scene after the vehicle is deflected facilitates the driver's intuitive observation of the driving environment and has a high intelligent assisted driving effect.
  • the embodiment of the present invention further provides an onboard controller, as shown in FIG. 13, which may include a processor 1301 and a memory 1302.
  • the memory 1302 is configured to store a computer program.
  • the processor 1301 is configured to execute the The following steps are implemented in the program stored on the memory 1302.
  • the panoramic composite mapping table includes Corresponding relationship between the pixel coordinates of the video to be stitched and the pixel coordinates of the original video; according to the stitching relationship, the video to be stitched is stitched to generate a panoramic video; and the panoramic video is sent to the in-vehicle display.
  • the processor 1301 may further implement the following steps: acquiring a sample image collected by each camera in the in-vehicle video collector, and each camera internal reference of the vehicle video collector calibration And each camera external parameter; performing spherical projection on each sample image according to each camera internal parameter to generate a spherical projection image corresponding to each sample image; and obtaining a spherical projection image according to each camera external parameter a relative positional relationship; transforming and cropping the spherical projection images according to the relative positional relationship to obtain a panoramic composition mapping table.
  • the processor 1301 may further implement the following steps: acquiring a sample image collected by each camera in the in-vehicle video collector, and each camera internal reference of the vehicle video collector calibration And establishing a first panoramic mapping table, where the content of the first panoramic mapping table is empty; according to the external parameters of the cameras, obtaining a position between each preset area in the first panoramic mapping table And performing inverse transformation on the preset regions according to the positional relationship to obtain an inverse transform region of each preset region; performing spherical projection on each sample image according to each camera internal parameter, generating the A spherical projection image corresponding to each sample image; and a panoramic composition mapping table is determined according to a correspondence relationship between each spherical projection image and each inverse transformation region.
  • the processor 1301 may further implement the following steps: acquiring an external reference calibration image collected by each camera in the vehicle video capture device, and a calibration camera in each external reference calibration image. External parameter; detecting an outer edge of the effective region of the external reference calibration image; fitting the outer edge of the effective region of the external reference calibration image by curve fitting to obtain an effective region of the external reference calibration image a contour; determining, according to an outer contour of the effective region of the external reference calibration image, a camera internal reference for acquiring the calibration image of each external reference; and performing spherical projection on each of the external reference calibration images according to each camera internal reference, generating a a spherical projection image corresponding to each external reference calibration image; obtaining a relative positional relationship between each spherical projection image according to each camera external parameter; and transforming and cutting the spherical projection images according to the relative positional relationship Panorama synthesis map.
  • the processor 1301 may further implement the following steps: acquiring an external reference calibration image collected by each camera in the vehicle video capture device, and a calibration camera in each external reference calibration image. External parameter; detecting an outer edge of the effective region of the external reference calibration image; fitting the outer edge of the effective region of the external reference calibration image by curve fitting to obtain an effective region of the external reference calibration image a contour; determining, according to an outer contour of the effective area of the external reference calibration image, a camera internal parameter for collecting the external reference calibration image; establishing a first panoramic mapping table, wherein the content of the first panoramic mapping table is empty; Each camera external parameter obtains a positional relationship between each preset area in the first panoramic mapping table; and inversely transforms each preset area according to the positional relationship to obtain an inverse of each preset area Converting a region; performing spherical projection on each of the external reference calibration images according to each camera internal parameter, and generating a spherical projection image corresponding to each external reference calibration image
  • the panoramic synthesis mapping table includes a mapping table of each camera in the in-vehicle video collector, and there is an overlapping area between the two mapping tables.
  • the processor 1301 is configured to implement the pre-established panoramic synthesis mapping table, and respectively generate a splicing relationship between the video to be spliced corresponding to each original video and each video to be spliced.
  • the following steps may be implemented: searching for a pre-established panoramic synthesis mapping table, respectively generating a splicing relationship between the video frames to be spliced corresponding to each video frame in each original video and the video frames to be spliced;
  • the generated video frames to be spliced are generated by using a preset video generation technology to generate a video to be spliced.
  • the processor 1301 When the processor 1301 performs the step of splicing the video to be spliced according to the splicing relationship to generate a panoramic video, the following steps may be specifically implemented: according to the splicing relationship between the video frames to be spliced, Each of the corresponding video frames to be spliced in each video to be spliced is spliced to obtain a plurality of panoramic images; and based on each panoramic image, a panoramic video is generated by using a preset video generation technology.
  • the processor 1301 when the processor 1301 implements the step of splicing each video to be spliced according to the splicing relationship to generate a panoramic video, the following steps may be specifically implemented: The splicing relationship is performed, and the video to be spliced is spliced to generate a two-dimensional video; according to the preset three-dimensional projection mode, the two-dimensional video is three-dimensionally projected to obtain a panoramic video.
  • the processor 1301 may further implement the following steps: acquiring vehicle state information; determining, according to the vehicle state information, pending information of the panoramic video; according to the to-be-processed Information, processing the panoramic video to obtain a processed panoramic video.
  • the processor 1301 When the processor 1301 implements the step of transmitting the panoramic video to the in-vehicle display, specifically, the processor 1301 may be configured to: send the processed panoramic video to the in-vehicle display.
  • the vehicle status information includes steering wheel rotation information.
  • the processor 1301 may specifically implement the following steps: acquiring steering wheel rotation information collected by the vehicle body sensor.
  • the processor 1301 may specifically implement the following steps: according to the steering wheel rotation information, the rotation angle is converted to obtain the The perspective of the panoramic video rotates the information.
  • the processor 1301 When the processor 1301 performs the step of processing the panoramic video according to the to-be-processed information to obtain the processed panoramic video, specifically, the following steps may be implemented: rotating the information according to the viewing angle, The panoramic video is rotated and transformed to obtain a transformed panoramic video.
  • the processor 1301 may specifically implement the step of transmitting the converted panoramic video to the in-vehicle display when the step of transmitting the processed panoramic video to the in-vehicle display is implemented.
  • the vehicle state information includes driver posture change information.
  • the processor 1301 may specifically implement the following steps: acquiring driver posture change information recognized by the in-vehicle video collector.
  • the processor 1301 when implementing the step of determining the to-be-processed information of the panoramic video according to the vehicle state information, may specifically implement the following steps: determining, according to the driver attitude change information, a viewpoint transformation relationship The viewpoint conversion parameter of the panoramic video.
  • the processor 1301 When the processor 1301 performs the step of processing the panoramic video according to the to-be-processed information to obtain the processed panoramic video, specifically, the following steps may be implemented: according to the viewpoint transformation parameter, The panoramic video is updated by the viewpoint to obtain a panoramic video after the viewpoint is updated.
  • the processor 1301 implements the step of transmitting the processed panoramic video to the in-vehicle display
  • the processor 1301 may be configured to: send the panoramic video after the view update to the on-board display.
  • the vehicle status information includes: an off-vehicle obstacle target information.
  • the processor 1301 may specifically implement the following steps: acquiring the vehicle exterior obstacle target information identified by the vehicle video capture device.
  • the processor 1301 when implementing the step of determining the to-be-processed information of the panoramic video according to the vehicle state information, may specifically implement the following steps: determining an obstacle outside the vehicle according to the target information of the vehicle exterior obstacle Category information, and location information of the outer obstacle in the panoramic video.
  • the processor 1301 When the processor 1301 performs the step of processing the panoramic video according to the to-be-processed information to obtain the processed panoramic video, the following steps may be specifically implemented: the category information and the location information. Superimposed on the panoramic video to obtain a panoramic video after superimposing information.
  • the processor 1301 When the processor 1301 is configured to send the processed panoramic video to the in-vehicle display, the processor 1301 may specifically implement the step of: transmitting the panoramic video after the superimposed information to the in-vehicle display.
  • the in-vehicle video collector is a fisheye device.
  • the processor 1301 implements the step of acquiring the original video collected by each camera in the vehicle video capture device, the following steps may be specifically implemented: acquiring the original video separately collected by the two fisheye cameras in the fisheye device.
  • Data transmission between the memory 1302 and the processor 1301 may be performed by means of a wired connection or a wireless connection, and the computer device may communicate with other devices through a wired communication interface or a wireless communication interface. It should be noted that only an example of transmitting data between the processor 1301 and the memory 1302 via the bus is shown in FIG. 13, and is not limited to a specific transmission mode.
  • the above memory may include a RAM (Random Access Memory), and may also include NVM (Non-Volatile Memory), such as at least one disk storage.
  • NVM Non-Volatile Memory
  • the memory may also be at least one storage device located away from the processor.
  • the processor may be a general-purpose processor, including a CPU (Central Processing Unit), an NP (Network Processor), or the like; or a DSP (Digital Signal Processor) or an ASIC (Application) Specific Integrated Circuit, FPGA (Field-Programmable Gate Array) or other programmable logic device, discrete gate or transistor logic device, discrete hardware components.
  • CPU Central Processing Unit
  • NP Network Processor
  • DSP Digital Signal Processor
  • ASIC Application) Specific Integrated Circuit
  • FPGA Field-Programmable Gate Array
  • other programmable logic device discrete gate or transistor logic device, discrete hardware components.
  • the embodiment of the present application provides a computer readable storage medium, where the computer readable storage medium stores a computer program, a computer program, corresponding to the vehicle-mounted panoramic video display method applied to the vehicle-mounted controller provided by the above embodiment. All steps of the in-vehicle panoramic video display method applied to the on-vehicle controller of the embodiment of the present application are implemented when executed by the processor.
  • the vehicle controller splicing and synthesizing the video to be spliced corresponding to the original video collected by each camera
  • the panoramic video is then displayed by the on-board display, which effectively improves the display effect of the video, enables the driver to more intuitively observe the surroundings of the vehicle from the display content, and improves the driver's interactive experience.
  • the computer readable storage medium stores an application program that executes the on-vehicle panoramic video display method applied to the on-vehicle controller provided by the embodiment of the present application at runtime, and thus can realize: the total coverage area of all the cameras in the vehicle video capture device It is greater than or equal to 360 degrees.
  • the running of the vehicle not only the road ahead of the vehicle is photographed, but also the side of the vehicle and the inside of the vehicle are photographed. 360 degrees without panoramic view, the panoramic view of the surrounding conditions of the vehicle is effectively improved.
  • the vehicle controller synthesizes the panoramic video by splicing the video to be spliced corresponding to the original video collected by each camera, and then displaying the panoramic video by using the vehicle display, thereby effectively improving the display effect of the video. It can make the driver more intuitively observe the situation around the vehicle from the display content, and improve the driver's interactive experience.
  • the embodiment of the present application provides an application program for performing at runtime: the vehicle panoramic video image applied to the vehicle controller provided by the embodiment of the present application Display method.
  • the application performs the on-vehicle panoramic video display method applied to the on-board controller provided by the embodiment of the present application during operation, so that the total coverage area of all the cameras in the on-board video collector is greater than or equal to 360.
  • the process of driving the vehicle not only the road ahead of the vehicle is photographed, but also the side of the vehicle and the inside of the vehicle are photographed, and 360-degree panoramic view of the surroundings of the vehicle is performed without dead angle, thereby effectively improving the safety of the vehicle.
  • the vehicle controller synthesizes the panoramic video by splicing the video to be spliced corresponding to the original video collected by each camera, and then displaying the panoramic video by using the vehicle display, thereby effectively improving the display effect of the video and enabling driving
  • the staff intuitively observed the surrounding conditions of the vehicle from the display content and improved the driver's interactive experience.
  • the on-board controller the computer readable storage medium, and the application embodiment, since the method content involved is basically similar to the foregoing method embodiment, the description is relatively simple, and the relevant parts refer to the description of the method embodiment. Just fine.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Mechanical Engineering (AREA)
  • Closed-Circuit Television Systems (AREA)

Abstract

L'invention concerne un système et un procédé d'affichage de vidéo panoramique de véhicule, ainsi qu'un dispositif de commande de véhicule, le système comprenant : un dispositif de collecte de vidéo de véhicule (1010), un dispositif de commande de véhicule (1020) et un dispositif d'affichage de véhicule (1030), le dispositif de collecte de vidéo de véhicule (1010) comprenant une pluralité de caméras ; chaque caméra collecte séparément des vidéos d'origine dans une zone de couverture, et la zone de couverture totale de toutes les caméras dans le dispositif de collecte de vidéo de véhicule (1010) est supérieure ou égale à 360 degrés ; et les vidéos d'origine collectées par chaque caméra sont envoyées au dispositif de commande de véhicule (1020). Le dispositif de commande de véhicule (1020) est utilisé pour rechercher une table de mappage de synthèse panoramique préétablie et pour générer respectivement une vidéo à monter qui correspond à chaque vidéo d'origine et une relation de montage entre chaque vidéo à monter ; les vidéos à monter sont montées selon la relation de montage de façon à générer une vidéo panoramique ; et la vidéo panoramique est envoyée au dispositif d'affichage de véhicule (1030). Le dispositif d'affichage de véhicule (1030) est utilisé pour afficher la vidéo panoramique au moyen d'une politique d'affichage prédéfinie. Au moyen de la présente solution, la sécurité de conduite d'un véhicule et l'effet d'affichage d'une vidéo capturée peuvent être améliorés.
PCT/CN2019/079732 2018-04-02 2019-03-26 Système et procédé d'affichage de vidéo panoramique de véhicule, et dispositif de commande de véhicule WO2019192359A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201810284336.3A CN110341597B (zh) 2018-04-02 2018-04-02 一种车载全景视频显示***、方法及车载控制器
CN201810284336.3 2018-04-02

Publications (1)

Publication Number Publication Date
WO2019192359A1 true WO2019192359A1 (fr) 2019-10-10

Family

ID=68099752

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2019/079732 WO2019192359A1 (fr) 2018-04-02 2019-03-26 Système et procédé d'affichage de vidéo panoramique de véhicule, et dispositif de commande de véhicule

Country Status (2)

Country Link
CN (1) CN110341597B (fr)
WO (1) WO2019192359A1 (fr)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112929601A (zh) * 2021-01-22 2021-06-08 广东新粤交通投资有限公司 一种车辆监控视频传输***及传输方法
CN113240615A (zh) * 2021-05-20 2021-08-10 北京城市网邻信息技术有限公司 图像处理方法、装置、电子设备和计算机可读存储介质
CN115514751A (zh) * 2022-08-24 2022-12-23 网易（杭州）网络有限公司 用于挖掘机远程控制的图像获取方法及远程控制***

Families Citing this family (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112818726A (zh) * 2019-11-15 2021-05-18 杭州海康威视数字技术股份有限公司 车辆违章预警方法、装置、***及存储介质
US11055835B2 (en) 2019-11-19 2021-07-06 Ke.com (Beijing) Technology, Co., Ltd. Method and device for generating virtual reality data
CN111105347B (zh) * 2019-11-19 2020-11-13 贝壳找房(北京)科技有限公司 一种生成带深度信息的全景图的方法、装置及存储介质
CN111137210A (zh) * 2019-11-28 2020-05-12 天津津航技术物理研究所 车载全景观察***
CN111447431A (zh) * 2020-04-02 2020-07-24 深圳普捷利科技有限公司 一种应用于车载环视摄像的裸眼3d显示方法及***
CN111546986A (zh) * 2020-04-30 2020-08-18 北京大椽科技有限公司 一种拖挂车全景环视方法
CN111757057B (zh) * 2020-06-17 2022-06-17 广州市泰睿科技有限公司 一种全景环视显示方法、装置、设备及存储介质
CN111815752B (zh) * 2020-07-16 2022-11-29 展讯通信（上海）有限公司 图像处理方法、装置和电子设备
CN114425991B (zh) * 2020-10-29 2023-12-29 上海博泰悦臻网络技术服务有限公司 图像处理方法、介质、设备及图像处理***
CN112606829A (zh) * 2020-12-16 2021-04-06 广州市车智连电子有限公司 一种基于全景拼接数据映射的辅助泊车***
CN112550469A (zh) * 2020-12-25 2021-03-26 浙江零跑科技有限公司 一种汽车透视a柱模块及显示方法
CN113715826B (zh) * 2021-08-31 2023-03-21 深圳市同进视讯技术有限公司 一种基于车载实时监控的车辆驾驶辅助方法及装置
CN116095298A (zh) * 2021-11-06 2023-05-09 浙江吉利控股集团有限公司 车外场景显示方法、头戴显示设备、终端设备及存储介质

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1987357A (zh) * 2006-12-26 2007-06-27 浙江工业大学 基于全方位计算机视觉的智能停车辅助装置
US20090207045A1 (en) * 2008-02-14 2009-08-20 Mando Corporation Method and apparatus for detecting target parking position by using two reference points, and parking assist system using the same
CN102111610A (zh) * 2010-05-21 2011-06-29 深圳安科智能视频技术有限公司 用于辅助驾驶的全景式影像生成方法
CN104851076A (zh) * 2015-05-27 2015-08-19 武汉理工大学 用于商用车的全景环视泊车辅助***及摄像头安装方法
CN107627959A (zh) * 2017-09-20 2018-01-26 鹰驾科技（深圳）有限公司 机动车的全景视频监测方法及***

Family Cites Families (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH1165004A (ja) * 1997-08-12 1999-03-05 Sony Corp パノラマ撮像システム
CN102045546B (zh) * 2010-12-15 2013-07-31 广州致远电子股份有限公司 一种全景泊车辅助***
CN102629372A (zh) * 2012-02-22 2012-08-08 北京工业大学 一种用于辅助车辆驾驶的360度全景鸟瞰图生成方法
CN103617606B (zh) * 2013-11-26 2016-09-14 中科院微电子研究所昆山分所 用于辅助驾驶的车辆多视角全景生成方法
CN106856000B (zh) * 2015-12-09 2021-02-05 广州汽车集团股份有限公司 一种车载全景图像无缝拼接处理方法及***
CN106855999A (zh) * 2015-12-09 2017-06-16 宁波芯路通讯科技有限公司 汽车环视图像的生成方法及装置
CN105957010A (zh) * 2016-05-19 2016-09-21 沈祥明 一种车载图像拼接***
CN107644394B (zh) * 2016-07-21 2021-03-30 完美幻境(北京)科技有限公司 一种3d图像的处理方法及装置
CN107424120A (zh) * 2017-04-12 2017-12-01 湖南源信光电科技股份有限公司 一种全景环视***中的图像拼接方法
CN107507131B (zh) * 2017-08-08 2021-02-23 广州市安晓科技有限责任公司 基于单摄像头的360度全景倒车影像生成方法

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1987357A (zh) * 2006-12-26 2007-06-27 浙江工业大学 基于全方位计算机视觉的智能停车辅助装置
US20090207045A1 (en) * 2008-02-14 2009-08-20 Mando Corporation Method and apparatus for detecting target parking position by using two reference points, and parking assist system using the same
CN102111610A (zh) * 2010-05-21 2011-06-29 深圳安科智能视频技术有限公司 用于辅助驾驶的全景式影像生成方法
CN104851076A (zh) * 2015-05-27 2015-08-19 武汉理工大学 用于商用车的全景环视泊车辅助***及摄像头安装方法
CN107627959A (zh) * 2017-09-20 2018-01-26 鹰驾科技（深圳）有限公司 机动车的全景视频监测方法及***

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112929601A (zh) * 2021-01-22 2021-06-08 广东新粤交通投资有限公司 一种车辆监控视频传输***及传输方法
CN112929601B (zh) * 2021-01-22 2024-05-10 广东新粤交通投资有限公司 一种车辆监控视频传输***及传输方法
CN113240615A (zh) * 2021-05-20 2021-08-10 北京城市网邻信息技术有限公司 图像处理方法、装置、电子设备和计算机可读存储介质
CN113240615B (zh) * 2021-05-20 2022-06-07 北京城市网邻信息技术有限公司 图像处理方法、装置、电子设备和计算机可读存储介质
CN115514751A (zh) * 2022-08-24 2022-12-23 网易（杭州）网络有限公司 用于挖掘机远程控制的图像获取方法及远程控制***

Also Published As

Publication number Publication date
CN110341597A (zh) 2019-10-18
CN110341597B (zh) 2020-11-27

Similar Documents

Publication Publication Date Title
WO2019192359A1 (fr) Système et procédé d'affichage de vidéo panoramique de véhicule, et dispositif de commande de véhicule
US9858639B2 (en) Imaging surface modeling for camera modeling and virtual view synthesis
CN108765496A (zh) 一种多视点汽车环视辅助驾驶***及方法
JP5444338B2 (ja) 車両周囲監視装置
CN105763854B (zh) 一种基于单目摄像头的全景成像***及其成像方法
JP5682788B2 (ja) 車両周辺監視装置
CN104442567B (zh) 交通工具图像显示***中的物体突显和传感
US8576285B2 (en) In-vehicle image processing method and image processing apparatus
JP5455124B2 (ja) カメラ姿勢パラメータ推定装置
US20140114534A1 (en) Dynamic rearview mirror display features
JP4643860B2 (ja) 車両用視覚支援装置及び支援方法
JP5321711B2 (ja) 車両用周辺監視装置および映像表示方法
WO2010070920A1 (fr) Dispositif de génération d'image des environs d'un véhicule
JP5516998B2 (ja) 画像生成装置
WO2017043331A1 (fr) Dispositif et procédé de traitement d'images
JPWO2019003910A1 (ja) 画像処理装置、画像処理方法、およびプログラム
KR101278654B1 (ko) 차량의 주변 영상 디스플레이 장치 및 방법
CN206287929U (zh) 一种3d全景泊车装置及***
KR20130071842A (ko) 차량의 주변 정보 제공 장치 및 방법
JP6977725B2 (ja) 画像処理装置および画像処理方法
KR20130024066A (ko) 차량의 주차 보조 장치 및 그 동작 방법
JP2009020647A (ja) 立体物検出方法および立体物検出装置
JP4339749B2 (ja) 画像生成方法および画像生成装置
JP2017117357A (ja) 立体物検知装置
JP5436056B2 (ja) 障害物検出装置および障害物検出方法

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19781257

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 19781257

Country of ref document: EP

Kind code of ref document: A1

122 Ep: pct application non-entry in european phase

Ref document number: 19781257

Country of ref document: EP

Kind code of ref document: A1

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 19.04.2021)

122 Ep: pct application non-entry in european phase

Ref document number: 19781257

Country of ref document: EP

Kind code of ref document: A1