CN115179920B

CN115179920B - Vehicle running system adjusting method and system based on cross-country scene

Info

Publication number: CN115179920B
Application number: CN202211089141.6A
Authority: CN
Inventors: 杨超
Original assignee: Beijing Smarter Eye Technology Co Ltd
Current assignee: Beijing Smarter Eye Technology Co Ltd
Priority date: 2022-09-07
Filing date: 2022-09-07
Publication date: 2023-01-17
Anticipated expiration: 2042-09-07
Also published as: CN115179920A

Abstract

The invention discloses a method and a system for adjusting a vehicle running system based on a cross-country scene, wherein the method comprises the following steps: acquiring image information in a target area, calculating three-dimensional point cloud of the image information, and generating depth information based on the three-dimensional point cloud; generating RGB-D information based on the image information and the depth information, and reconstructing a three-dimensional space of a target area based on the three-dimensional point cloud; acquiring vehicle information of a target vehicle, and generating a driving path and a vehicle model based on the vehicle information; generating a three-dimensional space of a driving route according to the driving path and the three-dimensional point cloud; generating a passability judgment result of the target area based on the three-dimensional space of the driving route and the vehicle model; under the condition that the passability judgment result is passable, generating attribute information of the target area according to the RGB-D information and a pre-stored semantic segmentation model; and generating an adjusting instruction based on the attribute information, wherein the adjusting instruction is used for adjusting the running parameters of the vehicle running system. The road safety and smoothness when the vehicle is off-road are improved.

Description

Vehicle running system adjusting method and system based on cross-country scene

Technical Field

The invention relates to the technical field of automatic driving, in particular to a method and a system for adjusting a vehicle running system based on a cross-country scene.

Background

For users of off-road vehicles, there is a risk of vehicle damage and personal safety associated with enjoying the experience of conquering excitement from off-road.

Safety and ride comfort are two important indicators for evaluating the performance of an off-road vehicle during off-road driving. The safety refers to that whether the geometric passability of the current cross-country road surface, namely the current road and vehicles can pass or not is judged through a sensor; the smoothness refers to the improvement of the passing comfort of the off-road surface by adjusting the chassis, namely how comfortable the vehicle passes on the current road.

Therefore, it is an urgent need of the skilled in the art to provide a method and a system for detecting a travelable area based on an off-road scene, so as to adjust a vehicle travel system in real time according to a road condition of the travelable area, thereby improving road safety and smoothness when the vehicle travels off-road.

Disclosure of Invention

Therefore, the embodiment of the invention provides a method and a system for detecting a travelable area based on an off-road scene, which can adjust a vehicle travelling system in real time according to the road condition of the travelable area, thereby improving the road safety and smoothness when the vehicle is off-road.

In order to achieve the above object, the embodiments of the present invention provide the following technical solutions:

the invention provides a vehicle running system adjusting method based on an off-road scene, which comprises the following steps:

acquiring image information in a target area, calculating three-dimensional point cloud of the image information, and generating depth information based on the three-dimensional point cloud;

generating RGB-D information based on the image information and the depth information, and reconstructing a three-dimensional space of the target area based on the three-dimensional point cloud;

acquiring vehicle information of a target vehicle, and generating a driving path and a vehicle model based on the vehicle information;

generating a driving route three-dimensional space according to the driving path and the three-dimensional point cloud;

generating a passability judgment result of the target area based on the three-dimensional space of the driving route and the vehicle model;

generating attribute information of the target area according to the RGB-D information and a pre-stored semantic segmentation model under the condition that the passability judgment result is passable;

and generating an adjusting instruction based on the attribute information, wherein the adjusting instruction is used for adjusting the running parameters of the vehicle running system.

In some embodiments, based on the three-dimensional space of the driving route and the vehicle model, a passability determination result of the target area is generated, and then the method further includes:

and under the condition that the passability judgment result is that the three-dimensional point cloud is passable, adjusting a driving path, and generating a driving route three-dimensional space according to the adjusted driving path and the three-dimensional point cloud.

In some embodiments, when the passability determination result is that the network device is passable, the method further includes:

and generating an early warning instruction, wherein the early warning instruction is used for controlling an early warning system of the target vehicle to send out an early warning signal.

In some embodiments, the driving parameters of the vehicle driving system include at least one of:

the height of the chassis suspension, the rigidity of the chassis suspension, the damping of the chassis suspension and the driving braking force of the power system.

In some embodiments, generating the attribute information of the target region according to the RGB-D information and a pre-stored semantic segmentation model specifically includes:

acquiring the RGB-D information;

extracting and fusing color and spatial feature information based on the semantic segmentation model to obtain a feature fusion result;

and obtaining attribute information of the target area based on the feature fusion result.

In some embodiments, generating the adjustment instruction based on the attribute information specifically includes:

performing single-frame category statistics and multi-frame category tracking on the attribute information to obtain an output scene category;

and obtaining an output control signal based on the output scene category, and generating the adjusting instruction based on the output control signal.

The invention also provides a vehicle travel system adjustment system based on an off-road scenario, the system comprising:

an image acquisition unit for acquiring image information in a target area, calculating a three-dimensional point cloud of the image information, and generating depth information based on the three-dimensional point cloud;

an information generating unit for generating RGB-D information based on the image information and the depth information, and reconstructing a three-dimensional space of the target region based on the three-dimensional point cloud;

a model generation unit configured to acquire vehicle information of a target vehicle, and generate a travel path and a vehicle model based on the vehicle information;

the three-dimensional space generating unit is used for generating a three-dimensional space of a driving route according to the driving path and the three-dimensional point cloud;

a result output unit configured to generate a passability determination result of the target area based on the travel route three-dimensional space and the vehicle model;

an attribute generating unit, configured to generate attribute information of the target area according to the RGB-D information and a pre-stored semantic segmentation model if the passability determination result indicates passability;

and the instruction generating unit is used for generating an adjusting instruction based on the attribute information, and the adjusting instruction is used for adjusting the running parameters of the vehicle running system.

The present invention also provides an intelligent terminal, including: the device comprises a data acquisition device, a processor and a memory;

the data acquisition device is used for acquiring data; the memory is to store one or more program instructions; the processor is configured to execute one or more program instructions to perform the method as described above.

The present invention also provides a computer readable storage medium having embodied therein one or more program instructions for carrying out the method as described above.

The invention also provides a computer program product comprising a computer program which, when executed by a processor, performs the steps of the method as described in any one of the above.

According to the method and the system for adjusting the vehicle running system based on the off-road scene, provided by the invention, the binocular camera is used for obtaining the three-dimensional space information of the three-dimensional point cloud reconstructed road surface, and the passability of the vehicle on the current road is calculated, so that the chassis height is adjusted according to the fluctuation of the road surface, and the passing safety of the vehicle is ensured; RGB-D information of the accessible road surface is fused, attributes of the current accessible road surface are obtained through a semantic segmentation model and perception signal processing, and driving parameters such as chassis suspension rigidity, damping and driving braking force of a power system are dynamically adjusted according to the attribute information of the accessible road surface, so that smoothness when the accessible road surface passes through a drivable area is guaranteed. Therefore, the method and the system for detecting the travelable area based on the off-road scene adjust the vehicle traveling system in real time according to the road condition of the travelable area, thereby improving the road safety and smoothness when the vehicle is off-road.

Drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below. It should be apparent that the drawings in the following description are merely exemplary, and that other embodiments can be derived from the drawings provided by those of ordinary skill in the art without inventive effort.

The structures, ratios, sizes, and the like shown in the present specification are only used for matching with the contents disclosed in the specification, so as to be understood and read by those skilled in the art, and are not used to limit the conditions that the present invention can be implemented, so that the present invention has no technical significance, and any structural modifications, changes in the ratio relationship, or adjustments of the sizes, without affecting the effects and the achievable by the present invention, should still fall within the range that the technical contents disclosed in the present invention can cover.

FIG. 1 is a schematic flow diagram illustrating a method for adjusting a vehicle driving system based on an off-road scene according to the present invention;

FIG. 2 is a schematic view of a vehicle model provided by the present invention;

FIG. 3 is a second flowchart of the off-road vehicle driving system adjustment method provided by the present invention;

FIG. 4 is a third schematic flow chart illustrating a third method for adjusting a vehicle driving system based on an off-road scene according to the present invention;

FIG. 5 is a flow chart of the road surface semantic segmentation provided by the present invention;

FIG. 6 is a fourth flowchart illustrating a method for adjusting a vehicle driving system based on an off-road scene according to the present invention;

FIG. 7 is a flowchart of a method for generating attribute information according to the present invention;

fig. 8 is a schematic structural diagram of a vehicle driving system adjustment system based on an off-road scene provided by the invention.

Detailed Description

The present invention is described in terms of particular embodiments, other advantages and features of the invention will become apparent to those skilled in the art from the following disclosure, and it is to be understood that the described embodiments are merely exemplary of the invention and that it is not intended to limit the invention to the particular embodiments disclosed. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

The invention provides a vehicle running system adjusting method and system in order to solve two core problems of safety and smoothness in an off-road process. In the driving process, whether the chassis can be prompted to pass or not and adjusted intelligently through the sensing signals of the binocular sensor, driving smoothness is improved, and a strong technical guarantee is provided for vehicle and personal safety.

Referring to fig. 1, fig. 1 is a schematic flow chart of a method for adjusting a vehicle driving system based on an off-road scene according to the present invention.

In one embodiment, the invention provides a method for adjusting a vehicle running system based on an off-road scene, comprising the following steps:

s101: the method comprises the steps of obtaining image information in a target area, calculating three-dimensional point cloud of the image information, and generating depth information based on the three-dimensional point cloud. The target area is an area within a certain range on a driving road, and the image information comprises a left eye image acquired by a left camera of the binocular camera and a right eye image acquired by a right camera.

S102: RGB-D information is generated based on the image information and the depth information, and a three-dimensional space of the target area is reconstructed based on the three-dimensional point cloud. Specifically, the left eye image is an RGB image, the left eye image and the right eye image are subjected to stereo matching, a depth image is obtained through the stereo matching, and the RGB image and the depth image are fused to obtain RGB-D information; and reconstructing a three-dimensional space of the target area by using the three-dimensional point cloud obtained after stereo matching.

S103: vehicle information of a target vehicle is acquired, and a travel path and a vehicle model are generated based on the vehicle information. The vehicle information may include parameters such as vehicle speed, vehicle width, and chassis height, as shown in fig. 2, the reconstructed road surface and 3D model of the vehicle (taking the chassis of the vehicle as a world coordinate system) have two main indicators for trafficability consideration: the width that the horizontal road surface can pass is more than n times of the width of the off-road vehicle (mainly considering that the off-road vehicle has certain turning space); the protrusion below the chassis cannot be larger than the maximum ground clearance of the chassis.

S104: generating a driving route three-dimensional space according to the driving path and the three-dimensional point cloud;

s105: and generating a passability judgment result of the target area based on the three-dimensional space of the driving route and the vehicle model.

S106: and when the passability judgment result is passable, generating attribute information of the target area according to the RGB-D information and a pre-stored semantic segmentation model, and adjusting the height of a chassis according to the passability judgment result. That is, the semantic segmentation model mask perceives signal processing, thereby generating attribute information.

S107: and generating an adjusting instruction based on the attribute information, wherein the adjusting instruction is used for adjusting the running parameters of the vehicle running system. Wherein the running parameter of the vehicle running system comprises at least one of the height of the chassis suspension, the rigidity of the chassis suspension, the damping of the chassis suspension and the driving braking force of the power system.

In some embodiments, as shown in fig. 3, generating a passability determination result of the target area based on the three-dimensional space of the driving route and the vehicle model, and then further includes:

s301: and if the passability judgment result is that the vehicle can not pass through the three-dimensional point cloud, adjusting a driving path, and generating a driving route three-dimensional space according to the adjusted driving path and the three-dimensional point cloud.

That is, if the passability determination result indicates that the vehicle cannot pass, the travel route needs to be adjusted, and the process returns to step S104.

Further, when the passability determination result indicates that the communication is impossible, the method further includes:

s302: and generating an early warning instruction, wherein the early warning instruction is used for controlling an early warning system of the target vehicle to send out an early warning signal.

In a specific usage scenario, as shown in fig. 3, the specific implementation process of the method provided by the present invention is as follows:

obtaining image information and dense point cloud information of a road in front by a binocular camera, and fusing the image information and the depth information to obtain RGB-D information (RGB is a color channel; D can be depth information disparity, height information Y and position information XYZ);

reconstructing a three-dimensional space of a travel path according to the point cloud on the travel track;

then, carrying out passability judgment according to the three-dimensional travelable path and the 3D model of the vehicle, and outputting a chassis height adjusting signal;

if the vehicle can not pass through, outputting early warning to prompt a driver to adjust a driving path;

if the ROI can pass through, obtaining mask information of the ROI through a semantic segmentation model;

and (5) carrying out sensing signal processing by using the mask information, and outputting the rigidity and the damping of the chassis suspension, the driving braking force of a power system and the like.

Further, as shown in fig. 4, generating attribute information of the target region according to the RGB-D information and a pre-stored semantic segmentation model specifically includes the following steps:

s401: acquiring the RGB-D information;

s402: extracting and fusing color and spatial feature information based on the semantic segmentation model to obtain a feature fusion result;

s403: and obtaining attribute information of the target area based on the feature fusion result.

As shown in fig. 5, when performing semantic segmentation on a road surface, RGB-D information is fused as input of a model, and color and spatial feature information (see a network encoder and a decoder for feature extraction and fusion) is extracted and fused through a deep learning network feature extraction network, so that scenes that cannot be distinguished by single color or spatial information (for example, white vehicles are identified as snow, cement wall surfaces are identified as road surfaces, etc.) can be effectively solved; the semantic segmentation network is divided into two parts, an encoder: the feature resolution is reduced, the feature receptive field is improved, and the feature dimension is increased; a decoder: the resolution of the features is improved, the features are fused with the same-size features of the encoder (the corresponding positions are added), and the expression capability of the features is improved; the category (type) of each pixel on the final output image and the confidence (confidence) that it belongs to that category.

In some embodiments, as shown in fig. 6, generating an adjustment instruction based on the attribute information specifically includes the following steps:

s601: performing single-frame category statistics and multi-frame category tracking on the attribute information to obtain an output scene category;

s602: and obtaining an output control signal based on the output scene category, and generating the adjusting instruction based on the output control signal.

As shown in fig. 7, the single-frame category statistics is to count the confidence sums of different categories pixel by pixel, and select the category with the highest confidence sum as the category of the frame, as shown in formula (1):

wherein i is an attribute index, n is the number of regions of interest, x is a pixel index,

is the confidence that x pixel locations are the i attribute.

Multiple frame class tracking is performed when the confidence is accumulated for the same class

Number reaches the category switching threshold

And locking the current frame class and outputting, as formula (2):

wherein i is the category with the maximum current cumulative confidence, j is the category attribute of the current frame,

is the confidence of the current frame class.

The control signal output is the parameter of the corresponding output chassis control signal according to the current frame type information. For example, when the current road is snowy, the suspension rigidity is increased and the driving force of four wheels is adjusted.

In the above specific embodiment, the method for adjusting a vehicle driving system based on an off-road scene provided by the invention utilizes a binocular camera to obtain three-dimensional point cloud to reconstruct three-dimensional space information of a road surface, and calculates the passability of a current road of a vehicle, so that the height of a chassis is adjusted according to the fluctuation of the road surface to ensure the passing safety of the vehicle; RGB-D information of the accessible road surface is fused, attributes of the current accessible road surface are obtained through a semantic segmentation model and perception signal processing, and driving parameters such as chassis suspension rigidity, damping and driving braking force of a power system are dynamically adjusted according to the attribute information of the accessible road surface, so that smoothness when the accessible road surface passes through a drivable area is guaranteed. Therefore, the method and the system for detecting the travelable area based on the off-road scene adjust the vehicle traveling system in real time according to the road condition of the travelable area, thereby improving the road safety and smoothness when the vehicle is off-road.

In addition to the above method, the present invention also provides a vehicle driving system adjusting system based on an off-road scene, as shown in fig. 8, the system comprising:

an image acquisition unit 801 configured to acquire image information in a target area, calculate a three-dimensional point cloud of the image information, and generate depth information based on the three-dimensional point cloud;

an information generating unit 802, configured to generate RGB-D information based on the image information and the depth information, and reconstruct a three-dimensional space of the target area based on the three-dimensional point cloud;

a model generation unit 803 for acquiring vehicle information of a target vehicle, generating a travel path and a vehicle model based on the vehicle information;

a three-dimensional space generating unit 804, configured to generate a three-dimensional space of a driving route according to the driving path and the three-dimensional point cloud;

a result output unit 805 for generating a passability determination result of the target area based on the travel route three-dimensional space and the vehicle model;

an attribute generating unit 806, configured to generate attribute information of the target area according to the RGB-D information and a pre-stored semantic segmentation model if the passability determination result indicates passability;

an instruction generating unit 807 for generating an adjustment instruction for adjusting a running parameter of the vehicle running system based on the attribute information.

and if the passability judgment result is that the vehicle can not pass through the three-dimensional point cloud, adjusting a driving path, and generating a driving route three-dimensional space according to the adjusted driving path and the three-dimensional point cloud.

acquiring the RGB-D information;

and obtaining the attribute information of the target area based on the feature fusion result.

carrying out single-frame category statistics and multi-frame category tracking on the attribute information to obtain an output scene category;

In the above specific embodiment, the vehicle driving system adjusting system based on the off-road scene provided by the invention utilizes the binocular camera to obtain the three-dimensional point cloud to reconstruct the three-dimensional spatial information of the road surface, and calculates the passability of the current road of the vehicle, so that the chassis height is adjusted according to the fluctuation of the road surface to ensure the passing safety of the vehicle; RGB-D information of the accessible road surface is fused, attributes of the current accessible road surface are obtained through a semantic segmentation model and perception signal processing, and driving parameters such as chassis suspension rigidity, damping and driving braking force of a power system are dynamically adjusted according to the attribute information of the accessible road surface, so that smoothness when the accessible road surface passes through a drivable area is guaranteed. Therefore, the method and the system for detecting the travelable area based on the off-road scene adjust the vehicle traveling system in real time according to the road condition of the travelable area, thereby improving the road safety and smoothness when the vehicle is off-road.

In correspondence with the above embodiments, the present invention also provides a computer-readable storage medium containing one or more program instructions therein. Wherein the one or more program instructions are for executing the method as described above by a binocular camera depth calibration system.

The invention also provides a computer program product comprising a computer program, the computer program being storable on a non-transitory computer readable storage medium, the computer program being capable of performing the above-mentioned methods when executed by a processor.

In an embodiment of the present invention, the processor may be an integrated circuit chip having signal processing capability. The Processor may be a general purpose Processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other programmable logic device, discrete Gate or transistor logic device, discrete hardware component.

The various methods, steps and logic blocks disclosed in the embodiments of the present invention may be implemented or performed. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like. The steps of the method disclosed in connection with the embodiments of the present invention may be directly implemented by a hardware decoding processor, or implemented by a combination of hardware and software modules in the decoding processor. The software modules may be located in ram, flash, rom, prom, or eprom, registers, etc. as is well known in the art. The processor reads the information in the storage medium and completes the steps of the method in combination with the hardware.

The storage medium may be a memory, for example, which may be volatile memory or nonvolatile memory, or which may include both volatile and nonvolatile memory.

The nonvolatile Memory may be a Read-Only Memory (ROM), a Programmable ROM (PROM), an Erasable PROM (EPROM), an Electrically Erasable PROM (EEPROM), or a flash Memory.

The volatile Memory may be a Random Access Memory (RAM) which serves as an external cache. By way of example and not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double Data Rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synclink DRAM (SLDRAM), and Direct Rambus RAM (DRRAM).

The storage media described in connection with the embodiments of the invention are intended to comprise, without being limited to, these and any other suitable types of memory.

Those skilled in the art will appreciate that the functionality described in the present invention may be implemented in a combination of hardware and software in one or more of the examples described above. When software is applied, the corresponding functionality may be stored on or transmitted over as one or more instructions or code on a computer-readable medium. Computer-readable media includes both computer-readable storage media and communication media including any medium that facilitates transfer of a computer program from one place to another. A storage media may be any available media that can be accessed by a general purpose or special purpose computer.

The above embodiments are provided to further explain the objects, technical solutions and advantages of the present invention in detail, and it should be understood that the above embodiments are only examples of the present invention and are not intended to limit the scope of the present invention, and any modifications, equivalent substitutions, improvements and the like made on the basis of the technical solutions of the present invention should be included in the scope of the present invention.

Claims

1. A method of vehicle travel system adjustment based on an off-road scenario, the method comprising:

generating an adjusting instruction based on the attribute information, wherein the adjusting instruction is used for adjusting the running parameters of the vehicle running system;

generating an adjusting instruction based on the attribute information, specifically comprising:

2. The vehicle travel system adjustment method according to claim 1, wherein a passability determination result of the target area is generated based on the travel route three-dimensional space and the vehicle model, and thereafter further comprising:

3. The vehicle travel system adjustment method according to claim 2, further comprising, when the passability determination result is impassability,:

4. The vehicle running system adjusting method according to claim 1, wherein the running parameter of the vehicle running system includes at least one of:

5. The vehicle driving system adjusting method according to claim 1, wherein generating the attribute information of the target area according to the RGB-D information and a pre-stored semantic segmentation model specifically comprises:

acquiring the RGB-D information;

extracting and fusing color and space characteristic information based on the semantic segmentation model to obtain a characteristic fusion result;

6. A vehicle travel system adjustment system based on an off-road scenario, the system comprising:

the image acquisition unit is used for acquiring image information in a target area, calculating three-dimensional point cloud of the image information and generating depth information based on the three-dimensional point cloud;

an instruction generating unit configured to generate an adjustment instruction for adjusting a running parameter of a vehicle running system based on the attribute information;

generating an adjusting instruction based on the attribute information specifically comprises:

7. An intelligent terminal, characterized in that, intelligent terminal includes: the device comprises a data acquisition device, a processor and a memory;

the data acquisition device is used for acquiring data; the memory is to store one or more program instructions; the processor, configured to execute one or more program instructions to perform the method of any of claims 1-5.

8. A computer-readable storage medium having one or more program instructions embodied therein for performing the method of any of claims 1-5.