WO2021223166A1 - State information determination method, apparatus and system, and movable platform and storage medium - Google Patents

State information determination method, apparatus and system, and movable platform and storage medium Download PDF

Info

Publication number
WO2021223166A1
WO2021223166A1 PCT/CN2020/089002 CN2020089002W WO2021223166A1 WO 2021223166 A1 WO2021223166 A1 WO 2021223166A1 CN 2020089002 W CN2020089002 W CN 2020089002W WO 2021223166 A1 WO2021223166 A1 WO 2021223166A1
Authority
WO
WIPO (PCT)
Prior art keywords
detection frame
frame information
information
target
target object
Prior art date
Application number
PCT/CN2020/089002
Other languages
French (fr)
Chinese (zh)
Inventor
陆泽早
Original Assignee
深圳市大疆创新科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 深圳市大疆创新科技有限公司 filed Critical 深圳市大疆创新科技有限公司
Priority to CN202080004726.6A priority Critical patent/CN112639405A/en
Priority to PCT/CN2020/089002 priority patent/WO2021223166A1/en
Publication of WO2021223166A1 publication Critical patent/WO2021223166A1/en

Links

Images

Classifications

    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
    • G01C23/00—Combined instruments indicating more than one navigational value, e.g. for aircraft; Combined measuring devices for measuring two or more variables of movement, e.g. distance, speed or acceleration
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/52—Surveillance or monitoring of activities, e.g. for recognising suspicious objects

Definitions

  • the present disclosure relates to the technical field of movable platforms, and in particular to methods, devices, systems, movable platforms, and storage media for determining status information of target objects.
  • Movable platforms include unmanned aerial vehicles (“UAVs”), sometimes referred to as “unmanned aerial vehicles”. Users can remotely operate or program to achieve automatic flight. UAVs include unmanned aerial vehicles of various sizes and configurations. Of course, the movable platform is not limited to this. For example, the movable platform may also include unmanned vehicles, unmanned ships and other mobile equipment.
  • Movable platforms can be used for many purposes, such as various personal, commercial and tactical applications.
  • the movable platform may be equipped with imaging devices, such as cameras, video cameras, etc.
  • the movable platform equipped with the imaging device can determine the status information of the target object.
  • the status information may include position information, for example, and the target object may include people, moving objects, and stationary objects, for example.
  • the present disclosure provides a method for determining status information of a target object, including: obtaining multiple frames of images about the target object through an imaging device carried by a movable platform; Multiple detection frame information; obtain multiple ranging results between the movable platform and the target object; filter effective ranging results from multiple ranging results according to multiple detection frame information; and according to multiple detection frame information And the effective ranging result determines the status information of the target object.
  • the present disclosure also provides another method for determining the status information of a target object, including: obtaining multiple frames of images about the target object through an imaging device carried by a movable platform; Multiple detection frame information of the target object; determine one or more target detection frame information satisfying preset conditions among the multiple detection frame information; obtain multiple ranging results between the movable platform and the target object; and according to one or Multiple target detection frame information and multiple ranging results determine the status information of the target object.
  • the present disclosure also provides a device for determining status information of a target object, including: a processor; a readable storage medium for storing one or more programs, wherein when the one or more programs are executed by the processor, the processing
  • the device performs the following operations: obtains multi-frame images of the target object obtained by the imaging device carried by the movable platform; recognizes each frame of the multi-frame image to obtain multiple detection frame information about the target object; obtains Multiple ranging results between the mobile platform and the target object; filtering effective ranging results from multiple ranging results based on multiple detection frame information; and determining the target object based on multiple detection frame information and effective ranging results status information.
  • the present disclosure also provides another device for determining status information of a target object, including: a processor; a readable storage medium for storing one or more programs, wherein when the one or more programs are executed by the processor, The processor performs the following operations: obtaining multiple frames of images about the target object obtained by the imaging device carried by the movable platform; identifying each frame of the multiple frames of images to obtain multiple detection frame information about the target object; determining One or more target detection frame information that meets the preset conditions among the multiple detection frame information; obtain multiple ranging results between the movable platform and the target object; and according to one or more target detection frame information and multiple measurements The distance result determines the status information of the target object.
  • the present disclosure also provides a system for determining status information of a target object, including: an imaging device for obtaining multiple frames of images about the target object; and the status information determining device as described above.
  • the present disclosure also provides a movable platform, including: a movable body; and the state information determining system as described above.
  • the present disclosure also provides a readable storage medium having executable instructions stored thereon, and when the instructions are executed by a processor, the processor executes the method described above.
  • invalid ranging results are identified based on multiple detection frame information, so that effective ranging results can be filtered from multiple ranging results, and the target object is determined based on multiple detection frame information and effective ranging results
  • the state information of the target object at least partially solves the technical problem that the state information of the target object is contaminated by the wrong data, which reduces the false detection rate of the ranging result and improves the reliability of the state information estimation.
  • the state information of the target object is determined according to the one or more target detection frame information and the multiple ranging results . At least partially solves the technical problem that the state information of the target object is contaminated by the wrong data, which causes the state estimation of the target object to fail, and improves the reliability of the state information estimation.
  • Fig. 1 schematically shows an application scenario in which a method for determining state information of a target object can be applied according to an embodiment of the present disclosure.
  • FIG. 2 schematically shows a schematic diagram of a frame of image about a target object captured by an imaging device according to an embodiment of the present disclosure.
  • Fig. 3 schematically shows a flowchart of a method for determining status information of a target object according to an embodiment of the present disclosure.
  • Fig. 4 schematically shows a flowchart of screening one or more ranging results corresponding to each detection frame information according to an embodiment of the present disclosure.
  • Fig. 5 schematically shows a flowchart of determining the validity of each ranging result according to the detection frame information corresponding to each ranging result and the laser spot corresponding to each ranging result according to an embodiment of the present disclosure.
  • Fig. 6 schematically shows a flowchart of determining target detection frame information according to an embodiment of the present disclosure.
  • Fig. 7 schematically shows a flow chart of determining whether a target object moves that meets a preset condition according to multiple detection frame information according to an embodiment of the present disclosure.
  • FIG. 8 schematically shows a schematic diagram of a probability distribution of initial position information corresponding to detection frame information according to an embodiment of the present disclosure.
  • Fig. 9 schematically shows a schematic diagram of a probability distribution of initial position information corresponding to detection frame information according to another embodiment of the present disclosure.
  • FIG. 10 schematically shows a schematic diagram of superimposing the first probability distribution and the second probability distribution to determine the spatial position with the highest probability density according to an embodiment of the present disclosure.
  • FIG. 11 schematically shows a flowchart of determining the state information of a target object according to each target detection frame information and the effective ranging result corresponding to each target detection frame information according to an embodiment of the present disclosure.
  • FIG. 12 schematically shows a schematic diagram of a time axis for screening the initial position information of the target object corresponding to each target detection frame information according to an embodiment of the present disclosure.
  • FIG. 13 schematically shows a schematic diagram of predicting the position information of a target object according to an embodiment of the present disclosure.
  • Fig. 14 schematically shows a flowchart of a method for determining status information of a target object according to another embodiment of the present disclosure.
  • Fig. 15 schematically shows a block diagram of a system for determining status information of a target object according to an embodiment of the present disclosure.
  • the movable platform can be equipped with an imaging device to allow users to remotely determine the location information of the target object, track the target object or take pictures and other applications.
  • the target object can include, for example, people, moving objects, and stationary objects. Take tracking target objects as an example. This ability to track target objects allows the movable platform to autonomously operate imaging devices and other devices to facilitate imaging of the target objects while tracking or photographing moving target objects.
  • a movable platform e.g., UAV
  • UAV may be configured to autonomously track the movement of an object and adjust the speed and direction of its movement accordingly, while adjusting the orientation of the imaging device to maintain a predetermined distance from the object. relative position.
  • the UAV can maintain a predetermined field of view for the object, so that when the object moves, the image of the object can be captured with substantially the same range and accuracy.
  • machine learning algorithms can be used to identify the target object that needs to be tracked on the image collected by the imaging device to obtain the detection frame of the target object in the image, and determine the position of the target object according to the detection frame of the target object. And change the position of the movable platform, the posture of the imaging device, etc. according to the position of the target object, so as to track the target object.
  • the accuracy and reliability of determining the position of the target object according to the detection frame of the target object is not high.
  • the distance between the mobile platform and the target object makes it impossible to determine the position of the target object more accurately.
  • the detection frame of the target object can be combined with the distance measurement result of the target object, which can improve the accuracy and reliability of the status information of the target object. Therefore, how to combine the detection frame of the target object with the ranging result to obtain status information such as the position and trajectory of the target object is a problem to be solved urgently.
  • the position and distance information of the target object can be provided by the visual detection frame of the target object combined with a variety of ranging methods, common ones are: satellite navigation system, lidar, ToF (Time of Flight ranging method, referred to as ToF) ranging , Binocular distance measurement, triangulation distance measurement based on parallax, and distance measurement based on other prior knowledge (for example, the height of the target object to the ground, the height of the target object itself).
  • ToF Time of Flight ranging method
  • the positioning accuracy of satellite navigation system is low, but it is not affected by visual obstruction; Lidar and ToF ranging have high accuracy and large ranging range, but there is no obstacle between the UAV and the target object; binocular measurement Range accuracy is limited by the size of the UAV, and the accuracy of long-distance targets is poor; triangulation based on parallax is only suitable for static target objects, and the UAV needs to fly with a fixed trajectory.
  • the imaging device shoots the target object with two different perspectives. ;
  • the accuracy of distance measurement based on prior knowledge depends on the degree of conformity between the prior hypothesis and the actual situation. Therefore, an improved method for determining the status information of the target object is needed to enhance the adaptability and reliability of determining the status information of the target object.
  • an improved method for determining the status information of a target object is provided.
  • the detection frame of the target object can be recognized by a machine learning method, and the relative position of the target object can be obtained by measuring with a distance measuring device (for example, single-point lidar). Based on the distance of the movable platform, the two kinds of data are combined to obtain real-time or near real-time status information of the target object.
  • the status information of the target object may include, for example, current position, speed, historical trajectory, and so on.
  • FIG. 1 schematically shows an application scenario in which a method for determining state information of a target object can be applied according to an embodiment of the present disclosure. It should be noted that FIG. 1 is only an example of a scenario where the embodiment of the present disclosure can be applied to help those skilled in the art understand the technical content of the present disclosure, but it does not mean that the embodiment of the present disclosure cannot be used for other devices. , System, environment or scene.
  • the field of view of the imaging device of the drone 101 may include, but is not limited to, a target object 102, an object 103, an object 104, and the like.
  • the drone 101 can capture one or more frames of images about the target object 102 through an imaging device, and by recognizing each frame of image, the detection frame information about the target object 102 can be obtained.
  • the target object 102 may be in a moving state or in a stationary state.
  • the imaging device can capture images of the target object 102 in real time or at preset time intervals, and then recognize the captured images to obtain multiple detection frames information.
  • FIG. 2 schematically shows a schematic diagram of a frame of image about a target object captured by an imaging device according to an embodiment of the present disclosure.
  • the image 200 captured by the imaging device includes a target object 102, an object 103, and an object 104.
  • the image 200 can be recognized by a neural network based on machine learning, and a detection frame information 1021 about the target object 102 can be obtained.
  • the UAV 101 can obtain the distance of the target object 102 relative to the UAV 101 through a distance measuring device (for example, a single-point lidar).
  • a distance measuring device for example, a single-point lidar.
  • the distance measuring device can measure the distance of the target object 102 relative to the drone 101 in real time or according to a preset time interval, and obtain multiple distance measurement results.
  • real-time or near real-time status information of the target object 102 can be obtained, for example, the position information, speed information, historical trajectory, etc. of the target object can be obtained.
  • the embodiments of the present disclosure can be applied to various applications based on the status information of the target object.
  • the target state information obtained by the embodiments of the present disclosure can be further used for target object tracking, target object surrounding, imaging device orientation control, imaging device tracking focus, tracking zoom, target object position prediction, feedback optimization of visual target recognition , AR (Augmented Reality, Augmented Reality, AR for short) surveillance and other fields have realized scene applications such as controlling the movable platform and the imaging device carried by it to track the target object.
  • Fig. 3 schematically shows a flowchart of a method for determining status information of a target object according to an embodiment of the present disclosure.
  • operation S301 and operation S302 have an order in terms of technical implementation, while operation S303, operation S301 and operation S302 do not have an order in terms of technical implementation, and it is not clearly stated that operation S301 and operation S302 are in operation S303. Executed before. Therefore, although the operation S301 and the operation S302 precede the operation S303 in the flowchart shown in FIG. 3, when the operation S303 is performed, it may be performed before the operation S301 and the operation S302 are performed, or simultaneously with the operation S301 and the operation S302. .
  • the method for determining the status information of the target object includes operations S301 to S305.
  • a multi-frame image about a target object is obtained through an imaging device carried by a movable platform.
  • each of the multiple frames of images is recognized to obtain multiple detection frame information about the target object.
  • the imaging device can continuously capture images within its field of view. By performing image recognition on the captured image, a multi-frame image containing the target object is obtained.
  • the image captured by the imaging device may not include the target object.
  • each frame of the captured image can be recognized by a machine learning method, and if the target object is included in the image, a detection frame information about the target object can be obtained. If there is no target object in the image, the image can be marked as an invalid image, and further, the invalid image can be filtered out, or if there is no target object in the image, the image can be filtered out directly.
  • the detection frame information may include a variety of information, for example, it may include part or all of the following: the position and size of the target object on the image screen, the angle of view, the position of the imaging device, and the The posture, the sampling time point of the image, and so on.
  • the detection frame can be represented by a rectangular frame (as shown in FIG. 2), and the position and size of the target object on the image screen can be provided by the target recognition module.
  • the field angle of the imaging device can be provided by the zoom module of the imaging device, and the zoom module can execute the following algorithms (1) and (2) to obtain the field angle of the imaging device.
  • fov zx represents the field angle in the row direction of the image frame
  • fov zy represents the field angle in the column direction of the image frame
  • focal_length is the real-time focal length of the imaging device
  • W, H are the width and height of the image sensor of the imaging device.
  • the posture of the imaging device relative to the world coordinate system and the corresponding sampling time can be provided by the vehicle posture measurement module.
  • the position of the imaging device in the world coordinate system and the corresponding sampling time may be provided by the vehicle position measurement module.
  • multiple distance measurement results between the movable platform and the target object can be measured by the distance measurement device.
  • the type of the ranging device is not limited.
  • a single-point lidar can be used to obtain the distance of the target object relative to the UAV.
  • the single-point lidar can be set on a movable platform.
  • the present disclosure is not limited to obtaining ranging results through single-point lidar on a movable platform, and can also obtain the movable platform and the distance measurement device through other ranging devices such as ToF camera, single-line lidar, area array lidar, binocular camera, etc. Multiple ranging results between target objects.
  • the ranging device can measure each ranging result in a time sequence, and each ranging result has a corresponding sampling time point.
  • the distance measurement device may use a single-point lidar with the characteristics of long distance measurement and high accuracy, and the distance measurement result between the movable platform and the target object may be the laser distance measurement result.
  • the imaging device can use a monocular camera, and the cost is lower than that of lidar and camera arrays.
  • the use of single-point lidar to measure the real-time distance between the movable platform and the target object does not require a satellite navigation system to provide the spatial position of the target object, nor does it rely on the relative height of the imaging device and the target object to remain unchanged or basically unchanged.
  • the continuous ranging of long-distance target objects is more reliably achieved, which increases the adaptability and reliability of target object position estimation.
  • invalid laser measurement results can be filtered, and the false detection rate of lasers can be reduced.
  • an effective ranging result is filtered from the multiple ranging results according to the multiple detection frame information.
  • the sampling frequency of the imaging device and the sampling frequency of the ranging device may be the same or different.
  • Each detection frame information may have a corresponding ranging result, or each detection frame information may have corresponding multiple ranging results.
  • one or more ranging results corresponding to each detection frame information can be determined first; then one or more ranging results corresponding to each detection frame information Or multiple ranging results to filter.
  • each detection frame information has a corresponding sampling time point, and the sampling time point of the image corresponding to the detection frame information can be used as the sampling time point corresponding to the detection frame information.
  • each can be determined according to the sampling time point of each detection frame information and the sampling time point of each ranging result in the multiple obtained ranging results.
  • One or more ranging results corresponding to each detection frame information are determined according to the sampling time point of each detection frame information and the sampling time point of each ranging result in the multiple obtained ranging results.
  • each ranging result can be associated with the detection frame information closest to the sampling time point of the ranging result.
  • the sampling time point corresponding to the ranging result is subtracted from the sampling time point of the first detection frame information to obtain the first A time difference, subtract the sampling time point of the second detection frame information from the sampling time point corresponding to the ranging result to obtain the second time difference, compare the absolute value of the first time difference and the second time difference, and detect the smaller absolute value
  • the frame information is determined as the detection frame information closest to the sampling time point of the ranging result, so that one or more ranging results corresponding to each detection frame information can be determined.
  • invalid ranging results can be identified based on multiple detection frame information, so as to filter out effective ranging results from the multiple ranging results.
  • the state information of the target object is determined according to the multiple detection frame information and the effective ranging result.
  • the effective ranging result may include one or more.
  • the effective ranging result corresponding to each detection frame information can be determined first; then, the state information of the target object can be determined according to each detection frame information and the effective ranging result corresponding to each detection frame information.
  • the state information of the target object can be determined according to the field of view, position and posture of the imaging device corresponding to each detection frame, and the corresponding effective ranging result.
  • the effective ranging result corresponding to each detection frame information can be determined according to the sampling time point.
  • each effective ranging result can be associated with the detection frame information closest to the sampling time point of the effective ranging result.
  • the specific method can refer to the above example of associating the ranging result to a certain detection frame of the two detection frame information, which will not be repeated here.
  • the state information of the target object is determined according to each detection frame information and the effective ranging result corresponding to each detection frame information, Including: calculating the weighted average of multiple effective ranging results corresponding to each detection frame information to obtain the target ranging result corresponding to each detection frame information; and according to each detection frame information and the target corresponding to each detection frame information The results of ranging, determine the status information of the target object.
  • the result obtained by dividing the echo intensity of each effective ranging result corresponding to the detection frame information by the reference value can be used as the weight of the effective ranging result, where the reference value corresponds to the detection frame information
  • the reference value may be the sum of echo intensities of multiple effective ranging results corresponding to the detection frame information.
  • the corresponding effective ranging results include effective ranging results 1 to 4, and the echo intensity corresponding to the effective ranging results 1 to 4 is echo intensity 1 to echo intensity 4.
  • the weight 1 of the effective ranging result 1 is the echo strength 1/(echo strength 1+echo strength 2+the echo strength 3+the echo strength 4)
  • the weight 2 of the effective ranging result 2 is the echo strength 2/ (Echo Intensity 1+Echo Intensity 2+Echo Intensity 3+Echo Intensity 4)
  • the weight 3 of the effective ranging result 3 is the echo intensity 3/(Echo Intensity 1+Echo Intensity 2+Echo Intensity 3+Echo Intensity 4)
  • the weight 4 of the effective ranging result 4 is Echo Intensity 4/(Echo Intensity 1+Echo Intensity 2+Echo Intensity 3+Echo Intensity 4).
  • the target ranging result corresponding to the first detection frame information is equal to the effective ranging result 1 ⁇ weight 1+effective ranging result 2 ⁇ weight 2+effective ranging result 3 ⁇ weight 3+effective ranging result 4 ⁇ weight 4.
  • the state information of the target object includes, but is not limited to, position information, speed information, and historical trajectory of the target object in the world coordinate system.
  • the historical trajectory includes several trajectory points.
  • invalid ranging results are identified based on multiple detection frame information, so that effective ranging results can be filtered from multiple ranging results, and the target object is determined based on multiple detection frame information and effective ranging results
  • the state information of the target object at least partially solves the technical problem that the state information of the target object is contaminated by the wrong data, which reduces the false detection rate of the ranging result and improves the reliability of the state information estimation.
  • the detection frame information and the effective ranging result are combined to determine the status information of the target object, there is no excessive requirement on the obtained image data, and the depth image can be collected without using a depth sensor, which at least partially solves the limitations of the depth sensor. This leads to the problem that the effective range of the obtained depth map is small or the accuracy is low.
  • FIG. 3 The method shown in FIG. 3 will be further described below with reference to FIGS. 4 to 13 in conjunction with specific embodiments.
  • Fig. 4 schematically shows a flowchart of screening one or more ranging results corresponding to each detection frame information according to an embodiment of the present disclosure.
  • screening one or more ranging results corresponding to each detection frame information may include operations S401 to S403.
  • a laser spot corresponding to each of the one or more ranging results corresponding to each detection frame information is determined.
  • the field angle of the imaging device and the scattering angle of the lidar can be used to calculate the distance measurement results lidar m ...lidar n-1 , lidar n corresponds to the laser spot circle m ... circle n-1 and circle n in the picture.
  • each ranging result is determined according to the detection frame information corresponding to each ranging result and the laser spot corresponding to each ranging result.
  • the detection frame information corresponding to each ranging result and the area coincidence rate of the laser spot corresponding to each ranging result can be determined, and then the validity of each ranging result can be determined according to the area coincidence rate.
  • each ranging result has corresponding detection frame information and laser spot.
  • the area of the detection frame can be calculated according to the detection frame information corresponding to the distance measurement result (such as the length and width of the detection frame in the screen), and the area of the laser spot is divided by the area of the detection frame to obtain the detection frame information and corresponding to the distance measurement result.
  • the area coincidence rate between the laser spots, and then the validity of each ranging result is determined according to the area coincidence rate.
  • the area coincidence rate can be compared with a preset ratio threshold; the distance measurement result whose area coincidence ratio is greater than or equal to the preset ratio threshold is determined as the effective distance measurement result; the area coincidence ratio is less than the preset ratio The distance measurement result of the threshold value is determined to be an invalid distance measurement result.
  • the size of the preset ratio threshold can be preset based on experience.
  • the size of the preset ratio threshold may be 70%. If the area overlap rate is greater than or equal to 70%, the ranging result can be marked as valid, otherwise the ranging result can be marked as invalid, and then the invalid ranging result will be filtered out according to the marking result.
  • one or more ranging results corresponding to each detection frame information are filtered according to the validity of each ranging result.
  • invalid ranging results can be filtered out from one or more ranging results corresponding to the detection frame information, and the valid ranging results can be determined as the ranging results corresponding to the detection frame information.
  • Fig. 5 schematically shows a flowchart of determining the validity of each ranging result according to the detection frame information corresponding to each ranging result and the laser spot corresponding to each ranging result according to an embodiment of the present disclosure.
  • determining the validity of each ranging result may include operations S501 to S502.
  • operation S501 according to the information of two detection frames with adjacent sampling times, determine the interpolation detection frame corresponding to each of the multiple ranging results between the sampling times of the two detection frame information with adjacent sampling times. Information to obtain the detection frame information corresponding to each ranging result.
  • the corresponding sampling time points are respectively t a and t b , and t a and t are selected
  • the sampling time points corresponding to the ranging results are t m , t m+1 ...t n-1 , t n , where t a ⁇ t m ... t n-1 and t n ⁇ t b are satisfied.
  • the detection frame changes from box a to box b at a constant speed
  • the interpolation algorithm is used to obtain the interpolation detection frames box m ... t m ... t n-1 and t n respectively.
  • the laser spot can be calculated by using the field of view angle of the imaging device and the scattering angle of the lidar.
  • each ranging result is determined according to the laser spot corresponding to the ranging result and the detection frame information corresponding to the ranging result.
  • the validity of each ranging result can be determined according to the detection frame information corresponding to the ranging result and the area overlap ratio of the laser spot corresponding to the ranging result.
  • the specific method reference may be made to the above description of FIG. 4, which will not be repeated here.
  • the previous detection frame whose sampling time is closest to the newly generated detection frame can be determined, and the sampling time is the newly generated detection frame and the closest detection frame.
  • the results of the distance measurement between the previous detection frame are filtered. It can realize real-time or near real-time screening of the ranging results, so as to achieve the effect of more accurately determining the status information of the target object.
  • the detection frame information of the target object is obtained by the machine learning method
  • the multiple ranging results of the target object relative to the movable platform are obtained by the ranging device
  • the multiple ranging results are filtered by the detection frame information
  • Obtaining effective ranging results, combining the detection frame information of the target object and the effective ranging results, can obtain real-time or near real-time status information of the target object.
  • the physical estimated size of the target object corresponding to each detection frame information can also be determined;
  • the physical estimated size of the object, and the effective ranging results corresponding to each detection frame information are screened.
  • the field angle corresponding to each detection frame information can be determined according to each detection frame information and the field of view angle of the imaging device when each detection frame information is collected; and then corresponding to each detection frame information
  • the field of view angle and the effective ranging result corresponding to each detection frame information determine the physical estimated size corresponding to each detection frame information.
  • the physical estimated size is the estimated size of the target object in the real world.
  • calculating the physical estimated size of the target object corresponding to the detection frame information may include the following operations.
  • the angle formed by the connection between the optical center and the four corners of the top, bottom, left, and right corners of the rectangular frame can be determined by using the pixel offset and the real-time viewing angle of the screen), and calculate the upper edge and bottom of the rectangular frame of the target object on the screen
  • the edges respectively correspond to the pitch angle relative to the imaging device.
  • the angle range, the range of the heading angle and the range of the pitch angle of the target object relative to the imaging device are hereinafter referred to as the field of view angle of the target detection frame.
  • the field angle of the detection frame and the effective ranging result can be used to determine the arc length corresponding to the heading angle and the pitch angle.
  • the end points of the two arc lengths can be used to determine the rectangular frame The range of the four sides can thus determine the physical estimated size of the target detection frame.
  • the reasonable range of the physical size of the target object can be estimated based on the prior knowledge of the type of the target object, and unreasonable effective ranging results can be filtered.
  • the above-mentioned filtering the effective ranging result corresponding to each detection frame information according to the physical estimated size of the target object corresponding to each detection frame information may include: Compare the physical estimated size of the target object with the preset reasonable range; and filter out the effective ranging corresponding to the detection frame information when the physical estimated size of the target object corresponding to the detection frame information does not match the preset reasonable range result.
  • the object type of the target object can be determined first, where each object type has a corresponding preset reasonable range; then the target preset reasonable range is determined according to the object type of the target object; the information of each detection frame The corresponding physical estimated size of the target object is compared with the preset reasonable range of the target.
  • the preset reasonable range of the target can be 0.6 meters to 2 meters, and the calculated physical estimated size can be compared with 0.6 meters to 2 meters.
  • the preset reasonable range of the target can be 2 meters to 20 meters, and the calculated physical estimated size can be compared with 2 meters to 20 meters.
  • the detection frame information can be filtered out.
  • the detection information may not be filtered out.
  • the effective ranging result corresponding to the detection frame information adjacent to the sampling time of the detection frame information from which the effective ranging result is filtered out is determined to be the effective ranging result corresponding to the detection frame information from which the effective ranging result is filtered out.
  • one or more target detection frame information that meets the preset condition among the multiple detection frame information can be determined first, According to each target detection frame information and the effective ranging result corresponding to each target detection frame information, the state information of the target object is determined, and other detection frames except the target detection frame information may not be used.
  • the effective ranging result corresponding to each target detection frame information in one or more target detection frame information can be determined first, in other words, the target detection frame information and the corresponding effective ranging result are first determined. Associate, and then determine the status information of the target object according to each target detection frame information and the effective ranging result corresponding to each target detection frame information.
  • Fig. 6 schematically shows a flowchart of determining target detection frame information according to an embodiment of the present disclosure.
  • determining the target detection frame information may include operations S601 to S602.
  • operation S601 it is determined whether the target object has moved that meets a preset condition according to a plurality of detection frame information.
  • the imaging device can capture multiple frames of images, recognize the multiple frames of images to obtain multiple detection frame information, and can determine that the target object is within a period of time based on the multiple detection frame information Whether there is a large movement to determine whether the target object has a movement that meets the preset conditions.
  • the detection frame information corresponding to the movement of the target object that satisfies the preset condition is determined as the target detection frame information.
  • the detection frame information corresponding to the target object when the target object has a large movement can be determined as the target detection frame information.
  • the detection frame information corresponding to the target object may include multiple pieces of detection frame information when the target object moves substantially. Therefore, multiple pieces of target detection frame information may be obtained. Further, multiple target detection frame information can be marked for use when determining the status information of the target object.
  • Fig. 7 schematically shows a flow chart of determining whether a target object moves that meets a preset condition according to multiple detection frame information according to an embodiment of the present disclosure.
  • determining whether the target object has moved that meets a preset condition according to multiple detection frame information may include operations S701 to S704.
  • multiple detection frame information within a time period can be acquired, for the first detection frame information and the second detection frame information that are adjacent to any two sampling time points in the one time period, Obtain the state information of the imaging device when the information of each detection frame is collected.
  • the status information of the imaging device may include the orientation of the imaging device, the angle of view of the imaging device, and the position of the imaging device.
  • a first probability distribution of the initial position information of the target object corresponding to the first detection frame information is determined.
  • a second probability distribution of the initial position information of the target object corresponding to the second detection frame information is determined.
  • the present embodiment of the present disclosure e.g., a period of time of any two selected detection frame and box a box b, and looks toward the detection frame box a sampling time and box b t a and t b of the corresponding image forming apparatus,
  • detection frame information The angle of view of the imaging device, the position of the imaging device (hereinafter referred to as detection frame information), and the effective ranging result.
  • detection frame information The angle of view of the imaging device, the position of the imaging device (hereinafter referred to as detection frame information), and the effective ranging result.
  • the back projection algorithm is used to calculate the probability distribution of the initial position information corresponding to the detection frame information according to the detection frame information and the effective ranging result, and the probability distribution represents the detection frame within the physical space.
  • the probability of mapping to each spatial location e.g., a period of time of any two selected detection frame and box a box b, and looks toward the detection frame box box a sampling time and box b t a and t b
  • FIG. 8 schematically shows a schematic diagram of a probability distribution of initial position information corresponding to detection frame information according to an embodiment of the present disclosure.
  • the probability distribution of the initial position information is a flowerpot type (or called a truncated cone type).
  • the position of the target object can be used as the original position, and the part before the effective ranging result 0.95 times away from the target object and the part after the effective ranging result 1.05 times away from the target object can be cut to obtain the target object’s original position.
  • the probability distribution is a flowerpot type, and the effective ranging result can be the distance between the imaging device and the target object.
  • the upper and lower limits of the effective ranging result are positively correlated with the measurement error of the measuring device. The closer to the center, the higher the probability.
  • Fig. 9 schematically shows a schematic diagram of a probability distribution of initial position information corresponding to detection frame information according to another embodiment of the present disclosure.
  • the probability distribution of the initial position information is a cone Type, as shown in Figure 9, the closer to the cone axis, the higher the probability.
  • the effective ranging result corresponding to the detection frame information adjacent to the sampling time point of the detection frame 901 can be determined as the effective ranging result corresponding to the detection frame 901. Then, according to the effective ranging result corresponding to the detection frame 901, the method described in FIG. 8 is adopted to cut the cone-shaped probability distribution into a flowerpot-shaped probability distribution.
  • determining whether the target object moves that meets a preset condition includes the following operations.
  • the spatial location with the highest probability density is determined according to the first probability distribution and the second probability distribution.
  • the first probability distribution and the second probability distribution are superimposed, and the spatial position with the highest probability density is calculated.
  • the probability distribution superposition method can also be used to determine the spatial position with the highest probability density.
  • FIG. 10 schematically shows a schematic diagram of superimposing the first probability distribution and the second probability distribution to determine the spatial position with the highest probability density according to an embodiment of the present disclosure.
  • the first probability distribution corresponding to the first detection frame information 1001 and the second probability distribution corresponding to the second detection frame information 1002 are superimposed to determine the spatial position 1003 with the highest probability density.
  • the first distance between the space position with the highest probability density and the center position of the first probability distribution of the first probability distribution and the second distance between the space position with the highest probability density and the center position of the second probability distribution of the second probability distribution are respectively calculated.
  • the Mahalanobis distance Euclidean distance multiplied by the probability distribution coefficient
  • the probability distance between the first detection frame information and the second detection frame information can be determined according to the first distance and the second distance. For example, the sum of the first distance and the second distance is taken as the probability distance between the two detection frames.
  • the probability distance between the first detection frame information and the second detection frame information is compared with a preset threshold. If the probability distance is greater than or equal to the preset threshold, it can be determined that the target object has moved that meets the preset condition.
  • the size of the preset threshold may reflect the amplitude requirement of the target object's movement. For example, the larger the preset threshold value, the higher the amplitude requirement of the target object's movement.
  • the preset threshold may be preset according to actual effects, and optionally, the preset threshold may be 1.0.
  • the probability distance between any two detection frames in a period of time is greater than a preset threshold, it can be considered that the target object has a large movement in the period of time.
  • the length of the time period can be preset, for example, it can be 2 seconds, 5 seconds, and so on.
  • the state information of the target object can be determined according to each target detection frame information and the effective ranging result corresponding to each target detection frame information.
  • the initial position information of the target object can be determined according to the orientation, field of view, position of the imaging device and the effective ranging result when the target detection frame is collected.
  • the target detection frame information may be detection frame information that has undergone a large movement.
  • FIG. 11 schematically shows a flowchart of determining the state information of a target object according to each target detection frame information and the effective ranging result corresponding to each target detection frame information according to an embodiment of the present disclosure.
  • determining the state information of the target object may include operations S1101 to S1103.
  • each target detection frame information and the effective ranging result corresponding to each target detection frame information determine the initial position information about the target object corresponding to each target detection frame information, and obtain multiple initial position information.
  • a plurality of initial position information is filtered according to the sampling time point corresponding to each target detection frame information to obtain one or more effective initial position information.
  • the initial position information about the target object corresponding to each target detection frame information can be sequentially filtered according to the time sequence of the sampling time point.
  • the state information of the target object is determined according to one or more valid initial position information.
  • the movement trajectory of the target object can be generated according to a plurality of valid initial position information.
  • the moving speed, acceleration, etc. of the target object can be calculated based on multiple valid initial position information,
  • the one or more valid initial position information can be optimized to smooth the movement trajectory of the target object.
  • the final optimized position information about the target object can be obtained according to the final optimized motion trajectory.
  • sequentially screening the initial position information about the target object corresponding to each target detection frame information in a chronological order includes the following operations.
  • the target detection frames box m ...box n-1 , box n in which the target object has moved substantially are selected, and the sampling time of these detection frames t m ... t n-1, t n corresponding to the initial position information pos m ... pos n-1, pos n as the state variable to be screened.
  • back projection Algorithm to calculate the initial value of the initial position information pos m ... pos n-1 , pos n . If the target detection frame does not have a corresponding effective ranging result, the previous effective ranging result adjacent to the sampling time can be used for calculation to obtain the initial value.
  • the time difference between it and the next state variable to be screened is calculated, and from all the state variables to be screened, the state variable whose time difference with the next state variable to be screened is less than the state variable threshold is eliminated.
  • the next state variable to be filtered is the next initial position information adjacent to the sampling time of the initial position information currently being filtered.
  • the state variable threshold may be a fixed value set in advance, for example, set in advance based on experience.
  • the state variable threshold may also be changed with the determined effective initial position information.
  • the state variable threshold is the time difference between the initial location information currently being screened and the sampling time point of the effective initial location information adjacent to the sampling time of the initial location information currently being screened.
  • the corresponding sampling time point is t 1 to t 6 .
  • FIG. 12 schematically shows a schematic diagram of a time axis for screening the initial position information of the target object corresponding to each target detection frame information according to an embodiment of the present disclosure.
  • the sampling time t 6 the corresponding target detection frame box 6 may be a new current detection frame.
  • Sampling the above-described embodiment can be calculated from 1 to the initial value of POS 6 initial position information POS, the initial position information pos 1 ⁇ pos initial value of 6 corresponds to being screened state variable 1 to be screened, the state variables 6, to be screened state variables 6 as the last A state variable to be filtered.
  • the state variable 6 to be screened since it is the latest detection frame, it can be directly used as a valid state variable without being eliminated.
  • all the state variables to be filtered can be processed in reverse chronological order, as shown in Figure 12, that is, the state variables to be filtered 5 are processed first, and then the states to be filtered are processed in turn Variable 4 ⁇ state variable 1 to be filtered.
  • the to-be-filtered state variable 6 is the next to-be-filtered state variable of the to-be-filtered state variable 5
  • the to-be-filtered state variable 5 is the next to-be-filtered state variable of the to-be-filtered state variable 4
  • the to-be-filtered state variable 4 is the state variable to be filtered.
  • the state variable to be filtered after 3, and so on.
  • the time difference between it and the next state variable to be screened is calculated, and the time difference is compared with the corresponding preset threshold.
  • the time difference t 2 -t 1 between it and the state variable 2 to be screened is calculated, and t 2 -t 1 is compared with the first preset threshold. If t 2 -t 1 is less than the first preset threshold, If the threshold is set, the initial position information pos 1 is filtered out; if t 2 -t 1 is greater than or equal to the first preset threshold, the initial position information pos 1 is retained, and the retained initial position information pos 1 is used as a valid state variable.
  • the first preset threshold to the third preset threshold may be changed with the determined effective state variable.
  • the state variable threshold may be the time difference between the state variable currently being screened and the sampling time point of the valid state variable adjacent to the sampling time of the state variable currently being screened.
  • All the state variables to be filtered are processed in reverse chronological order. Specifically, for example, the current state variable to be filtered is the state variable to be filtered 5, and the last valid state variable is the state variable to be filtered 6, then the fifth preset threshold Equal to t 6 -t 5 , the time difference t 6 -t 5 between the state variable 5 to be screened and the state variable 6 to be screened is equal to the fifth preset threshold t 6 -t 5 , therefore, the state variable 5 to be screened is a valid state variable.
  • the last valid state variable is the state variable 5 to be filtered
  • the fourth preset threshold is equal to t 5 -t 4
  • t 4 -t 3 if the time difference between the state variable 4 to be filtered and the state variable 3 to be filtered is t 4 -t 3 is greater than t 5 -t 4 , then the state variable 4 to be filtered is a valid state variable, otherwise, the state variable 4 to be filtered is an invalid state variable and will be eliminated.
  • the third preset threshold is equal to t 4 -t 3 , if The state variable 4 to be screened is an invalid state variable. At this time, for the state variable 3 to be screened, the next valid state variable is the state variable 5 to be screened. Then, the third preset threshold is equal to t 5 -t 3 .
  • the third preset threshold varies with whether the state variable 4 to be screened is a valid state variable, in other words, the third preset threshold varies with the determined valid state variable.
  • the second preset threshold value and the first preset threshold value change with the determined effective state variable, which will not be repeated here.
  • the effective state variable is the effective initial position information.
  • one or more effective initial position information can be optimized.
  • one or more effective initial position information can be non-linearly optimized to minimize the target deviation, where the target deviation is related to the detection frame information and/or the effective ranging result, and each effective initial position information is non-linearly optimized Then there is the corresponding optimized location information.
  • the target deviation may include a first deviation and/or a second deviation; the first deviation includes information about the effective initial position and the target detection frame information used to calculate the effective initial position information and the effective ranging result.
  • the second deviation includes the deviation between the smoothness of the adjacent effective initial position information and the prior value.
  • the target deviation may be the following algorithm (3): Minimizing the target deviation can be Among them, ⁇ i (x) represents the first deviation, ⁇ j (x j , x j+1 , x j+2 ) represents the second deviation, and x is an optimized variable corresponding to a valid initial position information.
  • the first deviation ⁇ i (x) may be characterized by the target detection frame information used to calculate the effective initial position information and the probability density function of the effective ranging result.
  • the probability density function may be determined according to the probability distribution corresponding to the target detection frame information.
  • the prior value may be a preset fixed value.
  • one or more effective initial position information is nonlinearly optimized to minimize the target deviation.
  • the effective initial position information may be continuously changed, and the non-linear optimization method is used to iteratively solve the target deviation to obtain the target The optimized location information of the object.
  • the effective initial position information is non-linearly optimized and the corresponding optimized position information can minimize the target deviation.
  • optimizing one or more effective initial location information may further include: determining whether the corresponding optimized location information after nonlinear optimization of each effective initial location information is abnormal; and filtering the abnormal optimized location information remove.
  • the remaining optimized position information can be non-linearly optimized again to minimize the target deviation.
  • each remaining optimized position information has a corresponding final optimization after non-linear optimization. location information.
  • the final optimized position information corresponding to the non-linear optimization of each remaining optimized position information can be calculated to obtain the optimized speed of the target object. For example, first, a number of the latest final optimized positions are obtained, and then a difference algorithm (for example, position 2 minus position 1, and then divided by the time difference) is used to calculate the average target speed corresponding to these latest final optimized positions. Furthermore, it is possible to perform low-pass filtering on the average target speed to obtain an optimized speed that is smooth and jump-free.
  • a difference algorithm for example, position 2 minus position 1, and then divided by the time difference
  • the first deviation may include a first sub-deviation and/or a second sub-deviation.
  • the first sub-deviation is the observation deviation between the effective initial position information and the target detection frame information used for calculating the effective initial position information.
  • the target detection frame information used to calculate the effective initial position information may include, for example, the orientation, the angle of view, the position of the imaging device, and the effective ranging result.
  • the second sub-deviation is the observation deviation between the effective initial position information and the effective ranging result used to calculate the effective initial position information.
  • determining whether the optimized position information corresponding to each effective initial position information after nonlinear optimization is abnormal includes: if it is determined that the detection frame information corresponding to the first sub-deviation is abnormal, and/or determining the second sub-deviation If the corresponding effective ranging result is abnormal, the abnormal detection frame information and/or the optimized position corresponding to the abnormal effective ranging result is determined as the abnormal optimized position.
  • the method of determining the abnormality of the detection frame information corresponding to the first sub-deviation may be as follows: calculating the effective initial position information and performing nonlinear optimization on the corresponding optimized position information and the effective initial position information used to obtain the effective initial position information. Observe the observed deviation between the target detection frame information to obtain the first deviation value; compare the first deviation value with the first deviation threshold value to determine whether the first deviation value is greater than or equal to the first deviation threshold value, if the first deviation value is greater than or If it is equal to the first deviation threshold, it is determined that the detection frame information corresponding to the first sub-deviation is abnormal.
  • the method for determining the abnormality of the effective ranging result corresponding to the second sub-deviation may be as follows: calculating the effective initial position information and performing nonlinear optimization on the corresponding optimized position information and calculating the effective initial position information. Observe the deviation between the effective ranging results to obtain the second deviation value; compare the second deviation value with the second deviation threshold value to determine whether the second deviation value is greater than or equal to the second deviation threshold value, if the second deviation value is greater than or If it is equal to the second deviation threshold, it is determined that the effective ranging result corresponding to the second sub-deviation is abnormal.
  • the optimized position information related to the abnormal detection frame information and the abnormal effective ranging result can be filtered out, and then the remaining optimized position information can be non-linearly optimized again to minimize the target deviation.
  • each remaining optimized position information has corresponding final optimized position information after nonlinear optimization, and the final optimized position information obtained after nonlinear optimization is used as the historical state information of the target object, according to the target
  • the historical state information of the object can predict the state information of the target object, such as the position, speed, and orientation of the target object.
  • the state information of the target object can be predicted based on the historical state information of the target object, even if the target object is not located in the center of the screen, the distance to the target cannot be obtained by the distance measuring device, the position of the target object cannot be determined, or when the image of the imaging device is There are obstacles occluding the target object, and the position of the target object cannot be determined after the visual tracking is lost.
  • the continuity of the motion of the target object and the historical state information can also be used to calculate the corresponding detection frame lacking effective ranging results. The distance of the target object improves the availability of the status information of the target object.
  • the first deviation threshold and the second deviation threshold may be preset based on experience.
  • abnormal detection frame information and/or abnormal effective ranging results can also be filtered out.
  • abnormal target detection frames can be identified and filtered, and abnormal observations can prevent the state estimation of the target object from failing, and the reliability of the state estimation can be improved.
  • abnormal ranging results can be identified and filtered, and abnormal observations can prevent the state estimation of the target object from failing, and the reliability of the state estimation can be improved.
  • the historical motion trajectory of the target object can be optimized, and then the motion trajectory of the target object can be predicted based on the optimized motion trajectory.
  • the trajectory of the target object can be estimated on the basis of historical motion trajectory estimation data.
  • the invalid detection frame may include a detection frame that does not contain the target object obtained after recognizing the current image.
  • the position information of the target object when it is lost can be determined; then according to the position information when the target object is lost and the smoothed motion trajectory of the target object , Predict the status information of the target object.
  • predicting the state information of the target object may be based on the position information when the target object is lost and the smoothed motion trajectory about the target object to generate a probability distribution about the predicted position of the target object.
  • FIG. 13 schematically shows a schematic diagram of predicting the position information of a target object according to an embodiment of the present disclosure.
  • the lost position of the target object at time t 0 is shown in Figure 13.
  • the spatial proportion of the probability distribution of the predicted position of the target object increases as the loss time of the target object increases.
  • the spatial proportion of the probability distribution of the predicted position of the target object at t 1 is smaller than the spatial proportion of the probability distribution of the predicted position at t 2
  • the probability distribution of the predicted position of the target object at t 2 the proportion is less than the predicted spatial position of the time t 3 of the spatial probability distribution proportion.
  • the predicted position of the ellipse center representing the target object keeps deviating from the lost position of the target object, and the prediction error range represented by the area of the ellipse continues to increase.
  • the position and position error of the target object at any point in time after the loss can be predicted based on the historical trajectory information of the target object, which improves the continuity of the state estimation of the target object.
  • the variation parameter of the spatial proportion of the probability distribution of the predicted position is related to the type of the target object, and different variation parameters may be adopted for different types of target objects.
  • the change parameter includes the growth rate of the spatial proportion of the probability distribution of the predicted position.
  • the first growth rate of the spatial proportion of the probability distribution of the predicted position is the same in different directions.
  • the creature can be a human, a dog, a horse, and so on.
  • the second increase speed of the spatial proportion of the probability distribution of the predicted position increases along the movement direction of the mobile device.
  • mobile devices can be cars, trains, boats, and so on.
  • the first growth rate is less than the second growth rate.
  • the target object is a human
  • the growth rate of the prediction error is relatively slow, and the growth rate in each direction can be equal.
  • the target object is a car or a boat
  • the growth rate of the prediction error is faster, and the growth direction of the prediction error is mainly concentrated in the direction of the target movement.
  • FIG. 14 schematically shows a flowchart of a method for determining status information of a target object according to another embodiment of the present disclosure.
  • the method for determining the status information of the target object provided in this embodiment may refer to the description in part or all of the method for determining the status information of the target object provided in the previous embodiment. Specifically, for the same or similar technical solutions, reference may be made to the description of the above-mentioned Figures 4 to 13, which will not be repeated here.
  • the method for determining the status information of the target object includes operations S1401 to S1405.
  • each of the multiple frames of images is recognized to obtain multiple detection frame information about the target object.
  • one or more target detection frame information satisfying a preset condition among the plurality of detection frame information is determined.
  • the multiple detection frame information can be screened, and one or more target detection frames satisfying preset conditions can be determined from the multiple detection frame information. information.
  • the preset condition may be a condition for judging whether the detection frame information is an abnormal detection frame.
  • the preset condition can be a condition for judging whether the sampling time of the detection frame information is abnormal, or the preset condition can also be a condition for judging whether the physical estimated size of the target object calculated based on the detection frame information is abnormal. and many more.
  • state information of the target object is determined according to one or more target detection frame information and a plurality of ranging results.
  • the state information of the target object is determined according to the one or more target detection frame information and the multiple ranging results . At least partially solves the technical problem that the state information of the target object is contaminated by the wrong data, which causes the state estimation of the target object to fail, and improves the reliability of the state information estimation.
  • the detection frame can be filtered, it can be solved that when a screen position that does not belong to the current target object (for example, an object with a high similarity to the target object) is mistakenly detected as the position of the current target object, it cannot be filtered.
  • the wrong target detection frame causes the technical problem of deviation in the estimation of the target object's position.
  • determining the state information of the target object according to one or more target detection frame information and multiple ranging results includes: determining the ranging result corresponding to each target detection frame information; and detecting according to each target The frame information and the ranging result corresponding to each target detection frame information determine the status information of the target object.
  • the state information of the target object includes position information; wherein, determining one or more target detection frame information that meets a preset condition among the plurality of detection frame information includes: sampling according to each detection frame information At the time point, multiple detection frame information is screened to obtain one or more target detection frame information.
  • filtering multiple detection frame information according to the sampling time point corresponding to each detection frame information includes: sequentially filtering each detection frame information in a time sequence.
  • each detection frame information is sequentially filtered in chronological order, including: calculating the detection frame information currently being screened and the next detection frame adjacent to the sampling time of the detection frame information currently being screened The time difference between the sampling time points of the information; compare the time difference with the state variable threshold; if the time difference is less than the state variable threshold, filter out the detection box information currently being screened; and if the time difference is greater than or equal to the state variable threshold, keep the current positive screening
  • the detection frame information of, where the retained detection frame information currently being screened is the target detection frame information.
  • the sampling timing of these detection frames are t m ... t n-1, t n.
  • the time difference between it and the next state variable to be screened is calculated, and from all state variables to be screened, the state variable whose time difference with the next state variable to be screened is less than the state variable threshold is eliminated.
  • the next state variable to be screened is the next detection frame information adjacent to the sampling time of the detection frame information currently being screened.
  • the state variable threshold may be a fixed value set in advance, for example, set in advance based on experience.
  • the state variable threshold may also change with the determined target detection frame information.
  • the state variable threshold is the time difference between the detection frame information currently being screened and the sampling time point of the target detection frame information adjacent to the sampling time of the detection frame information currently being screened.
  • the state variable threshold changes with the determined target detection frame information can refer to the description of FIG. 12 above, which will not be repeated here.
  • determining multiple target detection frame information that meets a preset condition among the multiple detection frame information includes: determining whether the target object moves that meets the preset condition according to the multiple detection frame information; when the target object appears In the case of movement that satisfies the preset condition, the detection frame information corresponding to the movement of the target object that satisfies the preset condition is determined as the target detection frame information.
  • determining whether a target object has moved that meets a preset condition according to multiple detection frame information includes: for any adjacent first detection frame information and second detection frame information among the multiple detection frame information, Obtain the first state information of the imaging device when the first detection frame information is collected, and the second state information of the imaging device when the second detection frame information is collected; according to the ranging result and the first state information corresponding to the first detection frame information , Determine the first probability distribution of the initial position information of the target object corresponding to the first detection frame information; determine the information corresponding to the second detection frame information according to the ranging result and the second state information corresponding to the second detection frame information A second probability distribution of the initial position information of the target object; and according to the first probability distribution and the second probability distribution, it is determined whether the target object moves that meets a preset condition.
  • determining whether the target object has moved that meets a preset condition includes: determining the spatial position with the highest probability density according to the first probability distribution and the second probability distribution ; Calculate the first distance between the spatial position with the highest probability density and the center position of the first probability distribution of the first probability distribution; calculate the second distance between the spatial position with the highest probability density and the center position of the second probability distribution of the second probability distribution; according to The first distance and the second distance determine the probability distance between the first detection frame information and the second detection frame information; and if the probability distance is greater than or equal to a preset threshold, it is determined that the target object moves that meets the preset condition.
  • determining the state information of the target object according to one or more target detection frame information and multiple ranging results includes: optimizing the effective initial position information corresponding to the one or more target detection frame information to Smooth the trajectory of the target object.
  • optimizing the effective initial position information corresponding to one or more target detection frame information includes: performing nonlinear optimization on the effective initial position information corresponding to one or more target detection frame information to minimize The target deviation, where the target deviation is related to the detection frame information and/or the ranging result, and each effective initial position information has corresponding optimized position information after nonlinear optimization.
  • the target deviation includes a first deviation and/or a second deviation; the first deviation includes the difference between the effective initial position information and the target detection frame information used to obtain the effective initial position information and the ranging result.
  • the first deviation is characterized by the probability density function of the target detection frame information and the ranging result used to calculate the effective initial position information.
  • optimizing the effective initial position information corresponding to one or more target detection frame information further includes: determining whether the optimized position information corresponding to each effective initial position information after nonlinear optimization is abnormal; and Abnormal optimized location information is filtered out.
  • the target deviation includes a first deviation
  • the first deviation includes a first sub-deviation and/or a second sub-deviation
  • the second sub-deviation is the observation deviation between the effective initial position information and the ranging result used to calculate the effective initial position information.
  • determining whether the optimized position information corresponding to each effective initial position information after nonlinear optimization is abnormal includes: if it is determined that the detection frame information corresponding to the first sub-deviation is abnormal, and/or determining the ranging result corresponding to the second sub-deviation If it is abnormal, the optimized position corresponding to the abnormal detection frame information and/or the abnormal ranging result is determined as the abnormal optimized position.
  • optimizing the effective initial position information corresponding to one or more target detection frame information further includes: after filtering out abnormal optimized position information, performing nonlinear optimization on the remaining optimized position information , In order to minimize the target deviation, where each remaining optimized position information has corresponding final optimized position information after nonlinear optimization.
  • the position information when the target object is lost is determined; and according to the position information when the target object is lost and the smoothed motion trajectory about the target object, Predict the status information of the target object.
  • the state information of the target object is predicted based on the position information when the target object is lost and the smoothed motion trajectory about the target object, including: according to the position information when the target object is lost and the smoothed information about the target object
  • the motion trajectory of the target object generates a probability distribution about the predicted position of the target object.
  • the spatial proportion of the probability distribution of the predicted position increases as the loss time of the target object increases.
  • the variation parameter of the spatial proportion of the probability distribution of the predicted position is related to the type of the target object.
  • the change parameter includes the growth rate of the space proportion of the probability distribution of the predicted position
  • the change parameter of the space proportion of the predicted position probability distribution is related to the type of the target object, including: In the case of the predicted position, the first growth rate of the space proportion of the probability distribution of the predicted position is the same in different directions; when the target object type is a mobile device, the second growth rate of the space proportion of the predicted position’s probability distribution Increase in the direction of movement of the mobile device.
  • the first growth rate is less than the second growth rate.
  • determining the status information of the target object according to one or more target detection frame information and multiple ranging results includes: filtering out valid from the multiple ranging results according to the one or more target detection frame information Ranging results; and determining the status information of the target object based on one or more target detection frame information and effective ranging results.
  • filtering effective ranging results from multiple ranging results according to one or more target detection frame information includes: determining one or more ranging results corresponding to each target detection frame information; and Screen one or more ranging results corresponding to each target detection frame information.
  • determining one or more ranging results corresponding to each target detection frame information includes: according to the sampling time point of each target detection frame information and the value of each ranging result in the multiple ranging results The sampling time point determines one or more ranging results corresponding to each target detection frame information.
  • the ranging result includes a laser ranging result
  • screening one or more ranging results corresponding to each target detection frame information includes: determining each ranging result in the one or more ranging results The laser spot corresponding to the result; according to the target detection frame information corresponding to each ranging result and the laser spot corresponding to each ranging result, the validity of each ranging result is determined; and the validity of each ranging result is checked One or more ranging results corresponding to each target detection frame information are screened.
  • determining the validity of each ranging result includes: determining the corresponding The target detection frame information and the area coincidence rate of the laser spot corresponding to each distance measurement result; and the validity of each distance measurement result is determined according to the area coincidence ratio.
  • each ranging result can be determined with reference to the above description of FIG. 5, here No longer.
  • determining the validity of each ranging result according to the area coincidence rate includes: comparing the area coincidence rate with a preset ratio threshold; and comparing the distance measurement result whose area coincidence rate is greater than or equal to the preset ratio threshold It is determined as a valid ranging result; and the ranging result whose area coincidence rate is less than the preset ratio threshold is determined as an invalid ranging result.
  • each target detection frame information corresponding to each ranging result and the laser spot corresponding to each ranging result are determined.
  • the validity of each ranging result includes: determining each of the multiple ranging results between the sampling times of the two target detection frame information with adjacent sampling times according to the information of the two target detection frames adjacent to the sampling time.
  • determining the state information of the target object according to multiple target detection frame information and effective ranging results includes: determining the effective ranging result corresponding to each target detection frame information; and determining the effective ranging result corresponding to each target detection frame information; and according to each target detection frame information The effective ranging result corresponding to each target detection frame information determines the status information of the target object.
  • determining the effective ranging result corresponding to each target detection frame information includes: according to the sampling time point of each target detection frame information and the sampling time point of each effective ranging result, each valid The ranging result is related to the target detection frame information closest to the sampling time point of the effective ranging result.
  • each target detection frame information corresponds to multiple effective ranging results
  • the target object's position is determined according to each target detection frame information and the effective ranging result corresponding to each target detection frame information.
  • the status information includes: calculating the weighted average of multiple effective ranging results corresponding to each target detection frame information to obtain the target ranging result corresponding to each target detection frame information; and according to each target detection frame information and each The target ranging result corresponding to the target detection frame information determines the status information of the target object.
  • determining the status information of the target object according to one or more target detection frame information and multiple ranging results includes: determining the ranging result corresponding to each target detection frame information; determining each target detection frame The physical estimated size of the target object corresponding to the information; the ranging result corresponding to each target detection frame information is screened according to the physical estimated size of the target object corresponding to each target detection frame information; and according to one or more targets The detection frame information and the filtered ranging results determine the status information of the target object.
  • screening the ranging results corresponding to each target detection frame information includes: The physical estimated size of the target object is compared with the preset reasonable range; and when the physical estimated size of the target object corresponding to the target detection frame information does not match the preset reasonable range, the measurement corresponding to the target detection frame information is filtered out. Distance results.
  • determining the physical estimated size of the target object corresponding to each target detection frame information includes: according to each target detection frame information and the field angle of the imaging device when each target detection frame information is collected, Determine the field angle corresponding to each target detection frame information; and determine the physical estimation corresponding to each target detection frame information according to the field angle corresponding to each target detection frame information and the distance measurement result corresponding to each target detection frame information size.
  • an apparatus for determining status information of a target object including: a processor; a readable storage medium for storing one or more programs, wherein, when one or more programs are executed by the processor When executed, it causes the processor to perform the following operations:
  • the processor screens out effective ranging results from multiple ranging results according to multiple detection frame information, including: determining one or more ranging results corresponding to each detection frame information; and One or more ranging results corresponding to each detection frame information are filtered.
  • the processor determines one or more ranging results corresponding to each detection frame information, including: according to the sampling time point of each detection frame information and the value of each ranging result in the multiple ranging results The sampling time point determines one or more ranging results corresponding to each detection frame information.
  • the ranging result includes a laser ranging result
  • the processor screens one or more ranging results corresponding to each detection frame information, including: determining each of the one or more ranging results The laser spot corresponding to the ranging result; the validity of each ranging result is determined according to the detection frame information corresponding to each ranging result and the laser spot corresponding to each ranging result; and the validity of each ranging result is checked according to the validity of each ranging result
  • One or more ranging results corresponding to each detection frame information are screened.
  • the processor determines the validity of each ranging result according to the detection frame information corresponding to each ranging result and the laser spot corresponding to each ranging result, including: determining that each ranging result corresponds to The detection frame information and the area coincidence rate of the laser spot corresponding to each distance measurement result; and the validity of each distance measurement result is determined according to the area coincidence ratio.
  • the processor determines the validity of each ranging result according to the area coincidence rate, including: comparing the area coincidence rate with a preset ratio threshold; The distance result is determined as a valid distance measurement result; and the distance measurement result whose area coincidence rate is less than the preset ratio threshold is determined as an invalid distance measurement result.
  • the processor determines each detection frame information corresponding to each ranging result and the laser spot corresponding to each ranging result.
  • the validity of each ranging result includes: determining each of the multiple ranging results between the sampling times of the two adjacent detection frames at the sampling time according to the information of the two adjacent detection frames at the sampling time Corresponding interpolation detection frame information to obtain the detection frame information corresponding to each ranging result; and determine each ranging result according to the laser spot corresponding to each ranging result and the detection frame information corresponding to each ranging result The validity of the results.
  • the processor determines the state information of the target object according to multiple detection frame information and effective ranging results, including: determining the effective ranging result corresponding to each detection frame information; and according to each detection frame information and The effective ranging result corresponding to each detection frame information determines the status information of the target object.
  • the processor determining the effective ranging result corresponding to each detection frame information includes: according to the sampling time point of each detection frame information and the sampling time point of each effective ranging result, the effective The ranging result is related to the detection frame information closest to the sampling time point of the effective ranging result.
  • the processor determines the state of the target object according to each detection frame information and the effective ranging result corresponding to each detection frame information Information, including: calculating the weighted average of multiple effective ranging results corresponding to each detection frame information to obtain the target ranging result corresponding to each detection frame information; and corresponding to each detection frame information and each detection frame information
  • the target ranging result is determined to determine the status information of the target object.
  • the processor further performs the following operations: determining the effective ranging result corresponding to each detection frame information; determining the physical estimated size of the target object corresponding to each detection frame information; and according to each detection frame information Corresponding to the physical estimated size of the target object, the effective ranging result corresponding to each detection frame information is screened.
  • the processor screens the effective ranging result corresponding to each detection frame information according to the physical estimated size of the target object corresponding to each detection frame information, including: Compare the physical estimated size of the target object with the preset reasonable range; and filter out the effective ranging corresponding to the detection frame information when the physical estimated size of the target object corresponding to the detection frame information does not match the preset reasonable range result.
  • the processor determining the physical estimated size of the target object corresponding to each detection frame information includes: determining, according to each detection frame information and the field angle of the imaging device when each detection frame information is collected The field angle corresponding to each detection frame information; and the physical estimated size corresponding to each detection frame information is determined according to the field angle corresponding to each detection frame information and the effective ranging result corresponding to each detection frame information.
  • the processor further performs the following operations: determining the object type of the target object, where each object type has a corresponding preset reasonable range; and calculating the physical estimated size of the target object corresponding to each detection frame information
  • the comparison with the preset reasonable range includes: determining the preset reasonable range of the target according to the object type of the target object; and comparing the physical estimated size of the target object corresponding to each detection frame information with the preset reasonable range of the target.
  • the processor further performs the following operations: after filtering out the effective ranging result corresponding to the detection frame information, it corresponds to the detection frame information adjacent to the sampling time of the detection frame information from which the effective ranging result is filtered out
  • the effective ranging result of is determined as the effective ranging result corresponding to the detection frame information filtered out of the effective ranging result.
  • the processor determines the state information of the target object according to the multiple detection frame information and the effective ranging result, including: determining that each target detection frame information in the one or more target detection frame information corresponds to the valid The ranging result, one or more target detection frame information is the detection frame information that meets the preset conditions among the multiple detection frame information; and the effective ranging result corresponding to each target detection frame information and each target detection frame information, Determine the status information of the target object.
  • the state information includes position information; wherein, the processor determines the state information of the target object according to each target detection frame information and the effective ranging result corresponding to each target detection frame information, including: Target detection frame information and the effective ranging result corresponding to each target detection frame information, determine the initial position information about the target object corresponding to each target detection frame information, and obtain multiple initial position information; and according to each target detection frame information At the corresponding sampling time point, multiple initial position information is screened to obtain one or more effective initial position information; the state information of the target object is determined according to the one or more effective initial position information.
  • the processor screens a plurality of initial position information according to the sampling time point corresponding to each target detection frame information, including: sequentially, in chronological order, corresponding to the target object corresponding to each target detection frame information
  • the initial location information is filtered.
  • the processor sequentially screens the initial position information about the target object corresponding to each target detection frame information in chronological order, including: calculating the initial position information currently being screened and the initial location information that is currently being screened The time difference between the sampling time of the position information and the sampling time point of the next initial position information; compare the time difference with the state variable threshold; if the time difference is less than the state variable threshold, filter out the initial position information currently being filtered; and if The time difference is greater than or equal to the state variable threshold, and the initial position information currently being screened is retained, where the retained initial position information currently being screened is the effective initial position information.
  • the state variable threshold changes with the determined effective initial position information.
  • the state variable threshold is the time difference between the initial location information currently being screened and the sampling time point of the effective initial location information adjacent to the sampling time of the initial location information currently being screened.
  • the processor further performs the following operations: determining whether the target object has a movement that meets a preset condition according to multiple detection frame information; and when the target object has a movement that meets the preset condition, the target The detection frame information corresponding to the movement when the object meets the preset condition is determined as the target detection frame information.
  • the processor determines whether the target object has moved that satisfies a preset condition according to multiple detection frame information, including: for any adjacent first detection frame information and second detection frame in the multiple detection frame information Information, obtain the first state information of the imaging device when the first detection frame information is collected, and the second state information of the imaging device when the second detection frame information is collected; State information, determining the first probability distribution of the initial position information of the target object corresponding to the first detection frame information; determining the first probability distribution corresponding to the second detection frame information according to the effective ranging result corresponding to the second detection frame information and the second state information
  • the information corresponds to a second probability distribution of the initial position information of the target object; and according to the first probability distribution and the second probability distribution, it is determined whether the target object moves that meets the preset condition.
  • the processor determines whether the target object has moved that meets a preset condition according to the first probability distribution and the second probability distribution, including: determining the one with the highest probability density according to the first probability distribution and the second probability distribution Spatial position; calculate the first distance between the spatial position with the highest probability density and the center of the first probability distribution of the first probability distribution; calculate the second distance between the spatial position with the highest probability density and the center of the second probability distribution of the second probability distribution ; Determine the probability distance between the first detection frame information and the second detection frame information according to the first distance and the second distance; and if the probability distance is greater than or equal to a preset threshold, it is determined that the target object appears to move that meets the preset conditions.
  • the processor determining the state information of the target object according to the one or more effective initial position information includes: optimizing the one or more effective initial position information to smooth the movement trajectory of the target object.
  • the processor optimizing one or more effective initial position information includes: performing non-linear optimization on one or more effective initial position information to minimize target deviation, wherein the target deviation and the detection frame The information is related to the effective ranging result, and each effective initial position information has corresponding optimized position information after nonlinear optimization.
  • the target deviation includes a first deviation and/or a second deviation; the first deviation includes information about the effective initial position and the target detection frame information used to calculate the effective initial position information and the effective ranging result.
  • the observation deviation of; the second deviation includes the deviation of the smoothness between adjacent valid initial position information and the prior value.
  • the first deviation is characterized by the probability density function of the target detection frame information used to calculate the effective initial position information and the effective ranging result.
  • the processor optimizes one or more effective initial position information, and further includes: determining whether the optimized position information corresponding to each effective initial position information after nonlinear optimization is abnormal; and determining the abnormal optimized position Information is filtered out.
  • the target deviation includes a first deviation
  • the first deviation includes a first sub-deviation and/or a second sub-deviation
  • the second sub-deviation is the observation deviation between the effective initial position information and the effective ranging result used to calculate the effective initial position information
  • the processor determines each effective initial position Whether the corresponding optimized position information is abnormal after the information is non-linearly optimized, including: if it is determined that the detection frame information corresponding to the first sub-deviation is abnormal, and/or the effective ranging result corresponding to the second sub-deviation is determined to be abnormal, then the abnormal detection
  • the optimized position corresponding to the frame information and/or the abnormal effective ranging result is determined as the abnormal optimized position.
  • the processor optimizes one or more effective initial position information, and further includes: after filtering out abnormal optimized position information, performing nonlinear optimization on the remaining optimized position information to minimize Target deviation, where each remaining optimized position information has corresponding final optimized position information after non-linear optimization.
  • the processor further performs the following operations: when the target object is not recognized in the obtained image, determine the position information when the target object is lost; and according to the position information when the target object is lost and the smoothed Regarding the motion trajectory of the target object, the state information of the target object is predicted.
  • the processor predicts the state information of the target object according to the position information when the target object is lost and the smoothed motion trajectory of the target object, including: according to the position information when the target object is lost and the smoothed Regarding the motion trajectory of the target object, a probability distribution about the predicted position of the target object is generated.
  • the spatial proportion of the probability distribution of the predicted position increases as the loss time of the target object increases.
  • the variation parameter of the spatial proportion of the probability distribution of the predicted position is related to the type of the target object.
  • the change parameter includes the growth rate of the spatial proportion of the probability distribution of the predicted position; in the case where the type of the target object is a creature, the first growth rate of the spatial proportion of the probability distribution of the predicted position is different The directions are the same; in the case where the type of the target object is a mobile device, the second increase speed of the spatial proportion of the probability distribution of the predicted position increases along the movement direction of the mobile device.
  • the first growth rate is less than the second growth rate.
  • another device for determining status information of a target object including: a processor; a readable storage medium for storing one or more programs, wherein when one or more programs are executed by the processor When executed, it causes the processor to perform the following operations:
  • the processor determines the state information of the target object according to one or more target detection frame information and multiple ranging results, including: determining the ranging result corresponding to each target detection frame information; and according to each The target detection frame information and the ranging result corresponding to each target detection frame information determine the state information of the target object.
  • the status information includes position information; wherein, the processor determines one or more target detection frame information that meets a preset condition among the plurality of detection frame information, including: according to the sampling time corresponding to each detection frame information Click to filter multiple detection frame information to obtain one or more target detection frame information.
  • the processor screens multiple detection frame information according to the sampling time point corresponding to each detection frame information, including: sequentially screening each detection frame information in a time sequence.
  • the processor sequentially screens each detection frame information in a chronological order, including: calculating the detection frame information currently being screened and the next detection adjacent to the sampling time of the detection frame information currently being screened The time difference between the sampling time points of the frame information; compare the time difference with the state variable threshold; if the time difference is less than the state variable threshold, filter out the detection frame information currently being screened; and if the time difference is greater than or equal to the state variable threshold, keep the current positive
  • the screened detection frame information wherein the retained detection frame information currently being screened is the target detection frame information.
  • the state variable threshold changes with the determined target detection frame information.
  • the state variable threshold is the time difference between the detection frame information currently being screened and the sampling time point of the target detection frame information adjacent to the sampling time of the detection frame information currently being screened.
  • the processor determining multiple target detection frame information that meets a preset condition among the multiple detection frame information includes: determining whether the target object has moved that meets the preset condition according to the multiple detection frame information; and In the case where the target object moves that meets the preset condition, the detection frame information corresponding to the movement of the target object that meets the preset condition is determined as the target detection frame information.
  • the processor determines whether the target object has moved that satisfies a preset condition according to multiple detection frame information, including: for any adjacent first detection frame information and second detection frame in the multiple detection frame information Information, obtain the first state information of the imaging device when the first detection frame information is collected, and the second state information of the imaging device when the second detection frame information is collected; State information, determine the first probability distribution of the initial position information of the target object corresponding to the first detection frame information; determine the corresponding to the second detection frame information according to the ranging result corresponding to the second detection frame information and the second state information The second probability distribution about the initial position information of the target object; and according to the first probability distribution and the second probability distribution, it is determined whether the target object appears to move that meets the preset condition.
  • the processor determining whether the target object has moved that meets a preset condition according to the first probability distribution and the second probability distribution includes: determining the space with the highest probability density according to the first probability distribution and the second probability distribution Position; calculate the first distance between the space position with the highest probability density and the center position of the first probability distribution of the first probability distribution; calculate the second distance between the space position with the highest probability density and the center position of the second probability distribution of the second probability distribution; The probability distance between the first detection frame information and the second detection frame information is determined according to the first distance and the second distance; and if the probability distance is greater than or equal to a preset threshold, it is determined that the target object has moved that meets the preset condition.
  • the processor determines the state information of the target object according to one or more target detection frame information and multiple ranging results, including: optimizing effective initial position information corresponding to the one or more target detection frame information To smooth the trajectory of the target object.
  • the processor optimizing the effective initial position information corresponding to the one or more target detection frame information includes: performing nonlinear optimization on the effective initial position information corresponding to the one or more target detection frame information to Minimize the target deviation, where the target deviation is related to the detection frame information and/or the ranging result, and each effective initial position information has corresponding optimized position information after nonlinear optimization.
  • the target deviation includes a first deviation and/or a second deviation; the first deviation includes the difference between the effective initial position information and the target detection frame information used to obtain the effective initial position information and the ranging result.
  • the first deviation is characterized by the probability density function of the target detection frame information and the ranging result used to calculate the effective initial position information.
  • the processor optimizing the effective initial position information corresponding to one or more target detection frame information further includes: determining whether the optimized position information corresponding to each effective initial position information after nonlinear optimization is abnormal; And filter out abnormal optimized location information.
  • the target deviation includes a first deviation
  • the first deviation includes a first sub-deviation and/or a second sub-deviation
  • the second sub-deviation is the observation deviation between the effective initial position information and the ranging result used to calculate the effective initial position information
  • the processor determines each effective initial position information Whether the corresponding optimized position information after nonlinear optimization is abnormal includes: if it is determined that the detection frame information corresponding to the first sub-deviation is abnormal, and/or the ranging result corresponding to the second sub-deviation is determined to be abnormal, then the abnormal detection frame information And/or the optimized position corresponding to the abnormal ranging result is determined as the abnormal optimized position.
  • the processor optimizes the effective initial position information corresponding to one or more target detection frame information, and further includes: after filtering out abnormal optimized position information, performing non-processing on the remaining optimized position information. Linear optimization to minimize the target deviation, where each remaining optimized position information has corresponding final optimized position information after nonlinear optimization.
  • the processor further performs the following operations: when the target object is not recognized in the obtained image, determine the position information when the target object is lost; and according to the position information when the target object is lost and the smoothed Regarding the motion trajectory of the target object, the state information of the target object is predicted.
  • the processor predicts the state information of the target object according to the position information when the target object is lost and the smoothed motion trajectory about the target object, including: according to the position information when the target object is lost and smoothed The trajectory of the target object is generated to generate the probability distribution of the predicted position of the target object.
  • the spatial proportion of the probability distribution of the predicted position increases as the loss time of the target object increases.
  • the variation parameter of the spatial proportion of the probability distribution of the predicted position is related to the type of the target object.
  • the change parameter includes the growth rate of the space proportion of the probability distribution of the predicted position
  • the change parameter of the space proportion of the predicted position probability distribution is related to the type of the target object, including: In the case of the predicted position, the first growth rate of the space proportion of the probability distribution of the predicted position is the same in different directions; when the target object type is a mobile device, the second growth rate of the space proportion of the predicted position’s probability distribution Increase in the direction of movement of the mobile device.
  • the first growth rate is less than the second growth rate.
  • the processor determines the state information of the target object according to the one or more target detection frame information and the multiple ranging results, including: filtering from the multiple ranging results according to the one or more target detection frame information A valid ranging result is obtained; and the status information of the target object is determined according to one or more target detection frame information and the valid ranging result.
  • the processor screens out effective ranging results from multiple ranging results according to one or more target detection frame information, including: determining one or more ranging results corresponding to each target detection frame information ; And screening one or more ranging results corresponding to each target detection frame information.
  • the processor determining one or more ranging results corresponding to each target detection frame information includes: according to the sampling time point of each target detection frame information and each ranging result in the multiple ranging results The sampling time point of the result determines one or more ranging results corresponding to each target detection frame information.
  • the ranging result includes a laser ranging result
  • the processor screens one or more ranging results corresponding to each target detection frame information, including: determining each of the one or more ranging results The laser spot corresponding to the ranging result; determine the validity of each ranging result according to the target detection frame information corresponding to each ranging result and the laser spot corresponding to each ranging result; and according to the validity of each ranging result Screen one or more ranging results corresponding to each target detection frame information.
  • the processor determines the validity of each ranging result according to the target detection frame information corresponding to each ranging result and the laser spot corresponding to each ranging result, including: determining each ranging result The corresponding target detection frame information and the area coincidence rate of the laser spot corresponding to each ranging result; and the validity of each ranging result is determined according to the area coincidence rate.
  • the processor determining the validity of each ranging result according to the area coincidence rate includes: comparing the area coincidence rate with a preset ratio threshold; The result is determined as a valid ranging result; and the ranging result whose area coincidence rate is less than the preset ratio threshold is determined as an invalid ranging result.
  • Determining the validity of each ranging result includes: determining each of the multiple ranging results between the sampling times of the two target detection frame information adjacent to the sampling time according to the information of the two target detection frames adjacent to the sampling time The interpolated target detection frame information corresponding to the ranging results respectively, to obtain the target detection frame information corresponding to each ranging result; and according to the laser spot corresponding to each ranging result and the target detection frame information corresponding to each ranging result, Determine the validity of each ranging result.
  • the processor determining the state information of the target object according to the multiple target detection frame information and the effective ranging result includes: determining the effective ranging result corresponding to each target detection frame information; and determining the effective ranging result according to each target detection frame The information and the effective ranging result corresponding to each target detection frame information determine the status information of the target object.
  • the processor determines the effective ranging result corresponding to each target detection frame information, including: according to the sampling time point of each target detection frame information and the sampling time point of each effective ranging result, each Each effective ranging result is associated with the target detection frame information closest to the sampling time point of the effective ranging result.
  • the processor determines the target according to each target detection frame information and the effective ranging result corresponding to each target detection frame information.
  • the state information of the object includes: calculating the weighted average of multiple effective ranging results corresponding to each target detection frame information to obtain the target ranging result corresponding to each target detection frame information; and according to each target detection frame information and The target ranging result corresponding to each target detection frame information determines the status information of the target object.
  • the processor determines the state information of the target object according to one or more target detection frame information and multiple ranging results, including: determining the ranging result corresponding to each target detection frame information; determining each target The physical estimated size of the target object corresponding to the detection frame information; the ranging result corresponding to each target detection frame information is screened according to the physical estimated size of the target object corresponding to each target detection frame information; and according to one or more The target detection frame information and the filtered ranging results determine the status information of the target object.
  • the processor screens the distance measurement results corresponding to each target detection frame information according to the physical estimated size of the target object corresponding to each target detection frame information, including: The corresponding physical estimated size of the target object is compared with the preset reasonable range; and when the physical estimated size of the target object corresponding to the target detection frame information does not match the preset reasonable range, the target detection frame information is filtered out. ⁇ ranging results.
  • the processor determines the physical estimated size of the target object corresponding to each target detection frame information, including: according to each target detection frame information and the field of view of the imaging device when each target detection frame information is collected Angle, determine the field of view corresponding to each target detection frame information; and determine the field of view corresponding to each target detection frame information and the distance measurement result corresponding to each target detection frame information. Physically estimated size.
  • the processor further performs the following operations: determining the object type of the target object, wherein each object type has a corresponding preset reasonable range; wherein, the information about the target object corresponding to each target detection frame information
  • the comparison of the physical estimated size with the preset reasonable range includes: determining the target preset reasonable range according to the object type of the target object; and comparing the physical estimated size of the target object corresponding to each target detection frame information with the target preset reasonable range .
  • the processor may include, for example, a general-purpose microprocessor, an instruction set processor and/or a related chipset and/or a special-purpose microprocessor (for example, an application specific integrated circuit (ASIC)), etc. .
  • the processor may also include on-board memory for caching purposes.
  • the processor may be a single processing unit or multiple processing units for executing different actions of the method flow according to the embodiments of the present disclosure.
  • a system for determining state information of a target object including: an imaging device for obtaining multi-frame images of the target object and the state information determining device mentioned in the above embodiment.
  • a movable platform including: a movable body and a system for determining status information of a target object.
  • the movable platform can have different components.
  • the movable platform when the movable platform is an unmanned aerial vehicle, it may also include a rotor and a rotating mechanism, etc. The description of components such as the rotor and the rotating mechanism is omitted in this disclosure.
  • the movable platform when the movable platform is an unmanned vehicle, it may also include an engine and wheels, etc. The description of components such as the engine and wheels is omitted in this disclosure.
  • a readable storage medium having executable instructions stored thereon, and when the instructions are executed by a processor, the processor executes the method for determining state information mentioned in the foregoing embodiment.
  • the readable storage medium may be included in the device/device/system described in the above embodiments; or it may exist alone without being assembled into the device/device/system.
  • the aforementioned readable storage medium carries one or more programs, and when the aforementioned one or more programs are executed, the method according to the embodiments of the present disclosure is implemented.
  • the readable storage medium may be a nonvolatile readable storage medium, for example, it may include but not limited to: portable computer disk, hard disk, random access memory (RAM), read only memory (ROM), Erasable programmable read-only memory (EPROM or flash memory), portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination of the above.
  • a readable storage medium may be any tangible medium that contains or stores a program, and the program may be used by or in combination with an instruction execution system, apparatus, or device.
  • Fig. 15 schematically shows a block diagram of a system for determining status information of a target object according to an embodiment of the present disclosure.
  • system 1500 for determining the status information of the target object may also have some or all of the hardware modules shown in FIG. 15.
  • a system 1500 for determining status information of a target object includes a processor 1501, which can be loaded into a random access memory according to a program stored in a read-only memory (ROM) 1502 or from a storage part 1508 (RAM)
  • the program in 1503 executes various appropriate actions and processing.
  • the processor 1501 may include, for example, a general-purpose microprocessor (for example, a CPU), an instruction set processor and/or a related chipset and/or a special purpose microprocessor (for example, an application specific integrated circuit (ASIC)), and so on.
  • the processor 1501 may also include on-board memory for caching purposes.
  • the processor 1501 may include a single processing unit or multiple processing units for performing different actions of a method flow according to an embodiment of the present disclosure.
  • various programs and data required for the operation of the system 1500 are determined by the state information of the target object.
  • the processor 1501, the ROM 1502, and the RAM 1503 are connected to each other through a bus 1504.
  • the processor 1501 executes various operations of the method flow according to the embodiments of the present disclosure by executing programs in the ROM 1502 and/or RAM 1503. It should be noted that the program can also be stored in one or more memories other than ROM 1502 and RAM 1503.
  • the processor 1501 may also execute various operations of the method flow according to the embodiments of the present disclosure by executing programs stored in the one or more memories.
  • the state information determining system 1500 of the target object may further include an input/output (I/O) interface 1505, and the input/output (I/O) interface 1505 is also connected to the bus 1504.
  • the state information determining system 1500 of the target object may also include one or more of the following components connected to the I/O interface 1505: an input part 1506 including a keyboard, a mouse, etc.; including a cathode ray tube (CRT), a liquid crystal display, etc. (LCD), etc. and output section 1507 of speakers, etc.; storage section 1508 including hard disks, etc.; and communication section 1509 including network interface cards such as LAN cards, modems, and the like.
  • the communication section 1509 performs communication processing via a network such as the Internet.
  • the driver 1510 is also connected to the I/O interface 1505 as needed.
  • a removable medium 1511 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 1510 as required, so that the computer program read therefrom is installed into the storage portion 1508 as required.
  • the method flow according to the embodiment of the present disclosure may be implemented as a computer software program.
  • an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable storage medium, and the computer program contains program code for executing the method shown in the flowchart.
  • the computer program may be downloaded and installed from the network through the communication section 1509, and/or installed from the removable medium 1511.
  • the computer program executes the above-mentioned functions defined in the system of the embodiment of the present disclosure.
  • each block in the flowchart or block diagram may represent a module, program segment, or part of the code, and the above-mentioned module, program segment, or part of the code contains one or more for realizing the specified logic function.
  • Executable instructions may also occur in a different order from the order marked in the drawings. For example, two blocks shown one after another can actually be executed substantially in parallel, and they can sometimes be executed in the reverse order, depending on the functions involved.
  • each block in the block diagram or flowchart, and the combination of blocks in the block diagram or flowchart can be implemented by a dedicated hardware-based system that performs the specified function or operation, or can be implemented by It is realized by a combination of dedicated hardware and computer instructions.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Radar, Positioning & Navigation (AREA)
  • Remote Sensing (AREA)
  • Multimedia (AREA)
  • Theoretical Computer Science (AREA)
  • Aviation & Aerospace Engineering (AREA)
  • Image Analysis (AREA)

Abstract

Disclosed are a state information determination method, apparatus and system for a target object (102), and a movable platform and a storage medium. The state information determination method for a target object (102) comprises: obtaining multiple frames of images regarding a target object (102) by means of an imaging apparatus carried by a movable platform; identifying each frame of image in the multiple frames of images to obtain a plurality of pieces of detection box information regarding the target object (102); obtaining a plurality of ranging results between the movable platform and the target object (102); selecting valid ranging results from among the plurality of ranging results according to the plurality of pieces of detection box information; and determining state information of the target object (102) according to the plurality of pieces of detection box information and the valid ranging results.

Description

状态信息确定方法、装置、***、可移动平台和存储介质Method, device, system, removable platform and storage medium for determining status information 技术领域Technical field
本公开涉及可移动平台技术领域，尤其涉及目标对象的状态信息确定方法、装置、***、可移动平台和存储介质。The present disclosure relates to the technical field of movable platforms, and in particular to methods, devices, systems, movable platforms, and storage media for determining status information of target objects.
背景技术Background technique
可移动平台包括无人机(“UAV”)，有时被称为“无人驾驶飞机”，用户可以远程操作或编程实现自动飞行，UAV包括各种尺寸和配置的无人驾驶飞行器。当然，可移动平台并不限于此，例如，可移动平台还可以包括无人车、无人船等移动设备。Movable platforms include unmanned aerial vehicles ("UAVs"), sometimes referred to as "unmanned aerial vehicles". Users can remotely operate or program to achieve automatic flight. UAVs include unmanned aerial vehicles of various sizes and configurations. Of course, the movable platform is not limited to this. For example, the movable platform may also include unmanned vehicles, unmanned ships and other mobile equipment.
可移动平台可以被用于许多目的，例如被用于各种个人、商业和战术应用。可移动平台可以装备有成像装置，诸如相机、摄像机等。装备有成像装置的可移动平台可以确定目标对象的状态信息，状态信息例如可以包括位置信息等，目标对象例如可以包括人、移动物体、静止物体等。Movable platforms can be used for many purposes, such as various personal, commercial and tactical applications. The movable platform may be equipped with imaging devices, such as cameras, video cameras, etc. The movable platform equipped with the imaging device can determine the status information of the target object. The status information may include position information, for example, and the target object may include people, moving objects, and stationary objects, for example.
公开内容Public content
本公开提供了一种目标对象的状态信息确定方法，包括：通过可移动平台携带的成像装置获得关于目标对象的多帧图像；对多帧图像中的每一帧图像进行识别，得到关于目标对象的多个检测框信息；获得可移动平台与目标对象之间的多个测距结果；根据多个检测框信息从多个测距结果中筛选出有效测距结果；以及根据多个检测框信息和有效测距结果确定目标对象的状态信息。The present disclosure provides a method for determining status information of a target object, including: obtaining multiple frames of images about the target object through an imaging device carried by a movable platform; Multiple detection frame information; obtain multiple ranging results between the movable platform and the target object; filter effective ranging results from multiple ranging results according to multiple detection frame information; and according to multiple detection frame information And the effective ranging result determines the status information of the target object.
本公开还提供了另一种目标对象的状态信息确定方法，包括：通过可移动平台携带的成像装置获得关于目标对象的多帧图像；对多帧图像中的每一帧图像进行识别，得到关于目标对象的多个检测框信息；确定多个检测框信息中满足预设条件的一个或多个目标检测框信息；获得可移动平台与目标对象之间的多个测距结果；以及根据一个或多个目标检测框信息和多个测距结果确定目标对象的状态信息。The present disclosure also provides another method for determining the status information of a target object, including: obtaining multiple frames of images about the target object through an imaging device carried by a movable platform; Multiple detection frame information of the target object; determine one or more target detection frame information satisfying preset conditions among the multiple detection frame information; obtain multiple ranging results between the movable platform and the target object; and according to one or Multiple target detection frame information and multiple ranging results determine the status information of the target object.
本公开还提供了一种目标对象的状态信息确定装置，包括：处理器；可读存储介质，用于存储一个或多个程序，其中，当一个或多个程序被处理器执行时，使得处理器执行以下操作：获得通过可移动平台携带的成像装置获得的关于目标对象的多帧图像；对多帧图像中的每一帧图像进行识别，得到关于目标对象的多个检测框信息；获得可移动平台与目标对象之间的多个测距结果；根据多个检测框信息从多个测距结果中筛选出有效测距结果；以及根据多个检测框信息和有效测距结果确定目标对象的状态信息。The present disclosure also provides a device for determining status information of a target object, including: a processor; a readable storage medium for storing one or more programs, wherein when the one or more programs are executed by the processor, the processing The device performs the following operations: obtains multi-frame images of the target object obtained by the imaging device carried by the movable platform; recognizes each frame of the multi-frame image to obtain multiple detection frame information about the target object; obtains Multiple ranging results between the mobile platform and the target object; filtering effective ranging results from multiple ranging results based on multiple detection frame information; and determining the target object based on multiple detection frame information and effective ranging results status information.
本公开还提供了另一种目标对象的状态信息确定装置，包括：处理器；可读存储介质，用于存储一个或多个程序，其中，当一个或多个程序被处理器执行时，使得处理器执行以下操作：获得通过可移动平台携带的成像装置获得的关于目标对象的多帧图像；对多帧图像中的每一帧图像进行识别，得到关于目标对象的多个检测框信息；确定多个检测框信息中满足预设条件的一个或多个目标检测框信息；获得可移动平台与目标对象之间的多个测距结果；以及根据一个或多个目标检测框信息和多个测距结果确定目标对象的状态信息。The present disclosure also provides another device for determining status information of a target object, including: a processor; a readable storage medium for storing one or more programs, wherein when the one or more programs are executed by the processor, The processor performs the following operations: obtaining multiple frames of images about the target object obtained by the imaging device carried by the movable platform; identifying each frame of the multiple frames of images to obtain multiple detection frame information about the target object; determining One or more target detection frame information that meets the preset conditions among the multiple detection frame information; obtain multiple ranging results between the movable platform and the target object; and according to one or more target detection frame information and multiple measurements The distance result determines the status information of the target object.
本公开还提供了一种目标对象的状态信息确定***，包括：成像装置，用于获得关于目标对象的多帧图像；如上所述的状态信息确定装置。The present disclosure also provides a system for determining status information of a target object, including: an imaging device for obtaining multiple frames of images about the target object; and the status information determining device as described above.
本公开还提供了一种可移动平台，包括：可移动本体；以及如上所述的状态信息确定***。The present disclosure also provides a movable platform, including: a movable body; and the state information determining system as described above.
本公开还提供了一种可读存储介质，其上存储有可执行指令，该指令被处理器执行时使处理器执行如上所述的方法。The present disclosure also provides a readable storage medium having executable instructions stored thereon, and when the instructions are executed by a processor, the processor executes the method described above.
通过本公开的实施例，根据多个检测框信息识别出无效的测距结果，以便从多个测距结果中筛选出有效测距结果，根据多个检测框信息和有效测距结果确定目标对象的状态信息，至少部分解决了错误数据污染目标对象的状态信息而导致目标对象的状态估计失效的技术问题，降低了测距结果的误检测率，提高了状态信息估计的可靠性。Through the embodiments of the present disclosure, invalid ranging results are identified based on multiple detection frame information, so that effective ranging results can be filtered from multiple ranging results, and the target object is determined based on multiple detection frame information and effective ranging results The state information of the target object at least partially solves the technical problem that the state information of the target object is contaminated by the wrong data, which reduces the false detection rate of the ranging result and improves the reliability of the state information estimation.
通过本公开的实施例，通过确定多个检测框信息中满足预设条件的一个或多个目标检测框信息，根据一个或多个目标检测框信息和多个测距结果确定目标对象的状态信息，至少部分解决了错误数据污染目标对象的状 态信息而导致目标对象的状态估计失效的技术问题，提高了状态信息估计的可靠性。Through the embodiments of the present disclosure, by determining one or more target detection frame information satisfying preset conditions among the plurality of detection frame information, the state information of the target object is determined according to the one or more target detection frame information and the multiple ranging results , At least partially solves the technical problem that the state information of the target object is contaminated by the wrong data, which causes the state estimation of the target object to fail, and improves the reliability of the state information estimation.
附图说明Description of the drawings
附图是用来提供对本公开的进一步理解，并且构成说明书的一部分，与下面的具体实施方式一起用于解释本公开，但并不构成对本公开的限制。在附图中：The accompanying drawings are used to provide a further understanding of the present disclosure and constitute a part of the specification. Together with the following specific embodiments, they are used to explain the present disclosure, but do not constitute a limitation to the present disclosure. In the attached picture:
图1示意性示出了根据本公开实施例的可以应用目标对象的状态信息确定方法的应用场景。Fig. 1 schematically shows an application scenario in which a method for determining state information of a target object can be applied according to an embodiment of the present disclosure.
图2示意性示出了根据本公开实施例的通过成像装置捕捉到的关于目标对象的一帧图像的示意图。FIG. 2 schematically shows a schematic diagram of a frame of image about a target object captured by an imaging device according to an embodiment of the present disclosure.
图3示意性示出了根据本公开实施例的目标对象的状态信息确定方法的流程图。Fig. 3 schematically shows a flowchart of a method for determining status information of a target object according to an embodiment of the present disclosure.
图4示意性示出了根据本公开实施例的对每个检测框信息对应的一个或多个测距结果进行筛选的流程图。Fig. 4 schematically shows a flowchart of screening one or more ranging results corresponding to each detection frame information according to an embodiment of the present disclosure.
图5示意性示出了根据本公开实施例的根据每个测距结果对应的检测框信息和每个测距结果对应的激光光斑，确定每个测距结果的有效性的流程图。Fig. 5 schematically shows a flowchart of determining the validity of each ranging result according to the detection frame information corresponding to each ranging result and the laser spot corresponding to each ranging result according to an embodiment of the present disclosure.
图6示意性示出了根据本公开实施例的确定目标检测框信息的流程图。Fig. 6 schematically shows a flowchart of determining target detection frame information according to an embodiment of the present disclosure.
图7示意性示出了根据本公开实施例的根据多个检测框信息确定目标对象是否出现满足预设条件的移动的流程图。Fig. 7 schematically shows a flow chart of determining whether a target object moves that meets a preset condition according to multiple detection frame information according to an embodiment of the present disclosure.
图8示意性示出了根据本公开实施例的检测框信息对应的初始位置信息的概率分布的示意图。FIG. 8 schematically shows a schematic diagram of a probability distribution of initial position information corresponding to detection frame information according to an embodiment of the present disclosure.
图9示意性示出了根据本公开另一实施例的检测框信息对应的初始位置信息的概率分布的示意图。Fig. 9 schematically shows a schematic diagram of a probability distribution of initial position information corresponding to detection frame information according to another embodiment of the present disclosure.
图10示意性示出了根据本公开实施例的根据第一概率分布和第二概率分布进行叠加确定概率密度最高的空间位置的示意图。FIG. 10 schematically shows a schematic diagram of superimposing the first probability distribution and the second probability distribution to determine the spatial position with the highest probability density according to an embodiment of the present disclosure.
图11示意性示出了根据本公开实施例的根据每个目标检测框信息和每个目标检测框信息对应的有效测距结果，确定目标对象的状态信息的流程图。FIG. 11 schematically shows a flowchart of determining the state information of a target object according to each target detection frame information and the effective ranging result corresponding to each target detection frame information according to an embodiment of the present disclosure.
图12示意性示出了根据本公开实施例的筛选每个目标检测框信息对应的关于目标对象的初始位置信息的时间轴示意图。FIG. 12 schematically shows a schematic diagram of a time axis for screening the initial position information of the target object corresponding to each target detection frame information according to an embodiment of the present disclosure.
图13示意性示出了根据本公开实施例的对目标对象的位置信息进行预测的示意图。FIG. 13 schematically shows a schematic diagram of predicting the position information of a target object according to an embodiment of the present disclosure.
图14示意性示出了根据本公开另一实施例的目标对象的状态信息确定方法的流程图。Fig. 14 schematically shows a flowchart of a method for determining status information of a target object according to another embodiment of the present disclosure.
图15示意性示出了根据本公开实施例的目标对象的状态信息确定***的框图。Fig. 15 schematically shows a block diagram of a system for determining status information of a target object according to an embodiment of the present disclosure.
具体实施方式Detailed ways
下面将结合实施例和实施例中的附图，对本公开技术方案进行清楚、完整的描述。显然，所描述的实施例仅仅是本公开一部分实施例，而不是全部的实施例。基于本公开中的实施例，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例，都属于本公开保护的范围。在下面的详细描述中，为便于解释，阐述了许多具体的细节以提供对本公开实施例的全面理解。然而，明显地，一个或多个实施例在没有这些具体细节的情况下也可以被实施。此外，在以下说明中，省略了对公知结构和技术的描述，以避免不必要地混淆本公开的概念。The technical solutions of the present disclosure will be clearly and completely described below in conjunction with the embodiments and the drawings in the embodiments. Obviously, the described embodiments are only a part of the embodiments of the present disclosure, rather than all the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by those of ordinary skill in the art without creative work shall fall within the protection scope of the present disclosure. In the following detailed description, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of the embodiments of the present disclosure. However, it is obvious that one or more embodiments can also be implemented without these specific details. In addition, in the following description, descriptions of well-known structures and technologies are omitted to avoid unnecessarily obscuring the concept of the present disclosure.
附图中示出了一些方框图和/或流程图。应理解，方框图和/或流程图中的一些方框或其组合可以由计算机程序指令来实现。这些计算机程序指令可以提供给通用计算机、专用计算机或其他可编程数据处理装置的处理器，从而这些指令在由该处理器执行时可以创建用于实现这些方框图和/或流程图中所说明的功能/操作的装置。本公开的技术可以硬件和/或软件(包括固件、微代码等)的形式来实现。另外，本公开的技术可以采取存储有指令的计算机可读存储介质上的计算机程序产品的形式，该计算机程序产品可供指令执行***使用或者结合指令执行***使用。Some block diagrams and/or flowcharts are shown in the drawings. It should be understood that some blocks in the block diagrams and/or flowcharts or combinations thereof can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that these instructions can be used to implement the functions described in these block diagrams and/or flowcharts when executed by the processor. /Operation device. The technology of the present disclosure can be implemented in the form of hardware and/or software (including firmware, microcode, etc.). In addition, the technology of the present disclosure may take the form of a computer program product on a computer-readable storage medium storing instructions, and the computer program product can be used by an instruction execution system or in combination with an instruction execution system.
可移动平台可以配置成像装置，允许用户远程确定目标对象的位置信息，对目标对象进行跟踪或拍照等其他应用，目标对象例如可以包括人、移动物体、静止对象等。以跟踪目标对象为例，这种跟踪目标对象的能力允许可移动平台在跟踪或拍摄移动的目标对象的同时，可以自主地操作成 像装置等其它装置以便于目标对象的成像。The movable platform can be equipped with an imaging device to allow users to remotely determine the location information of the target object, track the target object or take pictures and other applications. The target object can include, for example, people, moving objects, and stationary objects. Take tracking target objects as an example. This ability to track target objects allows the movable platform to autonomously operate imaging devices and other devices to facilitate imaging of the target objects while tracking or photographing moving target objects.
作为示意性的说明示例，可移动平台(例如，UAV)可以被配置为自主地跟踪物体的移动并相应地调整其移动的速度和方向，同时调整成像装置的朝向，以维持距物体的预定的相对位置。通过这种配置，UAV可以维持针对物体的预定的视场，使得当物体运动时，可以以基本相同的范围和精度来捕捉物体的图像。As a schematic illustrative example, a movable platform (e.g., UAV) may be configured to autonomously track the movement of an object and adjust the speed and direction of its movement accordingly, while adjusting the orientation of the imaging device to maintain a predetermined distance from the object. relative position. With this configuration, the UAV can maintain a predetermined field of view for the object, so that when the object moves, the image of the object can be captured with substantially the same range and accuracy.
在实际应用中，可以使用机器学***台的位置、成像装置的姿态等等，从而对目标对象进行跟踪。In practical applications, machine learning algorithms can be used to identify the target object that needs to be tracked on the image collected by the imaging device to obtain the detection frame of the target object in the image, and determine the position of the target object according to the detection frame of the target object. And change the position of the movable platform, the posture of the imaging device, etc. according to the position of the target object, so as to track the target object.
在实现本公开的过程中发现，根据目标对象的检测框确定出目标对象的位置，其准确性和可靠性不高，例如，当目标对象不位于采集的画面中心时，无法较为准确地获得可移动平台与目标对象之间的距离，导致无法较为准确地确定目标对象的位置。In the process of implementing the present disclosure, it is found that the accuracy and reliability of determining the position of the target object according to the detection frame of the target object is not high. The distance between the mobile platform and the target object makes it impossible to determine the position of the target object more accurately.
但是，在确定目标对象的位置时，能够将目标对象的检测框与目标对象的测距结果结合起来，可提高目标对象的状态信息的准确性和可靠性。因此，如何结合目标对象的检测框与测距结果，来获取目标对象的位置和轨迹等状态信息是目前亟待解决的问题。However, when determining the position of the target object, the detection frame of the target object can be combined with the distance measurement result of the target object, which can improve the accuracy and reliability of the status information of the target object. Therefore, how to combine the detection frame of the target object with the ranging result to obtain status information such as the position and trajectory of the target object is a problem to be solved urgently.
目标对象的位置与距离信息可以由目标对象的视觉检测框结合多种测距方法提供，常见的有：卫星导航***、激光雷达、ToF(Time of Flight飞行时间测距法，简称ToF)测距、双目测距、根据视差进行三角测距、以及根据其他先验知识(例如，目标对象的对地高度、目标对象的自身高度)进行测距。The position and distance information of the target object can be provided by the visual detection frame of the target object combined with a variety of ranging methods, common ones are: satellite navigation system, lidar, ToF (Time of Flight ranging method, referred to as ToF) ranging , Binocular distance measurement, triangulation distance measurement based on parallax, and distance measurement based on other prior knowledge (for example, the height of the target object to the ground, the height of the target object itself).
这些方法各有优劣，对融合多种方式确定目标对象位置的过程提出了挑战。例如，卫星导航***的定位精度较低，但不受到视觉障碍遮挡的影响；激光雷达、ToF测距的精度高，测距范围大，但要求UAV与目标对象之间没有障碍物；双目测距精度受限于UAV的尺寸，远距离目标对象精度较差；根据视差进行三角测距，仅适用于静态目标对象，并需要UAV以固定轨迹飞行，成像装置以两种不同的视角拍摄目标对象；根据先验知识进行测距的精度，取决于先验假设与实际情况的符合程度。因此，需要 改进的目标对象的状态信息确定方法，以增强确定目标对象状态信息的适应性与可靠性。These methods have their own advantages and disadvantages, which pose a challenge to the process of merging multiple methods to determine the location of the target object. For example, the positioning accuracy of satellite navigation system is low, but it is not affected by visual obstruction; Lidar and ToF ranging have high accuracy and large ranging range, but there is no obstacle between the UAV and the target object; binocular measurement Range accuracy is limited by the size of the UAV, and the accuracy of long-distance targets is poor; triangulation based on parallax is only suitable for static target objects, and the UAV needs to fly with a fixed trajectory. The imaging device shoots the target object with two different perspectives. ; The accuracy of distance measurement based on prior knowledge depends on the degree of conformity between the prior hypothesis and the actual situation. Therefore, an improved method for determining the status information of the target object is needed to enhance the adaptability and reliability of determining the status information of the target object.
根据本公开的实施例，提供了改进的目标对象的状态信息确定方法，可以通过机器学***台的距离，结合两种数据得到实时或近似实时的目标对象的状态信息，目标对象的状态信息例如可以包括当前位置、速度、历史轨迹等等。According to an embodiment of the present disclosure, an improved method for determining the status information of a target object is provided. The detection frame of the target object can be recognized by a machine learning method, and the relative position of the target object can be obtained by measuring with a distance measuring device (for example, single-point lidar). Based on the distance of the movable platform, the two kinds of data are combined to obtain real-time or near real-time status information of the target object. The status information of the target object may include, for example, current position, speed, historical trajectory, and so on.
以下将以可移动平台为无人机UAV为例，对可以应用本公开实施例提供的目标对象的状态信息确定方法的应用场景进行说明。The following will take the UAV UAV as the movable platform as an example to describe the application scenarios in which the method for determining the status information of the target object provided in the embodiments of the present disclosure can be applied.
图1示意性示出了根据本公开实施例的可以应用目标对象的状态信息确定方法的应用场景。需要注意的是，图1所示仅为可以应用本公开实施例的场景的示例，以帮助本领域技术人员理解本公开的技术内容，但并不意味着本公开实施例不可以用于其他设备、***、环境或场景。Fig. 1 schematically shows an application scenario in which a method for determining state information of a target object can be applied according to an embodiment of the present disclosure. It should be noted that FIG. 1 is only an example of a scenario where the embodiment of the present disclosure can be applied to help those skilled in the art understand the technical content of the present disclosure, but it does not mean that the embodiment of the present disclosure cannot be used for other devices. , System, environment or scene.
如图1所示，无人机101的成像装置的视野中可以包括但不限于目标对象102、对象103和对象104等。示例性地，无人机101可以通过成像装置捕捉关于目标对象102的一帧或多帧图像，通过对每一帧图像进行识别，可以得到关于目标对象102的检测框信息。根据本公开的实施例，目标对象102可以处于运动状态，也可以处于静止状态。在目标对象102处于某一状态下，尤其是运动状态的情况下，成像装置可以实时或按照预设时间间隔捕捉关于目标对象102的图像，然后对捕捉到的图像进行识别，得到多个检测框信息。As shown in FIG. 1, the field of view of the imaging device of the drone 101 may include, but is not limited to, a target object 102, an object 103, an object 104, and the like. Exemplarily, the drone 101 can capture one or more frames of images about the target object 102 through an imaging device, and by recognizing each frame of image, the detection frame information about the target object 102 can be obtained. According to an embodiment of the present disclosure, the target object 102 may be in a moving state or in a stationary state. When the target object 102 is in a certain state, especially when it is in motion, the imaging device can capture images of the target object 102 in real time or at preset time intervals, and then recognize the captured images to obtain multiple detection frames information.
图2示意性示出了根据本公开实施例的通过成像装置捕捉到的关于目标对象的一帧图像的示意图。FIG. 2 schematically shows a schematic diagram of a frame of image about a target object captured by an imaging device according to an embodiment of the present disclosure.
如图2所示，通过成像装置捕捉到的图像200包括目标对象102、对象103和对象104。可以利用基于机器学习的神经网络对图像200进行识别，得到关于目标对象102的一个检测框信息1021。As shown in FIG. 2, the image 200 captured by the imaging device includes a target object 102, an object 103, and an object 104. The image 200 can be recognized by a neural network based on machine learning, and a detection frame information 1021 about the target object 102 can be obtained.
无人机101可以通过测距装置测量(例如，单点激光雷达)得到目标对象102相对于无人机101的距离。在目标对象102处于某一状态下，尤其是运动状态的情况下，测距装置可以实时或按照预设时间间隔测量得到目标对象102相对于无人机101的距离，得到多个测距结果。The UAV 101 can obtain the distance of the target object 102 relative to the UAV 101 through a distance measuring device (for example, a single-point lidar). When the target object 102 is in a certain state, especially when it is in a moving state, the distance measuring device can measure the distance of the target object 102 relative to the drone 101 in real time or according to a preset time interval, and obtain multiple distance measurement results.
通过结合目标对象102的检测框信息和测距结果可以得到实时或近似实时的目标对象102的状态信息，例如，可以得到目标对象的位置信息、速度信息和历史轨迹等等。By combining the detection frame information of the target object 102 and the ranging result, real-time or near real-time status information of the target object 102 can be obtained, for example, the position information, speed information, historical trajectory, etc. of the target object can be obtained.
本公开的实施例可以适用于基于目标对象的状态信息的多种应用。例如，通过本公开的实施方式获得的目标状态信息，能够进一步用于目标对象跟踪、目标对象环绕、成像装置朝向控制、成像装置跟踪对焦、跟踪变焦、目标对象的位置预测、反馈优化视觉目标识别、AR(Augmented Reality，增强现实，简称AR)监视等领域，实现了控制可移动平台及其承载的成像装置对目标对象进行跟踪等场景应用。The embodiments of the present disclosure can be applied to various applications based on the status information of the target object. For example, the target state information obtained by the embodiments of the present disclosure can be further used for target object tracking, target object surrounding, imaging device orientation control, imaging device tracking focus, tracking zoom, target object position prediction, feedback optimization of visual target recognition , AR (Augmented Reality, Augmented Reality, AR for short) surveillance and other fields have realized scene applications such as controlling the movable platform and the imaging device carried by it to track the target object.
图3示意性示出了根据本公开实施例的目标对象的状态信息确定方法的流程图。Fig. 3 schematically shows a flowchart of a method for determining status information of a target object according to an embodiment of the present disclosure.
需要说明的是，本公开实施例中的流程图所示的操作除非明确说明不同操作之间存在执行的先后顺序，或者不同操作在技术实现上存在执行的先后顺序，否则，多个操作之间的执行顺序可以不分先后，多个操作也可以同时执行。It should be noted that the operations shown in the flowchart in the embodiments of the present disclosure, unless it is clearly stated that there is a sequence of execution between different operations, or there is a sequence of execution of different operations in technical implementation, otherwise, between multiple operations The order of execution can be in no particular order, and multiple operations can also be executed at the same time.
例如，在图3中，操作S301和操作S302在技术实现上存在先后顺序，而操作S303与操作S301和操作S302在技术实现上不存在先后顺序，也没有明确说明操作S301和操作S302在操作S303之前执行。因此，尽管图3所示的流程图中操作S301和操作S302在操作S303的前面，但是在执行操作S303时，也可以在操作S301和操作S302执行之前，或者与在操作S301和操作S302同时执行。For example, in FIG. 3, operation S301 and operation S302 have an order in terms of technical implementation, while operation S303, operation S301 and operation S302 do not have an order in terms of technical implementation, and it is not clearly stated that operation S301 and operation S302 are in operation S303. Executed before. Therefore, although the operation S301 and the operation S302 precede the operation S303 in the flowchart shown in FIG. 3, when the operation S303 is performed, it may be performed before the operation S301 and the operation S302 are performed, or simultaneously with the operation S301 and the operation S302. .
如图3所示，该目标对象的状态信息确定方法包括操作S301～S305。As shown in FIG. 3, the method for determining the status information of the target object includes operations S301 to S305.
在操作S301，通过可移动平台携带的成像装置获得关于目标对象的多帧图像。In operation S301, a multi-frame image about a target object is obtained through an imaging device carried by a movable platform.
在操作S302，对多帧图像中的每一帧图像进行识别，得到关于目标对象的多个检测框信息。In operation S302, each of the multiple frames of images is recognized to obtain multiple detection frame information about the target object.
根据本公开的实施例，成像装置可以在其视野范围内不断的捕捉图像。通过对捕捉到的图像进行图像识别，得到包含目标对象的多帧图像，当然，成像装置捕捉到的图像中也可能不包含目标对象。According to the embodiments of the present disclosure, the imaging device can continuously capture images within its field of view. By performing image recognition on the captured image, a multi-frame image containing the target object is obtained. Of course, the image captured by the imaging device may not include the target object.
根据本公开的实施例，例如，可以通过机器学习方法对捕捉到的每一帧图像进行识别，如果图像中包括目标对象，可以得到关于目标对象的一个检测框信息。如果图像中没有目标对象，则可以将该图像标记为无效图像，进一步地，可以将该无效图像进行滤除，或者，如果该图像中没有目标对象，可以直接该图像进行滤除。According to the embodiments of the present disclosure, for example, each frame of the captured image can be recognized by a machine learning method, and if the target object is included in the image, a detection frame information about the target object can be obtained. If there is no target object in the image, the image can be marked as an invalid image, and further, the invalid image can be filtered out, or if there is no target object in the image, the image can be filtered out directly.
根据本公开的实施例，检测框信息可以包括多种信息，例如，可以包括以下多种信息中的部分或全部：目标对象在图像画面上的位置和大小、成像装置的视场角、位置和姿态、图像的采样时间点等等。According to an embodiment of the present disclosure, the detection frame information may include a variety of information, for example, it may include part or all of the following: the position and size of the target object on the image screen, the angle of view, the position of the imaging device, and the The posture, the sampling time point of the image, and so on.
根据本公开的实施例，可以以矩形框表示检测框(如图2所示)，目标对象在图像画面上的位置和大小可以由目标识别模块提供。成像装置的视场角可以由成像装置的变焦模块提供，变焦模块可以执行如下算法(一)和(二)得到成像装置的视场角。According to an embodiment of the present disclosure, the detection frame can be represented by a rectangular frame (as shown in FIG. 2), and the position and size of the target object on the image screen can be provided by the target recognition module. The field angle of the imaging device can be provided by the zoom module of the imaging device, and the zoom module can execute the following algorithms (1) and (2) to obtain the field angle of the imaging device.
Figure PCTCN2020089002-appb-000001
Figure PCTCN2020089002-appb-000001
Figure PCTCN2020089002-appb-000002
Figure PCTCN2020089002-appb-000002
其中，fov zx表示图像画面行方向的视场角，fov zy表示图像画面列方向的视场角，focal_length是成像装置实时的焦距，W，H是成像装置的图像传感器的宽和高。 Among them, fov zx represents the field angle in the row direction of the image frame, fov zy represents the field angle in the column direction of the image frame, focal_length is the real-time focal length of the imaging device, and W, H are the width and height of the image sensor of the imaging device.
根据本公开的实施例，成像装置相对于世界坐标系的姿态，以及对应的采样时间，可以由载具姿态测量模块提供。根据本公开的实施例，成像装置在世界坐标系中的位置，以及对应的采样时间，可以由载具位置测量模块提供。According to an embodiment of the present disclosure, the posture of the imaging device relative to the world coordinate system and the corresponding sampling time can be provided by the vehicle posture measurement module. According to an embodiment of the present disclosure, the position of the imaging device in the world coordinate system and the corresponding sampling time may be provided by the vehicle position measurement module.
在操作S303，获得可移动平台与目标对象之间的多个测距结果。In operation S303, a plurality of ranging results between the movable platform and the target object are obtained.
根据本公开的实施例，可以通过测距装置测量得到可移动平台与目标对象之间的多个测距结果。其中，测距装置的类型不做限定。According to the embodiments of the present disclosure, multiple distance measurement results between the movable platform and the target object can be measured by the distance measurement device. Among them, the type of the ranging device is not limited.
例如，可以通过单点激光雷达得到目标对象相对于无人机的距离。其中，单点激光雷达可以设置在可移动平台上。当然，本公开并不限于通过可移动平台上的单点激光雷达获得测距结果，也可以通过ToF相机、单线激光雷达、面阵激光雷达、双目相机等其他测距装置获得可移动平台与目标对象之间的多个测距结果。For example, a single-point lidar can be used to obtain the distance of the target object relative to the UAV. Among them, the single-point lidar can be set on a movable platform. Of course, the present disclosure is not limited to obtaining ranging results through single-point lidar on a movable platform, and can also obtain the movable platform and the distance measurement device through other ranging devices such as ToF camera, single-line lidar, area array lidar, binocular camera, etc. Multiple ranging results between target objects.
根据本公开的实施例，测距装置可以按照时间顺序测量得到每个测距结果，每个测距结果具有对应的采样时间点。According to an embodiment of the present disclosure, the ranging device can measure each ranging result in a time sequence, and each ranging result has a corresponding sampling time point.
根据本公开的实施例，测距装置可以使用具有测距距离长和精度高的特点的单点激光雷达，可移动平台与目标对象之间的测距结果可以是激光测距结果。成像装置可以使用单目相机，成本低于激光雷达与相机阵列。采用单点激光雷达测量可移动平台与目标对象之间的实时距离，不需要卫星导航***提供目标对象的空间位置，也不依赖成像装置与目标对象的相对高度不变或基本不变。利用低成本的单目相机和单点激光雷达，较为可靠地实现了远距离目标对象的连续测距，增加了目标对象的位置估计的适应性与可靠性。通过本公开的实施例，能够过滤无效的激光测量结果，降低了激光的误检测率。According to an embodiment of the present disclosure, the distance measurement device may use a single-point lidar with the characteristics of long distance measurement and high accuracy, and the distance measurement result between the movable platform and the target object may be the laser distance measurement result. The imaging device can use a monocular camera, and the cost is lower than that of lidar and camera arrays. The use of single-point lidar to measure the real-time distance between the movable platform and the target object does not require a satellite navigation system to provide the spatial position of the target object, nor does it rely on the relative height of the imaging device and the target object to remain unchanged or basically unchanged. Using low-cost monocular cameras and single-point lidars, the continuous ranging of long-distance target objects is more reliably achieved, which increases the adaptability and reliability of target object position estimation. Through the embodiments of the present disclosure, invalid laser measurement results can be filtered, and the false detection rate of lasers can be reduced.
在操作S304，根据多个检测框信息从多个测距结果中筛选出有效测距结果。In operation S304, an effective ranging result is filtered from the multiple ranging results according to the multiple detection frame information.
根据本公开的实施例，成像装置的采样频率和测距装置的采样频率可以相同，也可以不同。每个检测框信息可以具有对应的一个测距结果，或者，每个检测框信息可以具有对应的多个测距结果。According to an embodiment of the present disclosure, the sampling frequency of the imaging device and the sampling frequency of the ranging device may be the same or different. Each detection frame information may have a corresponding ranging result, or each detection frame information may have corresponding multiple ranging results.
在根据多个检测框信息从多个测距结果中筛选出有效测距结果时，可以先确定每个检测框信息对应的一个或多个测距结果；然后对每个检测框信息对应的一个或多个测距结果进行筛选。When selecting effective ranging results from multiple ranging results based on multiple detection frame information, one or more ranging results corresponding to each detection frame information can be determined first; then one or more ranging results corresponding to each detection frame information Or multiple ranging results to filter.
根据本公开的实施例，每个检测框信息具有对应的采样时间点，可以将检测框信息对应的图像的采样时间点作为检测框信息对应的采样时间点。According to the embodiment of the present disclosure, each detection frame information has a corresponding sampling time point, and the sampling time point of the image corresponding to the detection frame information can be used as the sampling time point corresponding to the detection frame information.
在确定每个检测框信息对应的一个或多个测距结果时，可以根据每个检测框信息的采样时间点和获取到的多个测距结果中每个测距结果的采样时间点确定每个检测框信息对应的一个或多个测距结果。When determining one or more ranging results corresponding to each detection frame information, each can be determined according to the sampling time point of each detection frame information and the sampling time point of each ranging result in the multiple obtained ranging results. One or more ranging results corresponding to each detection frame information.
根据本公开的实施例，可以将每个测距结果关联至与测距结果的采样时间点最接近的检测框信息。以将测距结果关联至两个检测框信息中的某一个检测框为例，针对某一测距结果，将测距结果对应的采样时间点减去第一检测框信息的采样时间点得到第一时间差，将测距结果对应的采样时间点减去第二检测框信息的采样时间点得到第二时间差，比较第一时间差 和第二时间差的绝对值的大小，将绝对值更小的对应检测框信息确定为与该测距结果的采样时间点最接近的检测框信息，从而可以确定每个检测框信息对应的一个或多个测距结果。According to an embodiment of the present disclosure, each ranging result can be associated with the detection frame information closest to the sampling time point of the ranging result. Taking the ranging result to one of the two detection frame information as an example, for a certain ranging result, the sampling time point corresponding to the ranging result is subtracted from the sampling time point of the first detection frame information to obtain the first A time difference, subtract the sampling time point of the second detection frame information from the sampling time point corresponding to the ranging result to obtain the second time difference, compare the absolute value of the first time difference and the second time difference, and detect the smaller absolute value The frame information is determined as the detection frame information closest to the sampling time point of the ranging result, so that one or more ranging results corresponding to each detection frame information can be determined.
根据本公开的实施例，可以根据多个检测框信息识别出无效的测距结果，以便从多个测距结果中筛选出有效测距结果。According to the embodiments of the present disclosure, invalid ranging results can be identified based on multiple detection frame information, so as to filter out effective ranging results from the multiple ranging results.
在操作S305，根据多个检测框信息和有效测距结果确定目标对象的状态信息。In operation S305, the state information of the target object is determined according to the multiple detection frame information and the effective ranging result.
根据本公开的实施例，有效测距结果可以包括一个或多个。可以先确定每个检测框信息对应的有效测距结果；然后根据每个检测框信息和每个检测框信息对应的有效测距结果，确定目标对象的状态信息。According to an embodiment of the present disclosure, the effective ranging result may include one or more. The effective ranging result corresponding to each detection frame information can be determined first; then, the state information of the target object can be determined according to each detection frame information and the effective ranging result corresponding to each detection frame information.
根据本公开的实施例，例如，可以根据每个检测框对应的成像装置的视场角，位置和姿态以及对应的有效测距结果确定目标对象的状态信息。According to the embodiments of the present disclosure, for example, the state information of the target object can be determined according to the field of view, position and posture of the imaging device corresponding to each detection frame, and the corresponding effective ranging result.
根据本公开的实施例，可以根据采样时间点确定每个检测框信息对应的有效测距结果。According to the embodiments of the present disclosure, the effective ranging result corresponding to each detection frame information can be determined according to the sampling time point.
例如，可以根据每个检测框信息的采样时间点和每个有效测距结果的采样时间点，将每个有效测距结果关联至与有效测距结果的采样时间点最接近的检测框信息。具体方法可以参考上述将测距结果关联至两个检测框信息中的某一个检测框的示例，在此不再赘述。For example, according to the sampling time point of each detection frame information and the sampling time point of each effective ranging result, each effective ranging result can be associated with the detection frame information closest to the sampling time point of the effective ranging result. The specific method can refer to the above example of associating the ranging result to a certain detection frame of the two detection frame information, which will not be repeated here.
根据本公开的实施例，在每个检测框信息对应多个有效测距结果的情况下，根据每个检测框信息和每个检测框信息对应的有效测距结果，确定目标对象的状态信息，包括：计算每个检测框信息对应的多个有效测距结果的加权平均值，得到每个检测框信息对应的目标测距结果；以及根据每个检测框信息和每个检测框信息对应的目标测距结果，确定目标对象的状态信息。According to an embodiment of the present disclosure, when each detection frame information corresponds to multiple valid ranging results, the state information of the target object is determined according to each detection frame information and the effective ranging result corresponding to each detection frame information, Including: calculating the weighted average of multiple effective ranging results corresponding to each detection frame information to obtain the target ranging result corresponding to each detection frame information; and according to each detection frame information and the target corresponding to each detection frame information The results of ranging, determine the status information of the target object.
根据本公开的实施例，可以将检测框信息对应的每个有效测距结果的回波强度除以基准值后得到的结果作为该有效测距结果的权重，其中，基准值为检测框信息对应的所有有效测距结果的回波强度之和；然后根据多个有效测距结果和每个有效测距结果的权重计算得到加权平均值。According to the embodiments of the present disclosure, the result obtained by dividing the echo intensity of each effective ranging result corresponding to the detection frame information by the reference value can be used as the weight of the effective ranging result, where the reference value corresponds to the detection frame information The sum of the echo intensities of all effective ranging results; then a weighted average is calculated according to the multiple effective ranging results and the weight of each effective ranging result.
根据本公开的实施例，基准值可以是与检测框信息对应的多个有效测距结果的回波强度之和。According to an embodiment of the present disclosure, the reference value may be the sum of echo intensities of multiple effective ranging results corresponding to the detection frame information.
例如，针对第一检测框信息，与其对应的有效测距结果包括有效测距结果1～有效测距结果4，有效测距结果1～4对应的回波强度为回波强度1～回波强度4。有效测距结果1的权重1为回波强度1/(回波强度1+回波强度2+回波强度3+回波强度4)，有效测距结果2的权重2为回波强度2/(回波强度1+回波强度2+回波强度3+回波强度4)，有效测距结果3的权重3为回波强度3/(回波强度1+回波强度2+回波强度3+回波强度4)，有效测距结果4的权重4为回波强度4/(回波强度1+回波强度2+回波强度3+回波强度4)。For example, for the first detection frame information, the corresponding effective ranging results include effective ranging results 1 to 4, and the echo intensity corresponding to the effective ranging results 1 to 4 is echo intensity 1 to echo intensity 4. The weight 1 of the effective ranging result 1 is the echo strength 1/(echo strength 1+echo strength 2+the echo strength 3+the echo strength 4), and the weight 2 of the effective ranging result 2 is the echo strength 2/ (Echo Intensity 1+Echo Intensity 2+Echo Intensity 3+Echo Intensity 4), the weight 3 of the effective ranging result 3 is the echo intensity 3/(Echo Intensity 1+Echo Intensity 2+Echo Intensity 3+Echo Intensity 4), the weight 4 of the effective ranging result 4 is Echo Intensity 4/(Echo Intensity 1+Echo Intensity 2+Echo Intensity 3+Echo Intensity 4).
那么，第一检测框信息对应的目标测距结果等于有效测距结果1×权重1+有效测距结果2×权重2+有效测距结果3×权重3+有效测距结果4×权重4。Then, the target ranging result corresponding to the first detection frame information is equal to the effective ranging result 1×weight 1+effective ranging result 2×weight 2+effective ranging result 3×weight 3+effective ranging result 4×weight 4.
根据本公开的实施例，目标对象的状态信息包括但不限于目标对象在世界坐标系中的位置信息、速度信息、以及历史轨迹。其中，历史轨迹包括若干个轨迹点。According to an embodiment of the present disclosure, the state information of the target object includes, but is not limited to, position information, speed information, and historical trajectory of the target object in the world coordinate system. Among them, the historical trajectory includes several trajectory points.
通过本公开的实施例，根据多个检测框信息识别出无效的测距结果，以便从多个测距结果中筛选出有效测距结果，根据多个检测框信息和有效测距结果确定目标对象的状态信息，至少部分解决了错误数据污染目标对象的状态信息而导致目标对象的状态估计失效的技术问题，降低了测距结果的误检测率，提高了状态信息估计的可靠性。Through the embodiments of the present disclosure, invalid ranging results are identified based on multiple detection frame information, so that effective ranging results can be filtered from multiple ranging results, and the target object is determined based on multiple detection frame information and effective ranging results The state information of the target object at least partially solves the technical problem that the state information of the target object is contaminated by the wrong data, which reduces the false detection rate of the ranging result and improves the reliability of the state information estimation.
同时，由于结合检测框信息和有效测距结果确定目标对象的状态信息，对获得的图像数据没有过高的要求，可以不使用深度传感器采集深度图像，至少部分解决了由于深度传感器的局限性，导致获得的深度图有效范围小或精度低的问题。At the same time, because the detection frame information and the effective ranging result are combined to determine the status information of the target object, there is no excessive requirement on the obtained image data, and the depth image can be collected without using a depth sensor, which at least partially solves the limitations of the depth sensor. This leads to the problem that the effective range of the obtained depth map is small or the accuracy is low.
下面参考图4～图13，结合具体实施例对图3所示的方法做进一步说明。The method shown in FIG. 3 will be further described below with reference to FIGS. 4 to 13 in conjunction with specific embodiments.
图4示意性示出了根据本公开实施例的对每个检测框信息对应的一个或多个测距结果进行筛选的流程图。Fig. 4 schematically shows a flowchart of screening one or more ranging results corresponding to each detection frame information according to an embodiment of the present disclosure.
如图4所示，对每个检测框信息对应的一个或多个测距结果进行筛选可以包括操作S401～操作S403。As shown in FIG. 4, screening one or more ranging results corresponding to each detection frame information may include operations S401 to S403.
在操作S401，确定每个检测框信息对应的一个或多个测距结果中每个 测距结果对应的激光光斑。In operation S401, a laser spot corresponding to each of the one or more ranging results corresponding to each detection frame information is determined.
根据本公开的实施例，以测距结果为激光测距结果为例，例如，可以利用成像装置的视场角和激光雷达的散射角分别计算测距结果lidar m...lidar n-1、lidar n对应的在画面中的激光光斑circle m...circle n-1、circle n。 According to an embodiment of the present disclosure, taking the distance measurement result as the laser distance measurement result as an example, for example, the field angle of the imaging device and the scattering angle of the lidar can be used to calculate the distance measurement results lidar m ...lidar n-1 , lidar n corresponds to the laser spot circle m ... circle n-1 and circle n in the picture.
在操作S402，根据每个测距结果对应的检测框信息和每个测距结果对应的激光光斑，确定每个测距结果的有效性。In operation S402, the validity of each ranging result is determined according to the detection frame information corresponding to each ranging result and the laser spot corresponding to each ranging result.
根据本公开的实施例，可以确定每个测距结果对应的检测框信息和每个测距结果对应的激光光斑的面积重合率，然后根据面积重合率确定每个测距结果的有效性。According to the embodiments of the present disclosure, the detection frame information corresponding to each ranging result and the area coincidence rate of the laser spot corresponding to each ranging result can be determined, and then the validity of each ranging result can be determined according to the area coincidence rate.
根据本公开的实施例，每个测距结果具有对应的检测框信息和激光光斑。可以根据测距结果对应的检测框信息(如检测框在画面中的长和宽)计算检测框的面积，将激光光斑的面积除以检测框的面积，得到测距结果对应的检测框信息和激光光斑之间的面积重合率，然后根据面积重合率确定每个测距结果的有效性。According to an embodiment of the present disclosure, each ranging result has corresponding detection frame information and laser spot. The area of the detection frame can be calculated according to the detection frame information corresponding to the distance measurement result (such as the length and width of the detection frame in the screen), and the area of the laser spot is divided by the area of the detection frame to obtain the detection frame information and corresponding to the distance measurement result. The area coincidence rate between the laser spots, and then the validity of each ranging result is determined according to the area coincidence rate.
根据本公开的实施例，可以将面积重合率与预设比例阈值进行比较；将面积重合率大于或等于预设比例阈值的测距结果确定为有效测距结果；将面积重合率小于预设比例阈值的测距结果确定为无效测距结果。According to the embodiments of the present disclosure, the area coincidence rate can be compared with a preset ratio threshold; the distance measurement result whose area coincidence ratio is greater than or equal to the preset ratio threshold is determined as the effective distance measurement result; the area coincidence ratio is less than the preset ratio The distance measurement result of the threshold value is determined to be an invalid distance measurement result.
根据本公开的实施例，预设比例阈值的大小可以根据经验预先设定。示意性的，例如，预设比例阈值的大小可以是70％。若面积重合率≥70％，则可以将该测距结果标记为有效，否则标记为无效，然后根据标记结果将无效测距结果进行滤除。According to an embodiment of the present disclosure, the size of the preset ratio threshold can be preset based on experience. Illustratively, for example, the size of the preset ratio threshold may be 70%. If the area overlap rate is greater than or equal to 70%, the ranging result can be marked as valid, otherwise the ranging result can be marked as invalid, and then the invalid ranging result will be filtered out according to the marking result.
在操作S403，根据每个测距结果的有效性对每个检测框信息对应的一个或多个测距结果进行筛选。In operation S403, one or more ranging results corresponding to each detection frame information are filtered according to the validity of each ranging result.
根据本公开的实施例，可以将无效测距结果从检测框信息对应的一个或多个测距结果中滤除，将有效测距结果确定为与检测框信息对应的测距结果。According to the embodiments of the present disclosure, invalid ranging results can be filtered out from one or more ranging results corresponding to the detection frame information, and the valid ranging results can be determined as the ranging results corresponding to the detection frame information.
图5示意性示出了根据本公开实施例的根据每个测距结果对应的检测框信息和每个测距结果对应的激光光斑，确定每个测距结果的有效性的流程图。Fig. 5 schematically shows a flowchart of determining the validity of each ranging result according to the detection frame information corresponding to each ranging result and the laser spot corresponding to each ranging result according to an embodiment of the present disclosure.
如图5所示，根据每个测距结果对应的检测框信息和每个测距结果对 应的激光光斑，确定每个测距结果的有效性可以包括操作S501～操作S502。As shown in Figure 5, according to the detection frame information corresponding to each ranging result and the laser spot corresponding to each ranging result, determining the validity of each ranging result may include operations S501 to S502.
在操作S501，根据采样时间相邻的两个检测框信息，确定采样时间相邻的两个检测框信息的采样时间之间的多个测距结果中每个测距结果分别对应的插值检测框信息，得到每个测距结果对应的检测框信息。In operation S501, according to the information of two detection frames with adjacent sampling times, determine the interpolation detection frame corresponding to each of the multiple ranging results between the sampling times of the two detection frame information with adjacent sampling times. Information to obtain the detection frame information corresponding to each ranging result.
根据本公开的实施例，例如，针对任意两个采样时间相邻的包括目标对象的检测框信息box a和box b，对应的采样时间点分别为t a和t b，并选中t a和t b之间的时间段中所有测距结果lidar m...lidar n-1、lidar n，测距结果对应的采样时间点分别为t m、t m+1...t n-1、t n，其中，满足t a＜t m...t n-1、t n≤t b。 According to the embodiment of the present disclosure, for example, for any two adjacent detection frame information box a and box b including the target object at the sampling time, the corresponding sampling time points are respectively t a and t b , and t a and t are selected In the time period between b, all ranging results lidar m ...lidar n-1 , lidar n , the sampling time points corresponding to the ranging results are t m , t m+1 ...t n-1 , t n , where t a <t m ... t n-1 and t n ≤ t b are satisfied.
假设在该时间段t a到t b中，检测框从box a匀速变化为box b，使用插值算法得到t m...t n-1、t n分别对应的插值检测框box m...box n-1、box n。 Assuming that during the time period t a to t b , the detection frame changes from box a to box b at a constant speed, and the interpolation algorithm is used to obtain the interpolation detection frames box m ... t m ... t n-1 and t n respectively. box n-1 , box n .
分别计算测距结果lidar m……lidar n-1、lidar n对应的在画面中的激光光斑circle m……circle n-1、circle n。例如，可以利用成像装置的视场角和激光雷达的散射角计算得到激光光斑。 Calculate the laser spot circle m ……circle n-1 and circle n corresponding to lidar m ……lidar n-1 and lidar n in the screen respectively. For example, the laser spot can be calculated by using the field of view angle of the imaging device and the scattering angle of the lidar.
在操作S502，根据测距结果对应的激光光斑和与该测距结果对应的检测框信息，确定每个测距结果的有效性。In operation S502, the validity of each ranging result is determined according to the laser spot corresponding to the ranging result and the detection frame information corresponding to the ranging result.
根据本公开的实施例，可以根据测距结果对应的检测框信息和该测距结果对应的激光光斑的面积重合率确定每个测距结果的有效性。具体方法可以参考上述对图4的描述，在此不再赘述。According to the embodiments of the present disclosure, the validity of each ranging result can be determined according to the detection frame information corresponding to the ranging result and the area overlap ratio of the laser spot corresponding to the ranging result. For the specific method, reference may be made to the above description of FIG. 4, which will not be repeated here.
根据本公开的实施例，特别地，针对每一个新生成的检测框，可以确定采样时间与新生成的检测框最靠近的前一个检测框，对采样时间为新生成的检测框和最靠近的前一个检测框之间的测距结果进行筛选。可以实现实时或近似实时的筛选测距结果，从而达到较为准确的确定目标对象的状态信息的效果。According to the embodiments of the present disclosure, in particular, for each newly generated detection frame, the previous detection frame whose sampling time is closest to the newly generated detection frame can be determined, and the sampling time is the newly generated detection frame and the closest detection frame. The results of the distance measurement between the previous detection frame are filtered. It can realize real-time or near real-time screening of the ranging results, so as to achieve the effect of more accurately determining the status information of the target object.
根据本公开的实施例，通过机器学***台的多个测距结果，通过检测框信息对多个测距结果进行筛选得到有效测距结果，结合目标对象的检测框信息和有效测距结果，可以得到实时或近似实时的目标对象的状态信息。According to the embodiments of the present disclosure, the detection frame information of the target object is obtained by the machine learning method, the multiple ranging results of the target object relative to the movable platform are obtained by the ranging device, and the multiple ranging results are filtered by the detection frame information Obtaining effective ranging results, combining the detection frame information of the target object and the effective ranging results, can obtain real-time or near real-time status information of the target object.
根据本公开的实施例，在确定每个检测框信息对应的有效测距结果之后，还可以确定每个检测框信息对应的关于目标对象的物理估计尺寸；根 据每个检测框信息对应的关于目标对象的物理估计尺寸，对每个检测框信息对应的有效测距结果进行筛选。According to the embodiment of the present disclosure, after the effective ranging result corresponding to each detection frame information is determined, the physical estimated size of the target object corresponding to each detection frame information can also be determined; The physical estimated size of the object, and the effective ranging results corresponding to each detection frame information are screened.
根据本公开的实施例，可以根据每个检测框信息和每个检测框信息采集时的成像装置的视场角，确定每个检测框信息对应的视场角；然后根据每个检测框信息对应的视场角和每个检测框信息对应的有效测距结果，确定每个检测框信息对应的物理估计尺寸。According to the embodiments of the present disclosure, the field angle corresponding to each detection frame information can be determined according to each detection frame information and the field of view angle of the imaging device when each detection frame information is collected; and then corresponding to each detection frame information The field of view angle and the effective ranging result corresponding to each detection frame information determine the physical estimated size corresponding to each detection frame information.
对于一个检测框信息，首先根据成像装置的视场角以及检测框的大小计算检测框的视场角，然后，根据有效测距结果与检测框的视场角，计算检测框信息对应的物理估计尺寸，该物理估计尺寸是关于目标对象在现实世界的估计尺寸。For a detection frame information, first calculate the field angle of the detection frame according to the field angle of the imaging device and the size of the detection frame, and then calculate the physical estimation corresponding to the detection frame information according to the effective ranging result and the field angle of the detection frame Size, the physical estimated size is the estimated size of the target object in the real world.
根据本公开的实施例，具体地，计算检测框信息对应的关于目标对象的物理估计尺寸可以包括如下操作。According to an embodiment of the present disclosure, specifically, calculating the physical estimated size of the target object corresponding to the detection frame information may include the following operations.
首先，根据图像画面的实时视场角和目标对象在画面上的位置与大小，计算目标对象在画面上的矩形框的左边沿和右边沿分别对应的相对于成像装置的航向角(即成像装置的光心到矩形框的上下左右四个角的连线形成的夹角，可以利用像素偏移以及画面的实时视场角确定)，以及计算目标对象在画面上的矩形框的上边沿和下边沿分别对应的相对于成像装置的俯仰角。First, according to the real-time field of view of the image frame and the position and size of the target object on the screen, calculate the heading angle of the target object relative to the imaging device corresponding to the left edge and the right edge of the rectangular frame on the screen (ie the imaging device The angle formed by the connection between the optical center and the four corners of the top, bottom, left, and right corners of the rectangular frame can be determined by using the pixel offset and the real-time viewing angle of the screen), and calculate the upper edge and bottom of the rectangular frame of the target object on the screen The edges respectively correspond to the pitch angle relative to the imaging device.
然后，将左边沿和右边沿分别对应的航向角相减，得到目标相对于成像装置的航向角范围，并将上边沿和下边沿分别对应的俯仰角相减，得到目标相对于成像装置的俯仰角范围，目标对象相对于成像装置的航向角范围与俯仰角范围下面简称为目标检测框的视场角。Then, the heading angles corresponding to the left and right edges are subtracted to obtain the heading angle range of the target relative to the imaging device, and the pitch angles corresponding to the upper and lower edges are subtracted to obtain the pitch of the target relative to the imaging device. The angle range, the range of the heading angle and the range of the pitch angle of the target object relative to the imaging device are hereinafter referred to as the field of view angle of the target detection frame.
接着，在检测框的视场角确定后，可以利用检测框的视场角以及有效测距结果，确定对应于航向角以及俯仰角的弧长，利用两个弧长的端点可以确定矩形框的四边的范围，从而可以确定目标检测框的物理估计尺寸。Then, after the field angle of the detection frame is determined, the field angle of the detection frame and the effective ranging result can be used to determine the arc length corresponding to the heading angle and the pitch angle. The end points of the two arc lengths can be used to determine the rectangular frame The range of the four sides can thus determine the physical estimated size of the target detection frame.
可以根据目标对象的类型的先验知识估计目标对象的物理尺寸的合理范围，并过滤不合理的有效测距结果。The reasonable range of the physical size of the target object can be estimated based on the prior knowledge of the type of the target object, and unreasonable effective ranging results can be filtered.
根据本公开的实施例，上述根据每个检测框信息对应的关于目标对象的物理估计尺寸，对每个检测框信息对应的有效测距结果进行筛选，可以包括：将每个检测框信息对应的关于目标对象的物理估计尺寸与预设合理 范围进行比较；以及在检测框信息对应的关于目标对象的物理估计尺寸与预设合理范围不相符的情况下，滤除检测框信息对应的有效测距结果。According to the embodiment of the present disclosure, the above-mentioned filtering the effective ranging result corresponding to each detection frame information according to the physical estimated size of the target object corresponding to each detection frame information may include: Compare the physical estimated size of the target object with the preset reasonable range; and filter out the effective ranging corresponding to the detection frame information when the physical estimated size of the target object corresponding to the detection frame information does not match the preset reasonable range result.
根据本公开的实施例，可以先确定目标对象的对象类型，其中，每个对象类型具有对应的预设合理范围；然后根据目标对象的对象类型确定目标预设合理范围；将每个检测框信息对应的关于目标对象的物理估计尺寸与目标预设合理范围进行比较。According to the embodiments of the present disclosure, the object type of the target object can be determined first, where each object type has a corresponding preset reasonable range; then the target preset reasonable range is determined according to the object type of the target object; the information of each detection frame The corresponding physical estimated size of the target object is compared with the preset reasonable range of the target.
例如，若目标对象为人，则目标预设合理范围可以是0.6米到2米，可以将计算得到的物理估计尺寸与0.6米到2米进行比较。若目标对象为车辆，则目标预设合理范围可以是2米到20米，可以将计算得到的物理估计尺寸与2米到20米进行比较。For example, if the target object is a person, the preset reasonable range of the target can be 0.6 meters to 2 meters, and the calculated physical estimated size can be compared with 0.6 meters to 2 meters. If the target object is a vehicle, the preset reasonable range of the target can be 2 meters to 20 meters, and the calculated physical estimated size can be compared with 2 meters to 20 meters.
根据本公开的实施例，在滤除检测框信息对应的有效测距结果之后，如果该检测框信息没有对应的有效测距结果，可以将检测框信息进行滤除。According to the embodiment of the present disclosure, after the effective ranging result corresponding to the detection frame information is filtered out, if the detection frame information does not have a corresponding effective ranging result, the detection frame information can be filtered out.
当然，根据本公开的实施例，也可以不用将该检测信息进行滤除，在滤除检测框信息对应的有效测距结果之后，如果该检测框信息没有对应的有效测距结果，则可以将与滤除了有效测距结果的检测框信息的采样时间相邻的检测框信息对应的有效测距结果，确定为该滤除了有效测距结果的检测框信息对应的有效测距结果。Of course, according to the embodiments of the present disclosure, the detection information may not be filtered out. After filtering out the effective ranging result corresponding to the detection frame information, if the detection frame information does not have a corresponding effective ranging result, then The effective ranging result corresponding to the detection frame information adjacent to the sampling time of the detection frame information from which the effective ranging result is filtered out is determined to be the effective ranging result corresponding to the detection frame information from which the effective ranging result is filtered out.
根据本公开的实施例，在根据多个检测框信息和有效测距结果确定目标对象的状态信息时，可以先确定多个检测框信息中满足预设条件的一个或多个目标检测框信息，根据每个目标检测框信息和每个目标检测框信息对应的有效测距结果，确定目标对象的状态信息，而对于除目标检测框信息之外的其它检测框可以不做使用。According to the embodiment of the present disclosure, when determining the state information of the target object according to the multiple detection frame information and the effective ranging result, one or more target detection frame information that meets the preset condition among the multiple detection frame information can be determined first, According to each target detection frame information and the effective ranging result corresponding to each target detection frame information, the state information of the target object is determined, and other detection frames except the target detection frame information may not be used.
根据本公开的实施例，可以先确定一个或多个目标检测框信息中的每个目标检测框信息分别对应的有效测距结果，换言之，先将目标检测框信息与对应的有效测距结果进行关联，然后根据每个目标检测框信息和每个目标检测框信息对应的有效测距结果，确定目标对象的状态信息。According to the embodiment of the present disclosure, the effective ranging result corresponding to each target detection frame information in one or more target detection frame information can be determined first, in other words, the target detection frame information and the corresponding effective ranging result are first determined. Associate, and then determine the status information of the target object according to each target detection frame information and the effective ranging result corresponding to each target detection frame information.
图6示意性示出了根据本公开实施例的确定目标检测框信息的流程图。Fig. 6 schematically shows a flowchart of determining target detection frame information according to an embodiment of the present disclosure.
如图6所示，确定目标检测框信息可以包括操作S601～操作S602。As shown in FIG. 6, determining the target detection frame information may include operations S601 to S602.
在操作S601，根据多个检测框信息确定目标对象是否出现满足预设条件的移动。In operation S601, it is determined whether the target object has moved that meets a preset condition according to a plurality of detection frame information.
根据本公开的实施例，在一个时间段内，成像装置可以捕捉多帧图像，对多帧图像进行识别可以得到多个检测框信息，可以根据多个检测框信息判断目标对象在一个时间段内是否有大幅运动，以确定目标对象是否出现满足预设条件的移动。According to the embodiments of the present disclosure, within a period of time, the imaging device can capture multiple frames of images, recognize the multiple frames of images to obtain multiple detection frame information, and can determine that the target object is within a period of time based on the multiple detection frame information Whether there is a large movement to determine whether the target object has a movement that meets the preset conditions.
在操作S602，在目标对象出现了满足预设条件的移动的情况下，将目标对象满足预设条件的移动时对应的检测框信息确定为目标检测框信息。In operation S602, in a case where the target object has a movement that satisfies a preset condition, the detection frame information corresponding to the movement of the target object that satisfies the preset condition is determined as the target detection frame information.
根据本公开的实施例，例如，如果目标对象在一个时间段内有大幅运动，可以将目标对象在大幅运动时对应的检测框信息确定为目标检测框信息。需要说明的是，目标对象在大幅运动时对应的检测框信息可以包括多个，因此，可以得到多个目标检测框信息。进一步的，可以对多个目标检测框信息进行标记，用于确定目标对象的状态信息时使用。According to the embodiments of the present disclosure, for example, if the target object has a large movement in a period of time, the detection frame information corresponding to the target object when the target object has a large movement can be determined as the target detection frame information. It should be noted that the detection frame information corresponding to the target object may include multiple pieces of detection frame information when the target object moves substantially. Therefore, multiple pieces of target detection frame information may be obtained. Further, multiple target detection frame information can be marked for use when determining the status information of the target object.
根据本公开的实施例，接下来对目标对象是否出现满足预设条件的移动进行说明。According to an embodiment of the present disclosure, whether the target object moves that meets a preset condition will be described next.
图7示意性示出了根据本公开实施例的根据多个检测框信息确定目标对象是否出现满足预设条件的移动的流程图。Fig. 7 schematically shows a flow chart of determining whether a target object moves that meets a preset condition according to multiple detection frame information according to an embodiment of the present disclosure.
如图7所示，根据多个检测框信息确定目标对象是否出现满足预设条件的移动可以包括操作S701～操作S704。As shown in FIG. 7, determining whether the target object has moved that meets a preset condition according to multiple detection frame information may include operations S701 to S704.
在操作S701，针对多个检测框信息中任意相邻的第一检测框信息和第二检测框信息，获得第一检测框信息采集时的成像装置的第一状态信息，和第二检测框信息采集时的成像装置的第二状态信息。In operation S701, for any adjacent first detection frame information and second detection frame information among a plurality of detection frame information, first state information and second detection frame information of the imaging device when the first detection frame information is collected are obtained The second state information of the imaging device at the time of collection.
根据本公开的实施例，例如，可以获取一个时间段内的多个检测框信息，针对该一个时间段内的任意两个采样时间点相邻的第一检测框信息和第二检测框信息，获得每个检测框信息采集时的成像装置的状态信息。其中，成像装置的状态信息可以包括成像装置的朝向、成像装置的视场角、成像装置的位置。According to the embodiment of the present disclosure, for example, multiple detection frame information within a time period can be acquired, for the first detection frame information and the second detection frame information that are adjacent to any two sampling time points in the one time period, Obtain the state information of the imaging device when the information of each detection frame is collected. Wherein, the status information of the imaging device may include the orientation of the imaging device, the angle of view of the imaging device, and the position of the imaging device.
在操作S702，根据第一检测框信息对应的有效测距结果和第一状态信息，确定与第一检测框信息对应的关于目标对象的初始位置信息的第一概率分布。In operation S702, according to the effective ranging result corresponding to the first detection frame information and the first state information, a first probability distribution of the initial position information of the target object corresponding to the first detection frame information is determined.
在操作S703，根据第二检测框信息对应的有效测距结果和第二状态信息，确定与第二检测框信息对应的关于目标对象的初始位置信息的第二概率分布。In operation S703, according to the effective ranging result corresponding to the second detection frame information and the second state information, a second probability distribution of the initial position information of the target object corresponding to the second detection frame information is determined.
根据本公开的实施例，例如，选中一个时间段内任意两个检测框box a和box b，并查找与检测框box a和box b的采样时间t a和t b对应的成像装置的朝向、成像装置的视场角、成像装置的位置(以下称为检测框信息)以及有效测距结果。然后，对每一个检测框，使用反投影算法，根据检测框信息和有效测距结果，计算该检测框信息对应的初始位置信息的概率分布，该概率分布表征了在物理空间范围内该检测框映射到各个空间位置的概率。 According to the present embodiment of the present disclosure, e.g., a period of time of any two selected detection frame and box a box b, and looks toward the detection frame box a sampling time and box b t a and t b of the corresponding image forming apparatus, The angle of view of the imaging device, the position of the imaging device (hereinafter referred to as detection frame information), and the effective ranging result. Then, for each detection frame, the back projection algorithm is used to calculate the probability distribution of the initial position information corresponding to the detection frame information according to the detection frame information and the effective ranging result, and the probability distribution represents the detection frame within the physical space. The probability of mapping to each spatial location.
图8示意性示出了根据本公开实施例的检测框信息对应的初始位置信息的概率分布的示意图。FIG. 8 schematically shows a schematic diagram of a probability distribution of initial position information corresponding to detection frame information according to an embodiment of the present disclosure.
当检测框801有对应的有效测距结果时，初始位置信息的概率分布为花盆型(或称之为圆台型)。如图8所示，可以以目标对象的位置为原始位置，裁掉距离目标对象0.95倍的有效测距结果之前的部分以及距离目标对象1.05倍的有效测距结果之后的部分，得到目标对象的概率分布为花盆型，有效测距结果可以是成像装置与目标对象之间的距离，有效测距结果的上下限与测量装置的测量误差正相关，越靠近中心，概率越高。When the detection frame 801 has a corresponding effective ranging result, the probability distribution of the initial position information is a flowerpot type (or called a truncated cone type). As shown in Figure 8, the position of the target object can be used as the original position, and the part before the effective ranging result 0.95 times away from the target object and the part after the effective ranging result 1.05 times away from the target object can be cut to obtain the target object’s original position. The probability distribution is a flowerpot type, and the effective ranging result can be the distance between the imaging device and the target object. The upper and lower limits of the effective ranging result are positively correlated with the measurement error of the measuring device. The closer to the center, the higher the probability.
图9示意性示出了根据本公开另一实施例的检测框信息对应的初始位置信息的概率分布的示意图。Fig. 9 schematically shows a schematic diagram of a probability distribution of initial position information corresponding to detection frame information according to another embodiment of the present disclosure.
当检测框901没有对应的有效测距结果时(例如，根据检测框信息对应的关于目标对象的物理估计尺寸，确定该检测框信息对应的测距结果无效)，初始位置信息的概率分布为圆锥型，如图9所示，越靠近圆锥轴线，概率越高。根据本公开的实施例，可以将与检测框901的采样时间点相邻的检测框信息对应的有效测距结果，确定为检测框901对应的有效测距结果。然后，根据该检测框901对应的有效测距结果采用对图8介绍的方法，将圆锥型概率分布裁减为花盆型概率分布。When the detection frame 901 does not have a corresponding valid ranging result (for example, according to the physical estimated size of the target object corresponding to the detection frame information, it is determined that the ranging result corresponding to the detection frame information is invalid), the probability distribution of the initial position information is a cone Type, as shown in Figure 9, the closer to the cone axis, the higher the probability. According to the embodiment of the present disclosure, the effective ranging result corresponding to the detection frame information adjacent to the sampling time point of the detection frame 901 can be determined as the effective ranging result corresponding to the detection frame 901. Then, according to the effective ranging result corresponding to the detection frame 901, the method described in FIG. 8 is adopted to cut the cone-shaped probability distribution into a flowerpot-shaped probability distribution.
在操作S704，根据第一概率分布和第二概率分布，确定目标对象是否出现满足预设条件的移动。In operation S704, according to the first probability distribution and the second probability distribution, it is determined whether the target object moves that meets a preset condition.
根据本公开的实施例，根据第一概率分布和第二概率分布，确定目标对象是否出现满足预设条件的移动包括如下操作。According to an embodiment of the present disclosure, according to the first probability distribution and the second probability distribution, determining whether the target object moves that meets a preset condition includes the following operations.
首先，根据第一概率分布和第二概率分布确定概率密度最高的空间位置。例如，将第一概率分布和第二概率分布进行叠加，计算得到概率密度最高的空间位置。需要说明的是，对于没有对应的有效激光测距结果的检测框信息，也可以采用概率分布叠加的方式确定概率密度最高的空间位置。First, the spatial location with the highest probability density is determined according to the first probability distribution and the second probability distribution. For example, the first probability distribution and the second probability distribution are superimposed, and the spatial position with the highest probability density is calculated. It should be noted that, for the detection frame information for which there is no corresponding effective laser ranging result, the probability distribution superposition method can also be used to determine the spatial position with the highest probability density.
图10示意性示出了根据本公开实施例的根据第一概率分布和第二概率分布进行叠加确定概率密度最高的空间位置的示意图。FIG. 10 schematically shows a schematic diagram of superimposing the first probability distribution and the second probability distribution to determine the spatial position with the highest probability density according to an embodiment of the present disclosure.
如图10所示，首先，将第一检测框信息1001对应的第一概率分布和第二检测框信息1002对应的第二概率分布进行叠加，可以确定概率密度最高的空间位置1003。As shown in FIG. 10, first, the first probability distribution corresponding to the first detection frame information 1001 and the second probability distribution corresponding to the second detection frame information 1002 are superimposed to determine the spatial position 1003 with the highest probability density.
需要说明的是，无论两个检测框信息对应的概率分布是否有重叠，都可以确定出概率密度最高的空间位置。It should be noted that regardless of whether the probability distributions corresponding to the two detection frame information overlap, the spatial position with the highest probability density can be determined.
然后，分别计算概率密度最高的空间位置距离第一概率分布的第一概率分布中心位置的第一距离和概率密度最高的空间位置距离第二概率分布的第二概率分布中心位置的第二距离。例如，可以计算概率密度最高的空间位置在两种概率分布中，分别距离概率分布中心位置的马氏距离(欧式距离乘以概率分布系数)。Then, the first distance between the space position with the highest probability density and the center position of the first probability distribution of the first probability distribution and the second distance between the space position with the highest probability density and the center position of the second probability distribution of the second probability distribution are respectively calculated. For example, it is possible to calculate the Mahalanobis distance (Euclidean distance multiplied by the probability distribution coefficient) from the center position of the probability distribution in the two probability distributions of the spatial position with the highest probability density.
接下来，可以根据第一距离和第二距离确定第一检测框信息和第二检测框信息之间的概率距离。例如，将第一距离和第二距离之和作为这两个检测框之间的概率距离。Next, the probability distance between the first detection frame information and the second detection frame information can be determined according to the first distance and the second distance. For example, the sum of the first distance and the second distance is taken as the probability distance between the two detection frames.
最后，将第一检测框信息和第二检测框信息之间的概率距离与预设阈值进行比较，如果概率距离大于或等于预设阈值，可以确定目标对象出现了满足预设条件的移动。预设阈值的大小可以反映目标对象移动的幅度要求，例如，预设阈值越大，目标对象移动的幅度要求越高。预设阈值可以根据实际效果进行预设设定，可选地，预设阈值可以为1.0。Finally, the probability distance between the first detection frame information and the second detection frame information is compared with a preset threshold. If the probability distance is greater than or equal to the preset threshold, it can be determined that the target object has moved that meets the preset condition. The size of the preset threshold may reflect the amplitude requirement of the target object's movement. For example, the larger the preset threshold value, the higher the amplitude requirement of the target object's movement. The preset threshold may be preset according to actual effects, and optionally, the preset threshold may be 1.0.
根据本公开的实施例，若一时间段内任意两个检测框之间的概率距离大于预设阈值，则可以认为该一时间段内目标对象有大幅运动。其中，时间段的长度可以预先设定，例如，可以是2秒，5秒等等。According to an embodiment of the present disclosure, if the probability distance between any two detection frames in a period of time is greater than a preset threshold, it can be considered that the target object has a large movement in the period of time. The length of the time period can be preset, for example, it can be 2 seconds, 5 seconds, and so on.
根据本公开的实施例，根据每个目标检测框信息和每个目标检测框信息对应的有效测距结果，可以确定目标对象的状态信息。例如，可以根据目标检测框采集时的成像装置的朝向、视场角、位置以及有效测距结果，可以确定目标对象的初始位置信息。其中，目标检测框信息可以是发生大幅运动的检测框信息。According to the embodiments of the present disclosure, the state information of the target object can be determined according to each target detection frame information and the effective ranging result corresponding to each target detection frame information. For example, the initial position information of the target object can be determined according to the orientation, field of view, position of the imaging device and the effective ranging result when the target detection frame is collected. Among them, the target detection frame information may be detection frame information that has undergone a large movement.
图11示意性示出了根据本公开实施例的根据每个目标检测框信息和每个目标检测框信息对应的有效测距结果，确定目标对象的状态信息的流程图。FIG. 11 schematically shows a flowchart of determining the state information of a target object according to each target detection frame information and the effective ranging result corresponding to each target detection frame information according to an embodiment of the present disclosure.
如图11所示，根据每个目标检测框信息和每个目标检测框信息对应的有效测距结果，确定目标对象的状态信息可以包括操作S1101～操作S1103。As shown in FIG. 11, according to each target detection frame information and the effective ranging result corresponding to each target detection frame information, determining the state information of the target object may include operations S1101 to S1103.
在操作S1101，根据每个目标检测框信息和每个目标检测框信息对应的有效测距结果，确定每个目标检测框信息对应的关于目标对象的初始位置信息，得到多个初始位置信息。In operation S1101, according to each target detection frame information and the effective ranging result corresponding to each target detection frame information, determine the initial position information about the target object corresponding to each target detection frame information, and obtain multiple initial position information.
在操作S1102，根据每个目标检测框信息对应的采样时间点，对多个初始位置信息进行筛选，得到一个或多个有效初始位置信息。In operation S1102, a plurality of initial position information is filtered according to the sampling time point corresponding to each target detection frame information to obtain one or more effective initial position information.
根据本公开的实施例，可以按照采样时间点的时间顺序，依次对每个目标检测框信息对应的关于目标对象的初始位置信息进行筛选。According to the embodiment of the present disclosure, the initial position information about the target object corresponding to each target detection frame information can be sequentially filtered according to the time sequence of the sampling time point.
在操作S1103，根据一个或多个有效初始位置信息，确定目标对象的状态信息。In operation S1103, the state information of the target object is determined according to one or more valid initial position information.
根据本公开的实施例，例如，可以根据多个有效初始位置信息生成目标对象的运动轨迹。或者，可以根据多个有效初始位置信息计算目标对象的移动速度、加速度等等，According to the embodiments of the present disclosure, for example, the movement trajectory of the target object can be generated according to a plurality of valid initial position information. Alternatively, the moving speed, acceleration, etc. of the target object can be calculated based on multiple valid initial position information,
根据本公开的实施例，在根据一个或多个有效初始位置信息确定目标对象的状态信息时，可以对一个或多个有效初始位置信息进行优化，以平滑关于目标对象的运动轨迹。可以根据最终优化后的运动轨迹得到关于目标对象最终的优化位置信息。According to an embodiment of the present disclosure, when the state information of the target object is determined according to one or more valid initial position information, the one or more valid initial position information can be optimized to smooth the movement trajectory of the target object. The final optimized position information about the target object can be obtained according to the final optimized motion trajectory.
根据本公开的实施例，按照时间顺序，依次对每个目标检测框信息对应的关于目标对象的初始位置信息进行筛选包括如下操作。According to the embodiment of the present disclosure, sequentially screening the initial position information about the target object corresponding to each target detection frame information in a chronological order includes the following operations.
首先，计算当前正筛选的初始位置信息和与当前正筛选的初始位置信息的采样时间相邻的后一个初始位置信息的采样时间点之间的时间差；然后将时间差与状态变量阈值进行比较；如果时间差小于状态变量阈值，可以滤除当前正筛选的初始位置信息；如果时间差大于或等于状态变量阈值，可以保留当前正筛选的初始位置信息，其中，被保留的当前正筛选的初始位置信息为有效初始位置信息。First, calculate the time difference between the initial location information currently being screened and the sampling time point of the next initial location information adjacent to the sampling time of the initial location information currently being screened; then compare the time difference with the state variable threshold; if If the time difference is less than the state variable threshold, the initial position information currently being screened can be filtered out; if the time difference is greater than or equal to the state variable threshold, the initial position information currently being screened can be retained, where the retained initial position information currently being screened is valid Initial location information.
根据本公开的实施例，例如，从过去一段时间范围内的检测框中，选取出目标对象发生大幅运动的目标检测框box m…box n-1、box n，并将这些检测框的采样时间t m…t n-1、t n对应的初始位置信息pos m…pos n-1、pos n作为待筛选状态变量。 According to the embodiment of the present disclosure, for example, from the detection frames within a period of time in the past, the target detection frames box m …box n-1 , box n in which the target object has moved substantially are selected, and the sampling time of these detection frames t m ... t n-1, t n corresponding to the initial position information pos m ... pos n-1, pos n as the state variable to be screened.
然后，根据与目标检测框box m…box n-1、box n对应的相关数据(例如，成像装置的朝向、成像装置的视场角、成像装置的位置以及有效测距结果)，利用反投影算法，计算初始位置信息pos m…pos n-1、pos n的初始值，如果目标检测框没有对应的有效测距结果，可以采用采样时间相邻的前一个有效测距结果进行计算，得到初始值。 Then, according to the relevant data corresponding to the target detection frames box m ... box n-1 , box n (for example, the orientation of the imaging device, the angle of view of the imaging device, the position of the imaging device, and the effective ranging result), back projection Algorithm to calculate the initial value of the initial position information pos m … pos n-1 , pos n . If the target detection frame does not have a corresponding effective ranging result, the previous effective ranging result adjacent to the sampling time can be used for calculation to obtain the initial value.
最后，针对每个待筛选状态变量，计算其与后一个待筛选状态变量的时间差，从所有待筛选状态变量中，剔除与后一个待筛选状态变量的时间差小于状态变量阈值的状态变量。后一个待筛选状态变量是与当前正筛选的初始位置信息的采样时间相邻的后一个初始位置信息。Finally, for each state variable to be screened, the time difference between it and the next state variable to be screened is calculated, and from all the state variables to be screened, the state variable whose time difference with the next state variable to be screened is less than the state variable threshold is eliminated. The next state variable to be filtered is the next initial position information adjacent to the sampling time of the initial position information currently being filtered.
根据本公开的实施例，状态变量阈值可以是预先设定的固定值，例如，根据经验预先设定。According to an embodiment of the present disclosure, the state variable threshold may be a fixed value set in advance, for example, set in advance based on experience.
根据本公开的实施例，状态变量阈值也可以随确定的有效初始位置信息而发生变化。According to an embodiment of the present disclosure, the state variable threshold may also be changed with the determined effective initial position information.
根据本公开的实施例，状态变量阈值为当前正筛选的初始位置信息和与当前正筛选的初始位置信息的采样时间相邻的有效初始位置信息的采样时间点之间的时间差。According to an embodiment of the present disclosure, the state variable threshold is the time difference between the initial location information currently being screened and the sampling time point of the effective initial location information adjacent to the sampling time of the initial location information currently being screened.
下面以具体示例对按照时间顺序，依次筛选每个目标检测框信息对应的关于目标对象的初始位置信息进行说明。In the following, specific examples are used to describe the initial position information of the target object corresponding to each target detection frame information in chronological order.
例如，选取出目标对象发生大幅运动的目标检测框box 1～box 6，其对应的采样时间点为t 1～t 6。 For example, if the target detection frame box 1 to box 6 where the target object undergoes a large movement is selected, the corresponding sampling time point is t 1 to t 6 .
图12示意性示出了根据本公开实施例的筛选每个目标检测框信息对应的关于目标对象的初始位置信息的时间轴示意图。FIG. 12 schematically shows a schematic diagram of a time axis for screening the initial position information of the target object corresponding to each target detection frame information according to an embodiment of the present disclosure.
如图12所示，t 1～t 6按照时间顺序依次递增，采样时间t 6所对应的目标检测框box 6可以是当前一次最新的检测框。采样上述方式可以计算得到初始位置信息pos 1～pos 6的初始值，初始位置信息pos 1～pos 6的初始值对应于待筛选状态变量1～待筛选状态变量6，待筛选状态变量6作为最后一个待筛选状态变量。根据本公开的实施例，针对待筛选状态变量6，由于是当前最新的检测框，可以直接将其作为有效的状态变量，不进行剔除。 As shown in FIG. 12, t 1 ~ t 6 in ascending order of time, the sampling time t 6 the corresponding target detection frame box 6 may be a new current detection frame. Sampling the above-described embodiment can be calculated from 1 to the initial value of POS 6 initial position information POS, the initial position information pos 1 ~ pos initial value of 6 corresponds to being screened state variable 1 to be screened, the state variables 6, to be screened state variables 6 as the last A state variable to be filtered. According to the embodiment of the present disclosure, for the state variable 6 to be screened, since it is the latest detection frame, it can be directly used as a valid state variable without being eliminated.
在随目标对象的初始位置信息进行筛选的过程中，可以按照逆时间顺序，对所有待筛选状态变量进行处理，如图12所示，即先处理待筛选状态变量5、然后依次处理待筛选状态变量4～待筛选状态变量1。其中，待筛选状态变量6为待筛选状态变量5的后一个待筛选状态变量，待筛选状态变量5为待筛选状态变量4的后一个待筛选状态变量，待筛选状态变量4为待筛选状态变量3的后一个待筛选状态变量，以此类推。In the process of filtering with the initial position information of the target object, all the state variables to be filtered can be processed in reverse chronological order, as shown in Figure 12, that is, the state variables to be filtered 5 are processed first, and then the states to be filtered are processed in turn Variable 4~state variable 1 to be filtered. Among them, the to-be-filtered state variable 6 is the next to-be-filtered state variable of the to-be-filtered state variable 5, the to-be-filtered state variable 5 is the next to-be-filtered state variable of the to-be-filtered state variable 4, and the to-be-filtered state variable 4 is the state variable to be filtered. The state variable to be filtered after 3, and so on.
针对待筛选状态变量5，计算其与待筛选状态变量6的时间差t 6-t 5，将t 6-t 5与第五预设阈值进行比较，如果t 6-t 5小于第五预设阈值，则滤除初始位置信息pos 5；如果t 6-t 5大于或等于第五预设阈值，则保留初始位置信息pos 5，保留的初始位置信息pos 5作为有效的状态变量。 For the state variable 5 to be screened, calculate the time difference t 6 -t 5 between it and the state variable 6 to be screened, and compare t 6 -t 5 with the fifth preset threshold. If t 6 -t 5 is less than the fifth preset threshold , The initial position information pos 5 is filtered out; if t 6 -t 5 is greater than or equal to the fifth preset threshold, the initial position information pos 5 is retained, and the retained initial position information pos 5 is used as a valid state variable.
针对待筛选状态变量4，计算其与待筛选状态变量5的时间差t 5-t 4，将t 5-t 4与第四预设阈值进行比较，如果t 5-t 4小于第四预设阈值，则滤除初始位置信息pos 4；如果t 5-t 4大于或等于第四预设阈值，则保留初始位置信息pos 4，保留的初始位置信息pos 4作为有效的状态变量。 For the state variable 4 to be screened, calculate the time difference t 5 -t 4 between it and the state variable 5 to be screened, and compare t 5 -t 4 with the fourth preset threshold. If t 5 -t 4 is less than the fourth preset threshold , The initial position information pos 4 is filtered out; if t 5 -t 4 is greater than or equal to the fourth preset threshold, the initial position information pos 4 is retained, and the retained initial position information pos 4 is used as a valid state variable.
以此类推，对于剩余的待筛选状态变量，计算其与后一个待筛选状态变量的时间差，并将时间差与对应的预设阈值进行比较。By analogy, for the remaining state variables to be screened, the time difference between it and the next state variable to be screened is calculated, and the time difference is compared with the corresponding preset threshold.
例如，针对待筛选状态变量1，计算其与待筛选状态变量2的时间差t 2-t 1，将t 2-t 1与第一预设阈值进行比较，如果t 2-t 1小于第一预设阈值，则滤除初始位置信息pos 1；如果t 2-t 1大于或等于第一预设阈值，则保留初始位置信息pos 1，保留的初始位置信息pos 1作为有效的状态变量。 For example, for the state variable 1 to be screened, the time difference t 2 -t 1 between it and the state variable 2 to be screened is calculated, and t 2 -t 1 is compared with the first preset threshold. If t 2 -t 1 is less than the first preset threshold, If the threshold is set, the initial position information pos 1 is filtered out; if t 2 -t 1 is greater than or equal to the first preset threshold, the initial position information pos 1 is retained, and the retained initial position information pos 1 is used as a valid state variable.
进一步地，第一预设阈值～第三预设阈值可以是随确定的有效状态变量进行变化的。例如，状态变量阈值可以是当前正筛选的状态变量和与当 前正筛选的状态变量的采样时间相邻的有效状态变量的采样时间点之间的时间差。Further, the first preset threshold to the third preset threshold may be changed with the determined effective state variable. For example, the state variable threshold may be the time difference between the state variable currently being screened and the sampling time point of the valid state variable adjacent to the sampling time of the state variable currently being screened.
按照逆时间顺序对所有待筛选状态变量进行处理，具体地，例如，当前待筛选状态变量为待筛选状态变量5，后一个有效的状态变量为待筛选状态变量6，那么，第五预设阈值等于t 6-t 5，待筛选状态变量5与待筛选状态变量6的时间差t 6-t 5等于第五预设阈值t 6-t 5，因此，待筛选状态变量5为有效的状态变量。 All the state variables to be filtered are processed in reverse chronological order. Specifically, for example, the current state variable to be filtered is the state variable to be filtered 5, and the last valid state variable is the state variable to be filtered 6, then the fifth preset threshold Equal to t 6 -t 5 , the time difference t 6 -t 5 between the state variable 5 to be screened and the state variable 6 to be screened is equal to the fifth preset threshold t 6 -t 5 , therefore, the state variable 5 to be screened is a valid state variable.
针对待筛选状态变量4，后一个有效的状态变量为待筛选状态变量5，那么，第四预设阈值等于t 5-t 4，如果待筛选状态变量4与待筛选状态变量3的时间差t 4-t 3大于t 5-t 4，那么待筛选状态变量4为有效的状态变量，否则，待筛选状态变量4为无效的状态变量，将会被剔除。如果待筛选状态变量4为有效的状态变量，此时，针对待筛选状态变量3，后一个有效的状态变量为待筛选状态变量4，那么，第三预设阈值等于t 4-t 3，如果待筛选状态变量4为无效的状态变量，此时，针对待筛选状态变量3，后一个有效的状态变量为待筛选状态变量5，那么，第三预设阈值等于t 5-t 3。 For the state variable 4 to be filtered, the last valid state variable is the state variable 5 to be filtered, then the fourth preset threshold is equal to t 5 -t 4 , if the time difference between the state variable 4 to be filtered and the state variable 3 to be filtered is t 4 -t 3 is greater than t 5 -t 4 , then the state variable 4 to be filtered is a valid state variable, otherwise, the state variable 4 to be filtered is an invalid state variable and will be eliminated. If the state variable 4 to be filtered is a valid state variable, at this time, for the state variable 3 to be filtered, the next valid state variable is the state variable 4 to be filtered, then the third preset threshold is equal to t 4 -t 3 , if The state variable 4 to be screened is an invalid state variable. At this time, for the state variable 3 to be screened, the next valid state variable is the state variable 5 to be screened. Then, the third preset threshold is equal to t 5 -t 3 .
因此，第三预设阈值随待筛选状态变量4是否为有效的状态变量而变化，换言之，第三预设阈值随确定的有效状态变量进行变化。同样地，第二预设阈值和第一预设阈值随确定的有效状态变量进行变化，在此不再赘述。其中，有效状态变量即为有效初始位置信息。Therefore, the third preset threshold varies with whether the state variable 4 to be screened is a valid state variable, in other words, the third preset threshold varies with the determined valid state variable. Similarly, the second preset threshold value and the first preset threshold value change with the determined effective state variable, which will not be repeated here. Among them, the effective state variable is the effective initial position information.
根据本公开的实施例，在得到一个或多个有效初始位置信息之后，可以对一个或多个有效初始位置信息进行优化。例如，可以对一个或多个有效初始位置信息进行非线性优化，以最小化目标偏差，其中，目标偏差与检测框信息和/或有效测距结果相关，每个有效初始位置信息进行非线性优化后具有对应的优化位置信息。According to an embodiment of the present disclosure, after obtaining one or more effective initial position information, one or more effective initial position information can be optimized. For example, one or more effective initial position information can be non-linearly optimized to minimize the target deviation, where the target deviation is related to the detection frame information and/or the effective ranging result, and each effective initial position information is non-linearly optimized Then there is the corresponding optimized location information.
根据本公开的实施例，目标偏差可以包括第一偏差和/或第二偏差；第一偏差包括关于有效初始位置信息与用于计算得到有效初始位置信息的目标检测框信息和有效测距结果之间的观测偏差；第二偏差包括关于相邻有效初始位置信息之间的平滑程度与先验值的偏差。According to an embodiment of the present disclosure, the target deviation may include a first deviation and/or a second deviation; the first deviation includes information about the effective initial position and the target detection frame information used to calculate the effective initial position information and the effective ranging result. The second deviation includes the deviation between the smoothness of the adjacent effective initial position information and the prior value.
根据本公开的实施例，例如，目标偏差可以是如下算法(三)：
Figure PCTCN2020089002-appb-000003
最小化目标偏差可以是
Figure PCTCN2020089002-appb-000004
Figure PCTCN2020089002-appb-000005
其中，α i(x)表征第一偏差，β j(x j，x j+1，x j+2)表征第二偏差，x为优化变量，对应于一个有效初始位置信息。 According to an embodiment of the present disclosure, for example, the target deviation may be the following algorithm (3):
Figure PCTCN2020089002-appb-000003
Minimizing the target deviation can be
Figure PCTCN2020089002-appb-000004
Figure PCTCN2020089002-appb-000005
Among them, α i (x) represents the first deviation, β j (x j , x j+1 , x j+2 ) represents the second deviation, and x is an optimized variable corresponding to a valid initial position information.
根据本公开的实施例，第一偏差α i(x)可以通过用于计算得到有效初始位置信息的目标检测框信息和有效测距结果的概率密度函数进行表征。根据本公开的实施例，概率密度函数可以根据目标检测框信息对应的概率分布进行确定。 According to an embodiment of the present disclosure, the first deviation α i (x) may be characterized by the target detection frame information used to calculate the effective initial position information and the probability density function of the effective ranging result. According to an embodiment of the present disclosure, the probability density function may be determined according to the probability distribution corresponding to the target detection frame information.
根据本公开的实施例，可以利用差分公式，根据3个相邻的有效初始位置信息pos m...pos m+1、pos m+2之间的位置差，得到目标对象的运动速度vel m到m+1和vel m+1到m+2，再得到目标对象的加速度acc m+1，将目标对象的加速度与先验值的偏差＝(acc/先验值)作为相邻的有效初始位置信息之间的平滑程度与先验值的偏差β j(x j，x j+1，x j+2)。其中，先验值可以是预先设定的固定值。 According to the embodiment of the present disclosure, a difference formula can be used to obtain the movement velocity vel m of the target object according to the position difference between 3 adjacent effective initial position information pos m ... pos m+1 and pos m+2 To m+1 and vel m+1 to m+2 , the acceleration acc m+1 of the target object is obtained, and the deviation between the acceleration of the target object and the prior value = (acc/prior value) is regarded as the adjacent effective initial The deviation β j (x j , x j+1 , x j+2 ) of the degree of smoothness between the position information and the prior value. Among them, the prior value may be a preset fixed value.
根据本公开的实施例，对一个或多个有效初始位置信息进行非线性优化，以最小化目标偏差，可以是不断的改变有效初始位置信息，使用非线性优化方法迭代求解上述目标偏差，得到目标对象的优化位置信息。According to the embodiment of the present disclosure, one or more effective initial position information is nonlinearly optimized to minimize the target deviation. The effective initial position information may be continuously changed, and the non-linear optimization method is used to iteratively solve the target deviation to obtain the target The optimized location information of the object.
根据本公开的实施例，将有效初始位置信息进行非线性优化后对应的优化位置信息，可以使得目标偏差最小。According to the embodiment of the present disclosure, the effective initial position information is non-linearly optimized and the corresponding optimized position information can minimize the target deviation.
根据本公开的实施例，对一个或多个有效初始位置信息进行优化还可以包括：确定每个有效初始位置信息进行非线性优化后对应的优化位置信息是否异常；将异常的优化位置信息进行滤除。According to an embodiment of the present disclosure, optimizing one or more effective initial location information may further include: determining whether the corresponding optimized location information after nonlinear optimization of each effective initial location information is abnormal; and filtering the abnormal optimized location information remove.
在将异常的优化位置信息进行滤除之后，可以对剩余的优化位置信息再次进行非线性优化，以最小化目标偏差，其中，每个剩余的优化位置信息进行非线性优化后具有对应的最终优化位置信息。After filtering out the abnormal optimized position information, the remaining optimized position information can be non-linearly optimized again to minimize the target deviation. Among them, each remaining optimized position information has a corresponding final optimization after non-linear optimization. location information.
进一步地，可以根据每个剩余的优化位置信息进行非线性优化后对应的最终优化位置信息，计算得到关于目标对象的优化速度。例如，首先，获取若干个最新的最终优化位置，然后利用差分算法(例如，位置2减去位置1，然后除以时间差)，计算这些最新的最终优化位置对应的平均目标速度。进一步的，还可以对平均目标速度进行低通滤波，得到平滑无跳变的优化速度。Further, the final optimized position information corresponding to the non-linear optimization of each remaining optimized position information can be calculated to obtain the optimized speed of the target object. For example, first, a number of the latest final optimized positions are obtained, and then a difference algorithm (for example, position 2 minus position 1, and then divided by the time difference) is used to calculate the average target speed corresponding to these latest final optimized positions. Furthermore, it is possible to perform low-pass filtering on the average target speed to obtain an optimized speed that is smooth and jump-free.
根据本公开的实施例，第一偏差可以包括第一子偏差和/或第二子偏差。According to an embodiment of the present disclosure, the first deviation may include a first sub-deviation and/or a second sub-deviation.
第一子偏差为关于有效初始位置信息与用于计算得到有效初始位置信息的目标检测框信息之间的观测偏差。其中，用于计算得到有效初始位置信息的目标检测框信息例如可以包括成像装置的朝向、视场角、位置以及有效测距结果。The first sub-deviation is the observation deviation between the effective initial position information and the target detection frame information used for calculating the effective initial position information. Wherein, the target detection frame information used to calculate the effective initial position information may include, for example, the orientation, the angle of view, the position of the imaging device, and the effective ranging result.
第二子偏差为关于有效初始位置信息与用于计算得到有效初始位置信息的有效测距结果之间的观测偏差。The second sub-deviation is the observation deviation between the effective initial position information and the effective ranging result used to calculate the effective initial position information.
根据本公开的实施例，确定每个有效初始位置信息进行非线性优化后对应的优化位置信息是否异常，包括：若确定第一子偏差对应的检测框信息异常，和/或确定第二子偏差对应的有效测距结果异常，则将异常的检测框信息和/或异常的有效测距结果对应的优化位置确定为异常的优化位置。According to an embodiment of the present disclosure, determining whether the optimized position information corresponding to each effective initial position information after nonlinear optimization is abnormal includes: if it is determined that the detection frame information corresponding to the first sub-deviation is abnormal, and/or determining the second sub-deviation If the corresponding effective ranging result is abnormal, the abnormal detection frame information and/or the optimized position corresponding to the abnormal effective ranging result is determined as the abnormal optimized position.
根据本公开的实施例，确定第一子偏差对应的检测框信息异常的方式可以是如下：计算有效初始位置信息进行非线性优化后对应的优化位置信息与用于计算得到该有效初始位置信息的目标检测框信息之间的观测偏差，得到第一偏差值；将第一偏差值与第一偏差阈值进行比较，确定第一偏差值是否大于或等于第一偏差阈值，如果第一偏差值大于或等于第一偏差阈值，则确定第一子偏差对应的检测框信息异常。According to the embodiment of the present disclosure, the method of determining the abnormality of the detection frame information corresponding to the first sub-deviation may be as follows: calculating the effective initial position information and performing nonlinear optimization on the corresponding optimized position information and the effective initial position information used to obtain the effective initial position information. Observe the observed deviation between the target detection frame information to obtain the first deviation value; compare the first deviation value with the first deviation threshold value to determine whether the first deviation value is greater than or equal to the first deviation threshold value, if the first deviation value is greater than or If it is equal to the first deviation threshold, it is determined that the detection frame information corresponding to the first sub-deviation is abnormal.
根据本公开的实施例，确定第二子偏差对应的有效测距结果异常的方式可以是如下：计算有效初始位置信息进行非线性优化后对应的优化位置信息与用于计算得到有效初始位置信息的有效测距结果之间的观测偏差，得到第二偏差值；将第二偏差值与第二偏差阈值进行比较，确定第二偏差值是否大于或等于第二偏差阈值，如果第二偏差值大于或等于第二偏差阈值，则确定第二子偏差对应的有效测距结果异常。According to the embodiment of the present disclosure, the method for determining the abnormality of the effective ranging result corresponding to the second sub-deviation may be as follows: calculating the effective initial position information and performing nonlinear optimization on the corresponding optimized position information and calculating the effective initial position information. Observe the deviation between the effective ranging results to obtain the second deviation value; compare the second deviation value with the second deviation threshold value to determine whether the second deviation value is greater than or equal to the second deviation threshold value, if the second deviation value is greater than or If it is equal to the second deviation threshold, it is determined that the effective ranging result corresponding to the second sub-deviation is abnormal.
根据本公开的实施例，可以将检测框信息异常和有效测距结果异常相关的优化位置信息进行滤除，然后对剩余的优化位置信息再次进行非线性优化，以最小化目标偏差。在最小化目标偏差之后，每个剩余的优化位置信息进行非线性优化后具有对应的最终优化位置信息，将再次进行非线性优化后得到的最终优化位置信息作为目标对象的历史状态信息，根据目标对象的历史状态信息可以对目标对象的状态信息进行预测，如目标对象的位置、速度以及朝向等。According to the embodiment of the present disclosure, the optimized position information related to the abnormal detection frame information and the abnormal effective ranging result can be filtered out, and then the remaining optimized position information can be non-linearly optimized again to minimize the target deviation. After minimizing the target deviation, each remaining optimized position information has corresponding final optimized position information after nonlinear optimization, and the final optimized position information obtained after nonlinear optimization is used as the historical state information of the target object, according to the target The historical state information of the object can predict the state information of the target object, such as the position, speed, and orientation of the target object.
由于可以根据目标对象的历史状态信息对目标对象的状态信息进行预测，即使目标对象不位于画面中心时无法通过测距装置获得目标距离，无法确定目标对象的位置，或者，当成像装置的画面中存在障碍物遮挡目标对象，视觉跟踪丢失后无法确定目标对象的位置，通过本公开的实施例，也可以根据目标对象运动的连续性以及历史状态信息，推算缺少有效测距结果的检测框对应的目标对象的距离，提高了目标对象的状态信息的可用率。Since the state information of the target object can be predicted based on the historical state information of the target object, even if the target object is not located in the center of the screen, the distance to the target cannot be obtained by the distance measuring device, the position of the target object cannot be determined, or when the image of the imaging device is There are obstacles occluding the target object, and the position of the target object cannot be determined after the visual tracking is lost. Through the embodiments of the present disclosure, the continuity of the motion of the target object and the historical state information can also be used to calculate the corresponding detection frame lacking effective ranging results. The distance of the target object improves the availability of the status information of the target object.
根据本公开的实施例，第一偏差阈值和第二偏差阈值可以根据经验预先设定。According to an embodiment of the present disclosure, the first deviation threshold and the second deviation threshold may be preset based on experience.
根据本公开的实施例，进一步地，还可以将异常的检测框信息和/或异常的有效测距结果进行滤除。According to the embodiments of the present disclosure, further, abnormal detection frame information and/or abnormal effective ranging results can also be filtered out.
通过本公开的实施例，能够识别并过滤异常的目标检测框，避免异常的观测导致目标对象的状态估计失效，提高了状态估计的可靠性。并且，能够识别并过滤异常的测距结果，避免异常的观测导致目标对象的状态估计失效，提高了状态估计的可靠性。Through the embodiments of the present disclosure, abnormal target detection frames can be identified and filtered, and abnormal observations can prevent the state estimation of the target object from failing, and the reliability of the state estimation can be improved. In addition, abnormal ranging results can be identified and filtered, and abnormal observations can prevent the state estimation of the target object from failing, and the reliability of the state estimation can be improved.
通过本公开的实施例，可以对目标对象的历史运动轨迹进行优化，然后基于优化后的运动轨迹对目标对象的运动轨迹进行预测。Through the embodiments of the present disclosure, the historical motion trajectory of the target object can be optimized, and then the motion trajectory of the target object can be predicted based on the optimized motion trajectory.
根据本公开的实施例，每次获得一个新的检测框后，如果检测框无效或异常，可以在历史运动轨迹估计数据的基础上，进行目标对象的轨迹估计。其中，无效的检测框可以包括指对当前图像进行识别后，得到的不包含目标对象的检测框。According to the embodiments of the present disclosure, every time a new detection frame is obtained, if the detection frame is invalid or abnormal, the trajectory of the target object can be estimated on the basis of historical motion trajectory estimation data. Wherein, the invalid detection frame may include a detection frame that does not contain the target object obtained after recognizing the current image.
根据本公开的实施例，在获得的图像中未识别到目标对象的情况下，可以确定目标对象丢失时的位置信息；然后根据目标对象丢失时的位置信息和平滑后的关于目标对象的运动轨迹，对目标对象的状态信息进行预测。According to an embodiment of the present disclosure, when the target object is not recognized in the obtained image, the position information of the target object when it is lost can be determined; then according to the position information when the target object is lost and the smoothed motion trajectory of the target object , Predict the status information of the target object.
根据本公开的实施例，对目标对象的状态信息进行预测可以是根据目标对象丢失时的位置信息和平滑后的关于目标对象的运动轨迹，生成关于目标对象的预测位置的概率分布。According to an embodiment of the present disclosure, predicting the state information of the target object may be based on the position information when the target object is lost and the smoothed motion trajectory about the target object to generate a probability distribution about the predicted position of the target object.
图13示意性示出了根据本公开实施例的对目标对象的位置信息进行预测的示意图。FIG. 13 schematically shows a schematic diagram of predicting the position information of a target object according to an embodiment of the present disclosure.
目标对象在t 0时刻的丢失位置如图13所示。根据本公开的实施例，目标对象的预测位置的概率分布的空间占比随着目标对象的丢失时间的增长而增加。如图13所示，目标对象在t 1时刻的预测位置的概率分布的空间占比小于在t 2时刻的预测位置的概率分布的空间占比，目标对象在t 2时刻的预测位置的概率分布的空间占比小于在t 3时刻的预测位置的概率分布的空间占比。随着丢失时间的增长，椭圆中心代表目标对象的预测位置不断偏离目标对象丢失的位置，椭圆面积代表的预测误差范围也不断增加。 The lost position of the target object at time t 0 is shown in Figure 13. According to an embodiment of the present disclosure, the spatial proportion of the probability distribution of the predicted position of the target object increases as the loss time of the target object increases. As shown in Figure 13, the spatial proportion of the probability distribution of the predicted position of the target object at t 1 is smaller than the spatial proportion of the probability distribution of the predicted position at t 2 , and the probability distribution of the predicted position of the target object at t 2 the proportion is less than the predicted spatial position of the time t 3 of the spatial probability distribution proportion. As the loss time increases, the predicted position of the ellipse center representing the target object keeps deviating from the lost position of the target object, and the prediction error range represented by the area of the ellipse continues to increase.
根据本公开的实施例，当目标对象丢失后，能够根据目标对象的历史轨迹信息，预测目标对象在丢失后任意时间点的位置与位置误差，提高了目标对象的状态估计的连续性。According to the embodiments of the present disclosure, when the target object is lost, the position and position error of the target object at any point in time after the loss can be predicted based on the historical trajectory information of the target object, which improves the continuity of the state estimation of the target object.
根据本公开的实施例，预测位置的概率分布的空间占比的变化参数与目标对象的类型相关，对于不同种类的目标对象可以采用不同的变化参数。根据本公开的实施例，变化参数包括预测位置的概率分布的空间占比的增长速度。According to the embodiment of the present disclosure, the variation parameter of the spatial proportion of the probability distribution of the predicted position is related to the type of the target object, and different variation parameters may be adopted for different types of target objects. According to an embodiment of the present disclosure, the change parameter includes the growth rate of the spatial proportion of the probability distribution of the predicted position.
根据本公开的实施例，在目标对象的类型为生物的情况下，预测位置的概率分布的空间占比的第一增长速度在不同方向上相同。例如，生物可以是人，狗，马等等。According to an embodiment of the present disclosure, in a case where the type of the target object is a living thing, the first growth rate of the spatial proportion of the probability distribution of the predicted position is the same in different directions. For example, the creature can be a human, a dog, a horse, and so on.
根据本公开的实施例，在目标对象的类型为移动设备的情况下，预测位置的概率分布的空间占比的第二增长速度沿移动设备的运动方向增加。例如，移动设备可以是汽车，火车，船等等。According to an embodiment of the present disclosure, in a case where the type of the target object is a mobile device, the second increase speed of the spatial proportion of the probability distribution of the predicted position increases along the movement direction of the mobile device. For example, mobile devices can be cars, trains, boats, and so on.
根据本公开的实施例，第一增长速度小于第二增长速度。具体地，若目标对象为人，则预测误差的增长速度较慢，且各方向的增长速度可以相等。若目标对象为车、船，则预测误差的增长速度较快，且预测误差的增长方向主要集中于目标运动方向。According to an embodiment of the present disclosure, the first growth rate is less than the second growth rate. Specifically, if the target object is a human, the growth rate of the prediction error is relatively slow, and the growth rate in each direction can be equal. If the target object is a car or a boat, the growth rate of the prediction error is faster, and the growth direction of the prediction error is mainly concentrated in the direction of the target movement.
根据本公开的实施例，还提供了另一种目标对象的状态信息确定方法，图14示意性示出了根据本公开另一实施例的目标对象的状态信息确定方法的流程图。According to an embodiment of the present disclosure, another method for determining status information of a target object is also provided. FIG. 14 schematically shows a flowchart of a method for determining status information of a target object according to another embodiment of the present disclosure.
需要说明的是，本公开实施例中的流程图所示的操作除非明确说明不同操作之间存在执行的先后顺序，或者不同操作在技术实现上存在执行的先后顺序，否则，多个操作之间的执行顺序可以不分先后，多个操作也可 以同时执行。It should be noted that the operations shown in the flowchart in the embodiments of the present disclosure, unless it is clearly stated that there is a sequence of execution between different operations, or there is a sequence of execution of different operations in technical implementation, otherwise, between multiple operations The order of execution can be in no particular order, and multiple operations can also be executed at the same time.
另外，本实施例提供的目标对象的状态信息确定方法可以参考部分或全部上一实施例提供的目标对象的状态信息确定方法中的描述。具体地，涉及相同或类似的技术方案可以参考对上述附图4～13的描述，在此不再赘述。In addition, the method for determining the status information of the target object provided in this embodiment may refer to the description in part or all of the method for determining the status information of the target object provided in the previous embodiment. Specifically, for the same or similar technical solutions, reference may be made to the description of the above-mentioned Figures 4 to 13, which will not be repeated here.
如图14所示，该目标对象的状态信息确定方法包括操作S1401～S1405。As shown in FIG. 14, the method for determining the status information of the target object includes operations S1401 to S1405.
在操作S1401，通过可移动平台携带的成像装置获得关于目标对象的多帧图像。In operation S1401, multiple frames of images about the target object are obtained through the imaging device carried by the movable platform.
在操作S1402，对多帧图像中的每一帧图像进行识别，得到关于目标对象的多个检测框信息。In operation S1402, each of the multiple frames of images is recognized to obtain multiple detection frame information about the target object.
根据本公开的实施例，关于操作S1401和操作S1402的描述可以参考上述对图3的相关描述，在此不再赘述。According to an embodiment of the present disclosure, for the description of operation S1401 and operation S1402, reference may be made to the above-mentioned related description of FIG. 3, which will not be repeated here.
在操作S1403，确定多个检测框信息中满足预设条件的一个或多个目标检测框信息。In operation S1403, one or more target detection frame information satisfying a preset condition among the plurality of detection frame information is determined.
根据本公开的实施例，在得到关于目标对象的多个检测框信息之后，可以对多个检测框信息进行筛选，从多个检测框信息中确定满足预设条件的一个或多个目标检测框信息。According to the embodiments of the present disclosure, after multiple detection frame information about the target object is obtained, the multiple detection frame information can be screened, and one or more target detection frames satisfying preset conditions can be determined from the multiple detection frame information. information.
根据本公开的实施例，预设条件可以是用于判断检测框信息是否为异常检测框的条件。例如，预设条件可以是用于判断检测框信息的采样时间是否异常的条件，或者，预设条件还可以是用于判断基于检测框信息计算得到的关于目标对象的物理估计尺寸是否异常的条件等等。According to an embodiment of the present disclosure, the preset condition may be a condition for judging whether the detection frame information is an abnormal detection frame. For example, the preset condition can be a condition for judging whether the sampling time of the detection frame information is abnormal, or the preset condition can also be a condition for judging whether the physical estimated size of the target object calculated based on the detection frame information is abnormal. and many more.
在操作S1404，获得可移动平台与目标对象之间的多个测距结果。In operation S1404, a plurality of ranging results between the movable platform and the target object are obtained.
根据本公开的实施例，关于操作S1404的描述可以参考上述对图3的相关描述，在此不再赘述。According to an embodiment of the present disclosure, for the description of operation S1404, reference may be made to the above-mentioned related description of FIG. 3, which will not be repeated here.
在操作S1405，根据一个或多个目标检测框信息和多个测距结果确定目标对象的状态信息。In operation S1405, state information of the target object is determined according to one or more target detection frame information and a plurality of ranging results.
通过本公开的实施例，通过确定多个检测框信息中满足预设条件的一个或多个目标检测框信息，根据一个或多个目标检测框信息和多个测距结果确定目标对象的状态信息，至少部分解决了错误数据污染目标对象的状态信息而导致目标对象的状态估计失效的技术问题，提高了状态信息估计 的可靠性。通过本公开的实施例，由于可以对检测框进行筛选，可以解决将不属于当前目标对象的画面位置(例如与目标对象相似度较高的对象)误检测为当前目标对象的位置时，无法过滤错误的目标检测框，导致目标对象的位置估计出现偏差的技术问题。Through the embodiments of the present disclosure, by determining one or more target detection frame information satisfying preset conditions among the plurality of detection frame information, the state information of the target object is determined according to the one or more target detection frame information and the multiple ranging results , At least partially solves the technical problem that the state information of the target object is contaminated by the wrong data, which causes the state estimation of the target object to fail, and improves the reliability of the state information estimation. Through the embodiments of the present disclosure, since the detection frame can be filtered, it can be solved that when a screen position that does not belong to the current target object (for example, an object with a high similarity to the target object) is mistakenly detected as the position of the current target object, it cannot be filtered. The wrong target detection frame causes the technical problem of deviation in the estimation of the target object's position.
根据本公开的实施例，根据一个或多个目标检测框信息和多个测距结果确定目标对象的状态信息，包括：确定每个目标检测框信息对应的测距结果；以及根据每个目标检测框信息和每个目标检测框信息对应的测距结果，确定目标对象的状态信息。According to an embodiment of the present disclosure, determining the state information of the target object according to one or more target detection frame information and multiple ranging results includes: determining the ranging result corresponding to each target detection frame information; and detecting according to each target The frame information and the ranging result corresponding to each target detection frame information determine the status information of the target object.
根据本公开的实施例，目标对象的状态信息包括位置信息；其中，确定多个检测框信息中满足预设条件的一个或多个目标检测框信息，包括：根据每个检测框信息对应的采样时间点，对多个检测框信息进行筛选，得到一个或多个目标检测框信息。According to an embodiment of the present disclosure, the state information of the target object includes position information; wherein, determining one or more target detection frame information that meets a preset condition among the plurality of detection frame information includes: sampling according to each detection frame information At the time point, multiple detection frame information is screened to obtain one or more target detection frame information.
根据本公开的实施例，根据每个检测框信息对应的采样时间点，对多个检测框信息进行筛选，包括：按照时间顺序，依次对每个检测框信息进行筛选。According to an embodiment of the present disclosure, filtering multiple detection frame information according to the sampling time point corresponding to each detection frame information includes: sequentially filtering each detection frame information in a time sequence.
根据本公开的实施例，按照时间顺序，依次对每个检测框信息进行筛选，包括：计算当前正筛选的检测框信息和与当前正筛选的检测框信息的采样时间相邻的后一个检测框信息的采样时间点之间的时间差；将时间差与状态变量阈值进行比较；如果时间差小于状态变量阈值，滤除当前正筛选的检测框信息；以及如果时间差大于或等于状态变量阈值，保留当前正筛选的检测框信息，其中，被保留的当前正筛选的检测框信息为目标检测框信息。According to the embodiment of the present disclosure, each detection frame information is sequentially filtered in chronological order, including: calculating the detection frame information currently being screened and the next detection frame adjacent to the sampling time of the detection frame information currently being screened The time difference between the sampling time points of the information; compare the time difference with the state variable threshold; if the time difference is less than the state variable threshold, filter out the detection box information currently being screened; and if the time difference is greater than or equal to the state variable threshold, keep the current positive screening The detection frame information of, where the retained detection frame information currently being screened is the target detection frame information.
根据本公开的实施例，按照时间顺序，依次对每个检测框信息进行筛选可以参考上述图11中对每个目标检测框信息对应的关于目标对象的初始位置信息进行筛选的介绍过程。According to the embodiment of the present disclosure, in order to filter each detection frame information in sequence, refer to the introduction process of screening the initial position information of the target object corresponding to each target detection frame information in FIG. 11.
根据本公开的实施例，例如，选取出过去一段时间范围内的检测框box m…box n-1、box n，这些检测框的采样时间分别为t m…t n-1、t n。将检测框box m…box n-1、box n作为待筛选状态变量。 According to the present embodiment of the present disclosure, e.g., selecting a detection frame box m over a period of time ... box n-1, box n , the sampling timing of these detection frames are t m ... t n-1, t n. Use the detection boxes box m …box n-1 and box n as the state variables to be filtered.
针对每个待筛选状态变量，计算其与后一个待筛选状态变量的时间差，从所有待筛选状态变量中，剔除与后一个待筛选状态变量的时间差小于状 态变量阈值的状态变量。后一个待筛选状态变量是与当前正筛选的检测框信息的采样时间相邻的后一个检测框信息。For each state variable to be screened, the time difference between it and the next state variable to be screened is calculated, and from all state variables to be screened, the state variable whose time difference with the next state variable to be screened is less than the state variable threshold is eliminated. The next state variable to be screened is the next detection frame information adjacent to the sampling time of the detection frame information currently being screened.
根据本公开的实施例，状态变量阈值可以是预先设定的固定值，例如，根据经验预先设定。According to an embodiment of the present disclosure, the state variable threshold may be a fixed value set in advance, for example, set in advance based on experience.
根据本公开的实施例，状态变量阈值也可以随确定的目标检测框信息而发生变化。According to an embodiment of the present disclosure, the state variable threshold may also change with the determined target detection frame information.
根据本公开的实施例，状态变量阈值为当前正筛选的检测框信息和与当前正筛选的检测框信息的采样时间相邻的目标检测框信息的采样时间点之间的时间差。According to an embodiment of the present disclosure, the state variable threshold is the time difference between the detection frame information currently being screened and the sampling time point of the target detection frame information adjacent to the sampling time of the detection frame information currently being screened.
根据本公开的实施例，具体地，状态变量阈值随确定的目标检测框信息而发生变化可以参考上述图12的描述，在此不再赘述。According to the embodiment of the present disclosure, specifically, the state variable threshold changes with the determined target detection frame information can refer to the description of FIG. 12 above, which will not be repeated here.
根据本公开的实施例，确定多个检测框信息中满足预设条件的多个目标检测框信息，包括：根据多个检测框信息确定目标对象是否出现满足预设条件的移动；在目标对象出现了满足预设条件的移动的情况下，将目标对象满足预设条件的移动时对应的检测框信息确定为目标检测框信息。According to an embodiment of the present disclosure, determining multiple target detection frame information that meets a preset condition among the multiple detection frame information includes: determining whether the target object moves that meets the preset condition according to the multiple detection frame information; when the target object appears In the case of movement that satisfies the preset condition, the detection frame information corresponding to the movement of the target object that satisfies the preset condition is determined as the target detection frame information.
根据本公开的实施例，根据多个检测框信息确定目标对象是否出现满足预设条件的移动，包括：针对多个检测框信息中任意相邻的第一检测框信息和第二检测框信息，获得第一检测框信息采集时的成像装置的第一状态信息，和第二检测框信息采集时的成像装置的第二状态信息；根据第一检测框信息对应的测距结果和第一状态信息，确定与第一检测框信息对应的关于目标对象的初始位置信息的第一概率分布；根据第二检测框信息对应的测距结果和第二状态信息，确定与第二检测框信息对应的关于目标对象的初始位置信息的第二概率分布；以及根据第一概率分布和第二概率分布，确定目标对象是否出现满足预设条件的移动。According to an embodiment of the present disclosure, determining whether a target object has moved that meets a preset condition according to multiple detection frame information includes: for any adjacent first detection frame information and second detection frame information among the multiple detection frame information, Obtain the first state information of the imaging device when the first detection frame information is collected, and the second state information of the imaging device when the second detection frame information is collected; according to the ranging result and the first state information corresponding to the first detection frame information , Determine the first probability distribution of the initial position information of the target object corresponding to the first detection frame information; determine the information corresponding to the second detection frame information according to the ranging result and the second state information corresponding to the second detection frame information A second probability distribution of the initial position information of the target object; and according to the first probability distribution and the second probability distribution, it is determined whether the target object moves that meets a preset condition.
根据本公开的实施例，根据第一概率分布和第二概率分布，确定目标对象是否出现满足预设条件的移动，包括：根据第一概率分布和第二概率分布，确定概率密度最高的空间位置；计算概率密度最高的空间位置距离第一概率分布的第一概率分布中心位置的第一距离；计算概率密度最高的空间位置距离第二概率分布的第二概率分布中心位置的第二距离；根据第一距离和第二距离确定第一检测框信息和第二检测框信息之间的概率距 离；以及如果概率距离大于或等于预设阈值，确定目标对象出现满足预设条件的移动。According to an embodiment of the present disclosure, according to the first probability distribution and the second probability distribution, determining whether the target object has moved that meets a preset condition includes: determining the spatial position with the highest probability density according to the first probability distribution and the second probability distribution ; Calculate the first distance between the spatial position with the highest probability density and the center position of the first probability distribution of the first probability distribution; calculate the second distance between the spatial position with the highest probability density and the center position of the second probability distribution of the second probability distribution; according to The first distance and the second distance determine the probability distance between the first detection frame information and the second detection frame information; and if the probability distance is greater than or equal to a preset threshold, it is determined that the target object moves that meets the preset condition.
根据本公开的实施例，根据多个检测框信息确定目标对象是否出现满足预设条件的移动，可以参考对上述图6至图10的描述，在此不再赘述。According to the embodiment of the present disclosure, to determine whether the target object has moved that meets a preset condition according to multiple detection frame information, reference may be made to the description of FIGS. 6 to 10 above, which will not be repeated here.
根据本公开的实施例，根据一个或多个目标检测框信息和多个测距结果确定目标对象的状态信息，包括：对一个或多个目标检测框信息对应的有效初始位置信息进行优化，以平滑关于目标对象的运动轨迹。According to an embodiment of the present disclosure, determining the state information of the target object according to one or more target detection frame information and multiple ranging results includes: optimizing the effective initial position information corresponding to the one or more target detection frame information to Smooth the trajectory of the target object.
根据本公开的实施例，对一个或多个目标检测框信息对应的有效初始位置信息进行优化，包括：对一个或多个目标检测框信息对应的有效初始位置信息进行非线性优化，以最小化目标偏差，其中，目标偏差与检测框信息和/或测距结果相关，每个有效初始位置信息进行非线性优化后具有对应的优化位置信息。According to an embodiment of the present disclosure, optimizing the effective initial position information corresponding to one or more target detection frame information includes: performing nonlinear optimization on the effective initial position information corresponding to one or more target detection frame information to minimize The target deviation, where the target deviation is related to the detection frame information and/or the ranging result, and each effective initial position information has corresponding optimized position information after nonlinear optimization.
根据本公开的实施例，目标偏差包括第一偏差和/或第二偏差；第一偏差包括关于有效初始位置信息与用于计算得到有效初始位置信息的目标检测框信息和测距结果之间的观测偏差；第二偏差包括关于多个有效初始位置信息中的相邻有效初始位置信息之间的平滑程度与先验值的偏差。According to an embodiment of the present disclosure, the target deviation includes a first deviation and/or a second deviation; the first deviation includes the difference between the effective initial position information and the target detection frame information used to obtain the effective initial position information and the ranging result. Observation deviation; the second deviation includes a deviation between the smoothness of adjacent effective initial position information in the plurality of effective initial position information and the prior value.
根据本公开的实施例，第一偏差通过用于计算得到有效初始位置信息的目标检测框信息和测距结果的概率密度函数进行表征。According to the embodiment of the present disclosure, the first deviation is characterized by the probability density function of the target detection frame information and the ranging result used to calculate the effective initial position information.
根据本公开的实施例，对一个或多个目标检测框信息对应的有效初始位置信息进行优化，还包括：确定每个有效初始位置信息进行非线性优化后对应的优化位置信息是否异常；以及将异常的优化位置信息进行滤除。According to an embodiment of the present disclosure, optimizing the effective initial position information corresponding to one or more target detection frame information further includes: determining whether the optimized position information corresponding to each effective initial position information after nonlinear optimization is abnormal; and Abnormal optimized location information is filtered out.
根据本公开的实施例，目标偏差包括第一偏差，第一偏差包括第一子偏差和/或第二子偏差；第一子偏差为关于有效初始位置信息与用于计算得到有效初始位置信息的目标检测框信息之间的观测偏差；第二子偏差为关于有效初始位置信息与用于计算得到有效初始位置信息的测距结果之间的观测偏差。According to an embodiment of the present disclosure, the target deviation includes a first deviation, and the first deviation includes a first sub-deviation and/or a second sub-deviation; Observation deviation between target detection frame information; the second sub-deviation is the observation deviation between the effective initial position information and the ranging result used to calculate the effective initial position information.
其中，确定每个有效初始位置信息进行非线性优化后对应的优化位置信息是否异常，包括：若确定第一子偏差对应的检测框信息异常，和/或确定第二子偏差对应的测距结果异常，则将异常的检测框信息和/或异常的测距结果对应的优化位置确定为异常的优化位置。Wherein, determining whether the optimized position information corresponding to each effective initial position information after nonlinear optimization is abnormal includes: if it is determined that the detection frame information corresponding to the first sub-deviation is abnormal, and/or determining the ranging result corresponding to the second sub-deviation If it is abnormal, the optimized position corresponding to the abnormal detection frame information and/or the abnormal ranging result is determined as the abnormal optimized position.
根据本公开的实施例，对一个或多个目标检测框信息对应的有效初始位置信息进行优化，还包括：在将异常的优化位置信息进行滤除之后，对剩余的优化位置信息进行非线性优化，以最小化目标偏差，其中，每个剩余的优化位置信息进行非线性优化后具有对应的最终优化位置信息。According to an embodiment of the present disclosure, optimizing the effective initial position information corresponding to one or more target detection frame information further includes: after filtering out abnormal optimized position information, performing nonlinear optimization on the remaining optimized position information , In order to minimize the target deviation, where each remaining optimized position information has corresponding final optimized position information after nonlinear optimization.
根据本公开的实施例，在获得的图像中未识别到目标对象的情况下，确定目标对象丢失时的位置信息；以及根据目标对象丢失时的位置信息和平滑后的关于目标对象的运动轨迹，对目标对象的状态信息进行预测。According to an embodiment of the present disclosure, when the target object is not recognized in the obtained image, the position information when the target object is lost is determined; and according to the position information when the target object is lost and the smoothed motion trajectory about the target object, Predict the status information of the target object.
根据本公开的实施例，根据目标对象丢失时的位置信息和平滑后的关于目标对象的运动轨迹，对目标对象的状态信息进行预测，包括：根据目标对象丢失时的位置信息和平滑后的关于目标对象的运动轨迹，生成关于目标对象的预测位置的概率分布。According to the embodiments of the present disclosure, the state information of the target object is predicted based on the position information when the target object is lost and the smoothed motion trajectory about the target object, including: according to the position information when the target object is lost and the smoothed information about the target object The motion trajectory of the target object generates a probability distribution about the predicted position of the target object.
根据本公开的实施例，预测位置的概率分布的空间占比随着目标对象的丢失时间的增长而增加。According to an embodiment of the present disclosure, the spatial proportion of the probability distribution of the predicted position increases as the loss time of the target object increases.
根据本公开的实施例，预测位置的概率分布的空间占比的变化参数与目标对象的类型相关。According to an embodiment of the present disclosure, the variation parameter of the spatial proportion of the probability distribution of the predicted position is related to the type of the target object.
根据本公开的实施例，变化参数包括预测位置的概率分布的空间占比的增长速度，预测位置的概率分布的空间占比的变化参数与目标对象的类型相关包括：在目标对象的类型为生物的情况下，预测位置的概率分布的空间占比的第一增长速度在不同方向上相同；在目标对象的类型为移动设备的情况下，预测位置的概率分布的空间占比的第二增长速度沿移动设备的运动方向增加。According to an embodiment of the present disclosure, the change parameter includes the growth rate of the space proportion of the probability distribution of the predicted position, and the change parameter of the space proportion of the predicted position probability distribution is related to the type of the target object, including: In the case of the predicted position, the first growth rate of the space proportion of the probability distribution of the predicted position is the same in different directions; when the target object type is a mobile device, the second growth rate of the space proportion of the predicted position’s probability distribution Increase in the direction of movement of the mobile device.
根据本公开的实施例，第一增长速度小于第二增长速度。According to an embodiment of the present disclosure, the first growth rate is less than the second growth rate.
根据本公开的实施例，关于对目标对象的状态信息进行预测的过程可以参考对上述图13的描述，在此不再赘述。According to the embodiment of the present disclosure, for the process of predicting the state information of the target object, reference may be made to the description of the above-mentioned FIG. 13, which will not be repeated here.
根据本公开的实施例，根据一个或多个目标检测框信息和多个测距结果确定目标对象的状态信息，包括：根据一个或多个目标检测框信息从多个测距结果中筛选出有效测距结果；以及根据一个或多个目标检测框信息和有效测距结果确定目标对象的状态信息。According to an embodiment of the present disclosure, determining the status information of the target object according to one or more target detection frame information and multiple ranging results includes: filtering out valid from the multiple ranging results according to the one or more target detection frame information Ranging results; and determining the status information of the target object based on one or more target detection frame information and effective ranging results.
根据本公开的实施例，根据一个或多个目标检测框信息从多个测距结果中筛选出有效测距结果，包括：确定每个目标检测框信息对应的一个或 多个测距结果；以及对每个目标检测框信息对应的一个或多个测距结果进行筛选。According to an embodiment of the present disclosure, filtering effective ranging results from multiple ranging results according to one or more target detection frame information includes: determining one or more ranging results corresponding to each target detection frame information; and Screen one or more ranging results corresponding to each target detection frame information.
根据本公开的实施例，确定每个目标检测框信息对应的一个或多个测距结果，包括：根据每个目标检测框信息的采样时间点和多个测距结果中每个测距结果的采样时间点确定每个目标检测框信息对应的一个或多个测距结果。According to an embodiment of the present disclosure, determining one or more ranging results corresponding to each target detection frame information includes: according to the sampling time point of each target detection frame information and the value of each ranging result in the multiple ranging results The sampling time point determines one or more ranging results corresponding to each target detection frame information.
根据本公开的实施例，测距结果包括激光测距结果，对每个目标检测框信息对应的一个或多个测距结果进行筛选，包括：确定一个或多个测距结果中每个测距结果对应的激光光斑；根据每个测距结果对应的目标检测框信息和每个测距结果对应的激光光斑，确定每个测距结果的有效性；以及根据每个测距结果的有效性对每个目标检测框信息对应的一个或多个测距结果进行筛选。According to an embodiment of the present disclosure, the ranging result includes a laser ranging result, and screening one or more ranging results corresponding to each target detection frame information includes: determining each ranging result in the one or more ranging results The laser spot corresponding to the result; according to the target detection frame information corresponding to each ranging result and the laser spot corresponding to each ranging result, the validity of each ranging result is determined; and the validity of each ranging result is checked One or more ranging results corresponding to each target detection frame information are screened.
根据本公开的实施例，根据每个测距结果对应的目标检测框信息和每个测距结果对应的激光光斑，确定每个测距结果的有效性，包括：确定每个测距结果对应的目标检测框信息和每个测距结果对应的激光光斑的面积重合率；以及根据面积重合率确定每个测距结果的有效性。According to the embodiments of the present disclosure, according to the target detection frame information corresponding to each ranging result and the laser spot corresponding to each ranging result, determining the validity of each ranging result includes: determining the corresponding The target detection frame information and the area coincidence rate of the laser spot corresponding to each distance measurement result; and the validity of each distance measurement result is determined according to the area coincidence ratio.
根据本公开的实施例，根据每个测距结果对应的目标检测框信息和每个测距结果对应的激光光斑，确定每个测距结果的有效性可以参考上述对图5的描述，在此不再赘述。According to the embodiment of the present disclosure, according to the target detection frame information corresponding to each ranging result and the laser spot corresponding to each ranging result, the validity of each ranging result can be determined with reference to the above description of FIG. 5, here No longer.
根据本公开的实施例，根据面积重合率确定每个测距结果的有效性，包括：将面积重合率与预设比例阈值进行比较；将面积重合率大于或等于预设比例阈值的测距结果确定为有效测距结果；以及将面积重合率小于预设比例阈值的测距结果确定为无效测距结果。According to an embodiment of the present disclosure, determining the validity of each ranging result according to the area coincidence rate includes: comparing the area coincidence rate with a preset ratio threshold; and comparing the distance measurement result whose area coincidence rate is greater than or equal to the preset ratio threshold It is determined as a valid ranging result; and the ranging result whose area coincidence rate is less than the preset ratio threshold is determined as an invalid ranging result.
根据本公开的实施例，在确定每个目标检测框信息对应多个测距结果的情况下，根据每个测距结果对应的目标检测框信息和每个测距结果对应的激光光斑，确定每个测距结果的有效性，包括：根据采样时间相邻的两个目标检测框信息，确定采样时间相邻的两个目标检测框信息的采样时间之间的多个测距结果中每个测距结果分别对应的插值目标检测框信息，得到每个测距结果对应的的目标检测框信息；以及根据每个测距结果对应的 激光光斑和与每个测距结果对应的目标检测框信息，确定每个测距结果的有效性。According to an embodiment of the present disclosure, when it is determined that each target detection frame information corresponds to multiple ranging results, each target detection frame information corresponding to each ranging result and the laser spot corresponding to each ranging result are determined. The validity of each ranging result includes: determining each of the multiple ranging results between the sampling times of the two target detection frame information with adjacent sampling times according to the information of the two target detection frames adjacent to the sampling time. The interpolated target detection frame information corresponding to the distance results respectively, to obtain the target detection frame information corresponding to each ranging result; and according to the laser spot corresponding to each ranging result and the target detection frame information corresponding to each ranging result, Determine the validity of each ranging result.
根据本公开的实施例，根据多个目标检测框信息和有效测距结果确定目标对象的状态信息，包括：确定每个目标检测框信息对应的有效测距结果；以及根据每个目标检测框信息和每个目标检测框信息对应的有效测距结果，确定目标对象的状态信息。According to an embodiment of the present disclosure, determining the state information of the target object according to multiple target detection frame information and effective ranging results includes: determining the effective ranging result corresponding to each target detection frame information; and determining the effective ranging result corresponding to each target detection frame information; and according to each target detection frame information The effective ranging result corresponding to each target detection frame information determines the status information of the target object.
根据本公开的实施例，确定每个目标检测框信息对应的有效测距结果，包括：根据每个目标检测框信息的采样时间点和每个有效测距结果的采样时间点，将每个有效测距结果关联至与有效测距结果的采样时间点最接近的目标检测框信息。According to the embodiment of the present disclosure, determining the effective ranging result corresponding to each target detection frame information includes: according to the sampling time point of each target detection frame information and the sampling time point of each effective ranging result, each valid The ranging result is related to the target detection frame information closest to the sampling time point of the effective ranging result.
根据本公开的实施例，在每个目标检测框信息对应多个有效测距结果的情况下，根据每个目标检测框信息和每个目标检测框信息对应的有效测距结果，确定目标对象的状态信息，包括：计算每个目标检测框信息对应的多个有效测距结果的加权平均值，得到每个目标检测框信息对应的目标测距结果；以及根据每个目标检测框信息和每个目标检测框信息对应的目标测距结果，确定目标对象的状态信息。According to the embodiments of the present disclosure, in the case that each target detection frame information corresponds to multiple effective ranging results, the target object's position is determined according to each target detection frame information and the effective ranging result corresponding to each target detection frame information. The status information includes: calculating the weighted average of multiple effective ranging results corresponding to each target detection frame information to obtain the target ranging result corresponding to each target detection frame information; and according to each target detection frame information and each The target ranging result corresponding to the target detection frame information determines the status information of the target object.
根据本公开的实施例，根据一个或多个目标检测框信息和多个测距结果确定目标对象的状态信息，包括：确定每个目标检测框信息对应的测距结果；确定每个目标检测框信息对应的关于目标对象的物理估计尺寸；根据每个目标检测框信息对应的关于目标对象的物理估计尺寸，对每个目标检测框信息对应的测距结果进行筛选；以及根据一个或多个目标检测框信息以及筛选后的测距结果，确定目标对象的状态信息。According to an embodiment of the present disclosure, determining the status information of the target object according to one or more target detection frame information and multiple ranging results includes: determining the ranging result corresponding to each target detection frame information; determining each target detection frame The physical estimated size of the target object corresponding to the information; the ranging result corresponding to each target detection frame information is screened according to the physical estimated size of the target object corresponding to each target detection frame information; and according to one or more targets The detection frame information and the filtered ranging results determine the status information of the target object.
根据本公开的实施例，根据每个目标检测框信息对应的关于目标对象的物理估计尺寸，对每个目标检测框信息对应的测距结果进行筛选，包括：将每个目标检测框信息对应的关于目标对象的物理估计尺寸与预设合理范围进行比较；以及在目标检测框信息对应的关于目标对象的物理估计尺寸与预设合理范围不相符的情况下，滤除目标检测框信息对应的测距结果。According to the embodiment of the present disclosure, according to the physical estimated size of the target object corresponding to each target detection frame information, screening the ranging results corresponding to each target detection frame information includes: The physical estimated size of the target object is compared with the preset reasonable range; and when the physical estimated size of the target object corresponding to the target detection frame information does not match the preset reasonable range, the measurement corresponding to the target detection frame information is filtered out. Distance results.
根据本公开的实施例，确定每个目标检测框信息对应的关于目标对象的物理估计尺寸，包括：根据每个目标检测框信息和每个目标检测框信息采集时的成像装置的视场角，确定每个目标检测框信息对应的视场角；以 及根据每个目标检测框信息对应的视场角和每个目标检测框信息对应的测距结果，确定每个目标检测框信息对应的物理估计尺寸。According to an embodiment of the present disclosure, determining the physical estimated size of the target object corresponding to each target detection frame information includes: according to each target detection frame information and the field angle of the imaging device when each target detection frame information is collected, Determine the field angle corresponding to each target detection frame information; and determine the physical estimation corresponding to each target detection frame information according to the field angle corresponding to each target detection frame information and the distance measurement result corresponding to each target detection frame information size.
根据本公开的实施例，还可以确定目标对象的对象类型，其中，每个对象类型具有对应的预设合理范围。将每个目标检测框信息对应的关于目标对象的物理估计尺寸与预设合理范围进行比较包括：根据目标对象的对象类型确定目标预设合理范围；以及将每个目标检测框信息对应的关于目标对象的物理估计尺寸与目标预设合理范围进行比较。According to the embodiments of the present disclosure, the object type of the target object can also be determined, wherein each object type has a corresponding preset reasonable range. Comparing the physical estimated size of the target object corresponding to each target detection frame information with the preset reasonable range includes: determining the target preset reasonable range according to the object type of the target object; and comparing each target detection frame information corresponding to the target The estimated physical size of the object is compared with the preset reasonable range of the target.
根据本公开的实施例，还提供了一种目标对象的状态信息确定装置，包括：处理器；可读存储介质，用于存储一个或多个程序，其中，当一个或多个程序被处理器执行时，使得处理器执行以下操作：According to an embodiment of the present disclosure, there is also provided an apparatus for determining status information of a target object, including: a processor; a readable storage medium for storing one or more programs, wherein, when one or more programs are executed by the processor When executed, it causes the processor to perform the following operations:
获得通过可移动平台携带的成像装置获得的关于目标对象的多帧图像；对多帧图像中的每一帧图像进行识别，得到关于目标对象的多个检测框信息；获得可移动平台与目标对象之间的多个测距结果；根据多个检测框信息从多个测距结果中筛选出有效测距结果；以及根据多个检测框信息和有效测距结果确定目标对象的状态信息。Obtain multi-frame images of the target object obtained by the imaging device carried by the movable platform; recognize each frame of the multi-frame image to obtain multiple detection frame information about the target object; obtain the movable platform and the target object Multiple ranging results in between; screening effective ranging results from multiple ranging results based on multiple detection frame information; and determining the status information of the target object based on multiple detection frame information and effective ranging results.
根据本公开的实施例，处理器根据多个检测框信息从多个测距结果中筛选出有效测距结果，包括：确定每个检测框信息对应的一个或多个测距结果；以及对每个检测框信息对应的一个或多个测距结果进行筛选。According to an embodiment of the present disclosure, the processor screens out effective ranging results from multiple ranging results according to multiple detection frame information, including: determining one or more ranging results corresponding to each detection frame information; and One or more ranging results corresponding to each detection frame information are filtered.
根据本公开的实施例，处理器确定每个检测框信息对应的一个或多个测距结果，包括：根据每个检测框信息的采样时间点和多个测距结果中每个测距结果的采样时间点确定每个检测框信息对应的一个或多个测距结果。According to an embodiment of the present disclosure, the processor determines one or more ranging results corresponding to each detection frame information, including: according to the sampling time point of each detection frame information and the value of each ranging result in the multiple ranging results The sampling time point determines one or more ranging results corresponding to each detection frame information.
根据本公开的实施例，测距结果包括激光测距结果，处理器对每个检测框信息对应的一个或多个测距结果进行筛选，包括：确定一个或多个测距结果中每个测距结果对应的激光光斑；根据每个测距结果对应的检测框信息和每个测距结果对应的激光光斑，确定每个测距结果的有效性；以及根据每个测距结果的有效性对每个检测框信息对应的一个或多个测距结果进行筛选。According to an embodiment of the present disclosure, the ranging result includes a laser ranging result, and the processor screens one or more ranging results corresponding to each detection frame information, including: determining each of the one or more ranging results The laser spot corresponding to the ranging result; the validity of each ranging result is determined according to the detection frame information corresponding to each ranging result and the laser spot corresponding to each ranging result; and the validity of each ranging result is checked according to the validity of each ranging result One or more ranging results corresponding to each detection frame information are screened.
根据本公开的实施例，处理器根据每个测距结果对应的检测框信息和每个测距结果对应的激光光斑，确定每个测距结果的有效性，包括：确定 每个测距结果对应的检测框信息和每个测距结果对应的激光光斑的面积重合率；以及根据面积重合率确定每个测距结果的有效性。According to an embodiment of the present disclosure, the processor determines the validity of each ranging result according to the detection frame information corresponding to each ranging result and the laser spot corresponding to each ranging result, including: determining that each ranging result corresponds to The detection frame information and the area coincidence rate of the laser spot corresponding to each distance measurement result; and the validity of each distance measurement result is determined according to the area coincidence ratio.
根据本公开的实施例，处理器根据面积重合率确定每个测距结果的有效性，包括：将面积重合率与预设比例阈值进行比较；将面积重合率大于或等于预设比例阈值的测距结果确定为有效测距结果；以及将面积重合率小于预设比例阈值的测距结果确定为无效测距结果。According to an embodiment of the present disclosure, the processor determines the validity of each ranging result according to the area coincidence rate, including: comparing the area coincidence rate with a preset ratio threshold; The distance result is determined as a valid distance measurement result; and the distance measurement result whose area coincidence rate is less than the preset ratio threshold is determined as an invalid distance measurement result.
根据本公开的实施例，在确定每个检测框信息对应多个测距结果的情况下，处理器根据每个测距结果对应的检测框信息和每个测距结果对应的激光光斑，确定每个测距结果的有效性，包括：根据采样时间相邻的两个检测框信息，确定采样时间相邻的两个检测框信息的采样时间之间的多个测距结果中每个测距结果分别对应的插值检测框信息，得到每个测距结果对应的的检测框信息；以及根据每个测距结果对应的激光光斑和与每个测距结果对应的检测框信息，确定每个测距结果的有效性。According to an embodiment of the present disclosure, in the case of determining that each detection frame information corresponds to multiple ranging results, the processor determines each detection frame information corresponding to each ranging result and the laser spot corresponding to each ranging result. The validity of each ranging result includes: determining each of the multiple ranging results between the sampling times of the two adjacent detection frames at the sampling time according to the information of the two adjacent detection frames at the sampling time Corresponding interpolation detection frame information to obtain the detection frame information corresponding to each ranging result; and determine each ranging result according to the laser spot corresponding to each ranging result and the detection frame information corresponding to each ranging result The validity of the results.
根据本公开的实施例，处理器根据多个检测框信息和有效测距结果确定目标对象的状态信息，包括：确定每个检测框信息对应的有效测距结果；以及根据每个检测框信息和每个检测框信息对应的有效测距结果，确定目标对象的状态信息。According to an embodiment of the present disclosure, the processor determines the state information of the target object according to multiple detection frame information and effective ranging results, including: determining the effective ranging result corresponding to each detection frame information; and according to each detection frame information and The effective ranging result corresponding to each detection frame information determines the status information of the target object.
根据本公开的实施例，处理器确定每个检测框信息对应的有效测距结果，包括：根据每个检测框信息的采样时间点和每个有效测距结果的采样时间点，将每个有效测距结果关联至与有效测距结果的采样时间点最接近的检测框信息。According to the embodiment of the present disclosure, the processor determining the effective ranging result corresponding to each detection frame information includes: according to the sampling time point of each detection frame information and the sampling time point of each effective ranging result, the effective The ranging result is related to the detection frame information closest to the sampling time point of the effective ranging result.
根据本公开的实施例，在每个检测框信息对应多个有效测距结果的情况下，处理器根据每个检测框信息和每个检测框信息对应的有效测距结果，确定目标对象的状态信息，包括：计算每个检测框信息对应的多个有效测距结果的加权平均值，得到每个检测框信息对应的目标测距结果；以及根据每个检测框信息和每个检测框信息对应的目标测距结果，确定目标对象的状态信息。According to an embodiment of the present disclosure, in the case that each detection frame information corresponds to multiple valid ranging results, the processor determines the state of the target object according to each detection frame information and the effective ranging result corresponding to each detection frame information Information, including: calculating the weighted average of multiple effective ranging results corresponding to each detection frame information to obtain the target ranging result corresponding to each detection frame information; and corresponding to each detection frame information and each detection frame information The target ranging result is determined to determine the status information of the target object.
根据本公开的实施例，处理器还执行以下操作：确定每个检测框信息对应的有效测距结果；确定每个检测框信息对应的关于目标对象的物理估 计尺寸；以及根据每个检测框信息对应的关于目标对象的物理估计尺寸，对每个检测框信息对应的有效测距结果进行筛选。According to an embodiment of the present disclosure, the processor further performs the following operations: determining the effective ranging result corresponding to each detection frame information; determining the physical estimated size of the target object corresponding to each detection frame information; and according to each detection frame information Corresponding to the physical estimated size of the target object, the effective ranging result corresponding to each detection frame information is screened.
根据本公开的实施例，处理器根据每个检测框信息对应的关于目标对象的物理估计尺寸，对每个检测框信息对应的有效测距结果进行筛选，包括：将每个检测框信息对应的关于目标对象的物理估计尺寸与预设合理范围进行比较；以及在检测框信息对应的关于目标对象的物理估计尺寸与预设合理范围不相符的情况下，滤除检测框信息对应的有效测距结果。According to an embodiment of the present disclosure, the processor screens the effective ranging result corresponding to each detection frame information according to the physical estimated size of the target object corresponding to each detection frame information, including: Compare the physical estimated size of the target object with the preset reasonable range; and filter out the effective ranging corresponding to the detection frame information when the physical estimated size of the target object corresponding to the detection frame information does not match the preset reasonable range result.
根据本公开的实施例，处理器确定每个检测框信息对应的关于目标对象的物理估计尺寸，包括：根据每个检测框信息和每个检测框信息采集时的成像装置的视场角，确定每个检测框信息对应的视场角；以及根据每个检测框信息对应的视场角和每个检测框信息对应的有效测距结果，确定每个检测框信息对应的物理估计尺寸。According to an embodiment of the present disclosure, the processor determining the physical estimated size of the target object corresponding to each detection frame information includes: determining, according to each detection frame information and the field angle of the imaging device when each detection frame information is collected The field angle corresponding to each detection frame information; and the physical estimated size corresponding to each detection frame information is determined according to the field angle corresponding to each detection frame information and the effective ranging result corresponding to each detection frame information.
根据本公开的实施例，处理器还执行以下操作：确定目标对象的对象类型，其中，每个对象类型具有对应的预设合理范围；将每个检测框信息对应的关于目标对象的物理估计尺寸与预设合理范围进行比较，包括：根据目标对象的对象类型确定目标预设合理范围；以及将每个检测框信息对应的关于目标对象的物理估计尺寸与目标预设合理范围进行比较。According to an embodiment of the present disclosure, the processor further performs the following operations: determining the object type of the target object, where each object type has a corresponding preset reasonable range; and calculating the physical estimated size of the target object corresponding to each detection frame information The comparison with the preset reasonable range includes: determining the preset reasonable range of the target according to the object type of the target object; and comparing the physical estimated size of the target object corresponding to each detection frame information with the preset reasonable range of the target.
根据本公开的实施例，处理器还执行以下操作：在滤除检测框信息对应的有效测距结果之后，将与滤除了有效测距结果的检测框信息的采样时间相邻的检测框信息对应的有效测距结果，确定为滤除了有效测距结果的检测框信息对应的有效测距结果。According to the embodiment of the present disclosure, the processor further performs the following operations: after filtering out the effective ranging result corresponding to the detection frame information, it corresponds to the detection frame information adjacent to the sampling time of the detection frame information from which the effective ranging result is filtered out The effective ranging result of is determined as the effective ranging result corresponding to the detection frame information filtered out of the effective ranging result.
根据本公开的实施例，处理器根据多个检测框信息和有效测距结果确定目标对象的状态信息，包括：确定一个或多个目标检测框信息中的每个目标检测框信息分别对应的有效测距结果，一个或多个目标检测框信息为多个检测框信息中满足预设条件的检测框信息；以及根据每个目标检测框信息和每个目标检测框信息对应的有效测距结果，确定目标对象的状态信息。According to an embodiment of the present disclosure, the processor determines the state information of the target object according to the multiple detection frame information and the effective ranging result, including: determining that each target detection frame information in the one or more target detection frame information corresponds to the valid The ranging result, one or more target detection frame information is the detection frame information that meets the preset conditions among the multiple detection frame information; and the effective ranging result corresponding to each target detection frame information and each target detection frame information, Determine the status information of the target object.
根据本公开的实施例，状态信息包括位置信息；其中，处理器根据每个目标检测框信息和每个目标检测框信息对应的有效测距结果，确定目标对象的状态信息，包括：根据每个目标检测框信息和每个目标检测框信息 对应的有效测距结果，确定每个目标检测框信息对应的关于目标对象的初始位置信息，得到多个初始位置信息；以及根据每个目标检测框信息对应的采样时间点，对多个初始位置信息进行筛选，得到一个或多个有效初始位置信息；根据一个或多个有效初始位置信息，确定目标对象的状态信息。According to an embodiment of the present disclosure, the state information includes position information; wherein, the processor determines the state information of the target object according to each target detection frame information and the effective ranging result corresponding to each target detection frame information, including: Target detection frame information and the effective ranging result corresponding to each target detection frame information, determine the initial position information about the target object corresponding to each target detection frame information, and obtain multiple initial position information; and according to each target detection frame information At the corresponding sampling time point, multiple initial position information is screened to obtain one or more effective initial position information; the state information of the target object is determined according to the one or more effective initial position information.
根据本公开的实施例，处理器根据每个目标检测框信息对应的采样时间点，对多个初始位置信息进行筛选，包括：按照时间顺序，依次对每个目标检测框信息对应的关于目标对象的初始位置信息进行筛选。According to an embodiment of the present disclosure, the processor screens a plurality of initial position information according to the sampling time point corresponding to each target detection frame information, including: sequentially, in chronological order, corresponding to the target object corresponding to each target detection frame information The initial location information is filtered.
根据本公开的实施例，处理器按照时间顺序，依次对每个目标检测框信息对应的关于目标对象的初始位置信息进行筛选，包括：计算当前正筛选的初始位置信息和与当前正筛选的初始位置信息的采样时间相邻的后一个初始位置信息的采样时间点之间的时间差；将时间差与状态变量阈值进行比较；如果时间差小于状态变量阈值，滤除当前正筛选的初始位置信息；以及如果时间差大于或等于状态变量阈值，保留当前正筛选的初始位置信息，其中，被保留的当前正筛选的初始位置信息为有效初始位置信息。According to the embodiment of the present disclosure, the processor sequentially screens the initial position information about the target object corresponding to each target detection frame information in chronological order, including: calculating the initial position information currently being screened and the initial location information that is currently being screened The time difference between the sampling time of the position information and the sampling time point of the next initial position information; compare the time difference with the state variable threshold; if the time difference is less than the state variable threshold, filter out the initial position information currently being filtered; and if The time difference is greater than or equal to the state variable threshold, and the initial position information currently being screened is retained, where the retained initial position information currently being screened is the effective initial position information.
根据本公开的实施例，状态变量阈值随确定的有效初始位置信息发生变化。According to an embodiment of the present disclosure, the state variable threshold changes with the determined effective initial position information.
根据本公开的实施例，状态变量阈值为当前正筛选的初始位置信息和与当前正筛选的初始位置信息的采样时间相邻的有效初始位置信息的采样时间点之间的时间差。According to an embodiment of the present disclosure, the state variable threshold is the time difference between the initial location information currently being screened and the sampling time point of the effective initial location information adjacent to the sampling time of the initial location information currently being screened.
根据本公开的实施例，处理器还执行以下操作：根据多个检测框信息确定目标对象是否出现满足预设条件的移动；以及在目标对象出现了满足预设条件的移动的情况下，将目标对象满足预设条件的移动时对应的检测框信息确定为目标检测框信息。According to an embodiment of the present disclosure, the processor further performs the following operations: determining whether the target object has a movement that meets a preset condition according to multiple detection frame information; and when the target object has a movement that meets the preset condition, the target The detection frame information corresponding to the movement when the object meets the preset condition is determined as the target detection frame information.
根据本公开的实施例，处理器根据多个检测框信息确定目标对象是否出现满足预设条件的移动，包括：针对多个检测框信息中任意相邻的第一检测框信息和第二检测框信息，获得第一检测框信息采集时的成像装置的第一状态信息，和第二检测框信息采集时的成像装置的第二状态信息；根据第一检测框信息对应的有效测距结果和第一状态信息，确定与第一检测框信息对应的关于目标对象的初始位置信息的第一概率分布；根据第二检测框信息对应的有效测距结果和第二状态信息，确定与第二检测框信息对 应的关于目标对象的初始位置信息的第二概率分布；以及根据第一概率分布和第二概率分布，确定目标对象是否出现满足预设条件的移动。According to an embodiment of the present disclosure, the processor determines whether the target object has moved that satisfies a preset condition according to multiple detection frame information, including: for any adjacent first detection frame information and second detection frame in the multiple detection frame information Information, obtain the first state information of the imaging device when the first detection frame information is collected, and the second state information of the imaging device when the second detection frame information is collected; State information, determining the first probability distribution of the initial position information of the target object corresponding to the first detection frame information; determining the first probability distribution corresponding to the second detection frame information according to the effective ranging result corresponding to the second detection frame information and the second state information The information corresponds to a second probability distribution of the initial position information of the target object; and according to the first probability distribution and the second probability distribution, it is determined whether the target object moves that meets the preset condition.
根据本公开的实施例，处理器根据第一概率分布和第二概率分布，确定目标对象是否出现满足预设条件的移动，包括：根据第一概率分布和第二概率分布，确定概率密度最高的空间位置；计算概率密度最高的空间位置距离第一概率分布的第一概率分布中心位置的第一距离；计算概率密度最高的空间位置距离第二概率分布的第二概率分布中心位置的第二距离；根据第一距离和第二距离确定第一检测框信息和第二检测框信息之间的概率距离；以及如果概率距离大于或等于预设阈值，确定目标对象出现满足预设条件的移动。According to an embodiment of the present disclosure, the processor determines whether the target object has moved that meets a preset condition according to the first probability distribution and the second probability distribution, including: determining the one with the highest probability density according to the first probability distribution and the second probability distribution Spatial position; calculate the first distance between the spatial position with the highest probability density and the center of the first probability distribution of the first probability distribution; calculate the second distance between the spatial position with the highest probability density and the center of the second probability distribution of the second probability distribution ; Determine the probability distance between the first detection frame information and the second detection frame information according to the first distance and the second distance; and if the probability distance is greater than or equal to a preset threshold, it is determined that the target object appears to move that meets the preset conditions.
根据本公开的实施例，处理器根据一个或多个有效初始位置信息，确定目标对象的状态信息，包括：对一个或多个有效初始位置信息进行优化，以平滑关于目标对象的运动轨迹。According to an embodiment of the present disclosure, the processor determining the state information of the target object according to the one or more effective initial position information includes: optimizing the one or more effective initial position information to smooth the movement trajectory of the target object.
根据本公开的实施例，处理器对一个或多个有效初始位置信息进行优化，包括：对一个或多个有效初始位置信息进行非线性优化，以最小化目标偏差，其中，目标偏差与检测框信息和/或有效测距结果相关，每个有效初始位置信息进行非线性优化后具有对应的优化位置信息。According to an embodiment of the present disclosure, the processor optimizing one or more effective initial position information includes: performing non-linear optimization on one or more effective initial position information to minimize target deviation, wherein the target deviation and the detection frame The information is related to the effective ranging result, and each effective initial position information has corresponding optimized position information after nonlinear optimization.
根据本公开的实施例，目标偏差包括第一偏差和/或第二偏差；第一偏差包括关于有效初始位置信息与用于计算得到有效初始位置信息的目标检测框信息和有效测距结果之间的观测偏差；第二偏差包括关于相邻有效初始位置信息之间的平滑程度与先验值的偏差。According to an embodiment of the present disclosure, the target deviation includes a first deviation and/or a second deviation; the first deviation includes information about the effective initial position and the target detection frame information used to calculate the effective initial position information and the effective ranging result. The observation deviation of; the second deviation includes the deviation of the smoothness between adjacent valid initial position information and the prior value.
根据本公开的实施例，第一偏差通过用于计算得到有效初始位置信息的目标检测框信息和有效测距结果的概率密度函数进行表征。According to the embodiment of the present disclosure, the first deviation is characterized by the probability density function of the target detection frame information used to calculate the effective initial position information and the effective ranging result.
根据本公开的实施例，处理器对一个或多个有效初始位置信息进行优化，还包括：确定每个有效初始位置信息进行非线性优化后对应的优化位置信息是否异常；以及将异常的优化位置信息进行滤除。According to an embodiment of the present disclosure, the processor optimizes one or more effective initial position information, and further includes: determining whether the optimized position information corresponding to each effective initial position information after nonlinear optimization is abnormal; and determining the abnormal optimized position Information is filtered out.
根据本公开的实施例，目标偏差包括第一偏差，第一偏差包括第一子偏差和/或第二子偏差；第一子偏差为关于有效初始位置信息与用于计算得到有效初始位置信息的目标检测框信息之间的观测偏差；第二子偏差为关于有效初始位置信息与用于计算得到有效初始位置信息的有效测距结果 之间的观测偏差；其中，处理器确定每个有效初始位置信息进行非线性优化后对应的优化位置信息是否异常，包括：若确定第一子偏差对应的检测框信息异常，和/或确定第二子偏差对应的有效测距结果异常，则将异常的检测框信息和/或异常的有效测距结果对应的优化位置确定为异常的优化位置。According to an embodiment of the present disclosure, the target deviation includes a first deviation, and the first deviation includes a first sub-deviation and/or a second sub-deviation; Observation deviation between the target detection frame information; the second sub-deviation is the observation deviation between the effective initial position information and the effective ranging result used to calculate the effective initial position information; wherein the processor determines each effective initial position Whether the corresponding optimized position information is abnormal after the information is non-linearly optimized, including: if it is determined that the detection frame information corresponding to the first sub-deviation is abnormal, and/or the effective ranging result corresponding to the second sub-deviation is determined to be abnormal, then the abnormal detection The optimized position corresponding to the frame information and/or the abnormal effective ranging result is determined as the abnormal optimized position.
根据本公开的实施例，处理器对一个或多个有效初始位置信息进行优化，还包括：在将异常的优化位置信息进行滤除之后，对剩余的优化位置信息进行非线性优化，以最小化目标偏差，其中，每个剩余的优化位置信息进行非线性优化后具有对应的最终优化位置信息。According to an embodiment of the present disclosure, the processor optimizes one or more effective initial position information, and further includes: after filtering out abnormal optimized position information, performing nonlinear optimization on the remaining optimized position information to minimize Target deviation, where each remaining optimized position information has corresponding final optimized position information after non-linear optimization.
根据本公开的实施例，处理器还执行以下操作：在获得的图像中未识别到目标对象的情况下，确定目标对象丢失时的位置信息；以及根据目标对象丢失时的位置信息和平滑后的关于目标对象的运动轨迹，对目标对象的状态信息进行预测。According to an embodiment of the present disclosure, the processor further performs the following operations: when the target object is not recognized in the obtained image, determine the position information when the target object is lost; and according to the position information when the target object is lost and the smoothed Regarding the motion trajectory of the target object, the state information of the target object is predicted.
根据本公开的实施例，处理器根据目标对象丢失时的位置信息和平滑后的关于目标对象的运动轨迹，对目标对象的状态信息进行预测包括：根据目标对象丢失时的位置信息和平滑后的关于目标对象的运动轨迹，生成关于目标对象的预测位置的概率分布。According to an embodiment of the present disclosure, the processor predicts the state information of the target object according to the position information when the target object is lost and the smoothed motion trajectory of the target object, including: according to the position information when the target object is lost and the smoothed Regarding the motion trajectory of the target object, a probability distribution about the predicted position of the target object is generated.
根据本公开的实施例，预测位置的概率分布的空间占比随着目标对象的丢失时间的增长而增加。According to an embodiment of the present disclosure, the spatial proportion of the probability distribution of the predicted position increases as the loss time of the target object increases.
根据本公开的实施例，预测位置的概率分布的空间占比的变化参数与目标对象的类型相关。According to an embodiment of the present disclosure, the variation parameter of the spatial proportion of the probability distribution of the predicted position is related to the type of the target object.
根据本公开的实施例，变化参数包括预测位置的概率分布的空间占比的增长速度；在目标对象的类型为生物的情况下，预测位置的概率分布的空间占比的第一增长速度在不同方向上相同；在目标对象的类型为移动设备的情况下，预测位置的概率分布的空间占比的第二增长速度沿移动设备的运动方向增加。According to an embodiment of the present disclosure, the change parameter includes the growth rate of the spatial proportion of the probability distribution of the predicted position; in the case where the type of the target object is a creature, the first growth rate of the spatial proportion of the probability distribution of the predicted position is different The directions are the same; in the case where the type of the target object is a mobile device, the second increase speed of the spatial proportion of the probability distribution of the predicted position increases along the movement direction of the mobile device.
根据本公开的实施例，第一增长速度小于第二增长速度。According to an embodiment of the present disclosure, the first growth rate is less than the second growth rate.
根据本公开的实施例，还提供另一种目标对象的状态信息确定装置，包括：处理器；可读存储介质，用于存储一个或多个程序，其中，当一个或多个程序被处理器执行时，使得处理器执行以下操作：According to an embodiment of the present disclosure, another device for determining status information of a target object is also provided, including: a processor; a readable storage medium for storing one or more programs, wherein when one or more programs are executed by the processor When executed, it causes the processor to perform the following operations:
获得通过可移动平台携带的成像装置获得的关于目标对象的多帧图像；对多帧图像中的每一帧图像进行识别，得到关于目标对象的多个检测框信息；确定多个检测框信息中满足预设条件的一个或多个目标检测框信息；获得可移动平台与目标对象之间的多个测距结果；以及根据一个或多个目标检测框信息和多个测距结果确定目标对象的状态信息。Obtain multi-frame images of the target object obtained by the imaging device carried by the movable platform; identify each frame of the multi-frame image to obtain multiple detection frame information about the target object; determine the multiple detection frame information One or more target detection frame information that meets preset conditions; obtain multiple ranging results between the movable platform and the target object; and determine the target object based on one or more target detection frame information and multiple ranging results status information.
根据本公开的实施例，处理器根据一个或多个目标检测框信息和多个测距结果确定目标对象的状态信息，包括：确定每个目标检测框信息对应的测距结果；以及根据每个目标检测框信息和每个目标检测框信息对应的测距结果，确定目标对象的状态信息。According to an embodiment of the present disclosure, the processor determines the state information of the target object according to one or more target detection frame information and multiple ranging results, including: determining the ranging result corresponding to each target detection frame information; and according to each The target detection frame information and the ranging result corresponding to each target detection frame information determine the state information of the target object.
根据本公开的实施例，状态信息包括位置信息；其中，处理器确定多个检测框信息中满足预设条件的一个或多个目标检测框信息，包括：根据每个检测框信息对应的采样时间点，对多个检测框信息进行筛选，得到一个或多个目标检测框信息。According to an embodiment of the present disclosure, the status information includes position information; wherein, the processor determines one or more target detection frame information that meets a preset condition among the plurality of detection frame information, including: according to the sampling time corresponding to each detection frame information Click to filter multiple detection frame information to obtain one or more target detection frame information.
根据本公开的实施例，处理器根据每个检测框信息对应的采样时间点，对多个检测框信息进行筛选，包括：按照时间顺序，依次对每个检测框信息进行筛选。According to an embodiment of the present disclosure, the processor screens multiple detection frame information according to the sampling time point corresponding to each detection frame information, including: sequentially screening each detection frame information in a time sequence.
根据本公开的实施例，处理器按照时间顺序，依次对每个检测框信息进行筛选包括：计算当前正筛选的检测框信息和与当前正筛选的检测框信息的采样时间相邻的后一个检测框信息的采样时间点之间的时间差；将时间差与状态变量阈值进行比较；如果时间差小于状态变量阈值，滤除当前正筛选的检测框信息；以及如果时间差大于或等于状态变量阈值，保留当前正筛选的检测框信息，其中，被保留的当前正筛选的检测框信息为目标检测框信息。According to an embodiment of the present disclosure, the processor sequentially screens each detection frame information in a chronological order, including: calculating the detection frame information currently being screened and the next detection adjacent to the sampling time of the detection frame information currently being screened The time difference between the sampling time points of the frame information; compare the time difference with the state variable threshold; if the time difference is less than the state variable threshold, filter out the detection frame information currently being screened; and if the time difference is greater than or equal to the state variable threshold, keep the current positive The screened detection frame information, wherein the retained detection frame information currently being screened is the target detection frame information.
根据本公开的实施例，状态变量阈值随确定的目标检测框信息发生变化。According to an embodiment of the present disclosure, the state variable threshold changes with the determined target detection frame information.
根据本公开的实施例，状态变量阈值为当前正筛选的检测框信息和与当前正筛选的检测框信息的采样时间相邻的目标检测框信息的采样时间点之间的时间差。According to an embodiment of the present disclosure, the state variable threshold is the time difference between the detection frame information currently being screened and the sampling time point of the target detection frame information adjacent to the sampling time of the detection frame information currently being screened.
根据本公开的实施例，处理器确定多个检测框信息中满足预设条件的多个目标检测框信息，包括：根据多个检测框信息确定目标对象是否出现 满足预设条件的移动；以及在目标对象出现了满足预设条件的移动的情况下，将目标对象满足预设条件的移动时对应的检测框信息确定为目标检测框信息。According to an embodiment of the present disclosure, the processor determining multiple target detection frame information that meets a preset condition among the multiple detection frame information includes: determining whether the target object has moved that meets the preset condition according to the multiple detection frame information; and In the case where the target object moves that meets the preset condition, the detection frame information corresponding to the movement of the target object that meets the preset condition is determined as the target detection frame information.
根据本公开的实施例，处理器根据多个检测框信息确定目标对象是否出现满足预设条件的移动，包括：针对多个检测框信息中任意相邻的第一检测框信息和第二检测框信息，获得第一检测框信息采集时的成像装置的第一状态信息，和第二检测框信息采集时的成像装置的第二状态信息；根据第一检测框信息对应的测距结果和第一状态信息，确定与第一检测框信息对应的关于目标对象的初始位置信息的第一概率分布；根据第二检测框信息对应的测距结果和第二状态信息，确定与第二检测框信息对应的关于目标对象的初始位置信息的第二概率分布；以及根据第一概率分布和第二概率分布，确定目标对象是否出现满足预设条件的移动。According to an embodiment of the present disclosure, the processor determines whether the target object has moved that satisfies a preset condition according to multiple detection frame information, including: for any adjacent first detection frame information and second detection frame in the multiple detection frame information Information, obtain the first state information of the imaging device when the first detection frame information is collected, and the second state information of the imaging device when the second detection frame information is collected; State information, determine the first probability distribution of the initial position information of the target object corresponding to the first detection frame information; determine the corresponding to the second detection frame information according to the ranging result corresponding to the second detection frame information and the second state information The second probability distribution about the initial position information of the target object; and according to the first probability distribution and the second probability distribution, it is determined whether the target object appears to move that meets the preset condition.
根据本公开的实施例，处理器根据第一概率分布和第二概率分布，确定目标对象是否出现满足预设条件的移动包括：根据第一概率分布和第二概率分布，确定概率密度最高的空间位置；计算概率密度最高的空间位置距离第一概率分布的第一概率分布中心位置的第一距离；计算概率密度最高的空间位置距离第二概率分布的第二概率分布中心位置的第二距离；根据第一距离和第二距离确定第一检测框信息和第二检测框信息之间的概率距离；以及如果概率距离大于或等于预设阈值，确定目标对象出现满足预设条件的移动。According to an embodiment of the present disclosure, the processor determining whether the target object has moved that meets a preset condition according to the first probability distribution and the second probability distribution includes: determining the space with the highest probability density according to the first probability distribution and the second probability distribution Position; calculate the first distance between the space position with the highest probability density and the center position of the first probability distribution of the first probability distribution; calculate the second distance between the space position with the highest probability density and the center position of the second probability distribution of the second probability distribution; The probability distance between the first detection frame information and the second detection frame information is determined according to the first distance and the second distance; and if the probability distance is greater than or equal to a preset threshold, it is determined that the target object has moved that meets the preset condition.
根据本公开的实施例，处理器根据一个或多个目标检测框信息和多个测距结果确定目标对象的状态信息，包括：对一个或多个目标检测框信息对应的有效初始位置信息进行优化，以平滑关于目标对象的运动轨迹。According to an embodiment of the present disclosure, the processor determines the state information of the target object according to one or more target detection frame information and multiple ranging results, including: optimizing effective initial position information corresponding to the one or more target detection frame information To smooth the trajectory of the target object.
根据本公开的实施例，处理器对一个或多个目标检测框信息对应的有效初始位置信息进行优化，包括：对一个或多个目标检测框信息对应的有效初始位置信息进行非线性优化，以最小化目标偏差，其中，目标偏差与检测框信息和/或测距结果相关，每个有效初始位置信息进行非线性优化后具有对应的优化位置信息。According to an embodiment of the present disclosure, the processor optimizing the effective initial position information corresponding to the one or more target detection frame information includes: performing nonlinear optimization on the effective initial position information corresponding to the one or more target detection frame information to Minimize the target deviation, where the target deviation is related to the detection frame information and/or the ranging result, and each effective initial position information has corresponding optimized position information after nonlinear optimization.
根据本公开的实施例，目标偏差包括第一偏差和/或第二偏差；第一偏差包括关于有效初始位置信息与用于计算得到有效初始位置信息的目标 检测框信息和测距结果之间的观测偏差；第二偏差包括关于多个有效初始位置信息中的相邻有效初始位置信息之间的平滑程度与先验值的偏差。According to an embodiment of the present disclosure, the target deviation includes a first deviation and/or a second deviation; the first deviation includes the difference between the effective initial position information and the target detection frame information used to obtain the effective initial position information and the ranging result. Observation deviation; the second deviation includes a deviation between the smoothness of adjacent effective initial position information in the plurality of effective initial position information and the prior value.
根据本公开的实施例，第一偏差通过用于计算得到有效初始位置信息的目标检测框信息和测距结果的概率密度函数进行表征。According to the embodiment of the present disclosure, the first deviation is characterized by the probability density function of the target detection frame information and the ranging result used to calculate the effective initial position information.
根据本公开的实施例，处理器对一个或多个目标检测框信息对应的有效初始位置信息进行优化，还包括：确定每个有效初始位置信息进行非线性优化后对应的优化位置信息是否异常；以及将异常的优化位置信息进行滤除。According to an embodiment of the present disclosure, the processor optimizing the effective initial position information corresponding to one or more target detection frame information further includes: determining whether the optimized position information corresponding to each effective initial position information after nonlinear optimization is abnormal; And filter out abnormal optimized location information.
根据本公开的实施例，目标偏差包括第一偏差，第一偏差包括第一子偏差和/或第二子偏差；第一子偏差为关于有效初始位置信息与用于计算得到有效初始位置信息的目标检测框信息之间的观测偏差；第二子偏差为关于有效初始位置信息与用于计算得到有效初始位置信息的测距结果之间的观测偏差；其中，处理器确定每个有效初始位置信息进行非线性优化后对应的优化位置信息是否异常，包括：若确定第一子偏差对应的检测框信息异常，和/或确定第二子偏差对应的测距结果异常，则将异常的检测框信息和/或异常的测距结果对应的优化位置确定为异常的优化位置。According to an embodiment of the present disclosure, the target deviation includes a first deviation, and the first deviation includes a first sub-deviation and/or a second sub-deviation; Observation deviation between target detection frame information; the second sub-deviation is the observation deviation between the effective initial position information and the ranging result used to calculate the effective initial position information; wherein the processor determines each effective initial position information Whether the corresponding optimized position information after nonlinear optimization is abnormal includes: if it is determined that the detection frame information corresponding to the first sub-deviation is abnormal, and/or the ranging result corresponding to the second sub-deviation is determined to be abnormal, then the abnormal detection frame information And/or the optimized position corresponding to the abnormal ranging result is determined as the abnormal optimized position.
根据本公开的实施例，处理器对一个或多个目标检测框信息对应的有效初始位置信息进行优化，还包括：在将异常的优化位置信息进行滤除之后，对剩余的优化位置信息进行非线性优化，以最小化目标偏差，其中，每个剩余的优化位置信息进行非线性优化后具有对应的最终优化位置信息。According to an embodiment of the present disclosure, the processor optimizes the effective initial position information corresponding to one or more target detection frame information, and further includes: after filtering out abnormal optimized position information, performing non-processing on the remaining optimized position information. Linear optimization to minimize the target deviation, where each remaining optimized position information has corresponding final optimized position information after nonlinear optimization.
根据本公开的实施例，处理器还执行以下操作：在获得的图像中未识别到目标对象的情况下，确定目标对象丢失时的位置信息；以及根据目标对象丢失时的位置信息和平滑后的关于目标对象的运动轨迹，对目标对象的状态信息进行预测。According to an embodiment of the present disclosure, the processor further performs the following operations: when the target object is not recognized in the obtained image, determine the position information when the target object is lost; and according to the position information when the target object is lost and the smoothed Regarding the motion trajectory of the target object, the state information of the target object is predicted.
根据本公开的实施例，处理器根据目标对象丢失时的位置信息和平滑后的关于目标对象的运动轨迹，对目标对象的状态信息进行预测，包括：根据目标对象丢失时的位置信息和平滑后的关于目标对象的运动轨迹，生成关于目标对象的预测位置的概率分布。According to an embodiment of the present disclosure, the processor predicts the state information of the target object according to the position information when the target object is lost and the smoothed motion trajectory about the target object, including: according to the position information when the target object is lost and smoothed The trajectory of the target object is generated to generate the probability distribution of the predicted position of the target object.
根据本公开的实施例，预测位置的概率分布的空间占比随着目标对象的丢失时间的增长而增加。According to an embodiment of the present disclosure, the spatial proportion of the probability distribution of the predicted position increases as the loss time of the target object increases.
根据本公开的实施例，预测位置的概率分布的空间占比的变化参数与目标对象的类型相关。According to an embodiment of the present disclosure, the variation parameter of the spatial proportion of the probability distribution of the predicted position is related to the type of the target object.
根据本公开的实施例，变化参数包括预测位置的概率分布的空间占比的增长速度，预测位置的概率分布的空间占比的变化参数与目标对象的类型相关包括：在目标对象的类型为生物的情况下，预测位置的概率分布的空间占比的第一增长速度在不同方向上相同；在目标对象的类型为移动设备的情况下，预测位置的概率分布的空间占比的第二增长速度沿移动设备的运动方向增加。According to an embodiment of the present disclosure, the change parameter includes the growth rate of the space proportion of the probability distribution of the predicted position, and the change parameter of the space proportion of the predicted position probability distribution is related to the type of the target object, including: In the case of the predicted position, the first growth rate of the space proportion of the probability distribution of the predicted position is the same in different directions; when the target object type is a mobile device, the second growth rate of the space proportion of the predicted position’s probability distribution Increase in the direction of movement of the mobile device.
根据本公开的实施例，第一增长速度小于第二增长速度。According to an embodiment of the present disclosure, the first growth rate is less than the second growth rate.
根据本公开的实施例，处理器根据一个或多个目标检测框信息和多个测距结果确定目标对象的状态信息，包括：根据一个或多个目标检测框信息从多个测距结果中筛选出有效测距结果；以及根据一个或多个目标检测框信息和有效测距结果确定目标对象的状态信息。According to an embodiment of the present disclosure, the processor determines the state information of the target object according to the one or more target detection frame information and the multiple ranging results, including: filtering from the multiple ranging results according to the one or more target detection frame information A valid ranging result is obtained; and the status information of the target object is determined according to one or more target detection frame information and the valid ranging result.
根据本公开的实施例，处理器根据一个或多个目标检测框信息从多个测距结果中筛选出有效测距结果，包括：确定每个目标检测框信息对应的一个或多个测距结果；以及对每个目标检测框信息对应的一个或多个测距结果进行筛选。According to an embodiment of the present disclosure, the processor screens out effective ranging results from multiple ranging results according to one or more target detection frame information, including: determining one or more ranging results corresponding to each target detection frame information ; And screening one or more ranging results corresponding to each target detection frame information.
根据本公开的实施例，处理器确定每个目标检测框信息对应的一个或多个测距结果，包括：根据每个目标检测框信息的采样时间点和多个测距结果中每个测距结果的采样时间点确定每个目标检测框信息对应的一个或多个测距结果。According to an embodiment of the present disclosure, the processor determining one or more ranging results corresponding to each target detection frame information includes: according to the sampling time point of each target detection frame information and each ranging result in the multiple ranging results The sampling time point of the result determines one or more ranging results corresponding to each target detection frame information.
根据本公开的实施例，测距结果包括激光测距结果，处理器对每个目标检测框信息对应的一个或多个测距结果进行筛选，包括：确定一个或多个测距结果中每个测距结果对应的激光光斑；根据每个测距结果对应的目标检测框信息和每个测距结果对应的激光光斑，确定每个测距结果的有效性；以及根据每个测距结果的有效性对每个目标检测框信息对应的一个或多个测距结果进行筛选。According to an embodiment of the present disclosure, the ranging result includes a laser ranging result, and the processor screens one or more ranging results corresponding to each target detection frame information, including: determining each of the one or more ranging results The laser spot corresponding to the ranging result; determine the validity of each ranging result according to the target detection frame information corresponding to each ranging result and the laser spot corresponding to each ranging result; and according to the validity of each ranging result Screen one or more ranging results corresponding to each target detection frame information.
根据本公开的实施例，处理器根据每个测距结果对应的目标检测框信息和每个测距结果对应的激光光斑，确定每个测距结果的有效性，包括：确定每个测距结果对应的目标检测框信息和每个测距结果对应的激光光斑的面积重合率；以及根据面积重合率确定每个测距结果的有效性。According to an embodiment of the present disclosure, the processor determines the validity of each ranging result according to the target detection frame information corresponding to each ranging result and the laser spot corresponding to each ranging result, including: determining each ranging result The corresponding target detection frame information and the area coincidence rate of the laser spot corresponding to each ranging result; and the validity of each ranging result is determined according to the area coincidence rate.
根据本公开的实施例，处理器根据面积重合率确定每个测距结果的有效性包括：将面积重合率与预设比例阈值进行比较；将面积重合率大于或等于预设比例阈值的测距结果确定为有效测距结果；以及将面积重合率小于预设比例阈值的测距结果确定为无效测距结果。According to an embodiment of the present disclosure, the processor determining the validity of each ranging result according to the area coincidence rate includes: comparing the area coincidence rate with a preset ratio threshold; The result is determined as a valid ranging result; and the ranging result whose area coincidence rate is less than the preset ratio threshold is determined as an invalid ranging result.
根据本公开的实施例，在确定每个目标检测框信息对应多个测距结果的情况下，处理器根据每个测距结果对应的目标检测框信息和每个测距结果对应的激光光斑，确定每个测距结果的有效性包括：根据采样时间相邻的两个目标检测框信息，确定采样时间相邻的两个目标检测框信息的采样时间之间的多个测距结果中每个测距结果分别对应的插值目标检测框信息，得到每个测距结果对应的目标检测框信息；以及根据每个测距结果对应的激光光斑和与每个测距结果对应的目标检测框信息，确定每个测距结果的有效性。According to an embodiment of the present disclosure, in a case where it is determined that each target detection frame information corresponds to multiple ranging results, the processor according to the target detection frame information corresponding to each ranging result and the laser spot corresponding to each ranging result, Determining the validity of each ranging result includes: determining each of the multiple ranging results between the sampling times of the two target detection frame information adjacent to the sampling time according to the information of the two target detection frames adjacent to the sampling time The interpolated target detection frame information corresponding to the ranging results respectively, to obtain the target detection frame information corresponding to each ranging result; and according to the laser spot corresponding to each ranging result and the target detection frame information corresponding to each ranging result, Determine the validity of each ranging result.
根据本公开的实施例，处理器根据多个目标检测框信息和有效测距结果确定目标对象的状态信息包括：确定每个目标检测框信息对应的有效测距结果；以及根据每个目标检测框信息和每个目标检测框信息对应的有效测距结果，确定目标对象的状态信息。According to an embodiment of the present disclosure, the processor determining the state information of the target object according to the multiple target detection frame information and the effective ranging result includes: determining the effective ranging result corresponding to each target detection frame information; and determining the effective ranging result according to each target detection frame The information and the effective ranging result corresponding to each target detection frame information determine the status information of the target object.
根据本公开的实施例，处理器确定每个目标检测框信息对应的有效测距结果，包括：根据每个目标检测框信息的采样时间点和每个有效测距结果的采样时间点，将每个有效测距结果关联至与有效测距结果的采样时间点最接近的目标检测框信息。According to an embodiment of the present disclosure, the processor determines the effective ranging result corresponding to each target detection frame information, including: according to the sampling time point of each target detection frame information and the sampling time point of each effective ranging result, each Each effective ranging result is associated with the target detection frame information closest to the sampling time point of the effective ranging result.
根据本公开的实施例，在每个目标检测框信息对应多个有效测距结果的情况下，处理器根据每个目标检测框信息和每个目标检测框信息对应的有效测距结果，确定目标对象的状态信息，包括：计算每个目标检测框信息对应的多个有效测距结果的加权平均值，得到每个目标检测框信息对应的目标测距结果；以及根据每个目标检测框信息和每个目标检测框信息对应的目标测距结果，确定目标对象的状态信息。According to an embodiment of the present disclosure, in the case that each target detection frame information corresponds to multiple effective ranging results, the processor determines the target according to each target detection frame information and the effective ranging result corresponding to each target detection frame information. The state information of the object includes: calculating the weighted average of multiple effective ranging results corresponding to each target detection frame information to obtain the target ranging result corresponding to each target detection frame information; and according to each target detection frame information and The target ranging result corresponding to each target detection frame information determines the status information of the target object.
根据本公开的实施例，处理器根据一个或多个目标检测框信息和多个测距结果确定目标对象的状态信息，包括：确定每个目标检测框信息对应的测距结果；确定每个目标检测框信息对应的关于目标对象的物理估计尺寸；根据每个目标检测框信息对应的关于目标对象的物理估计尺寸，对每个目标检测框信息对应的测距结果进行筛选；以及根据一个或多个目标检测框信息以及筛选后的测距结果，确定目标对象的状态信息。According to an embodiment of the present disclosure, the processor determines the state information of the target object according to one or more target detection frame information and multiple ranging results, including: determining the ranging result corresponding to each target detection frame information; determining each target The physical estimated size of the target object corresponding to the detection frame information; the ranging result corresponding to each target detection frame information is screened according to the physical estimated size of the target object corresponding to each target detection frame information; and according to one or more The target detection frame information and the filtered ranging results determine the status information of the target object.
根据本公开的实施例，处理器根据每个目标检测框信息对应的关于目标对象的物理估计尺寸，对每个目标检测框信息对应的测距结果进行筛选，包括：将每个目标检测框信息对应的关于目标对象的物理估计尺寸与预设合理范围进行比较；以及在目标检测框信息对应的关于目标对象的物理估计尺寸与预设合理范围不相符的情况下，滤除目标检测框信息对应的测距结果。According to an embodiment of the present disclosure, the processor screens the distance measurement results corresponding to each target detection frame information according to the physical estimated size of the target object corresponding to each target detection frame information, including: The corresponding physical estimated size of the target object is compared with the preset reasonable range; and when the physical estimated size of the target object corresponding to the target detection frame information does not match the preset reasonable range, the target detection frame information is filtered out.的ranging results.
根据本公开的实施例，处理器确定每个目标检测框信息对应的关于目标对象的物理估计尺寸，包括：根据每个目标检测框信息和每个目标检测框信息采集时的成像装置的视场角，确定每个目标检测框信息对应的视场角；以及根据每个目标检测框信息对应的视场角和每个目标检测框信息对应的测距结果，确定每个目标检测框信息对应的物理估计尺寸。According to an embodiment of the present disclosure, the processor determines the physical estimated size of the target object corresponding to each target detection frame information, including: according to each target detection frame information and the field of view of the imaging device when each target detection frame information is collected Angle, determine the field of view corresponding to each target detection frame information; and determine the field of view corresponding to each target detection frame information and the distance measurement result corresponding to each target detection frame information. Physically estimated size.
根据本公开的实施例，处理器还执行以下操作：确定目标对象的对象类型，其中，每个对象类型具有对应的预设合理范围；其中，将每个目标检测框信息对应的关于目标对象的物理估计尺寸与预设合理范围进行比较包括：根据目标对象的对象类型确定目标预设合理范围；以及将每个目标检测框信息对应的关于目标对象的物理估计尺寸与目标预设合理范围进行比较。According to an embodiment of the present disclosure, the processor further performs the following operations: determining the object type of the target object, wherein each object type has a corresponding preset reasonable range; wherein, the information about the target object corresponding to each target detection frame information The comparison of the physical estimated size with the preset reasonable range includes: determining the target preset reasonable range according to the object type of the target object; and comparing the physical estimated size of the target object corresponding to each target detection frame information with the target preset reasonable range .
根据本公开的实施例，具体地，处理器例如可以包括通用微处理器、指令集处理器和/或相关芯片组和/或专用微处理器(例如，专用集成电路(ASIC))，等等。处理器还可以包括用于缓存用途的板载存储器。处理器可以是用于执行根据本公开实施例的方法流程的不同动作的单一处理单元或者是多个处理单元。According to an embodiment of the present disclosure, specifically, the processor may include, for example, a general-purpose microprocessor, an instruction set processor and/or a related chipset and/or a special-purpose microprocessor (for example, an application specific integrated circuit (ASIC)), etc. . The processor may also include on-board memory for caching purposes. The processor may be a single processing unit or multiple processing units for executing different actions of the method flow according to the embodiments of the present disclosure.
根据本公开的实施例，还提供了一种目标对象的状态信息确定***，包括：用于获得关于目标对象的多帧图像的成像装置和上述实施例中提及的状态信息确定装置。According to an embodiment of the present disclosure, there is also provided a system for determining state information of a target object, including: an imaging device for obtaining multi-frame images of the target object and the state information determining device mentioned in the above embodiment.
根据本公开的实施例，还提供了一种可移动平台，包括：可移动本体和目标对象的状态信息确定***。According to an embodiment of the present disclosure, a movable platform is also provided, including: a movable body and a system for determining status information of a target object.
需要说明的是，本领域技术人员可以清楚地了解到，本公开省略了一些关于可移动平台的公知部件的描述。由于针对不同的设备情况，可移动平台可以具有不同的部件。例如，可移动平台为无人机时，还可以包括旋翼和旋转机构等等，本公开省略了旋翼和旋转机构等部件的描述。可移动平台为无人车时，还可以包括发动机和车轮等等，本公开省略了发动机和车轮等部件的描述。It should be noted that those skilled in the art can clearly understand that the present disclosure omits some descriptions of well-known components of the movable platform. Due to different equipment conditions, the movable platform can have different components. For example, when the movable platform is an unmanned aerial vehicle, it may also include a rotor and a rotating mechanism, etc. The description of components such as the rotor and the rotating mechanism is omitted in this disclosure. When the movable platform is an unmanned vehicle, it may also include an engine and wheels, etc. The description of components such as the engine and wheels is omitted in this disclosure.
根据本公开的实施例，还提供了一种可读存储介质，其上存储有可执行指令，该指令被处理器执行时使处理器执行上述实施例中提及的状态信息确定方法。According to an embodiment of the present disclosure, there is also provided a readable storage medium having executable instructions stored thereon, and when the instructions are executed by a processor, the processor executes the method for determining state information mentioned in the foregoing embodiment.
该可读存储介质可以是上述实施例中描述的设备/装置/***中所包含的；也可以是单独存在，而未装配入该设备/装置/***中。上述可读存储介质承载有一个或者多个程序，当上述一个或者多个程序被执行时，实现根据本公开实施例的方法。The readable storage medium may be included in the device/device/system described in the above embodiments; or it may exist alone without being assembled into the device/device/system. The aforementioned readable storage medium carries one or more programs, and when the aforementioned one or more programs are executed, the method according to the embodiments of the present disclosure is implemented.
根据本公开的实施例，可读存储介质可以是非易失性的可读存储介质，例如可以包括但不限于：便携式计算机磁盘、硬盘、随机访问存储器(RAM)、只读存储器(ROM)、可擦式可编程只读存储器(EPROM或闪存)、便携式紧凑磁盘只读存储器(CD-ROM)、光存储器件、磁存储器件、或者上述的任意合适的组合。在本公开中，可读存储介质可以是任何包含或存储程序的有形介质，该程序可以被指令执行***、装置或者器件使用或者与其结合使用。According to an embodiment of the present disclosure, the readable storage medium may be a nonvolatile readable storage medium, for example, it may include but not limited to: portable computer disk, hard disk, random access memory (RAM), read only memory (ROM), Erasable programmable read-only memory (EPROM or flash memory), portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination of the above. In the present disclosure, a readable storage medium may be any tangible medium that contains or stores a program, and the program may be used by or in combination with an instruction execution system, apparatus, or device.
图15示意性示出了根据本公开实施例的目标对象的状态信息确定***的框图。Fig. 15 schematically shows a block diagram of a system for determining status information of a target object according to an embodiment of the present disclosure.
需要说明的是，目标对象的状态信息确定***1500也可以具有图15中所示的部分或全部硬件模块。It should be noted that the system 1500 for determining the status information of the target object may also have some or all of the hardware modules shown in FIG. 15.
如图15所示，根据本公开实施例的目标对象的状态信息确定***1500包括处理器1501，其可以根据存储在只读存储器(ROM)1502中的程序或者从存储部分1508加载到随机访问存储器(RAM)1503中的程序而执行各种适当的动作和处理。处理器1501例如可以包括通用微处理器(例如CPU)、指令集处理器和/或相关芯片组和/或专用微处理器(例如，专用集成电路(ASIC))，等等。处理器1501还可以包括用于缓存用途的板载存储器。处理器1501可以包括用于执行根据本公开实施例的方法流程的不同动作的单一处理单元或者是多个处理单元。As shown in FIG. 15, a system 1500 for determining status information of a target object according to an embodiment of the present disclosure includes a processor 1501, which can be loaded into a random access memory according to a program stored in a read-only memory (ROM) 1502 or from a storage part 1508 (RAM) The program in 1503 executes various appropriate actions and processing. The processor 1501 may include, for example, a general-purpose microprocessor (for example, a CPU), an instruction set processor and/or a related chipset and/or a special purpose microprocessor (for example, an application specific integrated circuit (ASIC)), and so on. The processor 1501 may also include on-board memory for caching purposes. The processor 1501 may include a single processing unit or multiple processing units for performing different actions of a method flow according to an embodiment of the present disclosure.
在RAM 1503中，存储有目标对象的状态信息确定***1500操作所需的各种程序和数据。处理器1501、ROM 1502以及RAM 1503通过总线1504彼此相连。处理器1501通过执行ROM 1502和/或RAM 1503中的程序来执行根据本公开实施例的方法流程的各种操作。需要注意，所述程序也可以存储在除ROM 1502和RAM 1503以外的一个或多个存储器中。处理器1501也可以通过执行存储在所述一个或多个存储器中的程序来执行根据本公开实施例的方法流程的各种操作。In the RAM 1503, various programs and data required for the operation of the system 1500 are determined by the state information of the target object. The processor 1501, the ROM 1502, and the RAM 1503 are connected to each other through a bus 1504. The processor 1501 executes various operations of the method flow according to the embodiments of the present disclosure by executing programs in the ROM 1502 and/or RAM 1503. It should be noted that the program can also be stored in one or more memories other than ROM 1502 and RAM 1503. The processor 1501 may also execute various operations of the method flow according to the embodiments of the present disclosure by executing programs stored in the one or more memories.
根据本公开的实施例，目标对象的状态信息确定***1500还可以包括输入/输出(I/O)接口1505，输入/输出(I/O)接口1505也连接至总线1504。目标对象的状态信息确定***1500还可以包括连接至I/O接口1505的以下部件中的一项或多项：包括键盘、鼠标等的输入部分1506；包括诸如阴极射线管(CRT)、液晶显示器(LCD)等以及扬声器等的输出部分1507；包括硬盘等的存储部分1508；以及包括诸如LAN卡、调制解调器等的网络接口卡的通信部分1509。通信部分1509经由诸如因特网的网络执行通信处理。驱动器1510也根据需要连接至I/O接口1505。可拆卸介质1511，诸如磁盘、光盘、磁光盘、半导体存储器等等，根据需要安装在驱动器1510上，以便于从其上读出的计算机程序根据需要被安装入存储部分1508。According to an embodiment of the present disclosure, the state information determining system 1500 of the target object may further include an input/output (I/O) interface 1505, and the input/output (I/O) interface 1505 is also connected to the bus 1504. The state information determining system 1500 of the target object may also include one or more of the following components connected to the I/O interface 1505: an input part 1506 including a keyboard, a mouse, etc.; including a cathode ray tube (CRT), a liquid crystal display, etc. (LCD), etc. and output section 1507 of speakers, etc.; storage section 1508 including hard disks, etc.; and communication section 1509 including network interface cards such as LAN cards, modems, and the like. The communication section 1509 performs communication processing via a network such as the Internet. The driver 1510 is also connected to the I/O interface 1505 as needed. A removable medium 1511, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 1510 as required, so that the computer program read therefrom is installed into the storage portion 1508 as required.
根据本公开的实施例，根据本公开实施例的方法流程可以被实现为计算机软件程序。例如，本公开的实施例包括一种计算机程序产品，其包括承载在计算机可读存储介质上的计算机程序，该计算机程序包含用于执行流程图所示的方法的程序代码。在这样的实施例中，该计算机程序可以通 过通信部分1509从网络上被下载和安装，和/或从可拆卸介质1511被安装。在该计算机程序被处理器1501执行时，执行本公开实施例的***中限定的上述功能。According to the embodiment of the present disclosure, the method flow according to the embodiment of the present disclosure may be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable storage medium, and the computer program contains program code for executing the method shown in the flowchart. In such an embodiment, the computer program may be downloaded and installed from the network through the communication section 1509, and/or installed from the removable medium 1511. When the computer program is executed by the processor 1501, it executes the above-mentioned functions defined in the system of the embodiment of the present disclosure.
附图中的流程图和框图，图示了按照本公开各种实施例的***、方法和计算机程序产品的可能实现的体系架构、功能和操作。在这点上，流程图或框图中的每个方框可以代表一个模块、程序段、或代码的一部分，上述模块、程序段、或代码的一部分包含一个或多个用于实现规定的逻辑功能的可执行指令。也应当注意，在有些作为替换的实现中，方框中所标注的功能也可以以不同于附图中所标注的顺序发生。例如，两个接连地表示的方框实际上可以基本并行地执行，它们有时也可以按相反的顺序执行，这依所涉及的功能而定。也要注意的是，框图或流程图中的每个方框、以及框图或流程图中的方框的组合，可以用执行规定的功能或操作的专用的基于硬件的***来实现，或者可以用专用硬件与计算机指令的组合来实现。The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions, and operations of the system, method, and computer program product according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagram may represent a module, program segment, or part of the code, and the above-mentioned module, program segment, or part of the code contains one or more for realizing the specified logic function. Executable instructions. It should also be noted that, in some alternative implementations, the functions marked in the block may also occur in a different order from the order marked in the drawings. For example, two blocks shown one after another can actually be executed substantially in parallel, and they can sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram or flowchart, and the combination of blocks in the block diagram or flowchart, can be implemented by a dedicated hardware-based system that performs the specified function or operation, or can be implemented by It is realized by a combination of dedicated hardware and computer instructions.
本领域技术人员可以清楚地了解到，为描述的方便和简洁，仅以上述各功能模块的划分进行举例说明，实际应用中，可以根据需要而将上述功能分配由不同的功能模块完成，即将装置的内部结构划分成不同的功能模块，以完成以上描述的全部或者部分功能。上述描述的装置的具体工作过程，可以参考前述方法实施例中的对应过程，在此不再赘述。Those skilled in the art can clearly understand that for the convenience and conciseness of the description, only the division of the above-mentioned functional modules is used as an example. In practical applications, the above-mentioned functions can be allocated by different functional modules as required, that is, the device The internal structure is divided into different functional modules to complete all or part of the functions described above. For the specific working process of the device described above, reference may be made to the corresponding process in the foregoing method embodiment, which is not repeated here.
最后应说明的是：以上各实施例仅用以说明本公开的技术方案，而非对其限制；尽管参照前述各实施例对本公开进行了详细的说明，本领域的普通技术人员应当理解：其依然可以对前述各实施例所记载的技术方案进行修改，或者对其中部分或者全部技术特征进行等同替换；在不冲突的情况下，本公开实施例中的特征可以任意组合；而这些修改或者替换，并不使相应技术方案的本质脱离本公开各实施例技术方案的范围。Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present disclosure, not to limit it; although the present disclosure has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: The technical solutions described in the foregoing embodiments can still be modified, or some or all of the technical features can be equivalently replaced; in the case of no conflict, the features in the embodiments of the present disclosure can be combined arbitrarily; and these modifications or replacements It does not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present disclosure.

Claims (151)

  1. 一种目标对象的状态信息确定方法，其特征在于，包括：A method for determining status information of a target object, which is characterized in that it includes:
    通过可移动平台携带的成像装置获得关于所述目标对象的多帧图像；Obtaining multiple frames of images about the target object through an imaging device carried by a movable platform;
    对多帧所述图像中的每一帧图像进行识别，得到关于所述目标对象的多个检测框信息；Recognizing each of the multiple frames of the images to obtain multiple detection frame information about the target object;
    获得所述可移动平台与所述目标对象之间的多个测距结果；Obtaining multiple distance measurement results between the movable platform and the target object;
    根据多个所述检测框信息从多个所述测距结果中筛选出有效测距结果；以及Filtering out effective ranging results from the plurality of ranging results according to the plurality of detection frame information; and
    根据多个所述检测框信息和所述有效测距结果确定所述目标对象的状态信息。The state information of the target object is determined according to a plurality of the detection frame information and the effective ranging result.
  2. 根据权利要求1所述的方法，其特征在于，所述根据多个所述检测框信息从多个所述测距结果中筛选出有效测距结果，包括：The method according to claim 1, wherein said screening effective ranging results from a plurality of said ranging results according to a plurality of said detection frame information comprises:
    确定每个所述检测框信息对应的一个或多个测距结果；以及Determine one or more ranging results corresponding to each of the detection frame information; and
    对每个所述检测框信息对应的一个或多个测距结果进行筛选。One or more ranging results corresponding to each detection frame information are screened.
  3. 根据权利要求2所述的方法，其特征在于，所述确定每个所述检测框信息对应的一个或多个测距结果，包括：The method according to claim 2, wherein the determining one or more ranging results corresponding to each of the detection frame information comprises:
    根据每个所述检测框信息的采样时间点和多个所述测距结果中每个测距结果的采样时间点确定每个所述检测框信息对应的一个或多个测距结果。Determine one or more ranging results corresponding to each detection frame information according to the sampling time point of each detection frame information and the sampling time point of each ranging result in the plurality of ranging results.
  4. 根据权利要求2所述的方法，其特征在于，所述测距结果包括激光测距结果，所述对每个所述检测框信息对应的一个或多个测距结果进行筛选，包括：The method according to claim 2, wherein the distance measurement result comprises a laser distance measurement result, and the screening of one or more distance measurement results corresponding to each of the detection frame information comprises:
    确定一个或多个所述测距结果中每个测距结果对应的激光光斑；Determining the laser spot corresponding to each of the one or more ranging results;
    根据每个所述测距结果对应的检测框信息和每个所述测距结果对应的激光光斑，确定每个所述测距结果的有效性；以及Determine the validity of each distance measurement result according to the detection frame information corresponding to each distance measurement result and the laser spot corresponding to each distance measurement result; and
    根据每个所述测距结果的有效性对每个所述检测框信息对应的一个或多个测距结果进行筛选。One or more ranging results corresponding to each of the detection frame information are screened according to the validity of each of the ranging results.
  5. 根据权利要求4所述的方法，其特征在于，所述根据每个所述测距结果对应的检测框信息和每个所述测距结果对应的激光光斑，确定每个所述测距结果的有效性，包括：The method according to claim 4, wherein the detection frame information corresponding to each of the ranging results and the laser spot corresponding to each of the ranging results are used to determine the value of each of the ranging results. Effectiveness, including:
    确定每个所述测距结果对应的检测框信息和每个所述测距结果对应的激光光斑的面积重合率；以及Determining the detection frame information corresponding to each of the distance measurement results and the area overlap ratio of the laser spot corresponding to each of the distance measurement results; and
    根据所述面积重合率确定每个所述测距结果的有效性。The validity of each distance measurement result is determined according to the area coincidence rate.
  6. 根据权利要求5所述的方法，其特征在于，所述根据所述面积重合率确定每个所述测距结果的有效性，包括：The method according to claim 5, wherein the determining the validity of each of the ranging results according to the area coincidence rate comprises:
    将所述面积重合率与预设比例阈值进行比较；Comparing the area coincidence rate with a preset ratio threshold;
    将所述面积重合率大于或等于所述预设比例阈值的测距结果确定为有效测距结果；以及Determining a ranging result whose area coincidence rate is greater than or equal to the preset ratio threshold value as a valid ranging result; and
    将所述面积重合率小于所述预设比例阈值的测距结果确定为无效测距结果。The ranging result whose area coincidence rate is less than the preset ratio threshold is determined as an invalid ranging result.
  7. 根据权利要求4所述的方法，其特征在于，在确定每个所述检测框信息对应多个测距结果的情况下，所述根据每个所述测距结果对应的检测框信息和每个所述测距结果对应的激光光斑，确定每个所述测距结果的有效性，包括：The method according to claim 4, wherein in the case where it is determined that each of the detection frame information corresponds to multiple ranging results, the detection frame information corresponding to each of the ranging results and each The laser spot corresponding to the distance measurement result to determine the validity of each distance measurement result includes:
    根据采样时间相邻的两个所述检测框信息，确定采样时间相邻的两个所述检测框信息的采样时间之间的多个所述测距结果中每个测距结果分别对应的插值检测框信息，得到每个所述测距结果对应的检测框信息；以及Determine the interpolation value corresponding to each of the multiple distance measurement results between the sampling times of the two detection frame information adjacent to the sampling time according to the information of the two detection frame information adjacent to the sampling time Detection frame information to obtain detection frame information corresponding to each of the ranging results; and
    根据每个所述测距结果对应的激光光斑和与每个所述测距结果对应的检测框信息，确定每个所述测距结果的有效性。The validity of each distance measurement result is determined according to the laser spot corresponding to each distance measurement result and the detection frame information corresponding to each distance measurement result.
  8. 根据权利要求1所述的方法，其特征在于，所述根据多个所述检测框信息和所述有效测距结果确定所述目标对象的状态信息，包括：The method according to claim 1, wherein the determining the state information of the target object according to a plurality of the detection frame information and the effective ranging result comprises:
    确定每个所述检测框信息对应的有效测距结果；以及Determine the effective ranging result corresponding to each of the detection frame information; and
    根据每个所述检测框信息和每个所述检测框信息对应的有效测距结果，确定所述目标对象的状态信息。Determine the state information of the target object according to each of the detection frame information and the effective ranging result corresponding to each of the detection frame information.
  9. 根据权利要求8所述的方法，其特征在于，所述确定每个所述检测框信息对应的有效测距结果，包括：The method according to claim 8, wherein said determining the effective ranging result corresponding to each said detection frame information comprises:
    根据每个所述检测框信息的采样时间点和每个所述有效测距结果的采样时间点，将每个所述有效测距结果关联至与所述有效测距结果的采样时间点最接近的检测框信息。According to the sampling time point of each detection frame information and the sampling time point of each effective ranging result, each effective ranging result is associated with the sampling time point closest to the effective ranging result The detection frame information.
  10. 根据权利要求8所述的方法，其特征在于，在每个所述检测框信息对应多个有效测距结果的情况下，所述根据每个所述检测框信息和每个所述检测框信息对应的有效测距结果，确定所述目标对象的状态信息，包括：The method according to claim 8, characterized in that, in the case that each of the detection frame information corresponds to a plurality of valid ranging results, the method according to each of the detection frame information and each of the detection frame information The corresponding effective ranging result, determining the state information of the target object, includes:
    计算每个所述检测框信息对应的多个有效测距结果的加权平均值，得到每个所述检测框信息对应的目标测距结果；以及Calculating the weighted average of the multiple effective ranging results corresponding to each of the detection frame information to obtain the target ranging result corresponding to each of the detection frame information; and
    根据每个所述检测框信息和每个所述检测框信息对应的目标测距结果，确定所述目标对象的状态信息。Determine the state information of the target object according to each detection frame information and a target ranging result corresponding to each detection frame information.
  11. 根据权利要求1所述的方法，其特征在于，还包括：The method according to claim 1, further comprising:
    确定每个所述检测框信息对应的有效测距结果；Determining an effective ranging result corresponding to each of the detection frame information;
    确定每个所述检测框信息对应的关于所述目标对象的物理估计尺寸；以及Determining the physical estimated size of the target object corresponding to each of the detection frame information; and
    根据每个所述检测框信息对应的关于所述目标对象的物理估计尺寸，对每个所述检测框信息对应的有效测距结果进行筛选。According to the physical estimated size of the target object corresponding to each detection frame information, the effective ranging result corresponding to each detection frame information is screened.
  12. 根据权利要求11所述的方法，其特征在于，所述根据每个所述检测框信息对应的关于所述目标对象的物理估计尺寸，对每个所述检测框信息对应的有效测距结果进行筛选，包括：The method according to claim 11, wherein the effective ranging result corresponding to each detection frame information is performed according to the physical estimated size of the target object corresponding to each detection frame information. Screening, including:
    将每个所述检测框信息对应的关于所述目标对象的物理估计尺寸与预设合理范围进行比较；以及Comparing the physical estimated size of the target object corresponding to each detection frame information with a preset reasonable range; and
    在所述检测框信息对应的关于所述目标对象的物理估计尺寸与所述预设合理范围不相符的情况下，滤除所述检测框信息对应的有效测距结果。In a case where the physical estimated size of the target object corresponding to the detection frame information does not match the preset reasonable range, the effective ranging result corresponding to the detection frame information is filtered out.
  13. 根据权利要求12所述的方法，其特征在于，所述确定每个所述检测框信息对应的关于所述目标对象的物理估计尺寸，包括：The method according to claim 12, wherein the determining the physical estimated size of the target object corresponding to each of the detection frame information comprises:
    根据每个所述检测框信息和每个所述检测框信息采集时的所述成像装置的视场角，确定每个所述检测框信息对应的视场角；以及Determine the field of view corresponding to each of the detection frame information according to each of the detection frame information and the field of view of the imaging device when each of the detection frame information is collected; and
    根据每个所述检测框信息对应的视场角和每个所述检测框信息对应的有效测距结果，确定每个所述检测框信息对应的物理估计尺寸。Determine the physical estimated size corresponding to each detection frame information according to the field of view angle corresponding to each detection frame information and the effective ranging result corresponding to each detection frame information.
  14. 根据权利要求12所述的方法，其特征在于，还包括：The method according to claim 12, further comprising:
    确定所述目标对象的对象类型，其中，每个对象类型具有对应的预设合理范围；Determine the object type of the target object, wherein each object type has a corresponding preset reasonable range;
    所述将每个所述检测框信息对应的关于所述目标对象的物理估计尺寸与预设合理范围进行比较，包括：The comparing the physical estimated size of the target object corresponding to each detection frame information with a preset reasonable range includes:
    根据所述目标对象的对象类型确定目标预设合理范围；以及Determine the preset reasonable range of the target according to the object type of the target object; and
    将每个所述检测框信息对应的关于所述目标对象的物理估计尺寸与所述目标预设合理范围进行比较。The physical estimated size of the target object corresponding to each detection frame information is compared with the preset reasonable range of the target.
  15. 根据权利要求12所述的方法，其特征在于，在所述滤除所述检测框信息对应的有效测距结果之后，还包括：The method according to claim 12, wherein after the filtering out the effective ranging result corresponding to the detection frame information, the method further comprises:
    将与滤除了有效测距结果的检测框信息的采样时间相邻的检测框信息对应的有效测距结果，确定为所述滤除了有效测距结果的检测框信息对应的有效测距结果。The effective ranging result corresponding to the detection frame information adjacent to the sampling time of the detection frame information from which the effective ranging result is filtered out is determined as the effective ranging result corresponding to the detection frame information from which the effective ranging result is filtered out.
  16. 根据权利要求1所述的方法，其特征在于，所述根据多个所述检测框信息和所述有效测距结果确定所述目标对象的状态信息，包括：The method according to claim 1, wherein the determining the state information of the target object according to a plurality of the detection frame information and the effective ranging result comprises:
    确定一个或多个目标检测框信息中的每个所述目标检测框信息分别对应的有效测距结果，一个或多个所述目标检测框信息为多个所述检测框信息中满足预设条件的检测框信息；以及Determine the effective ranging result corresponding to each target detection frame information in one or more target detection frame information, and one or more of the target detection frame information is that a preset condition is satisfied among the plurality of detection frame information The check box information of; and
    根据每个所述目标检测框信息和每个所述目标检测框信息对应的有效测距结果，确定所述目标对象的状态信息。Determine the state information of the target object according to each of the target detection frame information and the effective ranging result corresponding to each of the target detection frame information.
  17. 根据权利要求16所述的方法，其特征在于，所述状态信息包括位置信息；其中，所述根据每个所述目标检测框信息和每个所述目标检测框信息对应的有效测距结果，确定所述目标对象的状态信息，包括：The method according to claim 16, wherein the state information includes position information; wherein, the effective ranging result corresponding to each of the target detection frame information and each of the target detection frame information is used, Determining the status information of the target object includes:
    根据每个所述目标检测框信息和每个所述目标检测框信息对应的有效测距结果，确定每个所述目标检测框信息对应的关于所述目标对象的初始位置信息，得到多个初始位置信息；According to each of the target detection frame information and the effective ranging result corresponding to each of the target detection frame information, the initial position information about the target object corresponding to each target detection frame information is determined, and multiple initial positions are obtained. location information;
    根据每个所述目标检测框信息对应的采样时间点，对多个所述初始位置信息进行筛选，得到一个或多个有效初始位置信息；以及Filtering a plurality of the initial position information according to the sampling time point corresponding to each of the target detection frame information to obtain one or more effective initial position information; and
    根据一个或多个所述有效初始位置信息，确定所述目标对象的状态信息。Determine the state information of the target object according to one or more of the effective initial position information.
  18. 根据权利要求17所述的方法，其特征在于，所述根据每个所述目标检测框信息对应的采样时间点，对多个所述初始位置信息进行筛选，包括：The method according to claim 17, wherein said screening a plurality of said initial position information according to the sampling time point corresponding to each said target detection frame information comprises:
    按照时间顺序，依次对每个所述目标检测框信息对应的关于所述目标对象的初始位置信息进行筛选。According to the time sequence, the initial position information about the target object corresponding to each target detection frame information is sequentially filtered.
  19. 根据权利要求18所述的方法，其特征在于，所述按照时间顺序，依次对每个所述目标检测框信息对应的关于所述目标对象的初始位置信息进行筛选，包括：The method according to claim 18, wherein the step of sequentially screening the initial position information of the target object corresponding to each of the target detection frame information in a chronological order comprises:
    计算当前正筛选的初始位置信息和与所述当前正筛选的初始位置信息的采样时间相邻的后一个初始位置信息的采样时间点之间的时间差；Calculating the time difference between the initial location information currently being screened and the sampling time point of the next initial location information adjacent to the sampling time of the initial location information currently being screened;
    将所述时间差与状态变量阈值进行比较；Comparing the time difference with a state variable threshold;
    如果所述时间差小于所述状态变量阈值，滤除当前正筛选的初始位置信息；以及If the time difference is less than the state variable threshold, filter out the initial position information currently being screened; and
    如果所述时间差大于或等于所述状态变量阈值，保留所述当前正筛选的初始位置信息，其中，被保留的所述当前正筛选的初始位置信息为所述有效初始位置信息。If the time difference is greater than or equal to the state variable threshold, the initial location information currently being screened is retained, wherein the retained initial location information currently being screened is the effective initial location information.
  20. 根据权利要求19所述的方法，其特征在于，所述状态变量阈值随确定的有效初始位置信息发生变化。The method according to claim 19, wherein the state variable threshold changes with the determined effective initial position information.
  21. 根据权利要求20所述的方法，其特征在于，所述状态变量阈值为所述当前正筛选的初始位置信息和与所述当前正筛选的初始位置信息的采样时间相邻的有效初始位置信息的采样时间点之间的时间差。The method according to claim 20, wherein the state variable threshold is the value of the initial position information currently being screened and the effective initial position information adjacent to the sampling time of the initial position information being screened. The time difference between sampling time points.
  22. 根据权利要求16所述的方法，其特征在于，还包括：The method according to claim 16, further comprising:
    根据多个所述检测框信息确定所述目标对象是否出现满足预设条件的移动；以及Determining whether the target object has moved that meets a preset condition according to multiple pieces of detection frame information; and
    在所述目标对象出现了满足预设条件的移动的情况下，将所述目标对象满足所述预设条件的移动时对应的检测框信息确定为所述目标检测框信息。In the case that the target object has a movement that meets a preset condition, the detection frame information corresponding to the movement of the target object that meets the preset condition is determined as the target detection frame information.
  23. 根据权利要求22所述的方法，其特征在于，所述根据多个所述检测框信息确定所述目标对象是否出现满足预设条件的移动，包括：The method according to claim 22, wherein the determining whether the target object moves that meets a preset condition according to a plurality of the detection frame information comprises:
    针对多个所述检测框信息中任意相邻的第一检测框信息和第二检测框信息，获得所述第一检测框信息采集时的所述成像装置的第一状态信息，和所述第二检测框信息采集时的所述成像装置的第二状态信息；For any adjacent first detection frame information and second detection frame information among the plurality of detection frame information, obtain the first state information of the imaging device when the first detection frame information is collected, and the first state information of the imaging device when the first detection frame information is collected. 2. The second state information of the imaging device when the detection frame information is collected;
    根据所述第一检测框信息对应的有效测距结果和所述第一状态信息，确定与所述第一检测框信息对应的关于所述目标对象的初始位置信息的第一概率分布；Determine, according to the effective ranging result corresponding to the first detection frame information and the first state information, a first probability distribution of the initial position information of the target object corresponding to the first detection frame information;
    根据所述第二检测框信息对应的有效测距结果和所述第二状态信息，确定与所述第二检测框信息对应的关于所述目标对象的初始位置信息的第二概率分布；以及Determining a second probability distribution of the initial position information of the target object corresponding to the second detection frame information according to the effective ranging result corresponding to the second detection frame information and the second state information; and
    根据所述第一概率分布和所述第二概率分布，确定所述目标对象是否出现满足预设条件的移动。According to the first probability distribution and the second probability distribution, it is determined whether the target object moves that meets a preset condition.
  24. 根据权利要求23所述的方法，其特征在于，所述根据所述第一概率分布和所述第二概率分布，确定所述目标对象是否出现满足预设条件的移动，包括：The method according to claim 23, wherein the determining whether the target object moves that meets a preset condition according to the first probability distribution and the second probability distribution comprises:
    根据所述第一概率分布和所述第二概率分布，确定概率密度最高的空间位置；Determine the spatial position with the highest probability density according to the first probability distribution and the second probability distribution;
    计算所述概率密度最高的空间位置距离所述第一概率分布的第一概率分布中心位置的第一距离；Calculating the first distance between the spatial position with the highest probability density and the center position of the first probability distribution of the first probability distribution;
    计算所述概率密度最高的空间位置距离所述第二概率分布的第二概率分布中心位置的第二距离；Calculating the second distance between the spatial position with the highest probability density and the center position of the second probability distribution of the second probability distribution;
    根据所述第一距离和所述第二距离确定所述第一检测框信息和所述第二检测框信息之间的概率距离；以及Determining the probability distance between the first detection frame information and the second detection frame information according to the first distance and the second distance; and
    如果所述概率距离大于或等于预设阈值，确定所述目标对象出现满足预设条件的移动。If the probability distance is greater than or equal to a preset threshold, it is determined that the target object moves that meets a preset condition.
  25. 根据权利要求17所述的方法，其特征在于，所述根据一个或多个所述有效初始位置信息，确定所述目标对象的状态信息，包括：The method according to claim 17, wherein the determining the state information of the target object according to one or more of the effective initial position information comprises:
    对一个或多个所述有效初始位置信息进行优化，以平滑关于所述目标对象的运动轨迹。One or more of the effective initial position information is optimized to smooth the movement trajectory of the target object.
  26. 根据权利要求25所述的方法，其特征在于，所述对一个或多个所述有效初始位置信息进行优化，包括：The method according to claim 25, wherein the optimizing one or more of the effective initial position information comprises:
    对一个或多个所述有效初始位置信息进行非线性优化，以最小化目标偏差，其中，所述目标偏差与所述检测框信息和/或有效测距结果相关，每个所述有效初始位置信息进行非线性优化后具有对应的优化位置信息。Non-linear optimization is performed on one or more of the effective initial position information to minimize the target deviation, wherein the target deviation is related to the detection frame information and/or the effective ranging result, and each of the effective initial positions After non-linear optimization, the information has corresponding optimized location information.
  27. 根据权利要求26所述的方法，其特征在于，所述目标偏差包括第一偏差和/或第二偏差；The method according to claim 26, wherein the target deviation comprises a first deviation and/or a second deviation;
    所述第一偏差包括关于所述有效初始位置信息与用于计算得到所述有效初始位置信息的目标检测框信息和有效测距结果之间的观测偏差；The first deviation includes an observation deviation between the effective initial position information and the target detection frame information used for calculating the effective initial position information and the effective ranging result;
    所述第二偏差包括关于相邻所述有效初始位置信息之间的平滑程度与先验值的偏差。The second deviation includes a deviation between the degree of smoothness between adjacent effective initial position information and a priori value.
  28. 根据权利要求27所述的方法，其特征在于，所述第一偏差通过用于计算得到所述有效初始位置信息的目标检测框信息和有效测距结果的概率密度函数进行表征。The method according to claim 27, wherein the first deviation is characterized by a probability density function used to calculate the target detection frame information of the effective initial position information and the effective ranging result.
  29. 根据权利要求27所述的方法，其特征在于，所述对一个或多个所述有效初始位置信息进行优化，还包括：The method according to claim 27, wherein the optimizing one or more of the effective initial position information further comprises:
    确定每个所述有效初始位置信息进行非线性优化后对应的优化位置信息是否异常；以及Determine whether the optimized position information corresponding to each effective initial position information after nonlinear optimization is abnormal; and
    将异常的优化位置信息进行滤除。Filter out abnormal optimized location information.
  30. 根据权利要求29所述的方法，其特征在于，所述目标偏差包括所述第一偏差，所述第一偏差包括第一子偏差和/或第二子偏差；The method according to claim 29, wherein the target deviation includes the first deviation, and the first deviation includes a first sub-deviation and/or a second sub-deviation;
    所述第一子偏差为关于所述有效初始位置信息与用于计算得到所述有效初始位置信息的目标检测框信息之间的观测偏差；The first sub-deviation is an observation deviation between the effective initial position information and the target detection frame information used to calculate the effective initial position information;
    所述第二子偏差为关于所述有效初始位置信息与用于计算得到所述有效初始位置信息的有效测距结果之间的观测偏差；The second sub-deviation is an observation deviation between the effective initial position information and the effective ranging result used to calculate the effective initial position information;
    其中，所述确定每个所述有效初始位置信息进行非线性优化后对应的优化位置信息是否异常，包括：Wherein, the determining whether the optimized position information corresponding to each effective initial position information after nonlinear optimization is abnormal includes:
    若确定所述第一子偏差对应的检测框信息异常，和/或确定所述第二子偏差对应的有效测距结果异常，则将异常的检测框信息和/或异常的有效测距结果对应的优化位置确定为异常的优化位置。If it is determined that the detection frame information corresponding to the first sub-deviation is abnormal, and/or it is determined that the effective ranging result corresponding to the second sub-deviation is abnormal, then the abnormal detection frame information and/or the abnormal effective ranging result are corresponded to The optimized position of is determined as the abnormal optimized position.
  31. 根据权利要求29所述的方法，其特征在于，所述对一个或多个所述有效初始位置信息进行优化，还包括：The method according to claim 29, wherein the optimizing one or more of the effective initial position information further comprises:
    在将异常的优化位置信息进行滤除之后，对剩余的优化位置信息进行非线性优化，以最小化所述目标偏差，其中，每个所述剩余的优化位置信息进行非线性优化后具有对应的最终优化位置信息。After filtering out the abnormal optimized position information, the remaining optimized position information is nonlinearly optimized to minimize the target deviation, wherein each of the remaining optimized position information has a corresponding non-linear optimization Finally optimize the location information.
  32. 根据权利要求25所述的方法，其特征在于，还包括：The method according to claim 25, further comprising:
    在获得的图像中未识别到所述目标对象的情况下，确定所述目标对象丢失时的位置信息；以及If the target object is not recognized in the obtained image, determine the position information when the target object is lost; and
    根据所述目标对象丢失时的位置信息和平滑后的关于所述目标对象的运动轨迹，对所述目标对象的状态信息进行预测。The state information of the target object is predicted according to the position information when the target object is lost and the smoothed motion track of the target object.
  33. 根据权利要求32所述的方法，其特征在于，所述根据所述目标对象丢失时的位置信息和平滑后的关于所述目标对象的运动轨迹，对所述目标对象的状态信息进行预测包括：The method according to claim 32, wherein the predicting the state information of the target object according to the position information when the target object is lost and the smoothed motion trajectory of the target object comprises:
    根据所述目标对象丢失时的位置信息和平滑后的关于所述目标对象的运动轨迹，生成关于所述目标对象的预测位置的概率分布。According to the position information when the target object is lost and the smoothed motion trajectory about the target object, a probability distribution about the predicted position of the target object is generated.
  34. 根据权利要求33所述的方法，其特征在于，所述预测位置的概率分布的空间占比随着所述目标对象的丢失时间的增长而增加。The method according to claim 33, wherein the spatial proportion of the probability distribution of the predicted position increases as the loss time of the target object increases.
  35. 根据权利要求34所述的方法，其特征在于，所述预测位置的概率分布的空间占比的变化参数与所述目标对象的类型相关。The method according to claim 34, wherein the change parameter of the spatial proportion of the probability distribution of the predicted position is related to the type of the target object.
  36. 根据权利要求35所述的方法，其特征在于，所述变化参数包括所述预测位置的概率分布的空间占比的增长速度；The method according to claim 35, wherein the change parameter comprises the growth rate of the spatial proportion of the probability distribution of the predicted position;
    在所述目标对象的类型为生物的情况下，所述预测位置的概率分布的空间占比的第一增长速度在不同方向上相同；In a case where the type of the target object is a living thing, the first growth rate of the space proportion of the probability distribution of the predicted position is the same in different directions;
    在所述目标对象的类型为移动设备的情况下，所述预测位置的概率分布的空间占比的第二增长速度沿所述移动设备的运动方向增加。In a case where the type of the target object is a mobile device, the second increasing speed of the spatial proportion of the probability distribution of the predicted position increases along the movement direction of the mobile device.
  37. 根据权利要求36所述的方法，其特征在于，所述第一增长速度小于所述第二增长速度。The method of claim 36, wherein the first growth rate is less than the second growth rate.
  38. 一种目标对象的状态信息确定方法，其特征在于，包括：A method for determining status information of a target object, which is characterized in that it includes:
    通过可移动平台携带的成像装置获得关于所述目标对象的多帧图像；Obtaining multiple frames of images about the target object through an imaging device carried by a movable platform;
    对多帧所述图像中的每一帧图像进行识别，得到关于所述目标对象的多个检测框信息；Recognizing each of the multiple frames of the images to obtain multiple detection frame information about the target object;
    确定多个所述检测框信息中满足预设条件的一个或多个目标检测框信息；Determining one or more target detection frame information that meets a preset condition among the plurality of detection frame information;
    获得所述可移动平台与所述目标对象之间的多个测距结果；以及Obtaining a plurality of ranging results between the movable platform and the target object; and
    根据一个或多个所述目标检测框信息和多个所述测距结果确定所述目标对象的状态信息。The state information of the target object is determined according to one or more of the target detection frame information and a plurality of the ranging results.
  39. 根据权利要求38所述的方法，其特征在于，所述根据一个或多个所述目标检测框信息和多个所述测距结果确定所述目标对象的状态信息，包括：The method according to claim 38, wherein the determining the state information of the target object according to one or more of the target detection frame information and a plurality of the ranging results comprises:
    确定每个所述目标检测框信息对应的测距结果；以及Determining the ranging result corresponding to each of the target detection frame information; and
    根据每个所述目标检测框信息和每个所述目标检测框信息对应的测距结果，确定所述目标对象的状态信息。The state information of the target object is determined according to each of the target detection frame information and the ranging result corresponding to each of the target detection frame information.
  40. 根据权利要求39所述的方法，其特征在于，所述状态信息包括位置信息；其中，所述确定多个所述检测框信息中满足预设条件的一个或多个目标检测框信息，包括：The method according to claim 39, wherein the status information includes position information; wherein the determining one or more target detection frame information that meets a preset condition among the plurality of detection frame information includes:
    根据每个所述检测框信息对应的采样时间点，对多个所述检测框信息进行筛选，得到一个或多个目标检测框信息。According to the sampling time point corresponding to each detection frame information, a plurality of detection frame information is screened to obtain one or more target detection frame information.
  41. 根据权利要求40所述的方法，其特征在于，所述根据每个所述检测框信息对应的采样时间点，对多个所述检测框信息进行筛选，包括：The method according to claim 40, wherein said screening a plurality of said detection frame information according to the sampling time point corresponding to each said detection frame information comprises:
    按照时间顺序，依次对每个所述检测框信息进行筛选。According to the chronological order, the information of each detection frame is screened in turn.
  42. 根据权利要求41所述的方法，其特征在于，所述按照时间顺序，依次对每个所述检测框信息进行筛选，包括：The method according to claim 41, wherein the step of sequentially screening each of the detection frame information in chronological order comprises:
    计算当前正筛选的检测框信息和与所述当前正筛选的检测框信息的采样时间相邻的后一个检测框信息的采样时间点之间的时间差；Calculating the time difference between the detection frame information currently being screened and the sampling time point of the next detection frame information adjacent to the sampling time of the detection frame information currently being screened;
    将所述时间差与状态变量阈值进行比较；Comparing the time difference with a state variable threshold;
    如果所述时间差小于所述状态变量阈值，滤除当前正筛选的检测框信息；以及If the time difference is less than the state variable threshold, filter out the detection frame information currently being screened; and
    如果所述时间差大于或等于所述状态变量阈值，保留所述当前正筛选的检测框信息，其中，被保留的所述当前正筛选的检测框信息为目标检测框信息。If the time difference is greater than or equal to the state variable threshold, the detection frame information currently being screened is retained, wherein the retained detection frame information currently being screened is target detection frame information.
  43. 根据权利要求42所述的方法，其特征在于，所述状态变量阈值随确定的目标检测框信息发生变化。The method according to claim 42, wherein the state variable threshold changes with the determined target detection frame information.
  44. 根据权利要求43所述的方法，其特征在于，所述状态变量阈值为所述当前正筛选的检测框信息和与所述当前正筛选的检测框信息的采样时间相邻的目标检测框信息的采样时间点之间的时间差。The method according to claim 43, wherein the state variable threshold is the value of the detection frame information currently being screened and the target detection frame information adjacent to the sampling time of the detection frame information currently being screened. The time difference between sampling time points.
  45. 根据权利要求38所述的方法，其特征在于，所述确定多个所述检测框信息中满足预设条件的多个目标检测框信息，包括：The method according to claim 38, wherein the determining multiple target detection frame information that meets a preset condition among the multiple detection frame information comprises:
    根据多个所述检测框信息确定所述目标对象是否出现满足预设条件的移动；以及Determining whether the target object has moved that meets a preset condition according to multiple pieces of detection frame information; and
    在所述目标对象出现了满足预设条件的移动的情况下，将所述目标对象满足所述预设条件的移动时对应的检测框信息确定为所述目标检测框信息。In the case that the target object has a movement that meets a preset condition, the detection frame information corresponding to the movement of the target object that meets the preset condition is determined as the target detection frame information.
  46. 根据权利要求35所述的方法，其特征在于，所述根据多个所述检测框信息确定所述目标对象是否出现满足预设条件的移动，包括：The method according to claim 35, wherein the determining whether the target object has moved that meets a preset condition according to a plurality of the detection frame information comprises:
    针对多个所述检测框信息中任意相邻的第一检测框信息和第二检测框信息，获得所述第一检测框信息采集时的所述成像装置的第一状态信息，和所述第二检测框信息采集时的所述成像装置的第二状态信息；For any adjacent first detection frame information and second detection frame information among the plurality of detection frame information, obtain the first state information of the imaging device when the first detection frame information is collected, and the first state information of the imaging device when the first detection frame information is collected. 2. The second state information of the imaging device when the detection frame information is collected;
    根据所述第一检测框信息对应的测距结果和所述第一状态信息，确定与所述第一检测框信息对应的关于所述目标对象的初始位置信息的第一概率分布；Determine a first probability distribution of the initial position information of the target object corresponding to the first detection frame information according to the ranging result corresponding to the first detection frame information and the first state information;
    根据所述第二检测框信息对应的测距结果和所述第二状态信息，确定与所述第二检测框信息对应的关于所述目标对象的初始位置信息的第二概率分布；以及Determining a second probability distribution of the initial position information of the target object corresponding to the second detection frame information according to the ranging result corresponding to the second detection frame information and the second state information; and
    根据所述第一概率分布和所述第二概率分布，确定所述目标对象是否出现满足预设条件的移动。According to the first probability distribution and the second probability distribution, it is determined whether the target object moves that meets a preset condition.
  47. 根据权利要求46所述的方法，其特征在于，所述根据所述第一概率分布和所述第二概率分布，确定所述目标对象是否出现满足预设条件的移动，包括：The method according to claim 46, wherein the determining whether the target object has moved that meets a preset condition according to the first probability distribution and the second probability distribution comprises:
    根据所述第一概率分布和所述第二概率分布，确定概率密度最高的空间位置；Determine the spatial position with the highest probability density according to the first probability distribution and the second probability distribution;
    计算所述概率密度最高的空间位置距离所述第一概率分布的第一概率分布中心位置的第一距离；Calculating the first distance between the spatial position with the highest probability density and the center position of the first probability distribution of the first probability distribution;
    计算所述概率密度最高的空间位置距离所述第二概率分布的第二概率分布中心位置的第二距离；Calculating the second distance between the spatial position with the highest probability density and the center position of the second probability distribution of the second probability distribution;
    根据所述第一距离和所述第二距离确定所述第一检测框信息和所述第二检测框信息之间的概率距离；以及Determining the probability distance between the first detection frame information and the second detection frame information according to the first distance and the second distance; and
    如果所述概率距离大于或等于预设阈值，确定所述目标对象出现满足预设条件的移动。If the probability distance is greater than or equal to a preset threshold, it is determined that the target object moves that meets a preset condition.
  48. 根据权利要求40所述的方法，其特征在于，所述根据一个或多个所述目标检测框信息和多个所述测距结果确定所述目标对象的状态信息，包括：The method according to claim 40, wherein the determining the state information of the target object according to one or more of the target detection frame information and a plurality of the ranging results comprises:
    对一个或多个所述目标检测框信息对应的有效初始位置信息进行优化，以平滑关于所述目标对象的运动轨迹。The effective initial position information corresponding to one or more of the target detection frame information is optimized to smooth the movement track of the target object.
  49. 根据权利要求48所述的方法，其特征在于，所述对一个或多个所述目标检测框信息对应的有效初始位置信息进行优化，包括：The method according to claim 48, wherein the optimizing effective initial position information corresponding to one or more of the target detection frame information comprises:
    对一个或多个所述目标检测框信息对应的有效初始位置信息进行非线性优化，以最小化目标偏差，其中，所述目标偏差与所述检测框信息和/或测距结果相关，每个所述有效初始位置信息进行非线性优化后具有对应的优化位置信息。Non-linear optimization is performed on the effective initial position information corresponding to one or more of the target detection frame information to minimize the target deviation, wherein the target deviation is related to the detection frame information and/or the ranging result, each The effective initial position information has corresponding optimized position information after being non-linearly optimized.
  50. 根据权利要求49所述的方法，其特征在于，所述目标偏差包括第一偏差和/或第二偏差；The method according to claim 49, wherein the target deviation comprises a first deviation and/or a second deviation;
    所述第一偏差包括关于所述有效初始位置信息与用于计算得到所述有效初始位置信息的目标检测框信息和测距结果之间的观测偏差；The first deviation includes an observation deviation between the effective initial position information and the target detection frame information used for calculating the effective initial position information and the ranging result;
    所述第二偏差包括关于多个所述有效初始位置信息中的相邻有效初始位置信息之间的平滑程度与先验值的偏差。The second deviation includes a deviation between the degree of smoothing between adjacent effective initial position information in the plurality of effective initial position information and a prior value.
  51. 根据权利要求50所述的方法，其特征在于，所述第一偏差通过用于计算得到所述有效初始位置信息的目标检测框信息和测距结果的概率密度函数进行表征。The method according to claim 50, wherein the first deviation is characterized by the probability density function of the target detection frame information and the ranging result used to calculate the effective initial position information.
  52. 根据权利要求50所述的方法，其特征在于，所述对一个或多个所述目标检测框信息对应的有效初始位置信息进行优化，还包括：The method according to claim 50, wherein the optimizing effective initial position information corresponding to one or more of the target detection frame information further comprises:
    确定每个所述有效初始位置信息进行非线性优化后对应的优化位置信息是否异常；以及Determine whether the optimized position information corresponding to each effective initial position information after nonlinear optimization is abnormal; and
    将异常的优化位置信息进行滤除。Filter out abnormal optimized location information.
  53. 根据权利要求52所述的方法，其特征在于，所述目标偏差包括所述第一偏差，所述第一偏差包括第一子偏差和/或第二子偏差；The method according to claim 52, wherein the target deviation includes the first deviation, and the first deviation includes a first sub-deviation and/or a second sub-deviation;
    所述第一子偏差为关于所述有效初始位置信息与用于计算得到所述有效初始位置信息的目标检测框信息之间的观测偏差；The first sub-deviation is an observation deviation between the effective initial position information and the target detection frame information used to calculate the effective initial position information;
    所述第二子偏差为关于所述有效初始位置信息与用于计算得到所述有效初始位置信息的测距结果之间的观测偏差；The second sub-deviation is an observation deviation between the effective initial position information and the ranging result used to calculate the effective initial position information;
    其中，所述确定每个所述有效初始位置信息进行非线性优化后对应的优化位置信息是否异常，包括：Wherein, the determining whether the optimized position information corresponding to each effective initial position information after nonlinear optimization is abnormal includes:
    若确定所述第一子偏差对应的检测框信息异常，和/或确定所述第二子偏差对应的测距结果异常，则将异常的检测框信息和/或异常的测距结果对应的优化位置确定为异常的优化位置。If it is determined that the detection frame information corresponding to the first sub-deviation is abnormal, and/or the ranging result corresponding to the second sub-deviation is determined to be abnormal, then the abnormal detection frame information and/or the abnormal ranging result are optimized accordingly The location is determined to be an abnormally optimized location.
  54. 根据权利要求52所述的方法，其特征在于，所述对一个或多个所述目标检测框信息对应的有效初始位置信息进行优化，还包括：The method according to claim 52, wherein the optimizing effective initial position information corresponding to one or more of the target detection frame information further comprises:
    在将异常的优化位置信息进行滤除之后，对剩余的优化位置信息进行非线性优化，以最小化所述目标偏差，其中，每个所述剩余的优化位置信息进行非线性优化后具有对应的最终优化位置信息。After filtering out the abnormal optimized position information, the remaining optimized position information is nonlinearly optimized to minimize the target deviation, wherein each of the remaining optimized position information has a corresponding non-linear optimization Finally optimize the location information.
  55. 根据权利要求9所述的方法，其特征在于，还包括：The method according to claim 9, further comprising:
    在获得的所述图像中未识别到所述目标对象的情况下，确定所述目标对象丢失时的位置信息；以及If the target object is not recognized in the obtained image, determine the position information when the target object is lost; and
    根据所述目标对象丢失时的位置信息和平滑后的关于所述目标对象的运动轨迹，对所述目标对象的状态信息进行预测。The state information of the target object is predicted according to the position information when the target object is lost and the smoothed motion track of the target object.
  56. 根据权利要求55所述的方法，其特征在于，所述根据所述目标对象丢失时的位置信息和平滑后的关于所述目标对象的运动轨迹，对所述目标对象的状态信息进行预测，包括：The method according to claim 55, wherein the predicting the state information of the target object according to the position information when the target object is lost and the smoothed motion trajectory about the target object comprises :
    根据所述目标对象丢失时的位置信息和平滑后的关于所述目标对象的运动轨迹，生成关于所述目标对象的预测位置的概率分布。According to the position information when the target object is lost and the smoothed motion trajectory about the target object, a probability distribution about the predicted position of the target object is generated.
  57. 根据权利要求56所述的方法，其特征在于，所述预测位置的概率分布的空间占比随着所述目标对象的丢失时间的增长而增加。The method according to claim 56, wherein the spatial proportion of the probability distribution of the predicted position increases as the loss time of the target object increases.
  58. 根据权利要求57所述的方法，其特征在于，所述预测位置的概率分布的空间占比的变化参数与所述目标对象的类型相关。The method according to claim 57, wherein the variation parameter of the spatial proportion of the probability distribution of the predicted position is related to the type of the target object.
  59. 根据权利要求58所述的方法，其特征在于，所述变化参数包括所述预测位置的概率分布的空间占比的增长速度，所述预测位置的概率分布的空间占比的变化参数与所述目标对象的类型相关包括：The method according to claim 58, wherein the change parameter comprises the growth rate of the space proportion of the probability distribution of the predicted position, and the change parameter of the space proportion of the probability distribution of the predicted position is compared with the change parameter of the space proportion of the probability distribution of the predicted position. Related types of target audience include:
    在所述目标对象的类型为生物的情况下，所述预测位置的概率分布的空间占比的第一增长速度在不同方向上相同；In a case where the type of the target object is a living thing, the first growth rate of the space proportion of the probability distribution of the predicted position is the same in different directions;
    在所述目标对象的类型为移动设备的情况下，所述预测位置的概率分布的空间占比的第二增长速度沿所述移动设备的运动方向增加。In a case where the type of the target object is a mobile device, the second increasing speed of the spatial proportion of the probability distribution of the predicted position increases along the movement direction of the mobile device.
  60. 根据权利要求59所述的方法，其特征在于，所述第一增长速度小于所述第二增长速度。The method of claim 59, wherein the first growth rate is less than the second growth rate.
  61. 根据权利要求1所述的方法，其特征在于，所述根据一个或多个所述目标检测框信息和多个所述测距结果确定所述目标对象的状态信息，包括：The method according to claim 1, wherein the determining the state information of the target object according to one or more of the target detection frame information and a plurality of the ranging results comprises:
    根据一个或多个所述目标检测框信息从多个所述测距结果中筛选出有效测距结果；以及Filtering out effective ranging results from the multiple ranging results according to one or more of the target detection frame information; and
    根据一个或多个所述目标检测框信息和所述有效测距结果确定所述目标对象的状态信息。The state information of the target object is determined according to one or more of the target detection frame information and the effective ranging result.
  62. 根据权利要求61所述的方法，其特征在于，所述根据一个或多个所述目标检测框信息从多个所述测距结果中筛选出有效测距结果，包括：The method according to claim 61, wherein the filtering out effective ranging results from a plurality of ranging results according to one or more of the target detection frame information comprises:
    确定每个所述目标检测框信息对应的一个或多个测距结果；以及Determine one or more ranging results corresponding to each of the target detection frame information; and
    对每个所述目标检测框信息对应的一个或多个测距结果进行筛选。One or more ranging results corresponding to each target detection frame information are screened.
  63. 根据权利要求62所述的方法，其特征在于，所述确定每个所述目标检测框信息对应的一个或多个测距结果，包括：The method according to claim 62, wherein the determining one or more ranging results corresponding to each of the target detection frame information comprises:
    根据每个所述目标检测框信息的采样时间点和多个所述测距结果中每个测距结果的采样时间点确定每个所述目标检测框信息对应的一个或多个测距结果。One or more ranging results corresponding to each target detection frame information are determined according to the sampling time point of each target detection frame information and the sampling time point of each ranging result in the plurality of ranging results.
  64. 根据权利要求62所述的方法，其特征在于，所述测距结果包括激光测距结果，所述对每个所述目标检测框信息对应的一个或多个测距结果进行筛选，包括：The method according to claim 62, wherein the distance measurement result comprises a laser distance measurement result, and the screening of one or more distance measurement results corresponding to each of the target detection frame information comprises:
    确定一个或多个所述测距结果中每个测距结果对应的激光光斑；Determining the laser spot corresponding to each of the one or more ranging results;
    根据每个所述测距结果对应的目标检测框信息和每个所述测距结果对应的激光光斑，确定每个所述测距结果的有效性；以及Determine the validity of each distance measurement result according to the target detection frame information corresponding to each distance measurement result and the laser spot corresponding to each distance measurement result; and
    根据每个所述测距结果的有效性对每个所述目标检测框信息对应的一个或多个测距结果进行筛选。One or more ranging results corresponding to each of the target detection frame information are screened according to the validity of each of the ranging results.
  65. 根据权利要求64所述的方法，其特征在于，所述根据每个所述测距结果对应的目标检测框信息和每个所述测距结果对应的激光光斑，确定每个所述测距结果的有效性，包括：The method according to claim 64, wherein the determination of each of the ranging results is based on the target detection frame information corresponding to each of the ranging results and the laser spot corresponding to each of the ranging results The effectiveness of including:
    确定每个所述测距结果对应的目标检测框信息和每个所述测距结果对应的激光光斑的面积重合率；以及Determining the target detection frame information corresponding to each of the distance measurement results and the area overlap ratio of the laser spot corresponding to each of the distance measurement results; and
    根据所述面积重合率确定每个所述测距结果的有效性。The validity of each distance measurement result is determined according to the area coincidence rate.
  66. 根据权利要求65所述的方法，其特征在于，所述根据所述面积重合率确定每个所述测距结果的有效性，包括：The method according to claim 65, wherein the determining the validity of each of the ranging results according to the area coincidence rate comprises:
    将所述面积重合率与预设比例阈值进行比较；Comparing the area coincidence rate with a preset ratio threshold;
    将所述面积重合率大于或等于所述预设比例阈值的测距结果确定为有效测距结果；以及Determining a ranging result whose area coincidence rate is greater than or equal to the preset ratio threshold value as a valid ranging result; and
    将所述面积重合率小于所述预设比例阈值的测距结果确定为无效测距结果。The ranging result whose area coincidence rate is less than the preset ratio threshold is determined as an invalid ranging result.
  67. 根据权利要求64所述的方法，其特征在于，在确定每个所述目标检测框信息对应多个测距结果的情况下，所述根据每个所述测距结果对应的目标检测框信息和每个所述测距结果对应的激光光斑，确定每个所述测距结果的有效性，包括：The method according to claim 64, wherein when it is determined that each of the target detection frame information corresponds to multiple ranging results, the target detection frame information corresponding to each of the ranging results and The laser spot corresponding to each of the distance measurement results to determine the validity of each of the distance measurement results includes:
    根据采样时间相邻的两个所述目标检测框信息，确定采样时间相邻的两个所述目标检测框信息的采样时间之间的多个所述测距结果中每个测距结果分别对应的插值目标检测框信息，得到每个所述测距结果对应的目标检测框信息；以及According to the information of the two target detection frames adjacent to the sampling time, it is determined that each of the multiple ranging results between the sampling times of the two adjacent target detection frame information at the sampling time corresponds to each of the multiple ranging results. Interpolating the target detection frame information of, to obtain target detection frame information corresponding to each of the ranging results; and
    根据每个所述测距结果对应的激光光斑和与每个所述测距结果对应的目标检测框信息，确定每个所述测距结果的有效性。The validity of each ranging result is determined according to the laser spot corresponding to each ranging result and the target detection frame information corresponding to each ranging result.
  68. 根据权利要求61所述的方法，其特征在于，所述根据多个所述目标检测框信息和所述有效测距结果确定所述目标对象的状态信息，包括：The method according to claim 61, wherein the determining the state information of the target object according to a plurality of the target detection frame information and the effective ranging result comprises:
    确定每个所述目标检测框信息对应的有效测距结果；以及Determine the effective ranging result corresponding to each of the target detection frame information; and
    根据每个所述目标检测框信息和每个所述目标检测框信息对应的有效测距结果，确定所述目标对象的状态信息。Determine the state information of the target object according to each of the target detection frame information and the effective ranging result corresponding to each of the target detection frame information.
  69. 根据权利要求68所述的方法，其特征在于，所述确定每个所述目标检测框信息对应的有效测距结果，包括：The method according to claim 68, wherein said determining the effective ranging result corresponding to each said target detection frame information comprises:
    根据每个所述目标检测框信息的采样时间点和每个所述有效测距结果的采样时间点，将每个所述有效测距结果关联至与所述有效测距结果的采样时间点最接近的目标检测框信息。According to the sampling time point of each target detection frame information and the sampling time point of each effective ranging result, each effective ranging result is associated with the sampling time point of the effective ranging result. Information about the detection frame of the approaching target.
  70. 根据权利要求68所述的方法，其特征在于，在每个所述目标检测框信息对应多个有效测距结果的情况下，所述根据每个所述目标检测框信息和每个所述目标检测框信息对应的有效测距结果，确定所述目标对象的状态信息，包括：The method according to claim 68, wherein, in the case that each target detection frame information corresponds to a plurality of valid ranging results, the method according to each of the target detection frame information and each of the target The effective ranging result corresponding to the detection frame information and determining the state information of the target object includes:
    计算每个所述目标检测框信息对应的多个有效测距结果的加权平均值，得到每个所述目标检测框信息对应的目标测距结果；以及Calculating a weighted average of multiple effective ranging results corresponding to each of the target detection frame information to obtain a target ranging result corresponding to each of the target detection frame information; and
    根据每个所述目标检测框信息和每个所述目标检测框信息对应的目标测距结果，确定所述目标对象的状态信息。Determine the state information of the target object according to each of the target detection frame information and the target ranging result corresponding to each of the target detection frame information.
  71. 根据权利要求1所述的方法，其特征在于，所述根据一个或多个所述目标检测框信息和多个所述测距结果确定所述目标对象的状态信息，包括：The method according to claim 1, wherein the determining the state information of the target object according to one or more of the target detection frame information and a plurality of the ranging results comprises:
    确定每个所述目标检测框信息对应的测距结果；Determining a ranging result corresponding to each of the target detection frame information;
    确定每个所述目标检测框信息对应的关于所述目标对象的物理估计尺寸；Determine the physical estimated size of the target object corresponding to each target detection frame information;
    根据每个所述目标检测框信息对应的关于所述目标对象的物理估计尺寸，对每个所述目标检测框信息对应的测距结果进行筛选；以及Screening the distance measurement results corresponding to each target detection frame information according to the physical estimated size of the target object corresponding to each target detection frame information; and
    根据一个或多个所述目标检测框信息以及筛选后的测距结果，确定所述目标对象的状态信息。The state information of the target object is determined according to one or more of the target detection frame information and the filtered ranging result.
  72. 根据权利要求71所述的方法，其特征在于，所述根据每个所述目标检测框信息对应的关于所述目标对象的物理估计尺寸，对每个所述目标检测框信息对应的测距结果进行筛选，包括：The method according to claim 71, wherein the distance measurement result corresponding to each target detection frame information is determined according to the physical estimated size of the target object corresponding to each target detection frame information. Screening, including:
    将每个所述目标检测框信息对应的关于所述目标对象的物理估计尺寸与预设合理范围进行比较；以及Comparing the physical estimated size of the target object corresponding to each target detection frame information with a preset reasonable range; and
    在所述目标检测框信息对应的关于所述目标对象的物理估计尺寸与所述预设合理范围不相符的情况下，滤除所述目标检测框信息对应的测距结果。In the case that the physical estimated size of the target object corresponding to the target detection frame information does not match the preset reasonable range, the ranging result corresponding to the target detection frame information is filtered out.
  73. 根据权利要求72所述的方法，其特征在于，所述确定每个所述目标检测框信息对应的关于所述目标对象的物理估计尺寸，包括：The method according to claim 72, wherein the determining the physical estimated size of the target object corresponding to each of the target detection frame information comprises:
    根据每个所述目标检测框信息和每个所述目标检测框信息采集时的所述成像装置的视场角，确定每个所述目标检测框信息对应的视场角；以及Determine the field angle corresponding to each target detection frame information according to each of the target detection frame information and the field of view angle of the imaging device when each of the target detection frame information is collected; and
    根据每个所述目标检测框信息对应的视场角和每个所述目标检测框信息对应的测距结果，确定每个所述目标检测框信息对应的物理估计尺寸。Determine the physical estimated size corresponding to each target detection frame information according to the field angle corresponding to each target detection frame information and the ranging result corresponding to each target detection frame information.
  74. 根据权利要求72所述的方法，其特征在于，还包括：The method according to claim 72, further comprising:
    确定所述目标对象的对象类型，其中，每个对象类型具有对应的预设合理范围；Determine the object type of the target object, wherein each object type has a corresponding preset reasonable range;
    其中，将每个所述目标检测框信息对应的关于所述目标对象的物理估计尺寸与预设合理范围进行比较包括：Wherein, comparing the physical estimated size of the target object corresponding to each target detection frame information with a preset reasonable range includes:
    根据所述目标对象的对象类型确定目标预设合理范围；以及Determine the preset reasonable range of the target according to the object type of the target object; and
    将每个所述目标检测框信息对应的关于所述目标对象的物理估计尺寸与所述目标预设合理范围进行比较。The physical estimated size of the target object corresponding to each target detection frame information is compared with the preset reasonable range of the target.
  75. 一种目标对象的状态信息确定装置，其特征在于，包括：A device for determining status information of a target object, characterized in that it comprises:
    处理器；processor;
    可读存储介质，用于存储一个或多个程序，A readable storage medium for storing one or more programs,
    其中，当所述一个或多个程序被所述处理器执行时，使得所述处理器执行以下操作：Wherein, when the one or more programs are executed by the processor, the processor is caused to perform the following operations:
    获得通过可移动平台携带的成像装置获得的关于所述目标对象的多帧图像；Obtaining multiple frames of images about the target object obtained by an imaging device carried by a movable platform;
    对多帧所述图像中的每一帧图像进行识别，得到关于所述目标对象的多个检测框信息；Recognizing each of the multiple frames of the images to obtain multiple detection frame information about the target object;
    获得所述可移动平台与所述目标对象之间的多个测距结果；Obtaining multiple distance measurement results between the movable platform and the target object;
    根据多个所述检测框信息从多个所述测距结果中筛选出有效测距结果；以及Filtering out effective ranging results from the plurality of ranging results according to the plurality of detection frame information; and
    根据多个所述检测框信息和所述有效测距结果确定所述目标对象的状态信息。The state information of the target object is determined according to a plurality of the detection frame information and the effective ranging result.
  76. 根据权利要求75所述的装置，其特征在于，所述处理器根据多个所述检测框信息从多个所述测距结果中筛选出有效测距结果，包括：The device according to claim 75, wherein the processor screens the effective ranging results from the plurality of ranging results according to a plurality of the detection frame information, comprising:
    确定每个所述检测框信息对应的一个或多个测距结果；以及Determine one or more ranging results corresponding to each of the detection frame information; and
    对每个所述检测框信息对应的一个或多个测距结果进行筛选。One or more ranging results corresponding to each detection frame information are screened.
  77. 根据权利要求76所述的装置，其特征在于，所述处理器确定每个所述检测框信息对应的一个或多个测距结果，包括：The apparatus according to claim 76, wherein the processor determining one or more ranging results corresponding to each detection frame information comprises:
    根据每个所述检测框信息的采样时间点和多个所述测距结果中每个测距结果的采样时间点确定每个所述检测框信息对应的一个或多个测距结果。Determine one or more ranging results corresponding to each detection frame information according to the sampling time point of each detection frame information and the sampling time point of each ranging result in the plurality of ranging results.
  78. 根据权利要求76所述的装置，其特征在于，所述测距结果包括激光测距结果，所述处理器对每个所述检测框信息对应的一个或多个测距结果进行筛选，包括：The device according to claim 76, wherein the distance measurement result comprises a laser distance measurement result, and the processor screening one or more distance measurement results corresponding to each detection frame information comprises:
    确定一个或多个所述测距结果中每个测距结果对应的激光光斑；Determining the laser spot corresponding to each of the one or more ranging results;
    根据每个所述测距结果对应的检测框信息和每个所述测距结果对应的激光光斑，确定每个所述测距结果的有效性；以及Determine the validity of each distance measurement result according to the detection frame information corresponding to each distance measurement result and the laser spot corresponding to each distance measurement result; and
    根据每个所述测距结果的有效性对每个所述检测框信息对应的一个或多个测距结果进行筛选。One or more ranging results corresponding to each of the detection frame information are screened according to the validity of each of the ranging results.
  79. 根据权利要求78所述的装置，其特征在于，所述处理器根据每个所述测距结果对应的检测框信息和每个所述测距结果对应的激光光斑，确定每个所述测距结果的有效性，包括：The device according to claim 78, wherein the processor determines each of the distance measurement results according to the detection frame information corresponding to each of the distance measurement results and the laser spot corresponding to each of the distance measurement results. The validity of the results includes:
    确定每个所述测距结果对应的检测框信息和每个所述测距结果对应的激光光斑的面积重合率；以及Determining the detection frame information corresponding to each of the distance measurement results and the area overlap ratio of the laser spot corresponding to each of the distance measurement results; and
    根据所述面积重合率确定每个所述测距结果的有效性。The validity of each distance measurement result is determined according to the area coincidence rate.
  80. 根据权利要求79所述的装置，其特征在于，所述处理器根据所述面积重合率确定每个所述测距结果的有效性，包括：The device according to claim 79, wherein the processor determining the validity of each of the ranging results according to the area coincidence rate comprises:
    将所述面积重合率与预设比例阈值进行比较；Comparing the area coincidence rate with a preset ratio threshold;
    将所述面积重合率大于或等于所述预设比例阈值的测距结果确定为有效测距结果；以及Determining a ranging result whose area coincidence rate is greater than or equal to the preset ratio threshold value as a valid ranging result; and
    将所述面积重合率小于所述预设比例阈值的测距结果确定为无效测距结果。The ranging result whose area coincidence rate is less than the preset ratio threshold is determined as an invalid ranging result.
  81. 根据权利要求78所述的装置，其特征在于，在确定每个所述检测框信息对应多个测距结果的情况下，所述处理器根据每个所述测距结果对应的检测框信息和每个所述测距结果对应的激光光斑，确定每个所述测距结果的有效性，包括：The device according to claim 78, wherein, in the case where it is determined that each of the detection frame information corresponds to multiple ranging results, the processor is based on the detection frame information corresponding to each of the ranging results and The laser spot corresponding to each of the distance measurement results to determine the validity of each of the distance measurement results includes:
    根据采样时间相邻的两个所述检测框信息，确定采样时间相邻的两个所述检测框信息的采样时间之间的多个所述测距结果中每个测距结果分别对应的插值检测框信息，得到每个所述测距结果对应的检测框信息；以及Determine the interpolation value corresponding to each of the multiple distance measurement results between the sampling times of the two detection frame information adjacent to the sampling time according to the information of the two detection frame information adjacent to the sampling time Detection frame information to obtain detection frame information corresponding to each of the ranging results; and
    根据每个所述测距结果对应的激光光斑和与每个所述测距结果对应的检测框信息，确定每个所述测距结果的有效性。The validity of each distance measurement result is determined according to the laser spot corresponding to each distance measurement result and the detection frame information corresponding to each distance measurement result.
  82. 根据权利要求75所述的装置，其特征在于，所述处理器根据多个所述检测框信息和所述有效测距结果确定所述目标对象的状态信息，包括：The device according to claim 75, wherein the processor determining the state information of the target object according to a plurality of the detection frame information and the effective ranging result comprises:
    确定每个所述检测框信息对应的有效测距结果；以及Determine the effective ranging result corresponding to each of the detection frame information; and
    根据每个所述检测框信息和每个所述检测框信息对应的有效测距结果，确定所述目标对象的状态信息。Determine the state information of the target object according to each of the detection frame information and the effective ranging result corresponding to each of the detection frame information.
  83. 根据权利要求82所述的装置，其特征在于，所述处理器确定每个所述检测框信息对应的有效测距结果，包括：The device according to claim 82, wherein the processor determining the effective ranging result corresponding to each detection frame information comprises:
    根据每个所述检测框信息的采样时间点和每个所述有效测距结果的采样时间点，将每个所述有效测距结果关联至与所述有效测距结果的采样时间点最接近的检测框信息。According to the sampling time point of each detection frame information and the sampling time point of each effective ranging result, each effective ranging result is associated with the sampling time point closest to the effective ranging result The detection frame information.
  84. 根据权利要求82所述的装置，其特征在于，在每个所述检测框信息对应多个有效测距结果的情况下，所述处理器根据每个所述检测框信息和每个所述检测框信息对应的有效测距结果，确定所述目标对象的状态信息，包括：The device according to claim 82, wherein, in a case where each of the detection frame information corresponds to multiple valid ranging results, the processor is configured according to each of the detection frame information and each of the detection frames. The effective ranging result corresponding to the frame information and determining the state information of the target object includes:
    计算每个所述检测框信息对应的多个有效测距结果的加权平均值，得到每个所述检测框信息对应的目标测距结果；以及Calculating the weighted average of the multiple effective ranging results corresponding to each of the detection frame information to obtain the target ranging result corresponding to each of the detection frame information; and
    根据每个所述检测框信息和每个所述检测框信息对应的目标测距结果，确定所述目标对象的状态信息。Determine the state information of the target object according to each detection frame information and a target ranging result corresponding to each detection frame information.
  85. 根据权利要求75所述的装置，其特征在于，所述处理器还执行以下操作：The device according to claim 75, wherein the processor further performs the following operations:
    确定每个所述检测框信息对应的有效测距结果；Determining an effective ranging result corresponding to each of the detection frame information;
    确定每个所述检测框信息对应的关于所述目标对象的物理估计尺寸；以及Determining the physical estimated size of the target object corresponding to each of the detection frame information; and
    根据每个所述检测框信息对应的关于所述目标对象的物理估计尺寸，对每个所述检测框信息对应的有效测距结果进行筛选。According to the physical estimated size of the target object corresponding to each detection frame information, the effective ranging result corresponding to each detection frame information is screened.
  86. 根据权利要求85所述的装置，其特征在于，所述处理器根据每个所述检测框信息对应的关于所述目标对象的物理估计尺寸，对每个所述检测框信息对应的有效测距结果进行筛选，包括：The device according to claim 85, wherein the processor determines the effective distance measurement corresponding to each detection frame information according to the physical estimated size of the target object corresponding to each detection frame information. The results are filtered, including:
    将每个所述检测框信息对应的关于所述目标对象的物理估计尺寸与预设合理范围进行比较；以及Comparing the physical estimated size of the target object corresponding to each detection frame information with a preset reasonable range; and
    在所述检测框信息对应的关于所述目标对象的物理估计尺寸与所述预设合理范围不相符的情况下，滤除所述检测框信息对应的有效测距结果。In a case where the physical estimated size of the target object corresponding to the detection frame information does not match the preset reasonable range, the effective ranging result corresponding to the detection frame information is filtered out.
  87. 根据权利要求86所述的装置，其特征在于，所述处理器确定每个所述检测框信息对应的关于所述目标对象的物理估计尺寸，包括：The device according to claim 86, wherein the processor determining the physical estimated size of the target object corresponding to each detection frame information comprises:
    根据每个所述检测框信息和每个所述检测框信息采集时的所述成像装置的视场角，确定每个所述检测框信息对应的视场角；以及Determine the field of view corresponding to each of the detection frame information according to each of the detection frame information and the field of view of the imaging device when each of the detection frame information is collected; and
    根据每个所述检测框信息对应的视场角和每个所述检测框信息对应的有效测距结果，确定每个所述检测框信息对应的物理估计尺寸。Determine the physical estimated size corresponding to each detection frame information according to the field of view angle corresponding to each detection frame information and the effective ranging result corresponding to each detection frame information.
  88. 根据权利要求86所述的装置，其特征在于，所述处理器还执行以下操作：The device according to claim 86, wherein the processor further performs the following operations:
    确定所述目标对象的对象类型，其中，每个对象类型具有对应的预设合理范围；Determine the object type of the target object, wherein each object type has a corresponding preset reasonable range;
    所述将每个所述检测框信息对应的关于所述目标对象的物理估计尺寸与预设合理范围进行比较，包括：The comparing the physical estimated size of the target object corresponding to each detection frame information with a preset reasonable range includes:
    根据所述目标对象的对象类型确定目标预设合理范围；以及Determine the preset reasonable range of the target according to the object type of the target object; and
    将每个所述检测框信息对应的关于所述目标对象的物理估计尺寸与所述目标预设合理范围进行比较。The physical estimated size of the target object corresponding to each detection frame information is compared with the preset reasonable range of the target.
  89. 根据权利要求86所述的装置，其特征在于，所述处理器还执行以下操作：The device according to claim 86, wherein the processor further performs the following operations:
    在所述滤除所述检测框信息对应的有效测距结果之后，将与滤除了有效测距结果的检测框信息的采样时间相邻的检测框信息对应的有效测距结果，确定为所述滤除了有效测距结果的检测框信息对应的有效测距结果。After the effective ranging result corresponding to the detection frame information is filtered out, the effective ranging result corresponding to the detection frame information adjacent to the sampling time of the detection frame information from which the effective ranging result is filtered out is determined as the The effective ranging result corresponding to the detection frame information of the effective ranging result is filtered out.
  90. 根据权利要求75所述的装置，其特征在于，所述处理器根据多个所述检测框信息和所述有效测距结果确定所述目标对象的状态信息，包括：The device according to claim 75, wherein the processor determines the state information of the target object according to a plurality of the detection frame information and the effective ranging result, comprising:
    确定一个或多个目标检测框信息中的每个所述目标检测框信息分别对应的有效测距结果，一个或多个所述目标检测框信息为多个所述检测框信息中满足预设条件的检测框信息；以及Determine the effective ranging result corresponding to each target detection frame information in one or more target detection frame information, and one or more of the target detection frame information is that a preset condition is satisfied among the plurality of detection frame information The check box information of; and
    根据每个所述目标检测框信息和每个所述目标检测框信息对应的有效测距结果，确定所述目标对象的状态信息。Determine the state information of the target object according to each of the target detection frame information and the effective ranging result corresponding to each of the target detection frame information.
  91. 根据权利要求90所述的装置，其特征在于，所述状态信息包括位置信息；其中，所述处理器根据每个所述目标检测框信息和每个所述目标检测框信息对应的有效测距结果，确定所述目标对象的状态信息，包括：The device according to claim 90, wherein the status information includes position information; wherein, the processor is based on each of the target detection frame information and the effective ranging corresponding to each of the target detection frame information. As a result, the status information of the target object is determined, including:
    根据每个所述目标检测框信息和每个所述目标检测框信息对应的有效测距结果，确定每个所述目标检测框信息对应的关于所述目标对象的初始位置信息，得到多个初始位置信息；以及According to each of the target detection frame information and the effective ranging result corresponding to each of the target detection frame information, the initial position information about the target object corresponding to each target detection frame information is determined, and multiple initial positions are obtained. Location information; and
    根据每个所述目标检测框信息对应的采样时间点，对多个所述初始位置信息进行筛选，得到一个或多个有效初始位置信息；Filtering a plurality of the initial position information according to the sampling time point corresponding to each of the target detection frame information to obtain one or more effective initial position information;
    根据一个或多个所述有效初始位置信息，确定所述目标对象的状态信息。Determine the state information of the target object according to one or more of the effective initial position information.
  92. 根据权利要求91所述的装置，其特征在于，所述处理器根据每个所述目标检测框信息对应的采样时间点，对多个所述初始位置信息进行筛选，包括：The device according to claim 91, wherein the processor screens a plurality of the initial position information according to a sampling time point corresponding to each of the target detection frame information, comprising:
    按照时间顺序，依次对每个所述目标检测框信息对应的关于所述目标对象的初始位置信息进行筛选。According to the time sequence, the initial position information about the target object corresponding to each target detection frame information is sequentially filtered.
  93. 根据权利要求92所述的装置，其特征在于，所述处理器按照时间顺序，依次对每个所述目标检测框信息对应的关于所述目标对象的初始位置信息进行筛选，包括：The device according to claim 92, wherein the processor sequentially screens the initial position information of the target object corresponding to each of the target detection frame information in a chronological order, comprising:
    计算当前正筛选的初始位置信息和与所述当前正筛选的初始位置信息的采样时间相邻的后一个初始位置信息的采样时间点之间的时间差；Calculating the time difference between the initial location information currently being screened and the sampling time point of the next initial location information adjacent to the sampling time of the initial location information currently being screened;
    将所述时间差与状态变量阈值进行比较；Comparing the time difference with a state variable threshold;
    如果所述时间差小于所述状态变量阈值，滤除当前正筛选的初始位置信息；以及If the time difference is less than the state variable threshold, filter out the initial position information currently being screened; and
    如果所述时间差大于或等于所述状态变量阈值，保留所述当前正筛选的初始位置信息，其中，被保留的所述当前正筛选的初始位置信息为所述有效初始位置信息。If the time difference is greater than or equal to the state variable threshold, the initial location information currently being screened is retained, wherein the retained initial location information currently being screened is the effective initial location information.
  94. 根据权利要求93所述的装置，其特征在于，所述状态变量阈值随确定的有效初始位置信息发生变化。The device according to claim 93, wherein the state variable threshold changes with the determined effective initial position information.
  95. 根据权利要求94所述的装置，其特征在于，所述状态变量阈值为所述当前正筛选的初始位置信息和与所述当前正筛选的初始位置信息的采样时间相邻的有效初始位置信息的采样时间点之间的时间差。The device according to claim 94, wherein the state variable threshold is the value of the initial position information currently being screened and the effective initial position information adjacent to the sampling time of the initial position information being screened. The time difference between sampling time points.
  96. 根据权利要求90所述的装置，其特征在于，所述处理器还执行以下操作：The device according to claim 90, wherein the processor further performs the following operations:
    根据多个所述检测框信息确定所述目标对象是否出现满足预设条件的移动；以及Determining whether the target object has moved that meets a preset condition according to multiple pieces of detection frame information; and
    在所述目标对象出现了满足预设条件的移动的情况下，将所述目标对象满足所述预设条件的移动时对应的检测框信息确定为所述目标检测框信息。In the case that the target object has a movement that meets a preset condition, the detection frame information corresponding to the movement of the target object that meets the preset condition is determined as the target detection frame information.
  97. 根据权利要求96所述的装置，其特征在于，所述处理器根据多个所述检测框信息确定所述目标对象是否出现满足预设条件的移动，包括：The device according to claim 96, wherein the processor determining whether the target object moves that meets a preset condition according to a plurality of the detection frame information comprises:
    针对多个所述检测框信息中任意相邻的第一检测框信息和第二检测框信息，获得所述第一检测框信息采集时的所述成像装置的第一状态信息，和所述第二检测框信息采集时的所述成像装置的第二状态信息；For any adjacent first detection frame information and second detection frame information among the plurality of detection frame information, obtain the first state information of the imaging device when the first detection frame information is collected, and the first state information of the imaging device when the first detection frame information is collected. 2. The second state information of the imaging device when the detection frame information is collected;
    根据所述第一检测框信息对应的有效测距结果和所述第一状态信息，确定与所述第一检测框信息对应的关于所述目标对象的初始位置信息的第一概率分布；Determine, according to the effective ranging result corresponding to the first detection frame information and the first state information, a first probability distribution of the initial position information of the target object corresponding to the first detection frame information;
    根据所述第二检测框信息对应的有效测距结果和所述第二状态信息，确定与所述第二检测框信息对应的关于所述目标对象的初始位置信息的第二概率分布；以及Determining a second probability distribution of the initial position information of the target object corresponding to the second detection frame information according to the effective ranging result corresponding to the second detection frame information and the second state information; and
    根据所述第一概率分布和所述第二概率分布，确定所述目标对象是否出现满足预设条件的移动。According to the first probability distribution and the second probability distribution, it is determined whether the target object moves that meets a preset condition.
  98. 根据权利要求97所述的装置，其特征在于，所述处理器根据所述第一概率分布和所述第二概率分布，确定所述目标对象是否出现满足预设条件的移动，包括：The device according to claim 97, wherein the processor determining whether the target object moves that meets a preset condition according to the first probability distribution and the second probability distribution, comprises:
    根据所述第一概率分布和所述第二概率分布，确定概率密度最高的空间位置；Determine the spatial position with the highest probability density according to the first probability distribution and the second probability distribution;
    计算所述概率密度最高的空间位置距离所述第一概率分布的第一概率分布中心位置的第一距离；Calculating the first distance between the spatial position with the highest probability density and the center position of the first probability distribution of the first probability distribution;
    计算所述概率密度最高的空间位置距离所述第二概率分布的第二概率分布中心位置的第二距离；Calculating the second distance between the spatial position with the highest probability density and the center position of the second probability distribution of the second probability distribution;
    根据所述第一距离和所述第二距离确定所述第一检测框信息和所述第二检测框信息之间的概率距离；以及Determining the probability distance between the first detection frame information and the second detection frame information according to the first distance and the second distance; and
    如果所述概率距离大于或等于预设阈值，确定所述目标对象出现满足预设条件的移动。If the probability distance is greater than or equal to a preset threshold, it is determined that the target object moves that meets a preset condition.
  99. 根据权利要求91所述的装置，其特征在于，所述处理器根据一个或多个所述有效初始位置信息，确定所述目标对象的状态信息，包括：The device according to claim 91, wherein the processor determines the state information of the target object according to one or more of the effective initial position information, comprising:
    对一个或多个所述有效初始位置信息进行优化，以平滑关于所述目标对象的运动轨迹。One or more of the effective initial position information is optimized to smooth the movement trajectory of the target object.
  100. 根据权利要求99所述的装置，其特征在于，所述处理器对一个或多个所述有效初始位置信息进行优化，包括：The device of claim 99, wherein the processor optimizing one or more of the effective initial position information comprises:
    对一个或多个所述有效初始位置信息进行非线性优化，以最小化目标偏差，其中，所述目标偏差与所述检测框信息和/或有效测距结果相关，每个所述有效初始位置信息进行非线性优化后具有对应的优化位置信息。Non-linear optimization is performed on one or more of the effective initial position information to minimize the target deviation, wherein the target deviation is related to the detection frame information and/or the effective ranging result, and each of the effective initial positions After non-linear optimization, the information has corresponding optimized location information.
  101. 根据权利要求100所述的装置，其特征在于，所述目标偏差包括第一偏差和/或第二偏差；The device according to claim 100, wherein the target deviation comprises a first deviation and/or a second deviation;
    所述第一偏差包括关于所述有效初始位置信息与用于计算得到所述有效初始位置信息的目标检测框信息和有效测距结果之间的观测偏差；The first deviation includes an observation deviation between the effective initial position information and the target detection frame information used for calculating the effective initial position information and the effective ranging result;
    所述第二偏差包括关于相邻所述有效初始位置信息之间的平滑程度与先验值的偏差。The second deviation includes a deviation between the degree of smoothness between adjacent effective initial position information and a priori value.
  102. 根据权利要求101所述的装置，其特征在于，所述第一偏差通过用于计算得到所述有效初始位置信息的目标检测框信息和有效测距结果的概率密度函数进行表征。The device according to claim 101, wherein the first deviation is characterized by a probability density function used to calculate the target detection frame information of the effective initial position information and the effective ranging result.
  103. 根据权利要求101所述的装置，其特征在于，所述处理器对一个或多个所述有效初始位置信息进行优化，还包括：The device according to claim 101, wherein the processor optimizes one or more of the effective initial position information, further comprising:
    确定每个所述有效初始位置信息进行非线性优化后对应的优化位置信息是否异常；以及Determine whether the optimized position information corresponding to each effective initial position information after nonlinear optimization is abnormal; and
    将异常的优化位置信息进行滤除。Filter out abnormal optimized location information.
  104. 根据权利要求103所述的装置，其特征在于，所述目标偏差包括所述第一偏差，所述第一偏差包括第一子偏差和/或第二子偏差；The device according to claim 103, wherein the target deviation includes the first deviation, and the first deviation includes a first sub-deviation and/or a second sub-deviation;
    所述第一子偏差为关于所述有效初始位置信息与用于计算得到所述有效初始位置信息的目标检测框信息之间的观测偏差；The first sub-deviation is an observation deviation between the effective initial position information and the target detection frame information used to calculate the effective initial position information;
    所述第二子偏差为关于所述有效初始位置信息与用于计算得到所述有效初始位置信息的有效测距结果之间的观测偏差；The second sub-deviation is an observation deviation between the effective initial position information and the effective ranging result used to calculate the effective initial position information;
    其中，所述处理器确定每个所述有效初始位置信息进行非线性优化后对应的优化位置信息是否异常，包括：Wherein, the processor determining whether the optimized position information corresponding to each effective initial position information after non-linear optimization is abnormal, includes:
    若确定所述第一子偏差对应的检测框信息异常，和/或确定所述第二子偏差对应的有效测距结果异常，则将异常的检测框信息和/或异常的有效测距结果对应的优化位置确定为异常的优化位置。If it is determined that the detection frame information corresponding to the first sub-deviation is abnormal, and/or it is determined that the effective ranging result corresponding to the second sub-deviation is abnormal, then the abnormal detection frame information and/or the abnormal effective ranging result are corresponded to The optimized position of is determined as the abnormal optimized position.
  105. 根据权利要求103所述的装置，其特征在于，所述处理器对一个或多个所述有效初始位置信息进行优化，还包括：The device according to claim 103, wherein the processor optimizes one or more of the effective initial position information, further comprising:
    在将异常的优化位置信息进行滤除之后，对剩余的优化位置信息进行非线性优化，以最小化所述目标偏差，其中，每个所述剩余的优化位置信息进行非线性优化后具有对应的最终优化位置信息。After filtering out the abnormal optimized position information, the remaining optimized position information is nonlinearly optimized to minimize the target deviation, wherein each of the remaining optimized position information has a corresponding non-linear optimization Finally optimize the location information.
  106. 根据权利要求99所述的装置，其特征在于，所述处理器还执行以下操作：The device of claim 99, wherein the processor further performs the following operations:
    在获得的图像中未识别到所述目标对象的情况下，确定所述目标对象丢失时的位置信息；以及If the target object is not recognized in the obtained image, determine the position information when the target object is lost; and
    根据所述目标对象丢失时的位置信息和平滑后的关于所述目标对象的运动轨迹，对所述目标对象的状态信息进行预测。The state information of the target object is predicted according to the position information when the target object is lost and the smoothed motion track of the target object.
  107. 根据权利要求106所述的装置，其特征在于，所述处理器根据所述目标对象丢失时的位置信息和平滑后的关于所述目标对象的运动轨迹，对所述目标对象的状态信息进行预测包括：The device according to claim 106, wherein the processor predicts the state information of the target object based on the position information when the target object is lost and the smoothed motion trajectory about the target object include:
    根据所述目标对象丢失时的位置信息和平滑后的关于所述目标对象的运动轨迹，生成关于所述目标对象的预测位置的概率分布。According to the position information when the target object is lost and the smoothed motion trajectory about the target object, a probability distribution about the predicted position of the target object is generated.
  108. 根据权利要求107所述的装置，其特征在于，所述预测位置的概率分布的空间占比随着所述目标对象的丢失时间的增长而增加。The apparatus according to claim 107, wherein the spatial proportion of the probability distribution of the predicted position increases as the loss time of the target object increases.
  109. 根据权利要求108所述的装置，其特征在于，所述预测位置的概率分布的空间占比的变化参数与所述目标对象的类型相关。The apparatus according to claim 108, wherein the change parameter of the spatial proportion of the probability distribution of the predicted position is related to the type of the target object.
  110. 根据权利要求109所述的装置，其特征在于，所述变化参数包括所述预测位置的概率分布的空间占比的增长速度；The device according to claim 109, wherein the change parameter comprises a growth rate of the spatial proportion of the probability distribution of the predicted position;
    在所述目标对象的类型为生物的情况下，所述预测位置的概率分布的空间占比的第一增长速度在不同方向上相同；In a case where the type of the target object is a living thing, the first growth rate of the space proportion of the probability distribution of the predicted position is the same in different directions;
    在所述目标对象的类型为移动设备的情况下，所述预测位置的概率分布的空间占比的第二增长速度沿所述移动设备的运动方向增加。In a case where the type of the target object is a mobile device, the second increasing speed of the spatial proportion of the probability distribution of the predicted position increases along the movement direction of the mobile device.
  111. 根据权利要求110所述的装置，其特征在于，所述第一增长速度小于所述第二增长速度。The device of claim 110, wherein the first growth rate is less than the second growth rate.
  112. 一种目标对象的状态信息确定装置，其特征在于，包括：A device for determining status information of a target object, characterized in that it comprises:
    处理器；processor;
    可读存储介质，用于存储一个或多个程序，A readable storage medium for storing one or more programs,
    其中，当所述一个或多个程序被所述处理器执行时，使得所述处理器执行以下操作：Wherein, when the one or more programs are executed by the processor, the processor is caused to perform the following operations:
    获得通过可移动平台携带的成像装置获得的关于所述目标对象的多帧图像；Obtaining multiple frames of images about the target object obtained by an imaging device carried by a movable platform;
    对多帧所述图像中的每一帧图像进行识别，得到关于所述目标对象的多个检测框信息；Recognizing each of the multiple frames of the images to obtain multiple detection frame information about the target object;
    确定多个所述检测框信息中满足预设条件的一个或多个目标检测框信息；Determining one or more target detection frame information that meets a preset condition among the plurality of detection frame information;
    获得所述可移动平台与所述目标对象之间的多个测距结果；以及Obtaining a plurality of ranging results between the movable platform and the target object; and
    根据一个或多个所述目标检测框信息和多个所述测距结果确定所述目标对象的状态信息。The state information of the target object is determined according to one or more of the target detection frame information and a plurality of the ranging results.
  113. 根据权利要求112所述的装置，其特征在于，所述处理器根据一个或多个所述目标检测框信息和多个所述测距结果确定所述目标对象的状态信息，包括：The device according to claim 112, wherein the processor determines the state information of the target object according to one or more of the target detection frame information and a plurality of the ranging results, comprising:
    确定每个所述目标检测框信息对应的测距结果；以及Determining the ranging result corresponding to each of the target detection frame information; and
    根据每个所述目标检测框信息和每个所述目标检测框信息对应的测距结果，确定所述目标对象的状态信息。The state information of the target object is determined according to each of the target detection frame information and the ranging result corresponding to each of the target detection frame information.
  114. 根据权利要求113所述的装置，其特征在于，所述状态信息包括位置信息；其中，所述处理器确定多个所述检测框信息中满足预设条件的一个或多个目标检测框信息，包括：The device according to claim 113, wherein the state information includes position information; wherein the processor determines one or more target detection frame information that meets a preset condition among the plurality of detection frame information, include:
    根据每个所述检测框信息对应的采样时间点，对多个所述检测框信息进行筛选，得到一个或多个目标检测框信息。According to the sampling time point corresponding to each detection frame information, a plurality of detection frame information is screened to obtain one or more target detection frame information.
  115. 根据权利要求114所述的装置，其特征在于，所述处理器根据每个所述检测框信息对应的采样时间点，对多个所述检测框信息进行筛选，包括：The device according to claim 114, wherein the processor screens a plurality of the detection frame information according to the sampling time point corresponding to each detection frame information, comprising:
    按照时间顺序，依次对每个所述检测框信息进行筛选。According to the chronological order, the information of each detection frame is screened in turn.
  116. 根据权利要求115所述的装置，其特征在于，所述处理器按照时间顺序，依次对每个所述检测框信息进行筛选包括：The device according to claim 115, wherein the processor sequentially screening each of the detection frame information in a time sequence comprises:
    计算当前正筛选的检测框信息和与所述当前正筛选的检测框信息的采样时间相邻的后一个检测框信息的采样时间点之间的时间差；Calculating the time difference between the detection frame information currently being screened and the sampling time point of the next detection frame information adjacent to the sampling time of the detection frame information currently being screened;
    将所述时间差与状态变量阈值进行比较；Comparing the time difference with a state variable threshold;
    如果所述时间差小于所述状态变量阈值，滤除当前正筛选的检测框信息；以及If the time difference is less than the state variable threshold, filter out the detection frame information currently being screened; and
    如果所述时间差大于或等于所述状态变量阈值，保留所述当前正筛选的检测框信息，其中，被保留的所述当前正筛选的检测框信息为目标检测框信息。If the time difference is greater than or equal to the state variable threshold, the detection frame information currently being screened is retained, wherein the retained detection frame information currently being screened is target detection frame information.
  117. 根据权利要求116所述的装置，其特征在于，所述状态变量阈值随确定的目标检测框信息发生变化。The device of claim 116, wherein the state variable threshold changes with the determined target detection frame information.
  118. 根据权利要求117所述的装置，其特征在于，所述状态变量阈值为所述当前正筛选的检测框信息和与所述当前正筛选的检测框信息的采样时间相邻的目标检测框信息的采样时间点之间的时间差。The device according to claim 117, wherein the state variable threshold is the sum of the detection frame information currently being screened and the target detection frame information adjacent to the sampling time of the detection frame information currently being screened. The time difference between sampling time points.
  119. 根据权利要求112所述的装置，其特征在于，所述处理器确定多个所述检测框信息中满足预设条件的多个目标检测框信息，包括：The device according to claim 112, wherein the processor determining multiple target detection frame information that meets a preset condition among the multiple detection frame information comprises:
    根据多个所述检测框信息确定所述目标对象是否出现满足预设条件的移动；以及Determining whether the target object has moved that meets a preset condition according to multiple pieces of detection frame information; and
    在所述目标对象出现了满足预设条件的移动的情况下，将所述目标对象满足所述预设条件的移动时对应的检测框信息确定为所述目标检测框信息。In the case that the target object has a movement that meets a preset condition, the detection frame information corresponding to the movement of the target object that meets the preset condition is determined as the target detection frame information.
  120. 根据权利要求119所述的装置，其特征在于，所述处理器根据多个所述检测框信息确定所述目标对象是否出现满足预设条件的移动，包括：The device according to claim 119, wherein the processor determining whether the target object has moved that meets a preset condition according to a plurality of the detection frame information comprises:
    针对多个所述检测框信息中任意相邻的第一检测框信息和第二检测框信息，获得所述第一检测框信息采集时的所述成像装置的第一状态信息，和所述第二检测框信息采集时的所述成像装置的第二状态信息；For any adjacent first detection frame information and second detection frame information among the plurality of detection frame information, obtain the first state information of the imaging device when the first detection frame information is collected, and the first state information of the imaging device when the first detection frame information is collected. 2. The second state information of the imaging device when the detection frame information is collected;
    根据所述第一检测框信息对应的测距结果和所述第一状态信息，确定与所述第一检测框信息对应的关于所述目标对象的初始位置信息的第一概率分布；Determine a first probability distribution of the initial position information of the target object corresponding to the first detection frame information according to the ranging result corresponding to the first detection frame information and the first state information;
    根据所述第二检测框信息对应的测距结果和所述第二状态信息，确定与所述第二检测框信息对应的关于所述目标对象的初始位置信息的第二概率分布；以及Determining a second probability distribution of the initial position information of the target object corresponding to the second detection frame information according to the ranging result corresponding to the second detection frame information and the second state information; and
    根据所述第一概率分布和所述第二概率分布，确定所述目标对象是否出现满足预设条件的移动。According to the first probability distribution and the second probability distribution, it is determined whether the target object moves that meets a preset condition.
  121. 根据权利要求120所述的装置，其特征在于，所述处理器根据所述第一概率分布和所述第二概率分布，确定所述目标对象是否出现满足预设条件的移动包括：The device according to claim 120, wherein the processor determining whether the target object has moved that meets a preset condition according to the first probability distribution and the second probability distribution comprises:
    根据所述第一概率分布和所述第二概率分布，确定概率密度最高的空间位置；Determine the spatial position with the highest probability density according to the first probability distribution and the second probability distribution;
    计算所述概率密度最高的空间位置距离所述第一概率分布的第一概率分布中心位置的第一距离；Calculating the first distance between the spatial position with the highest probability density and the center position of the first probability distribution of the first probability distribution;
    计算所述概率密度最高的空间位置距离所述第二概率分布的第二概率分布中心位置的第二距离；Calculating the second distance between the spatial position with the highest probability density and the center position of the second probability distribution of the second probability distribution;
    根据所述第一距离和所述第二距离确定所述第一检测框信息和所述第二检测框信息之间的概率距离；以及Determining the probability distance between the first detection frame information and the second detection frame information according to the first distance and the second distance; and
    如果所述概率距离大于或等于预设阈值，确定所述目标对象出现满足预设条件的移动。If the probability distance is greater than or equal to a preset threshold, it is determined that the target object moves that meets a preset condition.
  122. 根据权利要求114所述的装置，其特征在于，所述处理器根据一个或多个所述目标检测框信息和多个所述测距结果确定所述目标对象的状态信息，包括：The device of claim 114, wherein the processor determines the state information of the target object according to one or more of the target detection frame information and a plurality of the ranging results, comprising:
    对一个或多个所述目标检测框信息对应的有效初始位置信息进行优化，以平滑关于所述目标对象的运动轨迹。The effective initial position information corresponding to one or more of the target detection frame information is optimized to smooth the movement track of the target object.
  123. 根据权利要求122所述的装置，其特征在于，所述处理器对一个或多个所述目标检测框信息对应的有效初始位置信息进行优化，包括：The device according to claim 122, wherein the processor optimizing effective initial position information corresponding to one or more of the target detection frame information comprises:
    对一个或多个所述目标检测框信息对应的有效初始位置信息进行非线性优化，以最小化目标偏差，其中，所述目标偏差与所述检测框信息和/或测距结果相关，每个所述有效初始位置信息进行非线性优化后具有对应的优化位置信息。Non-linear optimization is performed on the effective initial position information corresponding to one or more of the target detection frame information to minimize the target deviation, wherein the target deviation is related to the detection frame information and/or the ranging result, each The effective initial position information has corresponding optimized position information after being non-linearly optimized.
  124. 根据权利要求123所述的装置，其特征在于，所述目标偏差包括第一偏差和/或第二偏差；The device according to claim 123, wherein the target deviation comprises a first deviation and/or a second deviation;
    所述第一偏差包括关于所述有效初始位置信息与用于计算得到所述有效初始位置信息的目标检测框信息和测距结果之间的观测偏差；The first deviation includes an observation deviation between the effective initial position information and the target detection frame information used for calculating the effective initial position information and the ranging result;
    所述第二偏差包括关于多个所述有效初始位置信息中的相邻有效初始位置信息之间的平滑程度与先验值的偏差。The second deviation includes a deviation between the degree of smoothing between adjacent effective initial position information in the plurality of effective initial position information and a prior value.
  125. 根据权利要求124所述的装置，其特征在于，所述第一偏差通过用于计算得到所述有效初始位置信息的目标检测框信息和测距结果的概率密度函数进行表征。The device according to claim 124, wherein the first deviation is characterized by a probability density function used to calculate the target detection frame information of the effective initial position information and the ranging result.
  126. 根据权利要求124所述的装置，其特征在于，所述处理器对一个或多个所述目标检测框信息对应的有效初始位置信息进行优化，还包括：The device according to claim 124, wherein the processor optimizes effective initial position information corresponding to one or more of the target detection frame information, further comprising:
    确定每个所述有效初始位置信息进行非线性优化后对应的优化位置信息是否异常；以及Determine whether the optimized position information corresponding to each effective initial position information after nonlinear optimization is abnormal; and
    将异常的优化位置信息进行滤除。Filter out abnormal optimized location information.
  127. 根据权利要求126所述的装置，其特征在于，所述目标偏差包括所述第一偏差，所述第一偏差包括第一子偏差和/或第二子偏差；The device of claim 126, wherein the target deviation includes the first deviation, and the first deviation includes a first sub-deviation and/or a second sub-deviation;
    所述第一子偏差为关于所述有效初始位置信息与用于计算得到所述有效初始位置信息的目标检测框信息之间的观测偏差；The first sub-deviation is an observation deviation between the effective initial position information and the target detection frame information used to calculate the effective initial position information;
    所述第二子偏差为关于所述有效初始位置信息与用于计算得到所述有效初始位置信息的测距结果之间的观测偏差；The second sub-deviation is an observation deviation between the effective initial position information and the ranging result used to calculate the effective initial position information;
    其中，所述处理器确定每个所述有效初始位置信息进行非线性优化后对应的优化位置信息是否异常，包括：Wherein, the processor determining whether the optimized position information corresponding to each effective initial position information after non-linear optimization is abnormal, includes:
    若确定所述第一子偏差对应的检测框信息异常，和/或确定所述第二子偏差对应的测距结果异常，则将异常的检测框信息和/或异常的测距结果对应的优化位置确定为异常的优化位置。If it is determined that the detection frame information corresponding to the first sub-deviation is abnormal, and/or the ranging result corresponding to the second sub-deviation is determined to be abnormal, then the abnormal detection frame information and/or the abnormal ranging result are optimized accordingly The location is determined to be an abnormally optimized location.
  128. 根据权利要求126所述的装置，其特征在于，所述处理器对一个或多个所述目标检测框信息对应的有效初始位置信息进行优化，还包括：The device according to claim 126, wherein the processor optimizes effective initial position information corresponding to one or more of the target detection frame information, further comprising:
    在将异常的优化位置信息进行滤除之后，对剩余的优化位置信息进行非线性优化，以最小化所述目标偏差，其中，每个所述剩余的优化位置信息进行非线性优化后具有对应的最终优化位置信息。After filtering out the abnormal optimized position information, the remaining optimized position information is nonlinearly optimized to minimize the target deviation, wherein each of the remaining optimized position information has a corresponding non-linear optimization Finally optimize the location information.
  129. 根据权利要求120所述的装置，其特征在于，所述处理器还执行以下操作：The device of claim 120, wherein the processor further performs the following operations:
    在获得的所述图像中未识别到所述目标对象的情况下，确定所述目标对象丢失时的位置信息；以及If the target object is not recognized in the obtained image, determine the position information when the target object is lost; and
    根据所述目标对象丢失时的位置信息和平滑后的关于所述目标对象的运动轨迹，对所述目标对象的状态信息进行预测。The state information of the target object is predicted according to the position information when the target object is lost and the smoothed motion track of the target object.
  130. 根据权利要求129所述的装置，其特征在于，所述处理器根据所述目标对象丢失时的位置信息和平滑后的关于所述目标对象的运动轨迹，对所述目标对象的状态信息进行预测，包括：The device according to claim 129, wherein the processor predicts the state information of the target object based on the position information when the target object is lost and the smoothed motion trajectory about the target object ,include:
    根据所述目标对象丢失时的位置信息和平滑后的关于所述目标对象的运动轨迹，生成关于所述目标对象的预测位置的概率分布。According to the position information when the target object is lost and the smoothed motion trajectory about the target object, a probability distribution about the predicted position of the target object is generated.
  131. 根据权利要求130所述的装置，其特征在于，所述预测位置的概率分布的空间占比随着所述目标对象的丢失时间的增长而增加。The apparatus according to claim 130, wherein the spatial proportion of the probability distribution of the predicted position increases as the loss time of the target object increases.
  132. 根据权利要求131所述的装置，其特征在于，所述预测位置的概率分布的空间占比的变化参数与所述目标对象的类型相关。The device according to claim 131, wherein the change parameter of the spatial proportion of the probability distribution of the predicted position is related to the type of the target object.
  133. 根据权利要求132所述的装置，其特征在于，所述变化参数包括所述预测位置的概率分布的空间占比的增长速度，所述预测位置的概率分布的空间占比的变化参数与所述目标对象的类型相关包括：The device according to claim 132, wherein the change parameter comprises a growth rate of the space proportion of the probability distribution of the predicted position, and the change parameter of the space proportion of the probability distribution of the predicted position is compared with the change parameter of the space proportion of the probability distribution of the predicted position. Related types of target audience include:
    在所述目标对象的类型为生物的情况下，所述预测位置的概率分布的空间占比的第一增长速度在不同方向上相同；In a case where the type of the target object is a living thing, the first growth rate of the space proportion of the probability distribution of the predicted position is the same in different directions;
    在所述目标对象的类型为移动设备的情况下，所述预测位置的概率分布的空间占比的第二增长速度沿所述移动设备的运动方向增加。In a case where the type of the target object is a mobile device, the second increasing speed of the spatial proportion of the probability distribution of the predicted position increases along the movement direction of the mobile device.
  134. 根据权利要求133所述的装置，其特征在于，所述第一增长速度小于所述第二增长速度。The device of claim 133, wherein the first growth rate is less than the second growth rate.
  135. 根据权利要求112所述的装置，其特征在于，所述处理器根据一个或多个所述目标检测框信息和多个所述测距结果确定所述目标对象的状态信息，包括：The device according to claim 112, wherein the processor determines the state information of the target object according to one or more of the target detection frame information and a plurality of the ranging results, comprising:
    根据一个或多个所述目标检测框信息从多个所述测距结果中筛选出有效测距结果；以及Filtering out effective ranging results from the multiple ranging results according to one or more of the target detection frame information; and
    根据一个或多个所述目标检测框信息和所述有效测距结果确定所述目标对象的状态信息。The state information of the target object is determined according to one or more of the target detection frame information and the effective ranging result.
  136. 根据权利要求135所述的装置，其特征在于，所述处理器根据一个或多个所述目标检测框信息从多个所述测距结果中筛选出有效测距结果，包括：The device according to claim 135, wherein the processor screens out effective ranging results from a plurality of ranging results according to one or more of the target detection frame information, comprising:
    确定每个所述目标检测框信息对应的一个或多个测距结果；以及Determine one or more ranging results corresponding to each of the target detection frame information; and
    对每个所述目标检测框信息对应的一个或多个测距结果进行筛选。One or more ranging results corresponding to each target detection frame information are screened.
  137. 根据权利要求136所述的装置，其特征在于，所述处理器确定每个所述目标检测框信息对应的一个或多个测距结果，包括：The device according to claim 136, wherein the processor determining one or more ranging results corresponding to each of the target detection frame information comprises:
    根据每个所述目标检测框信息的采样时间点和多个所述测距结果中每个测距结果的采样时间点确定每个所述目标检测框信息对应的一个或多个测距结果。One or more ranging results corresponding to each target detection frame information are determined according to the sampling time point of each target detection frame information and the sampling time point of each ranging result in the plurality of ranging results.
  138. 根据权利要求136所述的装置，其特征在于，所述测距结果包括激光测距结果，所述处理器对每个所述目标检测框信息对应的一个或多个测距结果进行筛选，包括：The device according to claim 136, wherein the ranging result comprises a laser ranging result, and the processor screens one or more ranging results corresponding to each of the target detection frame information, comprising :
    确定一个或多个所述测距结果中每个测距结果对应的激光光斑；Determining the laser spot corresponding to each of the one or more ranging results;
    根据每个所述测距结果对应的目标检测框信息和每个所述测距结果对应的激光光斑，确定每个所述测距结果的有效性；以及Determine the validity of each distance measurement result according to the target detection frame information corresponding to each distance measurement result and the laser spot corresponding to each distance measurement result; and
    根据每个所述测距结果的有效性对每个所述目标检测框信息对应的一个或多个测距结果进行筛选。One or more ranging results corresponding to each of the target detection frame information are screened according to the validity of each of the ranging results.
  139. 根据权利要求138所述的装置，其特征在于，所述处理器根据每个所述测距结果对应的目标检测框信息和每个所述测距结果对应的激光光斑，确定每个所述测距结果的有效性，包括：The device according to claim 138, wherein the processor determines each of the measurement results according to the target detection frame information corresponding to each of the distance measurement results and the laser spot corresponding to each of the distance measurement results. The validity of the results includes:
    确定每个所述测距结果对应的目标检测框信息和每个所述测距结果对应的激光光斑的面积重合率；以及Determining the target detection frame information corresponding to each of the distance measurement results and the area overlap ratio of the laser spot corresponding to each of the distance measurement results; and
    根据所述面积重合率确定每个所述测距结果的有效性。The validity of each distance measurement result is determined according to the area coincidence rate.
  140. 根据权利要求139所述的装置，其特征在于，所述处理器根据所述面积重合率确定每个所述测距结果的有效性包括：The device according to claim 139, wherein the processor determining the validity of each of the ranging results according to the area coincidence rate comprises:
    将所述面积重合率与预设比例阈值进行比较；Comparing the area coincidence rate with a preset ratio threshold;
    将所述面积重合率大于或等于所述预设比例阈值的测距结果确定为有效测距结果；以及Determining a ranging result whose area coincidence rate is greater than or equal to the preset ratio threshold value as a valid ranging result; and
    将所述面积重合率小于所述预设比例阈值的测距结果确定为无效测距结果。The ranging result whose area coincidence rate is less than the preset ratio threshold is determined as an invalid ranging result.
  141. 根据权利要求138所述的装置，其特征在于，在确定每个所述目标检测框信息对应多个测距结果的情况下，所述处理器根据每个所述测距结果对应的目标检测框信息和每个所述测距结果对应的激光光斑，确定每个所述测距结果的有效性包括：The device according to claim 138, wherein, in the case where it is determined that each of the target detection frame information corresponds to multiple ranging results, the processor is based on the target detection frame corresponding to each of the ranging results. Information and the laser spot corresponding to each of the distance measurement results, and determining the validity of each of the distance measurement results includes:
    根据采样时间相邻的两个所述目标检测框信息，确定采样时间相邻的两个所述目标检测框信息的采样时间之间的多个所述测距结果中每个测距结果分别对应的插值目标检测框信息，得到每个所述测距结果对应的目标检测框信息；以及According to the information of the two target detection frames adjacent to the sampling time, it is determined that each of the multiple ranging results between the sampling times of the two adjacent target detection frame information at the sampling time corresponds to each of the multiple ranging results. Interpolating the target detection frame information of, to obtain target detection frame information corresponding to each of the ranging results; and
    根据每个所述测距结果对应的激光光斑和与每个所述测距结果对应的目标检测框信息，确定每个所述测距结果的有效性。The validity of each ranging result is determined according to the laser spot corresponding to each ranging result and the target detection frame information corresponding to each ranging result.
  142. 根据权利要求135所述的装置，其特征在于，所述处理器根据多个所述目标检测框信息和所述有效测距结果确定所述目标对象的状态信息包括：The apparatus according to claim 135, wherein the processor determining the state information of the target object according to a plurality of the target detection frame information and the effective ranging result comprises:
    确定每个所述目标检测框信息对应的有效测距结果；以及Determine the effective ranging result corresponding to each of the target detection frame information; and
    根据每个所述目标检测框信息和每个所述目标检测框信息对应的有效测距结果，确定所述目标对象的状态信息。Determine the state information of the target object according to each of the target detection frame information and the effective ranging result corresponding to each of the target detection frame information.
  143. 根据权利要求142所述的装置，其特征在于，所述处理器确定每个所述目标检测框信息对应的有效测距结果，包括：The device according to claim 142, wherein the processor determining the effective ranging result corresponding to each of the target detection frame information comprises:
    根据每个所述目标检测框信息的采样时间点和每个所述有效测距结果的采样时间点，将每个所述有效测距结果关联至与所述有效测距结果的采样时间点最接近的目标检测框信息。According to the sampling time point of each target detection frame information and the sampling time point of each effective ranging result, each effective ranging result is associated with the sampling time point of the effective ranging result. Information about the detection frame of the approaching target.
  144. 根据权利要求142所述的装置，其特征在于，在每个所述目标检测框信息对应多个有效测距结果的情况下，所述处理器根据每个所述目标检测框信息和每个所述目标检测框信息对应的有效测距结果，确定所述目标对象的状态信息，包括：The device according to claim 142, wherein, in a case where each target detection frame information corresponds to a plurality of valid ranging results, the processor is based on each target detection frame information and each target detection frame information. The effective ranging result corresponding to the target detection frame information and determining the state information of the target object include:
    计算每个所述目标检测框信息对应的多个有效测距结果的加权平均值，得到每个所述目标检测框信息对应的目标测距结果；以及Calculating a weighted average of multiple effective ranging results corresponding to each of the target detection frame information to obtain a target ranging result corresponding to each of the target detection frame information; and
    根据每个所述目标检测框信息和每个所述目标检测框信息对应的目标测距结果，确定所述目标对象的状态信息。Determine the state information of the target object according to each of the target detection frame information and the target ranging result corresponding to each of the target detection frame information.
  145. 根据权利要求112所述的装置，其特征在于，所述处理器根据一个或多个所述目标检测框信息和多个所述测距结果确定所述目标对象的状态信息，包括：The device according to claim 112, wherein the processor determines the state information of the target object according to one or more of the target detection frame information and a plurality of the ranging results, comprising:
    确定每个所述目标检测框信息对应的测距结果；Determining a ranging result corresponding to each of the target detection frame information;
    确定每个所述目标检测框信息对应的关于所述目标对象的物理估计尺寸；Determine the physical estimated size of the target object corresponding to each target detection frame information;
    根据每个所述目标检测框信息对应的关于所述目标对象的物理估计尺寸，对每个所述目标检测框信息对应的测距结果进行筛选；以及Screening the distance measurement results corresponding to each target detection frame information according to the physical estimated size of the target object corresponding to each target detection frame information; and
    根据一个或多个所述目标检测框信息以及筛选后的测距结果，确定所述目标对象的状态信息。The state information of the target object is determined according to one or more of the target detection frame information and the filtered ranging result.
  146. 根据权利要求145所述的装置，其特征在于，所述处理器根据每个所述目标检测框信息对应的关于所述目标对象的物理估计尺寸，对每个所述目标检测框信息对应的测距结果进行筛选，包括：The device according to claim 145, wherein the processor determines the measurement corresponding to each target detection frame information according to the physical estimated size of the target object corresponding to each target detection frame information. Filter the results, including:
    将每个所述目标检测框信息对应的关于所述目标对象的物理估计尺寸与预设合理范围进行比较；以及Comparing the physical estimated size of the target object corresponding to each target detection frame information with a preset reasonable range; and
    在所述目标检测框信息对应的关于所述目标对象的物理估计尺寸与所述预设合理范围不相符的情况下，滤除所述目标检测框信息对应的测距结果。In the case that the physical estimated size of the target object corresponding to the target detection frame information does not match the preset reasonable range, the ranging result corresponding to the target detection frame information is filtered out.
  147. 根据权利要求146所述的装置，其特征在于，所述处理器确定每个所述目标检测框信息对应的关于所述目标对象的物理估计尺寸，包括：The apparatus according to claim 146, wherein the processor determining the physical estimated size of the target object corresponding to each target detection frame information comprises:
    根据每个所述目标检测框信息和每个所述目标检测框信息采集时的所述成像装置的视场角，确定每个所述目标检测框信息对应的视场角；以及Determine the field angle corresponding to each target detection frame information according to each of the target detection frame information and the field of view angle of the imaging device when each of the target detection frame information is collected; and
    根据每个所述目标检测框信息对应的视场角和每个所述目标检测框信息对应的测距结果，确定每个所述目标检测框信息对应的物理估计尺寸。Determine the physical estimated size corresponding to each target detection frame information according to the field angle corresponding to each target detection frame information and the ranging result corresponding to each target detection frame information.
  148. 根据权利要求146所述的装置，其特征在于，所述处理器还执行以下操作：The device of claim 146, wherein the processor further performs the following operations:
    确定所述目标对象的对象类型，其中，每个对象类型具有对应的预设合理范围；Determine the object type of the target object, wherein each object type has a corresponding preset reasonable range;
    其中，将每个所述目标检测框信息对应的关于所述目标对象的物理估计尺寸与预设合理范围进行比较包括：Wherein, comparing the physical estimated size of the target object corresponding to each target detection frame information with a preset reasonable range includes:
    根据所述目标对象的对象类型确定目标预设合理范围；以及Determine the preset reasonable range of the target according to the object type of the target object; and
    将每个所述目标检测框信息对应的关于所述目标对象的物理估计尺寸与所述目标预设合理范围进行比较。The physical estimated size of the target object corresponding to each target detection frame information is compared with the preset reasonable range of the target.
  149. 一种目标对象的状态信息确定***，其特征在于，包括：A system for determining the status information of a target object, which is characterized in that it comprises:
    成像装置，用于获得关于所述目标对象的多帧图像；An imaging device for obtaining multiple frames of images about the target object;
    如权利要求75～148中任一项所述的状态信息确定装置。The status information determining device according to any one of claims 75 to 148.
  150. 一种可移动平台，其特征在于，包括：A movable platform, characterized in that it comprises:
    可移动本体；以及Removable body; and
    如权利要求149所述的***。The system of claim 149.
  151. 一种可读存储介质，其上存储有可执行指令，该指令被处理器执行时使处理器执行权利要求1至74中任一项所述的方法。A readable storage medium having executable instructions stored thereon, and when the instructions are executed by a processor, the processor executes the method according to any one of claims 1 to 74.
PCT/CN2020/089002 2020-05-07 2020-05-07 State information determination method, apparatus and system, and movable platform and storage medium WO2021223166A1 (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
CN202080004726.6A CN112639405A (en) 2020-05-07 2020-05-07 State information determination method, device, system, movable platform and storage medium
PCT/CN2020/089002 WO2021223166A1 (en) 2020-05-07 2020-05-07 State information determination method, apparatus and system, and movable platform and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2020/089002 WO2021223166A1 (en) 2020-05-07 2020-05-07 State information determination method, apparatus and system, and movable platform and storage medium

Publications (1)

Publication Number Publication Date
WO2021223166A1 true WO2021223166A1 (en) 2021-11-11

Family

ID=75291516

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/089002 WO2021223166A1 (en) 2020-05-07 2020-05-07 State information determination method, apparatus and system, and movable platform and storage medium

Country Status (2)

Country Link
CN (1) CN112639405A (en)
WO (1) WO2021223166A1 (en)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115963851A (en) * 2021-10-13 2023-04-14 北京三快在线科技有限公司 Unmanned aerial vehicle positioning method and device

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103389799A (en) * 2013-07-24 2013-11-13 清华大学深圳研究生院 Method for tracking motion trail of fingertip
CN106408591A (en) * 2016-09-09 2017-02-15 南京航空航天大学 Anti-blocking target tracking method
CN107066990A (en) * 2017-05-04 2017-08-18 厦门美图之家科技有限公司 A kind of method for tracking target and mobile device
US20180061076A1 (en) * 2016-08-30 2018-03-01 Canon Kabushiki Kaisha Fast multi-object detection and tracking system
CN108986138A (en) * 2018-05-24 2018-12-11 北京飞搜科技有限公司 Method for tracking target and equipment
CN110222206A (en) * 2019-05-20 2019-09-10 深圳供电局有限公司 Client, server and image recognition system

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110785993A (en) * 2018-11-30 2020-02-11 深圳市大疆创新科技有限公司 Control method and device of shooting equipment, equipment and storage medium

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103389799A (en) * 2013-07-24 2013-11-13 清华大学深圳研究生院 Method for tracking motion trail of fingertip
US20180061076A1 (en) * 2016-08-30 2018-03-01 Canon Kabushiki Kaisha Fast multi-object detection and tracking system
CN106408591A (en) * 2016-09-09 2017-02-15 南京航空航天大学 Anti-blocking target tracking method
CN107066990A (en) * 2017-05-04 2017-08-18 厦门美图之家科技有限公司 A kind of method for tracking target and mobile device
CN108986138A (en) * 2018-05-24 2018-12-11 北京飞搜科技有限公司 Method for tracking target and equipment
CN110222206A (en) * 2019-05-20 2019-09-10 深圳供电局有限公司 Client, server and image recognition system

Also Published As

Publication number Publication date
CN112639405A (en) 2021-04-09

Similar Documents

Publication Publication Date Title
US11915502B2 (en) Systems and methods for depth map sampling
KR102113955B1 (en) Device and method for monitoring ship and port
US20200208970A1 (en) Method and device for movable object distance detection, and aerial vehicle
US10860039B2 (en) Obstacle avoidance method and apparatus and unmanned aerial vehicle
US11887318B2 (en) Object tracking
KR102530691B1 (en) Device and method for monitoring a berthing
KR102151815B1 (en) Method and Apparatus for Vehicle Detection Using Lidar Sensor and Camera Convergence
JP2012185540A (en) Image processing device, image processing method, and image processing program
KR102265980B1 (en) Device and method for monitoring ship and port
CN112947419B (en) Obstacle avoidance method, device and equipment
CN112445204B (en) Object movement navigation method and device in construction site and computer equipment
JP7306766B2 (en) Target motion information detection method, apparatus, equipment and medium
KR102520844B1 (en) Method and device for monitoring harbor and ship considering sea level
KR102295183B1 (en) object tracking method for CCTV video by use of CCTV projection model
KR102645492B1 (en) Device and method for monitoring ship and port
CN114359714A (en) Unmanned body obstacle avoidance method and device based on event camera and intelligent unmanned body
WO2021223166A1 (en) State information determination method, apparatus and system, and movable platform and storage medium
KR20100066952A (en) Apparatus for tracking obstacle using stereo vision and method thereof
JP2013069045A (en) Image recognition device, image recognition method, and image recognition program
WO2023078445A1 (en) Target tracking method and apparatus for unmanned aerial vehicle, electronic device, and storage medium
WO2022088072A1 (en) Visual tracking method and apparatus, movable platform, and computer-readable storage medium
JP2018004435A (en) Movement amount calculation device and movement amount calculation method
JP2023115703A (en) Video monitoring device, video monitoring method and program
CN113661513A (en) Image processing method, image processing device, image processing system and storage medium
JP2018055521A (en) Detector, detection method and detection system

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20934411

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20934411

Country of ref document: EP

Kind code of ref document: A1