WO2023065165A1 - Procédé et appareil de commande de plateforme mobile, et plateforme mobile et support de stokage - Google Patents

Procédé et appareil de commande de plateforme mobile, et plateforme mobile et support de stokage Download PDF

Info

Publication number
WO2023065165A1
WO2023065165A1 PCT/CN2021/125060 CN2021125060W WO2023065165A1 WO 2023065165 A1 WO2023065165 A1 WO 2023065165A1 CN 2021125060 W CN2021125060 W CN 2021125060W WO 2023065165 A1 WO2023065165 A1 WO 2023065165A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
characteristic part
target object
characteristic
determining
Prior art date
Application number
PCT/CN2021/125060
Other languages
English (en)
Chinese (zh)
Inventor
程正喜
胡晓翔
关硕森
Original Assignee
深圳市大疆创新科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 深圳市大疆创新科技有限公司 filed Critical 深圳市大疆创新科技有限公司
Priority to PCT/CN2021/125060 priority Critical patent/WO2023065165A1/fr
Publication of WO2023065165A1 publication Critical patent/WO2023065165A1/fr

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion

Definitions

  • the first characteristic part of the target object in the second image cannot be determined according to the first characteristic part of the target object in the first image, and according to the target in the first image
  • the second characteristic part of the object determines the second characteristic part of the target object in the second image
  • controlling the photographing device to focus on the first characteristic part of the target object and switch to the second characteristic part of the target object Focusing on the second characteristic part of the target object for shooting.
  • an embodiment of the present invention provides a movable platform, and the movable platform includes a photographing device and a control device for the movable platform provided in the second aspect of the embodiment of the present invention.
  • an embodiment of the present invention provides a computer-readable storage medium, the storage medium is a computer-readable storage medium, and program instructions are stored in the computer-readable storage medium, and the program instructions are used to implement the implementation of the present invention.
  • the first characteristic part of the target object and the second characteristic part of the target object are determined in the first image obtained by focusing on the first characteristic part of the target object, if according to the determined target object In the second image obtained by focusing on the first characteristic part of the target object and the second characteristic part of the target object, the first characteristic part of the target object cannot be detected and the second characteristic part of the target object is detected part, the photographing device may be controlled to switch to the second characteristic part of the target object as the focus target for focus shooting.
  • the first characteristic part of the target object when the first characteristic part of the target object is blocked, it can switch to focus on the second characteristic part of the target object for shooting, avoiding the failure of the shooting device to focus on the target object, and improving the robustness of focusing on the target object and image capture quality.
  • FIG. 1 is a schematic flowchart of a method for controlling a mobile platform provided by an embodiment of the present invention
  • FIG. 2 is a schematic flow chart of determining a second characteristic part of a target object provided by an embodiment of the present invention
  • FIG. 3 is a schematic diagram of a key point detection provided by an embodiment of the present invention.
  • FIG. 4 is a schematic structural diagram of a control device for a movable platform provided by an embodiment of the present invention.
  • Fig. 5 is a schematic structural diagram of a mobile platform provided by an embodiment of the present invention.
  • the words “if”, “if” as used herein may be interpreted as “at” or “when” or “in response to determining” or “in response to detecting”.
  • the phrases “if determined” or “if detected (the stated condition or event)” could be interpreted as “when determined” or “in response to the determination” or “when detected (the stated condition or event) )” or “in response to detection of (a stated condition or event)”.
  • Fig. 1 is a flowchart of a method for controlling a mobile platform provided by an embodiment of the present invention.
  • the method can be applied to a movable platform, and the movable platform can be any device including a photographing device, and the movable platform can be any device that can move through external force or its own configured power system.
  • the movable platform may be a handheld device, and in some cases, the movable platform may be a drone, an intelligent robot, or the like.
  • the photographing device may be various types of optical imaging devices, and the photographing device may be used to focus and photograph a certain characteristic part of the target object. As shown in Figure 1, the method includes the following steps:
  • the first image and the second image are obtained by the photographing device focusing on the first characteristic part of the target object.
  • a depth sensor such as a TOF sensor
  • the first image and the second image are obtained by the shooting device according to the depth data of the target object.
  • the depth data of the first characteristic part of the target object is obtained by focusing and shooting the first characteristic part.
  • the first image may be acquired by focusing on the depth data of the first characteristic part of the target object obtained from the depth data according to the first characteristic part of the target object in the image captured before the first image.
  • the images captured before the first image may be images of the previous frame or several previous frames of the first image.
  • the second image may be acquired by focusing on the depth data of the first characteristic part of the target object acquired from the depth data according to the first characteristic part of the target object in the image captured before the second image.
  • the image obtained by shooting before the second image may be an image of a previous frame or several previous frames of the second image.
  • the second image may be acquired by focusing on the depth data of the first characteristic part of the target object obtained from the depth data according to the first characteristic part of the target object in the first image.
  • the above-mentioned target object can be a person.
  • the face or head of the person is prioritized. If the face or head is blocked and there is no face or head in the shooting field of view, you can retreat Select the main body of the character for focus processing.
  • the first characteristic part of the target object may be, for example, the face or head of the character
  • the second characteristic part of the target object may be, for example, the main body of the character.
  • the second feature of the target object may include the first feature of the target object.
  • the body of the Lord is the body part of the upper body of the human body except the arms. It should be noted that, in the embodiment of the present invention, selecting the above main body as the focus target can narrow the focus range and make the focus process more accurate than using the entire body as the focus target.
  • the process of focusing and shooting the second image it can be determined whether there is the first characteristic part of the target object in the second image according to the first characteristic part of the target object in the first image, that is, it can be determined in the image
  • the first feature part of the target object is tracked to determine whether there is a first feature part belonging to the target object in the second image.
  • the photographing device may be controlled to switch the focus target, that is, switch the focus target from the first characteristic part of the target object to the second characteristic part of the target object. Finally, in the process of focusing and shooting the second image, it may be adjusted to focus and shoot the second characteristic part of the target object.
  • the target depth data corresponding to the focus target needs to be used in the focus shooting process, and the target depth data is obtained based on the position of the focus target in the image, so it is necessary to determine the focus target in order to further obtain the focus target in the image In the position, finally to complete the focus shooting process.
  • the first characteristic part of the target object is preferentially selected as the focus target, and when the focus target disappears, it may switch to the second characteristic part of the target object for focus shooting.
  • the determination method includes:
  • a second characteristic part whose corresponding matching parameter satisfies a preset condition may be determined as the second characteristic part of the target object.
  • the first frame selection area where the first characteristic parts of the target object in the first image are located may be identified, and the second frame selection area where all the second feature parts of different objects in the first image are located may be identified.
  • key points are extracted in the first frame selection area to obtain key points corresponding to the first feature part
  • key points are extracted in the second frame selection area to obtain key points corresponding to the second feature part.
  • the key point can be the key point of the human body.
  • the key point can include the left eye, the right eye, the left ear, the right ear, the nose, the corner of the left mouth, the corner of the right mouth, etc.
  • the main body of the character can also include these key points.
  • FIG. 3 shows a character, and two framed areas are shown on the body of the character, one of which is the face, and the other framed area is the upper main body.
  • Several extracted key points are marked in the framed area corresponding to the face, and several extracted key points are also marked in the main body.
  • the second There is a corresponding relationship between a key point corresponding to a feature part and a key point corresponding to a second feature part, and the corresponding relationship is a key point of the same type, which may be called a key point with the same name.
  • the left eye corresponding to the first feature part and the left eye corresponding to the second feature part are key points with the same name
  • the right eye corresponding to the first feature part and the right eye corresponding to the second feature part are key points with the same name.
  • the key point with the same name belonging to key point X among the key points corresponding to the second feature part can be determined, and then calculate the distance between the key point with the same name and the key point X (ie pixel distance).
  • the distance corresponding to each key point of the first feature part can be obtained through the above method, and finally, the distance between the first feature part of the target object in the first image and the key point in the first image can be determined according to multiple distances. Matching parameters between the second feature parts.
  • the above-mentioned process of determining the matching parameters between the first characteristic part of the target object in the first image and each second characteristic part in the first image according to the distance between key points with the same name may be implemented as: According to the distance between the key points with the same name, determine the feature distance between the key points with the same name; determine the feature distance as between the first feature part of the target object in the first image and each second feature part in the first image matching parameters.
  • the process of determining the second characteristic part of the target object from the second characteristic parts in the first image according to the matching parameters can be implemented as: determining the second characteristic part in the first image with the smallest matching parameter as the first The second feature part of the target object in the image.
  • the feature distance can be determined according to the multiple distances.
  • the characteristic distance may be an average value, a median value, a maximum value, or a minimum value of multiple distances.
  • the characteristic distance is a matching parameter between the first characteristic part of the target object in the first image and the second characteristic part in the first image. After the matching parameters are obtained, among all the second characteristic parts contained in the first image, the second characteristic part with the smallest matching parameter may be determined as the second characteristic part of the target object in the first image.
  • the above solution is applicable to the case that the first image contains multiple objects, so that multiple second characteristic parts can be identified, and these second characteristic parts belong to different objects respectively.
  • the second characteristic part of the target object can be determined from all the second characteristic parts included in the first image according to the matching parameter.
  • the process of determining the matching parameters between the first characteristic part of the target object in the first image and each second characteristic part in the first image can also be implemented as: According to the distance between the key points with the same name, determine the feature distance between the key points with the same name; according to the feature distance and the size parameter of the first feature part of the target object in the first image, determine the first position of the target object in the first image A matching parameter between the feature part and each second feature part in the first image.
  • the size parameter of the first characteristic part of the target object in the above-mentioned first image can be, for example, the diagonal length, length or width of the first frame selection area corresponding to the first characteristic part, or it can also be the first frame Some other size parameters of the selected area are not limited in this embodiment of the present invention.
  • the method provided by the embodiment of the present invention may further include: acquiring a third image captured by the shooting device, wherein the third image is obtained by the shooting device focusing on the second characteristic part of the target object, and the third image is Acquired after the second image; determine the first characteristic part from the third image; determine the second characteristic part of the target object in the third image according to the second characteristic part of the target object in the second image; if it can be based on the second characteristic part of the target object in the second image
  • the second characteristic part of the target object in the three images determines the first characteristic part of the target object from the first characteristic part in the third image, and controls the shooting device to switch from focusing on the second characteristic part of the target object to focusing on the second characteristic part of the target object. Focusing is performed on the first characteristic part of the target object.
  • the above-mentioned third image may be a frame of image taken immediately after the second image is taken, or a frame of image taken several frames after the second image is taken.
  • the first characteristic part whose matching parameter is smaller than the preset threshold may be selected as the first characteristic part of the target object in the third image.
  • the first characteristic part with the smallest matching parameter may also be selected as the first characteristic part of the target object in the third image.
  • the focusing target used when taking the first image is the first characteristic part of the target object, and the first characteristic part may also be the upper main body.
  • the second characteristic The site can also be the face or the head. Therefore, optionally, the first feature location may also include the second feature location.
  • the first characteristic part of the target object when the first characteristic part of the target object is blocked, it can switch to focus on the second characteristic part of the target object for shooting, avoiding the failure of the shooting device to focus on the target object, and improving the robustness of focusing on the target object and image capture quality.
  • Yet another exemplary embodiment of the present invention provides a control device for a movable platform, the movable platform includes a shooting device, as shown in FIG. 4 , the device includes:
  • the processor 42 is used to run the computer program stored in the memory 41 to realize:
  • the first characteristic part of the target object in the second image cannot be determined according to the first characteristic part of the target object in the first image, and according to the target in the first image
  • the second characteristic part of the object determines the second characteristic part of the target object in the second image
  • controlling the photographing device to focus on the first characteristic part of the target object and switch to the second characteristic part of the target object Focusing on the second characteristic part of the target object for shooting.
  • the processor 42 is configured to:
  • the processor 42 is configured to:
  • the characteristic distance and the size parameter of the first characteristic part of the target object in the first image determine the difference between the first characteristic part of the target object in the first image and the first characteristic part in the first image The matching parameters between each of the second feature parts.
  • the processor 42 is further configured to:
  • the shooting device is controlled to Focusing on the second characteristic part of the target object is switched to focusing on the first characteristic part of the target object.
  • the second characteristic location includes the first characteristic location.
  • the second characteristic part is the upper main body.
  • the first characteristic part is the face or the head.
  • the first feature includes a second feature.
  • the first characteristic part is the upper main body.
  • the second characteristic part is the face or the head.
  • FIG. 5 is a schematic structural diagram of a movable platform provided by an embodiment of the present invention.
  • the movable platform may include a photographing device 51 and a control device 52 of the movable platform provided in the above-mentioned embodiments.
  • an embodiment of the present invention also provides a computer-readable storage medium, wherein executable code is stored in the computer-readable storage medium, and the executable code is used to realize the control of the mobile platform as provided in the foregoing embodiments. method.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Studio Devices (AREA)

Abstract

Sont prévus dans les modes de réalisation de la présente invention un procédé et un appareil de commande de plateforme mobile, et une plateforme mobile et un support de stockage. Le procédé consiste à : acquérir une première image et une seconde image, la seconde image étant acquise après la première image ; déterminer, à partir de la première image, une seconde partie de caractéristique, et une première partie de caractéristique d'un objet cible ; déterminer, à partir de la première image, une seconde partie de caractéristique de l'objet cible en fonction de la première partie de caractéristique de l'objet cible dans la première image ; et si la première partie de caractéristique de l'objet cible ne peut pas être déterminée à partir de la seconde image en fonction de la première partie de caractéristique de l'objet cible dans la première image, et lorsque la seconde partie de caractéristique de l'objet cible a été déterminée à partir de la seconde image en fonction de la seconde partie de caractéristique de l'objet cible dans la première image, commander un appareil de photographie pour passer de la réalisation d'une photographie de mise au point sur la première partie de caractéristique de l'objet cible à la réalisation d'une photographie de mise au point sur la seconde partie de caractéristique de l'objet cible. L'utilisation de la présente invention permet d'éviter une défaillance de traitement de mise au point.
PCT/CN2021/125060 2021-10-20 2021-10-20 Procédé et appareil de commande de plateforme mobile, et plateforme mobile et support de stokage WO2023065165A1 (fr)

Priority Applications (1)

Application Number Priority Date Filing Date Title
PCT/CN2021/125060 WO2023065165A1 (fr) 2021-10-20 2021-10-20 Procédé et appareil de commande de plateforme mobile, et plateforme mobile et support de stokage

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2021/125060 WO2023065165A1 (fr) 2021-10-20 2021-10-20 Procédé et appareil de commande de plateforme mobile, et plateforme mobile et support de stokage

Publications (1)

Publication Number Publication Date
WO2023065165A1 true WO2023065165A1 (fr) 2023-04-27

Family

ID=86057758

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/125060 WO2023065165A1 (fr) 2021-10-20 2021-10-20 Procédé et appareil de commande de plateforme mobile, et plateforme mobile et support de stokage

Country Status (1)

Country Link
WO (1) WO2023065165A1 (fr)

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20140247374A1 (en) * 2012-02-06 2014-09-04 Sony Corporation Image processing apparatus, image processing method, program, and recording medium
CN105659580A (zh) * 2014-09-30 2016-06-08 华为技术有限公司 一种自动对焦方法、装置及电子设备
CN109155067A (zh) * 2018-01-23 2019-01-04 深圳市大疆创新科技有限公司 可移动平台的控制方法、设备、计算机可读存储介质
CN110428449A (zh) * 2019-07-31 2019-11-08 腾讯科技（深圳）有限公司 目标检测跟踪方法、装置、设备及存储介质

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20140247374A1 (en) * 2012-02-06 2014-09-04 Sony Corporation Image processing apparatus, image processing method, program, and recording medium
CN105659580A (zh) * 2014-09-30 2016-06-08 华为技术有限公司 一种自动对焦方法、装置及电子设备
CN109155067A (zh) * 2018-01-23 2019-01-04 深圳市大疆创新科技有限公司 可移动平台的控制方法、设备、计算机可读存储介质
CN110428449A (zh) * 2019-07-31 2019-11-08 腾讯科技（深圳）有限公司 目标检测跟踪方法、装置、设备及存储介质

Similar Documents

Publication Publication Date Title
JP5099488B2 (ja) 撮像装置、顔認識方法およびそのプログラム
JP6961797B2 (ja) プレビュー写真をぼかすための方法および装置ならびにストレージ媒体
KR101679290B1 (ko) 영상 처리 방법 및 장치
US8879802B2 (en) Image processing apparatus and image processing method
JP4848965B2 (ja) 撮像装置
JP2004317699A (ja) デジタルカメラ
CN111492426A (zh) 注视启动的语音控制
JP2004320287A (ja) デジタルカメラ
WO2016187985A1 (fr) Dispositif photographique, procédé et système de photographie avec suivi, et support de stockage informatique
KR101537948B1 (ko) 얼굴 포즈 추정을 이용한 촬영 방법 및 장치
US20130070142A1 (en) Imaging Device and Imaging Method for Imaging Device
WO2019179468A1 (fr) Procédé de commande pour dispositif intelligent, appareil, dispositif et support d'informations
JP2011090413A (ja) 画像認識装置、その処理方法及びプログラム
JP2010056692A (ja) 撮像装置およびその制御方法
KR20100055946A (ko) 동영상 썸네일 생성 방법 및 장치
US10013632B2 (en) Object tracking apparatus, control method therefor and storage medium
JP2011166305A (ja) 画像処理装置および撮像装置
JP5771461B2 (ja) 追尾装置、追尾方法及び追尾プログラム
JP2004320285A (ja) デジタルカメラ
JP2007067559A (ja) 画像処理方法、画像処理装置、及び撮像装置の制御方法
JP2021128537A (ja) 画像処理装置および画像処理方法、プログラム並びに記憶媒体
JP2006259847A (ja) 自動追尾装置及び自動追尾方法
WO2023065165A1 (fr) Procédé et appareil de commande de plateforme mobile, et plateforme mobile et support de stokage
JP2011022927A (ja) 手画像認識装置
JP5499531B2 (ja) 電子カメラ

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21960921

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE