CN112866558A - Operation method of electronic equipment, control method of holder and holder system - Google Patents
Operation method of electronic equipment, control method of holder and holder system Download PDFInfo
- Publication number
- CN112866558A CN112866558A CN202011218182.1A CN202011218182A CN112866558A CN 112866558 A CN112866558 A CN 112866558A CN 202011218182 A CN202011218182 A CN 202011218182A CN 112866558 A CN112866558 A CN 112866558A
- Authority
- CN
- China
- Prior art keywords
- camera
- target object
- video stream
- electronic equipment
- head
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/60—Control of cameras or camera modules
- H04N23/61—Control of cameras or camera modules based on recognised objects
- H04N23/611—Control of cameras or camera modules based on recognised objects where the recognised objects include parts of the human body
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/60—Control of cameras or camera modules
- H04N23/69—Control of means for changing angle of the field of view, e.g. optical zoom objectives or electronic zooming
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/60—Control of cameras or camera modules
- H04N23/695—Control of camera direction for changing a field of view, e.g. pan, tilt or based on tracking of objects
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- User Interface Of Digital Computer (AREA)
- Studio Devices (AREA)
Abstract
The invention provides an operation method for electronic equipment, wherein the electronic equipment is provided with a camera and a display screen, and the operation method comprises the following steps: s101: acquiring a video stream of a screen recording image of the display screen; s102: acquiring first parameter information of a target object in the video stream; s103: and outputting the updated pose parameters of the electronic equipment according to the first parameter information of the target object. The embodiment provided by the invention enables the user to synchronously perform operations such as face recognition, face beautifying, human body temperature measurement, holder following and the like in the process of making WeChat calls and carrying out video conferences. The preferred embodiment of the invention can complete the identification and tracking of the target object without depending on other application programs, and practice proves that the invention has beneficial effects.
Description
Technical Field
The present invention relates generally to the field of pattern recognition technology, and more particularly, to an operation method for an electronic device based on pattern recognition.
Background
Under the situation that a plurality of application programs in an electronic device system (embedded system) run simultaneously, the plurality of application programs running simultaneously often cannot access a camera of the electronic device or simultaneously acquire a video stream shot by the camera, so that some of the application programs cannot perform subsequent operations. For example, only one app can be allowed to access the camera at a certain time in the mobile phone system, if a user is making a WeChat call at a certain time, another app also needs to acquire a camera image for a specific operation, such as face recognition, tracking or face beautifying operation, but at this time, the camera is occupied and cannot acquire a video stream shot by the camera, so that the later-enabled app cannot work normally unless the WeChat call is turned off.
The statements in this background section merely represent techniques known to the public and are not, of course, representative of the prior art.
Disclosure of Invention
In view of at least one of the drawbacks of the prior art, the present invention provides an operating method for an electronic device having a camera and a display screen, the operating method comprising:
s101: acquiring a video stream of a screen recording image of the display screen;
s102: acquiring first parameter information of a target object in the video stream;
s103: and outputting the updated pose parameters of the electronic equipment according to the first parameter information of the target object.
According to an aspect of the present invention, wherein in the step S101, the camera is called by a first process; the method of operation is performed by a second process, wherein the first process is different from the second process.
According to an aspect of the invention, wherein the first parameter information comprises a position parameter and/or a size parameter, an acceleration parameter.
According to an aspect of the invention, wherein step S103 further comprises:
according to the first parameter information of the target object, calculating an updated pose parameter of the electronic equipment, so that the position and/or the size of the target object in the image collected by the camera meet preset requirements.
According to an aspect of the present invention, wherein the electronic device is mounted on a pan/tilt head, the step S103 further includes: outputting the updated pose parameters of the electronic device to the pan-tilt head, the method of operation further comprising:
s104: and adjusting the pose of the electronic equipment according to the updated pose parameter of the electronic equipment through the cradle head.
According to one aspect of the invention, the electronic device comprises one or more of a mobile phone, a PAD, a motion camera, AR/VR glasses, and a home smart camera, and the target object comprises one or more of a human face, an iris, a heating element, and a dynamic target object.
According to an aspect of the invention, the method of operation further comprises:
judging whether a camera of the electronic equipment is called;
when the camera has been called, the step S101 is executed.
According to an aspect of the invention, the method of operation further comprises:
judging whether the video stream comprises an image collected by a camera or not;
when the video stream includes images captured by a camera, the step S102 is performed.
According to an aspect of the present invention, the step of determining whether the video stream is an image captured by a camera includes:
and acquiring the current display mode of the display screen from the bottom layer data of the electronic equipment so as to judge whether the video stream comprises the image acquired by the camera.
According to an aspect of the present invention, the step of determining whether the video stream is an image captured by a camera includes:
and performing living body detection on the image frames of the video stream to judge whether the video stream comprises images collected by a camera.
According to an aspect of the present invention, the step of determining whether the video stream is an image captured by a camera includes:
and judging the image frames of the video stream through a training scene model so as to judge whether the video stream comprises images acquired by a camera.
According to an aspect of the present invention, the step of determining whether the video stream is an image captured by a camera includes:
and judging whether the video stream comprises images collected by a camera or not by comparing the continuous frames of the video stream.
According to an aspect of the invention, the method of operation further comprises:
and when recognizing that an object in the video stream makes a preset gesture, taking the object as the target object.
According to an aspect of the invention, the method of operation further comprises:
and when the size of an object in the video stream is identified to be larger than a first threshold value or the size proportion of the object in the video stream exceeds a second threshold value, taking the object as the target object.
According to an aspect of the invention, the method of operation further comprises:
and when an object in the video stream is identified to be located in a preset area on a display screen, taking the object as the target object.
According to an aspect of the invention, the method of operation further comprises:
and when an object in the video stream is identified to be consistent with a preset object, taking the object as the target object.
According to an aspect of the invention, the method of operation further comprises:
and taking the object selected by the user of the electronic equipment in the video stream as the target object.
According to an aspect of the invention, the method of operation further comprises:
and selecting the target object according to the operation of a user on a hardware button arranged on the electronic equipment or the cradle head and/or a remote controller communicated with the electronic equipment or the cradle head.
According to an aspect of the invention, the method of operation further comprises:
and identifying the switching of the display screen of the electronic equipment, and when the display screen is switched, acquiring the first parameter information of the target object again or using the first parameter information of the target object acquired by switching the previous frame.
According to an aspect of the invention, the method of operation further comprises:
identifying a first image area and an additional second image area captured by the camera in the video stream; in step S102, the first parameter information of the target object is the first parameter information of the target object in the first image region.
The invention also provides a control method of the cradle head, wherein the cradle head is communicated with the electronic equipment arranged on the cradle head, and the control method comprises the following steps:
acquiring first parameter information of the target object in real time by using the operation method; and
and adjusting the pose of the electronic equipment in real time through the cradle head according to the first parameter information of the target object.
According to an aspect of the invention, the method of operation further comprises:
turning on and/or off control of the head according to one or more of a user's gesture, sound, motion, gesture, expression.
According to an aspect of the invention, the method of operation further comprises:
and according to the operation of a user on a hardware button arranged on the electronic equipment or the cradle head and/or a remote controller communicated with the electronic equipment or the cradle head, starting and/or closing the control of the cradle head.
According to an aspect of the invention, wherein the pan-tilt apparatus comprises a touch screen, the method further comprises:
the user may select the target object via a touch screen and/or turn on and/or off control of the head.
The invention also provides an operation method for the electronic equipment, wherein the electronic equipment is provided with a camera and a display screen, and the operation method comprises the following steps:
s201: when the camera is called by a first process, acquiring a video stream of a screen recording image of the display screen through a second process, wherein the first process is different from the second process;
s202: acquiring a position parameter and/or a size parameter of a target object in the video stream; and
s203: and adjusting the focal length of the camera according to the position parameter and/or the size parameter of the target object, so that the position and/or the size of the target object in the image acquired by the camera meet the preset requirement.
The invention also provides a computer-readable storage medium comprising computer-executable instructions stored thereon which, when executed by a processor, implement a method of operation of an electronic device as described above.
The preferred embodiment of the present invention provides an operation method of an electronic device (embedded system), where in a case where another application has invoked a camera, an application executing the operation method obtains first parameter information of a target object by obtaining a video stream of a screen recording image, and outputs a pose parameter updated by the electronic device according to the first parameter information. On the basis of the operation method of the electronic device, the preferred embodiment of the invention further provides a control method of the pan-tilt, a control method of the camera and a pan-tilt system. The embodiment provided by the invention enables the user to synchronously perform operations such as face recognition, face beautifying, human body temperature measurement, holder following and the like in the way of making WeChat calls and carrying out video conferences. The preferred embodiment of the invention can complete the identification and tracking of the target object without depending on other application programs, and practice proves that the invention has beneficial effects.
Drawings
The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this specification, illustrate embodiments of the invention and together with the description serve to explain the principles of the invention and not to limit the invention. In the drawings:
FIG. 1 schematically illustrates an electronic device having a camera and a display screen;
FIG. 2 illustrates a method of operation of an electronic device in accordance with a preferred embodiment of the present invention;
fig. 3 schematically illustrates an apparatus and a head for performing the method of operation of the electronic device of fig. 2, in accordance with a preferred embodiment of the present invention;
FIG. 4A schematically illustrates a method of selecting a target object by a user gesture in accordance with a preferred embodiment of the present invention;
FIG. 4B schematically illustrates a method of selecting a target object by a user gesture in accordance with a preferred embodiment of the present invention;
FIG. 5A schematically illustrates a method of selecting a target object by object size in accordance with a preferred embodiment of the present invention;
FIG. 5B schematically illustrates a method of selecting a target object by the proportion of the object size in the image, in accordance with a preferred embodiment of the present invention;
FIG. 6A schematically illustrates a method of selecting a target object through a preset area according to a preferred embodiment of the present invention;
FIG. 6B schematically illustrates a method of selecting a target object through a preset area according to a preferred embodiment of the present invention;
FIG. 7 schematically illustrates a method of selecting a target object via a hardware button in accordance with a preferred embodiment of the present invention;
FIG. 8 schematically illustrates a method of selecting a target object at the time of an on-screen transition in accordance with a preferred embodiment of the present invention;
FIG. 9 schematically illustrates a method of selecting a target object when information other than an image captured by a camera is contained within a display screen in accordance with a preferred embodiment of the present invention;
fig. 10 schematically illustrates a method for controlling a pan/tilt head by selecting a target object through a touch screen on the pan/tilt head according to a preferred embodiment of the present invention;
FIG. 11 illustrates a method of operation of an electronic device in accordance with another preferred embodiment of the present invention;
fig. 12 schematically shows a pan-tilt system according to a preferred embodiment of the present invention.
Detailed Description
In the following, only certain exemplary embodiments are briefly described. As those skilled in the art will recognize, the described embodiments may be modified in various different ways, all without departing from the spirit or scope of the present invention. Accordingly, the drawings and description are to be regarded as illustrative in nature, and not as restrictive.
In the description of the present invention, it is to be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", "clockwise", "counterclockwise", and the like, indicate orientations and positional relationships based on those shown in the drawings, and are used only for convenience of description and simplicity of description, and do not indicate or imply that the device or element being referred to must have a particular orientation, be constructed and operated in a particular orientation, and thus, should not be considered as limiting the present invention. Furthermore, the terms "first", "second" and "first" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, features defined as "first", "second", may explicitly or implicitly include one or more of the described features. In the description of the present invention, "a plurality" means two or more unless specifically defined otherwise.
In the description of the present invention, it should be noted that unless otherwise explicitly stated or limited, the terms "mounted," "connected," and "connected" are to be construed broadly, e.g., as meaning either a fixed connection, a removable connection, or an integral connection, either mechanically, electrically, or in communication with each other; either directly or indirectly through intervening media, either internally or in any other relationship. The specific meanings of the above terms in the present invention can be understood by those skilled in the art according to specific situations.
In the present invention, unless otherwise expressly stated or limited, "above" or "below" a first feature means that the first and second features are in direct contact, or that the first and second features are not in direct contact but are in contact with each other via another feature therebetween. Also, the first feature being "on," "above" and "over" the second feature includes the first feature being directly on and obliquely above the second feature, or merely indicating that the first feature is at a higher level than the second feature. A first feature being "under," "below," and "beneath" a second feature includes the first feature being directly above and obliquely above the second feature, or simply meaning that the first feature is at a lesser level than the second feature.
The following disclosure provides many different embodiments or examples for implementing different features of the invention. To simplify the disclosure of the present invention, the components and arrangements of specific examples are described below. Of course, they are merely examples and are not intended to limit the present invention. Furthermore, the present invention may repeat reference numerals and/or letters in the various examples, such repetition is for the purpose of simplicity and clarity and does not in itself dictate a relationship between the various embodiments and/or configurations discussed. In addition, the present invention provides examples of various specific processes and materials, but one of ordinary skill in the art may recognize applications of other processes and/or uses of other materials.
The embodiments of the present invention will be described in conjunction with the accompanying drawings, and it should be understood that the embodiments described herein are only for the purpose of illustrating and explaining the present invention, and are not intended to limit the present invention.
As shown in fig. 1, an electronic device (embedded system) 10 has a camera 11 and a display 12, where the camera 11 includes one or more of a front camera, a rear camera, and an external camera, and the display 12 is a generalized display including a solid display of the electronic device and/or a hologram projected by the electronic device. The camera 11 may be invoked by various applications of the electronic device 10, such as WeChat telephony, video conferencing, face recognition, pan-tilt control, and so forth. In the existing electronic device, the authority of multiple applications to access the camera at the same time cannot be opened, that is, when the camera 11 is called by the process of one application, other applications cannot acquire the video stream data of the camera 11. When the camera 11 is called, in some cases, the video content captured by the camera 11 in real time is displayed on the display screen 12, as shown in fig. 1, an image captured by the camera 11 in real time is displayed on the display screen 12, and the target object 13 is a human face, but as will be readily understood by those skilled in the art, the target object 13 also includes a human body, a human eye, a dynamic object, and other targets suitable for tracking. The preferred embodiment of the invention is to read the video stream data recorded on the desktop screen to perform subsequent processing and operation.
Fig. 2 shows a method 100 of operation for the electronic device 10 described above, the method 100 of operation comprising, in accordance with a preferred embodiment of the present invention:
in step S101, a video stream of a screen recording image of the display screen 12 is acquired;
in step S102, first parameter information of the target object 13 in the video stream is acquired;
in step S103, the updated pose parameter of the electronic device 10 is output according to the first parameter information of the target object 13.
According to a preferred embodiment of the present invention, in step S101, the camera 11 is called by a first process, and the operation method 100 is executed by a second process, where the process is a process executed by an application program, and the first process is different from the second process.
According to a preferred embodiment of the present invention, first parameter information is obtained for determining the position and/or orientation of the target object 13, and the first parameter information includes a position parameter and/or a size parameter, and also includes feature information that can indirectly obtain the position and/or orientation of the target object 13: for example, acceleration parameters, which predict the position and/or orientation of the target object 13; also for example, color information, the position and/or orientation of the target object 13 may be estimated by analyzing the color change using a color histogram or texture information. The first parameter information includes one or more of a position parameter, a size parameter, an acceleration parameter, color information, biometric information, and motion characteristic information, and a method for obtaining the position and/or orientation of the target object 13 through the above parameter information or a combination thereof is within the protection scope of the present invention.
According to a preferred embodiment of the present invention, step S103 further comprises: according to the first parameter information of the target object 13, the updated pose parameters of the electronic device 10 are calculated so that the position and/or size of the target object 13 in the image captured by the camera 11 meet preset requirements. Preferably, the preset requirements include: enabling the target object 13 to be located at the center of the image acquired by the camera 11, and/or enabling the size of the target object 13 in the image acquired by the camera 11 to be larger than a preset value, and/or enabling the target object 13 to face the camera 11; the target object 13 is positioned at the right side 1/3 of the image captured by the camera 11, and/or the size of the target object 13 in the image captured by the camera 11 is smaller than a preset value, and/or the target object 13 is faced to the camera 11 with a side turned by 45 °.
According to a preferred embodiment of the present invention, as shown in fig. 3, the electronic device 10 is mounted on a cradle head 20, the cradle head 20 is a device capable of implementing a stable shooting function, and the step S103 further includes: outputting the updated pose parameters of the electronic device 10 to the pan/tilt head 20, the operating method 100 further comprises step S104:
in step S104, the pose of the electronic device 10 is adjusted by the pan/tilt head 20 according to the updated pose parameter of the electronic device 10. According to the pose parameter updated in real time, the pose of the electronic device 10 is adjusted in real time by controlling the holder 20, so that the position and/or size of the target object 13 meet the preset requirements, that is, the target object 13 is followed in real time.
According to a preferred embodiment of the present invention, the electronic device 10 includes one or more of a mobile phone, a PAD, a motion camera, AR/VR glasses, and a home smart camera, and an electronic device (embedded system) having a display device adopts the operation method 100 provided by the present invention, which is within the protection scope of the present invention. The target object 13 includes one or more of a human face, an iris, a heating element, and a dynamic target object, and the selection of the target object may be determined according to the function of the application program and the application scenario.
According to a preferred embodiment of the present invention, the method of operation 100 further comprises:
judging whether the camera 11 of the electronic equipment 10 is called;
when the camera 11 has been called, step S101 is executed.
The operation method 100 provided by the preferred embodiment of the present invention first determines whether the camera 11 of the electronic device 10 (such as an android phone or an apple phone) is called, and if not, the application program executing the operation method 100 may directly call the camera 11 to obtain video stream data of the camera 11, and perform subsequent operations. If the process of the other application is calling the camera 11, the application executing the operating method 100 acquires the video stream of the current desktop screen recording of the electronic device 10, and the right is usually opened for the application of the electronic device 10. In the electronic device 10, there may be a case where a plurality of cameras 11 may be called by different applications, in which case, unoccupied cameras 11 may also be called directly, and when all the cameras 11 have been called, the application executing the operation method 100 acquires a video stream of a desktop screen recording.
According to a preferred embodiment of the present invention, the method of operation 100 further comprises:
judging whether the video stream comprises an image collected by a camera 11;
when the video stream includes the image captured by the camera 11, step S102 is performed.
When the camera 11 is called, in some cases, the video content shot by the camera 11 in real time is displayed on the display screen 12, where the video content shot by the camera 11 in real time is displayed on the display screen 12 in a full screen manner, or the video content shot by the camera 11 in real time is displayed in a partial area of the display screen 12, in other cases, the desktop content of the electronic device 10 is displayed on the display screen 12, and the application executing the operation method 100 needs to determine whether the video stream of the current desktop screen includes an image captured by the camera 11 or a static desktop operation image.
In the operation method 100, the method for determining whether the video stream includes an image captured by the camera 11 includes:
A. the current display mode of the display screen 12 is obtained from the underlying data of the electronic device 10 to determine whether the video stream includes images captured by the camera 11.
B. The image frames of the video stream are subjected to a live body detection to determine whether the video stream includes an image captured by the camera 11.
C. The image frames of the video stream are judged through a training scene model to judge whether the video stream includes the images collected by the camera 11.
The desktop of the electronic device 10 usually has some icons of different application programs, and a scene understanding model may be trained by inputting desktop images with icons of different application programs, and whether the video stream content is the desktop is determined according to the scene understanding model. And if the video stream of the current screen recording image is the desktop, outputting one instruction, and if the video stream of the current screen recording image is not the desktop and is generally the image collected by the camera 11, outputting another instruction.
D. By comparing successive frames of the video stream, it is determined whether the video stream comprises images captured by the camera 11.
And detecting the target in the current screen recording image in real time through a target detection algorithm, and judging whether the current screen recording image is a desktop or an image acquired by the camera 11 according to the change of a target detection result between mutually continuous frames. For desktop images, icons are static, and detection results between two frames are generally the same; for the image collected by the camera 11, the picture is dynamically changed, the detection result between two frames is usually changed, and the judgment is performed according to the relative position change condition of the detection result between the consecutive frames.
In some cases, such as WeChat telephone and video conference, the image content captured by the camera 11 in real time and the remote image content are displayed on the display 12, wherein the remote image content includes a plurality of objects (such as human faces) similar to the features of the target object 13, and in the preferred embodiment of the present invention, the operation method 100 further includes selecting the target object 13.
As shown in fig. 4A and 4B, the operation method 100 further includes, according to a preferred embodiment of the present invention:
when recognizing that an object in the video stream makes a preset gesture, the object is taken as a target object 13.
As shown in fig. 5A and 5B, the operation method 100 further includes, according to a preferred embodiment of the present invention:
when it is recognized that the size of an object in the video stream is larger than a first threshold (fig. 5A) or the size ratio of an object in the video stream exceeds a second threshold (fig. 5B), the object is taken as the target object 13.
As shown in fig. 6A and 6B, the operation method 100 further includes, according to a preferred embodiment of the present invention:
when an object in the video stream is identified to be located within a preset area on the display screen, the object is taken as a target object 13.
The area range of the initial state target object and/or the size of the target object are set in the application executing the operation method 100, for example: and in an initial state, setting the area range of the target object as the central area of the screen recording image, and/or setting the size of the target object to be larger than a preset value, and executing subsequent operation (acquiring first parameter information of the target object and outputting a pose parameter updated by the electronic equipment according to the first parameter information) when an object appears in the central area of the screen recording image and/or the size of the object is larger than the preset value. It may also be set to end execution of the operation method 100 when the target object moves out of the area range of the target object and/or the size of the target object is smaller than another preset value. These are all within the scope of the present invention.
According to a preferred embodiment of the present invention, the method of operation 100 further comprises:
when an object in the video stream is identified to match a preset object, the object is taken as a target object 13.
A preset object, that is, information of a target object to be followed (for example, information of a human face I D of a user) is recorded in advance in an application program for executing the operation method 100, after the application program for executing the operation method 100 is started, a screen-recorded image is analyzed in real time, if an object conforming to the preset object is recognized in the screen-recorded image, the object is taken as the target object, and a subsequent operation is executed.
As shown in fig. 7, the method of operation 100 further includes, in accordance with a preferred embodiment of the present invention:
the object selected by the user of the electronic device 10 in the video stream is taken as the target object 13.
According to a preferred embodiment of the present invention, the method of operation 100 further comprises:
the distance of the object is tested by means of a binocular or infrared ranging function of the electronic device 10, and objects within a certain distance range are selected and further matched with objects in the video stream to determine the target object 13.
The user may trigger the start or stop of the operation method 100 by holding the hardware button 21 of the pan/tilt head 20 or other external buttons or a remote controller, for example, in the mode of starting the recording screen, starting the application program by double-clicking the volume button to execute the operation method 100, and double-clicking again to exit the execution. The user can also select the target object through the hardware button, the external button and the remote controller.
As shown in fig. 8, the method of operation 100 further includes, in accordance with a preferred embodiment of the present invention:
the switching of the display screen 12 of the electronic device 10 is recognized, and when the display screen 12 is switched, the first parameter information of the target object 13 is re-acquired or the first parameter information of the target object 13 acquired using a frame before the switching is used.
In WeChat telephone and video conference, the user sometimes switches the screen, and at this time, the target object 13 needs to be repositioned, and the repositioning of the target object 13 after plane switching can adopt two modes: the first way includes restarting the application to perform the method 100 of operation, relocating the target object 13; the second way includes identifying the process of screen switching by an algorithm, automatically switching the target object 13, or switching according to the position of the screen, and continuing to use the target object 13 acquired by switching the previous frame.
As shown in fig. 9, the method of operation 100 further includes, in accordance with a preferred embodiment of the present invention:
identifying a first image area and an additional second image area captured by a camera 11 in the video stream; in step S102, the first parameter information of the target object 13 is the first parameter information of the target object 13 in the first image area.
In common live broadcast software, an application program executing the operation method 100 needs to determine which images are captured by the camera 11 and which live broadcast barrages are captured, acquire an image area captured by the camera 11, and then perform subsequent operations using the area.
The present invention further provides a control method 300 of the pan/tilt head 20, as shown in fig. 3, the pan/tilt head 20 communicates with the electronic device 10 disposed thereon, and the control method 300 includes:
using the operating method 100 as described above, first parameter information of the target object 13 is acquired in real time; and
and adjusting the pose of the electronic equipment 10 in real time through the cradle head 20 according to the first parameter information of the target object 13.
According to a preferred embodiment of the present invention, the control method 300 further comprises:
controls for the pan and tilt head 20 are turned on and/or off based on one or more of a user's gestures, sounds, actions, gestures, expressions.
According to a preferred embodiment of the present invention, the control method 300 further comprises:
a user may turn on and/or off control of the head 20 by holding hardware buttons 21 or other externally attached buttons, remote controls, of the head 20.
As shown in fig. 10, according to a preferred embodiment of the present invention, wherein the pan/tilt head 20 includes a touch screen 21, the control method 300 further includes:
the user may select the target object 13 via the touch screen 21 and/or turn on and/or off the control of the head 20.
The present invention also provides an operation method 200 for an electronic device 10, as shown in fig. 11, where the electronic device 10 has a camera 11 and a display 12, and the operation method 200 includes:
in step S201, when the camera 11 is called by a first process, acquiring a video stream of a screen recording image of the display screen 12 through a second process, where the process is a process executed by an application program, and the first process is different from the second process;
in step S202, a position parameter and/or a size parameter of the target object 13 in the video stream is obtained; and
in step S203, the focal length of the camera 11 is adjusted according to the position parameter and/or the size parameter of the target object 13, so that the position and/or the size of the target object 13 in the image captured by the camera 11 meet the preset requirement. The preset requirements include: the target object 13 is positioned at the center of the image acquired by the camera 11, the size of the target object 13 in the image acquired by the camera 11 is larger than a preset value, and the camera 11 is arranged on the front side of the target object 13.
The present invention also provides a pan-tilt system 40, as shown in fig. 12, the pan-tilt system 40 includes:
a cradle head 20 on which the electronic device 10 can be mounted, the electronic device 10 having a camera 11 and a display 12;
a control unit 30, the control unit 30 being in communication with the head 20 and being capable of performing the methods 100, 200 of operation of the electronic device 10 as described above.
According to a preferred embodiment of the present invention, the control unit 30 is an application or software process that is executable on the electronic device 10.
According to a preferred embodiment of the present invention, the pan/tilt head system 40 comprises a handle and a stabilizer, the electronic device 10 is loaded on the pan/tilt head system 40, the stabilizer is used to realize the stable shooting function of the electronic device 10, the handle has an accommodating space therein, and the load can be connected to the stabilizer, wherein the pan/tilt head has an extended state and a storage state, and in the extended state, the stabilizer is at least partially located outside the handle and connected to the handle; in the storage state, the stabilizer is stored in the containing space.
According to a preferred embodiment of the present invention, the head system 40 further comprises a battery and a wireless charging module connected to the battery and capable of wirelessly charging a load placed thereon.
According to a preferred embodiment of the present invention, the holder system 40 further includes a load protection shell, the load protection shell is used for mounting a load, the stabilizer is provided with a first quick release fixing structure, and the load protection shell is provided with a second quick release fixing structure connected to the first quick release fixing structure.
The present invention also provides a computer-readable storage medium comprising computer-executable instructions stored thereon which, when executed by a processor, implement the method of operation 100, 200 of an electronic device as described above.
The preferred embodiment of the present invention provides an operation method of an electronic device (embedded system), where in a case where another application has invoked a camera, an application executing the operation method obtains first parameter information of a target object by obtaining a video stream of a screen recording image, and outputs a pose parameter updated by the electronic device according to the first parameter information. On the basis of the operation method of the electronic device, the preferred embodiment of the invention further provides a control method of the pan-tilt, a control method of the camera and a pan-tilt system. The embodiment provided by the invention enables the user to synchronously perform operations such as face recognition, face beautifying, human body temperature measurement, holder following and the like in the process of making WeChat calls and carrying out video conferences. The preferred embodiment of the invention can complete the identification and tracking of the target object without depending on other application programs, and practice proves that the invention has beneficial effects.
Finally, it should be noted that: although the present invention has been described in detail with reference to the foregoing embodiments, it will be apparent to those skilled in the art that changes may be made in the embodiments and/or equivalents thereof without departing from the spirit and scope of the invention. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims (31)
1. An operating method for an electronic device, wherein the electronic device has a camera and a display screen, the operating method comprising:
s101: acquiring a video stream of a screen recording image of the display screen;
s102: acquiring first parameter information of a target object in the video stream;
s103: and outputting the updated pose parameters of the electronic equipment according to the first parameter information of the target object.
2. The operation method according to claim 1, wherein in the step S101, the camera is called by a first process; the method of operation is performed by a second process, wherein the first process is different from the second process.
3. Operating method according to claim 1 or 2, wherein the first parameter information comprises a position parameter and/or a size parameter, an acceleration parameter.
4. The operating method according to claim 1 or 2, wherein step S103 further comprises:
according to the first parameter information of the target object, calculating an updated pose parameter of the electronic equipment, so that the position and/or the size of the target object in the image collected by the camera meet preset requirements.
5. The operating method according to claim 1 or 2, wherein the electronic device is mounted on a pan-tilt head, the step S103 further comprising: outputting the updated pose parameters of the electronic device to the pan-tilt head, the method of operation further comprising:
s104: and adjusting the pose of the electronic equipment according to the updated pose parameter of the electronic equipment through the cradle head.
6. The method of claim 1 or 2, wherein the electronic device comprises one or more of a cell phone, a PAD, a motion camera, AR/VR glasses, a home smart camera, and the target object comprises one or more of a human face, an iris, a heat generator, and a dynamic target object.
7. The method of operation of claim 1 or 2, further comprising:
judging whether a camera of the electronic equipment is called;
when the camera has been called, the step S101 is executed.
8. The method of operation of claim 1 or 2, further comprising:
judging whether the video stream comprises an image collected by a camera or not;
when the video stream includes images captured by a camera, the step S102 is performed.
9. The method of operation of claim 8, wherein said step of determining whether the video stream is an image captured by a camera comprises:
and acquiring the current display mode of the display screen from the bottom layer data of the electronic equipment so as to judge whether the video stream comprises the image acquired by the camera.
10. The method of operation of claim 8, wherein the step of determining whether the video stream is an image captured by a camera comprises:
and performing living body detection on the image frames of the video stream to judge whether the video stream comprises images collected by a camera.
11. The method of operation of claim 8, wherein the step of determining whether the video stream is an image captured by a camera comprises:
and judging the image frames of the video stream through a training scene model so as to judge whether the video stream comprises images acquired by a camera.
12. The method of operation of claim 8, wherein the step of determining whether the video stream is an image captured by a camera comprises:
and judging whether the video stream comprises images collected by a camera or not by comparing the continuous frames of the video stream.
13. The method of operation of claim 1 or 2, further comprising:
and when recognizing that an object in the video stream makes a preset gesture, taking the object as the target object.
14. The method of operation of claim 1 or 2, further comprising:
and when the size of an object in the video stream is identified to be larger than a first threshold value or the size proportion of the object in the video stream exceeds a second threshold value, taking the object as the target object.
15. The method of operation of claim 1 or 2, further comprising:
and when an object in the video stream is identified to be located in a preset area on a display screen, taking the object as the target object.
16. The method of operation of claim 1 or 2, further comprising:
and when an object in the video stream is identified to be consistent with a preset object, taking the object as the target object.
17. The method of operation of claim 1 or 2, further comprising:
and taking the object selected by the user of the electronic equipment in the video stream as the target object.
18. The method of operation of claim 1 or 2, further comprising:
and selecting the target object according to the operation of a user on a hardware button arranged on the electronic equipment or the cradle head and/or a remote controller communicated with the electronic equipment or the cradle head.
19. The method of operation of claim 1 or 2, further comprising:
and identifying the switching of the display screen of the electronic equipment, and when the display screen is switched, acquiring the first parameter information of the target object again or using the first parameter information of the target object acquired by switching the previous frame.
20. The method of operation of claim 1 or 2, further comprising:
identifying a first image area and an additional second image area captured by the camera in the video stream; in step S102, the first parameter information of the target object is the first parameter information of the target object in the first image region.
21. A control method of a pan/tilt head, wherein the pan/tilt head communicates with an electronic device provided thereon, the control method comprising:
using the operating method according to any one of claims 1 to 20, acquiring first parameter information of the target object in real time; and
and adjusting the pose of the electronic equipment in real time through the cradle head according to the first parameter information of the target object.
22. The control method of claim 21, further comprising:
turning on and/or off control of the head according to one or more of a user's gesture, sound, motion, gesture, expression.
23. The control method of claim 21, further comprising:
and according to the operation of a user on a hardware button arranged on the electronic equipment or the cradle head and/or a remote controller communicated with the electronic equipment or the cradle head, starting and/or closing the control of the cradle head.
24. The control method of claim 21, wherein the pan-tilt device comprises a touch screen, the method further comprising:
the user may select the target object via a touch screen and/or turn on and/or off control of the head.
25. An operating method for an electronic device, wherein the electronic device has a camera and a display screen, the operating method comprising:
s201: when the camera is called by a first process, acquiring a video stream of a screen recording image of the display screen through a second process, wherein the first process is different from the second process;
s202: acquiring a position parameter and/or a size parameter of a target object in the video stream; and
s203: and adjusting the focal length of the camera according to the position parameter and/or the size parameter of the target object, so that the position and/or the size of the target object in the image acquired by the camera meet the preset requirement.
26. A pan-tilt system comprising:
a cradle head to which an electronic device is detachably attached as a load, the electronic device having a camera and a display screen;
a control unit in communication with the head and operable to perform the method of any one of claims 1-25.
27. A pan-tilt head system according to claim 26, wherein said control unit is an APP or software process operable on said electronic device.
28. A head system according to claim 26 or 27, wherein the head comprises a handle having a receiving space therein to which a load may be attached and a stabiliser, wherein the head has an extended state in which the stabiliser is at least partially located outside the handle and attached thereto and a stowed state; in the storage state, the stabilizer is stored in the containing space.
29. A tripod head system according to claim 28, wherein said tripod head further comprises a battery and a wireless charging module connected to said battery and capable of wirelessly charging a load placed thereon.
30. A head system as claimed in claim 28, wherein the head system further comprises a protective load shell for mounting a load, the stabilizer being provided with a first quick release fixing structure, the protective load shell being provided with a second quick release fixing structure for connection with the first quick release fixing structure.
31. A computer-readable storage medium comprising computer-executable instructions stored thereon which, when executed by a processor, implement the method of any one of claims 1-25.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011218182.1A CN112866558A (en) | 2020-11-04 | 2020-11-04 | Operation method of electronic equipment, control method of holder and holder system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011218182.1A CN112866558A (en) | 2020-11-04 | 2020-11-04 | Operation method of electronic equipment, control method of holder and holder system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112866558A true CN112866558A (en) | 2021-05-28 |
Family
ID=75995438
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011218182.1A Pending CN112866558A (en) | 2020-11-04 | 2020-11-04 | Operation method of electronic equipment, control method of holder and holder system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112866558A (en) |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20120117472A1 (en) * | 2010-11-10 | 2012-05-10 | Cyberlink Corp. | Systems and Methods for Application of Special Effects to a Captured Video Stream |
| CN107659768A (en) * | 2017-08-08 | 2018-02-02 | 珠海全志科技股份有限公司 | A kind of system and method based on the more Application share cameras of Android |
| CN108055501A (en) * | 2017-11-22 | 2018-05-18 | 天津市亚安科技有限公司 | A kind of target detection and the video monitoring system and method for tracking |
| CN111199172A (en) * | 2018-11-19 | 2020-05-26 | 中移动信息技术有限公司 | Terminal screen recording-based processing method and device and storage medium |
| CN111885303A (en) * | 2020-07-06 | 2020-11-03 | 雍朝良 | Active tracking recording and shooting visual method |
-
2020
- 2020-11-04 CN CN202011218182.1A patent/CN112866558A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20120117472A1 (en) * | 2010-11-10 | 2012-05-10 | Cyberlink Corp. | Systems and Methods for Application of Special Effects to a Captured Video Stream |
| CN107659768A (en) * | 2017-08-08 | 2018-02-02 | 珠海全志科技股份有限公司 | A kind of system and method based on the more Application share cameras of Android |
| CN108055501A (en) * | 2017-11-22 | 2018-05-18 | 天津市亚安科技有限公司 | A kind of target detection and the video monitoring system and method for tracking |
| CN111199172A (en) * | 2018-11-19 | 2020-05-26 | 中移动信息技术有限公司 | Terminal screen recording-based processing method and device and storage medium |
| CN111885303A (en) * | 2020-07-06 | 2020-11-03 | 雍朝良 | Active tracking recording and shooting visual method |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106572299B (en) | Camera opening method and device | |
| CN112118380B (en) | Camera control method, device, equipment and storage medium | |
| KR101537948B1 (en) | Photographing method and apparatus using pose estimation of face | |
| JP6103948B2 (en) | IMAGING DEVICE, REMOTE OPERATION TERMINAL, CAMERA SYSTEM, IMAGING DEVICE CONTROL METHOD AND PROGRAM, REMOTE OPERATION TERMINAL CONTROL METHOD AND PROGRAM | |
| CN105430262A (en) | Photographing control method and photographing control device | |
| EP3761627B1 (en) | Image processing method and apparatus | |
| KR20100055946A (en) | Method and apparatus for generating thumbnail of moving picture | |
| CN107426502A (en) | Image pickup method and device, electronic equipment | |
| CN111988493B (en) | Interaction processing method, device, equipment and storage medium | |
| KR102575196B1 (en) | Photography method, photography apparatus, electronic device, and storage medium | |
| CN108668080A (en) | Prompt method and device, the electronic equipment of camera lens degree of fouling | |
| EP3822742B1 (en) | Method, apparatus and device for triggering shooting mode, and storage medium | |
| CN108063859B (en) | Automatic photographing control method, terminal and computer storage medium | |
| KR101949218B1 (en) | Method and apparatus for photographing in portable terminal | |
| CN111984347A (en) | Interaction processing method, device, equipment and storage medium | |
| US20170244891A1 (en) | Method for automatically capturing photograph, electronic device and medium | |
| CN111314619B (en) | Shooting angle control method, terminal and storage medium | |
| CN111698600A (en) | Processing execution method and device and readable medium | |
| US8866934B2 (en) | Image pickup apparatus capable of deleting video effect superimposed on moving image, method of controlling the apparatus, and moving image-recording apparatus, as well as storage medium | |
| CN112866558A (en) | Operation method of electronic equipment, control method of holder and holder system | |
| CN111726531B (en) | Image shooting method, processing method, device, electronic equipment and storage medium | |
| CN113315903B (en) | Image acquisition method and device, electronic equipment and storage medium | |
| CN110493503B (en) | Image acquisition method, electronic equipment and storage medium | |
| CN109426758A (en) | Acquisition method and device, the computer readable storage medium of skin characteristic information | |
| CN110099160B (en) | Equipment control method and device and electronic equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20210528 |