CN115294631A - Adjustment method and device in panoramic video playing, electronic equipment and storage medium - Google Patents

Adjustment method and device in panoramic video playing, electronic equipment and storage medium Download PDF

Info

Publication number
CN115294631A
CN115294631A CN202210931139.2A CN202210931139A CN115294631A CN 115294631 A CN115294631 A CN 115294631A CN 202210931139 A CN202210931139 A CN 202210931139A CN 115294631 A CN115294631 A CN 115294631A
Authority
CN
China
Prior art keywords
user
eye position
face image
initial state
playing
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202210931139.2A
Other languages
Chinese (zh)
Inventor
李鸣
肖云
储磊
赵媛
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Future Tv Co ltd
Original Assignee
Future Tv Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Future Tv Co ltd filed Critical Future Tv Co ltd
Priority to CN202210931139.2A priority Critical patent/CN115294631A/en
Publication of CN115294631A publication Critical patent/CN115294631A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/161—Detection; Localisation; Normalisation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44—Arrangements for executing specific programs
    • G06F9/451—Execution arrangements for user interfaces
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/18—Eye characteristics, e.g. of the iris
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04M—TELEPHONIC COMMUNICATION
    • H04M1/00—Substation equipment, e.g. for use by subscribers
    • H04M1/72—Mobile telephones; Cordless telephones, i.e. devices for establishing wireless links to base stations without route selection
    • H04M1/724—User interfaces specially adapted for cordless or mobile telephones
    • H04M1/72448—User interfaces specially adapted for cordless or mobile telephones with means for adapting the functionality of the device according to specific conditions
    • H04M1/72454—User interfaces specially adapted for cordless or mobile telephones with means for adapting the functionality of the device according to specific conditions according to context-related or environment-related conditions

Landscapes

  • Engineering & Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Software Systems (AREA)
  • Physics & Mathematics (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Ophthalmology & Optometry (AREA)
  • Environmental & Geological Engineering (AREA)
  • General Engineering & Computer Science (AREA)
  • Image Analysis (AREA)

Abstract

The application provides an adjusting method and device in panoramic video playing, electronic equipment and a storage medium, and relates to the technical field of computers. The adjustment method in the panoramic video playing comprises the following steps: acquiring a face image of a user, wherein the face image comprises: the method comprises the steps of obtaining a face image in an initial state of a user and a face image in a multimedia resource playing process; detecting a face image of an initial state of a user and marking an eye position of the initial state; calculating and acquiring the eye position change of the user according to the eye position in the initial state and the eye position in the face image in the multimedia resource playing process; and adjusting the playing angle and/or size of the panoramic video of the multimedia resource according to the change of the eye position of the user. The adjustment method in the panoramic video playing can adjust the playing angle of the panoramic video more conveniently and more smoothly without upgrading hardware equipment, and the watching experience of a user is improved.

Description

Adjustment method and device in panoramic video playing, electronic equipment and storage medium
Technical Field
The present application relates to the field of computer technologies, and in particular, to an adjustment method and apparatus in panoramic video playing, an electronic device, and a storage medium.
Background
With the development of panoramic camera technology, panoramic video is more and more popular with people, the panoramic video is a video shot in all directions by a 3D camera, and can be displayed on film watching equipment such as a computer and a mobile phone for users to watch, and when the users watch the panoramic video, the playing angle can be adjusted at will, so that the users can watch in all directions.
At present, the playing angle of a panoramic video is mostly adjusted by acquiring the posture change of a sensor in a film watching device, namely, shaking the film watching device up and down, left and right, or adjusting the playing angle of the panoramic video by touching a virtual button on a screen.
However, no matter the viewing device is shaken, or the virtual button on the screen is touched, the user needs to adjust the playing angle of the panoramic video by hand or a mouse, and the user cannot adjust the playing angle of the panoramic video more conveniently and more smoothly without upgrading hardware, so that better viewing experience cannot be provided for the user.
Disclosure of Invention
An object of the present application is to provide an adjustment method, an adjustment device, a server, and a storage medium in panoramic video playing, so that a user can adjust the playing angle and/or size of a panoramic video more conveniently and more smoothly without upgrading hardware devices, thereby improving the viewing experience of the user.
In order to achieve the above purpose, the embodiments of the present application adopt the following technical solutions:
in a first aspect, in a possible implementation example, an embodiment of the present application provides an adjustment method in panoramic video playing, including:
capturing a user face image, wherein the user face image comprises: the face image of the user in the initial state and the face image in the multimedia resource playing process;
detecting a face image of the user in an initial state, and marking the eye position of the user in the initial state;
calculating and acquiring the eye position change of the user according to the eye position of the initial state and the eye position in the face image in the multimedia resource playing process;
and adjusting the playing angle and/or size of the panoramic video of the multimedia resource according to the change of the eye position of the user.
In a possible implementation example, the method includes: the positions of the central points of the irises of the two eyes of the user and the positions of the canthus of the two eyes of the user.
In a possible implementation example, the method, in which the calculating of the eye position change of the user according to the eye position of the initial state and the eye position in the face image during the playing of the multimedia resource includes:
acquiring a current face image in the multimedia resource playing process;
and calculating and acquiring the eye position change direction of the user according to the eye position of the initial state and the eye position in the current face image.
In a possible implementation example, the method, in which the calculating of the eye position change direction of the acquiring user according to the eye position of the initial state and the eye position in the current face image includes:
calculating the change of the connecting line of the iris center points of the two eyes of the user according to the eye position of the initial state and the eye position in the current face image;
if the change of the connecting line of the iris central points of the two eyes of the user is deviated from the horizontal direction, the change direction of the positions of the eyes of the user moves towards two sides;
if the change of the connecting line of the iris center points of the two eyes of the user is that the connecting line moves in the vertical direction, the change direction of the positions of the eyes of the user is that the connecting line moves in the vertical direction;
and if the ratio of the connecting line of the center points of the irises of the two eyes of the user in the current facial image to the connecting line of the center points of the irises of the two eyes of the user in the facial image in the initial state of the user is changed, the change direction of the positions of the eyes of the user is close to or far away from the film watching equipment.
In a possible implementation example, the method, in which the adjusting a panoramic video playing angle and/or a panoramic video playing size of the multimedia resource according to the change in the eye position of the user includes:
if the eye position of the user changes towards the two sides, adjusting the panoramic video playing angle of the multimedia resource according to the angle of the central point connecting line deviating from the horizontal direction;
if the eye position change direction of the user is moving in the vertical direction, adjusting the panoramic video playing angle of the multimedia resource according to the moving distance of the central point connecting line in the vertical direction;
and if the eye position change direction of the user is close to or far away from the film watching equipment, adjusting the playing size of the multimedia resource according to the proportion change of the central point connecting line on the face image of the user.
In a possible implementation example, after the acquiring the face image of the user, the method further includes:
if the face image in the multimedia resource playing process does not meet the preset conditions, the face image is taken as an invalid image to be removed, wherein the preset conditions comprise: and detecting that the position change of the front face and the eyes is larger than a preset range.
In a possible implementation example, the method further comprises:
when the multimedia resource playing is started, detecting the change range of the eye position in the current face image and the eye position in the initial state;
if the variation range is larger than a preset threshold value, displaying a prompt message;
if the facial image of the user is detected not to be facing the film watching device, the facial image is invalid data;
if the eye position of the user is not detected, the face image is invalid data.
In a second aspect, in a possible implementation example, an embodiment of the present application further provides an adjusting apparatus in panoramic video playing, including:
the acquisition module is used for acquiring a face image of a user;
the detection module is used for detecting the facial image of the user in the initial state and marking the eye position of the user in the initial state;
the computing module is used for computing and acquiring the eye position change of the user according to the eye position of the initial state and the eye position in the face image in the multimedia resource playing process;
and the control module is used for adjusting the panoramic video playing angle and/or size of the multimedia resource according to the change of the eye position of the user.
In a third aspect, an embodiment of the present application provides an electronic device, including: the device comprises a processor, a storage medium and a bus, wherein the storage medium stores program instructions executable by the electronic equipment, when a server runs, the processor and the storage medium are communicated through the bus, and the processor executes the program instructions to execute the steps of the adjusting method in the panoramic video playing.
In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, where a computer program is stored on the storage medium, and when the computer program is executed by a processor, the computer program performs an adjustment method in the panoramic video playing.
To sum up, the beneficial effect of this application is:
the application provides an adjusting method and device in panoramic video playing, an electronic device and a storage medium, wherein the adjusting method in the panoramic video playing comprises the following steps: acquiring a face image of a user, wherein the face image comprises: the face image of the user in the initial state and the face image in the multimedia resource playing process; detecting a face image of an initial state of a user and marking an eye position of the initial state; calculating and acquiring the eye position change of the user according to the eye position in the initial state and the eye position in the face image in the multimedia resource playing process; and adjusting the playing angle and/or size of the panoramic video of the multimedia resource according to the change of the eye position of the user. The application provides a method for adjusting panoramic video playing angle and/or size by detecting eye position change, compare in prior art through rocking the sight shadow equipment and come to adjust panoramic video playing angle and/or size, the method that this application embodiment provided is more convenient, the user only need the face remove can, need not to adjust panoramic video playing angle and/or size with hand or mouse when seeing the shadow, under the condition of not upgrading hardware equipment, can be more convenient, adjust panoramic video's playing angle more smoothly, user watching experience has been promoted.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are required to be used in the embodiments will be briefly described below, it should be understood that the following drawings only illustrate some embodiments of the present application and therefore should not be considered as limiting the scope, and for those skilled in the art, other related drawings can be obtained from the drawings without inventive effort.
Fig. 1 is a schematic flowchart illustrating an adjustment method in playing a panoramic video according to an embodiment of the present disclosure;
FIG. 2 is a schematic diagram of an eye position marker provided by an embodiment of the present application;
fig. 3 is a schematic flowchart illustrating an adjustment method in playing a panoramic video according to another embodiment of the present application;
fig. 4 is a schematic structural diagram of an adjusting apparatus for playing panoramic video according to an embodiment of the present disclosure.
Description of the reference numerals: 1. an iris center point; 2. the inner canthus; 3. the external canthus; 401. an acquisition module; 402. a detection module; 403. a calculation module; 404. and a control module.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are some embodiments of the present application, but not all embodiments.
Thus, the following detailed description of the embodiments of the present application, presented in the accompanying drawings, is not intended to limit the scope of the claimed application, but is merely representative of selected embodiments of the application. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
In the description of the present application, it should be noted that if the terms "upper", "lower", etc. are used to indicate an orientation or a positional relationship based on an orientation or a positional relationship shown in the drawings or an orientation or a positional relationship which is usually placed when the product of the application is used, the description is merely for convenience of description and simplification of the application, but the indication or suggestion that the device or the element referred to must have a specific orientation, be constructed and operated in a specific orientation, and thus, cannot be understood as a limitation of the application.
Furthermore, the terms "first," "second," and the like in the description and in the claims of the present application and in the above-described drawings are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It should be understood that the data so used may be interchanged under appropriate circumstances such that embodiments of the application described herein may be implemented in sequences other than those illustrated or described herein. Moreover, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
It should be noted that the features of the embodiments of the present application may be combined with each other without conflict.
Fig. 1 is a schematic flowchart of an adjustment method in playing a panoramic video according to an embodiment of the present application, and with reference to fig. 1, the method includes:
s101, collecting a face image of a user.
Wherein the user face image includes: a face image in an initial state, and a face image in a multimedia asset playing process.
Specifically, for example, a camera device on the viewing device may be used to capture a facial image of a user, where an initial state refers to a comfortable distance or a comfortable position where the user is located to view a movie, before viewing the movie, the user needs to open a software setting option in the viewing device, a capture function of the facial image of the user is started through the software setting option, accordingly, the viewing device starts to detect whether the camera device is present, if so, the camera device is opened, the capture of the facial image in the initial state is started, and if not, the user is prompted to retry after externally connecting the camera device, so that the facial image in the initial state is recorded through the camera device, and in a multimedia resource playing process, the camera device may capture the current facial image of the user in real time or periodically.
The acquisition range of the facial image of the user may be determined according to the acquisition range of the image capturing device, for example, the acquisition range may include the facial range of the user, the upper body part region, and the like, which is not limited herein.
S102, detecting a face image in an initial state of a user, and marking the eye position in the initial state.
Specifically, after the camera device collects the face image of the user in the initial state, the film watching device detects the face image of the user in the initial state, obtains the eye position of the user according to the face image, marks the eye position of the user in the initial state, and marks the current eye position of the user according to the face image of the multimedia resource in the playing process in the process of continuously collecting the face image of the user.
S103, calculating and acquiring the eye position change of the user according to the eye position in the initial state and the eye position in the face image in the multimedia resource playing process.
Specifically, the viewing device may compare the marked eye position of the user in the initial state with the marked eye position of the user in the multimedia resource playing process to obtain a change of the current eye position of the user in the multimedia resource playing process compared with the eye position of the user in the initial state.
And S104, adjusting the panoramic video playing angle and/or size of the multimedia resource according to the change of the eye position of the user.
Specifically, the viewing device may determine that the face of the user may move or perform other actions according to the change of the current eye position of the user in the playing process of the obtained multimedia resource compared to the eye position of the user in the initial state, and adjust the playing angle and/or size of the panoramic video of the multimedia resource in order to enable the user to better view the panoramic video, thereby achieving the adjustment of the playing angle and/or size of the panoramic video.
The multimedia resource in this embodiment may be an image, a video, or the like played through a panoramic video.
In summary, in the embodiment of the present application, according to the eye position of the initial state and the eye position in the facial image during the playing of the multimedia resource, the change of the eye position of the user can be calculated and obtained, and then the playing angle and/or size of the panoramic video can be adjusted according to the change of the eye position, compared with the prior art in which the playing angle and/or size of the panoramic video is adjusted by shaking the viewing device, the method provided by the embodiment of the present application is more convenient and faster, the user only needs to move the face, and the playing angle and/or size of the panoramic video does not need to be adjusted by using a hand or a mouse during viewing, so that the playing angle of the panoramic video can be adjusted more conveniently and more smoothly without upgrading hardware devices, and the viewing experience of the user is improved.
Fig. 2 is a schematic diagram of an eye position marker provided in an embodiment of the present application, and referring to fig. 2, the eye position in the embodiment may include, for example: the positions of the center points of the irises of the two eyes of the user and the positions of the canthi of the two eyes of the user may be other positions of the eyes, and are not limited herein. Wherein the position of the canthus may comprise the inner canthus 2 or the outer canthus 3.
In addition, the positions of the eyes in the above embodiments may be marked, for example, the positions of the center points of the irises of the two eyes of the user and the positions of the inner canthus 2 or the outer canthus 3 of the two eyes of the user, or other positions of the eyes may be marked.
On the basis of the adjustment method in the panoramic video playing provided by the above embodiment, in the above embodiment, calculating and acquiring the eye position change of the user according to the eye position in the initial state and the eye position in the face image in the multimedia resource playing process includes:
and acquiring a current face image in the multimedia resource playing process.
And calculating and acquiring the eye position change direction of the user according to the eye position in the initial state and the eye position in the current face image.
Specifically, the viewing device may compare the eye position of the marked user in the initial state with the current eye position of the marked user in the multimedia resource playing process, and obtain a change direction of the current eye position of the user in the multimedia resource playing process compared with the eye position of the user in the initial state.
For example, the eye position of the user in the initial state of the marked user and the coordinate information of the current eye position of the user in the playing process of the marked multimedia resource may be compared to determine whether the eye position of the user has changed.
On the basis of the adjustment method in the panoramic video playing provided by the foregoing embodiment, an embodiment of the present application further provides a possible implementation manner for calculating a direction of change of the eye position of the user, where in the foregoing S103, the calculating and obtaining the direction of change of the eye position of the user according to the eye position of the initial state and the eye position in the current face image includes:
first, the change of the connecting line of the iris center points of the two eyes of the user is calculated according to the eye position of the initial state and the eye position in the current face image.
Specifically, the viewing device may obtain positions of center points of irises of the two eyes of the user, so as to obtain a connection line of the center points of the irises of the two eyes of the user, and compare the current connection line position of the center points of the irises of the two eyes of the user with the connection line position of the center points of the irises of the two eyes of the user in the initial state, so as to obtain a change of the current connection line position of the center points of the irises of the two eyes of the user compared with the connection line position of the center points of the irises of the two eyes of the user in the initial state.
Specifically, there may be one or more of the following:
1. if the change of the connecting line of the central points of the irises of the two eyes of the user is deviated from the horizontal direction, the change direction of the positions of the eyes of the user moves towards two sides.
Specifically, if it is detected that a change of a connecting line of iris center points of two eyes of the user deviates from a horizontal direction, it is determined that the eye position change direction of the user moves to both sides, where the horizontal direction is the same direction as a direction of a geodetic horizon.
In an actual application scene, when a user moves the face to approach or move away from shoulders on two sides, the connecting line of the center points of the irises of the eyes of the user changes to deviate from the horizontal direction, and at this time, the viewing device can judge that the change direction of the positions of the eyes of the user moves to two sides according to the change of the connecting line of the center points of the irises of the two eyes of the user.
2. If the change of the connecting line of the center points of the irises of the two eyes of the user is moving in the vertical direction, the change direction of the positions of the eyes of the user is moving in the vertical direction.
Specifically, if it is detected that the change of the connecting line of the center points of the irises of the two eyes of the user is moving in the vertical direction, it is determined that the change direction of the positions of the eyes of the user is moving in the vertical direction.
In an actual application scene, when the user squats, stands or tiptoes, the change of the connecting line of the iris central points of the eyes of the user is in a vertical direction, and at this time, the film watching equipment can judge that the change direction of the position of the eyes of the user is in a vertical direction according to the change of the connecting line of the iris central points of the two eyes of the user.
3. If the ratio of the connecting line of the iris central points of the two eyes in the current facial image to the connecting line of the iris central points of the two eyes in the facial image in the initial state of the user in the facial image of the user is changed, the change direction of the positions of the eyes of the user is close to or far away from the film watching equipment.
Specifically, the above-mentioned captured image of the face of the user may also include an image around the face of the user, that is, an image in a certain range may be captured according to the viewing range of the image capturing device. If the ratio of the connecting line of the center points of the irises of the two eyes of the user to the connecting line of the center points of the irises of the two eyes of the user in the face image of the initial state of the user is detected to be changed, the change direction of the positions of the eyes of the user is judged to be close to or far away from the film watching equipment.
In an actual application scene, when a user moves close to or far from the film watching equipment, the ratio of the connecting line of the central points of the irises of the two eyes of the user to the connecting line of the central points of the irises of the two eyes of the user in the facial image of the initial state of the user is changed, and at the moment, the film watching equipment can judge that the position change direction of the eyes of the user is close to or far from the film watching equipment according to the ratio change of the connecting line of the central points of the irises of the two eyes of the user in the collected facial image of the user.
On the basis of the adjustment method in the panoramic video playing provided in the foregoing embodiment, an embodiment of the present application further provides a possible implementation manner for adjusting the panoramic video playing angle and/or size of the multimedia according to the change of the eye position, where in the foregoing S104, the panoramic video playing angle and/or size of the multimedia resource is adjusted according to the change of the eye position of the user, and the method includes a combination of one or more of the following conditions:
1. and if the eye position of the user changes towards the two sides, adjusting the panoramic video playing angle of the multimedia resource according to the angle of the central point connecting line deviating from the horizontal direction.
Specifically, in the initial state, the angle of the connecting line of the central points of the irises of the two eyes of the user deviating from the horizontal direction is recorded as 0 °, and the playing angle of the panoramic video of the multimedia resource in the horizontal direction is recorded as 0 °. In the process of playing the multimedia resource, if the angle of the connecting line of the central points of the irises of the two eyes of the user deviating from the horizontal direction is 1 degree, correspondingly adjusting the playing angle of the panoramic video of the multimedia resource in the horizontal direction to be 1 degree; if the angle of the connecting line of the iris central points of the two eyes of the user deviating from the horizontal direction is-1 degrees, the playing angle of the panoramic video of the multimedia resource in the horizontal direction is correspondingly adjusted to-1 degrees, the playing angle of the panoramic video of the multimedia resource in the horizontal direction is kept the same as the angle of the connecting line of the iris central points of the two eyes of the user deviating from the horizontal direction, the playing angle range in the horizontal direction is determined by the maximum range of the two eyes of the user moving towards the two sides, namely the maximum range of the face of the user approaching to or departing from the shoulders of the two sides, and the range can be-90 degrees to 90 degrees, for example.
2. And if the change direction of the eye position of the user is moving in the vertical direction, adjusting the panoramic video playing angle of the multimedia resource according to the moving distance of the central point connecting line in the vertical direction.
Specifically, the position of the connecting line of the iris center points of the two eyes of the user in the initial state in the vertical direction is recorded as 0cm, and the playing angle of the panoramic video of the multimedia resource in the vertical direction is recorded as 0 °. In the process of playing the multimedia resource, if the connecting line of the central points of the irises of the two eyes of the user moves up by 1cm in the vertical direction in the image detected by the camera equipment, correspondingly adjusting the playing angle of the panoramic video of the multimedia resource in the vertical direction to be 1 degree; if the connecting line of the iris center points of the two eyes of the user moves downwards by 1cm in the vertical direction, the playing angle of the panoramic video of the multimedia resource in the vertical direction is correspondingly adjusted to-1 °, the playing angle of the panoramic video of the multimedia resource in the vertical direction is kept to be the same as the moving amplitude of the connecting line of the iris center points of the two eyes of the user in the vertical direction, the playing angle range in the vertical direction is determined by the maximum range of the two eyes of the user moving in the vertical direction, namely the maximum range of the user squatting, standing or standing on tiptoe, and the range can be-90 degrees, for example.
3. And if the eye position change direction of the user is close to or far away from the film watching equipment, adjusting the playing size of the multimedia resource according to the proportion change of the central point connecting line in the face image of the user.
Specifically, the ratio of the connection line of the iris central points of the two eyes of the user in the face image of the user in the initial state is recorded, and the playing size of the panoramic video of the multimedia resource on the film watching equipment in the initial state is recorded. In the process of playing the multimedia resources, if the proportion of the connecting line of the iris center points of the two eyes of the user in the face image of the user is doubled in the image detected by the camera equipment, correspondingly doubling the playing size of the panoramic video of the multimedia resources on the video watching equipment; if the proportion of the connecting line of the iris center points of the two eyes of the user in the face image of the user is reduced to one half of that in the initial state, correspondingly reducing the playing size of the panoramic video of the multimedia resources on the viewing equipment to one half of that in the initial state; the playing size of the panoramic video of the multimedia resource on the film watching equipment is kept to be the same as the change amplitude of the connecting line of the central points of the irises of the two eyes of the user in the face image of the user.
It should be noted that, the change of the position of the eyes is limited by the positions of the face and the shoulders, and the playing angle of the panoramic video cannot be adjusted by the positions of the eyes to 360 degrees, when the playing angle of the panoramic video exceeding the change range of the positions of the eyes needs to be adjusted, the playing angle can be specified through a touch screen or a mouse, meanwhile, in order to facilitate the matching of the touch screen or the mouse to specify the playing angle, when the playing angle is specified through the touch screen or the mouse each time, the playing angle and the size of the panoramic video are updated to the initial state, at this time, the playing angle is specified through the touch screen or the mouse, and the user can control the playing angle and/or the size of the panoramic video according to the face posture on the basis of specifying the playing angle.
On the basis of the adjustment method in the panoramic video playing provided in the foregoing embodiment, in order to reduce the situation of erroneous judgment and reduce the workload of data processing, in this embodiment, some collected images may be filtered and removed, for example, after the collecting the facial image of the user in S101, the method further includes:
if the face image in the multimedia resource playing process does not meet the preset conditions, the face image is taken as an invalid image to be removed, wherein the preset conditions comprise: and detecting that the position change of the front face and the eyes is larger than a preset range.
That is to say, the images that are not collected with the front face, or the collected images that include the front face in the area that is smaller than the preset proportion (that is, only a small part of the front face is collected), or the eye position change is too small (for example, smaller than the preset range), etc. may be removed, and no processing is performed.
Specifically, the preset range of the eye position change may be, for example: the deviation angle of the connecting line of the central points of the irises of the two eyes and the horizontal direction can be detected by the film watching equipment, the moving distance of the connecting line of the central points of the irises of the two eyes in the vertical direction can be detected by the film watching equipment, or the proportion change of the connecting line of the central points of the irises of the two eyes in the image can be detected by the film watching equipment. Only when the face image is detected to contain the front face of the user and the change of the eye position of the user is larger than the preset range, the film watching equipment can adjust the playing angle and/or size of the panoramic video of the multimedia resource according to the change of the eye position.
Fig. 3 is a schematic flowchart of an adjusting method in panoramic video playing according to another embodiment of the present application, where as shown in fig. 3, the adjusting method in panoramic video playing according to the foregoing embodiment further includes:
s301, when the multimedia resource playing is started, detecting the change range of the eye position in the current face image and the eye position in the initial state.
Specifically, when the multimedia resource playing is started, the eye position in the current face image is compared with the eye position in the initial state, and the variation range of the eye position in the current face image compared with the eye position in the initial state is obtained.
S302, judging whether the variation range is larger than a preset threshold value.
If yes, go to step S303.
And S303, displaying a prompt message.
Specifically, the preset threshold may be, for example, a minimum change range in which the eye position in the current face image detected by the viewing device changes from the eye position in the initial state, and when it is detected that the change range in the eye position in the current face image from the eye position in the initial state is greater than the preset threshold, it is determined that the user may adjust the face posture or move the image capturing device, a prompt message is displayed on the viewing device to prompt the user to adjust the face posture to the same state as the initial state or not to move the image capturing device, and when the user adjusts the face posture to the same state as the initial state, the playing of the multimedia resource is started.
If not, go to step S304.
And S304, starting to play the panoramic video of the multimedia resource.
Specifically, if it is not detected that the change range of the eye position in the current face image from the eye position in the initial state is larger than a preset threshold value, the panoramic video starts to be played, and the face image detection is started synchronously.
Executing the flow shown in fig. 3 can make the facial posture of the user the same as the initial state when the multimedia resource playing is started, thereby avoiding that the user has large deviation before the video playing and bad viewing effect, and adding an adjusting step.
If it is detected that the face image of the user is not directed to the viewing apparatus, the face image is invalid data.
Specifically, in an actual application scene, if the face of the user is not directly facing the viewing device, that is, the connecting line of the center points of the irises of the two eyes of the user is not in the same horizontal plane with the body of the user, but the camera device can detect the complete eyes, the image is removed as an invalid image and cannot be used, and when the face of the user is readjusted to be directly facing the viewing device, the change of the positions of the eyes is continuously detected according to the face image. If the distance between the center point of the left iris of the user and the left eye angle is more than 5% different from the distance between the center point of the left iris of the user and the left eye angle when the user faces the viewing device, or the distance between the center point of the right iris of the user and the right eye angle is more than 5% different from the distance between the center point of the right iris of the user and the right eye angle when the user faces the viewing device, the connection line of the center points of the irises of the two eyes of the user and the body of the user is not in the same horizontal plane, data is not collected, the eye angle refers to an inner eye angle or an outer eye angle, and no limitation is made on the connection line, so long as the connection line is consistent during detection, for example, the distance between the center point of the left iris of the user and the left inner eye angle when the user faces the viewing device is detected.
For example, if the user makes a head-twisting motion, that is, the user makes a motion that the eyes look at the shoulders on both sides, it is determined that the connecting line of the center points of the irises of the two eyes of the user is not on the same horizontal plane with the body of the user, and the image is rejected as an invalid image and is not adopted.
If the user's eye position is not detected, the face image is invalid data.
Specifically, in an actual application scene, if the face displacement of the user is too large, and the camera device cannot detect a complete eye, the image is removed as invalid data, and at this time, the user is considered not to be in a watching state, and the panoramic video played on the viewing device is adjusted to an initial playing angle, where the initial playing angle is an angle at which the panoramic video is displayed on the viewing device when the panoramic video is just played.
The embodiment of the present application further provides an adjusting apparatus in panoramic video playing, fig. 4 is a schematic structural diagram of the adjusting apparatus in panoramic video playing provided in the embodiment of the present application, the apparatus may be integrated in a terminal device having a panoramic video playing function, and referring to fig. 4, the adjusting apparatus in panoramic video playing includes:
an acquiring module 401, configured to acquire a user face image, where the user face image includes: a face image of the user in an initial state, and a face image in the process of playing the multimedia resource.
A detection module 402, configured to detect a facial image of an initial state of a user and mark an eye position of the initial state;
a calculating module 403, configured to calculate and acquire a change of an eye position of the user according to the eye position in the initial state and the eye position in the face image during the playing of the multimedia resource.
And the control module 404 is configured to adjust a playing angle and/or size of the panoramic video of the multimedia resource according to the change of the eye position of the user.
Optionally, the eye positions comprise: the positions of the central points of the irises of the two eyes of the user and the positions of the canthus of the two eyes of the user.
Optionally, the calculating module 403 is specifically configured to calculate, according to the eye position in the initial state and the eye position in the current face image, an eye position change direction of the user.
Optionally, the calculating module 403 is specifically configured to calculate a change of a connecting line between center points of irises of two eyes of the user according to the eye position in the initial state and the eye position in the current face image. If the change of the connecting line of the central points of the irises of the two eyes of the user is deviated from the horizontal direction, the change direction of the positions of the eyes of the user moves towards two sides. If the change of the connecting line of the iris center points of the two eyes of the user is moving in the vertical direction, the change direction of the positions of the eyes of the user is moving in the vertical direction. If the ratio of the connecting line of the center points of the irises of the two eyes of the user in the current facial image to the connecting line of the center points of the irises of the two eyes of the user in the facial image in the initial state of the user is changed in the facial image of the user, the change direction of the positions of the eyes of the user is close to or far away from the film watching equipment.
Optionally, the control module 404 is specifically configured to adjust a panoramic video playing angle of the multimedia resource according to an angle at which the iris center line deviates from the horizontal direction if the eye position of the user changes in direction to move to both sides. And if the change direction of the eye position of the user is moving in the vertical direction, adjusting the panoramic video playing angle of the multimedia resource according to the moving distance of the connecting line of the central points of the irises in the vertical direction. And if the eye position change direction of the user is close to or far away from the film watching equipment, adjusting the playing size of the multimedia resource according to the proportion change of the iris center point connecting line on the face image of the user.
And the judging module is used for judging whether the facial image in the multimedia resource playing process meets the preset condition.
The detecting module 402 is specifically configured to detect a variation range between an eye position in a current face image and an eye position in an initial state when the multimedia resource is started to be played.
Fig. 4 is a schematic structural diagram of a server according to an embodiment of the present application, and referring to fig. 4, an embodiment of the present application further provides an electronic device, where the electronic device may be a terminal device with a panoramic video playing function, and the electronic device includes: the device comprises a processor, a storage medium and a bus, wherein the storage medium stores program instructions executable by the processor, when the electronic device runs, the processor and the storage medium communicate through the bus, and the processor executes the program instructions to execute the steps of the adjustment method in the panoramic video playing provided by the above embodiment.
The embodiment of the present application further provides a computer-readable storage medium, where a computer program is stored on the storage medium, and when the computer program is executed by a processor, the method for adjusting in playing a panoramic video provided by the above embodiment is executed.
To sum up, the present application provides an adjustment method, an apparatus, an electronic device, and a storage medium in panoramic video playing, where the adjustment method in panoramic video playing includes: acquiring a face image of a user, wherein the face image comprises: the face image of the user in the initial state and the face image in the multimedia resource playing process; detecting a face image of an initial state of a user and marking an eye position of the initial state; calculating and acquiring the eye position change of the user according to the eye position in the initial state and the eye position in the face image in the multimedia resource playing process; and adjusting the playing angle and/or size of the panoramic video of the multimedia resource according to the change of the eye position of the user. The application provides a method for adjusting panoramic video playing angle and/or size by detecting eye position change, compare in prior art through rocking the sight shadow equipment and adjust panoramic video playing angle and/or size, the method that this application embodiment provided is more convenient, the user only need the face remove can, need not to adjust panoramic video playing angle and/or size with hand or mouse when seeing the shadow, under the condition of not upgrading hardware equipment, can be more convenient, adjust panoramic video's playing angle more smoothly, user watching experience has been promoted, and, can discern invalid data and reject invalid data, user watching experience has further been promoted.
In the several embodiments provided in the present application, it should be understood that the disclosed apparatus and method may be implemented in other ways. For example, the above-described apparatus embodiments are merely illustrative, and for example, the division of the units is only one logical division, and other divisions may be realized in practice, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit may be implemented in the form of hardware, or in the form of hardware plus a software functional unit.
The integrated unit implemented in the form of a software functional unit may be stored in a computer-readable storage medium. The software functional unit is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) or a processor (in english: processor) to execute some steps of the methods described in the embodiments of the present application. And the aforementioned storage medium includes: a U disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.
The above description is only for the specific embodiments of the present application, but the scope of the present application is not limited thereto, and any person skilled in the art can easily conceive of the changes or substitutions within the technical scope of the present application, and shall be covered by the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims (10)

1. An adjustment method in panoramic video playing, comprising:
capturing a user face image, wherein the user face image comprises: the face image of the user in the initial state and the face image in the multimedia resource playing process;
detecting the face image of the user in the initial state, and marking the eye position of the user in the initial state;
calculating and acquiring the eye position change of the user according to the eye position of the initial state and the eye position in the face image in the multimedia resource playing process;
and adjusting the playing angle and/or size of the panoramic video of the multimedia resource according to the change of the eye position of the user.
2. The method of claim 1, wherein the eye position comprises: the positions of the central points of the irises of the two eyes of the user and the positions of the canthus of the two eyes of the user.
3. The method of claim 1, wherein calculating the eye position change of the user according to the eye position of the initial state and the eye position in the face image during the playing of the multimedia resource comprises:
acquiring a current face image in the multimedia resource playing process;
and calculating and acquiring the eye position change direction of the user according to the eye position of the initial state and the eye position in the current face image.
4. The method of claim 3, wherein calculating the eye position change direction of the capturing user based on the eye positions of the initial state and the eye positions in the current face image comprises:
calculating the change of the connecting line of the iris center points of the two eyes of the user according to the eye position of the initial state and the eye position in the current face image;
if the change of the connecting line of the iris center points of the two eyes of the user is deviated from the horizontal direction, the change direction of the positions of the eyes of the user is moved to two sides;
if the change of the connecting line of the iris center points of the two eyes of the user is that the connecting line moves in the vertical direction, the change direction of the positions of the eyes of the user is that the connecting line moves in the vertical direction;
and if the ratio of the connecting line of the iris central points of the two eyes of the user in the current facial image to the connecting line of the iris central points of the two eyes of the user in the facial image in the initial state of the user in the facial image of the user is changed, the change direction of the eye positions of the user is close to or far away from the film watching equipment.
5. The method of claim 4, wherein the adjusting the panoramic video playback angle and/or size of the multimedia asset according to the change in the eye position of the user comprises:
if the eye position of the user changes towards the two sides, adjusting the panoramic video playing angle of the multimedia resource according to the angle of the central point connecting line deviating from the horizontal direction;
if the eye position of the user changes in a vertical direction, adjusting the panoramic video playing angle of the multimedia resource according to the moving distance of the central point connecting line in the vertical direction;
and if the eye position change direction of the user is close to or far away from the film watching equipment, adjusting the playing size of the multimedia resource according to the proportion change of the central point connecting line on the face image of the user.
6. The method of any one of claims 1-5, wherein after the capturing the image of the user's face, further comprising:
if the face image in the multimedia resource playing process does not meet the preset conditions, the face image is taken as an invalid image to be removed, wherein the preset conditions comprise: and detecting that the position change of the front face and the eyes is larger than a preset range.
7. The method of any one of claims 1-5, further comprising:
when the multimedia resource playing is started, detecting the change range of the eye position in the current face image and the eye position in the initial state;
if the variation range is larger than a preset threshold value, displaying a prompt message;
if the facial image of the user is detected not to be directly facing the film watching device, the facial image is invalid data;
if the eye position of the user is not detected, the face image is invalid data.
8. An adjusting device in panoramic video playing, comprising:
an acquisition module for acquiring a user facial image, wherein the user facial image comprises: the method comprises the steps of obtaining a face image in an initial state of a user and a face image in a multimedia resource playing process;
the detection module is used for detecting the facial image in the initial state of the user and marking the eye position in the initial state;
the calculation module is used for calculating and acquiring the eye position change of the user according to the eye position in the initial state and the eye position in the face image in the multimedia resource playing process;
and the control module is used for adjusting the panoramic video playing angle and/or size of the multimedia resource according to the change of the eye position of the user.
9. An electronic device, comprising: a processor, a storage medium and a bus, wherein the storage medium stores program instructions executable by the processor, when the electronic device runs, the processor and the storage medium communicate through the bus, and the processor executes the program instructions to execute the steps of the adjusting method in the panoramic video playing according to any one of claims 1 to 7.
10. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, performs the method of adjusting in panoramic video playback according to any one of claims 1 to 7.
CN202210931139.2A 2022-08-04 2022-08-04 Adjustment method and device in panoramic video playing, electronic equipment and storage medium Pending CN115294631A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210931139.2A CN115294631A (en) 2022-08-04 2022-08-04 Adjustment method and device in panoramic video playing, electronic equipment and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210931139.2A CN115294631A (en) 2022-08-04 2022-08-04 Adjustment method and device in panoramic video playing, electronic equipment and storage medium

Publications (1)

Publication Number Publication Date
CN115294631A true CN115294631A (en) 2022-11-04

Family

ID=83825694

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210931139.2A Pending CN115294631A (en) 2022-08-04 2022-08-04 Adjustment method and device in panoramic video playing, electronic equipment and storage medium

Country Status (1)

Country Link
CN (1) CN115294631A (en)

Similar Documents

Publication Publication Date Title
CN108734736B (en) Camera posture tracking method, device, equipment and storage medium
CN108615248B (en) Method, device and equipment for relocating camera attitude tracking process and storage medium
CN110555882B (en) Interface display method, device and storage medium
CN110599549B (en) Interface display method, device and storage medium
CN108710525B (en) Map display method, device, equipment and storage medium in virtual scene
US9651782B2 (en) Wearable tracking device
EP2950180B1 (en) Method for determining screen display mode and terminal device
WO2017113668A1 (en) Method and device for controlling terminal according to eye movement
CN110148178B (en) Camera positioning method, device, terminal and storage medium
CN108712603B (en) Image processing method and mobile terminal
EP2991027B1 (en) Image processing program, image processing method and information terminal
KR102461232B1 (en) Image processing method and apparatus, electronic device, and storage medium
CN108090463B (en) Object control method, device, storage medium and computer equipment
CN110706283B (en) Calibration method and device for sight tracking, mobile terminal and storage medium
WO2013178151A1 (en) Screen turnover method and device, and mobile terminal
CN111629242B (en) Image rendering method, device, system, equipment and storage medium
JP2023017920A (en) Image processing device
CN111857910A (en) Information display method and device and electronic equipment
CN114741559A (en) Method, apparatus and storage medium for determining video cover
CN104866809B (en) Picture playing method and device
CN112702533A (en) Sight line correction method and sight line correction device
CN113342157A (en) Eyeball tracking processing method and related device
CN115294631A (en) Adjustment method and device in panoramic video playing, electronic equipment and storage medium
CN106921826B (en) Photographing mode processing method and device
CN111766947A (en) Display method, display device, wearable device and medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination