WO2023246302A9 - 字幕的显示方法、装置、设备及介质 - Google Patents

字幕的显示方法、装置、设备及介质 Download PDF

Info

Publication number
WO2023246302A9
WO2023246302A9 PCT/CN2023/091203 CN2023091203W WO2023246302A9 WO 2023246302 A9 WO2023246302 A9 WO 2023246302A9 CN 2023091203 W CN2023091203 W CN 2023091203W WO 2023246302 A9 WO2023246302 A9 WO 2023246302A9
Authority
WO
WIPO (PCT)
Prior art keywords
subtitle
layer
display
preset
virtual reality
Prior art date
Application number
PCT/CN2023/091203
Other languages
English (en)
French (fr)
Other versions
WO2023246302A1 (zh
Inventor
庞娜
杨毅平
方迟
Original Assignee
北京字跳网络技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 北京字跳网络技术有限公司 filed Critical 北京字跳网络技术有限公司
Publication of WO2023246302A1 publication Critical patent/WO2023246302A1/zh
Publication of WO2023246302A9 publication Critical patent/WO2023246302A9/zh

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/431—Generation of visual interfaces for content selection or interaction; Content or additional data rendering
    • H04N21/4312—Generation of visual interfaces for content selection or interaction; Content or additional data rendering involving specific graphical features, e.g. screen layout, special fonts or colors, blinking icons, highlights or animations
    • H04N21/4316—Generation of visual interfaces for content selection or interaction; Content or additional data rendering involving specific graphical features, e.g. screen layout, special fonts or colors, blinking icons, highlights or animations for displaying supplemental content in a region of the screen, e.g. an advertisement in a separate window
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/4302—Content synchronisation processes, e.g. decoder synchronisation
    • H04N21/4307—Synchronising the rendering of multiple content streams or additional data on devices, e.g. synchronisation of audio on a mobile phone with the video output on the TV screen
    • H04N21/43074—Synchronising the rendering of multiple content streams or additional data on devices, e.g. synchronisation of audio on a mobile phone with the video output on the TV screen of additional data with content streams on the same device, e.g. of EPG data or interactive icon with a TV program
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/431—Generation of visual interfaces for content selection or interaction; Content or additional data rendering
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/47—End-user applications
    • H04N21/488—Data services, e.g. news ticker
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/47—End-user applications
    • H04N21/488—Data services, e.g. news ticker
    • H04N21/4884—Data services, e.g. news ticker for displaying subtitles

Definitions

  • the present disclosure relates to the field of virtual reality technology, and in particular to a method, device, equipment and medium for displaying subtitles.
  • VR virtual reality
  • Its basic implementation method is to use computers to simulate virtual environments to give people a sense of immersion.
  • With the popularization of VR applications it has become a common scenario to display subtitles in virtual reality space in scenarios such as video display.
  • a translation function is performed on the audio stream corresponding to the video source based on tools such as a web player to obtain the translated subtitle content, and the translated subtitle content is displayed at a fixed position on the corresponding video frame.
  • tools such as a web player to obtain the translated subtitle content
  • the translated subtitle content is displayed at a fixed position on the corresponding video frame.
  • the display orientation of the video frame will change, thereby causing the subtitle content that is fixed relative to the video frame to appear visually distorted, affecting the viewing effect.
  • the present disclosure provides a subtitle display method, device, equipment and medium, which follow the user's line of sight, determine the position of the subtitle content rendered in the virtual reality space, improve the user's viewing experience, and render the subtitle content at the corresponding position to be independent of the virtual reality video frame.
  • the subtitle layer is used as a carrier for displaying subtitle content, which avoids the distortion of subtitle content display and improves the display effect of subtitle content.
  • the embodiment of the present disclosure provides a method for displaying subtitles, the method comprising: acquiring subtitle content corresponding to a currently played virtual reality video frame; determining a target spatial position in a virtual reality panoramic space according to a user's current line of sight; rendering a subtitle layer according to the subtitle content at the target spatial position, and synchronously rendering the subtitle content in the subtitle layer.
  • the embodiment of the present disclosure also provides a subtitle display device, which includes: an acquisition module, used to acquire subtitle content corresponding to a currently played virtual reality video frame; a position determination module, used to determine a target spatial position in a virtual reality panoramic space according to a user's current line of sight; and a rendering and display module, used to render a subtitle layer according to the subtitle content at the target spatial position, and synchronously render the subtitle content in the subtitle layer.
  • a subtitle display device which includes: an acquisition module, used to acquire subtitle content corresponding to a currently played virtual reality video frame; a position determination module, used to determine a target spatial position in a virtual reality panoramic space according to a user's current line of sight; and a rendering and display module, used to render a subtitle layer according to the subtitle content at the target spatial position, and synchronously render the subtitle content in the subtitle layer.
  • An embodiment of the present disclosure further provides an electronic device, comprising: a processor; a memory for storing instructions executable by the processor; the processor is configured to read the executable instructions from the memory and execute the instructions to implement a subtitle display method as provided in an embodiment of the present disclosure.
  • the embodiment of the present disclosure further provides a computer-readable storage medium, wherein the storage medium stores a computer program, and the computer program is used to execute the subtitle display method provided by the embodiment of the present disclosure.
  • the display scheme of subtitles obtained by the embodiment of the present disclosure obtains the subtitle content corresponding to the currently played virtual reality video frame, determines the target space position in the virtual reality panoramic space according to the user's current line of sight, renders the subtitle layer according to the subtitle content at the target space position, and synchronously renders the subtitle content in the subtitle layer.
  • the position of the subtitle content rendered in the virtual reality space is determined following the user's line of sight, and on the basis of realizing the display of the subtitle content, the user's viewing experience is improved, and the subtitle content is rendered at the corresponding position, with the subtitle layer independent of the virtual reality video frame as the display subtitle.
  • the carrier of the content avoids the distortion of the subtitle content display and improves the display effect of the subtitle content.
  • FIG1 is a schematic flow chart of a subtitle display method provided by an embodiment of the present disclosure.
  • FIG2 is a schematic diagram of a subtitle display scene provided by an embodiment of the present disclosure.
  • FIG3 is a schematic diagram of another subtitle display scenario provided by an embodiment of the present disclosure.
  • FIG4 is a schematic diagram of another subtitle display scenario provided by an embodiment of the present disclosure.
  • FIG5 is a schematic diagram of another subtitle display scenario provided by an embodiment of the present disclosure.
  • FIG6 is a schematic diagram of another subtitle display scenario provided by an embodiment of the present disclosure.
  • FIG7 is a flow chart of another method for displaying subtitles provided by an embodiment of the present disclosure.
  • FIG8 is a schematic diagram of another subtitle display scenario provided by an embodiment of the present disclosure.
  • FIG9 is a schematic flow chart of another subtitle display method provided by an embodiment of the present disclosure.
  • FIG10 is a schematic diagram of another subtitle display scenario provided by an embodiment of the present disclosure.
  • FIG11 is a schematic diagram of another subtitle display scene provided by an embodiment of the present disclosure.
  • FIG12 is a schematic diagram of another subtitle display scenario provided by an embodiment of the present disclosure.
  • FIG13 is a flow chart of another method for displaying subtitles provided by an embodiment of the present disclosure.
  • FIG14A is a schematic diagram of another subtitle display scenario provided by an embodiment of the present disclosure.
  • FIG14B is a schematic diagram of another subtitle display scenario provided by an embodiment of the present disclosure.
  • FIG15 is a schematic diagram of another subtitle display scenario provided by an embodiment of the present disclosure.
  • FIG16 is a schematic structural diagram of a subtitle display device provided by an embodiment of the present disclosure.
  • FIG. 17 is a schematic diagram of the structure of an electronic device provided in an embodiment of the present disclosure.
  • the embodiments of the present disclosure provide a subtitle display method, in which the method no longer relies on a fixed mode of subtitle content display, but instead releases the fixed position relationship between the subtitle content and the picture content, and adaptively adjusts the display position of the subtitle content following the user's line of sight, thereby avoiding visual distortion of the subtitle content and improving the user's visual viewing experience.
  • FIG1 is a flow chart of a method for displaying subtitles provided by an embodiment of the present disclosure, which method can be executed by a subtitle display device, wherein the device can be implemented by software and/or hardware and can generally be integrated in an electronic device. As shown in FIG1 , the method includes:
  • Step 101 obtaining subtitle content corresponding to the currently playing virtual reality video frame.
  • corresponding subtitle content can be obtained based on different methods.
  • an audio stream corresponding to a currently played virtual reality video frame is determined, and speech recognition processing is performed on the audio stream to obtain subtitle content. For example, timestamp information corresponding to each audio stream is obtained, and subtitle content corresponding to the virtual reality video frame is matched based on the timestamp information and the timestamp information of the virtual video frame.
  • the correspondence between each virtual reality video frame and the corresponding subtitle content can be pre-identified and constructed, and the correspondence can be stored in a corresponding preset database, so that the subtitle content corresponding to the currently playing virtual reality video frame can be obtained by querying the preset database.
  • Step 102 determining the target space position in the virtual reality panoramic space according to the user's current sight direction.
  • the virtual video frame is no longer used as the basis for determining the display position of the subtitle content.
  • the target space position is determined in the virtual reality panoramic space according to the user's current line of sight.
  • the target spatial position is made consistent with the user's current line of sight direction.
  • the target spatial position is located in the user's current line of sight direction.
  • the method of determining the target space position in the virtual reality panoramic space according to the user's current line of sight direction can be referred to in subsequent embodiments and will not be repeated here.
  • Step 103 Rendering a subtitle layer at the target spatial position according to the subtitle content, and synchronously rendering the subtitle content in the subtitle layer.
  • the subtitle layer is rendered at a certain position in the space, and the subtitle content is rendered synchronously in the subtitle layer. For example, referring to FIG. 2, if the user's line of sight changes horizontally, the subtitle layer follows the change in the line of sight and displays the subtitle layer at a spatial position in the line of sight, wherein the subtitle content is rendered synchronously in the subtitle layer. Since the subtitle layer is in the line of sight of the user, it is ensured that the subtitle content can be presented to the user without distortion, thereby ensuring the visual display effect.
  • the subtitle content is rendered separately with a subtitle layer independent of the virtual reality video frame, which avoids display distortion of the subtitle content.
  • the subtitle layer is rendered according to the subtitle content, which has a stronger visual sense of technology and improves the user's visual viewing experience.
  • the subtitle layer in order to further enhance the intelligence of the subtitle layer, background display processing can be performed on the subtitle layer, and background special effects can be added to the subtitle layer to enhance the intelligence of the display.
  • the subtitle layer in order to avoid blocking the virtual reality video frame, can also be processed as a layer with higher transparency.
  • the subtitle layer is a backlight sublayer and a quantum dot matrix sublayer, wherein the quantum color of the quantum dot matrix sublayer can be any color such as blue, and the quantum dot matrix can be displayed in a gradient form or with uniform brightness.
  • the subtitle layer formed by the superposition of the backlight sublayer and the quantum dot matrix sublayer can visually give the user a stronger sense of technology.
  • light animations can be rendered in the subtitle layer, such as aurora animations, white light flow animations, etc., which will not be described one by one here. By adding light animations, the sense of technology displayed is enhanced.
  • the subtitle layer in order to further improve the visual display effect, can also be dynamically rendered according to the subtitle content.
  • the display quantity of subtitle content is obtained, and a subtitle layer matching the display quantity is rendered, thereby achieving an effect of dynamically displaying the subtitle layer along with the change of the subtitle display quantity.
  • the number of subtitle contents displayed is obtained, and the real-time width and height of the subtitles are determined according to the number of subtitle contents displayed, the preset unit subtitle width, and the preset unit subtitle height.
  • the number of subtitle contents displayed refers to the number of all subtitle contents displayed in the current virtual reality video frame.
  • the preset unit subtitle width refers to the preset width of each subtitle content
  • the preset unit subtitle height refers to the preset height of each subtitle content.
  • the subtitle implementation height and the subtitle real-time width are determined so as to dynamically render the subtitle layer based on the subtitle implementation height and the subtitle real-time width.
  • a preset database may be queried according to the display quantity, the preset unit subtitle width, and the preset unit subtitle height to obtain the real-time subtitle width and the real-time subtitle height.
  • the display quantity is greater than a preset subtitle quantity threshold in the width direction, wherein the preset subtitle quantity threshold can be understood as a predetermined maximum number of subtitle contents displayed in each line. If it is greater than the preset subtitle quantity threshold, it indicates that the subtitle content needs to be wrapped. Therefore, the real-time width of the subtitles is determined based on the preset subtitle quantity threshold and the unit subtitle width. For example, the product of the preset subtitle quantity threshold and the unit subtitle width is calculated to determine the real-time width of the subtitles.
  • the number of subtitle rows can be determined based on a preset subtitle number threshold and the display number. For example, the ratio of the display number to the preset subtitle number threshold is calculated, and the smallest integer greater than or equal to the ratio is determined as the number of subtitle rows.
  • the real-time height of the subtitles can be determined based on the number of subtitle rows and the unit subtitle height. For example, the product of the number of subtitle rows and the unit subtitle height is calculated to determine the real-time height of the subtitles.
  • the subtitle real-time width is determined based on the unit subtitle width and the display quantity, that is, the product of the unit subtitle width and the display quantity is calculated to determine the subtitle real-time width, and the unit subtitle height is determined as the subtitle real-time height.
  • the real-time subtitle layer width matching the subtitle content is rendered according to the layer width of the unit subtitles and the real-time width of the subtitles, wherein the layer width of the unit subtitles is the width of each unit subtitle in the width direction.
  • the unit width of the subtitle layer is increased.
  • the real-time subtitle layer height is the unit height that the subtitle layer increases when each unit subtitle increases in height upward. That is, the subtitle layer can expand and change horizontally and vertically along with the subtitle content.
  • the subtitle layer width is determined according to the real-time width of the subtitles displayed in real time (the subtitle content in the figure is "the *th line of subtitle content is displayed here", and the subtitle layer is a gradient brightness). For example, each time one or more subtitle contents are newly displayed in real time, the corresponding subtitle content is displayed according to the unit subtitle width, and the subtitle layer is expanded according to the layer width of the unit subtitle.
  • the unit subtitle width is smaller than the layer width of the unit subtitle to ensure that the subtitle content can be visually wrapped by the subtitle layer. Thus, visually, it is achieved that in the width direction, the subtitle background is displayed according to the display content of the subtitle.
  • the subtitle content displayed in real time is greater than a preset subtitle quantity threshold
  • the subtitle content is displayed in a line break
  • the real-time subtitle layer height that matches the subtitle content is rendered according to the height change of the real-time height of the subtitle.
  • the width of the subtitle input in real time is greater than the preset maximum line width of the subtitle, that is, it needs to be displayed in a line break
  • the corresponding subtitle content is displayed according to the unit subtitle height
  • the display height of the subtitle layer is expanded according to the layer height of the unit subtitle.
  • the unit subtitle height is smaller than the layer height of the unit subtitle to ensure that the subtitle content can be visually wrapped by the subtitle layer.
  • the subtitle layer is displayed in the height direction according to the display content of the subtitle, forming a visual scrolling display effect.
  • the subtitle layer height gradually expands downward, forming a scrolling display effect.
  • the layer heights of the unit subtitles corresponding to different lines can be the same or different.
  • the first line of unit subtitles can be specified as The layer height of the first line of unit subtitles is 72, the layer height of the second line of unit subtitles is 100, the layer height of the third line of unit subtitles is 120, the layer height of the fourth line of unit subtitles is 132, the layer height of the fifth line of unit subtitles is 156, the layer height of the sixth line of unit subtitles is 180, and so on.
  • the subtitle content can also be displayed by reducing the preset unit subtitle width and/or the preset unit subtitle height.
  • the subtitle content of the preset line threshold is displayed first, and the remaining subtitle content is scrolled.
  • the preset line threshold is 6 lines, but the calculated number of subtitle lines is 7, the first 6 lines of subtitle content can be displayed first. After the preset display time, the subtitle content of the first line is no longer displayed, and the subtitle content of lines 2-6 is scrolled up and the original subtitle content of line 7 is displayed on line 6.
  • the preset display time the subtitle content of lines 1-6 is no longer displayed, and only the original subtitle content of line 7 is displayed, etc.
  • control subtitle layer when the above-mentioned control subtitle layer is dynamically rendered according to the subtitle content, when the rendered last line of subtitle content does not reach the preset subtitle quantity threshold, an entire row of layers can be rendered according to the preset subtitle quantity threshold as shown in the above Figure 5, or the subtitle layer width of the last row can be determined according to the number of subtitle contents in the last row and the preset unit subtitle width as shown in Figure 6, so as to further improve the intelligence of the subtitle layer following rendering.
  • the subtitle display method of the disclosed embodiment obtains the subtitle content corresponding to the currently played virtual reality video frame, determines the target space position in the virtual reality panoramic space according to the user's current line of sight, renders the subtitle layer according to the subtitle content at the target space position, and synchronously renders the subtitle content in the subtitle layer.
  • the position of the subtitle content rendered in the virtual reality space is determined following the user's line of sight, and on the basis of realizing the display of the subtitle content, the user's viewing experience is improved, and the subtitle content is rendered at the corresponding position, using the subtitle layer independent of the virtual reality video frame as the carrier for displaying the subtitle content, avoiding the distortion of the subtitle content display and improving the display effect of the subtitle content.
  • the method of determining the target space position in the virtual reality panoramic space is different according to the user's current line of sight direction.
  • the examples are as follows:
  • determining the target space position in the virtual reality panoramic space according to the current sight line direction of the user includes:
  • Step 701 determine the center point position of the virtual reality panoramic space, and obtain a preset radius distance.
  • the center point position of the virtual reality panoramic space is determined, and the center point position is located at the center of the virtual reality panoramic space.
  • the center point position is related to the shape of the virtual reality panoramic space.
  • a preset radius distance is obtained, wherein the preset radius distance can be set in advance according to the size of the virtual reality panoramic space.
  • the length from the preset radius distance to the center point position will not exceed the length from the center point position in the virtual reality panoramic space to the surface of the virtual reality panoramic space, thereby ensuring that the displayed subtitle layer is located in front of the virtual reality video frame, forming a barrier, and enhancing the sense of technology of the subtitle layer display.
  • Step 702 starting from the center point position, extending to a position with a preset radius distance according to the user's current line of sight as the target space position.
  • the position extending to the preset radius distance according to the user's current line of sight is used as the target space position.
  • the target space position is inside the virtual space, ensuring the display effect.
  • it can ensure the consistency of the target space position and the user's line of sight direction, so that the subtitle content displayed at the target space position is consistent with the user's line of sight direction, ensuring that the user can clearly view the subtitle content, thereby improving the viewing experience.
  • the virtual reality panoramic space is a “box-shaped” cube space
  • the preset radius distance is R1
  • the center point position of the virtual reality panoramic space is O1.
  • the position of the preset radius distance is directly extended to the user's current line of sight direction as the target space position.
  • determining the target space position in the virtual reality panoramic space according to the current sight line direction of the user includes:
  • Step 901 obtaining the historical spatial position corresponding to the subtitle content of the previous frame displayed in the virtual reality panoramic space.
  • the historical spatial position corresponding to the subtitle content of the previous frame displayed in the virtual reality panoramic space is obtained, and the historical spatial position is determined according to the user's line of sight under the previous frame of subtitles.
  • Step 902 Obtain the sight line change information between the user's current sight line direction and the sight line direction of the previous frame viewed.
  • the line of sight change information is used to indicate the change in the angle of the user's current line of sight relative to the line of sight when viewing the previous frame, etc.
  • the line of sight change information may include the change direction and change angle of the user's current line of sight relative to the line of sight when viewing the previous frame, etc.
  • a rotation angle of a camera in a virtual reality device worn by a user in a horizontal direction relative to a horizontal axis of a previous frame is obtained, and the rotation angle of the horizontal axis is determined as information on a change in the user's horizontal line of sight from a previous frame to a current frame.
  • the line of sight change information includes information on a change in the horizontal line of sight direction, it can be ensured that when the user rotates left and right to view, the target space position can follow the user's line of sight direction, further ensuring the consistency of the subtitle content displayed at the target space position with the user's line of sight direction.
  • Step 903 determining the target spatial position according to the sight line change information and the historical spatial position.
  • the target spatial position can be determined according to the sight line change information and the historical spatial position.
  • the historical spatial position can be moved according to the sight line change information to determine the target spatial position.
  • the target space position is determined only based on the line of sight change information between the user's current line of sight and the line of sight of the previous frame, without considering the current user's position in the virtual space and the user's position in the virtual space in the previous frame. No matter the current user's position in the virtual reality space is closer to the surface of the virtual space or further away from the surface of the virtual space relative to the previous frame, it does not affect the determination of the target space position.
  • the target space position is only related to the line of sight change information between the user's current line of sight and the line of sight of the previous frame. Therefore, visually, the user will not feel that there is a fixed display distance between the displayed subtitle content and the user's eyes, which further ensures the user's visual effect.
  • the virtual reality panoramic space is a “box-shaped” cubic space.
  • the historical spatial position corresponding to the subtitle content of the previous frame displayed in the virtual reality panoramic space is W1.
  • the line of sight change information between the user's current line of sight direction S2 and the line of sight direction S1 of the previous frame is obtained as shown in the figure.
  • the movement of the historical spatial position is controlled according to the line of sight change information to determine that the moved position W2 is the target spatial position.
  • a circle center position pre-set in a virtual reality panoramic space is obtained, wherein the circle center position is usually set at the center position of the virtual reality space as the circle center to ensure that when the subtitles are surrounded and followed as the circle center, the subtitle content can be displayed in the virtual reality space.
  • the initial position of the virtual reality device in the virtual reality panoramic space can be obtained, wherein the initial position is the center point position in the virtual reality panoramic space, and the virtual reality panoramic space is constructed using the initial position as the midpoint position in the virtual reality panoramic space, and then, the center point position in the virtual reality panoramic space is set as the center position of the circle.
  • the initial position of the virtual reality device in the virtual reality panoramic space is used as the center position of the circle
  • the center point position is continued to be set as the center position of the circle
  • the user's VR head-mounted display device is not continued as the center position of the circle in the virtual reality panoramic space.
  • the parameters unrelated to the line of sight such as the moving distance of the VR head-mounted display device, are separated from the display position of the subtitle content, thereby improving the visual display effect.
  • the result is to avoid the subtitle content being always associated with the moving distance of the VR head-mounted display device, etc., so that the subtitle content is always linked to the VR head-mounted display device, affecting the user's visual experience.
  • the horizontal axis rotation angle is used as the center rotation angle from the previous frame to the current frame, and the historical spatial position of the subtitle content of the previous frame relative to the center of the circle is determined. Then, the target spatial position is determined based on the center position, the historical spatial position, and the center rotation angle.
  • the center position pre-set in the virtual reality panoramic space is obtained, and the horizontal axis rotation angle is set as the center rotation angle of the rotation root node, and then, the historical spatial position of the subtitle content of the previous frame relative to the center of the circle is determined, and the subtitle content is rotated according to the center rotation angle, driving the subtitle content to rotate with the center position as the rotation center, and rotating the subtitle content from the historical spatial position to the target spatial position consistent with the user's line of sight, thereby ensuring the display effect of the subtitle content subsequently displayed at the target spatial position.
  • the neural network model can be trained in advance according to sample data, and the center position of the circle, the historical spatial position, and the rotation angle about the horizontal axis are input into the neural network model to obtain the target spatial position corresponding to the current line of sight in the virtual reality panoramic space.
  • a preset algorithm may be used to calculate the target space position corresponding to the current line of sight in the virtual reality panoramic space, wherein the preset algorithm may be as shown in the following formula (1), wherein in formula (1), x, y, z are the coordinates of the historical space position on the three coordinate axes, x1, y1, z1 are the coordinates of the target space position on the three coordinate axes, ⁇ is the rotation angle of the circle center, and Tx, Ty, Tz are the coordinates of the circle center position on the three coordinate axes.
  • the preset radius distance can be obtained, and the initial spatial position of the subtitle content of the initial frame relative to the center of the circle can be determined according to the center position of the circle, the radius distance, and the user's initial line of sight direction, and the initial spatial position is used as the rendering position of the first frame of subtitle content.
  • the subtitle display method of the embodiment of the present disclosure can flexibly determine the target spatial position of the subtitle content rendered in the virtual reality panoramic space according to the scene requirements, ensure that the target spatial position follows the user's current line of sight, realize the movement of the subtitle content following the user's current line of sight, and ensure the visual display effect of the subtitle content.
  • the subtitle layer is displayed as a layer independently of the virtual reality video frame, it is possible to block other displayed layers. For example, as shown in FIG12, if the user performs a display operation of the play panel layer while watching a video, the above-mentioned determination principle of the target space position will be used in the direction of the user's line of sight to determine the display position of the play panel layer and display it. As a result, the play panel layer may be blocked, and it is difficult for the user to perform playback control operations on the play panel.
  • the display status information of the other layers is identified, and the display status information of the subtitle layer is adjusted according to the display status information of the other layers to ensure that the user can clearly view the subtitle content and other layers such as the playback panel layer.
  • the display status information includes a displayed spatial position, including:
  • adjusting the display status information of the subtitle layer according to the display status information of other layers includes:
  • Step 1301 determining whether the reference space position and the target space position of other layers meet a preset occlusion condition.
  • the other layers may include any layers that follow the user's direction of display, such as the playback control panel layer, etc.
  • the other layers are closer to the virtual reality video than the subtitle layer.
  • Frame determine whether other layers and the subtitle layer meet the preset occlusion conditions, that is, determine whether the subtitle layer will occlude other layers.
  • the first center point position of other layers on the plane perpendicular to the line of sight is identified based on the reference space position
  • the second center point position of the subtitle layer on the plane perpendicular to the line of sight is identified based on the target space position
  • the plane perpendicular to the line of sight can be understood as the plane that the user is viewing directly.
  • the plane perpendicular to the line of sight can be understood as the XY plane.
  • the overlapping area of the subtitle layer and other layers is determined on a plane perpendicular to the line of sight based on the reference space position and the target space position.
  • the overlapping area can be determined based on the size of the overlapping area.
  • the overlapping area can be determined based on the number of pixels in the overlapping area, etc.
  • the ratio of the overlapping area to the layer area of the other layer is calculated to determine whether the area ratio is greater than a preset area ratio threshold. If it is greater than the preset area ratio threshold, it is determined that the preset occlusion condition is met.
  • the layer levels of other layers can also be determined, and it can be determined that the layer levels are higher than the preset level threshold, wherein the layer level can be obtained by identifying the layer types of the current other layers and querying the preset corresponding relationship.
  • Step 1302 If the occlusion condition is met, determine the target moving position of the subtitle layer and/or the target layer display size, wherein the subtitle layer corresponding to the target moving position and/or the target layer display size and other layers do not meet the occlusion condition.
  • the target moving position of the subtitle layer is determined, wherein the subtitle layer and other layers at the target moving position do not meet the occlusion condition.
  • the target moving position can be any position that is as close as possible to the user's line of sight but does not block other layers.
  • first height information of the upper edge line of other layers is determined, and the first height information can be understood as a Y-axis coordinate value, etc.
  • second height information of the lower edge line of the subtitle layer is determined based on the first height information, wherein the second height information is greater than or equal to the first height information, that is, the second height information can be slightly higher than the first height information or can be exactly on the first height information, and then, the target moving position is determined based on the second height information.
  • the first height information of the upper edge line of the other layers can be determined, and the subtitle layer can be translated upward, so that the subtitle layer moves from in front of the other layers to above the other layers, thereby avoiding occlusion of the other layers.
  • first width information of the left edge line of other layers is determined, and the first width information can be understood as an X-axis coordinate value, etc.
  • second width information of the right edge line of the subtitle layer is determined based on the first width information, wherein the second width information is less than or equal to the first width information, that is, the second width information can be slightly less than the first width information or can be exactly on the first width information, and then, the target moving position is determined based on the second width information.
  • the first width information of the left edge line of the other layer can be determined, and the subtitle layer is shifted to the left, so that the subtitle layer moves from the front of the other layers to the left of the other layers, thereby avoiding the occlusion of the other layers. Occlusion of other layers.
  • the display size of the subtitle layer is determined.
  • the reference display size of other layers can be determined according to the reference spatial position
  • the target layer display size of the subtitle layer can be determined according to the reference display size.
  • the product of the reference display size and a preset reduction ratio is calculated, wherein the preset reduction ratio is less than 1, and the target layer display size is determined according to the product value.
  • the size difference between the reference display size and the initial display size of the subtitle layer is calculated, and the scaling ratio is determined by querying a preset corresponding relationship according to the size difference, wherein the larger the size difference, the smaller the scaling ratio, and the initial display size is scaled according to the scaling ratio to obtain the target layer display size.
  • the target layer display size of the subtitle layer can be determined based on the reference display size of the other layers.
  • the subtitle layer of the target layer display size has a smaller occlusion area for the other layers, thereby not affecting the viewing of the other layers.
  • the subtitle layer is controlled to move to the target moving position and/or the target layer display size, wherein the subtitle content rendered on the subtitle layer also moves synchronously, and the user can not only view the subtitle content but also the content of other layers.
  • the subtitle layer will be moved to the target space position for display.
  • the above-mentioned display status information may also include a display time point, and the display time point of the subtitle layer is adjusted by the display time point of other layers to ensure that the user can view the display content of other layers.
  • adjusting the display time point of the subtitle layer can control the subtitle layer to be displayed intermittently according to a preset display frequency at the display time point of other layers. For example, if the display time point of other layers is a-b, other layers can be controlled to display for a duration of d every c time at the a-b time point, thereby ensuring that the user can view the display content of other layers.
  • the subtitle display method of the disclosed embodiment can change the display state of the subtitle layer after the subtitle layer may block other layers, so as to further ensure that the user can watch the content of other layers, thereby improving the flexibility of display.
  • the present disclosure also proposes a subtitle display device.
  • FIG16 is a schematic diagram of the structure of a subtitle display device provided by an embodiment of the present disclosure.
  • the device can be implemented by software and/or hardware and can generally be integrated in an electronic device to display subtitles.
  • the device includes: an acquisition module 1610, a position determination module 1620, and a rendering display module 1630, wherein:
  • the acquisition module 1610 is used to acquire the subtitle content corresponding to the currently played virtual reality video frame
  • the position determination module 1620 is used to determine the target space position in the virtual reality panoramic space according to the user's current sight line direction;
  • the rendering and display module 1630 is used to render a subtitle layer according to the subtitle content at a target spatial position, and synchronously render the subtitle content in the subtitle layer.
  • the present disclosure further proposes a computer program product, including a computer program/instruction, which implements the subtitle display method in the above embodiments when executed by a processor.
  • FIG. 17 is a schematic diagram of the structure of an electronic device provided in an embodiment of the present disclosure.
  • the electronic device 1700 in the embodiment of the present disclosure may include, but is not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc.
  • mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc.
  • PDAs personal digital assistants
  • PADs tablet computers
  • PMPs portable multimedia players
  • vehicle-mounted terminals such as vehicle-mounted navigation terminals
  • fixed terminals such as digital TVs, desktop computers, etc.
  • the electronic device shown in FIG17 is only an example and should not bring any limitation to the functions and scope of
  • the electronic device 1700 may include a processor (e.g., a central processing unit, a graphics processing unit, etc.) 1701, which may perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1702 or a program loaded from a memory 1708 to a random access memory (RAM) 1703.
  • ROM read-only memory
  • RAM random access memory
  • Various programs and data required for the operation of the electronic device 1700 are also stored in the RAM 1703.
  • the processor 1701, the ROM 1702, and the RAM 1703 are connected to each other via a bus 1704.
  • An input/output (I/O) interface 1705 is also connected to the bus 1704.
  • the following devices may be connected to the I/O interface 1705: input devices 1706 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; output devices 1707 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 1708 including, for example, a magnetic tape, a hard disk, etc.; and communication devices 1709.
  • the communication devices 1709 may allow the electronic device 1700 to communicate wirelessly or wired with other devices to exchange data.
  • FIG. 17 shows an electronic device 1700 with various devices, it should be understood that it is not required to implement or have all the devices shown. More or fewer devices may be implemented or have alternatively.
  • an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program contains program code for executing the method shown in the flowchart.
  • the computer program can be downloaded and installed from the network through the communication device 1709, or installed from the memory 1708, or installed from the ROM 1702.
  • the program When the program is executed by the processor 1701, the above functions defined in the subtitle display method of the embodiment of the present disclosure are performed.
  • the computer-readable medium disclosed above may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two.
  • the computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above.
  • Computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.
  • a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, device or device.
  • a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried.
  • This propagated data signal may take a variety of forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above.
  • the computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device.
  • the program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.
  • the client and server may communicate using any currently known or future developed network protocol such as HTTP (HyperText Transfer Protocol), and may be interconnected with any form or medium of digital data communication (e.g., a communication network).
  • HTTP HyperText Transfer Protocol
  • Examples of communication networks include a local area network ("LAN”), a wide area network ("WAN”), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.
  • the computer-readable medium may be included in the electronic device, or may exist independently without being installed in the electronic device.
  • the computer-readable medium carries one or more programs.
  • the electronic device obtains the subtitle content corresponding to the currently played virtual reality video frame, determines the target space position in the virtual reality panoramic space according to the user's current line of sight, renders the subtitle layer according to the subtitle content at the target space position, and synchronously renders the subtitle content in the subtitle layer.
  • the position of the subtitle content rendered in the virtual reality space is determined following the user's line of sight, and on the basis of realizing the display of the subtitle content, the user's viewing experience is improved, and the subtitle content is rendered at the corresponding position, using the subtitle layer independent of the virtual reality video frame as the carrier for displaying the subtitle content, avoiding the distortion of the subtitle content display and improving the display effect of the subtitle content.
  • the electronic device may be written in one or more programming languages or a combination thereof to write computer program code for performing the operations of the present disclosure, including but not limited to object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages.
  • the program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server.
  • the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).
  • LAN local area network
  • WAN wide area network
  • Internet service provider e.g., via the Internet using an Internet service provider
  • each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the module, program segment, or a part of the code contains one or more executable instructions for realizing the specified logical function.
  • the functions marked in the box can also occur in a different order than that marked in the accompanying drawings. For example, two boxes shown in succession may actually be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved.
  • each box in the block diagram and/or flow chart, and the combination of boxes in the block diagram and/or flow chart can be implemented by a dedicated hardware-based system that performs the specified function or operation, or can be implemented by a combination of dedicated hardware and computer instructions.
  • the units involved in the embodiments described in the present disclosure may be implemented by software or hardware, wherein the name of a unit does not, in some cases, constitute a limitation on the unit itself.
  • exemplary types of hardware logic components include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chip (SOCs), complex programmable logic devices (CPLDs), and the like.
  • FPGAs field programmable gate arrays
  • ASICs application specific integrated circuits
  • ASSPs application specific standard products
  • SOCs systems on chip
  • CPLDs complex programmable logic devices
  • a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment.
  • a machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium.
  • a machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing.
  • a more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
  • RAM random access memory
  • ROM read-only memory
  • EPROM or flash memory erasable programmable read-only memory
  • CD-ROM portable compact disk read-only memory
  • CD-ROM compact disk read-only memory
  • magnetic storage device or any suitable combination of the foregoing.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Business, Economics & Management (AREA)
  • Marketing (AREA)
  • User Interface Of Digital Computer (AREA)
  • Studio Circuits (AREA)

Abstract

本公开实施例涉及一种字幕的显示方法、装置、设备及介质，其中该方法包括：获取与当前播放的虚拟现实视频帧对应的字幕内容；根据用户当前的视线方向，在虚拟现实全景空间中确定目标空间位置；在目标空间位置上根据字幕内容渲染字幕图层，并在字幕图层中同步渲染字幕内容。本公开的实施例中，确定字幕内容后，跟随用户的视线方向，确定在虚拟现实空间中渲染的字幕内容的位置，在实现字幕内容显示的基础上，提升了用户的观看体验，并在对应的位置渲染字幕内容，以独立于虚拟现实视频帧的字幕图层作为显示字幕内容的载体，避免了字幕内容显示扭曲，提升了字幕内容的显示效果。

Description

字幕的显示方法、装置、设备及介质
本申请要求申请日为2022年06月20日、名称为“字幕的显示方法、装置、设备及介质”、申请号为202210698095.3的中国发明专利申请的优先权。
技术领域
本公开涉及虚拟现实技术领域，尤其涉及一种字幕的显示方法、装置、设备及介质。
背景技术
虚拟现实技术(Virtual Reality，VR)，囊括计算机、电子信息、仿真技术，其基本实现方式是计算机模拟虚拟环境从而给人以环境沉浸感。随着VR应用的普及，在视频显示等场景下实现字幕在虚拟现实空间中的显示成为一种常见的场景。
相关技术中，对于一些不自带字幕内容的视频源而言，基于网页播放器等工具执行对视频源对应的音频流的翻译功能，获取翻译得到的字幕内容，在对应的视频帧上的固定位置显示翻译得到的字幕内容，然而，在VR显示场景下，为了实现立体显示效果，视频帧的显示方位会出现变化，从而，导致与视频帧相对固定设置的字幕内容可能会在视觉上出现扭曲，影响观看效果。
发明内容
为了解决上述技术问题或者至少部分地解决上述技术问题，本公开提供了一种字幕的显示方法、装置、设备及介质，跟随用户的视线方向，确定在虚拟现实空间中渲染的字幕内容的位置，提升了用户的观看体验，并在对应的位置渲染字幕内容，以独立于虚拟现实视频帧 的字幕图层作为显示字幕内容的载体，避免了字幕内容显示扭曲，提升了字幕内容的显示效果。
本公开实施例提供了一种字幕的显示方法，所述方法包括：获取与当前播放的虚拟现实视频帧对应的字幕内容；根据用户当前的视线方向，在虚拟现实全景空间中确定目标空间位置；在所述目标空间位置上根据所述字幕内容渲染字幕图层，并在所述字幕图层中同步渲染所述字幕内容。
本公开实施例还提供了一种字幕的显示装置，所述装置包括：获取模块，用于获取与当前播放的虚拟现实视频帧对应的字幕内容；位置确定模块，用于根据用户当前的视线方向，在虚拟现实全景空间中确定目标空间位置；渲染显示模块，用于在所述目标空间位置上根据所述字幕内容渲染字幕图层，并在所述字幕图层中同步渲染所述字幕内容。
本公开实施例还提供了一种电子设备，所述电子设备包括：处理器；用于存储所述处理器可执行指令的存储器；所述处理器，用于从所述存储器中读取所述可执行指令，并执行所述指令以实现如本公开实施例提供的字幕的显示方法。
本公开实施例还提供了一种计算机可读存储介质，所述存储介质存储有计算机程序，所述计算机程序用于执行如本公开实施例提供的字幕的显示方法。
本公开实施例提供的技术方案与现有技术相比具有如下优点：
本公开实施例提供的字幕的显示方案，获取与当前播放的虚拟现实视频帧对应的字幕内容，根据用户当前的视线方向，在虚拟现实全景空间中确定目标空间位置，在目标空间位置上根据字幕内容渲染字幕图层，并在字幕图层中同步渲染字幕内容。确定字幕内容后，跟随用户的视线方向，确定在虚拟现实空间中渲染的字幕内容的位置，在实现字幕内容显示的基础上，提升了用户的观看体验，并在对应的位置渲染字幕内容，以独立于虚拟现实视频帧的字幕图层作为显示字幕 内容的载体，避免了字幕内容显示扭曲，提升了字幕内容的显示效果。
附图说明
结合附图并参考以下具体实施方式，本公开各实施例的上述和其他特征、优点及方面将变得更加明显。贯穿附图中，相同或相似的附图标记表示相同或相似的元素。应当理解附图是示意性的，原件和元素不一定按照比例绘制。
图1为本公开实施例提供的一种字幕的显示方法的流程示意图；
图2为本公开实施例提供的一种字幕的显示场景示意图；
图3为本公开实施例提供的另一种字幕的显示场景示意图；
图4为本公开实施例提供的另一种字幕的显示场景示意图；
图5为本公开实施例提供的另一种字幕的显示场景示意图；
图6为本公开实施例提供的另一种字幕的显示场景示意图；
图7为本公开实施例提供的另一种字幕的显示方法的流程示意图；
图8为本公开实施例提供的另一种字幕的显示场景示意图；
图9为本公开实施例提供的另一种字幕的显示方法的流程示意图；
图10为本公开实施例提供的另一种字幕的显示场景示意图；
图11为本公开实施例提供的另一种字幕的显示场景示意图；
图12为本公开实施例提供的另一种字幕的显示场景示意图；
图13为本公开实施例提供的另一种字幕的显示方法的流程示意图；
图14A为本公开实施例提供的另一种字幕的显示场景示意图；
图14B为本公开实施例提供的另一种字幕的显示场景示意图；
图15为本公开实施例提供的另一种字幕的显示场景示意图；
图16为本公开实施例提供的一种字幕的显示装置的结构示意图；
图17为本公开实施例提供的一种电子设备的结构示意图。
具体实施方式
下面将参照附图更详细地描述本公开的实施例。虽然附图中显示 了本公开的某些实施例，然而应当理解的是，本公开可以通过各种形式来实现，而且不应该被解释为限于这里阐述的实施例，相反提供这些实施例是为了更加透彻和完整地理解本公开。应当理解的是，本公开的附图及实施例仅用于示例性作用，并非用于限制本公开的保护范围。
应当理解，本公开的方法实施方式中记载的各个步骤可以按照不同的顺序执行，和/或并行执行。此外，方法实施方式可以包括附加的步骤和/或省略执行示出的步骤。本公开的范围在此方面不受限制。
本文使用的术语“包括”及其变形是开放性包括，即“包括但不限于”。术语“基于”是“至少部分地基于”。术语“一个实施例”表示“至少一个实施例”；术语“另一实施例”表示“至少一个另外的实施例”；术语“一些实施例”表示“至少一些实施例”。其他术语的相关定义将在下文描述中给出。
需要注意，本公开中提及的“第一”、“第二”等概念仅用于对不同的装置、模块或单元进行区分，并非用于限定这些装置、模块或单元所执行的功能的顺序或者相互依存关系。
需要注意，本公开中提及的“一个”、“多个”的修饰是示意性而非限制性的，本领域技术人员应当理解，除非在上下文另有明确指出，否则应该理解为“一个或多个”。
本公开实施方式中的多个装置之间所交互的消息或者信息的名称仅用于说明性的目的，而并不是用于对这些消息或信息的范围进行限制。
为了解决上述问题，本公开实施例提供了一种字幕的显示方法，在该方法中，不再依赖于字幕内容显示的固定模式，而是解除字幕内容和画面内容的位置固定关系，跟随用户的视线方向适应性的调整字幕内容的显示位置，避免字幕内容在视觉上出现扭曲，提升了用户的视觉观看体验。
下面结合具体的实施例对该方法进行介绍。
图1为本公开实施例提供的一种字幕的显示方法的流程示意图，该方法可以由字幕的显示装置执行，其中该装置可以采用软件和/或硬件实现，一般可集成在电子设备中。如图1所示，该方法包括：
步骤101，获取与当前播放的虚拟现实视频帧对应的字幕内容。
在本实施例中，对于一些不自带字幕的虚拟现实视频帧而言，可以基于不同的方式来获取对应的字幕内容。
在一些可能的实施例中，确定与当前播放的虚拟现实视频帧对应的音频流，对音频流进行语音识别处理以获取字幕内容，比如，获取每段音频流对应的时间戳信息，基于时间戳信息和虚拟视频帧的时间戳信息匹配虚拟现实视频帧对应的字幕内容等。
在另一些可能的实施例中，可以预先识别并构建每个虚拟现实视频帧和对应的字幕内容的对应关系，并将该对应关系存储在对应的预设数据库中，从而，可以通过查询预设数据库以获取当前播放的虚拟现实视频帧对应的字幕内容。
步骤102，根据用户当前的视线方向，在虚拟现实全景空间中确定目标空间位置。
在本实施例中，为了避免字幕内容显示的扭曲，不再以虚拟视频帧作为字幕内容显示位置的确定依据，而是根据用户当前的视线方向，在虚拟现实全景空间中确定目标空间位置。
即实现目标空间位置与用户的当前视线方向一致，比如，如图2所示，目标空间位置位于用户当前的视线方向上，由此，可以保证在视线方向上的目标空间位置和视线方向的一致，进一步为字幕内容的显示效果提供了技术支撑。
其中，根据用户当前的视线方向在虚拟现实全景空间中确定目标空间位置的方式，可以参照后续实施例，在此不再赘述。
步骤103，在目标空间位置上根据字幕内容渲染字幕图层，并在字幕图层中同步渲染字幕内容。
在本公开的一个实施例中，在确定了目标空间位置后，在目标空 间位置渲染字幕图层，并在字幕图层中同步渲染字幕内容，比如，继续参照图2，若是用户的视线方向水平变化，则字幕图层跟随视线方向的变化，在视线方向上的空间位置显示字幕图层，其中，字幕内容同步在字幕图层渲染。由于字幕图层在用户的视线方向上，因此，保证了字幕内容可以无畸变扭曲的呈现给用户，保证了视觉显示效果。
在本实施例中，以独立于虚拟现实视频帧的字幕图层单独渲染字幕内容，避免了字幕内容的显示扭曲，且字幕图层根据字幕内容渲染，视觉上科技感更强，提升了用户的视觉观看体验。
在公开的一个本实施例中，为了进一步提升字幕图层的智能感，还可以对字幕图层进行背景添加显示处理，对字幕图层添加背景特效提升显示的智能感，在一些可能的实施例中，为了避免对虚拟现实视频帧的遮挡，还可以将字幕图层处理为透明度较高的图层，比如，在一些可能的实施例中，字幕图层为背光子图层和量子点阵子图层，其中，量子点阵子图层的量子颜色可以为蓝色等任意颜色，量子点阵子可以渐变形式显示，也可以均匀亮度显示，如图3所示，背光子图层和量子点阵子图层的叠加形成的字幕图层，在视觉上可以给用户更强的科技感。在另一些可能的实施例中，可以在字幕图层中渲染灯光动画，比如，极光动画，比如白光流转动画等，在此不一一赘述，通过灯光动画的添加，提升显示的科技感。
在实际执行过程中，为了进一步提升在视觉上的显示效果，还可以根据字幕内容动态渲染字幕图层。
在本公开的一个实施例中，获取字幕内容的显示数量，渲染与显示数量匹配的字幕图层，由此，实现了伴随字幕显示数量变化动态的显示字幕图层的效果。
在一些可能的实施例中，获取字幕内容的显示数量，根据显示数量、预设的单位字幕宽度和预设的单位字幕高度，确定字幕实时宽度和字幕实时高度。其中，字幕内容的显示数量指的是当前虚拟现实视频帧中显示的所有字幕内容的数量，比如，若是语音识别得到字幕内 容，则可以每识别或者是读取得到一个或多个文字及触发对字幕内容的显示事件，从而，字幕内容的显示数量发生变化。预设的单位字幕宽度指的是预先规定的每个字幕内容的宽度，预设的单位字幕高度指的是预先规定的每个字幕内容的高度。
在本实施例中，确定字幕实施高度和字幕实时宽度，以便于基于字幕实施高度和字幕实时宽度动态渲染字幕图层。
在一些可能的实施例中，可以根据显示数量、预设的单位字幕宽度和预设的单位字幕高度查询预设数据库以获取字幕实时宽度和字幕实时高度。
在另一些可能的实施例中，可以确定显示数量是否大于宽度方向上的预设字幕数量阈值，其中，预设字幕数量阈值可以理解为预先规定的每一行显示的最多的字幕内容数量，若是大于预设字幕数量阈值，则表明字幕内容需要换行处理，因此，根据预设字幕数量阈值和单位字幕宽度确定字幕实时宽度，比如计算预设字幕数量阈值和单位字幕宽度的乘积以确定字幕实时宽度。
同样的，可以根据预设字幕数量阈值和显示数量确定字幕行数，比如，计算显示数量和预设字幕数量阈值的比值，确定大于等于该比值的最小整数作为字幕行数，进而，根据字幕行数和单位字幕高度确定字幕实时高度，比如计算字幕行数和单位字幕高度的乘积以确定字幕实时高度。
在本实施例中，若是显示数量不大于预设字幕数量阈值，即字幕显示数量不足一行，则根据单位字幕宽度和显示数量确定字幕实时宽度，即计算单位字幕宽度和显示数量的乘积以确定字幕实时宽度，并确定单位字幕高度为字幕实时高度。
进一步地，在确定字幕实时宽度和字幕实时高度后，若是字幕实时宽度发生了宽度变化，则响应于字幕实时宽度的宽度变化，根据单位字幕的图层宽度和字幕实时宽度，渲染与字幕内容匹配的实时字幕图层宽度，其中，单位字幕的图层宽度为每个单位字幕在宽度方向上 增加时，字幕图层增加的单位宽度。
同样的，在确定字幕实时宽度和字幕实时高度后，若是字幕实时高度发生了高度变化，则响应于字幕实时高度的高度变化，根据预设的单位字幕的图层高度和字幕实时高度，渲染与字幕内容匹配的实时字幕图层高度。其中，单位字幕的图层高度为每个单位字幕在高度向上增加时，字幕图层增加的单位高度。即字幕图层可以伴随字幕内容进行横向和纵向的扩展变化。
即在本实施例中，如图4所示，当实时显示的字幕内容的宽度小于或者等于预设字幕数量阈值时，根据实时显示的字幕实时宽度确定字幕图层宽度(图中字幕内容为“这里显示的是第*行字幕内容”，字幕图层为渐变亮度)，比如，每次实时新增显示一个或多个字幕内容，则根据单位字幕宽度显示对应的字幕内容，同时根据单位字幕的图层宽度扩展字幕图层。通常单位字幕宽度小于单位字幕的图层宽度，以保证字幕内容在视觉上可以被字幕图层包裹。从而，在视觉上实现了在宽度方向上，根据字幕的显示内容跟随显示字幕背景。
在本公开的一个实施例中，如图5所示，若是实时显示的字幕内容的宽度大于预设字幕数量阈值，对字幕内容进行换行显示，根据字幕实时高度的高度变化，渲染与字幕内容匹配的实时字幕图层高度，比如，当实时输入的字幕宽度大于预先设置的字幕最大行宽时，即需要换行显示时，每次实时新增显示一个或多个字幕内容，则根据单位字幕高度显示对应的字幕内容，同时根据单位字幕的图层高度扩展字幕图层的显示高度，通常单位字幕高度小于单位字幕的图层高度，以保证字幕内容在视觉上可以被字幕图层包裹。
从而，在视觉上实现了在高度方向上，根据字幕的显示内容跟随显示字幕图层，在视觉上形成了滚动显示的效果，继续参照图5所示，伴随着字幕内容从第一行增加到了第六行，字幕图层高度逐渐向下扩展，形成了滚动显示的效果。其中，在实际执行过程中，不同行对应的单位字幕的图层高度可以相同也可不同，比如，可以规定第一行单 位字幕的图层高度为72，第二行单位字幕的图层高度为100，第三行单位字幕的图层高度为120，第四行单位字幕的图层高度为132，第五行单位字幕的图层高度为156，第六行单位字幕的图层高度为180等。
在实际执行过程中，若是显示数量较多，比如大于预设值，则为了避免对虚拟现实视频帧的遮挡，还可以通过缩小预设的单位字幕宽度，和/或，预设的单位字幕高度来显示字幕内容。
又或者，可以判断当前计算得到的字幕行数是否大于预设行数阈值，其中，若是大于该预设行数阈值，则首先显示预设行数阈值的字幕内容，将其他的剩余的字幕内容滚动显示，比如，若是预设行数阈值为6行，但是计算得到的字幕行数为7，则可以先显示前6行字幕内容，在显示预设时长后，不再显示第1行的字幕内容，并将第2-6行的字幕内容向上滚动并在第6行显示原来的第7行的字幕内容，或者也可以在显示预设时长后，将第1-6行的字幕内容不再显示，仅仅显示原来的第7行的字幕内容等。
另外，在上述控制字幕图层根据字幕内容动态渲染时，当渲染的最后一行字幕内容没有达到预设字幕数量阈值时，可以如上述图5所示，根据预设字幕数量阈值渲染一整行图层，也可以如图6所示，根据最后一行的字幕内容数量以及预设的单位字幕宽度确定最后一行的字幕图层宽度，以进一步提升字幕图层跟随渲染的智能化程度。
综上，本公开实施例的字幕的显示方法，获取与当前播放的虚拟现实视频帧对应的字幕内容，根据用户当前的视线方向，在虚拟现实全景空间中确定目标空间位置，在目标空间位置上根据字幕内容渲染字幕图层，并在字幕图层中同步渲染字幕内容。确定字幕内容后，跟随用户的视线方向，确定在虚拟现实空间中渲染的字幕内容的位置，在实现字幕内容显示的基础上，提升了用户的观看体验，并在对应的位置渲染字幕内容，以独立于虚拟现实视频帧的字幕图层作为显示字幕内容的载体，避免了字幕内容显示扭曲，提升了字幕内容的显示效果。
在不同的应用场景中，根据用户当前的视线方向在虚拟现实全景空间中确定目标空间位置的方式不同，示例如下：
在本公开的一个实施例中，如图7所示，根据用户当前的视线方向在虚拟现实全景空间中确定目标空间位置，包括：
步骤701，确定虚拟现实全景空间的中心点位置，以及获取预设的半径距离。
在本实施例中，确定虚拟现实全景空间的中心点位置，该中心点位置位于虚拟现实全景空间的中心处，中心点位置和虚拟现实全景空间的形状有关。在确定虚拟现实全景空间的中心点位置后，获取预设的半径距离，其中，预设半径距离可以是预先根据虚拟现实全景空间的尺寸设置，通常预设的半径距离到中心点位置的长度，不会超过虚拟现实全景空间中中心点位置到虚拟现实全景空间的表面的长度，从而，保证显示的字幕图层位于虚拟现实视频帧前面，形成一道屏障，提升字幕图层显示的科技感。
步骤702，从中心点位置开始，按照用户当前的视线方向延伸到预设的半径距离的位置作为目标空间位置。
在本实施例中，由于通常预设的半径距离到中心点位置的长度，不会超过虚拟现实全景空间中中心点位置到虚拟现实全景空间的表面的长度，因此，从中心点位置开始，按照用户当前的视线方向延伸到预设的半径距离的位置作为目标空间位置，一方面，可以保证目标空间位置在虚拟空间内部，保证了显示效果，另一方面，可以保证目标空间位置与用户视线方向的一致性，使得显示在目标空间位置的字幕内容与用户的视线方向一致，保证了用户可以清晰的观看到字幕内容，提升了观看体验。
举例而言，如图8所示，虚拟现实全景空间为“盒状”立方体空间，预设的半径距离为R1，虚拟现实全景空间的中心点位置为O1，则在确定了用户当前的视线方向后，直接按照用户当前的视线方向延伸到预设的半径距离的位置作为目标空间位置。
在本公开的一个实施例中，如图9所示，根据用户当前的视线方向在虚拟现实全景空间中确定目标空间位置，包括：
步骤901，获取在虚拟现实全景空间中显示上一帧的字幕内容对应的历史空间位置。
在本实施例中，获取在虚拟现实全景空间中显示上一帧的字幕内容对应的历史空间位置，该历史空间位置是根据上一帧字幕下用户的视线方向确定的。
步骤902，获取用户当前的视线方向相对于观看上一帧视线方向之间的视线变化信息。
其中，视线变化信息用于表示用户当前的视线方向相对于观看上一帧视线方向之间的视线的角度的变化等，比如，视线变化信息可以包括用户当前的视线方向相对于观看上一帧视线方向之间的视线的变化方向以及变化角度等。
在本公开的一个实施例中，获取用户穿戴的虚拟现实设备中的相机在水平方向上相对于上一帧的水平轴旋转角度，确定该水平轴旋转角度为用户从观看上一帧的水平视线方向到观看当前帧的水平视线方向的变化信息，在本实施例中，由于视线变化信息包括水平视线方向的变化信息，可以保证用户在左右旋转观看时，目标空间位置可以跟随用户的视线方向，进一步保证了显示在目标空间位置的字幕内容与用户的视线方向一致性。
步骤903，根据视线变化信息以及历史空间位置确定目标空间位置。
在本实施例中，在确定了视线变化信息后，可以根据视线变化信息以及历史空间位置确定目标空间位置，比如，可以根据视线变化信息移动历史空间位置以确定目标空间位置。
需要强调的是，在本实施例中，仅仅根据用户当前的视线方向相对于观看上一帧视线方向之间的视线变化信息确定目标空间位置，并不考虑当前用户在虚拟现实空间中的位置以及上一帧用户在虚拟空间 中位置的变化，无论当前用户在虚拟现实空间中的位置，相对于上一帧用户在虚拟空间中位置是进一步接近虚拟空间的表面，还是进一步远离了虚拟空间的表面，都不影响目标空间位置的确定，目标空间位置仅仅与用户当前的视线方向相对于观看上一帧视线方向之间的视线变化信息有关，因此，在视觉上使得用户不会感觉到显示的字幕内容与用户人眼之间具有固定显示距离，进一步保证了用户的视觉效果。
举例而言，如图10所示，虚拟现实全景空间为“盒状”立方体空间，在虚拟现实全景空间中显示上一帧的字幕内容对应的历史空间位置为W1，获取用户当前的视线方向S2相对于观看上一帧视线方向S1之间的视线变化信息如图中所示，则根据视线变化信息控制历史空间位置的移动，以确定移动后的位置W2为目标空间位置。
需要说明的是，在不同的应用场景中，根据视线变化信息以及历史空间位置确定目标空间位置的方式不同，示例如下：
在本公开的一个实施例中，获取在虚拟现实全景空间中预先设置的圆心位置，其中，该圆心位置作为圆心通常设置在虚拟现实空间的中心位置，以保证在作为圆心使字幕环绕跟随时，字幕内容可以显示在虚拟现实空间中。
在本实施例中，可获取虚拟现实设备在虚拟现实全景空间中的初始位置，其中，初始位置为虚拟现实全景空间中的中心点位置，以初始位置作为在虚拟现实全景空间中的中点位置进行虚拟现实全景空间的搭建，进而，将虚拟现实全景空间中的中心点位置设置为作为圆心位置。
其中，在本公开的一个实施例中，虽然将虚拟现实设备在虚拟现实全景空间中的初始位置作为圆心位置，但是在实际执行过程中，当虚拟现实设备在虚拟现实全景空间移动时，继续将该中心点位置设置为圆心位置，而不继续以用户的VR头戴显示设备作为虚拟现实全景空间中的圆心位置，视觉上将VR头戴显示设备的移动距离等和视线方向无关的参数与字幕内容的显示的位置剥离开，提升了视觉显示效 果，避免字幕内容始终与VR头戴显示设备的移动距离等关联时，形成字幕内容始终与VR头戴显示设备联动，影响用户的视觉体验。
进一步地，在获取了圆心位置后，将水平轴旋转角度作为从上一帧到当前帧的圆心旋转角度，并且确定上一帧的字幕内容相对于圆心的历史空间位置，进而，根据圆心位置、历史空间位置、以及圆心旋转角度，确定目标空间位置。
举例而言，如图11所示，获取用户穿戴的虚拟现实设备中的相机在水平方向上相对于上一帧的水平轴旋转角度从当前播放的视频帧相对于上一帧的水平轴旋转角度后，获取在虚拟现实全景空间中预先设置的圆心位置，将水平轴旋转角度设置为旋转根节点的圆心旋转角度，进而，确定上一帧的字幕内容相对于圆心的历史空间位置，根据该圆心旋转角度旋转，带动字幕内容以圆心位置为旋转中心，将字幕内容从历史空间位置旋转到与用户视线方向一致的目标空间位置，保证了后续显示在目标空间位置的字幕内容的显示效果。
其中，在一些可能的实施例中，可以预先根据样本数据训练神经网络模型，将圆心位置、历史空间位置、以水平轴旋转角度输入该神经网络模型，以当前的视线方向在虚拟现实全景空间中对应的目标空间位置。
在另一些可能的实施例中，可以采用预设算法计算得到当前的视线方向在虚拟现实全景空间中对应的目标空间位置，其中，预设算法可以如下述公式(1)所示，其中，在公式(1)中，x，y，z分别为历史空间位置在三个坐标轴上的坐标，x1，y1，z1分别为目标空间位置在三个坐标轴上的坐标，θ为圆心旋转角度，Tx，Ty，Tz分别为圆心位置在三个坐标轴上的坐标。
其中，若是当前为首次显示的第一帧字幕内容，可以获取预设的半径距离，根据圆心位置、半径距离，以及用户的初始视线方向确定初始帧的字幕内容相对于圆心的初始空间位置，将该初始空间位置作为第一帧字幕内容的渲染位置。
综上，本公开实施例的字幕的显示方法，可根据场景需要灵活的确定字幕内容在虚拟现实全景空间中渲染的目标空间位置，保证了目标空间位置跟随用户当前的视线方向，实现了字幕内容跟随用户当前的视线方向的移动，保证了字幕内容的视觉显示效果。
基于上述实施例，由于字幕图层独立于虚拟现实视频帧作为一个图层单独显示，因此，有可能对其他显示的图层造成遮挡，比如，如图12所示，若是在观看视频时，用户执行了播放面板图层的显示操作，则会在用户的视线方向上采用上述目标空间位置的确定原理，确定出播放面板图层的显示位置并进行显示。从而，可能对播放面板图层造成了遮挡，用户难以对播放面板执行播控操作。
因此，为了避免上述问题，在本公开的一个实施例中，若是监测到在虚拟现实全景空间中显示其他图层，则识别其他图层的显示状态信息，根据其他图层的显示状态信息调整字幕图层的显示状态信息，以保证用户对字幕内容以及播放面板图层等其他图层可以清晰的观看到。
在本公开的一个实施例中，显示状态信息包括显示的空间位置，包括：在本实施例中，如图13所示，根据其他图层的显示状态信息调整字幕图层的显示状态信息，包括：
步骤1301，确定其他图层所在的参考空间位置与目标空间位置是否满足预设的遮挡条件。
在本实施例中，若是监测到在虚拟现实全景空间中显示其他图层，其他图层可以包括任意跟随用户实现方向显示的图层，比如，播控面板图层等，通常其他图层相对于字幕图层来说更为贴近虚拟现实视频 帧，确定其他图层与字幕图层是否满足预设的遮挡条件，即判断字幕图层是否会遮挡其他图层。
需要说明的是，在不同的应用场景下，确定其他图层所在的参考空间位置与目标空间位置是否满足预设的遮挡条件的方式不同，示例如下：
在一些可能的实施例中，根据参考空间位置识别其他图层在垂直视线方向的平面上的第一中心点位置，根据目标空间位置识别字幕图层在垂直视线方向的平面上的第二中心点位置，其中，垂直视线方向的平面可以理解为用户视线上正对观看的平面，比如，若是以用户视线方向为Z轴建立坐标系，则垂直视线方向的平面可以理解为XY平面。
在本实施例中，确定第一中心点位置与第二中心点位置的位置差值是否小于预设距离阈值，其中，若是小于预设距离阈值，则确定其他图层和字幕图层的重合度较高，字幕图层可能会和其他图层重合，对其他图层造成遮挡，因此，确定满足预设的遮挡条件。
在另一些可能的实施例中，根据参考空间位置和目标空间位置，在垂直视线方向的平面上，确定字幕图层与其他图层的重叠区域面积，比如，可以根据重叠区域的尺寸确定重叠区域面积，又比如，可以根据重叠区域的像素的个数确定重叠区域面积等，计算重叠区域面积与其他图层的图层面积的面积比值，确定面积比值是否大于预设面积比值阈值，其中，若是大于预设面积比值阈值，则确定满足预设的遮挡条件。
当然，在本公开的一个实施例中，并非是对所有的其他图层都判断是否遮挡，对于一些可遮挡的图层比如广告图层等，并不需要判断字幕图层是否对其造成遮挡，因此，在本实施例中，还可以确定其他图层的图层等级，并确定图层等级高于预设等级阈值，其中，图层等级可以通过识别当前的其他图层的图层类型后，查询预设对应关系得到等。
步骤1302，若是满足遮挡条件，则确定字幕图层的目标移动位置和/或目标图层显示尺寸，其中，目标移动位置和/或目标图层显示尺寸对应的字幕图层与其他图层不满足遮挡条件。
在本公开的一个实施例中，若是满足预设遮挡条件，则确定字幕图层的目标移动位置，其中，目标移动位置的字幕图层与其他图层不满足遮挡条件。
在实际执行过程中，目标移动位置可以为尽量靠近用户视线方向但是不会对其他图层造成遮挡的任意位置。
在本公开的一个实施例中，确定其他图层的上边缘线的第一高度信息，该第一高度信息可以理解为Y轴坐标值等，根据第一高度信息确定字幕图层的下边缘线的第二高度信息，其中，第二高度信息大于等于第一高度信息，即第二高度信息可以稍微高于第一高度信息也可以就在第一高度信息上，进而，根据第二高度信息确定目标移动位置。
举例而言，如图14A所示，若是其他图层和字幕图层都为矩形，则其他图层与字幕图层满足遮挡条件时，则继续如图14A所示，可以确定其他图层的上边缘线的第一高度信息，将字幕图层向上平移，使得字幕图层从位于其他图层的前方移动到其他图层的上方，避免了对其他图层的遮挡。
在本公开的另一个实施例中，确定其他图层的左边缘线的第一宽度信息，该第一宽度信息可以理解为X轴坐标值等，根据第一宽度信息确定字幕图层的右边缘线的第二宽度信息，其中，第二宽度信息小于等于第一宽度信息，即第二宽度信息可以稍微小于第一宽度信息也可以就在第一宽度信息上，进而，根据第二宽度信息确定目标移动位置。
举例而言，如图14B所示，若是其他图层和字幕图层都为矩形，则其他图层与字幕图层满足遮挡条件时，则继续如图14B所示，可以确定其他图层的左边缘线的第一宽度信息，将字幕图层向左平移，使得字幕图层从位于其他图层的前方移动到其他图层的左方，避免了对 其他图层的遮挡。
当然，上述实施例中控制字幕图层向上移动或者向左移动仅仅是一种可能的示例，在实际执行时，还可以控制字幕图层相对于其他图层的下方或者右方移动，其移动原理类似，在此不一一列举。
在本公开的另一个实施例中，若是满足遮挡条件，则确定字幕图层的显示尺寸。
在本实施例中，可根据参考空间位置确定其他图层的参考显示尺寸，根据参考显示尺寸确定字幕图层的目标图层显示尺寸，比如，计算参考显示尺寸和预设缩小比例的乘积，其中，预设缩小比例小于1，根据乘积值确定目标图层显示尺寸；又比如，发计算参考显示尺寸和字幕图层的初始显示尺寸的尺寸差值，根据该尺寸差值查询预设对应关系确定缩放比例，其中，尺寸差值越大，则缩放比例越小，根据该缩放比例缩放初始显示尺寸以得到目标图层显示尺寸。
在本公开的实施例中，如图15所示，若是其他图层和字幕图层都为矩形，则其他图层与字幕图层满足遮挡条件时，则可以根据其他图层的参考显示尺寸确定字幕图层的目标图层显示尺寸，目标图层显示尺寸的字幕图层对其他图层的遮挡区域较小，从而，不影响对其他图层的观看。
在本公开的另一个实施例中，也可以同时确定字幕图层的目标移动位置和目标图层显示尺寸，即结合位置和尺寸的调整来避免对其他图层的遮挡。
步骤1303，根据目标移动位置和/或目标图层显示尺寸显示字幕图层。
在本实施例中，控制字幕图层移动到目标移动位置和/或目标图层显示尺寸显示，其中，渲染在字幕图层上的字幕内容也同步移动，用户不但可以观看到字幕内容也可以观看到其他图层的内容等。
考虑到其他图层可能都是暂时显示的图层，因此，为了进一步提升视觉显示效果，在本公开的一个实施例中，若是监测到其他图层的 关闭显示指令，则控制字幕图层移动到目标空间位置上显示。
在本公开的一个实施例中，上述显示状态信息也可以包括显示时间点，通过其他图层的显示时间点，调整字幕图层的显示时间点，保证用户可以观看到其他图层的显示内容。其中，调整字幕图层的显示时间点可以控制字幕图层在其他图层的显示时间点下，根据预设显示频次断续显示等，比如，若是其他图层的显示时间点为a-b，则可以控制其他图层在a-b时间点下每隔c时间显示d时长，从而，保证用户可以观看到其他图层的显示内容。
综上，本公开实施例的字幕的显示方法，在字幕图层可能会对其他图层造成遮挡后，还可以改变字幕图层的显示在状态，以进一步保证用户对其他图层的内容观看，提升了显示的灵活性。
为了实现上述实施例，本公开还提出了一种字幕的显示装置。
图16为本公开实施例提供的一种字幕的显示装置的结构示意图，该装置可由软件和/或硬件实现，一般可集成在电子设备中进行字幕的显示。如图16所示，该装置包括：获取模块1610、位置确定模块1620、渲染显示模块1630，其中，
获取模块1610，用于获取与当前播放的虚拟现实视频帧对应的字幕内容；
位置确定模块1620，用于根据用户当前的视线方向，在虚拟现实全景空间中确定目标空间位置；
渲染显示模块1630，用于在目标空间位置上根据字幕内容渲染字幕图层，并在字幕图层中同步渲染字幕内容。
本公开实施例所提供的字幕的显示装置可执行本公开任意实施例所提供的字幕的显示方法，具备执行方法相应的功能模块和有益效果。
为了实现上述实施例，本公开还提出一种计算机程序产品，包括计算机程序/指令，该计算机程序/指令被处理器执行时实现上述实施例中的字幕的显示方法。
图17为本公开实施例提供的一种电子设备的结构示意图。
下面具体参考图17，其示出了适于用来实现本公开实施例中的电子设备1700的结构示意图。本公开实施例中的电子设备1700可以包括但不限于诸如移动电话、笔记本电脑、数字广播接收器、PDA(个人数字助理)、PAD(平板电脑)、PMP(便携式多媒体播放器)、车载终端(例如车载导航终端)等等的移动终端以及诸如数字TV、台式计算机等等的固定终端。图17示出的电子设备仅仅是一个示例，不应对本公开实施例的功能和使用范围带来任何限制。
如图17所示，电子设备1700可以包括处理器(例如中央处理器、图形处理器等)1701，其可以根据存储在只读存储器(ROM)1702中的程序或者从存储器1708加载到随机访问存储器(RAM)1703中的程序而执行各种适当的动作和处理。在RAM 1703中，还存储有电子设备1700操作所需的各种程序和数据。处理器1701、ROM 1702以及RAM 1703通过总线1704彼此相连。输入/输出(I/O)接口1705也连接至总线1704。
通常，以下装置可以连接至I/O接口1705：包括例如触摸屏、触摸板、键盘、鼠标、摄像头、麦克风、加速度计、陀螺仪等的输入装置1706；包括例如液晶显示器(LCD)、扬声器、振动器等的输出装置1707；包括例如磁带、硬盘等的存储器1708；以及通信装置1709。通信装置1709可以允许电子设备1700与其他设备进行无线或有线通信以交换数据。虽然图17示出了具有各种装置的电子设备1700，但是应理解的是，并不要求实施或具备所有示出的装置。可以替代地实施或具备更多或更少的装置。
特别地，根据本公开的实施例，上文参考流程图描述的过程可以被实现为计算机软件程序。例如，本公开的实施例包括一种计算机程序产品，其包括承载在非暂态计算机可读介质上的计算机程序，该计算机程序包含用于执行流程图所示的方法的程序代码。在这样的实施例中，该计算机程序可以通过通信装置1709从网络上被下载和安装，或者从存储器1708被安装，或者从ROM 1702被安装。在该计算机程 序被处理器1701执行时，执行本公开实施例的字幕的显示方法中限定的上述功能。
需要说明的是，本公开上述的计算机可读介质可以是计算机可读信号介质或者计算机可读存储介质或者是上述两者的任意组合。计算机可读存储介质例如可以是——但不限于——电、磁、光、电磁、红外线、或半导体的***、装置或器件，或者任意以上的组合。计算机可读存储介质的更具体的例子可以包括但不限于：具有一个或多个导线的电连接、便携式计算机磁盘、硬盘、随机访问存储器(RAM)、只读存储器(ROM)、可擦式可编程只读存储器(EPROM或闪存)、光纤、便携式紧凑磁盘只读存储器(CD-ROM)、光存储器件、磁存储器件、或者上述的任意合适的组合。在本公开中，计算机可读存储介质可以是任何包含或存储程序的有形介质，该程序可以被指令执行***、装置或者器件使用或者与其结合使用。而在本公开中，计算机可读信号介质可以包括在基带中或者作为载波一部分传播的数据信号，其中承载了计算机可读的程序代码。这种传播的数据信号可以采用多种形式，包括但不限于电磁信号、光信号或上述的任意合适的组合。计算机可读信号介质还可以是计算机可读存储介质以外的任何计算机可读介质，该计算机可读信号介质可以发送、传播或者传输用于由指令执行***、装置或者器件使用或者与其结合使用的程序。计算机可读介质上包含的程序代码可以用任何适当的介质传输，包括但不限于：电线、光缆、RF(射频)等等，或者上述的任意合适的组合。
在一些实施方式中，客户端、服务器可以利用诸如HTTP(HyperText Transfer Protocol，超文本传输协议)之类的任何当前已知或未来研发的网络协议进行通信，并且可以与任意形式或介质的数字数据通信(例如，通信网络)互连。通信网络的示例包括局域网(“LAN”)，广域网(“WAN”)，网际网(例如，互联网)以及端对端网络(例如，ad hoc端对端网络)，以及任何当前已知或未来研发的网络。
上述计算机可读介质可以是上述电子设备中所包含的；也可以是单独存在，而未装配入该电子设备中。
上述计算机可读介质承载有一个或者多个程序，当上述一个或者多个程序被该电子设备执行时，使得该电子设备：获取与当前播放的虚拟现实视频帧对应的字幕内容，根据用户当前的视线方向，在虚拟现实全景空间中确定目标空间位置，在目标空间位置上根据字幕内容渲染字幕图层，并在字幕图层中同步渲染字幕内容。确定字幕内容后，跟随用户的视线方向，确定在虚拟现实空间中渲染的字幕内容的位置，在实现字幕内容显示的基础上，提升了用户的观看体验，并在对应的位置渲染字幕内容，以独立于虚拟现实视频帧的字幕图层作为显示字幕内容的载体，避免了字幕内容显示扭曲，提升了字幕内容的显示效果。
电子设备可以以一种或多种程序设计语言或其组合来编写用于执行本公开的操作的计算机程序代码，上述程序设计语言包括但不限于面向对象的程序设计语言—诸如Java、Smalltalk、C++，还包括常规的过程式程序设计语言—诸如“C”语言或类似的程序设计语言。程序代码可以完全地在用户计算机上执行、部分地在用户计算机上执行、作为一个独立的软件包执行、部分在用户计算机上部分在远程计算机上执行、或者完全在远程计算机或服务器上执行。在涉及远程计算机的情形中，远程计算机可以通过任意种类的网络——包括局域网(LAN)或广域网(WAN)—连接到用户计算机，或者，可以连接到外部计算机(例如利用因特网服务提供商来通过因特网连接)。
附图中的流程图和框图，图示了按照本公开各种实施例的***、方法和计算机程序产品的可能实现的体系架构、功能和操作。在这点上，流程图或框图中的每个方框可以代表一个模块、程序段、或代码的一部分，该模块、程序段、或代码的一部分包含一个或多个用于实现规定的逻辑功能的可执行指令。也应当注意，在有些作为替换的实现中，方框中所标注的功能也可以以不同于附图中所标注的顺序发生。 例如，两个接连地表示的方框实际上可以基本并行地执行，它们有时也可以按相反的顺序执行，这依所涉及的功能而定。也要注意的是，框图和/或流程图中的每个方框、以及框图和/或流程图中的方框的组合，可以用执行规定的功能或操作的专用的基于硬件的***来实现，或者可以用专用硬件与计算机指令的组合来实现。
描述于本公开实施例中所涉及到的单元可以通过软件的方式实现，也可以通过硬件的方式来实现。其中，单元的名称在某种情况下并不构成对该单元本身的限定。
本文中以上描述的功能可以至少部分地由一个或多个硬件逻辑部件来执行。例如，非限制性地，可以使用的示范类型的硬件逻辑部件包括：现场可编程门阵列(FPGA)、专用集成电路(ASIC)、专用标准产品(ASSP)、片上***(SOC)、复杂可编程逻辑设备(CPLD)等等。
在本公开的上下文中，机器可读介质可以是有形的介质，其可以包含或存储以供指令执行***、装置或设备使用或与指令执行***、装置或设备结合地使用的程序。机器可读介质可以是机器可读信号介质或机器可读储存介质。机器可读介质可以包括但不限于电子的、磁性的、光学的、电磁的、红外的、或半导体***、装置或设备，或者上述内容的任何合适组合。机器可读存储介质的更具体示例会包括基于一个或多个线的电气连接、便携式计算机盘、硬盘、随机存取存储器(RAM)、只读存储器(ROM)、可擦除可编程只读存储器(EPROM或快闪存储器)、光纤、便捷式紧凑盘只读存储器(CD-ROM)、光学储存设备、磁储存设备、或上述内容的任何合适组合。
以上描述仅为本公开的较佳实施例以及对所运用技术原理的说明。本领域技术人员应当理解，本公开中所涉及的公开范围，并不限于上述技术特征的特定组合而成的技术方案，同时也应涵盖在不脱离上述公开构思的情况下，由上述技术特征或其等同特征进行任意组合而形成的其它技术方案。例如上述特征与本公开中公开的(但不限于) 具有类似功能的技术特征进行互相替换而形成的技术方案。
此外，虽然采用特定次序描绘了各操作，但是这不应当理解为要求这些操作以所示出的特定次序或以顺序次序执行来执行。在一定环境下，多任务和并行处理可能是有利的。同样地，虽然在上面论述中包含了若干具体实现细节，但是这些不应当被解释为对本公开的范围的限制。在单独的实施例的上下文中描述的某些特征还可以组合地实现在单个实施例中。相反地，在单个实施例的上下文中描述的各种特征也可以单独地或以任何合适的子组合的方式实现在多个实施例中。
尽管已经采用特定于结构特征和/或方法逻辑动作的语言描述了本主题，但是应当理解所附权利要求书中所限定的主题未必局限于上面描述的特定特征或动作。相反，上面所描述的特定特征和动作仅仅是实现权利要求书的示例形式。

Claims (21)

  1. 一种字幕的显示方法，其特征在于，包括以下步骤：
    获取与当前播放的虚拟现实视频帧对应的字幕内容；
    根据用户当前的视线方向，在虚拟现实全景空间中确定目标空间位置；
    在所述目标空间位置上根据所述字幕内容渲染字幕图层，并在所述字幕图层中同步渲染所述字幕内容。
  2. 如权利要求1所述的方法，其特征在于，所述获取与当前播放的虚拟现实视频帧对应的字幕内容，包括：
    对所述当前播放的虚拟现实视频帧对应的音频流进行语音识别处理以获取所述字幕内容；或者，
    查询预设数据库以获取与所述当前播放的虚拟现实视频帧对应的字幕内容。
  3. 如权利要求1所述的方法，其特征在于，所述根据用户当前的视线方向，在虚拟现实全景空间中确定目标空间位置，包括：
    确定所述虚拟现实全景空间的中心点位置，以及获取预设的半径距离；
    从所述中心点位置开始，按照所述用户当前的视线方向延伸到所述预设的半径距离的位置作为所述目标空间位置。
  4. 如权利要求1所述的方法，其特征在于，所述根据用户当前的视线方向，在虚拟现实全景空间中确定目标空间位置，包括：
    获取在所述虚拟现实全景空间中显示上一帧的字幕内容对应的历史空间位置；
    获取所述用户当前的视线方向相对于观看上一帧视线方向之间的视线变化信息；
    根据所述视线变化信息以及所述历史空间位置确定所述目标空间位置。
  5. 如权利要求4所述的方法，其特征在于，所述获取所述用户当前的视线方向相对于观看上一帧视线方向之间的视线变化信息，包括：
    获取用户穿戴的虚拟现实设备中的相机在水平方向上相对于上一帧的水平轴旋转角度，其中，
    所述水平轴旋转角度为所述用户从观看上一帧的水平视线方向到观看当前帧的水平视线方向的变化信息。
  6. 如权利要求1所述的方法，其特征在于，所述在所述目标空间位置上根据所述字幕内容渲染字幕图层，包括：
    获取字幕内容的显示数量；
    渲染与所述显示数量匹配的字幕图层。
  7. 如权利要求6所述的方法，其特征在于，所述渲染与所述显示数量匹配的字幕图层，包括：根据所述显示数量、预设的单位字幕宽度和预设的单位字幕高度，确定字幕实时宽度和字幕实时高度；
    响应于所述字幕实时宽度的宽度变化，根据单位字幕的图层宽度和所述字幕实时宽度，渲染与所述字幕内容匹配的实时字幕图层宽度；和/或，
    响应于所述字幕实时高度的高度变化，根据预设的单位字幕的图层高度和所述字幕实时高度，渲染与所述字幕内容匹配的实时字幕图层高度。
  8. 如权利要求7所述的方法，其特征在于，所述根据所述显示数量、预设的单位字幕宽度和预设的单位字幕高度，确定字幕实时宽度和字幕实时高度，包括：
    确定所述显示数量是否大于宽度方向上的预设字幕数量阈值；
    若是大于所述预设字幕数量阈值，则根据所述预设字幕数量阈值和所述单位字幕宽度确定所述字幕实时宽度；
    根据所述预设字幕数量阈值和所述显示数量确定字幕行数，并根据所述字幕行数和所述单位字幕高度确定所述字幕实时高度。
  9. 如权利要求8所述的方法，其特征在于，还包括：
    若是不大于所述预设字幕数量阈值，则根据所述单位字幕宽度和所述显示数量确定所述字幕实时宽度；
    确定所述单位字幕高度为所述字幕实时高度。
  10. 如权利要求1-9任一所述的方法，其特征在于，还包括：
    若是监测到在所述虚拟现实全景空间中显示其他图层，则识别所述其他图层的显示状态信息；
    根据所述其他图层的显示状态信息调整所述字幕图层的显示状态信息。
  11. 如权利要求10所述的方法，其特征在于，所述显示状态信息包括显示的空间位置，所述根据所述其他图层的显示状态信息调整所述字幕图层的显示状态信息，包括：
    确定所述其他图层所在的参考空间位置与所述目标空间位置是否满足预设的遮挡条件；
    若是满足所述遮挡条件，则确定所述字幕图层的目标移动位置和/或目标图层显示尺寸，其中，所述目标移动位置和/或目标图层显示尺寸对应的所述字幕图层与所述其他图层不满足所述遮挡条件；
    根据所述目标移动位置和/或目标图层显示尺寸显示所述字幕图层。
  12. 如权利要求10所述的方法，其特征在于，在所述根据所述其他图层的显示状态信息调整所述字幕图层的显示状态信息之前，还包括：
    确定所述其他图层的图层等级，并确定所述图层等级高于预设等级阈值。
  13. 如权利要求11所述的方法，其特征在于，所述确定所述其他图层所在的参考空间位置与所述目标空间位置是否满足预设的遮挡条件，包括：
    根据所述参考空间位置，识别所述其他图层在垂直所述视线方向的平面上的第一中心点位置；
    根据所述目标空间位置，识别所述字幕图层在垂直所述视线方向的平面上的第二中心点位置；
    确定所述第一中心点位置与所述第二中心点位置的位置差值是否小于预设距离阈值，其中，若是小于所述预设距离阈值，则确定满足所述预设的遮挡条件。
  14. 如权利要求11所述的方法，其特征在于，所述确定所述其他图层所在的参考空间位置与所述目标空间位置是否满足预设的遮挡条件，包括：
    根据所述参考空间位置和所述目标空间位置，确定所述字幕图层与所述其他图层在垂直所述视线方向的平面上的重叠区域面积；
    计算所述重叠区域面积与所述其他图层的图层面积的面积比值；
    确定所述面积比值是否大于预设面积比值阈值，其中，若是大于所述预设面积比值阈值，则确定满足所述预设的遮挡条件。
  15. 如权利要求11所述的方法，其特征在于，所述确定所述字幕图层的目标移动位置和/或目标图层显示尺寸，包括：
    根据所述参考空间位置，确定所述其他图层的上边缘线的第一高度信息，
    根据所述第一高度信息确定所述字幕图层的下边缘线的第二高度信息，其中，所述第二高度信息大于等于所述第一高度信息，
    根据所述第二高度信息确定所述目标移动位置；和/或，
    根据所述参考空间位置，确定所述其他图层的参考显示尺寸，
    根据所述参考显示尺寸确定所述字幕图层的目标图层显示尺寸。
  16. 如权利要求11所述的方法，其特征在于，还包括：
    若是监测到所述其他图层的关闭显示指令，则控制所述字幕图层移动到所述目标空间位置上显示。
  17. 如权利要求1-9任一所述的方法，其特征在于，还包括：
    对所述字幕图层进行背景添加显示处理。
  18. 如权利要求17所述的方法，其特征在于，所述对所述字幕图 层进行背景添加显示处理，包括：
    在所述字幕图层下添加背光子图层和量子点阵子图层；和/或，
    在所述字幕图层上渲染灯光动画。
  19. 一种字幕的显示装置，其特征在于，包括：
    获取模块，用于获取与当前播放的虚拟现实视频帧对应的字幕内容；
    位置确定模块，用于根据用户当前的视线方向，在虚拟现实全景空间中确定目标空间位置；
    渲染显示模块，用于在所述目标空间位置上根据所述字幕内容渲染字幕图层，并在所述字幕图层中同步渲染所述字幕内容。
  20. 一种电子设备，其特征在于，所述电子设备包括：
    处理器；
    用于存储所述处理器可执行指令的存储器；
    所述处理器，用于从所述存储器中读取所述可执行指令，并执行所述可执行指令以实现上述权利要求1-18中任一所述的字幕的显示方法。
  21. 一种计算机可读存储介质，其特征在于，所述计算机可读存储介质存储有计算机程序，所述计算机程序用于执行上述权利要求1-18中任一所述的字幕的显示方法。
PCT/CN2023/091203 2022-06-20 2023-04-27 字幕的显示方法、装置、设备及介质 WO2023246302A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202210698095.3 2022-06-20
CN202210698095.3A CN117319725A (zh) 2022-06-20 2022-06-20 字幕的显示方法、装置、设备及介质

Publications (2)

Publication Number Publication Date
WO2023246302A1 WO2023246302A1 (zh) 2023-12-28
WO2023246302A9 true WO2023246302A9 (zh) 2024-06-20

Family

ID=89259027

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2023/091203 WO2023246302A1 (zh) 2022-06-20 2023-04-27 字幕的显示方法、装置、设备及介质

Country Status (2)

Country Link
CN (1) CN117319725A (zh)
WO (1) WO2023246302A1 (zh)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117676053B (zh) * 2024-01-31 2024-04-16 成都华栖云科技有限公司 一种动态字幕渲染方法及***

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104918097B (zh) * 2015-06-01 2018-05-08 无锡天脉聚源传媒科技有限公司 一种字幕生成方法及装置
CN106412718A (zh) * 2016-09-18 2017-02-15 乐视控股（北京）有限公司 一种字幕在3d空间中的渲染方法及其装置
CN107613405B (zh) * 2017-09-30 2019-12-27 福州瑞芯微电子股份有限公司 一种vr视频字幕显示方法和装置
EP3493028A1 (en) * 2017-12-04 2019-06-05 Thomson Licensing Devices, systems and methods for enhancing content viewer experience using augmented reality
CN112019772A (zh) * 2020-09-07 2020-12-01 中图云创智能科技(北京)有限公司 一种全景视频中纠正字幕桶状变形的方法

Also Published As

Publication number Publication date
CN117319725A (zh) 2023-12-29
WO2023246302A1 (zh) 2023-12-28

Similar Documents

Publication Publication Date Title
CN110766777B (zh) 虚拟形象的生成方法、装置、电子设备及存储介质
US11842425B2 (en) Interaction method and apparatus, and electronic device and computer-readable storage medium
WO2021139382A1 (zh) 人脸图像的处理方法、装置、可读介质和电子设备
WO2022170958A1 (zh) 基于增强现实的显示方法、设备、存储介质及程序产品
WO2023221409A1 (zh) 虚拟现实空间的字幕渲染方法、装置、设备及介质
US12026896B2 (en) Method and apparatus for displaying object in video, and electronic device and computer readable storage medium
US11589027B2 (en) Methods, systems, and media for generating and rendering immersive video content
WO2023246302A9 (zh) 字幕的显示方法、装置、设备及介质
EP4175307A1 (en) Interaction method and apparatus, and electronic device
US20220283697A1 (en) Video editing and playing method, apparatus, device and medium
WO2023138559A1 (zh) 虚拟现实交互方法、装置、设备和存储介质
WO2023138548A1 (zh) 图像处理方法、装置、设备和存储介质
WO2023193642A1 (zh) 视频处理方法、装置、设备及介质
US20230133416A1 (en) Image processing method and apparatus, and device and medium
WO2023116801A1 (zh) 一种粒子效果渲染方法、装置、设备及介质
WO2023216936A1 (zh) 视频播放方法、装置、电子设备、存储介质和程序产品
WO2023179362A1 (zh) 控件的显示方法、装置、电子设备、存储介质和程序产品
WO2023169305A1 (zh) 特效视频生成方法、装置、电子设备及存储介质
CN114679628B (zh) 一种弹幕添加方法、装置、电子设备和存储介质
CN113589926B (zh) 虚拟界面操作方法、头戴式显示设备和计算机可读介质
WO2021244651A1 (zh) 信息显示方法、装置、终端及存储介质
US20230267664A1 (en) Animation processing method and apparatus, electronic device and storage medium
WO2024067159A1 (zh) 视频生成方法、装置、电子设备及存储介质
CN111726666A (zh) 视频显示控制方法和装置
EP4071725A1 (en) Augmented reality-based display method and device, storage medium, and program product

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 23825955

Country of ref document: EP

Kind code of ref document: A1