WO2023142370A1 - 投屏显示方法、装置、电子设备及可读存储介质 - Google Patents

投屏显示方法、装置、电子设备及可读存储介质 Download PDF

Info

Publication number
WO2023142370A1
WO2023142370A1 PCT/CN2022/102415 CN2022102415W WO2023142370A1 WO 2023142370 A1 WO2023142370 A1 WO 2023142370A1 CN 2022102415 W CN2022102415 W CN 2022102415W WO 2023142370 A1 WO2023142370 A1 WO 2023142370A1
Authority
WO
WIPO (PCT)
Prior art keywords
image data
picture
line
cropping
screen
Prior art date
Application number
PCT/CN2022/102415
Other languages
English (en)
French (fr)
Inventor
李宾
周强
Original Assignee
深圳创维-Rgb电子有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 深圳创维-Rgb电子有限公司 filed Critical 深圳创维-Rgb电子有限公司
Publication of WO2023142370A1 publication Critical patent/WO2023142370A1/zh

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/10Segmentation; Edge detection
    • G06T7/13Edge detection
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/14Digital output to display device ; Cooperation and interconnection of the display device with other functional units
    • G06F3/1454Digital output to display device ; Cooperation and interconnection of the display device with other functional units involving copying of the display data of a local workstation or window to a remote workstation or window so that an actual copy of the data is displayed simultaneously on two or more displays, e.g. teledisplay
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00Image enhancement or restoration
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/10Segmentation; Edge detection
    • G06T7/12Edge-based segmentation
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/10Segmentation; Edge detection
    • G06T7/136Segmentation; Edge detection involving thresholding

Definitions

  • the present application relates to the technical field of screen projection, and in particular to a screen projection display method, device, and readable storage medium.
  • the main purpose of this application is to provide a screen projection method, which aims to solve the technical problem that other information about the screen may be lost when the screen on the mobile terminal is rearranged to a large-screen device.
  • the present application provides a screen projection display method, which includes the following steps:
  • Integrating the additional information into the clipped image data to obtain the integrated image data Integrating the additional information into the clipped image data to obtain the integrated image data.
  • the preset extraction model includes preset extraction elements, and the step of extracting additional information in the image data according to the preset extraction model includes:
  • the extracted elements are used as the additional information.
  • the method further includes:
  • Image processing is performed on the additional information, and the image processing includes one or more of color processing, outline processing, and mask processing.
  • the step of integrating the additional information into the trimmed image data, and obtaining the integrated image data includes:
  • the processed additional information is placed in a preset position of the trimmed image data, so as to obtain the integrated image data.
  • the step of acquiring the integrated image data it includes:
  • the integrated image data is displayed, and the additional information is hidden after being displayed for a preset period of time, and the additional information is displayed again when a wake-up operation is triggered.
  • the step of performing black border recognition on the image data, and clipping the image data according to the black border recognition result includes:
  • a target cropping position is generated according to the first cropping position and the second cropping position, and the image data is cropped according to the target cropping position.
  • the step of identifying black borders on the first picture and the second picture respectively includes:
  • obtaining the first frame and the second frame from the image data is obtaining two pictures from the image data in the projection data.
  • the step of performing black border recognition on the image data, and clipping the image data according to the black border recognition result includes:
  • the first cropping position and the second cropping position will be combined to generate an average cropping position, and then the black border of the image data in the projection data will be adjusted according to the target cropping position Partial crop removal.
  • the color processing is to perform color filling on the extracted text elements and graphic elements, and the filling color can be preset or determined according to the color value of pixels in the preset area where the text elements and graphic elements are extracted;
  • the outline processing is when the image recognition technology cannot recognize the definite text elements and graphic elements, color-processing the identified outlines as text elements or graphic elements;
  • the masking process adds a masked background to the additional information text elements and graphic elements.
  • the generating a target cropping position according to the first cropping position and the second cropping position, and cropping the image data according to the target cropping position includes:
  • the first trimming position or the second trimming position Any one of the clipping positions is used as the target clipping position;
  • the target trimming position includes the average first line and The average second line is used to crop the image data according to the average first line and the average second line of the target cropping position.
  • the present application also provides a projection screen display device, the projection screen display device includes:
  • An acquisition module configured to acquire screen projection data, where the screen projection data includes image data
  • a cropping module configured to perform black edge recognition on the image data, clip the image data according to the black edge recognition result, and obtain the cropped image data
  • An extraction module configured to extract additional information in the image data according to a preset extraction model
  • An integration module configured to integrate the additional information into the cropped image data, and obtain the integrated image data.
  • the preset extraction model includes a preset extraction area
  • the extraction module is configured to extract an area corresponding to the preset extraction area in the image data; and use the extracted area as the additional information.
  • the present application also provides an electronic device, the electronic device includes: a memory, a processor, and a screen projection program stored in the memory and operable on the processor, the When the screen projection display program is executed by the processor, the above-mentioned steps of the screen projection display method are realized.
  • the present application also provides a readable storage medium, on which a screen projection display program is stored, and when the screen projection display program is executed by a processor, the above screen projection display is realized method steps.
  • FIG. 1 is a schematic diagram of a screen projection connection scene in the screen projection display method of the present application
  • FIG. 2 is a schematic structural diagram of electronic equipment in the hardware operating environment involved in the solution of the embodiment of the present application;
  • FIG. 3 is a schematic flow chart of the first embodiment of the projection screen display method of the present application.
  • FIG. 4 is a schematic diagram of the display effect of the first picture on the display device before cropping in the projection screen display method of the present application;
  • FIG. 5 is a schematic diagram of a display screen of image data on a display device in the projection screen display method of the present application
  • FIG. 6 is a schematic diagram of image data after integrating additional information in the projection screen display method of the present application.
  • FIG. 7 is a schematic diagram of a scene of an implementable method in the projection screen display method of the present application.
  • FIG. 8 is a schematic structural diagram of a device involved in the projection screen display method of the present application.
  • the main solution of the embodiment of the present application is: after the screen projection data is cropped, information is extracted from the image data in the screen projection data again, and the extracted additional information is integrated into the cropped graphic data.
  • the present application provides a solution, so that relevant information of image data is complete when displayed, avoiding loss of core information, and improving viewing experience of users.
  • FIG. 1 is a schematic diagram of a screen projection connection involved in an embodiment of the present application.
  • the screen projector includes: a screen projector transmitter and a screen projector receiver, wherein the first electronic device can be a personal computer, or a smart phone, a tablet computer, an e-book reader, a portable computer and other mobile terminal devices, the second electronic device may be a television, or a display device with a display function such as a monitor or a projector.
  • the first electronic device can be a personal computer, or a smart phone, a tablet computer, an e-book reader, a portable computer and other mobile terminal devices
  • the second electronic device may be a television, or a display device with a display function such as a monitor or a projector.
  • the first electronic device is connected to the screen projector transmitter or screen projector receiver through a wired or wireless network for data transmission and interaction;
  • the screen projector transmitter is connected to the screen projector receiver through a wireless network for data transmission and interaction, or the screen projector transmitter is connected to the second electronic device through a wireless network for data transmission and interaction;
  • the screen projector receiver is connected to the second electronic device through a wired or wireless network for data transmission and interaction.
  • the wired connection between the screen projector receiver and the second electronic device and between the screen projector transmitter and the first electronic device may include HDMI (High Definition Multimedia Interface, USB (Universal Serial Bus, Universal Serial Bus), DP (DisplayPort, Display Interface) and VGA (Video Graphics Array, Video Graphics Array) and other wired connections.
  • HDMI High Definition Multimedia Interface
  • USB Universal Serial Bus
  • DP DisplayPort
  • VGA Video Graphics Array, Video Graphics Array
  • the wireless communication methods between the first electronic device and the screen projector transmitter and the screen projector receiver, and between the second electronic device and the screen projector transmitter and the screen projector receiver may include Bluetooth, wireless Broadband (Wi-Fi), infrared communication and other wireless communications.
  • connection methods for implementing screen projection between the first electronic device, the screen projector transmitter, the screen projector receiver and the second electronic device at least include:
  • the screen projector transmitter is connected to the screen projector receiver, and the screen of the first electronic device is finally projected to the second electronic device through the screen projector transmitter and the screen projector receiver;
  • the transmitter of the screen projector is wirelessly connected with the second electronic device, and the screen of the first electronic device is projected to the second electronic device through the transmitter of the screen projector.
  • the communication modules are connected through a wireless network;
  • the first electronic device is connected to the screen projector receiver, and the screen of the first electronic device is projected to the second electronic device through the screen projector receiver.
  • FIG. 2 is a schematic diagram of a terminal structure of a hardware operating environment involved in the solution of the embodiment of the present application.
  • the electronic device in this embodiment of the present application may be a screen projector, or an electronic device with functions of data receiving, data processing, and data sending, such as a TV, a mobile phone, a PC, a tablet computer, and a portable computer.
  • the device may include: a processor 1001 , such as a CPU, a network interface 1004 , a user interface 1003 , a memory 1005 , and a communication bus 1002 .
  • the communication bus 1002 is used to realize connection and communication between these components.
  • the user interface 1003 may include a display screen (Display), an input unit such as a keyboard (Keyboard), and the optional user interface 1003 may also include a standard wired interface and a wireless interface.
  • the network interface 1004 may include a standard wired interface or a wireless interface (such as a WI-FI interface).
  • the memory 1005 may be a high-speed RAM memory, or a stable memory (non-volatile memory), such as a disk memory.
  • the memory 1005 may also be a storage device independent of the aforementioned processor 1001 .
  • the device may further include a camera, an RF (Radio Frequency, radio frequency) circuit, a sensor, an audio circuit, a WiFi module, and the like.
  • sensors such as light sensors, motion sensors and other sensors.
  • the light sensor may include an ambient light sensor and a proximity sensor, wherein the ambient light sensor may adjust the brightness of the display screen according to the brightness of the ambient light, and the proximity sensor may turn off the display screen and/or backlight.
  • the gravitational acceleration sensor can detect the magnitude of acceleration in various directions (generally three axes), and can detect the magnitude and direction of gravity when it is stationary, and can be used to identify the application of mobile terminal posture (such as horizontal and vertical screen switching, Related games, magnetometer attitude calibration), vibration recognition related functions (such as pedometer, tap), etc.
  • mobile terminal posture such as horizontal and vertical screen switching, Related games, magnetometer attitude calibration), vibration recognition related functions (such as pedometer, tap), etc.
  • mobile devices can also be equipped with other sensors such as gyroscope, barometer, hygrometer, thermometer, infrared sensor, etc. Here No longer.
  • FIG. 1 does not constitute a limitation to the device, and may include more or less components than shown in the figure, or combine some components, or arrange different components.
  • the memory 1005 as a computer storage medium may include an operating system, a network communication module, a user interface module, and a screen projection display program.
  • the network interface 1004 is mainly used to connect to the background server and perform data communication with the background server;
  • the user interface 1003 is mainly used to connect to the client (client) and perform data communication with the client;
  • the processor 1001 can be used to call the projection screen display program stored in memory 1005, and perform the following operations:
  • Integrating the additional information into the clipped image data to obtain the integrated image data Integrating the additional information into the clipped image data to obtain the integrated image data.
  • the processor 1001 may invoke the screen projection display program stored in the memory 1005, and further perform the following operations:
  • the preset extraction model includes a preset extraction area, and the step of extracting additional information in the image data according to the preset extraction model includes:
  • the extracted area is used as the additional information.
  • the processor 1001 may invoke the screen projection display program stored in the memory 1005, and further perform the following operations:
  • the preset extraction model includes preset extraction elements, and the step of extracting additional information in the image data according to the preset extraction model includes:
  • the extracted elements are used as the additional information.
  • the processor 1001 may invoke the screen projection display program stored in the memory 1005, and further perform the following operations:
  • the method further includes:
  • Image processing is performed on the additional information, and the image processing includes one or more of color processing, outline processing, and mask processing.
  • the processor 1001 may invoke the screen projection display program stored in the memory 1005, and further perform the following operations:
  • the step of integrating the additional information into the trimmed image data and obtaining the integrated image data includes:
  • the processed additional information is placed in a preset position of the trimmed image data, so as to obtain the integrated image data.
  • the processor 1001 may invoke the screen projection display program stored in the memory 1005, and further perform the following operations:
  • the integrated image data is displayed, and the additional information is hidden after being displayed for a preset period of time, and the additional information is displayed again when a wake-up operation is triggered.
  • the processor 1001 may invoke the screen projection display program stored in the memory 1005, and further perform the following operations:
  • the step of performing black border recognition on the image data, and cutting the image data according to the black border recognition result includes:
  • a target cropping position is generated according to the first cropping position and the second cropping position, and the image data is cropped according to the target cropping position.
  • the processor 1001 may invoke the screen projection display program stored in the memory 1005, and further perform the following operations:
  • the step of identifying black borders on the first picture and the second picture respectively includes:
  • the screen projection display method includes:
  • Step S10 acquiring screen projection data, where the screen projection data includes image data
  • the subject of implementation can be an intelligent electronic device participating in the screen projection process, such as a screen projection device transmitter, a screen projection device receiver, a TV, a mobile phone, etc., and this embodiment takes the screen projection device as the optimal embodiment.
  • the screen projector acquires screen projection data, and the screen projection data may generally include video data or several continuous pictures, etc., and the screen projection data may also include audio data.
  • the screen projector acquires image data from the projection data. It can be understood that if the projection data is video data, the image can be intercepted, and if it is graphic data, it can be directly received.
  • Step S20 performing black edge recognition on the image data, cutting the image data according to the black edge recognition result, and obtaining the cut image data;
  • the image data in the projection data may have a large area of black borders, which affects the viewing experience, and then black border recognition is performed on the image data.
  • the first picture and the second picture are obtained from the image data, and black border recognition is performed on the first picture and the second picture respectively, and the first cropping of the first picture is determined position and a second cropping position of the second frame; generating a target cropping position according to the first cropping position and the second cropping position, and cropping the image data according to the target cropping position.
  • the screen projection device when performing screen projection, will obtain two pictures from the image data in the screen projection data, that is, the first picture and the second picture. Identify the area with black borders in the first picture and the area of the actual playback screen, use the boundary between the black border area in the first picture and the area of the actual playback screen as the first cropping position, and identify the area with black borders in the second picture and the actual playback area In the area of the screen, the boundary line between the black border area in the second screen and the actual playing screen area is used as the second cropping position.
  • the color values of the pixels in the first picture are compared, the pixels in the first picture are divided into first comparison units, and the color values of the pixels in the first comparison units are divided into A position whose degree of change exceeds a preset threshold is identified as a black border and used as the first cropping position; comparing the pixel color values of the second picture, and dividing the pixel points of the second picture into a second comparison A unit for identifying a position where the change degree of the color value of the pixel in the second comparison unit exceeds a preset threshold as a black boundary line and as the second cropping position.
  • the first picture is taken as an example for illustration, and the processing process of the second picture is similar to that of the first picture, and details will not be described later.
  • the first picture is composed of a large number of pixels.
  • the pixels of the first picture Points are divided into different first comparison units according to different division rules (for example, if a row of pixels is used as a comparison unit, it is the first row comparison unit; if a column of pixels is used as a comparison unit, it is the first column comparison unit).
  • comparison unit compare the pixel color values of each comparison unit, for example, compare unit a with comparison unit b, if the difference between the two color values exceeds the preset threshold, then compare a
  • the unit is used as the first cropping position, or the comparison unit b is used as the first cropping position, or the position between the comparison unit a and the comparison unit b is used as the first cropping position.
  • the first comparison unit may be the first row comparison unit, the first column comparison unit, or the two comparison units respectively once.
  • the black borders distributed on the upper and lower sides of the screen can be identified; when divided into column comparison units, the black borders distributed on the left and right sides of the screen can be recognized, and the two comparison units are compared separately Once, you can recognize the black borders distributed on the four sides of the screen.
  • the first comparison unit is a line comparison unit, and the current comparison unit is obtained according to the first preset comparison order; the current comparison unit’s The pixel point color value is compared with the pixel point color value of the adjacent comparison unit of the current comparison unit; if the pixel point color value of the current comparison unit is compared with the pixel point color value of the adjacent comparison unit If the number of different values exceeds the preset threshold, the position between the current comparison unit and the adjacent comparison unit is used as the first cropping position; if the pixel point color of the current comparison unit is If the value differs from the color value of the pixel point of the adjacent comparison unit and does not exceed the preset threshold, the current comparison unit is reacquired until the pixel points of the first frame are traversed.
  • the first preset comparison order includes: an order from the upper and lower edges of the screen to the middle of the screen; or, an order from the middle of the screen to the upper and lower edges of the
  • the acquired first picture includes Area 1 and Area 2 as shown in Figure 4, where Figure 4 shows the display effect of the first picture on the display device before cropping, and Area 3 is the adaptive display of the first picture by the display device
  • a black border is generated
  • the first line a and the second line b are the dividing line between area 1 and area 2
  • area 1 is the black border
  • area 2 is the actual playback screen.
  • the first comparison unit is a line comparison unit. According to the first preset comparison sequence, the pixel points of a certain row are obtained from the first picture as the current comparison row, and the color of each pixel point of the current comparison row is further extracted. Value, compare it with the color value of the pixels in the adjacent row of the current comparison row.
  • the interval line between the current comparison row and the adjacent row is used as above crop line. For example, if the total number of pixels in a row is 100, it corresponds to 100 color values. If the preset ratio threshold is 80%, then when there are 81 color values that are different between the current comparison row and the adjacent row (default The proportion threshold can be set according to the actual situation), that is, the first cropping position can be determined according to the position of the current comparison row and the position of the adjacent row, such as the first line a and the second line b in Figure 4.
  • the current comparison sequence is reacquired according to the preset first comparison sequence.
  • the comparison row performs pixel point color value comparison with the adjacent row of the current comparison row at this time. All pixels up to the first frame are compared in a row manner.
  • the first cropping position according to the first frame
  • the second cropping position according to the second frame
  • the target cropping position includes the average first line a and the average second line b
  • the image data is cropped according to the average first line a and the average second line b of the target cropping position, that is, if the image data includes As shown in Area 1 and Area 2, remove Area 1 (black border area) and leave only Area 2 (actual play area). It can be understood that the reserved actual playback area is the cropped image data.
  • the specific number of images to be acquired, the classification of comparison units, and the comparison order can all be set according to actual conditions, and will not be repeated here.
  • Step S30 extracting additional information in the image data according to a preset extraction model
  • the preset extraction model includes a preset extraction area, and an area corresponding to the preset extraction area is extracted from the image data; the extracted area is used as the additional information.
  • the actually cropped part of the image data may contain information related to the image data, for example, the title, author, built-in icon, introduction and other information of the image data will be lost .
  • the preset extraction model will extract the above missing information.
  • the preset extraction model may randomly select one of the first picture and the second picture as an extraction object, and may also re-acquire the picture from the image data.
  • it is the image data on the display screen of the display device, including the playback area and the black border area.
  • the playback area displays image data
  • the preset 1 area and the preset 2 area in the playback area are the preset extraction models.
  • the default extraction area for It can be understood that the preset area 1 and the preset area 2 shown in Figure 5 do not impose specific restrictions on the position, quantity and size of the preset extraction area, and the position, number and size of the preset area can be set according to the actual situation .
  • the pictures of the preset area 1 and the preset area 2 are further extracted and used as additional information.
  • the preset extraction model includes preset extraction elements, elements corresponding to the preset extraction elements are extracted from the image data, and the extracted elements are used as the additional information.
  • the above preset extraction elements may include text elements and graphic elements. Based on the above example, the preset extraction model extracts elements from the playback area in Figure 5. The extraction of specific text elements and graphic elements can be extracted using image recognition technology, and the extracted The obtained text elements and graphic elements are used as additional information of the cropped image data.
  • image processing is performed on the additional information, and the image processing includes one or more of color processing, outline processing, and mask processing.
  • the additional information may be the aforementioned preset 1 and preset 2 areas, or text elements and graphic elements, etc.
  • the image processing includes one or more of color processing, outline processing, and mask processing.
  • color processing the extracted text elements and graphic elements are filled with color, the filling color can be set in advance, and can also be determined according to the pixel color value in the preset area where the text elements and graphic elements are extracted.
  • Contour processing is that if the image recognition technology cannot recognize certain text elements and graphic elements, the directly recognized contours are color-processed and used as text elements or graphic elements.
  • Mask processing adding a mask background to additional information text elements and graphic elements, such as setting a fixed color or a transparent mask, to prevent text elements and graphic elements from being distinguished from the actual playback area and affecting the user's viewing effect.
  • the processed additional information is placed in a preset position of the cropped image data, so as to obtain the integrated image data.
  • the additional information is processed through the above steps, and will be placed in the preset position in the cropped image data, as shown in Figure 6, which is a schematic diagram of the image data after integrating the additional information, including the title text area, profile information area and icon area, the reserved image area is the area reserved after the image data is cut, the title text area, brief information area and icon area are the preset positions of the above-mentioned image data after cutting, if the additional information is in the text element and Graphical elements, the text elements should be placed in the title text area and the brief information area, and the graphic elements should be placed in the icon area. If the additional information is the above-mentioned preset area 1 or preset The frame of area 2 is placed at the preset position of the trimmed image data. It can be understood that, the title text area, brief information area and icon area shown in FIG. 6 do not limit the size, quantity and area of the preset positions, and can be set according to actual conditions.
  • the step of acquiring the integrated image data includes: displaying the integrated image data, and hiding the additional information after displaying for a preset duration, and triggering The additional information is redisplayed upon wakeup operation.
  • the additional information when the integrated image data is displayed, the additional information will block the actual video content to a certain extent, especially when the video time is long, which will affect the user's actual viewing experience.
  • the additional information after the additional information is displayed for a certain period of time (such as 1 minute), the additional information is hidden, so that the actual playback content is completely displayed, improving the user's viewing experience.
  • the additional information is redisplayed, such as the user performs operations such as pause, fast forward, and double-speed playback on the mobile phone, screen projector or television, and the additional information is redisplayed.
  • the actual screen projection scene includes: a mobile phone 100, a screen projector transmitter 200, a screen projector receiver 300, and a television set 400.
  • the mobile phone 100 and the projector The screen projector transmitter 200 is in communication connection, the screen projector receiver 300 is in communication connection with the screen projector transmitter 200 , and the screen projector receiver 300 is in communication connection with the TV set 400 .
  • the screen projection device transmitter 200 obtains the projection data pushed by the mobile phone 100, cuts the image data in the projection data, and generates attachments and information based on the image data before cutting, and then integrates the generated attachments and information
  • the cropped image data is sent to the screen projector receiver 300, and the screen projector receiver 300 pushes the integrated image data to the TV 400 for display.
  • this embodiment also provides a projection screen display device, as shown in FIG. 8 , the projection screen display device includes:
  • An acquisition module 10 configured to acquire screen projection data, where the screen projection data includes image data
  • a cropping module 20 configured to perform black border recognition on the image data, clip the image data according to the black border recognition result, and obtain the cropped image data
  • An extraction module 30, configured to extract additional information in the image data according to a preset extraction model
  • the integration module 40 is configured to integrate the additional information into the cropped image data, and obtain the integrated image data.
  • the preset extraction model includes a preset extraction area
  • the extraction module 30 is further configured to:
  • the extracted area is used as the additional information.
  • the preset extraction model includes preset extraction elements
  • the extraction module 30 is further configured to:
  • the extracted elements are used as the additional information.
  • the extracting module 30 is also used for:
  • Image processing is performed on the additional information, and the image processing includes one or more of color processing, outline processing, and mask processing.
  • the integration module 40 is also used for:
  • the processed additional information is placed in a preset position of the trimmed image data, so as to obtain the integrated image data.
  • the integration module 40 is also used for:
  • the integrated image data is displayed, and the additional information is hidden after being displayed for a preset period of time, and the additional information is displayed again when a wake-up operation is triggered.
  • the clipping module 20 is also used for:
  • a target cropping position is generated according to the first cropping position and the second cropping position, and the image data is cropped according to the target cropping position.
  • the clipping module 20 is also used for:
  • the screen projection display device provided by the present application adopts the screen projection method in the above-mentioned embodiments, and solves the technical problem that other information about the screen may be lost when the screen on the mobile terminal is rearranged to a large-screen device.
  • the beneficial effect of the screen projection display device provided by the embodiment of the present application is the same as that of the screen projection method provided by the above embodiment, and other technical features of the screen projection display device are the same as those of the above embodiment method The disclosed features are the same and will not be repeated here.
  • this embodiment also provides an electronic device, the electronic device includes: a memory, a processor, and a screen projection display program stored in the memory and operable on the processor, the screen projection display When the program is executed by the processor, the steps of the above screen projection method are realized.
  • this embodiment also provides a readable storage medium, the readable storage medium stores a screen projection display program, and when the screen projection display program is executed by a processor, the steps of the above screen projection display method are implemented .
  • the methods of the above embodiments can be implemented by means of software plus a necessary general-purpose hardware platform, and of course also by hardware, but in many cases the former is better implementation.
  • the technical solution of the present application can be embodied in the form of a software product in essence or the part that contributes to the prior art, and the computer software product is stored in a storage medium (such as ROM/RAM) as described above. , magnetic disk, optical disk), including several instructions to make a terminal device (which may be a mobile phone, computer, server, screen projector, or network device, etc.) execute the methods described in various embodiments of the present application.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Human Computer Interaction (AREA)
  • General Engineering & Computer Science (AREA)
  • Controls And Circuits For Display Device (AREA)
  • Editing Of Facsimile Originals (AREA)

Abstract

一种投屏显示方法、装置、电子设备及可读存储介质,涉及投屏技术领域,投屏显示方法包括:获取投屏数据,投屏数据包括图像数据;对图像数据进行黑边识别,根据黑边识别结果对图像数据进行裁剪,获取裁剪后的图像数据;根据预设提取模型,提取图像数据中的附加信息;将附加信息整合到裁剪后的图像数据,获取整合后的图像数据。将丢失的附加信息重新在显示设备上显示,保证图像数据的信息完整性,避免裁剪造成的信息丢失,提升用户的观看体验。

Description

投屏显示方法、装置、电子设备及可读存储介质
相关申请
本申请要求于2022年1月30日提交中国专利局、申请号为202210114218.4、名称为“投屏显示方法、装置、电子设备及可读存储介质”的中国专利申请的优先权。
技术领域
本申请涉及投屏技术领域,尤其涉及一种投屏显示方法、设备及可读存储介质。
背景技术
随着时代的变化,现在将电视端的大屏当作显示或者投屏设备的需求越来越多。且随着移动端设备功能不断创新升级,用户对多设备之间的交互的要求越来越高,而电视的大屏端有着比移动端更佳的观看体验,所以将移动端的页面能够布局到大屏端的诉求是越来越强烈。但在将移动端的画面重新布局至大屏设备时,可能会造成有关画面的其他信息丢失,导致用户无法获知画面的核心信息,影响用户的观看体验。
因此,亟需一种能够将移动端画面重新布局到大屏端且不会丢失画面相关信息的方法。
上述内容仅用于辅助理解本申请的技术方案,并不代表承认上述内容是现有技术。
技术问题
本申请的主要目的在于提供一种投屏显示方法,旨在解决将动端的画面重新布局至大屏设备时,可能会造成有关画面的其他信息丢失的技术问题。
技术解决方案
为实现上述目的,本申请提供一种投屏显示方法,所述投屏显示方法包括以下步骤:
获取投屏数据,所述投屏数据包括图像数据;
对所述图像数据进行黑边识别,根据所述黑边识别结果对所述图像数据进行裁剪,获取裁剪后的所述图像数据;
根据预设提取模型,提取所述图像数据中的附加信息;
将所述附加信息整合到所述裁剪后的所述图像数据,获取整合后的所述图像数据。
在一实施例中,所述预设提取模型包括预设提取元素,所述根据预设提取模型,提取所述图像数据中的附加信息的步骤包括:
在所述图像数据中提取与所述预设提取元素对应的元素;
将所述提取的元素作为所述附加信息。
在一实施例中,在所述根据预设提取模型,提取所述图像数据中的附加信息的步骤之后,所述方法还包括:
对所述附加信息进行图像处理,所述图像处理包括色彩处理、轮廓处理、蒙版处理中的一种或多种。
在一实施例中,所述将所述附加信息整合到所述裁剪后的所述图像数据,获取整合后的所述图像数据的步骤包括:
将处理后的所述附加信息放置于所述裁剪后的所述图像数据的预设位置,以获取整合后的所述图像数据。
在一实施例中,在所述获取整合后的所述图像数据的步骤之后,包括:
将整合后的所述图像数据显示,并在显示预设时长后,将所述附加信息隐藏,且触发唤醒操作时重新显示所述附加信息。
在一实施例中,所述对所述图像数据进行黑边识别,根据所述黑边识别结果对所述图像数据进行裁剪的步骤包括:
从所述图像数据中获取第一画面和第二画面,并分别对所述第一画面和所述第二画面进行黑边识别,确定所述第一画面的第一裁剪位置和所述第二画面的第二裁剪位置;
根据所述第一裁剪位置和所述第二裁剪位置,生成目标裁剪位置,并根据所述目标裁剪位置对所述图像数据进行裁剪。
在一实施例中,所述分别对所述第一画面和所述第二画面进行黑边识别的步骤包括:
对所述第一画面的像素点色值进行比对,将所述第一画面的像素点分成第一比对单元,将所述第一比对单元中像素点色值变化程度超过预设阈值的位置识别为黑边并作为所述第一裁剪位置;
对所述第二画面的像素点色值进行比对,将所述第二画面的像素点分成第二比对单元,将所述第二比对单元中像素点色值变化程度超过预设阈值的位置识别为黑边界线并作为所述第二裁剪位置。
在一实施例中,从所述图像数据中获取第一画面和第二画面为从投屏数据中图像数据获取两张图片。
在一实施例中,所述对所述图像数据进行黑边识别,根据所述黑边识别结果对所述图像数据进行裁剪的步骤包括:
识别第一画面中黑边的区域和实际播放画面的区域,将第一画面中黑边区域和实际播放画面区域的分界线作为第一裁剪位置;
识别第二画面中黑边的区域和实际播放画面的区域,将第二画面中黑边区域和实际播放画面区域的分界线作为第二裁剪位置;
判断第一裁剪位置和第二裁剪位置是否相同;
响应于所述第一裁剪位置和第二裁剪位置相同的情况,直接将第一裁剪位置或者第二裁剪位置作为上述目标位置;
响应于所述第一裁剪位置和第二裁剪位置不相同的情况,则将结合第一裁剪位置和第二裁剪位置生成平均裁剪位置,再根据目标裁剪位置将投屏数据中图像数据的黑边部分裁剪去除。
在一实施例中,所述色彩处理为对提取到文字元素和图形元素进行颜色填充,所述填充颜色可预先设置或根据在提取文字元素和图形元素的预设区域像素点色值确定;
所述轮廓处理为在图像识别技术无法识别出确定文字元素和图形元素时,对识别出的轮廓进行色彩处理后作为文字元素或者图形元素;
所述蒙版处理为附加信息文字元素和图形元素添加蒙版背景。
在一实施例中,所述根据所述第一裁剪位置和所述第二裁剪位置,生成目标裁剪位置,并根据所述目标裁剪位置对所述图像数据进行裁剪,包括:
判断第一裁剪位置中第一线与第二裁剪位置中第一线的位置是否相同,及判断第一裁剪位置中第二线与第二裁剪位置中第二线的位置是否相同;
响应于第一裁剪位置中第一线与第二裁剪位置中第一线的位置相同及第一裁剪位置中第二线与第二裁剪位置中第二线的位置相同,将第一裁剪位置或者第二裁剪位置中任意一个作为所述目标裁剪位置;
响应于第一裁剪位置中第一线与第二裁剪位置中第一线的位置不相同及第一裁剪位置中第二线与第二裁剪位置中第二线的位置不相同,取第一裁剪位置中第一线与第二裁剪位置中第一线的平均第一线,取第一裁剪位置中第二线与第二裁剪位置中第二线的平均第二线,所述目标裁剪位置包括平均第一线和平均第二线,根据所述目标裁剪位置的均第一线和平均第二线对图像数据进行裁剪。
此外,为实现上述目的,本申请还提供一种投屏显示装置,所述投屏显示装置包括:
获取模块,用于获取投屏数据,所述投屏数据包括图像数据;
裁剪模块,用于对所述图像数据进行黑边识别,根据所述黑边识别结果对所述图像数据进行裁剪,获取裁剪后的所述图像数据;
提取模块,用于根据预设提取模型,提取所述图像数据中的附加信息;
整合模块,用于将所述附加信息整合到所述裁剪后的所述图像数据,获取整合后的所述图像数据。
在一实施例中,所述预设提取模型包括预设提取区域,所述提取模块用于在所述图像数据中提取与所述预设提取区域对应的区域;以及将所述提取的区域作为所述附加信息。
此外,为实现上述目的,本申请还提供一种电子设备,所述电子设备包括:存储器、处理器及存储在所述存储器上并可在所述处理器上运行的投屏显示程序,所述投屏显示程序被所述处理器执行时实现如上述的投屏显示方法的步骤。
此外,为实现上述目的,本申请还提供一种可读存储介质,所述可读存储介质上存储有投屏显示程序,所述投屏显示程序被处理器执行时实现如上述的投屏显示方法的步骤。
有益效果
本申请实施例提出的一种投屏显示方法,在对投屏数据进行裁剪操作后,重新从投屏数据中的图像数据进行信息提取,并将提取到的附加信息整合至裁剪后的图形数据,使得图像数据在显示时相关信息是完整的,避免核心信息丢失,提升了用户的观看的体验。
附图说明
图1为本申请投屏显示方法中投屏连接场景示意图;
图2为本申请实施例方案涉及的硬件运行环境的电子设备结构示意图;
图3为本申请投屏显示方法第一实施例的流程示意图;
图4为本申请投屏显示方法中裁剪前第一画面在显示设备上的显示效果示意图;
图5为本申请投屏显示方法中图像数据在显示设备的显示画面示意图;
图6为本申请投屏显示方法中整合附加信息后图像数据的示意图;
图7为本申请投屏显示方法中一种可实施方式的场景示意图;
图8为本申请投屏显示方法涉及的装置结构示意图。
本申请目的的实现、功能特点及优点将结合实施例,参照附图做进一步说明。
本发明的实施方式
应当理解,此处所描述的具体实施例仅仅用以解释本申请,并不用于限定本申请。
本申请实施例的主要解决方案是:在对投屏数据进行裁剪操作后,重新从投屏数据中的图像数据进行信息提取,并将提取到的附加信息整合至裁剪后的图形数据。
由于投屏技术将移动端的画面重新布局至大屏设备时,可能会造成有关画面的其他信息丢失,导致用户无法获知画面的核心信息。
本申请提供一种解决方案,使得图像数据在显示时相关信息是完整的,避免核心信息丢失,提升了用户的观看的体验。
参照图1,图1为本申请实施例涉及的投屏连接示意图。
如图1所示,投屏器包括:投屏器发射器以及投屏器接收器,其中,第一电子设备可以是个人电脑,也可以是智能手机、平板电脑、电子书阅读器、便携计算机等可移动式终端设备,第二电子设备可以是电视,也可以是显示器,投影仪等具有显示功能的显示设备。
在一些实施例中,第一电子设备通过有线或无线网络连接投屏器发射器或投屏器接收器进行数据传输与交互;投屏器发射器通过无线网络连接投屏器接收器进行数据传输与交互,或者投屏器发射器通过无线网络连接第二电子设备进行数据传输与交互;投屏器接收器通过有线或无线网络连接第二电子设备进行数据传输与交互。在一可选实现方式中,投屏器接收器与第二电子设备之间以及投屏器发射器与第一电子设备之间的有线连接方式,可以包括基于HDMI(High Definition Multimedia Interface,高清多媒体接口)、USB(Universal Serial Bus,通用串行总线)、DP(DisplayPort,显示接口)和VGA(Video Graphics Array,视频图形阵列)等有线连接。第一电子设备与投屏器发射器和投屏器接收器之间、第二电子设备与投屏器发射器和投屏器接收器之间的无线通信方式,可以包括蓝牙(Bluetooth)、无线宽带(Wi-Fi)、红外通讯等无线通信。
所以,第一电子设备、投屏器发射器、投屏器接收器和第二电子设备之间实现投屏时可能的连接方式至少包括:
(1)投屏器发射器与投屏器接收器连接,第一电子设备画面经过投屏器发射器、投屏器接收器最终投屏到第二电子设备;
(2)投屏器发射器与第二电子设备无线连接,第一电子设备画面经过投屏器发射器投屏到第二电子设备,此时投屏器发射器与第二电子设备内置的无线通信模块通过无线网络相连;
(3)第一电子设备与投屏器接收器连接,第一电子设备画面经过投屏器接收器投屏到第二电子设备。
如图2所示,图2是本申请实施例方案涉及的硬件运行环境的终端结构示意图。
本申请实施例电子设备可以是投屏器,也可以是电视、手机、PC、平板电脑、便携计算机等具有数据接收、数据处理和数据发送功能的电子设备。
如图1所示,该设备可以包括:处理器1001,例如CPU,网络接口1004,用户接口1003,存储器1005,通信总线1002。其中,通信总线1002用于实现这些组件之间的连接通信。用户接口1003可以包括显示屏(Display)、输入单元比如键盘(Keyboard),可选用户接口1003还可以包括标准的有线接口、无线接口。网络接口1004在一实施例中可以包括标准的有线接口或无线接口(如WI-FI接口)。存储器1005可以是高速RAM存储器,也可以是稳定的存储器(non-volatile memory),例如磁盘存储器。存储器1005还可以是独立于前述处理器1001的存储装置。
在一实施例中,设备还可以包括摄像头、RF(Radio Frequency,射频)电路,传感器、音频电路、WiFi模块等等。其中,传感器比如光传感器、运动传感器以及其他传感器。具体地,光传感器可包括环境光传感器及接近传感器,其中,环境光传感器可根据环境光线的明暗来调节显示屏的亮度,接近传感器可在移动终端移动到耳边时,关闭显示屏和/或背光。作为运动传感器的一种,重力加速度传感器可检测各个方向上(一般为三轴)加速度的大小,静止时可检测出重力的大小及方向,可用于识别移动终端姿态的应用(比如横竖屏切换、相关游戏、磁力计姿态校准)、振动识别相关功能(比如计步器、敲击)等;当然,移动设备还可配置陀螺仪、气压计、湿度计、温度计、红外线传感器等其他传感器,在此不再赘述。
本领域技术人员可以理解,图1中示出的设备结构并不构成对设备的限定,可以包括比图示更多或更少的部件,或者组合某些部件,或者不同的部件布置。
如图2所示,作为一种计算机存储介质的存储器1005中可以包括操作***、网络通信模块、用户接口模块以及投屏显示程序。
在图2所示的终端中,网络接口1004主要用于连接后台服务器,与后台服务器进行数据通信;用户接口1003主要用于连接客户端(用户端),与客户端进行数据通信;而处理器1001可以用于调用存储器1005中存储的投屏显示程序,并执行以下操作:
获取投屏数据,所述投屏数据包括图像数据;
对所述图像数据进行黑边识别,根据所述黑边识别结果对所述图像数据进行裁剪,获取裁剪后的所述图像数据;
根据预设提取模型,提取所述图像数据中的附加信息;
将所述附加信息整合到所述裁剪后的所述图像数据,获取整合后的所述图像数据。
在一实施例中,处理器1001可以调用存储器1005中存储的投屏显示程序,还执行以下操作:
所述预设提取模型包括预设提取区域,所述根据预设提取模型,提取所述图像数据中的附加信息的步骤包括:
在所述图像数据中提取与所述预设提取区域对应的区域;
将所述提取的区域作为所述附加信息。
在一实施例中,处理器1001可以调用存储器1005中存储的投屏显示程序,还执行以下操作:
所述预设提取模型包括预设提取元素,所述根据预设提取模型,提取所述图像数据中的附加信息的步骤包括:
在所述图像数据中提取与所述预设提取元素对应的元素;
将所述提取的元素作为所述附加信息。
在一实施例中,处理器1001可以调用存储器1005中存储的投屏显示程序,还执行以下操作:
在所述根据预设提取模型,提取所述图像数据中的附加信息的步骤之后,所述方法还包括:
对所述附加信息进行图像处理,所述图像处理包括色彩处理、轮廓处理、蒙版处理中的一种或多种。
在一实施例中,处理器1001可以调用存储器1005中存储的投屏显示程序,还执行以下操作:
所述将所述附加信息整合到所述裁剪后的所述图像数据,获取整合后的所述图像数据的步骤包括:
将处理后的所述附加信息放置于所述裁剪后的所述图像数据的预设位置,以获取整合后的所述图像数据。
在一实施例中,处理器1001可以调用存储器1005中存储的投屏显示程序,还执行以下操作:
在所述获取整合后的所述图像数据的步骤之后,包括:
将整合后的所述图像数据显示,并在显示预设时长后,将所述附加信息隐藏,且触发唤醒操作时重新显示所述附加信息。
在一实施例中,处理器1001可以调用存储器1005中存储的投屏显示程序,还执行以下操作:
所述对所述图像数据进行黑边识别,根据所述黑边识别结果对所述图像数据进行裁剪的步骤包括:
从所述图像数据中获取第一画面和第二画面,并分别对所述第一画面和所述第二画面进行黑边识别,确定所述第一画面的第一裁剪位置和所述第二画面的第二裁剪位置;
根据所述第一裁剪位置和所述第二裁剪位置,生成目标裁剪位置,并根据所述目标裁剪位置对所述图像数据进行裁剪。
在一实施例中,处理器1001可以调用存储器1005中存储的投屏显示程序,还执行以下操作:
所述分别对所述第一画面和所述第二画面进行黑边识别的步骤包括:
对所述第一画面的像素点色值进行比对,将所述第一画面的像素点分成第一比对单元,将所述第一比对单元中像素点色值变化程度超过预设阈值的位置识别为黑边并作为所述第一裁剪位置;
对所述第二画面的像素点色值进行比对,将所述第二画面的像素点分成第二比对单元,将所述第二比对单元中像素点色值变化程度超过预设阈值的位置识别为黑边界线并作为所述第二裁剪位置。
参照图3,本申请投屏显示方法中的第一实施例,所述投屏显示方法包括:
步骤S10,获取投屏数据,所述投屏数据包括图像数据;
在本实施例中,实施主体可为参与投屏过程的智能电子设备,如投屏器发射器、投屏器接收器、电视、手机等,本实施例以投屏器为最优实施例进行说明。投屏器获取投屏数据,投屏数据通常可以包括视频数据或者若干连续的图片等,此外投屏数据还可以包括音频数据。投屏器从投屏数据获取图像数据,可以理解的是,若投屏数据为视频数据则从中截取图像,若为图形数据则直接接收即可。
步骤S20,对所述图像数据进行黑边识别,根据所述黑边识别结果对所述图像数据进行裁剪,获取裁剪后的所述图像数据;
在实际投屏过程中,若手机作为投屏数据的源设备,投屏数据中的图像数据可能存在大面积黑边情况,影响用观看体验,进而对图像数据进行黑边识别。
在一实施例中,从所述图像数据中获取第一画面和第二画面,并分别对所述第一画面和所述第二画面进行黑边识别,确定所述第一画面的第一裁剪位置和所述第二画面的第二裁剪位置;根据所述第一裁剪位置和所述第二裁剪位置,生成目标裁剪位置,并根据所述目标裁剪位置对所述图像数据进行裁剪。
具体的,在进行投屏时,投屏器将从投屏数据中图像数据获取两张图片,即第一画面和第二画面。识别第一画面中黑边的区域和实际播放画面的区域,将第一画面中黑边区域和实际播放画面区域的分界线作为第一裁剪位置,识别第二画面中黑边的区域和实际播放画面的区域,将第二画面中黑边区域和实际播放画面区域的分界线作为第二裁剪位置。判断第一裁剪位置和第二裁剪位置是否相同,若相同,则直接将第一裁剪位置或者第二裁剪位置作为上述目标位置,若不相同,则将结合第一裁剪位置和第二裁剪位置生成平均裁剪位置。再根据目标裁剪位置将投屏数据中图像数据的黑边部分裁剪去除。
在一实施例中,对所述第一画面的像素点色值进行比对,将所述第一画面的像素点分成第一比对单元,将所述第一比对单元中像素点色值变化程度超过预设阈值的位置识别为黑边并作为所述第一裁剪位置;对所述第二画面的像素点色值进行比对,将所述第二画面的像素点分成第二比对单元,将所述第二比对单元中像素点色值变化程度超过预设阈值的位置识别为黑边界线并作为所述第二裁剪位置。
具体的,以第一画面为例进行说明,第二画面处理过程与第一画面类似,后续不在赘述。第一画面由大量的像素点组成,在本实施例中对像素点进行色值(RGB,颜色值,如白色色值为:(255,255,255))比较时,将第一画面的像素点按不同划分规则分成不同的第一比对单元(如将一行像素点作为一个比对单元,则为第一行比对单元,将一列像素点作为一个比对单元,则为第一列比对单元),对各个比对单元的像素点色值进行比对,如将a比对单元与b比对单元进行比对,若两者色值不同程度超过预设阈值,则将a比对单元作为第一裁剪位置或者将b比对单元作为第一裁剪位置或者将a比对单元和b比对单元之间的位置作为第一裁剪位置。可以理解的是,对第一画面进行第一比对单元比对时,第一比对单元可以第一行比对单元,也可是第一列比对单元,或者两种比对单元分别比对一次。即划分成行比对单元时,可识别出分布在画面中上下侧的黑边,划分成列比对单元时,可识别出分布在画面中左右侧的黑边,两种比对单元分别比对一次,则可以识别出分布在画面四边的黑边。
以第一画面仅存在上下黑边的情况为例进行说明,且第一比对单元为行比对单元,按照第一预设比对顺序获取当前比对单元;将所述当前比对单元的像素点色值与所述当前比对单元的相邻比对单元的像素点色值进行比对;若所述当前比对单元的像素点色值与所述相邻比对单元的像素点色值不同的数量超过所述预设阈值,则将所述当前比对单元和所述相邻比对单元之间的位置作为所述第一裁剪位置;若所述当前比对单元的像素点色值与所述相邻比对单元的像素点色值不同未超过所述预设阈值,则重新获取所述当前比对单元,直至遍历所述第一画面的像素点。其中,所述第一预设比对顺序包括:从画面上下边缘向画面中间的顺序;或者,从所述画面中间向所述画面上下边缘的顺序。
如若获取到的第一画面包括如图4所示的1区和2区,其中,图4为裁剪前第一画面在显示设备上的显示效果,3区为显示设备对第一画面适应性显示产生黑边,第一线a和第二线b为1区和2区的分界线,1区为黑边2区为实际播放画面。第一比对单元为行比对单元,按第一预设的比较顺序,从第一画面中获取某一行的像素点作为当前比对行,进一步提取当前比对行的每个像素点的色值,与当前比对行的相邻行的像素点的色值进行比对。当前比对行中色值与相邻行色值不相同的像素点的数量占一行像素点总数的比值超过预设占比阈值时,将当前比对行和相邻行之间的间隔线作为上述裁剪线。如一行像素点的总量为100对应着100个色值,若预设占比阈值为百分之八十,则当当前比对行和相邻行存在81个色值不相同时(预设占比阈值可根据实际设置),即可根据当前比对行的位置和相邻行的位置确定第一裁剪位置,如图4中的第一线a和第二线b。若当前比对行中色值与相邻行色值不相同的像素点的数量占一行像素点总数的比值未超过预设占比阈值时,则重新按照预设第一比较顺序获取所述当前比对行与此时的当前比对行的相邻行进行像素点色值比对。直到第一画面的所有像素点均被以行的方式比对完成。
对应的,根据第一画面生成第一裁剪位置,根据第二画面生成第二裁剪位置,判断第一裁剪位置和第二裁剪位置是否相同,具体为,判断第一裁剪位置中第一线a与第二裁剪位置中第一线a的位置是否相同,判断第一裁剪位置中第二线b与第二裁剪位置中第二线b的位置是否相同。若相同,将第一裁剪位置或者第二裁剪位置中任意一个作为目标裁剪位置。若不相同,取第一裁剪位置中第一线a与第二裁剪位置中第一线a的平均第一线a,取第一裁剪位置中第二线b与第二裁剪位置中第二线b的平均第二线b,则目标裁剪位置包括平均第一线a和平均第二线b,根据目标裁剪位置的均第一线a和平均第二线b对图像数据进行裁剪,即若图像数据包括如图4所示的1区和2区,则将1区(黑边区),只留下2区(实际播放区)。可以理解的是,保留的实际播放区即为裁剪后的所述图像数据。
在本实施例中,具体的画面获取张数、比对单元的划分类别、比对顺序,均可根据实际情况设置,此处不再赘述。
步骤S30,根据预设提取模型,提取所述图像数据中的附加信息;
在一实施例中,所述预设提取模型包括预设提取区域,在所述图像数据中提取与所述预设提取区域对应的区域;将所述提取的区域作为所述附加信息。
可以理解的是,经过上述裁剪步骤后,图像数据实际被裁剪的部分可能包含了与图像数据相关的信息,例如,图像数据的标题、作者、自带的图标、简介等等信息,都将丢失。而预设提取模型则将提取上述丢失的信息。
具体的,预设提取模型可从上述第一画面、第二画面中任选取一张作为提取对象,也可重新从图像数据中再获取画面。如图5所示,为图像数据在显示设备的显示画面,包括播放区和黑边区,播放区显示图像数据,而播放区中的预设1区和预设2区即为预设提取模型中的预设提取区域。可以理解的是,图5所示预设1区和预设2区,并不对预设提取区域的位置、数量和大小做具体限制,预设区域的位置、数量和大小可根据实际情况设定。进一步将预设1区和预设2区的画面提取并作为附加信息。
在一实施例中,所述预设提取模型包括预设提取元素,在所述图像数据中提取与所述预设提取元素对应的元素;将所述提取的元素作为所述附加信息。
上述预设提取元素可以包括文字元素和图形元素,基于上述例子,预设提取模型对图5中的播放区进行元素提取,具体的文字元素和图形元素的提取可使用图像识别技术提取,将提取到的文字元素和图形元素作为裁剪后图像数据的附加信息。
在一实施例中,对所述附加信息进行图像处理,所述图像处理包括色彩处理、轮廓处理、蒙版处理中的一种或多种。
具体的,对附加信息可以是上述预设1与和预设2区,也可以是文字元素和图形元素等,图像处理包括色彩处理、轮廓处理、蒙版处理中的一种或多种。如色彩处理,对提取到文字元素和图形元素进行颜色填充,填充颜色可预先设置,也根据在提取文字元素和图形元素的预设区域像素点色值确定。轮廓处理为,若图像识别技术无法识别出确定文字元素和图形元素时,则直接识别出的轮廓进行色彩处理后作为文字元素或者图形元素。蒙版处理,为附加信息文字元素和图形元素添加蒙版背景,如设置固定颜色或者透明蒙版,避免文字元素和图形元素无法与实际播放区区分开,影响用户观看效果。
S40,将所述附加信息整合到所述裁剪后的所述图像数据,获取整合后的所述图像数据。
在一实施例中,将处理后的所述附加信息放置于所述裁剪后的所述图像数据的预设位置,以获取整合后的所述图像数据。
具体的,附加信息经上述步骤处理完成,将放置于裁剪后的图像数据中的预设位置,如图6所示,图6为整合附加信息后图像数据的示意图,包括标题文字区、简介信息区和图标区,保留画面区即为图像数据经过裁剪后保留的区域,标题文字区、简介信息区和图标区即为上述裁剪后的图像数据的预设位置,若附加信息为在文字元素和图形元素,则文字元素对应放置于标题文字区和简介信息区,图形元素放置于图标区,若附加信息为上述预设1区或者预设2区,则直接将上述预设1区或者预设2区的画面放置于裁剪后的图像数据的预设位置。可以理解的是,图6所示标题文字区、简介信息区和图标区,也不对预设位置的大小、数量以及处于画面中的区域进行限制,均可根据实际情况设置。
在一实施例中,在所述获取整合后的所述图像数据的步骤之后,包括:将整合后的所述图像数据显示,并在显示预设时长后,将所述附加信息隐藏,且触发唤醒操作时重新显示所述附加信息。
具体的,在整合后的图像数据显示时,附加信息在一定程度上会对实际播放视频内容造成遮挡,尤其是在视频时间较长时,将影响用户的实际观看体验。对此,附加信息显示一定时长后(如1分钟),将附加信息进行隐藏,是的实际播放内容完全显示,提升用户观看体验。且在用户进行唤醒操作时重新显示附加信息,如用户对手机、投屏器或者电视机进行暂停、快进、倍速播放等操作,重新显示附加信息。
可以理解的是,本实施例中,在对投屏数据进行裁剪操作后,重新从投屏数据中的图像数据进行信息提取,并将提取到的附加信息整合至裁剪后的图形数据,使得图像数据在显示时相关信息是完整的,避免核心信息丢失,提升了用户的观看的体验。
在一种可实施的方式中,参照图7,在实际投屏场景中包括:手机100、投屏器发射器200、投屏器接收器300和电视机400,所述手机100和所述投屏器发射器200通信连接,所述投屏器接收器300与所述投屏器发射器200通信连接,所述投屏器接收器300与所述电视机400通讯连接。在投屏时,投屏器发射器200获取手机100推送的投屏数据,对投屏数据中的图像数据进行裁剪,并基于裁剪前的图像数据生成附件加信息,再将生成附件加信息整合至裁剪后的图像数据并推送至投屏器接收器300,投屏器接收器300将整合后的图像数据推送至电视机400显示。
此外,本实施例中还提供一种投屏显示装置,如图8所示,所述投屏显示装置包括:
获取模块10,用于获取投屏数据,所述投屏数据包括图像数据;
裁剪模块20,用于对所述图像数据进行黑边识别,根据所述黑边识别结果对所述图像数据进行裁剪,获取裁剪后的所述图像数据;
提取模块30,用于根据预设提取模型,提取所述图像数据中的附加信息;
整合模块40,用于将所述附加信息整合到所述裁剪后的所述图像数据,获取整合后的所述图像数据。
在一实施例中,所述预设提取模型包括预设提取区域,所述提取模块30,还用于:
在所述图像数据中提取与所述预设提取区域对应的区域;
将所述提取的区域作为所述附加信息。
在一实施例中,所述预设提取模型包括预设提取元素,所述提取模块30,还用于:
在所述图像数据中提取与所述预设提取元素对应的元素;
将所述提取的元素作为所述附加信息。
在一实施例中,所述提取模块30,还用于:
对所述附加信息进行图像处理,所述图像处理包括色彩处理、轮廓处理、蒙版处理中的一种或多种。
在一实施例中,所述整合模块40,还用于:
将处理后的所述附加信息放置于所述裁剪后的所述图像数据的预设位置,以获取整合后的所述图像数据。
在一实施例中,所述整合模块40,还用于:
将整合后的所述图像数据显示,并在显示预设时长后,将所述附加信息隐藏,且触发唤醒操作时重新显示所述附加信息。
在一实施例中,所述裁剪模块20,还用于:
从所述图像数据中获取第一画面和第二画面,并分别对所述第一画面和所述第二画面进行黑边识别,确定所述第一画面的第一裁剪位置和所述第二画面的第二裁剪位置;
根据所述第一裁剪位置和所述第二裁剪位置,生成目标裁剪位置,并根据所述目标裁剪位置对所述图像数据进行裁剪。
在一实施例中,所述裁剪模块20,还用于:
对所述第一画面的像素点色值进行比对,将所述第一画面的像素点分成第一比对单元,将所述第一比对单元中像素点色值变化程度超过预设阈值的位置识别为黑边并作为所述第一裁剪位置;
对所述第二画面的像素点色值进行比对,将所述第二画面的像素点分成第二比对单元,将所述第二比对单元中像素点色值变化程度超过预设阈值的位置识别为黑边界线并作为所述第二裁剪位置。
本申请提供的投屏显示装置,采用上述实施例中的投屏方法,解决了将动端的画面重新布局至大屏设备时,可能会造成有关画面的其他信息丢失的技术问题。与现有技术相比,本申请实施例提供的投屏显示装置的有益效果与上述实施例提供的投屏方法的有益效果相同,且该投屏显示装置中的其他技术特征与上述实施例方法公开的特征相同,在此不做赘述。
此外,本实施例中还提供一种电子设备,所述电子设备包括:存储器、处理器及存储在所述存储器上并可在所述处理器上运行的投屏显示程序,所述投屏显示程序被所述处理器执行时实现如上述的投屏显示方法的步骤。
本申请电子设备的具体实施方式与上述投屏显示方法各实施例基本相同,在此不再赘述。
此外,本实施例中还提供一种可读存储介质,所述可读存储介质上存储有投屏显示程序,所述投屏显示程序被处理器执行时实现如上述的投屏显示方法的步骤。
本申请介质具体实施方式与上述投屏显示方法各实施例基本相同,在此不再赘述。
需要说明的是,在本文中,术语“包括”、“包含”或者其任何其他变体意在涵盖非排他性的包含,从而使得包括一系列要素的过程、方法、物品或者***不仅包括那些要素,而且还包括没有明确列出的其他要素,或者是还包括为这种过程、方法、物品或者***所固有的要素。在没有更多限制的情况下,由语句“包括一个……”限定的要素,并不排除在包括该要素的过程、方法、物品或者***中还存在另外的相同要素。
上述本申请实施例序号仅仅为了描述,不代表实施例的优劣。
通过以上的实施方式的描述,本领域的技术人员可以清楚地了解到上述实施例方法可借助软件加必需的通用硬件平台的方式来实现,当然也可以通过硬件,但很多情况下前者是更佳的实施方式。基于这样的理解,本申请的技术方案本质上或者说对现有技术做出贡献的部分可以以软件产品的形式体现出来,该计算机软件产品存储在如上所述的一个存储介质(如ROM/RAM、磁碟、光盘)中,包括若干指令用以使得一台终端设备(可以是手机,计算机,服务器,投屏器,或者网络设备等)执行本申请各个实施例所述的方法。
以上仅为本申请的优选实施例,并非因此限制本申请的专利范围,凡是利用本申请说明书及附图内容所作的等效结构或等效流程变换,或直接或间接运用在其他相关的技术领域,均同理包括在本申请的专利保护范围内。

Claims (15)

  1. 一种投屏显示方法,其中,所述投屏显示方法包括以下步骤:
    获取投屏数据,所述投屏数据包括图像数据;
    对所述图像数据进行黑边识别,根据所述黑边识别结果对所述图像数据进行裁剪,获取裁剪后的所述图像数据;
    根据预设提取模型,提取所述图像数据中的附加信息;
    将所述附加信息整合到所述裁剪后的所述图像数据,获取整合后的所述图像数据。
  2. 如权利要求1所述的投屏显示方法,其中,所述预设提取模型包括预设提取区域,所述根据预设提取模型,提取所述图像数据中的附加信息的步骤包括:
    在所述图像数据中提取与所述预设提取区域对应的区域;
    将所述提取的区域作为所述附加信息。
  3. 如权利要求1所述的投屏显示方法,其中,所述预设提取模型包括预设提取元素,所述根据预设提取模型,提取所述图像数据中的附加信息的步骤包括:
    在所述图像数据中提取与所述预设提取元素对应的元素;
    将所述提取的元素作为所述附加信息。
  4. 如权利要求1所述的投屏显示方法,其中,在所述根据预设提取模型,提取所述图像数据中的附加信息的步骤之后,所述方法还包括:
    对所述附加信息进行图像处理,所述图像处理包括色彩处理、轮廓处理、蒙版处理中的一种或多种。
  5. 如权利要求4所述的投屏显示方法,其中,所述将所述附加信息整合到所述裁剪后的所述图像数据,获取整合后的所述图像数据的步骤包括:
    将处理后的所述附加信息放置于所述裁剪后的所述图像数据的预设位置,以获取整合后的所述图像数据。
  6. 如权利要求1所述的投屏显示方法,其中,在所述获取整合后的所述图像数据的步骤之后,包括:
    将整合后的所述图像数据显示,并在显示预设时长后,将所述附加信息隐藏,且触发唤醒操作时重新显示所述附加信息。
  7. 如权利要求1所述的投屏显示方法,其中,所述对所述图像数据进行黑边识别,根据所述黑边识别结果对所述图像数据进行裁剪的步骤包括:
    从所述图像数据中获取第一画面和第二画面,并分别对所述第一画面和所述第二画面进行黑边识别,确定所述第一画面的第一裁剪位置和所述第二画面的第二裁剪位置;
    根据所述第一裁剪位置和所述第二裁剪位置,生成目标裁剪位置,并根据所述目标裁剪位置对所述图像数据进行裁剪。
  8. 如权利要求7所述的投屏显示方法,其中,所述分别对所述第一画面和所述第二画面进行黑边识别的步骤包括:
    对所述第一画面的像素点色值进行比对,将所述第一画面的像素点分成第一比对单元,将所述第一比对单元中像素点色值变化程度超过预设阈值的位置识别为黑边并作为所述第一裁剪位置;
    对所述第二画面的像素点色值进行比对,将所述第二画面的像素点分成第二比对单元,将所述第二比对单元中像素点色值变化程度超过预设阈值的位置识别为黑边界线并作为所述第二裁剪位置。
  9. 如权利要求7所述的投屏显示方法,其中,从所述图像数据中获取第一画面和第二画面为从投屏数据中图像数据获取两张图片。
  10. 如权利要求7所述的投屏显示方法,其中,所述对所述图像数据进行黑边识别,根据所述黑边识别结果对所述图像数据进行裁剪的步骤包括:
    识别第一画面中黑边的区域和实际播放画面的区域,将第一画面中黑边区域和实际播放画面区域的分界线作为第一裁剪位置;
    识别第二画面中黑边的区域和实际播放画面的区域,将第二画面中黑边区域和实际播放画面区域的分界线作为第二裁剪位置;
    判断第一裁剪位置和第二裁剪位置是否相同;
    响应于所述第一裁剪位置和第二裁剪位置相同的情况,直接将第一裁剪位置或者第二裁剪位置作为上述目标位置;
    响应于所述第一裁剪位置和第二裁剪位置不相同的情况,则将结合第一裁剪位置和第二裁剪位置生成平均裁剪位置,再根据目标裁剪位置将投屏数据中图像数据的黑边部分裁剪去除。
  11. 如权利要求4所述的投屏显示方法,其中,所述色彩处理为对提取到文字元素和图形元素进行颜色填充,所述填充颜色可预先设置或根据在提取文字元素和图形元素的预设区域像素点色值确定;
    所述轮廓处理为在图像识别技术无法识别出确定文字元素和图形元素时,对识别出的轮廓进行色彩处理后作为文字元素或者图形元素;
    所述蒙版处理为附加信息文字元素和图形元素添加蒙版背景。
  12. 如权利要求7所述的投屏显示方法,其中,所述根据所述第一裁剪位置和所述第二裁剪位置,生成目标裁剪位置,并根据所述目标裁剪位置对所述图像数据进行裁剪,包括:
    判断第一裁剪位置中第一线与第二裁剪位置中第一线的位置是否相同,及判断第一裁剪位置中第二线与第二裁剪位置中第二线的位置是否相同;
    响应于第一裁剪位置中第一线与第二裁剪位置中第一线的位置相同及第一裁剪位置中第二线与第二裁剪位置中第二线的位置相同,将第一裁剪位置或者第二裁剪位置中任意一个作为所述目标裁剪位置;
    响应于第一裁剪位置中第一线与第二裁剪位置中第一线的位置不相同及第一裁剪位置中第二线与第二裁剪位置中第二线的位置不相同,取第一裁剪位置中第一线与第二裁剪位置中第一线的平均第一线,取第一裁剪位置中第二线与第二裁剪位置中第二线的平均第二线,所述目标裁剪位置包括平均第一线和平均第二线,根据所述目标裁剪位置的均第一线和平均第二线对图像数据进行裁剪。
  13. 一种投屏显示装置,其中,所述投屏显示装置包括:
    获取模块,用于获取投屏数据,所述投屏数据包括图像数据;
    裁剪模块,用于对所述图像数据进行黑边识别,根据所述黑边识别结果对所述图像数据进行裁剪,获取裁剪后的所述图像数据;
    提取模块,用于根据预设提取模型,提取所述图像数据中的附加信息;
    整合模块,用于将所述附加信息整合到所述裁剪后的所述图像数据,获取整合后的所述图像数据。
  14. 如权利要求13所述的投屏显示装置,其中,所述预设提取模型包括预设提取区域,所述提取模块用于在所述图像数据中提取与所述预设提取区域对应的区域;以及将所述提取的区域作为所述附加信息。
  15. 一种电子设备,其中,所述电子设备包括:存储器、处理器及存储在所述存储器上并可在所述处理器上运行的投屏显示程序,所述投屏显示程序被所述处理器执行时实现如权利要求1至12中任一项所述的投屏显示方法的步骤。
PCT/CN2022/102415 2022-01-30 2022-06-29 投屏显示方法、装置、电子设备及可读存储介质 WO2023142370A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202210114218.4 2022-01-30
CN202210114218.4A CN114463359A (zh) 2022-01-30 2022-01-30 投屏显示方法、装置、电子设备及可读存储介质

Publications (1)

Publication Number Publication Date
WO2023142370A1 true WO2023142370A1 (zh) 2023-08-03

Family

ID=81411708

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/102415 WO2023142370A1 (zh) 2022-01-30 2022-06-29 投屏显示方法、装置、电子设备及可读存储介质

Country Status (2)

Country Link
CN (1) CN114463359A (zh)
WO (1) WO2023142370A1 (zh)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114463359A (zh) * 2022-01-30 2022-05-10 深圳创维-Rgb电子有限公司 投屏显示方法、装置、电子设备及可读存储介质

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6535633B1 (en) * 1999-09-24 2003-03-18 Bank One Method and apparatus for re-classifying color image pixels classified by single channel segmentation
CN106408518A (zh) * 2015-07-30 2017-02-15 展讯通信(上海)有限公司 图像融合方法、装置及终端设备
CN112995535A (zh) * 2021-02-05 2021-06-18 北京百度网讯科技有限公司 用于处理视频的方法、装置、设备以及存储介质
CN113221846A (zh) * 2021-06-07 2021-08-06 北京百度网讯科技有限公司 图像识别方法、装置、设备、存储介质以及程序产品
CN113891167A (zh) * 2021-08-27 2022-01-04 荣耀终端有限公司 投屏方法和电子设备
CN113905268A (zh) * 2021-09-28 2022-01-07 四川长虹电器股份有限公司 移动终端投屏显示的去黑边方法
CN113965813A (zh) * 2021-10-21 2022-01-21 广州方硅信息技术有限公司 直播间内的视频播放方法、***及计算机设备
CN114463359A (zh) * 2022-01-30 2022-05-10 深圳创维-Rgb电子有限公司 投屏显示方法、装置、电子设备及可读存储介质

Family Cites Families (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20170138279A (ko) * 2016-06-07 2017-12-15 엘지전자 주식회사 이동 단말기 및 그 제어방법
CN107862312A (zh) * 2017-11-22 2018-03-30 朱秋华 一种基于tesseract引擎的文字识别方法、装置、设备及存储介质
CN109829068A (zh) * 2019-03-11 2019-05-31 重庆菜鸽途讯科技有限公司 一种信息获取方法、终端及计算机可读存储介质
CN109919726A (zh) * 2019-03-11 2019-06-21 重庆菜鸽途讯科技有限公司 一种商业***
CN110232713B (zh) * 2019-06-13 2022-09-20 腾讯数码(天津)有限公司 一种图像目标定位修正方法及相关设备
CN112351331A (zh) * 2020-10-27 2021-02-09 深圳Tcl新技术有限公司 消息提示方法、装置、显示设备及计算机可读存储介质
CN112307967B (zh) * 2020-10-30 2024-06-11 浪潮金融信息技术有限公司 一种商品识别模型训练优化方法
CN112887781A (zh) * 2021-01-27 2021-06-01 维沃移动通信有限公司 字幕处理方法及装置
CN113206964B (zh) * 2021-04-29 2022-07-26 深圳康佳电子科技有限公司 一种投屏数据显示处理方法、装置、终端设备及存储介质

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6535633B1 (en) * 1999-09-24 2003-03-18 Bank One Method and apparatus for re-classifying color image pixels classified by single channel segmentation
CN106408518A (zh) * 2015-07-30 2017-02-15 展讯通信(上海)有限公司 图像融合方法、装置及终端设备
CN112995535A (zh) * 2021-02-05 2021-06-18 北京百度网讯科技有限公司 用于处理视频的方法、装置、设备以及存储介质
CN113221846A (zh) * 2021-06-07 2021-08-06 北京百度网讯科技有限公司 图像识别方法、装置、设备、存储介质以及程序产品
CN113891167A (zh) * 2021-08-27 2022-01-04 荣耀终端有限公司 投屏方法和电子设备
CN113905268A (zh) * 2021-09-28 2022-01-07 四川长虹电器股份有限公司 移动终端投屏显示的去黑边方法
CN113965813A (zh) * 2021-10-21 2022-01-21 广州方硅信息技术有限公司 直播间内的视频播放方法、***及计算机设备
CN114463359A (zh) * 2022-01-30 2022-05-10 深圳创维-Rgb电子有限公司 投屏显示方法、装置、电子设备及可读存储介质

Also Published As

Publication number Publication date
CN114463359A (zh) 2022-05-10

Similar Documents

Publication Publication Date Title
US20210281771A1 (en) Video processing method, electronic device and non-transitory computer readable medium
US9892716B2 (en) Image display program, image display method, and image display system
JP7270661B2 (ja) ビデオの処理方法および装置、電子機器、記憶媒体並びにコンピュータプログラム
US20240036792A1 (en) Picture displaying method and apparatus, and electronic device
EP3043343A1 (en) Information processing device, information processing method, and program
CN107371062B (zh) 视频画面自动缩放的方法
CN114450969B (zh) 视频截屏方法、终端及计算机可读存储介质
CN110796664B (zh) 图像处理方法、装置、电子设备及计算机可读存储介质
CN111083568A (zh) 视频数据处理方法及电子设备
CN114428597A (zh) 多路终端投屏控制方法、装置、投屏器及存储介质
CN113126862A (zh) 截屏方法、装置、电子设备以及可读存储介质
CN111176526B (zh) 图片显示方法和电子设备
WO2019119643A1 (zh) 移动直播的互动终端、方法及计算机可读存储介质
WO2023142370A1 (zh) 投屏显示方法、装置、电子设备及可读存储介质
CN114463358A (zh) 投屏显示方法、装置、电子设备及可读存储介质
US20140146086A1 (en) Image output apparatus, image output method, and program
CN114489546B (zh) 分屏展示方法、电子设备、存储介质及装置
CN114489549B (zh) 投屏图像处理方法、装置、电子设备及存储介质
TWI790560B (zh) 並排影像偵測方法與使用該方法的電子裝置
US11373340B2 (en) Display apparatus and controlling method thereof
CN114242023A (zh) 显示屏亮度调节方法、显示屏亮度调节装置和电子设备
CN112788425A (zh) 动态区域显示方法、装置、设备及计算机可读存储介质
CN113223186A (zh) 实现增强现实的处理方法、设备、产品和装置
CN112650872A (zh) 动态图片播放方法、装置、设备及计算机可读存储介质
US11467798B2 (en) Display apparatus for changing an advertisement area, server, electronic apparatus and control methods thereof

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22923178

Country of ref document: EP

Kind code of ref document: A1