WO2020244497A1 - Display method for flexible screen and electronic device - Google Patents
Display method for flexible screen and electronic device Download PDFInfo
- Publication number
- WO2020244497A1 WO2020244497A1 PCT/CN2020/093899 CN2020093899W WO2020244497A1 WO 2020244497 A1 WO2020244497 A1 WO 2020244497A1 CN 2020093899 W CN2020093899 W CN 2020093899W WO 2020244497 A1 WO2020244497 A1 WO 2020244497A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- screen
- interface
- electronic device
- controls
- application
- Prior art date
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
- G06F3/0487—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
- G06F3/0488—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/451—Execution arrangements for user interfaces
Definitions
- This application relates to the field of terminal technology, and in particular to a display method and electronic device of a flexible screen.
- Flexible screens can also be called flexible OLEDs (organic light-emitting diodes). Compared with traditional screens, flexible screens are not only lighter and thinner in volume, but also based on their bendability and flexibility. The durability is also much higher than traditional screens.
- the present application provides a display method and electronic device for a flexible screen, which can re-lay out various controls in the display interface according to the physical state of the flexible screen, so that the user can get a better experience when watching and using the flexible screen.
- the present application provides a method for displaying a flexible screen.
- the flexible screen can be divided into a first screen and a second screen by a folding line when folded.
- the above method includes: When the included angle is less than the first preset value (for example, 90°) or greater than the second preset value (for example, 180°), the electronic device may display the first interface on the first screen and/or the second screen; when the first screen is detected When the angle between the first screen and the second screen is within the closed interval formed by the first preset value and the second preset value, the flexible screen is in the unfolded state or the stand state.
- the electronic device can determine that the Further, the electronic device can display a second interface on the first screen, and the second interface includes operation controls; and, the electronic device can display a third interface on the second screen, and the third interface Including non-operational controls.
- the electronic device can display the operating controls and non-operating controls in the application interface being displayed on the first screen and the second screen of the flexible screen. in.
- the operating controls after the split-screen display will not block the non-operating controls, so that the user can obtain a better immersive visual and operating experience.
- the method further includes: the electronic device queries the first Whether the interface is a preset application interface; when the first interface is a preset application interface (such as a game interface), it means that there may be a need for split-screen display when the application is running, and the electronic device may further determine Operational controls and non-operational controls.
- the electronic device queries the first Whether the interface is a preset application interface; when the first interface is a preset application interface (such as a game interface), it means that there may be a need for split-screen display when the application is running, and the electronic device may further determine Operational controls and non-operational controls.
- the method further includes: the electronic device determines the first screen And the second screen are in the landscape state; when the first screen and the second screen are in the landscape state, it means that there may be a need for split-screen display when the application is running, and the electronic device can further determine the Operational controls and non-operational controls.
- the electronic device determining the operating controls and non-operating controls in the first interface includes: the electronic device can obtain the position and size of each control in the first interface; further, the electronic device can set the size larger than The control of the preset size is determined as a non-operation control, and the control located on the layer where the non-operation control is located is determined as the operation control.
- the electronic device may further determine whether the layer where the first control is located is the topmost layer. If the layer where the first control is located is not the topmost layer, the electronic device may confirm the first control as a non-operation control.
- the electronic device can recognize the control with a larger display size and close to the underlying layer as a non-operational control, and recognize the control above the non-operational control as an operational control according to the size and positional relationship of each control in the interface.
- the electronic device determining operation controls and non-operation controls in the first interface includes: the electronic device obtains a preset configuration file corresponding to the first interface, and the first interface is recorded in the configuration file The display position of the operation control in the middle; then, the electronic device can recognize the operation control in the first interface according to the configuration file, and determine the control except the operation control in the first interface as a non-operation control.
- the second screen is a screen facing the user.
- the second screen facing the user can display non-operational controls that do not require operation and interaction for the user, and the user can operate the above-mentioned operation controls on another screen (ie, the first screen), so that the user can get immersive viewing And operating experience.
- the electronic device may also call a camera, an infrared sensor, a proximity light sensor, or a touch device to identify the specific orientation of the first screen and the second screen in the flexible screen.
- the electronic device can use acceleration sensors and other devices to detect the angle between the first screen and the second screen and the horizontal plane. When the angle between the first screen and the horizontal plane is less than a certain angle, the second screen can be determined to be the screen facing the user. .
- the method further includes: the electronic device acquires first view information for drawing the first interface (for example, the view tree of the first interface), and The view information includes the layer order between the controls in the first interface; further, the electronic device can generate second view information corresponding to the second interface and third view information corresponding to the third interface according to the first view information.
- first view information for drawing the first interface (for example, the view tree of the first interface)
- the view information includes the layer order between the controls in the first interface
- the electronic device can generate second view information corresponding to the second interface and third view information corresponding to the third interface according to the first view information.
- the second view information includes the layer order between the operation controls
- the third view information includes the layer order between the non-operation controls
- the electronic device displays the second interface on the first screen
- the electronic device Displaying the third interface on the second screen includes: the electronic device draws the second interface on the first screen according to the second view information; and the electronic device draws the third interface on the second screen according to the third view information.
- the layer order of the operation controls in the second view information is the same or different from the layer order in the first view information, that is, the layer order of the operation controls in the interface before and after the split screen can be the same or different. different.
- the layer order of the non-operating controls in the third view information is the same or different from the layer order in the first view information, that is, the layer order of the non-operating controls in the interface before and after the split screen can be the same or different .
- the configuration file corresponding to the first interface may also record the display position of the operation control in the second interface and the display position of the non-operation control in the third interface; at this time, the electronic device Generating the second view information corresponding to the second interface and the third view information corresponding to the third interface according to the first view information includes: the electronic device can follow the display position of the operation control recorded in the configuration file in the second interface, Split and reorganize the operation controls in the first view information to obtain the second view information; similarly, the electronic device can compare the first view information according to the display position of the non-operation controls recorded in the configuration file in the third interface. The non-operational controls in the control panel are split and reorganized to obtain the third view information.
- the method further includes: the electronic device receives a first touch event input by the user to the operation control on the second interface; The event is mapped to a second touch event in the first interface; the electronic device executes an operation instruction corresponding to the second touch event. That is, after each operation control is displayed on the second interface, the user can operate each operation control in the second interface to continue to complete the related functions of the operation control.
- the electronic device mapping the first touch event to the second touch event in the first interface includes: the electronic device obtains the coordinates of the first touch point in the first touch event; And the relative position relationship on the second interface, and calculate the coordinates of the second touch point corresponding to the first touch point in the first interface; wherein the electronic device executes the operation instruction corresponding to the second touch event, including: The coordinates of the two touch points are carried in the second touch event and reported to the first application (the first interface is the interface of the first application), so that the first application executes the operation instruction corresponding to the second touch event.
- the electronic device after the electronic device detects that the user inputs the first touch event in the second interface after the split screen, it can map the first touch event to the second touch event in the first interface before the split screen, so that related applications Finally, it can respond to the second touch event, that is, respond to the first touch event input by the user on the second interface.
- the present application provides an electronic device, including: a flexible screen that is divided into a first screen and a second screen by a folding line when folded; one or more processors; one or more memories; And one or more computer programs; wherein, the processor is coupled with the flexible screen and the memory, and the above one or more computer programs are stored in the memory.
- the processor executes one or more stored in the memory.
- a computer program to make an electronic device execute the flexible screen display method described in any one of the above.
- the present application provides a computer storage medium including computer instructions, which when the computer instructions run on an electronic device, cause the electronic device to execute the flexible screen display method as described in any one of the first aspect.
- this application provides a computer program product, which when the computer program product runs on an electronic device, causes the electronic device to execute the flexible screen display method described in any one of the first aspect.
- the electronic equipment described in the second aspect, the computer storage medium described in the third aspect, and the computer program product described in the fourth aspect provided above are all used to execute the corresponding methods provided above.
- the beneficial effects that can be achieved please refer to the beneficial effects in the corresponding method provided above, which will not be repeated here.
- FIG. 2 is a first structural diagram of an electronic device according to an embodiment of the application.
- FIG. 3 is a first structural diagram of a flexible screen in an electronic device provided by an embodiment of the application.
- FIG. 4 is a schematic diagram 2 of the structure of a flexible screen in an electronic device provided by an embodiment of the application;
- FIG. 5 is a schematic diagram 1 of an application scenario of a flexible screen display method provided by an embodiment of the application;
- FIG. 6 is a schematic diagram 2 of an application scenario of a flexible screen display method provided by an embodiment of the application;
- FIG. 7 is a schematic structural diagram of an operating system in an electronic device provided by an embodiment of this application.
- FIG. 8 is a third schematic diagram of an application scenario of a flexible screen display method provided by an embodiment of the application.
- FIG. 9 is a schematic diagram 4 of an application scenario of a flexible screen display method provided by an embodiment of the application.
- FIG. 10 is a schematic diagram 5 of an application scenario of a flexible screen display method provided by an embodiment of the application.
- FIG. 11 is a sixth schematic diagram of an application scenario of a flexible screen display method provided by an embodiment of the application.
- FIG. 12 is a schematic diagram 7 of an application scenario of a flexible screen display method provided by an embodiment of the application.
- FIG. 13 is an eighth schematic diagram of an application scenario of a flexible screen display method provided by an embodiment of this application.
- FIG. 14 is a schematic diagram 9 of an application scenario of a flexible screen display method provided by an embodiment of this application.
- 15 is a schematic diagram ten of an application scenario of a flexible screen display method provided by an embodiment of this application.
- 16 is a schematic diagram eleventh of an application scenario of a flexible screen display method provided by an embodiment of this application.
- FIG. 17 is a twelfth schematic diagram of an application scenario of a flexible screen display method provided by an embodiment of this application.
- 18 is a schematic diagram 13 of an application scenario of a flexible screen display method provided by an embodiment of this application.
- FIG. 19 is a fourteenth schematic diagram of an application scenario of a flexible screen display method provided by an embodiment of this application.
- 20 is a fifteenth schematic diagram of an application scenario of a flexible screen display method provided by an embodiment of this application.
- FIG. 21 is a sixteenth schematic diagram of an application scenario of a flexible screen display method provided by an embodiment of this application.
- FIG. 22 is a second structural diagram of an electronic device provided by an embodiment of this application.
- the flexible screen display method provided in the embodiments of this application can be applied to mobile phones, tablet computers, notebook computers, ultra-mobile personal computers (UMPC), handheld computers, netbooks, and personal digital assistants (personal digital assistants).
- UMPC ultra-mobile personal computers
- PDA personal digital assistants
- wearable electronic devices virtual reality devices, and other electronic devices
- the embodiments of the present application do not make any restrictions on this.
- FIG. 2 shows a schematic structural diagram of the electronic device 100.
- the electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, and an antenna 2.
- Mobile communication module 150 Wireless communication module 160, audio module 170, speaker 170A, receiver 170B, microphone 170C, earphone interface 170D, sensor module 180, camera 193, display screen 194, etc.
- the structure illustrated in the embodiment of the present invention does not constitute a specific limitation on the electronic device 100.
- the electronic device 100 may include more or fewer components than shown, or combine certain components, or split certain components, or arrange different components.
- the illustrated components can be implemented in hardware, software, or a combination of software and hardware.
- the processor 110 may include one or more processing units.
- the processor 110 may include an application processor (AP), a modem processor, a graphics processing unit (GPU), and an image signal processor. (image signal processor, ISP), controller, video codec, digital signal processor (digital signal processor, DSP), baseband processor, and/or neural-network processing unit (NPU), etc.
- AP application processor
- modem processor modem processor
- GPU graphics processing unit
- image signal processor image signal processor
- ISP image signal processor
- controller video codec
- digital signal processor digital signal processor
- DSP digital signal processor
- NPU neural-network processing unit
- the different processing units may be independent devices or integrated in one or more processors.
- a memory may also be provided in the processor 110 to store instructions and data.
- the memory in the processor 110 is a cache memory.
- the memory can store instructions or data that have just been used or recycled by the processor 110. If the processor 110 needs to use the instruction or data again, it can be directly called from the memory. Repeated accesses are avoided, the waiting time of the processor 110 is reduced, and the efficiency of the system is improved.
- the processor 110 may include one or more interfaces.
- the interface may include an integrated circuit (inter-integrated circuit, I2C) interface, an integrated circuit built-in audio (inter-integrated circuit sound, I2S) interface, a pulse code modulation (PCM) interface, and a universal asynchronous transmitter receiver/transmitter, UART) interface, mobile industry processor interface (MIPI), general-purpose input/output (GPIO) interface, subscriber identity module (SIM) interface, and / Or Universal Serial Bus (USB) interface, etc.
- I2C integrated circuit
- I2S integrated circuit built-in audio
- PCM pulse code modulation
- UART universal asynchronous transmitter receiver/transmitter
- MIPI mobile industry processor interface
- GPIO general-purpose input/output
- SIM subscriber identity module
- USB Universal Serial Bus
- the charging management module 140 is used to receive charging input from the charger.
- the charger can be a wireless charger or a wired charger.
- the charging management module 140 may receive the charging input of the wired charger through the USB interface 130.
- the charging management module 140 may receive the wireless charging input through the wireless charging coil of the electronic device 100. While the charging management module 140 charges the battery 142, it can also supply power to the electronic device through the power management module 141.
- the power management module 141 is used to connect the battery 142, the charging management module 140 and the processor 110.
- the power management module 141 receives input from the battery 142 and/or the charging management module 140, and supplies power to the processor 110, the internal memory 121, the display screen 194, the camera 193, and the wireless communication module 160.
- the power management module 141 may also be provided in the processor 110.
- the power management module 141 and the charging management module 140 may also be provided in the same device.
- the wireless communication function of the electronic device 100 can be implemented by the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor, and the baseband processor.
- the antenna 1 and the antenna 2 are used to transmit and receive electromagnetic wave signals.
- Each antenna in the electronic device 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve antenna utilization.
- antenna 1 can be multiplexed as a diversity antenna of a wireless local area network.
- the antenna can be used in combination with a tuning switch.
- the mobile communication module 150 can provide a wireless communication solution including 2G/3G/4G/5G and the like applied to the electronic device 100.
- the mobile communication module 150 may include one or more filters, switches, power amplifiers, low noise amplifiers (LNA), etc.
- the mobile communication module 150 can receive electromagnetic waves by the antenna 1, and perform processing such as filtering and amplifying the received electromagnetic waves, and then transmitting them to the modem processor for demodulation.
- the mobile communication module 150 can also amplify the signal modulated by the modem processor, and convert it into electromagnetic waves for radiation via the antenna 1.
- at least part of the functional modules of the mobile communication module 150 may be provided in the processor 110.
- at least part of the functional modules of the mobile communication module 150 and at least part of the modules of the processor 110 may be provided in the same device.
- the modem processor may include a modulator and a demodulator.
- the modulator is used to modulate the low frequency baseband signal to be sent into a medium and high frequency signal.
- the demodulator is used to demodulate the received electromagnetic wave signal into a low-frequency baseband signal. Then the demodulator transmits the demodulated low-frequency baseband signal to the baseband processor for processing.
- the low-frequency baseband signal is processed by the baseband processor and then passed to the application processor.
- the application processor outputs a sound signal through an audio device (not limited to the speaker 170A, the receiver 170B, etc.), or displays an image or video through the display screen 194.
- the modem processor may be an independent device.
- the modem processor may be independent of the processor 110 and be provided in the same device as the mobile communication module 150 or other functional modules.
- the wireless communication module 160 can provide applications on the electronic device 100 including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), and global navigation satellites.
- WLAN wireless local area networks
- BT Bluetooth
- GNSS global navigation satellite system
- FM frequency modulation
- NFC near field communication technology
- infrared technology infrared, IR
- the wireless communication module 160 may be one or more devices integrating one or more communication processing modules.
- the wireless communication module 160 receives electromagnetic waves via the antenna 2, frequency modulates and filters the electromagnetic wave signals, and sends the processed signals to the processor 110.
- the wireless communication module 160 can also receive the signal to be sent from the processor 110, perform frequency modulation, amplify it, and convert it into electromagnetic wave radiation via the antenna 2.
- the antenna 1 of the electronic device 100 is coupled with the mobile communication module 150, and the antenna 2 is coupled with the wireless communication module 160, so that the electronic device 100 can communicate with the network and other devices through wireless communication technology.
- the wireless communication technologies may include global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), broadband Code division multiple access (wideband code division multiple access, WCDMA), time-division code division multiple access (TD-SCDMA), long term evolution (LTE), BT, GNSS, WLAN, NFC , FM, and/or IR technology, etc.
- the GNSS may include global positioning system (GPS), global navigation satellite system (GLONASS), Beidou navigation satellite system (BDS), quasi-zenith satellite system (quasi -zenith satellite system, QZSS) and/or satellite-based augmentation systems (SBAS).
- GPS global positioning system
- GLONASS global navigation satellite system
- BDS Beidou navigation satellite system
- QZSS quasi-zenith satellite system
- SBAS satellite-based augmentation systems
- the electronic device 100 implements a display function through a GPU, a display screen 194, and an application processor.
- the GPU is a microprocessor for image processing, connected to the display 194 and the application processor.
- the GPU is used to perform mathematical and geometric calculations for graphics rendering.
- the processor 110 may include one or more GPUs, which execute program instructions to generate or change display information.
- the display screen 194 may specifically be a flexible screen.
- the flexible screen may include a display device and a touch sensor.
- the display device is used to output display content to the user, and the touch sensor is used to receive a touch operation input by the user on the flexible screen.
- the flexible screen 301 can be displayed as a complete display area in the unfolded state.
- the user can fold the screen along one or more folding lines in the flexible screen 301.
- the position of the folding line may be preset, or may be formed by the user after folding any position of the flexible screen 301.
- the flexible screen 301 can be divided into two display areas along the fold line AB, namely, display area 1 and display area 2.
- the folded display area 1 and the display area 2 can be displayed as two independent display areas.
- the display area 1 may be referred to as the main screen of the mobile phone 100
- the display area 2 may be referred to as the secondary screen of the mobile phone 100.
- the display area of the main screen and the secondary screen can be the same or different.
- the size of the flexible screen is 2200*2480 (in pixels).
- the width of the folding line AB on the flexible screen is 166. After folding along the folding line AB, the area on the right side of the flexible screen with a size of 1144*2480 is divided into the main screen 41, and the area on the left side of the flexible screen with a size of 890*2480 is divided into the secondary screen 42.
- the mobile phone 100 can calculate the angle between the main screen 41 and the sub screen 42 based on the data detected by one or more sensors.
- a gyroscope and an acceleration sensor may be provided on the main screen 41 and the auxiliary screen 42 of the flexible screen, respectively.
- the gyroscope on the main screen 41 can detect the rotation angular velocity when the main screen 41 rotates, and the acceleration sensor on the main screen 41 can detect the acceleration generated when the main screen 41 moves.
- the mobile phone 100 can determine the magnitude and direction of the gravity G according to the data detected by the gyroscope and the acceleration sensor on the main screen 41.
- the mobile phone 100 can also determine the magnitude and direction of the gravity G according to the data detected by the gyroscope and the acceleration sensor on the secondary screen 42.
- the sensor A can be provided on the main screen 41 and the sub screen 42 respectively.
- a gyroscope and an acceleration sensor can be integrated into one sensor and arranged on the main screen 41 and the auxiliary screen 42 respectively.
- corresponding coordinate systems may be set on the main screen 41 and the sub screen 42 respectively.
- a Cartesian coordinate system O1 can be set in the secondary screen 42.
- the x axis is parallel to the shorter side of the secondary screen 42
- the y axis is parallel to the longer side of the secondary screen 42
- the z axis is vertical.
- the plane formed by the x-axis and the y-axis points out of the secondary screen 42.
- a Cartesian coordinate system O2 can be set in the main screen 41.
- the x-axis is parallel to the shorter side of the main screen 41
- the y-axis is parallel to the longer side of the main screen 41
- the z-axis is perpendicular to the x-axis.
- the plane formed by the y axis points to the inside of the main screen 41.
- the gyroscope and acceleration sensor in the secondary screen 42 can detect the magnitude and direction of gravity G in the Cartesian coordinate system O1, and the gyroscope and acceleration sensor in the main screen 41 can detect the magnitude and direction of the gravity G in the Cartesian coordinate system O2.
- the magnitude and direction of gravity G Since the direction of the y-axis in the Cartesian coordinate system O1 and the Cartesian coordinate system O2 is the same, the component G1 of the gravity G on the x-axis and z-axis planes in the Cartesian coordinate system O1 is the same as the gravity G in the Cartesian coordinate system O2
- the components G2 on the x-axis and z-axis planes are equal in magnitude but different in direction.
- the angle between the component G1 and the component G2 is ⁇
- the electronic device 100 calculates the angle ⁇ between the component G1 of the gravity G in the Cartesian coordinate system O1 and the component G2 of the gravity G in the Cartesian coordinate system O2 to obtain the difference between the secondary screen 42 and the main screen 41 Angle ⁇ .
- the angle ⁇ between the secondary screen 42 and the main screen 41 may vary within a closed interval formed by 0 to 180°.
- the included angle ⁇ between the secondary screen 42 and the main screen 41 may vary in a closed interval formed by 180° to 360°.
- the included angle ⁇ between the secondary screen 42 and the main screen 41 may also vary within a closed interval formed by 0 to 360°, which is not limited in the embodiment of the present application.
- the flexible screen is in the unfolded state at this time.
- the angle between the main screen 41 and the secondary screen 42 is reduced from 180° to 170°, and the primary screen 41 and the secondary screen 42 are both in the landscape state, it can be defined that the flexible screen is in the support state at this time.
- the embodiments of the application do not impose any restriction on this.
- the horizontal screen state refers to a state where the height of the main screen 41 and the sub screen 42 is less than the width.
- the aspect ratio (that is, the ratio of height to width) of the display screen (for example, the main screen 41 or the sub screen 42) is less than 1.
- the height of the display can also be understood as the vertical length of the display
- the width of the display can also be understood as the horizontal length of the display
- the aspect ratio of the display can also be understood as the aspect ratio of the display (ie, the vertical length and The ratio of lateral length).
- the main screen 41 and the sub screen 42 may also be in a vertical screen state.
- the vertical screen state refers to the state where the height of the main screen 41 and the secondary screen 42 is greater than the width
- the aspect ratio (that is, the ratio of height to width) of the display screen is greater than 1.
- the electronic device 100 can determine whether the main screen 41 is currently in the landscape state or the portrait state through the acceleration sensor and the gyroscope on the main screen 41; similarly, the electronic device 100 can determine the secondary screen through the acceleration sensor and the gyroscope on the secondary screen 42.
- the screen 42 is currently in a landscape state or a portrait state.
- the electronic device 100 can implement a shooting function through an ISP, a camera 193, a video codec, a GPU, a display screen 194, and an application processor.
- the ISP is used to process the data fed back from the camera 193. For example, when taking a picture, the shutter is opened, the light is transmitted to the photosensitive element of the camera through the lens, the light signal is converted into an electrical signal, and the photosensitive element of the camera transfers the electrical signal to the ISP for processing and is converted into an image visible to the naked eye.
- ISP can also optimize the image noise, brightness, and skin color. ISP can also optimize the exposure, color temperature and other parameters of the shooting scene.
- the ISP may be provided in the camera 193.
- the camera 193 is used to capture still images or videos.
- the object generates an optical image through the lens and projects it to the photosensitive element.
- the light-sensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor.
- CMOS complementary metal-oxide-semiconductor
- the photosensitive element converts the optical signal into an electrical signal, and then transmits the electrical signal to the ISP to convert it into a digital image signal.
- ISP outputs digital image signals to DSP for processing.
- DSP converts digital image signals into standard RGB, YUV and other formats.
- the electronic device 100 may include 1 or N cameras 193, and N is a positive integer greater than 1.
- Digital signal processors are used to process digital signals. In addition to digital image signals, they can also process other digital signals. For example, when the electronic device 100 selects the frequency point, the digital signal processor is used to perform Fourier transform on the energy of the frequency point.
- Video codecs are used to compress or decompress digital video.
- the electronic device 100 may support one or more video codecs. In this way, the electronic device 100 can play or record videos in a variety of encoding formats, such as: moving picture experts group (MPEG) 1, MPEG2, MPEG3, MPEG4, and so on.
- MPEG moving picture experts group
- the external memory interface 120 may be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100.
- the external memory card communicates with the processor 110 through the external memory interface 120 to realize the data storage function. For example, save music, video and other files in an external memory card.
- the internal memory 121 may be used to store one or more computer programs, and the one or more computer programs include instructions.
- the processor 110 can execute the above-mentioned instructions stored in the internal memory 121 to enable the electronic device 100 to execute the display methods provided in some embodiments of the present application, as well as various functional applications and data processing.
- the internal memory 121 may include a storage program area and a storage data area. Among them, the storage program area can store the operating system; the storage program area can also store one or more application programs (such as a gallery, contacts, etc.) and so on.
- the data storage area can store data (such as photos, contacts, etc.) created during the use of the electronic device 101.
- the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic disk storage devices, flash memory devices, universal flash storage (UFS), etc.
- the processor 110 executes the instructions stored in the internal memory 121 and/or the instructions stored in the memory provided in the processor to cause the electronic device 100 to execute the display method provided in the embodiments of the present application. , And various functional applications and data processing.
- the electronic device 100 can implement audio functions through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the earphone interface 170D, and the application processor. For example, music playback, recording, etc.
- the audio module 170 is used to convert digital audio information into an analog audio signal for output, and is also used to convert an analog audio input into a digital audio signal.
- the audio module 170 can also be used to encode and decode audio signals.
- the audio module 170 may be provided in the processor 110, or part of the functional modules of the audio module 170 may be provided in the processor 110.
- the speaker 170A also called a “speaker” is used to convert audio electrical signals into sound signals.
- the electronic device 100 can listen to music through the speaker 170A, or listen to a hands-free call.
- the receiver 170B also called “earpiece” is used to convert audio electrical signals into sound signals.
- the electronic device 100 answers a call or voice message, it can receive the voice by bringing the receiver 170B close to the human ear.
- the microphone 170C also called “microphone”, “microphone”, is used to convert sound signals into electrical signals.
- the user can approach the microphone 170C through the mouth to make a sound, and input the sound signal to the microphone 170C.
- the electronic device 100 may be provided with one or more microphones 170C.
- the electronic device 100 may be provided with two microphones 170C, which can implement noise reduction functions in addition to collecting sound signals.
- the electronic device 100 can also be provided with three, four or more microphones 170C to collect sound signals, reduce noise, identify sound sources, and realize directional recording functions.
- the earphone interface 170D is used to connect wired earphones.
- the earphone interface 170D may be a USB interface 130, or a 3.5mm open mobile terminal platform (OMTP) standard interface, and a cellular telecommunications industry association (cellular telecommunications industry association of the USA, CTIA) standard interface.
- OMTP open mobile terminal platform
- CTIA cellular telecommunications industry association of the USA, CTIA
- the sensor module 180 may include a pressure sensor, a gyroscope sensor, an air pressure sensor, a magnetic sensor, an acceleration sensor, a distance sensor, a proximity light sensor, a fingerprint sensor, a temperature sensor, a touch sensor, an ambient light sensor, a bone conduction sensor, etc.
- the aforementioned electronic device 100 may also include one or more components such as buttons, motors, indicators, and SIM card interfaces, which are not limited in the embodiment of the present application.
- the software system of the above electronic device 100 may adopt a layered architecture, an event-driven architecture, a microkernel architecture, a microservice architecture, or a cloud architecture.
- the embodiment of the present application takes a layered Android system as an example to illustrate the software structure of the electronic device 100.
- FIG. 7 is a software structure block diagram of the electronic device 100 according to an embodiment of the present application.
- the layered architecture divides the software into several layers, and each layer has a clear role and division of labor. Communication between layers through software interface.
- the Android system is divided into four layers, from top to bottom, the application layer, the application framework layer, the Android runtime and system library, and the kernel layer.
- the application layer can include a series of applications.
- the above-mentioned application programs may include APPs (applications) such as call, contact, camera, gallery, calendar, map, navigation, Bluetooth, music, video, short message, etc.
- APPs applications
- the application interface displayed during runtime of some APPs may include not only controls that cannot interact with users, such as pictures, but also controls that can interact with users, such as buttons and progress bars.
- the application interface when running an application interface of a game application, the application interface may include a game screen and an operating handle used to control the game.
- the playback interface of a video application when running, the playback interface may include a video screen being played and control buttons for controlling video playback (such as fast forward, pause, barrage, etc.).
- the application framework layer provides application programming interfaces (application programming interface, API) and programming frameworks for applications in the application layer.
- the application framework layer includes some predefined functions.
- the application framework layer can include a view system, a notification manager, an activity manager, a window manager, a content provider, a resource manager, an input method manager, and so on.
- the view system can be used to construct the display interface of the application.
- Each display interface can consist of one or more controls.
- controls can include interface elements such as icons, buttons, menus, tabs, text boxes, dialog boxes, status bars, navigation bars, and widgets.
- the view system can obtain view information of the corresponding display interface when drawing the display interface, and the view information records the layer relationship between the various controls in the display interface to be drawn.
- each control in the display interface is generally organized hierarchically according to a tree structure to form a complete ViewTree (view tree), which may be referred to as the view information of the above display interface.
- the view system can draw the display interface according to the layer relationship between the controls set in the ViewTree.
- Each control in the display interface corresponds to a set of drawing instructions, such as DrawLine, DrawPoint, DrawBitmap, etc.
- the view system can in turn call the drawing instructions of the corresponding control to draw the control according to the layer relationship between the controls in the ViewTree.
- (a) in Figure 8 is the chat interface 801 of the WeChat APP
- the bottom control of the chat interface 801 is the root node (root)
- the basemap 802 also includes the following Controls: title bar 803, chat background 804, and input bar 805.
- the title bar 803 further includes a return button 806 and a title 807
- the chat background 804 further includes an avatar 808 and a bubble 809
- the input bar 805 further includes a voice input button icon 810, an input box 811, and a send button 812.
- These controls are layered in order to form a view tree A as shown in Figure 8(b).
- the view tree A records the layer order between the various controls in the chat interface 801.
- the base map 802 is located under the root node, and the title bar 803, the chat background 804, and the input bar 805 are all child nodes of the base map 802. Both the return button 806 and the title 807 are child nodes of the title bar 803. Both the avatar 808 and the bubble 809 are child nodes of the chat background 804.
- the voice input button icon 810, the input box 811, and the sending button 812 are all child nodes of the input field 805.
- a new recognition module may be added to the view system for recognizing operation controls in the application interface.
- the operation control refers to a control that can interact with a user, that is, the control can present corresponding functions in response to a user's operation.
- the operation controls in the aforementioned chat interface 801 include a return button 806, an avatar 808, a bubble 809, a voice input button icon 810, an input box 811, and a send button 812.
- the aforementioned recognition module can also be set at the application framework layer independently of the view system, and the embodiment of the present application does not impose any limitation on this.
- a status monitoring service can be set in the underlying system below the application framework layer to identify the position and shape changes of the flexible screen.
- the condition monitoring service can be set in the system library and/or kernel layer.
- the status monitoring service may call a sensor service to start sensors such as a gyroscope and an acceleration sensor for detection.
- the condition monitoring service can calculate the angle between the current main screen 41 and the secondary screen 42 based on the detection data reported by each sensor.
- the status monitoring service can report split-screen events to the view system.
- the view system may periodically obtain the current physical state of the flexible screen from the state monitoring service.
- the recognition module in the view system can identify the operation controls in the current application interface, and display the identified one or more operation controls on a screen of the flexible screen (such as the main screen) In 41), at the same time, the controls in the application interface that cannot be interacted with the user (hereinafter referred to as non-operation controls) are displayed on another screen of the flexible screen (for example, the secondary screen 42).
- the electronic device can display the operating controls and non-operating controls in the application interface being displayed on the first screen and the second screen of the flexible screen.
- the first screen is the main screen 41
- the second screen is the secondary screen 42
- the first screen is the secondary screen 42
- the second screen is the main screen 41.
- the electronic device can display the progress bar, fast forward, pause and other operation controls in the video playback interface on the secondary screen, and display the video screen (ie, non-operation control) played in the video playback interface In the main screen.
- the electronic device may display operation controls such as an operation handle in the game interface on the secondary screen, and display the game screen (ie, non-operation controls) in the game interface on the main screen.
- operation controls such as an operation handle in the game interface on the secondary screen
- the game screen ie, non-operation controls
- the operation controls after the split-screen display will not block the above-mentioned video images or game images, so that the user can obtain an immersive visual and operating experience.
- the aforementioned activity manager can be used to manage the life cycle of each application.
- Applications usually run in the operating system in the form of activity.
- the activity manager can schedule the activity process of the application to manage the life cycle of each application.
- the window manager is used to manage window programs.
- the window manager can obtain the size of the display, determine whether there is a status bar, lock the screen, take a screenshot, etc.
- the content provider is used to store and retrieve data and make these data accessible to applications.
- the data may include video, image, audio, phone calls made and received, browsing history and bookmarks, phone book, etc.
- the resource manager provides various resources for the application, such as localized strings, icons, pictures, layout files, video files, etc.
- Android Runtime includes core libraries and virtual machines. Android runtime is responsible for the scheduling and management of the Android system.
- the core library consists of two parts: one part is the function functions that the java language needs to call, and the other part is the core library of Android.
- the application layer and the application framework layer run in a virtual machine.
- the virtual machine executes the java files in the application layer and application framework layer as binary files.
- the virtual machine is used to perform functions such as object life cycle management, stack management, thread management, security and exception management, and garbage collection.
- the system library can include multiple functional modules. For example: surface manager (surface manager), media library (Media Libraries), three-dimensional graphics processing library (for example: OpenGL ES), 2D graphics engine (for example: SGL), etc.
- the surface manager is used to manage the display subsystem, and provides a combination of 2D and 3D layers for multiple applications.
- the media library supports playback and recording of a variety of commonly used audio and video formats, as well as still image files.
- the media library can support multiple audio and video encoding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc.
- the 3D graphics processing library is used to realize 3D graphics drawing, image rendering, synthesis, and layer processing.
- the 2D graphics engine is a drawing engine for 2D drawing.
- the kernel layer is the layer between hardware and software.
- the kernel layer includes at least a display driver, a camera driver, an audio driver, a sensor driver, etc., which are not limited in the embodiment of the present application.
- a mobile phone is used as an example of an electronic device, and a method for displaying a flexible screen provided by an embodiment of the present application will be described in detail with reference to the accompanying drawings.
- the physical state of the flexible screen in the mobile phone may also include a folded state.
- a folded state As an example of an outward-folding flexible screen, as shown in (a) of FIG. 9, when the angle between the main screen 41 and the auxiliary screen 42 is 360° (or close to 360°), the flexible screen is in a folded state.
- an inward-folding flexible screen as an example, as shown in FIG. 9(b), when the angle between the main screen 41 and the sub screen 42 is 0 (or close to 0), the flexible screen is in a folded state.
- the mobile phone may display a desktop 1001 on the main screen 41, and at this time, the secondary screen 42 (not shown in FIG. 10) may be in a black screen state.
- the condition monitoring service in the mobile phone determines that the angle ⁇ between the main screen 41 and the secondary screen 42 is 360° according to the data reported by the sensor, the condition monitoring service can continue to call the camera, infrared sensor, proximity light sensor or touch device Wait to identify the specific orientation of the phone.
- cameras can be installed on the main screen 41 and the secondary screen 42. If the camera of the main screen 41 captures facial information, but the camera of the secondary screen 42 does not capture the facial information, the status monitoring service can determine the flexibility of the current folding state The screen is in a state where the main screen 41 faces the user.
- infrared sensors can be installed on the main screen 41 and the auxiliary screen 42 respectively. If the infrared sensor of the auxiliary screen 42 captures the infrared signal radiated by the human body, but the infrared sensor of the main screen 41 does not capture the infrared signal radiated by the human body, the status monitoring The service may determine that the flexible screen in the current folded state is in a state where the secondary screen 42 faces the user.
- the mobile phone can also report the user's current touch position through the touch device.
- a preset grasping algorithm can be used according to the touch position to determine the grasping posture of the user currently grasping the handset.
- the state monitoring service can also determine the specific orientation of the mobile phone.
- the touch control device may report the coordinates of the touch point to the state monitoring service after detecting the touch event.
- the state monitoring service determines the gripping posture of the mobile phone by counting the position and number of touch points in the touch device.
- the screen facing the user at this time is the secondary screen 42.
- the threshold indicating that the user's fingers and palm are grasped on the main screen 41
- the screen facing the user at this time is the secondary screen 42.
- the threshold it means that the user's fingers and palm are grasped on the secondary screen 42, and the screen facing the user is now the primary screen 41.
- the mobile phone can display the desktop 1001 shown in FIG. 10 on the main screen 41.
- the mobile phone can display the above-mentioned desktop 1001 on the secondary screen 42.
- the mobile phone can also display the application interface of other applications on the main screen 41 (or the secondary screen 42), which is not limited in the embodiment of the present application.
- the desktop 1001 includes an icon 1002 of a game application. If the user's click on the icon 1002 is detected, as shown in FIG. 11, the mobile phone can open the game application and display the game interface 1101 of the game application. Exemplarily, if the game application supports display in the horizontal screen state but not in the vertical screen state, the mobile phone may automatically display the game interface 1101 in the horizontal screen state. Or, if the game application supports both the horizontal screen state display and the vertical screen state display, the mobile phone may display the game interface 1101 in the horizontal screen state after detecting that the user has rotated the mobile phone from the vertical screen state to the horizontal screen state.
- the status monitoring service in the mobile phone can periodically detect the angle between the main screen 41 and the sub screen 42 in the flexible screen, and whether the main screen 41 and the sub screen 42 are in a landscape state.
- the included angle between the main screen 41 and the auxiliary screen 42 is 180°, it indicates that the flexible screen is switched from the above-mentioned folded state to the unfolded state.
- the included angle between the main screen 41 and the sub screen 42 is within the range of [90, 180°)
- the main screen 41 and the sub screen 42 are both in landscape mode, it indicates that the flexible screen is switched from the above-mentioned folded state to the stand state .
- the mobile phone can separate the operating controls and non-operating controls in the game interface 1101, and display the operating controls in the game interface 1101 on a screen of the flexible screen (such as a secondary In the screen 42), the non-operation controls in the game interface 1101 are displayed in another screen (for example, the main screen 41) of the flexible screen.
- an application whitelist may be preset in the mobile phone, and the application whitelist includes one or more application identities, such as application package names.
- the applications in the application whitelist all support the split-screen display of operating controls and non-operating controls in the application interface. Then, when the mobile phone detects that the flexible screen is switched to the expanded state (or stand state), the mobile phone can obtain the package name of the currently running application (such as the aforementioned game application), and then query whether the game application is in the aforementioned application whitelist Pre-set application.
- an interface whitelist can be preset in the mobile phone, and the interface whitelist includes one or more application interface identifiers, such as the activityname of the application interface.
- the application interfaces in the interface whitelist all support the function of split-screen display of their operating controls and non-operating controls. Then, when the mobile phone detects that the flexible screen is switched to the expanded state (or stand state), the mobile phone can obtain the activityname of the currently running application interface, and then query the whitelist of the interface that the currently displayed application interface is the preset application interface .
- the mobile phone when the flexible screen is in a folded state, the mobile phone can display the game interface 1101 in a horizontal screen state.
- the game interface 1101 when the flexible screen is detected to be switched to the expanded state, if the game interface 1101 is a preset application interface, the mobile phone can identify the operation controls and non-operation controls in the game interface 1101, and then The identified non-operating controls are displayed on the main screen 41, and the identified operating controls are displayed on the secondary screen 42 (that is, the non-operating controls and the operating controls are displayed on separate screens).
- the game interface 1101 is not a preset application or application interface, the mobile phone can continue to display the game interface 1101 according to the existing display mode, and the embodiment of the present application does not impose any limitation on this.
- the mobile phone can obtain the view tree 1300 corresponding to the view system drawing the above-mentioned game interface 1101 in the folded state.
- the view tree 1300 records the layer order among the various controls in the game interface 1101.
- the root node of the game interface 1101 includes a control of a game screen 1301, and the game screen 1301 includes the following controls: an operation handle 1302, an attack button 1303, and a setting bar 1304.
- the setting column 1304 further includes a first setting button 1305 and a second setting button 1306.
- the game screen 1301 is a child node of the root node
- the operation handle 1302, the attack button 1303, and the setting bar 1304 are all child nodes of the game screen 1301, located on the layer where the game screen 1301 is located.
- the first setting button 1305 and the second setting button 1306 are both child nodes of the setting column 1304 and are located on the layer where the setting column 1304 is located.
- the mobile phone can record the position and size of each control in the view tree 1300, and the corresponding drawing instruction when drawing each control.
- the phone When drawing the game interface 1101, the phone can first draw the game screen 1301 according to the view tree 1300, and then draw the operation handle 1302, the attack button 1303, and the setting bar 1304 on the game screen 1301, and then draw the first setting button 1305 on the setting bar 1304 And the second setting button 1306.
- the mobile phone when the mobile phone detects that the flexible screen is switched from the folded state shown in Figure 12 (a) to the expanded state shown in Figure 12 (b), the mobile phone can obtain the above-mentioned view tree 1300 and query the view tree 1300 The position and size of each control.
- operation controls are usually located in the upper layer of the non-operation control, and the non-operation control is usually displayed as a base map in the lower layer of the operation control, and the size of the non-operation control is generally relatively large.
- the mobile phone finds that the size of the control of the game screen 1301 is the same as or close to the size of the entire main screen 41, the mobile phone can confirm the game screen 1301 as a non-operation control in the game interface 1101. In addition, the mobile phone can recognize each child node of the game screen 1301 in the view tree 1300 as an operation control.
- the mobile phone when the mobile phone finds that the size of the control of the game screen 1301 is the same as or close to the size of the entire main screen 41, it can further determine whether the layer where the game screen 1301 is located is the top layer. If the layer where the game screen 1301 is located is not the top layer, the mobile phone can confirm the game screen 1301 as a non-operation control in the game interface 1101. In addition, the mobile phone may determine each control located on the layer where the game screen 1301 is located as an operation control.
- the mobile phone can recognize the control with a larger display size and close to the bottom layer of the application screen as a non-operational control according to the size and position relationship of each control when drawing the current application interface, and recognize the control located above the non-operational control It is an operation control.
- the mobile phone can also obtain the specific type of each control in the view tree 1300.
- the control may include image control (Image View), text control (Text View), button (Button), navigation bar (ActionBar), text box, dialog box, status bar and other types.
- the view system in the mobile phone may send the identification or location of each control in the view tree 1300 to the game application, requesting the game application to query the specific type of each control in the view tree 1300 according to the identification or location of the control.
- the mobile phone may record the type of the control in the corresponding field when drawing each control in the view tree 1300. Then, the mobile phone can search for the specific type of each control in the view tree 1300 from the corresponding field.
- the mobile phone can confirm that the control is a non-operation control.
- the game screen 1301 in the aforementioned game interface 1101 is an Image View type non-operation control.
- a certain control is a button (Button), navigation bar (ActionBar), text box, dialog box, status bar and other types of controls
- the mobile phone can confirm that the control is an operation control.
- the operation handle 1302, the attack button 1303, and the controls in the setting bar 1304 in the aforementioned game interface 1101 are all operation controls.
- the configuration file corresponding to the aforementioned game interface 1101 can also be saved in the mobile phone in advance.
- the configuration file records the operation controls in the aforementioned game interface 1101.
- the configuration file may record the specific display position of the operation control in the game interface 1101 before the split screen (that is, when the flexible screen is in the folded state).
- the configuration file may also record the specific position of the above-mentioned operation control on the main screen or the secondary screen after the screen is split (that is, when the flexible screen is in the expanded or stent state).
- the configuration file may also record information such as the translation distance, zoom ratio, and rotation angle of the operation control after split screen.
- the configuration file 1 corresponding to the aforementioned game interface 1101 may be:
- each configuration file records the corresponding packagename, activityname, and the position of the operation control before split screen.
- the mobile phone can search according to the packagename and activityname of the currently running game application To the corresponding configuration file (for example, configuration file 1 above).
- the mobile phone can recognize the operation control 1 in the game interface 1101 according to the configuration file 1.
- the mobile phone can identify which control is specifically the control 1 in the game interface 1101 according to the specific position of the control 1 before the split screen recorded in the configuration file 1. In this way, by traversing the positions of all operation controls before splitting the screen in the configuration file 1, the mobile phone can identify each operation control in the game interface 1101.
- the mobile phone can recognize controls other than the operation controls in the game interface 1101 as non-operation controls.
- the mobile phone can recognize through the above method that the operation controls in the game interface 1101 include: an operation handle 1302, an attack button 1303, and various controls in the setting bar 1304.
- the non-operation controls in the game interface 1101 include the game screen 1301.
- the mobile phone can display non-operation controls on the main screen 41 and operation controls on the secondary screen 42.
- the mobile phone may also display the non-operation controls on the secondary screen 42 and the operation controls on the main screen 41.
- the above embodiment is an example of the flexible screen switching from the folded state to the unfolded state.
- the mobile phone can also call the camera, infrared sensor, proximity light sensor or touch device to identify the specific orientation of the mobile phone.
- the mobile phone can display the identified non-operation controls on the main screen 41 and the identified operation controls on the secondary screen 42. If the secondary screen 42 faces the user at this time, the mobile phone can display the identified non-operation controls on the secondary screen 42 and the identified operation controls on the main screen 41. In this way, the screen facing the user can display controls that do not require operation and interaction for the user (for example, the above-mentioned game screen 1301), and the user can use the operation controls on another screen to update or edit the content in the game screen 1301. Operation controls The game screen 1301 will not be blocked, so that the user can get an immersive viewing experience.
- the configuration file 1 corresponding to the game interface 1101 not only records the pre-split operation control 1 (For example, the position of the operating handle 1302) in the main screen 41, and the position of the operating handle 1302 in the secondary screen 42 after the split screen is also recorded. Then, according to the specific positions of the operation controls in the configuration file 1 before and after the split screen, the mobile phone can determine the view tree corresponding to the main screen 41 after the split screen and the view tree corresponding to the secondary screen 42 after the split screen.
- FIG. 15 shows a view tree 1300 of the game interface 1101 in a folded state.
- the operation control and the non-operation control in the view tree 1300 can be split into two view trees.
- the mobile phone can use the layer relationship of the operation controls in the view tree 1300 to generate the corresponding view tree 1401. At this time, the layer relationship of the operation controls in the view tree 1300 and the view tree 1401 is the same. Similarly, the mobile phone can use the layer relationship of the non-operational controls in the view tree 1300 to generate the corresponding view tree 1402. At this time, the layer relationship of the non-operational controls in the view tree 1300 and the view tree 1402 is also the same.
- the mobile phone can also split the operation control from the view tree 1401, and according to the record in the configuration file 1.
- Each operation control generates a corresponding view tree 1401 at the display position after the split screen.
- the layer relationship of the operation control in the view tree 1300 and the view tree 1401 may be the same or different, and the embodiment of the present application does not impose any limitation on this.
- the operation handle 1302, the attack button 1303, and the controls in the setting bar 1304 in the view tree 1300 are all operation controls, these operation controls are displayed on the secondary screen 42 after splitting the screen. Then, the mobile phone can generate a view tree 1401 corresponding to the secondary screen 42.
- the view tree 1401 includes an operating handle 1302, an attack button 1303, and a setting bar 1304 set under the root node.
- the setting column 1304 also includes two sub-nodes, namely the first setting button 1305 and the second setting button 1306. The position of each control in the view tree 1401 in the secondary screen 42 is recorded in the configuration file 1.
- the non-operation control is displayed in the main screen 41 after split screen.
- the mobile phone can generate a view tree 1402 corresponding to the main screen 41, and the view tree 1402 includes a game screen 1301 set under the root node.
- the position of the game screen 1301 in the main screen 41 after the split screen is the same as the position of the game screen 1301 when the view tree 1300 is drawn.
- the mobile phone can redraw the game screen 1301 on the main screen 41 according to the view tree 1402.
- the mobile phone since the mobile phone records the drawing instructions corresponding to each control when drawing the above-mentioned game interface 1101, when the mobile phone redraws the game screen 1301 in the main screen 41, the game screen 1301 when drawing the above-mentioned game interface 1101 can still be used. Redraw the game screen 1301 with the drawing instructions.
- the mobile phone can draw various operation controls in the view tree 1403 on the secondary screen 42 according to the view tree 1403.
- the mobile phone can use the drawing instruction of drawing the operating handle 1302 when drawing the aforementioned game interface 1101, and draw the operating handle 1302 at the position of the operating handle 1302 after the split screen indicated by the configuration file 1.
- each control in each level of control in the view tree 1403 can be drawn in a corresponding position in the secondary screen 42 in order.
- the main screen 41 and the sub-screen 42 are the same size, when the mobile phone draws each operation control in the view tree 1403 on the sub-screen 42, it can be adjusted according to the coordinates of each operation control in the main screen 41. Operation controls are drawn in the secondary screen 42.
- the first setting button 1305 in the game interface 1101 is a 40*30 rectangle, and the coordinates of the upper left corner of the rectangle on the main screen 41 are (380, 10), and the coordinates of the lower right corner are (420, 40).
- the first setting button 1305 can also be drawn as a 40*30 rectangle, and the rectangle is drawn on the secondary screen 42 with the upper left corner coordinates (380, 10 ), the area with coordinates (420,40) in the lower right corner.
- the size or shape of the operation controls can also be changed.
- the above configuration file 1 records that the zoom ratio of the operating handle 1302 on the x-axis is 2 and the zoom ratio on the y-axis is also 2, it means that the size of the operating handle 1302 in the secondary screen 42 after the split screen is before the split screen 2 times the operation handle 1302 in the game interface 1101.
- the mobile phone can magnify the operating handle 1302 in the game interface 1101 by 2 times to obtain a new operating handle 1503, and draw the operating handle 1503 on the secondary screen 42.
- the mobile phone can take up more or less display space when drawing operation controls after split screen, so that users can get a better experience when operating these operation controls after split screen.
- the mobile phone after the mobile phone detects that the flexible screen is switched from the folded state to the stand (or unfolded) state, it can also determine the operation controls and non-operations in the application interface to be drawn before drawing the application interface according to the above configuration file 1. Control. Furthermore, as shown in (a)-(b) of FIG. 15, the mobile phone can split the view tree 1300 that originally needs to be drawn into a view tree 1401 and a view tree 1402. The view tree 1401 is all operation controls that need to be drawn, and the view tree 1402 is all the non-operation controls that need to be drawn. In this way, as still shown in FIG.
- the mobile phone can draw various operation controls on the secondary screen 42 according to the view tree 1401, and draw various non-operation controls on the main screen 41 according to the view tree 1402.
- the mobile phone can identify the operating controls and non-operating controls in the application interface before drawing the application interface according to the configuration file, and then display the operating controls and non-operating controls on the main screen 41 and the secondary screen 42 separately, without drawing After the above-mentioned game interface 1101 is displayed, the split-screen interface is drawn twice according to the configuration file.
- the mobile phone draws the operation controls in the game interface 1101 on the secondary screen 42 to form a first interface 1501, and the mobile phone draws the non-operation controls in the game interface 1101 on the main screen to form a second interface. 1502.
- the user can operate each operation control on the secondary screen 42 to continue to complete related functions of the game application.
- the touch sensor on the secondary screen 42 can detect the user's touch operation on the first interface 1501 in real time.
- the user can touch the operating handle 1503 in the first interface 1501 to control the game character to move up, down, left, and right.
- the secondary screen 42 can sequentially report the touch information (for example, the coordinate information of the touch point, the touch time, etc.) of the detected series of touch events to the kernel layer of the mobile phone and the application program. Framework layer and application layer.
- the secondary screen 42 Take the coordinate system in the secondary screen 42 as the first coordinate system, and the coordinate system in the main screen 41 as the second coordinate system. If the coordinates of the touch point in a touch event detected by the secondary screen 42 are P(x, y), combined with the architecture diagram of the Android system shown in FIG. 7, as shown in FIG. 17, the secondary screen 42 can be detected by driving
- the received touch event is encapsulated as an original touch event and reported to the kernel layer, which then encapsulates the original touch event as an upper-level readable advanced touch event and reports it to the application framework layer.
- the secondary screen 42 is the first coordinate system set at the lower left corner as the origin, and the mobile phone
- the operating system may be the first coordinate system set at the upper left corner as the origin.
- the application framework layer After the application framework layer receives the advanced touch event carrying the coordinates P(x,y) or P'(x',y'), it can follow the positional relationship of each operation control recorded in the configuration file 1 after the split screen , It is determined that this touch event is an operation performed on the operating handle 1503. Furthermore, the application framework layer can convert the first touch event whose touch point is P point to the first touch event whose touch point is Q point in the second coordinate system according to the positional relationship of the operating handle 1503 before and after the split screen in the configuration file 1. Two touch events.
- the operating handle 1302 before split screen is located in a circular area with a center of A (40, 40) and a radius of 40 in the game interface 1101.
- the mobile phone displays the operating handle 1503 in the secondary screen 42 in a circular area with a radius of 80 and point B (80, 80) as the center according to the configuration file 1.
- the operating handle 1503 is enlarged twice in both the x-axis and the y-axis.
- the application framework layer can determine the touch point corresponding to point P in the second coordinate system
- the coordinates of Q are (50,40). That is, the touch point P (x, y) on the operating handle 1503 in the first coordinate system corresponds to the touch point Q (0.5x, 0.5y) on the operating handle 1302 in the second coordinate system.
- the application framework layer can report the second touch event carrying the touch point Q (0.5x, 0.5y) to the running game application in the application layer, so that the game application can be based on the touch point Q (0.5x, 0.5y). ) Respond to the user's touch operation this time.
- the game application After the mobile phone detects the first touch event of the user inputting the touch point P(x, y) on the secondary screen 42, the game application finally responds to the user at the Q(0.5x, 0.5y) point on the primary screen 41
- the application can also normally respond to the touch operations performed by the user on the operating controls after the split screen.
- the foregoing game interface 1101 is used as an example to illustrate the display method of the game interface 1101 before and after the split screen. It is understandable that the mobile phone can also display the operating controls and non-operating controls in the application interface in a split screen according to the above method when running other applications or application interfaces.
- the outer-folding flexible screen is exemplified. It can be understood that for the inner-folding flexible screen, the mobile phone can also display the operating controls and non-operating controls in the application interface according to the above method. The embodiments of the application do not impose any restriction on this.
- the inward-folding flexible screen is in a folded state.
- both the main screen 41 and the sub screen 42 can be in a black screen state. If it is detected that the included angle between the main screen 41 and the secondary screen 42 is 180°, it means that the flexible screen is switched from the folded state to the unfolded state at this time. If it is detected that the angle between the main screen 41 and the sub screen 42 is within the range of [90, 180), and the main screen 41 and the sub screen 42 are both in landscape mode, it means that the flexible screen is switched from the folded state to the stand state. .
- the mobile phone can unlock the main screen 41 or the sub-screen 42 of the flexible screen, and can also use the main screen 41 and the sub-screen 42 as a complete display area for display after unlocking.
- the desktop 1801 can be displayed on the main screen 41.
- the desktop 1801 includes an icon 1802 of a video application. If the user's operation to open the video application is detected, the mobile phone can open the video application to display various application interfaces of the video application.
- the mobile phone can automatically identify the operation controls and non-operation controls in the playback interface 1901 according to the method in the foregoing embodiment. If it is recognized that the video screen 1902 in the playback interface 1901 is a non-operation control, the progress bar 1903, the playback button 1904, the fast forward button 1905, the barrage button 1906, and the return button 1907 in the playback interface 1901 are the operation controls, as shown in Figure 20
- the mobile phone can display the video screen 1902 (ie, non-operation control) on the main screen 41 facing the user, and display the progress bar 1903, the play button 1904, the fast forward button 1905, the pop-up button 1906 and the return The button 1907 (that is, the operation control) is displayed in the secondary screen 42. Subsequently, the user can operate various operation controls on the secondary screen 42 to control the video application. These operation controls will not block the video screen 1902 on the primary
- the mobile phone when it is detected that the flexible screen is switched from the folded state to the unfolded state (or stand state), and the screen is unlocked, the mobile phone can automatically display the video screen 1902 (that is, non- The operation controls) are displayed on the main screen 41, and the progress bar 1903, the play button 1904, the fast forward button 1905, the popup button 1906, and the return button (ie, operation controls) are displayed on the secondary screen 42. Subsequently, the user can operate various operation controls on the secondary screen 42 to control the video application. These operation controls will not block the video screen 1902 on the primary screen 41, so that the user can obtain an immersive viewing and using experience.
- the mobile phone can also use the entire flexible screen as a complete display area to display the first interface (for example, the aforementioned playback interface 1901 or Game interface 1101).
- the flexible screen is switched to the expanded state or the stand state, that is, when the angle between the main screen 41 and the secondary screen 42 is in the range of [90, 180°]
- the mobile phone can also recognize the first interface according to the display method provided in the above embodiment Display the identified operation controls on one of the main screen 41 and the secondary screen 42, and display the identified non-operation controls on the other screen of the primary screen 41 and the secondary screen 42
- the embodiments of the present application do not impose any restriction on this.
- the embodiment of the present application discloses an electronic device including a processor, and a memory, an input device, an output device, and a communication module connected to the processor.
- the input device and the output device may be integrated into one device.
- the input device and the output device may be a flexible screen, and the flexible screen may include a display device and a touch sensor.
- the aforementioned electronic device may include: a flexible screen 2201; one or more processors 2202; a memory 2203; one or more application programs (not shown); and one or more computer programs 2204
- the above-mentioned devices may be connected through one or more communication buses 2205.
- the one or more computer programs 2204 are stored in the aforementioned memory 2203 and configured to be executed by the one or more processors 2202.
- the one or more computer programs 2204 include instructions, and the aforementioned instructions can be used to execute the aforementioned implementations.
- the steps in the example Among them, all relevant content of the steps involved in the above method embodiments can be cited in the functional description of the corresponding physical device, which will not be repeated here.
- the foregoing processor 2202 may specifically be the processor 110 shown in FIG. 2
- the foregoing memory 2203 may specifically be the internal memory 121 and/or the external memory 122 shown in FIG. 2
- the foregoing flexible screen 2201 may specifically include FIG. 2
- the display screen 194 and the touch sensor in the sensor module 180 shown are not limited in the embodiment of the present application.
- the functional units in the various embodiments of the embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units may be integrated into one unit.
- the above-mentioned integrated unit can be implemented in the form of hardware or software functional unit.
- the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer readable storage medium.
- a computer readable storage medium includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) or a processor to execute all or part of the steps of the methods described in the various embodiments of the present application.
- the aforementioned storage media include: flash memory, mobile hard disk, read-only memory, random access memory, magnetic disk or optical disk and other media that can store program codes.
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- User Interface Of Digital Computer (AREA)
- Controls And Circuits For Display Device (AREA)
Abstract
Provided in the embodiments of the present application are a display method for a flexible screen and an electronic device, relating to the technical field of terminals, and capable of rearranging the various controls in the display interface on the basis of the physical state of the flexible screen, such that the user has a better experience when viewing or using the flexible screen. The method comprises: when the included angle between a first screen of the flexible screen and a second screen of the flexible screen is less than a first preset value or more than a second preset value, the electronic device displays a first interface in the first screen and/or the second screen; when the included angle between the first screen and the second screen is in a closed range formed by the first preset value and the second preset value, the electronic device determines an operating control and a non-operating control in the first interface; the electronic device displays a second interface on the first screen, the second interface comprising the operating control; and the electronic device displays a third interface on the second screen, the third interface comprising the non-operating control.
Description
本申请要求在2019年6月5日提交中国国家知识产权局、申请号为201910488438.1的中国专利申请的优先权,发明名称为“一种柔性屏幕的显示方法及电子设备”的中国专利申请的优先权,其全部内容通过引用结合在本申请中。This application claims the priority of the Chinese patent application filed with the State Intellectual Property Office of China with application number 201910488438.1 on June 5, 2019, and the priority of the Chinese patent application with the title of "a flexible screen display method and electronic device" Right, the entire contents of which are incorporated in this application by reference.
本申请涉及终端技术领域,尤其涉及一种柔性屏幕的显示方法及电子设备。This application relates to the field of terminal technology, and in particular to a display method and electronic device of a flexible screen.
柔性屏幕也可称为柔性OLED(organic light-emitting diode,有机发光二极管),相较于传统屏幕,柔性屏幕不仅在体积上更加轻薄,同时基于其可弯曲、柔韧性佳的特性,柔性屏幕的耐用程度也大大高于传统屏幕。Flexible screens can also be called flexible OLEDs (organic light-emitting diodes). Compared with traditional screens, flexible screens are not only lighter and thinner in volume, but also based on their bendability and flexibility. The durability is also much higher than traditional screens.
目前,一些厂商已经将柔性屏幕应用在手机、平板电脑等电子设备中。如图1所示,用户在使用具有柔性屏幕的手机时可以折叠屏幕,使手机的灵活性和便携性更高。但屏幕被折叠后屏幕中的显示内容也随屏幕被分割,屏幕折叠后的显示内容分别显示在两个平面(即平面01和平面02)上,由于平面01和平面02呈一定夹角,因此导致用户在观看和操作手机时非常不方便。At present, some manufacturers have applied flexible screens to electronic devices such as mobile phones and tablet computers. As shown in Figure 1, users can fold the screen when using a mobile phone with a flexible screen, making the mobile phone more flexible and portable. However, after the screen is folded, the display content on the screen is also divided with the screen. The displayed content after the screen is folded are displayed on two planes (that is, plane 01 and plane 02). Since plane 01 and plane 02 form a certain angle, It is very inconvenient for users to watch and operate mobile phones.
发明内容Summary of the invention
本申请提供一种柔性屏幕的显示方法及电子设备,可根据柔性屏幕的物理状态重新布局显示界面中的各个控件,使得用户在观看和使用柔性屏幕时得到较好的体验。The present application provides a display method and electronic device for a flexible screen, which can re-lay out various controls in the display interface according to the physical state of the flexible screen, so that the user can get a better experience when watching and using the flexible screen.
为达到上述目的,本申请采用如下技术方案:In order to achieve the above objectives, this application adopts the following technical solutions:
第一方面,本申请提供一种柔性屏幕的显示方法,该柔性屏幕在折叠时可被折叠线划分为第一屏幕和第二屏幕,上述方法包括:当第一屏幕和第二屏幕之间的夹角小于第一预设值(例如90°)或大于第二预设值(例如180°)时,电子设备可在第一屏幕和/或第二屏幕中显示第一界面;当检测到第一屏幕和第二屏幕之间的夹角在第一预设值和第二预设值构成的闭区间内时,柔性屏幕处于开展状态或支架状态,此时,电子设备可确定第一界面中的操作控件和非操作控件;进而,电子设备可在第一屏幕中显示第二界面,第二界面中包括操作控件;并且,电子设备可在第二屏幕中显示第三界面,第三界面中包括非操作控件。In the first aspect, the present application provides a method for displaying a flexible screen. The flexible screen can be divided into a first screen and a second screen by a folding line when folded. The above method includes: When the included angle is less than the first preset value (for example, 90°) or greater than the second preset value (for example, 180°), the electronic device may display the first interface on the first screen and/or the second screen; when the first screen is detected When the angle between the first screen and the second screen is within the closed interval formed by the first preset value and the second preset value, the flexible screen is in the unfolded state or the stand state. At this time, the electronic device can determine that the Further, the electronic device can display a second interface on the first screen, and the second interface includes operation controls; and, the electronic device can display a third interface on the second screen, and the third interface Including non-operational controls.
也就是说,当柔性屏幕的物理状态转变为展开状态或支架状态时,电子设备可将正在显示的应用界面中的操作控件与非操作控件分屏显示在柔性屏幕的第一屏幕和第二屏幕中。这样,分屏显示后的操作控件不会对非操作控件产生遮挡,从而使得户获得较好的沉浸式的视觉和操作体验。In other words, when the physical state of the flexible screen changes to the unfolded state or the stand state, the electronic device can display the operating controls and non-operating controls in the application interface being displayed on the first screen and the second screen of the flexible screen. in. In this way, the operating controls after the split-screen display will not block the non-operating controls, so that the user can obtain a better immersive visual and operating experience.
在一种可能的实现方式中,当第一屏幕和第二屏幕之间的夹角在第一预设值和第二预设值构成的闭区间内时,还包括:电子设备查询上述第一界面是否为预设的应用界面;当第一界面为预设的应用界面(例如游戏界面)时,说明此时应用运行时可能有分屏显示的需求,则电子设备可进一步确定第一界面中的操作控件和非操作控件。In a possible implementation manner, when the angle between the first screen and the second screen is within a closed interval formed by the first preset value and the second preset value, the method further includes: the electronic device queries the first Whether the interface is a preset application interface; when the first interface is a preset application interface (such as a game interface), it means that there may be a need for split-screen display when the application is running, and the electronic device may further determine Operational controls and non-operational controls.
在一种可能的实现方式中,当第一屏幕和第二屏幕之间的夹角在第一预设值和第二预设值构成的闭区间内时,还包括:电子设备确定第一屏幕和第二屏幕是否为横屏状态;当第一屏幕和第二屏幕均为横屏状态时,说明此时应用运行时可能有分屏显示的需求,则电子设备可进一步确定第一界面中的操作控件和非操作控件。In a possible implementation manner, when the angle between the first screen and the second screen is within a closed interval formed by the first preset value and the second preset value, the method further includes: the electronic device determines the first screen And the second screen are in the landscape state; when the first screen and the second screen are in the landscape state, it means that there may be a need for split-screen display when the application is running, and the electronic device can further determine the Operational controls and non-operational controls.
在一种可能的实现方式中,电子设备确定第一界面中的操作控件和非操作控件,包括:电子设备可以获取第一界面中每个控件的位置和尺寸;进而,电子设备可将尺寸大于预设尺寸的控件确定为非操作控件,并将位于非操作控件所在图层之上控件确定为操作控件。In a possible implementation manner, the electronic device determining the operating controls and non-operating controls in the first interface includes: the electronic device can obtain the position and size of each control in the first interface; further, the electronic device can set the size larger than The control of the preset size is determined as a non-operation control, and the control located on the layer where the non-operation control is located is determined as the operation control.
又或者,当第一控件的尺寸大于预设尺寸时,电子设备还可以进一步确定第一控件所在的图层是否为最顶层的图层。如果第一控件所在的图层不是最顶层的图层,则电子设备可将第一控件确认为非操作控件。Or, when the size of the first control is larger than the preset size, the electronic device may further determine whether the layer where the first control is located is the topmost layer. If the layer where the first control is located is not the topmost layer, the electronic device may confirm the first control as a non-operation control.
也就是说,电子设备可以根据界面中各个控件的尺寸和位置关系,将显示尺寸较大且靠近底层图层的控件识别为非操作控件,将位于非操作控件之上的控件识别为操作控件。That is to say, the electronic device can recognize the control with a larger display size and close to the underlying layer as a non-operational control, and recognize the control above the non-operational control as an operational control according to the size and positional relationship of each control in the interface.
在一种可能的实现方式中,电子设备确定第一界面中的操作控件和非操作控件,包括:电子设备获取与第一界面对应的预设的配置文件,该配置文件中记录有第一界面中操作控件的显示位置;那么,电子设备按照该配置文件可识别出第一界面中的操作控件,并将第一界面中除操作控件外的控件确定为非操作控件。In a possible implementation manner, the electronic device determining operation controls and non-operation controls in the first interface includes: the electronic device obtains a preset configuration file corresponding to the first interface, and the first interface is recorded in the configuration file The display position of the operation control in the middle; then, the electronic device can recognize the operation control in the first interface according to the configuration file, and determine the control except the operation control in the first interface as a non-operation control.
在一种可能的实现方式中,第二屏幕为朝向用户的屏幕。这样一来,朝向用户的第二屏幕可为用户显示不需要进行操作和交互的非操作控件,用户可在另一屏幕(即第一屏幕)中操作上述操作控件,使用户获得沉浸式的观看和操作体验。In a possible implementation manner, the second screen is a screen facing the user. In this way, the second screen facing the user can display non-operational controls that do not require operation and interaction for the user, and the user can operate the above-mentioned operation controls on another screen (ie, the first screen), so that the user can get immersive viewing And operating experience.
示例性的,电子设备还可以调用摄像头、红外传感器、接近光传感器或触控器件等识别柔性屏幕中第一屏幕和第二屏幕的具体朝向。又或者,电子设备可以使用加速度传感器等器件检测第一屏幕和第二屏幕与水平面的夹角,当第一屏幕与水平面的夹角小于某一角度时,可确定第二屏幕为朝向用户的屏幕。Exemplarily, the electronic device may also call a camera, an infrared sensor, a proximity light sensor, or a touch device to identify the specific orientation of the first screen and the second screen in the flexible screen. Or, the electronic device can use acceleration sensors and other devices to detect the angle between the first screen and the second screen and the horizontal plane. When the angle between the first screen and the horizontal plane is less than a certain angle, the second screen can be determined to be the screen facing the user. .
在一种可能的实现方式中,在电子设备在第一屏幕中显示第二界面之前,还包括:电子设备获取绘制第一界面的第一视图信息(例如第一界面的视图树),第一视图信息包括第一界面内各个控件之间的图层顺序;进而,电子设备可根据第一视图信息生成与第二界面对应的第二视图信息以及与第三界面对应的第三视图信息,第二视图信息包括该操作控件之间的图层顺序,第三视图信息包括该非操作控件之间的图层顺序;此时,电子设备在第一屏幕中显示第二界面,并且,电子设备在第二屏幕中显示第三界面,包括:电子设备按照第二视图信息在第一屏幕中绘制第二界面;并且,电子设备按照第三视图信息在第二屏幕中绘制第三界面。In a possible implementation manner, before the electronic device displays the second interface on the first screen, the method further includes: the electronic device acquires first view information for drawing the first interface (for example, the view tree of the first interface), and The view information includes the layer order between the controls in the first interface; further, the electronic device can generate second view information corresponding to the second interface and third view information corresponding to the third interface according to the first view information. The second view information includes the layer order between the operation controls, and the third view information includes the layer order between the non-operation controls; at this time, the electronic device displays the second interface on the first screen, and the electronic device Displaying the third interface on the second screen includes: the electronic device draws the second interface on the first screen according to the second view information; and the electronic device draws the third interface on the second screen according to the third view information.
可以理解的是,操作控件在第二视图信息中的图层顺序,与在第一视图信息中的图层顺序相同或不同,即操作控件在分屏前后的界面中的图层顺序可以相同或不同。同样,非操作控件在第三视图信息中的图层顺序,与在第一视图信息中的图层顺序相同或不同,即非操作控件在分屏前后的界面中的图层顺序可以相同或不同。It is understandable that the layer order of the operation controls in the second view information is the same or different from the layer order in the first view information, that is, the layer order of the operation controls in the interface before and after the split screen can be the same or different. different. Similarly, the layer order of the non-operating controls in the third view information is the same or different from the layer order in the first view information, that is, the layer order of the non-operating controls in the interface before and after the split screen can be the same or different .
在一种可能的实现方式中,与第一界面对应的配置文件中还可以记录操作控件在第二界面中的显示位置,以及非操作控件在第三界面中的显示位置;此时,电子设备根据第一视图信息生成与第二界面对应的第二视图信息以及与第三界面对应的第三视图信息,包括:电子设备可按照配置文件中记录的操作控件在第二界面中的显示位置,对第一视图信息中的操作控件进行拆分和重组,得到第二视图信息;类似的,电子设备可按照配置文件中记录的非操作控件在第三界面中的显示位置,对第一视图信息中的非操作控件进行拆分和重组,得到第三视图信息。In a possible implementation, the configuration file corresponding to the first interface may also record the display position of the operation control in the second interface and the display position of the non-operation control in the third interface; at this time, the electronic device Generating the second view information corresponding to the second interface and the third view information corresponding to the third interface according to the first view information includes: the electronic device can follow the display position of the operation control recorded in the configuration file in the second interface, Split and reorganize the operation controls in the first view information to obtain the second view information; similarly, the electronic device can compare the first view information according to the display position of the non-operation controls recorded in the configuration file in the third interface. The non-operational controls in the control panel are split and reorganized to obtain the third view information.
在一种可能的实现方式中,在电子设备在第一屏幕中显示第二界面之后,还包括:电子设备接收用户在第二界面向操作控件输入的第一触摸事件;电子设备将第一触摸事件映射为在第一界面中的第二触摸事件;电子设备执行与第二触摸事件对应的操作指令。也就是说, 各个操作控件被显示在第二界面后,用户可在第二界面中操作各个操作控件继续完成该操作控件的相关功能。In a possible implementation manner, after the electronic device displays the second interface on the first screen, the method further includes: the electronic device receives a first touch event input by the user to the operation control on the second interface; The event is mapped to a second touch event in the first interface; the electronic device executes an operation instruction corresponding to the second touch event. That is, after each operation control is displayed on the second interface, the user can operate each operation control in the second interface to continue to complete the related functions of the operation control.
示例性的,电子设备将第一触摸事件映射为在第一界面中的第二触摸事件,包括:电子设备获取第一触摸事件中第一触摸点的坐标;电子设备根据操作控件在第一界面和第二界面上的相对位置关系,计算第一触摸点在第一界面中对应的第二触摸点的坐标;其中,电子设备执行与第二触摸事件对应的操作指令,包括:电子设备将第二触摸点的坐标携带在第二触摸事件中上报给第一应用(第一界面为第一应用的界面),使得第一应用执行与第二触摸事件对应的操作指令。Exemplarily, the electronic device mapping the first touch event to the second touch event in the first interface includes: the electronic device obtains the coordinates of the first touch point in the first touch event; And the relative position relationship on the second interface, and calculate the coordinates of the second touch point corresponding to the first touch point in the first interface; wherein the electronic device executes the operation instruction corresponding to the second touch event, including: The coordinates of the two touch points are carried in the second touch event and reported to the first application (the first interface is the interface of the first application), so that the first application executes the operation instruction corresponding to the second touch event.
也就是说,电子设备检测到用户在分屏后的第二界面中输入第一触摸事件后,可将第一触摸事件映射为分屏前在第一界面中的第二触摸事件,使得相关应用最终可响应该第二触摸事件,也即响应了用户在第二界面中输入的第一触摸事件。In other words, after the electronic device detects that the user inputs the first touch event in the second interface after the split screen, it can map the first touch event to the second touch event in the first interface before the split screen, so that related applications Finally, it can respond to the second touch event, that is, respond to the first touch event input by the user on the second interface.
第二方面,本申请提供一种电子设备,包括:柔性屏幕,所述柔性屏幕在折叠时被折叠线划分为第一屏幕和第二屏幕;一个或多个处理器;一个或多个存储器;以及一个或多个计算机程序;其中,处理器与柔性屏幕以及存储器均耦合,上述一个或多个计算机程序被存储在存储器中,当电子设备运行时,该处理器执行该存储器存储的一个或多个计算机程序,以使电子设备执行上述任一项所述的柔性屏幕的显示方法。In a second aspect, the present application provides an electronic device, including: a flexible screen that is divided into a first screen and a second screen by a folding line when folded; one or more processors; one or more memories; And one or more computer programs; wherein, the processor is coupled with the flexible screen and the memory, and the above one or more computer programs are stored in the memory. When the electronic device is running, the processor executes one or more stored in the memory. A computer program to make an electronic device execute the flexible screen display method described in any one of the above.
第三方面,本申请提供一种计算机存储介质,包括计算机指令,当计算机指令在电子设备上运行时,使得电子设备执行如第一方面中任一项所述的柔性屏幕的显示方法。In a third aspect, the present application provides a computer storage medium including computer instructions, which when the computer instructions run on an electronic device, cause the electronic device to execute the flexible screen display method as described in any one of the first aspect.
第四方面,本申请提供一种计算机程序产品,当计算机程序产品在电子设备上运行时,使得电子设备执行如第一方面中任一项所述的柔性屏幕的显示方法。In a fourth aspect, this application provides a computer program product, which when the computer program product runs on an electronic device, causes the electronic device to execute the flexible screen display method described in any one of the first aspect.
可以理解地,上述提供的第二方面所述的电子设备、第三方面所述的计算机存储介质,以及第四方面所述的计算机程序产品均用于执行上文所提供的对应的方法,因此,其所能达到的有益效果可参考上文所提供的对应的方法中的有益效果,此处不再赘述。Understandably, the electronic equipment described in the second aspect, the computer storage medium described in the third aspect, and the computer program product described in the fourth aspect provided above are all used to execute the corresponding methods provided above. For the beneficial effects that can be achieved, please refer to the beneficial effects in the corresponding method provided above, which will not be repeated here.
图1为现有技术中柔性屏幕的应用场景示意图;Figure 1 is a schematic diagram of an application scenario of a flexible screen in the prior art;
图2为本申请实施例提供的一种电子设备的结构示意图一;FIG. 2 is a first structural diagram of an electronic device according to an embodiment of the application;
图3为本申请实施例提供的一种电子设备中柔性屏幕的结构示意图一;FIG. 3 is a first structural diagram of a flexible screen in an electronic device provided by an embodiment of the application;
图4为本申请实施例提供的一种电子设备中柔性屏幕的结构示意图二;4 is a schematic diagram 2 of the structure of a flexible screen in an electronic device provided by an embodiment of the application;
图5为本申请实施例提供的一种柔性屏幕的显示方法的应用场景示意图一;5 is a schematic diagram 1 of an application scenario of a flexible screen display method provided by an embodiment of the application;
图6为本申请实施例提供的一种柔性屏幕的显示方法的应用场景示意图二;6 is a schematic diagram 2 of an application scenario of a flexible screen display method provided by an embodiment of the application;
图7为本申请实施例提供的一种电子设备内操作***的结构示意图;FIG. 7 is a schematic structural diagram of an operating system in an electronic device provided by an embodiment of this application;
图8为本申请实施例提供的一种柔性屏幕的显示方法的应用场景示意图三;FIG. 8 is a third schematic diagram of an application scenario of a flexible screen display method provided by an embodiment of the application;
图9为本申请实施例提供的一种柔性屏幕的显示方法的应用场景示意图四;9 is a schematic diagram 4 of an application scenario of a flexible screen display method provided by an embodiment of the application;
图10为本申请实施例提供的一种柔性屏幕的显示方法的应用场景示意图五;10 is a schematic diagram 5 of an application scenario of a flexible screen display method provided by an embodiment of the application;
图11为本申请实施例提供的一种柔性屏幕的显示方法的应用场景示意图六;11 is a sixth schematic diagram of an application scenario of a flexible screen display method provided by an embodiment of the application;
图12为本申请实施例提供的一种柔性屏幕的显示方法的应用场景示意图七;12 is a schematic diagram 7 of an application scenario of a flexible screen display method provided by an embodiment of the application;
图13为本申请实施例提供的一种柔性屏幕的显示方法的应用场景示意图八;FIG. 13 is an eighth schematic diagram of an application scenario of a flexible screen display method provided by an embodiment of this application;
图14为本申请实施例提供的一种柔性屏幕的显示方法的应用场景示意图九;14 is a schematic diagram 9 of an application scenario of a flexible screen display method provided by an embodiment of this application;
图15为本申请实施例提供的一种柔性屏幕的显示方法的应用场景示意图十;15 is a schematic diagram ten of an application scenario of a flexible screen display method provided by an embodiment of this application;
图16为本申请实施例提供的一种柔性屏幕的显示方法的应用场景示意图十一;16 is a schematic diagram eleventh of an application scenario of a flexible screen display method provided by an embodiment of this application;
图17为本申请实施例提供的一种柔性屏幕的显示方法的应用场景示意图十二;FIG. 17 is a twelfth schematic diagram of an application scenario of a flexible screen display method provided by an embodiment of this application;
图18为本申请实施例提供的一种柔性屏幕的显示方法的应用场景示意图十三;18 is a schematic diagram 13 of an application scenario of a flexible screen display method provided by an embodiment of this application;
图19为本申请实施例提供的一种柔性屏幕的显示方法的应用场景示意图十四;FIG. 19 is a fourteenth schematic diagram of an application scenario of a flexible screen display method provided by an embodiment of this application;
图20为本申请实施例提供的一种柔性屏幕的显示方法的应用场景示意图十五;20 is a fifteenth schematic diagram of an application scenario of a flexible screen display method provided by an embodiment of this application;
图21为本申请实施例提供的一种柔性屏幕的显示方法的应用场景示意图十六;FIG. 21 is a sixteenth schematic diagram of an application scenario of a flexible screen display method provided by an embodiment of this application;
图22为本申请实施例提供的一种电子设备的结构示意图二。FIG. 22 is a second structural diagram of an electronic device provided by an embodiment of this application.
下面将结合附图对本实施例的实施方式进行详细描述。The implementation of this embodiment will be described in detail below with reference to the accompanying drawings.
应当理解,在本申请以下各实施例中,“至少一个”、“一个或多个”是指一个、两个或两个以上。术语“和/或”,用于描述关联对象的关联关系,表示可以存在三种关系;例如,A和/或B,可以表示:单独存在A,同时存在A和B,单独存在B的情况,其中A、B可以是单数或者复数。It should be understood that in the following embodiments of the present application, "at least one" and "one or more" refer to one, two or more than two. The term "and/or" is used to describe the association relationship of associated objects, which means that there can be three kinds of relationships; for example, A and/or B can mean: A alone exists, A and B exist at the same time, and B exists alone. Among them, A and B can be singular or plural.
本申请实施例提供的一种柔性屏幕的显示方法可应用于手机、平板电脑、笔记本电脑、超级移动个人计算机(ultra-mobile personal computer,UMPC)、手持计算机、上网本、个人数字助理(personal digital assistant,PDA)、可穿戴电子设备、虚拟现实设备等电子设备中,本申请实施例对此不做任何限制。The flexible screen display method provided in the embodiments of this application can be applied to mobile phones, tablet computers, notebook computers, ultra-mobile personal computers (UMPC), handheld computers, netbooks, and personal digital assistants (personal digital assistants). , PDA), wearable electronic devices, virtual reality devices, and other electronic devices, the embodiments of the present application do not make any restrictions on this.
示例性的,图2示出了电子设备100的结构示意图。Exemplarily, FIG. 2 shows a schematic structural diagram of the electronic device 100.
电子设备100可以包括处理器110,外部存储器接口120,内部存储器121,通用串行总线(universal serial bus,USB)接口130,充电管理模块140,电源管理模块141,电池142,天线1,天线2,移动通信模块150,无线通信模块160,音频模块170,扬声器170A,受话器170B,麦克风170C,耳机接口170D,传感器模块180,摄像头193,显示屏194等。The electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, and an antenna 2. , Mobile communication module 150, wireless communication module 160, audio module 170, speaker 170A, receiver 170B, microphone 170C, earphone interface 170D, sensor module 180, camera 193, display screen 194, etc.
可以理解的是,本发明实施例示意的结构并不构成对电子设备100的具体限定。在本申请另一些实施例中,电子设备100可以包括比图示更多或更少的部件,或者组合某些部件,或者拆分某些部件,或者不同的部件布置。图示的部件可以以硬件,软件或软件和硬件的组合实现。It can be understood that the structure illustrated in the embodiment of the present invention does not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer components than shown, or combine certain components, or split certain components, or arrange different components. The illustrated components can be implemented in hardware, software, or a combination of software and hardware.
处理器110可以包括一个或多个处理单元,例如:处理器110可以包括应用处理器(application processor,AP),调制解调处理器,图形处理器(graphics processing unit,GPU),图像信号处理器(image signal processor,ISP),控制器,视频编解码器,数字信号处理器(digital signal processor,DSP),基带处理器,和/或神经网络处理器(neural-network processing unit,NPU)等。其中,不同的处理单元可以是独立的器件,也可以集成在一个或多个处理器中。The processor 110 may include one or more processing units. For example, the processor 110 may include an application processor (AP), a modem processor, a graphics processing unit (GPU), and an image signal processor. (image signal processor, ISP), controller, video codec, digital signal processor (digital signal processor, DSP), baseband processor, and/or neural-network processing unit (NPU), etc. Among them, the different processing units may be independent devices or integrated in one or more processors.
处理器110中还可以设置存储器,用于存储指令和数据。在一些实施例中,处理器110中的存储器为高速缓冲存储器。该存储器可以保存处理器110刚用过或循环使用的指令或数据。如果处理器110需要再次使用该指令或数据,可从所述存储器中直接调用。避免了重复存取,减少了处理器110的等待时间,因而提高了***的效率。A memory may also be provided in the processor 110 to store instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory can store instructions or data that have just been used or recycled by the processor 110. If the processor 110 needs to use the instruction or data again, it can be directly called from the memory. Repeated accesses are avoided, the waiting time of the processor 110 is reduced, and the efficiency of the system is improved.
在一些实施例中,处理器110可以包括一个或多个接口。接口可以包括集成电路(inter-integrated circuit,I2C)接口,集成电路内置音频(inter-integrated circuit sound,I2S)接口,脉冲编码调制(pulse code modulation,PCM)接口,通用异步收发传输器(universal asynchronous receiver/transmitter,UART)接口,移动产业处理器接口(mobile industry processor interface,MIPI),通用输入输出(general-purpose input/output,GPIO)接口,用户标识模块(subscriber identity module,SIM)接口,和/或通用串行总线(universal serial bus,USB)接口等。In some embodiments, the processor 110 may include one or more interfaces. The interface may include an integrated circuit (inter-integrated circuit, I2C) interface, an integrated circuit built-in audio (inter-integrated circuit sound, I2S) interface, a pulse code modulation (PCM) interface, and a universal asynchronous transmitter receiver/transmitter, UART) interface, mobile industry processor interface (MIPI), general-purpose input/output (GPIO) interface, subscriber identity module (SIM) interface, and / Or Universal Serial Bus (USB) interface, etc.
充电管理模块140用于从充电器接收充电输入。其中,充电器可以是无线充电器,也可以是有线充电器。在一些有线充电的实施例中,充电管理模块140可以通过USB接口130接 收有线充电器的充电输入。在一些无线充电的实施例中,充电管理模块140可以通过电子设备100的无线充电线圈接收无线充电输入。充电管理模块140为电池142充电的同时,还可以通过电源管理模块141为电子设备供电。The charging management module 140 is used to receive charging input from the charger. Among them, the charger can be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 140 may receive the charging input of the wired charger through the USB interface 130. In some embodiments of wireless charging, the charging management module 140 may receive the wireless charging input through the wireless charging coil of the electronic device 100. While the charging management module 140 charges the battery 142, it can also supply power to the electronic device through the power management module 141.
电源管理模块141用于连接电池142,充电管理模块140与处理器110。电源管理模块141接收电池142和/或充电管理模块140的输入,为处理器110,内部存储器121,显示屏194,摄像头193,和无线通信模块160等供电。在其他一些实施例中,电源管理模块141也可以设置于处理器110中。在另一些实施例中,电源管理模块141和充电管理模块140也可以设置于同一个器件中。The power management module 141 is used to connect the battery 142, the charging management module 140 and the processor 110. The power management module 141 receives input from the battery 142 and/or the charging management module 140, and supplies power to the processor 110, the internal memory 121, the display screen 194, the camera 193, and the wireless communication module 160. In some other embodiments, the power management module 141 may also be provided in the processor 110. In other embodiments, the power management module 141 and the charging management module 140 may also be provided in the same device.
电子设备100的无线通信功能可以通过天线1,天线2,移动通信模块150,无线通信模块160,调制解调处理器以及基带处理器等实现。The wireless communication function of the electronic device 100 can be implemented by the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor, and the baseband processor.
天线1和天线2用于发射和接收电磁波信号。电子设备100中的每个天线可用于覆盖单个或多个通信频带。不同的天线还可以复用,以提高天线的利用率。例如:可以将天线1复用为无线局域网的分集天线。在另外一些实施例中,天线可以和调谐开关结合使用。The antenna 1 and the antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in the electronic device 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve antenna utilization. For example, antenna 1 can be multiplexed as a diversity antenna of a wireless local area network. In other embodiments, the antenna can be used in combination with a tuning switch.
移动通信模块150可以提供应用在电子设备100上的包括2G/3G/4G/5G等无线通信的解决方案。移动通信模块150可以包括一个或多个滤波器,开关,功率放大器,低噪声放大器(low noise amplifier,LNA)等。移动通信模块150可以由天线1接收电磁波,并对接收的电磁波进行滤波,放大等处理,传送至调制解调处理器进行解调。移动通信模块150还可以对经调制解调处理器调制后的信号放大,经天线1转为电磁波辐射出去。在一些实施例中,移动通信模块150的至少部分功能模块可以被设置于处理器110中。在一些实施例中,移动通信模块150的至少部分功能模块可以与处理器110的至少部分模块被设置在同一个器件中。The mobile communication module 150 can provide a wireless communication solution including 2G/3G/4G/5G and the like applied to the electronic device 100. The mobile communication module 150 may include one or more filters, switches, power amplifiers, low noise amplifiers (LNA), etc. The mobile communication module 150 can receive electromagnetic waves by the antenna 1, and perform processing such as filtering and amplifying the received electromagnetic waves, and then transmitting them to the modem processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modem processor, and convert it into electromagnetic waves for radiation via the antenna 1. In some embodiments, at least part of the functional modules of the mobile communication module 150 may be provided in the processor 110. In some embodiments, at least part of the functional modules of the mobile communication module 150 and at least part of the modules of the processor 110 may be provided in the same device.
调制解调处理器可以包括调制器和解调器。其中,调制器用于将待发送的低频基带信号调制成中高频信号。解调器用于将接收的电磁波信号解调为低频基带信号。随后解调器将解调得到的低频基带信号传送至基带处理器处理。低频基带信号经基带处理器处理后,被传递给应用处理器。应用处理器通过音频设备(不限于扬声器170A,受话器170B等)输出声音信号,或通过显示屏194显示图像或视频。在一些实施例中,调制解调处理器可以是独立的器件。在另一些实施例中,调制解调处理器可以独立于处理器110,与移动通信模块150或其他功能模块设置在同一个器件中。The modem processor may include a modulator and a demodulator. Among them, the modulator is used to modulate the low frequency baseband signal to be sent into a medium and high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low-frequency baseband signal. Then the demodulator transmits the demodulated low-frequency baseband signal to the baseband processor for processing. The low-frequency baseband signal is processed by the baseband processor and then passed to the application processor. The application processor outputs a sound signal through an audio device (not limited to the speaker 170A, the receiver 170B, etc.), or displays an image or video through the display screen 194. In some embodiments, the modem processor may be an independent device. In other embodiments, the modem processor may be independent of the processor 110 and be provided in the same device as the mobile communication module 150 or other functional modules.
无线通信模块160可以提供应用在电子设备100上的包括无线局域网(wireless local area networks,WLAN)(如无线保真(wireless fidelity,Wi-Fi)网络),蓝牙(Bluetooth,BT),全球导航卫星***(global navigation satellite system,GNSS),调频(frequency modulation,FM),近距离无线通信技术(near field communication,NFC),红外技术(infrared,IR)等无线通信的解决方案。无线通信模块160可以是集成一个或多个通信处理模块的一个或多个器件。无线通信模块160经由天线2接收电磁波,将电磁波信号调频以及滤波处理,将处理后的信号发送到处理器110。无线通信模块160还可以从处理器110接收待发送的信号,对其进行调频,放大,经天线2转为电磁波辐射出去。The wireless communication module 160 can provide applications on the electronic device 100 including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), and global navigation satellites. System (global navigation satellite system, GNSS), frequency modulation (frequency modulation, FM), near field communication technology (near field communication, NFC), infrared technology (infrared, IR) and other wireless communication solutions. The wireless communication module 160 may be one or more devices integrating one or more communication processing modules. The wireless communication module 160 receives electromagnetic waves via the antenna 2, frequency modulates and filters the electromagnetic wave signals, and sends the processed signals to the processor 110. The wireless communication module 160 can also receive the signal to be sent from the processor 110, perform frequency modulation, amplify it, and convert it into electromagnetic wave radiation via the antenna 2.
在一些实施例中,电子设备100的天线1和移动通信模块150耦合,天线2和无线通信模块160耦合,使得电子设备100可以通过无线通信技术与网络以及其他设备通信。所述无线通信技术可以包括全球移动通讯***(global system for mobile communications,GSM),通用分组无线服务(general packet radio service,GPRS),码分多址接入(code division multiple access,CDMA),宽带码分多址(wideband code division multiple access,WCDMA),时分码分多址(time-division code division multiple access,TD-SCDMA),长期演进(long term evolution, LTE),BT,GNSS,WLAN,NFC,FM,和/或IR技术等。所述GNSS可以包括全球卫星定位***(global positioning system,GPS),全球导航卫星***(global navigation satellite system,GLONASS),北斗卫星导航***(beidou navigation satellite system,BDS),准天顶卫星***(quasi-zenith satellite system,QZSS)和/或星基增强***(satellite based augmentation systems,SBAS)。In some embodiments, the antenna 1 of the electronic device 100 is coupled with the mobile communication module 150, and the antenna 2 is coupled with the wireless communication module 160, so that the electronic device 100 can communicate with the network and other devices through wireless communication technology. The wireless communication technologies may include global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), broadband Code division multiple access (wideband code division multiple access, WCDMA), time-division code division multiple access (TD-SCDMA), long term evolution (LTE), BT, GNSS, WLAN, NFC , FM, and/or IR technology, etc. The GNSS may include global positioning system (GPS), global navigation satellite system (GLONASS), Beidou navigation satellite system (BDS), quasi-zenith satellite system (quasi -zenith satellite system, QZSS) and/or satellite-based augmentation systems (SBAS).
电子设备100通过GPU,显示屏194,以及应用处理器等实现显示功能。GPU为图像处理的微处理器,连接显示屏194和应用处理器。GPU用于执行数学和几何计算,用于图形渲染。处理器110可包括一个或多个GPU,其执行程序指令以生成或改变显示信息。The electronic device 100 implements a display function through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, connected to the display 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs, which execute program instructions to generate or change display information.
在本申请实施例中,显示屏194具体可以为柔性屏幕。柔性屏幕中可包括显示器件和触摸传感器。显示器件用于向用户输出显示内容,触摸传感器用于接收用户在柔性屏幕上输入的触摸操作。In the embodiment of the present application, the display screen 194 may specifically be a flexible screen. The flexible screen may include a display device and a touch sensor. The display device is used to output display content to the user, and the touch sensor is used to receive a touch operation input by the user on the flexible screen.
如图3中的(a)所示,柔性屏幕301在展开状态下可作为一块完整的显示区域进行显示。用户可以沿柔性屏幕301中的一条或多条折叠线折叠屏幕。其中,折叠线的位置可以是预先设置的,也可以是用户在柔性屏幕301的任意位置折叠后形成的。As shown in Fig. 3(a), the flexible screen 301 can be displayed as a complete display area in the unfolded state. The user can fold the screen along one or more folding lines in the flexible screen 301. Wherein, the position of the folding line may be preset, or may be formed by the user after folding any position of the flexible screen 301.
如图3中的(b)所示,用户沿柔性屏幕301中的折叠线AB折叠柔性屏幕301后,柔性屏幕301可沿折叠线AB被划分为两个显示区域,即显示区域1和显示区域2。在本申请实施例中,折叠后的显示区域1和显示区域2可以作为两个独立的显示区域进行显示。例如,可以将显示区域1称为手机100的主屏,将显示区域2称为手机100的副屏。主屏和副屏的显示面积可以相同或不同。As shown in Figure 3(b), after the user folds the flexible screen 301 along the fold line AB in the flexible screen 301, the flexible screen 301 can be divided into two display areas along the fold line AB, namely, display area 1 and display area 2. In the embodiment of the present application, the folded display area 1 and the display area 2 can be displayed as two independent display areas. For example, the display area 1 may be referred to as the main screen of the mobile phone 100, and the display area 2 may be referred to as the secondary screen of the mobile phone 100. The display area of the main screen and the secondary screen can be the same or different.
示例性的,如图4所示,柔性屏幕的大小为2200*2480(单位为像素)。其中,柔性屏幕上折叠线AB的宽度为166。沿折叠线AB折叠后,柔性屏幕右侧大小为1144*2480的区域被划分为主屏41,柔性屏幕左侧大小为890*2480的区域被划分为副屏42。Exemplarily, as shown in FIG. 4, the size of the flexible screen is 2200*2480 (in pixels). The width of the folding line AB on the flexible screen is 166. After folding along the folding line AB, the area on the right side of the flexible screen with a size of 1144*2480 is divided into the main screen 41, and the area on the left side of the flexible screen with a size of 890*2480 is divided into the secondary screen 42.
当用户折叠柔性屏幕之后,被划分出的主屏41和副屏42之间呈一定夹角。在本申请实施例中,手机100可通过一个或多个传感器检测到的数据计算主屏41和副屏42之间的夹角。After the user folds the flexible screen, the divided main screen 41 and the secondary screen 42 form a certain angle. In the embodiment of the present application, the mobile phone 100 can calculate the angle between the main screen 41 and the sub screen 42 based on the data detected by one or more sensors.
示例性的,如图5所示,可以在柔性屏幕的主屏41和副屏42上分别设置陀螺仪和加速度传感器。主屏41上的陀螺仪可检测主屏41转动时的转动角速度,主屏41上的加速度传感器可检测主屏41运动时产生的加速度。进而,手机100可根据主屏41上陀螺仪和加速度传感器检测到的数据确定重力G的大小和方向。同样,手机100也可根据副屏42上陀螺仪和加速度传感器检测到的数据确定重力G的大小和方向。或者,如果使用传感器A能够检测出重力G的大小和方向,则可在主屏41和副屏42上分别设置该传感器A。例如,可以将陀螺仪和加速度传感器集成为一个传感器分别设置在主屏41和副屏42上。Exemplarily, as shown in FIG. 5, a gyroscope and an acceleration sensor may be provided on the main screen 41 and the auxiliary screen 42 of the flexible screen, respectively. The gyroscope on the main screen 41 can detect the rotation angular velocity when the main screen 41 rotates, and the acceleration sensor on the main screen 41 can detect the acceleration generated when the main screen 41 moves. Furthermore, the mobile phone 100 can determine the magnitude and direction of the gravity G according to the data detected by the gyroscope and the acceleration sensor on the main screen 41. Similarly, the mobile phone 100 can also determine the magnitude and direction of the gravity G according to the data detected by the gyroscope and the acceleration sensor on the secondary screen 42. Alternatively, if the size and direction of the gravity G can be detected using the sensor A, the sensor A can be provided on the main screen 41 and the sub screen 42 respectively. For example, a gyroscope and an acceleration sensor can be integrated into one sensor and arranged on the main screen 41 and the auxiliary screen 42 respectively.
仍如图5所示,可以在主屏41和副屏42上分别设置有对应的坐标系。例如,可以在副屏42中设置笛卡尔坐标系O1,笛卡尔坐标系O1中x轴与副屏42较短的侧边平行,y轴与副屏42较长的侧边平行,z轴垂直于x轴和y轴组成的平面指向副屏42外。同样,可以在主屏41中设置笛卡尔坐标系O2,笛卡尔坐标系O2中x轴与主屏41较短的侧边平行,y轴与主屏41较长的侧边平行,z轴垂直于x轴和y轴组成的平面指向主屏41内。As still shown in FIG. 5, corresponding coordinate systems may be set on the main screen 41 and the sub screen 42 respectively. For example, a Cartesian coordinate system O1 can be set in the secondary screen 42. In the Cartesian coordinate system O1, the x axis is parallel to the shorter side of the secondary screen 42, the y axis is parallel to the longer side of the secondary screen 42, and the z axis is vertical. The plane formed by the x-axis and the y-axis points out of the secondary screen 42. Similarly, a Cartesian coordinate system O2 can be set in the main screen 41. In the Cartesian coordinate system O2, the x-axis is parallel to the shorter side of the main screen 41, the y-axis is parallel to the longer side of the main screen 41, and the z-axis is perpendicular to the x-axis. The plane formed by the y axis points to the inside of the main screen 41.
示例性的,副屏42中的陀螺仪和加速度传感器可在笛卡尔坐标系O1中检测出重力G的大小和方向,主屏41中的陀螺仪和加速度传感器可在笛卡尔坐标系O2中检测出重力G的大小和方向。由于笛卡尔坐标系O1与笛卡尔坐标系O2中y轴的指向相同,因此,重力G在笛卡尔坐标系O1中x轴和z轴平面上的分量G1,与重力G在笛卡尔坐标系O2中x轴和z轴平面上的分量G2大小相等但方向不同。此时,分量G1与分量G2之间的夹角为β,副屏 42与主屏41之间夹角α=360°-β。Exemplarily, the gyroscope and acceleration sensor in the secondary screen 42 can detect the magnitude and direction of gravity G in the Cartesian coordinate system O1, and the gyroscope and acceleration sensor in the main screen 41 can detect the magnitude and direction of the gravity G in the Cartesian coordinate system O2. The magnitude and direction of gravity G. Since the direction of the y-axis in the Cartesian coordinate system O1 and the Cartesian coordinate system O2 is the same, the component G1 of the gravity G on the x-axis and z-axis planes in the Cartesian coordinate system O1 is the same as the gravity G in the Cartesian coordinate system O2 The components G2 on the x-axis and z-axis planes are equal in magnitude but different in direction. At this time, the angle between the component G1 and the component G2 is β, and the angle between the secondary screen 42 and the main screen 41 is α=360°-β.
因此,电子设备100通过计算重力G在笛卡尔坐标系O1中的分量G1与重力G在笛卡尔坐标系O2中的分量G2之间的夹角β,可得到副屏42与主屏41之间的夹角α。Therefore, the electronic device 100 calculates the angle β between the component G1 of the gravity G in the Cartesian coordinate system O1 and the component G2 of the gravity G in the Cartesian coordinate system O2 to obtain the difference between the secondary screen 42 and the main screen 41 Angle α.
在另一些实施例中,仍如图5所示,由于笛卡尔坐标系O1与笛卡尔坐标系O2中y轴的指向相同,那么,笛卡尔坐标系O1中x轴与笛卡尔坐标系O2中x轴的夹角也等于β。那么,电子设备100还可以检测笛卡尔坐标系O1中x轴与笛卡尔坐标系O2中x轴的夹角β,进而计算副屏42与主屏41之间夹角α=360°-β。In other embodiments, as shown in FIG. 5, since the direction of the y-axis in the Cartesian coordinate system O1 and the Cartesian coordinate system O2 are the same, then the x-axis in the Cartesian coordinate system O1 and the Cartesian coordinate system O2 The included angle of the x-axis is also equal to β. Then, the electronic device 100 can also detect the included angle β between the x-axis in the Cartesian coordinate system O1 and the x-axis in the Cartesian coordinate system O2, and then calculate the included angle α=360°-β between the secondary screen 42 and the main screen 41.
在另一些实施例中,仍如图5所示,由于笛卡尔坐标系O1与笛卡尔坐标系O2中y轴的指向相同,那么,笛卡尔坐标系O1中z轴与笛卡尔坐标系O2中z轴的夹角也等于β。那么,电子设备100还可以检测笛卡尔坐标系O1中z轴与笛卡尔坐标系O2中z轴的夹角β,进而计算副屏42与主屏41之间夹角α=360°-β。In other embodiments, as shown in FIG. 5, since the direction of the y-axis in the Cartesian coordinate system O1 and the Cartesian coordinate system O2 are the same, then the z-axis in the Cartesian coordinate system O1 and the Cartesian coordinate system O2 The included angle of the z axis is also equal to β. Then, the electronic device 100 can also detect the included angle β between the z-axis in the Cartesian coordinate system O1 and the z-axis in the Cartesian coordinate system O2, and then calculate the included angle α=360°-β between the secondary screen 42 and the main screen 41.
示例性的,当电子设备100的柔性屏幕为内折式柔性屏幕时,副屏42与主屏41之间的夹角α可在0至180°构成的闭区间内变化。当电子设备100的柔性屏幕为外折式柔性屏幕时,副屏42与主屏41之间的夹角α可在180°至360°构成的闭区间内变化。又或者,副屏42与主屏41之间的夹角α也可以在0至360°构成的闭区间内变化,本申请实施例对此不做任何限制。Exemplarily, when the flexible screen of the electronic device 100 is an inward-folding flexible screen, the angle α between the secondary screen 42 and the main screen 41 may vary within a closed interval formed by 0 to 180°. When the flexible screen of the electronic device 100 is an outward-folding flexible screen, the included angle α between the secondary screen 42 and the main screen 41 may vary in a closed interval formed by 180° to 360°. Alternatively, the included angle α between the secondary screen 42 and the main screen 41 may also vary within a closed interval formed by 0 to 360°, which is not limited in the embodiment of the present application.
如图6中的(a)所示,当柔性屏幕中主屏41和副屏42之间的夹角α在[90°,180°)的范围内,且主屏41和副屏42均为横屏状态时,可将此时柔性屏幕的物理状态称为支架状态。如图6中的(b)所示,当主屏41和副屏42之间的夹角α为180°时,可将此时柔性屏幕的物理状态称为展开状态。需要说明的是,上述判断柔性屏幕的物理状态的预设值(例如90°,180°)均为举例说明,在实际应用中本领域技术人员可以根据实际需要设置上述预设值。例如,当检测到主屏41和副屏42之间的夹角增大至175°时,可定义此时柔性屏幕处于展开状态。又例如,当检测到主屏41和副屏42之间的夹角从180°减小至170°,且主屏41和副屏42均为横屏状态时,可定义此时柔性屏幕处于支架状态,本申请实施例对此不做任何限制。As shown in (a) in Figure 6, when the angle α between the main screen 41 and the secondary screen 42 in the flexible screen is in the range of [90°, 180°), and the primary screen 41 and the secondary screen 42 are both horizontal screens In the state, the physical state of the flexible screen at this time can be called the stand state. As shown in FIG. 6(b), when the included angle α between the main screen 41 and the sub screen 42 is 180°, the physical state of the flexible screen at this time can be referred to as the unfolded state. It should be noted that the foregoing preset values (for example, 90°, 180°) for determining the physical state of the flexible screen are all examples, and those skilled in the art can set the foregoing preset values according to actual needs in practical applications. For example, when it is detected that the included angle between the main screen 41 and the sub screen 42 increases to 175°, it can be defined that the flexible screen is in the unfolded state at this time. For another example, when it is detected that the angle between the main screen 41 and the secondary screen 42 is reduced from 180° to 170°, and the primary screen 41 and the secondary screen 42 are both in the landscape state, it can be defined that the flexible screen is in the support state at this time. The embodiments of the application do not impose any restriction on this.
其中,横屏状态是指:主屏41和副屏42的高小于宽的状态,此时显示屏(例如主屏41或副屏42)的高宽比(即高和宽的比)小于1。其中,显示屏的高也可以理解为显示屏的纵向长度,显示屏的宽也可以理解为显示屏的横向长度,显示屏的高宽比也可以理解为显示屏的纵横比(即纵向长度与横向长度的比)。相应的,主屏41和副屏42也可处于竖屏状态。竖屏状态是指:主屏41和副屏42的高大于宽状态,此时显示屏的高宽比(即高和宽的比)大于1。Among them, the horizontal screen state refers to a state where the height of the main screen 41 and the sub screen 42 is less than the width. At this time, the aspect ratio (that is, the ratio of height to width) of the display screen (for example, the main screen 41 or the sub screen 42) is less than 1. Among them, the height of the display can also be understood as the vertical length of the display, the width of the display can also be understood as the horizontal length of the display, and the aspect ratio of the display can also be understood as the aspect ratio of the display (ie, the vertical length and The ratio of lateral length). Correspondingly, the main screen 41 and the sub screen 42 may also be in a vertical screen state. The vertical screen state refers to the state where the height of the main screen 41 and the secondary screen 42 is greater than the width, and the aspect ratio (that is, the ratio of height to width) of the display screen is greater than 1.
示例性的,电子设备100可以通过主屏41上的加速度传感器和陀螺仪确定主屏41当前是横屏状态还是竖屏状态;同样,电子设备100可以通过副屏42上的加速度传感器和陀螺仪确定副屏42当前是横屏状态还是竖屏状态。Exemplarily, the electronic device 100 can determine whether the main screen 41 is currently in the landscape state or the portrait state through the acceleration sensor and the gyroscope on the main screen 41; similarly, the electronic device 100 can determine the secondary screen through the acceleration sensor and the gyroscope on the secondary screen 42. The screen 42 is currently in a landscape state or a portrait state.
电子设备100可以通过ISP,摄像头193,视频编解码器,GPU,显示屏194以及应用处理器等实现拍摄功能。The electronic device 100 can implement a shooting function through an ISP, a camera 193, a video codec, a GPU, a display screen 194, and an application processor.
ISP用于处理摄像头193反馈的数据。例如,拍照时,打开快门,光线通过镜头被传递到摄像头感光元件上,光信号转换为电信号,摄像头感光元件将所述电信号传递给ISP处理,转化为肉眼可见的图像。ISP还可以对图像的噪点,亮度,肤色进行算法优化。ISP还可以对拍摄场景的曝光,色温等参数优化。在一些实施例中,ISP可以设置在摄像头193中。The ISP is used to process the data fed back from the camera 193. For example, when taking a picture, the shutter is opened, the light is transmitted to the photosensitive element of the camera through the lens, the light signal is converted into an electrical signal, and the photosensitive element of the camera transfers the electrical signal to the ISP for processing and is converted into an image visible to the naked eye. ISP can also optimize the image noise, brightness, and skin color. ISP can also optimize the exposure, color temperature and other parameters of the shooting scene. In some embodiments, the ISP may be provided in the camera 193.
摄像头193用于捕获静态图像或视频。物体通过镜头生成光学图像投射到感光元件。感 光元件可以是电荷耦合器件(charge coupled device,CCD)或互补金属氧化物半导体(complementary metal-oxide-semiconductor,CMOS)光电晶体管。感光元件把光信号转换成电信号,之后将电信号传递给ISP转换成数字图像信号。ISP将数字图像信号输出到DSP加工处理。DSP将数字图像信号转换成标准的RGB,YUV等格式的图像信号。在一些实施例中,电子设备100可以包括1个或N个摄像头193,N为大于1的正整数。The camera 193 is used to capture still images or videos. The object generates an optical image through the lens and projects it to the photosensitive element. The light-sensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal, and then transmits the electrical signal to the ISP to convert it into a digital image signal. ISP outputs digital image signals to DSP for processing. DSP converts digital image signals into standard RGB, YUV and other formats. In some embodiments, the electronic device 100 may include 1 or N cameras 193, and N is a positive integer greater than 1.
数字信号处理器用于处理数字信号,除了可以处理数字图像信号,还可以处理其他数字信号。例如,当电子设备100在频点选择时,数字信号处理器用于对频点能量进行傅里叶变换等。Digital signal processors are used to process digital signals. In addition to digital image signals, they can also process other digital signals. For example, when the electronic device 100 selects the frequency point, the digital signal processor is used to perform Fourier transform on the energy of the frequency point.
视频编解码器用于对数字视频压缩或解压缩。电子设备100可以支持一种或多种视频编解码器。这样,电子设备100可以播放或录制多种编码格式的视频,例如:动态图像专家组(moving picture experts group,MPEG)1,MPEG2,MPEG3,MPEG4等。Video codecs are used to compress or decompress digital video. The electronic device 100 may support one or more video codecs. In this way, the electronic device 100 can play or record videos in a variety of encoding formats, such as: moving picture experts group (MPEG) 1, MPEG2, MPEG3, MPEG4, and so on.
外部存储器接口120可以用于连接外部存储卡,例如Micro SD卡,实现扩展电子设备100的存储能力。外部存储卡通过外部存储器接口120与处理器110通信,实现数据存储功能。例如将音乐,视频等文件保存在外部存储卡中。The external memory interface 120 may be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 through the external memory interface 120 to realize the data storage function. For example, save music, video and other files in an external memory card.
内部存储器121可以用于存储一个或多个计算机程序,该一个或多个计算机程序包括指令。处理器110可以通过运行存储在内部存储器121的上述指令,从而使得电子设备100执行本申请一些实施例中所提供的显示方法,以及各种功能应用和数据处理等。内部存储器121可以包括存储程序区和存储数据区。其中,存储程序区可存储操作***;该存储程序区还可以存储一个或多个应用程序(比如图库、联系人等)等。存储数据区可存储电子设备101使用过程中所创建的数据(比如照片,联系人等)等。此外,内部存储器121可以包括高速随机存取存储器,还可以包括非易失性存储器,例如一个或多个磁盘存储器件,闪存器件,通用闪存存储器(universal flash storage,UFS)等。在另一些实施例中,处理器110通过运行存储在内部存储器121的指令,和/或存储在设置于处理器中的存储器的指令,来使得电子设备100执行本申请实施例中提供的显示方法,以及各种功能应用和数据处理。The internal memory 121 may be used to store one or more computer programs, and the one or more computer programs include instructions. The processor 110 can execute the above-mentioned instructions stored in the internal memory 121 to enable the electronic device 100 to execute the display methods provided in some embodiments of the present application, as well as various functional applications and data processing. The internal memory 121 may include a storage program area and a storage data area. Among them, the storage program area can store the operating system; the storage program area can also store one or more application programs (such as a gallery, contacts, etc.) and so on. The data storage area can store data (such as photos, contacts, etc.) created during the use of the electronic device 101. In addition, the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic disk storage devices, flash memory devices, universal flash storage (UFS), etc. In other embodiments, the processor 110 executes the instructions stored in the internal memory 121 and/or the instructions stored in the memory provided in the processor to cause the electronic device 100 to execute the display method provided in the embodiments of the present application. , And various functional applications and data processing.
电子设备100可以通过音频模块170,扬声器170A,受话器170B,麦克风170C,耳机接口170D,以及应用处理器等实现音频功能。例如音乐播放,录音等。The electronic device 100 can implement audio functions through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the earphone interface 170D, and the application processor. For example, music playback, recording, etc.
音频模块170用于将数字音频信息转换成模拟音频信号输出,也用于将模拟音频输入转换为数字音频信号。音频模块170还可以用于对音频信号编码和解码。在一些实施例中,音频模块170可以设置于处理器110中,或将音频模块170的部分功能模块设置于处理器110中。The audio module 170 is used to convert digital audio information into an analog audio signal for output, and is also used to convert an analog audio input into a digital audio signal. The audio module 170 can also be used to encode and decode audio signals. In some embodiments, the audio module 170 may be provided in the processor 110, or part of the functional modules of the audio module 170 may be provided in the processor 110.
扬声器170A,也称“喇叭”,用于将音频电信号转换为声音信号。电子设备100可以通过扬声器170A收听音乐,或收听免提通话。The speaker 170A, also called a "speaker", is used to convert audio electrical signals into sound signals. The electronic device 100 can listen to music through the speaker 170A, or listen to a hands-free call.
受话器170B,也称“听筒”,用于将音频电信号转换成声音信号。当电子设备100接听电话或语音信息时,可以通过将受话器170B靠近人耳接听语音。The receiver 170B, also called "earpiece", is used to convert audio electrical signals into sound signals. When the electronic device 100 answers a call or voice message, it can receive the voice by bringing the receiver 170B close to the human ear.
麦克风170C,也称“话筒”,“传声器”,用于将声音信号转换为电信号。当拨打电话或发送语音信息时,用户可以通过人嘴靠近麦克风170C发声,将声音信号输入到麦克风170C。电子设备100可以设置一个或多个麦克风170C。在另一些实施例中,电子设备100可以设置两个麦克风170C,除了采集声音信号,还可以实现降噪功能。在另一些实施例中,电子设备100还可以设置三个,四个或更多麦克风170C,实现采集声音信号,降噪,还可以识别声音来源,实现定向录音功能等。The microphone 170C, also called "microphone", "microphone", is used to convert sound signals into electrical signals. When making a call or sending a voice message, the user can approach the microphone 170C through the mouth to make a sound, and input the sound signal to the microphone 170C. The electronic device 100 may be provided with one or more microphones 170C. In other embodiments, the electronic device 100 may be provided with two microphones 170C, which can implement noise reduction functions in addition to collecting sound signals. In some other embodiments, the electronic device 100 can also be provided with three, four or more microphones 170C to collect sound signals, reduce noise, identify sound sources, and realize directional recording functions.
耳机接口170D用于连接有线耳机。耳机接口170D可以是USB接口130,也可以是3.5mm 的开放移动电子设备平台(open mobile terminal platform,OMTP)标准接口,美国蜂窝电信工业协会(cellular telecommunications industry association of the USA,CTIA)标准接口。The earphone interface 170D is used to connect wired earphones. The earphone interface 170D may be a USB interface 130, or a 3.5mm open mobile terminal platform (OMTP) standard interface, and a cellular telecommunications industry association (cellular telecommunications industry association of the USA, CTIA) standard interface.
传感器模块180可以包括压力传感器,陀螺仪传感器,气压传感器,磁传感器,加速度传感器,距离传感器,接近光传感器,指纹传感器,温度传感器,触摸传感器,环境光传感器,骨传导传感器等。The sensor module 180 may include a pressure sensor, a gyroscope sensor, an air pressure sensor, a magnetic sensor, an acceleration sensor, a distance sensor, a proximity light sensor, a fingerprint sensor, a temperature sensor, a touch sensor, an ambient light sensor, a bone conduction sensor, etc.
另外,上述电子设备100中还可以包括按键、马达、指示器以及SIM卡接口等一种或多种部件,本申请实施例对此不做任何限制。In addition, the aforementioned electronic device 100 may also include one or more components such as buttons, motors, indicators, and SIM card interfaces, which are not limited in the embodiment of the present application.
上述电子设备100的软件***可以采用分层架构,事件驱动架构,微核架构,微服务架构,或云架构。本申请实施例以分层架构的Android***为例,示例性说明电子设备100的软件结构。The software system of the above electronic device 100 may adopt a layered architecture, an event-driven architecture, a microkernel architecture, a microservice architecture, or a cloud architecture. The embodiment of the present application takes a layered Android system as an example to illustrate the software structure of the electronic device 100.
图7是本申请实施例的电子设备100的软件结构框图。FIG. 7 is a software structure block diagram of the electronic device 100 according to an embodiment of the present application.
分层架构将软件分成若干个层,每一层都有清晰的角色和分工。层与层之间通过软件接口通信。在一些实施例中,将Android***分为四层,从上至下分别为应用程序层,应用程序框架层,安卓运行时(Android runtime)和***库,以及内核层。The layered architecture divides the software into several layers, and each layer has a clear role and division of labor. Communication between layers through software interface. In some embodiments, the Android system is divided into four layers, from top to bottom, the application layer, the application framework layer, the Android runtime and system library, and the kernel layer.
1、应用程序层1. Application layer
应用程序层可以包括一系列应用程序。The application layer can include a series of applications.
如图7所示,上述应用程序可以包括通话,联系人,相机,图库,日历,地图,导航,蓝牙,音乐,视频,短信息等APP(应用,application)。As shown in FIG. 7, the above-mentioned application programs may include APPs (applications) such as call, contact, camera, gallery, calendar, map, navigation, Bluetooth, music, video, short message, etc.
在本申请实施例中,一些APP在运行时显示的应用界面中可以既包括图片等无法与用户交互的控件,又包括按钮、进度条等能够与用户交互控件。例如,在运行游戏类应用的应用界面时,该应用界面中可以包括游戏画面以及用于操控游戏的操作手柄。又例如,在运行视频类应用的播放界面时,该播放界面中可以包括正在播放的视频画面以及控制视频播放的控制按钮(例如快进、暂停、弹幕等)。In the embodiments of the present application, the application interface displayed during runtime of some APPs may include not only controls that cannot interact with users, such as pictures, but also controls that can interact with users, such as buttons and progress bars. For example, when running an application interface of a game application, the application interface may include a game screen and an operating handle used to control the game. For another example, when the playback interface of a video application is running, the playback interface may include a video screen being played and control buttons for controlling video playback (such as fast forward, pause, barrage, etc.).
2、应用程序框架层2. Application framework layer
应用程序框架层为应用程序层的应用程序提供应用编程接口(application programming interface,API)和编程框架。应用程序框架层包括一些预先定义的函数。The application framework layer provides application programming interfaces (application programming interface, API) and programming frameworks for applications in the application layer. The application framework layer includes some predefined functions.
如图7所示,应用程序框架层中可以包括视图***(view system),通知管理器,活动管理器,窗口管理器,内容提供器,资源管理器,输入法管理器等。As shown in Figure 7, the application framework layer can include a view system, a notification manager, an activity manager, a window manager, a content provider, a resource manager, an input method manager, and so on.
其中,视图***可用于构建应用程序的显示界面。每个显示界面可以由一个或多个控件组成。一般而言,控件可以包括图标、按钮、菜单、选项卡、文本框、对话框、状态栏、导航栏、微件(Widget)等界面元素。Among them, the view system can be used to construct the display interface of the application. Each display interface can consist of one or more controls. Generally speaking, controls can include interface elements such as icons, buttons, menus, tabs, text boxes, dialog boxes, status bars, navigation bars, and widgets.
视图***在绘制显示界面时可获取对应显示界面的视图信息,该视图信息中记录了需要绘制的显示界面中各个控件之间的图层关系。示例性的,显示界面中的各个控件一般按照树状结构分层组织,形成一个完整的ViewTree(视图树),该视图树可称为上述显示界面的视图信息。视图***可根据ViewTree中设置好的各个控件之间的图层关系绘制显示界面。在绘制显示界面中的每一个控件时都对应一组绘制指令,例如DrawLine、DrawPoint、DrawBitmap等。视图***可按照ViewTree中各个控件之间的图层关系,依次调用相应控件的绘制指令绘制该控件。The view system can obtain view information of the corresponding display interface when drawing the display interface, and the view information records the layer relationship between the various controls in the display interface to be drawn. Exemplarily, each control in the display interface is generally organized hierarchically according to a tree structure to form a complete ViewTree (view tree), which may be referred to as the view information of the above display interface. The view system can draw the display interface according to the layer relationship between the controls set in the ViewTree. Each control in the display interface corresponds to a set of drawing instructions, such as DrawLine, DrawPoint, DrawBitmap, etc. The view system can in turn call the drawing instructions of the corresponding control to draw the control according to the layer relationship between the controls in the ViewTree.
例如,图8中的(a)为微信APP的聊天界面801,聊天界面801最底层的控件为根节点(root),根节点下设置有底图802这一控件,底图802中还包括以下控件:标题栏803、聊天背景804以及输入栏805。其中,标题栏803中进一步包括返回按钮806和标题807,聊天 背景804中进一步包括头像808和气泡809,输入栏805中进一步包括语音输入按钮图标810、输入框811以及发送按钮812。这些控件按照顺序分层可形成如图8中(b)所示的视图树A。视图树A中记录了聊天界面801内各个控件之间的图层顺序。其中,底图802位于根节点下,标题栏803、聊天背景804以及输入栏805均为底图802的子节点。返回按钮806和标题807均为标题栏803的子节点。头像808和气泡809均为聊天背景804的子节点。语音输入按钮图标810、输入框811以及发送按钮812均为输入栏805的子节点。For example, (a) in Figure 8 is the chat interface 801 of the WeChat APP, the bottom control of the chat interface 801 is the root node (root), and there is a basemap 802 control under the root node. The basemap 802 also includes the following Controls: title bar 803, chat background 804, and input bar 805. Wherein, the title bar 803 further includes a return button 806 and a title 807, the chat background 804 further includes an avatar 808 and a bubble 809, and the input bar 805 further includes a voice input button icon 810, an input box 811, and a send button 812. These controls are layered in order to form a view tree A as shown in Figure 8(b). The view tree A records the layer order between the various controls in the chat interface 801. Among them, the base map 802 is located under the root node, and the title bar 803, the chat background 804, and the input bar 805 are all child nodes of the base map 802. Both the return button 806 and the title 807 are child nodes of the title bar 803. Both the avatar 808 and the bubble 809 are child nodes of the chat background 804. The voice input button icon 810, the input box 811, and the sending button 812 are all child nodes of the input field 805.
在本申请实施例中,可以在视图***增加新的识别模块,用于识别应用界面中的操作控件。其中,操作控件是指可与用户交互的控件,即该控件可响应用户的操作呈现相应的功能。例如,上述聊天界面801中的操作控件包括返回按钮806、头像808、气泡809、语音输入按钮图标810、输入框811以及发送按钮812。当然,上述识别模块也可以独立于视图***单独设置在应用程序框架层,本申请实施例对此不做任何限制。In the embodiment of the present application, a new recognition module may be added to the view system for recognizing operation controls in the application interface. The operation control refers to a control that can interact with a user, that is, the control can present corresponding functions in response to a user's operation. For example, the operation controls in the aforementioned chat interface 801 include a return button 806, an avatar 808, a bubble 809, a voice input button icon 810, an input box 811, and a send button 812. Of course, the aforementioned recognition module can also be set at the application framework layer independently of the view system, and the embodiment of the present application does not impose any limitation on this.
另外,在应用程序框架层以下的底层***中还可以设置状态监测服务,用于识别柔性屏幕的位置和形态变化。例如,如图7所示,可将状态监测服务设置在***库和/或内核层内。In addition, a status monitoring service can be set in the underlying system below the application framework layer to identify the position and shape changes of the flexible screen. For example, as shown in Figure 7, the condition monitoring service can be set in the system library and/or kernel layer.
示例性的,状态监测服务可调用传感器服务(sensor service)启动陀螺仪、加速度传感器等传感器进行检测。状态监测服务可根据各个传感器上报的检测数据计算当前主屏41和副屏42之间的夹角。Exemplarily, the status monitoring service may call a sensor service to start sensors such as a gyroscope and an acceleration sensor for detection. The condition monitoring service can calculate the angle between the current main screen 41 and the secondary screen 42 based on the detection data reported by each sensor.
那么,如果状态监测服务检测到上述柔性屏幕的主屏41和副屏42之间的夹角在第一预设值(例如90°)和第二预设值(例如180°)构成的闭区间内时,状态监测服务可向视图***上报分屏事件。或者,视图***可定期从状态监测服务中获取当前柔性屏幕的物理状态。当获取到柔性屏幕处于支架状态或展开状态时,视图***中的识别模块可识别当前应用界面中的操作控件,并将识别出的一个或多个操作控件显示在柔性屏幕的一个屏幕(例如主屏41)中,同时,将该应用界面中不可与用户交互的控件(后续称为非操作控件)显示在柔性屏幕的另一个屏幕(例如副屏42)中。Then, if the condition monitoring service detects that the angle between the main screen 41 and the secondary screen 42 of the flexible screen is within the closed interval formed by the first preset value (for example, 90°) and the second preset value (for example, 180°) At the time, the status monitoring service can report split-screen events to the view system. Alternatively, the view system may periodically obtain the current physical state of the flexible screen from the state monitoring service. When it is obtained that the flexible screen is in the stand or unfolded state, the recognition module in the view system can identify the operation controls in the current application interface, and display the identified one or more operation controls on a screen of the flexible screen (such as the main screen) In 41), at the same time, the controls in the application interface that cannot be interacted with the user (hereinafter referred to as non-operation controls) are displayed on another screen of the flexible screen (for example, the secondary screen 42).
也就是说,当柔性屏幕的物理状态转变为展开状态或支架状态时,电子设备可将正在显示的应用界面中的操作控件与非操作控件分屏显示在柔性屏幕的第一屏幕和第二屏幕中。其中,当第一屏幕为主屏41时第二屏幕为副屏42,当第一屏幕为副屏42时第二屏幕为主屏41。例如,在显示视频播放界面时,电子设备可将视频播放界面中进度条、快进、暂停等操作控件显示在副屏中,并将视频播放界面中播放的视频画面(即非操作控件)显示在主屏中。又例如,在显示游戏界面时,电子设备可将游戏界面中的操作手柄等操作控件显示在副屏中,并将游戏界面中的游戏画面(即非操作控件)显示在主屏中。这样,分屏显示后的操作控件不会遮挡上述视频画面或游戏画面,使用户获得沉浸式的视觉和操作体验。In other words, when the physical state of the flexible screen changes to the unfolded state or the stand state, the electronic device can display the operating controls and non-operating controls in the application interface being displayed on the first screen and the second screen of the flexible screen. in. Among them, when the first screen is the main screen 41, the second screen is the secondary screen 42, and when the first screen is the secondary screen 42, the second screen is the main screen 41. For example, when displaying a video playback interface, the electronic device can display the progress bar, fast forward, pause and other operation controls in the video playback interface on the secondary screen, and display the video screen (ie, non-operation control) played in the video playback interface In the main screen. For another example, when displaying a game interface, the electronic device may display operation controls such as an operation handle in the game interface on the secondary screen, and display the game screen (ie, non-operation controls) in the game interface on the main screen. In this way, the operation controls after the split-screen display will not block the above-mentioned video images or game images, so that the user can obtain an immersive visual and operating experience.
另外,上述活动管理器可用于管理每个应用的生命周期。应用通常以activity的形式运行在操作***中。活动管理器可以调度应用的activity进程管理每个应用的生命周期。窗口管理器用于管理窗口程序。窗口管理器可以获取显示屏大小,判断是否有状态栏,锁定屏幕,截取屏幕等。内容提供器用来存放和获取数据,并使这些数据可以被应用程序访问。所述数据可以包括视频,图像,音频,拨打和接听的电话,浏览历史和书签,电话簿等。资源管理器为应用程序提供各种资源,比如本地化字符串,图标,图片,布局文件,视频文件等。In addition, the aforementioned activity manager can be used to manage the life cycle of each application. Applications usually run in the operating system in the form of activity. The activity manager can schedule the activity process of the application to manage the life cycle of each application. The window manager is used to manage window programs. The window manager can obtain the size of the display, determine whether there is a status bar, lock the screen, take a screenshot, etc. The content provider is used to store and retrieve data and make these data accessible to applications. The data may include video, image, audio, phone calls made and received, browsing history and bookmarks, phone book, etc. The resource manager provides various resources for the application, such as localized strings, icons, pictures, layout files, video files, etc.
3、Android runtime和***库3. Android runtime and system libraries
Android Runtime包括核心库和虚拟机。Android runtime负责安卓***的调度和管理。Android Runtime includes core libraries and virtual machines. Android runtime is responsible for the scheduling and management of the Android system.
核心库包含两部分:一部分是java语言需要调用的功能函数,另一部分是安卓的核心库。The core library consists of two parts: one part is the function functions that the java language needs to call, and the other part is the core library of Android.
应用程序层和应用程序框架层运行在虚拟机中。虚拟机将应用程序层和应用程序框架层 的java文件执行为二进制文件。虚拟机用于执行对象生命周期的管理,堆栈管理,线程管理,安全和异常的管理,以及垃圾回收等功能。The application layer and the application framework layer run in a virtual machine. The virtual machine executes the java files in the application layer and application framework layer as binary files. The virtual machine is used to perform functions such as object life cycle management, stack management, thread management, security and exception management, and garbage collection.
***库可以包括多个功能模块。例如:表面管理器(surface manager),媒体库(Media Libraries),三维图形处理库(例如:OpenGL ES),2D图形引擎(例如:SGL)等。The system library can include multiple functional modules. For example: surface manager (surface manager), media library (Media Libraries), three-dimensional graphics processing library (for example: OpenGL ES), 2D graphics engine (for example: SGL), etc.
其中,表面管理器用于对显示子***进行管理,并且为多个应用程序提供了2D和3D图层的融合。媒体库支持多种常用的音频,视频格式回放和录制,以及静态图像文件等。媒体库可以支持多种音视频编码格式,例如:MPEG4,H.264,MP3,AAC,AMR,JPG,PNG等。三维图形处理库用于实现三维图形绘图,图像渲染,合成,和图层处理等。2D图形引擎是2D绘图的绘图引擎。Among them, the surface manager is used to manage the display subsystem, and provides a combination of 2D and 3D layers for multiple applications. The media library supports playback and recording of a variety of commonly used audio and video formats, as well as still image files. The media library can support multiple audio and video encoding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc. The 3D graphics processing library is used to realize 3D graphics drawing, image rendering, synthesis, and layer processing. The 2D graphics engine is a drawing engine for 2D drawing.
4、内核层4. The kernel layer
内核层是硬件和软件之间的层。内核层至少包含显示驱动,摄像头驱动,音频驱动,传感器驱动等,本申请实施例对此不做任何限制。The kernel layer is the layer between hardware and software. The kernel layer includes at least a display driver, a camera driver, an audio driver, a sensor driver, etc., which are not limited in the embodiment of the present application.
以下将以手机作为电子设备举例,结合附图详细阐述本申请实施例提供的一种柔性屏幕的显示方法。In the following, a mobile phone is used as an example of an electronic device, and a method for displaying a flexible screen provided by an embodiment of the present application will be described in detail with reference to the accompanying drawings.
示例性的,在本申请实施例中,除了上述支架状态和展开状态外,手机中柔性屏幕的物理状态还可包含折叠状态。以外折式柔性屏幕举例,如图9中的(a)所示,当主屏41和副屏42之间的夹角为360°(或接近360°)时,柔性屏幕处于折叠状态。以内折式柔性屏幕举例,如图9中的(b)所示,当主屏41和副屏42之间的夹角为0(或接近0)时,柔性屏幕处于折叠状态。Exemplarily, in the embodiment of the present application, in addition to the above-mentioned bracket state and unfolded state, the physical state of the flexible screen in the mobile phone may also include a folded state. As an example of an outward-folding flexible screen, as shown in (a) of FIG. 9, when the angle between the main screen 41 and the auxiliary screen 42 is 360° (or close to 360°), the flexible screen is in a folded state. Taking an inward-folding flexible screen as an example, as shown in FIG. 9(b), when the angle between the main screen 41 and the sub screen 42 is 0 (or close to 0), the flexible screen is in a folded state.
以外折式柔性屏幕举例,如图10所示,手机可在主屏41中显示桌面1001,此时副屏42(图10中未示出)可以为黑屏状态。例如,当手机中的状态监测服务根据传感器上报的数据确定主屏41和副屏42之间的夹角β为360°时,状态监测服务可继续调用摄像头、红外传感器、接近光传感器或触控器件等识别手机的具体朝向。As an example of an outward-folding flexible screen, as shown in FIG. 10, the mobile phone may display a desktop 1001 on the main screen 41, and at this time, the secondary screen 42 (not shown in FIG. 10) may be in a black screen state. For example, when the condition monitoring service in the mobile phone determines that the angle β between the main screen 41 and the secondary screen 42 is 360° according to the data reported by the sensor, the condition monitoring service can continue to call the camera, infrared sensor, proximity light sensor or touch device Wait to identify the specific orientation of the phone.
例如,可以在主屏41和副屏42上分别安装摄像头,如果主屏41的摄像头捕捉到人脸信息,而副屏42的摄像头没有捕捉到人脸信息,则状态监测服务可确定当前折叠状态的柔性屏幕处于主屏41朝向用户的状态。For example, cameras can be installed on the main screen 41 and the secondary screen 42. If the camera of the main screen 41 captures facial information, but the camera of the secondary screen 42 does not capture the facial information, the status monitoring service can determine the flexibility of the current folding state The screen is in a state where the main screen 41 faces the user.
又例如,可以在主屏41和副屏42上分别安装红外传感器,如果副屏42的红外传感器捕捉到人体辐射的红外信号,而主屏41的红外传感器没有捕捉到人体辐射的红外信号,则状态监测服务可确定当前折叠状态的柔性屏幕处于副屏42朝向用户的状态。For another example, infrared sensors can be installed on the main screen 41 and the auxiliary screen 42 respectively. If the infrared sensor of the auxiliary screen 42 captures the infrared signal radiated by the human body, but the infrared sensor of the main screen 41 does not capture the infrared signal radiated by the human body, the status monitoring The service may determine that the flexible screen in the current folded state is in a state where the secondary screen 42 faces the user.
又例如,手机还可以通过触控器件上报的用户当前的触摸位置。进而,根据该触摸位置使用预设的抓握算法可确定用户当前抓握手机的抓握姿势。那么,结合手机的抓握姿势,状态监测服务也可确定出手机的具体朝向。示例性的,触控器件检测到触摸事件后可将触摸点的坐标上报给状态监测服务。状态监测服务通过统计触摸点在触控器件中的位置和个数确定手机的抓握姿势。例如,如果检测到落在主屏41中的触摸点的数目大于阈值,说明用户的手指和手掌抓握在主屏41上,则此时朝向用户的屏幕为副屏42。相应的,如果检测到落在副屏42中的触摸点的数目大于阈值,说明用户的手指和手掌抓握在副屏42上,则此时朝向用户的屏幕为主屏41。For another example, the mobile phone can also report the user's current touch position through the touch device. Furthermore, a preset grasping algorithm can be used according to the touch position to determine the grasping posture of the user currently grasping the handset. Then, combined with the gripping posture of the mobile phone, the state monitoring service can also determine the specific orientation of the mobile phone. Exemplarily, the touch control device may report the coordinates of the touch point to the state monitoring service after detecting the touch event. The state monitoring service determines the gripping posture of the mobile phone by counting the position and number of touch points in the touch device. For example, if it is detected that the number of touch points falling on the main screen 41 is greater than the threshold, indicating that the user's fingers and palm are grasped on the main screen 41, the screen facing the user at this time is the secondary screen 42. Correspondingly, if it is detected that the number of touch points falling on the secondary screen 42 is greater than the threshold, it means that the user's fingers and palm are grasped on the secondary screen 42, and the screen facing the user is now the primary screen 41.
那么,在折叠状态下,当检测到柔性屏幕的主屏41朝向用户时,手机可在主屏41中显示图10所示的桌面1001。同样,当检测到柔性屏幕的副屏42朝向用户时,手机可在副屏42中显示上述桌面1001。当然,手机也可以在主屏41(或副屏42)中显示其他应用的应用界面,本申请实施例对此不做任何限制。Then, in the folded state, when it is detected that the main screen 41 of the flexible screen faces the user, the mobile phone can display the desktop 1001 shown in FIG. 10 on the main screen 41. Similarly, when it is detected that the secondary screen 42 of the flexible screen faces the user, the mobile phone can display the above-mentioned desktop 1001 on the secondary screen 42. Of course, the mobile phone can also display the application interface of other applications on the main screen 41 (or the secondary screen 42), which is not limited in the embodiment of the present application.
仍以手机在主屏41中显示上述桌面1001举例,桌面1001中包含游戏应用的图标1002。如果检测到用户点击图标1002的操作,如图11所示,手机可打开游戏应用并显示游戏应用的游戏界面1101。示例性的,如果该游戏应用支持通过横屏状态显示而不支持通过竖屏状态显示,则手机可自动将游戏界面1101显示为横屏状态。又或者,如果游戏应用既支持横屏状态显示又支持竖屏状态显示,则手机可在检测到用户将手机从竖屏状态旋转为横屏状态后,将游戏界面1101显示为横屏状态。Still taking the example of the mobile phone displaying the above-mentioned desktop 1001 on the main screen 41, the desktop 1001 includes an icon 1002 of a game application. If the user's click on the icon 1002 is detected, as shown in FIG. 11, the mobile phone can open the game application and display the game interface 1101 of the game application. Exemplarily, if the game application supports display in the horizontal screen state but not in the vertical screen state, the mobile phone may automatically display the game interface 1101 in the horizontal screen state. Or, if the game application supports both the horizontal screen state display and the vertical screen state display, the mobile phone may display the game interface 1101 in the horizontal screen state after detecting that the user has rotated the mobile phone from the vertical screen state to the horizontal screen state.
在一些实施例中,手机中的状态监测服务可定期检测柔性屏幕中主屏41和副屏42之间的夹角,以及主屏41和副屏42是否处于横屏状态。当检测到主屏41和副屏42之间的夹角为180°时,说明柔性屏幕从上述折叠状态切换为展开状态。当检测到主屏41和副屏42之间的夹角在[90,180°)的范围内,且主屏41和副屏42均为横屏状态时,说明柔性屏幕从上述折叠状态切换为支架状态。In some embodiments, the status monitoring service in the mobile phone can periodically detect the angle between the main screen 41 and the sub screen 42 in the flexible screen, and whether the main screen 41 and the sub screen 42 are in a landscape state. When it is detected that the included angle between the main screen 41 and the auxiliary screen 42 is 180°, it indicates that the flexible screen is switched from the above-mentioned folded state to the unfolded state. When it is detected that the included angle between the main screen 41 and the sub screen 42 is within the range of [90, 180°), and the main screen 41 and the sub screen 42 are both in landscape mode, it indicates that the flexible screen is switched from the above-mentioned folded state to the stand state .
当柔性屏幕从折叠状态切换为展开状态或支架状态后,手机可将游戏界面1101中的操作控件和非操作控件分离,并将游戏界面1101中的操作控件显示在柔性屏幕的一个屏幕(例如副屏42)内,将游戏界面1101中的非操作控件显示在柔性屏幕的另一个屏幕(例如主屏41)内。When the flexible screen is switched from the folded state to the unfolded state or the stand state, the mobile phone can separate the operating controls and non-operating controls in the game interface 1101, and display the operating controls in the game interface 1101 on a screen of the flexible screen (such as a secondary In the screen 42), the non-operation controls in the game interface 1101 are displayed in another screen (for example, the main screen 41) of the flexible screen.
示例性的,可以在手机内预先设置一个应用白名单,该应用白名单中包括一个或多个应用的标识,例如应用的包名(packagename)。在应用白名单中的应用均支持将应用界面中操作控件和非操作控件分屏显示这一功能。那么,当手机检测到柔性屏幕切换为展开状态(或支架状态)时,手机可获取当前正在运行的应用(例如上述游戏应用)的包名,进而在上述应用白名单中查询该游戏应用是否为预先设置的应用。Exemplarily, an application whitelist may be preset in the mobile phone, and the application whitelist includes one or more application identities, such as application package names. The applications in the application whitelist all support the split-screen display of operating controls and non-operating controls in the application interface. Then, when the mobile phone detects that the flexible screen is switched to the expanded state (or stand state), the mobile phone can obtain the package name of the currently running application (such as the aforementioned game application), and then query whether the game application is in the aforementioned application whitelist Pre-set application.
又或者,可以在手机内预先设置一个界面白名单,该界面白名单中包括一个或多个应用界面的标识,例如应用界面的activityname。在界面白名单中的应用界面均支持将其操作控件和非操作控件分屏显示这一功能。那么,当手机检测到柔性屏幕切换为展开状态(或支架状态)时,手机可获取当前正在运行的应用界面的activityname,进而在上述界面白名单中查询当前显示的应用界面为预先设置的应用界面。Alternatively, an interface whitelist can be preset in the mobile phone, and the interface whitelist includes one or more application interface identifiers, such as the activityname of the application interface. The application interfaces in the interface whitelist all support the function of split-screen display of their operating controls and non-operating controls. Then, when the mobile phone detects that the flexible screen is switched to the expanded state (or stand state), the mobile phone can obtain the activityname of the currently running application interface, and then query the whitelist of the interface that the currently displayed application interface is the preset application interface .
仍以上述游戏界面1101举例,如图12中的(a)所示,当柔性屏幕为折叠状态时,手机可以横屏状态显示游戏界面1101。如图12中的(b)所示,当检测到柔性屏幕切换为展开状态时,如果游戏界面1101为预先设置的应用界面,则手机可识别游戏界面1101中的操作控件和非操作控件,进而将识别出的非操作控件显示在主屏41中,将识别出的操作控件显示在副屏42中(即将非操作控件和操作控件分屏显示)。当然,如果游戏界面1101不是预先设置的应用或应用界面,则手机可按照现有的显示方式继续显示游戏界面1101,本申请实施例对此不做任何限制。Still taking the above-mentioned game interface 1101 as an example, as shown in (a) of FIG. 12, when the flexible screen is in a folded state, the mobile phone can display the game interface 1101 in a horizontal screen state. As shown in Figure 12(b), when the flexible screen is detected to be switched to the expanded state, if the game interface 1101 is a preset application interface, the mobile phone can identify the operation controls and non-operation controls in the game interface 1101, and then The identified non-operating controls are displayed on the main screen 41, and the identified operating controls are displayed on the secondary screen 42 (that is, the non-operating controls and the operating controls are displayed on separate screens). Of course, if the game interface 1101 is not a preset application or application interface, the mobile phone can continue to display the game interface 1101 according to the existing display mode, and the embodiment of the present application does not impose any limitation on this.
以下将举例说明手机识别上述游戏界面1101中操作控件和非操作控件的实现方式。The following will give an example of how the mobile phone recognizes the operation control and the non-operation control in the aforementioned game interface 1101.
在一些实施例中,如图13所示,手机可以获取在折叠状态下viewsystem绘制上述游戏界面1101时对应的视图树1300。视图树1300中记录了游戏界面1101中各个控件的之间的图层顺序。例如,如图11所示,游戏界面1101的根节点下包括游戏画面1301这一控件,游戏画面1301下包括以下控件:操作手柄1302、攻击按钮1303以及设置栏1304。其中,设置栏1304中又包括第一设置按钮1305、第二设置按钮1306。In some embodiments, as shown in FIG. 13, the mobile phone can obtain the view tree 1300 corresponding to the view system drawing the above-mentioned game interface 1101 in the folded state. The view tree 1300 records the layer order among the various controls in the game interface 1101. For example, as shown in FIG. 11, the root node of the game interface 1101 includes a control of a game screen 1301, and the game screen 1301 includes the following controls: an operation handle 1302, an attack button 1303, and a setting bar 1304. Wherein, the setting column 1304 further includes a first setting button 1305 and a second setting button 1306.
在视图树1300中,如图13所示,游戏画面1301为根节点的子节点,操作手柄1302、攻击按钮1303以及设置栏1304均为游戏画面1301的子节点,位于游戏画面1301所在的图层之上。第一设置按钮1305和第二设置按钮1306均为设置栏1304的子节点,位于设置栏 1304所在的图层之上。并且,手机可记录视图树1300中每个控件的位置和大小,以及绘制每个控件时对应的绘图指令。在绘制游戏界面1101时,手机可按照视图树1300先绘制游戏画面1301,进而在游戏画面1301上绘制操作手柄1302、攻击按钮1303以及设置栏1304,再在设置栏1304上绘制第一设置按钮1305和第二设置按钮1306。In the view tree 1300, as shown in Figure 13, the game screen 1301 is a child node of the root node, and the operation handle 1302, the attack button 1303, and the setting bar 1304 are all child nodes of the game screen 1301, located on the layer where the game screen 1301 is located. Above. The first setting button 1305 and the second setting button 1306 are both child nodes of the setting column 1304 and are located on the layer where the setting column 1304 is located. In addition, the mobile phone can record the position and size of each control in the view tree 1300, and the corresponding drawing instruction when drawing each control. When drawing the game interface 1101, the phone can first draw the game screen 1301 according to the view tree 1300, and then draw the operation handle 1302, the attack button 1303, and the setting bar 1304 on the game screen 1301, and then draw the first setting button 1305 on the setting bar 1304 And the second setting button 1306.
那么,手机检测到柔性屏幕从图12中的(a)所示的折叠状态切换为图12中的(b)所示的展开状态时,手机可获取上述视图树1300,并查询视图树1300中每个控件的位置和大小。一般,操作控件通常位于非操作控件上层的图层中,非操作控件通常作为底图显示在操作控件下层的图层中,且非操作控件的尺寸一般比较大。当手机查询到游戏画面1301这一控件的大小与整个主屏41的大小相同或接近时,手机可将游戏画面1301确认为游戏界面1101中的非操作控件。并且,手机可将视图树1300中游戏画面1301的各个子节点识别为操作控件。Then, when the mobile phone detects that the flexible screen is switched from the folded state shown in Figure 12 (a) to the expanded state shown in Figure 12 (b), the mobile phone can obtain the above-mentioned view tree 1300 and query the view tree 1300 The position and size of each control. Generally, operation controls are usually located in the upper layer of the non-operation control, and the non-operation control is usually displayed as a base map in the lower layer of the operation control, and the size of the non-operation control is generally relatively large. When the mobile phone finds that the size of the control of the game screen 1301 is the same as or close to the size of the entire main screen 41, the mobile phone can confirm the game screen 1301 as a non-operation control in the game interface 1101. In addition, the mobile phone can recognize each child node of the game screen 1301 in the view tree 1300 as an operation control.
又或者,当手机查询到游戏画面1301这一控件的大小与整个主屏41的大小相同或接近时,还可以进一步确定游戏画面1301所在的图层是否为最顶层的图层。如果游戏画面1301所在的图层不是最顶层的图层,则手机可将游戏画面1301确认为游戏界面1101中的非操作控件。并且,手机可将位于游戏画面1301所在的图层上的各个控件确定为操作控件。Or, when the mobile phone finds that the size of the control of the game screen 1301 is the same as or close to the size of the entire main screen 41, it can further determine whether the layer where the game screen 1301 is located is the top layer. If the layer where the game screen 1301 is located is not the top layer, the mobile phone can confirm the game screen 1301 as a non-operation control in the game interface 1101. In addition, the mobile phone may determine each control located on the layer where the game screen 1301 is located as an operation control.
也就是说,手机可以根据绘制当前应用界面时各个控件的尺寸和位置关系,将显示尺寸较大且靠近应用画面底层图层的控件识别为非操作控件,将位于非操作控件之上的控件识别为操作控件。In other words, the mobile phone can recognize the control with a larger display size and close to the bottom layer of the application screen as a non-operational control according to the size and position relationship of each control when drawing the current application interface, and recognize the control located above the non-operational control It is an operation control.
在另一些实施例中,手机获取到柔性屏幕在折叠状态下绘制上述游戏界面1101时对应的视图树1300后,手机还可以获取视图树1300中每个控件的具体类型。例如,控件可包括图像控件(Image View)、文本控件(Text View)、按钮(Button)、导航栏(ActionBar)、文本框、对话框、状态栏等类型。In other embodiments, after the mobile phone obtains the view tree 1300 corresponding to the game interface 1101 when the flexible screen is drawn in the folded state, the mobile phone can also obtain the specific type of each control in the view tree 1300. For example, the control may include image control (Image View), text control (Text View), button (Button), navigation bar (ActionBar), text box, dialog box, status bar and other types.
例如,手机中的视图***可以将视图树1300中各个控件的标识或位置发送给游戏应用,请求游戏应用根据控件的标识或位置查询视图树1300中各个控件的具体类型。又例如,手机可在绘制视图树1300中的每个控件时在对应的字段中记录控件的类型。那么,手机可从对应的字段中查找视图树1300中各个控件的具体类型。For example, the view system in the mobile phone may send the identification or location of each control in the view tree 1300 to the game application, requesting the game application to query the specific type of each control in the view tree 1300 according to the identification or location of the control. For another example, the mobile phone may record the type of the control in the corresponding field when drawing each control in the view tree 1300. Then, the mobile phone can search for the specific type of each control in the view tree 1300 from the corresponding field.
示例性的,当某一控件为图像控件(Image View)或文本控件(Text View)等类型的控件时,手机可确认该控件为非操作控件。例如,上述游戏界面1101中的游戏画面1301为Image View类型的非操作控件。当某一控件为按钮(Button)、导航栏(ActionBar)、文本框、对话框、状态栏等类型的控件时,手机可确认该控件为操作控件。例如,上述游戏界面1101中的操作手柄1302、攻击按钮1303以及设置栏1304中的各个控件均为操作控件。Exemplarily, when a certain control is an image control (Image View) or a text control (Text View), the mobile phone can confirm that the control is a non-operation control. For example, the game screen 1301 in the aforementioned game interface 1101 is an Image View type non-operation control. When a certain control is a button (Button), navigation bar (ActionBar), text box, dialog box, status bar and other types of controls, the mobile phone can confirm that the control is an operation control. For example, the operation handle 1302, the attack button 1303, and the controls in the setting bar 1304 in the aforementioned game interface 1101 are all operation controls.
在另一些实施例中,还可以预先在手机内保存与上述游戏界面1101对应的配置文件。该配置文件中记录有上述游戏界面1101中的操作控件。例如,该配置文件中可以记录分屏前(即柔性屏幕处于折叠状态时)游戏界面1101中操作控件的具体显示位置。又例如,该配置文件中还可以记录分屏后(即柔性屏幕处于展开或支架状态时)上述操作控件在主屏或副屏中的具***置。又例如,该配置文件中还可以记录分屏后操作控件的平移距离、缩放比例、旋转角度等信息。In other embodiments, the configuration file corresponding to the aforementioned game interface 1101 can also be saved in the mobile phone in advance. The configuration file records the operation controls in the aforementioned game interface 1101. For example, the configuration file may record the specific display position of the operation control in the game interface 1101 before the split screen (that is, when the flexible screen is in the folded state). For another example, the configuration file may also record the specific position of the above-mentioned operation control on the main screen or the secondary screen after the screen is split (that is, when the flexible screen is in the expanded or stent state). For another example, the configuration file may also record information such as the translation distance, zoom ratio, and rotation angle of the operation control after split screen.
示例性的,与上述游戏界面1101对应的配置文件1可以为:Exemplarily, the configuration file 1 corresponding to the aforementioned game interface 1101 may be:
一般,每个配置文件中均记录了对应的packagename、activityname以及分屏前操作控件的位置。那么,当手机检测到柔性屏幕从图12中的(a)所示的折叠状态切换为图12中的(b)所示的展开状态时,手机可以根据当前运行的游戏应用的packagename和activityname查找到对应的配置文件(例如上述配置文件1)。进而,手机根据配置文件1可识别出游戏界面1101内的操作控件1。例如,手机可根据配置文件1中记录的分屏前操作控件1的具***置识别游戏界面1101中的操作控件1具体为哪个控件。这样,通过遍历配置文件1中分屏前所有操作控件的位置,手机可识别出游戏界面1101中的各个操作控件。另外,手机可将游戏界面1101中除操作控件外的控件识别为非操作控件。Generally, each configuration file records the corresponding packagename, activityname, and the position of the operation control before split screen. Then, when the mobile phone detects that the flexible screen is switched from the folded state shown in Figure 12 (a) to the expanded state shown in Figure 12 (b), the mobile phone can search according to the packagename and activityname of the currently running game application To the corresponding configuration file (for example, configuration file 1 above). Furthermore, the mobile phone can recognize the operation control 1 in the game interface 1101 according to the configuration file 1. For example, the mobile phone can identify which control is specifically the control 1 in the game interface 1101 according to the specific position of the control 1 before the split screen recorded in the configuration file 1. In this way, by traversing the positions of all operation controls before splitting the screen in the configuration file 1, the mobile phone can identify each operation control in the game interface 1101. In addition, the mobile phone can recognize controls other than the operation controls in the game interface 1101 as non-operation controls.
示例性的,手机通过上述方法可以识别出游戏界面1101中的操作控件包括:操作手柄1302、攻击按钮1303以及设置栏1304中的各个控件,游戏界面1101中的非操作控件包括游戏画面1301。进而,仍如图12中的(b)所示,手机可将非操作控件显示主屏41中,将操作控件显示在副屏42中。或者,手机也可将非操作控件显示副屏42中,将操作控件显示在主屏41中。Exemplarily, the mobile phone can recognize through the above method that the operation controls in the game interface 1101 include: an operation handle 1302, an attack button 1303, and various controls in the setting bar 1304. The non-operation controls in the game interface 1101 include the game screen 1301. Furthermore, as shown in (b) in FIG. 12, the mobile phone can display non-operation controls on the main screen 41 and operation controls on the secondary screen 42. Alternatively, the mobile phone may also display the non-operation controls on the secondary screen 42 and the operation controls on the main screen 41.
上述实施例是以柔性屏幕从折叠状态切换为展开状态举例说明的,如图14所示,如果检测到柔性屏幕从折叠状态切换为支架状态,即主屏41和副屏42之间的夹角处于[90,180)的范围内,且主屏41和副屏42均为横屏状态,手机还可以调用摄像头、红外传感器、接近光传感器或触控器件等识别手机的具体朝向。The above embodiment is an example of the flexible screen switching from the folded state to the unfolded state. As shown in FIG. 14, if it is detected that the flexible screen is switched from the folded state to the stand state, that is, the angle between the main screen 41 and the auxiliary screen 42 is Within the range of [90, 180), and the main screen 41 and the secondary screen 42 are both in the horizontal screen state, the mobile phone can also call the camera, infrared sensor, proximity light sensor or touch device to identify the specific orientation of the mobile phone.
如果此时主屏41朝向用户,则仍如图14所示,手机可将识别出的非操作控件显示在主屏41中,将识别出的操作控件显示在副屏42中。如果此时副屏42朝向用户,则手机可将识别出的非操作控件显示在副屏42中,将识别出的操作控件显示在主屏41中。这样一来,朝向用户的屏幕可为用户显示不需要进行操作和交互的控件(例如上述游戏画面1301),用户 可使用另一屏幕中的操作控件更新或编辑游戏画面1301中的内容,操作控件不会遮挡游戏画面1301,使用户获得沉浸式的观看体验。If the main screen 41 faces the user at this time, as shown in FIG. 14, the mobile phone can display the identified non-operation controls on the main screen 41 and the identified operation controls on the secondary screen 42. If the secondary screen 42 faces the user at this time, the mobile phone can display the identified non-operation controls on the secondary screen 42 and the identified operation controls on the main screen 41. In this way, the screen facing the user can display controls that do not require operation and interaction for the user (for example, the above-mentioned game screen 1301), and the user can use the operation controls on another screen to update or edit the content in the game screen 1301. Operation controls The game screen 1301 will not be blocked, so that the user can get an immersive viewing experience.
仍以图12中(a)-(b)示出的将上述游戏界面1101中的操作控件显示在副屏42举例,与游戏界面1101对应的配置文件1中不仅记录有分屏前操作控件1(例如操作手柄1302)在主屏41中的位置,还记录有分屏后操作手柄1302在副屏42中的位置。那么,根据配置文件1中各个操作控件在分屏前后的具***置,手机可确定出分屏后与主屏41对应的视图树以及分屏后与副屏42对应的视图树。Still taking the example shown in Figure 12 (a)-(b) to display the operation controls in the game interface 1101 on the secondary screen 42, the configuration file 1 corresponding to the game interface 1101 not only records the pre-split operation control 1 (For example, the position of the operating handle 1302) in the main screen 41, and the position of the operating handle 1302 in the secondary screen 42 after the split screen is also recorded. Then, according to the specific positions of the operation controls in the configuration file 1 before and after the split screen, the mobile phone can determine the view tree corresponding to the main screen 41 after the split screen and the view tree corresponding to the secondary screen 42 after the split screen.
示例性的,图15中的(a)示出了折叠状态下游戏界面1101的视图树1300。手机识别出视图树1300中的操作控件和非操作控件后,可将视图树1300中的操作控件和非操作控件拆分为两个视图树。Exemplarily, (a) in FIG. 15 shows a view tree 1300 of the game interface 1101 in a folded state. After the mobile phone recognizes the operation control and the non-operation control in the view tree 1300, the operation control and the non-operation control in the view tree 1300 can be split into two view trees.
示例性的,手机可沿用视图树1300中操作控件的图层关系生成对应的视图树1401,此时,操作控件在视图树1300和视图树1401中的图层关系相同。同样,手机可沿用视图树1300中非操作控件的图层关系生成对应的视图树1402,此时,非操作控件在视图树1300和视图树1402中的图层关系也相同。Exemplarily, the mobile phone can use the layer relationship of the operation controls in the view tree 1300 to generate the corresponding view tree 1401. At this time, the layer relationship of the operation controls in the view tree 1300 and the view tree 1401 is the same. Similarly, the mobile phone can use the layer relationship of the non-operational controls in the view tree 1300 to generate the corresponding view tree 1402. At this time, the layer relationship of the non-operational controls in the view tree 1300 and the view tree 1402 is also the same.
又或者,由于配置文件1中记录有分屏后各个操作控件在副屏42中的显示位置,那么,手机也可以将操作控件从视图树1401中拆分出来,并根据配置文件1中记录的各个操作控件在分屏后的显示位置生成对应的视图树1401。此时,操作控件在视图树1300和视图树1401中的图层关系可以相同或不同,本申请实施例对此不做任何限制。Or, because the configuration file 1 records the display position of each operation control in the secondary screen 42 after splitting the screen, then the mobile phone can also split the operation control from the view tree 1401, and according to the record in the configuration file 1. Each operation control generates a corresponding view tree 1401 at the display position after the split screen. At this time, the layer relationship of the operation control in the view tree 1300 and the view tree 1401 may be the same or different, and the embodiment of the present application does not impose any limitation on this.
如图15中的(b)所示,由于视图树1300中的操作手柄1302、攻击按钮1303以及设置栏1304中的各个控件均为操作控件,这些操作控件分屏后显示在副屏42中。那么,手机可生成与副屏42对应的视图树1401,视图树1401中包括设置在根节点下的操作手柄1302、攻击按钮1303以及设置栏1304。设置栏1304下还包括两个子节点,即第一设置按钮1305和第二设置按钮1306。视图树1401中各个控件在副屏42中的位置均记录在配置文件1中。As shown in (b) of FIG. 15, since the operation handle 1302, the attack button 1303, and the controls in the setting bar 1304 in the view tree 1300 are all operation controls, these operation controls are displayed on the secondary screen 42 after splitting the screen. Then, the mobile phone can generate a view tree 1401 corresponding to the secondary screen 42. The view tree 1401 includes an operating handle 1302, an attack button 1303, and a setting bar 1304 set under the root node. The setting column 1304 also includes two sub-nodes, namely the first setting button 1305 and the second setting button 1306. The position of each control in the view tree 1401 in the secondary screen 42 is recorded in the configuration file 1.
类似的,仍如图15中的(b)所示,由于视图树1300中的游戏画面1301为非操作控件,非操作控件分屏后显示在主屏41中。那么,手机可生成与主屏41对应的视图树1402,视图树1402中包括设置在根节点下的游戏画面1301。游戏画面1301分屏后在主屏41中的位置与绘制视图树1300时游戏画面1301所在的位置相同。Similarly, as shown in (b) of FIG. 15, since the game screen 1301 in the view tree 1300 is a non-operation control, the non-operation control is displayed in the main screen 41 after split screen. Then, the mobile phone can generate a view tree 1402 corresponding to the main screen 41, and the view tree 1402 includes a game screen 1301 set under the root node. The position of the game screen 1301 in the main screen 41 after the split screen is the same as the position of the game screen 1301 when the view tree 1300 is drawn.
进而,手机可根据视图树1402重新在主屏41中绘制游戏画面1301。示例性的,由于手机在绘制上述游戏界面1101时记录了与每个控件对应的绘图指令,因此,手机在主屏41中重新绘制游戏画面1301时,可仍然使用绘制上述游戏界面1101时游戏画面1301的绘图指令重新绘制游戏画面1301。Furthermore, the mobile phone can redraw the game screen 1301 on the main screen 41 according to the view tree 1402. Exemplarily, since the mobile phone records the drawing instructions corresponding to each control when drawing the above-mentioned game interface 1101, when the mobile phone redraws the game screen 1301 in the main screen 41, the game screen 1301 when drawing the above-mentioned game interface 1101 can still be used. Redraw the game screen 1301 with the drawing instructions.
同样,手机可根据视图树1403在副屏42中绘制视图树1403中的各个操作控件。以绘制操作手柄1302举例,手机可使用绘制上述游戏界面1101时绘制操作手柄1302的绘制指令,在配置文件1所指示的分屏后操作手柄1302的位置绘制该操作手柄1302。类似的,视图树1403中每层控件中的每个控件均可按照顺序被绘制在副屏42中的相应位置。Similarly, the mobile phone can draw various operation controls in the view tree 1403 on the secondary screen 42 according to the view tree 1403. Taking drawing the operating handle 1302 as an example, the mobile phone can use the drawing instruction of drawing the operating handle 1302 when drawing the aforementioned game interface 1101, and draw the operating handle 1302 at the position of the operating handle 1302 after the split screen indicated by the configuration file 1. Similarly, each control in each level of control in the view tree 1403 can be drawn in a corresponding position in the secondary screen 42 in order.
在一些实施例中,如果主屏41与副屏42的大小相同,则手机在副屏42中绘制视图树1403中的各个操作控件时,可按照每个操作控件在主屏41中的坐标,将该操作控件绘制在副屏42中。例如,上述游戏界面1101中第一设置按钮1305为40*30的矩形,该矩形在主屏41中左上角的坐标为(380,10),右下角的坐标为(420,40)。那么,手机在副屏42中绘制第一设置按钮1305时,同样可将第一设置按钮1305绘制为40*30的矩形,且将该矩形绘制在副屏42中左上角坐标为(380,10),右下角坐标为(420,40)的区域。In some embodiments, if the main screen 41 and the sub-screen 42 are the same size, when the mobile phone draws each operation control in the view tree 1403 on the sub-screen 42, it can be adjusted according to the coordinates of each operation control in the main screen 41. Operation controls are drawn in the secondary screen 42. For example, the first setting button 1305 in the game interface 1101 is a 40*30 rectangle, and the coordinates of the upper left corner of the rectangle on the main screen 41 are (380, 10), and the coordinates of the lower right corner are (420, 40). Then, when the mobile phone draws the first setting button 1305 on the secondary screen 42, the first setting button 1305 can also be drawn as a 40*30 rectangle, and the rectangle is drawn on the secondary screen 42 with the upper left corner coordinates (380, 10 ), the area with coordinates (420,40) in the lower right corner.
在另一些实施例中,手机将游戏界面1101中的操作控件绘制在副屏42时,还可以改变操作控件的大小或形状。例如,如果上述配置文件1中记录了操作手柄1302在x轴的缩放比例为2,在y轴的缩放比例也为2,则说明分屏后副屏42中操作手柄1302的大小是分屏前游戏界面1101中操作手柄1302的2倍。那么,如图16所示,手机可将游戏界面1101中的操作手柄1302放大2倍后得到新的操作手柄1503,并将操作手柄1503绘制在副屏42中。也就是说,分屏后手机在绘制操作控件时可以占用多或更少的显示空间,使得用户在分屏后操作这些操作控件时获得更好的使用体验。In other embodiments, when the mobile phone draws the operation controls in the game interface 1101 on the secondary screen 42, the size or shape of the operation controls can also be changed. For example, if the above configuration file 1 records that the zoom ratio of the operating handle 1302 on the x-axis is 2 and the zoom ratio on the y-axis is also 2, it means that the size of the operating handle 1302 in the secondary screen 42 after the split screen is before the split screen 2 times the operation handle 1302 in the game interface 1101. Then, as shown in FIG. 16, the mobile phone can magnify the operating handle 1302 in the game interface 1101 by 2 times to obtain a new operating handle 1503, and draw the operating handle 1503 on the secondary screen 42. In other words, the mobile phone can take up more or less display space when drawing operation controls after split screen, so that users can get a better experience when operating these operation controls after split screen.
在另一些实施例中,手机检测到柔性屏幕从折叠状态切换为支架(或展开)状态后,也可根据上述配置文件1在绘制应用界面之前确定即将绘制的应用界面中的操作控件和非操作控件。进而,如图15中的(a)-(b)所示,手机可将原本需要绘制的视图树1300拆分为视图树1401和视图树1402。视图树1401中均为需要绘制的操作控件,视图树1402中均为需要绘制的非操作控件。这样,仍如图15中的(b)所示,手机可按照视图树1401在副屏42中绘制各个操作控件,并按照视图树1402在主屏41中绘制各个非操作控件。也就是说,手机可根据配置文件在绘制应用界面之前识别出应用界面中的操作控件和非操作控件,进而将操作控件和非操作控件分屏显示在主屏41和副屏42中,无需在绘制出上述游戏界面1101后在根据配置文件二次绘制分屏后的界面。In other embodiments, after the mobile phone detects that the flexible screen is switched from the folded state to the stand (or unfolded) state, it can also determine the operation controls and non-operations in the application interface to be drawn before drawing the application interface according to the above configuration file 1. Control. Furthermore, as shown in (a)-(b) of FIG. 15, the mobile phone can split the view tree 1300 that originally needs to be drawn into a view tree 1401 and a view tree 1402. The view tree 1401 is all operation controls that need to be drawn, and the view tree 1402 is all the non-operation controls that need to be drawn. In this way, as still shown in FIG. 15(b), the mobile phone can draw various operation controls on the secondary screen 42 according to the view tree 1401, and draw various non-operation controls on the main screen 41 according to the view tree 1402. In other words, the mobile phone can identify the operating controls and non-operating controls in the application interface before drawing the application interface according to the configuration file, and then display the operating controls and non-operating controls on the main screen 41 and the secondary screen 42 separately, without drawing After the above-mentioned game interface 1101 is displayed, the split-screen interface is drawn twice according to the configuration file.
仍如图16所示,手机将上述游戏界面1101中的操作控件绘制在副屏42后形成了第一界面1501,手机将上述游戏界面1101中的非操作控件绘制在主屏后形成了第二界面1502。在本申请实施例中,各个操作控件被分屏至副屏42后,用户可在副屏42中操作各个操作控件继续完成游戏应用的相关功能。As shown in FIG. 16, the mobile phone draws the operation controls in the game interface 1101 on the secondary screen 42 to form a first interface 1501, and the mobile phone draws the non-operation controls in the game interface 1101 on the main screen to form a second interface. 1502. In the embodiment of the present application, after each operation control is divided into the secondary screen 42, the user can operate each operation control on the secondary screen 42 to continue to complete related functions of the game application.
示例性的,手机将上述游戏界面1101中的操作控件绘制在副屏42中后,副屏42上的触摸传感器可实时检测用户在第一界面1501上的触摸操作。如图16所示,用户可触摸第一界面1501中的操作手柄1503控制游戏人物上下左右移动。当用户使用手指移动操作手柄1503开始滑动时,副屏42可将检测到的一系列触摸事件的触摸信息(例如,触摸点的坐标信息、触摸时间等)依次上报给手机的内核层、应用程序框架层以及应用程序层。Exemplarily, after the mobile phone draws the operation controls in the aforementioned game interface 1101 on the secondary screen 42, the touch sensor on the secondary screen 42 can detect the user's touch operation on the first interface 1501 in real time. As shown in FIG. 16, the user can touch the operating handle 1503 in the first interface 1501 to control the game character to move up, down, left, and right. When the user uses the finger to move the operating handle 1503 to start sliding, the secondary screen 42 can sequentially report the touch information (for example, the coordinate information of the touch point, the touch time, etc.) of the detected series of touch events to the kernel layer of the mobile phone and the application program. Framework layer and application layer.
以副屏42中的坐标系为第一坐标系,主屏41中的坐标系为第二坐标系举例。如果副屏42检测到的某一触摸事件中触摸点的坐标为P(x,y),结合图7所示的安卓***的架构图,如图17所示,副屏42通过驱动可将检测到的该触摸事件封装为原始触摸事件上报给内核层,进而由内核层将该原始触摸事件封装为上层可读取的高级触摸事件上报给应用程序框架层。另外,如果副屏42中定义的第一坐标系与手机的操作***为副屏42定义的第一坐标系不同,例如,副屏42是以左下角为原点设置的第一坐标系,而手机的操作***可能是以左上角为原点设置的第一坐标系,那么,在将上述原始触摸事件封装为高级触摸事件时,内核层还可以将触摸点的坐标P(x,y)转换为操作***定义的第一坐标系内的坐标P’(x’,y’)。Take the coordinate system in the secondary screen 42 as the first coordinate system, and the coordinate system in the main screen 41 as the second coordinate system. If the coordinates of the touch point in a touch event detected by the secondary screen 42 are P(x, y), combined with the architecture diagram of the Android system shown in FIG. 7, as shown in FIG. 17, the secondary screen 42 can be detected by driving The received touch event is encapsulated as an original touch event and reported to the kernel layer, which then encapsulates the original touch event as an upper-level readable advanced touch event and reports it to the application framework layer. In addition, if the first coordinate system defined in the secondary screen 42 is different from the first coordinate system defined for the secondary screen 42 by the operating system of the mobile phone, for example, the secondary screen 42 is the first coordinate system set at the lower left corner as the origin, and the mobile phone The operating system may be the first coordinate system set at the upper left corner as the origin. Then, when the original touch event is encapsulated as an advanced touch event, the kernel layer can also convert the coordinates of the touch point P(x, y) into an operation The coordinates P'(x',y') in the first coordinate system defined by the system.
应用程序框架层接收到携带有坐标P(x,y)或P’(x’,y’)的高级触摸事件后,可根据上述配置文件1中记录的各个操作控件在分屏后的位置关系,确定出本次的触摸事件是对操作手柄1503执行的操作。进而,应用程序框架层可根据配置文件1中操作手柄1503在分屏前后的位置关系,将上述触摸点为P点的第一触摸事件转换为在第二坐标系内触摸点为Q点的第二触摸事件。After the application framework layer receives the advanced touch event carrying the coordinates P(x,y) or P'(x',y'), it can follow the positional relationship of each operation control recorded in the configuration file 1 after the split screen , It is determined that this touch event is an operation performed on the operating handle 1503. Furthermore, the application framework layer can convert the first touch event whose touch point is P point to the first touch event whose touch point is Q point in the second coordinate system according to the positional relationship of the operating handle 1503 before and after the split screen in the configuration file 1. Two touch events.
示例性的,如图18中的(a)所示,分屏前操作手柄1302在游戏界面1101中位于以A点(40,40)为圆心,半径为40的圆形区域内。如图18中的(b)所示,分屏后手机按照配置文件1将操作手柄1503显示在副屏42中以B点(80,80)为圆心,半径为80的圆形区域 内。相比于操作手柄1302,操作手柄1503在x轴和y轴均被放大了两倍。那么,如果检测到上述第一触摸事件中触摸点P在第一坐标系中的坐标为(100,80),则应用程序框架层可以确定出在第二坐标系中与P点对应的触摸点Q的坐标为(50,40)。也就是说,第一坐标系中操作手柄1503上的触摸点P(x,y)对应于第二坐标系中操作手柄1302上的触摸点Q(0.5x,0.5y)。Exemplarily, as shown in (a) of FIG. 18, the operating handle 1302 before split screen is located in a circular area with a center of A (40, 40) and a radius of 40 in the game interface 1101. As shown in (b) in Figure 18, after split screen, the mobile phone displays the operating handle 1503 in the secondary screen 42 in a circular area with a radius of 80 and point B (80, 80) as the center according to the configuration file 1. Compared with the operating handle 1302, the operating handle 1503 is enlarged twice in both the x-axis and the y-axis. Then, if it is detected that the coordinates of the touch point P in the first coordinate system in the first touch event mentioned above are (100, 80), the application framework layer can determine the touch point corresponding to point P in the second coordinate system The coordinates of Q are (50,40). That is, the touch point P (x, y) on the operating handle 1503 in the first coordinate system corresponds to the touch point Q (0.5x, 0.5y) on the operating handle 1302 in the second coordinate system.
进而,应用程序框架层可将携带触摸点Q(0.5x,0.5y)的第二触摸事件中上报给应用层中正在运行的游戏应用,使得游戏应用可以根据触摸点Q(0.5x,0.5y)对用户本次的触摸操作进行响应。也就是说,手机检测到用户在副屏42中输入触摸点P(x,y)的第一触摸事件后,游戏应用最终向用户响应的是在主屏41中Q(0.5x,0.5y)点的第二触摸事件,这样,手机将操作控件和非操作控件分屏显示后,应用也能够正常响应分屏后用户对操作控件执行的触摸操作。Furthermore, the application framework layer can report the second touch event carrying the touch point Q (0.5x, 0.5y) to the running game application in the application layer, so that the game application can be based on the touch point Q (0.5x, 0.5y). ) Respond to the user's touch operation this time. In other words, after the mobile phone detects the first touch event of the user inputting the touch point P(x, y) on the secondary screen 42, the game application finally responds to the user at the Q(0.5x, 0.5y) point on the primary screen 41 In this way, after the mobile phone displays the operating controls and non-operating controls on separate screens, the application can also normally respond to the touch operations performed by the user on the operating controls after the split screen.
需要说明的是,上述实施例中是以上述游戏界面1101举例阐述游戏界面1101分屏前后的显示方法。可以理解的是,手机还可以在运行其他应用或应用界面时按照上述方法对应用界面中的操作控件和非操作控件进行分屏显示。另外,上述实施例中是以外折式柔性屏幕举例说明的,可以理解的是,对于内折式柔性屏幕,手机也可按照上述方法对应用界面中的操作控件和非操作控件进行分屏显示,本申请实施例对此不做任何限制。It should be noted that, in the foregoing embodiment, the foregoing game interface 1101 is used as an example to illustrate the display method of the game interface 1101 before and after the split screen. It is understandable that the mobile phone can also display the operating controls and non-operating controls in the application interface in a split screen according to the above method when running other applications or application interfaces. In addition, in the above-mentioned embodiment, the outer-folding flexible screen is exemplified. It can be understood that for the inner-folding flexible screen, the mobile phone can also display the operating controls and non-operating controls in the application interface according to the above method. The embodiments of the application do not impose any restriction on this.
示例性的,如图19中的(a)所示,当主屏41和副屏42之间的夹角为0(或接近0)时,内折式柔性屏幕处于折叠状态。此时主屏41和副屏42均可为黑屏状态。如果检测到主屏41和副屏42之间的夹角为180°,说明此时柔性屏幕从折叠状态切换为展开状态。如果检测到主屏41和副屏42之间的夹角在[90,180)的范围内,且主屏41和副屏42均为横屏状态时,说明此时柔性屏幕从折叠状态切换为支架状态。Exemplarily, as shown in (a) of FIG. 19, when the included angle between the main screen 41 and the sub screen 42 is 0 (or close to 0), the inward-folding flexible screen is in a folded state. At this time, both the main screen 41 and the sub screen 42 can be in a black screen state. If it is detected that the included angle between the main screen 41 and the secondary screen 42 is 180°, it means that the flexible screen is switched from the folded state to the unfolded state at this time. If it is detected that the angle between the main screen 41 and the sub screen 42 is within the range of [90, 180), and the main screen 41 and the sub screen 42 are both in landscape mode, it means that the flexible screen is switched from the folded state to the stand state. .
在支架状态下,手机可解锁柔性屏幕的主屏41或副屏42,也可将主屏41和副屏42作为一个完整的显示区域解锁后进行显示。如图19中的(b)所示,手机解锁主屏41后可在主屏41中显示桌面1801。桌面1801中包括视频应用的图标1802。如果检测到用户打开该视频应用的操作,手机可打开视频应用显示视频应用的各个应用界面。In the stand state, the mobile phone can unlock the main screen 41 or the sub-screen 42 of the flexible screen, and can also use the main screen 41 and the sub-screen 42 as a complete display area for display after unlocking. As shown in (b) in FIG. 19, after the mobile phone unlocks the main screen 41, the desktop 1801 can be displayed on the main screen 41. The desktop 1801 includes an icon 1802 of a video application. If the user's operation to open the video application is detected, the mobile phone can open the video application to display various application interfaces of the video application.
如图20中的(a)所示,当检测到主屏41显示视频应用的播放界面1901时,手机可自动按照上述实施例中的方法识别播放界面1901中的操作控件和非操作控件。如果识别出播放界面1901中的视频画面1902为非操作控件,播放界面1901中的进度条1903、播放按钮1904、快进按钮1905、弹幕按钮1906以及返回按钮1907为操作控件,则如图20中的(b)所示,手机可将视频画面1902(即非操作控件)显示在朝向用户的主屏41中,并将进度条1903、播放按钮1904、快进按钮1905、弹幕按钮1906以及返回按钮1907(即操作控件)显示在副屏42中。后续,用户可以在副屏42中操作各个操作控件控制视频应用,这些操作控件不会对主屏41中的视频画面1902产生遮挡,使用户获得沉浸式的观看和使用体验。As shown in (a) of FIG. 20, when it is detected that the main screen 41 displays the playback interface 1901 of the video application, the mobile phone can automatically identify the operation controls and non-operation controls in the playback interface 1901 according to the method in the foregoing embodiment. If it is recognized that the video screen 1902 in the playback interface 1901 is a non-operation control, the progress bar 1903, the playback button 1904, the fast forward button 1905, the barrage button 1906, and the return button 1907 in the playback interface 1901 are the operation controls, as shown in Figure 20 As shown in (b), the mobile phone can display the video screen 1902 (ie, non-operation control) on the main screen 41 facing the user, and display the progress bar 1903, the play button 1904, the fast forward button 1905, the pop-up button 1906 and the return The button 1907 (that is, the operation control) is displayed in the secondary screen 42. Subsequently, the user can operate various operation controls on the secondary screen 42 to control the video application. These operation controls will not block the video screen 1902 on the primary screen 41, so that the user can obtain an immersive viewing and using experience.
又或者,仍以内折式柔性屏幕举例,如果手机在锁屏前正在前台运行上述视频应用的播放界面1901,如图21中的(a)所示,用户锁屏后可将柔性屏幕折叠,或用户将柔性屏幕折叠后手机可自动锁屏。如图21中的(b)所示,当检测到柔性屏幕从折叠状态切换为展开状态(或支架状态),且屏幕解锁后,手机可自动将上述播放界面1901中的视频画面1902(即非操作控件)显示在主屏41中,并将进度条1903、播放按钮1904、快进按钮1905、弹幕按钮1906以及返回按钮(即操作控件)显示在副屏42中。后续,用户可以在副屏42中操作各个操作控件控制视频应用,这些操作控件不会对主屏41中的视频画面1902产生遮挡,使用户获得沉浸式的观看和使用体验。Or, taking the example of an inward-folding flexible screen, if the mobile phone is running the playback interface 1901 of the above video application in the foreground before the screen is locked, as shown in (a) in Figure 21, the user can fold the flexible screen after locking the screen, or After the user folds the flexible screen, the phone can automatically lock the screen. As shown in (b) in Figure 21, when it is detected that the flexible screen is switched from the folded state to the unfolded state (or stand state), and the screen is unlocked, the mobile phone can automatically display the video screen 1902 (that is, non- The operation controls) are displayed on the main screen 41, and the progress bar 1903, the play button 1904, the fast forward button 1905, the popup button 1906, and the return button (ie, operation controls) are displayed on the secondary screen 42. Subsequently, the user can operate various operation controls on the secondary screen 42 to control the video application. These operation controls will not block the video screen 1902 on the primary screen 41, so that the user can obtain an immersive viewing and using experience.
又或者,当柔性屏幕的主屏41和副屏42之间的夹角小于90°或大于180°时,手机还可以将整个柔性屏幕作为完整的显示区域显示第一界面(例如上述播放界面1901或游戏界面1101)。如果柔性屏幕切换至展开状态或支架状态,即主屏41和副屏42之间的夹角在[90,180°]的范围内时,手机也可按照上述实施例提供的显示方法识别第一界面中的操作控件和非操作控件,并将识别出的操作控件显示在主屏41和副屏42中的一个屏幕中,将识别出的非操作控件显示在主屏41和副屏42中的另一个屏幕中,本申请实施例对此不做任何限制。Or, when the included angle between the main screen 41 and the secondary screen 42 of the flexible screen is less than 90° or greater than 180°, the mobile phone can also use the entire flexible screen as a complete display area to display the first interface (for example, the aforementioned playback interface 1901 or Game interface 1101). If the flexible screen is switched to the expanded state or the stand state, that is, when the angle between the main screen 41 and the secondary screen 42 is in the range of [90, 180°], the mobile phone can also recognize the first interface according to the display method provided in the above embodiment Display the identified operation controls on one of the main screen 41 and the secondary screen 42, and display the identified non-operation controls on the other screen of the primary screen 41 and the secondary screen 42 However, the embodiments of the present application do not impose any restriction on this.
本申请实施例公开了一种电子设备,包括处理器,以及与处理器相连的存储器、输入设备、输出设备和通信模块。其中,输入设备和输出设备可集成为一个设备,例如,输入设备和输出设备可为柔性屏幕,柔性屏幕中可包括显示器件和触摸传感器。The embodiment of the present application discloses an electronic device including a processor, and a memory, an input device, an output device, and a communication module connected to the processor. Among them, the input device and the output device may be integrated into one device. For example, the input device and the output device may be a flexible screen, and the flexible screen may include a display device and a touch sensor.
此时,如图22所示,上述电子设备可以包括:柔性屏幕2201;一个或多个处理器2202;存储器2203;一个或多个应用程序(未示出);以及一个或多个计算机程序2204,上述各器件可以通过一个或多个通信总线2205连接。其中该一个或多个计算机程序2204被存储在上述存储器2203中并被配置为被该一个或多个处理器2202执行,该一个或多个计算机程序2204包括指令,上述指令可以用于执行上述实施例中的各个步骤。其中,上述方法实施例涉及的各步骤的所有相关内容均可以援引到对应实体器件的功能描述,在此不再赘述。At this time, as shown in FIG. 22, the aforementioned electronic device may include: a flexible screen 2201; one or more processors 2202; a memory 2203; one or more application programs (not shown); and one or more computer programs 2204 The above-mentioned devices may be connected through one or more communication buses 2205. The one or more computer programs 2204 are stored in the aforementioned memory 2203 and configured to be executed by the one or more processors 2202. The one or more computer programs 2204 include instructions, and the aforementioned instructions can be used to execute the aforementioned implementations. The steps in the example. Among them, all relevant content of the steps involved in the above method embodiments can be cited in the functional description of the corresponding physical device, which will not be repeated here.
示例性的,上述处理器2202具体可以为图2所示的处理器110,上述存储器2203具体可以为图2所示的内部存储器121和/或外部存储器122,上述柔性屏幕2201具体可以包括图2所示的显示屏194和传感器模块180中的触摸传感器,本申请实施例对此不做任何限制。Exemplarily, the foregoing processor 2202 may specifically be the processor 110 shown in FIG. 2, the foregoing memory 2203 may specifically be the internal memory 121 and/or the external memory 122 shown in FIG. 2, and the foregoing flexible screen 2201 may specifically include FIG. 2 The display screen 194 and the touch sensor in the sensor module 180 shown are not limited in the embodiment of the present application.
通过以上的实施方式的描述,所属领域的技术人员可以清楚地了解到,为描述的方便和简洁,仅以上述各功能模块的划分进行举例说明,实际应用中,可以根据需要而将上述功能分配由不同的功能模块完成,即将装置的内部结构划分成不同的功能模块,以完成以上描述的全部或者部分功能。上述描述的***,装置和单元的具体工作过程,可以参考前述方法实施例中的对应过程,在此不再赘述。Through the description of the above embodiments, those skilled in the art can clearly understand that for the convenience and brevity of the description, only the division of the above-mentioned functional modules is used as an example for illustration. In practical applications, the above-mentioned functions can be allocated as needed. It is completed by different functional modules, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. For the specific working process of the system, device, and unit described above, reference may be made to the corresponding process in the foregoing method embodiment, which is not repeated here.
在本申请实施例各个实施例中的各功能单元可以集成在一个处理单元中,也可以是各个单元单独物理存在,也可以两个或两个以上单元集成在一个单元中。上述集成的单元既可以采用硬件的形式实现,也可以采用软件功能单元的形式实现。The functional units in the various embodiments of the embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units may be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or software functional unit.
所述集成的单元如果以软件功能单元的形式实现并作为独立的产品销售或使用时,可以存储在一个计算机可读取存储介质中。基于这样的理解,本申请实施例的技术方案本质上或者说对现有技术做出贡献的部分或者该技术方案的全部或部分可以以软件产品的形式体现出来,该计算机软件产品存储在一个存储介质中,包括若干指令用以使得一台计算机设备(可以是个人计算机,服务器,或者网络设备等)或处理器执行本申请各个实施例所述方法的全部或部分步骤。而前述的存储介质包括:快闪存储器、移动硬盘、只读存储器、随机存取存储器、磁碟或者光盘等各种可以存储程序代码的介质。If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the embodiments of the present application essentially or the part that contributes to the prior art or all or part of the technical solutions can be embodied in the form of software products, and the computer software products are stored in a storage The medium includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) or a processor to execute all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage media include: flash memory, mobile hard disk, read-only memory, random access memory, magnetic disk or optical disk and other media that can store program codes.
以上所述,仅为本申请实施例的具体实施方式,但本申请实施例的保护范围并不局限于此,任何在本申请实施例揭露的技术范围内的变化或替换,都应涵盖在本申请实施例的保护范围之内。因此,本申请实施例的保护范围应以所述权利要求的保护范围为准。The above are only specific implementations of the embodiments of the present application, but the protection scope of the embodiments of the present application is not limited to this. Any changes or substitutions within the technical scope disclosed in the embodiments of the present application shall be covered by this Within the protection scope of the application embodiments. Therefore, the protection scope of the embodiments of the present application should be subject to the protection scope of the claims.
Claims (13)
- 一种柔性屏幕的显示方法,其特征在于,所述柔性屏幕在折叠时被折叠线划分为第一屏幕和第二屏幕,所述方法包括:A display method of a flexible screen, wherein the flexible screen is divided into a first screen and a second screen by a folding line when the flexible screen is folded, and the method includes:当所述第一屏幕和所述第二屏幕之间的夹角小于第一预设值或大于第二预设值时,电子设备在所述第一屏幕和/或所述第二屏幕中显示第一界面;When the angle between the first screen and the second screen is less than a first preset value or greater than a second preset value, the electronic device displays on the first screen and/or the second screen First interface当所述第一屏幕和所述第二屏幕之间的夹角在所述第一预设值和所述第二预设值构成的闭区间内时,所述电子设备确定所述第一界面中的操作控件和非操作控件;When the angle between the first screen and the second screen is within the closed interval formed by the first preset value and the second preset value, the electronic device determines the first interface Operational controls and non-operational controls in所述电子设备在所述第一屏幕中显示第二界面,所述第二界面中包括所述操作控件;并且,所述电子设备在所述第二屏幕中显示第三界面,所述第三界面中包括所述非操作控件。The electronic device displays a second interface on the first screen, and the second interface includes the operation controls; and, the electronic device displays a third interface on the second screen, and the third interface The interface includes the non-operation control.
- 根据权利要求1所述的方法,其特征在于,当所述第一屏幕和所述第二屏幕之间的夹角在所述第一预设值和所述第二预设值构成的闭区间内时,所述方法还包括:The method according to claim 1, wherein when the angle between the first screen and the second screen is within a closed interval formed by the first preset value and the second preset value When internally, the method further includes:所述电子设备查询所述第一界面是否为预设的应用界面;The electronic device queries whether the first interface is a preset application interface;其中,所述电子设备确定所述第一界面中的操作控件和非操作控件,包括:Wherein, the electronic device determining operation controls and non-operation controls in the first interface includes:若所述第一界面为预设的应用界面,则所述电子设备确定所述第一界面中的操作控件和非操作控件。If the first interface is a preset application interface, the electronic device determines operation controls and non-operation controls in the first interface.
- 根据权利要求1所述的方法,其特征在于,当所述第一屏幕和所述第二屏幕之间的夹角在所述第一预设值和所述第二预设值构成的闭区间内时,所述方法还包括:The method according to claim 1, wherein when the angle between the first screen and the second screen is within a closed interval formed by the first preset value and the second preset value When internally, the method further includes:所述电子设备确定所述第一屏幕和所述第二屏幕是否为横屏状态;The electronic device determines whether the first screen and the second screen are in a landscape state;其中,所述电子设备确定所述第一界面中的操作控件和非操作控件,包括:Wherein, the electronic device determining operation controls and non-operation controls in the first interface includes:若所述第一屏幕和所述第二屏幕均为横屏状态,则所述电子设备确定所述第一界面中的操作控件和非操作控件。If the first screen and the second screen are both in a landscape state, the electronic device determines operation controls and non-operation controls in the first interface.
- 根据权利要求1-3中任一项所述的方法,其特征在于,所述电子设备确定所述第一界面中的操作控件和非操作控件,包括:The method according to any one of claims 1-3, wherein the electronic device determining operation controls and non-operation controls in the first interface comprises:所述电子设备获取所述第一界面中每个控件的位置和尺寸;Acquiring, by the electronic device, the position and size of each control in the first interface;所述电子设备将尺寸大于预设尺寸的控件确定为所述非操作控件,并将所述非操作控件所在图层之上控件确定为所述操作控件。The electronic device determines a control whose size is larger than a preset size as the non-operation control, and determines the control on the layer where the non-operation control is located as the operation control.
- 根据权利要求1-3中任一项所述的方法,其特征在于,所述电子设备确定所述第一界面中的操作控件和非操作控件,包括:The method according to any one of claims 1-3, wherein the electronic device determining operation controls and non-operation controls in the first interface comprises:所述电子设备获取与所述第一界面对应的预设的配置文件,所述配置文件中记录有所述第一界面中操作控件的显示位置;Acquiring, by the electronic device, a preset configuration file corresponding to the first interface, and the configuration file records the display position of the operation control in the first interface;所述电子设备按照所述配置文件识别所述第一界面中的操作控件,并将所述第一界面中除所述操作控件外的控件确定为非操作控件。The electronic device recognizes the operation controls in the first interface according to the configuration file, and determines the controls in the first interface except the operation controls as non-operation controls.
- 根据权利要求1-5中任一项所述的方法,其特征在于,所述第二屏幕为朝向用户的屏幕。The method according to any one of claims 1-5, wherein the second screen is a screen facing the user.
- 根据权利要求1-6中任一项所述的方法,其特征在于,在所述电子设备在所述第一屏幕中显示第二界面之前,还包括:The method according to any one of claims 1-6, wherein before the electronic device displays the second interface on the first screen, further comprising:所述电子设备获取绘制所述第一界面的第一视图信息,所述第一视图信息包括所述第一界面内各个控件之间的图层顺序;Acquiring, by the electronic device, first view information for drawing the first interface, where the first view information includes the order of layers between controls in the first interface;所述电子设备根据所述第一视图信息生成与所述第二界面对应的第二视图信息以及与所述第三界面对应的第三视图信息,所述第二视图信息包括所述操作控件之间的图层顺序,所述第三视图信息包括所述非操作控件之间的图层顺序;The electronic device generates second view information corresponding to the second interface and third view information corresponding to the third interface according to the first view information, and the second view information includes one of the operation controls The layer order between the non-operation controls, and the third view information includes the layer order between the non-operation controls;其中,所述电子设备在所述第一屏幕中显示第二界面,并且,所述电子设备在所述第二屏幕中显示第三界面,包括:Wherein, the electronic device displaying a second interface on the first screen, and the electronic device displaying a third interface on the second screen, includes:所述电子设备按照所述第二视图信息在所述第一屏幕中绘制所述第二界面;并且,所述电子设备按照所述第三视图信息在所述第二屏幕中绘制所述第三界面。The electronic device draws the second interface on the first screen according to the second view information; and, the electronic device draws the third interface on the second screen according to the third view information. interface.
- 根据权利要求7所述的方法,其特征在于,与所述第一界面对应的配置文件中记录有所述操作控件在所述第二界面中的显示位置,以及所述非操作控件在所述第三界面中的显示位置;The method according to claim 7, wherein the configuration file corresponding to the first interface records the display position of the operation control in the second interface, and the display position of the non-operation control in the The display position in the third interface;其中,所述电子设备根据所述第一视图信息生成与所述第二界面对应的第二视图信息以及与所述第三界面对应的第三视图信息,包括:Wherein, the electronic device generating second view information corresponding to the second interface and third view information corresponding to the third interface according to the first view information includes:所述电子设备按照所述配置文件中记录的所述操作控件在所述第二界面中的显示位置,对所述第一视图信息中的操作控件进行拆分和重组,得到所述第二视图信息;The electronic device splits and reorganizes the operation control in the first view information according to the display position of the operation control recorded in the configuration file in the second interface to obtain the second view information;所述电子设备按照所述配置文件中记录的所述非操作控件在所述第三界面中的显示位置,对所述第一视图信息中的非操作控件进行拆分和重组,得到所述第三视图信息。The electronic device splits and reorganizes the non-operational control in the first view information according to the display position of the non-operational control in the third interface recorded in the configuration file to obtain the first Three views of information.
- 根据权利要求1-8中任一项所述的方法,其特征在于,在所述电子设备在所述第一屏幕中显示第二界面之后,还包括:The method according to any one of claims 1-8, wherein after the electronic device displays a second interface on the first screen, the method further comprises:所述电子设备接收用户在所述第二界面向所述操作控件输入的第一触摸事件;Receiving, by the electronic device, a first touch event input by a user to the operation control on the second interface;所述电子设备将所述第一触摸事件映射为在所述第一界面中的第二触摸事件;The electronic device maps the first touch event to a second touch event in the first interface;所述电子设备执行与所述第二触摸事件对应的操作指令。The electronic device executes an operation instruction corresponding to the second touch event.
- 根据权利要求9所述的方法,其特征在于,所述电子设备将所述第一触摸事件映射为在所述第一界面中的第二触摸事件,包括:The method according to claim 9, wherein the electronic device mapping the first touch event to a second touch event in the first interface comprises:所述电子设备获取所述第一触摸事件中第一触摸点的坐标;Acquiring, by the electronic device, the coordinates of the first touch point in the first touch event;所述电子设备根据所述操作控件在所述第一界面和所述第二界面上的相对位置关系,计算所述第一触摸点在所述第一界面中对应的第二触摸点的坐标;The electronic device calculates the coordinates of the second touch point corresponding to the first touch point in the first interface according to the relative position relationship of the operation control on the first interface and the second interface;其中,所述所述电子设备执行与所述第二触摸事件对应的操作指令,包括:The execution of the operation instruction corresponding to the second touch event by the electronic device includes:所述电子设备将所述第二触摸点的坐标携带在第二触摸事件中上报给第一应用,所述第一界面为所述第一应用的界面。The electronic device carries the coordinates of the second touch point in a second touch event and reports it to the first application, and the first interface is an interface of the first application.
- 一种电子设备,其特征在于,包括:An electronic device, characterized in that it comprises:柔性屏幕,所述柔性屏幕在折叠时被折叠线划分为第一屏幕和第二屏幕;A flexible screen, which is divided into a first screen and a second screen by folding lines when the flexible screen is folded;一个或多个处理器;One or more processors;一个或多个存储器;One or more memories;以及一个或多个计算机程序,其中所述一个或多个计算机程序被存储在所述一个或多个存储器中,所述一个或多个计算机程序包括指令,当所述指令被所述电子设备执行时,使得所述电子设备执行如权利要求1-10中任一项所述的柔性屏幕的显示方法。And one or more computer programs, wherein the one or more computer programs are stored in the one or more memories, the one or more computer programs include instructions, when the instructions are executed by the electronic device At this time, the electronic device is caused to execute the flexible screen display method according to any one of claims 1-10.
- 一种计算机可读存储介质,所述计算机可读存储介质中存储有指令,其特征在于,当所述指令在电子设备上运行时,使得所述电子设备执行如权利要求1-10中任一项所述的柔性屏幕的显示方法。A computer-readable storage medium having instructions stored in the computer-readable storage medium, wherein when the instructions are executed on an electronic device, the electronic device executes any one of claims 1-10 The flexible screen display method described in the item.
- 一种包含指令的计算机程序产品,其特征在于,当所述计算机程序产品在电子设备上运行时,使得所述电子设备执行如权利要求1-10中任一项所述的柔性屏幕的显示方法。A computer program product containing instructions, characterized in that, when the computer program product runs on an electronic device, the electronic device is caused to execute the flexible screen display method according to any one of claims 1-10 .
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201910488438.1 | 2019-06-05 | ||
CN201910488438.1A CN110389802B (en) | 2019-06-05 | 2019-06-05 | Display method of flexible screen and electronic equipment |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2020244497A1 true WO2020244497A1 (en) | 2020-12-10 |
Family
ID=68285277
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/CN2020/093899 WO2020244497A1 (en) | 2019-06-05 | 2020-06-02 | Display method for flexible screen and electronic device |
Country Status (2)
Country | Link |
---|---|
CN (3) | CN110389802B (en) |
WO (1) | WO2020244497A1 (en) |
Cited By (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN113129753A (en) * | 2021-04-15 | 2021-07-16 | 维沃移动通信有限公司 | Inward folding type electronic equipment |
CN114428534A (en) * | 2022-01-12 | 2022-05-03 | Oppo广东移动通信有限公司 | Electronic device and control method thereof |
CN116055597A (en) * | 2022-08-08 | 2023-05-02 | 荣耀终端有限公司 | Folding angle detection method and electronic equipment |
CN116257235A (en) * | 2021-12-10 | 2023-06-13 | 华为技术有限公司 | Drawing method and electronic equipment |
WO2024017090A1 (en) * | 2022-07-21 | 2024-01-25 | 华为技术有限公司 | Information display method and electronic device |
CN117519864A (en) * | 2023-09-19 | 2024-02-06 | 荣耀终端有限公司 | Interface display method, electronic device and storage medium |
Families Citing this family (21)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN110389802B (en) * | 2019-06-05 | 2021-05-18 | 华为技术有限公司 | Display method of flexible screen and electronic equipment |
CN110955373A (en) * | 2019-10-30 | 2020-04-03 | 华为技术有限公司 | Display element display method and electronic equipment |
CN111124561B (en) * | 2019-11-08 | 2021-02-12 | 华为技术有限公司 | Display method applied to electronic equipment with folding screen and electronic equipment |
KR20210068853A (en) * | 2019-12-02 | 2021-06-10 | 삼성전자주식회사 | A foldable electronic device comprising a camera |
CN111190559A (en) * | 2019-12-04 | 2020-05-22 | 深圳市东向同人科技有限公司 | Screen projection control synchronization method, mobile terminal and computer readable storage medium |
CN110968252B (en) * | 2019-12-18 | 2021-10-22 | 华为技术有限公司 | Display method of interactive system, interactive system and electronic equipment |
CN114237530A (en) * | 2020-01-21 | 2022-03-25 | 华为技术有限公司 | Display method and related device of folding screen |
CN111405339B (en) * | 2020-03-11 | 2022-08-12 | 咪咕互动娱乐有限公司 | Split screen display method, electronic equipment and storage medium |
CN113542453A (en) * | 2020-03-31 | 2021-10-22 | 北京小米移动软件有限公司 | Folding screen terminal device, state detection method thereof and electronic device |
CN111522523A (en) * | 2020-04-30 | 2020-08-11 | 北京小米移动软件有限公司 | Display processing method and device and computer storage medium |
CN114356196B (en) * | 2020-09-30 | 2022-12-20 | 荣耀终端有限公司 | Display method and electronic equipment |
CN113867663B (en) * | 2020-10-22 | 2024-04-09 | 华为技术有限公司 | Display method and electronic equipment |
CN112565490B (en) * | 2020-12-10 | 2022-08-19 | 维沃移动通信有限公司 | Electronic device |
CN114238677A (en) * | 2021-07-28 | 2022-03-25 | 中国科学院深圳先进技术研究院 | Multi-view display method, device, equipment and medium |
CN114125137A (en) * | 2021-11-08 | 2022-03-01 | 维沃移动通信有限公司 | Video display method and device, electronic equipment and readable storage medium |
CN116107482A (en) * | 2021-11-08 | 2023-05-12 | Oppo广东移动通信有限公司 | Application control method, device, terminal, storage medium and computer program product |
CN115562535B (en) * | 2022-01-14 | 2024-05-17 | 荣耀终端有限公司 | Application control method and electronic equipment |
CN115686407B (en) * | 2022-06-30 | 2023-08-29 | 荣耀终端有限公司 | Display method and electronic equipment |
CN117590982A (en) * | 2022-08-09 | 2024-02-23 | 荣耀终端有限公司 | Display method of application interface, electronic equipment and storage medium |
CN115686334B (en) * | 2022-10-31 | 2023-11-28 | 荣耀终端有限公司 | Operation control method, electronic device and readable storage medium |
CN115904151A (en) * | 2022-11-30 | 2023-04-04 | 南京欧珀软件科技有限公司 | Display method and device and electronic equipment |
Citations (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN105549785A (en) * | 2015-12-29 | 2016-05-04 | 广东欧珀移动通信有限公司 | Terminal |
US20160246558A1 (en) * | 2015-02-24 | 2016-08-25 | Samsung Display Co., Ltd. | Foldable display |
CN107728901A (en) * | 2017-10-24 | 2018-02-23 | 广东欧珀移动通信有限公司 | interface display method, device and terminal |
US20180059721A1 (en) * | 2016-08-26 | 2018-03-01 | Semiconductor Energy Laboratory Co., Ltd. | Data processing device, display method, input/output method, server system, and computer program |
CN109542380A (en) * | 2018-11-26 | 2019-03-29 | Oppo广东移动通信有限公司 | Control method, device, storage medium and the terminal of display pattern |
CN110389802A (en) * | 2019-06-05 | 2019-10-29 | 华为技术有限公司 | A kind of display methods and electronic equipment of flexible screen |
Family Cites Families (8)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
KR101301032B1 (en) * | 2007-03-08 | 2013-08-28 | 엘지전자 주식회사 | Communication Device And Method of transmitting Data File Using the Same |
CN100478881C (en) * | 2007-05-16 | 2009-04-15 | 珠海金山软件股份有限公司 | Device and method for user operating sheltered area |
KR20130080937A (en) * | 2012-01-06 | 2013-07-16 | 삼성전자주식회사 | Apparatus and method for dislplaying a screen of portable device having a flexible display |
KR102377785B1 (en) * | 2014-02-10 | 2022-03-23 | 삼성전자주식회사 | User terminal device and method for displaying thereof |
US10579238B2 (en) * | 2016-05-13 | 2020-03-03 | Sap Se | Flexible screen layout across multiple platforms |
CN107678724A (en) * | 2017-10-19 | 2018-02-09 | 广东欧珀移动通信有限公司 | A kind of method for information display, device, mobile terminal and storage medium |
CN108459815B (en) * | 2018-03-16 | 2020-06-02 | 维沃移动通信有限公司 | Display control method and mobile terminal |
CN109710161A (en) * | 2018-12-29 | 2019-05-03 | 中兴通讯股份有限公司 | A kind of display control method and device, electronic equipment |
-
2019
- 2019-06-05 CN CN201910488438.1A patent/CN110389802B/en active Active
- 2019-06-05 CN CN202110383617.6A patent/CN113157130A/en active Pending
- 2019-06-05 CN CN202110502309.0A patent/CN113268196A/en active Pending
-
2020
- 2020-06-02 WO PCT/CN2020/093899 patent/WO2020244497A1/en active Application Filing
Patent Citations (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20160246558A1 (en) * | 2015-02-24 | 2016-08-25 | Samsung Display Co., Ltd. | Foldable display |
CN105549785A (en) * | 2015-12-29 | 2016-05-04 | 广东欧珀移动通信有限公司 | Terminal |
US20180059721A1 (en) * | 2016-08-26 | 2018-03-01 | Semiconductor Energy Laboratory Co., Ltd. | Data processing device, display method, input/output method, server system, and computer program |
CN107728901A (en) * | 2017-10-24 | 2018-02-23 | 广东欧珀移动通信有限公司 | interface display method, device and terminal |
CN109542380A (en) * | 2018-11-26 | 2019-03-29 | Oppo广东移动通信有限公司 | Control method, device, storage medium and the terminal of display pattern |
CN110389802A (en) * | 2019-06-05 | 2019-10-29 | 华为技术有限公司 | A kind of display methods and electronic equipment of flexible screen |
Cited By (9)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN113129753A (en) * | 2021-04-15 | 2021-07-16 | 维沃移动通信有限公司 | Inward folding type electronic equipment |
CN113129753B (en) * | 2021-04-15 | 2023-04-07 | 维沃移动通信有限公司 | Inward folding type electronic equipment |
CN116257235A (en) * | 2021-12-10 | 2023-06-13 | 华为技术有限公司 | Drawing method and electronic equipment |
CN114428534A (en) * | 2022-01-12 | 2022-05-03 | Oppo广东移动通信有限公司 | Electronic device and control method thereof |
CN114428534B (en) * | 2022-01-12 | 2023-11-28 | Oppo广东移动通信有限公司 | Electronic apparatus and control method thereof |
WO2024017090A1 (en) * | 2022-07-21 | 2024-01-25 | 华为技术有限公司 | Information display method and electronic device |
CN116055597A (en) * | 2022-08-08 | 2023-05-02 | 荣耀终端有限公司 | Folding angle detection method and electronic equipment |
CN116055597B (en) * | 2022-08-08 | 2023-11-14 | 荣耀终端有限公司 | Folding angle detection method and electronic equipment |
CN117519864A (en) * | 2023-09-19 | 2024-02-06 | 荣耀终端有限公司 | Interface display method, electronic device and storage medium |
Also Published As
Publication number | Publication date |
---|---|
CN110389802B (en) | 2021-05-18 |
CN110389802A (en) | 2019-10-29 |
CN113268196A (en) | 2021-08-17 |
CN113157130A (en) | 2021-07-23 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
WO2020244497A1 (en) | Display method for flexible screen and electronic device | |
WO2020244492A1 (en) | Screen projection display method and electronic device | |
WO2020244495A1 (en) | Screen projection display method and electronic device | |
CN112714901B (en) | Display control method of system navigation bar, graphical user interface and electronic equipment | |
CN113645351B (en) | Application interface interaction method, electronic device and computer-readable storage medium | |
WO2020000448A1 (en) | Flexible screen display method and terminal | |
WO2020253758A1 (en) | User interface layout method and electronic device | |
CN110495819B (en) | Robot control method, robot, terminal, server and control system | |
CN111669459B (en) | Keyboard display method, electronic device and computer readable storage medium | |
WO2021036770A1 (en) | Split-screen processing method and terminal device | |
WO2022068483A1 (en) | Application startup method and apparatus, and electronic device | |
WO2022022575A1 (en) | Display control method and apparatus, and storage medium | |
WO2021078032A1 (en) | User interface display method and electronic device | |
WO2021057699A1 (en) | Method for controlling electronic device with flexible screen, and electronic device | |
CN112130788A (en) | Content sharing method and device | |
WO2021190524A1 (en) | Screenshot processing method, graphic user interface and terminal | |
WO2022143180A1 (en) | Collaborative display method, terminal device, and computer readable storage medium | |
CN114579016A (en) | Method for sharing input equipment, electronic equipment and system | |
CN115016697A (en) | Screen projection method, computer device, readable storage medium, and program product | |
WO2021196980A1 (en) | Multi-screen interaction method, electronic device, and computer-readable storage medium | |
US20220317841A1 (en) | Screenshot Method and Related Device | |
WO2022228043A1 (en) | Display method, electronic device, storage medium and program product | |
WO2022111690A1 (en) | Method for sharing input device, electronic devices, and system | |
WO2022152174A1 (en) | Screen projection method and electronic device | |
WO2022062902A1 (en) | File transfer method and electronic device |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 20818985 Country of ref document: EP Kind code of ref document: A1 |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
122 | Ep: pct application non-entry in european phase |
Ref document number: 20818985 Country of ref document: EP Kind code of ref document: A1 |