WO2023125518A1 - Image encoding method and device - Google Patents

Image encoding method and device Download PDF

Info

Publication number
WO2023125518A1
WO2023125518A1 PCT/CN2022/142277 CN2022142277W WO2023125518A1 WO 2023125518 A1 WO2023125518 A1 WO 2023125518A1 CN 2022142277 W CN2022142277 W CN 2022142277W WO 2023125518 A1 WO2023125518 A1 WO 2023125518A1
Authority
WO
WIPO (PCT)
Prior art keywords
memory
slice
preset
height
storage
Prior art date
Application number
PCT/CN2022/142277
Other languages
French (fr)
Chinese (zh)
Inventor
李国怀
Original Assignee
华为技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 华为技术有限公司 filed Critical 华为技术有限公司
Publication of WO2023125518A1 publication Critical patent/WO2023125518A1/en

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/20—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using video object coding
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/42—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation
    • H04N19/423—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation characterised by memory arrangements

Definitions

  • the present application relates to the field of artificial intelligence, and in particular to an image encoding method and device.
  • the image is divided into multiple slices (Grid), and then each slice is encoded, which can improve the compression efficiency of the image and improve the encoding performance, so the process of image slicing is particularly important
  • the present application provides an image coding method and device, which are used to code stored images by means of memory mapping, reduce memory copying, and improve coding efficiency.
  • the present application provides an image encoding method, comprising: first, allocating a storage memory for an input image; dividing the storage memory into a plurality of memory slices; The address of at least one memory slice reads data and performs encoding to obtain encoded data.
  • the address of the data to be encoded can be directly mapped to the encoder, so that the encoder can directly extract the data to be encoded based on the address, so memory copying is not required, and the memory overhead of memory copying can be reduced , and can improve the coding efficiency.
  • the aforementioned dividing the storage memory into multiple memory slices may include: dividing the storage memory into multiple memory slices according to a preset slice size, where the preset slice size includes a preset height and preset width.
  • the division when dividing the storage memory, the division may be performed according to a preset slice size, so that subsequent encoders may perform encoding in units of the preset slice size.
  • the aforementioned allocating storage memory for the input image may include: allocating storage memory for the input image based on a preset slice size.
  • the storage memory may include a first memory and a second memory, the first memory stores the information of each pixel in the input image, and the second memory is used to make the size of the storage memory according to the preset Slice size alignment so that the encoder reads data from the input address in units of the preset slice size.
  • the address of the obtained memory slice can be mapped to the encoder for encoding to obtain the final encoded data, reducing the amount of memory copy , which can reduce memory overhead for memory copying and improve coding efficiency.
  • the storage memory is arranged in an array, the width of the storage memory is divisible by a preset width, and the height of the storage memory is divisible by a preset height.
  • the size of each memory slice is the preset slice size, Avoid the coder reading the address out of bounds, and only need to allocate a small amount of memory, which reduces the amount of memory copying, reduces the memory overhead of memory copying, and improves coding efficiency.
  • the second memory includes a third memory and a fourth memory
  • the first memory and the third memory are combined and arranged in an array
  • the height of the array is divisible by the preset height
  • the last row of the array is divided by the preset height.
  • the remainder after setting the width is not smaller than the preset width after being combined with the fourth memory.
  • the height of the allocated storage memory can be aligned with the preset height.
  • additional memory can be allocated for the last row, so that the array
  • the remainder after dividing the last line by the preset width is not smaller than the preset width after being combined with the fourth memory, to prevent the encoder from crossing the boundary when reading the address, and only need to allocate a small amount of memory, which reduces the amount of memory copy and can reduce the Memory copy memory overhead, and can improve coding efficiency.
  • multiple memory slices are divided into a first memory slice and a second memory slice, the height of the first memory slice is a preset height, and the height of the second memory slice is smaller than the preset height or the second memory slice
  • the last slice of the last line in the array composed of multiple memory slices, the first memory slice and the second memory slice are different memory slices; the above method may also include: allocating the second memory slice for the second memory slice among the multiple memory slices Five memory, and copy the data in the second memory to the fifth memory, the width of the fifth memory is divisible by the preset width, and the height of the fifth memory is divisible by the preset height; the aforementioned at least one of the multiple memory slices
  • Inputting the address of the memory slice to the encoder may include: inputting the address of the first memory slice and the address of the fifth memory among the plurality of memory slices to the encoder.
  • the storage memory occupied by the input image can be allocated based on the size of the input image, and then the storage memory can be divided.
  • the storage memory of the input image is not divisible by the preset slice size, the obtained and The address of the memory that matches the preset slice size is mapped to the encoder for encoding, and the data is copied from the remainder after division and used as the input of the encoder to obtain the final encoded data without copying all the memory in the storage memory.
  • the amount of memory copy is reduced, the memory overhead of memory copy can be reduced, and the coding efficiency can be improved.
  • the height of the storage memory is evenly divisible by the preset height.
  • the height of the allocated memory can be aligned according to the preset height, thereby reducing multiple The number of slices whose dimensions did not match the preset slice size. Then you can map the address of the memory that matches the preset slice size to the encoder for encoding, and copy the data from the remainder after integer division and use it as the input of the encoder to obtain the final encoded data without needing to store the data in the memory. Copy all the memory of the memory, which reduces the amount of memory copy, can reduce the memory overhead of memory copy, and can improve the coding efficiency.
  • an image encoding device including:
  • a division module configured to divide the storage memory into multiple memory slices
  • the encoding module is configured to input the address of at least one memory slice in the plurality of memory slices to the encoder, and the encoder is used to read and encode data based on the input address to obtain encoded data.
  • the division module is specifically configured to divide the storage memory into multiple memory slices according to a preset slice size, where the preset slice size includes a preset height and a preset width.
  • the storage memory includes a first memory and a second memory, the first memory stores the information of each pixel in the input image, and the second memory is used to make the size of the storage memory according to the preset Slice size alignment so that the encoder reads data from the input address in units of the preset slice size.
  • the storage memory is arranged in an array, the width of the storage memory is divisible by a preset width, and the height of the storage memory is divisible by a preset height.
  • the second memory includes a third memory and a fourth memory
  • the first memory and the third memory are combined and arranged in an array
  • the height of the array is divisible by the preset height
  • the last row of the array is divided by the preset height.
  • the remainder after setting the width is not smaller than the preset width after being combined with the fourth memory.
  • multiple memory slices are divided into a first memory slice and a second memory slice, the height of the first memory slice is a preset height, and the height of the second memory slice is smaller than the preset height or the second memory slice It is the last slice of the last line in the array composed of multiple memory slices, the first memory slice and the second memory slice are different memory slices;
  • the allocation module is also used to allocate a fifth memory for the second memory slice in the plurality of memory slices, and copy the data in the second memory to the fifth memory, the width of the fifth memory is divisible by the preset width, and the fifth memory The height of the memory is divisible by the preset height;
  • the encoding module is specifically configured to input the address of the first memory slice and the address of the fifth memory among the plurality of memory slices to the encoder.
  • the height of the storage memory is evenly divisible by the preset height.
  • an embodiment of the present application provides an electronic device, including: a processor and a memory, wherein the processor and the memory are interconnected through a line, and the processor calls the program code in the memory to execute any one of the above-mentioned first aspects. Functions related to processing in the image coding method shown.
  • the electronic device may be a chip.
  • the embodiment of the present application provides an electronic device, which can also be called a digital processing chip or a chip.
  • the chip includes a processing unit and a communication interface.
  • the processing unit obtains program instructions through the communication interface, and the program instructions are processed by the processing unit.
  • the processing unit is configured to execute the processing-related functions in the first aspect or any optional implementation manner of the first aspect.
  • an embodiment of the present application provides a computer-readable storage medium, including instructions, which, when run on a computer, cause the computer to execute the method in the above-mentioned first aspect or any optional implementation manner of the first aspect.
  • the embodiments of the present application provide a computer program product containing instructions, which, when run on a computer, cause the computer to execute the method in the above-mentioned first aspect or any optional implementation manner of the first aspect.
  • FIG. 1 is a schematic structural diagram of an electronic device provided by the present application.
  • FIG. 2 is a schematic structural diagram of another electronic device provided by the present application.
  • FIG. 3 is a schematic flowchart of an image encoding method provided by the present application.
  • FIG. 4 is a schematic structural diagram of a storage memory provided by the present application.
  • FIG. 5 is a schematic structural diagram of another electronic device provided by the present application.
  • FIG. 6 is a schematic flowchart of another image coding method provided by the present application.
  • FIG. 7 is a schematic structural diagram of another storage memory provided by the present application.
  • FIG. 8 is a schematic structural diagram of another storage memory provided by the present application.
  • FIG. 9 is a schematic structural diagram of another storage memory provided by the present application.
  • FIG. 10 is a schematic structural diagram of another storage memory provided by the present application.
  • FIG. 11 is a schematic structural diagram of another storage memory provided by the present application.
  • FIG. 12 is a schematic structural diagram of another storage memory provided by the present application.
  • FIG. 13 is a schematic structural diagram of an image encoding device provided by the present application.
  • FIG. 14 is a schematic structural diagram of another electronic device provided by the present application.
  • the method provided by this application can be deployed in various electronic devices, such as deployed in various terminals or servers.
  • the servers can be various independent servers, distributed servers, or centralized servers.
  • the terminals can include but not Limited to: smart mobile phones, TVs, tablet computers, wristbands, head mounted display devices (Head Mount Display, HMD), augmented reality (augmented reality, AR) devices, mixed reality (mixed reality, MR) devices, cellular phones (cellular phone), smart phone (smart phone), personal digital assistant (personal digital assistant, PDA), tablet computer, vehicle terminal, laptop computer (laptop computer) (or notebook computer or laptop computer, etc.), personal computer (personal computer, PC) and so on.
  • FIG. 1 a specific structure is taken as an example below to illustrate the structure of the electronic device provided in this application.
  • the electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (universal serial bus, USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, and an antenna 2 , mobile communication module 150, wireless communication module 160, audio module 170, speaker 170A, receiver 170B, microphone 170C, earphone jack 170D, sensor module 180, button 190, motor 191, indicator 192, camera 193, display screen 194, and A subscriber identification module (subscriber identification module, SIM) card interface 195 and the like.
  • SIM subscriber identification module
  • the sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, bone conduction sensor 180M, motion sensor 180N, etc.
  • the structure illustrated in the embodiment of the present invention does not constitute a specific limitation on the electronic device 100 .
  • the electronic device 100 may include more or fewer components than shown in the figure, or combine certain components, or separate certain components, or arrange different components.
  • the illustrated components can be realized in hardware, software or a combination of software and hardware.
  • the processor 110 may include one or more processing units, for example: the processor 110 may include an application processor (application processor, AP), a modem processor, a graphics processing unit (graphics processing unit, GPU), an image signal processor (image signal processor, ISP), controller, video codec, digital signal processor (digital signal processor, DSP), baseband processor, and/or neural network processor (neural-network processing unit, NPU), etc. Wherein, different processing units may be independent devices, or may be integrated in one or more processors.
  • application processor application processor, AP
  • modem processor graphics processing unit
  • GPU graphics processing unit
  • image signal processor image signal processor
  • ISP image signal processor
  • controller video codec
  • digital signal processor digital signal processor
  • baseband processor baseband processor
  • neural network processor neural-network processing unit
  • the controller can generate an operation control signal according to the instruction opcode and timing signal, and complete the control of fetching and executing the instruction.
  • a memory may also be provided in the processor 110 for storing instructions and data.
  • the memory in processor 110 is a cache memory.
  • the memory may hold instructions or data that the processor 110 has just used or recycled. If the processor 110 needs to use the instruction or data again, it can be called directly from the memory. Repeated access is avoided, and the waiting time of the processor 110 is reduced, thereby improving the efficiency of the system.
  • processor 110 may include one or more interfaces.
  • the interface may include an integrated circuit (inter-integrated circuit, I2C) interface, an integrated circuit built-in audio (inter-integrated circuit sound, I2S) interface, a pulse code modulation (pulse code modulation, PCM) interface, a universal asynchronous transmitter (universal asynchronous receiver/transmitter, UART) interface, mobile industry processor interface (mobile industry processor interface, MIPI), general-purpose input and output (general-purpose input/output, GPIO) interface, subscriber identity module (subscriber identity module, SIM) interface, and /or universal serial bus (universal serial bus, USB) interface, etc.
  • I2C integrated circuit
  • I2S integrated circuit built-in audio
  • PCM pulse code modulation
  • PCM pulse code modulation
  • UART universal asynchronous transmitter
  • MIPI mobile industry processor interface
  • GPIO general-purpose input and output
  • subscriber identity module subscriber identity module
  • SIM subscriber identity module
  • USB universal serial bus
  • the I2C interface is a bidirectional synchronous serial bus, including a serial data line (serial data line, SDA) and a serial clock line (derail clock line, SCL).
  • processor 110 may include multiple sets of I2C buses.
  • the processor 110 can be respectively coupled to the touch sensor 180K, the charger, the flashlight, the camera 193 and the like through different I2C bus interfaces.
  • the processor 110 may be coupled to the touch sensor 180K through the I2C interface, so that the processor 110 and the touch sensor 180K communicate through the I2C bus interface to realize the touch function of the electronic device 100 .
  • the I2S interface can be used for audio communication.
  • processor 110 may include multiple sets of I2S buses.
  • the processor 110 may be coupled to the audio module 170 through an I2S bus to implement communication between the processor 110 and the audio module 170 .
  • the audio module 170 can transmit audio signals to the wireless communication module 160 through the I2S interface, so as to realize the function of answering calls through the Bluetooth headset.
  • the PCM interface can also be used for audio communication, sampling, quantizing and encoding the analog signal.
  • the audio module 170 and the wireless communication module 160 may be coupled through a PCM bus interface.
  • the audio module 170 can also transmit audio signals to the wireless communication module 160 through the PCM interface, so as to realize the function of answering calls through the Bluetooth headset. Both the I2S interface and the PCM interface can be used for audio communication.
  • the UART interface is a universal serial data bus used for asynchronous communication.
  • the bus can be a bidirectional communication bus. It converts the data to be transmitted between serial communication and parallel communication.
  • a UART interface is generally used to connect the processor 110 and the wireless communication module 160 .
  • the processor 110 communicates with the Bluetooth module in the wireless communication module 160 through the UART interface to realize the Bluetooth function.
  • the audio module 170 can transmit audio signals to the wireless communication module 160 through the UART interface, so as to realize the function of playing music through the Bluetooth headset.
  • the MIPI interface can be used to connect the processor 110 with peripheral devices such as the display screen 194 and the camera 193 .
  • MIPI interface includes camera serial interface (camera serial interface, CSI), display serial interface (display serial interface, DSI), etc.
  • the processor 110 communicates with the camera 193 through the CSI interface to realize the shooting function of the electronic device 100 .
  • the processor 110 communicates with the display screen 194 through the DSI interface to realize the display function of the electronic device 100 .
  • the GPIO interface can be configured by software.
  • the GPIO interface can be configured as a control signal or as a data signal.
  • the GPIO interface can be used to connect the processor 110 with the camera 193 , the display screen 194 , the wireless communication module 160 , the audio module 170 , the sensor module 180 and so on.
  • the GPIO interface can also be configured as an I2C interface, I2S interface, UART interface, MIPI interface, etc.
  • the USB interface 130 is an interface conforming to the USB standard specification, specifically, it can be a Mini USB interface, a Micro USB interface, a USB Type C interface, and the like.
  • the USB interface 130 can be used to connect a charger to charge the electronic device 100 , and can also be used to transmit data between the electronic device 100 and peripheral devices. It can also be used to connect headphones and play audio through them. This interface can also be used to connect other electronic devices, such as AR devices.
  • the USB interface 130 here can also be replaced with other interfaces, such as Type-c or Lighting, which can realize charging or data transmission, and only the USB interface 130 is used as an example for illustration here.
  • the interface connection relationship between the modules shown in the embodiment of the present invention is only a schematic illustration, and does not constitute a structural limitation of the electronic device 100 .
  • the electronic device 100 may also adopt different interface connection manners in the foregoing embodiments, or a combination of multiple interface connection manners.
  • the charging management module 140 is configured to receive a charging input from a charger.
  • the charger may be a wireless charger or a wired charger.
  • the charging management module 140 can receive charging input from the wired charger through the USB interface 130 .
  • the charging management module 140 may receive a wireless charging input through a wireless charging coil of the electronic device 100 . While the charging management module 140 is charging the battery 142 , it can also supply power to the electronic device through the power management module 141 .
  • the power management module 141 is used for connecting the battery 142 , the charging management module 140 and the processor 110 .
  • the power management module 141 receives the input from the battery 142 and/or the charging management module 140 to provide power for the processor 110 , the internal memory 121 , the display screen 194 , the camera 193 , and the wireless communication module 160 .
  • the power management module 141 can also be used to monitor parameters such as battery capacity, battery cycle times, and battery health status (leakage, impedance).
  • the power management module 141 may also be disposed in the processor 110 .
  • the power management module 141 and the charging management module 140 may also be set in the same device.
  • the wireless communication function of the electronic device 100 can be realized by the antenna 1 , the antenna 2 , the mobile communication module 150 , the wireless communication module 160 , a modem processor, a baseband processor, and the like.
  • Antenna 1 and Antenna 2 are used to transmit and receive electromagnetic wave signals.
  • Each antenna in electronic device 100 may be used to cover single or multiple communication frequency bands. Different antennas can also be multiplexed to improve the utilization of the antennas.
  • Antenna 1 can be multiplexed as a diversity antenna for a WLAN.
  • the antenna may be used in conjunction with a tuning switch.
  • the mobile communication module 150 can provide wireless communication solutions including 2G/3G/4G/5G applied on the electronic device 100 .
  • the mobile communication module 150 may include at least one filter, switch, power amplifier, low noise amplifier (low noise amplifier, LNA) and the like.
  • the mobile communication module 150 can receive electromagnetic waves through the antenna 1, filter and amplify the received electromagnetic waves, and send them to the modem processor for demodulation.
  • the mobile communication module 150 can also amplify the signals modulated by the modem processor, and convert them into electromagnetic waves and radiate them through the antenna 1 .
  • at least part of the functional modules of the mobile communication module 150 may be set in the processor 110 .
  • at least part of the functional modules of the mobile communication module 150 and at least part of the modules of the processor 110 may be set in the same device.
  • a modem processor may include a modulator and a demodulator.
  • the modulator is used for modulating the low-frequency baseband signal to be transmitted into a medium-high frequency signal.
  • the demodulator is used to demodulate the received electromagnetic wave signal into a low frequency baseband signal. Then the demodulator sends the demodulated low-frequency baseband signal to the baseband processor for processing.
  • the low-frequency baseband signal is passed to the application processor after being processed by the baseband processor.
  • the application processor outputs sound signals through audio equipment (not limited to speaker 170A, receiver 170B, etc.), or displays images or videos through display screen 194 .
  • the modem processor may be a stand-alone device.
  • the modem processor may be independent from the processor 110, and be set in the same device as the mobile communication module 150 or other functional modules.
  • the wireless communication module 160 can provide wireless local area networks (wireless local area networks, WLAN) (such as wireless fidelity (Wireless Fidelity, Wi-Fi) network), bluetooth (bluetooth, BT), global navigation satellite, etc. applied on the electronic device 100.
  • System global navigation satellite system, GNSS
  • frequency modulation frequency modulation, FM
  • near field communication technology near field communication, NFC
  • ultra wideband ultra wideband
  • UWB ultra wideband
  • infrared technology infrared, IR
  • the wireless communication module 160 may be one or more devices integrating at least one communication processing module.
  • the wireless communication module 160 receives electromagnetic waves via the antenna 2 , frequency-modulates and filters the electromagnetic wave signals, and sends the processed signals to the processor 110 .
  • the wireless communication module 160 can also receive the signal to be sent from the processor 110 , frequency-modulate it, amplify it, and convert it into electromagnetic waves through the antenna 2 for radiation.
  • the antenna 1 of the electronic device 100 is coupled to the mobile communication module 150, and the antenna 2 is coupled to the wireless communication module 160, so that the electronic device 100 can communicate with the network and other devices through wireless communication technology.
  • the wireless communication technology may include but not limited to: the fifth generation mobile communication technology (5th-Generation, 5G) system, the global system for mobile communications (global system for mobile communications, GSM), general packet radio service (general packet radio service, GPRS), code division multiple access (CDMA), wideband code division multiple access (WCDMA), time-division code division multiple access (TD-SCDMA) ), long term evolution (long term evolution, LTE), bluetooth (bluetooth), global navigation satellite system (the global navigation satellite system, GNSS), wireless fidelity (wireless fidelity, WiFi), near field communication (near field communication, NFC), FM (also called FM radio), Zigbee (Zigbee), radio frequency identification (radio frequency identification, RFID) and/or infrared (infrared, IR) technology, etc
  • the GNSS may include a global positioning system (global positioning system, GPS), a global navigation satellite system (global navigation satellite system, GLONASS), a Beidou navigation satellite system (beidou navigation satellite system, BDS), a quasi-zenith satellite system (quasi -zenith satellite system, QZSS) and/or satellite based augmentation systems (satellite based augmentation systems, SBAS), etc.
  • GPS global positioning system
  • GLONASS global navigation satellite system
  • Beidou navigation satellite system beidou navigation satellite system
  • BDS Beidou navigation satellite system
  • QZSS quasi-zenith satellite system
  • SBAS satellite based augmentation systems
  • the electronic device 100 may also include a wired communication module (not shown in FIG. 1 ), or the mobile communication module 150 or wireless communication module 160 here may be replaced by a wired communication module (not shown in FIG. output), the wired communication module can enable the electronic device to communicate with other devices through a wired network.
  • the wired network may include but not limited to one or more of the following: Optical Transport Network (OTN), Synchronous Digital Hierarchy (SDH), Passive Optical Network (PON), Ethernet network (Ethernet), or flexible Ethernet (flex Ethernet, FlexE), etc.
  • the electronic device 100 realizes the display function through the GPU, the display screen 194 , and the application processor.
  • the GPU is a microprocessor for interface display, and is connected to the display screen 194 and the application processor. GPUs are used to perform mathematical and geometric calculations for graphics rendering.
  • Processor 110 may include one or more GPUs that execute program instructions to generate or change display information.
  • the display screen 194 is used to display images, videos and the like.
  • the display screen 194 includes a display panel.
  • the display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active matrix organic light emitting diode or an active matrix organic light emitting diode (active-matrix organic light emitting diode, AMOLED), flexible light-emitting diode (flex light-emitting diode, FLED), Miniled, MicroLed, Micro-oLed, quantum dot light emitting diodes (quantum dot light emitting diodes, QLED), etc.
  • the electronic device 100 may include 1 or N display screens 194 , where N is a positive integer greater than 1.
  • the electronic device 100 can realize the shooting function through the ISP, the camera 193 , the video codec, the GPU, the display screen 194 and the application processor.
  • the ISP is used for processing the data fed back by the camera 193 .
  • the light is transmitted to the photosensitive element of the camera through the lens, and the light signal is converted into an electrical signal, and the photosensitive element of the camera transmits the electrical signal to the ISP for processing, and converts it into an image visible to the naked eye.
  • ISP can also perform algorithm optimization on image noise, brightness, and skin color.
  • ISP can also optimize the exposure, color temperature and other parameters of the shooting scene.
  • the ISP may be located in the camera 193 .
  • Camera 193 is used to capture still images or video.
  • the object generates an optical image through the lens and projects it to the photosensitive element.
  • the photosensitive element may be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor.
  • CMOS complementary metal-oxide-semiconductor
  • the photosensitive element converts the light signal into an electrical signal, and then transmits the electrical signal to the ISP to convert it into a digital image signal.
  • the ISP outputs the digital image signal to the DSP for processing.
  • DSP converts digital image signals into image signals in standard RGB camera, YUV and other formats.
  • the electronic device 100 may include 1 or N cameras 193 , where N is a positive integer greater than 1.
  • Digital signal processors are used to process digital signals. In addition to digital image signals, they can also process other digital signals. For example, when the electronic device 100 selects a frequency point, the digital signal processor is used to perform Fourier transform on the energy of the frequency point.
  • Video codecs are used to compress or decompress digital video.
  • the electronic device 100 may support one or more video codecs.
  • the electronic device 100 can play or record video in various encoding formats, for example: moving picture experts group (moving picture experts group, MPEG) 1, MPEG2, MPEG3, MPEG4, etc.
  • the NPU is a neural-network (NN) computing processor.
  • NN neural-network
  • Applications such as intelligent cognition of the electronic device 100 can be realized through the NPU, such as image recognition, face recognition, speech recognition, text understanding, and the like.
  • the external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, so as to expand the storage capacity of the electronic device 100.
  • the external memory card communicates with the processor 110 through the external memory interface 120 to implement a data storage function. Such as saving music, video and other files in the external memory card.
  • the internal memory 121 may be used to store computer-executable program codes including instructions.
  • the internal memory 121 may include an area for storing programs and an area for storing data.
  • the stored program area can store an operating system, at least one application program required by a function (such as a sound playing function, an image playing function, etc.) and the like.
  • the storage data area can store data created during the use of the electronic device 100 (such as audio data, phonebook, etc.) and the like.
  • the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one magnetic disk storage device, flash memory device, universal flash storage (universal flash storage, UFS) and the like.
  • the processor 110 executes various functional applications and data processing of the electronic device 100 by executing instructions stored in the internal memory 121 and/or instructions stored in a memory provided in the processor.
  • the electronic device 100 can implement audio functions through the audio module 170 , the speaker 170A, the receiver 170B, the microphone 170C, the earphone interface 170D, and the application processor. Such as music playback, recording, etc.
  • the audio module 170 is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signal.
  • the audio module 170 may also be used to encode and decode audio signals.
  • the audio module 170 may be set in the processor 110 , or some functional modules of the audio module 170 may be set in the processor 110 .
  • Speaker 170A also referred to as a "horn" is used to convert audio electrical signals into sound signals.
  • Electronic device 100 can listen to music through speaker 170A, or listen to hands-free calls.
  • Receiver 170B also called “earpiece” is used to convert audio electrical signals into sound signals.
  • the receiver 170B can be placed close to the human ear to receive the voice.
  • the microphone 170C also called “microphone” or “microphone” is used to convert sound signals into electrical signals. When making a phone call or sending a voice message, the user can put his mouth close to the microphone 170C to make a sound, and input the sound signal to the microphone 170C.
  • the electronic device 100 may be provided with at least one microphone 170C. In some other embodiments, the electronic device 100 may be provided with two microphones 170C, which may also implement a noise reduction function in addition to collecting sound signals. In some other embodiments, the electronic device 100 can also be provided with three, four or more microphones 170C to collect sound signals, reduce noise, identify sound sources, and realize directional recording functions, etc.
  • the earphone interface 170D is used for connecting wired earphones.
  • the earphone interface 170D can be a USB interface 130, or a 3.5mm open mobile terminal platform (OMTP) standard interface, or a cellular telecommunications industry association of the USA (CTIA) standard interface.
  • OMTP open mobile terminal platform
  • CTIA cellular telecommunications industry association of the USA
  • the pressure sensor 180A is used to sense the pressure signal and convert the pressure signal into an electrical signal.
  • the pressure sensor 180A may be located on the display screen 194 .
  • pressure sensors 180A such as resistive pressure sensors, inductive pressure sensors, and capacitive pressure sensors.
  • a capacitive pressure sensor may be comprised of at least two parallel plates with conductive material.
  • the electronic device 100 determines the intensity of pressure according to the change in capacitance.
  • the electronic device 100 detects the intensity of the touch operation according to the pressure sensor 180A.
  • the electronic device 100 may also calculate the touched position according to the detection signal of the pressure sensor 180A.
  • touch operations acting on the same touch position but with different touch operation intensities may correspond to different operation instructions. For example: when a touch operation with a touch operation intensity less than the first pressure threshold acts on the short message application icon, an instruction to view short messages is executed. When a touch operation whose intensity is greater than or equal to the first pressure threshold acts on the icon of the short message application, the instruction of creating a new short message is executed.
  • the gyro sensor 180B can be used to determine the motion posture of the electronic device 100 .
  • the angular velocity of the electronic device 100 around three axes may be determined by the gyro sensor 180B.
  • the gyro sensor 180B can be used for image stabilization. Exemplarily, when the shutter is pressed, the gyro sensor 180B detects the shaking angle of the electronic device 100, calculates the distance that the lens module needs to compensate according to the angle, and allows the lens to counteract the shaking of the electronic device 100 through reverse movement to achieve anti-shake.
  • the gyro sensor 180B can also be used for navigation and somatosensory game scenes.
  • the air pressure sensor 180C is used to measure air pressure.
  • the electronic device 100 calculates the altitude based on the air pressure value measured by the air pressure sensor 180C to assist positioning and navigation.
  • the magnetic sensor 180D includes a Hall sensor.
  • the electronic device 100 may use the magnetic sensor 180D to detect the opening and closing of the flip leather case.
  • the electronic device 100 when the electronic device 100 is a clamshell machine, the electronic device 100 can detect opening and closing of the clamshell according to the magnetic sensor 180D.
  • features such as automatic unlocking of the flip cover are set.
  • the acceleration sensor 180E can detect the acceleration of the electronic device 100 in various directions (generally three axes).
  • the magnitude and direction of gravity can be detected when the electronic device 100 is stationary. It can also be used to identify the posture of electronic devices, and can be used in applications such as horizontal and vertical screen switching, pedometers, etc.
  • the distance sensor 180F is used to measure the distance.
  • the electronic device 100 may measure the distance by infrared or laser. In some embodiments, when shooting a scene, the electronic device 100 may use the distance sensor 180F for distance measurement to achieve fast focusing.
  • Proximity light sensor 180G may include, for example, light emitting diodes (LEDs) and light detectors, such as photodiodes.
  • the light emitting diodes may be infrared light emitting diodes.
  • the electronic device 100 emits infrared light through the light emitting diode.
  • Electronic device 100 uses photodiodes to detect infrared reflected light from nearby objects. When sufficient reflected light is detected, it may be determined that there is an object near the electronic device 100 . When insufficient reflected light is detected, the electronic device 100 may determine that there is no object near the electronic device 100 .
  • the electronic device 100 can use the proximity light sensor 180G to detect that the user is holding the electronic device 100 close to the ear to make a call, so as to automatically turn off the screen to save power.
  • the proximity light sensor 180G can also be used in leather case mode, automatic unlock and lock screen in pocket mode.
  • the ambient light sensor 180L is used for sensing ambient light brightness.
  • the electronic device 100 can adaptively adjust the brightness of the display screen 194 according to the perceived ambient light brightness.
  • the ambient light sensor 180L can also be used to automatically adjust the white balance when taking pictures.
  • the ambient light sensor 180L can also cooperate with the proximity light sensor 180G to detect whether the electronic device 100 is in the pocket, so as to prevent accidental touch.
  • the fingerprint sensor 180H is used to collect fingerprints.
  • the electronic device 100 can use the collected fingerprint characteristics to implement fingerprint unlocking, access to application locks, take pictures with fingerprints, answer incoming calls with fingerprints, and the like.
  • the temperature sensor 180J is used to detect temperature.
  • the electronic device 100 uses the temperature detected by the temperature sensor 180J to implement a temperature treatment strategy. For example, when the temperature reported by the temperature sensor 180J exceeds the threshold, the electronic device 100 may reduce the performance of the processor located near the temperature sensor 180J, so as to reduce power consumption and implement thermal protection.
  • the electronic device 100 when the temperature is lower than another threshold, the electronic device 100 heats the battery 142 to avoid abnormal shutdown of the electronic device 100 caused by the low temperature.
  • the electronic device 100 boosts the output voltage of the battery 142 to avoid abnormal shutdown caused by low temperature.
  • the touch sensor 180K is also called “touch device”.
  • the touch sensor 180K can be disposed on the display screen 194, and the touch sensor 180K and the display screen 194 form a touch screen, also called a “touch screen”.
  • the touch sensor 180K is used to detect a touch operation on or near it.
  • the touch sensor can pass the detected touch operation to the application processor to determine the type of touch event.
  • Visual output related to the touch operation can be provided through the display screen 194 .
  • the touch sensor 180K may also be disposed on the surface of the electronic device 100 , which is different from the position of the display screen 194 .
  • the bone conduction sensor 180M can acquire vibration signals. In some embodiments, the bone conduction sensor 180M can acquire the vibration signal of the vibrating bone mass of the human voice. The bone conduction sensor 180M can also contact the human pulse and receive the blood pressure beating signal. In some embodiments, the bone conduction sensor 180M can also be disposed in the earphone, combined into a bone conduction earphone.
  • the audio module 170 can analyze the voice signal based on the vibration signal of the vibrating bone mass of the vocal part acquired by the bone conduction sensor 180M, so as to realize the voice function.
  • the application processor can analyze the heart rate information based on the blood pressure beating signal acquired by the bone conduction sensor 180M, so as to realize the heart rate detection function.
  • the motion sensor 180N can be used to detect moving objects within the range captured by the camera, and collect motion profiles or trajectory of the moving objects.
  • the motion sensor 180N can be an infrared sensor, a laser sensor, a dynamic vision sensor (dynamic vision sensor, DVS), etc.
  • the DVS can specifically include DAVIS (Dynamic and Active-pixel Vision Sensor), ATIS (Asynchronous Time-based Image Sensor ) or sensors such as CeleX sensors.
  • DAVIS Dynamic and Active-pixel Vision Sensor
  • ATIS Asynchronous Time-based Image Sensor
  • CeleX sensors CeleX sensors.
  • DVS draws lessons from the characteristics of biological vision, each pixel simulates a neuron, and independently responds to relative changes in light intensity (hereinafter referred to as "light intensity"). When the relative change of light intensity exceeds the threshold, the pixel will output an event signal, including the pixel's position, time stamp and feature information of light intensity.
  • the keys 190 include a power key, a volume key and the like.
  • the key 190 may be a mechanical key. It can also be a touch button.
  • the electronic device 100 may receive key input and generate key signal input related to user settings and function control of the electronic device 100 .
  • the motor 191 can generate a vibrating reminder.
  • the motor 191 can be used for incoming call vibration prompts, and can also be used for touch vibration feedback.
  • touch operations applied to different applications may correspond to different vibration feedback effects.
  • the motor 191 may also correspond to different vibration feedback effects for touch operations acting on different areas of the display screen 194 .
  • Different application scenarios for example: time reminder, receiving information, alarm clock, games, etc.
  • the touch vibration feedback effect can also support customization.
  • the indicator 192 can be an indicator light, and can be used to indicate charging status, power change, and can also be used to indicate messages, missed calls, notifications, and the like.
  • the SIM card interface 195 is used for connecting a SIM card.
  • the SIM card can be connected and separated from the electronic device 100 by inserting it into the SIM card interface 195 or pulling it out from the SIM card interface 195 .
  • the electronic device 100 may support 1 or N SIM card interfaces, where N is a positive integer greater than 1.
  • SIM card interface 195 can support Nano SIM card, Micro SIM card, SIM card etc. Multiple cards can be inserted into the same SIM card interface 195 at the same time. The types of the multiple cards may be the same or different.
  • the SIM card interface 195 is also compatible with different types of SIM cards.
  • the SIM card interface 195 is also compatible with external memory cards.
  • the electronic device 100 interacts with the network through the SIM card to implement functions such as calling and data communication.
  • the electronic device 100 adopts an eSIM, that is, an embedded SIM card.
  • the eSIM card can be embedded in the electronic device 100 and cannot be separated from the electronic device 100 .
  • the electronic device may include more or fewer components than those shown in FIG. 1 , which may be adjusted according to actual application scenarios, which is not limited in this application.
  • the electronic device 100 can be logically divided into a hardware layer 21 , an operating system 261 , and an application layer 31 .
  • the hardware layer 21 includes hardware resources such as an application program processor 201 , a microcontroller unit 203 , a modem 207 , a Wi-Fi module 211 , a sensor 214 , and a positioning module 150 .
  • the application layer 31 includes one or more applications (applications, APPs), such as the application program 263, and the application program 263 may be any type of application program such as a social application program, an e-commerce application program, or a browser.
  • the operating system 261 is a software middleware between the hardware layer 21 and the application layer 31, and is a computer program for managing and controlling hardware and software resources.
  • the operating system 261 includes a kernel 23, a hardware abstraction layer (hardware abstraction layer, HAL) 25, libraries and runtime (libraries and runtime) 27, and a framework (framework) 29.
  • the kernel 23 is used to provide underlying system components and services, such as: power management, memory management, thread management, hardware drivers, etc.; hardware drivers include Wi-Fi drivers, sensor drivers, positioning module drivers, etc.
  • the hardware abstraction layer 25 encapsulates the kernel driver, provides an interface to the framework 29, and shields low-level implementation details.
  • the hardware abstraction layer 25 runs in user space, while the kernel driver runs in kernel space.
  • Library and runtime 27 is also called runtime library, which provides required library files and execution environment for executable programs at runtime.
  • the library and runtime 27 includes an Android runtime (Android Runtime, ART) 271 and a library 273, etc.
  • ART 271 is a virtual machine or virtual machine instance capable of converting an application's bytecode into machine code.
  • the library 273 is a program library that provides support for executable programs at runtime, including browser engines (such as webkit), script execution engines (such as JavaScript engines), graphics processing engines, and the like.
  • the framework 27 is used to provide various basic public components and services for applications in the application layer 31, such as window management, location management and so on.
  • the framework 27 may include a phone manager 291, a resource manager 293, a location manager 295, and the like.
  • each component of the operating system 261 described above can be realized by the application program processor 201 executing the programs stored in the memory 205 .
  • the electronic device 100 may include fewer or more components than those shown in FIG. 2.
  • the aforementioned electronic devices have the ability to store images and encode image data.
  • the images can be divided into multiple slices (Grid), and then each slice can be encoded, so that Improve image compression efficiency and improve encoding performance.
  • slicing methods can be divided into two modes: FrameWork layer slicing and slice sinking.
  • the original image is: 4032*3024 high efficiency image file format (high efficiency image file format, HEIF) coded picture (usually called Grid): 512*512;
  • HEIF high efficiency image file format
  • Grid coded picture
  • 512*512 the actual input to the encoder is the storage memory of 4032*3024, and it is divided into 512*512 Grid, that is, it is necessary to allocate memory for the image, copy the image data to the storage memory of 4032*3024, and then encode
  • the device reads data from this part of the memory for encoding, requiring a large number of memory copies, which affects performance and power consumption.
  • a common solution is to use specific instructions (such as NEON instructions) for data parallel processing to speed up the copy process. Or use line-by-line and multi-line copying instead of pixel-by-pixel copying to speed up the copying process.
  • HEVC high efficiency video coding
  • the present application provides an image encoding method.
  • the address of the storage memory of the input image is directly mapped to the encoder, so that the encoder can directly read and encode data based on the mapped address without performing Memory copy, without allocating additional memory to carry slice data, reduces the performance and power consumption overhead caused by memory copy, and improves coding efficiency.
  • FIG. 3 it is a schematic flowchart of an image encoding method provided by the present application, as described below.
  • the input image may be a received image, or may be an image captured by a local camera, and the input image may be stored in a memory for subsequent encoding.
  • the information of the storage memory of the input image may include information such as an address or a length of the storage memory.
  • a continuous memory can be allocated for the input image, and the information of each pixel in the input image can be stored in the memory.
  • the input image can be stored in the memory in the form of an array.
  • a continuous memory is allocated for the input image, and the information of the input image is stored in the allocated memory according to the arrangement of pixels.
  • the input image can be stored in the form of an array in the memory, and information such as the size and address of the storage memory can be recorded.
  • the allocated memory can be used to save the information of each pixel of the input image, such as chrominance value, brightness value and other information.
  • additional memory can be allocated for the input image.
  • a continuous storage memory can be allocated for the input image based on the preset slice size, and the storage memory can be divided into two parts, which are called the first memory and the second memory for easy distinction.
  • the memory is used to save the information of each pixel in the input image, and the second memory is used to align the storage memory according to the preset slice size, so that the subsequent encoder can read data from the input address in units of the preset slice size .
  • the storage memory can be allocated for the input image based on the preset slice size, wherein the storage memory can be understood as two parts, which are called the first memory and the second memory for easy distinction,
  • the first memory is used to save the data of the input image, such as the brightness value, gray value or chrominance value of each pixel in the input image
  • the second memory is used to make the size of the memory allocated for the input image according to the preset
  • the slice size is aligned so that subsequent slices can be sliced according to the preset slice size, so that the encoder can completely read the memory address of each memory slice, and encode in units of each memory slice.
  • the preset slice size when allocating memory for the input image, it can be allocated according to a preset slice size, the preset slice size includes a preset height and a preset width, and the height of the memory allocated for the input image is aligned according to the preset height, The width of the storage memory allocated for the input image is aligned according to the preset width, so that when the subsequent memory slice is performed, the memory slice with the size of the preset slice size can be directly sliced, so that the subsequent encoder can read the memory address avoid crossing the line.
  • the height of the allocated storage memory is aligned according to the preset height, and the width of the allocated storage memory is allocated according to the actual required memory of the input image, that is, the second memory can be divided into multiple
  • the part is called the third memory and the fourth memory for easy distinction.
  • the third memory is combined with the first memory and arranged in an array, that is, the height of the storage memory is aligned with the preset height through the third memory.
  • the last row of the array except Combining the remainder after the preset width with the fourth memory is not less than the preset width. That is, in order to reduce the amount of allocated memory, the width of the array may not be aligned according to the preset width, and an extra section of memory is allocated in the last line of the array to avoid subsequent encoder reading out of bounds.
  • one of the foregoing slicing manners may be pre-deployed in the electronic device, and multiple foregoing slicing manners may also be pre-deployed.
  • one of the slicing methods can be selected from the various slicing methods according to the preset method for slicing, which can be performed according to the actual application scenario. Adjustment.
  • the height of the storage memory can be aligned and allocated only according to the preset height, thereby forming an array, And additionally allocate multiple bytes for the last row of the array, so that the remainder of the last row of the array divided by the preset width is not smaller than the preset width after combining with the additional allocated multiple bytes, so as to avoid the encoder from reading Out of bounds;
  • the available memory of the electronic device is greater than the second threshold but not greater than the third threshold, it can be aligned according to the preset slice size, that is, the height of the storage memory is divisible by the preset height, and the width of the storage memory is divisible by the preset width , so that the follow-up can be sliced according to the preset slice size, so that the encoder performs encoding in units of memory slices without memory copying;
  • the available memory of the electronic device is greater than the third threshold, it means
  • the input image may be segmented to obtain multiple memory slices. It is equivalent to segmenting the storage memory of the input image, so that the storage memory is divided into multiple grids, that is, memory slices.
  • the storage memory may be divided according to a preset size, ie, a preset slice size, to obtain multiple memory slices (Grid). That is, it is equivalent to segmenting the input image, dividing the input image into multiple parts, so that the subsequent encoder can encode the input image.
  • a preset size ie, a preset slice size
  • the preset slice size may include a size preset by the user, a size negotiated between the upper layer and the lower layer of the electronic device, a size set according to the encoding length of the encoder, or a random size, etc., which may be determined according to actual application scenarios.
  • each memory slice After memory slicing is performed, the address of each memory slice is input to an encoder for encoding, thereby obtaining encoded data.
  • the data to be encoded can be extracted from each memory slice based on the input address, thereby encoding the data in each memory slice to obtain encoded data.
  • the encoder may be a pre-deployed encoder in the electronic device, and may perform encoding according to a preset encoding rule. For example, a piece of code can be deployed and run in an electronic device to read data from the address input to the encoder, and encode the image according to a preset encoding method, which can be HEVC encoding or advanced video encoding (advanced video coding, AVC) and other encoding methods, etc., can be adjusted according to actual application scenarios, and this application does not limit the specific encoding methods.
  • Grid is the encoding unit of the encoder, and the encoder encodes the data stored in the memory of a Grid as a unit, and can map the memory address corresponding to each Grid to the encoder, so that the encoder can obtain the memory of the Grid Information, including width, height, Stride, etc.
  • the address of the data to be encoded can be directly input to the encoder, so that the encoder can directly extract the data to be encoded based on the address, thus reducing the memory overhead for memory copying and improving the encoding efficiency. efficiency.
  • the storage memory is not divisible by the preset slice size, then the storage memory is divided to obtain multiple memory slices and the first slice, the first slice is the storage memory according to the preset The slice size is the remainder after slicing. Subsequently, the addresses of multiple memory slices may be mapped to the encoder for encoding, and the first data may be copied from the address corresponding to the first slice, and the first data may be used as an input of the encoder to obtain encoded data.
  • the address of the obtained memory matching the preset slice size can be mapped to the encoder for encoding, and the remaining Partially copy the data and use it as the input of the encoder to obtain the final encoded data without copying all the memory in the storage memory, reducing the amount of memory copying, reducing the memory overhead of memory copying, and improving encoding efficiency.
  • the storage memory is not completely aligned according to the preset slice size when allocating memory for the input image, for example, only the height is aligned but the width is not aligned, or the height and width are not aligned, etc.
  • the storage memory can be divided according to the preset slice size. Then the obtained memory slices can be divided into two types, which are called first memory slices and second memory slices for easy distinction.
  • the height of the first memory slice is the preset height, and the height of the second memory slice is smaller than the preset height, or
  • the second memory slice is the last slice in the last row, and the first memory slice and the second memory slice are different memory slices, that is, the type of the same memory is either the first memory slice or the second memory slice.
  • the fifth memory can be allocated for the second memory slice, and the data in the second memory can be copied to the fifth memory.
  • the width of the fifth memory is divisible by the preset width, and the height of the fifth memory is divisible by the preset height.
  • the data stored in the second memory slice is copied to the fifth memory.
  • the memory slice of the last row and last column can be used as the second memory slice, and the remaining memory slices can be divided into the first memory slice .
  • a fifth memory is allocated for the second memory slice, the size of the fifth memory may be a preset slice size, and the data stored in the second memory slice is copied to the fifth memory. Then, the addresses of the first memory slice and the fifth memory are input to the encoder, so that the encoder reads and encodes data from the input addresses to obtain final encoded data.
  • the part whose height is not divisible by the preset height that is, the part corresponding to the last row in the storage memory
  • the memory slice acts as the second memory slice. And allocate a fifth memory for the second memory slice, and copy the data stored in the second memory slice to the fifth memory. Then, the addresses of the first memory slice and the fifth memory are input to the encoder, so that the encoder reads and encodes data from the input addresses to obtain the final encoded data, avoiding the situation that the encoder reads out of bounds.
  • the system architecture can be based on the MediaCodec/OMX framework and the HEVC encoder to implement HEIC encoding, and can include multiple layers, such as the application layer, FrameWork layer and HAL/vendor layer, etc.
  • APP can be installed in the application layer, and can be implemented by the APP. Trigger the encoder to encode.
  • some APPs that can obtain input images can be installed in the electronic device, such as cameras or communication software that can generate or receive images.
  • the camera can be used to capture images and trigger the step of saving images; Images sent by other electronic devices, and trigger steps to save images, etc.
  • HEIF encoding In Android FrameWork, HEIFWriter and HEIFEncoder have been implemented. HEIFWriter is used to provide an interface for the application layer (APK) for the application layer to input original image data. HEIFWriter can call the interface of HEIFEncoder. HEIFWriter is used to complete the file packaging and generate . HEIC file; HEIFEncoder can call MediaCodec interface to create HEIC encoder and start HEIF encoding.
  • API application layer
  • HEIFWriter can call the interface of HEIFEncoder.
  • HEIFWriter is used to complete the file packaging and generate .
  • HEIC file; HEIFEncoder can call MediaCodec interface to create HEIC encoder and start HEIF encoding.
  • MediaCodec is a class for encoding and decoding audio and video, and realizes the function of encoding and decoding by accessing the underlying codec. It is part of the media infrastructure of electronic devices.
  • OMX OpenMAX (Open Media Acceleration)
  • OpenMAX Open Media Acceleration
  • the HEIC encoder as the OMX IL (OpenMAX Integration Layer, interface between the OMX framework layer and the component layer) layer, completes the docking with the OMX framework.
  • OMX IL OpenMAX Integration Layer, interface between the OMX framework layer and the component layer
  • its core function is to complete HEIF slicing.
  • Gralloc Graphic Buffer Allocator
  • Android Graphic Buffer Allocator
  • HEVC encoder Its input is the memory information of each Grid obtained after slicing by the HEIC encoder, such as the width of the grid corresponding to the picture, the height of the grid corresponding to the picture, stride and other information, used to read data from the input Grid memory information, And perform HEVC encoding on the data in each Grid respectively.
  • the process of encoding an image may be as shown in FIG. 6 .
  • the usage is set by the HEIC encoder, and the usage is an identifier for identifying whether the encoding type of the input image is HEIC encoding.
  • the HEIC encoder sets specific attributes for the Input Buffer, such as marking whether to slice and encode according to the method provided by this application through the graphic buffer usage. It can be understood that, in the process of negotiating Buffer attributes between the HEIC encoder module and OMX, this application adds a specific graphic buffer usage, which is used to identify the slice pre-alignment attribute of the HEIC encoding slice scene, that is, which slice method is used for alignment And slice. Which method is selected from the slicing methods in step 603 below for slicing.
  • OMX can read the graphic buffer usage, and after identifying the type of usage, it can know that the current HEIC encoding scene is. At this point, you can set a new usage, which also needs to be globally unique, to identify the HEIC encoding scene, as an attribute of the Input Buffer, and can be recognized by Gralloc, or directly pass the graphic buffer usage to Gralloc, so that Gralloc recognizes the current HEIC Code the scene.
  • Gralloc allocates memory based on usage
  • one or more slicing methods can be pre-deployed. After Gralloc gets the usage, it can recognize the current HEIC encoding scene and allocate corresponding memory.
  • Gralloc can recognize the current HEIC encoding scene after getting the usage, and then perform memory allocation.
  • Gralloc gets the usage, it can identify which method to use for slicing based on the usage, and then allocate the corresponding memory.
  • the memory allocated by each slicing method may be the same or may not be the same, and the matching slicing method can be selected according to the actual application scenario, for example, when the available memory of the electronic device is sufficient, it can Randomly select one of the slicing methods.
  • the available memory of the electronic device is insufficient, you can choose the slicing method that consumes the least memory, or when the coding efficiency is high, you can choose the slicing method that consumes less memory.
  • Adjustments may be made according to actual application scenarios, and this application is only an illustration and not a limitation.
  • Gralloc allocates memory based on usage
  • the information of the allocated memory can be passed to the HEIC encoder, and the HEIC encoder will slice it, and transmit the information of each slice obtained after slicing to the HEVC encoder, so that HEVC An encoder reads data from a slice and encodes it.
  • the Android system can be sliced by the FrameWork layer (or sliced by the application layer) or sliced by the encoder (usually called slice sinking).
  • the HEVC encoder is created by the upper layer: the upper layer can negotiate the slice size with the lower layer through Mediacodec/OMX, or preset the slice size, and then the upper layer slices based on the slice size, and the address of each slice is passed through Mediacodec /OMX is passed to the encoder, and the encoder extracts and encodes the data to be encoded based on the incoming address.
  • the encoded data can be carried by the Output Buffer and passed to the upper layer.
  • the upper layer generates and feeds back the end of stream (EOS), which indicates that the encoding is completed. After the upper layer receives the encoded data of all slices of the image, it can be encapsulated into a HEIC file.
  • EOS end of stream
  • the HEIC encoder is created by the upper layer.
  • the upper layer can negotiate the slice size with the lower layer through Mediacodec/OMX, or pre-set the slice size, and then the upper layer transmits the image to the lower layer, and the encoder slices the image and completes the encoding. , after the encoding of the last slice is completed, EOS is reported, and after the upper layer receives the encoded data of all slices of the image, it can be encapsulated into a HEIC file.
  • the upper layer and the lower layer are relative concepts.
  • the layer above the FrameWork layer can be understood as the upper layer
  • the layer below the FrameWork layer can be called the lower layer.
  • it can be divided according to the actual application scenario. Not as limiting.
  • the input image can be divided into multiple parts of the same size, that is, the storage memory corresponding to the input image can be divided into multiple grids of the same size, and then related parameters can be configured as the input of the HEVC encoder.
  • the width of the allocated memory is called stride
  • the height of the allocated memory is called Height
  • the width of each Grid is called Grid width
  • the height of each Grid is called Grid height
  • the Grid width*Grid height can be understood as the aforementioned preset slice size.
  • the input image can be allocated memory and sliced in various ways, which will be introduced as examples below.
  • the stride of the memory can be aligned according to the Grid width, which can be understood as the stride of the memory can be divisible by the Grid width, and the height of the memory can be aligned according to the Grid height.
  • the size of the memory occupied by the data in the input image may not be divisible by the preset slice size, and the size of each Grid usually needs to be consistent.
  • the size of the memory occupied by the data in the input image is not divisible by the size of the Grid, then when allocating memory for the Input Buffer, the stride of the allocated memory is aligned according to the Grid width, which can be the last A column of Grid and the last row of Grid allocate memory, and the memory information of each Grid (including the storage memory address allocated for the data included in the input image and the address of additional allocated memory) is passed to the encoder, so that the encoder can read When fetching memory, you can read data from the memory information corresponding to each Grid, so as to avoid reading illegal memory and causing the program to crash or the encoder to fail to work properly. And without memory copying, the encoder can directly read the input image from the input address, which reduces the process of memory copying, reduces memory overhead, and improves coding efficiency.
  • the encoder can encode in the unit of Grid, and the storage memory of the input image is usually a continuous memory, so the encoder can read the data of each Grid according to the starting address of each Grid, based on memory mapping
  • the encoder can read the data of each Grid according to the starting address of each Grid, based on memory mapping
  • N bytes can be allocated for the last Grid, that is, the width of the last line of the last Grid is Grid width, so that the encoder can read the data of a complete grid, thereby avoiding reading illegal memory , so that the encoder generates encoding errors or affects the encoding process of the encoder, etc.
  • the Height of the allocated memory is aligned according to the Grid height, and the Stride does not perform Grid width alignment; on this basis, more N bytes of memory are allocated.
  • N N ⁇ ((Grid width)–(picture width%Grid width))*4
  • the obtained N value is the minimum value to ensure that the encoder does not cross the boundary when reading the memory, so as to avoid reading Access to illegal memory, so that the encoder produces encoding errors or affects the encoding process of the encoder, etc.
  • the Input Buffer allocates memory
  • the Height is aligned according to the size of the Grid
  • the Stride is valued according to the width of the storage memory (or conventionally, aligned according to 16 bytes, 32 bytes, or 64 bytes, and the specific number is determined by the system itself. )
  • the Grid in the lower right corner of Figure 9 is mapped to the encoder, therefore, more N bytes of memory can be allocated to avoid reading illegal memory, which will cause encoding errors in the encoder or affect encoding The encoding process of the device, etc.
  • Method 3 Some parts use memory address mapping, and the rest use memory copy
  • the address of the part of the storage memory of the input image that can be divisible by the size of the Grid can be mapped to the encoder, and the remaining part can be input to the encoder by means of memory copy.
  • the memory occupied by the data of the input image may be divided into many situations, for example, the height is divisible by the height of the Grid, or the height is not divisible by the height of the Grid.
  • the storage memory can be allocated according to the actual memory required by the input image, and the storage memory can be divided. It is the first memory slice and the second memory slice, as shown in Figure 10, the second memory slice is the last memory slice in the last line, its width is not divisible by the Grid width, and the rest are the first memory slice.
  • the memory copy of the last slice of the last line can be performed, that is, additional memory is allocated for the last memory slice (ie, the fifth memory slice) ), and then copy the data in the last slice to the additionally allocated memory. Then both the address of the first memory slice and the address of the fifth memory are used as the input of the encoder.
  • the storage memory of the input image can be divided according to the preset Grid size, and the memory of the divisible part can be transferred to the encoder through memory address mapping, while the rest of the non-divisible part can be allocated to it.
  • the memory height required by the input image data is not divisible by the grid height
  • no additional memory may be allocated for the input image
  • the storage memory may be allocated according to the actual size of the input image.
  • the storage memory is directly divided according to the size of the Grid.
  • the obtained memory slices can be divided into the first memory slice and the second memory slice.
  • the first memory slice is the slice whose height is the Grid height
  • the second memory slice is the slice whose height is smaller than the Grid height slice.
  • allocate additional memory that is, the fifth memory
  • copy the data in the second memory slice to the additionally allocated memory.
  • both the address of the first memory slice and the address of the fifth memory are used as the input of the encoder.
  • the divisible part of the storage memory can be transmitted to the encoder through address mapping, and it is only necessary to allocate memory for the part that is not divisible for memory copying, which reduces the amount of memory copying and Memory overhead, improving coding efficiency.
  • the HEVC encoder After the HEVC encoder receives the configuration information of each Grid, such as the starting address and length of each Grid, it can perform HEVC encoding.
  • the configuration information may include information such as the start address, width or length of each Grid, so that the encoder can read and encode data in each Grid.
  • the received configuration information is as follows:
  • Stride stride of Input Buffer
  • Width Grid width
  • Stride represents the width of the memory
  • Width represents the width of each Grid
  • Height represents the height of each Grid.
  • the encoding is performed by means of memory copy and memory address mapping provided by this application, and the comparison is made by calculating the encoding time.
  • HEIC encoding time the moment when HEIC completes encoding (all grids complete HEVC encoding and report EOS) - the moment when HEIC starts encoding (the moment when the HEIC encoder receives the original image).
  • the memory copy method takes more than 80ms, while the encoding method provided by this application only needs 45ms at least, which greatly improves the encoding efficiency.
  • the above is based on a certain chip test, and the time-consuming data of different hardware may be different.
  • the storage memory of the image can be segmented, and the grid address obtained by the segmentation can be mapped to the encoder, so that additional allocated memory can be reduced, memory overhead can be reduced, and encoding efficiency can be improved. Furthermore, when encoding, in order to prevent the encoder from reading illegal memory addresses, a certain amount of memory can be allocated in various ways, which can prevent the encoder from reading illegal memory, thereby causing encoding errors or affecting the encoder encoding process, etc.
  • FIG. 13 it is a schematic structural diagram of an image coding device provided by the present application, and the image coding device is used to execute the method steps corresponding to the aforementioned FIGS. 3-12 .
  • the image encoding device includes:
  • An allocation module 1301, configured to allocate storage memory for the input image
  • the encoding module 1303 is configured to input the address of at least one memory slice in the plurality of memory slices to the encoder, and the encoder is configured to read and encode data based on the input address to obtain encoded data.
  • the division module 1302 is specifically configured to divide the storage memory into multiple memory slices according to a preset slice size, where the preset slice size includes a preset height and a preset width.
  • the division module 1302 is specifically configured to: allocate storage memory for the input image based on a preset slice size.
  • the storage memory includes a first memory and a second memory, the first memory stores the information of each pixel in the input image, and the second memory is used to make the size of the storage memory according to the preset Slice size alignment so that the encoder reads data from the input address in units of the preset slice size.
  • the storage memory is arranged in an array, the width of the storage memory is divisible by a preset width, and the height of the storage memory is divisible by a preset height.
  • the second memory includes a third memory and a fourth memory
  • the first memory and the third memory are combined and arranged in an array
  • the height of the array is divisible by the preset height
  • the last row of the array is divided by the preset height.
  • the remainder after setting the width is not smaller than the preset width after being combined with the fourth memory.
  • multiple memory slices are divided into a first memory slice and a second memory slice, the height of the first memory slice is a preset height, and the height of the second memory slice is smaller than the preset height or the second memory slice It is the last slice of the last line in the array composed of multiple memory slices, the first memory slice and the second memory slice are different memory slices;
  • the allocation module 1301 is further configured to allocate a fifth memory for the second memory slice among the plurality of memory slices, and copy the data in the second memory to the fifth memory.
  • the width of the fifth memory is divisible by the preset width.
  • the height of the five memory is divisible by the preset height;
  • the encoding module 1303 is specifically configured to input the address of the first memory slice and the address of the fifth memory in the plurality of memory slices to the encoder.
  • the height of the storage memory is evenly divisible by the preset height.
  • FIG. 14 is a schematic structural diagram of another electronic device provided by the present application, as described below.
  • the electronic device may include the aforementioned wearable device, terminal, or vehicle, etc., and the electronic device may include a processor 1401 , a memory 1402 , and a transceiver 1403 .
  • the processor 1401 and the memory 1402 are interconnected by wires. Wherein, program instructions and data are stored in the memory 1402 .
  • the memory 1402 stores program instructions and data corresponding to the aforementioned steps in FIGS. 3-12 .
  • the processor 1401 is configured to execute the method steps performed by the first device or the electronic device shown in any one of the foregoing embodiments in FIG. 3 to FIG. 12 .
  • the transceiver 1403 is configured to execute the step of receiving or sending data performed by the first device or the electronic device shown in any one of the embodiments in FIG. 3 to FIG. 12 .
  • the embodiment of the present application also provides a computer-readable storage medium, the computer-readable storage medium is stored with a program for generating the driving speed of the vehicle.
  • the illustrated embodiments describe steps in methods.
  • the aforementioned electronic device shown in FIG. 14 is a chip.
  • the embodiment of the present application also provides an electronic device, which can also be called a digital processing chip or a chip.
  • the chip includes a processing unit and a communication interface.
  • the processing unit obtains program instructions through the communication interface.
  • the program instructions are executed by the processing unit.
  • the unit is configured to execute the method steps executed by the electronic device shown in any one of the foregoing embodiments in FIG. 3 to FIG. 12 .
  • the embodiment of the present application also provides a digital processing chip.
  • the digital processing chip integrates circuits and one or more interfaces for realizing the above processor 1401 or the functions of the processor 1401 .
  • the digital processing chip can complete the method steps in any one or more of the foregoing embodiments.
  • no memory is integrated in the digital processing chip, it can be connected to an external memory through a communication interface.
  • the digital processing chip implements the actions performed by the electronic device in the above-mentioned embodiments according to the program code stored in the external memory.
  • the embodiment of the present application also provides a computer program product that, when running on a computer, causes the computer to execute the steps performed by the electronic device in the method described in the foregoing embodiments shown in FIGS. 3-12 .
  • the electronic device provided in the embodiment of the present application may be a chip, and the chip includes: a processing unit and a communication unit, the processing unit may be, for example, a processor, and the communication unit may be, for example, an input/output interface, a pin, or a circuit.
  • the processing unit can execute the computer-executed instructions stored in the storage unit, so that the chip in the server executes the device search method described in the embodiments shown in FIGS. 3-12 above.
  • the storage unit is a storage unit in the chip, such as a register, a cache, etc.
  • the storage unit may also be a storage unit located outside the chip in the wireless access device, such as only Read-only memory (ROM) or other types of static storage devices that can store static information and instructions, random access memory (random access memory, RAM), etc.
  • ROM Read-only memory
  • RAM random access memory
  • the aforementioned processing unit or processor may be a central processing unit (central processing unit, CPU), a network processor (neural-network processing unit, NPU), a graphics processing unit (graphics processing unit, GPU), digital signal processing (digital signal processor, DSP), application specific integrated circuit (ASIC) or field programmable logic gate array (field programmable gate array, FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.
  • a general-purpose processor may be a microprocessor or any conventional processor or the like.
  • the processor mentioned in any of the above-mentioned places may be a general-purpose central processing unit, a microprocessor, an ASIC, or one or more integrated circuits used to control the program execution of the above-mentioned methods in FIGS. 3-12 .
  • the device embodiments described above are only illustrative, and the units described as separate components may or may not be physically separated, and the components shown as units may or may not be A physical unit can be located in one place, or it can be distributed to multiple network units. Part or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment.
  • the connection relationship between the modules indicates that they have communication connections, which can be specifically implemented as one or more communication buses or signal lines.
  • the essence of the technical solution of this application or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product is stored in a readable storage medium, such as a floppy disk of a computer , U disk, mobile hard disk, read only memory (read only memory, ROM), random access memory (random access memory, RAM), magnetic disk or optical disk, etc., including several instructions to make a computer device (which can be a personal A computer, server, or network device, etc.) executes the methods described in various embodiments of the present application.
  • a computer device which can be a personal A computer, server, or network device, etc.
  • all or part of them may be implemented by software, hardware, firmware or any combination thereof.
  • software When implemented using software, it may be implemented in whole or in part in the form of a computer program product.
  • the computer program product includes one or more computer instructions.
  • the computer can be a general purpose computer, a special purpose computer, a computer network, or other programmable devices.
  • the computer instructions may be stored in or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from a website, computer, server, or data center Transmission to another website site, computer, server, or data center by wired (eg, coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (eg, infrared, wireless, microwave, etc.).
  • wired eg, coaxial cable, optical fiber, digital subscriber line (DSL)
  • wireless eg, infrared, wireless, microwave, etc.
  • the computer-readable storage medium may be any available medium that can be stored by a computer, or a data storage device such as a server or a data center integrated with one or more available media.
  • the available medium may be a magnetic medium (such as a floppy disk, a hard disk, or a magnetic tape), an optical medium (such as a DVD), or a semiconductor medium (such as a solid state disk (solid state disk, SSD)), etc.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)

Abstract

The present application relates to an image encoding method and a device, which are used for encoding a stored image by an internal memory mapping means, reducing internal memory copying, and improving encoding efficiency. The method comprises: first allocating storage internal memory for an input image; dividing the storage internal memory into a plurality of internal memory grid sections; then reading data from the address of at least one internal memory grid section among the plurality of internal memory grid sections and performing encoding by means of an encoder, and obtaining encoded data.

Description

一种图像编码方法以及装置An image encoding method and device
本申请要求于2021年12月29日提交中国专利局、申请号为“202111640001.9”、申请名称为“一种图像编码方法以及装置”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。This application claims the priority of the Chinese patent application with the application number "202111640001.9" and the application title "An Image Coding Method and Device" filed with the China Patent Office on December 29, 2021, the entire contents of which are incorporated herein by reference. Applying.
技术领域technical field
本申请涉及人工智能领域，尤其涉及一种图像编码方法以及装置。The present application relates to the field of artificial intelligence, and in particular to an image encoding method and device.
背景技术Background technique
在一些与图像相关的场景中，如拍照、图像互传等场景中，通常需要对图像进行编码以进行保存。而通常，在编码的过程中，将图像切分为多个切片(Grid)，然后对每个切片进行编码，从而可以提高图像的压缩效率，提高编码性能，因此图像切片的过程尤其重要In some image-related scenarios, such as taking pictures and transferring images to each other, it is usually necessary to encode the image for preservation. Usually, in the process of encoding, the image is divided into multiple slices (Grid), and then each slice is encoded, which can improve the compression efficiency of the image and improve the encoding performance, so the process of image slicing is particularly important
然而，在对图像进行切分时，存在大量的内存拷贝，将带来额外的开销，并且将消耗额外的内存来承载切片数据，切片过程开销大。因此，如何降低切片过程的开销，成为亟待解决的问题。However, when the image is segmented, there is a large amount of memory copy, which will bring additional overhead, and will consume additional memory to carry the slice data, and the slice process has a large overhead. Therefore, how to reduce the overhead of the slicing process has become an urgent problem to be solved.
发明内容Contents of the invention
本申请提供一种图像编码方法以及装置，用于通过内存映射的方式对存储的图像进行编码，减少内存拷贝，提高编码效率。The present application provides an image coding method and device, which are used to code stored images by means of memory mapping, reduce memory copying, and improve coding efficiency.
有鉴于此，第一方面，本申请提供一种图像编码方法，包括：首先，为输入图像分配存储内存；将存储内存划分为多个内存切片；随后通过编码器从所述多个内存切片中至少一个内存切片的地址读取数据并进行编码，得到编码数据。In view of this, in the first aspect, the present application provides an image encoding method, comprising: first, allocating a storage memory for an input image; dividing the storage memory into a plurality of memory slices; The address of at least one memory slice reads data and performs encoding to obtain encoded data.
因此，本申请实施方式中，可以直接将需要编码的数据的地址映射至编码器，使编码器可以直接基于地址提取到需要编码的数据，因此无需进行内存拷贝，可以减少进行内存拷贝的内存开销，且可以提高编码效率。Therefore, in the embodiment of the present application, the address of the data to be encoded can be directly mapped to the encoder, so that the encoder can directly extract the data to be encoded based on the address, so memory copying is not required, and the memory overhead of memory copying can be reduced , and can improve the coding efficiency.
在一种可能的实施方式中，前述的将存储内存划分为多个内存切片存储内存，可以包括：按照预设切片大小，将存储内存划分为多个内存切片，预设切片大小包括预设高度和预设宽度。In a possible implementation manner, the aforementioned dividing the storage memory into multiple memory slices may include: dividing the storage memory into multiple memory slices according to a preset slice size, where the preset slice size includes a preset height and preset width.
因此，本申请实施方式中，在划分存储内存时，可以按照预先设定的切片大小来进行划分，以使后续编码器可以以预设切片大小为单位进行编码。Therefore, in the embodiment of the present application, when dividing the storage memory, the division may be performed according to a preset slice size, so that subsequent encoders may perform encoding in units of the preset slice size.
在一种可能的实施方式中，前述的为输入图像分配存储内存，可以包括：基于预设切片大小为输入图像分配存储内存。In a possible implementation manner, the aforementioned allocating storage memory for the input image may include: allocating storage memory for the input image based on a preset slice size.
在一种可能的实施方式中，存储内存可以包括第一内存和第二内存，第一内存中保存了输入图像中的各个像素点的信息，第二内存用于使存储内存的尺寸按照预设切片大小对齐，以使编码器以预设切片大小为单位从输入的地址中读取数据。In a possible implementation manner, the storage memory may include a first memory and a second memory, the first memory stores the information of each pixel in the input image, and the second memory is used to make the size of the storage memory according to the preset Slice size alignment so that the encoder reads data from the input address in units of the preset slice size.
因此，本申请实施方式中，当输入图像的存储内存不被预设切片大小整除时，可以将得到的内存切片的地址映射至编码器进行编码，得到最终的编码数据，减少了内存拷贝的量，可以减少进行内存拷贝的内存开销，且可以提高编码效率。Therefore, in the embodiment of the present application, when the storage memory of the input image is not divisible by the preset slice size, the address of the obtained memory slice can be mapped to the encoder for encoding to obtain the final encoded data, reducing the amount of memory copy , which can reduce memory overhead for memory copying and improve coding efficiency.
在一种可能的实施方式中，存储内存按照阵列排列，存储内存的宽度被预设宽度整除，存储内存的高度被预设高度整除。In a possible implementation manner, the storage memory is arranged in an array, the width of the storage memory is divisible by a preset width, and the height of the storage memory is divisible by a preset height.
因此，本申请实施方式中，在为输入图像分配内存时，可以按照预设宽度和预设高度来进行对齐，从而在划分内存切片时，使每个内存切片的尺寸都为预设切片大小，避免编码器读取地址时越界，且仅需分配少量内存即可，减少了内存拷贝的量，可以减少进行内存拷贝的内存开销，且可以提高编码效率。Therefore, in the embodiment of the present application, when allocating memory for the input image, it can be aligned according to the preset width and preset height, so that when dividing the memory slice, the size of each memory slice is the preset slice size, Avoid the coder reading the address out of bounds, and only need to allocate a small amount of memory, which reduces the amount of memory copying, reduces the memory overhead of memory copying, and improves coding efficiency.
在一种可能的实施方式中，第二内存包括第三内存和第四内存，第一内存和第三内存组合后按照阵列排列，阵列的高度被预设高度整除，阵列的最后一行除以预设宽度后的余数与第四内存组合后不小于预设宽度。In a possible implementation manner, the second memory includes a third memory and a fourth memory, and the first memory and the third memory are combined and arranged in an array, the height of the array is divisible by the preset height, and the last row of the array is divided by the preset height. The remainder after setting the width is not smaller than the preset width after being combined with the fourth memory.
因此，本申请实施方式中，在为输入图像分配内存时，分配的存储内存的高度可以与预设高度对齐，为了避免编码器读取时越界，可以为最后一行分配额外的内存，使阵列的最后一行除以预设宽度后的余数与第四内存组合后不小于预设宽度，避免编码器读取地址时越界，且仅需分配少量内存即可，减少了内存拷贝的量，可以减少进行内存拷贝的内存开销，且可以提高编码效率。Therefore, in the embodiment of this application, when allocating memory for the input image, the height of the allocated storage memory can be aligned with the preset height. In order to prevent the encoder from crossing the boundary when reading, additional memory can be allocated for the last row, so that the array The remainder after dividing the last line by the preset width is not smaller than the preset width after being combined with the fourth memory, to prevent the encoder from crossing the boundary when reading the address, and only need to allocate a small amount of memory, which reduces the amount of memory copy and can reduce the Memory copy memory overhead, and can improve coding efficiency.
在一种可能的实施方式中，多个内存切片分为第一内存切片和第二内存切片，第一内存切片的高度为预设高度，第二内存切片高度小于预设高度或者第二内存切片为多个内存切片组成的阵列中的最后一行最后一个切片，第一内存切片和第二内存切片为不同的内存切片；上述方法还可以包括：为多个内存切片中的第二内存切片分配第五内存，并将第二内存中的数据拷贝至第五内存中，第五内存的宽度被预设宽度整除，第五内存的高度被预设高度整除；前述的将多个内存切片中至少一个内存切片的地址输入至编码器，可以包括：将多个内存切片中的第一内存切片的地址以及第五内存的地址输入至编码器。In a possible implementation manner, multiple memory slices are divided into a first memory slice and a second memory slice, the height of the first memory slice is a preset height, and the height of the second memory slice is smaller than the preset height or the second memory slice The last slice of the last line in the array composed of multiple memory slices, the first memory slice and the second memory slice are different memory slices; the above method may also include: allocating the second memory slice for the second memory slice among the multiple memory slices Five memory, and copy the data in the second memory to the fifth memory, the width of the fifth memory is divisible by the preset width, and the height of the fifth memory is divisible by the preset height; the aforementioned at least one of the multiple memory slices Inputting the address of the memory slice to the encoder may include: inputting the address of the first memory slice and the address of the fifth memory among the plurality of memory slices to the encoder.
因此，本申请实施方式中，可以基于输入图像的尺寸为输入图像分配占用的存储内存，然后对存储内存进行划分，当输入图像的存储内存不被预设切片大小整除时，可以将得到的与预设切片大小匹配的内存的地址映射至编码器进行编码，以及从整除后余的部分中拷贝数据并作为编码器的输入，得到最终的编码数据，无需对存储内存中的所有内存进行拷贝，减少了内存拷贝的量，可以减少进行内存拷贝的内存开销，且可以提高编码效率。Therefore, in the embodiment of the present application, the storage memory occupied by the input image can be allocated based on the size of the input image, and then the storage memory can be divided. When the storage memory of the input image is not divisible by the preset slice size, the obtained and The address of the memory that matches the preset slice size is mapped to the encoder for encoding, and the data is copied from the remainder after division and used as the input of the encoder to obtain the final encoded data without copying all the memory in the storage memory. The amount of memory copy is reduced, the memory overhead of memory copy can be reduced, and the coding efficiency can be improved.
在一种可能的实施方式中，存储内存的高度被预设高度整除。In a possible implementation manner, the height of the storage memory is evenly divisible by the preset height.
因此，本申请实施方式中，当输入图像的存储内存不被预设切片大小整除时，在为输入图像分配存储内存的过程中，分配内存的高度可以按照预设高度对齐，从而可以减少多个切片中尺寸与预设切片大小不符的数量。随后可以将得到的与预设切片大小匹配的内存的地址映射至编码器进行编码，以及从整除后余的部分中拷贝数据并作为编码器的输入，得到最终的编码数据，无需对存储内存中的所有内存进行拷贝，减少了内存拷贝的量，可以减少进行内存拷贝的内存开销，且可以提高编码效率。Therefore, in the embodiment of the present application, when the storage memory of the input image is not divisible by the preset slice size, in the process of allocating the storage memory for the input image, the height of the allocated memory can be aligned according to the preset height, thereby reducing multiple The number of slices whose dimensions did not match the preset slice size. Then you can map the address of the memory that matches the preset slice size to the encoder for encoding, and copy the data from the remainder after integer division and use it as the input of the encoder to obtain the final encoded data without needing to store the data in the memory. Copy all the memory of the memory, which reduces the amount of memory copy, can reduce the memory overhead of memory copy, and can improve the coding efficiency.
第二方面，本申请提供一种图像编码装置，包括：In a second aspect, the present application provides an image encoding device, including:
分配模块，用于为输入图像分配存储内存；An allocation module for allocating storage memory for the input image;
划分模块，用于将存储内存划分为多个内存切片；A division module, configured to divide the storage memory into multiple memory slices;
编码模块，用于将多个内存切片中至少一个内存切片的地址输入至编码器，编码器用 于基于输入的地址读取数据并进行编码，得到编码数据。The encoding module is configured to input the address of at least one memory slice in the plurality of memory slices to the encoder, and the encoder is used to read and encode data based on the input address to obtain encoded data.
在一种可能的实施方式中，划分模块，具体用于按照预设切片大小，将存储内存划分为多个内存切片，预设切片大小包括预设高度和预设宽度。In a possible implementation manner, the division module is specifically configured to divide the storage memory into multiple memory slices according to a preset slice size, where the preset slice size includes a preset height and a preset width.
在一种可能的实施方式中，具体用于：基于预设切片大小为输入图像分配存储内存。In a possible implementation manner, it is specifically used for: allocating storage memory for the input image based on a preset slice size.
在一种可能的实施方式中，存储内存中包括第一内存和第二内存，第一内存中保存了输入图像中的各个像素点的信息，第二内存用于使存储内存的尺寸按照预设切片大小对齐，以使编码器以预设切片大小为单位从输入的地址中读取数据。In a possible implementation manner, the storage memory includes a first memory and a second memory, the first memory stores the information of each pixel in the input image, and the second memory is used to make the size of the storage memory according to the preset Slice size alignment so that the encoder reads data from the input address in units of the preset slice size.
在一种可能的实施方式中，存储内存按照阵列排列，存储内存的宽度被预设宽度整除，存储内存的高度被预设高度整除。In a possible implementation manner, the storage memory is arranged in an array, the width of the storage memory is divisible by a preset width, and the height of the storage memory is divisible by a preset height.
在一种可能的实施方式中，第二内存包括第三内存和第四内存，第一内存和第三内存组合后按照阵列排列，阵列的高度被预设高度整除，阵列的最后一行除以预设宽度后的余数与第四内存组合后不小于预设宽度。In a possible implementation manner, the second memory includes a third memory and a fourth memory, and the first memory and the third memory are combined and arranged in an array, the height of the array is divisible by the preset height, and the last row of the array is divided by the preset height. The remainder after setting the width is not smaller than the preset width after being combined with the fourth memory.
在一种可能的实施方式中，多个内存切片分为第一内存切片和第二内存切片，第一内存切片的高度为预设高度，第二内存切片高度小于预设高度或者第二内存切片为多个内存切片组成的阵列中的最后一行最后一个切片，第一内存切片和第二内存切片为不同的内存切片；In a possible implementation manner, multiple memory slices are divided into a first memory slice and a second memory slice, the height of the first memory slice is a preset height, and the height of the second memory slice is smaller than the preset height or the second memory slice It is the last slice of the last line in the array composed of multiple memory slices, the first memory slice and the second memory slice are different memory slices;
分配模块，还用于为多个内存切片中的第二内存切片分配第五内存，并将第二内存中的数据拷贝至第五内存中，第五内存的宽度被预设宽度整除，第五内存的高度被预设高度整除；The allocation module is also used to allocate a fifth memory for the second memory slice in the plurality of memory slices, and copy the data in the second memory to the fifth memory, the width of the fifth memory is divisible by the preset width, and the fifth memory The height of the memory is divisible by the preset height;
编码模块，具体用于将多个内存切片中的第一内存切片的地址以及第五内存的地址输入至编码器。The encoding module is specifically configured to input the address of the first memory slice and the address of the fifth memory among the plurality of memory slices to the encoder.
在一种可能的实施方式中，存储内存的高度被预设高度整除。In a possible implementation manner, the height of the storage memory is evenly divisible by the preset height.
第三方面，本申请实施例提供一种电子设备，包括：处理器和存储器，其中，处理器和存储器通过线路互联，处理器调用存储器中的程序代码用于执行上述第一方面任一项所示的图像编码方法中与处理相关的功能。可选地，该电子设备可以是芯片。In a third aspect, an embodiment of the present application provides an electronic device, including: a processor and a memory, wherein the processor and the memory are interconnected through a line, and the processor calls the program code in the memory to execute any one of the above-mentioned first aspects. Functions related to processing in the image coding method shown. Alternatively, the electronic device may be a chip.
第四方面，本申请实施例提供了一种电子设备，该电子设备也可以称为数字处理芯片或者芯片，芯片包括处理单元和通信接口，处理单元通过通信接口获取程序指令，程序指令被处理单元执行，处理单元用于执行如上述第一方面或第一方面任一可选实施方式中与处理相关的功能。In the fourth aspect, the embodiment of the present application provides an electronic device, which can also be called a digital processing chip or a chip. The chip includes a processing unit and a communication interface. The processing unit obtains program instructions through the communication interface, and the program instructions are processed by the processing unit. For execution, the processing unit is configured to execute the processing-related functions in the first aspect or any optional implementation manner of the first aspect.
第五方面，本申请实施例提供了一种计算机可读存储介质，包括指令，当其在计算机上运行时，使得计算机执行上述第一方面或第一方面任一可选实施方式中的方法。In a fifth aspect, an embodiment of the present application provides a computer-readable storage medium, including instructions, which, when run on a computer, cause the computer to execute the method in the above-mentioned first aspect or any optional implementation manner of the first aspect.
第六方面，本申请实施例提供了一种包含指令的计算机程序产品，当其在计算机上运行时，使得计算机执行上述第一方面或第一方面任一可选实施方式中的方法。In a sixth aspect, the embodiments of the present application provide a computer program product containing instructions, which, when run on a computer, cause the computer to execute the method in the above-mentioned first aspect or any optional implementation manner of the first aspect.
附图说明Description of drawings
图1为本申请提供的一种电子设备的结构示意图；FIG. 1 is a schematic structural diagram of an electronic device provided by the present application;
图2为本申请提供的另一种电子设备的结构示意图；FIG. 2 is a schematic structural diagram of another electronic device provided by the present application;
图3为本申请提供的一种图像编码方法的流程示意图；FIG. 3 is a schematic flowchart of an image encoding method provided by the present application;
图4为本申请提供的一种存储内存的结构示意图；FIG. 4 is a schematic structural diagram of a storage memory provided by the present application;
图5为本申请提供的另一种电子设备的结构示意图；FIG. 5 is a schematic structural diagram of another electronic device provided by the present application;
图6为本申请提供的另一种图像编码方法的流程示意图；FIG. 6 is a schematic flowchart of another image coding method provided by the present application;
图7为本申请提供的另一种存储内存的结构示意图；FIG. 7 is a schematic structural diagram of another storage memory provided by the present application;
图8为本申请提供的另一种存储内存的结构示意图；FIG. 8 is a schematic structural diagram of another storage memory provided by the present application;
图9为本申请提供的另一种存储内存的结构示意图；FIG. 9 is a schematic structural diagram of another storage memory provided by the present application;
图10为本申请提供的另一种存储内存的结构示意图；FIG. 10 is a schematic structural diagram of another storage memory provided by the present application;
图11为本申请提供的另一种存储内存的结构示意图；FIG. 11 is a schematic structural diagram of another storage memory provided by the present application;
图12为本申请提供的另一种存储内存的结构示意图；FIG. 12 is a schematic structural diagram of another storage memory provided by the present application;
图13为本申请提供的一种图像编码装置的结构示意图；FIG. 13 is a schematic structural diagram of an image encoding device provided by the present application;
图14为本申请提供的另一种电子设备的结构示意图。FIG. 14 is a schematic structural diagram of another electronic device provided by the present application.
具体实施方式Detailed ways
下面将结合本申请实施例中的附图，对本申请实施例中的技术方案进行描述，显然，所描述的实施例仅仅是本申请一部分实施例，而不是全部的实施例。基于本申请中的实施例，本领域技术人员在没有作出创造性劳动前提下所获得的所有其他实施例，都属于本申请保护的范围。The following will describe the technical solutions in the embodiments of the application with reference to the drawings in the embodiments of the application. Apparently, the described embodiments are only some of the embodiments of the application, not all of them. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without making creative efforts belong to the scope of protection of this application.
首先，本申请提供的方法可以部署于各种电子设备中，如部署于各种终端或者服务器中，该服务器可以是各种独立服务器、分布式服务器或者集中式服务器等，该终端可以包括但不限于：智能移动电话、电视、平板电脑、手环、头戴显示设备(Head Mount Display，HMD)、增强现实(augmented reality，AR)设备，混合现实(mixed reality，MR)设备、蜂窝电话(cellular phone)、智能电话(smart phone)、个人数字助理(personal digital assistant，PDA)、平板型电脑、车载终端、膝上型电脑(laptop computer)(或者称为笔记本电脑或者手提电脑等)、个人电脑(personal computer，PC)等。First of all, the method provided by this application can be deployed in various electronic devices, such as deployed in various terminals or servers. The servers can be various independent servers, distributed servers, or centralized servers. The terminals can include but not Limited to: smart mobile phones, TVs, tablet computers, wristbands, head mounted display devices (Head Mount Display, HMD), augmented reality (augmented reality, AR) devices, mixed reality (mixed reality, MR) devices, cellular phones (cellular phone), smart phone (smart phone), personal digital assistant (personal digital assistant, PDA), tablet computer, vehicle terminal, laptop computer (laptop computer) (or notebook computer or laptop computer, etc.), personal computer (personal computer, PC) and so on.
示例性地，参阅图1，下面以一个具体的结构为例，对本申请提供的电子设备的结构进行示例性说明。For example, referring to FIG. 1 , a specific structure is taken as an example below to illustrate the structure of the electronic device provided in this application.
电子设备100可以包括处理器110，外部存储器接口120，内部存储器121，通用串行总线(universal serial bus，USB)接口130，充电管理模块140，电源管理模块141，电池142，天线1，天线2，移动通信模块150，无线通信模块160，音频模块170，扬声器170A，受话器170B，麦克风170C，耳机接口170D，传感器模块180，按键190，马达191，指示器192，摄像头193，显示屏194，以及用户标识模块(subscriber identification module，SIM)卡接口195等。其中传感器模块180可以包括压力传感器180A，陀螺仪传感器180B，气压传感器180C，磁传感器180D，加速度传感器180E，距离传感器180F，接近光传感器180G，指纹传感器180H，温度传感器180J，触摸传感器180K，环境光传感器180L，骨传导传感器180M，运动传感器180N等。The electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (universal serial bus, USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, and an antenna 2 , mobile communication module 150, wireless communication module 160, audio module 170, speaker 170A, receiver 170B, microphone 170C, earphone jack 170D, sensor module 180, button 190, motor 191, indicator 192, camera 193, display screen 194, and A subscriber identification module (subscriber identification module, SIM) card interface 195 and the like. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, bone conduction sensor 180M, motion sensor 180N, etc.
可以理解的是，本发明实施例示意的结构并不构成对电子设备100的具体限定。在本申请另一些实施例中，电子设备100可以包括比图示更多或更少的部件，或者组合某些部件，或者拆分某些部件，或者不同的部件布置。图示的部件可以以硬件，软件或软件和硬件的组合实现。It can be understood that, the structure illustrated in the embodiment of the present invention does not constitute a specific limitation on the electronic device 100 . In other embodiments of the present application, the electronic device 100 may include more or fewer components than shown in the figure, or combine certain components, or separate certain components, or arrange different components. The illustrated components can be realized in hardware, software or a combination of software and hardware.
处理器110可以包括一个或多个处理单元，例如：处理器110可以包括应用处理器(application processor，AP)，调制解调处理器，图形处理器(graphics processing unit，GPU)，图像信号处理器(image signal processor，ISP)，控制器，视频编解码器，数字信号处理器(digital signal processor，DSP)，基带处理器，和/或神经网络处理器(neural-network processing unit，NPU)等。其中，不同的处理单元可以是独立的器件，也可以集成在一个或多个处理器中。The processor 110 may include one or more processing units, for example: the processor 110 may include an application processor (application processor, AP), a modem processor, a graphics processing unit (graphics processing unit, GPU), an image signal processor (image signal processor, ISP), controller, video codec, digital signal processor (digital signal processor, DSP), baseband processor, and/or neural network processor (neural-network processing unit, NPU), etc. Wherein, different processing units may be independent devices, or may be integrated in one or more processors.
控制器可以根据指令操作码和时序信号，产生操作控制信号，完成取指令和执行指令的控制。The controller can generate an operation control signal according to the instruction opcode and timing signal, and complete the control of fetching and executing the instruction.
处理器110中还可以设置存储器，用于存储指令和数据。在一些实施例中，处理器110中的存储器为高速缓冲存储器。该存储器可以保存处理器110刚用过或循环使用的指令或数据。如果处理器110需要再次使用该指令或数据，可从所述存储器中直接调用。避免了重复存取，减少了处理器110的等待时间，因而提高了***的效率。A memory may also be provided in the processor 110 for storing instructions and data. In some embodiments, the memory in processor 110 is a cache memory. The memory may hold instructions or data that the processor 110 has just used or recycled. If the processor 110 needs to use the instruction or data again, it can be called directly from the memory. Repeated access is avoided, and the waiting time of the processor 110 is reduced, thereby improving the efficiency of the system.
在一些实施例中，处理器110可以包括一个或多个接口。接口可以包括集成电路(inter-integrated circuit，I2C)接口，集成电路内置音频(inter-integrated circuit sound，I2S)接口，脉冲编码调制(pulse code modulation，PCM)接口，通用异步收发传输器(universal asynchronous receiver/transmitter，UART)接口，移动产业处理器接口(mobile industry processor interface，MIPI)，通用输入输出(general-purpose input/output，GPIO)接口，用户标识模块(subscriber identity module，SIM)接口，和/或通用串行总线(universal serial bus，USB)接口等。In some embodiments, processor 110 may include one or more interfaces. The interface may include an integrated circuit (inter-integrated circuit, I2C) interface, an integrated circuit built-in audio (inter-integrated circuit sound, I2S) interface, a pulse code modulation (pulse code modulation, PCM) interface, a universal asynchronous transmitter (universal asynchronous receiver/transmitter, UART) interface, mobile industry processor interface (mobile industry processor interface, MIPI), general-purpose input and output (general-purpose input/output, GPIO) interface, subscriber identity module (subscriber identity module, SIM) interface, and /or universal serial bus (universal serial bus, USB) interface, etc.
I2C接口是一种双向同步串行总线，包括一根串行数据线(serial data line，SDA)和一根串行时钟线(derail clock line，SCL)。在一些实施例中，处理器110可以包含多组I2C总线。处理器110可以通过不同的I2C总线接口分别耦合触摸传感器180K，充电器，闪光灯，摄像头193等。例如：处理器110可以通过I2C接口耦合触摸传感器180K，使处理器110与触摸传感器180K通过I2C总线接口通信，实现电子设备100的触摸功能。The I2C interface is a bidirectional synchronous serial bus, including a serial data line (serial data line, SDA) and a serial clock line (derail clock line, SCL). In some embodiments, processor 110 may include multiple sets of I2C buses. The processor 110 can be respectively coupled to the touch sensor 180K, the charger, the flashlight, the camera 193 and the like through different I2C bus interfaces. For example, the processor 110 may be coupled to the touch sensor 180K through the I2C interface, so that the processor 110 and the touch sensor 180K communicate through the I2C bus interface to realize the touch function of the electronic device 100 .
I2S接口可以用于音频通信。在一些实施例中，处理器110可以包含多组I2S总线。处理器110可以通过I2S总线与音频模块170耦合，实现处理器110与音频模块170之间的通信。在一些实施例中，音频模块170可以通过I2S接口向无线通信模块160传递音频信号，实现通过蓝牙耳机接听电话的功能。The I2S interface can be used for audio communication. In some embodiments, processor 110 may include multiple sets of I2S buses. The processor 110 may be coupled to the audio module 170 through an I2S bus to implement communication between the processor 110 and the audio module 170 . In some embodiments, the audio module 170 can transmit audio signals to the wireless communication module 160 through the I2S interface, so as to realize the function of answering calls through the Bluetooth headset.
PCM接口也可以用于音频通信，将模拟信号抽样，量化和编码。在一些实施例中，音频模块170与无线通信模块160可以通过PCM总线接口耦合。在一些实施例中，音频模块170也可以通过PCM接口向无线通信模块160传递音频信号，实现通过蓝牙耳机接听电话的功能。所述I2S接口和所述PCM接口都可以用于音频通信。The PCM interface can also be used for audio communication, sampling, quantizing and encoding the analog signal. In some embodiments, the audio module 170 and the wireless communication module 160 may be coupled through a PCM bus interface. In some embodiments, the audio module 170 can also transmit audio signals to the wireless communication module 160 through the PCM interface, so as to realize the function of answering calls through the Bluetooth headset. Both the I2S interface and the PCM interface can be used for audio communication.
UART接口是一种通用串行数据总线，用于异步通信。该总线可以为双向通信总线。它 将要传输的数据在串行通信与并行通信之间转换。在一些实施例中，UART接口通常被用于连接处理器110与无线通信模块160。例如：处理器110通过UART接口与无线通信模块160中的蓝牙模块通信，实现蓝牙功能。在一些实施例中，音频模块170可以通过UART接口向无线通信模块160传递音频信号，实现通过蓝牙耳机播放音乐的功能。The UART interface is a universal serial data bus used for asynchronous communication. The bus can be a bidirectional communication bus. It converts the data to be transmitted between serial communication and parallel communication. In some embodiments, a UART interface is generally used to connect the processor 110 and the wireless communication module 160 . For example: the processor 110 communicates with the Bluetooth module in the wireless communication module 160 through the UART interface to realize the Bluetooth function. In some embodiments, the audio module 170 can transmit audio signals to the wireless communication module 160 through the UART interface, so as to realize the function of playing music through the Bluetooth headset.
MIPI接口可以被用于连接处理器110与显示屏194，摄像头193等***器件。MIPI接口包括摄像头串行接口(camera serial interface，CSI)，显示屏串行接口(display serial interface，DSI)等。在一些实施例中，处理器110和摄像头193通过CSI接口通信，实现电子设备100的拍摄功能。处理器110和显示屏194通过DSI接口通信，实现电子设备100的显示功能。The MIPI interface can be used to connect the processor 110 with peripheral devices such as the display screen 194 and the camera 193 . MIPI interface includes camera serial interface (camera serial interface, CSI), display serial interface (display serial interface, DSI), etc. In some embodiments, the processor 110 communicates with the camera 193 through the CSI interface to realize the shooting function of the electronic device 100 . The processor 110 communicates with the display screen 194 through the DSI interface to realize the display function of the electronic device 100 .
GPIO接口可以通过软件配置。GPIO接口可以被配置为控制信号，也可被配置为数据信号。在一些实施例中，GPIO接口可以用于连接处理器110与摄像头193，显示屏194，无线通信模块160，音频模块170，传感器模块180等。GPIO接口还可以被配置为I2C接口，I2S接口，UART接口，MIPI接口等。The GPIO interface can be configured by software. The GPIO interface can be configured as a control signal or as a data signal. In some embodiments, the GPIO interface can be used to connect the processor 110 with the camera 193 , the display screen 194 , the wireless communication module 160 , the audio module 170 , the sensor module 180 and so on. The GPIO interface can also be configured as an I2C interface, I2S interface, UART interface, MIPI interface, etc.
USB接口130是符合USB标准规范的接口，具体可以是Mini USB接口，Micro USB接口，USB Type C接口等。USB接口130可以用于连接充电器为电子设备100充电，也可以用于电子设备100与***设备之间传输数据。也可以用于连接耳机，通过耳机播放音频。该接口还可以用于连接其他电子设备，例如AR设备等。应理解，此处的USB接口130也可以替换为其他的接口，如Type-c或者Lighting等可实现充电或者数据传输的接口，此处仅仅以USB接口130为例进行示例性说明。The USB interface 130 is an interface conforming to the USB standard specification, specifically, it can be a Mini USB interface, a Micro USB interface, a USB Type C interface, and the like. The USB interface 130 can be used to connect a charger to charge the electronic device 100 , and can also be used to transmit data between the electronic device 100 and peripheral devices. It can also be used to connect headphones and play audio through them. This interface can also be used to connect other electronic devices, such as AR devices. It should be understood that the USB interface 130 here can also be replaced with other interfaces, such as Type-c or Lighting, which can realize charging or data transmission, and only the USB interface 130 is used as an example for illustration here.
可以理解的是，本发明实施例示意的各模块间的接口连接关系，只是示意性说明，并不构成对电子设备100的结构限定。在本申请另一些实施例中，电子设备100也可以采用上述实施例中不同的接口连接方式，或多种接口连接方式的组合。It can be understood that the interface connection relationship between the modules shown in the embodiment of the present invention is only a schematic illustration, and does not constitute a structural limitation of the electronic device 100 . In other embodiments of the present application, the electronic device 100 may also adopt different interface connection manners in the foregoing embodiments, or a combination of multiple interface connection manners.
充电管理模块140用于从充电器接收充电输入。其中，充电器可以是无线充电器，也可以是有线充电器。在一些有线充电的实施例中，充电管理模块140可以通过USB接口130接收有线充电器的充电输入。在一些无线充电的实施例中，充电管理模块140可以通过电子设备100的无线充电线圈接收无线充电输入。充电管理模块140为电池142充电的同时，还可以通过电源管理模块141为电子设备供电。The charging management module 140 is configured to receive a charging input from a charger. Wherein, the charger may be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 140 can receive charging input from the wired charger through the USB interface 130 . In some wireless charging embodiments, the charging management module 140 may receive a wireless charging input through a wireless charging coil of the electronic device 100 . While the charging management module 140 is charging the battery 142 , it can also supply power to the electronic device through the power management module 141 .
电源管理模块141用于连接电池142，充电管理模块140与处理器110。电源管理模块141接收电池142和/或充电管理模块140的输入，为处理器110，内部存储器121，显示屏194，摄像头193，和无线通信模块160等供电。电源管理模块141还可以用于监测电池容量，电池循环次数，电池健康状态(漏电，阻抗)等参数。在其他一些实施例中，电源管理模块141也可以设置于处理器110中。在另一些实施例中，电源管理模块141和充电管理模块140也可以设置于同一个器件中。The power management module 141 is used for connecting the battery 142 , the charging management module 140 and the processor 110 . The power management module 141 receives the input from the battery 142 and/or the charging management module 140 to provide power for the processor 110 , the internal memory 121 , the display screen 194 , the camera 193 , and the wireless communication module 160 . The power management module 141 can also be used to monitor parameters such as battery capacity, battery cycle times, and battery health status (leakage, impedance). In some other embodiments, the power management module 141 may also be disposed in the processor 110 . In some other embodiments, the power management module 141 and the charging management module 140 may also be set in the same device.
电子设备100的无线通信功能可以通过天线1，天线2，移动通信模块150，无线通信模块160，调制解调处理器以及基带处理器等实现。The wireless communication function of the electronic device 100 can be realized by the antenna 1 , the antenna 2 , the mobile communication module 150 , the wireless communication module 160 , a modem processor, a baseband processor, and the like.
天线1和天线2用于发射和接收电磁波信号。电子设备100中的每个天线可用于覆盖单个或多个通信频带。不同的天线还可以复用，以提高天线的利用率。例如：可以将天线 1复用为无线局域网的分集天线。在另外一些实施例中，天线可以和调谐开关结合使用。Antenna 1 and Antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 100 may be used to cover single or multiple communication frequency bands. Different antennas can also be multiplexed to improve the utilization of the antennas. For example: Antenna 1 can be multiplexed as a diversity antenna for a WLAN. In other embodiments, the antenna may be used in conjunction with a tuning switch.
移动通信模块150可以提供应用在电子设备100上的包括2G/3G/4G/5G等无线通信的解决方案。移动通信模块150可以包括至少一个滤波器，开关，功率放大器，低噪声放大器(low noise amplifier，LNA)等。移动通信模块150可以由天线1接收电磁波，并对接收的电磁波进行滤波，放大等处理，传送至调制解调处理器进行解调。移动通信模块150还可以对经调制解调处理器调制后的信号放大，经天线1转为电磁波辐射出去。在一些实施例中，移动通信模块150的至少部分功能模块可以被设置于处理器110中。在一些实施例中，移动通信模块150的至少部分功能模块可以与处理器110的至少部分模块被设置在同一个器件中。The mobile communication module 150 can provide wireless communication solutions including 2G/3G/4G/5G applied on the electronic device 100 . The mobile communication module 150 may include at least one filter, switch, power amplifier, low noise amplifier (low noise amplifier, LNA) and the like. The mobile communication module 150 can receive electromagnetic waves through the antenna 1, filter and amplify the received electromagnetic waves, and send them to the modem processor for demodulation. The mobile communication module 150 can also amplify the signals modulated by the modem processor, and convert them into electromagnetic waves and radiate them through the antenna 1 . In some embodiments, at least part of the functional modules of the mobile communication module 150 may be set in the processor 110 . In some embodiments, at least part of the functional modules of the mobile communication module 150 and at least part of the modules of the processor 110 may be set in the same device.
调制解调处理器可以包括调制器和解调器。其中，调制器用于将待发送的低频基带信号调制成中高频信号。解调器用于将接收的电磁波信号解调为低频基带信号。随后解调器将解调得到的低频基带信号传送至基带处理器处理。低频基带信号经基带处理器处理后，被传递给应用处理器。应用处理器通过音频设备(不限于扬声器170A，受话器170B等)输出声音信号，或通过显示屏194显示图像或视频。在一些实施例中，调制解调处理器可以是独立的器件。在另一些实施例中，调制解调处理器可以独立于处理器110，与移动通信模块150或其他功能模块设置在同一个器件中。A modem processor may include a modulator and a demodulator. Wherein, the modulator is used for modulating the low-frequency baseband signal to be transmitted into a medium-high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low frequency baseband signal. Then the demodulator sends the demodulated low-frequency baseband signal to the baseband processor for processing. The low-frequency baseband signal is passed to the application processor after being processed by the baseband processor. The application processor outputs sound signals through audio equipment (not limited to speaker 170A, receiver 170B, etc.), or displays images or videos through display screen 194 . In some embodiments, the modem processor may be a stand-alone device. In some other embodiments, the modem processor may be independent from the processor 110, and be set in the same device as the mobile communication module 150 or other functional modules.
无线通信模块160可以提供应用在电子设备100上的包括无线局域网(wireless local area networks，WLAN)(如无线保真(wireless fidelity，Wi-Fi)网络)，蓝牙(bluetooth，BT)，全球导航卫星***(global navigation satellite system，GNSS)，调频(frequency modulation，FM)，近距离无线通信技术(near field communication，NFC)，超宽带(ultra wideband，UWB)，红外技术(infrared，IR)等无线通信的解决方案。无线通信模块160可以是集成至少一个通信处理模块的一个或多个器件。无线通信模块160经由天线2接收电磁波，将电磁波信号调频以及滤波处理，将处理后的信号发送到处理器110。无线通信模块160还可以从处理器110接收待发送的信号，对其进行调频，放大，经天线2转为电磁波辐射出去。The wireless communication module 160 can provide wireless local area networks (wireless local area networks, WLAN) (such as wireless fidelity (Wireless Fidelity, Wi-Fi) network), bluetooth (bluetooth, BT), global navigation satellite, etc. applied on the electronic device 100. System (global navigation satellite system, GNSS), frequency modulation (frequency modulation, FM), near field communication technology (near field communication, NFC), ultra wideband (ultra wideband, UWB), infrared technology (infrared, IR) and other wireless communications s solution. The wireless communication module 160 may be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2 , frequency-modulates and filters the electromagnetic wave signals, and sends the processed signals to the processor 110 . The wireless communication module 160 can also receive the signal to be sent from the processor 110 , frequency-modulate it, amplify it, and convert it into electromagnetic waves through the antenna 2 for radiation.
在一些实施例中，电子设备100的天线1和移动通信模块150耦合，天线2和无线通信模块160耦合，使得电子设备100可以通过无线通信技术与网络以及其他设备通信。所述无线通信技术可以包括但不限于：第五代移动通信技术(5th-Generation，5G)***，全球移动通讯***(global system for mobile communications，GSM)，通用分组无线服务(general packet radio service，GPRS)，码分多址接入(code division multiple access，CDMA)，宽带码分多址(wideband code division multiple access，WCDMA)，时分码分多址(time-division code division multiple access，TD-SCDMA)，长期演进(long term evolution，LTE)，蓝牙(bluetooth)，全球导航卫星***(the global navigation satellite system，GNSS)，无线保真(wireless fidelity，WiFi)，近距离无线通信(near field communication，NFC)，FM(也可以称为调频广播)，紫蜂协议(Zigbee)，射频识别技术(radio frequency identification，RFID)和/或红外(infrared，IR)技术等。所述GNSS可以包括全球卫星定位***(global positioning system，GPS)，全球导航卫星***(global  navigation satellite system，GLONASS)，北斗卫星导航***(beidou navigation satellite system，BDS)，准天顶卫星***(quasi-zenith satellite system，QZSS)和/或星基增强***(satellite based augmentation systems，SBAS)等。In some embodiments, the antenna 1 of the electronic device 100 is coupled to the mobile communication module 150, and the antenna 2 is coupled to the wireless communication module 160, so that the electronic device 100 can communicate with the network and other devices through wireless communication technology. The wireless communication technology may include but not limited to: the fifth generation mobile communication technology (5th-Generation, 5G) system, the global system for mobile communications (global system for mobile communications, GSM), general packet radio service (general packet radio service, GPRS), code division multiple access (CDMA), wideband code division multiple access (WCDMA), time-division code division multiple access (TD-SCDMA) ), long term evolution (long term evolution, LTE), bluetooth (bluetooth), global navigation satellite system (the global navigation satellite system, GNSS), wireless fidelity (wireless fidelity, WiFi), near field communication (near field communication, NFC), FM (also called FM radio), Zigbee (Zigbee), radio frequency identification (radio frequency identification, RFID) and/or infrared (infrared, IR) technology, etc. The GNSS may include a global positioning system (global positioning system, GPS), a global navigation satellite system (global navigation satellite system, GLONASS), a Beidou navigation satellite system (beidou navigation satellite system, BDS), a quasi-zenith satellite system (quasi -zenith satellite system, QZSS) and/or satellite based augmentation systems (satellite based augmentation systems, SBAS), etc.
在一些实施方式中，电子设备100也可以包括有线通信模块(图1中未示出)，或者，此处的移动通信模块150或者无线通信模块160可以替换为有线通信模块(图1中未示出)，该有线通信模块可以使电子设备通过有线网络与其他设备进行通信。该有线网络可以包括但不限于以下一项或者多项：光传送网(optical transport network，OTN)、同步数字体系(synchronous digital hierarchy，SDH)、无源光网络(passive optical network，PON)、以太网(Ethernet)、或灵活以太网(flex Ethernet，FlexE)等。In some implementations, the electronic device 100 may also include a wired communication module (not shown in FIG. 1 ), or the mobile communication module 150 or wireless communication module 160 here may be replaced by a wired communication module (not shown in FIG. output), the wired communication module can enable the electronic device to communicate with other devices through a wired network. The wired network may include but not limited to one or more of the following: Optical Transport Network (OTN), Synchronous Digital Hierarchy (SDH), Passive Optical Network (PON), Ethernet network (Ethernet), or flexible Ethernet (flex Ethernet, FlexE), etc.
电子设备100通过GPU，显示屏194，以及应用处理器等实现显示功能。GPU为界面显示的微处理器，连接显示屏194和应用处理器。GPU用于执行数学和几何计算，用于图形渲染。处理器110可包括一个或多个GPU，其执行程序指令以生成或改变显示信息。The electronic device 100 realizes the display function through the GPU, the display screen 194 , and the application processor. The GPU is a microprocessor for interface display, and is connected to the display screen 194 and the application processor. GPUs are used to perform mathematical and geometric calculations for graphics rendering. Processor 110 may include one or more GPUs that execute program instructions to generate or change display information.
显示屏194用于显示图像，视频等。显示屏194包括显示面板。显示面板可以采用液晶显示屏(liquid crystal display，LCD)，有机发光二极管(organic light-emitting diode，OLED)，有源矩阵有机发光二极体或主动矩阵有机发光二极体(active-matrix organic light emitting diode的，AMOLED)，柔性发光二极管(flex light-emitting diode，FLED)，Miniled，MicroLed，Micro-oLed，量子点发光二极管(quantum dot light emitting diodes，QLED)等。在一些实施例中，电子设备100可以包括1个或N个显示屏194，N为大于1的正整数。The display screen 194 is used to display images, videos and the like. The display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active matrix organic light emitting diode or an active matrix organic light emitting diode (active-matrix organic light emitting diode, AMOLED), flexible light-emitting diode (flex light-emitting diode, FLED), Miniled, MicroLed, Micro-oLed, quantum dot light emitting diodes (quantum dot light emitting diodes, QLED), etc. In some embodiments, the electronic device 100 may include 1 or N display screens 194 , where N is a positive integer greater than 1.
电子设备100可以通过ISP，摄像头193，视频编解码器，GPU，显示屏194以及应用处理器等实现拍摄功能。The electronic device 100 can realize the shooting function through the ISP, the camera 193 , the video codec, the GPU, the display screen 194 and the application processor.
ISP用于处理摄像头193反馈的数据。例如，拍照时，打开快门，光线通过镜头被传递到摄像头感光元件上，光信号转换为电信号，摄像头感光元件将所述电信号传递给ISP处理，转化为肉眼可见的图像。ISP还可以对图像的噪点，亮度，肤色进行算法优化。ISP还可以对拍摄场景的曝光，色温等参数优化。在一些实施例中，ISP可以设置在摄像头193中。The ISP is used for processing the data fed back by the camera 193 . For example, when taking a picture, open the shutter, the light is transmitted to the photosensitive element of the camera through the lens, and the light signal is converted into an electrical signal, and the photosensitive element of the camera transmits the electrical signal to the ISP for processing, and converts it into an image visible to the naked eye. ISP can also perform algorithm optimization on image noise, brightness, and skin color. ISP can also optimize the exposure, color temperature and other parameters of the shooting scene. In some embodiments, the ISP may be located in the camera 193 .
摄像头193用于捕获静态图像或视频。物体通过镜头生成光学图像投射到感光元件。感光元件可以是电荷耦合器件(charge coupled device，CCD)或互补金属氧化物半导体(complementary metal-oxide-semiconductor，CMOS)光电晶体管。感光元件把光信号转换成电信号，之后将电信号传递给ISP转换成数字图像信号。ISP将数字图像信号输出到DSP加工处理。DSP将数字图像信号转换成标准的RGB摄像头，YUV等格式的图像信号。在一些实施例中，电子设备100可以包括1个或N个摄像头193，N为大于1的正整数。Camera 193 is used to capture still images or video. The object generates an optical image through the lens and projects it to the photosensitive element. The photosensitive element may be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, and then transmits the electrical signal to the ISP to convert it into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. DSP converts digital image signals into image signals in standard RGB camera, YUV and other formats. In some embodiments, the electronic device 100 may include 1 or N cameras 193 , where N is a positive integer greater than 1.
数字信号处理器用于处理数字信号，除了可以处理数字图像信号，还可以处理其他数字信号。例如，当电子设备100在频点选择时，数字信号处理器用于对频点能量进行傅里叶变换等。Digital signal processors are used to process digital signals. In addition to digital image signals, they can also process other digital signals. For example, when the electronic device 100 selects a frequency point, the digital signal processor is used to perform Fourier transform on the energy of the frequency point.
视频编解码器用于对数字视频压缩或解压缩。电子设备100可以支持一种或多种视频编解码器。这样，电子设备100可以播放或录制多种编码格式的视频，例如：动态图像专 家组(moving picture experts group，MPEG)1，MPEG2，MPEG3，MPEG4等。Video codecs are used to compress or decompress digital video. The electronic device 100 may support one or more video codecs. In this way, the electronic device 100 can play or record video in various encoding formats, for example: moving picture experts group (moving picture experts group, MPEG) 1, MPEG2, MPEG3, MPEG4, etc.
NPU为神经网络(neural-network，NN)计算处理器，通过借鉴生物神经网络结构，例如借鉴人脑神经元之间传递模式，对输入信息快速处理，还可以不断的自学习。通过NPU可以实现电子设备100的智能认知等应用，例如：图像识别，人脸识别，语音识别，文本理解等。The NPU is a neural-network (NN) computing processor. By referring to the structure of biological neural networks, such as the transfer mode between neurons in the human brain, it can quickly process input information and continuously learn by itself. Applications such as intelligent cognition of the electronic device 100 can be realized through the NPU, such as image recognition, face recognition, speech recognition, text understanding, and the like.
外部存储器接口120可以用于连接外部存储卡，例如Micro SD卡，实现扩展电子设备100的存储能力。外部存储卡通过外部存储器接口120与处理器110通信，实现数据存储功能。例如将音乐，视频等文件保存在外部存储卡中。The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, so as to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 through the external memory interface 120 to implement a data storage function. Such as saving music, video and other files in the external memory card.
内部存储器121可以用于存储计算机可执行程序代码，所述可执行程序代码包括指令。内部存储器121可以包括存储程序区和存储数据区。其中，存储程序区可存储操作***，至少一个功能所需的应用程序(比如声音播放功能，图像播放功能等)等。存储数据区可存储电子设备100使用过程中所创建的数据(比如音频数据，电话本等)等。此外，内部存储器121可以包括高速随机存取存储器，还可以包括非易失性存储器，例如至少一个磁盘存储器件，闪存器件，通用闪存存储器(universal flash storage，UFS)等。处理器110通过运行存储在内部存储器121的指令，和/或存储在设置于处理器中的存储器的指令，执行电子设备100的各种功能应用以及数据处理。The internal memory 121 may be used to store computer-executable program codes including instructions. The internal memory 121 may include an area for storing programs and an area for storing data. Wherein, the stored program area can store an operating system, at least one application program required by a function (such as a sound playing function, an image playing function, etc.) and the like. The storage data area can store data created during the use of the electronic device 100 (such as audio data, phonebook, etc.) and the like. In addition, the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one magnetic disk storage device, flash memory device, universal flash storage (universal flash storage, UFS) and the like. The processor 110 executes various functional applications and data processing of the electronic device 100 by executing instructions stored in the internal memory 121 and/or instructions stored in a memory provided in the processor.
电子设备100可以通过音频模块170，扬声器170A，受话器170B，麦克风170C，耳机接口170D，以及应用处理器等实现音频功能。例如音乐播放，录音等。The electronic device 100 can implement audio functions through the audio module 170 , the speaker 170A, the receiver 170B, the microphone 170C, the earphone interface 170D, and the application processor. Such as music playback, recording, etc.
音频模块170用于将数字音频信息转换成模拟音频信号输出，也用于将模拟音频输入转换为数字音频信号。音频模块170还可以用于对音频信号编码和解码。在一些实施例中，音频模块170可以设置于处理器110中，或将音频模块170的部分功能模块设置于处理器110中。The audio module 170 is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signal. The audio module 170 may also be used to encode and decode audio signals. In some embodiments, the audio module 170 may be set in the processor 110 , or some functional modules of the audio module 170 may be set in the processor 110 .
扬声器170A，也称“喇叭”，用于将音频电信号转换为声音信号。电子设备100可以通过扬声器170A收听音乐，或收听免提通话。Speaker 170A, also referred to as a "horn", is used to convert audio electrical signals into sound signals. Electronic device 100 can listen to music through speaker 170A, or listen to hands-free calls.
受话器170B，也称“听筒”，用于将音频电信号转换成声音信号。当电子设备100接听电话或语音信息时，可以通过将受话器170B靠近人耳接听语音。Receiver 170B, also called "earpiece", is used to convert audio electrical signals into sound signals. When the electronic device 100 receives a call or a voice message, the receiver 170B can be placed close to the human ear to receive the voice.
麦克风170C，也称“话筒”，“传声器”，用于将声音信号转换为电信号。当拨打电话或发送语音信息时，用户可以通过人嘴靠近麦克风170C发声，将声音信号输入到麦克风170C。电子设备100可以设置至少一个麦克风170C。在另一些实施例中，电子设备100可以设置两个麦克风170C，除了采集声音信号，还可以实现降噪功能。在另一些实施例中，电子设备100还可以设置三个，四个或更多麦克风170C，实现采集声音信号，降噪，还可以识别声音来源，实现定向录音功能等。The microphone 170C, also called "microphone" or "microphone", is used to convert sound signals into electrical signals. When making a phone call or sending a voice message, the user can put his mouth close to the microphone 170C to make a sound, and input the sound signal to the microphone 170C. The electronic device 100 may be provided with at least one microphone 170C. In some other embodiments, the electronic device 100 may be provided with two microphones 170C, which may also implement a noise reduction function in addition to collecting sound signals. In some other embodiments, the electronic device 100 can also be provided with three, four or more microphones 170C to collect sound signals, reduce noise, identify sound sources, and realize directional recording functions, etc.
耳机接口170D用于连接有线耳机。耳机接口170D可以是USB接口130，也可以是3.5mm的开放移动电子设备平台(open mobile terminal platform，OMTP)标准接口，美国蜂窝电信工业协会(cellular telecommunications industry association of the USA，CTIA)标准接口。The earphone interface 170D is used for connecting wired earphones. The earphone interface 170D can be a USB interface 130, or a 3.5mm open mobile terminal platform (OMTP) standard interface, or a cellular telecommunications industry association of the USA (CTIA) standard interface.
压力传感器180A用于感受压力信号，可以将压力信号转换成电信号。在一些实施例中， 压力传感器180A可以设置于显示屏194。压力传感器180A的种类很多，如电阻式压力传感器，电感式压力传感器，电容式压力传感器等。电容式压力传感器可以是包括至少两个具有导电材料的平行板。当有力作用于压力传感器180A，电极之间的电容改变。电子设备100根据电容的变化确定压力的强度。当有触摸操作作用于显示屏194，电子设备100根据压力传感器180A检测所述触摸操作强度。电子设备100也可以根据压力传感器180A的检测信号计算触摸的位置。在一些实施例中，作用于相同触摸位置，但不同触摸操作强度的触摸操作，可以对应不同的操作指令。例如：当有触摸操作强度小于第一压力阈值的触摸操作作用于短消息应用图标时，执行查看短消息的指令。当有触摸操作强度大于或等于第一压力阈值的触摸操作作用于短消息应用图标时，执行新建短消息的指令。The pressure sensor 180A is used to sense the pressure signal and convert the pressure signal into an electrical signal. In some embodiments, the pressure sensor 180A may be located on the display screen 194 . There are many types of pressure sensors 180A, such as resistive pressure sensors, inductive pressure sensors, and capacitive pressure sensors. A capacitive pressure sensor may be comprised of at least two parallel plates with conductive material. When a force is applied to the pressure sensor 180A, the capacitance between the electrodes changes. The electronic device 100 determines the intensity of pressure according to the change in capacitance. When a touch operation acts on the display screen 194, the electronic device 100 detects the intensity of the touch operation according to the pressure sensor 180A. The electronic device 100 may also calculate the touched position according to the detection signal of the pressure sensor 180A. In some embodiments, touch operations acting on the same touch position but with different touch operation intensities may correspond to different operation instructions. For example: when a touch operation with a touch operation intensity less than the first pressure threshold acts on the short message application icon, an instruction to view short messages is executed. When a touch operation whose intensity is greater than or equal to the first pressure threshold acts on the icon of the short message application, the instruction of creating a new short message is executed.
陀螺仪传感器180B可以用于确定电子设备100的运动姿态。在一些实施例中，可以通过陀螺仪传感器180B确定电子设备100围绕三个轴(即，x，y和z轴)的角速度。陀螺仪传感器180B可以用于拍摄防抖。示例性的，当按下快门，陀螺仪传感器180B检测电子设备100抖动的角度，根据角度计算出镜头模组需要补偿的距离，让镜头通过反向运动抵消电子设备100的抖动，实现防抖。陀螺仪传感器180B还可以用于导航，体感游戏场景。The gyro sensor 180B can be used to determine the motion posture of the electronic device 100 . In some embodiments, the angular velocity of the electronic device 100 around three axes (ie, x, y and z axes) may be determined by the gyro sensor 180B. The gyro sensor 180B can be used for image stabilization. Exemplarily, when the shutter is pressed, the gyro sensor 180B detects the shaking angle of the electronic device 100, calculates the distance that the lens module needs to compensate according to the angle, and allows the lens to counteract the shaking of the electronic device 100 through reverse movement to achieve anti-shake. The gyro sensor 180B can also be used for navigation and somatosensory game scenes.
气压传感器180C用于测量气压。在一些实施例中，电子设备100通过气压传感器180C测得的气压值计算海拔高度，辅助定位和导航。The air pressure sensor 180C is used to measure air pressure. In some embodiments, the electronic device 100 calculates the altitude based on the air pressure value measured by the air pressure sensor 180C to assist positioning and navigation.
磁传感器180D包括霍尔传感器。电子设备100可以利用磁传感器180D检测翻盖皮套的开合。在一些实施例中，当电子设备100是翻盖机时，电子设备100可以根据磁传感器180D检测翻盖的开合。进而根据检测到的皮套的开合状态或翻盖的开合状态，设置翻盖自动解锁等特性。The magnetic sensor 180D includes a Hall sensor. The electronic device 100 may use the magnetic sensor 180D to detect the opening and closing of the flip leather case. In some embodiments, when the electronic device 100 is a clamshell machine, the electronic device 100 can detect opening and closing of the clamshell according to the magnetic sensor 180D. Furthermore, according to the detected opening and closing state of the leather case or the opening and closing state of the flip cover, features such as automatic unlocking of the flip cover are set.
加速度传感器180E可检测电子设备100在各个方向上(一般为三轴)加速度的大小。当电子设备100静止时可检测出重力的大小及方向。还可以用于识别电子设备姿态，应用于横竖屏切换，计步器等应用。The acceleration sensor 180E can detect the acceleration of the electronic device 100 in various directions (generally three axes). The magnitude and direction of gravity can be detected when the electronic device 100 is stationary. It can also be used to identify the posture of electronic devices, and can be used in applications such as horizontal and vertical screen switching, pedometers, etc.
距离传感器180F，用于测量距离。电子设备100可以通过红外或激光测量距离。在一些实施例中，拍摄场景，电子设备100可以利用距离传感器180F测距以实现快速对焦。The distance sensor 180F is used to measure the distance. The electronic device 100 may measure the distance by infrared or laser. In some embodiments, when shooting a scene, the electronic device 100 may use the distance sensor 180F for distance measurement to achieve fast focusing.
接近光传感器180G可以包括例如发光二极管(LED)和光检测器，例如光电二极管。发光二极管可以是红外发光二极管。电子设备100通过发光二极管向外发射红外光。电子设备100使用光电二极管检测来自附近物体的红外反射光。当检测到充分的反射光时，可以确定电子设备100附近有物体。当检测到不充分的反射光时，电子设备100可以确定电子设备100附近没有物体。电子设备100可以利用接近光传感器180G检测用户手持电子设备100贴近耳朵通话，以便自动熄灭屏幕达到省电的目的。接近光传感器180G也可用于皮套模式，口袋模式自动解锁与锁屏。Proximity light sensor 180G may include, for example, light emitting diodes (LEDs) and light detectors, such as photodiodes. The light emitting diodes may be infrared light emitting diodes. The electronic device 100 emits infrared light through the light emitting diode. Electronic device 100 uses photodiodes to detect infrared reflected light from nearby objects. When sufficient reflected light is detected, it may be determined that there is an object near the electronic device 100 . When insufficient reflected light is detected, the electronic device 100 may determine that there is no object near the electronic device 100 . The electronic device 100 can use the proximity light sensor 180G to detect that the user is holding the electronic device 100 close to the ear to make a call, so as to automatically turn off the screen to save power. The proximity light sensor 180G can also be used in leather case mode, automatic unlock and lock screen in pocket mode.
环境光传感器180L用于感知环境光亮度。电子设备100可以根据感知的环境光亮度自适应调节显示屏194亮度。环境光传感器180L也可用于拍照时自动调节白平衡。环境光传感器180L还可以与接近光传感器180G配合，检测电子设备100是否在口袋里，以防误触。The ambient light sensor 180L is used for sensing ambient light brightness. The electronic device 100 can adaptively adjust the brightness of the display screen 194 according to the perceived ambient light brightness. The ambient light sensor 180L can also be used to automatically adjust the white balance when taking pictures. The ambient light sensor 180L can also cooperate with the proximity light sensor 180G to detect whether the electronic device 100 is in the pocket, so as to prevent accidental touch.
指纹传感器180H用于采集指纹。电子设备100可以利用采集的指纹特性实现指纹解锁，访问应用锁，指纹拍照，指纹接听来电等。The fingerprint sensor 180H is used to collect fingerprints. The electronic device 100 can use the collected fingerprint characteristics to implement fingerprint unlocking, access to application locks, take pictures with fingerprints, answer incoming calls with fingerprints, and the like.
温度传感器180J用于检测温度。在一些实施例中，电子设备100利用温度传感器180J检测的温度，执行温度处理策略。例如，当温度传感器180J上报的温度超过阈值，电子设备100执行降低位于温度传感器180J附近的处理器的性能，以便降低功耗实施热保护。在另一些实施例中，当温度低于另一阈值时，电子设备100对电池142加热，以避免低温导致电子设备100异常关机。在其他一些实施例中，当温度低于又一阈值时，电子设备100对电池142的输出电压执行升压，以避免低温导致的异常关机。The temperature sensor 180J is used to detect temperature. In some embodiments, the electronic device 100 uses the temperature detected by the temperature sensor 180J to implement a temperature treatment strategy. For example, when the temperature reported by the temperature sensor 180J exceeds the threshold, the electronic device 100 may reduce the performance of the processor located near the temperature sensor 180J, so as to reduce power consumption and implement thermal protection. In some other embodiments, when the temperature is lower than another threshold, the electronic device 100 heats the battery 142 to avoid abnormal shutdown of the electronic device 100 caused by the low temperature. In some other embodiments, when the temperature is lower than another threshold, the electronic device 100 boosts the output voltage of the battery 142 to avoid abnormal shutdown caused by low temperature.
触摸传感器180K，也称“触控器件”。触摸传感器180K可以设置于显示屏194，由触摸传感器180K与显示屏194组成触摸屏，也称“触控屏”。触摸传感器180K用于检测作用于其上或附近的触摸操作。触摸传感器可以将检测到的触摸操作传递给应用处理器，以确定触摸事件类型。可以通过显示屏194提供与触摸操作相关的视觉输出。在另一些实施例中，触摸传感器180K也可以设置于电子设备100的表面，与显示屏194所处的位置不同。The touch sensor 180K is also called "touch device". The touch sensor 180K can be disposed on the display screen 194, and the touch sensor 180K and the display screen 194 form a touch screen, also called a “touch screen”. The touch sensor 180K is used to detect a touch operation on or near it. The touch sensor can pass the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through the display screen 194 . In other embodiments, the touch sensor 180K may also be disposed on the surface of the electronic device 100 , which is different from the position of the display screen 194 .
骨传导传感器180M可以获取振动信号。在一些实施例中，骨传导传感器180M可以获取人体声部振动骨块的振动信号。骨传导传感器180M也可以接触人体脉搏，接收血压跳动信号。在一些实施例中，骨传导传感器180M也可以设置于耳机中，结合成骨传导耳机。音频模块170可以基于所述骨传导传感器180M获取的声部振动骨块的振动信号，解析出语音信号，实现语音功能。应用处理器可以基于所述骨传导传感器180M获取的血压跳动信号解析心率信息，实现心率检测功能。The bone conduction sensor 180M can acquire vibration signals. In some embodiments, the bone conduction sensor 180M can acquire the vibration signal of the vibrating bone mass of the human voice. The bone conduction sensor 180M can also contact the human pulse and receive the blood pressure beating signal. In some embodiments, the bone conduction sensor 180M can also be disposed in the earphone, combined into a bone conduction earphone. The audio module 170 can analyze the voice signal based on the vibration signal of the vibrating bone mass of the vocal part acquired by the bone conduction sensor 180M, so as to realize the voice function. The application processor can analyze the heart rate information based on the blood pressure beating signal acquired by the bone conduction sensor 180M, so as to realize the heart rate detection function.
运动传感器180N，可以用于对摄像头拍摄的范围内的运动物体进行检测，采集运动物体的运动轮廓或者运动轨迹等。例如，该运动传感器180N可以是红外传感器、激光传感器、动态视觉传感器(dynamic vision sensor，DVS)等，该DVS具体可以包括DAVIS(Dynamic and Active-pixel Vision Sensor)、ATIS(Asynchronous Time-based Image Sensor)或者CeleX传感器等传感器。DVS借鉴了生物视觉的特性，每个像素模拟一个神经元，独立地对光照强度(以下简称“光强”)的相对变化做出响应。当光强的相对变化超过阈值时，像素会输出一个事件信号，包括像素的位置、时间戳以及光强的特征信息。The motion sensor 180N can be used to detect moving objects within the range captured by the camera, and collect motion profiles or trajectory of the moving objects. For example, the motion sensor 180N can be an infrared sensor, a laser sensor, a dynamic vision sensor (dynamic vision sensor, DVS), etc., and the DVS can specifically include DAVIS (Dynamic and Active-pixel Vision Sensor), ATIS (Asynchronous Time-based Image Sensor ) or sensors such as CeleX sensors. DVS draws lessons from the characteristics of biological vision, each pixel simulates a neuron, and independently responds to relative changes in light intensity (hereinafter referred to as "light intensity"). When the relative change of light intensity exceeds the threshold, the pixel will output an event signal, including the pixel's position, time stamp and feature information of light intensity.
按键190包括开机键，音量键等。按键190可以是机械按键。也可以是触摸式按键。电子设备100可以接收按键输入，产生与电子设备100的用户设置以及功能控制有关的键信号输入。The keys 190 include a power key, a volume key and the like. The key 190 may be a mechanical key. It can also be a touch button. The electronic device 100 may receive key input and generate key signal input related to user settings and function control of the electronic device 100 .
马达191可以产生振动提示。马达191可以用于来电振动提示，也可以用于触摸振动反馈。例如，作用于不同应用(例如拍照，音频播放等)的触摸操作，可以对应不同的振动反馈效果。作用于显示屏194不同区域的触摸操作，马达191也可对应不同的振动反馈效果。不同的应用场景(例如：时间提醒，接收信息，闹钟，游戏等)也可以对应不同的振动反馈效果。触摸振动反馈效果还可以支持自定义。The motor 191 can generate a vibrating reminder. The motor 191 can be used for incoming call vibration prompts, and can also be used for touch vibration feedback. For example, touch operations applied to different applications (such as taking pictures, playing audio, etc.) may correspond to different vibration feedback effects. The motor 191 may also correspond to different vibration feedback effects for touch operations acting on different areas of the display screen 194 . Different application scenarios (for example: time reminder, receiving information, alarm clock, games, etc.) can also correspond to different vibration feedback effects. The touch vibration feedback effect can also support customization.
指示器192可以是指示灯，可以用于指示充电状态，电量变化，也可以用于指示消息，未接来电，通知等。The indicator 192 can be an indicator light, and can be used to indicate charging status, power change, and can also be used to indicate messages, missed calls, notifications, and the like.
SIM卡接口195用于连接SIM卡。SIM卡可以通过***SIM卡接口195，或从SIM卡接口195拔出，实现和电子设备100的接触和分离。电子设备100可以支持1个或N个SIM卡接口，N为大于1的正整数。SIM卡接口195可以支持Nano SIM卡，Micro SIM卡，SIM 卡等。同一个SIM卡接口195可以同时***多张卡。所述多张卡的类型可以相同，也可以不同。SIM卡接口195也可以兼容不同类型的SIM卡。SIM卡接口195也可以兼容外部存储卡。电子设备100通过SIM卡和网络交互，实现通话以及数据通信等功能。在一些实施例中，电子设备100采用eSIM，即：嵌入式SIM卡。eSIM卡可以嵌在电子设备100中，不能和电子设备100分离。The SIM card interface 195 is used for connecting a SIM card. The SIM card can be connected and separated from the electronic device 100 by inserting it into the SIM card interface 195 or pulling it out from the SIM card interface 195 . The electronic device 100 may support 1 or N SIM card interfaces, where N is a positive integer greater than 1. SIM card interface 195 can support Nano SIM card, Micro SIM card, SIM card etc. Multiple cards can be inserted into the same SIM card interface 195 at the same time. The types of the multiple cards may be the same or different. The SIM card interface 195 is also compatible with different types of SIM cards. The SIM card interface 195 is also compatible with external memory cards. The electronic device 100 interacts with the network through the SIM card to implement functions such as calling and data communication. In some embodiments, the electronic device 100 adopts an eSIM, that is, an embedded SIM card. The eSIM card can be embedded in the electronic device 100 and cannot be separated from the electronic device 100 .
需要说明的是，在一些实际应用场景中，电子设备可以包括比前述图1中更多或者更少的部件，具体可以根据实际应用场景调整，本申请对此不作限定。It should be noted that, in some actual application scenarios, the electronic device may include more or fewer components than those shown in FIG. 1 , which may be adjusted according to actual application scenarios, which is not limited in this application.
前述对本申请提供的电子设备的硬件结构进行了示例性介绍，该电子设备可以搭载的***可以包括
Figure PCTCN2022142277-appb-000001
鸿蒙或者其它操作***等，本申请实施例对此不作任何限制。 The foregoing has given an exemplary introduction to the hardware structure of the electronic device provided by this application, and the system that the electronic device can carry may include
Figure PCTCN2022142277-appb-000001
Hongmeng or other operating systems, etc., the embodiment of this application does not make any restrictions on this.
以搭载
Figure PCTCN2022142277-appb-000002
操作***的电子设备100为例，如图2所示，电子设备100从逻辑上可划分为硬件层21、操作***261，以及应用层31。硬件层21包括应用程序处理器201、微控制器单元203、调制调解器207、Wi-Fi模块211、传感器214、定位模块150等硬件资源。应用层31包括一个或多个应用程序(application，APP)，比如应用程序263，应用程序263可以为社交类应用程序、电子商务类应用程序、浏览器等任意类型的应用程序。操作***261作为硬件层21和应用层31之间的软件中间件，是管理和控制硬件与软件资源的计算机程序。 to carry
Figure PCTCN2022142277-appb-000002
An electronic device 100 with an operating system is taken as an example. As shown in FIG. 2 , the electronic device 100 can be logically divided into a hardware layer 21 , an operating system 261 , and an application layer 31 . The hardware layer 21 includes hardware resources such as an application program processor 201 , a microcontroller unit 203 , a modem 207 , a Wi-Fi module 211 , a sensor 214 , and a positioning module 150 . The application layer 31 includes one or more applications (applications, APPs), such as the application program 263, and the application program 263 may be any type of application program such as a social application program, an e-commerce application program, or a browser. The operating system 261 is a software middleware between the hardware layer 21 and the application layer 31, and is a computer program for managing and controlling hardware and software resources.
在一个实施例中，操作***261包括内核23，硬件抽象层(hardware abstraction layer，HAL)25、库和运行时(librariesand runtime)27以及框架(framework)29。其中，内核23用于提供底层***组件和服务，例如：电源管理、内存管理、线程管理、硬件驱动程序等；硬件驱动程序包括Wi-Fi驱动、传感器驱动、定位模块驱动等。硬件抽象层25是对内核驱动程序的封装，向框架29提供接口，屏蔽低层的实现细节。硬件抽象层25运行在用户空间，而内核驱动程序运行在内核空间。In one embodiment, the operating system 261 includes a kernel 23, a hardware abstraction layer (hardware abstraction layer, HAL) 25, libraries and runtime (libraries and runtime) 27, and a framework (framework) 29. Among them, the kernel 23 is used to provide underlying system components and services, such as: power management, memory management, thread management, hardware drivers, etc.; hardware drivers include Wi-Fi drivers, sensor drivers, positioning module drivers, etc. The hardware abstraction layer 25 encapsulates the kernel driver, provides an interface to the framework 29, and shields low-level implementation details. The hardware abstraction layer 25 runs in user space, while the kernel driver runs in kernel space.
库和运行时27也叫做运行时库，它为可执行程序在运行时提供所需要的库文件和执行环境。库与运行时27包括安卓运行时(Android Runtime，ART)271以及库273等。ART 271是能够把应用程序的字节码转换为机器码的虚拟机或虚拟机实例。库273是为可执行程序在运行时提供支持的程序库，包括浏览器引擎(比如webkit)、脚本执行引擎(比如JavaScript引擎)、图形处理引擎等。Library and runtime 27 is also called runtime library, which provides required library files and execution environment for executable programs at runtime. The library and runtime 27 includes an Android runtime (Android Runtime, ART) 271 and a library 273, etc. ART 271 is a virtual machine or virtual machine instance capable of converting an application's bytecode into machine code. The library 273 is a program library that provides support for executable programs at runtime, including browser engines (such as webkit), script execution engines (such as JavaScript engines), graphics processing engines, and the like.
框架27用于为应用层31中的应用程序提供各种基础的公共组件和服务，比如窗口管理、位置管理等等。框架27可以包括电话管理器291，资源管理器293，位置管理器295等。The framework 27 is used to provide various basic public components and services for applications in the application layer 31, such as window management, location management and so on. The framework 27 may include a phone manager 291, a resource manager 293, a location manager 295, and the like.
以上描述的操作***261的各个组件的功能均可以由应用程序处理器201执行存储器205中存储的程序来实现。The functions of each component of the operating system 261 described above can be realized by the application program processor 201 executing the programs stored in the memory 205 .
所属领域的技术人员可以理解电子设备100可包括比图2所示的更少或更多的部件，图2所示的该电子设备仅包括与本申请实施例所公开的多个实现方式更加相关的部件。Those skilled in the art can understand that the electronic device 100 may include fewer or more components than those shown in FIG. 2. The electronic device shown in FIG. parts.
前述的电子设备具有存储图像并对图像数据进行编码等能力，在对图像的存储或者编码的过程中，可以将图像切分为多个切片(Grid)，然后对每个切片进行编码，从而可以提高图像的压缩效率，提高编码性能。The aforementioned electronic devices have the ability to store images and encode image data. During the process of storing or encoding images, the images can be divided into multiple slices (Grid), and then each slice can be encoded, so that Improve image compression efficiency and improve encoding performance.
常用的切片方式可以分为FrameWork层切片和切片下沉两种模式。例如，原始图像为:4032*3024尺寸的高效图像文件格式(high efficiency image file format，HEIF)编码的图片(通常称为Grid):512*512；而在进行编码时，为了避免编码器读取到非法内存，实际传入编码器的是4032*3024的存储内存，并切分为512*512的Grid，即需要为图像分配内存，将图像数据拷贝至4032*3024的存储内存，然后又编码器从这部分内存中读取数据进行编码，需要大量内存拷贝，影响性能和功耗。常用方案是使用特定指令(如NEON指令)做数据并行处理，加速拷贝过程。或者使用逐行、多行拷贝的方式取代逐像素的拷贝方式，加速拷贝过程。使用切片拷贝和高效率视频编码(high efficiency video coding，HEVC)并行的模式,从而加速端到端的编码过程。然而，在切片以及编码的过程中，存在大量的内存拷贝，将带来额外的开销，并且将消耗额外的内存来承载切片数据，切片过程开销大。Commonly used slicing methods can be divided into two modes: FrameWork layer slicing and slice sinking. For example, the original image is: 4032*3024 high efficiency image file format (high efficiency image file format, HEIF) coded picture (usually called Grid): 512*512; while encoding, in order to avoid the encoder from reading To the illegal memory, the actual input to the encoder is the storage memory of 4032*3024, and it is divided into 512*512 Grid, that is, it is necessary to allocate memory for the image, copy the image data to the storage memory of 4032*3024, and then encode The device reads data from this part of the memory for encoding, requiring a large number of memory copies, which affects performance and power consumption. A common solution is to use specific instructions (such as NEON instructions) for data parallel processing to speed up the copy process. Or use line-by-line and multi-line copying instead of pixel-by-pixel copying to speed up the copying process. Use slice copy and high efficiency video coding (HEVC) parallel mode to speed up the end-to-end encoding process. However, in the process of slicing and encoding, there are a large number of memory copies, which will bring additional overhead, and will consume additional memory to carry the sliced data, and the slicing process is expensive.
因此，本申请提供一种图像编码方法，在进行编码时，将输入图像的存储内存的地址直接映射至编码器，从而使编码器可以直接基于映射的地址来读取数据并进行编码，无需进行内存拷贝，且无需分配额外的内存的承载切片数据，降低内存拷贝带来的性能和功耗开销，提高编码效率。Therefore, the present application provides an image encoding method. When encoding, the address of the storage memory of the input image is directly mapped to the encoder, so that the encoder can directly read and encode data based on the mapped address without performing Memory copy, without allocating additional memory to carry slice data, reduces the performance and power consumption overhead caused by memory copy, and improves coding efficiency.
参阅图3，本申请提供的一种图像编码方法的流程示意图，如下所述。Referring to FIG. 3 , it is a schematic flowchart of an image encoding method provided by the present application, as described below.
301、为输入图像分配存储内存。301. Allocate storage memory for an input image.
其中，该输入图像可以是接收到的图像，也可以是本地摄像头拍摄后成像得到的图像，该输入图像可以存储于内存中，用于进行后续编码。该输入图像的存储内存的信息可以包括存储内存的地址或长度等信息。Wherein, the input image may be a received image, or may be an image captured by a local camera, and the input image may be stored in a memory for subsequent encoding. The information of the storage memory of the input image may include information such as an address or a length of the storage memory.
通常，在电子设备得到输入图像时，如生成或者接收到输入图像时，可以为输入图像分配一段连续的内存，输入图像中各个像素点的信息即可存储于内存中。可以理解为，输入图像可以按照阵列的形式存储于内存中，如在得到输入图像时，为输入图像分配一段连续的内存，输入图像的信息按照像素的排列方式存储于分配的内存中。例如，如图4所示，该输入图像在内存中可以按照阵列的形式来存储，并记录存储内存的大小、地址等信息。Generally, when an electronic device obtains an input image, such as generating or receiving an input image, a continuous memory can be allocated for the input image, and the information of each pixel in the input image can be stored in the memory. It can be understood that the input image can be stored in the memory in the form of an array. For example, when the input image is obtained, a continuous memory is allocated for the input image, and the information of the input image is stored in the allocated memory according to the arrangement of pixels. For example, as shown in FIG. 4 , the input image can be stored in the form of an array in the memory, and information such as the size and address of the storage memory can be recorded.
通常，分配的内存可以用于保存输入图像各个像素点的信息，如色度值、亮度值等信息。一些场景中，为了避免编码器读取数据越界，可以为输入图像分配额外的内存。Usually, the allocated memory can be used to save the information of each pixel of the input image, such as chrominance value, brightness value and other information. In some scenarios, in order to prevent the encoder from reading data out of bounds, additional memory can be allocated for the input image.
具体地，为输入图像分配内存的方式可以包括多种，下面示例性地，以几种可行的实施方式进行示例性说明。Specifically, there may be various manners of allocating memory for the input image, and the following exemplifies several feasible implementation manners for illustration.
在一种可能的实施方式中，可以基于预设切片大小为输入图像来分配一段连续的存储内存，可以将存储内存分为两部分，为便于区分称为第一内存和第二内存，第一内存用于保存输入图像中各个像素点的信息，第二内存用于使存储内存按照预设切片大小进行对齐，从而使后续编码器可以以预设切片大小为单位从输入的地址中读取数据。In a possible implementation manner, a continuous storage memory can be allocated for the input image based on the preset slice size, and the storage memory can be divided into two parts, which are called the first memory and the second memory for easy distinction. The memory is used to save the information of each pixel in the input image, and the second memory is used to align the storage memory according to the preset slice size, so that the subsequent encoder can read data from the input address in units of the preset slice size .
可选地，在为输入图像分配内存时，可以基于预设切片大小来为输入图像分配存储内存，其中，可以将存储内存理解为两部分，为便于区分称为第一内存和第二内存，第一内存用于保存保存输入图像的数据，如输入图像中每个像素点的亮度值、灰度值或者色度值等，第二内存则用于使为输入图像分配的内存的尺寸按照预设切片大小对齐，以便于后续 切片时可以按照预设切片大小进行切片，从而使编码器可以完整地读取每个内存切片的内存地址，以每个内存切片为单位来进行编码。Optionally, when allocating memory for the input image, the storage memory can be allocated for the input image based on the preset slice size, wherein the storage memory can be understood as two parts, which are called the first memory and the second memory for easy distinction, The first memory is used to save the data of the input image, such as the brightness value, gray value or chrominance value of each pixel in the input image, and the second memory is used to make the size of the memory allocated for the input image according to the preset The slice size is aligned so that subsequent slices can be sliced according to the preset slice size, so that the encoder can completely read the memory address of each memory slice, and encode in units of each memory slice.
可选地，在为输入图像分配内存时，可以按照预设切片大小进行分配,该预设切片大小包括预设高度和预设宽度，为输入图像分配的内存的高度按照预设高度进行对齐，为输入图像分配的存储内存的宽度按照预设宽度进行对齐，从而使后续在进行内存切片时，可以直接切分得到尺寸为预设切片大小的内存切片，从而使后续编码器在读取内存地址时避免越界。Optionally, when allocating memory for the input image, it can be allocated according to a preset slice size, the preset slice size includes a preset height and a preset width, and the height of the memory allocated for the input image is aligned according to the preset height, The width of the storage memory allocated for the input image is aligned according to the preset width, so that when the subsequent memory slice is performed, the memory slice with the size of the preset slice size can be directly sliced, so that the subsequent encoder can read the memory address avoid crossing the line.
可选地，在为输入图像分配内存时，分配的存储内存的高度按照预设高度进行对齐，分配的存储内存的宽度按照输入图像的实际所需内存分配，即第二内存可以分为多个部分，为便于区分称为第三内存和第四内存，第三内存与第一内存组合后按照阵列排列，即通过第三内存来使存储内存的高度和预设高度对齐，阵列的最后一行除以预设宽度后的余数与第四内存组合后不小于预设宽度。即为了减少分配的内存量，阵列的宽度可以不按照预设宽度进行对齐，在阵列的最后一行多分配一段内存，以避免后续的编码器读取越界。Optionally, when allocating memory for the input image, the height of the allocated storage memory is aligned according to the preset height, and the width of the allocated storage memory is allocated according to the actual required memory of the input image, that is, the second memory can be divided into multiple The part is called the third memory and the fourth memory for easy distinction. The third memory is combined with the first memory and arranged in an array, that is, the height of the storage memory is aligned with the preset height through the third memory. The last row of the array except Combining the remainder after the preset width with the fourth memory is not less than the preset width. That is, in order to reduce the amount of allocated memory, the width of the array may not be aligned according to the preset width, and an extra section of memory is allocated in the last line of the array to avoid subsequent encoder reading out of bounds.
此外，在为输入图像分配内存的过程中，可以仅对高度进行对齐，使得存储内存的高度被预设高度整除，当然，也可以直接按照输入图像所包括的数据来分配占用的内存，具体可以根据实际应用场景调整。此外，电子设备中可以预先部署前述的其中一种切片方式，也可以预先部署前述的多种切片方式。当电子设备中部署了多种切片方式时，在得到需要编码的输入图像之后，可以根据预先设定的方式从多种切片方式中选择其中一种切片方式进行切片，具体可以根据实际应用场景进行调整。In addition, in the process of allocating memory for the input image, you can only align the height so that the height of the storage memory is divisible by the preset height. Of course, you can also directly allocate the occupied memory according to the data included in the input image. Specifically, you can Adjust according to actual application scenarios. In addition, one of the foregoing slicing manners may be pre-deployed in the electronic device, and multiple foregoing slicing manners may also be pre-deployed. When multiple slicing methods are deployed in the electronic device, after obtaining the input image that needs to be encoded, one of the slicing methods can be selected from the various slicing methods according to the preset method for slicing, which can be performed according to the actual application scenario. Adjustment.
例如，当电子设备的可用内存低于第一阈值时，为了降低内存拷贝量，在为输入图像分配内存时，可以仅按照预设高度对存储内存的高度进行对齐并分配内存，从而形成阵列，并为阵列的最后一行额外分配多个字节，以使阵列的最后一行除以预设宽度后的余数与额外分配的多个字节组合后不小于该预设宽度，以避免编码器读取越界；当电子设备的可用内存大于第二阈值但不大于第三阈值时，则可以按照预设切片大小进行对齐，即存储内存的高度被预设高度整除，存储内存的宽度被预设宽度整除,从而使后续可以按照预设切片大小进行切片，从而使编码器以内存切片为单位进行编码，无需进行内存拷贝；当电子设备的可用内存大于第三阈值时，表示电子设备的可用内存充足，此时可以选择直接对该阵列进行切分，整除的部分得到多个切片，剩余的部分在编码时可以使用内存拷贝进行编码。因此，可以根据实际应用场景来选择匹配的切片方式，从而使图像的编码方式与电子设备的可用内存情况或者用户需求匹配，提高编码效率，提高用户体验。For example, when the available memory of the electronic device is lower than the first threshold, in order to reduce the amount of memory copy, when allocating memory for the input image, the height of the storage memory can be aligned and allocated only according to the preset height, thereby forming an array, And additionally allocate multiple bytes for the last row of the array, so that the remainder of the last row of the array divided by the preset width is not smaller than the preset width after combining with the additional allocated multiple bytes, so as to avoid the encoder from reading Out of bounds; when the available memory of the electronic device is greater than the second threshold but not greater than the third threshold, it can be aligned according to the preset slice size, that is, the height of the storage memory is divisible by the preset height, and the width of the storage memory is divisible by the preset width , so that the follow-up can be sliced according to the preset slice size, so that the encoder performs encoding in units of memory slices without memory copying; when the available memory of the electronic device is greater than the third threshold, it means that the available memory of the electronic device is sufficient, At this point, you can choose to split the array directly, and the divisible part can be divided into multiple slices, and the remaining part can be encoded using memory copy during encoding. Therefore, the matching slice method can be selected according to the actual application scenario, so that the encoding method of the image matches the available memory of the electronic device or the user's demand, thereby improving the encoding efficiency and user experience.
302、将存储内存划分为多个内存切片。302. Divide the storage memory into multiple memory slices.
其中，在确定待编码的输入图像的存储内存之后，可以对输入图像进行切分，得到多个内存切片。相当于对输入图像的存储内存进行切分，从而将存储内存分为多个栅格，即内存切片。Wherein, after determining the storage memory of the input image to be encoded, the input image may be segmented to obtain multiple memory slices. It is equivalent to segmenting the storage memory of the input image, so that the storage memory is divided into multiple grids, that is, memory slices.
具体地，可以按照预先设定的大小，即预设切片大小来对存储内存进行切分，得到多个内存切片(Grid)。即相当于对输入图像进行切分，将输入图像分为多个部分，以便于后续编码器可以对输入图像进行编码。Specifically, the storage memory may be divided according to a preset size, ie, a preset slice size, to obtain multiple memory slices (Grid). That is, it is equivalent to segmenting the input image, dividing the input image into multiple parts, so that the subsequent encoder can encode the input image.
该预设切片大小可以包括由用户预先设定的大小、由电子设备的上层与下层协商的大小、根据编码器的编码长度设定的大小或者随机大小等，具体可以根据实际应用场景来确定。The preset slice size may include a size preset by the user, a size negotiated between the upper layer and the lower layer of the electronic device, a size set according to the encoding length of the encoder, or a random size, etc., which may be determined according to actual application scenarios.
303、将多个内存切片的地址输入至编码器进行编码，得到编码数据。303. Input addresses of multiple memory slices to an encoder for encoding to obtain encoded data.
在进行了内存切片之后，将每个内存切片的地址输入至编码器进行编码，从而得到编码数据。在进行编码时，即可基于输入的地址从每个内存切片中提取到需要进行编码的数据，从而对每个内存切片中的数据进行编码，得到编码数据。After memory slicing is performed, the address of each memory slice is input to an encoder for encoding, thereby obtaining encoded data. When encoding, the data to be encoded can be extracted from each memory slice based on the input address, thereby encoding the data in each memory slice to obtain encoded data.
该编码器可以是电子设备中预先部署的编码器，可以按照预先设定的编码规则进行编码。如可以在电子设备中部署并运行一段代码，用于从输入至编码器的地址中读取数据，并按照预先设定的编码方式对图像进行编码，该编码方式可以HEVC编码、或者高级视频编码(advanced video coding，AVC)等编码方式等，具体可以根据实际应用场景进行调整，本申请对具体的编码方式不作限定。The encoder may be a pre-deployed encoder in the electronic device, and may perform encoding according to a preset encoding rule. For example, a piece of code can be deployed and run in an electronic device to read data from the address input to the encoder, and encode the image according to a preset encoding method, which can be HEVC encoding or advanced video encoding (advanced video coding, AVC) and other encoding methods, etc., can be adjusted according to actual application scenarios, and this application does not limit the specific encoding methods.
可以理解为，Grid为编码器的编码单位，编码器以一个Grid的内存保存的数据为单位进行编码，可以将每个Grid对应的内存地址映射至编码器，从而使编码器可以获取Grid的内存信息,包括宽、高、Stride等。It can be understood that Grid is the encoding unit of the encoder, and the encoder encodes the data stored in the memory of a Grid as a unit, and can map the memory address corresponding to each Grid to the encoder, so that the encoder can obtain the memory of the Grid Information, including width, height, Stride, etc.
因此，本申请实施方式中，可以直接将需要编码的数据的地址输入至编码器，使编码器可以直接基于地址提取到需要编码的数据，因此可以减少进行内存拷贝的内存开销，且可以提高编码效率。Therefore, in the embodiment of the present application, the address of the data to be encoded can be directly input to the encoder, so that the encoder can directly extract the data to be encoded based on the address, thus reducing the memory overhead for memory copying and improving the encoding efficiency. efficiency.
在一种可能的实施方式中，若存储内存不被预设切片大小整除，则对存储内存进行切分，得到多个内存切片以及第一切片，第一切片为存储内存中按照预设切片大小切分后余的部分。随后可以将多个内存切片的地址映射至编码器进行编码，以及从第一切片对应的地址中拷贝第一数据，并将第一数据作为编码器的输入，得到编码数据。In a possible implementation manner, if the storage memory is not divisible by the preset slice size, then the storage memory is divided to obtain multiple memory slices and the first slice, the first slice is the storage memory according to the preset The slice size is the remainder after slicing. Subsequently, the addresses of multiple memory slices may be mapped to the encoder for encoding, and the first data may be copied from the address corresponding to the first slice, and the first data may be used as an input of the encoder to obtain encoded data.
因此，本申请实施方式中，当输入图像的存储内存不被预设切片大小整除时，可以将得到的与预设切片大小匹配的内存的地址映射至编码器进行编码，以及从整除后余的部分中拷贝数据并作为编码器的输入，得到最终的编码数据，无需对存储内存中的所有内存进行拷贝，减少了内存拷贝的量，可以减少进行内存拷贝的内存开销，且可以提高编码效率。Therefore, in the embodiment of the present application, when the storage memory of the input image is not divisible by the preset slice size, the address of the obtained memory matching the preset slice size can be mapped to the encoder for encoding, and the remaining Partially copy the data and use it as the input of the encoder to obtain the final encoded data without copying all the memory in the storage memory, reducing the amount of memory copying, reducing the memory overhead of memory copying, and improving encoding efficiency.
在一种可能的实施方式中，若前述为输入图像分配内存时，未将存储内存完全按照预设切片大小对齐，如仅对齐了高度而未对齐宽度，或者高度与宽度均未对齐等，则在进行内存切片时，可以按照预设切片大小对存储内存进行切分。然后可以将得到的内存切片分为两类，便于区分称为第一内存切片和第二内存切片，第一内存切片的高度为预设高度，第二内存切片的高度小于该预设高度，或者该第二内存切片为最后一行的最后一个切片，第一内存切片和第二内存切片为不同的内存切片，即同一个内存的类型是第一内存切片或第二内存切片中的一种。In a possible implementation, if the storage memory is not completely aligned according to the preset slice size when allocating memory for the input image, for example, only the height is aligned but the width is not aligned, or the height and width are not aligned, etc., then When performing memory slicing, the storage memory can be divided according to the preset slice size. Then the obtained memory slices can be divided into two types, which are called first memory slices and second memory slices for easy distinction. The height of the first memory slice is the preset height, and the height of the second memory slice is smaller than the preset height, or The second memory slice is the last slice in the last row, and the first memory slice and the second memory slice are different memory slices, that is, the type of the same memory is either the first memory slice or the second memory slice.
可以为第二内存切片分配第五内存，并将第二内存中的数据拷贝至第五内存中，第五内存的宽度被预设宽度整除，第五内存的高度被预设高度整除，将第二内存切片中保存的数据拷贝至第五内存中。在进行编码时，即可将第一内存切片和第五内存的地址均输入至编码器，以使编码器从输入的地址中读取到数据并进行编码，得到最终的编码数据，避免 出现编码器读取越界的情况。The fifth memory can be allocated for the second memory slice, and the data in the second memory can be copied to the fifth memory. The width of the fifth memory is divisible by the preset width, and the height of the fifth memory is divisible by the preset height. The data stored in the second memory slice is copied to the fifth memory. When encoding, the addresses of the first memory slice and the fifth memory can be input to the encoder, so that the encoder can read the data from the input address and encode it to obtain the final encoded data, avoiding the occurrence of encoding The case where the device reads out of bounds.
例如，若存储内存的高度被预设高度整除，而存储内存的宽度不被预设宽度整除，则可以将最后一行最后一列的内存切片作为第二内存切片，其余内存切片分为第一内存切片。为第二内存切片分配第五内存，该第五内存的尺寸可以为预设切片大小，将第二内存切片中保存的数据拷贝至第五内存中。然后将第一内存切片和第五内存的地址输入至编码器，以使编码器从输入的地址中读取到数据并进行编码，得到最终的编码数据。For example, if the height of the storage memory is divisible by the preset height, but the width of the storage memory is not divisible by the preset width, the memory slice of the last row and last column can be used as the second memory slice, and the remaining memory slices can be divided into the first memory slice . A fifth memory is allocated for the second memory slice, the size of the fifth memory may be a preset slice size, and the data stored in the second memory slice is copied to the fifth memory. Then, the addresses of the first memory slice and the fifth memory are input to the encoder, so that the encoder reads and encodes data from the input addresses to obtain final encoded data.
又例如，若存储内存的高度不被预设高度整除，且存储内存的宽度也不被预设宽度整除，此时可以将高度不被预设高度整除的部分，即存储内存中最后一行对应的内存切片作为第二内存切片。并为第二内存切片分配第五内存，将第二内存切片中保存的数据拷贝至第五内存中。然后将第一内存切片和第五内存的地址输入至编码器，以使编码器从输入的地址中读取到数据并进行编码，得到最终的编码数据，避免出现编码器读取越界的情况。For another example, if the height of the storage memory is not divisible by the preset height, and the width of the storage memory is not divisible by the preset width, then the part whose height is not divisible by the preset height, that is, the part corresponding to the last row in the storage memory The memory slice acts as the second memory slice. And allocate a fifth memory for the second memory slice, and copy the data stored in the second memory slice to the fifth memory. Then, the addresses of the first memory slice and the fifth memory are input to the encoder, so that the encoder reads and encodes data from the input addresses to obtain the final encoded data, avoiding the situation that the encoder reads out of bounds.
前述对本申请提供的方法流程进行了介绍，为便于理解，下面结合具体的应用场景进行示例性介绍。The foregoing has introduced the flow of the method provided by the present application. For ease of understanding, an exemplary introduction will be given below in combination with specific application scenarios.
首先，以一个具体的应用场景为例，其适配的电子设备的部分***架构可以如图5所示。其中，该***架构可以基于MediaCodec/OMX框架以及基于HEVC编码器，实现HEIC编码，可以包括多层，如应用层、FrameWork层以及HAL/vendor层等，应用层中可以安装APP，可以由APP来触发编码器进行编码。First, taking a specific application scenario as an example, a partial system architecture of an electronic device adapted thereto may be shown in FIG. 5 . Among them, the system architecture can be based on the MediaCodec/OMX framework and the HEVC encoder to implement HEIC encoding, and can include multiple layers, such as the application layer, FrameWork layer and HAL/vendor layer, etc. APP can be installed in the application layer, and can be implemented by the APP. Trigger the encoder to encode.
其中，该电子设备中可以安装一些可以得到输入图像的APP，如相机或者通信软件等可以生成或者接收图像的APP，具体例如相机可以用于拍摄图像，并触发保存图像的步骤；通信软件可以接收其他电子设备发送的图像，并触发保存图像的步骤等。Among them, some APPs that can obtain input images can be installed in the electronic device, such as cameras or communication software that can generate or receive images. Specifically, for example, the camera can be used to capture images and trigger the step of saving images; Images sent by other electronic devices, and trigger steps to save images, etc.
HEIF编码：在Android FrameWork中，已经实现HEIFWriter和HEIFEncoder，HEIFWriter用于为应用层(APK)提供接口供应用层输入原始图片数据，HEIFWriter可以调用HEIFEncoder的接口，HEIFWriter用于完成文件的封装，生成.HEIC文件；HEIFEncoder可以调用MediaCodec的接口，用于创建HEIC编码器,启动HEIF编码。HEIF encoding: In Android FrameWork, HEIFWriter and HEIFEncoder have been implemented. HEIFWriter is used to provide an interface for the application layer (APK) for the application layer to input original image data. HEIFWriter can call the interface of HEIFEncoder. HEIFWriter is used to complete the file packaging and generate . HEIC file; HEIFEncoder can call MediaCodec interface to create HEIC encoder and start HEIF encoding.
MediaCodec是一种用于对音视频进行编解码的类，通过访问底层的codec来实现编解码的功能。是电子设备的媒体(media)基础框架的一部分。MediaCodec is a class for encoding and decoding audio and video, and realizes the function of encoding and decoding by accessing the underlying codec. It is part of the media infrastructure of electronic devices.
OMX，或者称为OpenMAX(Open Media Acceleration)，是一种开放的跨平台媒体框架，可以用于传递Input Buffer(即输入的内存信息)的Usage(即标识)。OMX, or OpenMAX (Open Media Acceleration), is an open cross-platform media framework that can be used to transfer the Usage (identification) of Input Buffer (ie, input memory information).
HEIC编码器，作为OMX IL(OpenMAX Integration Layer，OMX框架层和组件层的接口)层，完成和OMX框架的对接，本实施例中其核心功能是完成HEIF切片。The HEIC encoder, as the OMX IL (OpenMAX Integration Layer, interface between the OMX framework layer and the component layer) layer, completes the docking with the OMX framework. In this embodiment, its core function is to complete HEIF slicing.
Gralloc(Graphic Buffer Allocator)，是一种Android中负责申请和释放GraphicBuffer的HAL层模块，用于进行Graphic Buffer的内存分配，其中优化的功能点：识别HEIC编码的特定Usage属性后，内存大小基于Grid大小进行对齐。Gralloc (Graphic Buffer Allocator), is a HAL layer module responsible for applying and releasing Graphic Buffer in Android. It is used for memory allocation of Graphic Buffer. The optimized function point: After identifying the specific Usage attribute of HEIC encoding, the memory size is based on Grid size to align.
HEVC编码器：其输入为HEIC编码器切片后得到的每个Grid的内存信息，如Grid对应图片的宽度、Grid对应图片的高度、stride等信息，用于从输入Grid内存信息中读取数据，并分别对每个Grid中的数据进行HEVC编码。HEVC encoder: Its input is the memory information of each Grid obtained after slicing by the HEIC encoder, such as the width of the grid corresponding to the picture, the height of the grid corresponding to the picture, stride and other information, used to read data from the input Grid memory information, And perform HEVC encoding on the data in each Grid respectively.
具体地，对图像进行编码的过程可以如图6所示。Specifically, the process of encoding an image may be as shown in FIG. 6 .
601、HEIC编码器设置标识(usage)601. HEIC encoder setting flag (usage)
首先由HEIC编码器设置usage，该usage是一种标识，用于标识输入的图像的编码类型是否为HEIC编码。First, the usage is set by the HEIC encoder, and the usage is an identifier for identifying whether the encoding type of the input image is HEIC encoding.
如HEIC编码器对Input Buffer设置特定的属性，如通过graphic buffer usage标识是否按照本申请提供的方式进行切片并进行编码。可以理解为，在HEIC编码器模块可以和OMX协商Buffer属性的过程，本申请增加一种特定的graphic buffer usage，用于标识HEIC编码切片场景的切片预对齐属性，即采用那种切片方式进行对齐并切片。如从以下步骤603的切片方式中选择哪一种方式来进行切片。For example, the HEIC encoder sets specific attributes for the Input Buffer, such as marking whether to slice and encode according to the method provided by this application through the graphic buffer usage. It can be understood that, in the process of negotiating Buffer attributes between the HEIC encoder module and OMX, this application adds a specific graphic buffer usage, which is used to identify the slice pre-alignment attribute of the HEIC encoding slice scene, that is, which slice method is used for alignment And slice. Which method is selected from the slicing methods in step 603 below for slicing.
602、OMX传递usage给Gralloc602. OMX passes usage to Gralloc
随后在OMX模块发起graphic buffer内存申请之前，OMX可以读取graphic buffer usage，识别到是哪种类型的usage后，即可获知当前为HEIC编码场景。此时可以设置一个新的usage，这个usage也需要全局唯一，用来标识HEIC编码场景，作为Input Buffer的属性，并且能被Gralloc识别，或者直接向Gralloc传递graphic buffer usage，使Gralloc识别当前为HEIC编码场景。Then, before the OMX module initiates the graphic buffer memory application, OMX can read the graphic buffer usage, and after identifying the type of usage, it can know that the current HEIC encoding scene is. At this point, you can set a new usage, which also needs to be globally unique, to identify the HEIC encoding scene, as an attribute of the Input Buffer, and can be recognized by Gralloc, or directly pass the graphic buffer usage to Gralloc, so that Gralloc recognizes the current HEIC Code the scene.
603、Gralloc基于usage分配内存603. Gralloc allocates memory based on usage
其中，可以预先部署一种或者多种切片方式，Gralloc得到usage之后，即可识别出当前为HEIC编码场景，并分配相应的内存。Among them, one or more slicing methods can be pre-deployed. After Gralloc gets the usage, it can recognize the current HEIC encoding scene and allocate corresponding memory.
当仅部署了一种切片方式时，Gralloc得到usage之后，即可识别出当前为HEIC编码场景，然后进行内存分配。When only one slicing method is deployed, Gralloc can recognize the current HEIC encoding scene after getting the usage, and then perform memory allocation.
当部署了多种切片方式时，Gralloc得到usage之后，可以基于该usage识别出采用哪种方式进行切片，随后分配相应的内存。When multiple slicing methods are deployed, after Gralloc gets the usage, it can identify which method to use for slicing based on the usage, and then allocate the corresponding memory.
通常，当存在多种切片方式时，每种切片方式所需分配的内存可能相同也可能不相同，可以根据实际应用场景来选择匹配的切片方式，如当电子设备的可用内存充足时，在可以随机选择其中一种切片方式，当电子设备的可用内存不足时，则可以选择消耗内存最少的切片方式，或者在对编码效率需求较高时，则可以选择消耗内存较少的切片方式来，具体可以根据实际应用场景进行调整，本申请仅仅是示例性说明，对此并不作限定。Usually, when there are multiple slicing methods, the memory allocated by each slicing method may be the same or may not be the same, and the matching slicing method can be selected according to the actual application scenario, for example, when the available memory of the electronic device is sufficient, it can Randomly select one of the slicing methods. When the available memory of the electronic device is insufficient, you can choose the slicing method that consumes the least memory, or when the coding efficiency is high, you can choose the slicing method that consumes less memory. Specifically Adjustments may be made according to actual application scenarios, and this application is only an illustration and not a limitation.
604、HEIC编码器进行切片604. HEIC encoder slices
在Gralloc基于usage分配内存之后，即可将分配的内存的信息传入HEIC编码器，由HEIC编码器来进行切片，并将切片后得到的每个切片的信息传输给HEVC编码器，以使HEVC编码器从切片中读取数据并进行编码。After Gralloc allocates memory based on usage, the information of the allocated memory can be passed to the HEIC encoder, and the HEIC encoder will slice it, and transmit the information of each slice obtained after slicing to the HEVC encoder, so that HEVC An encoder reads data from a slice and encodes it.
在进行切片的过程中，以Android***为例，可以由FrameWork层切片(或者应用层切片)或者由编码器切片(通常称为切片下沉)。In the process of slicing, taking the Android system as an example, it can be sliced by the FrameWork layer (or sliced by the application layer) or sliced by the encoder (usually called slice sinking).
若由FrameWork层切片，则由上层创建HEVC编码器：上层可以通过Mediacodec/OMX和下层协商切片大小，或者预先设定切片大小，然后由上层基于切片大小来进行切片，每个切片的地址通过Mediacodec/OMX传递给编码器，由编码器基于传入的地址来提取需要编码的数据并进行编码。编码后的数据可以通过Output Buffer承载，传递给上层。上层通过生成并反馈流结束(end of stream，EOS)，表示编码完成，上层收到图像的所有切片的 编码数据之后，即可封装为HEIC文件。If sliced by the FrameWork layer, the HEVC encoder is created by the upper layer: the upper layer can negotiate the slice size with the lower layer through Mediacodec/OMX, or preset the slice size, and then the upper layer slices based on the slice size, and the address of each slice is passed through Mediacodec /OMX is passed to the encoder, and the encoder extracts and encodes the data to be encoded based on the incoming address. The encoded data can be carried by the Output Buffer and passed to the upper layer. The upper layer generates and feeds back the end of stream (EOS), which indicates that the encoding is completed. After the upper layer receives the encoded data of all slices of the image, it can be encapsulated into a HEIC file.
若由编码器切片，由上层创建HEIC编码器，上层可以通过Mediacodec/OMX和下层协商切片大小，或者预先设定切片大小，然后上层传输图像至下层，由编码器对图像进行切片，并完成编码，在对最后一个切片进行编码完成之后，上报EOS，上层收到图像的所有切片的编码数据之后，即可封装为HEIC文件。If sliced by the encoder, the HEIC encoder is created by the upper layer. The upper layer can negotiate the slice size with the lower layer through Mediacodec/OMX, or pre-set the slice size, and then the upper layer transmits the image to the lower layer, and the encoder slices the image and completes the encoding. , after the encoding of the last slice is completed, EOS is reported, and after the upper layer receives the encoded data of all slices of the image, it can be encapsulated into a HEIC file.
其中，上层与下层为相对概念，如FrameWork层或者以上的层可以理解为上层，FrameWork层以下的层可以称为下层，具体可以根据实际应用场景来进行划分，本申请仅仅是示例性说明，并不作为限定。Among them, the upper layer and the lower layer are relative concepts. For example, the layer above the FrameWork layer can be understood as the upper layer, and the layer below the FrameWork layer can be called the lower layer. Specifically, it can be divided according to the actual application scenario. Not as limiting.
例如，可以将输入图像切分为多个相同大小的部分，即对输入图像对应的存储内存切分为多个相同大小的Grid，然后配置相关参数作为HEVC编码器的输入。示例性地，在进行切片时，如图7所示，将分配的内存的宽度称为stride，分配的内存的高度称为Height；每个Grid的宽度称为Grid width，每个Grid的高度称为Grid height，该Grid width*Grid height即可理解为前述的预设切片大小。For example, the input image can be divided into multiple parts of the same size, that is, the storage memory corresponding to the input image can be divided into multiple grids of the same size, and then related parameters can be configured as the input of the HEVC encoder. Exemplarily, when slicing, as shown in Figure 7, the width of the allocated memory is called stride, the height of the allocated memory is called Height; the width of each Grid is called Grid width, and the height of each Grid is called Grid height, the Grid width*Grid height can be understood as the aforementioned preset slice size.
结合前述步骤603和步骤604，本申请提供的方法中，可以按照多种方式来为输入图像分配内存并进行切片，下面示例性地分别进行介绍。In combination with the aforementioned step 603 and step 604, in the method provided by the present application, the input image can be allocated memory and sliced in various ways, which will be introduced as examples below.
方式一、按照Grid width分配strideMethod 1. Assign stride according to Grid width
其中，在分配内存时，内存的stride可以按照Grid width来进行对齐，可以理解为内存的stride可以被Grid width整除，内存的Height可以按照Grid height进行对齐。Among them, when allocating memory, the stride of the memory can be aligned according to the Grid width, which can be understood as the stride of the memory can be divisible by the Grid width, and the height of the memory can be aligned according to the Grid height.
通常，输入图像中的数据所占用的内存的尺寸可能不被预设切片大小整除，而每个Grid的大小通常需要保持一致。Usually, the size of the memory occupied by the data in the input image may not be divisible by the preset slice size, and the size of each Grid usually needs to be consistent.
示例性地，如图8所示，输入图像中的数据所占用的内存的尺寸不被Grid的尺寸整除，则在为Input Buffer分配内存时，分配内存的stride按照Grid width对齐，即可以为最后一列Grid以及最后一行Grid分配内存，并将每个Grid的内存信息(包括为输入图像所包括的数据分配的存储内存地址和额外分配的内存的地址)传入编码器，从而使编码器在读取内存时，可以从每个Grid对应的内存信息中读取数据，避免读取到非法内存而导致程序崩溃或者编码器无法正常工作。且无需进行内存拷贝，编码器可以直接从输入地址中读取到输入图像，减少了内存拷贝的过程，减少内存开销，提高编码效率。Exemplarily, as shown in Figure 8, the size of the memory occupied by the data in the input image is not divisible by the size of the Grid, then when allocating memory for the Input Buffer, the stride of the allocated memory is aligned according to the Grid width, which can be the last A column of Grid and the last row of Grid allocate memory, and the memory information of each Grid (including the storage memory address allocated for the data included in the input image and the address of additional allocated memory) is passed to the encoder, so that the encoder can read When fetching memory, you can read data from the memory information corresponding to each Grid, so as to avoid reading illegal memory and causing the program to crash or the encoder to fail to work properly. And without memory copying, the encoder can directly read the input image from the input address, which reduces the process of memory copying, reduces memory overhead, and improves coding efficiency.
以4032*3024分辨率的输入图像为例，若通过一些常用的切分方式进行切分，如切分为512*512的Grid，切片时需要大量内存拷贝，影响性能和功耗，并且需要分配额外的内存承载Grid数据。如需要拷贝48个Grid，每个Grid内存为512*512*4＝1M Bytes(每个Grid需要拷贝512次，每次512*4Bytes)。而通过本申请提供的内存分配以及切片方式，不需要额外的内存拷贝，消除了额外的性能和功耗开销，并且不需要分配额外的内存承载Grid数据。Taking an input image with a resolution of 4032*3024 as an example, if it is segmented by some commonly used segmentation methods, such as dividing into a grid of 512*512, a large amount of memory copy is required when slicing, which affects performance and power consumption, and needs to be allocated Additional memory hosts Grid data. If you need to copy 48 Grids, the memory of each Grid is 512*512*4=1M Bytes (each Grid needs to be copied 512 times, each time 512*4Bytes). However, the memory allocation and slicing methods provided by this application do not require additional memory copying, eliminate additional performance and power consumption overhead, and do not need to allocate additional memory to carry Grid data.
方式二、多分配N字节Method 2: allocate more N bytes
通常，编码器可以以Grid为单位进行编码，输入图像的存储内存通常是一段连续的内存，因此编码器可以按照每个Grid的起始地址来读取每个Grid的数据，在基于内存映射 进行编码的情况下，当读取到最后一行的最后一个Grid的时，超出该Grid的尺寸的部分可以从下一行Grid的地址中读取，从而读取到完整的一个Grid的内容，后续在进行图像恢复时，可以裁除最后一列Grid中多余的部分。而当读取到最后一行最后一个Grid的地址时，若读取超过Grid范围则可能读取到非法内存，从而使编码器无法正常工作或者影响编码器的编码过程等。Usually, the encoder can encode in the unit of Grid, and the storage memory of the input image is usually a continuous memory, so the encoder can read the data of each Grid according to the starting address of each Grid, based on memory mapping In the case of encoding, when the last Grid of the last line is read, the part exceeding the size of the Grid can be read from the address of the next line of Grid, so as to read the content of a complete Grid, and follow-up When the image is restored, the redundant part in the last grid column can be trimmed. And when the address of the last Grid of the last line is read, if the reading exceeds the range of the Grid, illegal memory may be read, thus making the encoder unable to work normally or affecting the encoding process of the encoder.
因此，可以针对最后一个Grid多分配N个字节，即最后一个Grid的最后一行的宽度为Grid width，从而使编码器可以读取到一个完整的gird的数据，从而可以避免读取到非法内存，从而使编码器产生编码错误或者影响编码器的编码过程等。Therefore, N bytes can be allocated for the last Grid, that is, the width of the last line of the last Grid is Grid width, so that the encoder can read the data of a complete grid, thereby avoiding reading illegal memory , so that the encoder generates encoding errors or affects the encoding process of the encoder, etc.
例如，如图9所示，在在使用Input Buffer分配内存时，分配内存的Height按照Grid height对齐，Stride不做Grid width对齐处理,；在此基础上，多分配N字节的内存。以RGBA格式的图像为例，N≥((Grid width)–(图片width％Grid width))*4，得到的N值为确保编码器读取内存时不越界的最小取值，从而可以避免读取到非法内存，从而使编码器产生编码错误或者影响编码器的编码过程等。For example, as shown in Figure 9, when using the Input Buffer to allocate memory, the Height of the allocated memory is aligned according to the Grid height, and the Stride does not perform Grid width alignment; on this basis, more N bytes of memory are allocated. Taking an image in RGBA format as an example, N≥((Grid width)–(picture width%Grid width))*4, the obtained N value is the minimum value to ensure that the encoder does not cross the boundary when reading the memory, so as to avoid reading Access to illegal memory, so that the encoder produces encoding errors or affects the encoding process of the encoder, etc.
可以理解为，Input Buffer分配内存时，Height按照Grid大小对齐，Stride按照存储内存的宽度进行取值(或者按照常规,按照16字节或者32字节或者64字节对齐，具体数目由***自行决定)，因图9中右下角的Grid映射至编码器时存在内存不足的问题，因此，可以多分配N字节的内存，以避免读取到非法内存，从而使编码器产生编码错误或者影响编码器的编码过程等。It can be understood that when the Input Buffer allocates memory, the Height is aligned according to the size of the Grid, and the Stride is valued according to the width of the storage memory (or conventionally, aligned according to 16 bytes, 32 bytes, or 64 bytes, and the specific number is determined by the system itself. ), because there is a problem of insufficient memory when the Grid in the lower right corner of Figure 9 is mapped to the encoder, therefore, more N bytes of memory can be allocated to avoid reading illegal memory, which will cause encoding errors in the encoder or affect encoding The encoding process of the device, etc.
以4032*3024分辨率的输入图像为例，可以切分为512*512的Grid,切片时需要大量内存拷贝，影响性能和功耗，并且需要分配额外的内存承载Grid数据。如需要拷贝48个Grid，每个Grid内存为512*512*4＝1M Bytes(每个Grid需要拷贝512次，每次512*4Bytes)。而通过本申请提供的切片方式，不需要额外的内存拷贝，消除了额外的性能和功耗开销，并且不需要分配额外的内存承载Grid数据。HEIF切片时，最后一行的Grid不需要额外的内存拷贝，因此所有的Grid都不需要内存拷贝。优化了性能和功耗，并且不需要分配额外的内存承载Grid数据。Taking an input image with a resolution of 4032*3024 as an example, it can be divided into 512*512 Grids. When slicing, a large amount of memory copy is required, which affects performance and power consumption, and additional memory needs to be allocated to carry Grid data. If you need to copy 48 Grids, the memory of each Grid is 512*512*4=1M Bytes (each Grid needs to be copied 512 times, each time 512*4Bytes). However, the slicing method provided by this application does not require additional memory copy, eliminates additional performance and power consumption overhead, and does not need to allocate additional memory to carry Grid data. When HEIF slices, the grid in the last row does not need additional memory copy, so all grids do not need memory copy. Optimized performance and power consumption, and does not need to allocate additional memory to carry Grid data.
方式三、部分采用内存地址映射，剩余部分采用内存拷贝Method 3: Some parts use memory address mapping, and the rest use memory copy
可以将输入图像的存储内存中可以被Grid的尺寸整除的部分的地址映射至编码器中，而余下的部分则可以通过内存拷贝的方式来输入至编码器中。The address of the part of the storage memory of the input image that can be divisible by the size of the Grid can be mapped to the encoder, and the remaining part can be input to the encoder by means of memory copy.
其中，输入图像的数据需占用的内存可能分为多种情况，如高度被Grid高度整除，或者高度不被Grid高度整除。Among them, the memory occupied by the data of the input image may be divided into many situations, for example, the height is divisible by the height of the Grid, or the height is not divisible by the height of the Grid.
例如，如图10所示，如若输入图像的数据需占用的内存高度被Grid高度整除，则可以按照输入图像的实际所需内存来分配存储内存，并对存储内存进行划分，得到的内存切片分为第一内存切片和第二内存切片，如图10中所示，第二内存切片为最后一行最后一个内存切片，其宽度不被Grid宽度整除，其余为第一内存切片。为了避免编码器在读取最后一行最后一个内存切片(即第二内存切片)时越界，可以对最后一行的最后一个切片进行内存拷贝，即为最后一个内存切片分配额外的内存(即第五内存)，然后将该最后一个切片 中的数据拷贝至额外分配的内存中。然后将第一内存切片的地址和第五内存的地址均作为编码器的输入。For example, as shown in Figure 10, if the memory height to be occupied by the input image data is divisible by the height of the Grid, the storage memory can be allocated according to the actual memory required by the input image, and the storage memory can be divided. It is the first memory slice and the second memory slice, as shown in Figure 10, the second memory slice is the last memory slice in the last line, its width is not divisible by the Grid width, and the rest are the first memory slice. In order to prevent the encoder from crossing the boundary when reading the last memory slice of the last line (ie, the second memory slice), the memory copy of the last slice of the last line can be performed, that is, additional memory is allocated for the last memory slice (ie, the fifth memory slice) ), and then copy the data in the last slice to the additionally allocated memory. Then both the address of the first memory slice and the address of the fifth memory are used as the input of the encoder.
即可以按照预先设定的Grid大小对输入图像的存储内存进行切分，将其中被整除的部分的内存通过内存地址映射的方式传入编码器，而其余不被整除的部分则可以为其分配额外的内存，通过内存拷贝的方式将不被整除部分内存中的数据传入编码器中。That is, the storage memory of the input image can be divided according to the preset Grid size, and the memory of the divisible part can be transferred to the encoder through memory address mapping, while the rest of the non-divisible part can be allocated to it. For extra memory, transfer the data in the non-divisible part of the memory to the encoder by means of memory copy.
可以理解为，在将每个Grid地址映射至编码器时，如图10中所示出的右下角Grid存在内存不足的问题，因此，对于右下角Grid，进行切片时，可以进行内存拷贝，即额外分配内存来承载这一个Grid中的数据，以避免读取到非法内存，从而使编码器产生编码错误或者影响编码器的编码过程等。又例如，如图11所示，若输入图像的数据需占用的内存不被Grid高度整除，则在为输入图像分配内存时，可以将分配的存储内存的高度与Grid高度对齐，即分配的存储内存的高度被Grid高度整除。然后分配得到的存储内存的切分方式可以参阅前述图10，此处不再赘述。It can be understood that when mapping each Grid address to the encoder, there is a problem of insufficient memory for the Grid in the lower right corner as shown in Figure 10. Therefore, when slicing the Grid in the lower right corner, memory copying can be performed, that is Additional memory is allocated to carry the data in this Grid to avoid reading illegal memory, which may cause coding errors in the encoder or affect the encoding process of the encoder. For another example, as shown in Figure 11, if the memory occupied by the data of the input image is not divisible by the height of the Grid, when allocating memory for the input image, the height of the allocated storage memory can be aligned with the height of the Grid, that is, the allocated storage The memory height is divisible by the Grid height. Then, the division method of the allocated storage memory can refer to the above-mentioned FIG. 10 , which will not be repeated here.
还例如，如图12所示，若输入图像的数据需占用的内存高度不被Grid高度整除，则可以不为输入图像分配额外的内存，按照输入图像的实际尺寸分配存储内存。然后按照Grid大小直接对存储内存进行划分，得到的内存切片可以分为第一内存切片和第二内存切片，第一内存切片即高度为Grid高度的切片，第二内存切片即高度小于Grid高度的切片。然后为第二内存切片分配额外的内存(即第五内存)，然后将该第二内存切片中的数据拷贝至额外分配的内存中。然后将第一内存切片的地址和第五内存的地址均作为编码器的输入。Also for example, as shown in FIG. 12 , if the memory height required by the input image data is not divisible by the grid height, no additional memory may be allocated for the input image, and the storage memory may be allocated according to the actual size of the input image. Then, the storage memory is directly divided according to the size of the Grid. The obtained memory slices can be divided into the first memory slice and the second memory slice. The first memory slice is the slice whose height is the Grid height, and the second memory slice is the slice whose height is smaller than the Grid height slice. Then allocate additional memory (that is, the fifth memory) for the second memory slice, and then copy the data in the second memory slice to the additionally allocated memory. Then both the address of the first memory slice and the address of the fifth memory are used as the input of the encoder.
因此，本申请实施方式中，存储内存中被整除的部分可以通过地址映射的方式传入编码器，仅需为不被整除的部分分配内存进行内存拷贝即可，减少了内存拷贝量，减少了内存开销，提高了编码效率。Therefore, in the embodiment of the present application, the divisible part of the storage memory can be transmitted to the encoder through address mapping, and it is only necessary to allocate memory for the part that is not divisible for memory copying, which reduces the amount of memory copying and Memory overhead, improving coding efficiency.
605、HEVC编码器编码605. HEVC encoder encoding
在进行HEVC编码器收到每一个Grid的配置信息后，如每个Grid的起始地址、长度等地信息，即可进行HEVC编码。该配置信息可以包括每个Grid的起始地址、宽度或者长度等信息，从而使编码器可以读取每个Grid中的数据并进行编码。After the HEVC encoder receives the configuration information of each Grid, such as the starting address and length of each Grid, it can perform HEVC encoding. The configuration information may include information such as the start address, width or length of each Grid, so that the encoder can read and encode data in each Grid.
例如，如针对每个Grid，接收到的配置信息如下：For example, for each Grid, the received configuration information is as follows:
Start＝Grid内存起始地址；Start＝Grid memory start address;
Stride＝Input Buffer的stride；Stride = stride of Input Buffer;
Width＝Grid width；Width = Grid width;
Height＝Grid heightHeight＝Grid height
其中，Stride表示内存的宽度，Width表示每个Grid的宽度，Height表示每个Grid的高度。Among them, Stride represents the width of the memory, Width represents the width of each Grid, and Height represents the height of each Grid.
例如，以一帧分辨率为4032*3024的图像为例，分别通过内存拷贝和本申请提供的内存地址映射的方式来进行编码，通过计算编码耗时来进行对比。HEIC编码耗时＝HEIC完成编码的时刻(所有Grid完成HEVC编码，上报EOS)-HEIC开始编码的时刻(HEIC编码器收到原始图像的时刻)。通过内存拷贝的方式耗时为80ms以上，而通过本申请提供的编码 方式，则最低仅需45ms，大大提高了编码效率。当然，以上为基于某一芯片测试,不同的硬件耗时数据可能会不相同。For example, taking an image with a frame resolution of 4032*3024 as an example, the encoding is performed by means of memory copy and memory address mapping provided by this application, and the comparison is made by calculating the encoding time. HEIC encoding time = the moment when HEIC completes encoding (all grids complete HEVC encoding and report EOS) - the moment when HEIC starts encoding (the moment when the HEIC encoder receives the original image). The memory copy method takes more than 80ms, while the encoding method provided by this application only needs 45ms at least, which greatly improves the encoding efficiency. Of course, the above is based on a certain chip test, and the time-consuming data of different hardware may be different.
因此，本申请实施方式中，可以将图像的存储内存进行切分，并将切分得到的Grid地址映射至编码器中，因此可以减少额外分配的内存，减少内存开销，提高编码效率。进一步地，在进行编码时，为了避免编码器读取到非法内存地址，可以通过多种方式分配一定的内存，可以避免编码器读取到非法内存，从而使编码器产生编码错误或者影响编码器的编码过程等。Therefore, in the embodiment of the present application, the storage memory of the image can be segmented, and the grid address obtained by the segmentation can be mapped to the encoder, so that additional allocated memory can be reduced, memory overhead can be reduced, and encoding efficiency can be improved. Furthermore, when encoding, in order to prevent the encoder from reading illegal memory addresses, a certain amount of memory can be allocated in various ways, which can prevent the encoder from reading illegal memory, thereby causing encoding errors or affecting the encoder encoding process, etc.
前述对本申请提供的方法流程进行了介绍，下面对本申请提供的装置进行介绍。The foregoing has introduced the flow of the method provided by the present application, and the following will introduce the device provided by the present application.
参阅图13，本申请提供的一种图像编码装置的结构示意图，该图像编码装置用于执行前述图3-图12对应的方法步骤。该图像编码装置包括：Referring to FIG. 13 , it is a schematic structural diagram of an image coding device provided by the present application, and the image coding device is used to execute the method steps corresponding to the aforementioned FIGS. 3-12 . The image encoding device includes:
分配模块1301，用于为输入图像分配存储内存；An allocation module 1301, configured to allocate storage memory for the input image;
划分模块1302，用于将存储内存划分为多个内存切片；A division module 1302, configured to divide the storage memory into a plurality of memory slices;
编码模块1303，用于将多个内存切片中至少一个内存切片的地址输入至编码器，编码器用于基于输入的地址读取数据并进行编码，得到编码数据。The encoding module 1303 is configured to input the address of at least one memory slice in the plurality of memory slices to the encoder, and the encoder is configured to read and encode data based on the input address to obtain encoded data.
在一种可能的实施方式中，划分模块1302，具体用于按照预设切片大小，将存储内存划分为多个内存切片，预设切片大小包括预设高度和预设宽度。In a possible implementation manner, the division module 1302 is specifically configured to divide the storage memory into multiple memory slices according to a preset slice size, where the preset slice size includes a preset height and a preset width.
在一种可能的实施方式中，划分模块1302，具体用于：基于预设切片大小为输入图像分配存储内存。In a possible implementation manner, the division module 1302 is specifically configured to: allocate storage memory for the input image based on a preset slice size.
在一种可能的实施方式中，存储内存中包括第一内存和第二内存，第一内存中保存了输入图像中的各个像素点的信息，第二内存用于使存储内存的尺寸按照预设切片大小对齐，以使编码器以预设切片大小为单位从输入的地址中读取数据。In a possible implementation manner, the storage memory includes a first memory and a second memory, the first memory stores the information of each pixel in the input image, and the second memory is used to make the size of the storage memory according to the preset Slice size alignment so that the encoder reads data from the input address in units of the preset slice size.
在一种可能的实施方式中，存储内存按照阵列排列，存储内存的宽度被预设宽度整除，存储内存的高度被预设高度整除。In a possible implementation manner, the storage memory is arranged in an array, the width of the storage memory is divisible by a preset width, and the height of the storage memory is divisible by a preset height.
在一种可能的实施方式中，第二内存包括第三内存和第四内存，第一内存和第三内存组合后按照阵列排列，阵列的高度被预设高度整除，阵列的最后一行除以预设宽度后的余数与第四内存组合后不小于预设宽度。In a possible implementation manner, the second memory includes a third memory and a fourth memory, and the first memory and the third memory are combined and arranged in an array, the height of the array is divisible by the preset height, and the last row of the array is divided by the preset height. The remainder after setting the width is not smaller than the preset width after being combined with the fourth memory.
在一种可能的实施方式中，多个内存切片分为第一内存切片和第二内存切片，第一内存切片的高度为预设高度，第二内存切片高度小于预设高度或者第二内存切片为多个内存切片组成的阵列中的最后一行最后一个切片，第一内存切片和第二内存切片为不同的内存切片；In a possible implementation manner, multiple memory slices are divided into a first memory slice and a second memory slice, the height of the first memory slice is a preset height, and the height of the second memory slice is smaller than the preset height or the second memory slice It is the last slice of the last line in the array composed of multiple memory slices, the first memory slice and the second memory slice are different memory slices;
分配模块1301，还用于为多个内存切片中的第二内存切片分配第五内存，并将第二内存中的数据拷贝至第五内存中，第五内存的宽度被预设宽度整除，第五内存的高度被预设高度整除；The allocation module 1301 is further configured to allocate a fifth memory for the second memory slice among the plurality of memory slices, and copy the data in the second memory to the fifth memory. The width of the fifth memory is divisible by the preset width. The height of the five memory is divisible by the preset height;
编码模块1303，具体用于将多个内存切片中的第一内存切片的地址以及第五内存的地址输入至编码器。The encoding module 1303 is specifically configured to input the address of the first memory slice and the address of the fifth memory in the plurality of memory slices to the encoder.
在一种可能的实施方式中，存储内存的高度被预设高度整除。In a possible implementation manner, the height of the storage memory is evenly divisible by the preset height.
请参阅图14，本申请提供的另一种电子设备的结构示意图，如下所述。Please refer to FIG. 14 , which is a schematic structural diagram of another electronic device provided by the present application, as described below.
该电子设备可以包括前述的穿戴设备、终端或者车辆等，该电子设备可以包括处理器1401、存储器1402和收发器1403。该处理器1401和存储器1402通过线路互联。其中，存储器1402中存储有程序指令和数据。The electronic device may include the aforementioned wearable device, terminal, or vehicle, etc., and the electronic device may include a processor 1401 , a memory 1402 , and a transceiver 1403 . The processor 1401 and the memory 1402 are interconnected by wires. Wherein, program instructions and data are stored in the memory 1402 .
存储器1402中存储了前述图3-图12中的步骤对应的程序指令以及数据。The memory 1402 stores program instructions and data corresponding to the aforementioned steps in FIGS. 3-12 .
处理器1401用于执行前述图3-图12中任一实施例所示的第一设备或者电子设备执行的方法步骤。The processor 1401 is configured to execute the method steps performed by the first device or the electronic device shown in any one of the foregoing embodiments in FIG. 3 to FIG. 12 .
收发器1403，用于执行前述图3-图12中任一实施例所示的第一设备或者电子设备执行的接收或者发送数据的步骤。The transceiver 1403 is configured to execute the step of receiving or sending data performed by the first device or the electronic device shown in any one of the embodiments in FIG. 3 to FIG. 12 .
本申请实施例中还提供一种计算机可读存储介质，该计算机可读存储介质中存储有用于生成车辆行驶速度的程序，当其在计算机上运行时，使得计算机执行如前述图3-图12所示实施例描述的方法中的步骤。The embodiment of the present application also provides a computer-readable storage medium, the computer-readable storage medium is stored with a program for generating the driving speed of the vehicle. The illustrated embodiments describe steps in methods.
可选地，前述的图14中所示的电子设备为芯片。Optionally, the aforementioned electronic device shown in FIG. 14 is a chip.
本申请实施例还提供了一种电子设备，该电子设备也可以称为数字处理芯片或者芯片，芯片包括处理单元和通信接口，处理单元通过通信接口获取程序指令，程序指令被处理单元执行，处理单元用于执行前述图3-图12中任一实施例所示的电子设备执行的方法步骤。The embodiment of the present application also provides an electronic device, which can also be called a digital processing chip or a chip. The chip includes a processing unit and a communication interface. The processing unit obtains program instructions through the communication interface. The program instructions are executed by the processing unit. The unit is configured to execute the method steps executed by the electronic device shown in any one of the foregoing embodiments in FIG. 3 to FIG. 12 .
本申请实施例还提供一种数字处理芯片。该数字处理芯片中集成了用于实现上述处理器1401，或者处理器1401的功能的电路和一个或者多个接口。当该数字处理芯片中集成了存储器时，该数字处理芯片可以完成前述实施例中的任一个或多个实施例的方法步骤。当该数字处理芯片中未集成存储器时，可以通过通信接口与外置的存储器连接。该数字处理芯片根据外置的存储器中存储的程序代码来实现上述实施例中电子设备执行的动作。The embodiment of the present application also provides a digital processing chip. The digital processing chip integrates circuits and one or more interfaces for realizing the above processor 1401 or the functions of the processor 1401 . When the digital processing chip is integrated with a memory, the digital processing chip can complete the method steps in any one or more of the foregoing embodiments. When no memory is integrated in the digital processing chip, it can be connected to an external memory through a communication interface. The digital processing chip implements the actions performed by the electronic device in the above-mentioned embodiments according to the program code stored in the external memory.
本申请实施例中还提供一种包括计算机程序产品，当其在计算机上行驶时，使得计算机执行如前述图3-图12所示实施例描述的方法中电子设备所执行的步骤。The embodiment of the present application also provides a computer program product that, when running on a computer, causes the computer to execute the steps performed by the electronic device in the method described in the foregoing embodiments shown in FIGS. 3-12 .
本申请实施例提供的电子设备可以为芯片，芯片包括：处理单元和通信单元，所述处理单元例如可以是处理器，所述通信单元例如可以是输入/输出接口、管脚或电路等。该处理单元可执行存储单元存储的计算机执行指令，以使服务器内的芯片执行上述图3-图12所示实施例描述的设备搜寻方法。可选地，所述存储单元为所述芯片内的存储单元，如寄存器、缓存等，所述存储单元还可以是所述无线接入设备端内的位于所述芯片外部的存储单元，如只读存储器(read-only memory，ROM)或可存储静态信息和指令的其他类型的静态存储设备，随机存取存储器(random access memory，RAM)等。The electronic device provided in the embodiment of the present application may be a chip, and the chip includes: a processing unit and a communication unit, the processing unit may be, for example, a processor, and the communication unit may be, for example, an input/output interface, a pin, or a circuit. The processing unit can execute the computer-executed instructions stored in the storage unit, so that the chip in the server executes the device search method described in the embodiments shown in FIGS. 3-12 above. Optionally, the storage unit is a storage unit in the chip, such as a register, a cache, etc., and the storage unit may also be a storage unit located outside the chip in the wireless access device, such as only Read-only memory (ROM) or other types of static storage devices that can store static information and instructions, random access memory (random access memory, RAM), etc.
具体地，前述的处理单元或者处理器可以是中央处理器(central processing unit，CPU)、网络处理器(neural-network processing unit，NPU)、图形处理器(graphics processing unit，GPU)、数字信号处理器(digital signal processor，DSP)、专用集成电路(application specific integrated circuit,ASIC)或现场可编程逻辑门阵列(field programmable gate array，FPGA)或者其他可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件等。通用处理器可以是微处理器或者也可以是任何常规的处理器等。Specifically, the aforementioned processing unit or processor may be a central processing unit (central processing unit, CPU), a network processor (neural-network processing unit, NPU), a graphics processing unit (graphics processing unit, GPU), digital signal processing (digital signal processor, DSP), application specific integrated circuit (ASIC) or field programmable logic gate array (field programmable gate array, FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor or the like.
其中，上述任一处提到的处理器，可以是一个通用中央处理器，微处理器，ASIC，或 一个或多个用于控制上述图3-图12的方法的程序执行的集成电路。Wherein, the processor mentioned in any of the above-mentioned places may be a general-purpose central processing unit, a microprocessor, an ASIC, or one or more integrated circuits used to control the program execution of the above-mentioned methods in FIGS. 3-12 .
另外需说明的是，以上所描述的装置实施例仅仅是示意性的，其中所述作为分离部件说明的单元可以是或者也可以不是物理上分开的，作为单元显示的部件可以是或者也可以不是物理单元，即可以位于一个地方，或者也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部模块来实现本实施例方案的目的。另外，本申请提供的装置实施例附图中，模块之间的连接关系表示它们之间具有通信连接，具体可以实现为一条或多条通信总线或信号线。In addition, it should be noted that the device embodiments described above are only illustrative, and the units described as separate components may or may not be physically separated, and the components shown as units may or may not be A physical unit can be located in one place, or it can be distributed to multiple network units. Part or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. In addition, in the drawings of the device embodiments provided in the present application, the connection relationship between the modules indicates that they have communication connections, which can be specifically implemented as one or more communication buses or signal lines.
通过以上的实施方式的描述，所属领域的技术人员可以清楚地了解到本申请可借助软件加必需的通用硬件的方式来实现，当然也可以通过专用硬件包括专用集成电路、专用CPU、专用存储器、专用元器件等来实现。一般情况下，凡由计算机程序完成的功能都可以很容易地用相应的硬件来实现，而且，用来实现同一功能的具体硬件结构也可以是多种多样的，例如模拟电路、数字电路或专用电路等。但是，对本申请而言更多情况下软件程序实现是更佳的实施方式。基于这样的理解，本申请的技术方案本质上或者说对现有技术做出贡献的部分可以以软件产品的形式体现出来，该计算机软件产品存储在可读取的存储介质中，如计算机的软盘、U盘、移动硬盘、只读存储器(read only memory，ROM)、随机存取存储器(random access memory，RAM)、磁碟或者光盘等，包括若干指令用以使得一台计算机设备(可以是个人计算机，服务器，或者网络设备等)执行本申请各个实施例所述的方法。Through the description of the above embodiments, those skilled in the art can clearly understand that the present application can be implemented by means of software plus necessary general-purpose hardware, and of course it can also be realized by special hardware including application-specific integrated circuits, dedicated CPUs, dedicated memories, Special components, etc. to achieve. In general, all functions completed by computer programs can be easily realized by corresponding hardware, and the specific hardware structure used to realize the same function can also be varied, such as analog circuits, digital circuits or special-purpose circuit etc. However, for this application, software program implementation is a better implementation mode in most cases. Based on this understanding, the essence of the technical solution of this application or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product is stored in a readable storage medium, such as a floppy disk of a computer , U disk, mobile hard disk, read only memory (read only memory, ROM), random access memory (random access memory, RAM), magnetic disk or optical disk, etc., including several instructions to make a computer device (which can be a personal A computer, server, or network device, etc.) executes the methods described in various embodiments of the present application.
在上述实施例中，可以全部或部分地通过软件、硬件、固件或者其任意组合来实现。当使用软件实现时，可以全部或部分地以计算机程序产品的形式实现。In the above embodiments, all or part of them may be implemented by software, hardware, firmware or any combination thereof. When implemented using software, it may be implemented in whole or in part in the form of a computer program product.
所述计算机程序产品包括一个或多个计算机指令。在计算机上加载和执行所述计算机程序指令时，全部或部分地产生按照本申请实施例所述的流程或功能。所述计算机可以是通用计算机、专用计算机、计算机网络、或者其他可编程装置。所述计算机指令可以存储在计算机可读存储介质中，或者从一个计算机可读存储介质向另一计算机可读存储介质传输，例如，所述计算机指令可以从一个网站站点、计算机、服务器或数据中心通过有线(例如同轴电缆、光纤、数字用户线(DSL))或无线(例如红外、无线、微波等)方式向另一个网站站点、计算机、服务器或数据中心进行传输。所述计算机可读存储介质可以是计算机能够存储的任何可用介质或者是包含一个或多个可用介质集成的服务器、数据中心等数据存储设备。所述可用介质可以是磁性介质，(例如，软盘、硬盘、磁带)、光介质(例如，DVD)、或者半导体介质(例如固态硬盘(solid state disk，SSD))等。The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on the computer, the processes or functions according to the embodiments of the present application will be generated in whole or in part. The computer can be a general purpose computer, a special purpose computer, a computer network, or other programmable devices. The computer instructions may be stored in or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from a website, computer, server, or data center Transmission to another website site, computer, server, or data center by wired (eg, coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (eg, infrared, wireless, microwave, etc.). The computer-readable storage medium may be any available medium that can be stored by a computer, or a data storage device such as a server or a data center integrated with one or more available media. The available medium may be a magnetic medium (such as a floppy disk, a hard disk, or a magnetic tape), an optical medium (such as a DVD), or a semiconductor medium (such as a solid state disk (solid state disk, SSD)), etc.
本申请的说明书和权利要求书及上述附图中的术语“第一”、“第二”、“第三”、“第四”等(如果存在)是用于区别类似的对象，而不必用于描述特定的顺序或先后次序。应该理解这样使用的数据在适当情况下可以互换，以便这里描述的实施例能够以除了在这里图示或描述的内容以外的顺序实施。此外，术语“包括”和“具有”以及他们的任何变形，意图在于覆盖不排他的包含，例如，包含了一系列步骤或单元的过程、方法、***、产品或设备不必限于清楚地列出的那些步骤或单元，而是可包括没有清楚地列出的或对于这些过程、方法、产品或设备固有的其它步骤或单元。The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present application and the above drawings are used to distinguish similar objects, and not necessarily Used to describe a specific sequence or sequence. It is to be understood that the terms so used are interchangeable under appropriate circumstances such that the embodiments described herein can be practiced in sequences other than those illustrated or described herein. Furthermore, the terms "comprising" and "having", as well as any variations thereof, are intended to cover a non-exclusive inclusion, for example, a process, method, system, product or device comprising a sequence of steps or elements is not necessarily limited to the expressly listed instead, may include other steps or elements not explicitly listed or inherent to the process, method, product or apparatus.
最后应说明的是：以上，仅为本申请的具体实施方式，但本申请的保护范围并不局限于此，任何熟悉本技术领域的技术人员在本申请揭露的技术范围内，可轻易想到变化或替换，都应涵盖在本申请的保护范围之内。因此，本申请的保护范围应以权利要求的保护范围为准。Finally, it should be noted that: the above is only a specific embodiment of the application, but the scope of protection of the application is not limited thereto, and any skilled person familiar with the technical field can easily think of changes within the technical scope disclosed in the application Or replacement, should be covered within the protection scope of this application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims (20)

  1. 一种图像编码方法，其特征在于，包括：An image coding method, characterized in that, comprising:
    为输入图像分配存储内存；Allocate storage memory for the input image;
    将所述存储内存划分为多个内存切片；dividing the storage memory into a plurality of memory slices;
    通过编码器从所述多个内存切片中至少一个内存切片的地址读取数据并进行编码，得到编码数据。An encoder reads data from an address of at least one memory slice in the plurality of memory slices and performs encoding to obtain encoded data.
  2. 根据权利要求1所述的方法，其特征在于，所述将所述存储内存划分为多个内存切片存储内存，包括：The method according to claim 1, wherein said dividing said storage memory into a plurality of memory slice storage memory comprises:
    按照预设切片大小，将所述存储内存划分为所述多个内存切片，所述预设切片大小包括预设高度和预设宽度。The storage memory is divided into the plurality of memory slices according to a preset slice size, where the preset slice size includes a preset height and a preset width.
  3. 根据权利要求2所述的方法，其特征在于，所述为输入图像分配存储内存，包括：The method according to claim 2, wherein the allocating storage memory for the input image comprises:
    基于所述预设切片大小为所述输入图像分配所述存储内存。Allocating the storage memory for the input image based on the preset slice size.
  4. 根据权利要求3所述的方法，其特征在于，所述存储内存中包括第一内存和第二内存，所述第一内存中保存了所述输入图像中的各个像素点的信息，所述第二内存用于使所述存储内存的尺寸按照所述预设切片大小对齐，以使所述编码器以所述预设切片大小为单位从输入的地址中读取数据。The method according to claim 3, wherein the storage memory includes a first memory and a second memory, the first memory stores information of each pixel in the input image, and the second memory The second memory is used to align the size of the storage memory according to the preset slice size, so that the encoder reads data from the input address in units of the preset slice size.
  5. 根据权利要求4所述的方法，其特征在于，所述存储内存按照阵列排列，所述存储内存的宽度被所述预设宽度整除，所述存储内存的高度被所述预设高度整除。The method according to claim 4, wherein the storage memory is arranged in an array, the width of the storage memory is divisible by the preset width, and the height of the storage memory is divisible by the preset height.
  6. 根据权利要求4所述的方法，其特征在于，所述第二内存包括第三内存和第四内存，所述第一内存和所述第三内存组合后按照阵列排列，所述阵列的高度被所述预设高度整除，所述阵列的最后一行除以所述预设宽度后的余数与第四内存组合后不小于所述预设宽度。The method according to claim 4, wherein the second memory includes a third memory and a fourth memory, the first memory and the third memory are combined and arranged in an array, and the height of the array is determined by The preset height is evenly divisible, and the remainder after dividing the last row of the array by the preset width is not smaller than the preset width after being combined with the fourth memory.
  7. 根据权利要求2或3所述的方法，其特征在于，所述存储内存按照阵列排列，所述多个内存切片分为第一内存切片和第二内存切片，所述第一内存切片的高度为所述预设高度，所述第二内存切片高度小于所述预设高度或者所述第二内存切片为所述多个内存切片组成的阵列中的最后一行最后一个切片，所述第一内存切片和所述第二内存切片为不同的内存切片；The method according to claim 2 or 3, wherein the storage memory is arranged in an array, the plurality of memory slices are divided into a first memory slice and a second memory slice, and the height of the first memory slice is The preset height, the height of the second memory slice is smaller than the preset height or the second memory slice is the last slice in the last row in the array composed of the plurality of memory slices, the first memory slice and the second memory slice are different memory slices;
    所述方法还包括：The method also includes:
    为所述多个内存切片中的第二内存切片分配第五内存，并将所述第二内存中的数据拷贝至所述第五内存中，所述第五内存的宽度被所述预设宽度整除，所述第五内存的高度被所述预设高度整除；Allocating a fifth memory for the second memory slice in the plurality of memory slices, and copying the data in the second memory to the fifth memory, the width of the fifth memory being equal to the preset width divisible, the height of the fifth memory is divisible by the preset height;
    所述通过编码器从所述多个内存切片中至少一个内存切片的地址读取数据并进行编码，得到所述编码数据，包括：The coder reads data from the address of at least one memory slice in the plurality of memory slices and performs encoding to obtain the encoded data, including:
    通过编码器从所述多个内存切片中的第一内存切片的地址以及所述第五内存的地址读取数据并进行编码，得到所述编码数据。The encoded data is obtained by an encoder reading data from the address of the first memory slice and the address of the fifth memory among the plurality of memory slices and performing encoding.
  8. 根据权利要求7所述的方法，其特征在于，所述存储内存的高度被所述预设高度整除。The method according to claim 7, wherein the height of the storage memory is divisible by the preset height.
  9. 一种图像编码装置，其特征在于，包括：An image encoding device, characterized in that it comprises:
    分配模块，用于为输入图像分配存储内存；An allocation module for allocating storage memory for the input image;
    划分模块，用于将所述存储内存划分为多个内存切片；A division module, configured to divide the storage memory into multiple memory slices;
    编码模块，用于通过编码器从所述多个内存切片中至少一个内存切片的地址读取数据并进行编码，得到编码数据。An encoding module, configured to read data from an address of at least one memory slice in the plurality of memory slices through an encoder and perform encoding to obtain encoded data.
  10. 根据权利要求9所述的装置，其特征在于，The device according to claim 9, characterized in that,
    所述划分模块，具体用于按照预设切片大小，将所述存储内存划分为所述多个内存切片，所述预设切片大小包括预设高度和预设宽度。The dividing module is specifically configured to divide the storage memory into the plurality of memory slices according to a preset slice size, where the preset slice size includes a preset height and a preset width.
  11. 根据权利要求10所述的装置，其特征在于，所述划分模块，具体用于：基于所述预设切片大小为所述输入图像分配所述存储内存。The device according to claim 10, wherein the dividing module is specifically configured to: allocate the storage memory for the input image based on the preset slice size.
  12. 根据权利要求11所述的装置，其特征在于，The device according to claim 11, characterized in that,
    所述存储内存中包括第一内存和第二内存，所述第一内存中保存了所述输入图像中的各个像素点的信息，所述第二内存用于使所述存储内存的尺寸按照所述预设切片大小对齐，以使所述编码器以所述预设切片大小为单位从输入的地址中读取数据。The storage memory includes a first memory and a second memory, the first memory stores the information of each pixel in the input image, and the second memory is used to make the size of the storage memory according to the specified The preset slice size is aligned so that the encoder reads data from the input address in units of the preset slice size.
  13. 根据权利要求12所述的装置，其特征在于，所述存储内存按照阵列排列，所述存储内存的宽度被所述预设宽度整除，所述存储内存的高度被所述预设高度整除。The device according to claim 12, wherein the storage memory is arranged in an array, the width of the storage memory is divisible by the preset width, and the height of the storage memory is divisible by the preset height.
  14. 根据权利要求12所述的装置，其特征在于，所述第二内存包括第三内存和第四内存，所述第一内存和所述第三内存组合后按照阵列排列，所述阵列的高度被所述预设高度整除，所述阵列的最后一行除以所述预设宽度后的余数与第四内存组合后不小于所述预设宽度。The device according to claim 12, wherein the second memory includes a third memory and a fourth memory, the first memory and the third memory are combined and arranged in an array, and the height of the array is determined by The preset height is evenly divisible, and the remainder after dividing the last row of the array by the preset width is not smaller than the preset width after being combined with the fourth memory.
  15. 根据权利要求10或11所述的装置，其特征在于，所述多个内存切片分为第一内存切片和第二内存切片，所述第一内存切片的高度为所述预设高度，所述第二内存切片高度小于所述预设高度或者所述第二内存切片为所述多个内存切片组成的阵列中的最后一行最后一个切片，所述第一内存切片和所述第二内存切片为不同的内存切片；The device according to claim 10 or 11, wherein the plurality of memory slices are divided into a first memory slice and a second memory slice, the height of the first memory slice is the preset height, and the The height of the second memory slice is less than the preset height or the second memory slice is the last slice in the last row in the array composed of the plurality of memory slices, the first memory slice and the second memory slice are different memory slices;
    所述分配模块，还用于为所述多个内存切片中的第二内存切片分配第五内存，并将所述第二内存中的数据拷贝至所述第五内存中，所述第五内存的宽度被所述预设宽度整除，所述第五内存的高度被所述预设高度整除；The allocation module is further configured to allocate a fifth memory for a second memory slice among the plurality of memory slices, and copy data in the second memory to the fifth memory, and the fifth memory The width of the fifth memory is divisible by the preset width, and the height of the fifth memory is divisible by the preset height;
    所述编码模块，具体用于通过编码器从所述多个内存切片中的第一内存切片的地址以及所述第五内存的地址读取数据并进行编码，得到所述编码数据。The encoding module is specifically configured to use an encoder to read data from the address of the first memory slice and the address of the fifth memory among the plurality of memory slices and perform encoding to obtain the encoded data.
  16. 根据权利要求15所述的装置，其特征在于，所述存储内存的高度被所述预设高度整除。The device according to claim 15, wherein the height of the storage memory is divisible by the preset height.
  17. 一种电子设备，其特征在于，包括一个或多个处理器，所述一个或多个处理器和存储器耦合，所述存储器存储有程序，当所述存储器存储的程序指令被所述一个或多个处理器执行时实现权利要求1至8中任一项所述的方法的步骤。An electronic device, characterized in that it includes one or more processors, the one or more processors are coupled to a memory, the memory stores a program, and when the program instructions stored in the memory are used by the one or more When a processor is executed, the steps of the method according to any one of claims 1 to 8 are realized.
  18. 一种计算机可读存储介质，其特征在于，包括程序，当其被处理单元所执行时， 执行如权利要求1至8中任一项所述的方法。A computer-readable storage medium, characterized by comprising a program, which, when executed by a processing unit, executes the method according to any one of claims 1 to 8.
  19. 一种计算机程序产品，包括计算机程序/指令，其特征在于，所述计算机程序/指令被处理器执行时实现如权利要求1至8中任一项所述方法的步骤。A computer program product, comprising computer programs/instructions, characterized in that, when the computer program/instructions are executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.
  20. 一种芯片，其特征在于，所述芯片包括处理单元和通信接口，所述处理单元通过所述通信接口获取程序指令，程序指令被所述处理单元执行，所述处理单元用于执行如权利要求1至8中任一项所述方法的步骤。A chip, characterized in that the chip includes a processing unit and a communication interface, the processing unit obtains program instructions through the communication interface, the program instructions are executed by the processing unit, and the processing unit is used to perform the The step of any one of 1 to 8 methods.
PCT/CN2022/142277 2021-12-29 2022-12-27 Image encoding method and device WO2023125518A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202111640001.9A CN116418994A (en) 2021-12-29 2021-12-29 Image coding method and device
CN202111640001.9 2021-12-29

Publications (1)

Publication Number Publication Date
WO2023125518A1 true WO2023125518A1 (en) 2023-07-06

Family

ID=86997926

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/142277 WO2023125518A1 (en) 2021-12-29 2022-12-27 Image encoding method and device

Country Status (2)

Country Link
CN (1) CN116418994A (en)
WO (1) WO2023125518A1 (en)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117094879B (en) * 2023-10-18 2024-03-26 南京砺算科技有限公司 Data copying method and device, computer readable storage medium and electronic equipment

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20100061464A1 (en) * 2008-06-03 2010-03-11 Fujitsu Limited Moving picture decoding apparatus and encoding apparatus
CN104216835A (en) * 2014-08-25 2014-12-17 杨立群 Method and device for implementing memory fusion
CN108496364A (en) * 2017-06-21 2018-09-04 深圳市大疆创新科技有限公司 Video transmission method, video processor, network processing unit and video transmission equipment
CN111538677A (en) * 2020-04-26 2020-08-14 西安万像电子科技有限公司 Data processing method and device

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20100061464A1 (en) * 2008-06-03 2010-03-11 Fujitsu Limited Moving picture decoding apparatus and encoding apparatus
CN104216835A (en) * 2014-08-25 2014-12-17 杨立群 Method and device for implementing memory fusion
CN108496364A (en) * 2017-06-21 2018-09-04 深圳市大疆创新科技有限公司 Video transmission method, video processor, network processing unit and video transmission equipment
CN111538677A (en) * 2020-04-26 2020-08-14 西安万像电子科技有限公司 Data processing method and device

Also Published As

Publication number Publication date
CN116418994A (en) 2023-07-11

Similar Documents

Publication Publication Date Title
CN109559270B (en) Image processing method and electronic equipment
CN113448382B (en) Multi-screen display electronic device and multi-screen display method of electronic device
US20230247093A1 (en) Data Synchronization Method, Electronic Device, and Computer-Readable Storage Medium
WO2021047567A1 (en) Callback stream processing method and device
WO2021077878A1 (en) Image processing method and apparatus, and electronic device
WO2022100685A1 (en) Drawing command processing method and related device therefor
EP4361954A1 (en) Object reconstruction method and related device
CN114461375B (en) Memory resource management method and electronic equipment
WO2022222924A1 (en) Method for adjusting screen projection display parameters
WO2022062809A1 (en) Screen projection control method and apparatus
CN114727220B (en) Equipment searching method and electronic equipment
WO2023125518A1 (en) Image encoding method and device
CN113973398A (en) Wireless network connection method, electronic equipment and chip system
CN113126948A (en) Audio playing method and related equipment
CN116389884B (en) Thumbnail display method and terminal equipment
CN115119048B (en) Video stream processing method and electronic equipment
WO2023016059A1 (en) Data transmission control method and related apparatus
WO2023051094A1 (en) Memory recovery method and apparatus, electronic device, and readable storage medium
WO2022121988A1 (en) Display synchronization method, electronic device, and readable storage medium
CN113901485B (en) Application program loading method, electronic device and storage medium
WO2022135195A1 (en) Method and apparatus for displaying virtual reality interface, device, and readable storage medium
WO2022170866A1 (en) Data transmission method and apparatus, and storage medium
WO2022170854A1 (en) Video call method and related device
CN115482143B (en) Image data calling method and system for application, electronic equipment and storage medium
CN114125805B (en) Bluetooth reconnection method and terminal equipment

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22914771

Country of ref document: EP

Kind code of ref document: A1