WO2016130325A1 - Systems and methods for power optimization for imaging devices with dual cameras - Google Patents
Systems and methods for power optimization for imaging devices with dual cameras Download PDFInfo
- Publication number
- WO2016130325A1 WO2016130325A1 PCT/US2016/015172 US2016015172W WO2016130325A1 WO 2016130325 A1 WO2016130325 A1 WO 2016130325A1 US 2016015172 W US2016015172 W US 2016015172W WO 2016130325 A1 WO2016130325 A1 WO 2016130325A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- zoom factor
- sensor
- frames
- factor
- series
- Prior art date
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/45—Cameras or camera modules comprising electronic image sensors; Control thereof for generating image signals from two or more image sensors being of different type or operating in different modes, e.g. with a CMOS sensor for moving images in combination with a charge-coupled device [CCD] for still images
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/60—Control of cameras or camera modules
- H04N23/63—Control of cameras or camera modules by using electronic viewfinders
- H04N23/633—Control of cameras or camera modules by using electronic viewfinders for displaying additional information relating to control or operation of the camera
- H04N23/635—Region indicators; Field of view indicators
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/60—Control of cameras or camera modules
- H04N23/65—Control of camera operation in relation to power supply
- H04N23/651—Control of camera operation in relation to power supply for reducing power consumption by affecting camera operations, e.g. sleep mode, hibernation mode or power off of selective parts of the camera
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/60—Control of cameras or camera modules
- H04N23/69—Control of means for changing angle of the field of view, e.g. optical zoom objectives or electronic zooming
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/90—Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums
Definitions
- the present application relates generally to imaging devices, and more particularly to systems, methods, and devices for power optimization in imaging devices with dual cameras.
- Dual cameras are being increasingly used in cell phones and other mobile devices.
- a first camera has a wide angle lens while the second camera has a telephoto lens.
- Running two cameras simultaneously increases power consumption and decreases battery life of the mobile device.
- systems and methods for dual camera applications focusing on power optimization for an optical zoom feature of a dual camera application are needed.
- Dual camera systems on a mobile device, or other device may include both a telephoto and wide-angle lens. In such dual camera systems, both cameras may be operated at the same time to capture an image of a scene. However, operating the dual cameras simultaneously can increase power consumption and therefore can decrease the battery life of a mobile device.
- Embodiments and innovations discussed herein relate to systems and methods that may be run in a processor for an electronic device to optimize the power utilization of devices having and/or operating dual cameras.
- the power optimization systems and methods discussed herein can be implemented in existing hardware and software.
- specially designed hardware and software may improve speed or efficiencies of such processes.
- a method for power optimization for a dual camera imaging device includes the steps of determining a zoom factor selection for capturing image data; determining whether the zoom factor selection falls within a first zoom factor range, a second zoom factor range, or a third zoom factor range, the first, second and third zoom factor ranges being an amount of zoom; and sending a series of frames of a scene captured by a first sensor or a series of frames of the scene captured by a second sensor or both to a camera application based on the determined zoom factor selection.
- the method further includes determining a buffer factor from a warm up period of the second sensor.
- the first, second, and third zoom factor ranges are determined using the buffer factor, a field of view (FOV) ratio between the first sensor and the second sensor and a maximum zoom factor of the imaging device.
- FOV ratio is a ratio between the FOV of the first sensor and a FOV of the second sensor and the FOV ratio is less than the maximum zoom factor of the imaging device.
- the first zoom factor range is between 1.0 and a difference between the FOV ratio and the buffer factor.
- the second zoom factor range is between a difference between the FOV ratio and the buffer factor and the sum of the field of view ratio and the buffer factor.
- the third zoom factor range is between the sum of the FOV ratio and the buffer factor and the maximum zoom factor.
- the method further includes turning off one of the first and second sensors when the other of the first and second sensors is sending frames to the camera application when the zoom factor is within either of the first and the third zoom factor ranges.
- the method further includes sending frames to the camera application from both of the first and second sensors when the zoom factor is within the second zoom factor range.
- an apparatus for optimizing operation of a dual camera imaging device includes a processor; a first imaging sensor and a second imaging sensor; and a memory, operably coupled to the processor, and configured to store processor instructions that configure the processor to determine a zoom factor selection; determine whether the zoom factor selection falls within a first zoom factor range, a second zoom factor range, or a third zoom factor range; and send a series of frames of a scene captured by a first sensor or a series of frames of the scene captured by a second sensor or both to a camera application based on the determined zoom factor section.
- the apparatus further includes a touch device configured to sense a touch on a touch interface.
- the memory further is configured to determine a buffer factor from a warm up period of the second sensor. In some embodiments, the memory further is configured to determine first, second, and third zoom factor ranges using the buffer factor, a field of view (FOV) ratio between the first sensor and the second sensor, and a maximum zoom factor of the imaging device.
- the first zoom factor range is between 1.0 and a difference between the FOV ratio and the buffer factor.
- the second zoom factor range is between a difference between the FOV ratio and the buffer factor and the sum of the FOV ratio and the buffer factor. In some embodiments, the third zoom factor range is between the sum of the FOV ratio and the buffer factor and the maximum zoom factor.
- the memory further is configured to turn off one of the first and second sensors when the other of the first and second sensors is sending frames to the camera application when the zoom factor is within either of the first and the third zoom factor ranges. In some embodiments, the memory further is configured to send frames to the camera application from both of the first and second sensors when the zoom factor is within the second zoom factor range.
- a system for optimizing operation of dual camera imaging device includes a control module configured to determine a zoom factor selection; determine whether the zoom factor selection falls within a first zoom factor range, a second zoom factor range, or a third zoom factor range; and send a series of frames of a scene captured by a first sensor or a series of frames of the scene captured by a second sensor or both to a camera application based on the determined zoom factor section.
- the system further includes a touch interface configured to receive a zoom factor selection.
- the control module is further configured to determine a buffer factor from a warm up period of the second sensor.
- control module further is configured to determine first, second, and third zoom factor ranges using the buffer factor, a field of view ratio between the first sensor and the second sensor, and a maximum zoom factor of the imaging device.
- first zoom factor range is between 1.0 and a difference between the field of view ratio and the buffer factor.
- second zoom factor range is between a difference between the field of view ratio and the buffer factor and the sum of the field of view ratio and the buffer factor.
- third zoom factor range is between the sum of the field of view ratio and the buffer factor and the maximum zoom factor.
- control module further is configured to turn off one of the first and second sensors when the other of the first and second sensors is sending frames to the camera application when the zoom factor is within either of the first and the third zoom factor ranges.
- the memory further is configured to send frames to the camera application from both of the first and second sensors when the zoom factor is within the second zoom factor range.
- a non-transitory computer-readable medium stores instructions that, when executed, cause at least one physical computer processor to perform a method for power optimization of a dual camera imaging device.
- the method includes the steps of determining a zoom factor selection; determining whether the zoom factor selection falls within a first zoom factor range, a second zoom factor range, or a third zoom factor range; and sending a series of frames of a scene captured by a first sensor or a series of frames of the scene captured by a second sensor or both to a camera application based on the determined zoom factor section.
- the method further includes determining a buffer factor from a warm up period of the second sensor.
- the method further includes turning off one of the first and second sensors when the other of the first and second sensors is sending frames to the camera application when the zoom factor is within either of the first and the third zoom factor ranges and sending frames to the camera application from both of the first and second sensors when the zoom factor is within the second zoom factor range.
- FIG. 1 illustrates two views of a scene as captured by a wide-angle camera and a telephoto camera of a dual camera device according to one embodiment.
- FIG. 2 illustrates an example implementation of an dual camera architecture, according to some embodiments, in which one camera stream may be displayed to a user at a time but both cameras can be operating simultaneously.
- FIG. 3 illustrates an example of a dual camera architecture, according to some embodiments, in which one camera stream may be displayed to a user at a time and the other camera may not be streamed.
- FIG. 4 illustrates another example of the dual camera architecture embodiment illustrated in FIG. 3 which includes a buffer zone in which both cameras may be streamed to reduce latency.
- FIG. 5 illustrates a block diagram of an example of a mobile imaging device in accordance with one embodiment.
- FIG. 6 illustrates one example of a user interface for selecting an optical zoom factor on a mobile imaging device.
- FIG. 7 is a flow chart illustrating an exemplary method of optimizing power on a mobile imaging device having a dual camera architecture using optical zoom selection.
- Embodiments relate to power optimization systems and methods for electronic devices that include dual cameras.
- Devices having dual cameras may include two camera modules and lens systems, for example, a wide-angle camera module having a wide-angle lens and a telephoto camera module having a telephoto lens.
- the wide- angle lens of the wide-angle camera module has a larger field of view (FOV) than the telephoto lens of the telephoto camera module at the same resolution and size (for example, width and height) of an image.
- the telephoto camera module is used to capture a narrow field of view of an object in the scene (or scene).
- An electronic device having dual cameras may operate both cameras simultaneously, thereby increasing power consumption and decreasing battery life of the device.
- the systems and methods disclosed herein relate to power optimization for dual cameras, wherein a control system selectively operates either the wide-angle or the telephoto camera module depending on desired optical zoom, while maintaining the overall resolution of the captured image.
- a control system selectively operates either the wide-angle or the telephoto camera module depending on desired optical zoom, while maintaining the overall resolution of the captured image.
- the examples may be described as a process, which is depicted as a flowchart, a flow diagram, a finite state diagram, a structure diagram, or a block diagram. Although a flowchart may describe the operations as a sequential process, many of the operations can be performed in parallel, or concurrently, and the process can be repeated. In addition, the order of the operations may be re-arranged.
- a process is terminated when its operations are completed.
- a process may correspond to a method, a function, a procedure, a subroutine, a subprogram, etc. When a process corresponds to a software function, its termination corresponds to a return
- Embodiments may be implemented in System-on-Chip (SoC) or external hardware, software, firmware, or any combination thereof.
- SoC System-on-Chip
- Those of skill in the art will understand that information and signals may be represented using any of a variety of different technologies and techniques.
- data, instructions, commands, information, signals, bits, symbols, and chips that may be referenced throughout the above description may be represented by voltages, currents, electromagnetic waves, magnetic fields or particles, optical fields or particles, or any combination thereof.
- Figure 1 illustrates two views of a scene as captured by a wide-angle lens of a dual camera device and a telephoto lens of a dual camera device (for example, a system as illustrated in Figure 5).
- a first image 100 was taken using the wide-angle lens and a second image 110 was taken using the telephoto lens, the second image 110 corresponding to a portion 115 of the first image.
- a feature such as optical zoom
- dual camera architectures may stream frames from both the wide-angle and the telephoto cameras even though only one is displayed.
- the wide-angle lens used to take the first image 100 has about twice as much field-of- view as the telephoto lens used to take the second image 110. While both cameras may stream frames to the image processor of the device, until the user inputs a 2x optical zoom factor, the image captured by the wide-angle lens will be displayed. For optical zoom factors greater than a 2x zoom factor, the image captured by the telephoto lens will be displayed.
- FIG. 2 illustrates an example of an implementation of a dual camera architecture in which one camera stream is displayed to a user at a time but both cameras are operating simultaneously.
- the output from a wide-angle sensor 202 and a telephoto sensor 204 are sent to separate video front end (VFE) image processing modules. That is, the wide-angle sensor 202 provides image information 203 to a VFE wide-angle image processing module 206, and the telephoto sensor 204 provides image information 205 to the VFE telephoto image processing module 208.
- the VFE wide-angle processing module 206 provides full resolution of the captured image information as output 220 to a video multiplexer 214, and provides a reduced (or smaller) resolution output 207 to a calibration offset estimation module 210.
- the VFE telephoto image processing module 208 provides a full resolution output 224 of the telephoto sensor data to the rectification/ Alignment GPU 212, which provides telephoto input data to the video multiplexer 214.
- the VFE wide angle processing module 206 also provides a reduced (or small) resolution output of the telephoto sensor 204 data to the calibration offset estimation module 210, which aligns the telephoto image data and the wide angle image data.
- the calibration offset estimation module 210 then provides an output to the rectification / alignment module 212 to provide alignment of the full resolution telephoto data (so that it is aligned with the wide angle data).
- the video multiplexer 214 selects either the wide angle sensor data input or the telephoto sensor data input based on a selected zoom factor, and outputs image data 222 (wide angle or telephoto data).
- image data 222 is then sent to processing module 216 for upscaling or downscaling and the resulting output 217 is sent via additional image processing module 218 to a display (not shown). Operation of both cameras simultaneously draws unnecessary power from one of the cameras even if the camera is not being displayed or streamed.
- streaming may refer to the CCD array of the camera sending all of the captured frames to the camera application to be displayed to the user.
- FIG. 3 is a schematic illustrating an example of a dual camera architecture embodiment.
- the image data output (or streamed) from both of the wide-angle camera 302 and the telephoto camera 318 are not used together to form the image data 330 which is ultimately displayed (or stored). That is, data from only one of the wide-angle camera 302 or the telephoto camera 318 is used to form the display data 330, or be saved in a memory component (not shown).
- the wide-angle camera 302 or the telephoto camera 318 that is producing an image data output (stream) that is not being displayed can be turned off to reduce the overall power consumption of the device and thus reduce the overall system load.
- the streams of each of the cameras 302, 318 are sent to separate video front end modules 304, 320 respectively. Outputs from the video front end modules 304, 320 are sent to a video multiplexer module 306.
- the choice of which camera stream to display to the user is determined at position 308 based on the selected optical zoom factor of the imaging system.
- the zoom factor can range between l .Ox to Fx, where F is the maximum zoom factor.
- the output 310 from the wide-angle camera is streamed when the selected optical zoom factor is between l .Ox and Mx, where M is the FOV ratio between the wide-angle lens and the telephoto lens and M ⁇ F.
- the output 328 from the telephoto camera is streamed.
- M the FOV ratio between the wide-angle lens and the telephoto lens, may be, for example between 1.5x and 4. Ox.
- M is approximately 2.
- F the maximum zoom factor
- M is approximately 6.
- Both of the wide-angle and telephoto camera streams 310 and 328 may be passed through other image processing modules, for example, modules 314 and 316, before being output 330 and then displayed or stored.
- the full resolution image stream 326 from the telephoto camera may be provided to a rectification and alignment module 312 prior to the selection of the image stream at position 308.
- the telephoto camera may incur a delay in producing a scene image stream when the user selects the Mx zoom factor. Therefore, this embodiment introduces a period of delay in operation that may be undesirable.
- Figure 4 illustrates a flow diagram of another alternative to the dual camera architecture illustrated in Figure 2.
- the wide-angle camera 402 and the telephoto camera 416 are not used or displayed at the same time and therefore the stream from the camera not being displayed can be turned off to reduce the power consumption of the device.
- the streams of each camera 402, 416 are sent to separate video front end modules 404, 418 for processing and then to the video multiplexer module 406.
- the choice of which camera stream 408, 422, to display to the user is determined at position 410 based on the selected optical zoom factor.
- the zoom factor can range between l .Ox to Fx, where F is a maximum zoom factor.
- F may be, for example, between 4.0x and 7. Ox. In other embodiments, F is approximately 6. Ox.
- one limitation to streaming one camera stream at a time is the lag or delay caused by the telephoto camera's inability to start immediately at the Mx zoom factor.
- a buffer factor between 1.0 and Fx is a programmable factor which varies based on the telephoto camera sensor's warm up period.
- B a buffer factor between 1.0 and Fx
- both the wide angle camera and the telephoto cameras are streamed when the zoom factor is in the range of (Mx - Bx) to (Mx + Bx).
- the output 408 from the wide-angle camera is streamed when the selected optical zoom factor is between l .Ox and (Mx - Bx).
- the selected zoom factor is between (Mx + Bx) and Fx
- the output 422 from the telephoto camera is streamed.
- Both of the camera streams 408 and 422 may be passed through other image processing modules, such as modules 412 and 414, before being sent to a display.
- the full resolution image stream 420 from the telephoto camera may be sent to a rectification / alignment module 424 to provide alignment of the full resolution telephoto data prior to selection of the displayed image stream.
- the zoom range buffer zone (Mx - Bx) to (Mx + Bx) preferably absorbs any latency caused by delay in start up of the telephoto camera.
- the determination of B is based on the sensor start up scheme implemented. For example, if the wide-angle or telephoto sensor is simply on standby then B can be a small value, such as 0.5x.
- the value of B is set to a higher value, such as l .Ox. Setting B to a higher value increases the range at which data from both the wide-angle and the telephoto cameras are streamed, reducing the latency involved in sensor start-up.
- FIG. 5 illustrates various components of a mobile device 500 with two camera sensors 502, 504 in accordance with one embodiment of the present disclosure which could perform the optical zoom optimization techniques described herein.
- the mobile device 500 may be configured to capture, create, process, modify, scale, encode, decode, transmit, store, and display digital images and/or video sequences.
- the mobile device 500 may represent or be implemented in a device, such as a wireless communication device, a personal digital assistant (PDA), a laptop computer, a desktop computer, a digital camera, a digital recording device, a network-enabled digital television, a mobile phone, a cellular phone, a satellite telephone, a camera phone, a terrestrial-based radiotelephone, a direct two-way communication device (sometimes referred to as a "walkie-talkie"), etc.
- PDA personal digital assistant
- laptop computer a desktop computer
- a digital camera a digital recording device
- a network-enabled digital television such as a mobile phone, a cellular phone, a satellite telephone, a camera phone, a terrestrial-based radiotelephone, a direct two-way communication device (sometimes referred to as a "walkie-talkie"), etc.
- PDA personal digital assistant
- laptop computer a desktop computer
- a digital camera a digital recording device
- network-enabled digital television such as a
- the mobile device 500 may include a first imaging sensor 502, a second imaging sensor 504, a memory 510, a processor 528, a working memory 520, and a display 525.
- the memory 510 may include various imaging processing modules, including a camera process pipeline 530, a user interface module 550, and operating system 555.
- the architecture illustrated in Figure 1 is merely an example. The features and techniques described herein may be implemented with a variety of other architectures.
- the sensors 502, 504 may be digital camera sensors. In some embodiments, one of the sensors 502 may be a wide-angle sensor and the other sensor 504 may be a telephoto sensor. The sensors 502, 504 may capture still image snapshots and/or video sequences. Each sensor may include color filter arrays (CFAs) arranged on a surface of individual sensors or sensor elements.
- CFAs color filter arrays
- the memory 510 and working memory 520 may store images or video sequences before and after processing.
- the memory 510 and working memory 520 may include volatile storage and non-volatile storage.
- the memory 510 and working memory 520 may comprise any type of data storage means, such as dynamic random access memory (DRAM), FLASH memory, NOR or NAND gate memory, or any other data storage technology.
- DRAM dynamic random access memory
- FLASH memory FLASH memory
- NOR or NAND gate memory any other data storage technology.
- the program memory 510 stores a camera process pipeline 530, a user interface module 550, and operating system 555. These modules may include instructions that configure the processor 528 to perform various image processing and device management tasks.
- the program memory 510 can be any suitable computer-readable storage medium, such as a non-transitory storage medium.
- Working memory 520 may be used by processor 528 to store a working set of processor instructions contained in the modules of program memory 510. Alternatively, working memory 520 may also be used by processor 528 to store dynamic data created during the operation of the device 500.
- the camera process pipeline 530 may comprise a chip set for a mobile phone, which may include hardware, software, firmware, and/or one or more microprocessors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), or various combinations thereof.
- the camera process pipeline 530 may perform one or more image processing techniques to improve quality of an image and/or a video sequence.
- the pipeline 530 may perform techniques such as demosaicing, lens rolloff correction, scaling, color correction, color conversion, and spatial filtering.
- the pipeline 530 may also perform other techniques.
- the camera process pipeline 530 may be a processor and other imaging processing modules specially configured for use with dual camera architectures such as that shown in Figure 5.
- the host processor 528 may be configured to handle the general processing requirements of device 500.
- the processing pipeline 530 and the host processor 528 may be in communication with each other. Additionally, the camera process pipeline may perform calculations to determine which of a telephoto or wide-angle camera stream to send to a display. As discussed above with respect to Figures 3 and 4, the determination of which camera stream to send to the display may be made based on the optical zoom selected by the user.
- the sensor not associated with the stream being sent to the display may be placed in stand-by operation or turned off, depending on user selection or power optimization strategies.
- Memory 510 also contains user interface module 550.
- the user interface module 550 includes instructions that configure the processor 528 to provide a collection of on-display objects and soft controls that allow the user to interact with the device.
- the user interface module 550 also allows applications to interact with the rest of the system in a uniform, abstracted way.
- Operating system 555 configures the processor 528 to manage the memory and processing resources of device 500.
- operating system 555 may include device drivers to manage hardware resources such as the electronic display 525 or imaging sensors 502, 504.
- the display 525 may include a touch screen component.
- the display component 525 may be coupled to the host processor 528 for receiving information for visual display to a user.
- Certain embodiments of display component 514 may be any flat panel display technology, such as an LED, LCD, plasma, or projection screen.
- Display component 525 may be coupled to the host processor 528 for receiving information for visual display to a user.
- Such information includes, but is not limited to, visual representations of files stored in a memory of device 500, software applications installed on device 500, user interfaces, and network-accessible content objects.
- the processor 528 may comprise or be a component of a processing system implemented with one or more processors.
- the one or more processors may be implemented with any combination of general-purpose microprocessors, microcontrollers, digital signal processors (DSPs), field programmable gate array (FPGAs), programmable logic devices (PLDs), controllers, state machines, gated logic, discrete hardware components, dedicated hardware finite state machines, or any other suitable entities that can perform calculations or other manipulations of information.
- Figure 5 depicts a system comprising separate components including a processor, a touch sensor, an electronic display, and memory
- processor a touch sensor
- memory a memory
- the memory components may be combined with processor components to save cost and improve performance.
- Figure 5 illustrates two memory components, including memory component 510 comprising several modules and a separate memory 520 comprising a working memory
- memory component 510 comprising several modules
- a separate memory 520 comprising a working memory
- a design may utilize ROM or static RAM memory for the storage of processor instructions implementing the modules contained in memory 510.
- processor instructions may be read at system startup from a disk storage device that is integrated into touch sensitive display system 100 or connected via an external device port. The processor instructions may then be loaded into RAM to facilitate execution by the processor.
- working memory 520 may be a RAM memory, with instructions loaded into working memory 520 before execution by the processor 528.
- Figure 6 illustrates one embodiment of a user interface 603 for a device 602 having a dual camera architecture.
- the device 602 has a display screen 603 on which an optical zoom range 604 may be displayed.
- the user can touch the display to enable a zoom operation, using a pinch to zoom, zoom slide bar, or other means.
- a user can pinch the screen 603, swipe the range 604, or press a specific point on the range 604.
- the user may select an optical zoom factor from a menu listing.
- a bounded area 606 may be displayed to the user over the captured image to indicate the viewable area of the image at the selected optical zoom factor.
- the output from the wide-angle camera is streamed when the selected optical zoom factor is between l .Ox and (Mx - Bx).
- the selected zoom factor is between (Mx + Bx) and Fx
- the output from the telephoto camera is streamed.
- the selected zoom factor is within the range (Mx - Bx) and (Mx + Bx)
- output from both the wide-angle and the telephoto cameras are streamed to reduce any latency that may occur due to a start-up delay of one or the other of the sensors.
- Figure 7 illustrates a flowchart of an exemplary method 700 to optimize optical zoom operation for a dual camera architecture.
- Method 700 may be implemented by any suitable devices and systems. Although the method 700 is described herein with reference to a particular order, in various embodiments, blocks herein may be performed in a different order, or omitted, and additional blocks may be added. The illustrated process may be executed by the device and processor described above with respect to Figure 5.
- a user initiates a zoom factor operation and both the wide-angle and the telephoto cameras are instructed to stream.
- the user may, for example, initiate a zoom operation using a pinch to zoom gesture, selecting a position on a slide bar, or selecting a zoom factor from a menu.
- process 700 transitions from block 720 to decision block 730 wherein the selected zoom factor is analyzed.
- the zoom factor is between lx and (Mx - Bx) for embodiments including a buffer optical zoom factor range, only the wide-angle camera is streamed, as shown in block 740. Otherwise, the process 700 transitions to decision block 750 and if the selected zoom factor is instead between (Mx - Bx) and (Mx + Bx), both the wide-angle and the telephoto cameras are streamed, as shown in block 760. Otherwise, by process of elimination, the zoom factor is between (Mx + Bx) and Fx and only the telephoto camera is streamed, as shown in block 770. Clarifications Regarding Terminology
- any reference to an element herein using a designation such as "first,” “second,” and so forth does not generally limit the quantity or order of those elements. Rather, these designations may be used herein as a convenient method of distinguishing between two or more elements or instances of an element. Thus, a reference to first and second elements does not mean that only two elements may be employed there or that the first element must precede the second element in some manner. Also, unless stated otherwise a set of elements may include one or more elements.
- any of the various illustrative logical blocks, modules, processors, means, circuits, and algorithm steps described in connection with the aspects disclosed herein may be implemented as electronic hardware (e.g., a digital implementation, an analog implementation, or a combination of the two, which may be designed using source coding or some other technique), various forms of program or design code incorporating instructions (which may be referred to herein, for convenience, as "software” or a "software module), or combinations of both.
- software or a “software module”
- the various illustrative logical blocks, modules, and circuits described in connection with the aspects disclosed herein and in connection with FIGS. 1-3 may be implemented within or performed by an integrated circuit (IC), an access terminal, or an access point.
- the IC may include a general purpose processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, electrical components, optical components, mechanical components, or any combination thereof designed to perform the functions described herein, and may execute codes or instructions that reside within the IC, outside of the IC, or both.
- the logical blocks, modules, and circuits may include antennas and/or transceivers to communicate with various components within the network or within the device.
- a general purpose processor may be a microprocessor, but in the alternative, the processor may be any conventional processor, controller, microcontroller, or state machine.
- a processor may also be implemented as a combination of computing devices, such as a combination of a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors in conjunction with a DSP core, or any other such configuration.
- the functionality of the modules may be implemented in some other manner as taught herein.
- the functionality described herein (for example, with regard to one or more of the accompanying figures) may correspond in some aspects to similarly designated "means for" functionality in the appended claims.
- Computer-readable media includes both computer storage media and communication media including any medium that can be enabled to transfer a computer program from one place to another.
- a storage media may be any available media that may be accessed by a computer.
- such computer-readable media may include RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other medium that may be used to store desired program code in the form of instructions or data structures and that may be accessed by a computer.
- Disk and disc includes compact disc (CD), laser disc, optical disc, digital versatile disc (DVD), floppy disk, and blu-ray disc where disks usually reproduce data magnetically, while discs reproduce data optically with lasers. Combinations of the above should also be included within the scope of computer- readable media. Additionally, the operations of a method or algorithm may reside as one or any combination or set of codes and instructions on a machine readable medium and computer-readable medium, which may be incorporated into a computer program product.
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Human Computer Interaction (AREA)
- Studio Devices (AREA)
Abstract
Systems, methods, and devices for power optimization in imaging devices having dual cameras are contained herein. In one aspect, a method for power optimization for a dual camera imaging device is disclosed. The method includes determining a zoom factor selection, determining whether the zoom factor selection falls within a first zoom factor range, a second zoom factor range, or a third zoom factor range, and sending a series of frames of an image captured by a first sensor or a series of frames of an image captured by a second sensor or both to a camera application based on the determined zoom factor section.
Description
SYSTEMS AND METHODS FOR POWER OPTIMIZATION FOR IMAGING DEVICES WITH DUAL CAMERAS
BACKGROUND
Technical Field
[0001] The present application relates generally to imaging devices, and more particularly to systems, methods, and devices for power optimization in imaging devices with dual cameras.
Background
[0002] Many current devices are available for capturing images. For example, current digital cameras and modem cell phones and tablets can all capture and display digital images to a user.
[0003] Dual cameras are being increasingly used in cell phones and other mobile devices. Typically, a first camera has a wide angle lens while the second camera has a telephoto lens. Running two cameras simultaneously, however, increases power consumption and decreases battery life of the mobile device. Thus, systems and methods for dual camera applications focusing on power optimization for an optical zoom feature of a dual camera application are needed.
SUMMARY
[0004] Dual camera systems on a mobile device, or other device, may include both a telephoto and wide-angle lens. In such dual camera systems, both cameras may be operated at the same time to capture an image of a scene. However, operating the dual cameras simultaneously can increase power consumption and therefore can decrease the battery life of a mobile device.
[0005] The systems, methods, devices, and computer program products discussed herein each have several aspects, no single one of which is solely responsible for the desirable attributes disclosed herein. Combinations of the innovations, aspects and features described herein can be incorporated in various embodiments of systems, methods, and devices, and such combinations are not limited by the examples of embodiments described herein.
[0006] Embodiments and innovations discussed herein relate to systems and methods that may be run in a processor for an electronic device to optimize the power utilization of devices having and/or operating dual cameras. Preferably, the power
optimization systems and methods discussed herein can be implemented in existing hardware and software. However, in some embodiments, specially designed hardware and software may improve speed or efficiencies of such processes.
[0007] In one innovation, a method for power optimization for a dual camera imaging device is disclosed. The method includes the steps of determining a zoom factor selection for capturing image data; determining whether the zoom factor selection falls within a first zoom factor range, a second zoom factor range, or a third zoom factor range, the first, second and third zoom factor ranges being an amount of zoom; and sending a series of frames of a scene captured by a first sensor or a series of frames of the scene captured by a second sensor or both to a camera application based on the determined zoom factor selection. In some embodiments, the method further includes determining a buffer factor from a warm up period of the second sensor. In some embodiments, the first, second, and third zoom factor ranges are determined using the buffer factor, a field of view (FOV) ratio between the first sensor and the second sensor and a maximum zoom factor of the imaging device. In some embodiments, the FOV ratio is a ratio between the FOV of the first sensor and a FOV of the second sensor and the FOV ratio is less than the maximum zoom factor of the imaging device. In some embodiments, the first zoom factor range is between 1.0 and a difference between the FOV ratio and the buffer factor. In some embodiments, the second zoom factor range is between a difference between the FOV ratio and the buffer factor and the sum of the field of view ratio and the buffer factor. In some embodiments, the third zoom factor range is between the sum of the FOV ratio and the buffer factor and the maximum zoom factor. In some embodiments, the method further includes turning off one of the first and second sensors when the other of the first and second sensors is sending frames to the camera application when the zoom factor is within either of the first and the third zoom factor ranges. In some embodiments, the method further includes sending frames to the camera application from both of the first and second sensors when the zoom factor is within the second zoom factor range.
[0008] In another innovation, an apparatus for optimizing operation of a dual camera imaging device includes a processor; a first imaging sensor and a second imaging sensor; and a memory, operably coupled to the processor, and configured to store processor instructions that configure the processor to determine a zoom factor selection; determine whether the zoom factor selection falls within a first zoom factor range, a second zoom factor range, or a third zoom factor range; and send a series of frames of a
scene captured by a first sensor or a series of frames of the scene captured by a second sensor or both to a camera application based on the determined zoom factor section. In some embodiments, the apparatus further includes a touch device configured to sense a touch on a touch interface. In some embodiments, the memory further is configured to determine a buffer factor from a warm up period of the second sensor. In some embodiments, the memory further is configured to determine first, second, and third zoom factor ranges using the buffer factor, a field of view (FOV) ratio between the first sensor and the second sensor, and a maximum zoom factor of the imaging device. In some embodiments, the first zoom factor range is between 1.0 and a difference between the FOV ratio and the buffer factor. In some embodiments, the second zoom factor range is between a difference between the FOV ratio and the buffer factor and the sum of the FOV ratio and the buffer factor. In some embodiments, the third zoom factor range is between the sum of the FOV ratio and the buffer factor and the maximum zoom factor. In some embodiments, the memory further is configured to turn off one of the first and second sensors when the other of the first and second sensors is sending frames to the camera application when the zoom factor is within either of the first and the third zoom factor ranges. In some embodiments, the memory further is configured to send frames to the camera application from both of the first and second sensors when the zoom factor is within the second zoom factor range.
[0009] In another innovation, a system for optimizing operation of dual camera imaging device includes a control module configured to determine a zoom factor selection; determine whether the zoom factor selection falls within a first zoom factor range, a second zoom factor range, or a third zoom factor range; and send a series of frames of a scene captured by a first sensor or a series of frames of the scene captured by a second sensor or both to a camera application based on the determined zoom factor section. In some embodiments, the system further includes a touch interface configured to receive a zoom factor selection. In some embodiments, the control module is further configured to determine a buffer factor from a warm up period of the second sensor. In some embodiments, the control module further is configured to determine first, second, and third zoom factor ranges using the buffer factor, a field of view ratio between the first sensor and the second sensor, and a maximum zoom factor of the imaging device. In some embodiments, the first zoom factor range is between 1.0 and a difference between the field of view ratio and the buffer factor. In some embodiments, the second zoom
factor range is between a difference between the field of view ratio and the buffer factor and the sum of the field of view ratio and the buffer factor. In some embodiments, the third zoom factor range is between the sum of the field of view ratio and the buffer factor and the maximum zoom factor. In some embodiments, the control module further is configured to turn off one of the first and second sensors when the other of the first and second sensors is sending frames to the camera application when the zoom factor is within either of the first and the third zoom factor ranges. In some embodiments, the memory further is configured to send frames to the camera application from both of the first and second sensors when the zoom factor is within the second zoom factor range.
[0010] In another innovation, a non-transitory computer-readable medium stores instructions that, when executed, cause at least one physical computer processor to perform a method for power optimization of a dual camera imaging device. The method includes the steps of determining a zoom factor selection; determining whether the zoom factor selection falls within a first zoom factor range, a second zoom factor range, or a third zoom factor range; and sending a series of frames of a scene captured by a first sensor or a series of frames of the scene captured by a second sensor or both to a camera application based on the determined zoom factor section. In some embodiments, the method further includes determining a buffer factor from a warm up period of the second sensor. In some embodiments, the method further includes turning off one of the first and second sensors when the other of the first and second sensors is sending frames to the camera application when the zoom factor is within either of the first and the third zoom factor ranges and sending frames to the camera application from both of the first and second sensors when the zoom factor is within the second zoom factor range.
BRIEF DESCRIPTION OF THE DRAWINGS
[0011] The disclosed aspects will hereinafter be described in conjunction with the appended drawings, provided to illustrate and not to limit the disclosed aspects, wherein like designations denote like elements.
[0012] FIG. 1 illustrates two views of a scene as captured by a wide-angle camera and a telephoto camera of a dual camera device according to one embodiment.
[0013] FIG. 2 illustrates an example implementation of an dual camera architecture, according to some embodiments, in which one camera stream may be displayed to a user at a time but both cameras can be operating simultaneously.
[0014] FIG. 3 illustrates an example of a dual camera architecture, according to some embodiments, in which one camera stream may be displayed to a user at a time and the other camera may not be streamed.
[0015] FIG. 4 illustrates another example of the dual camera architecture embodiment illustrated in FIG. 3 which includes a buffer zone in which both cameras may be streamed to reduce latency.
[0016] FIG. 5 illustrates a block diagram of an example of a mobile imaging device in accordance with one embodiment.
[0017] FIG. 6 illustrates one example of a user interface for selecting an optical zoom factor on a mobile imaging device.
[0018] FIG. 7 is a flow chart illustrating an exemplary method of optimizing power on a mobile imaging device having a dual camera architecture using optical zoom selection.
DETAILED DESCRIPTION
[0019] Embodiments relate to power optimization systems and methods for electronic devices that include dual cameras. Devices having dual cameras may include two camera modules and lens systems, for example, a wide-angle camera module having a wide-angle lens and a telephoto camera module having a telephoto lens. The wide- angle lens of the wide-angle camera module has a larger field of view (FOV) than the telephoto lens of the telephoto camera module at the same resolution and size (for example, width and height) of an image. The telephoto camera module is used to capture a narrow field of view of an object in the scene (or scene). An electronic device having dual cameras may operate both cameras simultaneously, thereby increasing power consumption and decreasing battery life of the device.
[0020] The systems and methods disclosed herein relate to power optimization for dual cameras, wherein a control system selectively operates either the wide-angle or the telephoto camera module depending on desired optical zoom, while maintaining the overall resolution of the captured image.
[0021] It is noted that the examples may be described as a process, which is depicted as a flowchart, a flow diagram, a finite state diagram, a structure diagram, or a block diagram. Although a flowchart may describe the operations as a sequential process, many of the operations can be performed in parallel, or concurrently, and the process can be repeated. In addition, the order of the operations may be re-arranged. A process is terminated when its operations are completed. A process may correspond to a method, a function, a procedure, a subroutine, a subprogram, etc. When a process corresponds to a software function, its termination corresponds to a return of the function to the calling function or the main function.
[0022] Embodiments may be implemented in System-on-Chip (SoC) or external hardware, software, firmware, or any combination thereof. Those of skill in the art will understand that information and signals may be represented using any of a variety of different technologies and techniques. For example, data, instructions, commands, information, signals, bits, symbols, and chips that may be referenced throughout the above description may be represented by voltages, currents, electromagnetic waves, magnetic fields or particles, optical fields or particles, or any combination thereof.
[0023] In the following description, specific details are given to provide a thorough understanding of the examples. However, it will be understood by one of ordinary skill in the art that the examples may be practiced without these specific details. For example, electrical components/devices may be shown in block diagrams in order not to obscure the examples in unnecessary detail. In other instances, such components, other structures and techniques may be shown in detail to further explain the examples.
[0024] Figure 1 illustrates two views of a scene as captured by a wide-angle lens of a dual camera device and a telephoto lens of a dual camera device (for example, a system as illustrated in Figure 5). A first image 100 was taken using the wide-angle lens and a second image 110 was taken using the telephoto lens, the second image 110 corresponding to a portion 115 of the first image. When using a feature such as optical zoom, normally only one of the images captured by the cameras is displayed to the user. However, dual camera architectures may stream frames from both the wide-angle and the telephoto cameras even though only one is displayed. In the example illustrated in Figure 1, the wide-angle lens used to take the first image 100 has about twice as much field-of- view as the telephoto lens used to take the second image 110. While both cameras may stream frames to the image processor of the device, until the user inputs a 2x optical
zoom factor, the image captured by the wide-angle lens will be displayed. For optical zoom factors greater than a 2x zoom factor, the image captured by the telephoto lens will be displayed.
[0025] Figure 2 illustrates an example of an implementation of a dual camera architecture in which one camera stream is displayed to a user at a time but both cameras are operating simultaneously. After capturing an image, the output from a wide-angle sensor 202 and a telephoto sensor 204 are sent to separate video front end (VFE) image processing modules. That is, the wide-angle sensor 202 provides image information 203 to a VFE wide-angle image processing module 206, and the telephoto sensor 204 provides image information 205 to the VFE telephoto image processing module 208. The VFE wide-angle processing module 206 provides full resolution of the captured image information as output 220 to a video multiplexer 214, and provides a reduced (or smaller) resolution output 207 to a calibration offset estimation module 210.
[0026] The VFE telephoto image processing module 208 provides a full resolution output 224 of the telephoto sensor data to the rectification/ Alignment GPU 212, which provides telephoto input data to the video multiplexer 214. The VFE wide angle processing module 206 also provides a reduced (or small) resolution output of the telephoto sensor 204 data to the calibration offset estimation module 210, which aligns the telephoto image data and the wide angle image data. The calibration offset estimation module 210 then provides an output to the rectification / alignment module 212 to provide alignment of the full resolution telephoto data (so that it is aligned with the wide angle data).
[0027] The video multiplexer 214 selects either the wide angle sensor data input or the telephoto sensor data input based on a selected zoom factor, and outputs image data 222 (wide angle or telephoto data). The output image data 222 is then sent to processing module 216 for upscaling or downscaling and the resulting output 217 is sent via additional image processing module 218 to a display (not shown). Operation of both cameras simultaneously draws unnecessary power from one of the cameras even if the camera is not being displayed or streamed. As discussed herein, streaming may refer to the CCD array of the camera sending all of the captured frames to the camera application to be displayed to the user.
[0028] Figure 3 is a schematic illustrating an example of a dual camera architecture embodiment. In this embodiment, the image data output (or streamed) from
both of the wide-angle camera 302 and the telephoto camera 318 are not used together to form the image data 330 which is ultimately displayed (or stored). That is, data from only one of the wide-angle camera 302 or the telephoto camera 318 is used to form the display data 330, or be saved in a memory component (not shown). In this embodiment, the wide-angle camera 302 or the telephoto camera 318 that is producing an image data output (stream) that is not being displayed can be turned off to reduce the overall power consumption of the device and thus reduce the overall system load. The streams of each of the cameras 302, 318 are sent to separate video front end modules 304, 320 respectively. Outputs from the video front end modules 304, 320 are sent to a video multiplexer module 306.
[0029] As illustrated in Figure 3, the choice of which camera stream to display to the user is determined at position 308 based on the selected optical zoom factor of the imaging system. In the illustrated embodiment, the zoom factor can range between l .Ox to Fx, where F is the maximum zoom factor. The output 310 from the wide-angle camera is streamed when the selected optical zoom factor is between l .Ox and Mx, where M is the FOV ratio between the wide-angle lens and the telephoto lens and M < F. When the selected zoom factor is between Mx and Fx, the output 328 from the telephoto camera is streamed. For example, in one embodiment, M, the FOV ratio between the wide-angle lens and the telephoto lens, may be, for example between 1.5x and 4. Ox. In other embodiments, M is approximately 2. Ox. In some embodiments, F, the maximum zoom factor, may be, for example, between 4. Ox and 7. Ox. In other embodiments, F is approximately 6. Ox. Both of the wide-angle and telephoto camera streams 310 and 328 may be passed through other image processing modules, for example, modules 314 and 316, before being output 330 and then displayed or stored. In some embodiments, the full resolution image stream 326 from the telephoto camera may be provided to a rectification and alignment module 312 prior to the selection of the image stream at position 308. In some embodiments, the telephoto camera may incur a delay in producing a scene image stream when the user selects the Mx zoom factor. Therefore, this embodiment introduces a period of delay in operation that may be undesirable.
[0030] Figure 4 illustrates a flow diagram of another alternative to the dual camera architecture illustrated in Figure 2. As in the embodiment shown in Figure 3, the wide-angle camera 402 and the telephoto camera 416 are not used or displayed at the same time and therefore the stream from the camera not being displayed can be turned off
to reduce the power consumption of the device. The streams of each camera 402, 416 are sent to separate video front end modules 404, 418 for processing and then to the video multiplexer module 406. As illustrated in Figure 4, the choice of which camera stream 408, 422, to display to the user is determined at position 410 based on the selected optical zoom factor. In the illustrated embodiment, the zoom factor can range between l .Ox to Fx, where F is a maximum zoom factor. In some embodiments, F may be, for example, between 4.0x and 7. Ox. In other embodiments, F is approximately 6. Ox. As noted above, one limitation to streaming one camera stream at a time is the lag or delay caused by the telephoto camera's inability to start immediately at the Mx zoom factor.
[0031] In this embodiment, a buffer factor between 1.0 and Fx, defined as B, is a programmable factor which varies based on the telephoto camera sensor's warm up period. As shown in Figure 4, both the wide angle camera and the telephoto cameras are streamed when the zoom factor is in the range of (Mx - Bx) to (Mx + Bx). The output 408 from the wide-angle camera is streamed when the selected optical zoom factor is between l .Ox and (Mx - Bx). When the selected zoom factor is between (Mx + Bx) and Fx, the output 422 from the telephoto camera is streamed. Both of the camera streams 408 and 422 may be passed through other image processing modules, such as modules 412 and 414, before being sent to a display. In some embodiments, the full resolution image stream 420 from the telephoto camera may be sent to a rectification / alignment module 424 to provide alignment of the full resolution telephoto data prior to selection of the displayed image stream. The zoom range buffer zone (Mx - Bx) to (Mx + Bx) preferably absorbs any latency caused by delay in start up of the telephoto camera. The determination of B is based on the sensor start up scheme implemented. For example, if the wide-angle or telephoto sensor is simply on standby then B can be a small value, such as 0.5x. However, when the wide-angle or telephoto sensor is turned off and then started up, the value of B is set to a higher value, such as l .Ox. Setting B to a higher value increases the range at which data from both the wide-angle and the telephoto cameras are streamed, reducing the latency involved in sensor start-up.
[0032] Figure 5 illustrates various components of a mobile device 500 with two camera sensors 502, 504 in accordance with one embodiment of the present disclosure which could perform the optical zoom optimization techniques described herein. The mobile device 500 may be configured to capture, create, process, modify, scale, encode, decode, transmit, store, and display digital images and/or video sequences.
The mobile device 500 may represent or be implemented in a device, such as a wireless communication device, a personal digital assistant (PDA), a laptop computer, a desktop computer, a digital camera, a digital recording device, a network-enabled digital television, a mobile phone, a cellular phone, a satellite telephone, a camera phone, a terrestrial-based radiotelephone, a direct two-way communication device (sometimes referred to as a "walkie-talkie"), etc. The illustrated embodiment is not meant to be limitative and device 500 may include a variety of other components as required for other functions.
[0033] The mobile device 500 may include a first imaging sensor 502, a second imaging sensor 504, a memory 510, a processor 528, a working memory 520, and a display 525. The memory 510 may include various imaging processing modules, including a camera process pipeline 530, a user interface module 550, and operating system 555. The architecture illustrated in Figure 1 is merely an example. The features and techniques described herein may be implemented with a variety of other architectures.
[0034] The sensors 502, 504 may be digital camera sensors. In some embodiments, one of the sensors 502 may be a wide-angle sensor and the other sensor 504 may be a telephoto sensor. The sensors 502, 504 may capture still image snapshots and/or video sequences. Each sensor may include color filter arrays (CFAs) arranged on a surface of individual sensors or sensor elements.
[0035] The memory 510 and working memory 520 may store images or video sequences before and after processing. The memory 510 and working memory 520 may include volatile storage and non-volatile storage. The memory 510 and working memory 520 may comprise any type of data storage means, such as dynamic random access memory (DRAM), FLASH memory, NOR or NAND gate memory, or any other data storage technology.
[0036] In the illustrated embodiment, the program memory 510 stores a camera process pipeline 530, a user interface module 550, and operating system 555. These modules may include instructions that configure the processor 528 to perform various image processing and device management tasks. The program memory 510 can be any suitable computer-readable storage medium, such as a non-transitory storage medium. Working memory 520 may be used by processor 528 to store a working set of processor instructions contained in the modules of program memory 510. Alternatively,
working memory 520 may also be used by processor 528 to store dynamic data created during the operation of the device 500.
[0037] The camera process pipeline 530 (also called an engine, module, processing unit, etc.) may comprise a chip set for a mobile phone, which may include hardware, software, firmware, and/or one or more microprocessors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), or various combinations thereof. The camera process pipeline 530 may perform one or more image processing techniques to improve quality of an image and/or a video sequence. For example, the pipeline 530 may perform techniques such as demosaicing, lens rolloff correction, scaling, color correction, color conversion, and spatial filtering. The pipeline 530 may also perform other techniques. The camera process pipeline 530 may be a processor and other imaging processing modules specially configured for use with dual camera architectures such as that shown in Figure 5. The host processor 528 may be configured to handle the general processing requirements of device 500. The processing pipeline 530 and the host processor 528 may be in communication with each other. Additionally, the camera process pipeline may perform calculations to determine which of a telephoto or wide-angle camera stream to send to a display. As discussed above with respect to Figures 3 and 4, the determination of which camera stream to send to the display may be made based on the optical zoom selected by the user. In some embodiments, the sensor not associated with the stream being sent to the display may be placed in stand-by operation or turned off, depending on user selection or power optimization strategies.
[0038] Memory 510 also contains user interface module 550. The user interface module 550 includes instructions that configure the processor 528 to provide a collection of on-display objects and soft controls that allow the user to interact with the device. The user interface module 550 also allows applications to interact with the rest of the system in a uniform, abstracted way. Operating system 555 configures the processor 528 to manage the memory and processing resources of device 500. For example, operating system 555 may include device drivers to manage hardware resources such as the electronic display 525 or imaging sensors 502, 504.
[0039] The display 525 may include a touch screen component. The display component 525 may be coupled to the host processor 528 for receiving information for visual display to a user. Certain embodiments of display component 514 may be any flat
panel display technology, such as an LED, LCD, plasma, or projection screen. Display component 525 may be coupled to the host processor 528 for receiving information for visual display to a user. Such information includes, but is not limited to, visual representations of files stored in a memory of device 500, software applications installed on device 500, user interfaces, and network-accessible content objects.
[0040] The processor 528 may comprise or be a component of a processing system implemented with one or more processors. The one or more processors may be implemented with any combination of general-purpose microprocessors, microcontrollers, digital signal processors (DSPs), field programmable gate array (FPGAs), programmable logic devices (PLDs), controllers, state machines, gated logic, discrete hardware components, dedicated hardware finite state machines, or any other suitable entities that can perform calculations or other manipulations of information.
[0041] Although Figure 5 depicts a system comprising separate components including a processor, a touch sensor, an electronic display, and memory, one skilled in the art would recognize that these separate components may be combined a variety of ways to achieve particular design objectives. For example, in an alternative embodiment, the memory components may be combined with processor components to save cost and improve performance.
[0042] Additionally, although Figure 5 illustrates two memory components, including memory component 510 comprising several modules and a separate memory 520 comprising a working memory, one with skill in the art would recognize several embodiments utilizing different memory architectures. For example, a design may utilize ROM or static RAM memory for the storage of processor instructions implementing the modules contained in memory 510. Alternatively, processor instructions may be read at system startup from a disk storage device that is integrated into touch sensitive display system 100 or connected via an external device port. The processor instructions may then be loaded into RAM to facilitate execution by the processor. For example, working memory 520 may be a RAM memory, with instructions loaded into working memory 520 before execution by the processor 528.
[0043] Figure 6 illustrates one embodiment of a user interface 603 for a device 602 having a dual camera architecture. As illustrated, the device 602 has a display screen 603 on which an optical zoom range 604 may be displayed. The user can touch the display to enable a zoom operation, using a pinch to zoom, zoom slide bar, or other
means. To indicate a desired optical zoom factor, a user can pinch the screen 603, swipe the range 604, or press a specific point on the range 604. In other embodiments, the user may select an optical zoom factor from a menu listing. A bounded area 606 may be displayed to the user over the captured image to indicate the viewable area of the image at the selected optical zoom factor. As discussed above, the output from the wide-angle camera is streamed when the selected optical zoom factor is between l .Ox and (Mx - Bx). When the selected zoom factor is between (Mx + Bx) and Fx, the output from the telephoto camera is streamed. When the selected zoom factor is within the range (Mx - Bx) and (Mx + Bx), output from both the wide-angle and the telephoto cameras are streamed to reduce any latency that may occur due to a start-up delay of one or the other of the sensors.
[0044] Figure 7 illustrates a flowchart of an exemplary method 700 to optimize optical zoom operation for a dual camera architecture. Method 700 may be implemented by any suitable devices and systems. Although the method 700 is described herein with reference to a particular order, in various embodiments, blocks herein may be performed in a different order, or omitted, and additional blocks may be added. The illustrated process may be executed by the device and processor described above with respect to Figure 5.
[0045] In an operational block 710 of the method 700, a user initiates a zoom factor operation and both the wide-angle and the telephoto cameras are instructed to stream. The user may, for example, initiate a zoom operation using a pinch to zoom gesture, selecting a position on a slide bar, or selecting a zoom factor from a menu. When the user releases or finalizes the desired selection of a zoom operation by, for example, finishing the pinching gesture, raising a finger from the touch screen, or selecting the zoom factor from the menu, as indicated in operational block 720, process 700 transitions from block 720 to decision block 730 wherein the selected zoom factor is analyzed. If the zoom factor is between lx and (Mx - Bx) for embodiments including a buffer optical zoom factor range, only the wide-angle camera is streamed, as shown in block 740. Otherwise, the process 700 transitions to decision block 750 and if the selected zoom factor is instead between (Mx - Bx) and (Mx + Bx), both the wide-angle and the telephoto cameras are streamed, as shown in block 760. Otherwise, by process of elimination, the zoom factor is between (Mx + Bx) and Fx and only the telephoto camera is streamed, as shown in block 770.
Clarifications Regarding Terminology
[0046] It should be understood that any reference to an element herein using a designation such as "first," "second," and so forth does not generally limit the quantity or order of those elements. Rather, these designations may be used herein as a convenient method of distinguishing between two or more elements or instances of an element. Thus, a reference to first and second elements does not mean that only two elements may be employed there or that the first element must precede the second element in some manner. Also, unless stated otherwise a set of elements may include one or more elements.
[0047] A person/one having ordinary skill in the art would understand that information and signals may be represented using any of a variety of different technologies and techniques. For example, data, instructions, commands, information, signals, bits, symbols, and chips that may be referenced throughout the above description may be represented by voltages, currents, electromagnetic waves, magnetic fields or particles, optical fields or particles, or any combination thereof.
[0048] A person/one having ordinary skill in the art would further appreciate that any of the various illustrative logical blocks, modules, processors, means, circuits, and algorithm steps described in connection with the aspects disclosed herein may be implemented as electronic hardware (e.g., a digital implementation, an analog implementation, or a combination of the two, which may be designed using source coding or some other technique), various forms of program or design code incorporating instructions (which may be referred to herein, for convenience, as "software" or a "software module), or combinations of both. To clearly illustrate this interchangeability of hardware and software, various illustrative components, blocks, modules, circuits, and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the overall system. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present disclosure.
[0049] The various illustrative logical blocks, modules, and circuits described in connection with the aspects disclosed herein and in connection with FIGS. 1-3 may be implemented within or performed by an integrated circuit (IC), an access terminal, or an
access point. The IC may include a general purpose processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, electrical components, optical components, mechanical components, or any combination thereof designed to perform the functions described herein, and may execute codes or instructions that reside within the IC, outside of the IC, or both. The logical blocks, modules, and circuits may include antennas and/or transceivers to communicate with various components within the network or within the device. A general purpose processor may be a microprocessor, but in the alternative, the processor may be any conventional processor, controller, microcontroller, or state machine. A processor may also be implemented as a combination of computing devices, such as a combination of a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors in conjunction with a DSP core, or any other such configuration. The functionality of the modules may be implemented in some other manner as taught herein. The functionality described herein (for example, with regard to one or more of the accompanying figures) may correspond in some aspects to similarly designated "means for" functionality in the appended claims.
[0050] If implemented in software, the functions may be stored on or transmitted over as one or more instructions or code on a computer-readable medium. The steps of a method or algorithm disclosed herein may be implemented in a processor- executable software module which may reside on a computer-readable medium. Computer-readable media includes both computer storage media and communication media including any medium that can be enabled to transfer a computer program from one place to another. A storage media may be any available media that may be accessed by a computer. By way of example, and not limitation, such computer-readable media may include RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other medium that may be used to store desired program code in the form of instructions or data structures and that may be accessed by a computer. Also, any connection can be properly termed a computer- readable medium. Disk and disc, as used herein, includes compact disc (CD), laser disc, optical disc, digital versatile disc (DVD), floppy disk, and blu-ray disc where disks usually reproduce data magnetically, while discs reproduce data optically with lasers. Combinations of the above should also be included within the scope of computer-
readable media. Additionally, the operations of a method or algorithm may reside as one or any combination or set of codes and instructions on a machine readable medium and computer-readable medium, which may be incorporated into a computer program product.
[0051] It is understood that any specific order or hierarchy of steps in any disclosed process is an example of a sample approach. Based upon design preferences, it is understood that the specific order or hierarchy of steps in the processes may be rearranged while remaining within the scope of the present disclosure. The accompanying method claims present elements of the various steps in a sample order, and are not meant to be limited to the specific order or hierarchy presented.
[0052] Various modifications to the implementations described in this disclosure may be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other implementations without departing from the spirit or scope of this disclosure. Thus, the disclosure is not intended to be limited to the implementations shown herein, but is to be accorded the widest scope consistent with the claims, the principles and the novel features disclosed herein. The word "exemplary" is used exclusively herein to mean "serving as an example, instance, or illustration." Any implementation described herein as "exemplary" is not necessarily to be construed as preferred or advantageous over other implementations.
[0053] Certain features that are described in this specification in the context of separate implementations also can be implemented in combination in a single implementation. Conversely, various features that are described in the context of a single implementation also can be implemented in multiple implementations separately or in any suitable sub-combination. Moreover, although features may be described above as acting in certain combinations and even initially claimed as such, one or more features from a claimed combination can in some cases be excised from the combination, and the claimed combination may be directed to a sub-combination or variation of a sub-combination.
[0054] Similarly, while operations are depicted in the drawings in a particular order, this should not be understood as requiring that such operations be performed in the particular order shown or in sequential order, or that all illustrated operations be performed, to achieve desirable results. In certain circumstances, multitasking and parallel processing may be advantageous. Moreover, the separation of various system components in the implementations described above should not be understood as
requiring such separation in all implementations, and it should be understood that the described program components and systems can generally be integrated together in a single software product or packaged into multiple software products. Additionally, other implementations are within the scope of the following claims. In some cases, the actions recited in the claims can be performed in a different order and still achieve desirable results.
Claims
1. A method for power optimization for a dual camera imaging device, the method comprising:
determining a zoom factor selection for capturing image data; determining a zoom factor range that the zoom factor selection falls within;
providing a series of frames of a scene from a first sensor based on the determined zoom factor range; and
providing a series of frames of a scene from a second sensor based on the determined zoom factor range.
2. The method of claim 1 , wherein said providing further comprises providing the series of frames to a camera application.
3. The method of claim 2, wherein said providing further comprises:
providing a series of frames of the scene captured by the first sensor if the zoom factor selection is within a first zoom factor range; and providing a series of frames of the scene captured by the second sensor if the zoom factor selection is within a second zoom factor range.
4. The method of claim 3, wherein said providing further comprises providing a series of frames of the scene captured by the first sensor and providing a series of frames of the scene captured by the second sensor if the zoom factor selection is within a third zoom factor range.
5. The method of claim 4 further comprising determining a buffer factor from a warm up period of the second sensor, wherein the first, second, and third zoom factor ranges are determined using the buffer factor, a field of view (FOV) ratio between the first sensor and the second sensor and a maximum zoom factor of the imaging device.
6. The method of claim 5, wherein the FOV ratio is a ratio between the FOV of the first sensor and a FOV of the second sensor and the FOV ratio is less than the maximum zoom factor of the imaging device.
7. The method of claim 5, wherein the first zoom factor range is between 1.0 and a difference between the FOV ratio and the buffer factor.
8. The method of claim 5, wherein the second zoom factor range is between a difference between the FOV ratio and the buffer factor and the sum of the field of view ratio and the buffer factor.
9. The method of claim 5, wherein the third zoom factor range is between the sum of the FOV ratio and the buffer factor and the maximum zoom factor.
10. The method of claim 4, further comprising turning off one of the first and second sensors when the other of the first and second sensors is sending frames to the camera application when the zoom factor is within either of the first and the third zoom factor ranges.
1 1. An apparatus for optimizing operation of a dual camera imaging device, comprising:
a processor;
a first imaging sensor and a second imaging sensor; and
a memory, operably coupled to the processor, and configured to store processor instructions that configure the processor to:
determine a zoom factor selection for capturing image data;
determine a zoom factor range that the zoom factor selection falls within;
provide a series of frames of a scene from a first sensor based on the determined zoom factor range; and
provide a series of frames of a scene from a second sensor based on the determined zoom factor range.
12. The apparatus of claim 1 1 , wherein said providing further comprises providing the series of frames to a camera application.
13. The apparatus of claim 12, wherein said providing further comprises:
providing a series of frames of the scene captured by the first sensor if the zoom factor selection is within a first zoom factor range; and
providing a series of frames of the scene captured by the second sensor if the zoom factor selection is within a second zoom factor range.
14. The apparatus of claim 13, wherein said providing further comprises providing a series of frames of the scene captured by the first sensor and providing a series of frames of the scene captured by the second sensor if the zoom factor selection is within a third zoom factor range.
15. The apparatus of claim 14, wherein the memory further is configured to determine a buffer factor from a warm up period of the second sensor and to determine the first, second, and third zoom factor ranges using the buffer factor, a field of view (FOV) ratio between the first sensor and the second sensor, and a maximum zoom factor of the imaging device.
16. The apparatus of claim 15, wherein the first zoom factor range is between 1.0 and a difference between the FOV ratio and the buffer factor, the second zoom factor range is between a difference between the FOV ratio and the buffer factor and the sum of the FOV ratio and the buffer factor, and the third zoom factor range is between the sum of the FOV ratio and the buffer factor and the maximum zoom factor.
17. The apparatus of claim 14, wherein the memory further is configured to turn off one of the first and second sensors when the other of the first and second sensors is sending frames to the camera application when the zoom factor is within either of the first and the third zoom factor ranges.
18. A system for optimizing operation of dual camera imaging device, comprising;
a control module configured to
determine a zoom factor selection for capturing image data;
determine a zoom factor range that the zoom factor selection falls within;
provide a series of frames of a scene from a first sensor based on the determined zoom factor range; and
provide a series of frames of a scene from a second sensor based on the determined zoom factor range.
19. The system of claim 18 wherein said providing further comprises providing the series of frames to a camera application.
20. The system of claim 18, wherein said providing further comprises:
providing a series of frames of the scene captured by the first sensor if the zoom factor selection is within a first zoom factor range; and providing a series of frames of the scene captured by the second sensor if the zoom factor selection is within a second zoom factor range.
21. The system of claim 20, wherein said providing further comprises providing a series of frames of the scene captured by the first sensor and providing a series of frames of the scene captured by the second sensor if the zoom factor selection is within a third zoom factor range.
22. The system of claim 21, wherein the control module is further configured to determine a buffer factor from a warm up period of the second sensor, wherein the first, second, and third zoom factor ranges are determined using the buffer factor, a field of view (FOV) ratio between the first sensor and the second sensor, and a maximum zoom factor of the imaging device.
23. The system of claim 22, wherein the FOV ratio is a ratio between a FOV of the first sensor and a FOV of the second sensor, and the FOV ratio is less than the maximum zoom factor of the imaging device.
24. The system of claim 23, wherein the first zoom factor range is between 1.0 and a difference between the FOV ratio and the buffer factor, the second zoom factor range is between a difference between the field of view ratio and the buffer factor and the sum of the field of view ratio and the buffer factor, and the third zoom factor range is between the sum of the field of view ratio and the buffer factor and the maximum zoom factor.
25. The system of claim 21, wherein the control module is further configured to turn off one of the first and second sensors when the other of the first and second sensors is sending frames to the camera application when the zoom factor is within either of the first and the third zoom factor ranges.
26. A non-transitory computer-readable medium storing instructions that, when executed, cause at least one physical computer processor to perform a method for power optimization of a dual camera imaging device, the method comprising:
determining a zoom factor selection for capturing image data; determining a zoom factor range that the zoom factor selection falls within;
providing a series of frames of a scene from a first sensor based on the determined zoom factor range; and
providing a series of frames of a scene from a second sensor based on the determined zoom factor range.
27. The non-transitory computer-readable medium of claim 26 wherein said providing further comprises providing the series of frames to a camera application.
28. The non-transitory computer-readable medium of claim 26, wherein said providing further comprises:
providing a series of frames of the scene captured by the first sensor if the zoom factor selection is within a first zoom factor range; providing a series of frames of the scene captured by the second sensor if the zoom factor selection is within a second zoom factor range; and
providing a series of frames of the scene captured by the first sensor and providing a series of frames of the scene captured by the second sensor if the zoom factor selection is within a third zoom factor range.
29. The non-transitory computer-readable medium of claim 28 further comprising determining a buffer factor from a warm up period of the second sensor, wherein the first, second, and third zoom factor ranges are determined using the buffer factor, a field of
view (FOV) ratio between the first sensor and the second sensor and a maximum zoom factor of the imaging device, the FOV ratio is a ratio between the FOV of the first sensor and a FOV of the second sensor and the FOV ratio is less than the maximum zoom factor of the imaging device.
30. The non-transitory computer-readable medium of claim 29, wherein the first zoom factor range is between 1.0 and a difference between the FOV ratio and the buffer factor, the second zoom factor range is between a difference between the FOV ratio and the buffer factor and the sum of the field of view ratio and the buffer factor, and the third zoom factor range is between the sum of the FOV ratio and the buffer factor and the maximum zoom factor.
Priority Applications (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
EP16708490.4A EP3257239B1 (en) | 2015-02-13 | 2016-01-27 | Systems and methods for power optimization for imaging devices with dual cameras |
CN201680006562.4A CN107211093B (en) | 2015-02-13 | 2016-01-27 | System and method for power optimization for imaging device with dual cameras |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US14/622,121 | 2015-02-13 | ||
US14/622,121 US9800798B2 (en) | 2015-02-13 | 2015-02-13 | Systems and methods for power optimization for imaging devices with dual cameras |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2016130325A1 true WO2016130325A1 (en) | 2016-08-18 |
Family
ID=55485291
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/US2016/015172 WO2016130325A1 (en) | 2015-02-13 | 2016-01-27 | Systems and methods for power optimization for imaging devices with dual cameras |
Country Status (4)
Country | Link |
---|---|
US (1) | US9800798B2 (en) |
EP (1) | EP3257239B1 (en) |
CN (1) | CN107211093B (en) |
WO (1) | WO2016130325A1 (en) |
Cited By (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN107959773A (en) * | 2016-10-18 | 2018-04-24 | 三星电子株式会社 | The electronic device of shooting image |
EP3328063A1 (en) * | 2016-11-29 | 2018-05-30 | Guangdong Oppo Mobile Telecommunications Corp., Ltd. | Control method, control device and electronic device |
EP3328055A1 (en) * | 2016-11-29 | 2018-05-30 | Guangdong Oppo Mobile Telecommunications Corp., Ltd. | Control method, control device and electronic device |
WO2018113055A1 (en) * | 2016-12-24 | 2018-06-28 | 华为技术有限公司 | Camera control method and terminal |
WO2018156273A1 (en) * | 2017-02-23 | 2018-08-30 | Qualcomm Incorporated | Adjustment for cameras for low power mode operation |
EP3641301A4 (en) * | 2017-07-10 | 2020-05-20 | Guangdong Oppo Mobile Telecommunications Corp., Ltd. | Method and apparatus for inhibiting aec jump, and terminal device |
WO2021061139A1 (en) * | 2019-09-26 | 2021-04-01 | Google Llc | Transitioning between cameras of a multiple-camera system |
Families Citing this family (78)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN113472989A (en) | 2012-11-28 | 2021-10-01 | 核心光电有限公司 | Multi-aperture imaging system and method for acquiring images by multi-aperture imaging system |
CN109194849B (en) | 2013-06-13 | 2021-01-15 | 核心光电有限公司 | Double-aperture zooming digital camera |
WO2015001440A1 (en) | 2013-07-04 | 2015-01-08 | Corephotonics Ltd. | Miniature telephoto lens assembly |
EP3028443A2 (en) | 2013-08-01 | 2016-06-08 | Corephotonics Ltd. | Thin multi-aperture imaging system with auto-focus and methods for using same |
US9392188B2 (en) | 2014-08-10 | 2016-07-12 | Corephotonics Ltd. | Zoom dual-aperture camera with folded lens |
WO2016036412A1 (en) | 2014-09-02 | 2016-03-10 | Apple Inc. | Remote camera user interface |
CN112327463B (en) | 2015-01-03 | 2022-10-14 | 核心光电有限公司 | Miniature telephoto lens module and camera using the same |
KR101914894B1 (en) | 2015-04-02 | 2018-11-02 | 코어포토닉스 리미티드 | Dual voice coil motor structure of dual optical module camera |
KR102480080B1 (en) | 2015-04-16 | 2022-12-22 | 코어포토닉스 리미티드 | Auto focus and optical imagestabilization in a compact folded camera |
CN110769149B (en) | 2015-04-23 | 2021-05-11 | 苹果公司 | Method, electronic device, and storage medium for processing content from multiple cameras |
WO2016189455A1 (en) | 2015-05-28 | 2016-12-01 | Corephotonics Ltd. | Bi-directional stiffness for optical image stabilization and auto-focus in a dual-aperture digital camera |
EP3598737B1 (en) * | 2015-08-13 | 2020-12-09 | Corephotonics Ltd. | Dual aperture zoom camera with video support and switching / non-switching dynamic control |
KR101993077B1 (en) | 2015-09-06 | 2019-06-25 | 코어포토닉스 리미티드 | Automatic focus and optical image stabilization by roll compensation of compact folding camera |
EP4254926A3 (en) | 2015-12-29 | 2024-01-31 | Corephotonics Ltd. | Dual-aperture zoom digital camera with automatic adjustable tele field of view |
WO2017205700A1 (en) * | 2016-05-25 | 2017-11-30 | Arris Enterprises Llc | Binary, ternary and quad tree partitioning for jvet coding of video data |
EP3518520B1 (en) | 2016-05-30 | 2020-08-26 | Corephotonics Ltd. | Rotational ball-guided voice coil motor |
US10009536B2 (en) | 2016-06-12 | 2018-06-26 | Apple Inc. | Applying a simulated optical effect based on data received from multiple camera sensors |
EP4020958B1 (en) | 2016-06-19 | 2023-10-25 | Corephotonics Ltd. | Frame synchronization in a dual-aperture camera system |
KR102524498B1 (en) * | 2016-07-06 | 2023-04-24 | 삼성전자주식회사 | The Electronic Device including the Dual Camera and Method for controlling the Dual Camera |
KR102390572B1 (en) | 2016-07-07 | 2022-04-25 | 코어포토닉스 리미티드 | Linear ball guided voice coil motor for folded optic |
WO2018007951A1 (en) | 2016-07-07 | 2018-01-11 | Corephotonics Ltd. | Dual-camera system with improved video smooth transition by image blending |
KR102600504B1 (en) * | 2016-09-07 | 2023-11-10 | 삼성전자주식회사 | Electronic Apparatus and the Controlling Method thereof |
KR102560780B1 (en) * | 2016-10-05 | 2023-07-28 | 삼성전자주식회사 | Image processing system including plurality of image sensors and electronic device including thereof |
EP3337153A1 (en) * | 2016-12-16 | 2018-06-20 | Vestel Elektronik Sanayi ve Ticaret A.S. | Camera system and method |
EP4246993A3 (en) | 2016-12-28 | 2024-03-06 | Corephotonics Ltd. | Folded camera structure with an extended light-folding-element scanning range |
US10884321B2 (en) | 2017-01-12 | 2021-01-05 | Corephotonics Ltd. | Compact folded camera |
KR102611056B1 (en) * | 2017-01-16 | 2023-12-07 | 엘지전자 주식회사 | Mobile terminal and Method for controlling the same |
WO2018154421A1 (en) | 2017-02-23 | 2018-08-30 | Corephotonics Ltd. | Folded camera lens designs |
KR102530535B1 (en) | 2017-03-15 | 2023-05-08 | 코어포토닉스 리미티드 | Cameras with panoramic scanning range |
US10142543B1 (en) | 2017-05-12 | 2018-11-27 | Mediatek Inc. | Power reduction in a multi-sensor camera device by on-demand sensors activation |
CN107277355A (en) * | 2017-07-10 | 2017-10-20 | 广东欧珀移动通信有限公司 | camera switching method, device and terminal |
CN107277480B (en) | 2017-07-10 | 2019-03-29 | Oppo广东移动通信有限公司 | White balance synchronous method, device and terminal device |
US10917586B2 (en) | 2017-08-22 | 2021-02-09 | Samsung Electronics Co., Ltd. | Electronic device providing switching between signals output from concurrently operable image sensors |
US10904512B2 (en) | 2017-09-06 | 2021-01-26 | Corephotonics Ltd. | Combined stereoscopic and phase detection depth mapping in a dual aperture camera |
US10951834B2 (en) | 2017-10-03 | 2021-03-16 | Corephotonics Ltd. | Synthetically enlarged camera aperture |
CN113075776B (en) | 2017-11-23 | 2023-08-29 | 核心光电有限公司 | mobile electronic device |
CN114609746A (en) | 2018-02-05 | 2022-06-10 | 核心光电有限公司 | Folding camera device |
CN113568251B (en) | 2018-02-12 | 2022-08-30 | 核心光电有限公司 | Digital camera and method for providing focus and compensating for camera tilt |
US10694168B2 (en) | 2018-04-22 | 2020-06-23 | Corephotonics Ltd. | System and method for mitigating or preventing eye damage from structured light IR/NIR projector systems |
CN114153107B (en) | 2018-04-23 | 2024-07-02 | 核心光电有限公司 | Camera and actuator |
US10488371B1 (en) | 2018-05-04 | 2019-11-26 | United Technologies Corporation | Nondestructive inspection using thermoacoustic imagery and method therefor |
US10685433B2 (en) | 2018-05-04 | 2020-06-16 | Raytheon Technologies Corporation | Nondestructive coating imperfection detection system and method therefor |
US10902664B2 (en) | 2018-05-04 | 2021-01-26 | Raytheon Technologies Corporation | System and method for detecting damage using two-dimensional imagery and three-dimensional model |
US11268881B2 (en) | 2018-05-04 | 2022-03-08 | Raytheon Technologies Corporation | System and method for fan blade rotor disk and gear inspection |
US10943320B2 (en) | 2018-05-04 | 2021-03-09 | Raytheon Technologies Corporation | System and method for robotic inspection |
US11079285B2 (en) | 2018-05-04 | 2021-08-03 | Raytheon Technologies Corporation | Automated analysis of thermally-sensitive coating and method therefor |
US10958843B2 (en) * | 2018-05-04 | 2021-03-23 | Raytheon Technologies Corporation | Multi-camera system for simultaneous registration and zoomed imagery |
US10914191B2 (en) | 2018-05-04 | 2021-02-09 | Raytheon Technologies Corporation | System and method for in situ airfoil inspection |
US10928362B2 (en) | 2018-05-04 | 2021-02-23 | Raytheon Technologies Corporation | Nondestructive inspection using dual pulse-echo ultrasonics and method therefor |
US10473593B1 (en) | 2018-05-04 | 2019-11-12 | United Technologies Corporation | System and method for damage detection by cast shadows |
TWI693828B (en) * | 2018-06-28 | 2020-05-11 | 圓展科技股份有限公司 | Image-capturing device and method for operating the same |
WO2020015754A1 (en) * | 2018-07-19 | 2020-01-23 | 杭州海康威视数字技术股份有限公司 | Image capture method and image capture device |
JP7028983B2 (en) | 2018-08-04 | 2022-03-02 | コアフォトニクス リミテッド | Switchable continuous display information system on the camera |
WO2020039302A1 (en) | 2018-08-22 | 2020-02-27 | Corephotonics Ltd. | Two-state zoom folded camera |
DK201870623A1 (en) | 2018-09-11 | 2020-04-15 | Apple Inc. | User interfaces for simulated depth effects |
US10645294B1 (en) | 2019-05-06 | 2020-05-05 | Apple Inc. | User interfaces for capturing and managing visual media |
US11770601B2 (en) | 2019-05-06 | 2023-09-26 | Apple Inc. | User interfaces for capturing and managing visual media |
US11128792B2 (en) | 2018-09-28 | 2021-09-21 | Apple Inc. | Capturing and displaying images with multiple focal planes |
US11321857B2 (en) | 2018-09-28 | 2022-05-03 | Apple Inc. | Displaying and editing images with depth information |
US11287081B2 (en) | 2019-01-07 | 2022-03-29 | Corephotonics Ltd. | Rotation mechanism with sliding joint |
CN111971956B (en) | 2019-03-09 | 2021-12-03 | 核心光电有限公司 | Method and system for dynamic stereo calibration |
US11706521B2 (en) | 2019-05-06 | 2023-07-18 | Apple Inc. | User interfaces for capturing and managing visual media |
KR102425865B1 (en) | 2019-07-31 | 2022-07-27 | 코어포토닉스 리미티드 | System and method for creating background blur in camera panning or motion |
CN112422805B (en) * | 2019-08-22 | 2022-02-18 | 华为技术有限公司 | Shooting method and electronic equipment |
KR20210042621A (en) * | 2019-10-10 | 2021-04-20 | 삼성전자주식회사 | Electronic device and method for controlling zoom rate of camera thereof |
US11659135B2 (en) | 2019-10-30 | 2023-05-23 | Corephotonics Ltd. | Slow or fast motion video using depth information |
WO2021116851A2 (en) | 2019-12-09 | 2021-06-17 | Corephotonics Ltd. | Systems and methods for obtaining a smart panoramic image |
US11949976B2 (en) | 2019-12-09 | 2024-04-02 | Corephotonics Ltd. | Systems and methods for obtaining a smart panoramic image |
CN114641805A (en) | 2020-02-22 | 2022-06-17 | 核心光电有限公司 | Split screen feature for macro photography |
CN114144898B (en) | 2020-04-26 | 2022-11-04 | 核心光电有限公司 | Temperature control for Hall bar sensor calibration |
KR102495627B1 (en) | 2020-05-17 | 2023-02-06 | 코어포토닉스 리미티드 | Image stitching in the presence of a full-field reference image |
US11770609B2 (en) | 2020-05-30 | 2023-09-26 | Corephotonics Ltd. | Systems and methods for obtaining a super macro image |
US11637977B2 (en) | 2020-07-15 | 2023-04-25 | Corephotonics Ltd. | Image sensors and sensing methods to obtain time-of-flight and phase detection information |
CN114730064A (en) | 2020-07-15 | 2022-07-08 | 核心光电有限公司 | Viewpoint aberration correction for scanning folded cameras |
CN114270145B (en) | 2020-07-31 | 2024-05-17 | 核心光电有限公司 | Hall sensor-magnet geometry for large-stroke linear position sensing |
CN111970438B (en) * | 2020-08-03 | 2022-06-28 | Oppo广东移动通信有限公司 | Zoom processing method and device, equipment and storage medium |
CN116626960A (en) | 2020-08-12 | 2023-08-22 | 核心光电有限公司 | Method for optical anti-shake |
EP4204885A4 (en) | 2021-06-08 | 2024-03-06 | Corephotonics Ltd. | Systems and cameras for tilting a focal plane of a super-macro image |
Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20060187312A1 (en) * | 2005-02-18 | 2006-08-24 | Peter Labaziewicz | Digital camera using multiple lenses and image sensors to provide an extended zoom range |
US20070025713A1 (en) * | 2005-08-01 | 2007-02-01 | Eiji Hosono | Image-capturing device having multiple optical systems |
US20080030592A1 (en) * | 2006-08-01 | 2008-02-07 | Eastman Kodak Company | Producing digital image with different resolution portions |
US20140232905A1 (en) * | 2013-02-21 | 2014-08-21 | Samsung Electronics Co., Ltd. | Method for dual recording shooting and electronic device thereof |
WO2014199338A2 (en) * | 2013-06-13 | 2014-12-18 | Corephotonics Ltd. | Dual aperture zoom digital camera |
Family Cites Families (14)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5870139A (en) * | 1995-08-28 | 1999-02-09 | Ultrak, Inc. | Method and system for video multiplexing |
US7071971B2 (en) * | 1997-08-25 | 2006-07-04 | Elbex Video Ltd. | Apparatus for identifying the scene location viewed via remotely operated television camera |
US5974272A (en) * | 1997-10-29 | 1999-10-26 | Eastman Kodak Company | Parallax corrected image capture system |
WO2006067545A1 (en) * | 2004-12-23 | 2006-06-29 | Nokia Corporation | Multi-camera solution for electronic devices |
US7561191B2 (en) * | 2005-02-18 | 2009-07-14 | Eastman Kodak Company | Camera phone using multiple lenses and image sensors to provide an extended zoom range |
US8456515B2 (en) | 2006-07-25 | 2013-06-04 | Qualcomm Incorporated | Stereo image and video directional mapping of offset |
FI20085510L (en) | 2008-05-28 | 2009-11-29 | Valtion Teknillinen | A zoom-camera arrangement comprising several sub-cameras |
US8115831B2 (en) * | 2008-08-04 | 2012-02-14 | Freedom Scientific, Inc. | Portable multi position magnifier camera |
US8941706B2 (en) | 2010-04-07 | 2015-01-27 | Apple Inc. | Image processing for a dual camera mobile device |
CN102053475A (en) * | 2010-10-15 | 2011-05-11 | 肖继忠 | Single camera based omnibearing stereo vision system |
US20130235234A1 (en) * | 2012-03-12 | 2013-09-12 | Megan Lyn Cucci | Digital camera having multiple image capture systems |
JP2013198016A (en) * | 2012-03-21 | 2013-09-30 | Casio Comput Co Ltd | Imaging apparatus |
US8913138B2 (en) | 2012-12-21 | 2014-12-16 | Technologies Humanware Inc. | Handheld magnification device with a two-camera module |
US20140184854A1 (en) * | 2012-12-28 | 2014-07-03 | Motorola Mobility Llc | Front camera face detection for rear camera zoom function |
-
2015
- 2015-02-13 US US14/622,121 patent/US9800798B2/en active Active
-
2016
- 2016-01-27 WO PCT/US2016/015172 patent/WO2016130325A1/en active Application Filing
- 2016-01-27 CN CN201680006562.4A patent/CN107211093B/en not_active Expired - Fee Related
- 2016-01-27 EP EP16708490.4A patent/EP3257239B1/en active Active
Patent Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20060187312A1 (en) * | 2005-02-18 | 2006-08-24 | Peter Labaziewicz | Digital camera using multiple lenses and image sensors to provide an extended zoom range |
US20070025713A1 (en) * | 2005-08-01 | 2007-02-01 | Eiji Hosono | Image-capturing device having multiple optical systems |
US20080030592A1 (en) * | 2006-08-01 | 2008-02-07 | Eastman Kodak Company | Producing digital image with different resolution portions |
US20140232905A1 (en) * | 2013-02-21 | 2014-08-21 | Samsung Electronics Co., Ltd. | Method for dual recording shooting and electronic device thereof |
WO2014199338A2 (en) * | 2013-06-13 | 2014-12-18 | Corephotonics Ltd. | Dual aperture zoom digital camera |
Cited By (12)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN107959773A (en) * | 2016-10-18 | 2018-04-24 | 三星电子株式会社 | The electronic device of shooting image |
EP3328063A1 (en) * | 2016-11-29 | 2018-05-30 | Guangdong Oppo Mobile Telecommunications Corp., Ltd. | Control method, control device and electronic device |
EP3328055A1 (en) * | 2016-11-29 | 2018-05-30 | Guangdong Oppo Mobile Telecommunications Corp., Ltd. | Control method, control device and electronic device |
US10412298B2 (en) | 2016-11-29 | 2019-09-10 | Guangdong Oppo Mobile Telecommunications Corp, Ltd. | Control method, control device and electronic device |
WO2018113055A1 (en) * | 2016-12-24 | 2018-06-28 | 华为技术有限公司 | Camera control method and terminal |
EP3550822A4 (en) * | 2016-12-24 | 2020-01-29 | Huawei Technologies Co., Ltd. | Camera control method and terminal |
WO2018156273A1 (en) * | 2017-02-23 | 2018-08-30 | Qualcomm Incorporated | Adjustment for cameras for low power mode operation |
US10334149B2 (en) | 2017-02-23 | 2019-06-25 | Qualcomm Incorporated | Adjustment for cameras for low power mode operation |
EP3641301A4 (en) * | 2017-07-10 | 2020-05-20 | Guangdong Oppo Mobile Telecommunications Corp., Ltd. | Method and apparatus for inhibiting aec jump, and terminal device |
US10819913B2 (en) | 2017-07-10 | 2020-10-27 | Guangdong Oppo Mobile Telecomunnications Corp., Ltd. | Method and apparatus for inhibiting AEC jump, and terminal device |
EP4040776A1 (en) * | 2017-07-10 | 2022-08-10 | Guangdong Oppo Mobile Telecommunications Corp., Ltd. | Method and apparatus for inhibiting aec value jump, and terminal device |
WO2021061139A1 (en) * | 2019-09-26 | 2021-04-01 | Google Llc | Transitioning between cameras of a multiple-camera system |
Also Published As
Publication number | Publication date |
---|---|
CN107211093A (en) | 2017-09-26 |
EP3257239B1 (en) | 2021-05-26 |
US20160241793A1 (en) | 2016-08-18 |
EP3257239A1 (en) | 2017-12-20 |
US9800798B2 (en) | 2017-10-24 |
CN107211093B (en) | 2020-02-28 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US9800798B2 (en) | Systems and methods for power optimization for imaging devices with dual cameras | |
AU2020201023B2 (en) | Adaptive transfer function for video encoding and decoding | |
JP6297682B2 (en) | System and method for super-resolution of user-selected regions of interest | |
EP2878121B1 (en) | Method and apparatus for dual camera shutter | |
US9232125B2 (en) | Method of eliminating a shutter-lag, camera module, and mobile device having the same | |
JP6676539B2 (en) | System and method for delaying power consumption by post-processing sensor data | |
US20130201360A1 (en) | Method of changing an operation mode of a camera image sensor | |
US20100231735A1 (en) | Methods, Apparatuses, and Computer Program Products for Facilitating Concurrent Video Recording and Still Image Capture | |
US9971844B2 (en) | Adaptive image loading | |
US9626076B2 (en) | Display apparatus for displaying images and method thereof | |
US10681273B2 (en) | Mobile device including multiple cameras | |
US20150350533A1 (en) | Realtime capture exposure adjust gestures | |
US9189269B2 (en) | Apparatus and method for performing multi-tasking in portable terminal | |
EP4150894A1 (en) | Camera transition blending | |
JP2016116133A (en) | Imaging controller, control method, program therefor, and storage medium | |
US10769095B2 (en) | Image processing apparatus | |
KR102689351B1 (en) | Focusing method and apparatus applied to terminal devices and terminal devices | |
CN114979466B (en) | Shooting processing method and device and wireless communication module | |
KR20190022238A (en) | Mobile device including multiple cameras | |
US20110254995A1 (en) | Method and system for mitigating seesawing effect during autofocus | |
JP2022502970A (en) | Focusing methods and devices applicable to terminal devices, as well as terminal devices |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 16708490 Country of ref document: EP Kind code of ref document: A1 |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
REEP | Request for entry into the european phase |
Ref document number: 2016708490 Country of ref document: EP |