US20230014865A1 - Method and apparatus for creating and displaying visual media on a device - Google Patents
Method and apparatus for creating and displaying visual media on a device Download PDFInfo
- Publication number
- US20230014865A1 US20230014865A1 US17/951,358 US202217951358A US2023014865A1 US 20230014865 A1 US20230014865 A1 US 20230014865A1 US 202217951358 A US202217951358 A US 202217951358A US 2023014865 A1 US2023014865 A1 US 2023014865A1
- Authority
- US
- United States
- Prior art keywords
- visual content
- display
- viewership
- circular
- orientation
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Abandoned
Links
- 230000000007 visual effect Effects 0.000 title claims abstract description 307
- 238000000034 method Methods 0.000 title abstract description 30
- 230000033001 locomotion Effects 0.000 claims abstract description 73
- 238000004891 communication Methods 0.000 description 43
- 230000005484 gravity Effects 0.000 description 37
- 238000003384 imaging method Methods 0.000 description 27
- 230000000694 effects Effects 0.000 description 13
- 235000009508 confectionery Nutrition 0.000 description 12
- 230000004048 modification Effects 0.000 description 11
- 238000012986 modification Methods 0.000 description 11
- 230000008901 benefit Effects 0.000 description 7
- 230000000875 corresponding effect Effects 0.000 description 5
- 239000004615 ingredient Substances 0.000 description 5
- 230000008569 process Effects 0.000 description 5
- 239000013598 vector Substances 0.000 description 5
- 238000010586 diagram Methods 0.000 description 4
- 230000006870 function Effects 0.000 description 4
- 230000003287 optical effect Effects 0.000 description 4
- 230000007704 transition Effects 0.000 description 4
- 230000009191 jumping Effects 0.000 description 3
- 238000013507 mapping Methods 0.000 description 3
- 230000001133 acceleration Effects 0.000 description 2
- 230000004075 alteration Effects 0.000 description 2
- 230000010267 cellular communication Effects 0.000 description 2
- 230000008859 change Effects 0.000 description 2
- 230000001276 controlling effect Effects 0.000 description 2
- 238000013500 data storage Methods 0.000 description 2
- 230000001419 dependent effect Effects 0.000 description 2
- 238000001514 detection method Methods 0.000 description 2
- 206010009696 Clumsiness Diseases 0.000 description 1
- 230000009471 action Effects 0.000 description 1
- 230000001413 cellular effect Effects 0.000 description 1
- 230000002596 correlated effect Effects 0.000 description 1
- 230000003247 decreasing effect Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 210000004247 hand Anatomy 0.000 description 1
- 239000012535 impurity Substances 0.000 description 1
- 238000000691 measurement method Methods 0.000 description 1
- 239000000203 mixture Substances 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 210000000707 wrist Anatomy 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N5/00—Details of television systems
- H04N5/222—Studio circuitry; Studio devices; Studio equipment
- H04N5/262—Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects
- H04N5/2628—Alteration of picture size, shape, position or orientation, e.g. zooming, rotation, rolling, perspective, translation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
- G06F3/0484—Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
- G06F3/04845—Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range for image manipulation, e.g. dragging, rotation, expansion or change of colour
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
- G06F3/0487—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
- G06F3/0488—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/60—Rotation of whole images or parts thereof
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N5/00—Details of television systems
- H04N5/222—Studio circuitry; Studio devices; Studio equipment
- H04N5/262—Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects
- H04N5/268—Signal distribution or switching
Definitions
- the present exemplary embodiment relates to methods and devices for creating and displaying visual content. It finds particular application in conjunction with providing a video and image content format that (i) fills the entire display of a media device regardless of the screen size and orientation, and (ii) seamlessly transitions the displayed visual content as the viewership device is moved and will be described with particular reference thereto. However, it is to be appreciated that the present exemplary embodiment is also amenable to other like applications.
- the software of a media device processes the captured visual content data and the visual content may be displayed on an associated rectangular display or on another viewership device in either a fixed orientation, for devices that have a generally fixed position such as a television or in one of two discrete orientations relative to a rotational position of the media device (i.e., a portrait or landscape orientation).
- the displayed visual content either (i) fills the entire screen or (ii) is accompanied by margin effects (“margins”), generally embodied as a pair of top and bottom and/or right and left artifacts, that appear with the visual content.
- margin effects generally embodied as a pair of top and bottom and/or right and left artifacts, that appear with the visual content.
- the visual content is centered within the margin effects, commonly known as letterboxing and pillarboxing.
- visual content is formatted based on a desired end use and therefore a multitude of media formats exist.
- Each type of media format, cinema, television, internet, smart device, etc. is configured with an aspect ratio.
- the multitude of aspect ratios is inconvenient for both visual content creators and consumers of that visual content as the selected viewership device by a consumer may not match the format of the created visual content.
- Modifications, including margin effects are particularly prevalent when visual content is created with one capture device and the visual content is shared to other devices with a display having an aspect ratio that is different from the aspect ratio of the captured visual content. Due to the multitude of different form factors for media devices and displays, it is difficult to create an equal media experience for every user accessing the visual content. That is, those users with displays that match the aspect ratio of the visual content, view the visual content as filling the entire display. Those users with a display that does not match the aspect ratio view the visual content with undesirable margin effects.
- a user viewing a received visual content on a portable media device may rotate the media device from a portrait orientation to a landscape orientation (or vice versa) if the margins are present and the user would like to view the visual content while minimizing the margin effect, i.e., have the image fill as much of the screen as possible.
- many users of portable media devices do not bother rotating the media device to view the visual content in a manner that avoids the margin effect—for whatever reason (e.g., takes too long, don't want to be bothered, etc.).
- a media consumer is able to rotate a viewership device to selectively view portions of captured visual content.
- the present disclosure relates to devices and methods for formatting and viewing visual content to produce a format that may adapt and fill the entire, available dimensions of a display screen.
- the devices and methods find particular application in conjunction with images (photographs) and videos captured by smart devices incorporating a camera and will be described with particular thereto. However, it is to be appreciated that the present disclosure is also amenable to other like applications.
- a formatting device for visual content includes a motion sensor, an image sensor configured to capture visual content data, a storage medium that at least temporarily stores visual content data therein; and a processor configured to link visual content data to a horizon line defined by the motion sensor.
- the visual content data includes at least one image.
- Some methods include providing an electronic device including at least one motion sensor, an image sensor, a storage medium/memory configured to at least temporarily store visual content data captured by the image sensor and a processor configured to execute a set of instructions.
- the instructions include generating circular visual content (e.g. cropping rectangular content to a circle format where the cropped visual content fills an entirety of the area of the cropped circle) and linking the circular visual content to a motion sensor of the device.
- One preferred method includes the steps of (i) generating circularly formatted visual content, wherein the circularly formatted visual content is located within the confines of a circle having a circle edge, (ii) linking the circularly formatted visual content to one of a stable horizon line and a preferred direction with respect to gravity; and (iii) mapping an orientation dependent selection of the circularly formatted visual content data to a rectangular display of a display device, wherein at least one corner of the rectangular display abuts the circle edge of the circularly formatted visual content so that the orientation dependent selection of the circularly formatted visual content fills the entirety of the rectangular display.
- imaging devices including a circular image sensor
- the imaging device includes an electronic processor, a circular imaging sensor; and at least one circularly shaped optical lens.
- the circularly shaped optical lens is in optical communication with the circular imaging sensor such that the majority if not all information captured by the circular lens is communicated and captured by the circular image sensor.
- the electronic processor of the imaging device is configured to link a stable horizon line to captured visual content via a motion sensor of the imaging device, a user command, or a default orientation setting.
- FIG. 1 is a block diagram illustrating an example of an electronic device in accordance with the present disclosure
- FIG. 2 is an example of a user interface of an electronic device in accordance with the present disclosure
- FIG. 3 is a graphical representation of a viewer device presenting visual content in accordance with the present disclosure
- FIG. 4 is a graphical representation of a viewer device presenting visual content in accordance with the present disclosure
- FIG. 5 is a graphical representation of a viewer device presenting visual content in accordance with the present disclosure.
- FIG. 6 is a diagram of a method for formatting visual content in accordance with the present disclosure.
- FIG. 7 A-C is a diagram of prior art rectangular image sensors compared to circular image sensors in accordance with the present disclosure.
- FIGS. 8 A and 8 B illustrate a device and visual content in accordance with the present disclosure.
- the term “comprising” may include the embodiments “consisting of” and “consisting essentially of.”
- the terms “comprise(s),” “include(s),” “having,” “has,” “can,” “contain(s),” and variants thereof, as used herein, are intended to be open-ended transitional phrases, terms, or words that require the presence of the named ingredients/components/steps and permit the presence of other ingredients/components/steps.
- compositions, articles, or processes as “consisting of” and “consisting essentially of” the enumerated ingredients/components/steps, which allows the presence of only the named ingredients/components/steps, along with any impurities that might result therefrom, and excludes other ingredients/components/steps.
- Original aspect ratio is the aspect ratio in which visual content is originally produced.
- a user operating an image capture device such as a camera, may select the OAR of the visual content to be captured.
- the user may select a specific aspect ratio based on the desired end use display for the visual content.
- a display device as described herein is a device having a function of controlling visual content displayed on an associated display, such as a display panel.
- the desired end use may be a wide format television, cinema screen, personal computer display, tablet, smartphone display, smartwatch, etc.
- the OAR of the visual content may be modified.
- viewing a cinema format a substantially rectangular visual content format
- a standard television a substantially square display
- MAR modified aspect ratio
- the visual content is stretched to fill in the area according to the new ratio.
- Modified aspect ratio (MAR) is the aspect ratio assigned in order to fit a type of screen, and different from the dimensions of the device in which the visual content is captured.
- the visual content is letterboxed.
- the visual content is pillarboxed.
- the visual content is both letterboxed and pillarboxed.
- Letterboxing is a modification to the presentation of visual content captured in a widescreen format and transferred to a display device having a display of smaller width that preserves the visual content's OAR.
- the display device presents the visual content with bars, e.g., black bars positioned above and below the content.
- Pillarboxing is a modification of the presentation of visual content wherein the OAR width is smaller than the width of the display of the display device. In pillarboxing, black bars are placed on one or more sides of the visual content in order to preserve the OAR of the visual content.
- a display device may be a smart device, such as a smartphone or tablet, and can view visual content in a portrait mode and a landscape mode. That is, display devices are typically equipped with a sensor that detects the orientation of the display device and switches a display mode (portrait and landscape) based on the detected orientation of the display device.
- a portrait mode is perhaps best described as a thin, tall, rectangular-like shape. In other words, the height of the display is greater than the width, yielding a vertical aspect ratio, for example and without limitation 9:16 and 4:5 aspect ratios.
- a landscape mode is perhaps best described as a wide fat rectangular shape.
- the width of the display is greater than the height creating a horizontal aspect ratio, for example and without limitation 1.91:1.
- horizontal visual content having a horizontal aspect ratio and intended for horizontal displays
- the visual content is capable of filling the majority of the display.
- the viewer, rotating the viewership device to initiate a portrait mode views the horizontal visual content with modifications, often including margin effect combined with a zoom effect that makes the image appear smaller or further away.
- the portrait and landscape modes for capturing and viewing visual content are discrete. That is, the display presents the visual content in either a portrait mode or landscape mode and abruptly switches between the two modes, usually upon reaching a threshold rotation of the viewership device. In this manner, the visual content is viewed as “jumping” from one mode to another without intermediate states as the device rotates. This jumping effect is further emphasized to a user when the visual content is modified such that different margin effects are present in one or both orientations.
- an electronic device 100 is configured to generate and modify visual content such that the aspect ratio is preserved across display devices having different aspect ratios.
- a block diagram illustrates an example of an electronic device 100 .
- the electronic device 100 includes at least one motion sensor 110 configured to detect an orientation of the electronic device 100 , a processor 120 , and a storage medium/memory 130 . These components are configured together in a manner well known in the art.
- the electronic device 100 includes an image sensor 140 , such as a camera, configured to capture visual content. In some embodiments, the electronic device 100 also includes a display 150 , configured to display visual content to a viewer.
- an image sensor 140 such as a camera
- the electronic device 100 also includes a display 150 , configured to display visual content to a viewer.
- the electronic device 100 is equipped with a communication device 160 configured to send visual content to another device or storage (storage including a hard drive, memory, cloud storage, media platforms).
- the communication device 160 may include wired communication components, wireless communication components, cellular communication components, near field communication (NFC) components, Bluetooth® components, Wi-Fi® components, and other communication components to provide communication via other modalities.
- NFC near field communication
- Bluetooth® components Wi-Fi® components
- This list of exemplary communications devices is intended to be exemplary and not preclude the use of one or more of these components alternately or in combination or preclude the use of still other communication components that perform substantially the same function in substantially the same way.
- the electronic device 100 may also include a user interface 170 configured to receive commands from a user of the device 100 .
- the user interface 170 may include without limitation, a touchscreen device, a keyboard, a mouse, motion sensors, buttons, knobs, voice actuation, and the like.
- the motion sensor 110 may be a gravity sensor, an accelerometer, a gyroscope, and/or any sensor(s) configured to detect one or more motions and/or an orientation associated with the electronic device 100 .
- a motion capable of detection by the motion sensor 110 may be a tilt, wherein the device is moved to a sloping position, i.e., at an angle with respect to a horizontal plane.
- Another motion capable of detection by the motion sensor 110 may be a rotation, wherein a rotation is moving the electronic device 100 within a single plane, e.g. orientating the electronic device 100 from a portrait orientation to a landscape orientation, vice versa, or to intermediate orientations between a portrait and landscape orientation.
- a motion may also include an acceleration, wherein the motion sensor 110 may detect a change in velocity of the electronic device 100 in any direction.
- a motion may also be a combination of any or all movements of the electronic device 100 .
- the electronic device 100 may include multiple motion sensors, for example and without limitation, the electronic device 100 may include a gravity sensor, an accelerometer, and a gyroscope. While the motion sensors 110 may be located anywhere on the electronic device 100 , typically the motion sensors 110 are internal components of the electronic device 100 and in electronic communication with the processor 120 .
- a gravity sensor is a motion sensor 110 that is configured to measure an orientation with respect to the direction of gravity.
- the gravity sensor enables the electronic device 100 (via the processor 120 ) to recognize the direction of gravity relative to the device based, for example, on calculated three-dimensional vectors.
- the gravity sensor may indicate an orientation, such as a degree of rotation with respect to the direction of gravity, of the electronic device 100 .
- the detected orientation may be combined with the acquisition of visual content by the electronic device 100 . That is, the visual content may be tied to the direction of gravity such that the visual content is weighted.
- “weighted” visual content is visual content that is aligned with the direction of gravity.
- the visual content when capturing visual content via an electronic device 100 , (e.g., a tablet or a smartphone) orientated in a landscape mode, wherein the long axis of the electronic device is perpendicular to the direction of gravity, the visual content is assigned a gravity vector that is perpendicular to the long axis of the electronic device 100 .
- the electronic device 100 is rotated at an angle, for example 30 degrees with respect to gravity (detected by the motion sensor 110 )
- the visual content will be tagged with a gravity vector that is also offset by 30 degrees.
- the tagged gravity vector When the visual content is displayed on a viewership device, the tagged gravity vector will be aligned with a detected direction of gravity of a viewership device (described in greater detail below).
- the gravity sensor may also be used to determine a tilting direction of the electronic device 100 .
- An accelerometer is a motion sensor 110 configured to detect a change in velocity during a time period and senses an acceleration.
- a three-axis accelerometer may include multiple motion sensors positioned in the x, y, and z-axis directions.
- the processor 120 of the electronic device 100 receives from the accelerometer values measured in the multi-axis directions as vector values. The processor 120 may then determine a direction in which the electronic device 100 is rotated or tilted based on values obtained with respect to the three axes.
- a gyroscope is a motion sensor 110 configured to calculate an angle to which the device 100 rotates with respect to an axis. This may be represented as a numerical value.
- a three-axis gyroscope calculates the degree to which the device 100 rotates with respect to three axes.
- a motion sensor 110 may be an external sensor in communication with the electronic device 100 . That is, the motion sensor 110 may be a plug-and-play device configured to connect with an input/output (I/O) interface of an electronic device 100 and communicate with the processor 120 .
- I/O input/output
- the processor 120 is configured to control the operations of the electronic device 100 and may execute applications, apps, and instructions stored in the device memory 130 and/or accessible via the communication device 160 . Generally, the processor 120 determines whether to rotate visual content displayed on the display 150 based on data obtained from at least one motion sensor 110 .
- the electronic device 100 is equipped with an image sensor 140 , embodied as a camera or similar sensor for acquiring visual content.
- a user of the electronic device 100 may take photographs or record video by manipulating a user interface 170 to employ the image sensor 140 .
- a motion sensor 110 may apply a “weight” to the visual content.
- a viewership device displaying the visual content will orient the visual content such that the assigned weight or a preferred direction is in line with the true direction of gravity. More about the visual content and its association with a weight and/or a preferred direction is discussed in greater detail below.
- the electronic device 100 may store the visual content into the memory 130 .
- the electronic device 100 may upload the visual content to an external storage device, such as an external hard drive, cloud, internet storage, etc. via the communication device 160 .
- the communication device 160 may support two-way communication with other devices, cloud storage, networks, and external storage devices by using a variety of technologies.
- the communication device 160 may include a network interface component or other suitable device to interface with a network.
- the communication device 160 may include wired communication components, wireless communication components, cellular communication components, near field communication (NFC) components, Bluetooth® components, Wi-Fi® components, and other communication components to provide communication via other modalities. This list of communication devices is intended to be exemplary and not preclude the use of one or more of these components alternately or in combination or preclude the use of still other communication components.
- the communication via the communication device 160 may be for the purposes of storing visual content, obtaining visual content, and sharing visual content with other devices including viewership devices.
- the communication device 160 is configured to connect to a social media network wherein the visual content may be uploaded, stored, or otherwise accessible by one or more users of the social media network.
- FIG. 2 is an exemplary embodiment of a software application run on an electronic device 200 controlled by device processor 220 configured to capture visual content in accordance with the present disclosure.
- the electronic device 200 includes a touch screen display 250 which simultaneously displays visual content 252 and provides a user interface for user operation of the electronic device 200 and various features presented by the software program.
- the electronic device 200 may be operable in a video mode 254 or a photograph mode 256 by selecting the desired mode via the user interface.
- a video mode is an operational mode of an onboard image sensor configured to capture a series of images over time (frames).
- a photograph mode is an operational mode of an onboard image sensor configured to acquire at least one or several images over a short period of time.
- An image sensor 240 in electronic communication with the processor 220 is configured to capture visual content 252 which is displayed on the display 250 . That is, the electronic device 200 may be a visual content capture and formatting device as well as a visual content viewership device.
- an electronic device 200 for example and without limitation, a smartphone or tablet
- an image representing all the visual information captured by the image sensor of the camera is presented on the display 250 .
- visual content would fill both the circle portion 258 and blacked out portion (outside the perimeter of image 258 ) shown on display 250 to a user while being captured.
- FIG. 2 shows the result of cropping all visual information captured by the image sensor 240 .
- the software of the electronic device 200 projects a sweet spot target 266 (indicated by the dotted circle) on the visual information as it is captured.
- the sweet spot target 266 contains the visual content that will always be displayed on a viewership device regardless of orientation and aspect ratio for reasons which will become more apparent below.
- the sweet spot target 266 is generally the geometric centered portion of the visual information displayed. This may be the default position of the sweet spot target 266 . In other embodiments, a user may select the geometric position of the sweet spot target 266 within the bounds of the visual information.
- the processor 220 executes instructions stored in either the memory 230 or instructions accessible via communication device 260 to perform a cropping method on visual information that is captured by the image sensor 240 .
- the method includes (i) cropping the visual information captured by the image sensor 240 to a circle format 258 visual content and (ii) coding/syncing the cropped circular visual content.
- the circle format 258 does not exceed the narrower dimension of the visual information displayed on the electronic device in either portrait or landscape, or at an angle therebetween.
- the rectangular visual content may be the visual information acquired from the image sensor 240 of the electronic device 200 , or the rectangular visual content may be visual content captured by another imaging device (such a digital point and shoot camera or wide format cinema camera) and imported to the electronic device 200 for formatting purposes.
- the first part of the cropping method removes portions of the visual content such that the remaining visual content is circular in format.
- the visual information may fill the entire display 250 , including the parts that are represented as blacked out. Cropping the visual information removes portions of the visual information leaving the circular visual content 258 .
- the resolution of the visual content (the detail held by the visual content) stays the same.
- the storage size of the visual content file itself may be reduced, providing in some embodiments, a memory saving function.
- the processor 220 is also configured to sync the cropped visual content to the motion sensors 110 of the electronic device 200 that captures the visual content.
- the circle format 258 of the visual content is coded to the motion sensors 110 in order to link the visual content to a stable reference or “horizon line” 269 .
- the stable horizon line is a reference line that is configured to be perpendicular to the direction of gravity.
- the visual content is synced with a gyroscope sensor of the electronic device 200 .
- the software of the electronic device 200 may obtain an image and determine a most likely preferred orientation of the visual content and assign or link the visual content to a reference line (e.g., horizon, gravity).
- a reference line e.g., horizon, gravity
- the orientation of the coded visual content is modified with respect to a detected direction of gravity by a motion sensor on a viewership device. This provides a reference point that allows the coded visual content to rotate on the display of a viewership device counter to an opposite rotation of the viewership device such that the visual content appears to the user as being stationary. That is a determined bottom 264 of the visual content is always associated with the direction of gravity.
- FIGS. 3 - 5 The effect of a stable viewership is best illustrated in FIGS. 3 - 5 and is described in greater detail below.
- the motion sensors 110 of the electronic device 200 define the orientation of the visual content, such as defining the top 262 and bottom 264 of the visual content 252 .
- the visual content 252 is weighted such that the bottom 264 of the visual content is the closest to the bottom-most portion of the display 250 .
- the visual content may be further edited by a user.
- Editing includes any enhancements or changes to the visual content appearance, including, but not limited to, changing the sweet spot, resolution, and zoom, as well as changing the saturation, brightness, color, graphics and the like to the visual content.
- the visual content is shared to any type of platform, including, but not limited to, social media, such as Facebook®, Twitter®, iMessage®, etc. It is to be appreciated that editing may be performed on the electronic device 200 or may be distributed to another device configured to receive the visual content for editing. In some embodiments, the editing of the visual content is performed in the cloud.
- the circular visual content 258 linked to the motion sensors 110 (circular coded visual content) during creation has many advantages.
- the visual content may be opened by any viewership device with access to the visual content.
- a viewership device may be variously embodied, for example the image capture device 200 itself may store an access the visual content as a viewership device.
- the circular coded visual content may be shared to other viewership devices via communication components that distribute the circular visual content 258 to a viewership device and/or networks, direct linking, etc. It is to be appreciated that a viewership device may be an electronic device equipped similar to electronic device 200 .
- a viewership device may also be a television display, computer monitor, cinema screen, and the like.
- the circular coded visual content 358 is presented on a viewership device having a viewer display 350 .
- the circular coded visual content 358 is automatically configured to fit the size of the viewer display 350 of the viewership device. This configuration enables the viewer display 350 , having four corners 351 a - d to touch the circle format 358 in four respective places. While a rectangular viewer display 350 is shown, it is contemplated that the viewership display 350 may be embodied as other shapes. That is, a triangular display may have its three corners each touch the edge of the circle format in three respective places. As an additional example, the viewership display may also be circular in shape wherein the circumference of the display touches the circumference of the visual content 358 .
- the horizon line 369 coded to the visual content 358 defines the orientation on which the visual content is presented on the display 350 .
- the horizon line 369 is parallel to a defined bottom edge 371 of a display 350 .
- a user may define the bottom edge 371 as either the short edge, similar to the orientation of the display shown in FIG. 3 , or the long edge, similar to the orientation of the display in FIG. 5 . This is usually done by changing the display settings of the computer and associated monitor. A user may wish to have a monitor oriented in a traditional landscape orientation, or a user may wish to have the monitor oriented in a portrait orientation. With the orientation defined by a user in the display settings, the horizon line may be linked to the visual content accordingly.
- the viewer display 350 is incorporated into a viewership device similar in nature to the electronic device 100 and 200 of FIGS. 1 and 2 respectively. That is, the viewership device includes at least a viewer display 350 , a processor, and at least one motion sensor.
- the viewership device may be a portable device that is able to be manipulated into a plurality of orientations for viewing the visual content.
- the circular coded visual content 358 has a first captured image diameter 359 .
- the circular coded visual content 358 is synced to a motion sensor, analogous to motion sensor 110 , in communication with a processor, analogous to processor 120 of the viewership device.
- the motion sensor is a gyroscope.
- a portion of the circular coded visual content 358 is mapped to fit the viewer display 350 of the viewership device. That is, a circumference 359 is correlated with the four corners 351 a - d of the viewer display 350 so that each corner abuts the circumference 359 .
- Sweet spot target 266 is a smaller diameter portion of the circular coded visual content 358 and specifically is dimensioned so that a diameter of the sweet spot 266 preferably creates a tangent at two locations 266 a , 266 b that are diametrically opposite and in tangent contact with the narrow sides of the rectangular viewer display 350 .
- the mapping enables the viewer display 350 having four corners 351 a - d to preferably touch the circle format 358 along the circumference 359 in four respective places.
- the circular visual content 358 is rotated in a direction counter of the rotation of the viewer display 350 with respect to the stable horizon line 369 . That is, the motion sensor of the viewership device links or syncs to the circular coded visual content 358 and calculates an angle, tilt, and/or rotation of the display 350 with respect to the horizon line 369 .
- the viewership device, and along with it, the viewer display 350 may be rotated about a plane defined by the circle format 358 .
- a non-circular display such as the rectangular display 350 is a viewfinder of the entire visual circular visual content 358 .
- the viewer display 350 may only “view” rectangular portion of the circular coded visual content 358 at a time.
- the viewed area is represented by the darker image confined within the viewer display 350 .
- the selected rectangular portion viewed on the rectangular display 350 includes the entire sweet spot target 266 no matter what the orientation of the viewer display 350 .
- Select annular portions i.e. between the sweet sport 266 and circumference diameter 359 will be shown on the rectangular display 350 .
- the angular segment of the annular portion that is seen on the rectangular display 350 will vary depending on the physical orientation of the viewer device but in each instance the sweet spot 266 is displayed. Stated another way, the remaining portions of the circular coded visual content, illustrated as the lighter regions of the visual content 352 outside the confines of the display 350 , may be viewed and explored on the viewer display 350 by rotating the viewer display to a desired orientation.
- the visual content captured in sweet spot target 266 (or a portion thereof) is preferably always present no matter the orientation of the viewer display 350 .
- the user of the viewership device is provided with a unique choice in selecting the viewership of the visual content 358 allowing the viewer to play an active role in viewing visual content 358 .
- the circular coded visual content 358 is continuously rotated as the viewer display 350 is rotated. In this way, there is no abrupt jump in the orientation of the visual content while transitioning the viewer display 350 from a portrait orientation (shown in FIG. 3 ) to a landscape orientation (shown in FIG. 5 ). That is, the viewer display 350 presents full screen intermediate orientations during rotation of the viewer display 350 which appear as a seamless continuous transition (i.e. without jumping) at a constant image resolution.
- a preferred method 600 of circularizing and coding visual content is provided.
- the method 600 is illustrated in FIG. 6 and performed by a device (capturing device, intermediate device/cloud, and/or viewership device) including an electronic processor configured to execute instructions provided on a non-transitory storage medium.
- the electronic device 100 of FIG. 1 may be employed to perform the method 600 .
- the method 600 includes a first step 602 of acquiring visual content. After acquiring the visual content, the method provides cropping the visual content to a circle format, illustrated at block 604 . After the visual content is cropped, a horizon line 369 is coded, at block 606 , to the cropped visual content creating circular coded visual content.
- the circular coded visual content may then be displayed, at block 608 , on a display device 350 (which could be the image capturing device or another display device) such that the circular coded visual content is mapped to a desired portion of the display.
- Visual content may be acquired, at block 602 , in several ways.
- visual content is created by an imaging sensor, e.g., imaging sensor 140 , in electronic communication with the processor 120 of the device 100 .
- visual content is acquired by importing/downloading visual content via a communication device 160 in electronic communication with the processor 120 of the device 100 .
- the communication device 160 may connect to an internet network, a cellular network, or the like.
- the communication device 160 may also acquire visual content via wired connections and/or wireless connections.
- the visual content may be stored on a storage medium 150 of the device for further processing.
- the visual content is then cropped to a circular format as illustrated and described with respect to FIG. 3 .
- visual content is initially generated in a rectangular form factor by an image sensor that is also rectangular in shape.
- the processor 120 then crops the rectangular visual content to create a circular format.
- existing imaging hardware i.e., non-circular image sensors
- visual content may be created/generated from cameras other than those in direct electronic communication with the processor.
- a second device may capture visual content in rectangular form and upload the visual content to a storage device/network.
- the device processor equipped with the software described herein to perform the method 600 may access the uploaded visual content and, in some cases, download the visual content to a storage medium 150 in electronic communication with the processor 120 .
- the processor 120 may then crop the visual content acquired from the external source.
- a motion sensor in electronic communication with the processor of the device is used to define and code a horizon line to the circular visual content.
- the motion sensor may be any sensor known to detect an orientation of visual content or the device including a display configured to present visual content.
- the motion sensor is a gyroscope.
- the motion sensor is an accelerometer.
- the motion sensor is a gravity sensor.
- the motion sensor is a plurality of sensors including any combination of two or more sensors described herein.
- a horizon line is a user-defined line coded to the cropped visual content.
- a user controlling the processor via a user interface may manually select a horizon line for the visual content.
- this selection may override a horizon line potentially coded to the cropped visual content as determined by at least one motion sensor in electronic communication with the processor.
- the circular coded visual content may then be presented on a viewer display 350 .
- the viewer display is in electronic communication with a processor of a viewership device.
- the display is incorporated into a viewership device separate from the device performing the formatting method 600 .
- the horizon line coded to the circular coded visual content indicates to the viewership device at least the orientation the visual content is to be displayed. Generally, and perhaps preferably, the horizon line is to be parallel with a bottom edge of the display although other orientations are not precluded.
- the circular coded visual content is synced to the gyroscope of the viewership device.
- a rectangular portion of the circular visual content is selected to fit the size of the viewer display based on a corresponding detected orientation of the viewership device.
- a motion sensor such as a gyroscope
- senses rotations of the viewership device the selected rectangular portion of the circular content changes based upon the detected rotation. For example, in the reference frame of the visual content, when the motion sensor of the viewership device detects a viewership device portrait orientation as illustrated in FIG. 3 , a matching portrait selection of the circular visual content 358 is displayed.
- the current devices and methods may utilize pre-existing common hardware such as cameras integrated into smart devices and the like.
- the devices and methods operate without specialized equipment.
- video content may be captured and acquired, that is, already in a circular format.
- the device(s) and methods may either (i) further crop the circular format and code the visual content accordingly or (ii) code the captured circular visual content without first cropping the visual content.
- an imaging device includes an electronic processor and at least one image sensor and at least one motion sensor.
- the at least one image sensor is a camera.
- the at least one motion sensor is a gyroscope.
- the image sensor and gyroscope are linked such that when capturing visual content, the image is stabilized.
- an imaging device may capture visual content such as an image of a person standing.
- a person may appear as being parallel to the long axis (height) and perpendicular to a short axis (width).
- the imaging device is rotated during capture, the person is captured at an angle with respect to the axes of the orientation format.
- the rotated, captured content is subsequently displayed, the person standing is presented on a viewer display at an angle mirroring the rotation of the imaging device.
- the gyroscope When the gyroscope is linked to the imaging sensor, the gyroscope may detect rotations during capture of the visual content. In this way, when the video content is later viewed/replayed the video content is displayed such that the rotation during capture does not affect the orientation of the content displayed. Said another way and using the example of capturing an image of a person standing, when viewing the visual content of a person standing during rotation, the person standing will appear to be perpendicular to the short axis and parallel to the long axis. Thus, the captured visual content is rotated in response to a rotation of the imaging device.
- the imaging device processor is further configured to capture the visual content as a circle from the shot. In other embodiments, the imaging device processor is further configured to crop the captured visual content into a circle.
- FIGS. 7 A- 7 B illustrate various lens 700 and sensor 702 arrangements. If the circular lens 700 is too small for the corresponding rectangular image sensor 702 as shown in FIG. 7 A , then the final image may appear to have dark corners as the corners 704 of the image sensor 702 will have minimal exposure to incoming light. On the other hand, if the circular lens 700 is bigger than the image sensor 702 as shown in FIG. 7 B , then the information captured by the lens 700 outside the area of the image sensor 702 never makes it to the image sensor for capturing and is, therefore, lost. Said another way, when the lens format is larger than the image sensor, the field of view is reduced.
- Another potential issue is when the software described herein circularizes, via cropping, the captured visual content associated with rectangular imaging sensors. That is, some of the visual information 707 captured by the image sensor 702 is lost. This means that, in some cases, resolution of the final visual content is reduced. In other words, a cropped circle 706 mapped to the rectangular shape loses the visual content information captured by the ends of the imaging sensor 702 , for example, 4 k resolution image may be reduced to about 2.5 k resolution (these values are representative only, and should not be deemed to be limiting). Under some rare circumstances the visual content may appear to be pixelated.
- One advantage to the cropping is that the process provides a data storage savings of up to 50% as the information that is lost during the cropping step or process is not saved with the corresponding visual content file.
- FIG. 7 C The advantage of using a circular image sensor is apparent in FIG. 7 C . While the lens 700 , circular image sensor 703 , and cropped circle 706 are shown as decreasing in size respectively, it is noted that this reduction is merely for illustrative purposes and that the sizes of each or some of the elements are substantially the same and/or may slightly overlap.
- the circular lens 700 and circular image sensor 703 are configured such that the most visual information captured by the lens is directed to the circular sensor 703 . This effectively minimizes any losses that are present when a circular lens is paired with a rectangular image sensor such as those illustrated in FIG. 7 A and FIG. 7 B . In other words, the maximum amount of captured information is able to be presented on display 708 .
- the circular image sensor 703 generates a circular image without further cropping needed.
- a circular image generated from the circular image sensor 703 may be cropped to a circle 706 .
- the further cropping may be advantageous as it may provide data storage savings for the final visual content file.
- an imaging device includes at least one circular lens 700 in optical communication with a circular image sensor 703 .
- a motion sensor similar to that of sensor 110 illustrated in FIG. 1 , is configured to link a horizon, similar to horizon line 369 of FIG. 3 , to the visual content.
- fisheye type lens may produce a circular image.
- fisheye lenses also create a distortion known generally as the fisheye effect which is not always desirable.
- the fisheye lens is an ultra-wide angle lens that produces strong visual distorting intended to create a hemispherical image.
- a fisheye lens uses a special mapping which gives the characteristic convex non-rectilinear appearance.
- the circular imaging sensor is able to maximize the visual content captured by the lens with little to no distortion. That is, visual content produced by the circular image sensor has straight lines of perspective.
- the visual content and lenses used to create the captured image in accordance with the present invention is advantageously achieved without a fisheye type lens.
- FIGS. 8 A and 8 B illustrate an advantage of associating circular visual content to a motion sensor of a viewership device.
- a viewership device 800 has a top 802 and a bottom 804 .
- a user 820 holds the viewership device 800 in a portrait orientation as illustrated in FIG. 8 A . That is, the bottom 804 of the viewership device 800 is seated about the user's palm and the viewership device 800 is held upright.
- the visual content 852 is accompanied with a preferred orientation with respect to gravity. That is, the visual content 852 may be captured via an image sensor associated with a motion sensor as described above.
- the visual content may be tagged with a preferred direction via manually selecting a preferred direction of gravity 850 or via coding (either via the capturing device and/or via an intermediate device).
- the viewership device 800 includes at least one integrated motion sensor, analogous to motion sensor 110 , that may detect the viewership device's movements.
- the at least one integrated motion sensor detect information corresponding to the viewership device's 800 orientation with respect to the direction of gravity 850 .
- FIG. 8 A the viewership device 800 is held upright with respect to the direction of gravity 850 and in FIG. 8 B the viewership device 800 is held upside-down with respect to the direction of gravity 850 .
- the visual content 852 is orientated in the same way with respect to the direction of gravity 850 .
- the viewership device 800 when a user is viewing visual content on a viewership device 800 , the viewership device 800 is held upright as in FIG. 8 A .
- the user 820 would need to rotate their hand in an azimuth fashion such that the orientation of the viewership device 800 with respect to the direction of gravity 850 is constant throughout the rotation.
- the display of the viewership device 800 which was previously facing the user in an upright orientation, is rotated to face another person across from the user also in an upright orientation. Maintaining this hand position while showing the visual content 852 on the viewership display to another is uncomfortable for the user.
- the user 820 simply flips their wrist such that the viewership device 800 is orientated upside-down with respect to the direction of gravity 850 as illustrated in FIG. 8 B .
- this flipping hand motion would result in showing visual content 852 that is also upside down.
- the visual content 852 “weighted”, the visual content 852 is rotated by the processor of the viewership device 800 such that the preferred direction of the visual content is in alignment with the detected direction of gravity 850 by the integrated motion sensors.
- the user 820 may operate the application or operating system on the viewership device 800 easily in any direction.
- the orientation of a GUI operating system such as ANDROID® or iOS®, is fixed with respect to the top 802 and bottom 804 of a viewership device 800 .
- ANDROID® or iOS® is fixed with respect to the top 802 and bottom 804 of a viewership device 800 .
- the user 820 When a user 820 picks up the device in a random orientation, the user 820 must rotate the viewership device 800 in their hands until the bottom 802 of the device 800 is seated in the user's palm (if operating in a portrait mode). This rotation action takes times, risks potential clumsiness of the user 820 dropping the viewership device 800 , and is unnecessary.
- a user 820 may pick up the viewership device 800 from a table the ground, etc. and, no matter the orientation of the device (upright or upside down), the GUI is displayed upright with respect to gravity 850 . This allows a user 820 to start experiencing and using the GUI immediately upon picking up the viewership device 800 .
- motion sensors of a content creation, formatting and/or viewership device may be coupled to either circular visual content or rectangular visual content.
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Human Computer Interaction (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- User Interface Of Digital Computer (AREA)
Abstract
Devices and methods for formatting visual content to produce a visual content format that may adapt to and fill the dimensions of a viewer display screen are provided. The visual content is circularized and linked to a horizon line defined by a motion sensor of an associated device.
Description
- This application is a continuation of U.S. patent application Ser. No. 17/050,544, filed Oct. 26, 2020, which is a 371 of International Application Serial No. PCT/US2019/029304, filed Apr. 26, 2019, which claims the benefit of U.S. Provisional Patent Application Ser. No. 62/663,118, filed Apr. 26, 2018, and U.S. Provisional Patent Application Ser. No. 62/795,383, filed Jan. 22, 2019, which applications are hereby incorporated by reference.
- The present exemplary embodiment relates to methods and devices for creating and displaying visual content. It finds particular application in conjunction with providing a video and image content format that (i) fills the entire display of a media device regardless of the screen size and orientation, and (ii) seamlessly transitions the displayed visual content as the viewership device is moved and will be described with particular reference thereto. However, it is to be appreciated that the present exemplary embodiment is also amenable to other like applications.
- Currently, the majority of the displays present on media devices (smartphones, tablets, tablet PC, televisions, watches etc.) have a rectangular form factor. These devices are usually equipped with at least one imaging hardware component (e.g., an image sensor) that is also rectangular. Capturing an image or video (visual content) with current hardware results in visual content that is likewise, rectangular (in either a first/portrait/vertical orientation or a second/landscape/horizontal orientation). The software of a media device processes the captured visual content data and the visual content may be displayed on an associated rectangular display or on another viewership device in either a fixed orientation, for devices that have a generally fixed position such as a television or in one of two discrete orientations relative to a rotational position of the media device (i.e., a portrait or landscape orientation). Depending on the position of the device on which the visual content is captured and/or the orientation of the media device on which the visual content is viewed, the displayed visual content either (i) fills the entire screen or (ii) is accompanied by margin effects (“margins”), generally embodied as a pair of top and bottom and/or right and left artifacts, that appear with the visual content. In most applications, the visual content is centered within the margin effects, commonly known as letterboxing and pillarboxing.
- Generally, visual content is formatted based on a desired end use and therefore a multitude of media formats exist. Each type of media format, cinema, television, internet, smart device, etc., is configured with an aspect ratio. The multitude of aspect ratios is inconvenient for both visual content creators and consumers of that visual content as the selected viewership device by a consumer may not match the format of the created visual content. Thus, it is common for visual content to be presented with modifications. Modifications, including margin effects, are particularly prevalent when visual content is created with one capture device and the visual content is shared to other devices with a display having an aspect ratio that is different from the aspect ratio of the captured visual content. Due to the multitude of different form factors for media devices and displays, it is difficult to create an equal media experience for every user accessing the visual content. That is, those users with displays that match the aspect ratio of the visual content, view the visual content as filling the entire display. Those users with a display that does not match the aspect ratio view the visual content with undesirable margin effects.
- The equal media experience is even more difficult to achieve with portable media devices subject to user rotation. Generally, a user viewing a received visual content on a portable media device may rotate the media device from a portrait orientation to a landscape orientation (or vice versa) if the margins are present and the user would like to view the visual content while minimizing the margin effect, i.e., have the image fill as much of the screen as possible. However, it is also recognized that many users of portable media devices do not bother rotating the media device to view the visual content in a manner that avoids the margin effect—for whatever reason (e.g., takes too long, don't want to be bothered, etc.). Thus, a need exists for creating and/or displaying visual content in a manner that automatically utilizes the entire area of a display regardless of the media device used or viewing orientation of such device. That is, the viewership of the visual content is at least free of margin effects.
- It is desirable to meet the needs of the prior art using existing hardware that display captured visual content on a device without margin effects and where the displayed visual content seamlessly maintains and fills the entire display screen regardless of the orientation of the display device including during the transition from a first orientation to a second orientation, i.e., the displayed visual content does not “jump” between “portrait” and “landscape” modes where at least one of those display modes includes visually perceived margins.
- Furthermore, as will become apparent below, it is desirable to provide a visual content consumer with the opportunity to become an active participant in the media experience. Briefly, a media consumer is able to rotate a viewership device to selectively view portions of captured visual content.
- The present disclosure relates to devices and methods for formatting and viewing visual content to produce a format that may adapt and fill the entire, available dimensions of a display screen. The devices and methods find particular application in conjunction with images (photographs) and videos captured by smart devices incorporating a camera and will be described with particular thereto. However, it is to be appreciated that the present disclosure is also amenable to other like applications.
- In accordance with some aspects of the present disclosure, a formatting device for visual content is provided. The formatting device includes a motion sensor, an image sensor configured to capture visual content data, a storage medium that at least temporarily stores visual content data therein; and a processor configured to link visual content data to a horizon line defined by the motion sensor. The visual content data includes at least one image.
- In accordance with other aspects of the present disclosure, methods for formatting visual content are provided. Some methods include providing an electronic device including at least one motion sensor, an image sensor, a storage medium/memory configured to at least temporarily store visual content data captured by the image sensor and a processor configured to execute a set of instructions. The instructions include generating circular visual content (e.g. cropping rectangular content to a circle format where the cropped visual content fills an entirety of the area of the cropped circle) and linking the circular visual content to a motion sensor of the device.
- In accordance with some aspects of the present disclosure, methods for converting visual content from a rectangular display configuration to a universal mode that displays the visual content in at least both portrait and landscape modes without margins are provided. One preferred method includes the steps of (i) generating circularly formatted visual content, wherein the circularly formatted visual content is located within the confines of a circle having a circle edge, (ii) linking the circularly formatted visual content to one of a stable horizon line and a preferred direction with respect to gravity; and (iii) mapping an orientation dependent selection of the circularly formatted visual content data to a rectangular display of a display device, wherein at least one corner of the rectangular display abuts the circle edge of the circularly formatted visual content so that the orientation dependent selection of the circularly formatted visual content fills the entirety of the rectangular display.
- In accordance with other aspects of the present disclosure, imaging devices including a circular image sensor are provided. The imaging device includes an electronic processor, a circular imaging sensor; and at least one circularly shaped optical lens. The circularly shaped optical lens is in optical communication with the circular imaging sensor such that the majority if not all information captured by the circular lens is communicated and captured by the circular image sensor. The electronic processor of the imaging device is configured to link a stable horizon line to captured visual content via a motion sensor of the imaging device, a user command, or a default orientation setting.
- These and other non-limiting characteristics of the disclosure are more particularly disclosed below.
- The following is a brief description of the drawings, which are presented for the purposes of illustrating the exemplary embodiments disclosed herein and not for the purposes of limiting the same.
-
FIG. 1 is a block diagram illustrating an example of an electronic device in accordance with the present disclosure; -
FIG. 2 is an example of a user interface of an electronic device in accordance with the present disclosure; -
FIG. 3 is a graphical representation of a viewer device presenting visual content in accordance with the present disclosure; -
FIG. 4 is a graphical representation of a viewer device presenting visual content in accordance with the present disclosure; -
FIG. 5 is a graphical representation of a viewer device presenting visual content in accordance with the present disclosure; and -
FIG. 6 is a diagram of a method for formatting visual content in accordance with the present disclosure. -
FIG. 7A-C is a diagram of prior art rectangular image sensors compared to circular image sensors in accordance with the present disclosure. -
FIGS. 8A and 8B illustrate a device and visual content in accordance with the present disclosure. - A more complete understanding of the components, processes and apparatuses disclosed herein can be obtained by reference to the accompanying drawings. These figures are merely schematic representations based on convenience and the ease of demonstrating the present disclosure and are, therefore, not intended to indicate relative size and dimensions of the devices or components thereof and/or to define or limit the scope of the exemplary embodiments.
- Although specific terms are used in the following description for the sake of clarity, these terms are intended to refer only to the particular structure of the embodiments selected for illustration in the drawings and are not intended to define or limit the scope of the disclosure. In the drawings and the following description below, it is to be understood that like numeric designations refer to components of like function.
- The singular forms “a,” “an,” and “the” include plural referents unless the context clearly dictates otherwise.
- As used in the specification and in the claims, the term “comprising” may include the embodiments “consisting of” and “consisting essentially of.” The terms “comprise(s),” “include(s),” “having,” “has,” “can,” “contain(s),” and variants thereof, as used herein, are intended to be open-ended transitional phrases, terms, or words that require the presence of the named ingredients/components/steps and permit the presence of other ingredients/components/steps. However, such description should be construed as also describing compositions, articles, or processes as “consisting of” and “consisting essentially of” the enumerated ingredients/components/steps, which allows the presence of only the named ingredients/components/steps, along with any impurities that might result therefrom, and excludes other ingredients/components/steps.
- Numerical values in the specification and claims of this application should be understood to include numerical values which are the same when reduced to the same number of significant figures and numerical values which differ from the stated value by less than the experimental error of conventional measurement technique of the type described in the present application to determine the value.
- Original aspect ratio (OAR) is the aspect ratio in which visual content is originally produced. Oftentimes a user operating an image capture device, such as a camera, may select the OAR of the visual content to be captured. The user may select a specific aspect ratio based on the desired end use display for the visual content. A display device as described herein, is a device having a function of controlling visual content displayed on an associated display, such as a display panel. The desired end use may be a wide format television, cinema screen, personal computer display, tablet, smartphone display, smartwatch, etc. When the OAR of the visual content matches the aspect ratio of the display, the visual content fills the entire field of view of the display.
- When the OAR of the visual content does not match an aspect ratio of a display device the OAR may be modified. For example, viewing a cinema format (a substantially rectangular visual content format) on a standard television (a substantially square display) may require the OAR be converted to a modified aspect ratio (MAR) by enlarging the original image to fill the display area, while cutting off the excess area. In other modifications, the visual content is stretched to fill in the area according to the new ratio. Modified aspect ratio (MAR) is the aspect ratio assigned in order to fit a type of screen, and different from the dimensions of the device in which the visual content is captured.
- Sometimes it is desirable to maintain the OAR when viewing the visual content on a display device. In some modifications wherein the OAR is maintained, the visual content is letterboxed. In other modifications, the visual content is pillarboxed. In yet other modifications, the visual content is both letterboxed and pillarboxed. Letterboxing is a modification to the presentation of visual content captured in a widescreen format and transferred to a display device having a display of smaller width that preserves the visual content's OAR. In letterboxing, the display device presents the visual content with bars, e.g., black bars positioned above and below the content. Pillarboxing is a modification of the presentation of visual content wherein the OAR width is smaller than the width of the display of the display device. In pillarboxing, black bars are placed on one or more sides of the visual content in order to preserve the OAR of the visual content.
- Maintaining the aspect ratio becomes more difficult when the display device viewing the visual content is capable of having multiple display orientations. For example, a display device may be a smart device, such as a smartphone or tablet, and can view visual content in a portrait mode and a landscape mode. That is, display devices are typically equipped with a sensor that detects the orientation of the display device and switches a display mode (portrait and landscape) based on the detected orientation of the display device. A portrait mode is perhaps best described as a thin, tall, rectangular-like shape. In other words, the height of the display is greater than the width, yielding a vertical aspect ratio, for example and without limitation 9:16 and 4:5 aspect ratios. A landscape mode is perhaps best described as a wide fat rectangular shape. In other words, the width of the display is greater than the height creating a horizontal aspect ratio, for example and without limitation 1.91:1. When horizontal visual content, having a horizontal aspect ratio and intended for horizontal displays, is viewed in a landscape mode, the visual content is capable of filling the majority of the display. The viewer, rotating the viewership device to initiate a portrait mode, views the horizontal visual content with modifications, often including margin effect combined with a zoom effect that makes the image appear smaller or further away.
- The portrait and landscape modes for capturing and viewing visual content are discrete. That is, the display presents the visual content in either a portrait mode or landscape mode and abruptly switches between the two modes, usually upon reaching a threshold rotation of the viewership device. In this manner, the visual content is viewed as “jumping” from one mode to another without intermediate states as the device rotates. This jumping effect is further emphasized to a user when the visual content is modified such that different margin effects are present in one or both orientations.
- In accordance with the present disclosure, an
electronic device 100 is configured to generate and modify visual content such that the aspect ratio is preserved across display devices having different aspect ratios. With reference toFIG. 1 , a block diagram illustrates an example of anelectronic device 100. Theelectronic device 100 includes at least onemotion sensor 110 configured to detect an orientation of theelectronic device 100, aprocessor 120, and a storage medium/memory 130. These components are configured together in a manner well known in the art. - In some embodiments, the
electronic device 100 includes animage sensor 140, such as a camera, configured to capture visual content. In some embodiments, theelectronic device 100 also includes adisplay 150, configured to display visual content to a viewer. - In some embodiments, the
electronic device 100 is equipped with acommunication device 160 configured to send visual content to another device or storage (storage including a hard drive, memory, cloud storage, media platforms). Thecommunication device 160 may include wired communication components, wireless communication components, cellular communication components, near field communication (NFC) components, Bluetooth® components, Wi-Fi® components, and other communication components to provide communication via other modalities. This list of exemplary communications devices is intended to be exemplary and not preclude the use of one or more of these components alternately or in combination or preclude the use of still other communication components that perform substantially the same function in substantially the same way. - The
electronic device 100 may also include a user interface 170 configured to receive commands from a user of thedevice 100. The user interface 170 may include without limitation, a touchscreen device, a keyboard, a mouse, motion sensors, buttons, knobs, voice actuation, and the like. - The
motion sensor 110 may be a gravity sensor, an accelerometer, a gyroscope, and/or any sensor(s) configured to detect one or more motions and/or an orientation associated with theelectronic device 100. A motion capable of detection by themotion sensor 110 may be a tilt, wherein the device is moved to a sloping position, i.e., at an angle with respect to a horizontal plane. Another motion capable of detection by themotion sensor 110 may be a rotation, wherein a rotation is moving theelectronic device 100 within a single plane, e.g. orientating theelectronic device 100 from a portrait orientation to a landscape orientation, vice versa, or to intermediate orientations between a portrait and landscape orientation. A motion may also include an acceleration, wherein themotion sensor 110 may detect a change in velocity of theelectronic device 100 in any direction. A motion may also be a combination of any or all movements of theelectronic device 100. It is to be appreciated that theelectronic device 100 may include multiple motion sensors, for example and without limitation, theelectronic device 100 may include a gravity sensor, an accelerometer, and a gyroscope. While themotion sensors 110 may be located anywhere on theelectronic device 100, typically themotion sensors 110 are internal components of theelectronic device 100 and in electronic communication with theprocessor 120. - A gravity sensor is a
motion sensor 110 that is configured to measure an orientation with respect to the direction of gravity. The gravity sensor enables the electronic device 100 (via the processor 120) to recognize the direction of gravity relative to the device based, for example, on calculated three-dimensional vectors. The gravity sensor may indicate an orientation, such as a degree of rotation with respect to the direction of gravity, of theelectronic device 100. The detected orientation may be combined with the acquisition of visual content by theelectronic device 100. That is, the visual content may be tied to the direction of gravity such that the visual content is weighted. Here, “weighted” visual content is visual content that is aligned with the direction of gravity. For example, when capturing visual content via anelectronic device 100, (e.g., a tablet or a smartphone) orientated in a landscape mode, wherein the long axis of the electronic device is perpendicular to the direction of gravity, the visual content is assigned a gravity vector that is perpendicular to the long axis of theelectronic device 100. If theelectronic device 100 is rotated at an angle, for example 30 degrees with respect to gravity (detected by the motion sensor 110), the visual content will be tagged with a gravity vector that is also offset by 30 degrees. When the visual content is displayed on a viewership device, the tagged gravity vector will be aligned with a detected direction of gravity of a viewership device (described in greater detail below). The gravity sensor may also be used to determine a tilting direction of theelectronic device 100. - An accelerometer is a
motion sensor 110 configured to detect a change in velocity during a time period and senses an acceleration. A three-axis accelerometer may include multiple motion sensors positioned in the x, y, and z-axis directions. Theprocessor 120 of theelectronic device 100 receives from the accelerometer values measured in the multi-axis directions as vector values. Theprocessor 120 may then determine a direction in which theelectronic device 100 is rotated or tilted based on values obtained with respect to the three axes. - A gyroscope is a
motion sensor 110 configured to calculate an angle to which thedevice 100 rotates with respect to an axis. This may be represented as a numerical value. A three-axis gyroscope calculates the degree to which thedevice 100 rotates with respect to three axes. - In some embodiments, a
motion sensor 110 may be an external sensor in communication with theelectronic device 100. That is, themotion sensor 110 may be a plug-and-play device configured to connect with an input/output (I/O) interface of anelectronic device 100 and communicate with theprocessor 120. - The
processor 120 is configured to control the operations of theelectronic device 100 and may execute applications, apps, and instructions stored in thedevice memory 130 and/or accessible via thecommunication device 160. Generally, theprocessor 120 determines whether to rotate visual content displayed on thedisplay 150 based on data obtained from at least onemotion sensor 110. - In some embodiments, the
electronic device 100 is equipped with animage sensor 140, embodied as a camera or similar sensor for acquiring visual content. A user of theelectronic device 100 may take photographs or record video by manipulating a user interface 170 to employ theimage sensor 140. When theimage sensor 140 of theelectronic device 100 captures the visual content (photograph, image, video, etc.), amotion sensor 110 may apply a “weight” to the visual content. When visual content is shared or accessed, a viewership device displaying the visual content will orient the visual content such that the assigned weight or a preferred direction is in line with the true direction of gravity. More about the visual content and its association with a weight and/or a preferred direction is discussed in greater detail below. - The
electronic device 100 may store the visual content into thememory 130. In some embodiments, theelectronic device 100 may upload the visual content to an external storage device, such as an external hard drive, cloud, internet storage, etc. via thecommunication device 160. Thecommunication device 160 may support two-way communication with other devices, cloud storage, networks, and external storage devices by using a variety of technologies. Thecommunication device 160 may include a network interface component or other suitable device to interface with a network. In other examples, thecommunication device 160 may include wired communication components, wireless communication components, cellular communication components, near field communication (NFC) components, Bluetooth® components, Wi-Fi® components, and other communication components to provide communication via other modalities. This list of communication devices is intended to be exemplary and not preclude the use of one or more of these components alternately or in combination or preclude the use of still other communication components. - The communication via the
communication device 160 may be for the purposes of storing visual content, obtaining visual content, and sharing visual content with other devices including viewership devices. In some embodiments, thecommunication device 160 is configured to connect to a social media network wherein the visual content may be uploaded, stored, or otherwise accessible by one or more users of the social media network. -
FIG. 2 is an exemplary embodiment of a software application run on anelectronic device 200 controlled bydevice processor 220 configured to capture visual content in accordance with the present disclosure. Theelectronic device 200 includes atouch screen display 250 which simultaneously displaysvisual content 252 and provides a user interface for user operation of theelectronic device 200 and various features presented by the software program. Theelectronic device 200 may be operable in avideo mode 254 or aphotograph mode 256 by selecting the desired mode via the user interface. A video mode is an operational mode of an onboard image sensor configured to capture a series of images over time (frames). A photograph mode is an operational mode of an onboard image sensor configured to acquire at least one or several images over a short period of time. An image sensor 240 in electronic communication with theprocessor 220, is configured to capturevisual content 252 which is displayed on thedisplay 250. That is, theelectronic device 200 may be a visual content capture and formatting device as well as a visual content viewership device. - Generally, when an
electronic device 200, for example and without limitation, a smartphone or tablet, is used to capture visual content, an image representing all the visual information captured by the image sensor of the camera is presented on thedisplay 250. Thus, generally and in some embodiments disclosed herein, visual content would fill both thecircle portion 258 and blacked out portion (outside the perimeter of image 258) shown ondisplay 250 to a user while being captured. Here,FIG. 2 shows the result of cropping all visual information captured by the image sensor 240. - In some embodiments, the software of the
electronic device 200 projects a sweet spot target 266 (indicated by the dotted circle) on the visual information as it is captured. As will be discussed in more detail below, thesweet spot target 266 contains the visual content that will always be displayed on a viewership device regardless of orientation and aspect ratio for reasons which will become more apparent below. In some embodiments, thesweet spot target 266 is generally the geometric centered portion of the visual information displayed. This may be the default position of thesweet spot target 266. In other embodiments, a user may select the geometric position of thesweet spot target 266 within the bounds of the visual information. - The
processor 220 executes instructions stored in either thememory 230 or instructions accessible viacommunication device 260 to perform a cropping method on visual information that is captured by the image sensor 240. The method includes (i) cropping the visual information captured by the image sensor 240 to acircle format 258 visual content and (ii) coding/syncing the cropped circular visual content. Preferably, thecircle format 258 does not exceed the narrower dimension of the visual information displayed on the electronic device in either portrait or landscape, or at an angle therebetween. - Currently the majority of visual content is captured rectangular in form, either formatted to a portrait orientation or more commonly, a landscape orientation. The rectangular visual content may be the visual information acquired from the image sensor 240 of the
electronic device 200, or the rectangular visual content may be visual content captured by another imaging device (such a digital point and shoot camera or wide format cinema camera) and imported to theelectronic device 200 for formatting purposes. The first part of the cropping method removes portions of the visual content such that the remaining visual content is circular in format. For example and with reference toFIG. 2 , the visual information may fill theentire display 250, including the parts that are represented as blacked out. Cropping the visual information removes portions of the visual information leaving the circularvisual content 258. When parts of the visual content are removed, the resolution of the visual content (the detail held by the visual content) stays the same. However, it is to be appreciated that the storage size of the visual content file itself may be reduced, providing in some embodiments, a memory saving function. - The
processor 220 is also configured to sync the cropped visual content to themotion sensors 110 of theelectronic device 200 that captures the visual content. In some embodiments, and as illustrated inFIG. 2 , thecircle format 258 of the visual content is coded to themotion sensors 110 in order to link the visual content to a stable reference or “horizon line” 269. The stable horizon line is a reference line that is configured to be perpendicular to the direction of gravity. In some embodiments, the visual content is synced with a gyroscope sensor of theelectronic device 200. While this may be the most efficient way to do this, it is contemplated that the software of theelectronic device 200 may obtain an image and determine a most likely preferred orientation of the visual content and assign or link the visual content to a reference line (e.g., horizon, gravity). - The addition of a reference system to the circular format and/or gyroscope syncing allows for the visual content to have a preferred orientation. In some embodiments, the orientation of the coded visual content is modified with respect to a detected direction of gravity by a motion sensor on a viewership device. This provides a reference point that allows the coded visual content to rotate on the display of a viewership device counter to an opposite rotation of the viewership device such that the visual content appears to the user as being stationary. That is a
determined bottom 264 of the visual content is always associated with the direction of gravity. The effect of a stable viewership is best illustrated inFIGS. 3-5 and is described in greater detail below. - In some embodiments, the
motion sensors 110 of theelectronic device 200 define the orientation of the visual content, such as defining the top 262 andbottom 264 of thevisual content 252. In some embodiments, thevisual content 252 is weighted such that thebottom 264 of the visual content is the closest to the bottom-most portion of thedisplay 250. - In some embodiments, after the visual content is captured/recorded the visual content may be further edited by a user. Editing includes any enhancements or changes to the visual content appearance, including, but not limited to, changing the sweet spot, resolution, and zoom, as well as changing the saturation, brightness, color, graphics and the like to the visual content. In some embodiments, the visual content is shared to any type of platform, including, but not limited to, social media, such as Facebook®, Twitter®, iMessage®, etc. It is to be appreciated that editing may be performed on the
electronic device 200 or may be distributed to another device configured to receive the visual content for editing. In some embodiments, the editing of the visual content is performed in the cloud. - The circular
visual content 258 linked to the motion sensors 110 (circular coded visual content) during creation has many advantages. Once the visual content is finalized, meaning all editing done by the program and user are complete, the visual content may be opened by any viewership device with access to the visual content. A viewership device may be variously embodied, for example theimage capture device 200 itself may store an access the visual content as a viewership device. The circular coded visual content may be shared to other viewership devices via communication components that distribute the circularvisual content 258 to a viewership device and/or networks, direct linking, etc. It is to be appreciated that a viewership device may be an electronic device equipped similar toelectronic device 200. A viewership device may also be a television display, computer monitor, cinema screen, and the like. - With reference to
FIG. 3 , the circular codedvisual content 358 is presented on a viewership device having aviewer display 350. In some embodiments, the circular codedvisual content 358 is automatically configured to fit the size of theviewer display 350 of the viewership device. This configuration enables theviewer display 350, having four corners 351 a-d to touch thecircle format 358 in four respective places. While arectangular viewer display 350 is shown, it is contemplated that theviewership display 350 may be embodied as other shapes. That is, a triangular display may have its three corners each touch the edge of the circle format in three respective places. As an additional example, the viewership display may also be circular in shape wherein the circumference of the display touches the circumference of thevisual content 358. Thehorizon line 369 coded to thevisual content 358 defines the orientation on which the visual content is presented on thedisplay 350. - For fixed viewership devices, i.e. those not intended to rotate, the
horizon line 369 is parallel to a definedbottom edge 371 of adisplay 350. In the exemplary example of a widescreen/rectangular computer monitor as a viewership device having aviewer display 350, a user may define thebottom edge 371 as either the short edge, similar to the orientation of the display shown inFIG. 3 , or the long edge, similar to the orientation of the display inFIG. 5 . This is usually done by changing the display settings of the computer and associated monitor. A user may wish to have a monitor oriented in a traditional landscape orientation, or a user may wish to have the monitor oriented in a portrait orientation. With the orientation defined by a user in the display settings, the horizon line may be linked to the visual content accordingly. - In some embodiments, the
viewer display 350 is incorporated into a viewership device similar in nature to theelectronic device FIGS. 1 and 2 respectively. That is, the viewership device includes at least aviewer display 350, a processor, and at least one motion sensor. The viewership device may be a portable device that is able to be manipulated into a plurality of orientations for viewing the visual content. In these embodiments and as represented inFIG. 3 , the circular codedvisual content 358 has a first capturedimage diameter 359. The circular codedvisual content 358 is synced to a motion sensor, analogous tomotion sensor 110, in communication with a processor, analogous toprocessor 120 of the viewership device. In some embodiments, the motion sensor is a gyroscope. After the circular codedvisual content 358 is synced with the viewership device, a portion of the circular codedvisual content 358 is mapped to fit theviewer display 350 of the viewership device. That is, acircumference 359 is correlated with the four corners 351 a-d of theviewer display 350 so that each corner abuts thecircumference 359.Sweet spot target 266 is a smaller diameter portion of the circular codedvisual content 358 and specifically is dimensioned so that a diameter of thesweet spot 266 preferably creates a tangent at twolocations rectangular viewer display 350. The mapping enables theviewer display 350 having four corners 351 a-d to preferably touch thecircle format 358 along thecircumference 359 in four respective places. - When the viewership device is equipped with a motion sensor, such as
motion sensor 110, the circularvisual content 358 is rotated in a direction counter of the rotation of theviewer display 350 with respect to thestable horizon line 369. That is, the motion sensor of the viewership device links or syncs to the circular codedvisual content 358 and calculates an angle, tilt, and/or rotation of thedisplay 350 with respect to thehorizon line 369. - The viewership device, and along with it, the
viewer display 350, may be rotated about a plane defined by thecircle format 358. In this way, a non-circular display, such as therectangular display 350 is a viewfinder of the entire visual circularvisual content 358. Theviewer display 350, may only “view” rectangular portion of the circular codedvisual content 358 at a time. The viewed area is represented by the darker image confined within theviewer display 350. It is noted and illustrated inFIGS. 3-5 that the selected rectangular portion viewed on therectangular display 350 includes the entiresweet spot target 266 no matter what the orientation of theviewer display 350. Select annular portions (i.e. between thesweet sport 266 andcircumference diameter 359 will be shown on therectangular display 350. As will be appreciated, the angular segment of the annular portion that is seen on therectangular display 350 will vary depending on the physical orientation of the viewer device but in each instance thesweet spot 266 is displayed. Stated another way, the remaining portions of the circular coded visual content, illustrated as the lighter regions of thevisual content 352 outside the confines of thedisplay 350, may be viewed and explored on theviewer display 350 by rotating the viewer display to a desired orientation. - It is noted that the visual content captured in sweet spot target 266 (or a portion thereof) is preferably always present no matter the orientation of the
viewer display 350. The user of the viewership device is provided with a unique choice in selecting the viewership of thevisual content 358 allowing the viewer to play an active role in viewingvisual content 358. - The circular coded
visual content 358 is continuously rotated as theviewer display 350 is rotated. In this way, there is no abrupt jump in the orientation of the visual content while transitioning theviewer display 350 from a portrait orientation (shown inFIG. 3 ) to a landscape orientation (shown inFIG. 5 ). That is, theviewer display 350 presents full screen intermediate orientations during rotation of theviewer display 350 which appear as a seamless continuous transition (i.e. without jumping) at a constant image resolution. - In accordance with another aspect of the present disclosure, a
preferred method 600 of circularizing and coding visual content is provided. Themethod 600 is illustrated inFIG. 6 and performed by a device (capturing device, intermediate device/cloud, and/or viewership device) including an electronic processor configured to execute instructions provided on a non-transitory storage medium. For example, theelectronic device 100 ofFIG. 1 may be employed to perform themethod 600. Briefly, themethod 600 includes afirst step 602 of acquiring visual content. After acquiring the visual content, the method provides cropping the visual content to a circle format, illustrated atblock 604. After the visual content is cropped, ahorizon line 369 is coded, atblock 606, to the cropped visual content creating circular coded visual content. The circular coded visual content may then be displayed, atblock 608, on a display device 350 (which could be the image capturing device or another display device) such that the circular coded visual content is mapped to a desired portion of the display. - Visual content may be acquired, at
block 602, in several ways. In some embodiments, visual content is created by an imaging sensor, e.g.,imaging sensor 140, in electronic communication with theprocessor 120 of thedevice 100. In other embodiments, visual content is acquired by importing/downloading visual content via acommunication device 160 in electronic communication with theprocessor 120 of thedevice 100. Thecommunication device 160 may connect to an internet network, a cellular network, or the like. Thecommunication device 160 may also acquire visual content via wired connections and/or wireless connections. - Once the visual content is acquired it may be stored on a
storage medium 150 of the device for further processing. The visual content is then cropped to a circular format as illustrated and described with respect toFIG. 3 . Typically, visual content is initially generated in a rectangular form factor by an image sensor that is also rectangular in shape. Theprocessor 120 then crops the rectangular visual content to create a circular format. In this way, existing imaging hardware (i.e., non-circular image sensors) may be employed to create universal visual content. - However, it is to be appreciated that visual content may be created/generated from cameras other than those in direct electronic communication with the processor. As a non-limiting illustrative example, a second device may capture visual content in rectangular form and upload the visual content to a storage device/network. The device processor equipped with the software described herein to perform the
method 600 may access the uploaded visual content and, in some cases, download the visual content to astorage medium 150 in electronic communication with theprocessor 120. Theprocessor 120 may then crop the visual content acquired from the external source. - After the visual content is cropped a horizon line is coded, at
block 606, to the circularvisual content 358. In some embodiments, a motion sensor in electronic communication with the processor of the device is used to define and code a horizon line to the circular visual content. The motion sensor may be any sensor known to detect an orientation of visual content or the device including a display configured to present visual content. In some embodiments, the motion sensor is a gyroscope. In other embodiments, the motion sensor is an accelerometer. In yet other embodiments, the motion sensor is a gravity sensor. In still yet other embodiments, the motion sensor is a plurality of sensors including any combination of two or more sensors described herein. - In some embodiments, a horizon line is a user-defined line coded to the cropped visual content. In this way, a user controlling the processor via a user interface may manually select a horizon line for the visual content. In some embodiments, this selection may override a horizon line potentially coded to the cropped visual content as determined by at least one motion sensor in electronic communication with the processor.
- The circular coded visual content may then be presented on a
viewer display 350. In some embodiments the viewer display is in electronic communication with a processor of a viewership device. In other embodiments, the display is incorporated into a viewership device separate from the device performing theformatting method 600. - The horizon line coded to the circular coded visual content indicates to the viewership device at least the orientation the visual content is to be displayed. Generally, and perhaps preferably, the horizon line is to be parallel with a bottom edge of the display although other orientations are not precluded.
- When the viewership device is equipped with a motion sensor, such as a gyroscope, the circular coded visual content is synced to the gyroscope of the viewership device. A rectangular portion of the circular visual content is selected to fit the size of the viewer display based on a corresponding detected orientation of the viewership device. As a motion sensor, such as a gyroscope, senses rotations of the viewership device, the selected rectangular portion of the circular content changes based upon the detected rotation. For example, in the reference frame of the visual content, when the motion sensor of the viewership device detects a viewership device portrait orientation as illustrated in
FIG. 3 , a matching portrait selection of the circularvisual content 358 is displayed. When the motion sensor of the viewership device detects a rotated orientation, as illustrated inFIG. 4 , a corresponding angled rectangular selection of the circularvisual content 358 is displayed on the viewer display. When the motion sensor of the viewership device detects a viewership device landscape orientation as illustrated inFIG. 5 , a matching landscape selection of the circularvisual content 358 is displayed. Explained in the reference frame of a user of the viewership device, when the viewership device is rotated about an angle with respect to either thehorizon line 369 or direction of gravity, the circular visual content is rotated at the same angle in the opposite direction, such that a different rectangular portion of the circular visual content is presented on the viewer display. This allows for all visual content to be viewed on any display without the need to pillarbox, letterbox, or otherwise modify the visual content. It also provides for the seamless, continuous transition of the image/visual content as the viewing device is moved from a first orientation to a different, second orientation. - It is to be appreciated that the current devices and methods may utilize pre-existing common hardware such as cameras integrated into smart devices and the like. Thus, the devices and methods operate without specialized equipment. It is contemplated in this disclosure that video content may be captured and acquired, that is, already in a circular format. In these cases, the device(s) and methods may either (i) further crop the circular format and code the visual content accordingly or (ii) code the captured circular visual content without first cropping the visual content.
- In accordance with another aspect of the present disclosure, an imaging device includes an electronic processor and at least one image sensor and at least one motion sensor. In some embodiments, the at least one image sensor is a camera. In some embodiments, the at least one motion sensor is a gyroscope. Here, the image sensor and gyroscope are linked such that when capturing visual content, the image is stabilized. For example, an imaging device may capture visual content such as an image of a person standing. Generally, when capturing visual content of a person in a portrait orientation, wherein the height of the visual content is greater than the width, a person may appear as being parallel to the long axis (height) and perpendicular to a short axis (width). When the imaging device is rotated during capture, the person is captured at an angle with respect to the axes of the orientation format. When the rotated, captured content is subsequently displayed, the person standing is presented on a viewer display at an angle mirroring the rotation of the imaging device.
- When the gyroscope is linked to the imaging sensor, the gyroscope may detect rotations during capture of the visual content. In this way, when the video content is later viewed/replayed the video content is displayed such that the rotation during capture does not affect the orientation of the content displayed. Said another way and using the example of capturing an image of a person standing, when viewing the visual content of a person standing during rotation, the person standing will appear to be perpendicular to the short axis and parallel to the long axis. Thus, the captured visual content is rotated in response to a rotation of the imaging device.
- In some embodiments, the imaging device processor is further configured to capture the visual content as a circle from the shot. In other embodiments, the imaging device processor is further configured to crop the captured visual content into a circle.
- In some embodiments, software executed by the imaging device processor creates a non-rectangular visual content and incorporates the captured non-rectangular visual content with a motion sensor, e.g., gyroscope, gravity sensor, accelerometer, etc., associated with the imaging device as well a motion sensor associated with a viewer device. In this manner, the processor configures the visual content viewed on the viewer device to fill the entire display of the device. That is, there are no visible margins (pillarboxing, letterboxing, and the like) and the viewer can view the visual content in any orientation—landscape, portrait, or any degree in between—with the image adjusting in real-time using the gyroscope.
- It is recognized that current imaging devices use rectangular solid-state imaging sensors mounted behind at least one circular lens. There are several issues with the current systems. First, there is a mismatch in area between the lens and the image sensor, which makes selecting the proper lens an important consideration when creating the desired visual content. This is illustrated in
FIGS. 7A-7B . -
FIGS. 7A-7B illustratevarious lens 700 andsensor 702 arrangements. If thecircular lens 700 is too small for the correspondingrectangular image sensor 702 as shown inFIG. 7A , then the final image may appear to have dark corners as thecorners 704 of theimage sensor 702 will have minimal exposure to incoming light. On the other hand, if thecircular lens 700 is bigger than theimage sensor 702 as shown inFIG. 7B , then the information captured by thelens 700 outside the area of theimage sensor 702 never makes it to the image sensor for capturing and is, therefore, lost. Said another way, when the lens format is larger than the image sensor, the field of view is reduced. - Another potential issue is when the software described herein circularizes, via cropping, the captured visual content associated with rectangular imaging sensors. That is, some of the
visual information 707 captured by theimage sensor 702 is lost. This means that, in some cases, resolution of the final visual content is reduced. In other words, a croppedcircle 706 mapped to the rectangular shape loses the visual content information captured by the ends of theimaging sensor 702, for example, 4 k resolution image may be reduced to about 2.5 k resolution (these values are representative only, and should not be deemed to be limiting). Under some rare circumstances the visual content may appear to be pixelated. One advantage to the cropping is that the process provides a data storage savings of up to 50% as the information that is lost during the cropping step or process is not saved with the corresponding visual content file. - The advantage of using a circular image sensor is apparent in
FIG. 7C . While thelens 700,circular image sensor 703, and croppedcircle 706 are shown as decreasing in size respectively, it is noted that this reduction is merely for illustrative purposes and that the sizes of each or some of the elements are substantially the same and/or may slightly overlap. Thecircular lens 700 andcircular image sensor 703 are configured such that the most visual information captured by the lens is directed to thecircular sensor 703. This effectively minimizes any losses that are present when a circular lens is paired with a rectangular image sensor such as those illustrated inFIG. 7A andFIG. 7B . In other words, the maximum amount of captured information is able to be presented ondisplay 708. - In some embodiments, the
circular image sensor 703 generates a circular image without further cropping needed. In other embodiments, a circular image generated from thecircular image sensor 703 may be cropped to acircle 706. The further cropping may be advantageous as it may provide data storage savings for the final visual content file. - In some embodiments, an imaging device includes at least one
circular lens 700 in optical communication with acircular image sensor 703. A motion sensor, similar to that ofsensor 110 illustrated inFIG. 1 , is configured to link a horizon, similar tohorizon line 369 ofFIG. 3 , to the visual content. - It is noted that fisheye type lens may produce a circular image. However, fisheye lenses also create a distortion known generally as the fisheye effect which is not always desirable. The fisheye lens is an ultra-wide angle lens that produces strong visual distorting intended to create a hemispherical image. Instead of producing images with straight lines of perspective, a fisheye lens uses a special mapping which gives the characteristic convex non-rectilinear appearance. The circular imaging sensor is able to maximize the visual content captured by the lens with little to no distortion. That is, visual content produced by the circular image sensor has straight lines of perspective. Thus, the visual content and lenses used to create the captured image in accordance with the present invention is advantageously achieved without a fisheye type lens.
-
FIGS. 8A and 8B illustrate an advantage of associating circular visual content to a motion sensor of a viewership device. In some embodiments, aviewership device 800 has a top 802 and a bottom 804. Generally, auser 820 holds theviewership device 800 in a portrait orientation as illustrated inFIG. 8A . That is, thebottom 804 of theviewership device 800 is seated about the user's palm and theviewership device 800 is held upright. Thevisual content 852 is accompanied with a preferred orientation with respect to gravity. That is, thevisual content 852 may be captured via an image sensor associated with a motion sensor as described above. Alternatively, the visual content may be tagged with a preferred direction via manually selecting a preferred direction ofgravity 850 or via coding (either via the capturing device and/or via an intermediate device). Theviewership device 800 includes at least one integrated motion sensor, analogous tomotion sensor 110, that may detect the viewership device's movements. Preferably, the at least one integrated motion sensor, detect information corresponding to the viewership device's 800 orientation with respect to the direction ofgravity 850. InFIG. 8A , theviewership device 800 is held upright with respect to the direction ofgravity 850 and inFIG. 8B theviewership device 800 is held upside-down with respect to the direction ofgravity 850. However, as illustrated in bothFIG. 8A andFIG. 8B thevisual content 852 is orientated in the same way with respect to the direction ofgravity 850. Thus, there appears to be a “weight” tied to thevisual content 852 such that the visual content is always displayed upright with respect to the direction ofgravity 850. - Generally, when a user is viewing visual content on a
viewership device 800, theviewership device 800 is held upright as inFIG. 8A . Currently, if auser 820 wanted to show thevisual content 852 on the display of theviewership device 800 with another person standing across from the user, theuser 820 would need to rotate their hand in an azimuth fashion such that the orientation of theviewership device 800 with respect to the direction ofgravity 850 is constant throughout the rotation. In other words, the display of theviewership device 800, which was previously facing the user in an upright orientation, is rotated to face another person across from the user also in an upright orientation. Maintaining this hand position while showing thevisual content 852 on the viewership display to another is uncomfortable for the user. However, associating the visual content with a motion sensor of theviewership device 800 theuser 820 simply flips their wrist such that theviewership device 800 is orientated upside-down with respect to the direction ofgravity 850 as illustrated inFIG. 8B . With the visual content unconnected to the motion sensors of the device, this flipping hand motion would result in showingvisual content 852 that is also upside down. With thevisual content 852 “weighted”, thevisual content 852 is rotated by the processor of theviewership device 800 such that the preferred direction of the visual content is in alignment with the detected direction ofgravity 850 by the integrated motion sensors. - In the cases wherein the
visual content 852 is a graphical application or a graphical user interface (GUI) such as an operating system, theuser 820 may operate the application or operating system on theviewership device 800 easily in any direction. Generally, the orientation of a GUI operating system, such as ANDROID® or iOS®, is fixed with respect to the top 802 andbottom 804 of aviewership device 800. When auser 820 picks up the device in a random orientation, theuser 820 must rotate theviewership device 800 in their hands until thebottom 802 of thedevice 800 is seated in the user's palm (if operating in a portrait mode). This rotation action takes times, risks potential clumsiness of theuser 820 dropping theviewership device 800, and is unnecessary. With the GUI associated with a preferred direction and tied to at least one motion sensor of aviewership device 800, auser 820, may pick up theviewership device 800 from a table the ground, etc. and, no matter the orientation of the device (upright or upside down), the GUI is displayed upright with respect togravity 850. This allows auser 820 to start experiencing and using the GUI immediately upon picking up theviewership device 800. - It is to be appreciated, that the motion sensors of a content creation, formatting and/or viewership device may be coupled to either circular visual content or rectangular visual content.
- The exemplary embodiments have been described with reference to the preferred embodiments. Obviously, modifications and alterations will occur to others upon reading and understanding the preceding detailed description. It is intended that the exemplary embodiments be construed as including all such modifications and alterations insofar as they come within the scope of the appended claims or the equivalents thereof.
- Although specific advantages have been enumerated above, various embodiments may include some, none, or all of the enumerated advantages. Although exemplary embodiments are illustrated in the figures and description herein, the principles of the present disclosure may be implements using any number of techniques, whether currently known or not. Moreover, the operations of the system and apparatus disclosed herein may be performed by more, fewer, or other components and the methods described herein may include more, fewer or other steps. Additionally, steps may be performed in any suitable order.
- To aid the Patent Office and any readers of this application and any resulting patent in interpreting the claims appended hereto, applicants do not intend any of the appended claims or claim elements to invoke 35 U.S.C. 112(f) unless the words “means for” or “step for” are explicitly used in the particular claim.
Claims (1)
1. A formatting device for visual content comprising:
a motion sensor,
an image sensor, configured to capture visual content data;
a storage medium that at least temporarily stores visual content data therein; and
a processor configured to link visual content data to a horizon line defined by the motion sensor wherein the visual content data comprises at least one image.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US17/951,358 US20230014865A1 (en) | 2018-04-26 | 2022-09-23 | Method and apparatus for creating and displaying visual media on a device |
Applications Claiming Priority (5)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US201862663118P | 2018-04-26 | 2018-04-26 | |
US201962795383P | 2019-01-22 | 2019-01-22 | |
PCT/US2019/029304 WO2019210139A1 (en) | 2018-04-26 | 2019-04-26 | Method and apparatus for creating and displaying visual media on a device |
US202017050544A | 2020-10-26 | 2020-10-26 | |
US17/951,358 US20230014865A1 (en) | 2018-04-26 | 2022-09-23 | Method and apparatus for creating and displaying visual media on a device |
Related Parent Applications (2)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US17/050,544 Continuation US11490032B2 (en) | 2018-04-26 | 2019-04-26 | Method and apparatus for creating and displaying visual media on a device |
PCT/US2019/029304 Continuation WO2019210139A1 (en) | 2018-04-26 | 2019-04-26 | Method and apparatus for creating and displaying visual media on a device |
Publications (1)
Publication Number | Publication Date |
---|---|
US20230014865A1 true US20230014865A1 (en) | 2023-01-19 |
Family
ID=68294616
Family Applications (2)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US17/050,544 Active US11490032B2 (en) | 2018-04-26 | 2019-04-26 | Method and apparatus for creating and displaying visual media on a device |
US17/951,358 Abandoned US20230014865A1 (en) | 2018-04-26 | 2022-09-23 | Method and apparatus for creating and displaying visual media on a device |
Family Applications Before (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US17/050,544 Active US11490032B2 (en) | 2018-04-26 | 2019-04-26 | Method and apparatus for creating and displaying visual media on a device |
Country Status (2)
Country | Link |
---|---|
US (2) | US11490032B2 (en) |
WO (1) | WO2019210139A1 (en) |
Families Citing this family (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
KR20200127599A (en) * | 2019-05-03 | 2020-11-11 | 삼성전자주식회사 | Display apparatus, server, control method of display apparatus and control method of server |
DE102020126249A1 (en) * | 2020-10-07 | 2022-04-07 | Welbilt Deutschland GmbH | Cooking device, in particular commercial cooking device |
US11882368B1 (en) * | 2021-04-27 | 2024-01-23 | Apple Inc. | Circular image file |
US12039690B2 (en) * | 2021-06-29 | 2024-07-16 | Microsoft Technology Licensing, Llc | Image rotation for stream of input images |
Family Cites Families (15)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO1996027864A1 (en) | 1995-03-06 | 1996-09-12 | Bell, Addison | Computer based event capturing system and method having flexible storage |
WO2000049448A1 (en) | 1999-02-17 | 2000-08-24 | Cogent Light Technologies, Inc. | Imaging device having minimized packaging of remote sensors for video imaging and the like |
GB0208654D0 (en) | 2002-04-16 | 2002-05-29 | Koninkl Philips Electronics Nv | Image processing for video or photographic equipment |
US20060182437A1 (en) | 2005-02-11 | 2006-08-17 | Williams Karen E | Method and apparatus for previewing a panoramic image on a digital camera |
US20070081081A1 (en) | 2005-10-07 | 2007-04-12 | Cheng Brett A | Automated multi-frame image capture for panorama stitching using motion sensor |
US7593627B2 (en) * | 2006-08-18 | 2009-09-22 | Sony Ericsson Mobile Communications Ab | Angle correction for camera |
WO2014137368A1 (en) | 2013-03-08 | 2014-09-12 | Thomson Licensing | Method and system for stabilization and reframing |
US20140267806A1 (en) | 2013-03-12 | 2014-09-18 | Sony Corporation | Device and method for processing video content |
US9531970B2 (en) * | 2013-04-26 | 2016-12-27 | Omnivision Technologies, Inc. | Imaging systems and methods using square image sensor for flexible image orientation |
JP2014217032A (en) * | 2013-04-30 | 2014-11-17 | ソニー株式会社 | Image processing apparatus, image processing method and program |
US20150181123A1 (en) * | 2013-12-19 | 2015-06-25 | Lyve Minds, Inc. | Image orientation adjustment based on camera orientation |
US9723212B2 (en) | 2014-03-28 | 2017-08-01 | Intel Corporation | Image capture |
US20150341536A1 (en) | 2014-05-23 | 2015-11-26 | Mophie, Inc. | Systems and methods for orienting an image |
US10460765B2 (en) | 2015-08-26 | 2019-10-29 | JBF Interlude 2009 LTD | Systems and methods for adaptive and responsive video |
US20200296462A1 (en) | 2019-03-11 | 2020-09-17 | Wci One, Llc | Media content presentation |
-
2019
- 2019-04-26 US US17/050,544 patent/US11490032B2/en active Active
- 2019-04-26 WO PCT/US2019/029304 patent/WO2019210139A1/en active Application Filing
-
2022
- 2022-09-23 US US17/951,358 patent/US20230014865A1/en not_active Abandoned
Also Published As
Publication number | Publication date |
---|---|
WO2019210139A1 (en) | 2019-10-31 |
US11490032B2 (en) | 2022-11-01 |
US20210195118A1 (en) | 2021-06-24 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US20230014865A1 (en) | Method and apparatus for creating and displaying visual media on a device | |
US20220256081A1 (en) | Panoramic camera | |
US10674075B2 (en) | Image processing device, image processing method, and program | |
CN113038111B (en) | Method, system and medium for image capture and processing | |
JP5406813B2 (en) | Panorama image display device and panorama image display method | |
US8717412B2 (en) | Panoramic image production | |
EP2018049B1 (en) | Method of assembling a panoramic image and camera therefor | |
US11039109B2 (en) | System and method for adjusting an image for a vehicle mounted camera | |
US9986156B2 (en) | Image processing device, image processing method, and program | |
CN111145352A (en) | House live-action picture display method and device, terminal equipment and storage medium | |
US20030137506A1 (en) | Image-based rendering for 3D viewing | |
US20080253685A1 (en) | Image and video stitching and viewing method and system | |
US9462218B2 (en) | Video display device and method of displaying video | |
US20100039548A1 (en) | Image processing apparatus, image processing method, program and imaging apparatus | |
WO2017112800A1 (en) | Macro image stabilization method, system and devices | |
CN103795892A (en) | Method and apparatus for processing collected image data | |
CN110278368A (en) | Image processing apparatus, camera chain, image processing method | |
CN110392202A (en) | Image processing apparatus, camera chain, image processing method | |
CN110072059A (en) | Image shooting device and method and terminal | |
AU2019271924B2 (en) | System and method for adjusting an image for a vehicle mounted camera | |
KR101704362B1 (en) | System for real time making of panoramic video base on lookup table and Method for using the same | |
US20130044258A1 (en) | Method for presenting video content on a hand-held electronic device | |
JP2018033107A (en) | Video distribution device and distribution method | |
US20200351485A1 (en) | Systems and methods for stabilizing views of videos | |
CN115379195B (en) | Video generation method, device, electronic equipment and readable storage medium |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
STPP | Information on status: patent application and granting procedure in general |
Free format text: NON FINAL ACTION MAILED |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: RESPONSE TO NON-FINAL OFFICE ACTION ENTERED AND FORWARDED TO EXAMINER |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: FINAL REJECTION MAILED |
|
STCB | Information on status: application discontinuation |
Free format text: ABANDONED -- FAILURE TO RESPOND TO AN OFFICE ACTION |