US20220035450A1 - Surfacing related content based on user interaction with currently presented content - Google Patents
Surfacing related content based on user interaction with currently presented content Download PDFInfo
- Publication number
- US20220035450A1 US20220035450A1 US17/465,741 US202117465741A US2022035450A1 US 20220035450 A1 US20220035450 A1 US 20220035450A1 US 202117465741 A US202117465741 A US 202117465741A US 2022035450 A1 US2022035450 A1 US 2022035450A1
- Authority
- US
- United States
- Prior art keywords
- electronic device
- user
- media content
- content item
- selection
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Abandoned
Links
- 230000003993 interaction Effects 0.000 title description 20
- 230000009471 action Effects 0.000 claims abstract description 97
- 238000000034 method Methods 0.000 claims abstract description 63
- 230000015654 memory Effects 0.000 claims description 29
- 238000001514 detection method Methods 0.000 abstract description 13
- 230000004044 response Effects 0.000 abstract description 10
- 238000004458 analytical method Methods 0.000 description 25
- 238000010586 diagram Methods 0.000 description 25
- 230000033001 locomotion Effects 0.000 description 21
- 230000008569 process Effects 0.000 description 21
- 238000004891 communication Methods 0.000 description 7
- 230000008859 change Effects 0.000 description 6
- 230000006870 function Effects 0.000 description 5
- 238000007726 management method Methods 0.000 description 5
- 238000012545 processing Methods 0.000 description 5
- 241000208822 Lactuca Species 0.000 description 4
- 235000003228 Lactuca sativa Nutrition 0.000 description 4
- 230000004424 eye movement Effects 0.000 description 4
- 235000014101 wine Nutrition 0.000 description 4
- 241001147416 Ursus maritimus Species 0.000 description 3
- 230000001413 cellular effect Effects 0.000 description 3
- 230000000694 effects Effects 0.000 description 3
- 239000011521 glass Substances 0.000 description 3
- 230000007246 mechanism Effects 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 238000012546 transfer Methods 0.000 description 3
- 230000000007 visual effect Effects 0.000 description 3
- 241000283153 Cetacea Species 0.000 description 2
- 240000008415 Lactuca sativa Species 0.000 description 2
- 238000010411 cooking Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 230000007613 environmental effect Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000002265 prevention Effects 0.000 description 2
- 230000001737 promoting effect Effects 0.000 description 2
- RWSOTUBLDIXVET-UHFFFAOYSA-N Dihydrogen sulfide Chemical compound S RWSOTUBLDIXVET-UHFFFAOYSA-N 0.000 description 1
- 229910000831 Steel Inorganic materials 0.000 description 1
- 238000007792 addition Methods 0.000 description 1
- 244000052616 bacterial pathogen Species 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 230000007274 generation of a signal involved in cell-cell signaling Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 238000012423 maintenance Methods 0.000 description 1
- 230000006855 networking Effects 0.000 description 1
- 230000008439 repair process Effects 0.000 description 1
- 238000012552 review Methods 0.000 description 1
- 230000008054 signal transmission Effects 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 239000010959 steel Substances 0.000 description 1
- 239000000126 substance Substances 0.000 description 1
- 230000000153 supplemental effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
- G06F3/013—Eye tracking input arrangements
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/0093—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00 with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/40—Information retrieval; Database structures therefor; File system structures therefor of multimedia data, e.g. slideshows comprising image and additional audio data
- G06F16/43—Querying
- G06F16/435—Filtering based on additional data, e.g. user or group profiles
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
- G06F3/0481—Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
- G06F3/0483—Interaction with page-structured environments, e.g. book metaphor
Definitions
- This application relates generally to the field of computer technology and, in a specific example implementation, to a system and method for selecting and displaying additional content to a user.
- digital media can be dynamically altered to improve the user experience and allow advertisers to reach potential customers more effectively.
- websites often include areas that can be filled with advertisements, such as banner ads, that can be added to the webpage at the time the webpage is requested.
- banner ads advertisements that can be added to the webpage at the time the webpage is requested.
- users often find banner advertisements annoying and not related to their interests.
- FIG. 1 is a network diagram depicting a client-server system, within which one example implementation may be deployed.
- FIG. 2 is a block diagram illustrating content selection and user action analysis applications, in one example implementation.
- FIG. 3 illustrates an exemplary user interface of a client device presenting an electronic document (e-book).
- e-book an electronic document
- FIG. 4 illustrates an exemplary user interface of a client device presenting an electronic document (e-book).
- e-book an electronic document
- FIG. 5 illustrates an exemplary user interface of a client device presenting an electronic document (e-book).
- FIG. 6A illustrates an exemplary user interface of a display presenting an electronic document (e-magazine).
- FIG. 6B illustrates an exemplary user interface of a display presenting an electronic document (e-magazine).
- FIG. 7 is a flow diagram illustrating a process for surfacing related content based on user interactions with the currently presented content in accordance with some implementations.
- FIG. 8 is a flow diagram illustrating a process for identifying content viewed by the user by eye-tracking in accordance with some implementations.
- FIG. 9 is a flow diagram illustrating a process for determining user interest in a current page of content in accordance with some implementations.
- FIG. 10 is a flow diagram illustrating a process for selecting additional content to surface in accordance with some implementations.
- FIG. 11 is a flow diagram illustrating a process for surfacing additional content based on user interaction with currently presented content in accordance with some implementations.
- FIG. 12 is a flow diagram illustrating a process for surfacing additional content based on user interaction with currently presented content in accordance with some implementations.
- FIG. 13 is a flow diagram illustrating a process for surfacing additional content based on user interaction with currently presented content in accordance with some implementations.
- FIG. 14 is a flow diagram illustrating a process for surfacing additional content based on user interaction with currently presented content in accordance with some implementations.
- FIG. 15 is a block diagram illustrating a mobile device 1500 , according to an example implementation.
- FIG. 16 is a block diagram illustrating components of a machine 1600 , according to some example implementations.
- a media content item is presented on an electronic device.
- Media content items include, but are not limited to, textual content (e.g., an e-book, an electronic magazine, a website, etc.), video content (e.g., a movie, television show, animation, web clip, etc.), audio content (e.g., a music file or a podcast), an image, a video game, or any other media type.
- Media content items can be presented on an electronic device, such as a display device for visual media like images, text, or video. Audio media content items, such as music or podcasts, can be presented though speakers, earphones, and the like.
- a media content item e.g., a video, an audio clip, a digital book or e-book, or a video game or other animation
- the user of the electronic device is able to take actions in relation to the presented media content item.
- only a portion of a media content item is displayed at any given time (e.g., one page of a book or a frame of a movie).
- Some user actions include an interaction with a specific portion of the display section of the media content item (e.g., the user can click on a particular word in an e-book for a definition, right-click on an image to save it, or hover a cursor over a particular section or the rewind a video or audio presentation to repeat a particular section of a media content item.)
- Other user actions do not include direct user interaction with a specific portion of the presented media content item (e.g., a user looking at a portion of the display, turning up the volume during a particular section of a podcast, turning an e-book page or ceasing to walk while reading an article in a digital magazine.)
- the electronic device detects the user action.
- the electronic device includes a camera capable of tracking the positions of the user's eyes and then calculating the part of the display that the user is currently focused on.
- the electronic device detects direct input from a user, such as a mouse click input or a detected finger gesture on a touch screen.
- the electronic device can also measure motion of the device, through either an accelerometer or a global positioning system (GPS) device.
- GPS global positioning system
- the electronic device determines whether the detected user action selects a particular part of the currently presented portion of the media content item. For example, a mouse click selects the particular part of the currently presented portion of the media content item associated with the location of the mouse click. In contrast, a page turn gesture causes the section of the media content item that is currently displayed to change, but does not select a particular part of the currently presented portion of the media a content item.
- the electronic device analyses the user action to determine whether it is associated with a particular portion of the currently presented media item.
- the electronic device transmits the detected user action to the server system for analysis.
- the server system determines whether the user action is associated with a particular portion of the currently presented media content item. For example, the electronic device determines a gaze point for the user based on the user's eye positions and then transmits that gaze point to the server system. If the user is looking at a specific portion of the electronic display, the server system then determines that the user action (e.g., gaze point) is associated with a particular portion of the content media item (e.g., a particular section of text in a magazine or a portion of a screen during a movie).
- the user action e.g., gaze point
- the electronic device determines that the user action is associated with a particular portion of the currently presented media content item, the electronic device then identifies additional content associated with the particular portion of the currently presented media content item.
- the creator of the content has pre-designated specific additional content for specific sections of the content media item. For example, an author includes additional resources or further reading suggestions for particular sections of an e-book.
- a displayed advertisement has additional content associated with it that includes further information about an advertised product or a link to buy the product.
- the electronic device or the server system analyzes the content in the section of the media content item that is associated with the user action to determine one or more topics associated with the content. For example, the server system parses the text of an e-book to determine one or more associated topics based on the specific words included in the text. In other examples, the server system uses an image recognition algorithm to identify a particular image of interest in a section of an image or video. Once the image is identified, the server system uses a look-up table or other method to identify topics associated with the identified image.
- the server system determines that a particular image or section of an image is a wine glass
- the identified topics include wine, home furnishings, and glasses.
- the video or image has associated metadata that identifies topics associated with an image or section of a video.
- the electronic device uses the identified topics to select additional content to present to the user.
- Additional content includes, but is not limited to, advertisements, additional related content (e.g., making of documentaries, commentary, behind the scenes photos), supplemental content, additional learning resources, offers to sell, auction listings, and so forth.
- additional related content e.g., making of documentaries, commentary, behind the scenes photos
- supplemental content e.g., making of documentaries, commentary, behind the scenes photos
- additional content e.g., additional learning resources, offers to sell, auction listings, and so forth.
- the additional content includes one or more advertisements for wines, information on local wineries, and an advertisement for a current home furnishing auction listing.
- the electronic device selects one or more of additional content items and transmits it to the electronic device for presentation alongside the currently presented media content item.
- the additional content is presented concurrently with the currently presented media content item.
- the above implementations can be performed remotely by a server system (e.g., streams media data items and receives user action data from the electronic device over a network).
- a server system e.g., streams media data items and receives user action data from the electronic device over a network.
- the various steps described above can be split between a server system and an electronic device in any configuration that is useful.
- the electronic device is able to access a third party web service (e.g., a server system available over a network that provides a service).
- the electronic device first determines one or more topics (or keywords) that are associated with the user action. Then, the electronic device transmits the determined topics (or keywords) to the third party service provider.
- the topics are first translated into a search query and then transferred to the third party server system.
- the third party server system receives the topics from the electronic device.
- the third party server receives additional metadata about the request.
- the request includes a number of additional content media items needed, the type of media, the length of the media (in the case of video or audio), the size of the media (for text or images), the source of the media, and any other relevant metadata.
- the third party server system uses the received topics, keywords, or search query to identify relevant media content items. In some implementations the third party server system then ranks the identified media content items based on relevance and any associated metadata and sends back the requested number of additional media content items to the electronic device. In other implementations the server system filters out any media that does not match the received metadata and sends all identified media content items (or links to all identified media content items) and the electronic device ranks them and selects the best matches. In some implementations the third party server system clips text or video to an acceptable length before transmitting them to the client system.
- FIG. 1 is a network diagram depicting a client-server system 100 , within which one example implementation may be deployed.
- a networked system 102 in the example forms of a network-based marketplace or publication system, provides server-side functionality, via a network 104 (e.g., the Internet or a wide area network (WAN)), to one or more clients.
- FIG. 1 illustrates, for example, a web client 106 (e.g., a browser, such as the Internet Explorer® browser developed by Microsoft® Corporation of Redmond, Wash.) and a programmatic client 108 executing on respective devices 110 and 112 .
- a web client 106 e.g., a browser, such as the Internet Explorer® browser developed by Microsoft® Corporation of Redmond, Wash.
- programmatic client 108 executing on respective devices 110 and 112 .
- An Application Program Interface (API) server 114 and a web server 116 are coupled to, and provide programmatic and web interfaces respectively to, one or more application servers 118 .
- the application servers 118 host one or more content selection applications 120 and user action analysis applications 122 .
- the application servers 118 are, in turn, shown to be coupled to one or more database servers that facilitate access to one or more databases 126 .
- the user action analysis applications 122 receive one or more user actions from one or more client devices 110 .
- the user action analysis applications 122 then analyze the received user action to determine whether it pertains to specific portions of a media content item. If so, the user action analysis applications determine which one or more specific portions of the media content item relate to the user action.
- the user action analysis applications 122 then transmit the determined one or more specific portions to the content selection applications 120 .
- the content selection applications 120 use the received one or more specific portions to select additional content to be provided to a client device 110 based on analysis performed by the user action analysis applications 122 . While the content selection and user action analysis 120 and 122 are shown in FIG. 1 to both form part of the networked system 102 , it will be appreciated that other configurations can be used such that the content selection applications 120 and the user action selection applications 122 may each form part of a service that is separate and distinct from the networked system 102 .
- system 100 shown in FIG. 1 employs a client-server architecture
- the implementations are, of course, not limited to such an architecture, and could equally well find application in a distributed, or peer-to-peer, architecture system, for example.
- the various content selection and user action analysis applications 120 and 122 could also be implemented as standalone software programs, which do not necessarily have networking capabilities.
- the web client 106 accesses the various content selection and user action analysis applications 120 and 122 via the web interface supported by the web server 116 .
- the programmatic client 108 accesses the various services and functions provided by content selection and user action analysis applications 120 and 122 via the programmatic interface provided by the API server 114 .
- the programmatic client 108 may, for example, be a seller application (e.g., the Turbo Lister application developed by eBay Inc., of San Jose, Calif.) to enable sellers to author and manage listings on the networked system 102 in an offline manner, and to perform batch-mode communications between the programmatic client 108 and the networked system 102 .
- FIG. 1 also illustrates a third party application 128 , executing on a third party server machine 130 , as having programmatic access to the networked system 102 via the programmatic interface provided by the API server 114 .
- the third party application 128 may, utilizing information retrieved from the networked system 102 , support one or more features or functions on a website hosted by the third party.
- the third party website may, for example, provide one or more promotional, marketplace, or payment functions that are supported by the relevant applications of the networked system 102 .
- the applications 120 and 122 may be hosted on dedicated or shared server machines (not shown) that are communicatively coupled to enable communications between server machines.
- the applications 120 and 122 themselves are communicatively coupled (e.g., via appropriate interfaces) to each other and to various data sources, so as to allow information to be passed between the applications 120 and 122 or so as to allow the applications 120 and 122 to share and access common data.
- the applications 120 and 122 may furthermore access one or more databases 126 via the database servers 124 .
- FIG. 2 is a block diagram illustrating content selection and user action analysis applications 120 and 122 that, in one example implementation, are provided as part of application server(s) 118 in the networked system 102 .
- the networked system 102 provides services that select and transmit for presentation additional content related to media content already being presented at the electronic device of a user.
- the content selection application(s) 120 in some implementations, includes several modules or sub-applications to perform its role.
- a content selection application 120 uses one or more topics of interest to select one or more additional content items.
- the one or more topics of interest are based on a specific piece of currently displayed content (e.g., the text of a document currently displayed to a user).
- the content selection application 120 also uses information about the user, such as age, gender, location, interests, past purchasing activity, and past browsing/media consumption activity, to customize and personalize the selected additional content.
- this user information is submitted directly from the user.
- the information is gathered or inferred by the networked server system (e.g., the networked server system 102 in FIG. 1 ) and stored in a user profile.
- the server system has a privacy policy that prevents it from gathering or storing any personal information for a user without the user's express permission.
- the content selection application(s) 120 includes a content display application 202 .
- the content display application 202 transmits media content items for display at the client system.
- the network system stores a plurality of media content items and transmits them to clients for presentation as requested.
- the networked system hosts a plurality of e-books and transmits or streams to the users based on the user's requests.
- Metadata analysis applications 204 support the content selection applications 200 by analyzing content and determining matching topics based on metadata associated with the media content. For example, a movie has associated metadata (e.g., genre information, actor information, director information, length, country of origin, a plot synopsis, etc.). The metadata analysis applications 204 use this information to identify one or more topics associated with the content item.
- metadata e.g., genre information, actor information, director information, length, country of origin, a plot synopsis, etc.
- image recognition applications 206 support the content selection applications 120 by analyzing images or a section of an image to identify objects within the images. Any suitable image detection algorithm can be used to accomplish this task. In some implementations, once the image recognition applications 206 identify one or more objects within the image, it uses a lookup table or other suitable mechanism to identify topics associated with the identified object or objects. For example, if the identified object is a classic car, the identified topics include “Car Shows,” “Classic Cars,” and “Car Maintenance and Repairs.”
- Text analysis applications 208 support the content selection applications 200 by analyzing text content associated with a particular portion of a text based media content item.
- the text analysis application 208 parses the relevant text and, using appropriate text parsing algorithms, identifies one or more topics associated with the text content. This identification may be accomplished by identifying word frequencies, key words, and patterns within the text.
- the identified topics are then sent to the content selection applications 120 to a topic matching application 210 to assist in selecting additional content. For example, if a given portion of text included the text “20-16,” “Jerry Rice,” and “Jan. 22, 1989,” the text analysis application 208 would determine that the portion of text was discussing the 1989 SuperBowl and identify “Professional American Football,” “San Francisco 49ers,” and “the 80s” as topics of interest.
- the topic matching application 210 supports the content selection applications 120 by receiving one or more identified topics of interest from another module or application and using it to identify additional content items that have similar associated topics. For example, if the determined topics are “world history,” “books,” and “best-sellers,” the topic matching applications matches those topics with the book “Guns, Germs, and Steel,” which also includes those topics. The topic matching applications 210 then notify the content selection applications 120 of the matching additional content.
- the user action detection applications 122 receive a notification from a client device (e.g., client device 102 in FIG. 1 ) that a user action has occurred. The user action detection application 122 then determines whether the user action is associated with a specific section of the currently presented media content. In accordance with a determination that the user action is associated with a specific portion of the media content item, the user action detection application then determines the specific portion of the media content item that is associated with the user action.
- a touchscreen input application 214 receives and interprets touch screen input from a client device. For example, if the user makes a swipe gesture on the touch screen of a device currently displaying an electronic book, the touch screen input application 214 determines what specific portion of the electronic book is associated with the swiping motion (e.g., the page that is displayed after the swipe is complete).
- the reading speed detection application 216 calculates the approximate reading speed of a user.
- the reading speed can be calculated by determining the number of words on a page and how long it takes the user to turn to the next page. As more and more pages are read, the reading speed of the user is approximated more accurately. Then, when the user turns to a new page in an electronic document, the user action analysis application 212 can estimate the user's current reading position on the page by estimating the number of words the user has read since the page was turned.
- the reading speed detection application 216 determines that the user averages about 120 words read per minute and the user has been reading a page with about 500 words for two minutes, the reading speed detection application can then estimate that the reader is about half way through the text (e.g., about 250 words down the page). The user action analysis application 212 can then identify the nearby text as the specific portion of media content currently of interest to the user.
- the eye tracking application 218 uses a camera or infrared tracking device to measure the movement of a user's eyes. With this information, and information regarding the position of a display associated with the user of the client device, the eye tracking application 218 can estimate the particular portion of a display that the user is current focusing on. The eye tracking application 218 then identifies this currently viewed portion of the display as the portion of the media content item currently of interest to the user.
- device movement tracking applications 220 determine whether the client device is moving and in what direction. This can be tracked by way of a GPS tracking device or an accelerometer device internal to the client device. When the movement of the device changes (e.g., starts moving, ceases moving, or changes movement speeds), the device movement tracking application 220 records a user action. In some implementations the device movement tracking application 220 determines that the change in movement is the result of user interest in the presented media content (e.g., the user stops walking to focus on a particular scene). In this case, the device movement tracking applications 220 identify the currently displayed portion of the media content item as of interest to the user.
- the device movement tracking applications 220 identify the currently displayed portion of the media content item as of interest to the user.
- Interest determination applications 222 use the received user actions to determine whether the user action is indicative of user interest in a particular portion of the presented media content. For example, the client device reports a constant rate of movement for 25 minutes and then a deceleration to a stop. Based on the accompanying GPS data, the interest determination application 222 determines that the detected action is a result of a train ride that the user is on coming to an end, not increased user interest in the current content. Conversely, if the user inputs instructions to zoom in on a particular section of an image, the interest determination application 222 determines that the user is interested in the zoomed in portion of the image.
- a user increases the volume of current presented audio or video content.
- the interest determination application 222 determines whether there has been a corresponding increase in environmental noise using a microphone or other audio sensing device. If there has been an increase in environmental noise, the interest determination application 222 determines that the increase in volume level is the result of the user reacting to the increased background noise, not a result of increased interest in the currently present content. However, if the interest determination application 222 determines there has been no recent increase in background noise, then the user increasing the volume may be a result of increased user interest in the currently presented content.
- the networked system 102 includes one or more network commerce applications 224 for enabling commercial transactions over the networked system 102 .
- Listing management applications 226 allow sellers to create and manage listings. Specifically, where a particular seller has authored and/or published a large number of listings, the management of such listings may present a challenge.
- the listing management applications 226 provide a number of features (e.g., auto-relisting, inventory level monitors, etc.) to assist the seller in managing such listings. ⁇
- One or more post-listing management applications may also assist sellers with a number of activities that typically occur post-listing. For example, upon completion of an auction facilitated by one or more auction applications 228 , a seller may wish to leave feedback regarding a particular buyer. To this end, a post-listing management application may provide an interface to one or more reputation applications 230 , so as to allow the seller to conveniently provide feedback regarding multiple buyers to reputation applications 230 .
- Reputation applications 230 allow users who transact, utilizing the networked system 102 , to establish, build, and maintain reputations, which may be made available and published to potential trading partners.
- the reputation applications 230 allow a user (for example, through feedback provided by other transaction partners) to establish a reputation within the networked system 102 over time. Other potential trading partners may then reference such a reputation for the purposes of assessing credibility and trustworthiness.
- a number of fraud prevention applications 232 implement fraud detection and prevention mechanisms to reduce the occurrence of fraud within the networked system 102 .
- Messaging applications 234 are responsible for the generation and delivery of messages to users of the networked system 102 (such as, for example, messages advising users regarding the status of listings at the networked system 102 (e.g., providing “outbid” notices to bidders during an auction process or providing promotional and merchandising information to users)). Respective messaging applications 234 may utilize any one of a number of message delivery networks and platforms to deliver messages to users.
- messaging applications 228 may deliver electronic mail (e-mail), instant message (IM), Short Message Service (SMS), text, facsimile, or voice (e.g., Voice over IP (VoIP)) messages via the wired (e.g., the Internet), plain old telephone service (POTS), or wireless (e.g., mobile, cellular, Wi-Fi, WiMAX) networks 104 .
- e-mail electronic mail
- IM instant message
- SMS Short Message Service
- SMS Short Message Service
- text e.g., facsimile
- voice e.g., Voice over IP (VoIP)
- POTS plain old telephone service
- wireless e.g., mobile, cellular, Wi-Fi, WiMAX
- FIG. 2 shows content selection applications 120 , user action analysis applications 122 , and network commerce applications, some implementations do not include all these application.
- an e-book reader can include both the content selection applications 120 and the user action analysis applications 122 but not include any network commerce applications.
- FIG. 3 illustrates an exemplary user interface 302 of a client device 300 presenting an electronic document (e-book).
- the electronic device 300 includes a display 302 .
- digital text 306 in this case an article about polar bears.
- the user interface also displays a user interaction object 304 that represents user input at a particular part of the display from either a mouse input or a finger gesture on a particular section of a touch screen. In this case the user input initiates changing from a first page (e.g., page 1) to a second page (e.g., page 2).
- FIG. 4 illustrates an exemplary user interface 400 of a client device 300 presenting an electronic document (e-book).
- the electronic device 300 includes a display 302 .
- page 2 of the digital text 402 in this case an article about polar bears shown in FIG. 3 after the page turning input received previously.
- FIG. 5 illustrates an exemplary user interface 500 of a client device 300 presenting an electronic document (e-book).
- the electronic device 300 includes a display 302 .
- page 2 of the digital text 402 in this case an article about Polar bears
- additional content 502 displayed concurrently with the original content.
- the additional content takes the form of a link to additional information about early arctic explorers 502 .
- the additional content is displayed immediately.
- the additional content is displayed after a fixed amount of time (e.g., 5 seconds after the page turns).
- the additional content is displayed based upon the user's reading speed (e.g., displayed when the device estimates the user has just read the related content.)
- FIG. 6A illustrates an exemplary user interface 600 of a display presenting an electronic document (e-magazine).
- the display includes a section containing article content 602 and three sections containing advertisements (World Wide Airways Advertisement 604 , Cunningham Automobile Corporation Advertisement 606 , and Rondo Soda Advertisement 608 ).
- the display has an associated camera that tracks the eye movements of the user. In this case the left eye is pointed to a left eye focus point 610 on the display, and the right eye is pointed to a right eye focus point 612 on the display. Based on the two focus points, the device calculates a gaze area 614 in the area assigned to the Cunningham Automobile Corporation advertisement 606 .
- the network system can use this information to determine that user is currently focused on the Cunningham Automobile Corporation advertisement 606 .
- FIG. 6B illustrates an exemplary user interface 600 of a display presenting an electronic document (e-magazine).
- the display includes article content 602 , 3 advertisements ( 604 , 606 , and 608 ), and additional content 620 that was displayed in response to the electronic device associated with the display determining that the user is interested in further content related to the Cunningham Automobile Corporation Advertisement 606 based on the detected position of the user's eyes.
- the additional content 620 is displayed partially overlapping and simultaneous with the already displayed content.
- the additional content 620 includes information related to the Cunningham Automobile Corporation advertisement 606 , including three buttons ( 622 , 624 , 626 ) that offer additional information about specific automobiles sold by the company associated with the Cunningham Automobile Corporation advertisement 606 (presumably the Cunningham Automobile Corporation).
- FIGS. 7-14 are flow diagrams that represent a method for surfacing related content based on user interactions with the currently presented content. It should be noted that while each Figure is described as being performed by either an electronic device or by a server system communicating with that electronic device, the steps in each of the Figures may be performed either by the electronic device or by a server system communicating with that device. As such, the methods described below can be rearranged so that all the steps are performed at a single electronic device, all the steps are performed at a server system in communication with that device, or the steps are divided up between the two devices (server device and a client device) pieces in any way.
- FIG. 7 is a flow diagram illustrating a process for surfacing related content based on user interactions with the currently presented content in accordance with some implementations.
- Each of the operations shown in FIG. 7 may correspond to instructions stored in a computer memory or computer readable storage medium. Optional operations are indicated by dashed lines (e.g., boxes with dashed-line borders).
- the method described in FIG. 7 is performed by the electronic device (e.g., electronic device 110 in FIG. 1 ).
- the method is performed at a computer system including one or more processors and memory storing one or more programs for execution by the one or more processors.
- the electronic device e.g., electronic device 110 in FIG. 1
- the media content item is either displayed on a screen (in the case of a visual media content item such as a movie, webpage, e-book, etc.) or played over a speaker or headphones (in the case of an audio media content item such as music, podcasts, audiobooks, etc.).
- the electronic device identifies a user action 704 .
- An identified user action can include any user action that the electronic device (e.g., electronic device 110 in FIG. 1 ) is capable of detecting including, but not limited to, input received from the user from a keyboard, mouse, microphone, touchscreen, camera, or any other input device; a physical movement of the user; position of the user's body, including his or her eyes; control commands from the user (e.g., pausing, zooming in, turning up the volume); and sounds made by the user (detecting and analyzing user speech).
- Some user actions include specifically selecting a part of the media content item currently on the display.
- clicking with a mouse touching an icon with a figure gesture, hovering a pointer, and similar user actions all involve selecting a particular part of the content currently displayed.
- Other user actions do not involve selecting a particular part of the content that is currently displayed but may still be associated with a particular part.
- a swipe gesture on an e-reader will result in all of the currently displayed content being removed and a new section of the media content item (in this case an e-book) being displayed. The finger swipe does not select any portion of the previous page or the new page, but is associated with both.
- the electronic device analyses the user action to identify ( 706 ) a portion of the media content item associated with the user action. For example, if the user action is a mouse click on a displayed link, the electronic device (e.g., electronic device 110 in FIG. 1 ) identifies the link (and the content it links to) as being associated with the user action. If the user action is an eye movement to change from one gaze point to a second gaze point, the electronic device (e.g., electronic device 110 in Figure 1 ) identifies the content displayed in the part of the screen that the user is currently focused on as being associated with the changing gaze point.
- the electronic device e.g., electronic device 110 in FIG. 1
- the electronic device selects ( 708 ) one or more additional content items. These additional content items are selected based on the identified content. For example, if the identified content is a picture of a whale, the additional content may be an article about whales.
- the electronic device (e.g., electronic device 110 in FIG. 1 ) then presents ( 710 ) the one or more additional selected content items at the electronic device.
- the additional content is displayed simultaneously with the originally displayed content, either overlapping or partially displacing some of the content.
- FIG. 8 is a flow diagram illustrating a process for identifying content viewed by the user by eye-tracking in accordance with some implementations.
- Each of the operations shown in FIG. 8 may correspond to instructions stored in a computer memory or computer readable storage medium. Optional operations are indicated by dashed lines (e.g., boxes with dashed-line borders).
- the method described in FIG. 8 is performed by the networked server system (e.g., the networked server system in FIG. 1 ).
- the method is performed at a computer system including one or more processors and memory storing one or more programs for execution by the one or more processors.
- the electronic device e.g., electronic device 110 in FIG. 1
- the electronic device e.g., electronic device 110 in FIG. 1
- the electronic device determines ( 802 ) the left gaze point. Determining a gaze point includes using the position of an eye to determine a line of sight for that eye. Then the device determines at what point that line of sight intercepts a particular surface (in this case a display). The point where the line of sight crosses through the display is a gaze point. Each eye has its own specific gaze point.
- the electronic device determines ( 804 ) the right gaze point. Using the right and left gaze points, the electronic device (e.g., electronic device 110 in FIG. 1 ) determines ( 806 ) the user focus area.
- the user focus area is where the user is current focusing. This is determined by measuring the distance between the two gaze points. Gaze points that are relatively far apart indicate that the user focusing on something behind the display or just gazing off into space. Gaze points that are too close together or where the right gaze point is to the left of the left gaze point indicate that the user is focusing on something in front of the display.
- the electronic device determines ( 808 ) whether the user has a focus area that is currently on the display. If not, the electronic device (e.g., electronic device 110 in FIG. 1 ) continues to monitor user actions ( 810 ).
- the electronic device determines that the user focus area is on the display screen
- the electronic device determines ( 812 ) the portion of the display associated with the user focus area. For example, the electronic device (e.g., electronic device 110 in FIG. 1 ) determines whether the focus area is at the top of the screen, the bottom of the screen, or the left or right side of the screen.
- the electronic device e.g., electronic device 110 in FIG. 1
- the electronic device identifies ( 814 ) the particular content displayed at the part of the display that the user is current focusing on. For example, if the user focus area is above text, the electronic device (e.g., electronic device 110 in FIG. 1 ) determines the content of that text. If the user focus area is on an image or video, the electronic device (e.g., electronic device 110 in FIG. 1 ) analyzes the image or video to determine what is being displayed.
- FIG. 9 is a flow diagram illustrating a process for determining user interest in a current page of content in accordance with some implementations.
- Each of the operations shown in FIG. 9 may correspond to instructions stored in a computer memory or computer readable storage medium. Optional operations are indicated by dashed lines (e.g., boxes with dashed-line borders).
- the method described in FIG. 9 is performed by the electronic device (e.g., electronic device 110 in FIG. 1 ).
- the method is performed at a computer system including one or more processors and memory storing one or more programs for execution by the one or more processors.
- the electronic device e.g., electronic device 110 in FIG. 1
- the electronic device is an e-reader or other device capable of displaying textual media content.
- an e-reader can display e-books.
- the electronic device e.g., electronic device 110 in FIG. 1
- the electronic device (e.g., electronic device 110 in FIG. 1 ) first detects ( 902 ) a page change.
- a page change occurs whenever the user replaces previous content on the display with new content. For example, a user swipes on a touch screen that is currently displaying an e-book. The next page of the e-book is displayed in response. Once the next page is displayed, the electronic device (e.g., electronic device 110 in FIG. 1 ) begins ( 904 ) a page timer.
- a page timer tracks the amount of time that the user has been on a specific page. In some implementations page timers are used to build a reading speed estimate for a user.
- the electronic device e.g., electronic device 110 in FIG. 1
- the electronic device e.g., electronic device 110 in FIG. 1
- the electronic device can then generate an estimate reading time for the new page. For example, if the reading speed is 83.3 words per minute and the new page has 250 words, the electronic device (e.g., electronic device 110 in FIG. 1 ) can estimate that the user will take about 3 minutes to finish the page. The electronic device (e.g., electronic device 110 in FIG. 1 ) can then use this value as the estimated time for the user to finish the page. In some implementations the electronic device (e.g., electronic device 110 in FIG. 1 ) also adds a buffer amount to the estimate time (e.g., 20% of the estimated time) to account for minor variations in reading speed due to difficulty of content and various distractions that are unrelated to the content.
- a buffer amount e.g. 20% of the estimated time
- the electronic device e.g., electronic device 110 in FIG. 1
- the electronic device determines ( 912 ) that the user has increased interested in the current page and its contents.
- FIG. 10 is a flow diagram illustrating a process for selecting additional content to surface in accordance with some implementations.
- Each of the operations shown in FIG. 10 may correspond to instructions stored in a computer memory or computer readable storage medium. Optional operations are indicated by dashed lines (e.g., boxes with dashed-line borders).
- the method described in FIG. 10 is performed by the electronic device (e.g., electronic device 110 in FIG. 1 ).
- the method is performed at a computer system including one or more processors and memory storing one or more programs for execution by the one or more processors.
- the electronic device e.g., electronic device 110 in FIG. 1
- the user action can be anything the user does that the electronic device (e.g., electronic device 110 in FIG. 1 ) can detect.
- a user action includes the physical movement of the user (provided the device has a camera, accelerometer, or other means for tracking movement) and interactions the user has with the electronic device (e.g., electronic device 110 in FIG. 1 ) such as mouse inputs, touch inputs, or any other direct input into the system.
- the electronic device determines ( 1004 ) whether there is already preselected additional content available.
- the preselected additional content is chosen by the author or creator of the content to enhance the user's experience with the content.
- the preselected additional content is a form of advertisement inserted into the media content item as a result of a business transaction between the creator and the advertiser.
- the electronic device In accordance with a determination that preselected additional content is available, the electronic device (e.g., electronic device 110 in FIG. 1 ) then presents ( 1006 ) the preselected additional content. In accordance with a determination that the preselected additional content is not available, the electronic device (e.g., electronic device 110 in FIG. 1 ) then analyzes ( 1008 ) the portion of the media content item identified as associated with the user action to determine one or more topics of interest based on the content of the analyzed portion.
- the electronic device uses the topics of interest to identify ( 1010 ) one or more additional content items that match the determined topics of interest. For example, if the portion of the media content item is an essay about lettuce, the identified topics of interest include “lettuce,” “agriculture,” “cooking,” and “Spanish Exports.”
- the electronic device e.g., electronic device 110 in FIG. 1
- the electronic device e.g., electronic device 110 in Figure 1
- the electronic device (e.g., electronic device 110 in FIG. 1 ) then ranks ( 1012 ) the one or more additional content items based on suitability for the user.
- the electronic device e.g., electronic device 110 in FIG. 1
- the electronic device e.g., electronic device 110 in FIG. 1
- the electronic device selects ( 1014 ) one or more additional content items for presentation based on their respective ranking.
- the electronic device determines how many additional content items to display. This determination is based on what media content is currently displayed (e.g., if a video is playing the number of additional content items is very low, perhaps only 1), the space available on the display, and the preferences of the user. Once the number of additional content items to display is determined, the electronic device (e.g., electronic device 110 in FIG. 1 ) chooses the highest ranked additional content items to fulfil the needed number of additional content items.
- FIG. 11 is a flow diagram illustrating a process for surfacing additional content based on user interaction with currently presented content in accordance with some implementations.
- Each of the operations shown in FIG. 11 may correspond to instructions stored in a computer memory or computer readable storage medium. Optional operations are indicated by dashed lines (e.g., boxes with dashed-line borders).
- the method described in FIG. 11 is performed by the electronic device (e.g., electronic device 110 in FIG. 1 ).
- the method is performed at a computer system including one or more processors and memory storing one or more programs for execution by the one or more processors.
- the electronic device e.g., electronic device 110 in FIG. 1
- media content items include, but are not limited to, textual content (e.g., an e-book, an electronic magazine, a website, etc.), video content (e.g., a movie, TV show, animation, web clip, etc.), audio content (e.g., a music file or a podcast), images, video games, or any other media content.
- Media content items can be presented on an electronic device, such as a display device for visual media like images, text, or video. Audio media content items, such as music or podcasts, can be presented though speakers, earphones, or the like.
- the electronic device detects ( 1103 ) user action with a respective portion of the media content item, wherein the user action does not include explicit selection of the respective portion of the media content item.
- a user action that includes a direct click on a link includes explicit selection of the respective portion of the media content item
- a user action that includes turning up the volume does not explicitly select a respective portion of the media content item.
- detecting user action includes detecting ( 1104 ) user interactions with an electronic device associated with the electronic display. For example, any user action that involves the user intentionally using an input device, such as a mouse, keyboard, touch screen, microphone, camera, or other input device to interact with the electronic device (e.g., electronic device 110 in FIG. 1 ).
- the electronic device e.g., electronic device 110 in FIG. 1
- the electronic device measures ( 1108 ) physical movements of a user.
- the electronic device e.g., electronic device 110 in FIG. 1
- a user's movements can be considered a user action.
- the electronic device e.g., electronic device 110 in FIG. 1
- user actions include but are not limited to changing the currently displayed page, pausing a video at a particular part, rewinding to review or re-listen to a particular section, eye movement, changes in movement of a user, voice commands received by a user, volume changes by a user, and the like.
- the electronic device in response to detection of the user action, identifies ( 1112 ) additional content to present based on the content included in the portion of the respective portion of the media content item.
- identifying additional content includes determining ( 1114 ) whether any additional content has been preselected by the creator of the media content item to be associated with the respective portion of the media content item.
- an e-book is created by an author. In addition to the text of the book, the author compiles a list of additional media resources, such as additional books, recommendations, clips, scientific papers, web pages, and so forth. The author then chooses which specific section of the e-book each additional media resources is associated with. Then if the user demonstrates interest in a specific portion through a user action, the previously associated additional content item can be displayed.
- FIG. 12 is a flow diagram illustrating a process for surfacing additional content based on user interaction with currently presented content in accordance with some implementations.
- Each of the operations shown in FIG. 12 may correspond to instructions stored in a computer memory or computer readable storage medium. Optional operations are indicated by dashed lines (e.g., boxes with dashed-line borders).
- the method described in FIG. 12 is performed by the electronic device (e.g., electronic device 110 in FIG. 1 ).
- the method is performed at a computer system including one or more processors and memory storing one or more programs for execution by the one or more processors.
- the electronic device determines ( 1204 ) the preselected additional content as the identified additional content. For example, a particular scene in a movie contains a tablet computer, and the electronic device determines that the user is looking at the tablet.
- the electronic device e.g., electronic device 110 in FIG.
- the electronic device determines that the creator of the movie has preselected an advertisement for the tablet and associated it with the scene in the movie where the tablet appears.
- the electronic device e.g., electronic device 110 in FIG. 1
- the electronic device analyzes ( 1206 ) the respective portion of the media content item to determine one or more associated topics. For example, if the respective portion is text, the electronic device (e.g., electronic device 110 in FIG. 1 ) parses the text to identify keywords and then identifies specific topic using the keywords. Once the one or more topics have been determined, the electronic device (e.g., electronic device 110 in FIG. 1 ) identifies the particular portion of the currently presented media content item ( 1208 ) one or more additional content items that match the determined one or more associated topics.
- the electronic device (e.g., electronic device 110 in FIG. 1 ) then ranks ( 1210 ) the one or more determined additional content items.
- Ranking includes the electronic device (e.g., electronic device 110 in FIG. 1 ) generating ( 1212 ) a match score for each of the one or more determined additional content items.
- the electronic device e.g., electronic device 110 in FIG. 1
- each media content item includes a list of associated topics or keywords. Then the electronic device ( FIG.
- a match score based on the number of keywords or topics that each additional media content item has in common with the list of associated topics or keywords, such that additional media content items that have more topics or keywords in common have a higher match score.
- each associated topic has an associated relevance score that is used in calculating a match score.
- a match score is based, at least in part, on demographic information of the user. In some implementations a match score is based on user preference data for the user.
- the electronic device orders ( 1214 ) the one or more determined additional content items based on their associated match score. For example, the additional content items with the highest match score are listed first.
- FIG. 13 is a flow diagram illustrating a process for surfacing additional content based on user interaction with currently presented content in accordance with some implementations.
- Each of the operations shown in FIG. 13 may correspond to instructions stored in a computer memory or computer readable storage medium. Optional operations are indicated by dashed lines (e.g., boxes with dashed-line borders).
- the method described in FIG. 13 is performed by the electronic device (e.g., electronic device 110 in FIG. 1 ).
- the method is performed at an electronic device including one or more processors and memory storing one or more programs for execution by the one or more processors.
- the electronic device selects ( 1302 ) one or more additional content items based on the rankings of the additional content items. For example, the electronic device (e.g., electronic device 110 in FIG. 1 ) selects the highest ranked additional content item. In another example, the electronic device (e.g., electronic device 110 in FIG. 1 ) determines a specific number of needed additional content items and then selects that number of additional content items.
- the electronic device simultaneously present(s) the additional content and the media content item on the electronic display. This can be accomplished by presenting both the additional content and the media content item simultaneously, such that both can be at least partially seen.
- the additional content is displayed in a pop-up such that it partially obscures ( 1306 ) the original content item.
- the original content time is shifted such that space is created for the additional content item by moving part of the original content item off screen.
- the display originally includes room for additional content, and the original content item does not need to be displaced or partially covered up.
- FIG. 14 is a flow diagram illustrating a process for surfacing additional content based on user interaction with currently presented content in accordance with some implementations.
- Each of the operations shown in FIG. 14 may correspond to instructions stored in a computer memory or computer readable storage medium. Optional operations are indicated by dashed lines (e.g., boxes with dashed-line borders).
- the method described in FIG. 14 is performed by the server system (e.g., networked sever system 102 on FIG. 1 ).
- the method is performed at a server system including one or more processors and memory storing one or more programs for execution by the one or more processors.
- the server system performs the steps of the method, transmits the data to a client system for presentation, and then receives communications from the client system to notify the server system of user actions.
- the server system transmits ( 1402 ) a media content item for presentation at a client system.
- the server system streams video content to the client system.
- the server system receives ( 1404 ) notification of a user action associated with a portion of the presented media content item from the client system. For example, when the user changes the page of an e-book, the client system sends a notification to the server system.
- the server system In response to receiving notification of the user action, the server system identifies ( 1406 ) additional content to present based on the respective portion of the media content item. The server system then transmits ( 1408 ) the identified additional content to the client system for simultaneous presentation along with the original media content item.
- FIG. 15 is a block diagram illustrating a mobile device 1500 , according to an example implementation.
- the mobile device 1500 may include a processor 1502 .
- the processor 1502 may be any of a variety of different types of commercially available processors suitable for mobile devices (for example, an XScale architecture microprocessor, a Microprocessor without Interlocked Pipeline Stages (MIPS) architecture processor, or another type of processor).
- a memory 1504 such as a random access memory (RAM), a Flash memory, or other type of memory, is typically accessible to the processor.
- the memory 1504 may be adapted to store an operating system (OS) 1506 , as well as application programs 1508 , such as a mobile location enabled application that may provide Location Based Services (LBSs) to a user.
- OS operating system
- application programs 1508 such as a mobile location enabled application that may provide Location Based Services (LBSs) to a user.
- LBSs Location Based Services
- the processor 1502 may be coupled, either directly or via appropriate intermediary hardware, to a display 1510 and to one or more input/output (I/O) devices 1512 , such as a keypad, a touch panel sensor, a microphone, and the like.
- the processor 1502 may be coupled to a transceiver 1514 that interfaces with an antenna 1516 .
- the transceiver 1514 may be configured to both transmit and receive cellular network signals, wireless data signals, or other types of signals via the antenna 1516 , depending on the nature of the mobile device 1500 .
- a GPS receiver 1518 may also make use of the antenna 1516 to receive GPS signals.
- FIG. 16 is a block diagram illustrating components of a machine 1600 , according to some example implementations, able to read instructions 1624 from a machine-readable medium 1622 (e.g., a non-transitory machine-readable medium, a machine-readable storage medium, a computer-readable storage medium, or any suitable combination thereof) and perform any one or more of the methodologies discussed herein, in whole or in part.
- a machine-readable medium 1622 e.g., a non-transitory machine-readable medium, a machine-readable storage medium, a computer-readable storage medium, or any suitable combination thereof
- FIG. 16 shows the machine 1600 in the example form of a computer system (e.g., a computer) within which the instructions 1624 (e.g., software, a program, an application, an applet, an app, or other executable code) for causing the machine 1600 to perform any one or more of the methodologies discussed herein may be executed, in whole or in part.
- the instructions 1624 e.g., software,
- the machine 1600 operates as a standalone device or may be connected (e.g., networked) to other machines.
- the machine 1600 may operate in the capacity of a server machine or a client machine in a server-client network environment, or as a peer machine in a distributed (e.g., peer-to-peer) network environment.
- the machine 1600 may be a server computer, a client computer, a personal computer (PC), a tablet computer, a laptop computer, a netbook, a cellular telephone, a smartphone, a set-top box (STB), a personal digital assistant (PDA), a web appliance, a network router, a network switch, a network bridge, or any machine capable of executing the instructions 1624 , sequentially or otherwise, that specify actions to be taken by that machine.
- PC personal computer
- PDA personal digital assistant
- STB set-top box
- web appliance a network router, a network switch, a network bridge, or any machine capable of executing the instructions 1624 , sequentially or otherwise, that specify actions to be taken by that machine.
- STB set-top box
- PDA personal digital assistant
- a web appliance a network router, a network switch, a network bridge, or any machine capable of executing the instructions 1624 , sequentially or otherwise, that specify actions to be taken by that machine.
- STB set-top box
- the machine 1600 includes a processor 1602 (e.g., a central processing unit (CPU), a graphics processing unit (GPU), a digital signal processor (DSP), an application specific integrated circuit (ASIC), a radio-frequency integrated circuit (RFIC), or any suitable combination thereof), a main memory 1604 , and a static memory 1606 , which are configured to communicate with each other via a bus 1608 .
- the processor 1602 may contain microcircuits that are configurable, temporarily or permanently, by some or all of the instructions 1624 such that the processor 1602 is configurable to perform any one or more of the methodologies described herein, in whole or in part.
- a set of one or more microcircuits of the processor 1602 may be configurable to execute one or more modules (e.g., software modules) described herein.
- the machine 1600 may further include a video display 1610 (e.g., a plasma display panel (PDP), a light emitting diode (LED) display, a liquid crystal display (LCD), a projector, a cathode ray tube (CRT), or any other display capable of displaying graphics or video).
- a video display 1610 e.g., a plasma display panel (PDP), a light emitting diode (LED) display, a liquid crystal display (LCD), a projector, a cathode ray tube (CRT), or any other display capable of displaying graphics or video).
- PDP plasma display panel
- LED light emitting diode
- LCD liquid crystal display
- CRT cathode ray tube
- the machine 1600 may also include an alphanumeric input device 1612 (e.g., a keyboard or keypad), a UI navigation device 1614 (e.g., a mouse, a touchpad, a trackball, a joystick, a motion sensor, an eye tracking device, or other pointing instrument), a storage unit 1616 , an signal generation device 1618 (e.g., a sound card, an amplifier, a speaker, a headphone jack, or any suitable combination thereof), and a network interface device 1620 .
- an alphanumeric input device 1612 e.g., a keyboard or keypad
- a UI navigation device 1614 e.g., a mouse, a touchpad, a trackball, a joystick, a motion sensor, an eye tracking device, or other pointing instrument
- a storage unit 1616 e.g., an audio card, an amplifier, a speaker, a headphone jack, or any suitable combination thereof
- an signal generation device 1618 e.g
- the storage unit 1616 includes the machine-readable medium 1622 (e.g., a tangible and non-transitory machine-readable storage medium) on which are stored the instructions embodying any one or more of the methodologies or functions described herein.
- the instructions 1624 may also reside, completely or at least partially, within the main memory 1604 , within the processor 1602 (e.g., within the processor's cache memory), or both, before or during execution thereof by the machine 1600 . Accordingly, the main memory 1604 and the processor 1602 may be considered machine-readable media (e.g., tangible and non-transitory machine-readable media).
- the instructions 1624 may be transmitted or received over the network 1626 via the network interface device 1620 .
- the network interface device 1620 may communicate the instructions 1624 using any one or more transfer protocols (e.g., hypertext transfer protocol (HTTP)).
- HTTP hypertext transfer protocol
- the machine 1600 may be a portable computing device, such as a smart phone or tablet computer, and have one or more additional input components (e.g., sensors or gauges).
- additional input components e.g., sensors or gauges.
- input components 1630 include an image input component (e.g., one or more cameras), an audio input component (e.g., a microphone), a direction input component (e.g., a compass), a location input component (e.g., a global positioning system (GPS) receiver), an orientation component (e.g., a gyroscope), a motion detection component (e.g., one or more accelerometers), an altitude detection component (e.g., an altimeter), and a gas detection component (e.g., a gas sensor).
- Inputs harvested by any one or more of these input components may be accessible and available for use by any of the modules described herein.
- the term “memory” refers to a machine-readable medium able to store data temporarily or permanently and may be taken to include, but not be limited to, RAM), read-only memory (ROM, buffer memory, flash memory, and cache memory. While the machine-readable medium 1622 is shown in an example implementation to be a single medium, the term “machine-readable medium” should be taken to include a single medium or multiple media (e.g., a centralized or distributed database, or associated caches and servers) able to store instructions.
- machine-readable medium shall also be taken to include any medium, or combination of multiple media, that is capable of storing the instructions 1624 for execution by the machine 1600 , such that the instructions 1624 , when executed by one or more processors of the machine 1600 (e.g., processor 1602 ), cause the machine 1600 to perform any one or more of the methodologies described herein, in whole or in part.
- a “machine-readable medium” refers to a single storage apparatus or device, as well as cloud-based storage systems or storage networks that include multiple storage apparatus or devices.
- machine-readable medium shall accordingly be taken to include, but not be limited to, one or more tangible (e.g., non-transitory) data repositories in the form of a solid-state memory, an optical medium, a magnetic medium, or any suitable combination thereof.
- Modules may constitute software modules (e.g., code stored or otherwise embodied on a machine-readable medium or in a transmission medium), hardware modules, or any suitable combination thereof.
- a “hardware module” is a tangible (e.g., non-transitory) unit capable of performing certain operations and may be configured or arranged in a certain physical manner.
- one or more computer systems e.g., a standalone computer system, a client computer system, or a server computer system
- one or more hardware modules of a computer system e.g., a processor or a group of processors
- software e.g., an application or application portion
- a hardware module may be implemented mechanically, electronically, or any suitable combination thereof.
- a hardware module may include dedicated circuitry or logic that is permanently configured to perform certain operations.
- a hardware module may be a special-purpose processor, such as a field programmable gate array (FPGA) or an ASIC.
- a hardware module may also include programmable logic or circuitry that is temporarily configured by software to perform certain operations.
- a hardware module may include software encompassed within a general-purpose processor or other programmable processor. It will be appreciated that the decision to implement a hardware module mechanically, in dedicated and permanently configured circuitry, or in temporarily configured circuitry (e.g., configured by software) may be driven by cost and time considerations.
- hardware module should be understood to encompass a tangible entity, and such a tangible entity may be physically constructed, permanently configured (e.g., hardwired), or temporarily configured (e.g., programmed) to operate in a certain manner or to perform certain operations described herein.
- “hardware-implemented module” refers to a hardware module. Considering implementations in which hardware modules are temporarily configured (e.g., programmed), each of the hardware modules need not be configured or instantiated at any one instance in time. For example, where a hardware module comprises a general-purpose processor configured by software to become a special-purpose processor, the general-purpose processor may be configured as respectively different special-purpose processors (e.g., comprising different hardware modules) at different times. Software (e.g., a software module) may accordingly configure one or more processors, for example, to constitute a particular hardware module at one instance of time and to constitute a different hardware module at a different instance of time.
- Hardware modules can provide information to, and receive information from, other hardware modules. Accordingly, the described hardware modules may be regarded as being communicatively coupled. Where multiple hardware modules exist contemporaneously, communications may be achieved through signal transmission (e.g., over appropriate circuits and buses) between or among two or more of the hardware modules. In implementations in which multiple hardware modules are configured or instantiated at different times, communications between such hardware modules may be achieved, for example, through the storage and retrieval of information in memory structures to which the multiple hardware modules have access. For example, one hardware module may perform an operation and store the output of that operation in a memory device to which it is communicatively coupled. A further hardware module may then, at a later time, access the memory device to retrieve and process the stored output. Hardware modules may also initiate communications with input or output devices, and can operate on a resource (e.g., a collection of information).
- a resource e.g., a collection of information
- processors may be temporarily configured (e.g., by software) or permanently configured to perform the relevant operations. Whether temporarily or permanently configured, such processors may constitute processor-implemented modules that operate to perform one or more operations or functions described herein.
- processor-implemented module refers to a hardware module implemented using one or more processors.
- processor-implemented module refers to a hardware module in which the hardware includes one or more processors.
- processors may also operate to support performance of the relevant operations in a “cloud computing” environment or as a “software as a service” (SaaS).
- SaaS software as a service
- at least some of the operations may be performed by a group of computers (as examples of machines including processors), with these operations being accessible via a network (e.g., the Internet) and via one or more appropriate interfaces (e.g., an API).
- the performance of certain operations may be distributed among the one or more processors, not only residing within a single machine, but deployed across a number of machines.
- the one or more processors or processor-implemented modules may be located in a single geographic location (e.g., within a home environment, an office environment, or a server farm). In other example implementations, the one or more processors or processor-implemented modules may be distributed across a number of geographic locations.
- the term “if” may be construed to mean “when” or “upon” or “in response to determining” or “in response to detecting,” depending on the context.
- the phrase “if it is determined” or “if (a stated condition or event) is detected” may be construed to mean “upon determining” or “in response to determining” or “upon detecting (the stated condition or event)” or “in response to detecting (the stated condition or event),” depending on the context.
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Human Computer Interaction (AREA)
- Multimedia (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Optics & Photonics (AREA)
- User Interface Of Digital Computer (AREA)
- Information Transfer Between Computers (AREA)
Abstract
Description
- This application is a continuation of U.S. patent application Ser. No. 16/674,754, filed Nov. 5, 2019, which is a continuation of U.S. patent application Ser. No. 14/313,938, filed on Jun. 24, 2014, now U.S. Pat. No. 10,466,776, issued on Nov. 5, 2019; the disclosure of which are incorporated herein by reference in its entirety.
- This application relates generally to the field of computer technology and, in a specific example implementation, to a system and method for selecting and displaying additional content to a user.
- Electronic devices have revolutionized how media content is produced and consumed. Specifically, modern electronic devices have drastically simplified the process for creating, transferring, and consuming all kinds of data. This is possible because data can be stored in a digital form that is easy to create, edit, transfer, and present.
- Additionally, digital media can be dynamically altered to improve the user experience and allow advertisers to reach potential customers more effectively. For example, websites often include areas that can be filled with advertisements, such as banner ads, that can be added to the webpage at the time the webpage is requested. However, users often find banner advertisements annoying and not related to their interests.
- The present description is illustrated by way of example, and not by way of limitation, in the figures of the accompanying drawings in which:
-
FIG. 1 is a network diagram depicting a client-server system, within which one example implementation may be deployed. -
FIG. 2 is a block diagram illustrating content selection and user action analysis applications, in one example implementation. -
FIG. 3 illustrates an exemplary user interface of a client device presenting an electronic document (e-book). -
FIG. 4 illustrates an exemplary user interface of a client device presenting an electronic document (e-book). -
FIG. 5 illustrates an exemplary user interface of a client device presenting an electronic document (e-book). -
FIG. 6A illustrates an exemplary user interface of a display presenting an electronic document (e-magazine). -
FIG. 6B illustrates an exemplary user interface of a display presenting an electronic document (e-magazine). -
FIG. 7 is a flow diagram illustrating a process for surfacing related content based on user interactions with the currently presented content in accordance with some implementations. -
FIG. 8 is a flow diagram illustrating a process for identifying content viewed by the user by eye-tracking in accordance with some implementations. -
FIG. 9 is a flow diagram illustrating a process for determining user interest in a current page of content in accordance with some implementations. -
FIG. 10 is a flow diagram illustrating a process for selecting additional content to surface in accordance with some implementations. -
FIG. 11 is a flow diagram illustrating a process for surfacing additional content based on user interaction with currently presented content in accordance with some implementations. -
FIG. 12 is a flow diagram illustrating a process for surfacing additional content based on user interaction with currently presented content in accordance with some implementations. -
FIG. 13 is a flow diagram illustrating a process for surfacing additional content based on user interaction with currently presented content in accordance with some implementations. -
FIG. 14 is a flow diagram illustrating a process for surfacing additional content based on user interaction with currently presented content in accordance with some implementations. -
FIG. 15 is a block diagram illustrating amobile device 1500, according to an example implementation. -
FIG. 16 is a block diagram illustrating components of amachine 1600, according to some example implementations. - Like reference numerals refer to corresponding parts throughout the drawings.
- Although the implementations have been described with reference to specific example implementations, it will be evident that various modifications and changes may be made to these implementations without departing from the broader spirit and scope of the description. Accordingly, the specification and drawings are to be regarded in an illustrative rather than a restrictive sense.
- In various implementations, methods and systems for surfacing related content based on user interactions with the currently presented content are described. A media content item is presented on an electronic device. Media content items include, but are not limited to, textual content (e.g., an e-book, an electronic magazine, a website, etc.), video content (e.g., a movie, television show, animation, web clip, etc.), audio content (e.g., a music file or a podcast), an image, a video game, or any other media type. Media content items can be presented on an electronic device, such as a display device for visual media like images, text, or video. Audio media content items, such as music or podcasts, can be presented though speakers, earphones, and the like.
- When a media content item (e.g., a video, an audio clip, a digital book or e-book, or a video game or other animation) is presented at an electronic device, the user of the electronic device is able to take actions in relation to the presented media content item. In some implementations only a portion of a media content item is displayed at any given time (e.g., one page of a book or a frame of a movie). Some user actions include an interaction with a specific portion of the display section of the media content item (e.g., the user can click on a particular word in an e-book for a definition, right-click on an image to save it, or hover a cursor over a particular section or the rewind a video or audio presentation to repeat a particular section of a media content item.) Other user actions do not include direct user interaction with a specific portion of the presented media content item (e.g., a user looking at a portion of the display, turning up the volume during a particular section of a podcast, turning an e-book page or ceasing to walk while reading an article in a digital magazine.)
- In some implementations the electronic device detects the user action. For example, the electronic device includes a camera capable of tracking the positions of the user's eyes and then calculating the part of the display that the user is currently focused on. In other examples the electronic device detects direct input from a user, such as a mouse click input or a detected finger gesture on a touch screen. The electronic device can also measure motion of the device, through either an accelerometer or a global positioning system (GPS) device.
- Once user action has been detected, the electronic device determines whether the detected user action selects a particular part of the currently presented portion of the media content item. For example, a mouse click selects the particular part of the currently presented portion of the media content item associated with the location of the mouse click. In contrast, a page turn gesture causes the section of the media content item that is currently displayed to change, but does not select a particular part of the currently presented portion of the media a content item.
- Once the electronic device detects a user action and determines that is does not involve selection of a particular part of the currently presented section of the media content item, the electronic device analyses the user action to determine whether it is associated with a particular portion of the currently presented media item.
- In some implementations the electronic device transmits the detected user action to the server system for analysis. The server system then determines whether the user action is associated with a particular portion of the currently presented media content item. For example, the electronic device determines a gaze point for the user based on the user's eye positions and then transmits that gaze point to the server system. If the user is looking at a specific portion of the electronic display, the server system then determines that the user action (e.g., gaze point) is associated with a particular portion of the content media item (e.g., a particular section of text in a magazine or a portion of a screen during a movie).
- Once the electronic device determines that the user action is associated with a particular portion of the currently presented media content item, the electronic device then identifies additional content associated with the particular portion of the currently presented media content item. In some implementations the creator of the content has pre-designated specific additional content for specific sections of the content media item. For example, an author includes additional resources or further reading suggestions for particular sections of an e-book. In other examples, a displayed advertisement has additional content associated with it that includes further information about an advertised product or a link to buy the product.
- In other implementations there is no pre-designated additional content for a particular section of the currently presented media content item. In this case, the electronic device or the server system analyzes the content in the section of the media content item that is associated with the user action to determine one or more topics associated with the content. For example, the server system parses the text of an e-book to determine one or more associated topics based on the specific words included in the text. In other examples, the server system uses an image recognition algorithm to identify a particular image of interest in a section of an image or video. Once the image is identified, the server system uses a look-up table or other method to identify topics associated with the identified image. For example, if the server system determines that a particular image or section of an image is a wine glass, the identified topics include wine, home furnishings, and glasses. In other implementations the video or image has associated metadata that identifies topics associated with an image or section of a video.
- In some implementations the electronic device then uses the identified topics to select additional content to present to the user. Additional content includes, but is not limited to, advertisements, additional related content (e.g., making of documentaries, commentary, behind the scenes photos), supplemental content, additional learning resources, offers to sell, auction listings, and so forth. For example, if the determined topics are “wine,” “glasses,” and “home furnishings,” then the additional content includes one or more advertisements for wines, information on local wineries, and an advertisement for a current home furnishing auction listing.
- The electronic device then selects one or more of additional content items and transmits it to the electronic device for presentation alongside the currently presented media content item. In some implementations the additional content is presented concurrently with the currently presented media content item.
- In some implementations the above implementations can be performed remotely by a server system (e.g., streams media data items and receives user action data from the electronic device over a network). Indeed, the various steps described above can be split between a server system and an electronic device in any configuration that is useful.
- In some implementations the amount of potential additional media content is too great to store on a single electronic device. Instead, the electronic device is able to access a third party web service (e.g., a server system available over a network that provides a service). The electronic device first determines one or more topics (or keywords) that are associated with the user action. Then, the electronic device transmits the determined topics (or keywords) to the third party service provider. In some implementations the topics are first translated into a search query and then transferred to the third party server system.
- The third party server system receives the topics from the electronic device. In some implementations the third party server receives additional metadata about the request. For example, the request includes a number of additional content media items needed, the type of media, the length of the media (in the case of video or audio), the size of the media (for text or images), the source of the media, and any other relevant metadata.
- The third party server system then uses the received topics, keywords, or search query to identify relevant media content items. In some implementations the third party server system then ranks the identified media content items based on relevance and any associated metadata and sends back the requested number of additional media content items to the electronic device. In other implementations the server system filters out any media that does not match the received metadata and sends all identified media content items (or links to all identified media content items) and the electronic device ranks them and selects the best matches. In some implementations the third party server system clips text or video to an acceptable length before transmitting them to the client system.
-
FIG. 1 is a network diagram depicting a client-server system 100, within which one example implementation may be deployed. Anetworked system 102, in the example forms of a network-based marketplace or publication system, provides server-side functionality, via a network 104 (e.g., the Internet or a wide area network (WAN)), to one or more clients.FIG. 1 illustrates, for example, a web client 106 (e.g., a browser, such as the Internet Explorer® browser developed by Microsoft® Corporation of Redmond, Wash.) and aprogrammatic client 108 executing onrespective devices - An Application Program Interface (API)
server 114 and aweb server 116 are coupled to, and provide programmatic and web interfaces respectively to, one ormore application servers 118. Theapplication servers 118 host one or morecontent selection applications 120 and useraction analysis applications 122. Theapplication servers 118 are, in turn, shown to be coupled to one or more database servers that facilitate access to one ormore databases 126. - The user
action analysis applications 122 receive one or more user actions from one ormore client devices 110. The useraction analysis applications 122 then analyze the received user action to determine whether it pertains to specific portions of a media content item. If so, the user action analysis applications determine which one or more specific portions of the media content item relate to the user action. The useraction analysis applications 122 then transmit the determined one or more specific portions to thecontent selection applications 120. - The
content selection applications 120 use the received one or more specific portions to select additional content to be provided to aclient device 110 based on analysis performed by the useraction analysis applications 122. While the content selection anduser action analysis FIG. 1 to both form part of thenetworked system 102, it will be appreciated that other configurations can be used such that thecontent selection applications 120 and the useraction selection applications 122 may each form part of a service that is separate and distinct from thenetworked system 102. - Further, while the
system 100 shown inFIG. 1 employs a client-server architecture, the implementations are, of course, not limited to such an architecture, and could equally well find application in a distributed, or peer-to-peer, architecture system, for example. The various content selection and useraction analysis applications - The
web client 106 accesses the various content selection and useraction analysis applications web server 116. Similarly, theprogrammatic client 108 accesses the various services and functions provided by content selection and useraction analysis applications API server 114. Theprogrammatic client 108 may, for example, be a seller application (e.g., the Turbo Lister application developed by eBay Inc., of San Jose, Calif.) to enable sellers to author and manage listings on thenetworked system 102 in an offline manner, and to perform batch-mode communications between theprogrammatic client 108 and thenetworked system 102. -
FIG. 1 also illustrates athird party application 128, executing on a thirdparty server machine 130, as having programmatic access to thenetworked system 102 via the programmatic interface provided by theAPI server 114. For example, thethird party application 128 may, utilizing information retrieved from thenetworked system 102, support one or more features or functions on a website hosted by the third party. The third party website may, for example, provide one or more promotional, marketplace, or payment functions that are supported by the relevant applications of thenetworked system 102. - The
applications applications applications applications applications more databases 126 via thedatabase servers 124. -
FIG. 2 is a block diagram illustrating content selection and useraction analysis applications networked system 102. Thenetworked system 102 provides services that select and transmit for presentation additional content related to media content already being presented at the electronic device of a user. To this end, the content selection application(s) 120, in some implementations, includes several modules or sub-applications to perform its role. As discussed above, acontent selection application 120 uses one or more topics of interest to select one or more additional content items. In some implementations the one or more topics of interest are based on a specific piece of currently displayed content (e.g., the text of a document currently displayed to a user). In other implementations thecontent selection application 120 also uses information about the user, such as age, gender, location, interests, past purchasing activity, and past browsing/media consumption activity, to customize and personalize the selected additional content. In some implementations this user information is submitted directly from the user. In other implementations the information is gathered or inferred by the networked server system (e.g., thenetworked server system 102 inFIG. 1 ) and stored in a user profile. In some implementations the server system has a privacy policy that prevents it from gathering or storing any personal information for a user without the user's express permission. - In some implementations the content selection application(s) 120 includes a
content display application 202. Thecontent display application 202 transmits media content items for display at the client system. In some implementations the network system stores a plurality of media content items and transmits them to clients for presentation as requested. For example, the networked system hosts a plurality of e-books and transmits or streams to the users based on the user's requests. -
Metadata analysis applications 204 support the content selection applications 200 by analyzing content and determining matching topics based on metadata associated with the media content. For example, a movie has associated metadata (e.g., genre information, actor information, director information, length, country of origin, a plot synopsis, etc.). Themetadata analysis applications 204 use this information to identify one or more topics associated with the content item. - In some implementations
image recognition applications 206 support thecontent selection applications 120 by analyzing images or a section of an image to identify objects within the images. Any suitable image detection algorithm can be used to accomplish this task. In some implementations, once theimage recognition applications 206 identify one or more objects within the image, it uses a lookup table or other suitable mechanism to identify topics associated with the identified object or objects. For example, if the identified object is a classic car, the identified topics include “Car Shows,” “Classic Cars,” and “Car Maintenance and Repairs.” -
Text analysis applications 208 support the content selection applications 200 by analyzing text content associated with a particular portion of a text based media content item. Thetext analysis application 208 parses the relevant text and, using appropriate text parsing algorithms, identifies one or more topics associated with the text content. This identification may be accomplished by identifying word frequencies, key words, and patterns within the text. The identified topics are then sent to thecontent selection applications 120 to atopic matching application 210 to assist in selecting additional content. For example, if a given portion of text included the text “20-16,” “Jerry Rice,” and “Jan. 22, 1989,” thetext analysis application 208 would determine that the portion of text was discussing the 1989 SuperBowl and identify “Professional American Football,” “San Francisco 49ers,” and “the 80s” as topics of interest. - The
topic matching application 210 supports thecontent selection applications 120 by receiving one or more identified topics of interest from another module or application and using it to identify additional content items that have similar associated topics. For example, if the determined topics are “world history,” “books,” and “best-sellers,” the topic matching applications matches those topics with the book “Guns, Germs, and Steel,” which also includes those topics. Thetopic matching applications 210 then notify thecontent selection applications 120 of the matching additional content. - In some implementations the user
action detection applications 122 receive a notification from a client device (e.g.,client device 102 inFIG. 1 ) that a user action has occurred. The useraction detection application 122 then determines whether the user action is associated with a specific section of the currently presented media content. In accordance with a determination that the user action is associated with a specific portion of the media content item, the user action detection application then determines the specific portion of the media content item that is associated with the user action. - A
touchscreen input application 214 receives and interprets touch screen input from a client device. For example, if the user makes a swipe gesture on the touch screen of a device currently displaying an electronic book, the touchscreen input application 214 determines what specific portion of the electronic book is associated with the swiping motion (e.g., the page that is displayed after the swipe is complete). - The reading
speed detection application 216 calculates the approximate reading speed of a user. The reading speed can be calculated by determining the number of words on a page and how long it takes the user to turn to the next page. As more and more pages are read, the reading speed of the user is approximated more accurately. Then, when the user turns to a new page in an electronic document, the user action analysis application 212 can estimate the user's current reading position on the page by estimating the number of words the user has read since the page was turned. For example, if the readingspeed detection application 216 determines that the user averages about 120 words read per minute and the user has been reading a page with about 500 words for two minutes, the reading speed detection application can then estimate that the reader is about half way through the text (e.g., about 250 words down the page). The user action analysis application 212 can then identify the nearby text as the specific portion of media content currently of interest to the user. - The
eye tracking application 218 uses a camera or infrared tracking device to measure the movement of a user's eyes. With this information, and information regarding the position of a display associated with the user of the client device, theeye tracking application 218 can estimate the particular portion of a display that the user is current focusing on. Theeye tracking application 218 then identifies this currently viewed portion of the display as the portion of the media content item currently of interest to the user. - In some implementations device
movement tracking applications 220 determine whether the client device is moving and in what direction. This can be tracked by way of a GPS tracking device or an accelerometer device internal to the client device. When the movement of the device changes (e.g., starts moving, ceases moving, or changes movement speeds), the devicemovement tracking application 220 records a user action. In some implementations the devicemovement tracking application 220 determines that the change in movement is the result of user interest in the presented media content (e.g., the user stops walking to focus on a particular scene). In this case, the devicemovement tracking applications 220 identify the currently displayed portion of the media content item as of interest to the user. -
Interest determination applications 222 use the received user actions to determine whether the user action is indicative of user interest in a particular portion of the presented media content. For example, the client device reports a constant rate of movement for 25 minutes and then a deceleration to a stop. Based on the accompanying GPS data, theinterest determination application 222 determines that the detected action is a result of a train ride that the user is on coming to an end, not increased user interest in the current content. Conversely, if the user inputs instructions to zoom in on a particular section of an image, theinterest determination application 222 determines that the user is interested in the zoomed in portion of the image. - In another example, a user increases the volume of current presented audio or video content. The
interest determination application 222 determines whether there has been a corresponding increase in environmental noise using a microphone or other audio sensing device. If there has been an increase in environmental noise, theinterest determination application 222 determines that the increase in volume level is the result of the user reacting to the increased background noise, not a result of increased interest in the currently present content. However, if theinterest determination application 222 determines there has been no recent increase in background noise, then the user increasing the volume may be a result of increased user interest in the currently presented content. - In some implementations the
networked system 102 includes one or morenetwork commerce applications 224 for enabling commercial transactions over thenetworked system 102. -
Listing management applications 226 allow sellers to create and manage listings. Specifically, where a particular seller has authored and/or published a large number of listings, the management of such listings may present a challenge. Thelisting management applications 226 provide a number of features (e.g., auto-relisting, inventory level monitors, etc.) to assist the seller in managing such listings.\ - One or more post-listing management applications may also assist sellers with a number of activities that typically occur post-listing. For example, upon completion of an auction facilitated by one or
more auction applications 228, a seller may wish to leave feedback regarding a particular buyer. To this end, a post-listing management application may provide an interface to one ormore reputation applications 230, so as to allow the seller to conveniently provide feedback regarding multiple buyers toreputation applications 230. -
Reputation applications 230 allow users who transact, utilizing thenetworked system 102, to establish, build, and maintain reputations, which may be made available and published to potential trading partners. Consider that where, for example, thenetworked system 102 supports person-to-person trading, users may otherwise have no history or other reference information whereby the trustworthiness and credibility of potential trading partners may be assessed. Thereputation applications 230 allow a user (for example, through feedback provided by other transaction partners) to establish a reputation within thenetworked system 102 over time. Other potential trading partners may then reference such a reputation for the purposes of assessing credibility and trustworthiness. - A number of
fraud prevention applications 232 implement fraud detection and prevention mechanisms to reduce the occurrence of fraud within thenetworked system 102. -
Messaging applications 234 are responsible for the generation and delivery of messages to users of the networked system 102 (such as, for example, messages advising users regarding the status of listings at the networked system 102 (e.g., providing “outbid” notices to bidders during an auction process or providing promotional and merchandising information to users)).Respective messaging applications 234 may utilize any one of a number of message delivery networks and platforms to deliver messages to users. For example,messaging applications 228 may deliver electronic mail (e-mail), instant message (IM), Short Message Service (SMS), text, facsimile, or voice (e.g., Voice over IP (VoIP)) messages via the wired (e.g., the Internet), plain old telephone service (POTS), or wireless (e.g., mobile, cellular, Wi-Fi, WiMAX) networks 104. - It is important to note that although
FIG. 2 showscontent selection applications 120, useraction analysis applications 122, and network commerce applications, some implementations do not include all these application. For example, an e-book reader can include both thecontent selection applications 120 and the useraction analysis applications 122 but not include any network commerce applications. -
FIG. 3 illustrates anexemplary user interface 302 of aclient device 300 presenting an electronic document (e-book). Theelectronic device 300 includes adisplay 302. In the display is digital text 306 (in this case an article about polar bears). The user interface also displays auser interaction object 304 that represents user input at a particular part of the display from either a mouse input or a finger gesture on a particular section of a touch screen. In this case the user input initiates changing from a first page (e.g., page 1) to a second page (e.g., page 2). -
FIG. 4 illustrates anexemplary user interface 400 of aclient device 300 presenting an electronic document (e-book). Theelectronic device 300 includes adisplay 302. In the display ispage 2 of the digital text 402 (in this case an article about polar bears) shown inFIG. 3 after the page turning input received previously. -
FIG. 5 illustrates anexemplary user interface 500 of aclient device 300 presenting an electronic document (e-book). Theelectronic device 300 includes adisplay 302. In the display ispage 2 of the digital text 402 (in this case an article about Polar bears) as shown inFIG. 4 andadditional content 502 displayed concurrently with the original content. In this case the additional content takes the form of a link to additional information about earlyarctic explorers 502. In some implementations the additional content is displayed immediately. In other implementations the additional content is displayed after a fixed amount of time (e.g., 5 seconds after the page turns). In yet other implementations, the additional content is displayed based upon the user's reading speed (e.g., displayed when the device estimates the user has just read the related content.) -
FIG. 6A illustrates anexemplary user interface 600 of a display presenting an electronic document (e-magazine). The display includes a section containingarticle content 602 and three sections containing advertisements (WorldWide Airways Advertisement 604, CunninghamAutomobile Corporation Advertisement 606, and Rondo Soda Advertisement 608). The display has an associated camera that tracks the eye movements of the user. In this case the left eye is pointed to a lefteye focus point 610 on the display, and the right eye is pointed to a righteye focus point 612 on the display. Based on the two focus points, the device calculates agaze area 614 in the area assigned to the CunninghamAutomobile Corporation advertisement 606. The network system can use this information to determine that user is currently focused on the CunninghamAutomobile Corporation advertisement 606. -
FIG. 6B illustrates anexemplary user interface 600 of a display presenting an electronic document (e-magazine). The display includesarticle content additional content 620 that was displayed in response to the electronic device associated with the display determining that the user is interested in further content related to the CunninghamAutomobile Corporation Advertisement 606 based on the detected position of the user's eyes. Theadditional content 620 is displayed partially overlapping and simultaneous with the already displayed content. Theadditional content 620 includes information related to the CunninghamAutomobile Corporation advertisement 606, including three buttons (622, 624, 626) that offer additional information about specific automobiles sold by the company associated with the Cunningham Automobile Corporation advertisement 606 (presumably the Cunningham Automobile Corporation). -
FIGS. 7-14 are flow diagrams that represent a method for surfacing related content based on user interactions with the currently presented content. It should be noted that while each Figure is described as being performed by either an electronic device or by a server system communicating with that electronic device, the steps in each of the Figures may be performed either by the electronic device or by a server system communicating with that device. As such, the methods described below can be rearranged so that all the steps are performed at a single electronic device, all the steps are performed at a server system in communication with that device, or the steps are divided up between the two devices (server device and a client device) pieces in any way. -
FIG. 7 is a flow diagram illustrating a process for surfacing related content based on user interactions with the currently presented content in accordance with some implementations. Each of the operations shown inFIG. 7 may correspond to instructions stored in a computer memory or computer readable storage medium. Optional operations are indicated by dashed lines (e.g., boxes with dashed-line borders). In some implementations, the method described inFIG. 7 is performed by the electronic device (e.g.,electronic device 110 inFIG. 1 ). - In some implementations the method is performed at a computer system including one or more processors and memory storing one or more programs for execution by the one or more processors. The electronic device (e.g.,
electronic device 110 inFIG. 1 ) presents (702) a media content item. The media content item is either displayed on a screen (in the case of a visual media content item such as a movie, webpage, e-book, etc.) or played over a speaker or headphones (in the case of an audio media content item such as music, podcasts, audiobooks, etc.). - The electronic device (e.g.,
electronic device 110 inFIG. 1 ) then identifies auser action 704. An identified user action can include any user action that the electronic device (e.g.,electronic device 110 inFIG. 1 ) is capable of detecting including, but not limited to, input received from the user from a keyboard, mouse, microphone, touchscreen, camera, or any other input device; a physical movement of the user; position of the user's body, including his or her eyes; control commands from the user (e.g., pausing, zooming in, turning up the volume); and sounds made by the user (detecting and analyzing user speech). Some user actions include specifically selecting a part of the media content item currently on the display. For example, clicking with a mouse, touching an icon with a figure gesture, hovering a pointer, and similar user actions all involve selecting a particular part of the content currently displayed. Other user actions do not involve selecting a particular part of the content that is currently displayed but may still be associated with a particular part. For example, a swipe gesture on an e-reader will result in all of the currently displayed content being removed and a new section of the media content item (in this case an e-book) being displayed. The finger swipe does not select any portion of the previous page or the new page, but is associated with both. - Thus the electronic device (e.g.,
electronic device 110 inFIG. 1 ) analyses the user action to identify (706) a portion of the media content item associated with the user action. For example, if the user action is a mouse click on a displayed link, the electronic device (e.g.,electronic device 110 inFIG. 1 ) identifies the link (and the content it links to) as being associated with the user action. If the user action is an eye movement to change from one gaze point to a second gaze point, the electronic device (e.g.,electronic device 110 in Figure 1) identifies the content displayed in the part of the screen that the user is currently focused on as being associated with the changing gaze point. - In some implementations if the user action involves direct selection of a part of the currently displayed media content item, the electronic device (e.g.,
electronic device 110 inFIG. 1 ) does nothing. If the user action is merely associated with a portion of the currently presented media content item, the electronic device (e.g.,electronic device 110 inFIG. 1 ) selects (708) one or more additional content items. These additional content items are selected based on the identified content. For example, if the identified content is a picture of a whale, the additional content may be an article about whales. - The electronic device (e.g.,
electronic device 110 inFIG. 1 ) then presents (710) the one or more additional selected content items at the electronic device. In some implementations the additional content is displayed simultaneously with the originally displayed content, either overlapping or partially displacing some of the content. -
FIG. 8 is a flow diagram illustrating a process for identifying content viewed by the user by eye-tracking in accordance with some implementations. Each of the operations shown inFIG. 8 may correspond to instructions stored in a computer memory or computer readable storage medium. Optional operations are indicated by dashed lines (e.g., boxes with dashed-line borders). In some implementations, the method described inFIG. 8 is performed by the networked server system (e.g., the networked server system inFIG. 1 ). - In some implementations the method is performed at a computer system including one or more processors and memory storing one or more programs for execution by the one or more processors. The electronic device (e.g.,
electronic device 110 inFIG. 1 ) uses eye tracking technology to determine what content the user is currently focused on. First, the electronic device (e.g.,electronic device 110 inFIG. 1 ) has a camera or other means of determining eye position. - The electronic device (e.g.,
electronic device 110 inFIG. 1 ) then determines (802) the left gaze point. Determining a gaze point includes using the position of an eye to determine a line of sight for that eye. Then the device determines at what point that line of sight intercepts a particular surface (in this case a display). The point where the line of sight crosses through the display is a gaze point. Each eye has its own specific gaze point. - The electronic device (e.g.,
electronic device 110 inFIG. 1 ) then determines (804) the right gaze point. Using the right and left gaze points, the electronic device (e.g.,electronic device 110 inFIG. 1 ) determines (806) the user focus area. The user focus area is where the user is current focusing. This is determined by measuring the distance between the two gaze points. Gaze points that are relatively far apart indicate that the user focusing on something behind the display or just gazing off into space. Gaze points that are too close together or where the right gaze point is to the left of the left gaze point indicate that the user is focusing on something in front of the display. Thus, by determining the focus area, the electronic device (e.g.,electronic device 110 inFIG. 1 ) determines (808) whether the user has a focus area that is currently on the display. If not, the electronic device (e.g.,electronic device 110 inFIG. 1 ) continues to monitor user actions (810). - When the electronic device (e.g.,
electronic device 110 inFIG. 1 ) determines that the user focus area is on the display screen, the electronic device (e.g.,electronic device 110 inFIG. 1 ) then determines (812) the portion of the display associated with the user focus area. For example, the electronic device (e.g.,electronic device 110 inFIG. 1 ) determines whether the focus area is at the top of the screen, the bottom of the screen, or the left or right side of the screen. Depending on the precision of the camera that detects eye position, the electronic device (e.g.,electronic device 110 inFIG. 1 ) can narrow down the user focus area to a very specific portion of the screen (e.g., see 614 ofFIG. 6A ). - The electronic device (e.g.,
electronic device 110 inFIG. 1 ) then identifies (814) the particular content displayed at the part of the display that the user is current focusing on. For example, if the user focus area is above text, the electronic device (e.g.,electronic device 110 inFIG. 1 ) determines the content of that text. If the user focus area is on an image or video, the electronic device (e.g.,electronic device 110 inFIG. 1 ) analyzes the image or video to determine what is being displayed. -
FIG. 9 is a flow diagram illustrating a process for determining user interest in a current page of content in accordance with some implementations. Each of the operations shown inFIG. 9 may correspond to instructions stored in a computer memory or computer readable storage medium. Optional operations are indicated by dashed lines (e.g., boxes with dashed-line borders). In some implementations, the method described inFIG. 9 is performed by the electronic device (e.g.,electronic device 110 inFIG. 1 ). - In some implementations the method is performed at a computer system including one or more processors and memory storing one or more programs for execution by the one or more processors. In some implementations the electronic device (e.g.,
electronic device 110 inFIG. 1 ) is an e-reader or other device capable of displaying textual media content. For example, an e-reader can display e-books. The electronic device (e.g.,electronic device 110 inFIG. 1 ) can estimate a user's interest on a particular page by measuring the amount of time a user spends on that page relative to the amount of time the electronic device (e.g.,electronic device 110 inFIG. 1 ) predicts the user will spend on the page. - The electronic device (e.g.,
electronic device 110 inFIG. 1 ) first detects (902) a page change. A page change occurs whenever the user replaces previous content on the display with new content. For example, a user swipes on a touch screen that is currently displaying an e-book. The next page of the e-book is displayed in response. Once the next page is displayed, the electronic device (e.g.,electronic device 110 inFIG. 1 ) begins (904) a page timer. A page timer tracks the amount of time that the user has been on a specific page. In some implementations page timers are used to build a reading speed estimate for a user. For example, by keeping track of the amount of time spent on each page for the last 50 pages and the number of words on each page, the electronic device (e.g.,electronic device 110 inFIG. 1 ) can generate an average reading speed.(for example, 2500 words/30 minutes=83.3 words per minutes). - Using an estimate average reading speed, the electronic device (e.g.,
electronic device 110 inFIG. 1 ) can then generate an estimate reading time for the new page. For example, if the reading speed is 83.3 words per minute and the new page has 250 words, the electronic device (e.g.,electronic device 110 inFIG. 1 ) can estimate that the user will take about 3 minutes to finish the page. The electronic device (e.g.,electronic device 110 inFIG. 1 ) can then use this value as the estimated time for the user to finish the page. In some implementations the electronic device (e.g.,electronic device 110 inFIG. 1 ) also adds a buffer amount to the estimate time (e.g., 20% of the estimated time) to account for minor variations in reading speed due to difficulty of content and various distractions that are unrelated to the content. - The electronic device (e.g.,
electronic device 110 inFIG. 1 ) then compares (906) the current page timer to the estimated time to determine if the page time exceeds (908) the estimated time. If not, the electronic device (e.g.,electronic device 110 inFIG. 1 ) determines (910) whether a new page change command has been received. If so, it begins the process again. If not, it continues to compare (906) the page time to the estimated time. - If the page time has exceeded the estimated time, the electronic device (e.g.,
electronic device 110 inFIG. 1 ) determines (912) that the user has increased interested in the current page and its contents. -
FIG. 10 is a flow diagram illustrating a process for selecting additional content to surface in accordance with some implementations. Each of the operations shown inFIG. 10 may correspond to instructions stored in a computer memory or computer readable storage medium. Optional operations are indicated by dashed lines (e.g., boxes with dashed-line borders). In some implementations, the method described inFIG. 10 is performed by the electronic device (e.g.,electronic device 110 inFIG. 1 ). - In some implementations the method is performed at a computer system including one or more processors and memory storing one or more programs for execution by the one or more processors. The electronic device (e.g.,
electronic device 110 inFIG. 1 ) first identifies (1002) a user action that is associated with a portion of a presented media content item. As described in detail above, the user action can be anything the user does that the electronic device (e.g.,electronic device 110 inFIG. 1 ) can detect. For example, a user action includes the physical movement of the user (provided the device has a camera, accelerometer, or other means for tracking movement) and interactions the user has with the electronic device (e.g.,electronic device 110 inFIG. 1 ) such as mouse inputs, touch inputs, or any other direct input into the system. - Once the user action is detected and the associated content is identified, the electronic device (e.g.,
electronic device 110 inFIG. 1 ) then determines (1004) whether there is already preselected additional content available. In some implementations the preselected additional content is chosen by the author or creator of the content to enhance the user's experience with the content. In other implementations the preselected additional content is a form of advertisement inserted into the media content item as a result of a business transaction between the creator and the advertiser. - In accordance with a determination that preselected additional content is available, the electronic device (e.g.,
electronic device 110 inFIG. 1 ) then presents (1006) the preselected additional content. In accordance with a determination that the preselected additional content is not available, the electronic device (e.g.,electronic device 110 inFIG. 1 ) then analyzes (1008) the portion of the media content item identified as associated with the user action to determine one or more topics of interest based on the content of the analyzed portion. - The electronic device (e.g.,
electronic device 110 inFIG. 1 ) then uses the topics of interest to identify (1010) one or more additional content items that match the determined topics of interest. For example, if the portion of the media content item is an essay about lettuce, the identified topics of interest include “lettuce,” “agriculture,” “cooking,” and “Spanish Exports.” The electronic device (e.g.,electronic device 110 inFIG. 1 ) then matches those topics of interest against the topics associated with potential additional content items. As a result of the matching, the electronic device (e.g.,electronic device 110 inFigure 1 ) identifies two additional content items, a book called “The History of Lettuce” and a recipe for “Wilted Lettuce Salad.” - The electronic device (e.g.,
electronic device 110 inFIG. 1 ) then ranks (1012) the one or more additional content items based on suitability for the user. Continuing with the example above, the electronic device (e.g.,electronic device 110 inFIG. 1 ) determines, based on a user profile associated with the user, that the user is interested in cooking but not history. Thus, the electronic device (e.g.,electronic device 110 inFIG. 1 ) ranks the recipe for “Wilted Lettuce Salad” above the book entitled “The History of Lettuce.” - The electronic device (e.g.,
electronic device 110 inFIG. 1 ) selects (1014) one or more additional content items for presentation based on their respective ranking. In some implementations the electronic device (e.g.,electronic device 110 inFIG. 1 ) determines how many additional content items to display. This determination is based on what media content is currently displayed (e.g., if a video is playing the number of additional content items is very low, perhaps only 1), the space available on the display, and the preferences of the user. Once the number of additional content items to display is determined, the electronic device (e.g.,electronic device 110 inFIG. 1 ) chooses the highest ranked additional content items to fulfil the needed number of additional content items. -
FIG. 11 is a flow diagram illustrating a process for surfacing additional content based on user interaction with currently presented content in accordance with some implementations. Each of the operations shown inFIG. 11 may correspond to instructions stored in a computer memory or computer readable storage medium. Optional operations are indicated by dashed lines (e.g., boxes with dashed-line borders). In some implementations, the method described inFIG. 11 is performed by the electronic device (e.g.,electronic device 110 inFIG. 1 ). - In some implementations the method is performed at a computer system including one or more processors and memory storing one or more programs for execution by the one or more processors. The electronic device (e.g.,
electronic device 110 inFIG. 1 ) presents (1102) a media content item on an electronic display. As noted above, media content items include, but are not limited to, textual content (e.g., an e-book, an electronic magazine, a website, etc.), video content (e.g., a movie, TV show, animation, web clip, etc.), audio content (e.g., a music file or a podcast), images, video games, or any other media content. Media content items can be presented on an electronic device, such as a display device for visual media like images, text, or video. Audio media content items, such as music or podcasts, can be presented though speakers, earphones, or the like. - The electronic device (e.g.,
electronic device 110 inFIG. 1 ) detects (1103) user action with a respective portion of the media content item, wherein the user action does not include explicit selection of the respective portion of the media content item. For example, a user action that includes a direct click on a link includes explicit selection of the respective portion of the media content item, while a user action that includes turning up the volume does not explicitly select a respective portion of the media content item. - In some implementations detecting user action includes detecting (1104) user interactions with an electronic device associated with the electronic display. For example, any user action that involves the user intentionally using an input device, such as a mouse, keyboard, touch screen, microphone, camera, or other input device to interact with the electronic device (e.g.,
electronic device 110 inFIG. 1 ). Once a user action is detected, the electronic device (e.g.,electronic device 110 inFIG. 1 ) analyzes (1106) the detected user interactions to determine whether the detected user interaction are associated with the presented media content item. For example, if the user uses a mouse to hover a cursor over a particular image on a web page, the electronic device (e.g.,electronic device 110 inFIG. 1 ) determines (1106) that the user action (mouse hovering) is associated with the particular image on the web page. - In some implementations the electronic device (e.g.,
electronic device 110 inFIG. 1 ) measures (1108) physical movements of a user. For example, the electronic device (e.g.,electronic device 110 inFIG. 1 ) measures the eye movements of a user to track where the user is looking. A user's movements can be considered a user action. The electronic device (e.g.,electronic device 110 inFIG. 1 ) then analyzes (1110) the measured physical movements of the user to determine whether the measured physical movements are associated with the presented media content item. - It should be noted that user actions include but are not limited to changing the currently displayed page, pausing a video at a particular part, rewinding to review or re-listen to a particular section, eye movement, changes in movement of a user, voice commands received by a user, volume changes by a user, and the like.
- In some implementations, in response to detection of the user action, the electronic device (e.g.,
electronic device 110 inFIG. 1 ) identifies (1112) additional content to present based on the content included in the portion of the respective portion of the media content item. In some implementations identifying additional content includes determining (1114) whether any additional content has been preselected by the creator of the media content item to be associated with the respective portion of the media content item. For example, an e-book is created by an author. In addition to the text of the book, the author compiles a list of additional media resources, such as additional books, recommendations, clips, scientific papers, web pages, and so forth. The author then chooses which specific section of the e-book each additional media resources is associated with. Then if the user demonstrates interest in a specific portion through a user action, the previously associated additional content item can be displayed. -
FIG. 12 is a flow diagram illustrating a process for surfacing additional content based on user interaction with currently presented content in accordance with some implementations. Each of the operations shown inFIG. 12 may correspond to instructions stored in a computer memory or computer readable storage medium. Optional operations are indicated by dashed lines (e.g., boxes with dashed-line borders). In some implementations, the method described inFIG. 12 is performed by the electronic device (e.g.,electronic device 110 inFIG. 1 ). - In some implementations the method is performed at a computer system including one or more processors and memory storing one or more programs for execution by the one or more processors. In accordance with a determination that additional content has been preselected by the creator of the media content item to be associated with the respective portion of the media content item (1112), the electronic device (e.g.,
electronic device 110 inFIG. 1 ) determines (1204) the preselected additional content as the identified additional content. For example, a particular scene in a movie contains a tablet computer, and the electronic device determines that the user is looking at the tablet. The electronic device (e.g.,electronic device 110 inFIG. 1 ) then determines that the creator of the movie has preselected an advertisement for the tablet and associated it with the scene in the movie where the tablet appears. The electronic device (e.g.,electronic device 110 inFIG. 1 ) then selects the tablet advertisement as the additional content. - In some implementations the electronic device (e.g.,
electronic device 110 inFIG. 1 ) analyzes (1206) the respective portion of the media content item to determine one or more associated topics. For example, if the respective portion is text, the electronic device (e.g.,electronic device 110 inFIG. 1 ) parses the text to identify keywords and then identifies specific topic using the keywords. Once the one or more topics have been determined, the electronic device (e.g.,electronic device 110 inFIG. 1 ) identifies the particular portion of the currently presented media content item (1208) one or more additional content items that match the determined one or more associated topics. - In some implementations the electronic device (e.g.,
electronic device 110 inFIG. 1 ) then ranks (1210) the one or more determined additional content items. Ranking includes the electronic device (e.g.,electronic device 110 inFIG. 1 ) generating (1212) a match score for each of the one or more determined additional content items. For example, the electronic device (e.g.,electronic device 110 inFIG. 1 ) has a list of topics or keywords that were determined by analyzing a portion of the presented media content item. Similarly each media content item includes a list of associated topics or keywords. Then the electronic device (FIG. 1, 110 ) generates a match score based on the number of keywords or topics that each additional media content item has in common with the list of associated topics or keywords, such that additional media content items that have more topics or keywords in common have a higher match score. In some implementations each associated topic has an associated relevance score that is used in calculating a match score. In some implementations a match score is based, at least in part, on demographic information of the user. In some implementations a match score is based on user preference data for the user. - In some implementations the electronic device (e.g.,
electronic device 110 inFIG. 1 ) orders (1214) the one or more determined additional content items based on their associated match score. For example, the additional content items with the highest match score are listed first. -
FIG. 13 is a flow diagram illustrating a process for surfacing additional content based on user interaction with currently presented content in accordance with some implementations. Each of the operations shown inFIG. 13 may correspond to instructions stored in a computer memory or computer readable storage medium. Optional operations are indicated by dashed lines (e.g., boxes with dashed-line borders). In some implementations, the method described inFIG. 13 is performed by the electronic device (e.g.,electronic device 110 inFIG. 1 ). - In some implementations the method is performed at an electronic device including one or more processors and memory storing one or more programs for execution by the one or more processors. The electronic device (e.g.,
electronic device 110 inFIG. 1 ) selects (1302) one or more additional content items based on the rankings of the additional content items. For example, the electronic device (e.g.,electronic device 110 inFIG. 1 ) selects the highest ranked additional content item. In another example, the electronic device (e.g.,electronic device 110 inFIG. 1 ) determines a specific number of needed additional content items and then selects that number of additional content items. - In some implementations the electronic device (e.g.,
electronic device 110 inFIG. 1 ) simultaneously present(s) the additional content and the media content item on the electronic display. This can be accomplished by presenting both the additional content and the media content item simultaneously, such that both can be at least partially seen. For example, the additional content is displayed in a pop-up such that it partially obscures (1306) the original content item. In another example, the original content time is shifted such that space is created for the additional content item by moving part of the original content item off screen. In other examples, the display originally includes room for additional content, and the original content item does not need to be displaced or partially covered up. -
FIG. 14 is a flow diagram illustrating a process for surfacing additional content based on user interaction with currently presented content in accordance with some implementations. Each of the operations shown inFIG. 14 may correspond to instructions stored in a computer memory or computer readable storage medium. Optional operations are indicated by dashed lines (e.g., boxes with dashed-line borders). In some implementations, the method described inFIG. 14 is performed by the server system (e.g., networked seversystem 102 onFIG. 1 ). - In some implementations the method is performed at a server system including one or more processors and memory storing one or more programs for execution by the one or more processors. The server system performs the steps of the method, transmits the data to a client system for presentation, and then receives communications from the client system to notify the server system of user actions.
- In some implementations, the server system transmits (1402) a media content item for presentation at a client system. For example, the server system streams video content to the client system. In response, the server system receives (1404) notification of a user action associated with a portion of the presented media content item from the client system. For example, when the user changes the page of an e-book, the client system sends a notification to the server system.
- In response to receiving notification of the user action, the server system identifies (1406) additional content to present based on the respective portion of the media content item. The server system then transmits (1408) the identified additional content to the client system for simultaneous presentation along with the original media content item.
-
FIG. 15 is a block diagram illustrating amobile device 1500, according to an example implementation. Themobile device 1500 may include aprocessor 1502. Theprocessor 1502 may be any of a variety of different types of commercially available processors suitable for mobile devices (for example, an XScale architecture microprocessor, a Microprocessor without Interlocked Pipeline Stages (MIPS) architecture processor, or another type of processor). Amemory 1504, such as a random access memory (RAM), a Flash memory, or other type of memory, is typically accessible to the processor. Thememory 1504 may be adapted to store an operating system (OS) 1506, as well asapplication programs 1508, such as a mobile location enabled application that may provide Location Based Services (LBSs) to a user. Theprocessor 1502 may be coupled, either directly or via appropriate intermediary hardware, to adisplay 1510 and to one or more input/output (I/O)devices 1512, such as a keypad, a touch panel sensor, a microphone, and the like. Similarly, in some implementations, theprocessor 1502 may be coupled to atransceiver 1514 that interfaces with anantenna 1516. Thetransceiver 1514 may be configured to both transmit and receive cellular network signals, wireless data signals, or other types of signals via theantenna 1516, depending on the nature of themobile device 1500. Further, in some configurations, aGPS receiver 1518 may also make use of theantenna 1516 to receive GPS signals. -
FIG. 16 is a block diagram illustrating components of amachine 1600, according to some example implementations, able to readinstructions 1624 from a machine-readable medium 1622 (e.g., a non-transitory machine-readable medium, a machine-readable storage medium, a computer-readable storage medium, or any suitable combination thereof) and perform any one or more of the methodologies discussed herein, in whole or in part. Specifically,FIG. 16 shows themachine 1600 in the example form of a computer system (e.g., a computer) within which the instructions 1624 (e.g., software, a program, an application, an applet, an app, or other executable code) for causing themachine 1600 to perform any one or more of the methodologies discussed herein may be executed, in whole or in part. - In alternative implementations, the
machine 1600 operates as a standalone device or may be connected (e.g., networked) to other machines. In a networked deployment, themachine 1600 may operate in the capacity of a server machine or a client machine in a server-client network environment, or as a peer machine in a distributed (e.g., peer-to-peer) network environment. Themachine 1600 may be a server computer, a client computer, a personal computer (PC), a tablet computer, a laptop computer, a netbook, a cellular telephone, a smartphone, a set-top box (STB), a personal digital assistant (PDA), a web appliance, a network router, a network switch, a network bridge, or any machine capable of executing theinstructions 1624, sequentially or otherwise, that specify actions to be taken by that machine. Further, while only a single machine is illustrated, the term “machine” shall also be taken to include any collection of machines that individually or jointly execute theinstructions 1624 to perform all or part of any one or more of the methodologies discussed herein. - The
machine 1600 includes a processor 1602 (e.g., a central processing unit (CPU), a graphics processing unit (GPU), a digital signal processor (DSP), an application specific integrated circuit (ASIC), a radio-frequency integrated circuit (RFIC), or any suitable combination thereof), amain memory 1604, and astatic memory 1606, which are configured to communicate with each other via abus 1608. Theprocessor 1602 may contain microcircuits that are configurable, temporarily or permanently, by some or all of theinstructions 1624 such that theprocessor 1602 is configurable to perform any one or more of the methodologies described herein, in whole or in part. For example, a set of one or more microcircuits of theprocessor 1602 may be configurable to execute one or more modules (e.g., software modules) described herein. - The
machine 1600 may further include a video display 1610 (e.g., a plasma display panel (PDP), a light emitting diode (LED) display, a liquid crystal display (LCD), a projector, a cathode ray tube (CRT), or any other display capable of displaying graphics or video). Themachine 1600 may also include an alphanumeric input device 1612 (e.g., a keyboard or keypad), a UI navigation device 1614 (e.g., a mouse, a touchpad, a trackball, a joystick, a motion sensor, an eye tracking device, or other pointing instrument), astorage unit 1616, an signal generation device 1618 (e.g., a sound card, an amplifier, a speaker, a headphone jack, or any suitable combination thereof), and anetwork interface device 1620. - The
storage unit 1616 includes the machine-readable medium 1622 (e.g., a tangible and non-transitory machine-readable storage medium) on which are stored the instructions embodying any one or more of the methodologies or functions described herein. Theinstructions 1624 may also reside, completely or at least partially, within themain memory 1604, within the processor 1602 (e.g., within the processor's cache memory), or both, before or during execution thereof by themachine 1600. Accordingly, themain memory 1604 and theprocessor 1602 may be considered machine-readable media (e.g., tangible and non-transitory machine-readable media). Theinstructions 1624 may be transmitted or received over thenetwork 1626 via thenetwork interface device 1620. For example, thenetwork interface device 1620 may communicate theinstructions 1624 using any one or more transfer protocols (e.g., hypertext transfer protocol (HTTP)). - In some example implementations, the
machine 1600 may be a portable computing device, such as a smart phone or tablet computer, and have one or more additional input components (e.g., sensors or gauges). Examples of such input components 1630 include an image input component (e.g., one or more cameras), an audio input component (e.g., a microphone), a direction input component (e.g., a compass), a location input component (e.g., a global positioning system (GPS) receiver), an orientation component (e.g., a gyroscope), a motion detection component (e.g., one or more accelerometers), an altitude detection component (e.g., an altimeter), and a gas detection component (e.g., a gas sensor). Inputs harvested by any one or more of these input components may be accessible and available for use by any of the modules described herein. - As used herein, the term “memory” refers to a machine-readable medium able to store data temporarily or permanently and may be taken to include, but not be limited to, RAM), read-only memory (ROM, buffer memory, flash memory, and cache memory. While the machine-
readable medium 1622 is shown in an example implementation to be a single medium, the term “machine-readable medium” should be taken to include a single medium or multiple media (e.g., a centralized or distributed database, or associated caches and servers) able to store instructions. The term “machine-readable medium” shall also be taken to include any medium, or combination of multiple media, that is capable of storing theinstructions 1624 for execution by themachine 1600, such that theinstructions 1624, when executed by one or more processors of the machine 1600 (e.g., processor 1602), cause themachine 1600 to perform any one or more of the methodologies described herein, in whole or in part. Accordingly, a “machine-readable medium” refers to a single storage apparatus or device, as well as cloud-based storage systems or storage networks that include multiple storage apparatus or devices. The term “machine-readable medium” shall accordingly be taken to include, but not be limited to, one or more tangible (e.g., non-transitory) data repositories in the form of a solid-state memory, an optical medium, a magnetic medium, or any suitable combination thereof. - Throughout this specification, plural instances may implement components, operations, or structures described as a single instance. Although individual operations of one or more methods are illustrated and described as separate operations, one or more of the individual operations may be performed concurrently, and nothing requires that the operations be performed in the order illustrated. Structures and functionality presented as separate components in example configurations may be implemented as a combined structure or component. Similarly, structures and functionality presented as a single component may be implemented as separate components. These and other variations, modifications, additions, and improvements fall within the scope of the subject matter herein.
- Certain implementations are described herein as including logic or a number of components, modules, or mechanisms. Modules may constitute software modules (e.g., code stored or otherwise embodied on a machine-readable medium or in a transmission medium), hardware modules, or any suitable combination thereof. A “hardware module” is a tangible (e.g., non-transitory) unit capable of performing certain operations and may be configured or arranged in a certain physical manner. In various example implementations, one or more computer systems (e.g., a standalone computer system, a client computer system, or a server computer system) or one or more hardware modules of a computer system (e.g., a processor or a group of processors) may be configured by software (e.g., an application or application portion) as a hardware module that operates to perform certain operations as described herein.
- In some implementations, a hardware module may be implemented mechanically, electronically, or any suitable combination thereof. For example, a hardware module may include dedicated circuitry or logic that is permanently configured to perform certain operations. For example, a hardware module may be a special-purpose processor, such as a field programmable gate array (FPGA) or an ASIC. A hardware module may also include programmable logic or circuitry that is temporarily configured by software to perform certain operations. For example, a hardware module may include software encompassed within a general-purpose processor or other programmable processor. It will be appreciated that the decision to implement a hardware module mechanically, in dedicated and permanently configured circuitry, or in temporarily configured circuitry (e.g., configured by software) may be driven by cost and time considerations.
- Accordingly, the phrase “hardware module” should be understood to encompass a tangible entity, and such a tangible entity may be physically constructed, permanently configured (e.g., hardwired), or temporarily configured (e.g., programmed) to operate in a certain manner or to perform certain operations described herein. As used herein, “hardware-implemented module” refers to a hardware module. Considering implementations in which hardware modules are temporarily configured (e.g., programmed), each of the hardware modules need not be configured or instantiated at any one instance in time. For example, where a hardware module comprises a general-purpose processor configured by software to become a special-purpose processor, the general-purpose processor may be configured as respectively different special-purpose processors (e.g., comprising different hardware modules) at different times. Software (e.g., a software module) may accordingly configure one or more processors, for example, to constitute a particular hardware module at one instance of time and to constitute a different hardware module at a different instance of time.
- Hardware modules can provide information to, and receive information from, other hardware modules. Accordingly, the described hardware modules may be regarded as being communicatively coupled. Where multiple hardware modules exist contemporaneously, communications may be achieved through signal transmission (e.g., over appropriate circuits and buses) between or among two or more of the hardware modules. In implementations in which multiple hardware modules are configured or instantiated at different times, communications between such hardware modules may be achieved, for example, through the storage and retrieval of information in memory structures to which the multiple hardware modules have access. For example, one hardware module may perform an operation and store the output of that operation in a memory device to which it is communicatively coupled. A further hardware module may then, at a later time, access the memory device to retrieve and process the stored output. Hardware modules may also initiate communications with input or output devices, and can operate on a resource (e.g., a collection of information).
- The various operations of example methods described herein may be performed, at least partially, by one or more processors that are temporarily configured (e.g., by software) or permanently configured to perform the relevant operations. Whether temporarily or permanently configured, such processors may constitute processor-implemented modules that operate to perform one or more operations or functions described herein. As used herein, “processor-implemented module” refers to a hardware module implemented using one or more processors.
- Similarly, the methods described herein may be at least partially processor-implemented, a processor being an example of hardware. For example, at least some of the operations of a method may be performed by one or more processors or processor-implemented modules. As used herein, “processor-implemented module” refers to a hardware module in which the hardware includes one or more processors. Moreover, the one or more processors may also operate to support performance of the relevant operations in a “cloud computing” environment or as a “software as a service” (SaaS). For example, at least some of the operations may be performed by a group of computers (as examples of machines including processors), with these operations being accessible via a network (e.g., the Internet) and via one or more appropriate interfaces (e.g., an API).
- The performance of certain operations may be distributed among the one or more processors, not only residing within a single machine, but deployed across a number of machines. In some example implementations, the one or more processors or processor-implemented modules may be located in a single geographic location (e.g., within a home environment, an office environment, or a server farm). In other example implementations, the one or more processors or processor-implemented modules may be distributed across a number of geographic locations.
- Some portions of the subject matter discussed herein may be presented in terms of algorithms or symbolic representations of operations on data stored as bits or binary digital signals within a machine memory (e.g., a computer memory). Such algorithms or symbolic representations are examples of techniques used by those of ordinary skill in the data processing arts to convey the substance of their work to others skilled in the art. As used herein, an “algorithm” is a self-consistent sequence of operations or similar processing leading to a desired result. In this context, algorithms and operations involve physical manipulation of physical quantities. Typically, but not necessarily, such quantities may take the form of electrical, magnetic, or optical signals capable of being stored, accessed, transferred, combined, compared, or otherwise manipulated by a machine. It is convenient at times, principally for reasons of common usage, to refer to such signals using words such as “data,” “content,” “bits,” “values,” “elements,” “symbols,” “characters,” “terms,” “numbers,” “numerals,” or the like. These words, however, are merely convenient labels and are to be associated with appropriate physical quantities.
- Unless specifically stated otherwise, discussions herein using words such as “processing,” “computing,” “calculating,” “determining,” “presenting,” “displaying,” or the like may refer to actions or processes of a machine (e.g., a computer) that manipulates or transforms data represented as physical (e.g., electronic, magnetic, or optical) quantities within one or more memories (e.g., volatile memory, non-volatile memory, or any suitable combination thereof), registers, or other machine components that receive, store, transmit, or display information. Furthermore, unless specifically stated otherwise, the terms “a” or “an” are herein used, as is common in patent documents, to include one or more than one instance. Finally, as used herein, the conjunction “or” refers to a non-exclusive “or,” unless specifically stated otherwise.
- As used herein, the term “if” may be construed to mean “when” or “upon” or “in response to determining” or “in response to detecting,” depending on the context. Similarly, the phrase “if it is determined” or “if (a stated condition or event) is detected” may be construed to mean “upon determining” or “in response to determining” or “upon detecting (the stated condition or event)” or “in response to detecting (the stated condition or event),” depending on the context.
Claims (21)
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US17/465,741 US20220035450A1 (en) | 2014-06-24 | 2021-09-02 | Surfacing related content based on user interaction with currently presented content |
Applications Claiming Priority (3)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US14/313,938 US10466776B2 (en) | 2014-06-24 | 2014-06-24 | Surfacing related content based on user interaction with currently presented content |
US16/674,754 US11112867B2 (en) | 2014-06-24 | 2019-11-05 | Surfacing related content based on user interaction with currently presented content |
US17/465,741 US20220035450A1 (en) | 2014-06-24 | 2021-09-02 | Surfacing related content based on user interaction with currently presented content |
Related Parent Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US16/674,754 Continuation US11112867B2 (en) | 2014-06-24 | 2019-11-05 | Surfacing related content based on user interaction with currently presented content |
Publications (1)
Publication Number | Publication Date |
---|---|
US20220035450A1 true US20220035450A1 (en) | 2022-02-03 |
Family
ID=54869596
Family Applications (3)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US14/313,938 Active US10466776B2 (en) | 2014-06-24 | 2014-06-24 | Surfacing related content based on user interaction with currently presented content |
US16/674,754 Active US11112867B2 (en) | 2014-06-24 | 2019-11-05 | Surfacing related content based on user interaction with currently presented content |
US17/465,741 Abandoned US20220035450A1 (en) | 2014-06-24 | 2021-09-02 | Surfacing related content based on user interaction with currently presented content |
Family Applications Before (2)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US14/313,938 Active US10466776B2 (en) | 2014-06-24 | 2014-06-24 | Surfacing related content based on user interaction with currently presented content |
US16/674,754 Active US11112867B2 (en) | 2014-06-24 | 2019-11-05 | Surfacing related content based on user interaction with currently presented content |
Country Status (1)
Country | Link |
---|---|
US (3) | US10466776B2 (en) |
Families Citing this family (20)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US9971755B2 (en) * | 2015-02-03 | 2018-05-15 | Flipboard, Inc. | Selecting additional supplemental content for presentation in conjunction with a content item presented via a digital magazine |
US10515127B2 (en) * | 2015-04-09 | 2019-12-24 | Oath Inc. | Inductive matrix completion and graph proximity for content item recommendation |
US9558159B1 (en) * | 2015-05-15 | 2017-01-31 | Amazon Technologies, Inc. | Context-based dynamic rendering of digital content |
US10629086B2 (en) * | 2015-06-09 | 2020-04-21 | International Business Machines Corporation | Providing targeted, evidence-based recommendations to improve content by combining static analysis and usage analysis |
CN107220230A (en) * | 2016-03-22 | 2017-09-29 | 阿里巴巴集团控股有限公司 | A kind of information collecting method and device, and a kind of intelligent terminal |
US10366405B2 (en) * | 2016-04-15 | 2019-07-30 | Flipboard, Inc. | Content viewability based on user interaction in a flip-based digital magazine environment |
US9864737B1 (en) | 2016-04-29 | 2018-01-09 | Rich Media Ventures, Llc | Crowd sourcing-assisted self-publishing |
US10015244B1 (en) | 2016-04-29 | 2018-07-03 | Rich Media Ventures, Llc | Self-publishing workflow |
US9886172B1 (en) | 2016-04-29 | 2018-02-06 | Rich Media Ventures, Llc | Social media-based publishing and feedback |
US10083672B1 (en) | 2016-04-29 | 2018-09-25 | Rich Media Ventures, Llc | Automatic customization of e-books based on reader specifications |
US10642881B2 (en) * | 2016-06-30 | 2020-05-05 | Intel Corporation | System architecture for universal emotive autography |
US20180114596A1 (en) * | 2016-08-25 | 2018-04-26 | Vimedicus, Inc. | Systems and methods for generating custom user experiences based on health and occupational data |
US20180060506A1 (en) * | 2016-08-25 | 2018-03-01 | Vimedicus, Inc. | Systems and methods for generating custom user experiences based on processed claims data |
US10339215B2 (en) | 2016-12-14 | 2019-07-02 | International Business Machines Corporation | Determining a reading speed based on user behavior |
US11403881B2 (en) * | 2017-06-19 | 2022-08-02 | Paypal, Inc. | Content modification based on eye characteristics |
JP6976530B2 (en) * | 2017-06-19 | 2021-12-08 | 株式会社アロマビット | Composite data generation method and composite data generator |
US11023261B1 (en) * | 2018-12-28 | 2021-06-01 | Snap Inc. | 3RD party application management |
US11151616B2 (en) * | 2019-07-12 | 2021-10-19 | International Business Machines Corporation | Contextual information insertion in context with content |
CN111796741B (en) * | 2020-06-29 | 2022-03-29 | 上海连尚网络科技有限公司 | Method and equipment for providing recommended books |
US20230142904A1 (en) * | 2021-11-09 | 2023-05-11 | Honda Motor Co., Ltd. | Creation of notes for items of interest mentioned in audio content |
Citations (8)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20060251292A1 (en) * | 2005-05-09 | 2006-11-09 | Salih Burak Gokturk | System and method for recognizing objects from images and identifying relevancy amongst images and information |
US7975019B1 (en) * | 2005-07-15 | 2011-07-05 | Amazon Technologies, Inc. | Dynamic supplementation of rendered web pages with content supplied by a separate source |
US20110188713A1 (en) * | 2008-07-16 | 2011-08-04 | Imprezzeo Pty Ltd | Facial image recognition and retrieval |
US20140026156A1 (en) * | 2012-07-18 | 2014-01-23 | David Deephanphongs | Determining User Interest Through Detected Physical Indicia |
US20140118257A1 (en) * | 2012-10-29 | 2014-05-01 | Amazon Technologies, Inc. | Gesture detection systems |
US8947351B1 (en) * | 2011-09-27 | 2015-02-03 | Amazon Technologies, Inc. | Point of view determinations for finger tracking |
US20150134688A1 (en) * | 2013-11-12 | 2015-05-14 | Pinterest, Inc. | Image based search |
US9817477B1 (en) * | 2013-03-11 | 2017-11-14 | Amazon Technologies, Inc. | Eye event detection for electronic documents |
Family Cites Families (20)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US8479122B2 (en) * | 2004-07-30 | 2013-07-02 | Apple Inc. | Gestures for touch sensitive input devices |
US6931564B2 (en) * | 2001-07-13 | 2005-08-16 | International Business Machines Corporation | Failure isolation in a distributed processing system employing relative location information |
US7047502B2 (en) * | 2001-09-24 | 2006-05-16 | Ask Jeeves, Inc. | Methods and apparatus for mouse-over preview of contextually relevant information |
US7100123B1 (en) * | 2002-01-25 | 2006-08-29 | Microsoft Corporation | Electronic content search and delivery based on cursor location |
US20040203852A1 (en) * | 2002-03-07 | 2004-10-14 | Pradeep Janakiraman | System for delivering location-based notifications and method thereof |
FR2849029B1 (en) * | 2002-12-20 | 2005-03-18 | Lafon Labor | PROCESS FOR THE PREPARATION AND CRYSTALLINE FORMS OF OPTICAL ENANTIOMERS OF MODAFINIL. |
US7844354B2 (en) * | 2006-07-27 | 2010-11-30 | International Business Machines Corporation | Adjusting the volume of an audio element responsive to a user scrolling through a browser window |
US7646740B2 (en) * | 2006-10-13 | 2010-01-12 | At&T Intellectual Property I, L.P. | System and method of providing advertisements to vehicles |
US9715543B2 (en) * | 2007-02-28 | 2017-07-25 | Aol Inc. | Personalization techniques using image clouds |
US8296291B1 (en) * | 2007-12-12 | 2012-10-23 | Amazon Technologies, Inc. | Surfacing related user-provided content |
US20130246381A1 (en) * | 2008-10-03 | 2013-09-19 | Google Inc. | Geo signal search result |
US8595228B1 (en) * | 2009-01-09 | 2013-11-26 | Google Inc. | Preferred sites |
US8494215B2 (en) * | 2009-03-05 | 2013-07-23 | Microsoft Corporation | Augmenting a field of view in connection with vision-tracking |
US20120036014A1 (en) * | 2010-08-06 | 2012-02-09 | Verizon Patent And Licensing, Inc. | System for and method of location aware marketing |
US20120209987A1 (en) * | 2011-02-16 | 2012-08-16 | Rhinelander Edward D | Monitoring Use Of Tracking Objects on a Network Property |
JP5957318B2 (en) * | 2012-07-12 | 2016-07-27 | 株式会社日立製作所 | Network system, information relay device, and packet distribution method |
US9123053B2 (en) * | 2012-12-10 | 2015-09-01 | Google Inc. | Analyzing reading metrics to generate action information |
US10089639B2 (en) * | 2013-01-23 | 2018-10-02 | [24]7.ai, Inc. | Method and apparatus for building a user profile, for personalization using interaction data, and for generating, identifying, and capturing user data across interactions using unique user identification |
US20140255003A1 (en) * | 2013-03-05 | 2014-09-11 | Google Inc. | Surfacing information about items mentioned or presented in a film in association with viewing the film |
US9626361B2 (en) * | 2014-05-09 | 2017-04-18 | Webusal Llc | User-trained searching application system and method |
-
2014
- 2014-06-24 US US14/313,938 patent/US10466776B2/en active Active
-
2019
- 2019-11-05 US US16/674,754 patent/US11112867B2/en active Active
-
2021
- 2021-09-02 US US17/465,741 patent/US20220035450A1/en not_active Abandoned
Patent Citations (8)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20060251292A1 (en) * | 2005-05-09 | 2006-11-09 | Salih Burak Gokturk | System and method for recognizing objects from images and identifying relevancy amongst images and information |
US7975019B1 (en) * | 2005-07-15 | 2011-07-05 | Amazon Technologies, Inc. | Dynamic supplementation of rendered web pages with content supplied by a separate source |
US20110188713A1 (en) * | 2008-07-16 | 2011-08-04 | Imprezzeo Pty Ltd | Facial image recognition and retrieval |
US8947351B1 (en) * | 2011-09-27 | 2015-02-03 | Amazon Technologies, Inc. | Point of view determinations for finger tracking |
US20140026156A1 (en) * | 2012-07-18 | 2014-01-23 | David Deephanphongs | Determining User Interest Through Detected Physical Indicia |
US20140118257A1 (en) * | 2012-10-29 | 2014-05-01 | Amazon Technologies, Inc. | Gesture detection systems |
US9817477B1 (en) * | 2013-03-11 | 2017-11-14 | Amazon Technologies, Inc. | Eye event detection for electronic documents |
US20150134688A1 (en) * | 2013-11-12 | 2015-05-14 | Pinterest, Inc. | Image based search |
Also Published As
Publication number | Publication date |
---|---|
US11112867B2 (en) | 2021-09-07 |
US20200201429A1 (en) | 2020-06-25 |
US10466776B2 (en) | 2019-11-05 |
US20150370331A1 (en) | 2015-12-24 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US11112867B2 (en) | Surfacing related content based on user interaction with currently presented content | |
US20200351356A1 (en) | Systems and methods for automatically saving a state of a communication session | |
US11012753B2 (en) | Computerized system and method for determining media based on selected motion video inputs | |
KR102033189B1 (en) | Gesture-based tagging to view related content | |
US9069443B2 (en) | Method for dynamically displaying a personalized home screen on a user device | |
KR102335360B1 (en) | Optimized item attribute comparison | |
US9141702B2 (en) | Method for dynamically displaying a personalized home screen on a device | |
US11455348B2 (en) | Systems and methods for saving and presenting a state of a communication session | |
WO2015084877A1 (en) | Systems and methods to adapt search results | |
WO2016167930A1 (en) | Device dependent search experience |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: PAYPAL, INC., CALIFORNIA Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:EBAY INC.;REEL/FRAME:057377/0344 Effective date: 20150717 Owner name: EBAY INC., CALIFORNIA Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:GONZALES, SERGIO PINZON;REEL/FRAME:057377/0320 Effective date: 20140620 |
|
AS | Assignment |
Owner name: EBAY INC., CALIFORNIA Free format text: CORRECTIVE ASSIGNMENT TO CORRECT THE ASSIGNOR'S NAME PREVIOUSLY RECORDED AT REEL: 057377 FRAME: 0320. ASSIGNOR(S) HEREBY CONFIRMS THE ASSIGNMENT;ASSIGNOR:GONZALES, SERGIO PINZON, JR.;REEL/FRAME:059173/0069 Effective date: 20140620 |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: NON FINAL ACTION MAILED |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: FINAL REJECTION MAILED |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: NON FINAL ACTION MAILED |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: RESPONSE TO NON-FINAL OFFICE ACTION ENTERED AND FORWARDED TO EXAMINER |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: FINAL REJECTION MAILED |
|
STCB | Information on status: application discontinuation |
Free format text: ABANDONED -- FAILURE TO RESPOND TO AN OFFICE ACTION |