EP2612261B1 - Internet search related methods and apparatus - Google Patents
Internet search related methods and apparatus Download PDFInfo
- Publication number
- EP2612261B1 EP2612261B1 EP11758619.8A EP11758619A EP2612261B1 EP 2612261 B1 EP2612261 B1 EP 2612261B1 EP 11758619 A EP11758619 A EP 11758619A EP 2612261 B1 EP2612261 B1 EP 2612261B1
- Authority
- EP
- European Patent Office
- Prior art keywords
- search
- user
- query
- engines
- search engines
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Not-in-force
Links
- 238000000034 method Methods 0.000 title claims description 59
- 230000008569 process Effects 0.000 description 30
- 230000004044 response Effects 0.000 description 22
- 238000010586 diagram Methods 0.000 description 13
- 230000009471 action Effects 0.000 description 10
- 230000006399 behavior Effects 0.000 description 5
- 238000004590 computer program Methods 0.000 description 5
- 230000006870 function Effects 0.000 description 5
- 230000007812 deficiency Effects 0.000 description 4
- 238000012545 processing Methods 0.000 description 3
- 230000003068 static effect Effects 0.000 description 3
- 230000006978 adaptation Effects 0.000 description 2
- 238000004891 communication Methods 0.000 description 2
- 230000006835 compression Effects 0.000 description 2
- 238000007906 compression Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012544 monitoring process Methods 0.000 description 2
- 230000006855 networking Effects 0.000 description 2
- 238000009877 rendering Methods 0.000 description 2
- 238000012552 review Methods 0.000 description 2
- 230000005236 sound signal Effects 0.000 description 2
- 238000012549 training Methods 0.000 description 2
- 238000012546 transfer Methods 0.000 description 2
- 238000003491 array Methods 0.000 description 1
- 230000008901 benefit Effects 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000007781 pre-processing Methods 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 230000002123 temporal effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/95—Retrieval from the web
- G06F16/957—Browsing optimisation, e.g. caching or content distillation
- G06F16/9577—Optimising the visualization of content, e.g. distillation of HTML documents
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/95—Retrieval from the web
- G06F16/951—Indexing; Web crawling techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/95—Retrieval from the web
- G06F16/953—Querying, e.g. by the use of web search engines
Definitions
- the techniques described herein are directed generally to the field of issuing search queries via the Internet.
- the Internet is a global system of interconnected computer networks that store a vast array of information.
- the World Wide Web is an information sharing model built on top of the Internet, in which a system of interlinked hypertext documents are accessed using particular protocols (i.e., the Hypertext Transfer Protocol and its variants).
- search engines have been developed to aid users in locating desired content on the Internet.
- a search engine is a computer program that receives a search query from a user (e.g., in the form of a set of keywords) indicative of content desired by the user, and returns information and/or hyperlinks to information that the search engine determines to be relevant to the user's search query.
- Search engines typically work by retrieving a large number of WWW web pages and/or other content using a computer program called a webcrawler that browses the WWW in an automated fashion (e.g., following every hyperlink that it comes across in each web page that it browses).
- the retrieved web pages and/or content are analyzed and information about the web pages or content is stored in an index.
- the search engine uses the index to identify the web pages and/or content that it determines to best match the user's search query and returns a list of results with the best-matching web pages and/or content.
- this list is in the form of one or more web pages that include a set of hyperlinks to the web pages and/or content determined to best match the user's query.
- general-purpose search engines there are at least two general types of search engines accessible via the Internet: general-purpose search engines and site-specific search engines.
- general-purpose search engine is defined to mean a search engine that provides search results that include web pages and/or content (or hyperlinks to web pages and/or content) hosted on at least two different and independent web sites or domains.
- General purpose search engines attempt to index and provide search results from content distributed across a wide swath of the Internet. Examples of general purpose search engines include GoogleTM, operated by Google, Inc. of Mountain View, CA; Yahoo!TM, operated by Yahoo!, Inc. of Sunnyvale, CA; and BingTM, operated by Microsoft Corp. of Redmond, WA.
- EP1072984 discloses a method and system for providing a native language query service.
- site-specific search engine is defined to mean a search engine that provides search results that include web pages and/or content (or hyperlinks to web pages and/or content) hosted on only one web site or domain.
- Site-specific search engines are frequently used by operators of web sites to allow users to find specific web pages or content on their web sites.
- the web site of an online retailer may include a site-specific search engine that facilitates a user locating web pages for products sold by the retailer.
- the inventors have recognized that for a given user-generated search query, the user may be interested in different types of information pertaining to that search query. For example, for the search query "Miles Davis,” the user may be interested in obtaining biographical information about Miles Davis, listening to samples of or purchasing Miles Davis' music, and/or social networking with others who are interested in Miles Davis. Historically, to obtain these three different types of information, a user would enter the search string "Miles Davis" into three different search engines.
- the user would enter this search string into the search engine for an encyclopedia web site to obtain biographical information, would enter the search string into the search engine for a web site that sells music to listen to or purchase music, and would enter the search string into the search engine of a social networking site to connect with others.
- the inventors have recognized that this process is often time consuming and laborious, as the user must navigate to multiple different web sites (and potentially manage multiple different browser windows or tabs) and enter the same search string repeatedly.
- the inventors have also recognized that, because different search engines index web pages and/or content in different ways, index different universes of content, and/or use different algorithms to determine which web pages and/or content best match a particular search query, different search engines may provide different (though possibly overlapping) sets of search results in response to the same search query. This concept is illustrated by the Venn diagram in Figure 1 .
- Each set depicted in the Venn diagram of Figure 1 represents the search results from one of four hypothetical search engines (i.e., search engine one, search engine two, search engine three, and search engine four) provided in response to a user-provided search query for the keywords "Miles Davis.”
- set 101 includes results provided from search engine one and include search results "Q,” “R,” “S,” and “T.”
- Set 103 includes results provided from search engine two and includes results “R,” “S,” and “Y.”
- Set 105 includes results provided from search engine three and includes results "S,” “T,” and “X.”
- Set 107 includes results provided from search engine four and includes result "Z.”
- some search results are included only in one of the sets and, as such, are returned from only of the four search engines, while other search results are included in multiple of the sets and, as such, are returned from two or more of the search engines.
- a search engine When a search engine returns a set of search results in response to a search query, the search engine generally returns the results in an ordered list.
- the list may be ordered by relevance, may be ordered based on money paid for higher positions in the search results, and/or may be ordered based on other criteria. For example, web pages or content that a search engine has determined to be most relevant are at the top of the list of the results, while web pages or content that the search engine has determined to be less relevant are farther down in the list of results.
- a particular electronics manufacture may pay an electronics e-tailer to list its televisions higher up in the list when users enter search queries including the word "television" into the site-specific search engine for the e-tailer's web site.
- the inventors have appreciated that, because different search engines use different algorithms to determine in what order web pages and/or content are to be listed in the search results, even in a situation in which two different search engines include the same web page or piece of content in their search results in response to a particular search query (e.g., "Miles Davis"), that web page or piece of content may be at or near the top of the list of results provided by a first of the two search engines (e.g., because the first search engine has determined the web page or piece of content to be particularly relevant in the universe of content in evaluates), but may be further down the list of results provided by a second of the two search engines (e.g., because the second of the two search engines has determined the web page or piece of content to be less relevant in the universe of content it evaluates).
- a particular search query e.g., "Miles Davis”
- results that are listed near the top of the list of search results from the second search engine may be listed much lower in the list of results provided by the first engine.
- the inventors have recognized that users are more likely to notice and access results that are near the top of the list. Thus, by using only one of the two search engines, a user may not notice or access results that may be highly relevant to the user.
- the inventors have recognized that issuing a user-specified search query to only a single search engine increases the chances of some web pages and/or pieces of content that are potentially relevant not being included the search results or being listed low enough in the returned list of search results that the user disregards them.
- issuing such a search query to only a single search engine may limit the content that is returned to only the type of content that is search and/or indexed by that search engine, while the user may be interested in many different types of content pertaining to that search query.
- metasearch engines Some web-based software programs, referred to as metasearch engines or search engine aggregators, have been developed that receive a user-specified search query, issue the search query to multiple search engines, receive results from each of these search engines, remove duplicates, aggregate these search results into a single list, and display this list of aggregated search results to the user.
- metasearch engine is DogpileTM, operated by Infospace, Inc. of Bellevue, WA.
- the inventors have recognized that these metasearch engines have a number of disadvantages.
- the search engines that are queried by these metasearch engines include only general-purpose search engines, and do not include any site-specific search engines.
- metasearch engines in response to a user-specified search query, display to the user a single list of aggregated search results from a number of different search engines, the user has no ability to see which search engine provided which search result, to see how relevant each of the plurality of search engines deemed a particular search result to be, or to look at the full set of results returned from any particular search engine.
- metasearch engines run on one or more servers that receive a user-specified search query issued from a browser or other software application executing on the user's client device and issue the search query from the server(s) to the plurality of search engines that are to be queried. Because a metasearch engine may receive search queries from and provide aggregated search results to a large number of users, the server(s) on which the metasearch engine operates may regularly issue a very large number of search queries to the search engines whose results it aggregates. One consequence of this is that a search engine may receive hundreds of thousands or even millions of search queries daily from the same server or IP address, and may perceive this behavior as a single user issuing an enormous number of search queries to the search engine.
- search engine operators would consider this behavior to be an abuse of the search engine service and/or a potential denial of service attack and would take action to block search queries to their search engines from an IP address issuing such a large number of queries. For this reason, a metasearch engine operator must typically obtain contractual agreements with the search engine operators whose search engine results are aggregated by the metasearch engine to allow the metasearch engine to issue a large number of search queries to these search engines.
- metasearch engines do not provide any capability for the user to control to which search engines his or her search query is to be provided. Rather, existing metasearch engines have a fixed set of search engines to which every search query is provided. Thus, the user has no control over which search engines' results are provided in response to a search query and the metasearch engines do not perform any customization of which search engines are queries based on which user issued the search query or based on the content of the user's search query.
- Some embodiments described below are directed to techniques related to issuing a user-specified search query to multiple search engines and/or displaying the results of such queries to users. While some of the embodiments discussed below address all of the above-discussed deficiencies of existing metasearch engines, not every embodiment addresses all of the above-discussed deficiencies of metasearch engines, and some embodiments do not address any of these deficiencies. As such, it should be understood that the invention is not limited to embodiments that address all or any of the above-described deficiencies of metasearch engines.
- Figure 2 is a flow chart of an illustrative process 200 that may be performed, in some embodiments, by an application program executing on a user's client device to issue a search query to multiple search engines and to display the search results from each of these search engines to the user.
- the client device upon which the application program executes to perform process 200 may be any type of computing device that has hardware capable of executing a software computer program. Examples of types of client devices that may be used include a laptop or desktop personal computer, a personal digital assistant (PDA), a mobile phone, a server computer, and/or various other types of computing devices.
- PDA personal digital assistant
- the techniques described below may be implemented on a mobile telephone (e.g., a smartphone) and combined with a speech recognition capability so that the user may issue search queries by voice, but all aspects of the invention are not limited to use on mobile phones or to use with a speech recognition interface.
- process 200 permits a user to launch an application program on the client device, input a search query (e.g., by voice or by text), and view search results for that search query provided from multiple different search engines.
- a search query e.g., by voice or by text
- Process 200 begins at act 201, where the application program executing on the client device 303 receives a user-specified search query. As discussed in greater detail below, the query may be received in any of a variety possible ways via any of a variety of possible client device user interfaces.
- act 203 the application program determines, in response to receiving the user-specified query, which of search engines 307 to query.
- the set of search engines to be queried may be determined dynamically and may be user-configurable, though aspects of the invention are not limited in this respect as in some embodiments the set of search engines to be queried may be static and not configurable.
- these search engines may include general-purpose search engines and/or site-specific search engines, but the invention is not limited in this respect as in some embodiments only site-specific search engines or only general-purpose search engines may be queried.
- act 207 where the application program receives, via the Internet 305, search results from the search engines queried in act 205.
- process 200 depicts one illustrative order in which acts 201, 203, 205, 207, and 209 may be performed. Various other orders are possible, and these acts may be performed in different orders in various different embodiments. Additionally, various possible implementations of each of the acts of process 200 are possible and are discussed in greater detail below.
- the application program receives user input specifying a search query.
- This user input may be received in any of a variety of ways and in any of a variety of formats.
- the user input may be received as text and may be received via an input device integrated into or coupled to the client device. Examples of such input devices include a keyboard, a touch screen, a mouse, and/or any other input device via which a user may provide text input to a computing device.
- the user input may be received as audio via a microphone that is integrated into or coupled to the client device.
- a user may speak a search query into the microphone and the application program may receive the user's voice input as audio data.
- Automated speech recognition may be performed on the audio data to obtain a recognition result (e.g., in textual form) that may be used as the basis for querying search engines.
- one or more of the search engines to be queried may have a speech interface.
- the audio may be provided to such search engines instead of, for example, providing a text search query based on automated speech recognition of the audio.
- Automated speech recognition techniques are well known, and any of a variety of automated speech recognition techniques may be used to obtain a recognition result for the audio data.
- the client device may provide a number of different options from which the user may select to provide a search query. For example, in some embodiments, the client device may permit the user to input the query either in voice format (e.g., by speaking the query into a microphone) or in text format (e.g., by typing the query on a keyboard or a touchscreen).
- voice format e.g., by speaking the query into a microphone
- text format e.g., by typing the query on a keyboard or a touchscreen.
- the automated speech recognition of the user-supplied audio data may be accomplished in any of a variety of ways.
- the client device may execute an automated speech recognizer, which is a software program that performs automated speech recognition on audio data, and the application program may use the automated speech recognizer to obtain a recognition result of the audio data.
- client device 401 executes the application program 403 that performs process 200 and an automated speech recognizer 405.
- application program 403 receives user-supplied audio data 407, it may pass the audio data to automated speech recognizer 405.
- Automated speech recognizer 405 may perform automated speech recognition on the audio data to obtain a recognition result 409 and may return recognition result 409 to application program 403 for use in formatting queries for the speech engines.
- client devices may not have sufficient computing resources to execute an automated speech recognizer that operates with a desired level of accuracy and/or efficiency. That is, for example, a client device may not have sufficient memory to store acoustic models, language models, grammars, and/or other components of such a speech recognizer and/or may not have a processor powerful enough to perform automated speech recognition at a desired level of efficiency. This may occur in any number of situations, particularly when client device is a handheld device, such as a PDA or mobile phone.
- automated speech recognition on audio data supplied from a user may be performed by a computer external to the client device executing the application program that receives user input and manages searches based on such input.
- client device 501 executes application program 503 that receives voice input supplied by a user in the form of audio data.
- Automated speech recognition may be performed on the audio data by an automated speech recognizer 507 executing on a server 505.
- application program may send the user-supplied audio data 509 to automated speech recognizer 507 executing on server 505.
- the audio data 509 may be sent to the automated speech recognizer in any of a variety of possible ways.
- the user-supplied audio may be captured by a microphone and analog-to-digital conversion may be performed on the captured microphone audio to generate digital audio data.
- This digital audio data may be supplied to automated speech recognizer 507.
- some processing of the digital audio data may be performed prior to sending this data to the automated speech recognizer.
- the digital data may be compressed or some pre-processing may be performed to extract parameters of the audio that are useful in automated speech recognition.
- the extracted parameters may be passed to automated speech recognizer 507 with the audio data.
- any of a variety of possible compression algorithms may be used.
- the speexTM codec may be used.
- audio data may refer to an analog audio signal or a digital representation or compressed digital representation of an analog audio signal.
- Automated speech recognizer 507 may perform automated speech recognition on audio data 509 and return to application program 503 a response 511 that comprises results from audio data 509.
- the results may take any suitable form.
- the results may comprise a text recognition result obtained by automated speech recognizer 507 from performing automated speech recognition on the audio data.
- client device 501 is depicted as communicating directly with server 505. It should be appreciated that this depiction is provided merely to facilitate an understanding of the types of information sent between client device 501 and server 505, and that client device 501 may communicate with server 505 in any of a variety of ways and via any suitable communication medium, including, for example, via the Internet.
- automatic speech recognition on audio data received at a client device is performed on the client device
- audio data is provided from the client device to a server and the server perform automatic speech recognition on the server.
- automatic speech recognition may be performed partially by the client device and partially by the server. For example, a portion of an ASR may execute on the client and a portion may execute on the server. This may be done, for example, to enable the client device to perform a portion of the speech recognition process, but leave computationally intensive portions for the server.
- server 505 may provide automated speech recognition services for multiple client devices.
- Figure 6 shows a computing environment in which each of a plurality of users 601a, 601b, ... 601n operates a client device 603a, 603b, ..., 603n.
- Each client device 603 may execute an application program that performs process 200 of Figure 2 to send search queries via the Internet 607 to search engines 609a, 609b, 609c, ...., 609n, and to display the results of these search queries to their respective users.
- Server 605 may provide automated speech recognition services.
- client devices 603 may send the audio data to server 605 to have automated speech recognition performed on the audio data, and may receive a recognition result and/or other information based on the recognition result in response.
- server 605 provides automated speech recognition services for client devices 603.
- server 605 may provide additional functionality to facilitate issuance of search queries from a client device 603 to one or more of search engines 609. Some examples of this additional functionality are described in detail below. For the sake of simplicity, in the example of Figure 6 , only one server 605 is depicted. However, it should be understood that, as is conventional for Internet servers that receive a high volume of traffic, any number of servers may be used to provide the functionality of server 605 that is described herein and the traffic load from client devices 603 may be balanced across these servers using any of a variety of load balancing techniques.
- a number of techniques may be used to facilitate automated speech recognition of the audio data to generate search queries. These techniques may be employed both in embodiments in which automated speech recognition is performed on the client device and in embodiments in which automated speech recognition is performed on a computer external to the client device.
- the language model that is used by the automated speech recognizer in performing speech recognition on the audio data of the user-supplied search query may be selected based on the search engine(s) which are to be queried. For example, as shown in Figure 7 , in some embodiments, the automated speech recognizer may perform recognition on the same audio data using multiple different language models to yield multiple different recognition results.
- speech recognition is performed on audio data 701 using language models 703a, 703b, ..., 703n to generate recognition results 705a, 705b, ..., 705n.
- each of recognition results 705 may be the same, each may be different from the others, or some may be the same and some may be different.
- One or more of language models 703 may be associated with a particular search engine and the recognition result generated using a language model that is associated with a particular search engine may serve as the basis for the search query that is issued to that search engine.
- a language model that has been trained on the content that is indexed by a particular search engine may be used to generate the recognition result that serves as the basis for the query that is issued to that search engine. This results in a query that is more likely to yield relevant search results for that search engine.
- This technique may be particularly useful when issuing queries to site-specific search engines that index a limited amount of content that is generally directed to a particular subject area.
- site-specific search engines that index a limited amount of content that is generally directed to a particular subject area.
- an online shoe retailer may have a site-specific search engine for its e-commerce web site.
- the content of this web site may include shoe-specific terminology and phrases, such as the name of shoe brands and models, descriptions of shoe properties, and various other shoe-specific information.
- the recognizer is more likely to accurately recognize speech that includes the shoe-specific terminology and phrases.
- the audio input may be recognized using a first language model (e.g., a language model that has been trained on content related to the content indexed by the site-specific search engine) to generate a recognition result that serves as the basis for the query to be issued to the site-specific search engine and may also be recognized using a second language model (e.g., that has been trained on more general content), different from the first language model, to generate a recognition result that serves as the basis for the query to be issued to the general-purpose search engine.
- a first language model e.g., a language model that has been trained on content related to the content indexed by the site-specific search engine
- a second language model e.g., that has been trained on more general content
- each of recognition results 705 may be generated by performing automatic speech recognition on audio data 701 using a particular language model.
- one of recognition results 705 may be selected to be the result that serves as the basis for generating a search query to be issued to multiple search engines. This selection may be made in any suitable way. For example, as discussed above, the recognition result that has the highest score or confidence value as a result of the speech recognition may be selected or any other suitable criterion or combination of criteria may be used.
- the inventors have recognized that, in situations in which speech recognition is performed on the same audio data using multiple language models, performing multiple separate and independent speech recognition processes with each of the different language models may increase the latency in obtaining recognition results and/or may increase the amount of processing resources used in performing speech recognition.
- one generalized language model may be used to create a lattice of hypothesized word sequences, and this lattice may be re-scored using one or more selected language models to yield recognition result(s) particularly for the selected language model(s).
- this lattice may be re-scored using one or more selected language models to yield recognition result(s) particularly for the selected language model(s).
- audio data 701 is shown as being provided directly to each of language models 703. This depiction is provided merely to facilitate an understanding of how multiple language models may be used, and it should be understood that, in practice, the language models may not operate directly on audio data.
- a lattice with multiple hypothesized sequences of words may be obtained from audio data using an acoustic model and a general language model, and specialized language models 703 may be used to rescore the general lattice and obtain a sequence of words appropriate to a specific search engine.
- one language model may be selected from a plurality of available language models. For example, in some embodiments, one language model may be identified as being well-suited for recognizing a particular voice query, and that language model may be used in recognizing the query.
- any of a variety of possible criteria or combinations of criteria may be used to select a language model to be used in recognizing a particular voice query.
- the content of the query may be used as a basis for selecting a language model to be used.
- a two-pass process may be used whereby in a first pass automatic speech recognition of a voice query or a portion of a voice query is performed using a general language model that doesn't have a specialized vocabulary.
- Such a language model may recognize some of the words in the query, but may not accurately recognize all of the words.
- the recognition result obtained from the first-pass may be used to classify the query based on topic or category. Any of a variety of well-known classification techniques may be used.
- a language model that has a specialized vocabulary directed to the identified topic or category may be selected.
- a second-pass of automatic speech recognition may be performed using the selected language model. For example, if after the first-pass it is determined that the voice query includes word such as "directions,” "street,” or “road,” it may be determined that voice query is map-related query, and a map-related language model (e.g., a language model that includes city, street, and place names for the United States and/or other countries) may be selected and used in the second-pass.
- a map-related language model e.g., a language model that includes city, street, and place names for the United States and/or other countries
- the recognition result from the second-pass may serve as the basis for one or more search queries to be provided to one or more search engines.
- the search engine or engines that are queried may be selected based on the language model that was selected to recognize the query in the second-pass or the content of the recognition result obtained from the second-pass.
- a language model that is used to recognize a voice search query supplied from a user as audio data may be updated based on topics of interest. This may be done in any of a variety of possible ways.
- the inventors have recognized that current events frequently result in search engines receiving a large number of search queries that include keywords associated with a particular event, including keywords that may have been previously unused and not the subject of frequent searches. For example, if a popular new movie is playing in movie theaters, search engines may receive a large number of search queries from different users that include the name of the movie, the names of the actors in the movie, or other keywords associated with the movie.
- search engines may receive a large number of queries that include the names of the teams and/or athletes participating in the event, or an event in popular culture may thrust a person previously unknown to the general public into the limelight (e.g., a person winning the lottery, a person accused of a high-profile crime, a person involved in a high profile scandal, etc.)
- words and/or phrases associated with topics of recent user interest may be identified, and the language model may be updated to improve the accuracy of recognizing these words and/or phrases.
- These words and/or phrases may be identified in any of a variety of ways. For example, in some embodiments in which user-supplied queries (e.g., in audio format, text format, or some other format) are provided from a plurality of client devices to a server or set of servers, the server or set of servers may identify frequently used search terms or phrases in these search queries and the language model may be updated based on these frequently used search terms.
- words and/or phrases associated with topics of interest may be identified manually by one or more humans tasked with identifying such words or phrases.
- the language model may be updated in this manner with any desired degree of frequency.
- the language model may be updated at a regular interval, such as once per day, once every two days, twice per day, once per week, or any of a variety of other possible intervals.
- the language model may be updated in response to a user action, such as an indication from the user to update the language model or in any other suitable way.
- words provided in user search queries, words obtained from webcrawling, and/or words from other sources may be collected and used to update the language model.
- a word count indicating the number of occurrences of the words in the source data may be maintained and words whose word count exceeds a threshold value may be added to the language model.
- an auto-pronunciation e.g., a sequence of phonemes
- a human reviewer e.g., a linguist
- the language model used in automated speech recognition of search queries may be updated using feedback indicating which search results a user selected.
- an automated speech recognizer may generate a recognition result from audio data of a user-supplied voice search query. This recognition result may serve as the basis for one or more search queries that are provided to one or more search engines, and search results from the one or more search engines may be returned and displayed to the user in response to the query or queries.
- the user may select one or more of the search results to view a particular web page or piece of content.
- information about which search result(s) the user selected may be used to update the language model used by the automated speech recognizer.
- a search query for the phrase "Willy Maze” may be issued to one or more search engines.
- one or more of the search engines may return in its list of results a hyperlink to a web page with biographical information about Willie Mays. If the user selects this hyperlink to access that web page, the content of the web page may be used to update the language model. This may increase the likelihood that the next time a user speaks the phrase "Willie Mays," it will be correctly recognized.
- the language model that is used for a particular user may be customized to that user's speech, using previous utterances obtained from that user. This may be done in any of a variety of ways.
- voice data received from a particular speaker may be represented as a bag-of-word feature vector, with the goal of discovering a set of representative latent speakers to "explain" the data. The outcome is that, for example, similar speakers are clustered together in a probabilistic fashion. These clusters may be used to build a set of latent speaker language model components. Using these latent components, adaptation for a particular user is performed via estimating the speaker-specific linear interpolation weights of the language model components using the received voice data from the particular speaker for which adaptation is being performed.
- the language model that is used for a particular user may be updated based on the user's historical browsing information. This may be accomplished in any of a variety of ways. For example, based on the user's browsing history, web sites that are frequently visited may be identified. The language model that is used to recognize speech queries may be re-trained using a training corpus in which content from the frequently visited web sites is emphasized.
- any or all of the above-described techniques for selecting a language model to facilitate automated speech recognition can be used in combination with each other.
- these techniques for selecting a language model need not be used, and in some embodiments the same language model may be used to process all voice input.
- the application program determines how many and which search engines to query. This may be accomplished in any of a variety of ways.
- the set of search engines which are queried may be static.
- the application program may be hard coded or otherwise configured to query a specific set of search engines.
- the application program queries the same set of search engines in response to every user-supplied search query that it receives.
- act 203 may be considered to be performed by executing the software code of the application program that determines the search engines to be queried simply by those search engines being identified by static information in or accessible by the application program.
- the determination of which search engines to query may be made dynamically based on one or more of a variety of possible criteria.
- the search engines to be queried may be determined based on explicit instructions from the user indicating one or more search engines to query, may be determined based on the content of the query, may be determined based on historical browsing information or access patterns of the user that supplied the query, may be determined based on historical browsing information or access patterns of one or more other users different from the user that supplied the query, may be based on any combination of two or more of the above, and/or may be determined based on any one or more of a number of other possible criteria.
- the determination of which search engines to query using the above-listed criteria may be made by the application program executing on the client device. In other embodiments, the determination of which search engines to query using the above-listed criteria may be made by a computer external to the client device that executes the application program, and this computer may provide information identifying the search engines to be queried to the application program. In these embodiments, the application program determines which search engines to query from the information provided to it by the external computer.
- server 605 may instruct the application program executing on the client device 603 which search engine(s) to query.
- server 605 may instruct the application program as to which search engine(s) to query by providing to the application program Universal Resource Locators (URLs) for the search engines to be queried, wherein the URL for each search engine to be queried includes the search string, appropriately formatted for that search engine.
- URLs Universal Resource Locators
- the application program executing on client device 803 may receive a search query from a user 801, which may be in text format, audio format, or some other format.
- Client device 803 may pass the user-supplied search query to server 805, which may determine which search engines to query, generate URLs for the determined search engines with the appropriate search strings, and return the URLs to the application program on client device 803.
- server 805 may, in some embodiments, perform automated speech recognition on the audio data to generate a recognition result and determine which search engines to query based on the recognition result.
- automated speech recognition may be performed on client device 803, and the recognition result, rather than the audio data, may be sent from client device 803 to server 805.
- Figure 9 shows an example of URLs that may be generated for three different search engines for the search query "Miles Davis."
- the client device may generate the URLs for the different search engines.
- the server may instruct the client device which search engines to query and the client device may generate the URLs for these search engines.
- search engines cannot be queried with a simple URL.
- the client first establishes a connection or session with the search engine, and then sends the query to the search engine. If such a search engine is identified as a search engine to be queried, a query based on the content of the user-supplied query may be generated in a format suitable for that search engine. Like the URLs, the query for such a search engine may be generated by the application program on the client device and/or by a computer external to the client device (e.g., server 605).
- the determination of which search engines to query may be made based, at least in part, on direct input from a user identifying which search engines he or she desired to be queried. For example, the user may select which search engines he or she desires to query from a list provided by the application program, the user may identify the search engine in the query (e.g., "give me the Wikipedia page for George Washington,” or “give me the Yelp review for John Doe's Bar and Grill,” etc.), or may provide this input in some other way.
- such input may be supplied each time the user provides a search query to the application program.
- this input may be supplied once by the user, the application program may store information identifying the user-indicated search engines, and each time a user-supplied search query is received, the application program may use this stored information to determine which search engines to query.
- the application program may provide this information to an external computer (e.g., server 605 in Figure 6 ) and the information may be stored thereon.
- server 605 may determine which search engines to query using this information and may instruct the application program to query these search engines (e.g., by providing URLs or in some other way).
- one or more of the search engines to be queried may be determined based on the content of the user-supplied search query.
- a user-supplied search query includes the name of search engine from which the user-supplied search query requests content
- that search engine may be selected as a search engine to be queried. For example, a user may provide the search query, "get me the wikipedia page on Miles Davis.” Thus, the search engine for wikipedia.com may be selected as a search engine to be queried.
- one or more of the search engines to be queried may be determined based on the type of information that the user requests. For example, if the user provides a search query requesting information about shoes, a search engine for the web site of an online shoe retailer may be selected as a search engine to be queried. As another example, if a user provides a search query such as, "what is the current weather in New York City?" the search engine of a weather web site may be selected as a search engine to be queried.
- one or more of the search engines to be queried may be determined based on historical browsing information or access patterns of the user that issued the query and/or historical browsing information or access patterns of one or more other different users. For example, if the historical browsing information indicates that a user very frequently selects links returned in search results from search engine A but very infrequently selects links returned in search results from search engine B, search engine A may be selected as a search engine to be queried, while search engine B may not be selected.
- search engine A may be selected as a search engine to be queried.
- the application program executing on the client device may issue search queries to the search engines identified in act 203.
- the application program or a computer e.g., a server
- the application program or a computer external to the client device may generate URLs or other types of search queries corresponding to each of the search engines, where each URL or query includes a search string that is based on the search query received from the user (e.g., the search query received at act 201 of process 200) and is formatted appropriately for its corresponding search engine.
- the application program may issue the search queries (e.g., by accessing the URLs) to cause each of the selected search engines to perform a search on the search string included in its corresponding URL.
- the search queries are issued by the application program executing on the client device, such that search results are returned from the search engines to the application program on the client device.
- This provides the advantage that the search engines that are queried do not receive a large volume of queries from the same server or IP address.
- the search engines being queried may receive a very large number of queries from a single source, and may perceive this behavior as a single user issuing an enormous number of search queries to the search engine.
- search engine operators would consider this behavior to be an abuse of the search engine service and/or a potential denial of service attack and would take action to block search queries to their search engines from a source issuing such a large number of queries.
- the search engines may perceive the queries as being issued from the client device that caused them to be issued, and may treat the issuance of such queries as normal user behavior.
- the URLs or queries are generated by a computer external to the client device (e.g., server 605 in Figure 6 )
- that computer may issue the queries, receive the search results from the corresponding search engines, and return those search results to the application program on the client device.
- the queries for each of the search engines selected in act 203 may be issued automatically in response to receiving the search query from the user in act 201. In other embodiments, the queries for all or some of the search engines selected in act 203 may not be issued to those search engines until the user provides input requesting to view the search results from those search engines.
- the search results from some search engines may include a large amount of content.
- the search results from a search engine for a web site that hosts video clips may include a thumbnail image of each video included in the list of results. Consequently, transferring the search results from the search engine to the client device or external computer may take a relatively long time.
- the time taken to transfer the search results may be time that is not well spent if the user does not ever view the search results from that search engine.
- search queries to search engines that return results including bandwidth-intensive content such as a large number of images, video data, audio data, and/or other bandwidth-intensive content may not be issued automatically in response to receiving a user-supplied search query, while search queries to search engines that return results that do not include bandwidth-intensive content may be issued automatically.
- the size of the content (or the expected size of the content) of a set of search results is one example of a criterion that may be used to determine whether to issue a search query automatically to a particular search engine or whether to await some user indication before issuing the query to that search engine. Any of a variety of other criteria or combinations of criteria may be used.
- the application program executing on the client device may receive search engine results generated by the search engines queried in act 205.
- the search engine results may be received in any of a variety of possible ways.
- each search engine may provide its results directly to the client device.
- an external computer e.g., server 605 in Figure 6
- each search engine may provide its results to the external computer, and the external computer may provide these results to the client device.
- the results may be received in any of a variety of formats.
- Some search engines provide results as a hypertext markup language (HTML) web page that includes a list of hyperlinks to the content and/or web pages identified as relevant to the search query.
- Other search engines may provide results in different formats. It should be appreciated that the application program may receive results in any of a variety of formats and all of the search results received need not be in the same format.
- HTML hypertext markup language
- the application program may cause the search results received in act 207 to be displayed on a display device integrated into or coupled to the client device.
- the results may be displayed in any of a variety of formats and may be presented in any of a variety of possible ways.
- the search results from search engines that provide search results as an HTML web page may be displayed by rendering the HTML code in the web page. This is different from the displayed results from metasearch engines such as the above discussed DogpileTM, which do not render the HTML code received from a search engine, but rather extract links from this HTML code and generate new HTML code to display the extracted links.
- metasearch engines such as the above discussed DogpileTM, which do not render the HTML code received from a search engine, but rather extract links from this HTML code and generate new HTML code to display the extracted links.
- HTML code may be automatically generated to display the received search results, and these search results may be displayed by rendering the automatically generated HTML code.
- the search results from each search engine may be displayed as visually separated from the search results from other search engines, so that it is clear to the user viewing the search results which search engine provided each set of search results.
- the order of the list of search results returned from each search engine may be preserved in the displayed search results for each search engine, such that the user may view how relevant each search engine determined each search result in its list to be.
- Search results from each search engine may be visually separated from each other in any of a variety of possible ways. For example, in some embodiments, search results from multiple search engines may be displayed concurrently on the display, separated by columns, lines, or in some other way. In some embodiments, only one of the search engines' search results may be displayed on the display at any given time, and the user may control which of the search engines' search results are displayed by selecting an icon, button, or other user interface element.
- a carousel having a plurality of slots may be displayed on the display device, with each slot corresponding to one of the search engines from which search results were received. User selection of one of the slots of the carousel causes the search results from the search engine corresponding to the selected one of the slots to be displayed on the display.
- Figure 10 shows an example of a display 1000 that employs such a carousel.
- Figure 10 is a screenshot of a search result from the Dragon SearchTM application available from Nuance Communications, Inc., of Burlington, MA for the iPhoneTM available from Apple Computer, Inc. of Cupertino, CA.
- the screenshot of Figure 10 is one example of a variety of possibilities that may be used to display search results from multiple search engines.
- display 1000 comprises a first area in which a carousel 1001 is displayed.
- Carousel 1001 comprises a number of slots 1003a, 1003b, 1003c, 1003d, and 1003e, each of which corresponds to a search engine from which search results were received in act 207.
- the search engines to which the slots 1003 correspond may include general purpose search engines and/or site-specific search engines, such that the search results provided in response to one user-supplied query may come from multiple different search engines (some of which could be general purpose search engines and some of which could be site-specific search engines).
- the search engine to which each slot corresponds may be identified by displaying a symbol, icon, text, image or other information in each slot that is indicative of the search engine corresponding to the slot. Because, in some embodiments, the search engines that are queried in response to a user-supplied query received at the client device may be determined dynamically, the search engines that are allocated to the slots in the carousel may also be determined dynamically. That is, for example, a slot may be allocated in the carousel for each search engine selected in act 203 of process 200 as a search engine to be queried.
- a user may configure which search engines are to be allocated to the slots in the carousel.
- the allocation of carousel slots to particular search engines may be made based on the results returned by the search engines. For example, if one of the search engines selected at act 203 of process 200 as a search engine to be query does not return any search results or returns an indication that no search results were found as a result of the query, a slot in the carousel may not be allocated for that search engine, and the page returned from that search engine may not be displayed to the user.
- Display 1000 may also include an area 1005 in which search results are displayed.
- User selection of one of the slots 1003 in carousel 1001 causes the results received from the search engine corresponding to the selected slot to be displayed in area 1005.
- the content displayed in area 1005 may be different for each different user-issued query. That is, for example, when a user selects slot 1003b in carousel 1001, the content displayed in area 1005 may be different for the query "Miles Davis" than for the query "George Washington.”
- the content that is displayed is response to user selection of a carousel slot can be considered to be "dynamic.”
- Display 1000 may also comprise an indicator (e.g., an arrow or some other type of indicator) 1007 that indicates which slot 1003 in carousel 1001 is currently selected. This indicator identifies to the user which search engine's search results are being displayed in area 1005.
- an indicator e.g., an arrow or some other type of indicator
- Display 1000 may also comprise a query field 1009.
- the search query displayed in field 1009 indicates to the user the basis of the search query that was issued to each of the search engines from which search results were received.
- query field 1009 may display the user-supplied query that served as the basis for each of the different search queries issued in response to that query.
- a user may input a search query into field 1009 in any of a variety ways.
- a user may provide text input to field 1009 (e.g., via a physical or touch screen keyboard).
- a user may provide voice input to field 1009.
- the user may select button 1011 and speak his or her search query into a microphone on the client device.
- one carousel slot may be selected by default, such that one search engine's results are automatically displayed in area 1005.
- the default carousel slot may be selected in any of a variety ways. For example, the default carousel slot may be selected based on which search engine's results the user views most frequently, based on the content of the user's query, based on which search engine provides the most general results, and/or based on any other suitable criterion or criteria.
- not all of the results returned from a particular search engine in response to a query may fit in area 1005 at once.
- a user may be provided with the capability to scroll up and down the list of search results and/or adjust the zoom level at which results are displayed so as to adjust which portion of the search results are displayed in area 1005.
- the order of the search engines in the carousel slots may be selected in any of a variety of ways.
- the search engines may be ordered in the carousel based on frequency of use by the user, based on the content of the user's query, and/or based on any other suitable criterion or criteria.
- not all of the slots of carousel 1001 may be able to be displayed on the display at one time.
- slots 1003a and 1003e are only partially displayed.
- carousel 1001 may include additional slots that are not displayed at all in Figure 10 .
- the user may adjust which slots of the carousel are displayed by shifting the carousel to the right side and/or to the left side of the display. This may be accomplished in any of a variety of ways. For example, the user may drag the carousel in one direction using a pointer device such as a mouse. In embodiments in which display 1000 is displayed on a touch screen, a user may drag the carousel in one direction using his or her finger.
- carousel 1001 may wrap-around display 1000.
- the slots of carousel 1001 moving the left will move off of the left side of the display and will eventually re-enter the display on the right side.
- dragging carousel 1001 in a particular direction can be thought of as rotating the carousel in that direction.
- search results from one or more search engines are displayed to a user
- the user may view some or all of the lists of search results, and may access hyperlinks to view web pages or content identified in the lists of results.
- the application program may monitor user actions taken with respect to the search results and may store information about the monitored user actions.
- the monitored actions can take any suitable form, as the aspect of the invention relating to monitoring user actions is not limited in this respect.
- the monitored user actions may include, for example, which search engines' search results the user viewed and what web pages and/or content the user accessed.
- the information describing the monitored user actions may be subsequently used to update language models used in automatic speech recognition of voice queries, determine which search engines to query in response to subsequent user queries, determine how search engine results are to be presented (e.g., determine which carousel slot is the default slot and/or in what order the search engines are to be positioned in the carousel slots), and/or to facilitate various other aspects of querying search engines and/or displaying search engine results.
- the client device may send the monitored information to the external computer.
- the external computer may associate this information with the client device, the user of the client device, and/or the network address from which the information was sent, and may store it for subsequent use.
- Figure 11 is a block diagram an illustrative computing device 1100 that may be used to implement any of the above-discussed computing devices.
- the computing device 1100 may include one or more processors 1101 and one or more tangible, non-transitory computer-readable storage media (e.g., memory 1103).
- Memory 1103 may store, in tangible non-transitory computer-readable storage media computer instructions that implement any of the above-described functionality.
- Processor(s) 1101 may be coupled to memory 1103 and may execute such computer instructions to cause the functionality to be realized and performed.
- Computing device 1100 may also include a network input/output (I/O) interface 1105 via which the computing device may communicate with other computers (e.g., over a network), and may also include one or more user I/O interfaces, via which the computer may provide output to and receive input from a user.
- the user I/O interfaces may include devices such as a keyboard, a mouse, a microphone, a display device (e.g., a monitor or touch screen), speakers, a camera, and/or various other types of I/O devices.
- the above-described embodiments of the present invention can be implemented in any of numerous ways.
- the embodiments may be implemented using hardware, software or a combination thereof.
- the software code can be executed on any suitable processor or collection of processors, whether provided in a single computer or distributed among multiple computers.
- any component or collection of components that perform the functions described above can be generically considered as one or more controllers that control the above-discussed functions.
- the one or more controllers can be implemented in numerous ways, such as with dedicated hardware, or with general purpose hardware (e.g., one or more processors) that is programmed using microcode or software to perform the functions recited above.
- one implementation of various embodiments of the present invention comprises at least one tangible, non-transitory computer-readable storage medium (e.g., a computer memory, a floppy disk, a compact disk, and optical disk, a magnetic tape, a flash memory, circuit configurations in Field Programmable Gate Arrays or other semiconductor devices, etc.) encoded with one or more computer programs (i.e., a plurality of instructions) that, when executed on one or more computers or other processors, performs the above-discussed functions of various embodiments of the present invention.
- the computer-readable storage medium can be transportable such that the program(s) stored thereon can be loaded onto any computer resource to implement various aspects of the present invention discussed herein.
- references to a computer program which, when executed, performs the above-discussed functions is not limited to an application program running on a host computer. Rather, the term computer program is used herein in a generic sense to reference any type of computer code (e.g., software or microcode) that can be employed to program a processor to implement the above-discussed aspects of the present invention.
- embodiments of the invention may be implemented as one or more methods, of which an example has been provided.
- the acts performed as part of the method(s) may be ordered in any suitable way. Accordingly, embodiments may be constructed in which acts are performed in an order different than illustrated, which may include performing some acts simultaneously, even though shown as sequential acts in illustrative embodiments.
Landscapes
- Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
- Information Transfer Between Computers (AREA)
Applications Claiming Priority (6)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US12/877,549 US8239366B2 (en) | 2010-09-08 | 2010-09-08 | Method and apparatus for processing spoken search queries |
US12/877,690 US20120059658A1 (en) | 2010-09-08 | 2010-09-08 | Methods and apparatus for performing an internet search |
US12/877,647 US20120059814A1 (en) | 2010-09-08 | 2010-09-08 | Methods and apparatus for selecting a search engine to which to provide a search query |
US12/877,765 US20120060113A1 (en) | 2010-09-08 | 2010-09-08 | Methods and apparatus for displaying content |
US12/877,440 US8341142B2 (en) | 2010-09-08 | 2010-09-08 | Methods and apparatus for searching the Internet |
PCT/US2011/050669 WO2012033820A2 (en) | 2010-09-08 | 2011-09-07 | Internet search related methods and apparatus |
Publications (2)
Publication Number | Publication Date |
---|---|
EP2612261A2 EP2612261A2 (en) | 2013-07-10 |
EP2612261B1 true EP2612261B1 (en) | 2018-11-07 |
Family
ID=45811141
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
EP11758619.8A Not-in-force EP2612261B1 (en) | 2010-09-08 | 2011-09-07 | Internet search related methods and apparatus |
Country Status (5)
Country | Link |
---|---|
EP (1) | EP2612261B1 (zh) |
JP (1) | JP5851507B2 (zh) |
KR (1) | KR20130108563A (zh) |
CN (1) | CN103339623B (zh) |
WO (1) | WO2012033820A2 (zh) |
Families Citing this family (119)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US8677377B2 (en) | 2005-09-08 | 2014-03-18 | Apple Inc. | Method and apparatus for building an intelligent automated assistant |
US9318108B2 (en) | 2010-01-18 | 2016-04-19 | Apple Inc. | Intelligent automated assistant |
US8977255B2 (en) | 2007-04-03 | 2015-03-10 | Apple Inc. | Method and system for operating a multi-function portable electronic device using voice-activation |
US8676904B2 (en) | 2008-10-02 | 2014-03-18 | Apple Inc. | Electronic devices with voice command and contextual data processing capabilities |
US20120309363A1 (en) | 2011-06-03 | 2012-12-06 | Apple Inc. | Triggering notifications associated with tasks items that represent tasks to perform |
US10276170B2 (en) | 2010-01-18 | 2019-04-30 | Apple Inc. | Intelligent automated assistant |
US8682667B2 (en) | 2010-02-25 | 2014-03-25 | Apple Inc. | User profiling for selecting user specific voice input processing information |
US8769624B2 (en) | 2011-09-29 | 2014-07-01 | Apple Inc. | Access control utilizing indirect authentication |
US9002322B2 (en) | 2011-09-29 | 2015-04-07 | Apple Inc. | Authentication with secondary approver |
US10417037B2 (en) | 2012-05-15 | 2019-09-17 | Apple Inc. | Systems and methods for integrating third party services with a digital assistant |
US8589164B1 (en) * | 2012-10-18 | 2013-11-19 | Google Inc. | Methods and systems for speech recognition processing using search query information |
CN104969289B (zh) | 2013-02-07 | 2021-05-28 | 苹果公司 | 数字助理的语音触发器 |
US10652394B2 (en) | 2013-03-14 | 2020-05-12 | Apple Inc. | System and method for processing voicemail |
US10748529B1 (en) | 2013-03-15 | 2020-08-18 | Apple Inc. | Voice activated device for use with a voice-based digital assistant |
WO2014143776A2 (en) | 2013-03-15 | 2014-09-18 | Bodhi Technology Ventures Llc | Providing remote interactions with host device using a wireless device |
EP3937002A1 (en) | 2013-06-09 | 2022-01-12 | Apple Inc. | Device, method, and graphical user interface for enabling conversation persistence across two or more instances of a digital assistant |
US10176167B2 (en) | 2013-06-09 | 2019-01-08 | Apple Inc. | System and method for inferring user intent from speech inputs |
WO2015020942A1 (en) | 2013-08-06 | 2015-02-12 | Apple Inc. | Auto-activating smart responses based on activities from remote devices |
CN104516887B (zh) * | 2013-09-27 | 2019-08-30 | 腾讯科技(深圳)有限公司 | 一种网页数据搜索方法、装置和*** |
US10296160B2 (en) | 2013-12-06 | 2019-05-21 | Apple Inc. | Method for extracting salient dialog usage from live data |
CN103617278A (zh) * | 2013-12-09 | 2014-03-05 | 北京奇虎科技有限公司 | 一种地址栏搜索的控制方法及装置 |
RU2580431C2 (ru) | 2014-03-27 | 2016-04-10 | Общество С Ограниченной Ответственностью "Яндекс" | Способ и сервер для обработки поискового запроса и машиночитаемый носитель |
US9928030B2 (en) * | 2014-03-28 | 2018-03-27 | Panasonic Intellectual Property Management Co., Ltd. | Speech retrieval device, speech retrieval method, and display device |
US11343335B2 (en) | 2014-05-29 | 2022-05-24 | Apple Inc. | Message processing by subscriber app prior to message forwarding |
US9430463B2 (en) | 2014-05-30 | 2016-08-30 | Apple Inc. | Exemplar-based natural language processing |
US9967401B2 (en) | 2014-05-30 | 2018-05-08 | Apple Inc. | User interface for phone call routing among devices |
WO2015184186A1 (en) | 2014-05-30 | 2015-12-03 | Apple Inc. | Multi-command single utterance input method |
US10170123B2 (en) | 2014-05-30 | 2019-01-01 | Apple Inc. | Intelligent assistant for home automation |
US9715875B2 (en) | 2014-05-30 | 2017-07-25 | Apple Inc. | Reducing the need for manual start/end-pointing and trigger phrases |
AU2015267671B2 (en) | 2014-05-30 | 2018-04-19 | Apple Inc. | Transition from use of one device to another |
US9338493B2 (en) | 2014-06-30 | 2016-05-10 | Apple Inc. | Intelligent automated assistant for TV user interactions |
US10339293B2 (en) | 2014-08-15 | 2019-07-02 | Apple Inc. | Authenticated device used to unlock another device |
US9668121B2 (en) | 2014-09-30 | 2017-05-30 | Apple Inc. | Social reminders |
US10127911B2 (en) | 2014-09-30 | 2018-11-13 | Apple Inc. | Speaker identification and unsupervised speaker adaptation techniques |
US10152299B2 (en) | 2015-03-06 | 2018-12-11 | Apple Inc. | Reducing response latency of intelligent automated assistants |
US10567477B2 (en) * | 2015-03-08 | 2020-02-18 | Apple Inc. | Virtual assistant continuity |
US9721566B2 (en) | 2015-03-08 | 2017-08-01 | Apple Inc. | Competing devices responding to voice triggers |
US9886953B2 (en) | 2015-03-08 | 2018-02-06 | Apple Inc. | Virtual assistant activation |
US10460227B2 (en) | 2015-05-15 | 2019-10-29 | Apple Inc. | Virtual assistant in a communication session |
US10083688B2 (en) | 2015-05-27 | 2018-09-25 | Apple Inc. | Device voice control for selecting a displayed affordance |
US10200824B2 (en) | 2015-05-27 | 2019-02-05 | Apple Inc. | Systems and methods for proactively identifying and surfacing relevant content on a touch-sensitive device |
US9578173B2 (en) | 2015-06-05 | 2017-02-21 | Apple Inc. | Virtual assistant aided communication with 3rd party service in a communication session |
US20160378747A1 (en) | 2015-06-29 | 2016-12-29 | Apple Inc. | Virtual assistant for media playback |
US10671428B2 (en) | 2015-09-08 | 2020-06-02 | Apple Inc. | Distributed personal assistant |
US10331312B2 (en) | 2015-09-08 | 2019-06-25 | Apple Inc. | Intelligent automated assistant in a media environment |
US10740384B2 (en) | 2015-09-08 | 2020-08-11 | Apple Inc. | Intelligent automated assistant for media search and playback |
US10747498B2 (en) | 2015-09-08 | 2020-08-18 | Apple Inc. | Zero latency digital assistant |
US10691473B2 (en) | 2015-11-06 | 2020-06-23 | Apple Inc. | Intelligent automated assistant in a messaging environment |
US10956666B2 (en) | 2015-11-09 | 2021-03-23 | Apple Inc. | Unconventional virtual assistant interactions |
US10223066B2 (en) | 2015-12-23 | 2019-03-05 | Apple Inc. | Proactive assistance based on dialog communication between devices |
DK179186B1 (en) | 2016-05-19 | 2018-01-15 | Apple Inc | REMOTE AUTHORIZATION TO CONTINUE WITH AN ACTION |
US11227589B2 (en) | 2016-06-06 | 2022-01-18 | Apple Inc. | Intelligent list reading |
US10586535B2 (en) | 2016-06-10 | 2020-03-10 | Apple Inc. | Intelligent digital assistant in a multi-tasking environment |
DK201670540A1 (en) | 2016-06-11 | 2018-01-08 | Apple Inc | Application integration with a digital assistant |
DK179415B1 (en) | 2016-06-11 | 2018-06-14 | Apple Inc | Intelligent device arbitration and control |
DK201670622A1 (en) | 2016-06-12 | 2018-02-12 | Apple Inc | User interfaces for transactions |
US10474753B2 (en) | 2016-09-07 | 2019-11-12 | Apple Inc. | Language identification using recurrent neural networks |
CN106649864A (zh) * | 2016-12-31 | 2017-05-10 | 珠海市魅族科技有限公司 | 一种信息的展示方法及装置 |
US11204787B2 (en) | 2017-01-09 | 2021-12-21 | Apple Inc. | Application integration with a digital assistant |
US10992795B2 (en) | 2017-05-16 | 2021-04-27 | Apple Inc. | Methods and interfaces for home media control |
US11431836B2 (en) | 2017-05-02 | 2022-08-30 | Apple Inc. | Methods and interfaces for initiating media playback |
DK201770383A1 (en) | 2017-05-09 | 2018-12-14 | Apple Inc. | USER INTERFACE FOR CORRECTING RECOGNITION ERRORS |
US10395654B2 (en) | 2017-05-11 | 2019-08-27 | Apple Inc. | Text normalization based on a data-driven learning network |
DK180048B1 (en) | 2017-05-11 | 2020-02-04 | Apple Inc. | MAINTAINING THE DATA PROTECTION OF PERSONAL INFORMATION |
US10726832B2 (en) | 2017-05-11 | 2020-07-28 | Apple Inc. | Maintaining privacy of personal information |
DK179496B1 (en) | 2017-05-12 | 2019-01-15 | Apple Inc. | USER-SPECIFIC Acoustic Models |
US11301477B2 (en) | 2017-05-12 | 2022-04-12 | Apple Inc. | Feedback analysis of a digital assistant |
DK201770429A1 (en) | 2017-05-12 | 2018-12-14 | Apple Inc. | LOW-LATENCY INTELLIGENT AUTOMATED ASSISTANT |
DK179745B1 (en) | 2017-05-12 | 2019-05-01 | Apple Inc. | SYNCHRONIZATION AND TASK DELEGATION OF A DIGITAL ASSISTANT |
DK201770411A1 (en) | 2017-05-15 | 2018-12-20 | Apple Inc. | MULTI-MODAL INTERFACES |
US20180336275A1 (en) | 2017-05-16 | 2018-11-22 | Apple Inc. | Intelligent automated assistant for media exploration |
DK179560B1 (en) | 2017-05-16 | 2019-02-18 | Apple Inc. | FAR-FIELD EXTENSION FOR DIGITAL ASSISTANT SERVICES |
CN111343060B (zh) | 2017-05-16 | 2022-02-11 | 苹果公司 | 用于家庭媒体控制的方法和界面 |
US10311144B2 (en) | 2017-05-16 | 2019-06-04 | Apple Inc. | Emoji word sense disambiguation |
US20220279063A1 (en) | 2017-05-16 | 2022-09-01 | Apple Inc. | Methods and interfaces for home media control |
US20180336892A1 (en) | 2017-05-16 | 2018-11-22 | Apple Inc. | Detecting a trigger of a digital assistant |
CN110741366B (zh) * | 2017-10-03 | 2023-12-01 | 谷歌有限责任公司 | 跨越多个设备协调音频查询的并行处理 |
JP6904225B2 (ja) * | 2017-11-15 | 2021-07-14 | トヨタ自動車株式会社 | 情報処理装置 |
US10592604B2 (en) | 2018-03-12 | 2020-03-17 | Apple Inc. | Inverse text normalization for automatic speech recognition |
US10818288B2 (en) | 2018-03-26 | 2020-10-27 | Apple Inc. | Natural assistant interaction |
US10928918B2 (en) | 2018-05-07 | 2021-02-23 | Apple Inc. | Raise to speak |
US11145294B2 (en) | 2018-05-07 | 2021-10-12 | Apple Inc. | Intelligent automated assistant for delivering content from user experiences |
US10892996B2 (en) | 2018-06-01 | 2021-01-12 | Apple Inc. | Variable latency device coordination |
DK179822B1 (da) | 2018-06-01 | 2019-07-12 | Apple Inc. | Voice interaction at a primary device to access call functionality of a companion device |
DK180639B1 (en) | 2018-06-01 | 2021-11-04 | Apple Inc | DISABILITY OF ATTENTION-ATTENTIVE VIRTUAL ASSISTANT |
US11010561B2 (en) | 2018-09-27 | 2021-05-18 | Apple Inc. | Sentiment prediction from textual data |
US11170166B2 (en) | 2018-09-28 | 2021-11-09 | Apple Inc. | Neural typographical error modeling via generative adversarial networks |
US10839159B2 (en) | 2018-09-28 | 2020-11-17 | Apple Inc. | Named entity normalization in a spoken dialog system |
US11462215B2 (en) | 2018-09-28 | 2022-10-04 | Apple Inc. | Multi-modal inputs for voice commands |
US11475898B2 (en) | 2018-10-26 | 2022-10-18 | Apple Inc. | Low-latency multi-speaker speech recognition |
KR102214990B1 (ko) * | 2018-11-26 | 2021-02-15 | 김준 | 북마크관리 및 정보검색 서비스 제공시스템 및 이를 이용한 북마크관리 및 정보검색 서비스 제공방법 |
JP6555838B1 (ja) * | 2018-12-19 | 2019-08-07 | Jeインターナショナル株式会社 | 音声問合せシステム、音声問合せ処理方法、スマートスピーカー運用サーバー装置、チャットボットポータルサーバー装置、およびプログラム。 |
US11638059B2 (en) | 2019-01-04 | 2023-04-25 | Apple Inc. | Content playback on multiple devices |
US11348573B2 (en) | 2019-03-18 | 2022-05-31 | Apple Inc. | Multimodality in digital assistant systems |
US11307752B2 (en) | 2019-05-06 | 2022-04-19 | Apple Inc. | User configurable task triggers |
DK201970509A1 (en) | 2019-05-06 | 2021-01-15 | Apple Inc | Spoken notifications |
US11423908B2 (en) | 2019-05-06 | 2022-08-23 | Apple Inc. | Interpreting spoken requests |
US11475884B2 (en) | 2019-05-06 | 2022-10-18 | Apple Inc. | Reducing digital assistant latency when a language is incorrectly determined |
US11140099B2 (en) | 2019-05-21 | 2021-10-05 | Apple Inc. | Providing message response suggestions |
DK180129B1 (en) | 2019-05-31 | 2020-06-02 | Apple Inc. | USER ACTIVITY SHORTCUT SUGGESTIONS |
US11010121B2 (en) | 2019-05-31 | 2021-05-18 | Apple Inc. | User interfaces for audio media control |
US11289073B2 (en) | 2019-05-31 | 2022-03-29 | Apple Inc. | Device text to speech |
EP4134811A1 (en) | 2019-05-31 | 2023-02-15 | Apple Inc. | User interfaces for audio media control |
DK201970510A1 (en) | 2019-05-31 | 2021-02-11 | Apple Inc | Voice identification in digital assistant systems |
US11496600B2 (en) | 2019-05-31 | 2022-11-08 | Apple Inc. | Remote execution of machine-learned models |
US11360641B2 (en) | 2019-06-01 | 2022-06-14 | Apple Inc. | Increasing the relevance of new available information |
US11477609B2 (en) | 2019-06-01 | 2022-10-18 | Apple Inc. | User interfaces for location-related communications |
US11481094B2 (en) | 2019-06-01 | 2022-10-25 | Apple Inc. | User interfaces for location-related communications |
US11227599B2 (en) | 2019-06-01 | 2022-01-18 | Apple Inc. | Methods and user interfaces for voice-based control of electronic devices |
US11488406B2 (en) | 2019-09-25 | 2022-11-01 | Apple Inc. | Text detection using global geometry estimators |
CN111343506B (zh) * | 2020-02-12 | 2022-11-29 | 咪咕文化科技有限公司 | 一种视频搜索方法、装置、***、服务器和可读存储介质 |
US11183193B1 (en) | 2020-05-11 | 2021-11-23 | Apple Inc. | Digital assistant hardware abstraction |
US11061543B1 (en) | 2020-05-11 | 2021-07-13 | Apple Inc. | Providing relevant data items based on context |
US11490204B2 (en) | 2020-07-20 | 2022-11-01 | Apple Inc. | Multi-device audio adjustment coordination |
US11438683B2 (en) | 2020-07-21 | 2022-09-06 | Apple Inc. | User identification using headphones |
US11392291B2 (en) | 2020-09-25 | 2022-07-19 | Apple Inc. | Methods and interfaces for media control with dynamic feedback |
US11847378B2 (en) | 2021-06-06 | 2023-12-19 | Apple Inc. | User interfaces for audio routing |
US12039471B2 (en) | 2021-11-29 | 2024-07-16 | T-Mobile Usa, Inc. | Tracking issues and resolution of same in a wireless communication network |
US11962455B2 (en) | 2021-11-29 | 2024-04-16 | T-Mobile Usa, Inc. | Prioritizing multiple issues associated with a wireless telecommunication network |
Family Cites Families (12)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN1176432C (zh) * | 1999-07-28 | 2004-11-17 | 国际商业机器公司 | 提供本国语言查询服务的方法和*** |
JP4325075B2 (ja) * | 2000-04-21 | 2009-09-02 | ソニー株式会社 | データオブジェクト管理装置 |
US7502774B2 (en) * | 2004-12-09 | 2009-03-10 | International Business Machines Corporation | Ring method, apparatus, and computer program product for managing federated search results in a heterogeneous environment |
US7921091B2 (en) * | 2004-12-16 | 2011-04-05 | At&T Intellectual Property Ii, L.P. | System and method for providing a natural language interface to a database |
CN100375094C (zh) * | 2005-08-23 | 2008-03-12 | 萧学文 | 实现移动终端搜索网络资源的***及方法 |
US8065286B2 (en) * | 2006-01-23 | 2011-11-22 | Chacha Search, Inc. | Scalable search system using human searchers |
JP4762063B2 (ja) * | 2006-06-29 | 2011-08-31 | 株式会社日立製作所 | 計算機システム、プログラム及び検索エンジン検索方法 |
US7930286B2 (en) * | 2007-02-16 | 2011-04-19 | Yahoo! Inc. | Federated searches implemented across multiple search engines |
CN101505328A (zh) * | 2008-02-04 | 2009-08-12 | 台达电子工业股份有限公司 | 应用语音识别的网络数据检索方法及其*** |
US8364481B2 (en) * | 2008-07-02 | 2013-01-29 | Google Inc. | Speech recognition with parallel recognition tasks |
CN101820592A (zh) * | 2009-02-27 | 2010-09-01 | 华为技术有限公司 | 移动搜索方法及装置 |
US9684741B2 (en) * | 2009-06-05 | 2017-06-20 | Microsoft Technology Licensing, Llc | Presenting search results according to query domains |
-
2011
- 2011-09-07 EP EP11758619.8A patent/EP2612261B1/en not_active Not-in-force
- 2011-09-07 WO PCT/US2011/050669 patent/WO2012033820A2/en active Application Filing
- 2011-09-07 CN CN201180043219.4A patent/CN103339623B/zh active Active
- 2011-09-07 KR KR1020137007487A patent/KR20130108563A/ko not_active Application Discontinuation
- 2011-09-07 JP JP2013528267A patent/JP5851507B2/ja active Active
Non-Patent Citations (1)
Title |
---|
None * |
Also Published As
Publication number | Publication date |
---|---|
WO2012033820A3 (en) | 2013-01-31 |
CN103339623A (zh) | 2013-10-02 |
WO2012033820A2 (en) | 2012-03-15 |
KR20130108563A (ko) | 2013-10-04 |
CN103339623B (zh) | 2018-05-25 |
JP5851507B2 (ja) | 2016-02-03 |
EP2612261A2 (en) | 2013-07-10 |
JP2013537332A (ja) | 2013-09-30 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
EP2612261B1 (en) | Internet search related methods and apparatus | |
US8745025B2 (en) | Methods and apparatus for searching the Internet | |
US8666963B2 (en) | Method and apparatus for processing spoken search queries | |
US20120060113A1 (en) | Methods and apparatus for displaying content | |
US20120059658A1 (en) | Methods and apparatus for performing an internet search | |
US20120059814A1 (en) | Methods and apparatus for selecting a search engine to which to provide a search query | |
US9489457B2 (en) | Methods and apparatus for initiating an action | |
US8812474B2 (en) | Methods and apparatus for identifying and providing information sought by a user | |
US9292603B2 (en) | Receipt and processing of user-specified queries | |
US8635201B2 (en) | Methods and apparatus for employing a user's location in providing information to the user | |
US20160055245A1 (en) | Systems and methods for providing information discovery and retrieval | |
US20130086028A1 (en) | Receiving and processing user-specified queries | |
US20130018864A1 (en) | Methods and apparatus for identifying and providing information of various types to a user | |
US9916396B2 (en) | Methods and systems for content-based search | |
JP2008287697A (ja) | 音声チャットシステム、情報処理装置およびプログラム | |
US20130086027A1 (en) | Techniques for the receipt and processing of user-specified queries | |
US20130086025A1 (en) | Techniques for receiving and processing one or more user-specified queries | |
US20130086026A1 (en) | Techniques relating to receiving and processing user-specified queries | |
CN109829117A (zh) | 用于推送信息的方法和装置 | |
WO2013009938A2 (en) | Methods and apparatus for identifying and providing information sought by a user | |
WO2013048729A1 (en) | Techniques for receiving and processing one or more user-specified queries |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PUAI | Public reference made under article 153(3) epc to a published international application that has entered the european phase |
Free format text: ORIGINAL CODE: 0009012 |
|
17P | Request for examination filed |
Effective date: 20130405 |
|
AK | Designated contracting states |
Kind code of ref document: A2 Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR |
|
RIN1 | Information on inventor provided before grant (corrected) |
Inventor name: BODENSTAB, NATHAN, M. Inventor name: EVERMANN, GUNNAR Inventor name: VOZILA, PAUL, J. Inventor name: CLAYTON, GARY, B. Inventor name: REGAN, MARC, W. Inventor name: GANONG, WILLIAM, F. Inventor name: CHEN, VICTOR, S. Inventor name: SEJNOHA, VLADIMIR Inventor name: HATCH, STEVEN Inventor name: TAM, YIK-CHEUNG Inventor name: LAVERTY, STEPHEN, W. |
|
DAX | Request for extension of the european patent (deleted) | ||
GRAP | Despatch of communication of intention to grant a patent |
Free format text: ORIGINAL CODE: EPIDOSNIGR1 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: GRANT OF PATENT IS INTENDED |
|
INTG | Intention to grant announced |
Effective date: 20180410 |
|
RIN1 | Information on inventor provided before grant (corrected) |
Inventor name: SEJNOHA, VLADIMIR Inventor name: LAVERTY, STEPHEN, W. Inventor name: CLAYTON, GARY, B. Inventor name: BODENSTAB, NATHAN, M. Inventor name: REGAN, MARC, W. Inventor name: HATCH, STEVEN Inventor name: TAM, YIK-CHEUNG Inventor name: VOZILA, PAUL, J. Inventor name: GANONG, WILLIAM, F. Inventor name: CHEN, VICTOR, S. Inventor name: EVERMANN, GUNNAR |
|
GRAS | Grant fee paid |
Free format text: ORIGINAL CODE: EPIDOSNIGR3 |
|
GRAA | (expected) grant |
Free format text: ORIGINAL CODE: 0009210 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: THE PATENT HAS BEEN GRANTED |
|
AK | Designated contracting states |
Kind code of ref document: B1 Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR |
|
REG | Reference to a national code |
Ref country code: GB Ref legal event code: FG4D |
|
REG | Reference to a national code |
Ref country code: CH Ref legal event code: EP Ref country code: AT Ref legal event code: REF Ref document number: 1062930 Country of ref document: AT Kind code of ref document: T Effective date: 20181115 |
|
REG | Reference to a national code |
Ref country code: DE Ref legal event code: R079 Ref document number: 602011053640 Country of ref document: DE Free format text: PREVIOUS MAIN CLASS: G06F0017300000 Ipc: G06F0016000000 |
|
REG | Reference to a national code |
Ref country code: IE Ref legal event code: FG4D |
|
REG | Reference to a national code |
Ref country code: DE Ref legal event code: R096 Ref document number: 602011053640 Country of ref document: DE |
|
REG | Reference to a national code |
Ref country code: NL Ref legal event code: MP Effective date: 20181107 |
|
REG | Reference to a national code |
Ref country code: LT Ref legal event code: MG4D |
|
REG | Reference to a national code |
Ref country code: AT Ref legal event code: MK05 Ref document number: 1062930 Country of ref document: AT Kind code of ref document: T Effective date: 20181107 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: FI Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20181107 Ref country code: HR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20181107 Ref country code: AT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20181107 Ref country code: LV Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20181107 Ref country code: NO Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20190207 Ref country code: ES Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20181107 Ref country code: IS Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20190307 Ref country code: BG Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20190207 Ref country code: LT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20181107 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: AL Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20181107 Ref country code: PT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20190307 Ref country code: SE Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20181107 Ref country code: NL Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20181107 Ref country code: GR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20190208 Ref country code: RS Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20181107 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: CZ Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20181107 Ref country code: PL Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20181107 Ref country code: IT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20181107 Ref country code: DK Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20181107 |
|
REG | Reference to a national code |
Ref country code: DE Ref legal event code: R097 Ref document number: 602011053640 Country of ref document: DE |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: SM Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20181107 Ref country code: EE Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20181107 Ref country code: SK Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20181107 Ref country code: RO Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20181107 |
|
PLBE | No opposition filed within time limit |
Free format text: ORIGINAL CODE: 0009261 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT |
|
26N | No opposition filed |
Effective date: 20190808 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: SI Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20181107 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: TR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20181107 |
|
REG | Reference to a national code |
Ref country code: DE Ref legal event code: R119 Ref document number: 602011053640 Country of ref document: DE |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: MC Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20181107 |
|
REG | Reference to a national code |
Ref country code: CH Ref legal event code: PL |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: LU Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20190907 Ref country code: LI Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20190930 Ref country code: DE Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20200401 Ref country code: IE Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20190907 Ref country code: CH Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20190930 |
|
REG | Reference to a national code |
Ref country code: BE Ref legal event code: MM Effective date: 20190930 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: BE Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20190930 |
|
GBPC | Gb: european patent ceased through non-payment of renewal fee |
Effective date: 20190907 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: FR Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20190930 Ref country code: GB Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20190907 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: CY Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20181107 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: HU Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT; INVALID AB INITIO Effective date: 20110907 Ref country code: MT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20181107 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: MK Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20181107 |
|
P01 | Opt-out of the competence of the unified patent court (upc) registered |
Effective date: 20230510 |