CN114287113A - Targeted fingerprinting of radio broadcast audio - Google Patents

Targeted fingerprinting of radio broadcast audio Download PDF

Info

Publication number
CN114287113A
CN114287113A CN201980099769.4A CN201980099769A CN114287113A CN 114287113 A CN114287113 A CN 114287113A CN 201980099769 A CN201980099769 A CN 201980099769A CN 114287113 A CN114287113 A CN 114287113A
Authority
CN
China
Prior art keywords
radio
audio
metadata
radio receiver
radio broadcast
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201980099769.4A
Other languages
Chinese (zh)
Inventor
P·维尼智亚
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Ibiquity Digital Corp
Original Assignee
Ibiquity Digital Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Ibiquity Digital Corp filed Critical Ibiquity Digital Corp
Publication of CN114287113A publication Critical patent/CN114287113A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04HBROADCAST COMMUNICATION
    • H04H60/00Arrangements for broadcast applications with a direct linking to broadcast information or broadcast space-time; Broadcast-related systems
    • H04H60/68Systems specially adapted for using specific information, e.g. geographical or meteorological information
    • H04H60/73Systems specially adapted for using specific information, e.g. geographical or meteorological information using meta-information
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04HBROADCAST COMMUNICATION
    • H04H60/00Arrangements for broadcast applications with a direct linking to broadcast information or broadcast space-time; Broadcast-related systems
    • H04H60/35Arrangements for identifying or recognising characteristics with a direct linkage to broadcast information or to broadcast space-time, e.g. for identifying broadcast stations or for identifying users
    • H04H60/38Arrangements for identifying or recognising characteristics with a direct linkage to broadcast information or to broadcast space-time, e.g. for identifying broadcast stations or for identifying users for identifying broadcast time or space
    • H04H60/41Arrangements for identifying or recognising characteristics with a direct linkage to broadcast information or to broadcast space-time, e.g. for identifying broadcast stations or for identifying users for identifying broadcast time or space for identifying broadcast space, i.e. broadcast channels, broadcast stations or broadcast areas
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04HBROADCAST COMMUNICATION
    • H04H60/00Arrangements for broadcast applications with a direct linking to broadcast information or broadcast space-time; Broadcast-related systems
    • H04H60/35Arrangements for identifying or recognising characteristics with a direct linkage to broadcast information or to broadcast space-time, e.g. for identifying broadcast stations or for identifying users
    • H04H60/49Arrangements for identifying or recognising characteristics with a direct linkage to broadcast information or to broadcast space-time, e.g. for identifying broadcast stations or for identifying users for identifying locations
    • H04H60/51Arrangements for identifying or recognising characteristics with a direct linkage to broadcast information or to broadcast space-time, e.g. for identifying broadcast stations or for identifying users for identifying locations of receiving stations
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04HBROADCAST COMMUNICATION
    • H04H60/00Arrangements for broadcast applications with a direct linking to broadcast information or broadcast space-time; Broadcast-related systems
    • H04H60/76Arrangements characterised by transmission systems other than for broadcast, e.g. the Internet
    • H04H60/81Arrangements characterised by transmission systems other than for broadcast, e.g. the Internet characterised by the transmission system itself
    • H04H60/82Arrangements characterised by transmission systems other than for broadcast, e.g. the Internet characterised by the transmission system itself the transmission system being the Internet
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04HBROADCAST COMMUNICATION
    • H04H2201/00Aspects of broadcast communication
    • H04H2201/90Aspects of broadcast communication characterised by the use of signatures

Landscapes

  • Engineering & Computer Science (AREA)
  • Signal Processing (AREA)
  • Circuits Of Receivers In General (AREA)

Abstract

A system comprising: an intermediate communication platform providing an interface to an internet network; and a first server, the first server comprising: a port operatively coupled to the intermediate communication platform, processing circuitry, and a service application for execution by a processor. The service application is configured to: receiving geographic location information of a radio receiver via the intermediate communication platform; determining one or more radio broadcasts available to the radio receiver from the geographical location information; and sending metadata of the radio broadcast to the radio receiver via the intermediate communication platform, the metadata including an indication of whether content of the radio broadcast is suitable for an audio fingerprinting process.

Description

Targeted fingerprinting of radio broadcast audio
Technical Field
The technology described in this patent document relates to systems and methods for providing supplemental data (e.g., metadata) associated with an over-the-air radio broadcast signal.
Background
Over-the-air radio broadcast signals are used to deliver a wide variety of programming content (e.g., audio, etc.) to radio receiver systems. Such over-the-air radio broadcast signals may include conventional AM (amplitude modulation) and FM (frequency modulation) analog broadcast signals, digital radio broadcast signals, mixed analog and digital broadcast signals, or other broadcast signals. Hybrid radio broadcast technology and digital radio broadcast technology can deliver audio and data services to mobile, portable, and fixed receivers.
Service data including multimedia programming may be included with the radio broadcast. The broadcast of service data may be contracted by a company to include multimedia content associated with primary or main radio program content. However, the service data may not always be available with the radio broadcast. In such a case, it may be desirable to identify the audio content being broadcast and match the service data with the audio content. Some current broadcast radio content information systems rely on digital "fingerprinting" of audio content. However, the audio fingerprinting process may consume a large amount of memory space of the radio receiver, and the identification service for the fingerprinting process may be costly and expensive in terms of resources (such as memory resources and processing resources of the radio receiver).
Disclosure of Invention
This summary is provided to introduce a selection of concepts in a simplified form that are further described below in the detailed description. This summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter.
In general, embodiments of radio broadcast metadata distribution systems and methods determine whether received audio is suitable for fingerprinting before fingerprinting is used to obtain audio metadata related to a radio broadcast. An example radio system includes: an intermediate communication platform providing an interface to an internet network; and a first server, the first server comprising: a port operatively coupled to the intermediate communication platform, processing circuitry, and a service application for execution by the processing circuitry. The service application is configured to: receiving geographic location information of a radio receiver via the intermediate communication platform; determining one or more radio broadcasts available to the radio receiver from the geographical location information; and sending metadata of the radio broadcast to the radio receiver via the intermediate communication platform, the metadata including an indication of whether content of the radio broadcast is suitable for an audio fingerprinting process.
An example radio receiver includes Radio Frequency (RF) receiver circuitry, an internet network interface, a display, processing circuitry, and a client application including instructions for execution by the processing circuitry. The RF circuitry is configured to receive a radio broadcast signal. The client application is configured to: sending geographic location information to an audio metadata service application via the internet network interface; and receiving, via the internet network interface, metadata of a radio broadcast available to the radio receiver, the metadata including an indication of whether content of the radio broadcast is suitable for an audio fingerprinting process.
It should be noted that alternative embodiments are possible, and that the steps and elements discussed herein may be changed, added, or eliminated depending on the particular embodiment. These alternative embodiments include structural changes that may be made, as well as alternative steps and alternative elements that may be used, without departing from the scope of the invention.
Drawings
In the drawings, which are not necessarily drawn to scale, like numerals may describe similar components in different views. Like numerals having different letter suffixes may represent different instances of similar components. The accompanying drawings illustrate generally, by way of example, but not by way of limitation, various embodiments discussed in this document.
Fig. 1 is a block diagram illustrating an overview of an embodiment of a radio system.
Fig. 2 is a block diagram of an example of a server providing internet protocol streams to a radio receiver.
Fig. 3 is a flow chart of an example of a method of distributing metadata to a radio receiver.
Fig. 4 is a block diagram of portions of an example of a radio receiver.
Detailed Description
In the following description of embodiments of the radio broadcast metadata distribution system, reference is made to the accompanying drawings. These drawings show by way of illustration specific examples of how embodiments of the metadata distribution system may be implemented. It is to be understood that other embodiments may be utilized and structural changes may be made without departing from the scope of the claimed subject matter.
Over-the-air radio broadcast signals are commonly used to deliver a wide variety of programming content (e.g., audio, etc.) to radio receiver systems. Main Program Service (MPS) data and Supplemental Program Service (SPS) data may be provided to a radio broadcast receiver system. Metadata associated with the programming content may be delivered in MPS data or SPS data via over-the-air radio signals. The metadata may be included in a subcarrier of the primary radio signal. In IBOC radio, the radio broadcast may be a hybrid radio signal, which may include streaming analog broadcast and digital audio broadcast. The subcarriers broadcast by the main channel may include digital information such as text or numerical information, and metadata may be included in the digital information of the subcarriers. Thus, a hybrid over-the-air radio broadcast may include analog audio broadcasts, digital audio broadcasts, and other textual and numerical digital information (such as metadata streamed with the over-the-air broadcast). The programming content may be broadcast according to a DAB standard, a world Digital Radio (DRM) standard, a Radio Data System (RDS) protocol, a Radio Broadcast Data System (RBDS) protocol, or a High Definition (HD) IBOC radio protocol.
The metadata may include both "static" metadata and "dynamic" metadata. Static metadata does not change frequently or does not change. The static metadata may include the call sign, name, logo (e.g., higher or lower logo resolution), slogan, station format, station genre, language, web page Uniform Resource Locator (URL), URL of social media (e.g., Facebook, Twitter), phone number, Short Message Service (SMS) number, SMS short code, Program Identification (PI) code, country, or other information of the radio station.
Dynamic metadata changes relatively frequently. Dynamic metadata may include song title, artist name, album image, artist image (e.g., related to content currently playing on the broadcast), advertisements, enhanced advertisements (e.g., title, slogan, image, phone number, SMS number, URL, search terms), programming (image, schedule, title, artist name, DJ name, phone number, URL), service tracking data, or other information. When a radio receiver system is receiving an over-the-air radio broadcast signal from a particular radio station, the receiver system may receive both static metadata and dynamic metadata.
Another approach to providing service data is to combine over-the-air (OTA) radio information broadcasting with Internet Protocol (IP) delivered content to provide an enhanced user experience. An example of this type of service is
Figure BDA0003520340670000041
Connected RadioTMService, the
Figure BDA0003520340670000042
Connected RadioTMThe service combines OTA analog/digital AM/FM radio with IP delivered content. The combined service receives dynamic metadata (such as artist information and song title, over-the-air radio program information, and station contact information) directly from the local radio broadcaster, which is then paired with IP-delivered content (e.g., metadata) and displayed in the vehicle. The DTS Connected Radio service supports all global broadcast standards including analog, DAB + and HD RadioTM. The vehicle's radio receiver integrates data from internet services with broadcast audio to create a rich media experience. One of the internet services provided is with respect to radio stationsInformation of what is currently playing and what has been played.
As explained previously herein, service data may not always be available with a radio broadcast, and it may be desirable to send an audio fingerprint to identify the content of an over-the-air radio broadcast and receive metadata of the identified radio broadcast for presentation to a user. The audio fingerprint may be transmitted from the in-vehicle radio receiver to a server that performs Automatic Content Recognition (ACR) to identify the content of the over-the-air radio broadcast.
However, digital fingerprinting and identification of audio fingerprints can be expensive both in terms of cost of service and in terms of resources (such as processor and memory usage) of the radio receiver. These resources can be wasted if the broadcast audio being received by the radio receiver is not suitable for the ACR fingerprinting and identification process. For example, if the broadcast audio does not include music, the ACR process may fail. This may occur if the broadcast audio contains content of a talk show or commercial. An improvement would be to verify that the broadcast audio is suitable for ACR fingerprinting and identification before the process starts.
Fig. 1 is a block diagram illustrating an overview of an embodiment of a radio system. A conventional broadcast radio station 100 transmits an OTA audio signal 105 to a radio receiver 110 of the vehicle. The OTA audio signal 105 can be an analog audio signal, a digital audio signal, or a mixed audio signal. The radio service provided is a combined OTA-IP radio service and the car radio receiver 110 can receive the OTA audio signal 105 and the IP stream. The IP streams are received from one or more servers 120 via the intermediate communication platform 108. The intermediate communication platform 108 may be a cellular telephone network or a telematics network.
Fig. 2 is a block diagram of an example of a radio system server providing IP streams to radio receivers. The server 220 includes processing circuitry 272, memory 274, and a service application 276 or Application Program Interface (API) for execution by the processor 272. Service application 276 may include software that operates using operating system software of server 220. The server 220 includes a port 270 operatively coupled to an interface of the intermediate communication platform 208 that provides IP streaming and receives information from the radio receiver.
Returning to fig. 1, in order to receive metadata of radio broadcasting, the in-vehicle radio receiver 110 transmits geographical location information to a service application of the server 120. One or more of the servers determine radio broadcasts available to the radio receivers from the geographical location information, and the service application sends metadata of the determined radio broadcasts to the radio receivers via the intermediate communication platform. The one or more servers also transmit an indication of the suitability of the content of the radio broadcast for the fingerprinting process. The indication may be included with the metadata (e.g., included in a specified field of the metadata). The indication is that the content is suitable or unsuitable for audio fingerprinting. An indication of suitability is transmitted for each of those radio broadcasts determined to be available to the radio receiver. The indication of suitability may be a flag or a digital codeword stored in memory for radio broadcasting. In a variant, an indication of the suitability of the fingerprinting process may be stored in association with metadata of the radio broadcast.
If the vehicle's radio receiver 110 subsequently tunes to a radio broadcast for which the radio receiver 110 does not have metadata (e.g., dynamic metadata), the radio receiver 110 checks for an indication of the suitability of the content of the radio broadcast for fingerprinting processing. If the indication for the radio broadcast is that the content is eligible for fingerprinting, the radio receiver generates an audio fingerprint and sends the audio fingerprint 130 to the service application. The service application determines audio metadata of the radio broadcast corresponding to the audio fingerprint and transmits the determined audio metadata 125 to the radio receiver 110. The service application may track the radio broadcast tuned to by the plurality of radio receivers. In response to determining the audio metadata from the audio fingerprint provided by the first radio receiver, the service application may transmit the audio metadata to a plurality of radio receivers that are receiving the radio broadcast. The audio metadata may be transmitted using the intermediate communication platform 108.
If the indication for the radio broadcast is that the content does not fit into the fingerprinting process, the radio receiver does not generate an audio fingerprint. The radio receiver 110 may do nothing or may simply send an acknowledgement back via the intermediate communication platform 108 according to the communication protocol. The radio receiver 110 may display the available metadata (e.g., static metadata). The radio receiver's resources are not wasted by generating and transmitting digital fingerprints that would fail the ACR fingerprinting process.
According to some embodiments, to determine audio metadata corresponding to a received audio fingerprint, the server 120 includes a memory that may store a fingerprint database that stores audio metadata in association with audio fingerprint information in the server memory. The service application determines the audio metadata by retrieving the audio metadata from memory using the audio fingerprint.
In some embodiments, the service application receives audio metadata from audio recognition source 150. The service application of the first server 120 receives the audio fingerprint 130 from the radio receiver 110 and forwards the audio fingerprint 140 to the audio identification source 150. The first server and the audio identification source may communicate using a communication network. The communication network may be the intermediate communication platform 108 or another communication network. As shown in the example of fig. 2, the server 220 may include a second port 260 operatively coupled to the internet network interface 215. In certain embodiments, the internet network interface 215 comprises an internet access point (e.g., a modem), and the port 260 may comprise, among other options, a Communications (COMM) port or a Universal Serial Bus (USB) port.
Audio recognition source 150 is shown in fig. 1 as residing in the cloud. The term "cloud" is used herein to refer to a hardware abstraction. Instead of a dedicated server processing the audio fingerprint and returning the audio metadata, sending the audio fingerprint to the cloud may include sending the audio fingerprint to a data center or processing center. The actual servers used to process the audio file content information are interchangeable at the data center or processing center. The audio recognition source 150 may include a second server that includes a fingerprint database. The audio identification source 150 receives the audio fingerprint 140 forwarded from the first server and returns audio metadata 160 to the first server, and may send other associated metadata to the first server 120. The first server 120 sends the audio metadata 125 to the radio receiver 110.
Fig. 3 is a flow chart of an example of a method 300 of providing metadata to a radio receiver. The method involves a vehicle contacting a Radio system (e.g., a DTS Connected Radio system) that combines OTA Radio broadcasting with IP delivered content. The method may be performed using a client program of a radio receiver of the vehicle. At 305, the in-vehicle radio receiver tunes to the radio station, and the radio receiver lacks dynamic metadata associated with the program currently being broadcast by the radio station. At 310, the radio receiver has queried the radio service system for static radio station metadata for the radio broadcast. The radio service system may include an Application Program Interface (API) or service application executing on a server of the radio service system.
The radio receiver may send the geographical location information to the radio system along with the query, and the API sends static metadata of any radio broadcast available for reception by the radio receiver. The API also includes in the metadata an indication of whether the content of the radio broadcast is suitable for audio fingerprinting and identification for the radio broadcast.
At 315, the radio receiver has received from the API an indication that ACR fingerprinting and identification is appropriate for the radio broadcast lacking dynamic metadata. As a result of the indication, the radio receiver generates an audio fingerprint and sends the audio fingerprint to the API at 320. The ACR fingerprinting process may be performed by a client program or by other software of the radio receiver.
The radio receiver transmits the audio fingerprint to the radio system using an internet connection. The radio system identifies an audio program corresponding to the audio fingerprint and determines audio metadata associated with the audio program. At 325, the API distributes the dynamic metadata to the radio receiver and may distribute the dynamic metadata to other radio receivers that are receiving the radio broadcast (e.g., via an intermediate communication platform). In some embodiments, dynamic metadata is distributed to radio receivers capable of receiving radio broadcasts regardless of the radio station to which the receiver is tuned.
In some embodiments, the radio receiver may store an audio fingerprint database, and before transmitting the audio fingerprint to the radio system, the radio receiver may first determine whether it is storing the missing dynamic metadata locally at 330. For example, the radio receiver may store the results of previous audio fingerprinting processes and may check a database to see if the corresponding audio program was previously identified. If the audio fingerprint is found in the local database, the radio receiver uses the corresponding dynamic metadata for the radio broadcast. This reduces the resources consumed at the radio receiver when determining the audio metadata.
At 340, in response to the query for the radio broadcast, the radio receiver receives an indication from the API that the content of the radio broadcast is unsuitable for ACR fingerprinting. In this case, the client program of the radio receiver does not spend any resources of the radio receiver, such as processing bandwidth, memory space, or communication time, in generating and transmitting the digital fingerprint of the radio broadcast at 345. The radio receiver may display the available metadata (e.g., static metadata).
Fig. 4 is a block diagram of portions of an example of a radio receiver 400. The radio receiver is capable of receiving OTA radio broadcasts and is capable of receiving IP delivered content. In certain variations, the Radio receiver is a DTS Connected Radio receiver. The radio receiver 400 may be the radio receiver 110 of the vehicle shown in the example of fig. 1. The radio receiver 400 includes a wireless internet network interface 440 for receiving metadata via wireless IP and other components for receiving over-the-air radio broadcast signals. The internet network interface 440 and the receiver controller 430 may be collectively referred to as a wireless internet protocol hardware communication module of the radio receiver.
The radio receiver 400 includes a Radio Frequency (RF) receiver circuit including a tuner 456 having an input 452 connected to an antenna 454. The antenna 454, tuner 456 and baseband processor 451 may be collectively referred to as an over-the-air radio broadcast hardware communication module of the radio receiver. The RF circuit is configured to receive an audio broadcast signal.
Within the baseband processor 451, the intermediate frequency signal 457 from the tuner 456 is provided to an analog-to-digital converter and digital down-converter 458 to produce a baseband signal comprising a series of complex signal samples at an output 460. The signal samples are complex in that each sample includes a "real" component and an "imaginary" component. An analog demodulator 462 demodulates the analog modulated portion of the baseband signal to produce an analog audio signal on line 464. The digitally modulated portion of the sampled baseband signal is filtered by an isolation filter 466, the isolation filter 466 having a filter function that includes the sub-carriers f present in the received OFDM signal1-fnThe passband frequency response of the total set. A First Adjacent Canceller (FAC)468 suppresses the effects of the first adjacent interference. Complex signal 469 is routed to an input of an acquisition module 470, which acquisition module 470 acquires or recovers the OFDM symbol timing offset/error and carrier frequency offset/error from the received OFDM symbols represented in received complex signal 469. The acquisition module 470 forms the symbol timing offset Δ t and the carrier frequency offset Δ f, as well as the status and control information. The signal is then demodulated (block 472) to demodulate the digitally modulated portion of the baseband signal. The digital signal is deinterleaved by a deinterleaver 474 and decoded by a viterbi decoder 476. Service demultiplexer 478 separates the main and supplemental program signals from the data signal. The supplemental program signal may comprise a digital audio file received in an IBOC DAB radio broadcast signal.
An audio processor 480 processes the received signal to produce an audio signal on line 482 and MPSD/SPSD 481. In an embodiment, the analog and main digital audio signals are mixed as shown in block 484 or the supplemental program signal is passed to produce an audio output on line 486. A data processor 488 processes the received data signals and produces data output signals on lines 490, 492 and 494. Data lines 490, 492, and 494 may be multiplexed together onto a suitable bus (such as I)2C. SPI, UART, or USB). The data signal may comprise data, for example representing metadata to be rendered at the radio receiver.
The internet network interface 440 may be managed by the receiver controller 430. As shown in fig. 4, the internet network interface 440 and the receiver controller 430 are operatively coupled via a line 442, and data transmitted between the internet network interface 440 and the receiver controller 430 is transmitted over this line 442. The selector 420 may be connected to the receiver controller 430 via line 436 to select particular data received from the internet network interface 440. The data may include metadata (e.g., text, images, video, etc.) and may be rendered at substantially the same time as the primary or supplemental programming content received over the air in rendering the IBOC DAB radio signal.
The receiver controller 430 receives and processes the data signals. The receiver controller 430 may include a microcontroller operatively coupled to a user interface 432 and a memory 434. The microcontroller may be an 8-bit RISC microprocessor, an advanced RISC machine 32-bit microprocessor, or any other suitable microprocessor or microcontroller. Further, some or all of the functions of receiver controller 430 may be performed in a baseband processor (e.g., audio processor 480 and/or data processor 488). The user interface 432 may include an input/output (I/O) processor that controls a display 444, which display 444 may be any suitable visual display, such as an LCD or LED display. In some embodiments, the user interface 432 may also control user input components via a touch screen display. In some embodiments, the user interface 432 may also control user input from a keyboard, dial, knob, or other suitable input. The memory 434 may include any suitable data storage medium, such as RAM, Flash ROM (e.g., SD memory card), and/or a hard disk drive. The radio receiver 400 may also include a Global Positioning System (GPS) receiver 496 to receive GPS coordinates.
The processing circuitry of the receiver controller 430 is configured to execute instructions included in a client application or "client" installed in the radio receiver. The client 446 is capable of generating an audio fingerprint from an audio broadcast received via the RF receiver circuitry. The client 446 also sends the geographic location information to the audio metadata service application via the internet network interface 440. The radio receiver may include a GPS receiver 496 and the client may send GPS coordinates as geographic location information. In response to sending the geographic information, client 446 receives metadata for any radio broadcasts available to the radio receiver at the geographic location it indicates. Included in this metadata is an indication for each radio broadcast of whether the content of the radio broadcast is suitable for ACR fingerprinting and identification procedures.
As explained earlier herein, when the current radio broadcast to which the radio receiver is tuned, audio metadata is missing or unavailable, the client generates an audio fingerprint for that radio broadcast when the indication for that radio broadcast is that its content is suitable for the audio fingerprinting process. The client 446 sends the generated audio fingerprint to the audio metadata service application via an internet network interface. The audio fingerprint is processed by the service application and the client 446 receives dynamic metadata associated with the radio broadcast corresponding to the audio fingerprint. The dynamic metadata may be received via an internet network. In some embodiments, the dynamic metadata is received via one or more subcarriers broadcast by the main channel OTA. Client 446 displays the information included in the received dynamic metadata.
When the content indicating that it is a radio broadcast signal is not suitable for the audio fingerprinting process, the client 446 does not generate an audio fingerprint. Client 446 may do nothing in response to missing audio metadata or may display available metadata (e.g., static metadata instead of dynamic metadata).
The described systems, devices and methods provide metadata to a radio receiver of a vehicle. The radio receiver is capable of performing audio fingerprinting, but the system, apparatus and method prevents the radio receiver from performing audio fingerprinting and requesting identification of an audio fingerprint when the content of the radio broadcast is not suitable for the fingerprinting and identification process. This saves a lot of computational resources and communication bandwidth of the radio receiver.
Alternate embodiments and exemplary operating Environment
Example 1 includes a subject matter (such as a system to provide audio metadata to a radio receiver) comprising: an intermediate communication platform providing an interface to an internet network; and a first server. The first server includes: a port operatively coupled to the intermediate communication platform, processing circuitry, and a service application for execution by a processor. The service application is configured to: receiving geographic location information of a radio receiver via the intermediate communication platform; determining one or more radio broadcasts available to the radio receiver from the geographical location information; and transmitting metadata of the radio broadcast to the radio receiver via the intermediate communication platform. The metadata includes an indication of whether the content of the radio broadcast is suitable for the audio fingerprinting process.
In example 2, the subject matter of example 1 optionally includes: a server configured to store indications of suitability of audio fingerprinting processes for a plurality of radio broadcasts in association with metadata of the plurality of radio broadcasts; and a server application configured to: determining all radio broadcasts available to the radio receiver from the geographical location information; and in response to receiving the geographical location information, transmitting metadata of the determined radio broadcast, the metadata of the determined radio broadcast including an indication of suitability of an audio fingerprinting process for the determined radio broadcast.
In example 3, the subject matter of one or both of examples 1 and 2 optionally includes a service application configured to: receiving an audio fingerprint from the radio receiver via the intermediary communication platform; determining audio metadata of a radio broadcast corresponding to the audio fingerprint; and transmitting the determined audio metadata to the radio receiver.
In example 4, the subject matter of one or any combination of examples 1-3 optionally includes a service application configured to transmit the determined audio metadata to a plurality of radio receivers via the intermediate communication platform.
In example 5, the subject matter of one or any combination of examples 1-4 optionally includes the server comprising a memory configured to store audio metadata in association with audio fingerprint information, and the service application is configured to determine the audio metadata by retrieving the audio metadata from the memory using the audio fingerprint.
In example 6, the subject matter of one or any combination of examples 1-4 optionally includes: a second server configured to store audio metadata; and a communication network operatively coupled to the first server and the second server. The service application of the first server is configured to determine audio metadata by forwarding the audio fingerprint to the second server via the communication network and receiving audio metadata from the second server.
In example 7, the subject matter of one or any combination of examples 1-6 optionally includes a service application configured to: sending, along with the static metadata, an indication of whether the content of the radio broadcast is suitable for the audio fingerprinting process; receiving an audio fingerprint from the radio receiver via the intermediary communication platform; determining dynamic metadata of a radio broadcast corresponding to the audio fingerprint; and transmitting the determined dynamic metadata to the radio receiver via the intermediate communication platform.
In example 8, the subject matter of one or any combination of examples 1-7 optionally includes the intermediate communication platform being a cellular telephone network.
In example 9, the subject matter of one or any combination of examples 1-7 optionally includes the intermediate communication platform being a telematics network.
Example 10 may include a subject matter (such as a radio receiver) or may optionally be combined with one or any combination of examples 1-9 to include a subject matter comprising: a Radio Frequency (RF) receiver circuit configured to receive a radio broadcast signal; an internet network interface; a display; a processing circuit; and a client application program comprising instructions for execution by the processing circuit. The client application is configured to: sending geographic location information to an audio metadata service application via the internet network interface; and receiving, via the internet network interface, metadata of a radio broadcast available to the radio receiver, the metadata including an indication of whether content of the radio broadcast is suitable for an audio fingerprinting process.
In example 11, the subject matter of example 10 optionally includes a client application configured to: determining that dynamic metadata associated with a radio broadcast is not available for presentation using the display; generating an audio fingerprint of the radio broadcast when the indication is that the content of the radio broadcast is suitable for an audio fingerprinting process; sending the audio fingerprint to the audio metadata service application via the internet network interface; receiving dynamic metadata associated with a radio broadcast corresponding to the audio fingerprint; and displaying information included in the dynamic metadata.
In example 12, the subject matter of one or both of examples 10 and 11 optionally includes: a memory; and a client application configured to: determining that dynamic metadata associated with a radio broadcast is not available for presentation using the display; generating an audio fingerprint of the radio broadcast when the indication is that the content of the radio broadcast is suitable for an audio fingerprinting process; identifying metadata stored in the memory using the generated audio fingerprint; and displaying information included in the identified metadata.
In example 13, the subject matter of one or any combination of examples 10-12 optionally includes a client application configured to: determining that metadata associated with a radio broadcast is not available for presentation using the display; and not generating an audio fingerprint of the radio broadcast when the indication is that the content of the radio broadcast signal is not suitable for the audio fingerprint processing procedure.
In example 14, the subject matter of one or any combination of examples 10-13 optionally includes a client application configured to: receiving, via the internet network interface, metadata for all radio broadcasts available to the radio receiver for the geographic location information; and receiving an indication of whether the content of the radio broadcast is suitable for the audio fingerprinting process for each available radio broadcast.
In example 15, the subject matter of one or any combination of examples 10-14 optionally includes the internet network interface being a cellular telephone network.
In example 16, the subject matter of one or any combination of examples 10-14 optionally includes the internet network interface being a telematics network.
Example 17 may include subject matter, or may optionally be combined with one or any combination of examples 1-16, to include subject matter, such as a computer-readable storage medium comprising instructions that, when executed by processing circuitry of a server, cause the processing circuitry to perform acts comprising: receiving geographic location information of a radio receiver via an intermediate communication platform, the intermediate communication platform providing an interface to an internet network; determining radio broadcasts available to the radio receiver based on the geographical location information; and sending metadata of the radio broadcast to the radio receiver via the intermediate communication platform, the metadata including an indication of whether content of the radio broadcast is suitable for an audio fingerprinting process.
In example 18, the subject matter of example 17 optionally includes a computer-readable storage medium comprising instructions to cause the processing circuit to perform acts comprising: determining all radio broadcasts available to the radio receiver from the geographical location information; and in response to receiving the geographical location information, transmitting metadata of the determined radio broadcast, the metadata of the determined radio broadcast including an indication of suitability of an audio fingerprinting process for the determined radio broadcast.
In example 19, the subject matter of one or both of examples 17 and 18 optionally includes a computer-readable storage medium comprising instructions to cause the processing circuit to perform acts comprising: receiving an audio fingerprint from the radio receiver; determining audio metadata of a radio broadcast corresponding to the audio fingerprint; and transmitting the determined audio metadata to the radio receiver.
In example 20, the subject matter of one or any combination of examples 17-19 optionally includes a computer-readable storage medium comprising instructions to cause the processing circuit to perform acts comprising: transmitting audio metadata determined from audio fingerprints received from the radio receivers to a plurality of other radio receivers via the intermediate communication platform.
These non-limiting examples may be combined in any permutation or combination. It will be apparent from the disclosure herein that there are many other variations in addition to those described herein. For example, depending on the embodiment, certain acts, events or functions of any methods and algorithms described herein can be performed in a different order, may be added, merged, or omitted altogether (such that not all described acts or events are necessary for the practice of the methods and algorithms). Moreover, in some embodiments, acts or events may be performed concurrently, such as through multi-threaded processing, interrupt processing, or multiple processors or processor cores, or on other parallel architectures, rather than sequentially. Further, different tasks or processes may be performed by different machines and computing systems that may function together.
The various illustrative logical blocks, modules, methods, and algorithm processes and sequences described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or combinations of both. To clearly illustrate this interchangeability of hardware and software, various illustrative components, blocks, modules, and process actions have been described above generally in terms of their functionality. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the overall system. The described functionality may be implemented in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of this document.
The various illustrative logical blocks and modules described in connection with the embodiments disclosed herein may be implemented or performed with a machine such as a general purpose processor, a processing device, a computing device with one or more processing devices, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other programmable logic device designed to perform the functions described herein, discrete gate or transistor logic, discrete hardware components, or any combination thereof. A general purpose processor and a processing device may be a microprocessor, but in the alternative, the processor may be a controller, microcontroller, or state machine, combinations of the same, or the like. A processor may also be implemented as a combination of computing devices, e.g., a combination of a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors in conjunction with a DSP core, or any other such configuration.
Embodiments of the targeted fingerprint processing radio system and method described herein are operational with numerous types of general purpose or special purpose computing system environments or configurations. In general, a computing environment may include any type of computer system, including but not limited to one or more microprocessor-based computer systems, mainframe computers, digital signal processors, portable computing devices, personal organizers, device controllers, computing engines in devices, mobile telephones, desktop computers, mobile computers, tablets, smart phones, devices with embedded computers, and so forth.
Such computing devices may generally be found in devices having at least some minimal computing power, including but not limited to personal computers, server computers, hand-held computing devices, laptop or mobile computers, communication devices such as cellular telephones and PDAs, multiprocessor systems, microprocessor-based systems, set top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, audio or video media players, and the like. In some embodiments, the computing device will include one or more processors. Each processor may be a dedicated microprocessor, such as a Digital Signal Processor (DSP), Very Long Instruction Word (VLIW), or other microcontroller, or may be a conventional Central Processing Unit (CPU) having one or more processing cores, including a dedicated Graphics Processing Unit (GPU) based core in a multi-core CPU.
The process actions or operations of a method, process, or algorithm described in connection with the embodiments disclosed herein may be embodied directly in hardware, in a software module executed by a processor, or in any combination of the two. The software modules may be embodied in computer-readable media that may be accessed by a computing device. Computer-readable media includes both volatile and nonvolatile media, which are either removable, non-removable, or some combination thereof. Computer-readable media are used to store information such as computer-readable or computer-executable instructions, data structures, program modules or other data. By way of example, and not limitation, computer readable media may comprise computer storage media and communication media.
Computer storage media includes, but is not limited to, computer or machine readable media or storage devices, such as blu-ray discs (BD), Digital Versatile Discs (DVD), Compact Discs (CD), floppy disks, tape drives, hard drives, optical drives, solid state memory devices, RAM memory, ROM memory, EPROM memory, EEPROM memory, flash memory or other memory technology, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other device that can be used to store the desired information and that can be accessed by one or more computing devices.
A software module may reside in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, hard disk, a removable disk, a CD-ROM, or any other form of non-transitory computer-readable storage medium, media, or physical computer storage known in the art. An exemplary storage medium may be coupled to the processor such the processor can read information from, and write information to, the storage medium. In the alternative, the storage medium may be integral to the processor. The processor and the storage medium may reside in an Application Specific Integrated Circuit (ASIC). The ASIC may reside in a user terminal. In the alternative, the processor and the storage medium may reside as discrete components in a user terminal.
As used in this document, the phrase "non-transient" refers to "persistent or long-lived". The phrase "non-transitory computer readable medium" includes any and all computer readable media, with the sole exception of transitory propagating signals. By way of example, and not limitation, this includes non-transitory computer-readable media such as register memory, processor cache, and Random Access Memory (RAM). The phrase "audio signal" is a signal representing a physical sound.
The retention of information such as computer-readable or computer-executable instructions, data structures, program modules, etc., may also be accomplished by the use of various communication media to encode one or more modulated data signals, electromagnetic waves (such as carrier waves), or other transmission mechanisms or communication protocols and include any wired or wireless information delivery mechanisms. Generally, these communications media refer to signals whose one or more characteristics are set or changed in such a manner as to encode information or instructions in the signal. For example, communication media includes wired media such as a wired network or direct-wired connection carrying one or more modulated data signals, and wireless media such as acoustic, Radio Frequency (RF), infrared, laser, and other wireless media for transmitting, receiving, or both one or more modulated data signals or electromagnetic waves. Combinations of any of the above should also be included within the scope of communication media.
In addition, one or any combination, or portions thereof, of the various embodiments implementing the in-vehicle real-time guide generation systems and methods described herein may be stored, received, transmitted, or read from any desired combination of computers or machine-readable media or storage devices and communication media in the form of computer-executable instructions or other data structures.
Embodiments of the in-vehicle real-time guidance generation systems and methods described herein may be further described in the general context of computer-executable instructions, such as program modules, being executed by a computing device. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform particular tasks or implement particular abstract data types. The embodiments described herein may also be practiced in distributed computing environments where tasks are performed by one or more remote processing devices, or in a cloud of one or more devices that are linked through one or more communications networks. In a distributed computing environment, program modules may be located in both local and remote computer storage media including media storage devices. Still further, the instructions described above may be implemented partially or fully as hardware logic circuits, which may or may not include a processor.
As used herein, conditional language (such as "can," "might," "can," "e.g.," among others) is generally intended to convey that certain embodiments include, while other embodiments do not include, certain features, elements, and/or states, unless otherwise indicated or otherwise understood in the context as used. Thus, such conditional language is not generally intended to imply that features, elements, and/or states are in any way required for one or more embodiments or that one or more embodiments necessarily include logic for deciding, with or without author input or prompting, that such features, elements, and/or states are included or are to be performed in any particular embodiment. The terms "comprising," "having," and the like are synonymous and are used inclusively in an open-ended fashion, and do not exclude additional elements, features, acts, operations, and the like. Also, the term "or" is used in its inclusive sense (and not in its exclusive sense) such that, when used with a list of, for example, connected elements, the term "or" refers to one, some, or all of the elements in the list.
While the above detailed description has shown, described, and pointed out novel features as applied to various embodiments, it will be understood that various omissions, substitutions, and changes in the form and details of the device or algorithm illustrated may be made without departing from the scope of the disclosure. As will be recognized, certain embodiments of the present invention described herein can be embodied within a form that does not provide the features and advantages set forth herein, as some features can be used or practiced separately from others.

Claims (20)

1. A system for providing audio metadata to a radio receiver, the system comprising:
an intermediate communication platform providing an interface to an internet network; and
a first server, the first server comprising: a port operatively coupled to the intermediate communication platform, processing circuitry, and a service application for execution by a processor, wherein the service application is configured to:
receiving geographic location information of a radio receiver via the intermediate communication platform;
determining one or more radio broadcasts available to the radio receiver from the geographical location information; and
sending metadata of a radio broadcast to the radio receiver via the intermediate communication platform, the metadata including an indication of whether content of the radio broadcast is suitable for an audio fingerprinting process.
2. The system of claim 1, wherein the first and second optical elements are selected from the group consisting of a laser, and a laser,
wherein the first server is configured to store an indication of suitability of an audio fingerprinting process for a plurality of radio broadcasts in association with metadata of the plurality of radio broadcasts;
wherein the service application is configured to:
determining all radio broadcasts available to the radio receiver from the geographical location information; and
in response to receiving the geographic location information, transmitting metadata of the determined radio broadcast including an indication of suitability of an audio fingerprinting process for the determined radio broadcast.
3. The system of claim 1, wherein the service application is configured to:
receiving an audio fingerprint from the radio receiver via the intermediary communication platform;
determining audio metadata of a radio broadcast corresponding to the audio fingerprint; and
transmitting the determined audio metadata to the radio receiver.
4. The system of claim 3, wherein the service application is configured to transmit the determined audio metadata to a plurality of radio receivers via the intermediate communication platform.
5. The system of claim 3, wherein the first server comprises a memory configured to store audio metadata in association with audio fingerprint information, and the service application is configured to determine audio metadata by retrieving audio metadata from the memory using the audio fingerprint.
6. The system of claim 3, comprising:
a second server configured to store audio metadata; and
a communication network operatively coupled to the first and second servers;
wherein the service application of the first server is configured to determine audio metadata by forwarding the audio fingerprint to the second server via the communication network and receiving audio metadata from the second server.
7. The system of claim 3, wherein the service application is configured to:
sending, along with the static metadata, an indication of whether the content of the radio broadcast is suitable for the audio fingerprinting process;
receiving an audio fingerprint from the radio receiver via the intermediary communication platform;
determining dynamic metadata of a radio broadcast corresponding to the audio fingerprint; and
transmitting the determined dynamic metadata to the radio receiver via the intermediate communication platform.
8. The system of claim 1, wherein the intermediate communication platform is a cellular telephone network.
9. The system of claim 1, wherein the intermediate communication platform is a telematics network.
10. A radio receiver, comprising:
a Radio Frequency (RF) receiver circuit configured to receive a radio broadcast signal;
an internet network interface;
a display;
a processing circuit; and
a client application comprising instructions for execution by the processing circuit, wherein the client application is configured to:
sending geographic location information to an audio metadata service application via the internet network interface; and
receiving, via the internet network interface, metadata of a radio broadcast available to the radio receiver, the metadata including an indication of whether content of the radio broadcast is suitable for an audio fingerprinting process.
11. The radio receiver of claim 10, wherein the client application is configured to:
determining that dynamic metadata associated with a radio broadcast is not available for presentation using the display;
generating an audio fingerprint of the radio broadcast when the indication is that the content of the radio broadcast is suitable for an audio fingerprinting process;
sending the audio fingerprint to the audio metadata service application via the internet network interface;
receiving dynamic metadata associated with a radio broadcast corresponding to the audio fingerprint; and
displaying information included in the dynamic metadata.
12. The radio receiver of claim 10, comprising a memory; wherein the client application is configured to:
determining that dynamic metadata associated with a radio broadcast is not available for presentation using the display;
generating an audio fingerprint of the radio broadcast when the indication is that the content of the radio broadcast is suitable for an audio fingerprinting process;
identifying metadata stored in the memory using the generated audio fingerprint; and
displaying information included in the identified metadata.
13. The radio receiver of claim 10, wherein the client application is configured to:
determining that metadata associated with a radio broadcast is not available for presentation using the display; and
not generating an audio fingerprint of the radio broadcast when the indication is that the content of the radio broadcast signal is not suitable for the audio fingerprinting process.
14. The radio receiver of claim 10, wherein the client application is configured to:
receiving, via the internet network interface, metadata for all radio broadcasts available to the radio receiver for the geographic location information; and
an indication of whether the content of the radio broadcast is suitable for the audio fingerprinting process is received for each available radio broadcast.
15. The radio receiver of claim 10, wherein the internet network interface is a cellular telephone network.
16. The radio receiver of claim 10, wherein the internet network interface is a telematics network.
17. A non-transitory computer-readable storage medium comprising instructions that, when executed by processing circuitry of a server, cause the processing circuitry to perform acts comprising:
receiving geographic location information of a radio receiver via an intermediate communication platform, the intermediate communication platform providing an interface to an internet network;
determining radio broadcasts available to the radio receiver based on the geographical location information; and
sending metadata of a radio broadcast to the radio receiver via the intermediate communication platform, the metadata including an indication of whether content of the radio broadcast is suitable for an audio fingerprinting process.
18. The non-transitory computer readable storage medium of claim 17, comprising instructions to cause the processing circuit to perform acts comprising:
determining all radio broadcasts available to the radio receiver from the geographical location information; and
in response to receiving the geographic location information, transmitting metadata of the determined radio broadcast including an indication of suitability of an audio fingerprinting process for the determined radio broadcast.
19. The non-transitory computer readable storage medium of claim 17, comprising instructions to cause the processing circuit to perform acts comprising:
receiving an audio fingerprint from the radio receiver;
determining audio metadata of a radio broadcast corresponding to the audio fingerprint; and
transmitting the determined audio metadata to the radio receiver.
20. The non-transitory computer readable storage medium of claim 17, comprising instructions to cause the processing circuit to perform acts comprising: transmitting audio metadata determined from audio fingerprints received from the radio receivers to a plurality of other radio receivers via the intermediate communication platform.
CN201980099769.4A 2019-07-19 2019-07-19 Targeted fingerprinting of radio broadcast audio Pending CN114287113A (en)

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/US2019/042619 WO2021015716A1 (en) 2019-07-19 2019-07-19 Targeted fingerprinting of radio broadcast audio

Publications (1)

Publication Number Publication Date
CN114287113A true CN114287113A (en) 2022-04-05

Family

ID=67515193

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201980099769.4A Pending CN114287113A (en) 2019-07-19 2019-07-19 Targeted fingerprinting of radio broadcast audio

Country Status (8)

Country Link
EP (1) EP4000192A1 (en)
JP (1) JP7381712B2 (en)
KR (1) KR102656973B1 (en)
CN (1) CN114287113A (en)
AU (1) AU2019457816A1 (en)
CA (1) CA3147864A1 (en)
MX (1) MX2022000751A (en)
WO (1) WO2021015716A1 (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20220094457A1 (en) * 2020-09-19 2022-03-24 Ibiquity Digital Corporation Content Linking Multicast Streaming for Broadcast Radio

Family Cites Families (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
FR2865298B1 (en) * 2004-01-16 2010-01-15 Musiwave Sa SYSTEM AND METHOD FOR RECOGNIZING SOUND SEQUENCE
EP2070231B1 (en) * 2006-10-03 2013-07-03 Shazam Entertainment, Ltd. Method for high throughput of identification of distributed broadcast content
JP4575518B1 (en) 2009-10-30 2010-11-04 Jvc・ケンウッド・ホールディングス株式会社 Information distribution management device, information distribution management method, information distribution management program, and information distribution system
JP5859245B2 (en) 2011-08-11 2016-02-10 株式会社ビデオリサーチ Information display system and method, survey system and method
JP5948773B2 (en) 2011-09-22 2016-07-06 ソニー株式会社 Receiving apparatus, receiving method, program, and information processing system
WO2015138798A1 (en) 2014-03-13 2015-09-17 Verance Corporation Interactive content acquisition using embedded codes
BR112018071591A2 (en) * 2016-04-22 2019-02-12 Ibiquity Digital Corporation broadcasting receiving system, and method for providing metadata associated with wireless broadcasting signals to a broadcasting receiving system.
US10574373B2 (en) 2017-08-08 2020-02-25 Ibiquity Digital Corporation ACR-based radio metadata in the cloud

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20220094457A1 (en) * 2020-09-19 2022-03-24 Ibiquity Digital Corporation Content Linking Multicast Streaming for Broadcast Radio
US12009909B2 (en) * 2020-09-19 2024-06-11 Ibiquity Digital Corporation Content linking multicast streaming for broadcast radio

Also Published As

Publication number Publication date
EP4000192A1 (en) 2022-05-25
AU2019457816A1 (en) 2022-03-03
JP2022547385A (en) 2022-11-14
CA3147864A1 (en) 2021-01-28
WO2021015716A1 (en) 2021-01-28
MX2022000751A (en) 2022-03-25
KR102656973B1 (en) 2024-04-11
KR20220035474A (en) 2022-03-22
JP7381712B2 (en) 2023-11-15

Similar Documents

Publication Publication Date Title
US8351843B2 (en) Digital radio broadcast receiver, broadcasting methods and methods for tagging content of interest
JP7068187B2 (en) Radio radio broadcast signal metadata
US11245482B2 (en) ACR-based radio metadata in the cloud
US8660479B2 (en) Digital radio broadcast receiver, broadcasting methods and methods for tagging content of interest
US10868621B1 (en) Connected Radio local, isolated, and hybrid implementation
US11817941B2 (en) Targeted fingerprinting of radio broadcast audio
US10826634B2 (en) System and method for in-vehicle live guide generation
US10673549B1 (en) Advertising measurement and conversion measurement for radio systems
KR102656973B1 (en) Targeted fingerprinting of radio broadcast audio
JP7490763B2 (en) Local, Separate and Hybrid Implementations of Connected Radio

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination