CN110752976A

CN110752976A - Device and method for controlling Internet of things equipment

Info

Publication number: CN110752976A
Application number: CN201910977197.7A
Authority: CN
Inventors: 尹硕铉
Original assignee: Samsung Electronics Co Ltd
Current assignee: Samsung Electronics Co Ltd
Priority date: 2014-05-28
Filing date: 2015-05-28
Publication date: 2020-02-04
Anticipated expiration: 2035-05-28
Also published as: CN106537849B; KR20150136981A; CN110752976B; KR102330255B1; CN106537849A

Abstract

Certain embodiments of an electronic device including a memory are disclosed. The memory may store instructions that, by operation of the application processor, cause the electronic device to: the method includes obtaining information about a first external electronic device, accessing a server storing a software program related to the first external electronic device through a wireless communication interface, receiving at least a portion of the software program related to the first external electronic device from the server through the communication interface, installing at least a portion of the software program, transmitting at least a portion of the information about the first external electronic device and/or at least a portion of the received at least a portion of the software program to a second external electronic device, and providing a user interface to a display using the at least a portion of the installed software program. The user interface may be configured to receive a user input for the second external electronic device to perform an operation related to the first external electronic device.

Description

Device and method for controlling Internet of things equipment

The application is a divisional application of Chinese patent application with the application date of 2015, 5 and 28, the application number of 201580028156.3 and the name of 'device and method for controlling Internet of things'.

Technical Field

The present disclosure relates generally to electronic devices, and more particularly, to apparatus and methods for controlling internet of things (IoT) devices.

Background

The internet enables information exchange between devices by connecting many computing devices, such as servers, Personal Computers (PCs), mobile devices, and the like, to one another. Currently, there is an attempt to connect items (e.g., sensors, home appliances, meters, etc.) that generally do not have a computing function to the internet to exchange various information or data therebetween. The mechanism by which items are connected via the internet is commonly referred to as the internet of things (IoT).

In the IoT space, a great deal of research and development is being conducted to provide protocols for communication between various devices, communication mechanisms between devices, and data collection. IoT is expected to facilitate technological evolution over the internet by combining various technologies.

Disclosure of Invention

Solution to the problem

In an IoT implementation, various items or electronic devices may be connected to the internet. However, due to the different functionality, computing power levels, and communication capabilities of these items, connecting such devices to the internet may present difficulties, which may affect product development and the availability of such devices. Accordingly, there is a need to address these problems and/or disadvantages as described above.

One embodiment of the present disclosure may provide a scheme and system method for solving the above-described problems. Other embodiments of the present disclosure may provide an identification and discovery, lightweight network protocol, and/or IoT middleware framework for low-end IoT devices with restricted resources for accessing the internet.

According to one embodiment of the present disclosure, an electronic device is provided that may include a wireless communication interface, a memory, a display, and an application processor electrically connected to the communication interface, the memory, and the display. The memory may store instructions that, when executed by the application processor, cause the electronic device to: the method includes obtaining information about a first external electronic device, accessing a server storing a software program related to the first external electronic device through a communication interface based on at least a portion of the information about the first external electronic device, receiving at least a portion of the software program associated with the first external electronic device from the server through the communication interface, installing the at least a portion of the software program received from the server on the electronic device, transmitting the at least a portion of the information about the first external electronic device and/or the at least a portion of the received at least a portion of the software program to a second external electronic device through the communication interface, and providing a user interface on a display using the at least a portion of the installed software program. The user interface may be configured to receive a user input for a second external electronic device (control device) to perform an operation associated with the first external electronic device.

In various embodiments, the processor may be configured to obtain information about the first external electronic device using at least one of a code or a tag attached to or provided with the first external electronic device. In various embodiments, the code may include at least one of a barcode or a Quick Response (QR) code. In various embodiments, the tag may include at least one of a Radio Frequency Identification (RFID), Near Field Communication (NFC) tag, or Bluetooth Low Energy (BLE) tag. In other embodiments, the instructions further cause the electronic device to: information about an application store that provides a software program associated with the first external electronic device is received through the communication interface, and at least one image or icon related to the software program is displayed on the display using the information about the application store.

In embodiments, the instructions further cause the electronic device to: the method includes receiving information about an application store providing a software program associated with a first external electronic device through a communication interface, displaying at least one category related to the software program on a display based on the received information about the application store, and displaying a plurality of images or icons including images or icons related to the software program in response to a user input selecting the category.

In embodiments, the instructions further cause the electronic device to: in response to a user input for selecting an icon displayed on the display, an application configured to provide a user interface is run. The application program may be configured to further provide a user interface for the second external electronic device to perform operations related to the third external electronic device.

In various embodiments, at least a portion of the received at least a portion of the software program may include identification information of the software program. In embodiments, the instructions further cause the electronic device to: the location of the first external electronic device is determined and information about the location is sent to the second external electronic device.

According to another embodiment, a portable electronic device may include a wireless communication interface, a memory device, a display, and an application processor electrically connected to the communication interface, the memory device, and the display. The storage device may store a single application program for controlling a plurality of external electronic devices, and store instructions that, through operation of the application processor, cause the electronic device to perform the following operations: the method includes accessing an external server providing an application store through a communication interface, displaying a first user interface associated with the application store on a display, receiving a first software program to be used for controlling a first external electronic device through the communication interface and installing the received first software program in the electronic device in response to a first user selection through the first user interface, receiving a second software program to be used for controlling a second external electronic device through the communication interface and installing the received second software program in the electronic device in response to a second user selection through the first user interface, and displaying a second user interface for receiving respective user inputs associated with the first and second external electronic devices on the display using a single application.

In embodiments, the second user interface may comprise a single screen, window, or page. A single screen, window, or page may include a first icon, first image, or first text for selecting a first external electronic device and a second icon, second image, or second text for selecting a second external electronic device. In various embodiments, the electronic device may display a sub-user interface for controlling the first external electronic device or the second external electronic device on the display in response to a selection of the first or second icon, the first or second image, or the first or second text on the second user interface.

According to another embodiment, an electronic device (control device) may include a housing including a first surface, a second surface facing away from the first surface, and a side surface surrounding a space between the first surface and the second surface, a major portion of the first surface, the second surface, and the side surface of the housing including an opaque or translucent material, a wired communication port located on one of the surfaces of the housing and configured to transmit audio/video (a/V) data to an external display device, a power connector located on the one of the surfaces of the housing and configured to receive power, a wireless communication interface located within the housing, wherein the wireless communication interface supports a first non-cellular communication protocol and a second non-cellular communication protocol, a non-volatile memory located within the housing, and a processor, located within the housing and electrically connected to the wireless communication interface, the wired communication port, and the memory. The memory may store Operating System (OS) software and store instructions that, when executed by the processor, cause the electronic device to: receiving a first software program for controlling a first external electronic device through a communication interface and installing the received first software program to run on OS software, receiving a second software program for controlling a second external electronic device through the communication interface and installing the received second software program to run on the OS software, generating a signal for controlling at least one of the first external electronic device or the second external electronic device using the at least one of the first software program or the second software program and the OS software, independently of the transmission of the a/V data and in response to a request from the external portable electronic device, and transmitting the signal for controlling the at least one of the first external electronic device or the second external electronic device to the at least one of the first external electronic device or the second external electronic device using the at least one of the first non-cellular communication protocol or the second non-cellular communication protocol A signal of at least one of the external electronic devices.

In various embodiments, the processor may be configured to receive at least a portion of the first software program or the second software program from an external portable electronic device. In embodiments, the instructions further cause the electronic device to: at least a portion of the identification information of the first external electronic device or the second external electronic device is received from the external portable electronic device. In embodiments, the instructions further cause the electronic device to: identification information associated with at least a portion of the first software program or the second software program is received from the external portable electronic device, and at least a portion of the first software program or the second software program is received from the external server based on the identification information. In various embodiments, the signal for controlling at least one of the first external electronic device or the second external electronic device may include: at least one of a first control signal including an internet protocol based address or a second control signal based on a non-internet communication protocol.

In various embodiments, the signal for controlling at least one of the first external electronic device or the second external electronic device may include a first control signal based on a first non-cellular communication protocol or a second control signal based on a second non-cellular communication protocol. The length of the header of the packet or frame of the first control signal is longer than the length of the header of the packet or frame of the second control signal. In embodiments, the electronic device comprises at least one of an internet gateway, a home networking control device, a set-top box, a media storage device, a security control device, a gaming console, or a thermostat.

According to a further embodiment, a method of operating an electronic device may comprise the operations of: acquiring, by an electronic device, information about a first external electronic device, wherein the electronic device comprises a wireless communication interface, a memory, a display and a processor, accessing, by the electronic device, a server storing a software program related to the first external electronic device based on at least a part of the information about the first external electronic device acquired through the communication interface, receiving, by the electronic device, at least a part of the software program related to the first external electronic device from the server through the communication interface, installing, by the electronic device, at least a part of the software program received from the server on the electronic device, transmitting, by the electronic device, at least a part of the information about the first external electronic device and/or at least a part of the received at least a part of the software program to a second external electronic device (control device) through the communication interface, and displaying, by the electronic device, a user interface on the display using the at least a part of the installed software program And (5) kneading. The displayed user interface is configured to receive user input for controlling the second external electronic device to perform an operation related to the first external electronic device.

In various embodiments, obtaining the information may use at least one of a code or a label attached to or provided with the first external electronic device. In various embodiments, the code may include at least one of a barcode or a Quick Response (QR) code. In various embodiments, the tag may include at least one of a Radio Frequency Identification (RFID), Near Field Communication (NFC) tag, or Bluetooth Low Energy (BLE) tag. In various embodiments, the method may further comprise: receiving, by the electronic device, information about an application store that provides a software program related to the first external electronic device through the wireless communication interface, and displaying, by the electronic device, at least one image or icon related to the software program on the display using the information about the application store.

In various embodiments, the method may further comprise: receiving, by the electronic device, information about an application store that provides a software program associated with the first external electronic device through the wireless communication interface, displaying, by the electronic device, at least one category associated with the software program on the display using at least a portion of the information about the application store, and displaying, by the electronic device, a plurality of images or icons including images or icons associated with the software program in response to a user input selecting the category.

In various embodiments, the method may further comprise: an application providing a user interface is executed by the electronic device in response to a user input for selecting an icon displayed on the display. The application program may be configured to further provide a user interface for the second external electronic device to perform operations related to the third external electronic device. In various embodiments, at least a portion of the received at least a portion of the software program may include identification information of the software program. In various embodiments, the method may further comprise: a location associated with a first external electronic device is acquired or determined by the electronic device, and information regarding the location is transmitted by the electronic device to a second external electronic device.

According to a further embodiment, a method of operating a portable electronic device may comprise: accessing, by an electronic device through a communication interface, an external server providing an application store, wherein the electronic device includes a wireless communication interface, a memory, a display, and a processor, displaying, by the electronic device, a first user interface associated with the application store on the display, receiving, by the electronic device, a first software program for controlling a first external electronic device and installing, by the electronic device, the received first software program in the electronic device in response to a first user selection through the first user interface, receiving, by the electronic device, a second software program for controlling a second external electronic device and installing, by the electronic device, the received second software program in the electronic device in response to a second user selection through the first user interface, and displaying, by the electronic device, on the display, using a single application stored in the memory, a second software program for receiving respective user inputs associated with the first external electronic device and the second external electronic device The single application for controlling a plurality of external electronic devices.

In embodiments, the second user interface may comprise a single screen, window, or page. A single screen, window, or page may include a first icon, first image, or first text for selecting a first external electronic device and a second icon, second image, or second text for selecting a second external electronic device. In various embodiments, the method may further comprise the operations of: displaying, by the electronic device, a sub-user interface for controlling the first external electronic device or the second external electronic device on the display in response to a selection of the first or second icon, the first or second image, or the first or second text on the second user interface.

According to yet another embodiment, a method of operating an electronic device (control device) may include the operations of: a housing including a first surface, a second surface facing away from the first surface, and a side surface surrounding a space between the first surface and the second surface, a major portion of the first surface, the second surface, and the side surface of the housing including an opaque or translucent material; a wired communication port located on one of the surfaces of the housing and configured to transmit audio/video (AV) data to an external display device; a power connector located on the one of the surfaces of the housing and configured to receive power; a wireless communication interface located within the housing and supporting a first non-cellular communication protocol and a second non-cellular communication protocol; a non-volatile memory located within the housing and storing Operating System (OS) software; and a processor located within the housing and electrically connected to the wireless communication interface, the wired communication port, and the memory: receiving, by the electronic device, a first software program for controlling a first external electronic device through the communication interface and installing the received first software program to run on the OS software, receiving, by the electronic device, a second software program for controlling a second external electronic device through the communication interface and installing the received second software program to run on the OS software, generating, by the electronic device, a signal for controlling the first external electronic device and/or the second external electronic device using at least a part of the first software program or the second software program and the OS software, independently of the transmission of the a/V data and in response to a request received from the external portable electronic device, and transmitting the control signal using at least one of the first non-cellular communication protocol or the second non-cellular communication protocol.

In various embodiments, the method may further comprise the operations of: the first software program or the second software program is received by the electronic device from an external portable electronic device. In various embodiments, the method may further comprise the operations of: at least a portion of the identification information of the first external electronic device or the second external electronic device is received by the electronic device from the external portable electronic device. In various embodiments, the method may further comprise the operations of: the method further includes receiving, by the electronic device, identification information associated with at least a portion of the first software program or the second software program from the external portable electronic device, and receiving, by the electronic device, at least a portion of the first software program or the second software program from the external server based on the identification information.

In various embodiments, the signal to control at least one of the first external electronic device or the second external electronic device may include at least one of a first control signal including an internet protocol based address or a second control signal based on a non-internet communication protocol. In various embodiments, the signal to control at least one of the first external electronic device or the second external electronic device may include a first control signal based on the first communication protocol or a second control signal based on the second communication protocol. The length of the header of the packet or frame of the first control signal may be longer than the length of the header of the packet or frame of the second control signal.

According to another embodiment of the present disclosure, there is provided an electronic device that may include a wireless communication interface; an application processor; a display; and a memory storing instructions that, when executed by the operations of the application processor, instruct the electronic device to: acquiring information about a first external electronic device; accessing a server storing a software program related to the first external electronic device through a communication interface based on at least a part of the information on the first external electronic device; receiving at least a portion of a software program associated with a first external electronic device from a server through a communication interface; installing at least a portion of the software program received from the server in the electronic device; transmitting at least one of at least a portion of the information about the first external electronic device and at least a portion of the received at least a portion of the software program to a second external electronic device through a communication interface; and providing a user interface to the display using at least a portion of the installed software program. The displayed user interface is configured to receive user input for controlling the second external electronic device to perform an operation related to the first external electronic device.

According to another embodiment of the present disclosure, an electronic device is provided that may include a wireless communication interface supporting a first non-cellular communication protocol and a second non-cellular communication protocol; a processor; and a memory storing Operating System (OS) software and instructions that, when executed by operation of the processor, instruct the electronic device to receive a first software program for controlling a first external electronic device through the communication interface; installing the received first software program to interoperate with the OS software; receiving a second software program for controlling a second external electronic device through the communication interface; installing the received second software program to interoperate with the OS software; generating a signal for controlling at least one of the first and second external electronic devices using the OS software and at least one of the first and second software programs in response to a signal for controlling at least one of the first and second external electronic devices in response to a request from the external portable electronic device; and transmitting a signal for controlling at least one of the first external electronic device and the second external electronic device to at least one of the first external electronic device and the second external electronic device using at least one of the first non-cellular communication protocol and the second non-cellular communication protocol.

According to another embodiment of the present disclosure, a method for operating an electronic device including a wireless communication interface and a display is provided. The method can comprise the following steps: acquiring information about a first external electronic device; accessing, through the communication interface, a server storing a software program related to the first external electronic device based on at least a part of the acquired information about the first external electronic device; receiving at least a portion of a software program associated with a first external electronic device from a server through a communication interface; installing at least a portion of the software program received from the server in the electronic device; transmitting at least one of at least a portion of the information about the first external electronic device and at least a portion of the received at least a portion of the software program to a second external electronic device through a communication interface; and displaying a user interface through the display using at least a portion of the installed software program. The displayed user interface is configured to receive user input for controlling the second external electronic device to perform an operation related to the first external electronic device.

According to another embodiment of the present disclosure, there is provided a method for operating an electronic device that includes a communication interface that supports a first non-cellular communication protocol and a second non-cellular communication protocol. The method can comprise the following steps: receiving a first software program for controlling a first external electronic device through a communication interface; installing the received first software program to interoperate with Operating System (OS) software; receiving a second software program for controlling a second external electronic device through the communication interface; installing the received second software program to interoperate with the OS software; receiving a signal requesting control of at least one of the first external electronic device and the second external electronic device from the external portable electronic device; generating a signal for controlling at least one of the first and second external electronic devices using the OS software and at least one of the first and second software programs in response to the received signal requesting control of the at least one of the first and second external electronic devices; and transmitting a signal for controlling at least one of the first external electronic device and the second external electronic device to at least one of the first external electronic device and the second external electronic device using at least one of the first non-cellular communication protocol and the second non-cellular communication protocol.

According to one embodiment of the present disclosure, there is provided an electronic device that may include a wireless communication interface; an application processor; a display; and a memory storing instructions that, when executed by the operations of the application processor, instruct the electronic device to: acquiring information about a first external electronic device; accessing a server storing a software program related to the first external electronic device through a communication interface based on the acquired information about the first external electronic device; receiving a software program related to a first external electronic device from a server through a communication interface; installing the software program received from the server in the electronic device; transmitting at least one of information about the first external electronic device and the received software program to the second external electronic device through the communication interface; and providing a user interface to the display using the installed software program. The displayed user interface is configured to receive user input for controlling the second external electronic device to perform an operation related to the first external electronic device.

According to another embodiment of the present disclosure, there is provided an electronic device that may include a wireless communication interface supporting a non-cellular communication protocol; a processor; and a memory storing Operating System (OS) software and instructions that, when executed by the operation of the processor, instruct the electronic device to: receiving a software program for controlling an external electronic device through a communication interface; installing the received software program to interoperate with the OS software; generating a signal for controlling the external electronic device using the OS software and the installed software program in response to a signal for controlling the external electronic device in response to a request from the external portable electronic device; and transmitting a signal for controlling the external electronic device to the external electronic device using the non-cellular communication protocol.

According to another embodiment of the present disclosure, a method for operating an electronic device including a wireless communication interface and a display is provided. The method can comprise the following steps: acquiring information about a first external electronic device; accessing a server storing a software program related to the first external electronic device through a communication interface based on the acquired information about the first external electronic device; receiving a software program related to a first external electronic device from a server through a communication interface; installing the software program received from the server in the electronic device; transmitting at least one of information about the first external electronic device and the received software program to the second external electronic device through the communication interface; and displaying a user interface through the display using the installed software program. The displayed user interface is configured to receive user input for controlling the second external electronic device to perform an operation related to the first external electronic device.

According to one embodiment of the present disclosure, a method is provided for operating an electronic device that includes a communication interface that supports a non-cellular communication protocol. The method can comprise the following steps: receiving a software program for controlling an external electronic device through a communication interface; installing the received software program to interoperate with Operating System (OS) software; receiving a signal requesting control of the external electronic device from the external portable electronic device; generating a signal for controlling the external electronic device using the OS software and the installed software program in response to the received signal for controlling the external electronic device; and transmitting a signal for controlling the external electronic device to the external electronic device using the non-cellular communication protocol.

Drawings

The above and other aspects, features and advantages of certain embodiments of the present disclosure will become more apparent from the following detailed description when taken in conjunction with the accompanying drawings in which:

fig. 1 illustrates a system including various items or IoT devices connected to the internet in accordance with one embodiment of the present disclosure;

fig. 2a is a schematic diagram of an internet-connectable IoT device, in accordance with an embodiment of the present disclosure;

fig. 2b is a schematic diagram of an internet-connectable IoT device, according to another embodiment of the present disclosure;

fig. 3 is a perspective view of a control device for controlling an IoT device in accordance with an embodiment of the present disclosure;

fig. 4 is a schematic block diagram of a control device for controlling an IoT device in accordance with an embodiment of the present disclosure;

fig. 5 is a schematic block diagram of a control device for controlling an IoT device in accordance with an embodiment of the present disclosure;

fig. 6 is a schematic block diagram of a software stack of a control device for controlling an IoT device, in accordance with an embodiment of the present disclosure;

fig. 7 is a schematic block diagram of an electronic device for providing a user interface for a control device for controlling IoT devices in accordance with an embodiment of the present disclosure;

fig. 8 is a schematic block diagram of a software stack of an electronic device for providing a user interface for a control device for controlling an IoT device, in accordance with an embodiment of the present disclosure;

fig. 9 is a schematic block diagram of an electronic device for providing a user interface for a control device for controlling IoT devices in accordance with an embodiment of the present disclosure;

figures 10a to 10f illustrate one embodiment of a method of connecting an IoT device to a control device;

fig. 11 is a flowchart illustrating a method of operating an electronic device for providing a user interface when connecting an IoT device to a control device in accordance with an embodiment of the present disclosure;

FIG. 12 is a flow chart illustrating another embodiment of a method of receiving a software program associated with an external device as part of the flow chart of FIG. 11;

FIG. 13 is a flow chart illustrating another embodiment of a method for displaying information about software programs related to an external device as part of the flow chart of FIG. 12;

fig. 14a and 14b illustrate examples of screen displays for an application store of IoT devices in accordance with embodiments of the present disclosure;

fig. 15 is a flowchart illustrating a method of controlling an IoT device in accordance with an embodiment of the present disclosure;

fig. 16 illustrates a user interface controlling an IoT device in accordance with an embodiment of the present disclosure;

fig. 17 is a flowchart illustrating a method of controlling an IoT device in accordance with another embodiment of the present disclosure;

fig. 18 illustrates a user interface controlling an IoT device in accordance with another embodiment of the present disclosure;

fig. 19 is a flowchart illustrating a method of providing a location of an IoT device to a control device of the IoT device in accordance with an embodiment of the present disclosure;

fig. 20 illustrates a scenario for providing a location of an IoT device to a control device of the IoT device, in accordance with an embodiment of the present disclosure;

fig. 21 is a flowchart illustrating a method of operating an IoT control device in accordance with an embodiment of the present disclosure;

fig. 22 is a flow diagram illustrating another embodiment of a method of operating an IoT control device as part of the flow diagram of fig. 21; and

fig. 23 is a flowchart illustrating another embodiment of a method of operating an IoT control device as part of the flowchart of fig. 21.

Detailed Description

Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. In the following description, only specific details (e.g., detailed configurations and components) are provided to assist in a general understanding of the embodiments of the disclosure. Thus, it will be apparent to those skilled in the art that various changes and modifications can be made to the embodiments described herein without departing from the scope of the disclosure. Moreover, descriptions of well-known functions and constructions are omitted for clarity and conciseness.

The present disclosure is susceptible to embodiments and modifications and variations therein. Accordingly, the present disclosure will be described in detail with reference to particular embodiments thereof as illustrated in the accompanying drawings. It is to be understood, however, that the present disclosure is not limited to the particular embodiments, but includes all modifications, alterations, equivalents, and/or substitutions falling within the scope of the present disclosure. In describing the drawings, like reference numerals may be used to designate like elements.

The terms "having," "may have," "including," or "may include" used in various embodiments of the present disclosure indicate the presence of the respective functions, operations, elements, etc., disclosed, and do not limit one or more additional functions, operations, elements, etc. Furthermore, it should be understood that the terms "comprises" or "comprising," when used in various embodiments of the present disclosure, are intended to specify the presence of stated features, amounts, steps, operations, elements, components, or combinations thereof, but do not preclude the presence or addition of one or more other features, amounts, steps, operations, elements, components, or combinations thereof.

The term "a or B", "at least one of a or/and B" or "one or more of a or/and B" as used in various embodiments of the present disclosure includes any and all combinations of the words listed therewith. For example, "a or B," "at least one of a and B," or "at least one of a or B" means (1) including at least one a, (2) including at least one B, or (3) including both at least one a and at least one B.

Although terms such as "first" and "second" used in embodiments of the present disclosure may modify various elements of the embodiments, the terms do not limit the corresponding elements. For example, these terms do not limit the order and/or importance of the corresponding elements. These terms may be used for the purpose of distinguishing elements from each other. For example, the first user equipment and the second user equipment both indicate user equipment and may indicate different user equipment. For example, a first element could be termed a second element, and, similarly, a second element could be termed a first element, without departing from the scope of the claims of the embodiments of the present disclosure.

It will be understood that when an element (e.g., a first element) is "connected" or "coupled" (operably or communicatively) to another element (e.g., a second element), the element can be directly connected or coupled to the other element or intervening elements (e.g., third elements) may be present between the element and the other element. In contrast, it will be understood that when an element (e.g., a first element) is "directly connected to" or "directly coupled to" another element (e.g., a second element), there are no intervening elements (e.g., third elements) between the element and the other element.

The expression "configured (set)" used in the embodiments of the present disclosure may be replaced with "adapted", "having … … capability", "designed", "adapted", "for", or "… … -enabled", as the case may be. The term "configured to (arranged to)" does not necessarily mean "specially designed" hardware. Conversely, the expression "an apparatus configured as … …" may denote that in a particular situation the apparatus "is capable of" … … "together with other devices or components. For example, a "processor configured to (arranged to) perform A, B and C" may be a dedicated processor (e.g., an embedded processor) for performing the respective operations or a general-purpose processor (e.g., a Central Processing Unit (CPU) or an Application Processor (AP)) capable of performing the respective operations by executing one or more software programs stored in a memory device.

The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the disclosure. As used herein, the singular forms may also include the plural forms unless the context clearly dictates otherwise. Furthermore, unless explicitly defined in various embodiments of the present disclosure, all terms (including technical and scientific terms) used herein should be interpreted as having the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure belongs, and should not be interpreted as having an ideal or excessively formal meaning.

The term "module" as used herein may refer to a unit comprising, for example, one of, or a combination of two or more of, hardware, software, and firmware. A "module" may be used interchangeably with terms such as unit, logic block, component, circuit, and the like. A "module" may be the smallest unit or part of an integrally formed component. A "module" may be the smallest unit or part thereof for performing one or more functions. The "module" may be implemented mechanically or electrically. For example, a "module" of the present disclosure may include at least one of: application Specific Integrated Circuit (ASIC) chips, Field Programmable Gate Arrays (FPGAs), and programmable logic devices that are known or will be open in the future and perform specific operations.

At least some portion of an apparatus (e.g., modules or their functionality) or at least some portion of a method (e.g., operations) according to embodiments of the present disclosure may be implemented using instructions, for example, stored in a computer-readable storage medium. If such instructions are executed by one or more processors, the one or more processors may perform the functions corresponding to the instructions. The computer readable storage medium may be, for example, a memory.

The computer-readable storage medium may include, for example, a hard disk, a magnetic medium (e.g., a floppy disk and a magnetic tape), an optical storage medium (e.g., a compact disc-read only memory (CD-ROM) or a DVD), a magneto-optical medium (e.g., a floppy disk), a hardware device (e.g., a ROM, a Random Access Memory (RAM), a flash memory), and the like. Further, examples of the program instructions include not only a machine language created by a compiler, but also a high-level language that a computer can execute using an interpreter, and the like. The hardware devices described above may be configured to operate as one or more software modules to perform the operations of the embodiments described below, and vice versa.

A module or programming module according to embodiments of the disclosure may also include at least one or more of the above-described elements, or may omit some of them, or may also include additional other elements. Operations performed by modules, programmed modules, or other elements according to embodiments of the disclosure may be performed sequentially, in parallel, repeatedly, or heuristically. Further, some operations may be performed in a different order, or may be omitted, or other operations may be added.

The electronic device according to embodiments may comprise, for example, at least one of: a smart phone; a tablet Personal Computer (PC); a mobile phone; video telephony; an electronic book reader; a desktop PC; a laptop PC; a netbook computer; workstations, servers, Personal Digital Assistants (PDAs); a Portable Multimedia Player (PMP); an MP3 player; an ambulatory medical device; a camera; or a wearable device (e.g., smart glasses, Head Mounted Device (HMD), electronic apparel, electronic bracelet, electronic necklace, electronic accessory, electronic tattoo, smart mirror, or smart watch), or a function thereof.

In other embodiments, the electronic device may be a smart home appliance. Such an appliance may include at least one of: a Television (TV); a Digital Video Disc (DVD) player; an audio player; a refrigerator; an air conditioner; a dust collector; an oven; a microwave oven; a washing machine; an air purifier; a set-top box; a home automation control panel; a security control panel; television box (e.g., three stars)

Apple

Or Google TV); a game console (e.g.,

or

) (ii) a An electronic dictionary; an electronic key; a camera; or at least one of the electronic framework or its functions.

In other embodiments, the electronic device may include at least one of: a medical device (e.g., an ambulatory medical device (e.g., a blood glucose monitoring device, a heart rate monitor, a blood pressure monitoring device, or a thermometer), a Magnetic Resonance Angiography (MRA) machine, a Magnetic Resonance Imaging (MRI) machine, a Computed Tomography (CT) scanner, or an ultrasound machine); a navigation device; a Global Positioning System (GPS) receiver; event Data Recorder (EDR); a Flight Data Recorder (FDR); a vehicle-mounted infotainment device; electronic devices for a vessel (e.g., a vessel navigation device and/or a gyrocompass); an avionics device; a security device; a vehicle head unit; industrial or domestic robots; at least one function of an Automatic Teller Machine (ATM) of a financial institution, a point of sale (POS) device of a retail store, or an internet of things device (e.g., a light bulb, various sensors, an electricity meter, a gas meter, a sprinkler, a fire alarm, a thermostat, a street light, an oven, sports equipment, a water heater, a heater, or a boiler, etc.), or a function thereof.

In some embodiments, the electronic device may include at least one of: a piece of furniture or building/structure; an electronic board; an electronic signature receiving device; digital signage devices, projectors; and various measuring instruments (e.g., a water meter, an electricity meter, a gas meter, or a wave meter) or at least one of the functions thereof.

Electronic devices according to embodiments of the present disclosure may also include combinations of two or more of the above devices. In some embodiments, the electronic device may be a flexible electronic device. Further, it will be apparent to those skilled in the art that the electronic device according to the embodiments of the present disclosure is not limited to the above-described device, and may have a novel device. Herein, the term "user" may indicate a person using an electronic device or a device using an electronic device (e.g., an artificial intelligence electronic device).

Overview of IoT System

Various embodiments for implementing IoT are described below. Fig. 1 illustrates a system including various items or IoT devices connected to the internet, in accordance with one embodiment.

Referring to fig. 1, the illustrated system may include a control device 110 connected to the internet 140, various IoT devices 120 to 130 connected to the control device 110, a first electronic device 100-1 connected to the control device 110 via short range communication and providing a user interface, and a second electronic device 100-2 remotely connected to the control device 110 through the internet 140. The first electronic device 100-1 may be identical to the second electronic device 100-2.

In one embodiment, the control device 110 may be a stand-alone device. In another embodiment, the control device 110 may include at least one of: home networking controller, set-top box, media device (e.g., Samsung Home)

Google

Apple

) Game console (e.g., Microsoft Windows)

Sony of the design reside in the sony

) A network access point, a security control panel, or a home climate controller, or any combination thereof.

The control device 110 may be wired and/or wirelessly connected to various external electronic devices (e.g., IoT devices 120-130) using various communication schemes. For example, the communication scheme may include at least one of wireless fidelity (WiFi), Bluetooth (BT), Bluetooth Low Energy (BLE), Zigbee, power line communication, infrared transmission (IR), and ultrasonic communication.

In one embodiment, the control device 110 may be connected to the IoT devices 120 to 130 and perform functions of controlling the IoT devices 120 to 130 and communicating data with the IoT devices 120 to 130. In other embodiments, the control device 110 may act as a gateway that collects data from the IoT devices 120-130 and forwards the collected data to other devices (e.g., servers or other gateway devices) on an external network over the internet 140.

In other embodiments, the control device 110 may be connected to at least one cloud. For example, the cloud may collect data from control device 110 and other devices similar to control device 110. The cloud may use the collected data to form big data. The collected data may be used for a specific purpose (e.g., advertising). In some embodiments, control device 110 may be connected to a personal cloud (e.g.,

etc.).

In the illustrated embodiment, the IoT devices 120-130 may include at least one of: household appliances (e.g., TV120, refrigerator 123, oven 125, washing machine, dryer, etc.), lighting system 122, fire alarm system 121, meters (e.g., electricity meter 126, gas meter, etc.), solar power generation system, sprinkler system 124, thermostat 127, or security system 128. In other embodiments, various other IoT devices may also or alternatively be connected to the control device 110.

Further, electronic devices 100-1 and 100-2 (e.g., smartphones or tablet computing devices) and/or optionally wearable devices (e.g., smart glasses 129 or smart watch 130) may be used to serve as a user interface for controlling device 110. For example, the electronic device 100-1 or 100-2 may control the various IoT devices 120-130 through the control device 110.

In one embodiment, electronic device 100-1 may connect directly or indirectly to control device 110 using a short-range communication scheme (e.g., WiFi, bluetooth, BLE, Zigbee, IR, ultrasound communication, etc.). In another embodiment, the electronic device 100-2 may be connected to the control device 110 through an internet network (e.g., the internet 140) or a cellular network.

The electronic device 100-1 may use different communication schemes depending on its location. For example, the electronic device 100-1 may communicate with the control device 110 using short-range communication when located near the control device 110, and the electronic device 100-1 may communicate with the control device 110 using an internet network or a cellular network when located at a location remote from the control device 110. In other embodiments, the control device 110 may be connected to the TV120, for example via a wire, and use the TV120 as a user interface.

In some embodiments, the IoT devices 120-1301 shown in fig. 1, or other IoT devices, may have various computing capabilities and/or communication capabilities. For example, IoT devices may have various performance capabilities categorized as in table 1 below.

TABLE 1

As shown in table 1 above, IoT devices 120-130 may use control signals according to one or more of the different protocols based on their respective categories. For example, class 4 ("regular") or higher class (i.e., classes 1-4) IP-enabled IoT devices (e.g., TV120, refrigerator 123, security system 128, etc.) include computing capabilities and communication means that can support internet protocol version 6(IPv6) protocol. In one embodiment, as shown in fig. 2a, the IoT device 200 may include at least one of: a media access control/physical (MAC/PHY) layer (e.g., radio), a receiver, a transmitter, a memory, an Operating System (OS), a processor, or input/output (I/O) logic, and thus may support the IPv6 protocol.

In another embodiment, class level 5 ("base") or lower class levels (i.e., class 5-7) of the IoT devices of table 1 (e.g., oven 125, washing machine, dryer, fire alarm system 121, meters (e.g., electricity meter 126, gas meter, etc.), sprinkler system 124, thermostat 127, etc.) may have only limited computing power and simple communication means, and thus may not support the IPv6 protocol. Such IoT devices (e.g., IoT device 210) may include only at least one of a PHY layer (e.g., radio), a receiver, a transmitter, a modulator, or a detector. Such IoT devices (e.g., IoT device 210) may not be able to support the IPv6 protocol due to their limited resources and communication capabilities.

In one embodiment, the control device 110 may perform one-way communication with low-power or low-end IoT devices (e.g., the fire alarm system 121). For example, the control device 110 may receive control signals from the low power IoT device using a BLE communication protocol.

In another embodiment, the control device 110 may perform bi-directional communication with high power and high end IoT devices (e.g., home appliances). For example, the control device 110 may send and receive signals to and from the high-end IoT devices using a WiFi communication protocol.

In yet another embodiment, the control device 110 may connect to the IoT devices using at least two or more different wireless communication schemes based on different characteristics (e.g., power consumption, communication range, etc.) of the IoT devices 120-130. For example, when connecting to multiple IoT devices, the control device 110 may connect to the IoT devices using different wireless communication schemes based on the power consumption (e.g., battery level) and/or communication range of the respective IoT devices.

In yet another embodiment, the control device 110 may send and receive data with multiple IoT devices on a time division basis. For example, when connecting to multiple IoT devices, the control device 110 may send and receive data with the respective IoT devices using different durations or intervals.

As described above, the control device 110 may be configured to efficiently control IoT devices 120-130 having different configurations and using different communication protocols at low cost. Further, the control device 110 may be located near the IoT device regardless of the location of the user. Thus, the control device 110 may collect data from the IoT devices continuously, at selected intervals, or during selected periods of time, and may control the IoT devices as needed.

Further, in the illustrated embodiment, the user interface of the control device 110 may be provided by other devices (e.g., the electronic devices 100-1 and 100-2, the smart watch 130, or the TV 120) for the convenience of the user. However, in other embodiments, the user interface may be integrated with the control device 110.

IoT control device

Referring to fig. 3-6, an IoT control device in accordance with various embodiments is described below. Fig. 3 shows an appearance of the IoT control device 300. The control device 110 shown in fig. 3 may be used as the control device 110 shown in fig. 1.

The control device 300 may include a housing 310, the housing 310 including a first surface 312 (e.g., a top surface), a second surface 313 (e.g., a bottom surface) facing away from the first surface 312, and at least one side surface 314 (e.g., a side surface), the side surface 314 at least partially surrounding a space between the first surface 312 and the second surface 313. At least one of the above surfaces of the housing 310 may have a curved surface or a flat surface. For example, at least a portion of the surfaces of the housing 310 may be formed of a translucent or opaque material.

According to one embodiment, one of the surfaces of housing 310 (e.g., side surface 314) may include a power connector 320 or a wired communication port (e.g., high-definition multimedia interface (HDMI) port 330, Universal Serial Bus (USB) port 340 (or a micro-USB port), optical audio output port 350, and Internet port 360 (e.g., an Ethernet port) configured to receive power for controlling device 300.

Fig. 4 illustrates an IoT control device in accordance with an embodiment. Control device 400 may include a bus 410, a processor 420, a memory 430 (e.g., Dynamic Random Access Memory (DRAM) and/or NAND flash memory), a power management module 450, an input/output interface 460, a communication interface 470, and an antenna (not shown) (e.g., a 2.4 gigahertz (GHz), 5GHz, or 60GHz directional antenna). The antennas may have a multiple-input multiple-output (MIMO) configuration. In some embodiments, the control device 400 may not include at least one of the above elements, or may also have other elements.

Bus 410 may include circuitry to connect the above-described elements (e.g., processor 420, memory 430, power management module 450, input/output interface 460, and communication interface 470, etc.) to one another and to provide communication (e.g., exchange control messages) between the elements.

Processor 420 may include one or more of the following: a Central Processing Unit (CPU), an Application Processor (AP), or a Communication Processor (CP). For example, processor 420 may perform operations or data processing for controlling the control and/or communication of at least one of the other elements of device 400.

According to one embodiment, processor 420 may communicate control signals and/or data with at least one of external

electronic devices

402 or 404 via communication interface 470. The external electronic device 402 may be an IoT device (e.g., any of the IoT devices 120-130 shown in fig. 1) or a device that provides a user interface (e.g., the electronic device 100-1 shown in fig. 1). The external electronic device 404 may also be an electronic device (e.g., the electronic device 100-2 shown in fig. 1) remotely connected to the control device 400. Processor 420 may also be connected through a communication interface 470 to a server 464 that is connected to a network 462 (e.g., a Local Area Network (LAN), a Wide Area Network (WAN), or the internet).

The memory 430 may include volatile and/or nonvolatile memory. The memory 430 may store instructions and/or data related to at least one of the other elements of the control device 400. As shown in fig. 4, the memory 430 may store software and/or programs 440. For example, programs 440 may include a kernel 441, middleware 443, an Application Programming Interface (API)445, and application programs 447. At least some of the kernel 441, middleware 443, or API445 may be referred to as an Operating System (OS).

The kernel 441 may control or manage system resources (e.g., the bus 410, the processor 420, the memory 430, etc.) for performing operations or functions implemented in other programs (e.g., the middleware 443, the API445, or the application programs 447). The kernel 441 may provide an interface for the middleware 443, the API445, or the application 447 to access various elements of the control device 400, thereby controlling or managing system resources.

The middleware 443 can function as a relay station for the API445 or the application 447 to communicate and exchange data with the kernel 441. The middleware 443 can perform control of work requests received from the application programs 447. For example, the middleware 443 can control (e.g., schedule or load balance) work requests using system resources of the control device 400 by assigning priorities to at least one of the applications 447.

The API445 may include interfaces or functions (e.g., instructions) for the application 447 to control the functions of the kernel 441 or the middleware 443. For example, the API445 may include at least one interface for file control, window control, image processing, or text control.

The power management module 450 may control power for driving the control device 400. For example, the power management module 450 may continuously supply the external power connected through the power connector 320 to the control device 400 such that the control device 400 is continuously driven. For example, to reduce power consumption of the control device 400, the power management module 450 may control an external power source connected through the power connector 320 to supply power to the control device 400 during the activation duration of the control device 400.

The input/output interface 460 may serve as an interface capable of forwarding instructions or data input from a user or other external devices to other elements of the control device 400. In addition, the input/output interface 460 may output instructions or data received from other elements of the control device 400 to a user or other external devices.

The communication interface 470 may establish communication between the control device 400 and an external device (e.g., the first external electronic device 402, the second external electronic device 404, or the server 464). For example, communication interface 470 may connect to network 462 via one or more or a combination of wireless communication or wired communication protocols and communicate with external devices.

The wireless communication may include short-range communication or long-range communication. The remote communication may use a cellular communication protocol, such as at least one of: long Term Evolution (LTE), LTE-advanced (LTE-a), Code Division Multiple Access (CDMA), wideband-CDMA (wcdma), Universal Mobile Telecommunications System (UMTS), wireless broadband (WiBro), or global system for mobile communications (GSM). Wireless communications may support existing wide area cells as well as small scale cells such as pico cells or femto cells. The short range wireless communication may include, for example, at least one of WiFi, bluetooth, BLE, Zigbee, IR, or ultrasound communication.

The wired communication may include, for example, at least one of Universal Serial Bus (USB), high-definition multimedia interface (HDMI), recommended standard 232(RS-232), power line communication, or Plain Old Telephone Service (POTS). Network 462 may include at least one of a telecommunications network, such as a computer network (e.g., a LAN or WAN), the internet, or a telephone network.

As described above, the first external electronic device 402 or the second external electronic device 404 may be an electronic device that provides a user interface for controlling the external electronic devices through the internet. In one embodiment, server 464 may comprise a group of one or more servers. In various embodiments, the control device 400 may connect and control external electronic devices using at least one module that is operatively or physically separate from the processor 420.

In one embodiment, the control device 400 may include an operating system, applications, and minimal storage for data. Most of the data acquired from the IoT devices may be stored in the cloud. In some embodiments, the software program of the application may include an IoT control-related portion and a user interface-related portion. In some embodiments, the control device 400 may not store the entire such software program, but may instead store or install only IoT control-related portions of the software program. The user interface related part of the software program may also be stored in the mobile device operating as a user interface.

In some cases, the control device 400 may be continuously connected to some IoT devices while periodically or intermittently accessing other IoT devices as needed. When the control device 400 is configured to further include the functionality of other devices (e.g., a TV set-top box, a media device, and/or a security panel), the control device 400 may also include hardware and/or software for performing the functionality of these other devices.

Fig. 5 is a schematic hardware block diagram illustrating a control device according to one embodiment. In the following description, the control device 500 may be the whole or a part of the control device 400 shown in fig. 4, for example. The illustrated control device 500 may include one or more of an Application Processor (AP)510, a communication module 520, a memory 530, a sensor module 540, an interface 550, a power management module 560, or an indicator 570.

The AP510 may drive an OS or an application program, thereby controlling a plurality of hardware or software components connected to the AP 510. The AP510 may also process various data including multimedia data or perform operations. For example, AP510 may be implemented as a system on a chip (SoC).

The communication module 520 (e.g., the communication interface 470) may perform data transmission/reception in communication between the control device 500 (e.g., the control device 400) and other electronic devices connected through a network. The communication module 520 may include a cellular module 521, a WiFi module 523, a Bluetooth (BT) module 525, an NFC module 527, and an RF module 529. Although the communication module 520 is illustrated in fig. 5 as including a number of different modules, the number and type of modules included therein may vary in other embodiments.

The cellular module 521 may provide internet services through a communication network (e.g., LTE-a, CDMA, WCDMA, UMTS, WiBro, GSM, etc.). In one embodiment, cellular module 521 may perform at least some of the functions that may be provided by AP 510. For example, the cellular module 521 may perform at least a portion of the multimedia control functions.

In another embodiment, the cellular module 521 may include a Communication Processor (CP). For example, the cellular module 521 may be implemented as a SoC. Although components such as cellular module 521 (e.g., a communications processor), memory 530, or power management module 560 are shown in fig. 5 as being separate from AP510, at least some of the functionality of these components (e.g., cellular module 521) may be implemented as part of AP 510.

In one embodiment, AP510 or cellular module 521 (e.g., a communications processor) may load instructions or data received from at least one of a non-volatile memory (which is connected to a respective one of AP510 or cellular module 521) or other component into a volatile memory and process the loaded instructions or data. AP510 or cellular module 521 may also store data received from or generated by at least one of the other components in a non-volatile memory.

At least one of the WiFi module 523, the BT module 525, or the NFC module 527 may include, for example, a processor for processing data transmitted/received through the respective module. Although the cellular module 521, the WiFi module 523, the BT module 525, or the NFC module 527 are shown as being separate from one another in fig. 5, at least some (e.g., two or more) of the cellular module 521, the WiFi module 523, the BT module 525, or the NFC module 527 may be included in a single Integrated Chip (IC) or IC package. For example, at least some of the processors for the cellular module 521, the WiFi module 523, the BT module 525, or the NFC module 527 may be implemented as one SoC (e.g., a communication processor corresponding to the cellular module 521 and a WiFi processor corresponding to the WiFi module 523 may be combined on one SoC).

The RF module 529 can perform transmission and reception of data using an RF signal. Although not shown, the RF module 529 may include a transceiver, a Power Amplification Module (PAM), a frequency filter, and a Low Noise Amplifier (LNA). Further, the RF module 529 may further include components (e.g., conductors, wires, etc.) for transmitting and receiving electromagnetic waves in free space using wireless communication.

Although the cellular module 521, the WiFi module 523, the BT module 525, and the NFC module 527 are illustrated as the shared RF module 529 in fig. 5, at least one of the cellular module 521, the WiFi module 523, the BT module 525, or the NFC module 527 may transmit and receive through a separate RF module.

Memory 530 may include internal memory 532 or external memory 534. For example, internal memory 532 may include at least one of: volatile memory (e.g., DRAM, Static Random Access Memory (SRAM), Synchronous Dynamic Random Access Memory (SDRAM), etc.) or non-volatile memory (e.g., one-time programmable read only memory (OTPROM), programmable ROM (prom), erasable programmable ROM (eprom), electrically erasable programmable ROM (eeprom), mask ROM, flash ROM, NAND flash, nor (nor) flash, etc.).

In one embodiment, the internal memory 532 may comprise a Solid State Drive (SSD). The external memory 534 may include a flash drive, such as Compact Flash (CF), Secure Digital (SD), micro secure digital (micro-SD), mini secure digital (mini-SD), speed card (xD), or memory stick. The external memory 534 may be operatively connected to the control device 500 through various interfaces. In another embodiment, the control device 500 may further include a storage device (or storage medium) such as a hard disk drive or the like.

The sensor module 540 may measure a physical quantity or detect an operation state of the control device 500 and then convert the measured or detected information or value into an electrical signal. The sensor module 540 may include, for example, an IR sensor 540A or an ultrasonic sensor 540B. The sensor module 540 may also include control circuitry for controlling one or more sensors therein.

The interface 550 may include, for example, HDMI 552, USB 554, optical interface 556, or internet port 558. The power management module 560 may manage power of the control device 500. Although not shown in fig. 5, the power management module 560 may include a power management IC (pmic), a charger IC, or a battery or fuel gauge.

For example, the PMIC may be mounted into an integrated circuit or SoC semiconductor. The charging scheme may be a wired charging scheme or a wireless charging scheme. The charger IC can charge the battery and can prevent the inflow of overvoltage or overcurrent from the charger. For example, the charger IC may include a charger IC for at least one of a wired charging scheme or a wireless charging scheme. Examples of the wireless charging scheme include a magnetic resonance scheme, a magnetic induction scheme, or an electromagnetic wave scheme. Additional circuitry for wireless charging, e.g., circuitry such as coil loops, resonant circuits, rectifiers, etc., may also be added. The indicator 570 may indicate a particular state of the control device 500 or a portion thereof (e.g., the AP 510), such as a startup state, a message state, a charging state, and the like.

At least one of the components of the control device according to the above-described embodiments may include one or more components or subcomponents, and the names of the components may be different according to the type of the control device. The control device may also comprise at least one of the above-mentioned components, omit some components, or further comprise additional components. In addition, some of the components of the above-described control apparatus may be combined and constructed as one entity, and may perform the same function as that of the corresponding component before the combination.

Fig. 6 illustrates a software stack of a control device for controlling an IoT device, according to one embodiment. In the illustrated embodiment, program modules 600 (e.g., programs 440) may include a kernel 610, middleware 620, an API660, and an application layer 670. At least a portion of the program module 600 may be preloaded onto a control device, such as the control devices 400 or 500 of fig. 4 and 5, or at least a portion of the program module 600 may be downloaded from a server of a particular manufacturer (e.g., IoT device manufacturer) or from an application store provided by a host platform (e.g., Android, iOS, Tizen, Firefox, Windows Mobile, Blackberry 10, etc.).

At least a portion of kernel 610 and middleware 620 may be referred to as an Operating System (OS) for controlling resources associated with a control device (e.g., control device 400). Various applications (e.g., application programs 447) running on the OS may be included on the application layer 670. Examples of operating systems may include

And

kernel 610 (e.g., kernel 441 of FIG. 4) may include, for example, a system resource manager (not shown) or a device driver (not shown). The system resource manager may control, allocate, or revoke system resources, etc. The system resource manager may include a process management unit, a memory management unit, a file system management unit, and the like. The device driver may include, for example, a bluetooth driver, a USB driver, a WiFi driver, or an inter-process communication (IPC) driver for managing each communication module 611a, 611b, 611c, or 611d to communicate with an external electronic device for internet communication with the control device.

In one embodiment, the control device (500 of fig. 5) may be configured to be removably coupled or connected to various communication modules (in hardware, firmware, and/or software). For example, when selling the control apparatus, the control apparatus includes only the first communication module. However, the control device may be configured to add a second communication module or replace the first module with the second module according to the user's needs. In this case, software for the communication module on kernel 610 may be added or updated. In some embodiments, to add a new communication hardware module, a separate slot may be included in the housing (e.g., 300 of fig. 3) of the control device, and corresponding software may be downloaded to the control device.

The middleware 620 may provide functions that are typically required by the application 670, or provide various functions to the application 670 through the API660 so that the application 670 can efficiently use limited system resources within the electronic device. In one embodiment, middleware 620 (e.g., middleware 443) can include at least one of: an application manager 641, a resource manager 642, a power manager 643, a database manager 644, a packet manager 645, a connection manager 646, an IoT control manager 647, a security manager 648, or a runtime repository 649.

The application manager 641 may manage, for example, a lifecycle of at least one of the applications 670. Resource manager 642 may manage resources, such as source code for at least any of applications 670, memory, storage space, and the like.

The power manager 643 may operate, for example, in conjunction with a basic input/output system (BIOS) to manage a battery or power source and provide power information for operating a control device. Database manager 644 may create, search, or change a database to be used by at least one of applications 670. The package manager 645 may manage installation or update of applications distributed in the form of package files.

The connection manager 646 may manage, for example, wireless connections, such as WiFi, bluetooth, etc. IoT control manager 647 may manage connections and controls with external electronic devices for control over IoT communications. The security manager 648 may provide general security functions for system security, user authentication, and the like.

Runtime library 649 may include, for example, library modules used by a compiler to add new functionality through a programming language during execution of application 670. In addition, the runtime library 649 may perform functions for input/output management, memory management, and/or arithmetic functions.

Middleware 620 may include a middleware module that forms a combination of the various functions of the above-described components. Middleware 620 may provide modules that are specific to a particular operating system to provide differentiated functionality. Further, middleware 620 may dynamically delete some of the existing components or add new components.

The API660 (e.g., API445) may include a collection of API programming functions and may have different configurations depending on the operating system. For example,or

An API set may be provided for a platform, and

two or more API sets may be provided for a platform.

In one embodiment, the application layer 670 may include one or more application programs (e.g., application programs 447). The application layer 670 may include one or more application programs 671a to 671c that may provide functionality such as media player, photo album, environmental information provision (e.g., air pressure, humidity, or temperature), and the like.

In the illustrated embodiment, the application layer 670 may include one or more application programs 672 a-672 b for controlling IoT devices. Each of the applications 672 a-672 b may include software appropriate for the respective IoT device or IoT device(s). In one embodiment, for example, the first application may include a program for controlling IoT lights from a first manufacturer, while the second application may include a program for controlling IoT lights from a second manufacturer.

In another embodiment, the first application may include a program for controlling IoT lights from various manufacturers, while the second application may include a program for controlling electronic home appliances from various manufacturers. In yet another embodiment, the application layer 670 may include applications, each configured to control a respective one of various types of IoT devices, as shown in table 1 above.

In one embodiment, at least a portion of the application program may be preloaded onto the control device (500 of FIG. 5) when the control device is manufactured. In another embodiment, at least a portion of the application may be installed on the control device after the user downloads the application from an application store or an internet website (e.g., a website of an IoT device manufacturer) after purchasing the control device.

In some embodiments, a general purpose application may be installed on the control device. In such embodiments, the control device may be provided with information regarding the type, class, classification, and/or manufacturer of the one or more IoT devices to be controlled by the control device. The control device may then use this information to support various IoT devices. In another embodiment, a combination of two or more of the above embodiments may be performed.

In various embodiments, at least a portion of program module 600 may be implemented in software, firmware, hardware, or a combination thereof. For example, at least a portion of the program modules 600 may be implemented (e.g., executed) by a processor (e.g., an application). At least a portion of the program modules 600 may include a module, program, routine, set of instructions, or process for performing one or more functions. Combinations of any of the above embodiments may also be implemented.

User interface for IoT control devices

Referring to fig. 7, an electronic device 700 for providing a user interface for an IoT control device (e.g., 500 of fig. 5) in accordance with one embodiment will be described below. The illustrated electronic device 700 may include a bus 710, a processor 720, a memory 730, an input/output interface 750, a display 760, and a communications interface 770. In certain embodiments, at least one of the above-described components may be omitted and/or additional or alternative components may be included. In one embodiment, the mobile device 700 may be a mobile device, such as a smartphone, tablet, laptop, or wearable device.

Bus 710 may include circuitry to interconnect the above-described components (e.g., processor 720, memory 730, input/output interface 750, display 760, or communication interface 770) and to provide communication (e.g., control messages) between the components.

Processor 720 may include one or more of a CPU, AP, or CP. For example, processor 720 may perform operations or data processing for control and/or communication of at least one other component of electronic device 700.

The memory 730 may include volatile and/or nonvolatile memory. Memory 730 may store instructions or data related to at least one other component of electronic device 700. In fig. 7, memory 730 may store software and/or programs 740. For example, programs 740 may include a kernel 741, middleware 743, an API745, and application programs 747 (e.g., applications), among others. At least a portion of the kernel 741, the middleware 743, or the API745 may be collectively referred to as the OS.

The kernel 741 may control or manage system resources (e.g., the bus 710, the processor 720, or the memory 730) for performing operations or functions performed in other programs (e.g., the middleware 743, the API745, or the application 747). The kernel 741 may provide an interface for the middleware 743, the API745, or the application 747 to access individual components of the electronic device 700 to control or manage system resources.

The middleware 743 may function as a relay station for the API745 or the application 747 to communicate with the kernel 741 and exchange data with the kernel 741. Middleware 743 can provide control over work requests received from applications 747. For example, middleware 733 may provide control (e.g., scheduling or load balancing) of work requests by allocating at least one of applications 747 to use system resources of electronic device 700.

The API745 may include interfaces or functions (e.g., instructions) for causing the application program 747 to control the functions of the kernel 741 or the middleware 743. For example, API745 may include at least one interface for file control, window control, image processing, or character control, for example.

The input/output interface 750 may serve as an interface that provides instructions or data input from a user or other external devices to other components of the electronic device 700 and may output instructions or data received from other components of the electronic device 700 to the user or other external devices.

For example, display 760 may include a Liquid Crystal Display (LCD), a Light Emitting Diode (LED) display, an Organic Light Emitting Diode (OLED) display, a micro-electro-mechanical system (MEMS) display, or an electronic paper display. For example, display 760 may display various information (e.g., text, images, videos, icons, symbols, etc.) to a user. Display 760 may include a touch screen that receives touch, gesture, and/or proximity inputs from a stylus (e.g., hovering inputs), an electronic pen, or a portion of a user's body (e.g., a finger).

Communication interface 770 may connect electronic device 700 to external devices (e.g., control device 702, external electronic device 704, or server 706). For example, the communication interface 770 may connect to the network 762 by wireless communication or wired communication, and thereby provide communication with external devices.

For example, the wireless communication may use a cellular communication protocol, such as at least one of: LTE, LTE-A, CDMA, WCDMA, UMTS, WiBro, or GSM. Further, the wireless communication may include at least one of WiFi, bluetooth, BLE, Zigbee, IR, or ultrasound communication. For example, the wired communication may utilize at least one of USB, HDMI, RS-232, or POTS (plain old telephone service). Network 762 may include at least one of a telecommunications network, such as a computer network (e.g., a LAN or WAN), the internet, or a telephone network.

The external electronic device 704 may be the same type of device as the electronic device 700 or a different type of device. In one embodiment, the server 706 may include a group of one or more servers.

In various embodiments, all or some of the operations that may be performed by the electronic device 700 may be performed with another electronic device or by multiple electronic devices (e.g., the control device 702, the external electronic device 704, or the server 706). In one embodiment, when the electronic device 700 performs a function or service automatically or in response to a request, the electronic device 700 may send a request to other devices (e.g., the control device 702, the external electronic device 704, or the server 706) to perform at least a portion of the function or service instead of or in addition to performing the function or service in person. Other electronic devices (e.g., the control device 702, the external electronic device 704, or the server 706) may perform functions requested by the electronic device 700 or additional functions and provide the results of the performance to the electronic device 700. The electronic device 700 may then use the received results as is or further process them and provide the requested function or service to other electronic devices. For example, cloud computing, distributed computing, or client-server computing techniques may be used for this purpose.

In various embodiments, electronic device 700 may obtain information about the external electronic device through at least one module operatively or physically separate from processor 720 and install software programs related to the external electronic device. In one embodiment, the processor 720 may obtain information about at least one external electronic device 704 (e.g., an IoT device). For example, the processor 720 may obtain such information by using at least one of a code or a tag attached to or provided with the at least one external electronic device 704. The code may include, for example, at least one of a barcode or a QR code. The tag may comprise at least one of an RFID, NFC tag or BLE tag, for example.

In one embodiment, the processor 720 may derive or estimate location information of at least one external electronic device 704. For example, such electronic devices 704 may be IoT devices that are at least temporarily at fixed locations. When obtaining information of at least one external electronic device 704 for control through the IoT environment, the processor 720 may consider the location of the electronic device 700 at that time as the location of the external electronic device 704. For example, the processor 720 may estimate the location of the electronic device 704 using a positioning technique such as simultaneous positioning and mapping (SLAM), LTE positioning protocol extensions (LPPe)2.0, and so on.

In one embodiment, the processor 720 may access the server 706 and receive at least a portion of a software program corresponding to information about the external electronic device 704 and then install the received portion of the software program onto the electronic device 700. For example, processor 720 may control display 760 to display a list of categories received from an application store via communication interface 770. If the user selects an IoT-related category from the list of categories displayed on display 760, processor 720 may then control display 760 to display a list of software programs included in the selected category.

If a software program associated with the external electronic device 704 is selected from the list of software programs, the processor 720 may receive at least a portion of the selected software program and install the received portion of the software program onto the electronic device 700. In one embodiment, the processor 720 may install at least a portion of the software program on the electronic device 700, and the at least a portion may provide a user interface for the external electronic device 704. In another embodiment, the processor 720 may install at least some portions of the software programs on the electronic device 700, and these portions may provide user interface and control for the external electronic device 704.

In one embodiment, the processor 720 may send information about the external electronic device 704 to the control device 702. For example, the processor 720 may send identification information of the external electronic device 704 or identification information of a software program related to the external electronic device 704 to the control device 702.

In other embodiments, processor 720 may display icons for IoT applications on display 760. When executing an application for an IoT, display 760 may then display a list of software programs for controlling the IoT device.

In another embodiment, the display 760 may display software program folders for the IoT. For example, when the software program folder is selected, the display 760 may display a software program list including items or icons controllable by the electronic device 700, the items or icons representing software programs for controlling an external electronic device.

FIG. 8 shows program modules of an electronic device according to one embodiment. In the illustrated embodiment, program modules 800 (e.g., programs 740) may include an OS for controlling resources associated with an electronic device (e.g., electronic device 700) and/or various applications (e.g., application programs 747) running on an operating system. The operating system may be, for example

And the like.

The program modules 800 may include a kernel 810, middleware 830, an API860, and/or an application (application program) 870. At least a portion of the program modules 800 may be preloaded on the electronic device or downloaded from a server.

Kernel 810 (e.g., kernel 741 of fig. 7) may include, for example, system resource manager 811 or device driver 812. The system resource manager 811 can provide control of, allocation of, revocation of, etc. system resources. For example, the system resource manager 811 may include a process management unit, a memory management unit, or a file system management unit. The device driver 812 may include a display driver, a camera driver, a bluetooth driver, a shared memory driver, a USB driver, a keypad driver, a WiFi driver, an audio driver, a battery driver, a touch driver, or an IPC driver.

Middleware 830 may provide functionality typically required by application 870 or provide various functionality to application 870 through API860 such that application 870 may efficiently utilize limited system resources within an electronic device. In fig. 8, middleware 830 (e.g., middleware 743) may include, for example, at least one of: runtime library 835, application manager 841, window manager 842, multimedia manager 843, resource manager 844, power manager 845, database manager 846, packet manager 847, connection manager 848, notification manager 849, location manager 850, graphics manager 851, or security manager 852.

Runtime library 835 may include library modules used by a compiler to add new functionality through a programming language while running application 870. The runtime library 835 may perform functions such as input/output, memory management, and arithmetic functions.

The application manager 841 may manage a lifecycle of at least one of the applications 870. The window manager 842 may manage GUI resources used in the screen. The multimedia manager 843 may determine formats required to play various media files and encode or decode the media files using codecs appropriate to the respective formats. The resource manager 844 may manage resources, such as source code for at least one of the applications 870, memory, storage space, and the like.

The power manager 845 may operate in conjunction with the BIOS, manage the battery or power supply, and may provide power information for operating the electronic device. Database manager 846 may create, search, or change a database to be used in at least one of applications 870. The package manager 847 may manage installation or update of applications distributed in the form of package files.

The connection manager 848 may manage wireless connections such as WiFi or bluetooth. The notification manager 849 may display or notify events such as incoming messages, appointments, proximity notifications, etc., preferably in a manner that does not disturb the user. The location manager 850 may manage location information of the electronic device. The graphic manager 851 may manage graphic effects, or user interfaces, to be provided to the user. The security manager 852 may provide general security functions for system security, user authentication, and the like. In one embodiment of the present disclosure, when an electronic device (e.g., electronic device 700) includes telephony functionality, middleware 830 may also include a telephony manager for managing voice or video telephony functionality of the electronic device.

Middleware 830 may include a middleware module that forms a combination of the various functions of the above-described components. For example, the middleware 830 may provide modules that are specialized according to an operating system to provide differentiated functionality. The middleware may dynamically delete some of the existing components or add new components.

The API860 (e.g., API 745) may include a set of API programming functions and may be set to have different configurations according to different operating systems. For example,

or

An API set may be provided for a platform, and

two or more API sets may be provided for a platform.

Applications 870 (e.g., applications 747) may include one or more applications, a respective one or combination of which provides at least one of the functions of IoT user interface 871, dialer 872, short message service/multimedia message service (SMS/MMS)873, Instant Message (IM)874, browser 875, camera 876, alarm 877, contacts or directory 878, device settings 879, email 880, calendar 881, media player 882, gallery 883, and watch 884. Although not shown, the applications 870 may also include one or more applications for providing functionality for healthcare (e.g., measuring an amount of exercise, blood glucose level, or blood pressure), providing environmental information, and so forth.

For example, the application 870 may include an application (hereinafter referred to as an "information exchange application") that supports information exchange between (1) an electronic device (e.g., the electronic device 700) and (2) an external electronic device or a control device. For example, the information exchange application may include a notification relay application for relaying specific information to the control device or a device management application for managing the external electronic device.

For example, the notification relay application may include functionality to relay notification information generated in other applications of the electronic device (e.g., an SMS/MMS application, an email application, a healthcare application, an environmental information application, etc.) to the external electronic device. Further, the notification relay application may receive notification information from the control device and provide the received notification information to the user.

In one embodiment, the device management application may manage (e.g., install, delete, or update) the functionality. The function may include, for example, turning on/off the external electronic device itself (or some component thereof). The functions may also or alternatively include adjusting the brightness (or resolution) of a display of an external electronic device in communication with the electronic device. In other embodiments, the functionality may include managing applications running on or services (e.g., telephony services or messaging services) provided by the external electronic device.

In one embodiment, the applications 870 may include applications based on attributes or types of external electronic devices. For example, when the electronic device is an ambulatory medical device, the application 870 may include a healthcare application.

The application 870 may include an application received from an external electronic device (e.g., a server or an electronic device). The applications 870 may include preloaded applications or third party applications that may be downloaded from a server. In other embodiments, various components of the program module 800 in the illustrated embodiment may be mentioned differently or configured differently depending on the operating system.

At least a portion of program module 800 may be implemented in software, firmware, hardware, or a combination thereof. For example, the portion of the program module 800 may be implemented (e.g., executed) by a processor (e.g., an application program). The portion of program module 800 may include, for example, a module, program, routine, set of instructions, or process for performing one or more functions.

FIG. 9 is a block diagram illustrating an electronic device according to one embodiment. In the following description, electronic device 900 may form all or a portion of electronic device 700, such as shown in FIG. 7. Referring to fig. 9, electronic device 900 may include one or more of the following: an Application Processor (AP)910, a communication module 920, a Subscriber Identity Module (SIM) card 924, memory 930, a sensor module 940, an input device 950, a display 960, an interface 970, an audio module 980, an image sensor module 991, a power management module 995, a battery 996, an indicator 997, or a motor 980.

The AP910 may drive an OS or application and control a number of hardware or software components connected or coupled to the AP 910. The API 910 may also process various types of data including multimedia data or perform operations. For example, AP910 may be implemented as a SoC. Although not shown, the AP910 may also include a Graphics Processing Unit (GPU).

The communication module 920 (e.g., the communication interface 770) may perform data transmission and reception in communication between the electronic device 900 (e.g., the control device 700) and other electronic devices (e.g., the control device 702) connected through a network. In one embodiment, the communication module 920 may include a cellular module 921, a WiFi module 923, a BT module 925, a GPS module 927, an NFC module 928, or an RF module 929.

The cellular module 921 may provide voice telephony, video telephony, text services, internet services, etc. through a communication network (e.g., LTE-a, CDMA, WCDMA, UMTS, WiBro, or GSM). Further, the cellular module 921 may use a subscriber identification module (e.g., SIM card 924) to identify or authenticate electronic devices within the communication network. Further, the cellular module 921 may perform at least some of the functions that may be provided by the AP 910. For example, the cellular module 921 may perform at least a portion of the multimedia control function.

In one embodiment, although not shown, the cellular module 921 may include a Communication Processor (CP). The cellular module 921 may also be implemented as an SoC. In fig. 9, although components such as the cellular module 921 (e.g., the communication processor), the memory 930, and the power management module 995 are shown separate from the AP910, the AP910 may be implemented to include at least some of the functions of the above-described components (e.g., the cellular module 921).

The AP910 or the cellular module 921 (e.g., a communication processor) may load instructions or data received from a non-volatile memory connected to at least one of the AP910 and the cellular module 921, or other components, to a volatile memory and process the loaded instructions or data. The AP910 or the cellular module 921 may also store data received from or generated by at least one of the other components in the non-volatile memory.

Although not shown in fig. 9, each of the WiFi module 923, the BT module 925, the GPS module 927, or the NFC module 928 may include a processor for processing data transmitted/received through the respective module. Further, although cellular module 921, WiFi module 923, BT module 925, GPS module 927, or NFC module 928 are shown as separate blocks, at least some (e.g., two or more) of cellular module 921, WiFi module 923, BT module 925, GPS module 927, or NFC module 928 may be included within one IC or IC package. For example, at least some of the processors (e.g., a communication processor corresponding to the cellular module 921 and a WiFi processor corresponding to the WiFi module 923) each corresponding to the cellular module 921, the WiFi module 923, the BT module 925, the GPS module 927, or the NFC module 928 may be implemented as one SoC.

The RF module 929 may perform transmission and reception of data using an RF signal. Although not shown, the RF module 929 may include a transceiver, a Power Amplifier Module (PAM), a frequency filter, and a Low Noise Amplifier (LNA). The RF module 929 may also include components (e.g., conductors, wires, etc.) for transmitting and receiving electromagnetic waves in free space in wireless communication. Although the cellular module 921, the WiFi module 923, the BT module 925, the GPS module 927, and the NFC module 928 are illustrated as sharing one RF module 929 in fig. 9, in other embodiments, at least one of the cellular module 921, the WiFi module 923, the BT module 925, the GPS module 927, or the NFC module 928 may transmit and receive through a separate RF module.

The RF module 929 may include at least one of a main antenna or a sub-antenna operatively connected to the electronic device 900. For example, the communication module 920 may support Multiple Input Multiple Output (MIMO) using the main antennas and the sub antennas.

The SIM card 924 may be a card that includes a subscriber identity module and may be inserted into a slot provided in the electronic device. The SIM card 924 may include unique identification information (e.g., an integrated circuit card id (iccid)) or subscriber information (e.g., an International Mobile Subscriber Identity (IMSI)).

The memory 930 includes an internal memory 932 or an external memory 934. For example, the internal memory 932 may include at least one of: volatile memory (e.g., DRAM, SRAM, SDRAM, etc.) or non-volatile memory (e.g., OTPROM, PROM, EPROM, EEPROM, mask ROM, flash ROM, NAND flash, NOR flash, etc.).

In one embodiment, the internal memory 932 may be a Solid State Drive (SSD), and the external memory 934 may include a flash drive, such as a CF (compact flash), SD (secure digital), micro-SD (micro secure digital), mini-SD (mini secure digital), xD (ultra speed card), or memory stick. The external memory 934 may be operatively connected to the electronic device 900 through various interfaces. For example, the electronic device 900 may also include a storage device (or storage medium) such as a hard disk drive or the like.

The sensor module 940 may meter a physical quantity or detect a state of the electronic device 900 and convert the metered or detected information into an electrical signal. The sensor module 940 may include at least one of: a gesture sensor 940A, a gyroscope sensor 940B, an air pressure sensor 940C, a magnetic sensor 940D, an acceleration sensor 940E, a grip sensor 940F, a proximity sensor 940G, a color sensor 940H (e.g., a red-green-blue (RGB) sensor), a biophysical sensor 940I, a temperature/humidity sensor 940J, an illumination sensor 940K, and an Ultraviolet (UV) sensor 940M. Although not shown, the sensor module 940 may additionally or alternatively include electronic nose (E nose) sensors, Electromyography (EMG) sensors, electroencephalography (EEG) sensors, and Electrocardiogram (ECG) sensors, IR sensors, ultrasound sensors, iris sensors, fingerprint sensors, and the like. The sensor module 940 may also include a control circuit for controlling at least one of the sensors included therein.

Input devices 950 may include a touch panel 952, a (digital) pen sensor 954, keys 956, or an ultrasonic input device 958. The touch panel 952 may recognize, for example, a touch input using at least one of a capacitive scheme, a pressure-sensitive scheme, an infrared scheme, or an ultrasonic scheme. The touch panel 952 may also include a control circuit. In capacitive schemes, physical touch or proximity (hover) recognition is possible. The touch panel 952 may also include a tactile layer so that the touch panel 952 can provide a tactile response to a user.

The (digital) pen sensor 954 may be implemented using the same or similar method as receiving a user's touch input or a separate identification patch (e.g., a pad manufactured by Wacom). Keys 956 may include physical or mechanical buttons, optical keys, or a keypad. The ultrasonic input device 958 includes an input device for generating an ultrasonic signal and a microphone for sensing sound waves or sound waves, thereby identifying the data so that it can perform wireless identification or sensing. In other embodiments, electronic device 900 may receive user input from an external device (e.g., a computer or server) connected to electronic device 900 using communication module 920.

Display 960 (e.g., display 760) may include a panel 962, a holographic device 964, and/or a projector 966. For example, the panel 962 may include an LCD, AMOLED, and the like. The panel 962 may be implemented as flexible, transparent, or wearable. The panel 962 may also be implemented as one module together with the touch panel 952. The hologram device 964 can display a three-dimensional image in the air by using interference of light. The projector 966 may display images by projecting light onto a screen, which may be internal or external to the electronic device 900. In one embodiment, the display 960 may also include control circuitry for the control panel 962, the holographic device 964, and/or the projector 966.

The interface 970 may include HDMI972, USB974, optical interface 976, or D-sub 978. Although not shown, interface 970 may additionally or alternatively include a mobile high definition link (MHL) interface, a Secure Digital (SD) card/multimedia card (MMC) interface, and/or an infrared data association (IrDA) standard interface.

The audio module 980 may convert sound into electrical signals and vice versa. For example, audio module 980 may process sound information input or output through speaker 982, receiver 984, headphones 986, and/or microphone 988.

The image sensor module 991 captures still images and moving images. For example, the image sensor module 991 may include one or more image sensors (e.g., front sensors or rear/back sensors), lenses, an image signal processor (ISP, not shown), and a flash (e.g., an LED or xenon lamp).

The power management module 995 can manage power for the electronic device 900. Although not shown, the power management module 995 may include a PMIC, a charger IC, or a battery or fuel gauge.

The PMIC may be mounted in an integrated circuit or SoC semiconductor, for example. The charging scheme may be a wired charging scheme or a wireless charging scheme. The charger IC can charge the battery and can prevent the inflow of overvoltage or overcurrent from the charger. The charger IC may include a charger IC for at least one of a wired charging scheme or a wireless charging scheme. Examples of the wireless charging scheme include a magnetic resonance scheme, a magnetic induction scheme, and an electromagnetic wave scheme. Supplementary circuits for wireless charging, such as coil loops, resonant circuits, rectifiers, etc. may also be added.

The battery gauge may measure the level of the battery 996, the voltage, current, and/or temperature during charging. The battery 996 may store or generate power, and may use the stored or generated power to power the electronic device 900. For example, the battery 996 may include a rechargeable battery or a solar cell.

The indicator 997 may indicate a particular state of the electronic device 900 or a component thereof (e.g., the AP910), such as a startup state, a message state, a charging state, etc. The motor 998 may convert the electrical signal into mechanical vibrations. Although not shown, the electronic device 900 may also include a processing device (e.g., GPU) for mobile TV support that processes standard media data for Digital Multimedia Broadcasting (DMB), Digital Video Broadcasting (DVB), media streaming, and the like.

Each of the above components of the electronic device of the above embodiments of the present disclosure may include one or more components. The names of the respective components may be different according to the electronic device. Further, the electronic device may include at least one of the above components, and some components may be omitted or additional components may be further included. In other embodiments, some of the components of the electronic device of the above-described embodiments may be combined and implemented as one entity, while still performing the functions of the respective components equally well before combination.

Process for connecting an IoT device to an IoT control device

In the embodiments described above with reference to fig. 1 to 9, various IoT devices may be controlled via a connection with an IoT control device. For user convenience, such IoT devices may be controlled using an electronic device such as a smartphone or the like. However, since various IoT devices are manufactured by many different manufacturers, it may be difficult for consumers to connect such various IoT devices to an IoT control device. It may also be desirable to easily connect multiple IoT devices to an IoT control device. In some embodiments, a simple low-end IoT device may not include a user interface, or even a connection port (e.g., a USB port). Therefore, a method of connecting such IoT devices to an IoT control device and/or a user interface device is needed. In addition, an efficient and simple method of connecting an IoT device to an IoT control device may be needed.

Referring to fig. 10a to 10f and fig. 11, one embodiment of a method of connecting an IoT device to an IoT control device is described below. Referring to fig. 10a, a first electronic device 1000 (e.g., electronic device 700 of fig. 7) may be a mobile device, such as a smartphone, and a second electronic device 1010 (e.g., control device 400 of fig. 4) may be an IoT control device.

As described above with reference to fig. 7-9, the mobile device 1000 may serve as a user interface for the IoT control device 1010. Additionally, as described above with reference to fig. 1-6, IoT control device 1010 may control various external electronic devices or IoT devices 1020. In the illustrated embodiment, IoT control device 1010 may control IoT device 1020, such as a smart light (alternatively referred to as a smart light bulb, a smart light, or smart lighting). However, in other embodiments, IoT device 1020 may be one or more of a variety of other IoT devices (e.g., 120-130 of fig. 1).

Referring back to fig. 10a, the mobile device 1000 and the IoT control device 1010 have previously connected to each other and communicated with each other. The smart lamp 1020 may not include an external port and may have a simple function and configuration (similar to the configuration shown in, for example, fig. 2 b). Therefore, in order to control the smart lamp 1020 using the IoT control device 1010, the smart lamp 1020 may need to be connected to the IoT control device 1010. In this case, IoT control device 1010 may not be easily moved because it is typically located in a particular location. In this case, one embodiment of a method for connecting and installing the smart electric lamp 1020 using the mobile device 1000 will be described below with reference to fig. 10b to 10f and fig. 11.

(1) IoT device identification

Referring to fig. 10b and 11, in operation 1101, the mobile device 1000 may acquire information about an IoT device (i.e., the smart lamp 1020). For example, the mobile device 1000 may obtain information about the IoT device 1020 (e.g., identification information of the smart lamp 1020) using at least one of a code or a tag attached to the smart lamp 1020 or provided with the smart lamp 1020. Such a code or label provided with the smart electric lamp 1020 may be, for example, in a manual or user guide, or located inside or on the box or package of the smart electric lamp 1020.

Referring to fig. 10b, the mobile device 1000 may obtain information 1032 about the smart electric lamp 1020. For example, the mobile device 1000 may acquire information about the smart lamp 1020 by recognizing the QR code 1030 attached to the smart lamp 1020 with a built-in camera.

In another embodiment, the code or label attached to the smart light 1020 or provided with the smart light 1020 (e.g., included within a usage manual) or attached to the packaging of the IoT device 1020 may be, for example, a barcode. In yet another embodiment, the mobile device 1000 may also recognize the smart electric lamp 1020 by performing an augmented reality mechanism or a three-dimensional (3D) recognition scheme using at least one of a camera, a wearable device, or smart glasses built into the mobile device 1000.

In yet another alternative, the mobile device 1000 may use a code or tag attached to the smart lamp 1020 or provided with the smart lamp 1020 or attached to the packaging of the smart lamp 1020 in conjunction with (1) an augmented reality mechanism or a 3D identification scheme to obtain information about the smart lamp 1020.

In another embodiment, the mobile device 1000 may use one or more RFIDs to obtain information about the smart lamp 1020 (e.g., identification information of the IoT device 1020). In yet another embodiment, the mobile device 1000 may obtain information about the smart lamp 1020 from an NFC tag or BLE tag attached to the smart lamp 1020. In yet another embodiment, the mobile device 1000 may obtain information about a single IoT device or about a set or group of IoT devices from a single code or tag.

(2) Downloading and installing IoT device-related software on mobile devices

Referring to fig. 10c and 11, at operation 1103, the mobile device 1000 may access a server 1040 (e.g., server 706 or 1040) that includes application storage.

Thereafter, at operation 1105, the mobile device 1000 may receive or download a software program related to the IoT device 1020 from a server. For example, the mobile device 1000 may transmit identification information of the smart lamp 1020 to a server and then receive a corresponding software program related to the smart lamp 1020 from the server.

At operation 1107, the mobile device 1000 may install the software program related to the IoT device 1020 received from the server. For example, as shown in fig. 10c, the mobile device 1000 may access an external server 1040 including an application Store (e.g., Google Play or Apple App Store) and receive at least a portion of the software programs related to the smart lamp 1020 from the external server 1040. Thereafter, the mobile device 1000 may install the received portion of the software program.

In one embodiment, the mobile device 1000 may receive and install at least a portion of a software program for a user interface of the smart light 1020. In another embodiment, the mobile device 1000 may receive and install at least a portion of the software program for the user interface of the smart lamp 1020 and at least a portion of the software program for controlling the smart lamp 1020 to be used by the IoT control device. In this case, portions of the software program for controlling the smart lamp 1020 may be temporarily stored in the mobile device 1000, then transmitted to the IoT control device 1010 (described below), and then removed from the mobile device 1000.

In another embodiment, the mobile device 1000 may (1) receive and install a software program for a user interface of the smart lamp 1020 and (2) receive, instead of the software, only identification information of the smart lamp 1020 for controlling the smart lamp 1020. In this case, the identification information of the smart lamp 1020 may be sent to the IoT control device 1010, as will be described in more detail below.

In another embodiment, the user may identify and select a desired application from an application Store (e.g., Google Play or Apple App Store) based on: the name of the IoT device and/or its class, and then download and install the identified application onto the mobile device 1000. In one embodiment, an app store may provide apps for IoT devices by category.

Referring to fig. 12, another embodiment of a method of using operations in place of operations 1101 through 1105 of fig. 11 is described. Referring to fig. 12, when a server (e.g., server 706 or 1040) is accessed in operation 1103 of fig. 11, the mobile device 1000 may receive information about application storage from the server 1040 in operation 1201.

At operation 1203, the mobile device 1000 may display information about the software program related to the smart light 1020 on its display screen (e.g., display 760) using the received information about the application storage.

Referring to fig. 13, 14a and 14b, one embodiment of implementation details of operation 1203 of fig. 12 is described below. Referring to fig. 13, after receiving information about an application storage from a server at operation 1201 of fig. 12, the mobile device 1000 may display a category list included in the application storage on a display (e.g., display 760) at operation 1301. For example, as shown in fig. 14a, the mobile device 1000 may display a list of categories 1400 included in an application store, such as games, IoT1410, education, traffic, finance, weather, news and magazines, decorations, and the like, on a display.

In operation 1303, the mobile device 1000 may receive a user input for selecting a category corresponding to an IoT in a list of categories displayed on the display. For example, referring again to fig. 14a, the mobile device 1000 may receive a selection of an IoT category 1410 from the category list 1400 shown on the display.

In operation 1305, in response to the selection of the category corresponding to the IoT, the mobile device 1000 may display a list of IoT-related software programs on the display. The list may also include information about software programs associated with various external electronic devices. In one embodiment, as shown in fig. 14b, the mobile device 1000 may display a list of software programs in the IoT category 1410 on its display, such as a light control application 1420, a security control application 1430, a gas control application 1440, and the like.

At operation 1205 of fig. 12, the mobile device 1000 may receive a selection input of a software program related to an external electronic device (e.g., a smart lamp 1020) displayed on a display. In operation 1207, in response to the selection, the mobile device 1000 may transmit a signal (e.g., the smart lamp 1020) requesting a software program related to the external electronic device to the server 1040. In operation 1209, the mobile device 1000 may receive a software program (e.g., the smart lamp 1020) related to the external electronic device from the server 1040 as a response to the request signal.

(3) Downloading and installing IoT device-related software on an IoT control device

Referring to fig. 10d and 11, at operation 1109, the mobile device 1000 may transmit information related to the IoT device 1020 to a control device (e.g., the control device 702 or the IoT control device 1010). In one embodiment, the mobile device 1000 may send identification information of the IoT device or identification information of a software program related to the IoT device to the control device. In another embodiment, the mobile device 1000 may send at least a portion of the software program received from the server-related IoT device to the control device. In such embodiments, the mobile device 1000 may delete or remove at least a portion of the software program (e.g., a control-related portion of the software program) after transferring the portion to the control device. In yet another embodiment, the mobile device 1000 may send an identifier of a software program related to the IoT device received from the server to the control device.

Referring to fig. 10d, for example, the mobile device 1000 may transmit information related to the smart lamp 1020 to the IoT control device 1010. For example, the mobile device 1000 may send identification information of the smart lamp 1020 or identification information of a software program related to the smart lamp 1020 (e.g., identification information of a software program for control) to the IoT control device 1010. In another embodiment, the mobile device 1000 may transmit at least a portion of the software program received with respect to the smart light 1020 to the IoT control device 1010 at operation 1105 (e.g., a control-related portion of the program). Further, the mobile device 1000 may then delete at least a portion of the software program (e.g., the control-related portion of the program) that was transmitted to the IoT control device 1010.

In yet another embodiment, the mobile device 1000 may send an identifier for the software program received by the smart light 1020 to the IoT control device 1010 at operation 1105. Thereafter, IoT control device 1010, for example, may retrieve the software program using the received identifier.

As shown in fig. 10e, IoT control device 1010 may install a software program 1091d for controlling smart lamp 1020 at application layer 1090 of program module 1050 based on information related to smart lamp 1020 or information related to at least a portion of smart lamp 1020.

In one embodiment, IoT control device 1010 may receive software programs related to smart lamp 1020 from mobile device 1000 through communication module 11061a of kernel 1060 and install the received software programs on application layer 1090. In another embodiment, IoT control device 1010 may receive a software program related to smart lamp 1020 from an application memory through communication module 31061c of kernel 1060 and install the received software program on application layer 1090.

In yet another embodiment, IoT control device 1010 may preload or download and install a generic application (e.g., I091a) (e.g., a software module capable of using a particular communication protocol). Using such a generic application, IoT control device 1010 may provide software functionality for the new IoT device by simply registering the new IoT device with the generic application based on information or messages received from mobile device 1000 for the IoT device (rather than installing the new software program for the corresponding new IoT device).

In another embodiment, IoT control device 1010 may be pre-loaded or downloaded in advance and then have default control software (e.g., 1092a) installed, which may be for a particular IoT device or a plurality of different kinds or brands of IoT devices. In this case, IoT control device 1010 may communicate with the new IoT device by simply registering the new IoT device with the default control software based on information or messages received from mobile device 1000, as described above.

As described above, after the software or control module for controlling the smart lamp 1020 is installed on the IoT control device 1010, using the mobile device 1000, the IoT control device 1010 may thereafter control the smart lamp 1020 as needed. For example, IoT control device 1010 and smart lamp 1020 may exchange control signals for control and status checking with each other.

In some embodiments, at least some of the IoT device-related software programs installed on the mobile device 1000 and/or the IoT control device 1010 may be associated with the app store that provided the program or the manufacturer's website IoT device.

(4) Providing an IoT device-related user interface on a mobile device

Referring to fig. 10f and 11, at operation 1111, mobile device 1000 may display user interface 1010 related to IoT device 1020 on a display (e.g., display 760) using a software program related to IoT device 1020 installed on IoT control device 1020. Thus, a user may control the smart lamp 1020 through a user interface displayed on the mobile device 1000 via the IoT control device 1010. As shown in fig. 10f, IoT control device 1010 may send a control command to smart lamp 1020 through communication module 1022 of smart lamp 1020.

Although fig. 10 a-14 b have been described above with reference to first electronic device 1000 as a mobile device and IoT device 1020 as a smart light, the above-described embodiments are not limited to these examples. For example, the first electronic device 1000 may be a smart watch or a television, and the IoT device may be a dishwasher or a laundry dryer.

Referring to fig. 15, 16, one embodiment of a method of running a software program specific to an IoT device on a mobile device will be described below. As shown in fig. 15, at operation 1501, a mobile device (e.g., electronic device 700 or mobile device 1000) may receive a selection of an icon of an IoT application displayed on a display (e.g., display 760) from a user. As shown in (a) of fig. 16, the mobile device may display icons representing various applications on the display. Of these icons, an icon dedicated to an application (e.g., IoT App1600) that controls an IoT device is displayed on the lower right side.

In operation 1503, the mobile device may open an application for IoT in response to selection of an icon of the application for IoT. When the application is run, as shown in (B) of fig. 16, a user interface may be displayed. The user interface may display a list of IoT devices that may be controlled by the mobile device, for example, in the form of icons, images, and/or text. As shown in fig. 16 (B), the mobile device may display on a display previously installed IoT devices, such as a lighting system 1612, a security alarm system, an electricity meter, a sprinkler system, and a gas valve meter that may be controlled by the mobile device. When an icon for the lighting system 1612 is selected, the mobile device can display a detailed menu 1620 for controlling the light.

With reference to fig. 17, 18, another embodiment of a method of executing an application of an IoT device on a mobile device will be described below. Referring to fig. 17, at operation 1701, a mobile device (e.g., electronic device 700 or mobile device 1000) may receive a selection of a folder (e.g., IoT folder 1800) displayed on a display (e.g., display 760) from a user. The folder may include icons of applications associated with the IoT devices.

At operation 1703, in response to selection of IoT folder 1800, the mobile device may display icons of applications of IoT devices already installed in the mobile device within extension area 1810 of folder 1800. For example, as shown in (B) of fig. 18, the mobile device may display icons of a lighting system application, a safety alarm system application, an electricity meter application, a sprinkler system application, and an air valve metering application on the display.

At operation 1705, the mobile device may receive a selection of an application among the applications for a particular IoT device from the user. In operation 1707, in response to the selection of the application, the mobile device may execute the selected application. For example, the mobile device may execute the selected application and display a user interface for controlling the corresponding IoT device.

In some embodiments, to identify and connect the IoT device with the IoT control device, the mobile device may use at least one of RFID, GPS, electromagnetic location indicators, accelerometers, or other environmental information. In another embodiment, when the IoT device does not include a communication module, the IoT device may be controlled using an after market device configured to connect the IoT device to an Alternating Current (AC) outlet.

According to various embodiments of the present disclosure, a mobile device may add or connect a new IoT device to an IoT control device using various methods.

Scene 1: adding a new IoT device to an installed IoT control device

In one embodiment, a mobile device used as a user interface may identify the user interface using, for example, an NFC tag, RFID, augmented reality, QR code, barcode, or a combination of two or more of the foregoing. The mobile device may then receive or download and install the software program (or just the portion of the program related to the user interface) related to the IoT device. Thereafter, the mobile device may send information about the software programs related to the IoT device (e.g., identification information of the IoT device or identification information of the software programs related to the IoT device) to the IoT control device. The IoT control device may receive or download and install at least a portion of the software programs related to the IoT device (e.g., a control-related portion of the programs) based on the information about the software programs related to the IoT device.

When the IoT device is activated and the IoT control device receives a signal from the IoT device, the IoT control device may complete registration with the IoT device and begin monitoring the IoT device. The IoT control device may send registration completion information to the mobile device.

When the IoT control device fails to receive a signal from the IoT device, the IoT control device may wait for a selected period of time (e.g., one or two days) for the signal from the IoT device. If there is no signal within the selected time period, the IoT control device may delete the software program associated with the IoT device. Alternatively, the control device may delete the software program related to the IoT device in response to a user's selection. As in the embodiments to be described below, the IoT control device may determine the location of the at least one IoT device with reference to the location of the IoT control device.

Scene 2: replacing an existing IoT control device (i.e., an existing control device) already connected to an IoT device with a new IoT control device (i.e., a new control device)

In one embodiment, an existing IoT control device may collect information about IoT devices that have been installed (e.g., identification information of all IoT devices or identification information of software programs related to the IoT devices) and store the collected information in a server (e.g., a cloud server) or in a mobile device. Thereafter, when the existing IoT control device is replaced with a new IoT control device, previously stored information about the IoT device may be provided to the new IoT control device from a server or mobile device, and the new IoT control device may connect to the existing IoT device. In another embodiment, the previously stored information of the IoT device may be provided back to the existing IoT control device from the server or the mobile device even when the existing IoT control device is reset or changed.

Scene 3: first installing a set of IoT devices using an IoT control device

In one embodiment, as described above, the IoT control device may install the relevant software for the new set of IoT devices using the wrapper or an identifier of the wrapper (e.g., barcode) from the set of IoT devices. Then, the IoT control device may connect to the set of IoT devices. For example, the IoT control device may receive a software package for a plurality of IoT devices from a server and install the received software package.

Scene 4: connecting a new mobile device to a network including an IoT control device

In one embodiment, the IoT control device may connect to a new mobile device that serves as a user interface. In this case, the IoT control device may provide information to the mobile device so that the mobile device may download and install software for the IoT user interface from the app store or the IoT control device. The IoT control device may send information of at least one IoT device that is controllable by the IoT control device (e.g., identification information of the IoT device or identification information of a software program related to the IoT device) to the new mobile device.

Process for providing location information about IoT devices to IoT control devices

In some embodiments, the IoT control device may require location information of the IoT device to control the IoT device. For example, when a plurality of indoor lighting devices (e.g., smart lamps) are connected to the IoT control device and the same signal is transmitted and received inside a building, the IoT control device may need location information of the lighting devices in order to control a specific lighting device among the lighting devices each time.

However, the IoT control device or the mobile device may not have location information for the low-end IoT device. Furthermore, although high-end IP-based IoT devices are connected to IoT control devices, their physical locations may not be readily available. Furthermore, when there are multiple same or similar IoT devices, location information of the IoT devices may need to be provided to an IoT control device or a user of the IoT devices.

In one embodiment, in the operations described with reference to fig. 10 a-10 f, the mobile device may determine the location of the IoT device when capturing a QR code (or barcode) associated with the IoT device to identify the IoT device. For example, in identifying the IoT device, the mobile device may determine the location of the mobile device, e.g., using GPS included in the mobile device, an indoor location checking mechanism, etc. The mobile device may then record the determined location and treat the determined location as the location of the IoT device. In another embodiment, the mobile device may use a multiple-input multiple-output (MIMO) mechanism of the mobile device to determine the relative direction of the IoT device with respect to the mobile device.

In another embodiment, when accurate location information of an IoT device is needed, the mobile device may determine, estimate, or derive a more accurate location of the IoT device than the above-described embodiments by comparing the size of the barcode or QR code captured for identification of the IoT device with the original size of the preset QR code or barcode. In this case, the mobile device may estimate a more accurate location of the IoT device by further using at least one of: (a) a direction detected by a compass of the mobile device (i.e., south, north, etc.); (b) a location of the mobile device; or (c) the orientation of the mobile device (e.g., as detected by a gyroscope sensor or an acceleration sensor).

In yet another embodiment, the IoT control device may selectively provide a signal to at least one of the plurality of IoT devices located at different locations based on the location information of the IoT device. Such IoT control devices may use, for example, directional antennas or beamforming techniques. The IoT control device may provide signals to the selected IoT device at once while having no impact on other IoT devices.

In other embodiments, the mobile device may determine the location of the IoT device using at least one of WiFi SLAM (simultaneous positioning and mapping), location measurement techniques using lppe2.0, or the location techniques disclosed in korean patent publication No.10-2013 and 0038757, the disclosures of which are incorporated herein by reference.

In one embodiment, when using SLAM, the mobile device may have a positioning function that receives input of 2-dimensional (2D) data (images) and even determines its own position while making a 3D map. For example, SLAM may provide a service in which a robotic vacuum cleaner having a camera moves throughout a house while the robotic vacuum cleaner is mapped to determine where it is located on the map. For example, SLAM may provide a service of creating a 3D map of a space through input from a camera included in a mobile device, and determine a position of the camera (i.e., the mobile device including the camera) and a direction in which the camera faces on the 3D map.

When using WiFi SLAM, the mobile device may have a positioning function that makes a 3D map through WiFi signals and determines the current location of the mobile device, similar to the principle of SLAM. Accordingly, when a mobile device using a WiFi SLAM moves in a space where an Access Point (AP) is installed, the mobile device may receive unique information (i.e., a fingerprint) formed from a WiFi signal and make a map of the space and then estimate a location of the mobile device. Based on the scheme, the mobile device may determine location information of the IoT device.

In one embodiment, when using WiFi SLAM, the mobile device and WiFi-enabled IoT devices may use WiFi unique information (i.e., fingerprints) to measure the respective locations. In this case, the IoT device may determine a relative distance and direction with respect to the mobile device.

In another embodiment, to improve indoor positioning performance based on lppe2.0 technology, at least one of the following methods may be used: radio characteristic information methods, Image Recognition (IRB) based methods, hand-held dead reckoning (PDR) methods, methods of locomotor status assisted by UE, map based attributes, UE's own crowd source support for location and measurement information, indoor/outdoor transmission area information, radio map data, UE based IRB positioning or improved WLAN positioning information provision. Such methods may be used to obtain location information of IoT devices by a mobile device.

In one embodiment, a mobile device may detect a relative movement distance and direction of an IoT device from a first location of the mobile device. This scheme may be implemented using the distance between the location of the mobile device and the location of the IoT device estimated using the pedometer/accelerometer sensor (e.g., when the user approaches the IoT device with the mobile device on hand).

In another embodiment, when using the technology of korean patent laid-open No. 10-2013-.

In another embodiment, the mobile device may previously recognize coordinates of each location corresponding to each pixel of the image by installing a fixed camera and installing markers at selected intervals. Then, the mobile device may estimate the location of the IoT device by checking whether a particular object (e.g., IoT device) is present anywhere on the image and estimating the coordinates of the object in space.

Fig. 19 is a flowchart illustrating a method of providing a location of an IoT device to a control device of the IoT device, according to one embodiment. As shown in fig. 20, the following embodiments may be used to determine the location of an IoT device under certain circumstances.

Referring to fig. 19, an electronic device (e.g., the electronic device 700 or the mobile device 1000) may determine a location of an external electronic device in operation 1901. For example, as shown in fig. 20, when an electronic device (e.g., the electronic device 2002) approaches an external electronic device (e.g., the electric lamp 2000) to be controlled by an IoT and acquires information of the electric lamp 2000, the location of the electronic device 2002 may be used as the location of the electric lamp 2000. In another embodiment, in fig. 20, the electronic device 2002 may estimate the location of the TV 2010 to be controlled through the IoT scheme using location techniques of SLAM, lppe2.0, and so on. In operation 1903, the electronic device may transmit location information of the external electronic device to a control device (e.g., control device 702 or IoT control device 1010).

Fig. 21 is a flow diagram illustrating a method of operating an IoT control device, in accordance with one embodiment. Referring to fig. 21, at operation 2101, an IoT control device (e.g., control device 702 or IoT control device 1010) may receive a software program for controlling an IoT device (e.g., smart lamp 1020). In one embodiment, the control device may receive a software program for controlling an external electronic device from a server (e.g., server 464 or 1040) using identification information of the IoT device or identification information of a software program corresponding to the server (e.g., server 464 or 1040), both of which are received from the electronic device (e.g., electronic device 700 or mobile device 1000). In another embodiment, the IoT control device may receive a software program from the electronic device for controlling the IoT device.

In operation 2103, the IoT control device may install the received software program for controlling the IoT device. In operation 2105, the IoT control device receives a signal from the electronic device requesting control of the IoT device. In operation 2107, in response to the signal requesting control of the IoT device, the IoT control device may generate a control signal for controlling the IoT device. For example, the IoT control device may convert control signals received from the electronic device into control signals corresponding to a scheme for communicating with the IoT device. In operation 2109, the IoT control device may send the converted control signal to the IoT device.

Fig. 22 is a flowchart illustrating a method of receiving a software program for controlling an external device according to one embodiment. In particular, FIG. 22 provides more detailed operation of step 2101 of FIG. 21. Referring to fig. 22, at operation 2201, an IoT control device (e.g., control device 702 or IoT control device 1010) may receive identification information of an IoT device (e.g., smart lamp 1020) provided from an electronic device (e.g., electronic device 700 or mobile device 1000).

At operation 2203, the IoT control device may send a request signal to a server (e.g., server 464 or 1040) for a software program related to the identification information of the IoT device. In operation 2205, in response to the request signal, the IoT control device may receive a software program related to the identification information of the IoT device from the server. In operation 2103 of fig. 21, the control device may install a software program for controlling the external electronic device, received from the server, in the control device.

Fig. 23 is a flowchart illustrating a method of receiving a software program for controlling an external device according to one embodiment. In particular, FIG. 23 provides more detailed operation of step 2101 of FIG. 21. Referring to fig. 23, at operation 2301, an IoT control device (e.g., control device 702 or IoT control device 1010) may receive identification information of a software program related to an IoT device (e.g., smart lamp 1020) provided from an electronic device (e.g., electronic device 700 or mobile device 1000).

At operation 2303, the IoT control device may send a request signal to a server (e.g., server 464 or 1040) that includes identification information of the used software program related to the IoT device. In operation 2305, in response to the request signal, the IoT control device may receive a software program related to the identification information of the IoT device from the server. In operation 2103 of fig. 21, the control device may install a software program for controlling the external electronic device, received from the server, in the control device.

In another embodiment, the mobile device may use voice instructions to control the IoT device. For example, when the mobile device receives voice instructions, the mobile device may analyze the voice instructions (or optionally, with the assistance of a separate server). The mobile device may send the analyzed voice instructions to the IoT control device. The IoT control device may send a control signal corresponding to the analyzed voice instruction to the IoT device.

In one embodiment, upon receiving the voice instruction "turn off the light," the mobile device may analyze the corresponding voice instruction based on context such as the time and location at which the voice instruction was received. For example, the mobile device may analyze a voice command as a command for turning off a lamp installed in room 1 of a house. The mobile device may send the analyzed voice instructions to the IoT control device. The IoT control device may send a control signal to the lights in room 1 to "turn off the lights" according to the analyzed voice instructions. The lights of room 1 may then be turned off based on the control signal.

In another embodiment, the IoT (e.g., home control) may be activated after or immediately after the mobile device and IoT control device are connected to each other. For example, when the mobile device is located within the service area of the IoT control device, selected or specific operations or services related to the selected IoT device may be activated. For example, when a user with a mobile device arrives home, the mobile device may connect to the IoT control device through a wireless connection (e.g., WiFi, BLE, bluetooth, etc.).

In one embodiment, when communication between the IoT control device and the mobile device is established, certain operations or services related to the IoT device may be activated. For example, when the IoT control device detects that the mobile device is connected to the IoT control device, the IoT control device may send a control signal to the previously selected IoT device, controlling the IoT device to perform the selected operation.

In another embodiment, the IoT control device may selectively activate operations or services related to the IoT device (e.g., air conditioning) based on the environment and/or context (e.g., based on needs such as environment, temperature, time, and lighting, etc.). In yet another embodiment, the IoT devices and the particular operations or services associated therewith (e.g., turning on or off lights in room 2) may be pre-selected by the user.

In another embodiment, the IoT device may activate certain operations even without a wireless connection to the mobile device. For example, when a user with a mobile device drives home, the mobile device may provide its location information to the IoT control device. If it is determined that the mobile device comes within a selected distance from home, the IoT control device may initiate a particular operation of the IoT device (e.g., turn on an air conditioner).

In the above-described embodiment, using a mobile device (serving as a user interface) and information on an IoT device, the IoT device and an IoT control device can be efficiently and easily connected to each other.

The electronic device and method according to the above embodiments may be used to easily connect various items to the internet to implement an IoT environment. Further, the electronic device and method may provide a system in which IoT devices may be controlled at a specific location or remotely, which is convenient for users.

While the disclosure has been shown and described with reference to certain embodiments thereof, it will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the scope of the disclosure as defined by the scope of the disclosure, the appended claims, and any equivalents thereof.

Claims

1. A system, comprising:

a mobile device, comprising:

a first wireless communication interface;

a camera;

a display;

a first non-volatile memory; and

a first processor electrically connected to the first wireless communication interface, the camera, the display, and the first non-volatile memory; and

a media device, comprising:

second oneA wireless communication interface configured to supportA protocol;

a third wireless communication interface configured to support a Wi-Fi protocol;

a second non-volatile memory;

a wired communication port configured to transmit audio/video A/V data to a television;

a power connector configured to receive power; and

a second processor electrically connected to the second wireless communication interface, the third wireless communication interface, the second non-volatile memory, the wired communication port, and the power connector,

wherein the first non-volatile memory stores instructions that, when executed, cause the mobile device to:

capturing an image of a Quick Response (QR) code associated with an Internet of things (IoT) device using the camera,

information is obtained from the image of the QR code,

receiving software code corresponding to the obtained information from a server through the first wireless communication interface,

transmitting at least a first portion of the received software code to the media device,

providing a user interface on the display using at least a second portion of the received software code, the user interface configured to control an IoT device,

receiving user input through the user interface provided on the display, an

Based on the user input, sending a control signal to the media device, the control signal used by the media device to remotely control an IoT device.

2. The system of claim 1, wherein the instructions further cause the mobile device to:

receiving, over the first wireless communication interface, information about an app store that provides the software code associated with an IoT device; and

displaying at least one image related to the software code on the display based on the received information about the application store.

3. The system of claim 1, wherein the instructions further cause the mobile device to:

receiving, over the first wireless communication interface, information about an app store that provides the software code associated with an IoT device;

displaying on the display at least one category related to the software code based on the received information about the application store; and

in response to a user input selecting one of the displayed at least one category, displaying a plurality of images including an image associated with the software code.

4. The system of claim 1, wherein the at least the first portion of the software code transmitted to the media device comprises at least one of identification information of the software code and identification information of an IoT device.

5. The system of claim 1, wherein the instructions further cause the mobile device to: displaying the location of the IoT device on the display.

6. The system of claim 1, wherein at least a portion of the first portion is the same as at least a portion of the second portion.

7. A mobile device, comprising:

a wireless communication interface;

a camera;

a display;

a processor; and

a memory storing instructions that, when executed by the processor, cause the mobile device to:

information is obtained from the image of the QR code,

receiving software code corresponding to the obtained information from a server through the wireless communication interface,

providing a user interface on the display using at least a first portion of the software code, the user interface configured to control an IoT device,

receiving user input through the user interface provided on the display, an

Based on the user input, sending a control signal to an external device, the control signal used by the external device to remotely control an IoT device.

8. The mobile device of claim 7, wherein the at least the first portion of the software code comprises at least one of identification information of the software code and identification information of an IoT device.

9. The mobile device of claim 7, wherein the instructions further cause the mobile device to display a location of an IoT device on the display.

10. The mobile device of claim 7, wherein the external device is configured to:

in response to receiving the control signal to control an IoT device via a wireless fidelity Wi-Fi communication interface, generate a signal to control the IoT device independent of sending audio/video a/V data via a wired communication interface.

11. The mobile device of claim 7, wherein the instructions further cause the mobile device to:

sending at least a second part of the software code received from the server to the external device, an

Wherein the external device is configured to:

transmitting audio/video A/V data to a connected display device via a wired communication interface, receiving the at least a second portion of the software code from the mobile device, and

in response to receiving the control signal to control an IoT device via a wireless communication interface, generate a signal to control an IoT device based on the at least a second portion of the software code independent of sending the A/V data via the wired communication interface.

12. A system, comprising:

a mobile device, comprising:

a first wireless communication interface;

a camera;

a touch screen display;

a first non-volatile memory; and

a media device, comprising:

a second wireless communication interface configured to supportA protocol;

a second non-volatile memory;

a power connector configured to receive power; and

wherein the first non-volatile memory stores instructions that, when executed, cause the first processor to:

scanning a Quick Response (QR) code associated with an Internet of things (IoT) device using the camera,

accessing, through the first wireless communication interface, a server based on the scanned QR code, the server storing a software program related to an IoT device;

receiving at least a portion of the software program from the server through the first wireless communication interface;

storing the at least a portion of the software program received from the server on the first non-volatile memory;

providing images and text on the display using the at least a portion of the software program;

receiving user input through the display; and

based on the user input, sending a signal to the media device via the internet requesting control of an IoT device, an

Wherein the second non-volatile memory stores control software for controlling a plurality of different IoT devices and instructions that, when executed, cause the second processor to:

generating, using the control software, a signal for controlling an IoT device independent of sending the A/V data via the wired communication port and in response to receiving the signal requesting control of an IoT device via the third wireless communication interface; and

transmitting a signal for controlling the IoT device to the IoT device.

13. The system of claim 12, wherein the media device comprises a fourth wireless communication interface configured to support a bluetooth low energy protocol,

One of a protocol, a power line communication protocol, an infrared transmission protocol and an ultrasonic communication protocol, and

wherein the second non-volatile memory further stores instructions that, when executed, cause the second processor to: transmitting a signal for controlling an IoT device to an IoT device using one of the second wireless communication interface, the third wireless communication interface, and the fourth wireless communication interface.

14. The system of claim 12, wherein the instructions further cause the mobile device to:

receiving, over the first wireless communication interface, information about an application store that provides the software program associated with an IoT device; and

displaying at least one image or icon related to the software program on the display based on the received information about the application store.

15. The system of claim 12, wherein the instructions further cause the mobile device to:

receiving, over the first wireless communication interface, information about an application store that provides the software program associated with an IoT device;

displaying on the display at least one category related to the software program based on the received information about the application store; and

in response to user input selecting the displayed category, a plurality of images or icons are displayed including images or icons associated with the software program.

16. The system of claim 12, wherein the media device is configured to connect to a personal cloud and is further configured to use a television as a media device user interface.

17. A system, comprising:

a mobile device, comprising:

a first wireless communication interface;

a camera;

a touch screen display;

a microphone;

a first non-volatile memory; and

a first processor electrically connected to the first wireless communication interface, the camera, the display, the microphone, and the first non-volatile memory; and

a media device connected to the personal cloud and configured to connect to a television and use the television as a media device user interface,

wherein the media device comprises:

a second wireless communication interface configured to support

A protocol;

a second non-volatile memory;

a wired communication port configured to transmit audio/video A/V data to the television;

a power connector configured to receive power; and

scanning, using the camera, a Quick Response (QR) code on an Internet of things (IoT) device or a QR code provided with an IoT device;

displaying, on the display, an icon dedicated to an application controlling a plurality of IoT devices;

receiving an input for selecting the icon;

in response to the input to select the icon, displaying on the display a user interface including a list of IoT devices controllable by the mobile device, wherein the list includes at least one image and/or at least one text;

receiving, by the display, user input on the user interface for an IoT device; and

transmitting, to the media device via the Internet, a signal requesting control of an IoT device based on the user input,

receiving information associated with an IoT device;

use of

A protocol or a Wi-Fi protocol, which transmits a signal for controlling the IoT device to the IoT device.

18. The system of claim 17, wherein the first non-volatile memory further stores instructions that, when executed, cause the first processor to:

receiving a user input selecting an application associated with an IoT device from an application store; and

downloading and installing the selected application on the mobile device.

19. The system of claim 18, wherein an IoT device comprises a lighting system or a thermostat.

20. The system of claim 17, wherein the second non-volatile memory further stores instructions that, when executed, cause the second processor to:

control signals for control and status checking are exchanged with the IoT devices.

21. The system of claim 17, wherein the at least one image comprises an icon representing an IoT device.

22. The system of claim 21, wherein the at least one text comprises a location associated with an IoT device.

23. The system of claim 17, wherein the first non-volatile memory further stores instructions that, when executed, cause the mobile device to:

receiving a voice instruction via the microphone and,

analyzing the voice command or sending the voice command to a server, an

Cause the voice instruction to be sent to an IoT device via the media device.

24. The system of claim 17, wherein the first non-volatile memory further stores instructions that, when executed, cause the mobile device to:

activating the selected IoT device-related operation or service when the mobile device is located within a service area of the media device.

25. The system of claim 24, wherein the first non-volatile memory further stores instructions that, when executed, cause the mobile device to:

activating a previously selected operation or service related to an IoT device when the mobile device connects with the media device via Wi-Fi or Bluetooth protocols.

26. The system of claim 25, wherein the first non-volatile memory further stores instructions that, when executed, cause the mobile device to:

a selection of the operation or service is received from a user in advance.