CN111401218B - Smart city monitoring method and system - Google Patents

Smart city monitoring method and system Download PDF

Info

Publication number
CN111401218B
CN111401218B CN202010170331.5A CN202010170331A CN111401218B CN 111401218 B CN111401218 B CN 111401218B CN 202010170331 A CN202010170331 A CN 202010170331A CN 111401218 B CN111401218 B CN 111401218B
Authority
CN
China
Prior art keywords
information
recognition result
feature
monitoring
numerical code
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202010170331.5A
Other languages
Chinese (zh)
Other versions
CN111401218A (en
Inventor
高宽友
王利
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shanghai Hongdian Intelligent Technology Co ltd
Original Assignee
Shanghai Hongdian Intelligent Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shanghai Hongdian Intelligent Technology Co ltd filed Critical Shanghai Hongdian Intelligent Technology Co ltd
Priority to CN202010170331.5A priority Critical patent/CN111401218B/en
Publication of CN111401218A publication Critical patent/CN111401218A/en
Application granted granted Critical
Publication of CN111401218B publication Critical patent/CN111401218B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/20—Movements or behaviour, e.g. gesture recognition
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q50/00—Information and communication technology [ICT] specially adapted for implementation of business processes of specific business sectors, e.g. utilities or tourism
    • G06Q50/10—Services
    • G06Q50/26—Government or public services
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/52—Surveillance or monitoring of activities, e.g. for recognising suspicious objects
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L15/00—Speech recognition
    • G10L15/22—Procedures used during a speech recognition process, e.g. man-machine dialogue
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L17/00—Speaker identification or verification techniques
    • G10L17/02—Preprocessing operations, e.g. segment selection; Pattern representation or modelling, e.g. based on linear discriminant analysis [LDA] or principal components; Feature selection or extraction
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00—Television systems
    • H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
    • H04N7/181—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast for receiving images from a plurality of remote sources
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02A—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
    • Y02A30/00—Adapting or protecting infrastructure or their operation
    • Y02A30/60—Planning or developing urban green infrastructure

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Physics & Mathematics (AREA)
  • Health & Medical Sciences (AREA)
  • Business, Economics & Management (AREA)
  • Theoretical Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • General Physics & Mathematics (AREA)
  • Acoustics & Sound (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Tourism & Hospitality (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • General Health & Medical Sciences (AREA)
  • Human Resources & Organizations (AREA)
  • Primary Health Care (AREA)
  • Strategic Management (AREA)
  • General Business, Economics & Management (AREA)
  • Marketing (AREA)
  • Psychiatry (AREA)
  • Social Psychology (AREA)
  • Economics (AREA)
  • Signal Processing (AREA)
  • Educational Administration (AREA)
  • Development Economics (AREA)
  • Computational Linguistics (AREA)
  • Telephonic Communication Services (AREA)

Abstract

The invention provides a smart city monitoring method and a system. Firstly, the acquisition equipment uploads the acquired monitoring information of the target area to the information processing equipment through a preset information transmission channel, so that the time and accuracy of the transmission of the monitoring information can be improved. Secondly, the information processing equipment performs feature extraction on the image information and the voice information in the monitoring information, and performs feature recognition to obtain a first recognition result and a second recognition result. And finally, determining a weighting coefficient based on the first confidence coefficient and the second confidence coefficient which are obtained according to the first recognition result and the second recognition result, realizing weighted summation of the first recognition result and the second recognition result to obtain a third recognition result, and judging that the target area is abnormal when the early warning level of the third recognition result exceeds the set level. Therefore, the monitoring information can be deeply mined from the association angle of the image information and the voice information to obtain a more comprehensive and reliable monitoring analysis result so as to ensure the safety of the building.

Description

Smart city monitoring method and system
Technical Field
The invention relates to the technical field of smart cities, in particular to a smart city monitoring method and a smart city monitoring system.
Background
Along with the development of the Internet of things, the application of smart cities is more and more mature. The intelligent city can realize the deep integration of informatization, industrialization and township, and is favorable for improving the problems of the city at the present stage. Taking building monitoring in smart city application as an example, building buildings can be monitored continuously by utilizing big data and the Internet of things, however, in the process of monitoring the buildings, the phenomenon that monitoring information is not mined in place still occurs.
Disclosure of Invention
In view of the above, the present invention provides a smart city monitoring method, apparatus and information processing device to improve the above-mentioned problems.
In a first aspect of an embodiment of the present invention, there is provided a smart city monitoring method applied to a smart city monitoring system, the smart city monitoring system including an information processing device and an acquisition device that are in communication with each other, the acquisition device being disposed in different areas of a building and configured to acquire monitoring information in the different areas of the building, the method including:
the method comprises the steps that the acquisition equipment monitors a target area of a building in real time, acquires monitoring information of the target area, and uploads the monitoring information to the information processing equipment in real time through a preset information transmission channel; the information transmission channel is generated by the information processing device according to the matching degree between first parameter structured information obtained by the information processing device and second parameter structured information of the acquisition device, wherein the first parameter structured information is used for representing communication parameter logic distribution of the information processing device, and the second parameter structured information is used for representing communication parameter logic distribution of the acquisition device; the monitoring information comprises image information corresponding to the target area and voice information corresponding to the target area;
The information processing device receives the monitoring information corresponding to the target area uploaded by the acquisition device through the information transmission channel, and carries out information category identification on the monitoring information to obtain first category information used for representing the image information and second category information used for representing the voice information, wherein the first category information is included in the monitoring information; classifying the monitoring information according to the first category information and the second category information to obtain the image information corresponding to the first category information and the voice information corresponding to the second category information in the monitoring information;
the information processing equipment extracts first information features of the image information and second information features of the voice information, performs feature recognition on the first information features based on a first preset feature database to obtain a first recognition result, and performs feature recognition on the second information features based on a second preset feature database to obtain a second recognition result; the first preset feature database is an image feature database, and the second preset feature database is a voice feature database;
the information processing equipment acquires a second confidence coefficient of the second recognition result obtained by taking the first recognition result as a reference and a first confidence coefficient of the first recognition result obtained by taking the second recognition result as a reference; obtaining a weighting coefficient for weighting the first recognition result and the second recognition result according to the first confidence coefficient and the second confidence coefficient; carrying out weighted summation on the first recognition result and the second recognition result based on the weighting coefficient to obtain a third recognition result, and determining an early warning grade corresponding to the third recognition result; and judging that the target area is abnormal when the early warning level exceeds a set level.
Optionally, the extracting the first information feature of the image information and the second information feature of the voice information includes:
determining image coding information of the image information, dividing the image coding information according to coding segmentation identifiers in the image coding information to obtain a plurality of continuous coding information segments, determining matching coefficients between every two coding information segments, and carrying out relevance correction on each coding information segment according to all the determined matching coefficients to obtain target coding information segments corresponding to each coding information segment;
determining the code character distribution corresponding to each target code information segment, listing the character structure topology of each code character distribution, determining the character distribution characteristics corresponding to each code character distribution according to the character structure topology, and integrating all the character distribution characteristics to obtain the first information characteristics of the image information;
extracting a spectrogram of the voice information, separating a voiceprint curve in the voice information from the spectrogram, and acquiring voiceprint characteristics corresponding to the voiceprint curve;
text information corresponding to the voice information is obtained according to the voice information, word segmentation processing is carried out on the text information to obtain a plurality of keywords, topic information corresponding to the text information is determined according to semantic connection relations among the keywords, and topic features of the topic information are extracted;
Determining a first association weight of the voiceprint feature relative to the theme feature and a second association weight of the theme feature relative to the voiceprint feature, and carrying out weighted summation on the voiceprint feature and the theme feature based on the first association weight and the second association weight to obtain a second information feature of the voice information.
Optionally, the performing feature recognition on the first information feature based on a first preset feature database to obtain a first recognition result includes:
acquiring a first target feature with the minimum cosine distance between the first target feature and the first information feature in the first preset feature database;
determining a behavior category corresponding to the first target feature from a preset first mapping relation list, wherein the behavior category is a person obtaining behavior category in the image information;
and generating the first recognition result based on the behavior category.
Optionally, the performing feature recognition on the second information feature based on a second preset feature database to obtain a second recognition result includes:
determining at least part of second target features, of which the similarity value between the second target features and the second information features in the second preset feature database is smaller than or equal to a set threshold value;
And determining a semantic result corresponding to each second target feature according to a preset second mapping relation list, and fusing all the semantic results to obtain the second identification result.
Optionally, the performing weighted summation on the first recognition result and the second recognition result based on the weighting coefficient to obtain a third recognition result includes:
respectively listing the first identification result and the second identification result in a numerical code form to obtain a first numerical code list corresponding to the first identification result and a second numerical code list corresponding to the second identification result;
pairing each first list unit in the first numerical code list and each second list unit in the second numerical code to obtain at least part of list unit groups; wherein each list unit group comprises a first list unit and a second list unit;
and generating a third numerical code list according to at least part of list unit groups, weighting the third numerical code list based on the weighting coefficient to obtain a fourth numerical code list, and converting the fourth numerical code list into a third identification result based on first conversion logic which lists the first identification result in a numerical code form to obtain a first numerical code list corresponding to the first identification result or based on second conversion logic which lists the second identification result in a numerical code form to obtain a second numerical code list corresponding to the second identification result.
In a second aspect of the embodiment of the invention, a smart city monitoring system is provided, which comprises an information processing device and an acquisition device which are communicated with each other, wherein the acquisition device is arranged in different areas of a building;
the acquisition equipment is used for monitoring a target area of a building in real time and acquiring monitoring information of the target area, and uploading the monitoring information to the information processing equipment in real time through a preset information transmission channel; the information transmission channel is generated by the information processing device according to the matching degree between first parameter structured information obtained by the information processing device and second parameter structured information of the acquisition device, wherein the first parameter structured information is used for representing communication parameter logic distribution of the information processing device, and the second parameter structured information is used for representing communication parameter logic distribution of the acquisition device; the monitoring information comprises image information corresponding to the target area and voice information corresponding to the target area;
the information processing device is used for receiving the monitoring information corresponding to the target area uploaded by the acquisition device through the information transmission channel, and carrying out information category identification on the monitoring information to obtain first category information which is included in the monitoring information and used for representing the image information and second category information which is used for representing the voice information; classifying the monitoring information according to the first category information and the second category information to obtain the image information corresponding to the first category information and the voice information corresponding to the second category information in the monitoring information; the information processing equipment extracts first information features of the image information and second information features of the voice information, performs feature recognition on the first information features based on a first preset feature database to obtain a first recognition result, and performs feature recognition on the second information features based on a second preset feature database to obtain a second recognition result; the first preset feature database is an image feature database, and the second preset feature database is a voice feature database; the information processing equipment acquires a second confidence coefficient of the second recognition result obtained by taking the first recognition result as a reference and a first confidence coefficient of the first recognition result obtained by taking the second recognition result as a reference; obtaining a weighting coefficient for weighting the first recognition result and the second recognition result according to the first confidence coefficient and the second confidence coefficient; carrying out weighted summation on the first recognition result and the second recognition result based on the weighting coefficient to obtain a third recognition result, and determining an early warning grade corresponding to the third recognition result; and judging that the target area is abnormal when the early warning level exceeds a set level.
Optionally, the information processing apparatus is specifically configured to:
determining image coding information of the image information, dividing the image coding information according to coding segmentation identifiers in the image coding information to obtain a plurality of continuous coding information segments, determining matching coefficients between every two coding information segments, and carrying out relevance correction on each coding information segment according to all the determined matching coefficients to obtain target coding information segments corresponding to each coding information segment;
determining the code character distribution corresponding to each target code information segment, listing the character structure topology of each code character distribution, determining the character distribution characteristics corresponding to each code character distribution according to the character structure topology, and integrating all the character distribution characteristics to obtain the first information characteristics of the image information;
extracting a spectrogram of the voice information, separating a voiceprint curve in the voice information from the spectrogram, and acquiring voiceprint characteristics corresponding to the voiceprint curve;
text information corresponding to the voice information is obtained according to the voice information, word segmentation processing is carried out on the text information to obtain a plurality of keywords, topic information corresponding to the text information is determined according to semantic connection relations among the keywords, and topic features of the topic information are extracted;
Determining a first association weight of the voiceprint feature relative to the theme feature and a second association weight of the theme feature relative to the voiceprint feature, and carrying out weighted summation on the voiceprint feature and the theme feature based on the first association weight and the second association weight to obtain a second information feature of the voice information.
Optionally, the information processing apparatus is specifically configured to:
acquiring a first target feature with the minimum cosine distance between the first target feature and the first information feature in the first preset feature database;
determining a behavior category corresponding to the first target feature from a preset first mapping relation list, wherein the behavior category is a person obtaining behavior category in the image information;
and generating the first recognition result based on the behavior category.
Optionally, the information processing apparatus is specifically configured to:
determining at least part of second target features, of which the similarity value between the second target features and the second information features in the second preset feature database is smaller than or equal to a set threshold value;
and determining a semantic result corresponding to each second target feature according to a preset second mapping relation list, and fusing all the semantic results to obtain the second identification result.
Optionally, the information processing apparatus is specifically configured to:
respectively listing the first identification result and the second identification result in a numerical code form to obtain a first numerical code list corresponding to the first identification result and a second numerical code list corresponding to the second identification result;
pairing each first list unit in the first numerical code list and each second list unit in the second numerical code to obtain at least part of list unit groups; wherein each list unit group comprises a first list unit and a second list unit;
and generating a third numerical code list according to at least part of list unit groups, weighting the third numerical code list based on the weighting coefficient to obtain a fourth numerical code list, and converting the fourth numerical code list into a third identification result based on first conversion logic which lists the first identification result in a numerical code form to obtain a first numerical code list corresponding to the first identification result or based on second conversion logic which lists the second identification result in a numerical code form to obtain a second numerical code list corresponding to the second identification result.
Advantageous effects
According to the smart city monitoring method and system provided by the embodiment of the invention, firstly, the acquisition equipment uploads the acquired monitoring information of the target area to the information processing equipment through the preset information transmission channel, so that the time and accuracy of the transmission of the monitoring information can be improved.
Secondly, the information processing equipment performs feature extraction on the image information and the voice information in the monitoring information, and performs feature recognition to obtain a first recognition result and a second recognition result.
And finally, determining a weighting coefficient based on the first confidence coefficient and the second confidence coefficient which are obtained according to the first recognition result and the second recognition result, realizing weighted summation of the first recognition result and the second recognition result to obtain a third recognition result, and then judging that the target area is abnormal when the early warning level of the third recognition result exceeds the set level.
Therefore, the monitoring information can be deeply mined from the association angle of the image information and the voice information, and further a more comprehensive and reliable monitoring analysis result is obtained, so that the safety of the building is ensured.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings that are needed in the embodiments will be briefly described below, it being understood that the following drawings only illustrate some embodiments of the present invention and therefore should not be considered as limiting the scope, and other related drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
Fig. 1 is a schematic diagram of a smart city monitoring system according to an embodiment of the present invention.
Fig. 2 is a flowchart of a smart city monitoring method according to an embodiment of the present invention.
Fig. 3 is a block diagram of an information processing apparatus according to an embodiment of the present invention.
Icon:
100-a smart city monitoring system;
200-an information processing apparatus; 201-a smart city monitoring device; 211-memory; 212-a processor; 213-network module;
300-acquisition device.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the embodiments of the present invention more apparent, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention, and it is apparent that the described embodiments are only some embodiments of the present invention, but not all embodiments of the present invention. The components of the embodiments of the present invention generally described and illustrated in the figures herein may be arranged and designed in a wide variety of different configurations.
Thus, the following detailed description of the embodiments of the invention, as presented in the figures, is not intended to limit the scope of the invention, as claimed, but is merely representative of selected embodiments of the invention. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
It should be noted that: like reference numerals and letters denote like items in the following figures, and thus once an item is defined in one figure, no further definition or explanation thereof is necessary in the following figures.
In order to improve the phenomenon that monitoring information is not mined in place in the process of monitoring a building, the embodiment of the invention provides a smart city monitoring method and a smart city monitoring system, which can deeply mine the monitoring information so as to obtain a more comprehensive and reliable monitoring analysis result to ensure the safety of the building.
Referring to fig. 1, a schematic architecture diagram of a smart city monitoring system 100 according to an embodiment of the present invention is provided, where the smart city monitoring system 100 includes an information processing device 200 and a collecting device 300 that communicate with each other. In this embodiment, the capturing device 300 may be a camera or a microphone, which is not limited herein. The acquisition device 300 may be disposed in different areas of a building structure for acquiring monitoring information in the different areas of the building structure.
Referring to fig. 2 in combination, a flowchart of a smart city monitoring method according to an embodiment of the invention is provided, and the method is applied to the smart city monitoring system 100 in fig. 1. Further, the method may be implemented specifically by what is described in the following steps.
Step S21, the acquisition equipment monitors a target area of a building in real time and acquires monitoring information of the target area, and the monitoring information is uploaded to the information processing equipment in real time through a preset information transmission channel; the information transmission channel is generated by the information processing device according to the matching degree between first parameter structured information obtained by the information processing device and second parameter structured information of the acquisition device, wherein the first parameter structured information is used for representing communication parameter logic distribution of the information processing device, and the second parameter structured information is used for representing communication parameter logic distribution of the acquisition device; the monitoring information comprises image information corresponding to the target area and voice information corresponding to the target area.
Step S22, the information processing equipment receives the monitoring information corresponding to the target area uploaded by the acquisition equipment through the information transmission channel, and carries out information category identification on the monitoring information to obtain first category information which is included in the monitoring information and is used for representing the image information and second category information which is used for representing the voice information; classifying the monitoring information according to the first category information and the second category information to obtain the image information corresponding to the first category information and the voice information corresponding to the second category information in the monitoring information.
Step S23, the information processing equipment extracts first information features of the image information and second information features of the voice information, performs feature recognition on the first information features based on a first preset feature database to obtain a first recognition result, and performs feature recognition on the second information features based on a second preset feature database to obtain a second recognition result; the first preset feature database is an image feature database, and the second preset feature database is a voice feature database;
step S24, the information processing equipment acquires a second confidence coefficient of the second recognition result obtained by taking the first recognition result as a reference and a first confidence coefficient of the first recognition result obtained by taking the second recognition result as a reference; obtaining a weighting coefficient for weighting the first recognition result and the second recognition result according to the first confidence coefficient and the second confidence coefficient; carrying out weighted summation on the first recognition result and the second recognition result based on the weighting coefficient to obtain a third recognition result, and determining an early warning grade corresponding to the third recognition result; and judging that the target area is abnormal when the early warning level exceeds a set level.
It can be understood that, based on the descriptions in the above steps S21 to S24, first, the collecting device uploads the collected monitoring information of the target area to the information processing device through the preset information transmission channel, so that the reliability and accuracy of the transmission of the monitoring information can be improved. Secondly, the information processing equipment performs feature extraction on the image information and the voice information in the monitoring information, and performs feature recognition to obtain a first recognition result and a second recognition result. And finally, determining a weighting coefficient based on the first confidence coefficient and the second confidence coefficient which are obtained according to the first recognition result and the second recognition result, realizing weighted summation of the first recognition result and the second recognition result to obtain a third recognition result, and then judging that the target area is abnormal when the early warning level of the third recognition result exceeds the set level. Therefore, the monitoring information can be deeply mined from the association angle of the image information and the voice information, and further a more comprehensive and reliable monitoring analysis result is obtained, so that the safety of the building is ensured.
In an alternative embodiment, in order to accurately determine the first information feature and the second information feature, in step S23, the extracting the first information feature of the image information and the second information feature of the voice information may specifically include what is described in the following steps.
Step S231, determining image coding information of the image information, dividing the image coding information according to coding division identifiers in the image coding information to obtain a plurality of continuous coding information segments, determining matching coefficients between every two coding information segments, and carrying out relevance correction on each coding information segment according to all the determined matching coefficients to obtain a target coding information segment corresponding to each coding information segment.
Step S232, determining the code character distribution corresponding to each target code information segment, listing the character structure topology of each code character distribution, determining the character distribution characteristics corresponding to each code character distribution according to the character structure topology, and integrating all the character distribution characteristics to obtain the first information characteristics of the image information.
Step S233, extracting a spectrogram of the voice information, separating a voiceprint curve in the voice information from the spectrogram, and obtaining voiceprint characteristics corresponding to the voiceprint curve.
Step S234, text information corresponding to the voice information is obtained according to the voice information, word segmentation processing is carried out on the text information to obtain a plurality of keywords, topic information corresponding to the text information is determined according to semantic connection relations among the keywords, and topic features of the topic information are extracted.
Step S235, determining a first association weight of the voiceprint feature relative to the theme feature and a second association weight of the theme feature relative to the voiceprint feature, and performing weighted summation on the voiceprint feature and the theme feature based on the first association weight and the second association weight to obtain a second information feature of the voice information.
It will be appreciated that the first information feature and the second information feature can be accurately determined by the description of steps S231 to S235.
In a specific implementation, in step S23, the feature recognition of the first information feature based on the first preset feature database may specifically include a method described in the following steps.
(11) And acquiring a first target feature with the minimum cosine distance between the first target feature and the first information feature in the first preset feature database.
(12) And determining a behavior category corresponding to the first target feature from a preset first mapping relation list, wherein the behavior category is a person obtained behavior category in the image information.
(13) And generating the first recognition result based on the behavior category.
In this embodiment, the first recognition result including the behavior category can be accurately determined through the descriptions in the steps (11) - (13).
On the basis of the above, in step S23, the feature recognition of the second information feature based on the second preset feature database may specifically include a method described in the following steps.
(21) And determining at least part of second target features, of which the similarity value between the second information features and the second preset feature database is smaller than or equal to a set threshold value.
(22) And determining a semantic result corresponding to each second target feature according to a preset second mapping relation list, and fusing all the semantic results to obtain the second identification result.
In the present embodiment, the second recognition result can be accurately determined by the contents described in the above steps (21) to (23).
In a specific implementation, in step S24, the weighted summation of the first recognition result and the second recognition result based on the weighting coefficient obtains a third recognition result, which may specifically include a method described in the following steps.
Step S241, listing the first recognition result and the second recognition result in a numerical code form to obtain a first numerical code list corresponding to the first recognition result and a second numerical code list corresponding to the second recognition result.
Step S242, pairing each first list unit in the first numerical code list and each second list unit in the second numerical code to obtain at least part of list unit groups; wherein each list element group comprises a first list element and a second list element.
Step S243, generating a third numerical code list according to at least part of the list unit groups, weighting the third numerical code list based on the weighting coefficient to obtain a fourth numerical code list, and converting the fourth numerical code list into a third recognition result based on first conversion logic that lists the first recognition result in a numerical code form to obtain a first numerical code list corresponding to the first recognition result or based on second conversion logic that lists the second recognition result in a numerical code form to obtain a second numerical code list corresponding to the second recognition result.
It is understood that the third recognition result can be accurately obtained based on the descriptions of the above steps S241 to S243.
Referring to fig. 3 in combination, a block diagram of an information processing apparatus 200 according to an embodiment of the invention is shown. The information processing apparatus 200 in the embodiment of the present invention has data storage, transmission, processing functions, and as shown in fig. 3, the information processing apparatus 200 includes: memory 211, processor 212, network module 213 and smart city monitoring device 201.
The memory 211, the processor 212 and the network module 213 are electrically connected directly or indirectly to enable transmission or interaction of data. For example, the components may be electrically connected to each other by one or more communication buses or signal lines. The memory 211 stores the smart city monitoring device 20, and the smart city monitoring device 20 includes at least one software function module stored in the memory 211 in the form of software or firmware (firmware), and the processor 212 executes various function applications and data processes by executing software programs and modules stored in the memory 211, such as the smart city monitoring device 20 in the embodiment of the present invention, that is, implements the smart city monitoring method in the embodiment of the present invention.
The Memory 211 may be, but is not limited to, a random access Memory (Random Access Memory, RAM), a Read Only Memory (ROM), a programmable Read Only Memory (Programmable Read-Only Memory, PROM), an erasable Read Only Memory (Erasable Programmable Read-Only Memory, EPROM), an electrically erasable Read Only Memory (Electric Erasable Programmable Read-Only Memory, EEPROM), etc. The memory 211 is used for storing a program, and the processor 212 executes the program after receiving an execution instruction.
The processor 212 may be an integrated circuit chip having data processing capabilities. The processor 212 may be a general-purpose processor including a central processing unit (Central Processing Unit, CPU), a network processor (Network Processor, NP), etc. The methods, steps and logic blocks disclosed in the embodiments of the present invention may be implemented or performed. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
The network module 213 is configured to establish a communication connection between the information processing apparatus 200 and other communication terminal apparatuses through a network, and implement a transceiving operation of network signals and data. The network signals may include wireless signals or wired signals.
It is to be understood that the structure shown in fig. 3 is merely illustrative, and the information processing apparatus 200 may further include more or less components than those shown in fig. 3, or have a different configuration from that shown in fig. 1. The components shown in fig. 1 may be implemented in hardware, software, or a combination thereof.
Embodiments of the present invention also provide a computer-readable storage medium including a computer program. The computer program controls the information processing apparatus 200 where the readable storage medium is located to execute the steps corresponding to the information processing apparatus 200 in the smart city monitoring method shown in fig. 2 when running.
In summary, according to the smart city monitoring method and system provided by the embodiment of the invention, firstly, the acquisition equipment uploads the acquired monitoring information of the target area to the information processing equipment through the preset information transmission channel, so that the information and accuracy in the transmission of the monitoring information can be improved.
Secondly, the information processing equipment performs feature extraction on the image information and the voice information in the monitoring information, and performs feature recognition to obtain a first recognition result and a second recognition result.
And finally, determining a weighting coefficient based on the first confidence coefficient and the second confidence coefficient which are obtained according to the first recognition result and the second recognition result, realizing weighted summation of the first recognition result and the second recognition result to obtain a third recognition result, and then judging that the target area is abnormal when the early warning level of the third recognition result exceeds the set level.
Therefore, the monitoring information can be deeply mined from the association angle of the image information and the voice information, and further a more comprehensive and reliable monitoring analysis result is obtained, so that the safety of the building is ensured.
In the embodiments provided in the present invention, it should be understood that the disclosed apparatus and method may be implemented in other manners. The apparatus and method embodiments described above are merely illustrative, for example, flow diagrams and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
In addition, functional modules in the embodiments of the present invention may be integrated together to form a single part, or each module may exist alone, or two or more modules may be integrated to form a single part.
The functions, if implemented in the form of software functional modules and sold or used as a stand-alone product, may be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention may be embodied essentially or in a part contributing to the prior art or in a part of the technical solution in the form of a software product stored in a storage medium, comprising several instructions for causing a computer device (which may be a personal computer, an information processing device 200, or a network device, etc.) to perform all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), a magnetic disk, or an optical disk, or other various media capable of storing program codes. It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.
The above description is only of the preferred embodiments of the present invention and is not intended to limit the present invention, but various modifications and variations can be made to the present invention by those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims (10)

1. A smart city monitoring method, characterized in that it is applied to a smart city monitoring system, the smart city monitoring system includes an information processing device and an acquisition device that communicate with each other, the acquisition device is disposed in different areas of a building and is used for acquiring monitoring information in different areas of the building, the method includes:
the method comprises the steps that the acquisition equipment monitors a target area of a building in real time, acquires monitoring information of the target area, and uploads the monitoring information to the information processing equipment in real time through a preset information transmission channel; the information transmission channel is generated by the information processing device according to the matching degree between first parameter structured information obtained by the information processing device and second parameter structured information of the acquisition device, wherein the first parameter structured information is used for representing communication parameter logic distribution of the information processing device, and the second parameter structured information is used for representing communication parameter logic distribution of the acquisition device; the monitoring information comprises image information corresponding to the target area and voice information corresponding to the target area;
The information processing device receives the monitoring information corresponding to the target area uploaded by the acquisition device through the information transmission channel, and carries out information category identification on the monitoring information to obtain first category information used for representing the image information and second category information used for representing the voice information, wherein the first category information is included in the monitoring information; classifying the monitoring information according to the first category information and the second category information to obtain the image information corresponding to the first category information and the voice information corresponding to the second category information in the monitoring information;
the information processing equipment extracts first information features of the image information and second information features of the voice information, performs feature recognition on the first information features based on a first preset feature database to obtain a first recognition result, and performs feature recognition on the second information features based on a second preset feature database to obtain a second recognition result; the first preset feature database is an image feature database, and the second preset feature database is a voice feature database;
the information processing equipment acquires a second confidence coefficient of the second recognition result obtained by taking the first recognition result as a reference and a first confidence coefficient of the first recognition result obtained by taking the second recognition result as a reference; obtaining a weighting coefficient for weighting the first recognition result and the second recognition result according to the first confidence coefficient and the second confidence coefficient; carrying out weighted summation on the first recognition result and the second recognition result based on the weighting coefficient to obtain a third recognition result, and determining an early warning grade corresponding to the third recognition result; and judging that the target area is abnormal when the early warning level exceeds a set level.
2. The method of claim 1, wherein the extracting the first information characteristic of the image information and the second information characteristic of the voice information comprises:
determining image coding information of the image information, dividing the image coding information according to coding segmentation identifiers in the image coding information to obtain a plurality of continuous coding information segments, determining matching coefficients between every two coding information segments, and carrying out relevance correction on each coding information segment according to all the determined matching coefficients to obtain target coding information segments corresponding to each coding information segment;
determining the code character distribution corresponding to each target code information segment, listing the character structure topology of each code character distribution, determining the character distribution characteristics corresponding to each code character distribution according to the character structure topology, and integrating all the character distribution characteristics to obtain the first information characteristics of the image information;
extracting a spectrogram of the voice information, separating a voiceprint curve in the voice information from the spectrogram, and acquiring voiceprint characteristics corresponding to the voiceprint curve;
text information corresponding to the voice information is obtained according to the voice information, word segmentation processing is carried out on the text information to obtain a plurality of keywords, topic information corresponding to the text information is determined according to semantic connection relations among the keywords, and topic features of the topic information are extracted;
Determining a first association weight of the voiceprint feature relative to the theme feature and a second association weight of the theme feature relative to the voiceprint feature, and carrying out weighted summation on the voiceprint feature and the theme feature based on the first association weight and the second association weight to obtain a second information feature of the voice information.
3. The method according to claim 2, wherein the performing feature recognition on the first information feature based on the first preset feature database to obtain a first recognition result includes:
acquiring a first target feature with the minimum cosine distance between the first target feature and the first information feature in the first preset feature database;
determining a behavior category corresponding to the first target feature from a preset first mapping relation list, wherein the behavior category is a person obtaining behavior category in the image information;
and generating the first recognition result based on the behavior category.
4. The method according to claim 2, wherein the performing feature recognition on the second information feature based on the second preset feature database to obtain a second recognition result includes:
determining at least part of second target features, of which the similarity value between the second target features and the second information features in the second preset feature database is smaller than or equal to a set threshold value;
And determining a semantic result corresponding to each second target feature according to a preset second mapping relation list, and fusing all the semantic results to obtain the second identification result.
5. The method of any of claims 1-4, wherein the weighted summing the first recognition result and the second recognition result based on the weighting coefficients to obtain a third recognition result comprises:
respectively listing the first identification result and the second identification result in a numerical code form to obtain a first numerical code list corresponding to the first identification result and a second numerical code list corresponding to the second identification result;
pairing each first list unit in the first numerical code list and each second list unit in the second numerical code to obtain at least part of list unit groups; wherein each list unit group comprises a first list unit and a second list unit;
and generating a third numerical code list according to at least part of list unit groups, weighting the third numerical code list based on the weighting coefficient to obtain a fourth numerical code list, and converting the fourth numerical code list into a third identification result based on first conversion logic which lists the first identification result in a numerical code form to obtain a first numerical code list corresponding to the first identification result or based on second conversion logic which lists the second identification result in a numerical code form to obtain a second numerical code list corresponding to the second identification result.
6. The intelligent city monitoring system is characterized by comprising information processing equipment and acquisition equipment which are communicated with each other, wherein the acquisition equipment is arranged in different areas of a building;
the acquisition equipment is used for monitoring a target area of a building in real time and acquiring monitoring information of the target area, and uploading the monitoring information to the information processing equipment in real time through a preset information transmission channel; the information transmission channel is generated by the information processing device according to the matching degree between first parameter structured information obtained by the information processing device and second parameter structured information of the acquisition device, wherein the first parameter structured information is used for representing communication parameter logic distribution of the information processing device, and the second parameter structured information is used for representing communication parameter logic distribution of the acquisition device; the monitoring information comprises image information corresponding to the target area and voice information corresponding to the target area;
the information processing device is used for receiving the monitoring information corresponding to the target area uploaded by the acquisition device through the information transmission channel, and carrying out information category identification on the monitoring information to obtain first category information which is included in the monitoring information and used for representing the image information and second category information which is used for representing the voice information; classifying the monitoring information according to the first category information and the second category information to obtain the image information corresponding to the first category information and the voice information corresponding to the second category information in the monitoring information; the information processing equipment extracts first information features of the image information and second information features of the voice information, performs feature recognition on the first information features based on a first preset feature database to obtain a first recognition result, and performs feature recognition on the second information features based on a second preset feature database to obtain a second recognition result; the first preset feature database is an image feature database, and the second preset feature database is a voice feature database; the information processing equipment acquires a second confidence coefficient of the second recognition result obtained by taking the first recognition result as a reference and a first confidence coefficient of the first recognition result obtained by taking the second recognition result as a reference; obtaining a weighting coefficient for weighting the first recognition result and the second recognition result according to the first confidence coefficient and the second confidence coefficient; carrying out weighted summation on the first recognition result and the second recognition result based on the weighting coefficient to obtain a third recognition result, and determining an early warning grade corresponding to the third recognition result; and judging that the target area is abnormal when the early warning level exceeds a set level.
7. The system according to claim 6, wherein the information processing device is specifically configured to:
determining image coding information of the image information, dividing the image coding information according to coding segmentation identifiers in the image coding information to obtain a plurality of continuous coding information segments, determining matching coefficients between every two coding information segments, and carrying out relevance correction on each coding information segment according to all the determined matching coefficients to obtain target coding information segments corresponding to each coding information segment;
determining the code character distribution corresponding to each target code information segment, listing the character structure topology of each code character distribution, determining the character distribution characteristics corresponding to each code character distribution according to the character structure topology, and integrating all the character distribution characteristics to obtain the first information characteristics of the image information;
extracting a spectrogram of the voice information, separating a voiceprint curve in the voice information from the spectrogram, and acquiring voiceprint characteristics corresponding to the voiceprint curve;
text information corresponding to the voice information is obtained according to the voice information, word segmentation processing is carried out on the text information to obtain a plurality of keywords, topic information corresponding to the text information is determined according to semantic connection relations among the keywords, and topic features of the topic information are extracted;
Determining a first association weight of the voiceprint feature relative to the theme feature and a second association weight of the theme feature relative to the voiceprint feature, and carrying out weighted summation on the voiceprint feature and the theme feature based on the first association weight and the second association weight to obtain a second information feature of the voice information.
8. The system according to claim 7, wherein the information processing device is specifically configured to:
acquiring a first target feature with the minimum cosine distance between the first target feature and the first information feature in the first preset feature database;
determining a behavior category corresponding to the first target feature from a preset first mapping relation list, wherein the behavior category is a person obtaining behavior category in the image information;
and generating the first recognition result based on the behavior category.
9. The system according to claim 7, wherein the information processing device is specifically configured to:
determining at least part of second target features, of which the similarity value between the second target features and the second information features in the second preset feature database is smaller than or equal to a set threshold value;
and determining a semantic result corresponding to each second target feature according to a preset second mapping relation list, and fusing all the semantic results to obtain the second identification result.
10. The system according to any of the claims 6-9, characterized in that the information processing device is specifically adapted to:
respectively listing the first identification result and the second identification result in a numerical code form to obtain a first numerical code list corresponding to the first identification result and a second numerical code list corresponding to the second identification result;
pairing each first list unit in the first numerical code list and each second list unit in the second numerical code to obtain at least part of list unit groups; wherein each list unit group comprises a first list unit and a second list unit;
and generating a third numerical code list according to at least part of list unit groups, weighting the third numerical code list based on the weighting coefficient to obtain a fourth numerical code list, and converting the fourth numerical code list into a third identification result based on first conversion logic which lists the first identification result in a numerical code form to obtain a first numerical code list corresponding to the first identification result or based on second conversion logic which lists the second identification result in a numerical code form to obtain a second numerical code list corresponding to the second identification result.
CN202010170331.5A 2020-03-12 2020-03-12 Smart city monitoring method and system Active CN111401218B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202010170331.5A CN111401218B (en) 2020-03-12 2020-03-12 Smart city monitoring method and system

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010170331.5A CN111401218B (en) 2020-03-12 2020-03-12 Smart city monitoring method and system

Publications (2)

Publication Number Publication Date
CN111401218A CN111401218A (en) 2020-07-10
CN111401218B true CN111401218B (en) 2023-05-26

Family

ID=71430731

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010170331.5A Active CN111401218B (en) 2020-03-12 2020-03-12 Smart city monitoring method and system

Country Status (1)

Country Link
CN (1) CN111401218B (en)

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112082475B (en) * 2020-08-25 2022-05-24 中国科学院空天信息创新研究院 Living stumpage species identification method and volume measurement method
CN112992154A (en) * 2021-05-08 2021-06-18 北京远鉴信息技术有限公司 Voice identity determination method and system based on enhanced voiceprint library
CN113420563A (en) * 2021-06-18 2021-09-21 北京易华录信息技术股份有限公司 City civilization monitoring method, system and device based on artificial intelligence

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2000067085A (en) * 1999-07-23 2000-03-03 Sharp Corp System for making non-coded information into data base
JP2001154693A (en) * 1999-11-30 2001-06-08 Sony Corp Robot controller and robot control method and recording medium
US6567775B1 (en) * 2000-04-26 2003-05-20 International Business Machines Corporation Fusion of audio and video based speaker identification for multimedia information access
CN106790054A (en) * 2016-12-20 2017-05-31 四川长虹电器股份有限公司 Interactive authentication system and method based on recognition of face and Application on Voiceprint Recognition
CN110569720A (en) * 2019-07-31 2019-12-13 安徽四创电子股份有限公司 audio and video intelligent identification processing method based on audio and video processing system

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2000067085A (en) * 1999-07-23 2000-03-03 Sharp Corp System for making non-coded information into data base
JP2001154693A (en) * 1999-11-30 2001-06-08 Sony Corp Robot controller and robot control method and recording medium
US6567775B1 (en) * 2000-04-26 2003-05-20 International Business Machines Corporation Fusion of audio and video based speaker identification for multimedia information access
CN106790054A (en) * 2016-12-20 2017-05-31 四川长虹电器股份有限公司 Interactive authentication system and method based on recognition of face and Application on Voiceprint Recognition
CN110569720A (en) * 2019-07-31 2019-12-13 安徽四创电子股份有限公司 audio and video intelligent identification processing method based on audio and video processing system

Also Published As

Publication number Publication date
CN111401218A (en) 2020-07-10

Similar Documents

Publication Publication Date Title
CN111401218B (en) Smart city monitoring method and system
CN109525595B (en) Black product account identification method and equipment based on time flow characteristics
CN109765539B (en) Indoor user behavior monitoring method and device, electrical equipment and home monitoring system
CN109088869B (en) APT attack detection method and device
CN110109998B (en) Intelligent data transaction integration system
CN112069276B (en) Address coding method, address coding device, computer equipment and computer readable storage medium
CN115098705B (en) Network security event analysis method and system based on knowledge graph reasoning
CN113159105B (en) Driving behavior unsupervised mode identification method and data acquisition monitoring system
CN109740573B (en) Video analysis method, device, equipment and server
KR20130107424A (en) Method, system and control system for detecting and forecasting fog based on ubiquitous sensor network and semantic technology
CN110443219B (en) Driving behavior abnormity detection method and device and industrial equipment
CN113450545A (en) Natural disaster early warning system and method, cloud platform and storable medium
CN112464951B (en) Fishery safety emergency disposal system and method based on Internet of things
CN113340353B (en) Monitoring method, equipment and medium for power transmission line
JP2018503183A (en) Vocabulary analyzer for neuro-language behavior recognition system
CN110533094B (en) Evaluation method and system for driver
CN110648172A (en) Identity recognition method and system fusing multiple mobile devices
WO2015027830A1 (en) Systems and methods for acquiring water depth information
CN113435753A (en) Enterprise risk judgment method, device, equipment and medium in high-risk industry
KR102470153B1 (en) Risks detecting system and method for using pattern analysis of layered tags in user log data
CN113569965A (en) User behavior analysis method and system based on Internet of things
CN115294682B (en) Data management method, device and equipment for intelligent door lock and storage medium
CN109447177B (en) Account clustering method and device and server
CN110968570A (en) Distributed big data mining system facing E-commerce platform
CN113486342A (en) Information security processing method and system based on user behavior analysis

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
CB02 Change of applicant information
CB02 Change of applicant information

Address after: Dingyi business center, 99 Keyuan Road, high tech Zone, Kunming, Yunnan 650033

Applicant after: Gao Kuanyou

Address before: 655604 No.9, Zaiyi village committee, Longhai Township, Luliang County, Qujing City, Yunnan Province

Applicant before: Gao Kuanyou

TA01 Transfer of patent application right
TA01 Transfer of patent application right

Effective date of registration: 20230508

Address after: No.536 Laolu Road, Lingang New District, China (Shanghai) pilot Free Trade Zone, Pudong New Area, Shanghai, 200120

Applicant after: Shanghai Hongdian Intelligent Technology Co.,Ltd.

Address before: Dingyi business center, 99 Keyuan Road, high tech Zone, Kunming, Yunnan 650033

Applicant before: Gao Kuanyou

GR01 Patent grant
GR01 Patent grant