CN117891589A - Distribution method, device and equipment of heterogeneous hard disk and computer readable storage medium - Google Patents
Distribution method, device and equipment of heterogeneous hard disk and computer readable storage medium Download PDFInfo
- Publication number
- CN117891589A CN117891589A CN202311585804.8A CN202311585804A CN117891589A CN 117891589 A CN117891589 A CN 117891589A CN 202311585804 A CN202311585804 A CN 202311585804A CN 117891589 A CN117891589 A CN 117891589A
- Authority
- CN
- China
- Prior art keywords
- label information
- node
- message
- partition
- hard disk
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 50
- 238000003860 storage Methods 0.000 title claims abstract description 25
- 238000009826 distribution Methods 0.000 title claims abstract description 17
- 238000005192 partition Methods 0.000 claims abstract description 75
- 230000007246 mechanism Effects 0.000 claims abstract description 44
- 238000004590 computer program Methods 0.000 claims description 16
- 238000002372 labelling Methods 0.000 claims description 8
- 238000012423 maintenance Methods 0.000 abstract description 6
- 238000010586 diagram Methods 0.000 description 9
- 230000003287 optical effect Effects 0.000 description 5
- 238000004891 communication Methods 0.000 description 4
- 230000006870 function Effects 0.000 description 4
- 239000007787 solid Substances 0.000 description 4
- 230000009471 action Effects 0.000 description 3
- 230000008569 process Effects 0.000 description 3
- 238000004519 manufacturing process Methods 0.000 description 2
- 239000013307 optical fiber Substances 0.000 description 2
- 238000012545 processing Methods 0.000 description 2
- 230000000644 propagated effect Effects 0.000 description 2
- 239000004065 semiconductor Substances 0.000 description 2
- 230000000694 effects Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 239000000835 fiber Substances 0.000 description 1
- 238000002955 isolation Methods 0.000 description 1
- 230000007774 longterm Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000002688 persistence Effects 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 238000000638 solvent extraction Methods 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Information Transfer Between Computers (AREA)
Abstract
The present disclosure relates to a method, an apparatus, a device, and a computer-readable storage medium for distributing heterogeneous hard disks, the method comprising: creating a plurality of message sets, and respectively determining the partition of each message set, wherein the message sets carry set label information; responding to the online of the nodes of the system, and determining node label information according to the hard disk configuration of the nodes; and selecting a first target node corresponding to the partition of the message set according to the set label information, the node label information and a preset label mechanism so as to realize the distribution of the heterogeneous hard disk. According to the method and the device, the first target node corresponding to the partition of the message set is selected through the set label information of the message set, the node label information determined by the hard disk configuration of the node and the preset label mechanism, so that the hardware cost and the manual operation and maintenance cost of the heterogeneous hard disk are reduced, the performance load of the heterogeneous hard disk node is balanced, and the user experience is improved.
Description
Technical Field
The disclosure relates to the field of computer technology, and in particular, to a method, a device, equipment and a computer readable storage medium for distributing heterogeneous hard disks.
Background
In the prior art, the Partition allocation mechanism of Kafka default allocation, that is, kafka orders all the Broker and Partition, and allocates the Partition to each Broker in turn. Disadvantages of this allocation mechanism include: when the heterogeneous hard disk has a mixed part of a mechanical hard disk and a solid state hard disk, it is difficult to balance the performance of each node (Broker) in the Kafka cluster; kafka unifies Partition partitioning strategies for message sets (topics), and does not take into account the different performance requirements of different topics.
Disclosure of Invention
In order to solve the above technical problems or at least partially solve the above technical problems, the present disclosure provides a method, an apparatus, a device, and a computer readable storage medium for distributing heterogeneous hard disks, so as to balance performance load of heterogeneous hard disk nodes and reduce artificial operation and maintenance costs.
In a first aspect, an embodiment of the present disclosure provides a method for allocating heterogeneous hard disks, including:
creating a plurality of message sets, and respectively determining the partition of each message set, wherein the message sets carry set label information;
responding to the online of the nodes of the system, and determining node label information according to the hard disk configuration of the nodes;
and selecting a first target node corresponding to the partition of the message set according to the set label information, the node label information and a preset label mechanism so as to realize the distribution of the heterogeneous hard disk.
In a second aspect, an embodiment of the present disclosure provides a device for distributing heterogeneous hard disks, including:
the system comprises a creation module, a storage module and a storage module, wherein the creation module is used for creating a plurality of message sets, respectively determining the partition of each message set, and the message sets carry set label information;
the determining module is used for responding to the online of the nodes of the system and determining node tag information according to the hard disk configuration of the nodes;
the selection module is used for selecting a first target node corresponding to the partition of the message set according to the set label information, the node label information and a preset label mechanism so as to realize the distribution of heterogeneous hard disks.
In a third aspect, an embodiment of the present disclosure provides an electronic device, including:
a memory;
a processor; and
a computer program;
wherein the computer program is stored in the memory and configured to be executed by the processor to implement the method according to the first aspect.
In a fourth aspect, embodiments of the present disclosure provide a computer-readable storage medium having stored thereon a computer program for execution by a processor to implement the method of the first aspect.
In a fifth aspect, embodiments of the present disclosure also provide a computer program product comprising a computer program or instructions which, when executed by a processor, implement the method of the first aspect.
The embodiment of the disclosure provides a method, a device, equipment and a computer readable storage medium for distributing heterogeneous hard disks, which are characterized in that a plurality of message sets are created to respectively determine the partition of each message set, and the message sets carry set label information; responding to the online of the nodes of the system, and determining node label information according to the hard disk configuration of the nodes; according to the set label information, the node label information and the preset label mechanism, a first target node corresponding to the partition of the message set is selected to realize the distribution of the heterogeneous hard disk, so that the hardware cost and the manual operation and maintenance cost of the heterogeneous hard disk are reduced, the performance load of the heterogeneous hard disk node is balanced, and the user experience is improved.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the disclosure and together with the description, serve to explain the principles of the disclosure.
In order to more clearly illustrate the embodiments of the present disclosure or the solutions in the prior art, the drawings that are required for the description of the embodiments or the prior art will be briefly described below, and it will be obvious to those skilled in the art that other drawings can be obtained from these drawings without inventive effort.
Fig. 1 is a flowchart of a method for distributing heterogeneous hard disks according to an embodiment of the present disclosure;
fig. 2 is a schematic diagram of an application scenario provided in an embodiment of the present disclosure;
fig. 3 is a schematic structural diagram of a distributing device for heterogeneous hard disks according to an embodiment of the present disclosure;
fig. 4 is a schematic structural diagram of an electronic device according to an embodiment of the disclosure.
Detailed Description
In order that the above objects, features and advantages of the present disclosure may be more clearly understood, a further description of aspects of the present disclosure will be provided below. It should be noted that, without conflict, the embodiments of the present disclosure and features in the embodiments may be combined with each other.
In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present disclosure, but the present disclosure may be practiced otherwise than as described herein; it will be apparent that the embodiments in the specification are only some, but not all, embodiments of the disclosure.
In the prior art, the Partition allocation mechanism of Kafka default allocation, that is, kafka orders all the Broker and Partition, and allocates the Partition to each Broker in turn. The disadvantages of this allocation mechanism include at least the following two points:
first, kafka is an IO intensive service, as the scale of the Kafka cluster increases, multiple heterogeneous hard disks may appear in the cluster, and when there is a mechanical hard disk and solid state hard disk hybrid in the heterogeneous hard disk, it is difficult to balance the performance of each node (Broker) in the Kafka cluster.
Second, as traffic grows, the number of topics in Kafka increases, the performance required by different topics is different, and the Partition policy of Kafka on topics is uniform, and the performance requirement of different topics is not considered.
In view of the foregoing, embodiments of the present disclosure provide a method for distributing heterogeneous hard disks, which is described below with reference to specific embodiments.
Fig. 1 is a flowchart of a method for distributing heterogeneous hard disks according to an embodiment of the present disclosure. The method can be executed by a distributing device of the heterogeneous hard disk, the distributing device of the heterogeneous hard disk can be realized in a software and/or hardware mode, the distributing device of the heterogeneous hard disk can be configured in electronic equipment, such as a server or a terminal, wherein the terminal specifically comprises a mobile phone, a computer or a tablet computer and the like. In addition, the method can be applied to application scenes of the distribution of the heterogeneous hard disk, and it can be understood that the method for distributing the heterogeneous hard disk provided by the embodiment of the disclosure can also be applied to other scenes.
The following describes a method for distributing heterogeneous hard disks shown in fig. 1, which includes the following specific steps:
s101, creating a plurality of message sets, and respectively determining the partition of each message set, wherein the message sets carry set label information.
The system creates a plurality of message sets carrying set tag information, and determines the partition of each message set respectively.
Kafka is a high throughput distributed publish-subscribe messaging system for large data products that provides persistence of messages through the disk data structure of O (1), which maintains long-term stable performance for message storage even in TB. The method can provide millions of message performance support per second on common server hardware, can provide a better production and consumption interface, and can realize the capabilities of effective consumption isolation and the like.
Broker: all system nodes in the Kafka cluster are called broaders, which are responsible for providing production and consumption requests for clients and for storing all data.
Topic is a collection of messages sent to Kafka clusters, which is a logical concept, a Kafka cluster can theoretically create any number of Topic, and all messages sent to Kafka must specify collection tag information.
Partition: namely, the partition of Topic, is a physical concept; each Topic needs to contain at least one Partition, which can create multiple partitions, each of which is equivalent to a commit log of a message, the messages are written to the partitions in an appended manner, the messages within a single Partition guarantee order, and each Partition has and can only be allocated to one Broker.
Illustratively, a plurality of message sets (Topic) may be created by the Kafka cluster in the system, each message set (Topic) created by the Kafka cluster carries set tag information when created, that is, each message set (Topic) may specify set tag information when created, where the set tag information may be node tag information contained in a Broker data directory or may be a default tag, and it is understood that when Topic creates unspecified set tag information, the Topic is automatically identified as a default tag, that is, the Topic set tag information is default. The partition of each message set (Topic) is determined separately.
S102, responding to the online of the nodes of the system, and determining node label information according to the hard disk configuration of the nodes.
And responding to the online of the nodes of the system, and determining node label information according to the hard disk configuration of the nodes by the system.
Specifically, when a node (Broker) of the Kafka cluster is online, the system configures node tag information for the node (Broker) according to a hard disk configuration of the node (Broker), where the node tag information may be set by the system or may be customized according to personal habits of a user, and the embodiment is not limited, and exemplarily, the hard disk of the node (Broker) is configured as a solid state disk, and the node tag information is a Solid State Drive (SSD); if the node (Broker) is not configured with node tag information, the node (Broker) is automatically configured as a default tag, i.e. the node tag information is default.
S103, selecting a first target node corresponding to the partition of the message set according to the set label information, the node label information and a preset label mechanism so as to realize the distribution of heterogeneous hard disks.
The system selects a first target node corresponding to the partition of the message set according to the set label information, the node label information and a preset label mechanism so as to realize the distribution of the heterogeneous hard disk.
Optionally, the preset labeling mechanism includes: when the set label information belongs to the node label information, the partition of the message set corresponding to the set label information is distributed to the node corresponding to the node label information.
Specifically, the preset labeling mechanism includes: when the set label information belongs to the node label information, the partition of the message set corresponding to the set label information is distributed to the node corresponding to the node label information.
Correspondingly, according to the set label information, the node label information and a preset label mechanism, selecting a first target node corresponding to the partition of the message set to realize the distribution of heterogeneous hard disks, including: and selecting node label information corresponding to the partition of the message set according to the preset label mechanism through the set label information, and determining a first target node according to the node label information.
When the set tag information belongs to the node tag information, as shown in fig. 2, for example, the set tag information carried by the Topic1 (message set 1) is SSD, the partitions of the Topic1 (message set 1) are Topic1 Partition1 and Topic1 Partition2, the Partition of the message set corresponding to the set tag information is allocated to the node corresponding to the node tag information according to the preset tag mechanism (i.e., when the set tag information belongs to the node tag information), the node tag information SSD corresponding to the Partition of the Topic1 Partition1 and the Partition of the Topic1 (message set 1) is selected by the set tag information SSD, and the first target node Broker1SSD and the Broker2 SSD are determined according to the node tag information.
For example, as shown in fig. 2, when the set tag information is not specified during the creation of the set tag information, the set tag information carried by the set tag 2 (message set 2) is identified as default, the Partition of the set tag 2 (message set 2) is identified as the default 1 and the Partition of the set tag 2, according to a preset tag mechanism (that is, when the set tag information belongs to the node tag information, the Partition of the message set corresponding to the set tag information is allocated to the node corresponding to the node tag information), the Partition of the set tag 2 (message set 2) is selected by the default, and the node tag information default corresponding to the Partition of the set 2 (message set 2) is selected by the default, and the first target nodes, namely the first target node, and the first target node, are determined according to the node tag 4 default.
Optionally, the preset labeling mechanism further includes: and when the set label information does not belong to the node label information, the partitions of the message set corresponding to the set label information are distributed to the nodes in sequence.
Specifically, the preset labeling mechanism further includes: and when the set label information does not belong to the node label information, the partitions of the message set corresponding to the set label information are distributed to the nodes in sequence.
Correspondingly, according to the set label information, the node label information and a preset label mechanism, selecting a first target node corresponding to the partition of the message set to realize the distribution of heterogeneous hard disks, including: and determining a first target node of the partition of the message set according to the preset label mechanism and the selected sequence of the node label information.
When the aggregate tag information does not belong to the node tag information, as shown in fig. 2, for example, aggregate tag information carried by Topic3 (message set 3) is SATA, partitions of Topic3 (message set 3) are Topic3Partition 1 and Topic3Partition2, partitions of the message set corresponding to the aggregate tag information are sequentially allocated to the node according to a preset tag mechanism (i.e., when the aggregate tag information does not belong to the node tag information), and the first target node Broker1SSD and Broker2 SSD of Topic3 (message set 3) are determined according to the selected order of the node tag information Broker1SSD, broker2 SSD, broker3 default, and Broker4 default.
According to the embodiment of the disclosure, a plurality of message sets are created, the partition of each message set is determined, and the message sets carry set tag information; responding to the online of the nodes of the system, and determining node label information according to the hard disk configuration of the nodes; according to the set label information, the node label information and the preset label mechanism, a first target node corresponding to the partition of the message set is selected to realize the distribution of the heterogeneous hard disk, so that the hardware cost and the manual operation and maintenance cost of the heterogeneous hard disk are reduced, the performance load of the heterogeneous hard disk node is balanced, and the user experience is improved.
In some embodiments, the method further comprises: modifying the set label information of at least one message set in the message sets to obtain modified set label information; and based on the modified set label information, validating, and selecting a second target node corresponding to the partition of the message set according to the modified set label information, the node label information and a preset label mechanism so as to realize the redistribution of the heterogeneous hard disk.
The system modifies the set label information of at least one message set in the message sets to obtain modified set label information; and based on the modified set label information, validating, and selecting a second target node corresponding to the partition of the message set according to the modified set label information, the node label information and a preset label mechanism so as to realize the redistribution of the heterogeneous hard disk.
According to the embodiment of the disclosure, the collection label information of at least one message collection in the plurality of message collections is modified, the modified collection label information is effective, and the second target node corresponding to the partition of the message collection is selected according to the modified collection label information, the node label information and the preset label mechanism, so that the redistribution of the heterogeneous hard disk is realized, the hardware cost and the manual operation and maintenance cost of the heterogeneous hard disk are further reduced, the performance load of the heterogeneous hard disk node is balanced, and the user experience is improved.
In some embodiments, the method further comprises: modifying the node label information to obtain modified node label information; and based on the modified node label information, validating, and selecting a third target node corresponding to the partition of the message set according to the set label information, the modified node label information and a preset label mechanism so as to realize the redistribution of the heterogeneous hard disk.
The system modifies the node label information to obtain modified node label information; and based on the modified node label information, validating, and selecting a third target node corresponding to the partition of the message set according to the set label information, the modified node label information and a preset label mechanism so as to realize the redistribution of the heterogeneous hard disk.
Optionally, after modifying the node tag information to obtain modified node tag information, the method further includes: the nodes of the system are restarted.
Specifically, the system modifies the node tag information to obtain modified node tag information; restarting a node of the system; and based on the modified node label information, validating, and selecting a third target node corresponding to the partition of the message set according to the set label information, the modified node label information and a preset label mechanism so as to realize the redistribution of the heterogeneous hard disk.
According to the embodiment of the disclosure, the node label information is modified, so that modified node label information is obtained; restarting a node of the system; based on the modified node label information, the third target node corresponding to the partition of the message set is selected according to the set label information, the modified node label information and the preset label mechanism, so that the redistribution of the heterogeneous hard disk is realized, the hardware cost and the manual operation and maintenance cost of the heterogeneous hard disk are further reduced, the performance load of the heterogeneous hard disk node is balanced, and the user experience is improved.
Fig. 3 is a schematic structural diagram of a distributing device for heterogeneous hard disks according to an embodiment of the present disclosure. The distribution device of the heterogeneous hard disk may be a terminal as described in the above embodiments, or the distribution device of the heterogeneous hard disk may be a part or component in the terminal. The heterogeneous hard disk allocation device provided in the embodiment of the present disclosure may execute the processing flow provided in the embodiment of the method for allocating heterogeneous hard disks, as shown in fig. 3, where the heterogeneous hard disk allocation device 30 includes: a creation module 31, a determination module 32, a selection module 33; the creating module 31 is configured to create a plurality of message sets, and determine a partition of each message set, where the message sets carry set tag information; a determining module 32, configured to determine node tag information according to a hard disk configuration of a node in response to a node of the system being online; and the selection module 33 is configured to select a first target node corresponding to the partition of the message set according to the set label information, the node label information, and a preset label mechanism, so as to implement allocation of heterogeneous hard disks.
Optionally, the preset labeling mechanism includes: when the set label information belongs to the node label information, the partition of the message set corresponding to the set label information is distributed to the node corresponding to the node label information.
Optionally, the preset labeling mechanism further includes: and when the set label information does not belong to the node label information, the partitions of the message set corresponding to the set label information are distributed to the nodes in sequence.
Optionally, the selecting module 33 is further configured to modify set tag information of at least one message set in the plurality of message sets, to obtain modified set tag information; and based on the modified set label information, validating, and selecting a second target node corresponding to the partition of the message set according to the modified set label information, the node label information and a preset label mechanism so as to realize the redistribution of the heterogeneous hard disk.
Optionally, the selecting module 33 is further configured to modify the node tag information to obtain modified node tag information; and based on the modified node label information, validating, and selecting a third target node corresponding to the partition of the message set according to the set label information, the modified node label information and a preset label mechanism so as to realize the redistribution of the heterogeneous hard disk.
Optionally, the selecting module 33 is further configured to restart the node of the system after modifying the node tag information to obtain modified node tag information.
The device for distributing heterogeneous hard disks in the embodiment shown in fig. 3 may be used to implement the technical solution of the above embodiment of the method for distributing heterogeneous hard disks, and its implementation principle and technical effects are similar, and are not repeated here.
Fig. 4 is a schematic structural diagram of an electronic device according to an embodiment of the disclosure. The electronic device may be a terminal as described in the above embodiments. The electronic device provided in the embodiment of the present disclosure may execute the processing flow provided in the embodiment of the method for distributing heterogeneous hard disks, as shown in fig. 4, where the electronic device 40 includes: memory 41, processor 42, computer programs and communication interface 43; wherein the computer program is stored in the memory 41 and configured to be executed by the processor 42 for the allocation method of heterogeneous hard disks as described above.
In addition, the embodiment of the present disclosure also provides a computer readable storage medium having stored thereon a computer program that is executed by a processor to implement the method for distributing heterogeneous hard disks described in the above embodiment.
Furthermore, the embodiments of the present disclosure also provide a computer program product comprising a computer program or instructions which, when executed by a processor, implement a method of assigning heterogeneous hard disks as described above.
It should be noted that the computer readable medium described in the present disclosure may be a computer readable signal medium or a computer readable storage medium, or any combination of the two. The computer readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples of the computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this disclosure, a computer-readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In the present disclosure, however, the computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, with the computer-readable program code embodied therein. Such a propagated data signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination of the foregoing. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: electrical wires, fiber optic cables, RF (radio frequency), and the like, or any suitable combination of the foregoing.
In some implementations, the clients, servers may communicate using any currently known or future developed network protocol, such as HTTP (HyperText Transfer Protocol ), and may be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), the internet (e.g., the internet), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future developed networks.
The computer readable medium may be contained in the electronic device; or may exist alone without being incorporated into the electronic device.
The computer readable medium carries one or more programs which, when executed by the electronic device, cause the electronic device to:
creating a plurality of message sets, and respectively determining the partition of each message set, wherein the message sets carry set label information;
responding to the online of the nodes of the system, and determining node label information according to the hard disk configuration of the nodes;
and selecting a first target node corresponding to the partition of the message set according to the set label information, the node label information and a preset label mechanism so as to realize the distribution of the heterogeneous hard disk.
In addition, the electronic device may also perform other steps in the method of assigning heterogeneous hard disks as described above.
Computer program code for carrying out operations of the present disclosure may be written in one or more programming languages, including, but not limited to, an object oriented programming language such as Java, smalltalk, C ++ and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computer (for example, through the Internet using an Internet service provider).
The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The units involved in the embodiments of the present disclosure may be implemented by means of software, or may be implemented by means of hardware. Wherein the names of the units do not constitute a limitation of the units themselves in some cases.
The functions described above herein may be performed, at least in part, by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: a Field Programmable Gate Array (FPGA), an Application Specific Integrated Circuit (ASIC), an Application Specific Standard Product (ASSP), a system on a chip (SOC), a Complex Programmable Logic Device (CPLD), and the like.
In the context of this disclosure, a machine-readable medium may be a tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. The machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
It should be noted that in this document, relational terms such as "first" and "second" and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.
The foregoing is merely a specific embodiment of the disclosure to enable one skilled in the art to understand or practice the disclosure. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the disclosure. Thus, the present disclosure is not intended to be limited to the embodiments shown and described herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims (10)
1. A method for distributing heterogeneous hard disks, the method comprising:
creating a plurality of message sets, and respectively determining the partition of each message set, wherein the message sets carry set label information;
responding to the online of the nodes of the system, and determining node label information according to the hard disk configuration of the nodes;
and selecting a first target node corresponding to the partition of the message set according to the set label information, the node label information and a preset label mechanism so as to realize the distribution of the heterogeneous hard disk.
2. The method of claim 1, wherein the preset labeling mechanism comprises:
when the set label information belongs to the node label information, the partition of the message set corresponding to the set label information is distributed to the node corresponding to the node label information.
3. The method of claim 1, wherein the preset labeling mechanism further comprises:
and when the set label information does not belong to the node label information, the partitions of the message set corresponding to the set label information are distributed to the nodes in sequence.
4. The method of claim 2, wherein selecting a first target node corresponding to a partition of the message set according to the set label information, the node label information, and a preset label mechanism, comprises:
and selecting node label information corresponding to the partition of the message set according to the preset label mechanism through the set label information, and determining a first target node according to the node label information.
5. The method according to claim 1, wherein the method further comprises:
modifying the set label information of at least one message set in the message sets to obtain modified set label information;
and based on the modified set label information, validating, and selecting a second target node corresponding to the partition of the message set according to the modified set label information, the node label information and a preset label mechanism so as to realize the redistribution of the heterogeneous hard disk.
6. The method according to claim 1, wherein the method further comprises:
modifying the node label information to obtain modified node label information;
and based on the modified node label information, validating, and selecting a third target node corresponding to the partition of the message set according to the set label information, the modified node label information and a preset label mechanism so as to realize the redistribution of the heterogeneous hard disk.
7. The method of claim 6, wherein after modifying the node tag information to obtain modified node tag information, the method further comprises:
the nodes of the system are restarted.
8. A device for distributing heterogeneous hard disks, the device comprising:
the system comprises a creation module, a storage module and a storage module, wherein the creation module is used for creating a plurality of message sets, respectively determining the partition of each message set, and the message sets carry set label information;
the determining module is used for responding to the online of the nodes of the system and determining node tag information according to the hard disk configuration of the nodes;
the selection module is used for selecting a first target node corresponding to the partition of the message set according to the set label information, the node label information and a preset label mechanism so as to realize the distribution of heterogeneous hard disks.
9. An electronic device, comprising:
a memory;
a processor; and
a computer program;
wherein the computer program is stored in the memory and configured to be executed by the processor to implement the method of any of claims 1-7.
10. A computer readable storage medium, on which a computer program is stored, which computer program, when being executed by a processor, implements the method according to any of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311585804.8A CN117891589A (en) | 2023-11-24 | 2023-11-24 | Distribution method, device and equipment of heterogeneous hard disk and computer readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311585804.8A CN117891589A (en) | 2023-11-24 | 2023-11-24 | Distribution method, device and equipment of heterogeneous hard disk and computer readable storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117891589A true CN117891589A (en) | 2024-04-16 |
Family
ID=90639975
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311585804.8A Pending CN117891589A (en) | 2023-11-24 | 2023-11-24 | Distribution method, device and equipment of heterogeneous hard disk and computer readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117891589A (en) |
-
2023
- 2023-11-24 CN CN202311585804.8A patent/CN117891589A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10614117B2 (en) | Sharing container images between mulitple hosts through container orchestration | |
| CN110896355B (en) | Network slice selection method and device | |
| US10545914B2 (en) | Distributed object storage | |
| CN107391629B (en) | Method, system, server and computer storage medium for data migration between clusters | |
| CN111597148B (en) | Distributed metadata management method for distributed file system | |
| CN110166507B (en) | Multi-resource scheduling method and device | |
| JP2022519810A (en) | Communication method, device, entity and computer readable storage medium | |
| CN109905431B (en) | Message processing method and system, storage medium and electronic equipment | |
| US10831716B2 (en) | Method and apparatus for configuring relevant parameters of MapReduce applications | |
| CN112905342A (en) | Resource scheduling method, device, equipment and computer readable storage medium | |
| US20210314371A1 (en) | Network-based media processing (nbmp) workflow management through 5g framework for live uplink streaming (flus) control | |
| CN111124299A (en) | Data storage management method, device, equipment, system and storage medium | |
| CN114490062A (en) | Local disk scheduling method and device, electronic equipment and storage medium | |
| JP2023545985A (en) | Managing task flows in edge computing environments | |
| CN109729110B (en) | Method, apparatus and computer readable medium for managing dedicated processing resources | |
| US11106514B2 (en) | Message stream processor microbatching | |
| KR20170014804A (en) | Virtual machine provisioning system and method for cloud service | |
| CN112860421B (en) | Method, apparatus and computer program product for job processing | |
| US20230283695A1 (en) | Communication Protocol for Knative Eventing's Kafka components | |
| CN117891589A (en) | Distribution method, device and equipment of heterogeneous hard disk and computer readable storage medium | |
| CN111694670B (en) | Resource allocation method, apparatus, device and computer readable medium | |
| CN112860422B (en) | Method, apparatus and computer program product for job processing | |
| CN114253730A (en) | Method, device and equipment for managing database memory and storage medium | |
| US10200301B1 (en) | Logical control groups for distributed system resources | |
| CN112616143A (en) | Method and device for distributing communication number, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |