CN112753016A - 神经网络中数据预处理阶段的计算资源的管理方法和装置 - Google Patents

神经网络中数据预处理阶段的计算资源的管理方法和装置 Download PDF

Info

Publication number
CN112753016A
CN112753016A CN201880098036.4A CN201880098036A CN112753016A CN 112753016 A CN112753016 A CN 112753016A CN 201880098036 A CN201880098036 A CN 201880098036A CN 112753016 A CN112753016 A CN 112753016A
Authority
CN
China
Prior art keywords
computing
resource
information
node
nodes
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201880098036.4A
Other languages
English (en)
Inventor
范礼
路石
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Huawei Technologies Co Ltd
Original Assignee
Huawei Technologies Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Huawei Technologies Co Ltd filed Critical Huawei Technologies Co Ltd
Publication of CN112753016A publication Critical patent/CN112753016A/zh
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]

Landscapes

  • Engineering & Computer Science (AREA)
  • Software Systems (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Debugging And Monitoring (AREA)
  • Data Exchanges In Wide-Area Networks (AREA)

Abstract

一种应用于神经网络的数据预处理阶段的计算资源的管理方法和装置，该计算资源包括多个异构的计算节点，该方法包括：分别监测所述多个计算节点的资源使用信息(S310)；根据所述资源使用信息，并基于预设的资源调度策略，生成与所述多个计算节点中的待调整节点对应的资源调整信息(S320)；根据所述资源调整信息，对所述待调整节点的计算资源进行动态调整(S330)，能够提高计算资源的利用率，有助于减少神经网络模型的训练时间。

Description

PCT国内申请，说明书已公开。

Claims (18)

  1. PCT国内申请，权利要求书已公开。
CN201880098036.4A 2018-09-30 2018-09-30 神经网络中数据预处理阶段的计算资源的管理方法和装置 Pending CN112753016A (zh)

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2018/109181 WO2020062277A1 (zh) 2018-09-30 2018-09-30 神经网络中数据预处理阶段的计算资源的管理方法和装置

Publications (1)

Publication Number Publication Date
CN112753016A true CN112753016A (zh) 2021-05-04

Family

ID=69950217

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201880098036.4A Pending CN112753016A (zh) 2018-09-30 2018-09-30 神经网络中数据预处理阶段的计算资源的管理方法和装置

Country Status (2)

Country Link
CN (1) CN112753016A (zh)
WO (1) WO2020062277A1 (zh)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113391907A (zh) * 2021-06-25 2021-09-14 中债金科信息技术有限公司 一种任务的放置方法、装置、设备和介质
CN114756372A (zh) * 2022-04-28 2022-07-15 北京百度网讯科技有限公司 用于负载均衡的方法、装置、设备和介质
WO2023093375A1 (zh) * 2021-11-25 2023-06-01 北京九章云极科技有限公司 一种计算资源获取方法、装置、电子设备和存储介质

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20040216114A1 (en) * 2003-04-22 2004-10-28 Lin Sheng Ling Balancing loads among computing nodes
CN103617086A (zh) * 2013-11-20 2014-03-05 东软集团股份有限公司 一种并行计算方法及***
CN103812895A (zh) * 2012-11-12 2014-05-21 华为技术有限公司 调度方法、管理节点以及云计算集群
CN104168332A (zh) * 2014-09-01 2014-11-26 广东电网公司信息中心 高性能计算中负载均衡与节点状态监控方法
CN108200156A (zh) * 2017-12-29 2018-06-22 南京邮电大学 一种云环境下分布式文件***的动态负载均衡方法

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20040216114A1 (en) * 2003-04-22 2004-10-28 Lin Sheng Ling Balancing loads among computing nodes
CN103812895A (zh) * 2012-11-12 2014-05-21 华为技术有限公司 调度方法、管理节点以及云计算集群
CN103617086A (zh) * 2013-11-20 2014-03-05 东软集团股份有限公司 一种并行计算方法及***
CN104168332A (zh) * 2014-09-01 2014-11-26 广东电网公司信息中心 高性能计算中负载均衡与节点状态监控方法
CN108200156A (zh) * 2017-12-29 2018-06-22 南京邮电大学 一种云环境下分布式文件***的动态负载均衡方法

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113391907A (zh) * 2021-06-25 2021-09-14 中债金科信息技术有限公司 一种任务的放置方法、装置、设备和介质
WO2023093375A1 (zh) * 2021-11-25 2023-06-01 北京九章云极科技有限公司 一种计算资源获取方法、装置、电子设备和存储介质
CN114756372A (zh) * 2022-04-28 2022-07-15 北京百度网讯科技有限公司 用于负载均衡的方法、装置、设备和介质

Also Published As

Publication number Publication date
WO2020062277A1 (zh) 2020-04-02

Similar Documents

Publication Publication Date Title
CN111626430B (zh) 一种数据处理方法及相关产品
KR102606825B1 (ko) 뉴럴 네트워크 모델을 변형하는 뉴럴 네트워크 시스템, 이를 포함하는 어플리케이션 프로세서 및 뉴럴 네트워크 시스템의 동작방법
US9424079B2 (en) Iteration support in a heterogeneous dataflow engine
CN110674936A (zh) 一种神经网络处理方法、装置、计算机设备及存储介质
US20180260700A1 (en) Method and system for implementing reinforcement learning agent using reinforcement learning processor
US20210295168A1 (en) Gradient compression for distributed training
CN107679625B (zh) 针对数据记录执行机器学习的分布式***及其方法
CN112753016A (zh) 神经网络中数据预处理阶段的计算资源的管理方法和装置
CN111191789B (zh) 模型优化部署***、芯片、电子设备及介质
WO2023246801A1 (zh) 算法流水线编排方法、装置、电子设备和存储介质
JP2021507345A (ja) 畳み込みニューラル・ネットワークの完全なカーネルを近似するためのスパース・カーネルの融合
CN114118433A (zh) 一种设备的配置参数的推荐方法及装置
US20210158131A1 (en) Hierarchical partitioning of operators
WO2018175164A1 (en) Resource-efficient machine learning
CN112099882B (zh) 一种业务处理方法、装置及设备
EP3926546A2 (en) Neural network model splitting method, apparatus, computer device and storage medium
US11551095B2 (en) Sharing preprocessing, computations, and hardware resources between multiple neural networks
Li et al. An intelligent collaborative inference approach of service partitioning and task offloading for deep learning based service in mobile edge computing networks
CN112099848A (zh) 一种业务处理方法、装置及设备
US11308396B2 (en) Neural network layer-by-layer debugging
US20220067495A1 (en) Intelligent processor, data processing method and storage medium
US12014202B2 (en) Method and apparatus with accelerator
US11631001B2 (en) Heterogeneous computing on a system-on-chip, including machine learning inference
Wang et al. Edge computing for artificial intelligence
Ting et al. System services for reconfigurable hardware acceleration in mobile devices

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination