Shared Memory-Contention-Aware Concurrent DNN Execution for Diversely Heterogeneous System-on-Chips

Details

• Personal Author:
  Belviranli ME ; Dagli I
• Description:
  Two distinguishing features of state-of-the-art mobile and autonomous systems are: 1) There are often multiple workloads, mainly deep neural network (DNN) inference, running concurrently and continuously. 2) They operate on shared memory System-on-Chips (SoC) that embed heterogeneous accelerators tailored for specific operations. State-of-the-art systems lack efficient performance and resource management techniques necessary to either maximize total system throughput or minimize end-to-end workload latency. In this work, we propose HaX-CoNN, a novel scheme that characterizes and maps layers in concurrently executing DNN inference workloads to a diverse set of accelerators within an SoC. Our scheme uniquely takes per-layer execution characteristics, shared memory (SM) contention, and inter-accelerator transitions into account to find optimal schedules. We evaluate HaX-CoNN on NVIDIA Orin, NVIDIA Xavier, and Qualcomm Snapdragon 865 SoCs. Our experimental results indicate that HaX-CoNN can minimize memory contention by up to 45% and improve total latency and throughput by up to 32% and 29%, respectively, compared to the state-of-the-art. [Description provided by NIOSH]
• Subjects:
  Automation Mining Occupational Health Safety
• Keywords:
  Artificial Intelligence Author Keywords: Concurrent DNN Inference Computer Assisted Mining Emerging Technology Heterogeneous SoCs Mining Industry Neural Networks Shared Memory Contention
• ISBN:
  9798400704352
• Publisher:
  Association for Computing Machinery (ACM)
• Document Type:
  Text
• Funding:
  Contract
• Genre:
  Proceedings
• Place as Subject:
  Colorado ; OSHA Region 8
• CIO:
  National Institute for Occupational Safety and Health (NIOSH)
• Topic:
  Workplace Safety & Health
• Location:
  North America
• Pages in Document:
  243-256
• NIOSHTIC Number:
  nn:20069482
• Citation:
  PPoPP '24: Proceedings of the 29th ACM SIGPLAN Annual Symposium on Principles and Practice of Parallel Programming, March 2-6, 2024, Edinburgh, United Kingdom. New York: Association for Computing Machinery (ACM), 2024 Mar; :243-256
• Federal Fiscal Year:
  2024
• Performing Organization:
  Colorado School of Mines
• Peer Reviewed:
  False
• Start Date:
  20190913
• Source Full Name:
  PPoPP '24: Proceedings of the 29th ACM SIGPLAN Annual Symposium on Principles and Practice of Parallel Programming, March 2-6, 2024, Edinburgh, United Kingdom
• Collection(s):
  National Institute for Occupational Safety and Health
• Main Document Checksum:
  urn:sha-512:13bf30f31d439b12395501c460002885f97f84519fd6fff4be32a8d17901a4abf1d536a0922dc058839f771c89c6fae8530d704315e43bd98462a9c8088cb3e0
• Download URL:
  https://stacks.cdc.gov/view/cdc/207917/cdc_207917_DS1.pdf
• File Type:
  [PDF - 904.91 KB ]