Energy-Aware Task Scheduling in Heterogeneous GPU/TPU–FPGA Embedded Platforms

Abstract

Dynamic performance needed in the edge and embedded systems has boosted demand of real time energy-efficient computing that requires integration of heterogeneous hardware accelerators, such as Graphics Processing Units (GPUs), Tensor Processing Units (TPUs), and Field-Programmable Gate Arrays (FPGAs). Such varied processing units each bring both complementary abilities to the table, with GPUs offering vast parallelism, TPUs specializing in neural-inference, and FPGAs offering low-power reconfigurable computing. Nevertheless, effective work of such heterogeneous platforms is rather challenging, especially when it comes to scheduling of tasks, owing to the existence of the energy-performance specifications and architectural differences between these accelerators. The proposed work is a new architecture of energy-aware task scheduling aimed at dynamically distributing work on GPU, TPU, and FPGA with multi-unit to exploit real-time profiling, workload classification, and cost-optimal scheduling policy. The scheduler uses light-weight machine learning models to predict the execution unit of the most fitting task on the basis of the computational complexity, memory requirement, and latency requirements. Broad assessment is carried out on a sample embedded system that includes an NVIDIA Jetson Xavier GPU, a Google Coral Edge TPU and an Intel Arria 10 FPGA. The system performance is measured using real-world tasks, i.e. image classification, signal transformation, and deep learning inference. According to the results, the proposed scheduler yields up to 35 percent energy savings and 28 percent gains in execution latency as compared with baseline scheme such as static round-robin and performance only scheduling. Moreover, the framework proves to be resilient at changing intensive workloads with the scheduling overhead of less than 2.5% and this nature of the framework qualifies it to be compatible with real-time tasks. Its contribution to the field is a scalable and intelligent way of scheduling that optimizes energy consumption within an acceptable impact on performance, so this paper is of particular interest to embedded AI computing of the future as well as IoT edge systems and low power-intensive cyber-physical systems. Future work Future expentions will find application in reinforcement learning based adaptive control and more integration with other processing aspects like NPUs and DSPs to increase the scope of the framework.