Zain Taufique "Efficient run-time systems for AI inference on heterogeneous computing platforms"

This talk can be viewed via zoom. (Note: this talk will be recorded)

Title: "Efficient run-time systems for AI inference on heterogeneous computing platforms”

Abstract:

Efficient AI orchestration across heterogeneous edge platforms is crucial for satisfying workload Quality-of-Service (QoS) requirements, such as latency, accuracy, and priority, while maintaining platform energy efficiency. The edge platforms integrate heterogeneous compute clusters, including CPUs, GPUs, and Neural Processing Units, each exhibiting asymmetric energy performance characteristics. However, these baseline platforms remain resource-constrained with limited computational capacity. At the same time, modern AI workloads are compound in nature, comprising diverse models such as Deep Neural Networks, transformers, diffusion models, and large language models. This diversity manifests at runtime as varied inference requests that arrive both continuously and in response to user prompts, each with distinct latency, accuracy, and priority requirements. Consequently, executing multiple AI models simultaneously while satisfying diverse QoS constraints introduces a complex design-space exploration problem that requires intelligent run-time resource scheduling. The existing scheduling mechanisms remain largely conservative, failing to fully exploit resource heterogeneity while often violating workload and system constraints. This talk presents my current research on efficient AI orchestration on resource-constrained edge platforms and outlines future directions toward integrated edge–cloud computing paradigms.