Felix ASPLOS 2024

Felix Optimizing Tensor Programs with Gradient Descent

Posted by Treaseven on December 27, 2024

Motivation

现有工具的搜索过程遭受低效率,由于搜索空间的大小需要数日时间来发现张量程序

面临的挑战: (1) the search space of schedule is discrete, with many of the tunable parameters constrained in a subset of integers (2) The objective function - the performance of the program as a function of the schedule - is highly complex, often discontinuous, and non-differentiable (3) differentiation of the feature vector with regard to the variables in the program schedule remains tremendously challenging as program generation exercises multiple components of the compiler

Felix Design

Symbolic schedule and symbolic program generation

Feature Formula Extraction and Rewriting

Optimizing Schedules with Gradient Descent

Evaluation

Reference

Felix: Optimizing Tensor Programs with Gradient Descent