AMOS ISCA 2022

Background and Motivation

Existing compilers use handtuned computation implementations and optimization templates, resulting in sub-optimal performance and heavy development costs 作者提出一个自动编译框架用于spatial hardware accelerators 手工设计模板的问题： (1) these templates rely on explicit intrinsic programming to use the dedicated computation units provided by spatial accelerators (2) the templates use fragile pattern matching rules to map operators to accelerators 面临的挑战： (1) specifies the behavior of spatial hardware instructions (2) defines the mapping problem from software to hardware 提出的创新点： (1) 提出硬件抽象层，将不同加速器的底层指令抽象为统一的高层表示，包括计算抽象和内存抽象，使得编译器能够自动分析和优化不同加速器的指令 (2) 开发自动映射生成和验证算法，系统地探索软件到硬件的映射空间，确保生成的映射是正确的 (3) 结合性能模型和调优技术，可以高效地在大规模搜索空间中找到高性能的映射方案

AMOS Overview

Hardware Abstraction

Compute Abstraction

操作数、算法运算、数据访问索引、索引访问约束

Memory Abstraction

Software-Hardware Mapping

计算映射：将软件迭代映射到硬件指令迭代内存映射：为每个操作数分配内存地址和步长

Mapping Generation, Validation, and Exploration

Software-Hardware Mapping Generation

这一块还没有太看懂，等过一两天，现在脑子转不过来了

Software-Hardware Mapping Validation

软件访问矩阵(X): 描述软件中每个迭代变量(n,k,p,q,c,r,s)和每个张量(image,weight,out)之间的关系指令访问矩阵(Z): 描述Tensor Core指令中每个迭代变量(i1,i2,r1)和每个操作数(src1,src2,dst)之间的访问关系迭代匹配矩阵(Y): 描述如软件迭代变量如何映射到硬件指令迭代变量

Exploration of Mapping and Schedule

优化目标：最小化总体执行时间、平衡计算和内存访问、充分利用硬件资源，在巨大的搜索空间中找到好的解决方案

Evaluation

Thinking

Reference

AMOS: Enabling Automatic Mapping for Tensor Computations On Spatial Accelerators with Hardware Abstraction

FEATURED TAGS

Tensor Compiler Compiler Optimization Code Generation Heterogeneous Systems Operator Fusion Deep Neural Network Recursive Tensor Execution Deep Learning Compiler Classical Machine Learning Compiler Optimizations Bayesian Optimization Autotuning Spatial Accelerators Tensor Computations Code Reproduction Neural Processing Units Polyhedral Model Auto-tuning Machine Learning Compiler Neural Network Program Transformations Tensor Programs Deep learning Tensor Program Optimizer Search Algorithm Compiler Infrastructure Scalalbe and Modular Compiler Systems Tensor Computation GPU Task Scheduling GPU Streams Tensor Expression Language Automated Program optimization Framework AI compiler memory hierarchy data locality tiling fusion polyhedral model scheduling domain-specific architectures memory intensive TVM Sparse Tensor Algebra Sparse Iteration Spaces Optimizing Transformations Tensor Operations Machine Learning Model Scoring AI Compiler Memory-Intensive Computation Fusion Neural Networks Dataflow Domain specific Language Programmable Domain-specific Acclerators Mapping Space Search Gradient-based Search Deep Learning Systems Systems for Machine Learning Programming Models Compilation Design Space Exploration Tile Size Optimization Performance Modeling High-Performance Tensor Program Tensor Language Model Tensor Expression GPU Loop Transformations Vectorization and Parallelization Hierarchical Classifier TVM API Optimizing Compilers Halide Pytorch Optimizing Tensor Programs Gradient Descent debug Automatic Tensor Program Tuning Operators Fusion Tensor Program Cost Model Weekly Schedule Spatio-temporal Schedule tensor compilers auto-tuning tensor program optimization compute schedules Tensor Compilers Data Processing Pipeline Mobile Devices Layout Transformations Transformer Design space exploration GPU kernel optimization Compilers Group Tuning Technique Tensor Processing Unit Hardware-software Codeisgn Data Analysis Adaptive Systems Program Auto-tuning python api Code Optimization Distributed Systems High Performance Computing code generation compiler optimization tensor computation Instructions Integration Code rewriting Tensor Computing DSL CodeReproduction Deep Learning Compiler Loop Program Analysis Nested Data Parallelism Compute-Intensive Automatic Exploration Loop Fusion Data Movement C++ Machine Learning System Decision Forest Optimizfing Compiler Decision Tree Ensemble Decision Tree Inference Parallelization Optimizing Compiler decision trees random forest machine learning parallel processing multithreading Tree Structure Performance Model Code generation Compiler optimization Tensor computation accelerator neural networks optimizing compilers autotuning performance models deep neural networks compilers auto-scheduling tensor programs Tile size optimization Performance modeling Program Functionalization affine transformations loop optimization Performance Optimization Subgraph Similarity deep learning compiler Intra- and Inter-Operator Parallelisms ILP tile-size operator fusion cost model graph partition zero-shot tuning tensor program kernel orchestration machine learning compiler Loop tiling Locality Polyhedral compilation Optimizing Transformation Sparse Tensors Asymptotic Analysis Automatic Scheduling Optimization Operation Fusion data reuse deep reuse Tensorize docker graph substitution compiler Just-in-time compiler graph Tensor program construction tensor compilation graph traversal Markov analysis Deep Learning Compilation Tensor Program Auto-Tuning Decision Tree Search-based code generation Domain specific lanuages Parallel architectures Dynamic neural network mobile device spatial accelerate software mapping reinforcement learning Computation Graph Graph Scheduling and Transformation Graph-level Optimization Operator-level Optimization Partitioning Algorithms IR Design Parallel programming languages Software performance Digitial signal processing Retargetable compilers Equational logic and rewriting Tensor-level Memory Management Code Generation and Optimizations Scheduling Sparse Tensor Auto-Scheduling Tensor Coarse-Grained Reconfigurable Architecture Graph Neural Network Reinforcement Learning Auto-Tuning Domain-Specific Accelerator Deep learning compiler Long context Memory optimization code analysis

AMOS Enabling Automatic Mapping for Tensor Computations On Spatial Accelerators with Hardware Abstraction