AST ² ransformer: An Adaptive Sparse Attention Framework for Efficient Clinical Risk Assessment on Heterogeneous Tabular Data

Abstract

Deep learning models for clinical tabular data often struggle with high dimensionality, extreme sparsity, and the quadratic computational complexity (<italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">O</i>(<italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">N</i><sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup>)) of standard attention mechanisms. To address these bottlenecks for efficient clinical risk assessment, this study proposes the Adaptive-Sparse-TabTransformer (AST<sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup>ransformer), a framework that reframes feature interaction as an adaptive topology learning task at the schema level. The AST<sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup>ransformer incorporates a Correlation Assessment Module (CAM) and a Dynamic Sparse Pattern Generator (DSPG) to selectively capture feature dependencies by constructing a probabilistic graph. Furthermore, an Optimized Attention Computation Unit (OACU) leverages block-sparse operations to significantly reduce computational complexity by limiting interactions to a bounded budget (<italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">O</i>(<italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">k · N · d</i>)). Extensive evaluations were conducted on a private Venous Thromboembolism (VTE) dataset and two public benchmarks: the cardiovascular disease (CVD) dataset and the high-sparsity ECHO-NOTE2NUM dataset derived from MIMIC-III. The proposed model achieves state-of-the-art AUROC scores of 0.9487 and 0.8466 on the high-dimensional VTE and ECHO-NOTE2NUM datasets, respectively. On the lower-dimensional CVD benchmark, while achieving a competitive AUROC (0.7958) comparable to Gradient Boosting Decision Trees, our model demonstrates superior sensitivity (0.7644), significantly outperforming baseline methods in identifying positive risk cases. Crucially, it reduces computational costs by approximately 32% compared to dense baselines, offering a robust engineering solution for deploying complex risk models in resource-constrained clinical environments.

Ähnliche Arbeiten