Attention Augmented Learning Machines: Theory and Applications

This book includes eight chapters introducing some interesting works on the attention mechanism. Chapter 1 is a review of the attention mechanism used in the deep learning area, while Chapter 2 and Chapter 3 present two models that integrate the attention mechanism into gated recurrent units (GRUs) and long short-term memory (LSTM), respectively, making them pay attention to important information in the sequences. Chapter 4 designs a multi-attention fusion mechanism and uses it for industrial surface defect detection. Chapter 5 enhances Transformer for object detection applications. Moreover, Chapter 6 proposes a dual-path architecture called dual-path mutual attention network (DPMAN) for medical image classification, and Chapter 7 proposes a novel graph model called attention-gated graph neural network (AGGNN) for text classification. In addition, Chapter 8 combines the generative adversarial networks (GANs), LSTM, and an attention mechanism to build a generative model for stock price prediction.