Transformer Self-Attention

What are the key trade-offs of Transformer Self-Attention? When would you choose it vs alternatives?

Answers use simple, clear English.