EfficientNet is evolved from the MobileNet V2 building blocks, with the key insight that scaling up the width, depth or resolution can improve a network’s performance, and a balanced scaling of all three is the key to maximizing improvements. It achieves the state of the art performance with much less parameters and FLOPS than other architectures.

The transformer architecture from Attention is all you need is the most important technology for natural language processing in recent years. This post is basically a TL;DR + ELI5 version explaining this ingenious model design.