TL;DR: More efficient text encoders emerge from an attempt (failed?) to do "GAN's for Text".
GAN Generative Adversarial Data Programming