Different architectures of neural network language models are described in this post, including feedforward neural network language models, recurrent neural network language models and long short term memory ones. Variants of these nerual network language models will also be discussed, like using different activation functions, with or without direct connections between input and output layers and adopting bias terms or not. General issues about training and testing neural network language models are also included in this post.