The widespread use of text over online social networks makes it quite suitable for steganography. Conventional text steganography usually transmits the secret data by either slightly modifying the given text or hiding the secret data through synonym replacement. The rapid development of deep neural networks (DNNs) has led automatically generating the steganographic text to become an important and promising topic. This has motivated us to propose a novel generative text steganographic method based on long short-term memory (LSTM) network in this paper. We apply attention mechanism with keywords to the LSTM network to generate the steganographic text. Experiments show that, compared to the related work, the steganographic text generated by the proposed method is of higher semantic quality and more capable of resisting against steganalysis, which has shown the superiority.