Large-scaled PLMs require a large corpus and unsupervised learning and only require fine-tuning for downstream tasks to achieve state-of-the-art performance. Although considered a black box, it can be considered a knowledge base, which makes it possible for generating text with specific constraints without requiring external domain knowledge.