Supervised NRE models have a lack of large-scale high-quality training data because manually labeling data is time-consuming and human intensive. To help with this, distant supervision (DS) assumption has been used to automatically label data by aligning existing knowledge graphs with plain text. For any entity pair in knowledge graphs, sentences mentioning both entities will be labeled with their corresponding relations in knowledge graphs. Large-scale training examples can be easily constructed using this training scheme. Although DS provides a feasible approach to use more data, it is inevitable for its automatic labeling mechanism to be accompanied with the wrong labeling problem, since not all sentences mentioning two given entities express their relations in knowledge graphs exactly.