A major challenge in transfer learning is determining whether the initial and current problems are similar enough. There is no strict determinant for what makes a source network sufficient or similar enough to the target application. If the algorithm fails to find a connection between the problems, the model can perform even worse than if it had never been trained in the first place.