So is this the solution to optimizing all non-convex problems?

• Not really. In theory, the effect that a complex non-convex feature space has on the application of the continuation method would result in a very large number of objective functions which, as a result, would make the problem computationally intractable. (NP-hard optimization problems remain NP-hard even in this case).
• Another factor to consider is that non-convex problems might remain non-convex whatever the extent of blurring in some cases. E.g. $J(\Theta)=-\Theta^{T}\Theta$.
• The pitfall of blurring from the intuitive standpoint works here as well i.e. blurring out space could lead to a false convex and in turn force the solution path to inferior local minima.

The initial problem with large feature spaces was believed to be that of the local minima and issues like saddle points, etc that surround them. But, high-dimensional spaces behave very differently. The bigger problem that surrounds them is that of sparsity.