We consider a smooth penalty algorithm to solve nonconvex optimization problem based on a family of smooth functions that approximate the usual exact penalty function. At each iteration in the algorithm we only need to find a stationary point of the smooth penalty function, so the difficulty of computing the global solution can be avoided. Under a generalized Mangasarian-Fromovitz constraint qualification condition (GMFCQ) that is weaker and more comprehensive than the traditional MFCQ, we prove that the sequence generated by this algorithm will enter the feasible solution set of the primal problem after finite times of iteration, and if the sequence of iteration points has an accumulation point, then it must be a Karush-Kuhn-Tucker (KKT) point. Furthermore, we obtain better convergence for convex optimization problem.
"On the Convergence of a Smooth Penalty Algorithm without Computing Global Solutions." J. Appl. Math. 2012 (SI03) 1 - 12, 2012. https://doi.org/10.1155/2012/620949