Completely positive factorization (CPF) is a critical task with applications in many fields. This paper proposes a novel method for the CPF. Based on the idea of exterior point iteration, an optimization model is given, which aims to orthogonally transform a symmetric lower rank factor to be nonnegative. The optimization problem can be solved via a modified nonlinear conjugate gradient method iteratively. The iteration points locate on the exterior of the orthonormal manifold and the closed set whose transformed matrices are nonnegative before convergence generally. Convergence analysis is given for the local or global optimum of the objective function, together with the iteration algorithm. Some potential issues that may affect the CPF are explored numerically. The exterior point method performs much better than other algorithms, not only in the efficiency of computational cost or accuracy, but also in the ability to address the CPF in some hard cases.