Designing quantum circuits for ground state preparation is a fundamental task in quantum information science. However, standard Variational Quantum Algorithms (VQAs) are often constrained by limited ansatz expressivity and difficult optimization landscapes. To address these issues, we adopt a geometric perspective, formulating the problem as the minimization of an energy cost function directly over the unitary group. We establish a retraction-based Riemannian optimization framework for this setting, ensuring that all algorithmic procedures are implementable on quantum hardware. Within this framework, we unify existing randomized gradient approaches under a Riemannian Random Subspace Gradient Projection (RRSGP) method. While recent geometric approaches have predominantly focused on such first-order gradient descent techniques, efficient second-order methods remain unexplored. To bridge this gap, we derive explicit expressions for the Riemannian Hessian and show that it can be estimated directly on quantum hardware via parameter-shift rules. Building on this, we propose the Riemannian Random Subspace Newton (RRSN) method, a scalable second-order algorithm that constructs a Newton system from measurement data. Numerical simulations indicate that RRSN achieves quadratic convergence, yielding high-precision ground states in significantly fewer iterations compared to both existing first-order approaches and standard VQA baselines. Ultimately, this work provides a systematic foundation for applying a broader class of efficient Riemannian algorithms to quantum circuit design.