This paper presents a mathematical model for analyzing long-term infrastructure investment decisions in a deregulated electricity market, such as the case in the United States. The interdependence between different decision entities in the system is captured in a network-based stochastic multi-agent optimization model, where new entrants of investors compete among themselves and with existing generators for natural resources, transmission capacities, and demand markets. To overcome computational challenges involved in stochastic multi-agent optimization problems, we have developed a solution method by combining stochastic decomposition and variational inequalities, which converts the original problem to many smaller problems that can be solved more easily.