The authors study singular perturbations of optimal stochastic control problems and differential games arising in the dimension reduction of system with multiple time scales. They analyze the uniform convergence of the value functions via the associated Hamilton-Jacobi-Bellman-Isaacs equations, in the framework of viscosity solutions. The crucial properties of ergodicity and stabilization to a constant that the Hamiltonian must possess are formulated as differential games with ergodic cost criteria. They are studied under various different assumptions and with PDE as well as control-theoretic methods. The authors also construct an explicit example where the convergence is not uniform. Finally they give some applications to the periodic homogenization of Hamilton-Jacobi equations with non-coercive Hamiltonian and of some degenerate parabolic PDEs.