We prove a very general Kobayashi-Hitchin correspondence on arbitrary compact Hermitian manifolds, and we discuss differential geometric properties of the corresponding moduli spaces. This correspondence refers to moduli spaces of “universal holomorphic oriented pairs”. Most of the classical moduli problems in complex geometry (e. g. holomorphic bundles with reductive structure groups, holomorphic pairs, holomorphic Higgs pairs, Witten triples, arbitrary quiver moduli problems) are special cases of this universal classification problem. Our Kobayashi-Hitchin correspondence relates the complex geometric concept “polystable oriented holomorphic pair” to the existence of a reduction solving a generalized Hermitian-Einstein equation. The proof is based on the Uhlenbeck-Yau continuity method. Using ideas from Donaldson theory, we further introduce and investigate canonical Hermitian metrics on such moduli spaces. We discuss in detail remarkable classes of moduli spaces in the non-Kählerian framework: Oriented holomorphic structures, Quot-spaces, oriented holomorphic pairs and oriented vortices, non-abelian Seiberg-Witten monopoles.