Numerous experimental data show that cortical networks of neurons are not silent in the absence of external inputs, but rather maintain a low spontaneous firing activity. This aspect of cortical networks is likely to be important for their computational function, but is hard to reproduce in models of cortical circuits of neurons, since the low activity regime is inherently unstable. We show in this article through theoretical analysis and extensive computer simulations that short-term synaptic plasticity endows models of cortical circuits with a remarkable stability in the low-activity regime. This short-term plasticity works as a homeostatic mechanism that stabilizes the over all activity level in spite of drastic changes in external inputs and internal circuit properties, while preserving reliable transient responses to signals. The contribution of synaptic dynamics to this stability can be predicted on the basis of general principles from control theory.