During early development, before the eyes open, synaptic refinement of sensory networks depends on activity generated by developing neurons themselves. In the mouse visual system, retinal cells spontaneously depolarize and recruit downstream neurons to bursts of activity, where the number of recruited cells determines the resolution of synaptic retinotopic refinement. Here we show that during the second post-natal week in mouse visual cortex, somatostatin (SST)-expressing interneurons control the recruitment of cells to retinally driven spontaneous activity. Suppressing SST interneurons increases cell participation and allows events to spread farther along the cortex. During the same developmental period, a second type of high-participation, retina-independent event occurs. During these events, cells receive such large excitatory charge that inhibition is overwhelmed and large parts of the cortex participate in each burst. These results reveal a role of SST interneurons in restricting retinally driven activity in the visual cortex, which may contribute to the refinement of retinotopy.