Free-ranging large grazers, such as cattle and horses, are increasingly reintroduced to former agricultural areas in Western Europe in order to restore natural and diverse habitats. In this review we outline mechanisms by which large grazers induce and maintain structural diversity in the vegetation (mosaics of grasslands, shrub thickets and trees). This variation in vegetation structure is considered to be important for the conservation of biodiversity of various plant and animal groups. The process of spatial association with unpalatable plants (associational resistance) enables palatable plants to establish in grasslands maintained by large grazers. In this way, short unattractive (thorny, low quality or toxic) species facilitate taller unattractive shrubs, which facilitate palatable trees, which in turn outshade the species that facilitated their recruitment. Established trees can, therefore, not regenerate under their own canopy, leading to cyclic patch dynamics. Since this cyclic dynamic occurs on a local scale, this contributes to shifting mosaics. The mechanisms involved in creating and maintaining the resulting shifting mosaics are described for temperate floodplain and heathland ecosystems, including the effects on nutrient transport within grazed landscapes. How grazing leads to shifting mosaics is described in terms of plant functional types, allowing potential generalisation to other ecosystems. The resulting interaction web of grasses, unpalatable forbs and shrubs, palatable light-demanding trees and shade-tolerant trees is discussed, and was found to contain various interesting direct and indirect effects. The key process contributing to spatial diversity in vegetation structure is the alternation of positive (facilitation) interactions between plant species at one life cycle stage, and competitive displacement at another stage. Crazing thus causes directional successional sequences to change to shifting mosaics. The implications of this theory for nature conservation are discussed, including the relevant management problems, possible choices and practical solutions. We conclude that the theoretical framework outlined in this review provides helpful insights when coping with nature conservation issues in temperate woodland habitats.