Perceiving numerosity, i.e. the set size of a group of items, is an evolutionarily preserved ability found in humans and animals. A useful method to infer the neural underpinnings of a given perceptual property is sensory adaptation. Like other primary perceptual attributes, numerosity is susceptible to adaptation. Recently, we have shown numerosity-selective neural populations with a topographic organization in the human brain. Here, we investigated whether numerosity adaptation can affect the numerosity selectivity of these populations using ultra-high field (7 Tesla) functional magnetic resonance imaging (fMRI). Participants viewed stimuli of changing numerosity (1 to 7 dots), which allowed the mapping of numerosity selectivity. We interleaved a low or high numerosity adapter stimulus with these mapping stimuli, repeatedly presenting 1 or 20 dots respectively to adapt the numerosity-selective neural populations. We analyzed the responses using custom-build population receptive field neural models of numerosity encoding and compared estimated numerosity preferences between adaptation conditions. We replicated our previous studies where we found several topographic maps of numerosity-selective responses. We found that overall, numerosity adaptation altered the preferred numerosities within the numerosity maps, resulting in predominantly attractive biases towards the numerosity of the adapter. The differential biases could be explained by the difference between the unadapted preferred numerosity and the numerosity of the adapter, with attractive biases being observed with higher difference. The results could link perceptual numerosity adaptation effects to changes in neural numerosity selectivity.