We used fMRI to study the effect of hiding the target of a grasping action on the cerebral activity of an observer whose task was to anticipate the size of the object being grasped. Activity in the putative mirror neuron system (pMNS) was higher when the target was concealed from the view of the observer and anticipating the size of the object being grasped requested paying attention to the hand kinematics. In contrast, activity in ventral visual areas outside the pMNS increased when the target was fully visible, and the performance improved in this condition. A repetition suppression analysis demonstrated that in full view, the size of the object being grasped by the actor was encoded in the ventral visual stream. Dynamic causal modeling showed that monitoring a grasping action increased the coupling between the parietal and ventral premotor nodes of the pMNS. The modulation of the functional connectivity between these nodes was correlated with the subject's capability to detect the size of hidden objects. In full view, synaptic activity increased within the ventral visual stream, and the connectivity with the pMNS was diminished. The re-enactment of observed actions in the pMNS is crucial when interpreting others' actions requires paying attention to the body kinematics. However, when the context permits, visual-spatial information processing may complement pMNS computations for improved action anticipation accuracy.