During the last few decades, the consumption of low-calorie sweeteners, as a substitute for caloric sweeteners, has sharply increased. Although research shows that caloric versus low-calorie sweeteners can have differential effects on the brain, it is unknown which neuronal populations are responsible for detecting the difference between the two types of sweeteners. Using in vivo two-photon calcium imaging, we investigated how drinking sucrose or sucralose (a low-calorie sweetener) affects the activity of glutamatergic neurons in the lateral hypothalamus. Furthermore, we explored the consequences of consuming a free-choice high fat diet on the calorie detection abilities of these glutamatergic neurons. We found that glutamatergic neurons indeed can discriminate sucrose from water and sucralose, and that consumption of a free-choice high fat diet shifts the glutamatergic neuronal response from sucrose-specific to sucralose-specific, thereby disrupting calorie detection. These results highlight the disruptive effects of a diet high in saturated fat on calorie detection in the lateral hypothalamus.