Grasslands are important ecosystems and make up 40% of the terrestrial ecosystems worldwide. The Inner Mongolia steppe is the main grassland region of China, and nitrogen (N) and water availability are two important factors that limit the productivity of these grasslands. We tested how N and water addition influence the composition of the microbial community in the soil using PLFA, and soil physical and chemical properties in two semiarid grassland sites in Inner Mongolia during two consecutive years. In both sites, a split-plot design was employed with two water treatments (natural precipitation, stimulated wet year precipitation) and three N treatments (0 kg N ha−1, 25 kg N ha−1, 50 kg N ha−1). Water addition greatly increased soil fungi and decreased bacteria while N had opposite effects. Water addition resulted in a significant increase in soil pH and electric conductivity. N addition did not lead to consistent changes in soil characteristics. Multivariate analysis showed that PLFA composition varied between all treatments but was mainly influenced by water addition. This study provides insight into how climatic changes such as alternations in rainfall and N deposition shape the soil microbial communities in Inner Mongolia steppes.