Invasive plant species are among the major threats to freshwater biodiversity. Few experimental studies have investigated whether native plant diversity can provide biotic resistance to invaders in freshwater ecosystems. At small spatial scales, invasion resistance may increase with plant species richness due to a better use of available resources, leaving less available for a potential invader (Complementarity effect) and/or the greater probability to have a highly competitive (or productive) native species in the community (Selection effect). In submerged aquatic plant communities, we tested the following hypotheses: (1) invader establishment success is greatest in the absence of a native plant community; (2) lower in plant communities with greater native species richness, due to complementary and/or selection effects; and (3) invader establishment success would be lowest in rooted plant communities, based on the limiting similarity theory as the invader is a rooted submerged species. In a greenhouse experiment, we established mesocosms planted with 0 (bare sediment), 1, 2, and 4 submerged plant species native to NW Europe and subjected these to the South African invader Lagarosiphon major (Ridl.) Moss. We used two rooted (Myriophyllum spicatum L., Potamogeton perfoliatus L.) and two non-rooted native species (Ceratophyllum demersum L., Utricularia vulgaris L.) representing two distinct functional groups considering their nutrient acquisition strategy which follows from their growth form, with, respectively, the sediment and water column as their main nutrient source. We found that the presence of native vegetation overall decreased the establishment success of an alien aquatic plant species. The strength of this observed biotic resistance increased with increasing species richness of the native community. Mainly due to a selection effect, the native biomass of mixed communities overyielded, and this further lowered the establishment success of the invader in our experiment. The strongest biotic resistance was caused by the two native plant species that were of the same functional group, i.e., functionally most similar to the invader. These results support the prediction of Elton’s biotic resistance hypothesis in aquatic ecosystems and indicate that both species richness and functional group identity can play an important role in decreasing establishment success of alien plant species.