Current AI technologies and data representations often reflect the popular or majority vote. This is an inherent artefact of the frequency bias of many statistical analysis methods that are used to create for example knowledge graphs, resulting in simplified representations of the world in which diverse perspectives are underrepresented. With the use of AI-infused tools ever increasing, as well as the diverse audiences using these tools, this bias needs to be addressed in both the algorithms analysing data, as well as in the resulting representations. In this problems to solve before you die submission, we explain the implications of the lack of polyvocality and contextual knowledge in the semantic web. We identify three challenges for the Semantic Web community on dealing with various voices and perspectives as well as our vision for addressing it.