RNA viruses are among the most prevalent pathogens and are a major burden on society. Although RNA viruses have been studied extensively, little is known about the processes that occur during the first several hours of infection because of a lack of sensitive assays. Here we develop a single-molecule imaging assay, virus infection real-time imaging (VIRIM), to study translation and replication of individual RNA viruses in live cells. VIRIM uncovered a striking heterogeneity in replication dynamics between cells and revealed extensive coordination between translation and replication of single viral RNAs. Furthermore, using VIRIM, we identify the replication step of the incoming viral RNA as a major bottleneck of successful infection and identify host genes that are responsible for inhibition of early virus replication. Single-molecule imaging of virus infection is a powerful tool to study virus replication and virus-host interactions that may be broadly applicable to RNA viruses.