While a ‘return of the repressed’ is commonly linked to neurotic symptoms, the title of this chapter reflects the argument that there can be a return of the repressed in and through discourse. The discussion is based on a reading of reactions to the performance and subsequent imprisonment of Russian feminist punk collective Pussy Riot in 2012. The strongly negative reactions to the group in Russia featured not only discursive reenactments of gendered fantasmatic violence; the language also frequently called to mind the linguistic repertoire of Stalinism. This – linguistic and other – violence was symptomatic of a collective unease with the ambiguity inherent in the multiple meanings of the group’s name and the nature of their performances, as they evoked a return to instability and chaos. It may seem self-evident that societal antagonisms are revealed by such ‘spontaneous’ linguistic outbursts, but it is worth paying attention to the language employed in order to understand which elements of the past are conjured by it, and why. When language is uprooted and retrieved from a previous historical context, it can retain a violent charge that comes back to haunt the speaking subject and its discourse. This chapter assumes a psychosocial perspective in order to reflect on the relationship between language and history, with the aim of finding a means of speaking of the social so as to understand the relationship between violent language and ‘unfinished history.’