Going to school and getting an education can both be important and tenuous for young people who, because they are refugees, are experiencing a precarious and fragile life. It is often considered an effective catalyst for integration, for future success in life, and as a possible haven. At the same time, schooling and education aimed at newly arrived refugee children and young people are rife with challenges. In this chapter, we use ethnographic data to investigate how young unaccompanied people talk about school and school life in relation to racism, and administrative violence. How the young people act, react to, and reflect on their education and schools are affected by how they are perceived and treated by society at large. Being labelled, questioned, and rejected can be considered elements of administrative violence. Another related form of violence affecting these young people is racist violence, both physical violence and violence expressed through institutions, such as schools. We argue that the school system must adapt to these young people’s wishes, experiences, and challenges, without homogenizing and reproducing inequalities and violence aimed at them. Further, racism must be addressed and fought methodically, on the policy level and locally within the schools.