Drawing on interviews with young refugees, 20–30 years old, mainly from Syria and Afghanistan, we discuss the effects that temporary residence permits have on their ability to build a life in Sweden. The article includes both unaccompanied and accompanied youth that at some point had been given temporary residence permits. These permits could later be renewed or turned into permanent permits if the youth fulfilled certain tough requirements. Through rich empirical data, we show how these temporal techniques of border control keep young refugees in a state where they fear deportation, which have detrimental effects for their ability to build a life in Sweden. They are not able to plan ahead and they feel forced to work although they would have preferred to study, which puts them out of sync with other young people around them, challenges their sense of agency and increases their vulnerability. Temporary residence permits severely limit the life opportunities of young refugees in Sweden, and thus hamper their ability to achieve the “migrant integration” that is expected of them.