In this paper we propose a new method for variational segmentation of image sequences containing nonrigid, moving objects. The method is based on the Chan-Vese model augmented with a novel frame-to-frame interaction term, which allow us to update the segmentation result from one image frame to the next using the previous segmentation result as a shape prior. The interaction term is constructed to be pose-invariant and to allow moderate deformations in shape. It can handle the appearance of occlusions which otherwise can make segmentation fail. The performance of the model is illustrated with experiments on synthetic and real image sequences.