Motion estimation has traditionally been approached either from a pure discrete point of view, using multi-view tensors, or from a pure continuous point of view, using optical flow. This paper builds upon a novel framework of hybrid matching constraints for motion estimation, combining the advantages of both discrete and continuous methods. We will derive both bifocal and trifocal hybrid constraints and use them together with a structure estimate based on filtering techniques. A feedback from the structure estimate will be used to further refine the motion estimate. This gives a complete iterative structure and motion estimation scheme. Its performance will be demonstrated in simulated experiments.