Deep neural networks for positioning can improve accuracy by adapting to inhomogeneous environments. However, they are still susceptible to noisy data, often resulting in invalid positions. A related task, map matching, can be used for reducing geographical invalid positions by aligning observations to a model of the real world. In this paper, we propose an approach for positioning, enhanced with map matching, within a single deep neural network model. We introduce a novel way of reducing the number of invalid position estimates by adding map information to the input of the model and using a map-based loss function. Evaluating on real-world Received Signal Strength Indicator data from an asset tracking application, we show that our approach gives both increased position accuracy and a decrease of one order of magnitude in the number of invalid positions.