This paper introduces a novel deep neural network tracker for robust object tracking. To this end, we employ a ranking loss which provides a fine-tuning of the target object position and returns more precise bounding boxes framing the target object. This is achieved by systematically learning to give higher scores to the candidate regions better framing the target object than the regions that frame the object with less accuracy. As a result, the risk of tracking error accumulation and drifts are largely mitigated, and the object is tracked more successfully: When the proposed network is used with a simple yet effective model update rule, our proposed tracker achieves the state-of-the-art results on all tested challenging tracking datasets. Especially, our results on the OTB (Object Tracking Benchmark) datasets are very promising. The proposed tracker outperforms both deep neural network and correlation filter based trackers, MDNet and ECO, by about 2%, which is a significant improvement over the previous state-of-the-art.