Purpose Expert evaluation is the backbone of the multi-criteria decision-making (MCDM) techniques. The experts make pairwise comparisons between criteria or alternatives in this evaluation. The mainstream research focus on the ambiguity in this process and use fuzzy logic. On the other hand, cognitive biases are the other but scarcely studied challenges to make accurate decisions. The purpose of this paper is to propose pilot filters - as a debiasing strategy - embedded in the MCDM techniques to reduce the effects of framing effect, loss aversion and status quo-type cognitive biases. The applicability of the proposed methodology is shown with analytic hierarchy process-based Technique for Order-Preference by Similarity to Ideal Solution method through a sustainable supplier selection problem. Design/methodology/approach The first filter's aim is to reduce framing bias with restructuring the questions. To manipulate the weights of criteria according to the degree of expected status quo and loss aversion biases is the second filter's aim. The second filter is implemented to a sustainable supplier selection problem. Findings The comparison of the results of biased and debiased ranking indicates that the best and worst suppliers did not change, but the ranking of suppliers changed. As a result, it is shown that, to obtain more accurate results, employing debiasing strategies is beneficial. Originality/value To the best of the author's knowledge, this approach is a novel way to cope with the cognitive biases. Applying this methodology easily to other MCDM techniques will help the decision makers to take more accurate decisions.