Quality inspection is an important process in green pepper production and trading. Traditionally, quality control is performed manually, based on features like size, shape wellness, texture, and color. In this study, we aim to develop an image-based algorithm to be used to grade green pepper quality with respect to flexure. We propose a novel method to segment the stem of the green pepper from its flesh. When the misleading effect of the stem is removed, the classification rate is increased to 88.8 percentage. K-Nearest Neighbor and multi-class Support Vector Machine methods are used for classifying features extracted from pepper image samples into their respective quality levels.