Diagnosing fashion outfit compatibility with deep learning techniques

Balim C., ÖZKAN K.

EXPERT SYSTEMS WITH APPLICATIONS, cilt.215, 2023 (SCI-Expanded) identifier identifier

  • Yayın Türü: Makale / Tam Makale
  • Cilt numarası: 215
  • Basım Tarihi: 2023
  • Doi Numarası: 10.1016/j.eswa.2022.119305
  • Dergi Adı: EXPERT SYSTEMS WITH APPLICATIONS
  • Derginin Tarandığı İndeksler: Science Citation Index Expanded (SCI-EXPANDED)
  • Anahtar Kelimeler: Outfit compatibility, Fashion recommendation, Image captioning, Image segmentation, Encoder -decoder networks, Deep learning
  • Eskişehir Osmangazi Üniversitesi Adresli: Evet

Özet

Fashion image understanding is a popular research field with many different machine learning applications. There have been many studies regarding outfit prediction and outfit composition in the field of fashion. However, there are few works that explain the prediction. This paper addresses a method of diagnosing outfit compatibility through clothing images. The proposed system not only predicts compatibility, but also diagnoses incompatible clothing items in outfits. First, a new dataset named ModAI, which has clothing images and compatibility comments from different users was created. After this, a common compatibility comment was created according to user comments for each clothing image. Lastly, image captioning techniques were used to generate compat-ibility suggestion texts from clothing images. Different segmentation techniques were also used to improve captioning capabilities. The model achieves a 0.62 BLEU-4 score. Experiments show that image captioning techniques can also be used to diagnose outfit compatibility.