A4 Artikkeli konferenssijulkaisussa
Zero-shot Semantic Segmentation using Relation Network (2021)

Zhang, Y., & Khriyenko, O. (2021). Zero-shot Semantic Segmentation using Relation Network. In S. Balandin, V. Deart, & T. Tyutina (Eds.), FRUCT '28 : Proceedings of the 28th Conference of Open Innovations Association FRUCT (pp. 516-527). FRUCT Oy. Proceedings of Conference of Open Innovations Association FRUCT. https://doi.org/10.23919/FRUCT50888.2021.9347619

JYU-tekijät tai -toimittajat

Julkaisun tiedot

Julkaisun kaikki tekijät tai toimittajat: Zhang, Yindong; Khriyenko, Oleksiy

Emojulkaisu: FRUCT '28 : Proceedings of the 28th Conference of Open Innovations Association FRUCT

Emojulkaisun toimittajat: Balandin, Sergey; Deart, Vladimir; Tyutina, Tatiana

Konferenssin paikka ja aika: Moscow, Russia, 25.-29.1.2021

eISBN: 978-952-69244-4-1

Lehti tai sarja: Proceedings of Conference of Open Innovations Association FRUCT

ISSN: 2305-7254

eISSN: 2343-0737

Julkaisuvuosi: 2021

Artikkelin sivunumerot: 516-527

Kustantaja: FRUCT Oy

Julkaisumaa: Suomi

Julkaisun kieli: englanti

DOI: https://doi.org/10.23919/FRUCT50888.2021.9347619

Pysyvä verkko-osoite: https://fruct.org/publications/fruct28/files/Zha.pdf

Julkaisun avoin saatavuus: Avoimesti saatavilla

Julkaisukanavan avoin saatavuus: Kokonaan avoin julkaisukanava

Julkaisu on rinnakkaistallennettu (JYX): https://jyx.jyu.fi/handle/123456789/74258

Tiivistelmä

Zero-shot learning (ZSL) is widely studied in recent years to solve the problem of lacking annotations. Currently, most studies on ZSL are for image classification and object detection. But, zero-shot semantic segmentation, pixel level classification, is still at its early stage. Therefore, this work proposes to extend a zero-shot image classification model, Relation Network (RN), to semantic segmentation tasks. We modified the structure of RN based on other state-of-the-arts semantic segmentation models (i.e. U-Net and DeepLab) and utilizes word embeddings from Caltech-UCSD Birds 200-2011 attributes and natural language processing models (i.e. word2vec and fastText). Because meta-learning is limited to binary tasks, this work proposes to join multiple binary semantic segmentation pipelines for multi-class semantic segmentation. It is proved by experiments that RN could improve accuracy of U-Net with the help of semantic side information on binary semantic segmentation and it could also be applied on multi-class semantic segmentation with simpler structure than the baseline model, SPNet, but higher accuracy under ZSL setting. However, the capability of RN under generalized zero-shot learning (GZSL) setting still needs improvement. We also studied on how different word embeddings, network structures and data affect RN and what could be done to improve its results.

YSO-asiasanat: konenäkö; hahmontunnistus (tietotekniikka); koneoppiminen

Vapaat asiasanat: deep learning; image segmentation; zero-shot semantic segmentation

Liittyvät organisaatiot

OKM-raportointi: Kyllä

Raportointivuosi: 2021

JUFO-taso: 1