Zero-Shot Ranking Socio-Political Texts with Transformer Language Models to Reduce Close Reading Time

Kiymet Akdemir; Ali Hürriyetoğlu

Zero-Shot Ranking Socio-Political Texts with Transformer Language Models to Reduce Close Reading Time

Kiymet Akdemir, Ali Hürriyetoğlu

Research output: Contribution to conference › Paper › Scientific › peer-review

1 Citation (Scopus)

Abstract

We approach the classification problem as an entailment problem and apply zero-shot ranking to socio-political texts. Documents that are ranked at the top can be considered positively classified documents and this reduces the close reading time for the information extraction process. We use Transformer Language Models to get the entailment probabilities and investigate different types of queries. We find that DeBERTa achieves higher mean average precision scores than RoBERTa and when declarative form of the class label is used as a query, it outperforms dictionary definition of the class label. We show that one can reduce the close reading time by taking some percentage of the ranked documents that the percentage depends on how much recall they want to achieve. However, our findings also show that percentage of the documents that should be read increases as the topic gets broader.

Original language	English
Pages	124-132
Number of pages	9
Publication status	Published - 2022

Access to Document

https://aclanthology.org/2022.case-1.17.pdf

Cite this

@conference{06b048d2fa844c149ad78c252106ff57,

title = "Zero-Shot Ranking Socio-Political Texts with Transformer Language Models to Reduce Close Reading Time",

abstract = "We approach the classification problem as an entailment problem and apply zero-shot ranking to socio-political texts. Documents that are ranked at the top can be considered positively classified documents and this reduces the close reading time for the information extraction process. We use Transformer Language Models to get the entailment probabilities and investigate different types of queries. We find that DeBERTa achieves higher mean average precision scores than RoBERTa and when declarative form of the class label is used as a query, it outperforms dictionary definition of the class label. We show that one can reduce the close reading time by taking some percentage of the ranked documents that the percentage depends on how much recall they want to achieve. However, our findings also show that percentage of the documents that should be read increases as the topic gets broader.",

author = "Kiymet Akdemir and Ali H{\"u}rriyetoğlu",

note = "Publisher Copyright: {\textcopyright} 2022 Association for Computational Linguistics.",

year = "2022",

language = "English",

pages = "124--132",

}

TY - CONF

T1 - Zero-Shot Ranking Socio-Political Texts with Transformer Language Models to Reduce Close Reading Time

AU - Akdemir, Kiymet

AU - Hürriyetoğlu, Ali

PY - 2022

Y1 - 2022

N2 - We approach the classification problem as an entailment problem and apply zero-shot ranking to socio-political texts. Documents that are ranked at the top can be considered positively classified documents and this reduces the close reading time for the information extraction process. We use Transformer Language Models to get the entailment probabilities and investigate different types of queries. We find that DeBERTa achieves higher mean average precision scores than RoBERTa and when declarative form of the class label is used as a query, it outperforms dictionary definition of the class label. We show that one can reduce the close reading time by taking some percentage of the ranked documents that the percentage depends on how much recall they want to achieve. However, our findings also show that percentage of the documents that should be read increases as the topic gets broader.

AB - We approach the classification problem as an entailment problem and apply zero-shot ranking to socio-political texts. Documents that are ranked at the top can be considered positively classified documents and this reduces the close reading time for the information extraction process. We use Transformer Language Models to get the entailment probabilities and investigate different types of queries. We find that DeBERTa achieves higher mean average precision scores than RoBERTa and when declarative form of the class label is used as a query, it outperforms dictionary definition of the class label. We show that one can reduce the close reading time by taking some percentage of the ranked documents that the percentage depends on how much recall they want to achieve. However, our findings also show that percentage of the documents that should be read increases as the topic gets broader.

UR - http://www.scopus.com/inward/record.url?scp=85143435297&partnerID=8YFLogxK

M3 - Paper

AN - SCOPUS:85143435297

SP - 124

EP - 132

ER -

Zero-Shot Ranking Socio-Political Texts with Transformer Language Models to Reduce Close Reading Time

Abstract

Access to Document

Other files and links

Fingerprint

Cite this