Detecting harassment in real-time as conversations develop

Wessel Stoop; Florian Kunneman; A. van den Bosch; Benjamin Miller

doi:10.18653/v1/W19-3503

Detecting harassment in real-time as conversations develop

Wessel Stoop, Florian Kunneman, A. van den Bosch, Benjamin Miller

Meertens Institute

Research output: Chapter in book/volume › Contribution to conference proceedings › Scientific › peer-review

Abstract

We developed a machine-learning-based method to detect video game players that harass teammates or opponents in chat earlier in the conversation. This real-time technology would allow gaming companies to intervene during games, such as issue warnings or muting or banning a player. In a proof-of-concept experiment on League of Legends data we compute and visualize evaluation metrics for a machine learning classifier as conversations unfold, and observe that the optimal precision and recall of detecting toxic players at each moment in the conversation depends on the confidence threshold of the classifier: the threshold should start low, and increase as the conversation unfolds. How fast this sliding threshold should increase depends on the training set size.

Original language	English
Title of host publication	Proceedings of the Third Workshop on Abusive Language Online
Place of Publication	Florence, Italy
Publisher	Association for Computational Linguistics (ACL)
Pages	19-24
DOIs	https://doi.org/10.18653/v1/W19-3503
Publication status	Published - 01 Aug 2019

Access to Document

10.18653/v1/W19-3503Licence: CC BY

Cite this

@inbook{fcee684c016c451bb68d826fefe2b16c,

title = "Detecting harassment in real-time as conversations develop",

abstract = "We developed a machine-learning-based method to detect video game players that harass teammates or opponents in chat earlier in the conversation. This real-time technology would allow gaming companies to intervene during games, such as issue warnings or muting or banning a player. In a proof-of-concept experiment on League of Legends data we compute and visualize evaluation metrics for a machine learning classifier as conversations unfold, and observe that the optimal precision and recall of detecting toxic players at each moment in the conversation depends on the confidence threshold of the classifier: the threshold should start low, and increase as the conversation unfolds. How fast this sliding threshold should increase depends on the training set size.",

author = "Wessel Stoop and Florian Kunneman and {van den Bosch}, A. and Benjamin Miller",

year = "2019",

month = aug,

day = "1",

doi = "10.18653/v1/W19-3503",

language = "English",

pages = "19--24",

booktitle = "Proceedings of the Third Workshop on Abusive Language Online",

publisher = "Association for Computational Linguistics (ACL)",

address = "United States",

}

Detecting harassment in real-time as conversations develop. / Stoop, Wessel; Kunneman, Florian; van den Bosch, A. et al.

Proceedings of the Third Workshop on Abusive Language Online. Florence, Italy : Association for Computational Linguistics (ACL), 2019. p. 19-24.

Research output: Chapter in book/volume › Contribution to conference proceedings › Scientific › peer-review

TY - CHAP

T1 - Detecting harassment in real-time as conversations develop

AU - Stoop, Wessel

AU - Kunneman, Florian

AU - van den Bosch, A.

AU - Miller, Benjamin

PY - 2019/8/1

Y1 - 2019/8/1

N2 - We developed a machine-learning-based method to detect video game players that harass teammates or opponents in chat earlier in the conversation. This real-time technology would allow gaming companies to intervene during games, such as issue warnings or muting or banning a player. In a proof-of-concept experiment on League of Legends data we compute and visualize evaluation metrics for a machine learning classifier as conversations unfold, and observe that the optimal precision and recall of detecting toxic players at each moment in the conversation depends on the confidence threshold of the classifier: the threshold should start low, and increase as the conversation unfolds. How fast this sliding threshold should increase depends on the training set size.

AB - We developed a machine-learning-based method to detect video game players that harass teammates or opponents in chat earlier in the conversation. This real-time technology would allow gaming companies to intervene during games, such as issue warnings or muting or banning a player. In a proof-of-concept experiment on League of Legends data we compute and visualize evaluation metrics for a machine learning classifier as conversations unfold, and observe that the optimal precision and recall of detecting toxic players at each moment in the conversation depends on the confidence threshold of the classifier: the threshold should start low, and increase as the conversation unfolds. How fast this sliding threshold should increase depends on the training set size.

U2 - 10.18653/v1/W19-3503

DO - 10.18653/v1/W19-3503

M3 - Contribution to conference proceedings

SP - 19

EP - 24

BT - Proceedings of the Third Workshop on Abusive Language Online

PB - Association for Computational Linguistics (ACL)

CY - Florence, Italy

ER -

Detecting harassment in real-time as conversations develop

Abstract

Access to Document

Fingerprint

Cite this