Home About Browse Search
Svenska


Towards Machine Recognition of Facial Expressions of Pain in Horses

Andersen, Pia Haubro and Broome, Sofia and Rashid, Maheen and Lundblad, Johan and Ask, Katrina and Li, Zhenghong and Hernlund, Elin and Rhodin, Marie and Kjellstrom, Hedvig (2021). Towards Machine Recognition of Facial Expressions of Pain in Horses. Animals. 11 , 1643
[Article Review/Survey]

[img] PDF
1MB

Abstract

Simple Summary Facial activity can convey valid information about the experience of pain in a horse. However, scoring of pain in horses based on facial activity is still in its infancy and accurate scoring can only be performed by trained assessors. Pain in humans can now be recognized reliably from video footage of faces, using computer vision and machine learning. We examine the hurdles in applying these technologies to horses and suggest two general approaches to automatic horse pain recognition. The first approach involves automatically detecting objectively defined facial expression aspects that do not involve any human judgment of what the expression "means". Automated classification of pain expressions can then be done according to a rule-based system since the facial expression aspects are defined with this information in mind. The other involves training very flexible machine learning methods with raw videos of horses with known true pain status. The upside of this approach is that the system has access to all the information in the video without engineered intermediate methods that have filtered out most of the variation. However, a large challenge is that large datasets with reliable pain annotation are required. We have obtained promising results from both approaches. Automated recognition of human facial expressions of pain and emotions is to a certain degree a solved problem, using approaches based on computer vision and machine learning. However, the application of such methods to horses has proven difficult. Major barriers are the lack of sufficiently large, annotated databases for horses and difficulties in obtaining correct classifications of pain because horses are non-verbal. This review describes our work to overcome these barriers, using two different approaches. One involves the use of a manual, but relatively objective, classification system for facial activity (Facial Action Coding System), where data are analyzed for pain expressions after coding using machine learning principles. We have devised tools that can aid manual labeling by identifying the faces and facial keypoints of horses. This approach provides promising results in the automated recognition of facial action units from images. The second approach, recurrent neural network end-to-end learning, requires less extraction of features and representations from the video but instead depends on large volumes of video data with ground truth. Our preliminary results suggest clearly that dynamics are important for pain recognition and show that combinations of recurrent neural networks can classify experimental pain in a small number of horses better than human raters.

Authors/Creators:Andersen, Pia Haubro and Broome, Sofia and Rashid, Maheen and Lundblad, Johan and Ask, Katrina and Li, Zhenghong and Hernlund, Elin and Rhodin, Marie and Kjellstrom, Hedvig
Title:Towards Machine Recognition of Facial Expressions of Pain in Horses
Series Name/Journal:Animals
Year of publishing :2021
Volume:11
Article number:1643
Number of Pages:18
Publisher:MDPI
ISSN:2076-2615
Language:English
Publication Type:Article Review/Survey
Article category:Scientific peer reviewed
Version:Published version
Copyright:Creative Commons: Attribution 4.0
Full Text Status:Public
Subjects:(A) Swedish standard research categories 2011 > 4 Agricultural Sciences > 403 Veterinary Science > Medical Bioscience
(A) Swedish standard research categories 2011 > 1 Natural sciences > 102 Computer and Information Science > Computer Vision and Robotics (Autonomous Systems)
Keywords:pain, facial expressions, objective methods, horse, computer vision, machine learning, deep recurrent two-stream network, convolutional networks, facial keypoint detection, facial action units
URN:NBN:urn:nbn:se:slu:epsilon-p-112767
Permanent URL:
http://urn.kb.se/resolve?urn=urn:nbn:se:slu:epsilon-p-112767
Additional ID:
Type of IDID
DOI10.3390/ani11061643
Web of Science (WoS)000665456000001
ID Code:24849
Faculty:VH - Faculty of Veterinary Medicine and Animal Science
Department:(VH) > Dept. of Clinical Sciences
(VH) > Dept. of Anatomy, Physiology and Biochemistry
Deposited By: SLUpub Connector
Deposited On:13 Jul 2021 09:25
Metadata Last Modified:13 Jul 2021 09:31

Repository Staff Only: item control page

Downloads

Downloads per year (since September 2012)

View more statistics

Downloads
Hits