Breakdown and Repair of Human-machine Communication: Video Analysis of Family Members’ Joint Media Engagement

LUO Longxiang; WANG Bing; WANG Xiuli

PDF(1528 KB)

Chinese Journal of Journalism & Communication ›› 2024, Vol. 46 ›› Issue (8) : 27-49.

Specific Topic / Human-Machine Communication Research

Breakdown and Repair of Human-machine Communication: Video Analysis of Family Members’ Joint Media Engagement

Author information +

History +

Abstract

This study advances research on human-machine communication (HMC) by exploring communication breakdown and its repair in the process of family members’ joint media engagement with smart speakers. Leveraging insights from interpersonal communication research and using video analysis and thematic analysis of 74 UGC (User Generated Content) videos, it identifies factors contributing to communication breakdown, including recognition sensitivity, interaction stability, corpus diversity, and content richness of smart speakers, as well as children’s language proficiency and intellectual development level. The study also highlights strategies employed by smart speakers to repair communication breakdowns, including both negative and positive measures, and that employed by children, consisting of a spectrum of verbal and non-verbal approaches, and parents adopt different repair strategies in response to communication breakdowns. Additionally, users tend to adapt to the logic of the machine during human-machine communication rather than the opposite. It argues that the concept of “process” bridges interpersonal communication and HMC, emphasizing developmental aspects of interpersonal communication is particularly relevant to HMC. The study also offers insights into human-machine relationship, human-machine civilization, and technopolitics.

Key words

interpersonal communication / human-machine relationship / communication process / smart speaker

Cite this article

EndNote

Ris (Procite)

Bibtex

Download Citations

LUO Longxiang , WANG Bing , WANG Xiuli. Breakdown and Repair of Human-machine Communication: Video Analysis of Family Members’ Joint Media Engagement[J]. Chinese Journal of Journalism & Communication. 2024, 46(8): 27-49

References

List( Publishing order | Descend order by publishing year | Descend order by cited within ) Chart analysis

[1]	邓建国(2022). 我们何以身临其境?——人机传播中社会在场感的建构与挑战. 《新闻与写作》,(10),17-28. Cited in this article [1]

[2]	甘雨梅(2023). 远程媒介化抚育中的“脚手架”、身体及物质性——以留守儿童与打工父母的视频通话研究为例. 《新闻与传播研究》,(1),85-102+128. Cited in this article [1]

[3]	甘雨梅(2020). “录像分析”作为社会研究方法:理论、应用与展望. 《新闻与传播研究》,(2),25-41+126. Cited in this article [1]

[4]	《国际新闻界》传播学年度课题组(2024). 2023年中国的传播学研究. 《国际新闻界》,(1),27-48. Cited in this article [1]

[5]	胡春阳(2009). 人际传播:学科与概念. 《国际新闻界》,(7),36-40. Cited in this article [2]

[6]	兰登·温纳(1977/1914). 《自主性技术》(杨海燕译). 北京: 北京大学出版社. Cited in this article [2]

[7]	刘蒙之(2009). 美国的人际传播研究及代表性理论. 《国际新闻界》,(3),123-128 Cited in this article [1]

[8]	罗龙翔, 王兵, 王秀丽(2021). 功能、关系、哲学:人机传播视域下用户与智能音箱的互动研究. 《全球传媒学刊》,(3),102-118. Cited in this article [3]

[9]	莫里斯·梅洛-庞蒂(1945/2021). 《知觉现象学》(杨大春,张尧均,关群德译). 北京: 商务印书馆. Cited in this article [2]

[10]	牟怡(2022). 《机器与传播:从计算机中介传播到人机传播》. 上海: 上海交通大学出版社. Cited in this article [1]

[11]	牟怡, 许坤(2018). 什么是人机传播?——一个新兴传播学领域之国际视域考察. 《江淮论坛》,(2),149-154. Cited in this article [1]

[12]	彭兰(2022). 人机传播与交流的未来. 《湖南师范大学社会科学学报》,(5),12-22. Cited in this article [3]

[13]	全燕(2023). 重组人机——行动者网络中的人机传播研究. 《新闻与传播研究》,(10),39-51+127. Cited in this article [1]

[14]	王广赞, 易显飞(2018). 人工智能研究的三大流派:比较与启示. 《长沙理工大学学报(社会科学版)》,(4),1-6. Cited in this article [1]

[15]	王晓升(2017). 现代性、现代主义和后现代主义——概念的梳理. 《华中科技大学学报(社会科学版)》,(5),1-8. Cited in this article [1]

[16]	王怡红(1996). 西方人际传播定义辨析. 《新闻与传播研究》,(4),72-79. Cited in this article [1]

[17]	王怡红(2015). 论“人际传播”的定名与定义问题. 《新闻与传播研究》,(7),112-125. Cited in this article [1]

[18]	王颖吉, 王袁欣(2021). 任务或闲聊?——人机交流的极限与聊天机器人的发展路径选择. 《国际新闻界》,(4),30-50. Cited in this article [1]

[19]	杨雅, 苏芳(2022). 后喻文化时代的家庭传播:新技术下的代际沟通与媒介素养. 《中华家教》,(1),25-32. Cited in this article [1]

[20]	朱秀凌(2018). 家庭传播研究的逻辑起点、历史演进和发展路径. 《国际新闻界》,(9),29-46. Cited in this article [1]

[21]	Baxter L. A., & Bullis C. (1986). Turning points in developing romantic relationships. Human Communication Research, 12(4), 469-493. https://doi.org/10.1111/hcre.1986.12.issue-4 http://www.blackwell-synergy.com/toc/hcre/12/4 Cited in this article [1]

[22]	Beneteau E., Richards O. K., Zhang M., Kientz J. A., Yip J., & Hiniker A. (2019, May). Communication breakdowns between families and alexa. Paper presented at the 2019 CHI Conference on Human Factors in Computing Systems. Glasgow. Cited in this article [3]

[23]

Berger

C. R.

, & Calabrese

R. J.

(1975). Some explorations in initial interaction and beyond: Toward a developmental theory of interpersonal communication. Human Communication Research, 1(2), 99-112.

https://doi.org/10.1111/hcre.1975.1.issue-2

http://www.blackwell-synergy.com/toc/hcre/1/2

Cited in this article [1]

[24]	Braithwaite D. O., & Schrodt P. (2022). Engaging theories in interpersonal communication: Multiple perspectives (3^rd ed.). New York, NY: Routledge. Cited in this article [4]

[25]	Braun V., & Clarke V. (2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3(2), 77-101. https://doi.org/10.1191/1478088706qp063oa https://www.tandfonline.com/doi/full/10.1191/1478088706qp063oa Cited in this article [1]

[26]	Cooley C. H. (1909). Social organization:A study of the larger mind. New York, NY: Charles Scribner’s Sons. Cited in this article [1]

[27]

Ewin

C. A.

, Reupert

A. E.

, McLean

L. A.

, & Ewin

C. J.

(2021). The impact of joint media engagement on parent-child interactions: A systematic review. Human Behavior and Emerging Technologies, 3(2), 230-254.

https://doi.org/10.1002/hbe2.v3.2

https://onlinelibrary.wiley.com/toc/25781863/3/2

Cited in this article [3]

[28]

Flippin

(2024). Communication breakdowns and repairs of children with autism spectrum disorder with fathers and mothers. International Journal of Disability, Development and Education, 71(4), 518-534.

https://doi.org/10.1080/1034912X.2022.2146069

https://www.tandfonline.com/doi/full/10.1080/1034912X.2022.2146069

Cited in this article [1]

[29]	Gagné J., Stelmacovich P., & Yovetich W. (1993). Reactions to requests for clarification used by hearing-impaired individuals. The Volta Review, 93(3), 129-143. Cited in this article [2]

[30]	Gallagher T. M. (1981). Contingent query sequences within adult-child discourse. Journal of Child Language, 8(1), 51-62. https://www.ncbi.nlm.nih.gov/pubmed/7204527 Cited in this article [1]

[31]

Garvey

(1975). Requests and responses in children’s speech. Journal of Child Language, 2(1), 41-63.

https://doi.org/10.1017/S030500090000088X

https://www.cambridge.org/core/product/identifier/S030500090000088X/type/journal_article

Cited in this article [1] Abstract

An investigation of children's ability to convey and respond to requests for action was based on the spontaneous speech of 36 dyads of nursery school children (3; 6–5; 7). Direct request forms (e.g. Give me the hammer) were frequent and the majority were acknowledged verbally. Examination of the contexts of direct requests indicated that speaker and addressee shared an understanding of the interpersonal meaning factors relevant to requesting. These meaning factors were invoked in justifying, refusing and in repeating or paraphrasing a request, and they also provided a basis for the communication of indirect requests. Examples of inferred requests are discussed, and a relationship between the structure of the speech act and conversational sequences is proposed.

[32]

Guzman

A. L.

, & Lewis

S. C.

(2020). Artificial intelligence and communication: A human-machine communication research agenda. New Media & Society, 22(1), 70-86.

https://doi.org/10.1177/1461444819858691

https://journals.sagepub.com/doi/10.1177/1461444819858691

Cited in this article [3] Abstract

Artificial intelligence (AI) and people’s interactions with it—through virtual agents, socialbots, and language-generation software—do not fit neatly into paradigms of communication theory that have long focused on human–human communication. To address this disconnect between communication theory and emerging technology, this article provides a starting point for articulating the differences between communicative AI and previous technologies and introduces a theoretical basis for navigating these conditions in the form of scholarship within human–machine communication (HMC). Drawing on an HMC framework, we outline a research agenda built around three key aspects of communicative AI technologies: (1) the functional dimensions through which people make sense of these devices and applications as communicators, (2) the relational dynamics through which people associate with these technologies and, in turn, relate to themselves and others, and (3) the metaphysical implications called up by blurring ontological boundaries surrounding what constitutes human, machine, and communication.

[33]	Hartley P. (1999). Interpersonal communication (2^nd ed.). New York, NY: Routledge. Cited in this article [1]

[34]	Heath C., Hindmarsh J., & Luff P. (2010). Video in qualitative research: Analysing social interaction in everyday life. London, UK: SAGE. Cited in this article [1]

[35]	Jefferson G. (2004). Glossary of transcript symbols with an introduction. In Lerner, G. H. (Ed.). Conversation analysis: Studies from the first generation (pp. 13-31). Philadelphia, PA: John Benjamins. Cited in this article [1]

[36]	Kim H., Kim J. Y., & Kang H. (2022). Effects of conversational agent’s self-repair strategy on user experience-focused on task criticality and conversational error. Journal of Digital Convergence, 20(2), 251-260. Cited in this article [1]

[37]	Liestøl G., Morrison A., & Rasmussen T. (Eds.). Digital media revisited:Theoretical and conceptual innovations in digital domains. Boston, MA: The MIT Press.

[38]	Lovato S., & Piper A. M. (2015, June). “Siri, is this you?”: Understanding young children’s interactions with voice input systems. Paper presented at the 14th International Conference on Interaction Design and Children. Boston. Cited in this article [1]

[39]	Luger E., & Sellen A. (2016, May). “Like having a really bad pa”: The gulf between user expectation and experience of conversational agents. Paper presented at the 2016 CHI Conference on Human Factors in Computing Systems. New York. Cited in this article [1]

[40]	Miller G. R. (1978). The current status of theory and research in interpersonal communication. Human Communication Research, 4(2), 164-178. https://doi.org/10.1111/hcre.1978.4.issue-2 http://www.blackwell-synergy.com/toc/hcre/4/2 Cited in this article [3]

[41]

Most

(2002). The use of repair strategies by children with and without hearing impairment. Language, Speech, and Hearing Services in Schools, 33(2), 112-123.

https://doi.org/10.1044/0161-1461(2002/009)

http://pubs.asha.org/doi/10.1044/0161-1461%282002/009%29

Cited in this article [1] Abstract

\n Purpose:\n This study examined how students with hearing impairments, having different levels of speech intelligibility, responded to a communication breakdown as compared to students with normal hearing.\n

[42]	Murdock G. P. (1949). Social structure. New York, NY: The Free Press. Cited in this article [1]

[43]	Nathanson A. I. (2002). The unintended effects of parental mediation of television on adolescents. Media Psychology, 4(3), 207-230. https://doi.org/10.1207/S1532785XMEP0403_01 http://www.tandfonline.com/doi/abs/10.1207/S1532785XMEP0403_01 Cited in this article [1]

[44]	Nowell L. S., Norris J. M., White D. E., & Moules N. J. (2017). Thematic analysis: Striving to meet the trustworthiness criteria. International Journal of Qualitative Methods, 16(1), 1609406917733847. Cited in this article [1]

[45]	Popper K. L. (1966). The open society and its enemies (5^th ed.). Princeton, NJ: Princeton University Press. Cited in this article [1]

[46]	Porcheron M., Fischer J. E., Reeves S., & Sharples S. (2018, April). Voice interfaces in everyday life. Paper presented at the 2018 CHI conference on human factors in computing systems. Montreal. Cited in this article [1]

[47]	Silverstone R. (1994). Television and everyday life. London, UK: Routledge. Cited in this article [1]

[48]	Suchman L. (2007). Human-machine reconfigurations: Plans and situated actions (2^nd ed.). Cambridge, UK: Cambridge University Press. Cited in this article [1]

[49]	Takeuchi L., & Stevens R. (2011). The new coviewing:Designing for learning through joint media engagement. New York, NY: The Joan Ganz Cooney Center at Sesame Workshop. Cited in this article [1]

[50]

Wang

, Luo

, & Wang

(2023). “Back to the living room era”: Smart speaker usage and family democracy from the family dynamic perspective. New Media & Society. Advance online publication. https://doi.org/10.1177/14614448231155624.

https://doi.org/10.1177/14614448231155624.

Cited in this article [2]

[51]	Yelland N., & Masters J. (2007). Rethinking scaffolding in the information age. Computers & Education, 48(3), 362-382. https://doi.org/10.1016/j.compedu.2005.01.010 https://linkinghub.elsevier.com/retrieve/pii/S0360131505000187 Cited in this article [1]

[52]	Yen K., Chen Y., Cheng Y., Chen S., Chen Y. Y., Ni Y., & Hiniker A. (2018). Joint media engagement between parents and preschoolers in the U.S., China, and Taiwan. Paper presented at the ACM on Human-Computer Interaction. New York. Cited in this article [1]

[53]

Yilmazyildiz

, Read

, Belpeame

, & Verhelst

(2015). Review of semantic-free utterances in social human-robot interaction. International Journal of Human-Computer Interaction, 32(1), 63-85.

https://doi.org/10.1080/10447318.2015.1093856

http://www.tandfonline.com/doi/full/10.1080/10447318.2015.1093856

Cited in this article [1]

[54]

Zhao

(2006). Humanoid social robots as a medium of communication. New Media & Society, 8(3), 401-419.

https://doi.org/10.1177/1461444806061951

https://journals.sagepub.com/doi/10.1177/1461444806061951

Cited in this article [1] Abstract

This article examines the emerging phenomenon of humanoid social robots and human-humanoid interactions. A central argument of this article is that humanoid social robots belong to a special type of robotic technology used for communicating and interacting with humans. These robotic entities, which can be in either mechanical or digital form, are autonomous, interactive and humanlike. Some of them are used to interact with humans for utilitarian purposes and others are designed to trigger human emotions. Incorporation of such robotic entities into the realm of social life invariably alters the condition as well as the dynamics of human interaction, giving rise to a synthetic society in which humans co-mingle with humanoids. More research is needed to investigate the social and cultural impact of this unfolding robotic revolution.

Footnotes

1. 参考转写规则（Jefferson，2004），录像转写时使用到的特殊符号说明如下：

符号	含义
（数字）	沉默时间秒数，中间无言语对话
> <	极快的语速
↑	音量增加
oo	非常轻柔的声音
－	突然停顿
（（文字））	研究者添加的描述