“Hearing” and “Appearing” in Human-computer Interaction: The Compensation and Conflict of the Avatar Image Presentation of Intelligent Voice Assistant to the User’s Auditory Image

JIAN Yufan, HUANG Yubo

Chinese Journal of Journalism & Communication ›› 2022, Vol. 44 ›› Issue (10) : 50-73.

PDF(1635 KB)
PDF(1635 KB)
Chinese Journal of Journalism & Communication ›› 2022, Vol. 44 ›› Issue (10) : 50-73.
Research Articles

“Hearing” and “Appearing” in Human-computer Interaction: The Compensation and Conflict of the Avatar Image Presentation of Intelligent Voice Assistant to the User’s Auditory Image

Author information +
History +

Abstract

Whether human-computer interaction is consistent with interpersonal communication mode is a hot topic in current human-computer relationship research. Network interpersonal communication can realize “hearing its voice” even in the absence of appearance. If human-computer interaction returns to network interpersonal communication mode, the voice language interaction between the user and the intelligent voice assistant can also produce a mental imagery, and this image may deviate from the avatar image of the intelligent voice assistant. This paper takes the auditory and visual senses (avatar image) of the intelligent voice assistant as an example as the starting point, it specifically discusses the avatar image presentation, interactive human-computer interaction relationship, impact potency and impact mechanism of intelligent voice assistant on user evaluation. Through two experiments, it is found that under the communal man-machine relationship, the avatar image of intelligent voice assistant presents, conflicts with the user’s auditory sensory psychological image, produces image distortion, and then weakens the user’s evaluation; under the exchange man-machine relationship, the avatar image of intelligent voice assistant is presented, which compensates the psychological image of users’ auditory senses, promotes image clarity, and improves users’ evaluation.

Key words

human-machine relationship / avatar image / intelligent voice assistant / sensory research / mental imagery

Cite this article

Download Citations
JIAN Yufan , HUANG Yubo. “Hearing” and “Appearing” in Human-computer Interaction: The Compensation and Conflict of the Avatar Image Presentation of Intelligent Voice Assistant to the User’s Auditory Image[J]. Chinese Journal of Journalism & Communication. 2022, 44(10): 50-73

References

[1]
孟伟(2006). 《声音传播:多媒介传播时代的广播听觉文本》. 北京: 中国传媒大学出版.
[2]
陈瑞, 郑毓煌, 刘文静(2013). 中介效应分析:原理、程序、Bootstrap方法及其应用. 《营销科学学报》,(4),120-135.
[3]
陈新民, 马廷魁(2017). “人机互动”还是“人际互动”?——对电视社交化生存的多维度思考. 《现代传播(中国传媒大学学报)》,(2),18-22.
[4]
曹忠鹏, 靳成雯, 马菁, 李雁晨(2020). 自助服务技术中虚拟代理人呈现对顾客准备的影响研究. 《南开管理评论》,(4),73-83.
[5]
何双百(2022). 人工移情:新型同伴关系中的自我、他者及程序意向性. 《现代传播(中国传媒大学学报)》,(2),162-168.
[6]
林升梁, 叶立(2019). 人机·交往·重塑:作为“第六媒介”的智能机器人. 《新闻与传播研究》,(10),87-104+128.
[7]
申琦, 王璐瑜(2021). 当“机器人”成为社会行动者:人机交互关系中的刻板印象. 《新闻与传播研究》,(2),37-52+127.
[8]
沈荟, 王学成(2015). 新媒体人际传播的议题、理论与方法选择——以美国三大传播学期刊为样本的分析. 《新闻与传播研究》,(12),81-100+128.
[9]
王海忠, 谢涛, 詹纯玉(2021). 服务失败情境下智能客服化身拟人化的负面影响:厌恶感的中介机制. 《南开管理评论》,(4),194-206.
[10]
王颖吉, 王袁欣(2021). 任务或闲聊?——人机交流的极限与聊天机器人的发展路径选择. 《国际新闻界》,(4),30-50.
[11]
姚晓鸥(2020). 面容媒介、道德意识与人际交往关系:基于现象学的交互主体性分析. 《新闻与传播研究》,(1),53-68+127.
[12]
喻国明, 王文轩, 冯菲, 修利超(2021). 合成语音新闻的传播效果评测——关于语速影响的EEG证据. 《国际新闻界》,(2),6-26.
[13]
张放(2011). 网络人际传播中印象形成效果的实验研究. 《国际新闻界》,(2),76-82.
[14]
张放(2012). 网络人际传播中印象形成机制的实验研究. 《新闻与传播研究》,(3),52-60+111.
[15]
朱华伟, 苏羽, 冯靖元(2021). 代言人类型和产品创新类型对创新产品购买意愿的交互影响,检索于https://kns.cnki.net/kcms/detail/12.1288.f.20210508.1610.005.html.
[16]
Aggarwal P. (2004). The effects of brand relationship norms on consumer attitudes and behavior. Journal of consumer research, 31(1), 87-101.
[17]
Babin L. A., & Burns A. C. (1997). Effects of print ad pictures and copy containing instructions to imagine on mental imagery that mediates attitudes. Journal of advertising, 26(3), 33-44.
[18]
Bartneck C., Kulić D., Croft E., & Zoghbi S. (2009). Measurement instruments for the anthropomorphism, animacy, likeability, perceived intelligence, and perceived safety of robots. International journal of social robotics, 1(1), 71-81.
[19]
Brahnam S., & De Angeli A. (2012). Gender affordances of conversational agents. Interacting with Computers, 24(3), 139-153.
[20]
Braun M., Mainz A., Chadowitz R., Pfleging B., & Alt F. (2019, May). At your service: Designing voice assistant personalities to improve automotive user interfaces. Paper presented at the 2019 CHI Conference on Human Factors in Computing Systems. Glasgow.
[21]
Cautela J. R., & McCullough L. (1978). Covert conditioning:A learning-theory perspective on imagery. In Singer, J. L., & PopeK. S. (Eds.). The power of human imagination (pp.227-254). Boston, MA: Springer,.
[22]
Chen Q., He Y., Hu M., & Kim J. (2020). Navigating relationship norms: An exploration of how content strategies improve brand valuation over time. Journal of Advertising, 49(4), 459-476.
[23]
Cheng X., Zhang X., Cohen J., & Mou J. (2022). Human vs. AI: Understanding the impact of anthropomorphism on consumer response to chatbots from the perspective of trust and relationship norms. Retrieved from http://gfcfb391f4815d8064db7s5bcvup0kncqc6qbc.fzzh.libproxy.ruc.edu.cn/10.1016/j.ipm.2022.102940.
[24]
Clark M. S., & Mils J. (1993). The difference between communal and exchange relationships: What it is and is not. Personality and social psychology bulletin, 19(6), 684-691.
Replying to Batson (this issue), who argues that the difference between communal and exchange relationships is less than might be imagined, the authors review the communal/exchange distinction, discuss what it does and does not mean, and address specific issues raised by Batson. They conclude, contrary to claims by Batson, that (a) ingratiation cannot account for all the findings supporting the communal/exchange distinction; (b) if desire for a communal relationship leads to ingratiation, that does not contradict the distinction; (c) if communal norms are followed for other than altruistic reasons, that does not undermine the distinction, (d) the difference between communal and exchange relationships is not limited to the breadth of benefits exchanged; (e) the difference between communal and exchange relationships is not limited to a difference in etiquette; rather, the communal/exchange distinction provides a theoretical explanation for the difference in etiquette.
[25]
Cassell J. (2000). Embodied conversational interface agents. Communications of the ACM, 43(4), 70-78.
[26]
Cox D., & Cox A. D. (2002). Beyond first impressions: The effects of repeated exposure on consumer liking of visually complex and simple product designs. Journal of the Academy of Marketing Science, 30(2), 119-130.
[27]
Chae S. W., Lee K. C., & Seo Y. W. (2016). Exploring the effect of avatar trust on learners’ perceived participation intentions in an e-learning environment. International Journal of Human-Computer Interaction, 32(5), 373-393.
[28]
Dagsdóttir L. K., Skyt I., Vase L., Baad-Hansen L., Castrillon E., & Svensson P. (2015). Experimental orofacial pain and sensory deprivation lead to perceptual distortion of the face in healthy volunteers. Experimental brain research, 233(9), 2597-2606.
[29]
Dawar N., & Bendle N. (2018). Marketing in the age of Alexa. Harvard Business Review, 96(3), 80-86.
[30]
Etemad-Sajadi R. (2016). The impact of online real-time interactivity on patronage intention: The use of avatars. Computers in human behavior, 61, 227-232.
[31]
Fernandes T., & Oliveira E. (2021). Understanding consumers’ acceptance of automated technologies in service encounters: Drivers of digital voice assistants adoption. Journal of Business Research, 122, 180-191.
[32]
Foehr J., & Germelmann C. C. (2020). Alexa, can I trust you? Exploring consumer paths to trust in smart voice-interaction technologies. Journal of the Association for Consumer Research, 5(2), 181-205.
[33]
Fox J., Ahn S. J., Janssen J. H., Yeykelis L., Segovia K. Y., & Bailenson J. N. (2015). Avatars versus agents: a meta-analysis quantifying the effect of agency on social influence. Human-Computer Interaction, 30(5), 401-432.
[34]
Johnson J. W., & Grimm P. E. (2010). Communal and exchange relationship perceptions as separate constructs and their role in motivations to donate. Journal of Consumer Psychology, 20(3), 282-294.
Researchers have operationalized communal and exchange relationship perceptions as either mutually exclusive categories or opposite ends of a continuum. This research conceptualizes these relationships as distinct constructs that should be measured separately. We develop multi‐item measures of communal and exchange relationship perceptions and find that they are actually positively correlated. We also examine the way communal and exchange relationship perceptions combine to influence intrinsic, extrinsic, and social motivations to donate, a category stipulated in economics, but not in psychology. We find that both relationship perceptions influence consumer attitudes toward donating through a mix of intrinsic, extrinsic, and social motivations.
[35]
Kiesler S., Powers A., Fussell S. R., & Torrey C. (2008). Anthropomorphic interactions with a robot and robot-like agent. Social Cognition, 26(2), 169-181.
[36]
Keeling K., McGoldrick P., & Beatty S. (2010). Avatars as salespeople: Communication style, trust, and intentions. Journal of Business Research, 63(8), 793-800.
[37]
Klaus P., & Zaichkowsky J. L. (2022). The convenience of shopping via voice AI: Introducing AIDM. Retrieved from http://gfcfb391f4815d8064db7s5bcvup0kncqc6qbc.fzzh.libproxy.ruc.edu.cn/10.1016/j.jretconser.2021.102490.
[38]
Lee W., & Gretzel U. (2012). Designing persuasive destination websites: A mental imagery processing perspective. Tourism management, 33(5), 1270-1280.
[39]
Lee J. E., & Shin E. (2020). The effects of apparel names and visual complexity of apparel design on consumers' apparel product attitudes: A mental imagery perspective. Journal of Business Research, 120, 407-417.
[40]
MacInnis D. J., & Price L. L. (1987). The role of imagery in information processing: Review and extensions. Journal of consumer research, 13(4), 473-491.
[41]
MacInnis D. J., & Jaworski B. J. (1989). Information processing from advertisements: Toward an integrative framework. Journal of marketing, 53(4), 1-23.
[42]
Moon J. H., Kim E., Choi S. M., & Sung Y. (2013). Keep the social in social media: The role of social interaction in avatar-based virtual shopping. Journal of Interactive Advertising, 13(1), 14-26.
[43]
McLean G., Osei-Frimpong K., & Barhorst J. (2021). Alexa, do voice assistants influence consumer brand engagement? -Examining the role of AI powered voice assistants in influencing consumer brand engagement. Journal of Business Research, 124, 312-328.
[44]
Mileva M., Tompkinson J., Watt D., & Burton A. M. (2020). The role of face and voice cues in predicting the outcome of student representative elections. Personality and Social Psychology Bulletin, 46(4), 617-625.
First impressions formed after seeing someone's face or hearing their voice can affect many social decisions, including voting in political elections. Despite the many studies investigating the independent contribution of face and voice cues to electoral success, their integration is still not well understood. Here, we examine a novel electoral context, student representative ballots, allowing us to test the generalizability of previous studies. We also examine the independent contributions of visual, auditory, and audiovisual information to social judgments of the candidates, and their relationship to election outcomes. Results showed that perceived trustworthiness was the only trait significantly related to election success. These findings contrast with previous reports on the importance of perceived competence using audio or visual cues only in the context of national political elections. The present study highlights the role of real-world context and emphasizes the importance of using ecologically valid stimulus presentation in understanding real-life social judgment.
[45]
Petty R. E., Cacioppo J. T., & Schumann D. (1983). Central and peripheral routes to advertising effectiveness: The moderating role of involvement. Journal of consumer research, 10(2), 135-146.
[46]
Rhee C. E., & Choi J. (2020). Effects of personalization and social role in voice shopping: An experimental study on product recommendation by a conversational voice agent. Retrieved from http://gfcfb391f4815d8064db7s5bcvup0kncqc6qbc.fzzh.libproxy.ruc.edu.cn/10.1016/j.chb.2020.106359.
[47]
Schlosser A. E. (2003). Experiencing products in the virtual world: The role of goal and imagery in influencing attitudes versus purchase intentions. Journal of consumer research, 30(2), 184-198.
[48]
Schacter D. L., Addis D. R., & Buckner R. L. (2007). Remembering the past to imagine the future: the prospective brain. Nature reviews neuroscience, 8(9), 657-661.
A rapidly growing number of recent studies show that imagining the future depends on much of the same neural machinery that is needed for remembering the past. These findings have led to the concept of the prospective brain; an idea that a crucial function of the brain is to use stored information to imagine, simulate and predict possible future events. We suggest that processes such as memory can be productively re-conceptualized in light of this idea.
[49]
Wan L. C., Hui M. K., & Wyer Jr R. S. (2011). The role of relationship norms in responses to service failures. Journal of Consumer Research, 38(2), 260-277.
[50]
Wang L. C., Baker J., Wagner J. A., & Wakefield K. (2007). Can a retail web site be social? Journal of marketing, 71(3), 143-157.
Avatars are lifelike characters created by technology. Research suggests that avatars can increase the persuasiveness of online sales channels. The authors investigate how the social cues inherent in avatars influence consumers' affect and shopping value. In Study 1, social cues induce perceptions of Web site socialness, leading to increased pleasure and arousal, both of which positively influence flow, hedonic and utilitarian value, and patronage intentions. Study 2 finds that social cue–induced arousal leads to increased pleasure only for consumers who are involved with the product category. Moreover, the influence of arousal on hedonic value is stronger for women, flow does not lead to pleasure for older consumers, and utilitarian value is less important for this group than for their younger counterparts. The findings suggest that there is a competitive advantage for online retailers that use social cues that provide consumers with enhanced perceptions of human connection and the formation of emotional bonds.
[51]
Weinstein N., Vansteenkiste M., & Paulmann S. (2020). Don’t you say it that way! Experimental evidence that controlling voices elicit defiance. Retrieved from http://gfcfb391f4815d8064db7s5bcvup0kncqc6qbc.fzzh.libproxy.ruc.edu.cn/10.1016/j.jesp.2019.103949.
[52]
Yoo J., & Kim M. (2014). The effects of online product presentation on consumer responses: A mental imagery perspective. Journal of Business Research, 67(11), 2464-2472.

Funding

National Natural Science Fund of China(72002135)
National Social Science Foundation of China(20BXW122)
Shenzhen Social Science Planning Project(SZ2021B042)
PDF(1635 KB)

Accesses

Citation

Detail

Sections
Recommended

/