Here's another snippet of the Brain-Computer Metaphor essay I'm working on. I'm finding it a little difficult to strike the right tone on this one. Nonetheless, below is the working intro, which I've edited down considerably from my previous version:
Last year, OpenAI’s latest language model, GPT-3, was tested as a viable healthcare chatbot, and promptly suggested that a fake patient should commit suicide because they “felt bad” (Daws, 2020). While in this instance GPT-3 did not perform as hoped, large language models in general are fluent enough to give a false impression of language understanding and mental modeling (Bender et al., 2021), thus epitomizing the concept of artificial intelligence, or AI. Cases like the above, however, call into question whether the intelligence and brain-based terminology used to market AI technology poses risks upon whom the technology is used.
At the core of this terminology is perhaps the most debated metaphor in all of science, the Brain-Computer Metaphor (Brette, 2019; Cisek, 1999; Cobb, 2020; Jonas & Kording, 2017; Marcus, 2015; Richards, 2018; Smith, 1993; Taylor & Dewsbury, 2018; Vlasits, 2017; von Neumann, 1958; Weizenbaum, 1976). Drawing parallels between computer and brain functions, neuroscientists frequently conceptualize that THE BRAIN IS A COMPUTER. Indeed, for neuroscientists, the implication of the Brain-Computer Metaphor is quite clear: it is needed to advance the field because scientific metaphors, in general, are valuable tools for explaining complex subject matter and generating useful ideas. But the Brain-Computer Metaphor is bigger than neuroscience because it is encountered in everyday life and implied in every mention of artificial intelligence. Thus, as the most critical users of that metaphor, neuroscientists shape how it is communicated both in and outside of the lab.
The purpose of this essay is not to pose a semantic argument for the sake of neuroscience, nor is it to suppress innovative thinking. Rather, it is to point out that the Brain-Computer Metaphor has tangible, and important social implications which have received little attention. As neuroscientists debate about whether the brain is a computer or not, AI is widely marketed with impressive measures of biometric recognition and language generation, demonstrating seemingly brain-like abilities. At the same time, however, it is known these intelligent technologies can perpetuate social harms that are often at the expense of those who are under-represented in the neuroscience and AI fields (Allyn, 2020; Angwin et al., 2016; Benjamin, 2019; Birhane & Guest, 2020; Crawford et al., 2019; O’Neil, 2017; Ong, 2017; Stark, 2019). The academic debate around the Brain-Computer Metaphor, therefore, should also include this crucial question: whom does it benefit, and whom does it harm?
Allyn, B. (2020, June 24). “The Computer Got It Wrong”: How Facial Recognition Led To False Arrest Of Black Man. NPR.Org. https://www.npr.org/2020/06/24/882683463/the-computer-got-it-wrong-how-facial-recognition-led-to-a-false-arrest-in-michig
Angwin, J., Larson, J., Mattu, S., & Kirchner, L. (2016, May 23). Machine Bias. ProPublica. https://www.propublica.org/article/machine-bias-risk-assessments-in-criminal-sentencing
Bender, E. M., Gebru, T., McMillan-Major, A., & Shmitchell, S. (2021). On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? 🦜. Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency, 610–623. https://doi.org/10.1145/3442188.3445922
Benjamin, R. (2019). Race after Technology; Abolitionist Tools for the New Jim Code. Polity.
Birhane, A., & Guest, O. (2020). Towards decolonising computational sciences. ArXiv:2009.14258 [Cs]. http://arxiv.org/abs/2009.14258
Brette, R. (2019). Is coding a relevant metaphor for the brain? Behavioral and Brain Sciences, 42. https://doi.org/10.1017/S0140525X19000049
Cisek, P. (1999). BEYOND THE COMPUTER METAPHOR: BEHAVIOR AS INTERACTION. Journal of Consciousness Studies, 6(11–12), 125–142.
Cobb, M. (2020, February 27). Why your brain is not a computer. The Guardian. http://www.theguardian.com/science/2020/feb/27/why-your-brain-is-not-a-computer-neuroscience-neural-networks-consciousness
Crawford, K., Dobbe, R., Dryer, T., Fried, G., Green, B., Kaziunas, E., Kak, A., Mathur, V., McElroy, E., Sánchez, A. N., Raji, D., Rankin, J. L., Richardson, R., Schultz, J., West, S. M., & Whittaker, M. (2019). AI Now 2019 Report. https://ainowinstitute.org/AI_Now_2019_Report.pdf
Daws, R. (2020, October 28). Medical chatbot using OpenAI’s GPT-3 told a fake patient to kill themselves. AI News. https://artificialintelligence-news.com/2020/10/28/medical-chatbot-openai-gpt3-patient-kill-themselves/
Jonas, E., & Kording, K. P. (2017). Could a Neuroscientist Understand a Microprocessor? PLOS Computational Biology, 13(1), e1005268. https://doi.org/10.1371/journal.pcbi.1005268
Marcus, G. (2015, June 27). Opinion | Face It, Your Brain Is a Computer. The New York Times. https://www.nytimes.com/2015/06/28/opinion/sunday/face-it-your-brain-is-a-computer.html
O’Neil, C. (2017). Weapons of Math Destruction. Penguin Books.
Ong, T. (2017, October 24). Facebook apologizes after wrong translation sees Palestinian man arrested for posting ‘good morning.’ The Verge. https://www.theverge.com/us-world/2017/10/24/16533496/facebook-apology-wrong-translation-palestinian-arrested-post-good-morning
Richards, B. (2018, October 1). Yes, the brain is a computer…. Medium. https://medium.com/the-spike/yes-the-brain-is-a-computer-11f630cad736
Smith, C. U. M. (1993). The use and abuse of metaphors in the history of brain science. Journal of the History of the Neurosciences, 2(4), 283–301. https://doi.org/10.1080/09647049309525577
Stark, L. (2019). Facial recognition is the plutonium of AI. XRDS: Crossroads, The ACM Magazine for Students, 25(3), 50–55. https://doi.org/10.1145/3313129
Taylor, C., & Dewsbury, B. M. (2018). On the Problem and Promise of Metaphor Use in Science and Science Communication. Journal of Microbiology & Biology Education, 19(1). https://doi.org/10.1128/jmbe.v19i1.1538
Vlasits, A. (2017, June 22). Tech Metaphors Are Holding Back Brain Research. Wired. https://www.wired.com/story/tech-metaphors-are-holding-back-brain-research/
von Neumann, J. (1958). The Computer and the Brain. Yale University Press.
Weizenbaum, J. (1976). Computer Power and Human Reason: From Judgment to Calculation. W.H. Freeman & Co.
Last year, OpenAI’s latest language model, GPT-3, was tested as a viable healthcare chatbot, and promptly suggested that a fake patient should commit suicide because they “felt bad” (Daws, 2020). While in this instance GPT-3 did not perform as hoped, large language models in general are fluent enough to give a false impression of language understanding and mental modeling (Bender et al., 2021), thus epitomizing the concept of artificial intelligence, or AI. Cases like the above, however, call into question whether the intelligence and brain-based terminology used to market AI technology poses risks upon whom the technology is used.
At the core of this terminology is perhaps the most debated metaphor in all of science, the Brain-Computer Metaphor (Brette, 2019; Cisek, 1999; Cobb, 2020; Jonas & Kording, 2017; Marcus, 2015; Richards, 2018; Smith, 1993; Taylor & Dewsbury, 2018; Vlasits, 2017; von Neumann, 1958; Weizenbaum, 1976). Drawing parallels between computer and brain functions, neuroscientists frequently conceptualize that THE BRAIN IS A COMPUTER. Indeed, for neuroscientists, the implication of the Brain-Computer Metaphor is quite clear: it is needed to advance the field because scientific metaphors, in general, are valuable tools for explaining complex subject matter and generating useful ideas. But the Brain-Computer Metaphor is bigger than neuroscience because it is encountered in everyday life and implied in every mention of artificial intelligence. Thus, as the most critical users of that metaphor, neuroscientists shape how it is communicated both in and outside of the lab.
The purpose of this essay is not to pose a semantic argument for the sake of neuroscience, nor is it to suppress innovative thinking. Rather, it is to point out that the Brain-Computer Metaphor has tangible, and important social implications which have received little attention. As neuroscientists debate about whether the brain is a computer or not, AI is widely marketed with impressive measures of biometric recognition and language generation, demonstrating seemingly brain-like abilities. At the same time, however, it is known these intelligent technologies can perpetuate social harms that are often at the expense of those who are under-represented in the neuroscience and AI fields (Allyn, 2020; Angwin et al., 2016; Benjamin, 2019; Birhane & Guest, 2020; Crawford et al., 2019; O’Neil, 2017; Ong, 2017; Stark, 2019). The academic debate around the Brain-Computer Metaphor, therefore, should also include this crucial question: whom does it benefit, and whom does it harm?
Allyn, B. (2020, June 24). “The Computer Got It Wrong”: How Facial Recognition Led To False Arrest Of Black Man. NPR.Org. https://www.npr.org/2020/06/24/882683463/the-computer-got-it-wrong-how-facial-recognition-led-to-a-false-arrest-in-michig
Angwin, J., Larson, J., Mattu, S., & Kirchner, L. (2016, May 23). Machine Bias. ProPublica. https://www.propublica.org/article/machine-bias-risk-assessments-in-criminal-sentencing
Bender, E. M., Gebru, T., McMillan-Major, A., & Shmitchell, S. (2021). On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? 🦜. Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency, 610–623. https://doi.org/10.1145/3442188.3445922
Benjamin, R. (2019). Race after Technology; Abolitionist Tools for the New Jim Code. Polity.
Birhane, A., & Guest, O. (2020). Towards decolonising computational sciences. ArXiv:2009.14258 [Cs]. http://arxiv.org/abs/2009.14258
Brette, R. (2019). Is coding a relevant metaphor for the brain? Behavioral and Brain Sciences, 42. https://doi.org/10.1017/S0140525X19000049
Cisek, P. (1999). BEYOND THE COMPUTER METAPHOR: BEHAVIOR AS INTERACTION. Journal of Consciousness Studies, 6(11–12), 125–142.
Cobb, M. (2020, February 27). Why your brain is not a computer. The Guardian. http://www.theguardian.com/science/2020/feb/27/why-your-brain-is-not-a-computer-neuroscience-neural-networks-consciousness
Crawford, K., Dobbe, R., Dryer, T., Fried, G., Green, B., Kaziunas, E., Kak, A., Mathur, V., McElroy, E., Sánchez, A. N., Raji, D., Rankin, J. L., Richardson, R., Schultz, J., West, S. M., & Whittaker, M. (2019). AI Now 2019 Report. https://ainowinstitute.org/AI_Now_2019_Report.pdf
Daws, R. (2020, October 28). Medical chatbot using OpenAI’s GPT-3 told a fake patient to kill themselves. AI News. https://artificialintelligence-news.com/2020/10/28/medical-chatbot-openai-gpt3-patient-kill-themselves/
Jonas, E., & Kording, K. P. (2017). Could a Neuroscientist Understand a Microprocessor? PLOS Computational Biology, 13(1), e1005268. https://doi.org/10.1371/journal.pcbi.1005268
Marcus, G. (2015, June 27). Opinion | Face It, Your Brain Is a Computer. The New York Times. https://www.nytimes.com/2015/06/28/opinion/sunday/face-it-your-brain-is-a-computer.html
O’Neil, C. (2017). Weapons of Math Destruction. Penguin Books.
Ong, T. (2017, October 24). Facebook apologizes after wrong translation sees Palestinian man arrested for posting ‘good morning.’ The Verge. https://www.theverge.com/us-world/2017/10/24/16533496/facebook-apology-wrong-translation-palestinian-arrested-post-good-morning
Richards, B. (2018, October 1). Yes, the brain is a computer…. Medium. https://medium.com/the-spike/yes-the-brain-is-a-computer-11f630cad736
Smith, C. U. M. (1993). The use and abuse of metaphors in the history of brain science. Journal of the History of the Neurosciences, 2(4), 283–301. https://doi.org/10.1080/09647049309525577
Stark, L. (2019). Facial recognition is the plutonium of AI. XRDS: Crossroads, The ACM Magazine for Students, 25(3), 50–55. https://doi.org/10.1145/3313129
Taylor, C., & Dewsbury, B. M. (2018). On the Problem and Promise of Metaphor Use in Science and Science Communication. Journal of Microbiology & Biology Education, 19(1). https://doi.org/10.1128/jmbe.v19i1.1538
Vlasits, A. (2017, June 22). Tech Metaphors Are Holding Back Brain Research. Wired. https://www.wired.com/story/tech-metaphors-are-holding-back-brain-research/
von Neumann, J. (1958). The Computer and the Brain. Yale University Press.
Weizenbaum, J. (1976). Computer Power and Human Reason: From Judgment to Calculation. W.H. Freeman & Co.
Comments
Post a Comment