The business of ethically using artificial intelligence

Editor's note: This expert Q&A is part of our “AI is everywhere ... now what?” special project exploring the potential (and potential pitfalls) of artificial intelligence in our lives. Explore more topics and takes on the project page.

When it comes to artificial intelligence, how can it be ethically measured through data? This is a question that serves as a guiding principle at the Center for AI and Data Analytics for Business and Society at the W. P. Carey School of Business at Arizona State University.

Pei-yu (Sharon) Chen is the Red Avenue Foundation Professor, the chair of the Department of Information Systems and the founding director and co-director of the Center for AI and Data Analytics for Business and Society. 

Her research examines how information technology interacts with business and where they can garner value from using AI while ethically utilizing it. These values are part of what formed the W. P. Carey School's master's degree in AI in business. The master's program was the first of its kind to be offered at a business school in the U.S. when it launched in March 2024 and now offers an undergraduate component. 

Chen is also a distinguished fellow of INFORMS Information Systems Society, which recognizes her outstanding intellectual contributions to the information systems discipline. Her work has been published in leading business journals, and she currently serves as senior editor for Information Systems Research and associate editor for Management Science.

Here, Chen provided her expertise on what AI can achieve when approached thoughtfully.

Note: Answers have been edited for length and/or clarity.

Question: How has your research with AI shaped your outlook on it?

Answer: Research and teaching about AI allows me to understand its potential as well as how the sausage is made, giving me mixed feelings. AI promises substantial benefits for individuals, businesses and society. However, these same capabilities that make AI powerful also raises serious concerns about privacy, security and the potential for misuse, which can create substantial damage to our lives and society. Unfortunately, our increasing reliance on advanced technology makes us even more vulnerable, and it is not clear if the future would indeed be better for us. What we do today would determine whether the future is better or worse. 

This is what motivates the creation of our Center for AI and Data Analytics (AIDA) for Business and Society, and our AI in business programs. We need to enhance awareness about the dual-edged nature of AI, we need to conduct research that mindfully advances (the) knowledge and technology frontier, and we need to develop a large AI workforce capable of leading responsible and ethical implementation of AI. Only then can we ensure that AI truly benefits our lives, society and the world.

Q: You’ve discussed “mindful AI” in your work. Can you expand on this? What metrics do you utilize when creating a framework for mindful AI in data analysis?

A: Mindful AI means being mindful about how our decisions and actions may impact others and society, specifically concerning the implications of how we develop, deploy and use AI. For example, while AI can reduce costs and automate decisions, any inherent bias contained can result in significantly amplified effects on affected groups, leading to inequality and unfairness due to the wider use of the application and increasing reliance on AI. 

Therefore, it is crucial to understand the implications of how an AI application is developed and used. Additionally, we need to understand the potential consequences if AI is not developed or used safely/responsibly.

Q: How can business owners prepare to work with AI? Why should they care?

A: Business owners can prepare to work with AI at two levels. The first level involves integrating AI into business processes to enhance efficiency, performance and welfare. This includes automating routine tasks, optimizing operations, and improving decision-making through data-driven insights.

The second, deeper level is the integration of AI into the organization to augment learning, intelligence and the capability to sense and respond quickly and accurately to changing conditions. This involves using AI to foster a more adaptive and intelligent organization that can continuously learn and evolve, thereby improving strategic agility and resilience in dynamic environments.

Business owners should care about AI adoption because the first level of AI use is becoming a competitive necessity, due to its capabilities to improve operational efficiency and performance. The second level of AI use can create a competitive advantage by capitalizing on new opportunities, driving growth and enhancing robustness and resilience in dynamic environments.

Q: Can you discuss the new Center for AI and Data Analytics (AIDA) for Business and Society?

A: AI is transforming every aspect of our lives and society, bringing abundant opportunities, but also potential risks and challenges. The current state of AI reveals a rapid embrace by individuals and organizations, driven by the fear of missing out on opportunities or competitive advantage if they delay adoption.

However, these private incentives are often shortsighted, self-centered and narrow focused, which are often not aligned with optimal social outcomes. This misalignment can lead to unfair results across different segments of the population and pose high risk and substantial costs to the society in the long run. Unfortunately, the private sector often lacks the motivation to invest in measures ensuring the responsible and ethical design and use of AI, whereas users often lack understanding of the processes behind AI applications and the implications of their reliance on or usage of such technology, thereby amplifying further concerns of unintended consequences to society.

AIDA’s vision is to ensure that AI can actually better our lives, society and the world. Our mission is to help individuals and organizations capitalize on the opportunities of AI and achieve success while also mitigating risks and preventing misuse. We seek to make an impact through pursuing research, education and solutions that aim to bring about positive and lasting transformations in how individuals, organizations and society view, develop and use AI. Central to this vision is the concept of “mindful AI,” which emphasizes a deep understanding of the purposes — what and why— for which AI is utilized and how it should be mindfully developed and used. 

Q: What skills will students specifically learn in a business program that focuses on AI?

A: Students can expect to learn three types of skills: 

• Technical skills, including AI foundations, machine learning and AI algorithms, and big data analytics.
• Business and professional skills. Students will learn how to analyze diverse business situations and apply AI to further business goals, skills needed to lead cross-functional conversations and collaboration for effective digital transformation and management of AI projects, and how to effectively communicate the impact of AI transformations. 
• Mindful AI and principled innovations. It’s important to note that we are not just training the next generation of AI experts, but we are preparing the leaders who will ensure that AI development and application is ethical, inclusive and responsible, and is driven by a desire to improve the world. Students will practice mindful AI and pay attention to ethics, bias, welfare, privacy and trust. 

Q: Your research looks at how AI can not only enhance business practices but also enhance social welfare. Can you explain in a couple of sentences what this means?

A: Over the years, we have found that, as technology progresses, our lives do not always improve. For example, the polarization and conflicts among people are partly due to technology encouraging us to consume information that reinforces our existing beliefs, building thicker walls around us. Social media addiction is another issue, as recommendation systems excel at suggesting content that captures our interest. Furthermore, many AI tools are being used to spread misinformation and disinformation, making it difficult to tell the truth. 

Is it possible for AI to enhance business practices while also improving social welfare? Yes, by raising awareness and aligning incentives to include social welfare as a key aspect of everyone's responsibility and evaluation. When the majority of people care and take action, it will compel companies to take corporate social responsibilities seriously and redefine their success beyond mere profitability.