Measuring the Impact of Business School Research
There have long been calls for business schools to focus on “responsible research” with relevance to society. Now, there is fresh scope to deploy technology to help assess their academics’ outputs at scale, albeit with limitations.
Limitations of Measuring Responsible Research
It is difficult, by any yardstick, to evaluate digitally the originality and depth of insight of academic writing, let alone assess its ultimate impact. Academic awards and assessments of papers do not necessarily coincide with the views of practitioners seeking practical applications. Ideas may take many years, and be significantly modified, before they are taken up. Some of the strongest may eventually prove to be erroneous and even harmful, or unexpectedly beneficial in unanticipated applications.
Technology Tools for Measuring Responsible Research
However, technology tools linked to large language models and artificial intelligence can at least start to measure the extent to which the topics researched and published by academics align with societal objectives. One framework gathering increasing popularity uses the UN Sustainable Development Goals (SDGs) — 17 objectives agreed by countries around the world for 2030, such as eradicating poverty, providing quality education and taking climate action. Within the 17 goals, there are 169 specific targets.
SDG-Impact Journal Rating and ChatSDG
David Steingard, associate professor of management at the Haub School of Business, Saint Joseph’s University, in the US, has developed an SDG-Impact Journal Rating to assess how far 100 leading business school academic journals match the goals. ChatSDG, the latest iteration, builds on the approach to show how far schools are aligning their research more broadly with the SDGs.
OpenAlex and Clarivate SDG Analytics
For its latest analysis on how many academics in each business school have published SDG-related articles, the FT has used OpenAlex, the open access data provider. It, in turn, takes into account a series of key SDG terms and definitions developed by the Aurora Universities network. These have been filtered to consider, as a measure of rigour, only articles published in the FT50 list of leading journals.
Clarivate, which operates the Web of Science database of journals, is one of several commercial providers also offering SDG analytics. As with OpenAlex, however, it focuses on tallying articles considered to have some SDG relevance, rather than trying to compare the depth of their analysis with other papers.
Limitations of Technology Tools
All these methodologies have their limitations, however. Mijnhardt’s list contained at least one historically focused article that had little obvious alignment with the SDGs: a paper from the Journal of Financial and Quantitative Analysis called “Access to finance and technological innovation: evidence from pre-civil war America”. Other articles are clearly relevant to the SDGs, but how far they are the most important contributions to scholarship and practice is a far more open debate.
Case Studies in Responsible Research
Supply Chains for Clothing
Meltem Denizel and Caroline Schumm from Iowa State University explored fashion in their 2023 Journal of the Operations Management article “Closed loop supply chains in apparel: current state and future directions”. The academics examined public sustainability reports to compare the low levels of recycling of clothing with the electronics industry, where moves towards a “circular economy” without requiring additional resource depletion is more advanced.
Marketplace Literacy
Madhubalan Viswanathan, professor of marketing at the College of Business Administration, Loyola Marymount University in the US, and co-authors including Arun Sreekumar at the Indian Institute of Management, Ahmedabad, published “Marketplace literacy as a pathway to a better world: evidence from field experiments in low-access subsistence marketplaces” in the Journal of Marketing in 2021.
Informal Water Vendors
Florence Dery, from Queen’s University, Canada, and Ophelia Soliku, from the SD Dombo University of Business and Integrated Development Studies in Ghana, plus colleagues studied private water sellers in low income regions in their paper “‘Quenching the thirst of others while suffering’: embodied experiences of water vendors in Ghana and Kenya”, published in Social Science & Medicine.
Clarivate SDG Pick
Nan Jia, at the University of Southern California, Xueming Luo, of Temple University in Philadelphia, and their fellow authors considered, “When and how artificial intelligence augments employee creativity” in the Academy of Management Journal. They studied a telemarketing company to find that AI assistance in generating sales leads on average increased employees’ creativity in answering customers’ questions during subsequent sales calls, leading to increased sales.
Conclusion
While measuring the impact of business school research is challenging, technology tools linked to large language models and artificial intelligence can at least start to measure the extent to which the topics researched and published by academics align with societal objectives. The SDGs offer a framework for assessing responsible research, and case studies demonstrate the potential for impactful research in areas such as supply chains, marketplace literacy, and informal water vendors.
FAQs
Q: What is the SDG-Impact Journal Rating?
A: The SDG-Impact Journal Rating is a framework developed by David Steingard to assess how far 100 leading business school academic journals match the UN Sustainable Development Goals (SDGs).
Q: What is ChatSDG?
A: ChatSDG is the latest iteration of the SDG-Impact Journal Rating, which builds on the approach to show how far schools are aligning their research more broadly with the SDGs.
Q: How do OpenAlex and Clarivate SDG Analytics work?
A: OpenAlex and Clarivate SDG Analytics use large language models and artificial intelligence to measure the extent to which articles published by academics align with the UN Sustainable Development Goals (SDGs). They focus on tallying articles considered to have some SDG relevance, rather than trying to compare the depth of their analysis with other papers.
