Ideation with GenAI in Organizations: A Large-scale Field Study with Sam’s Club

Abstract:

Ideations is a critical driver of organizational success, serving as the foundation for innovation, problem-solving, and competitive advantage. The advent of Generative AI (GenAI) is fundamentally transforming organizational ideation processes, particularly in how firms generate and evaluate new ideas. While GenAI has democratized idea generation by providing powerful language models capable of producing diverse and high-quality ideas, the role of managers has evolved from idea generators to idea evaluators. With increasing adoption of GenAI in organization ideation, we examine its impact on the sales performance in organizations with a typical hierarchal structure consisting of headquarter top-level managers and local branch frontline managers. Our research identifies two distinct managerial levels in the idea evaluation process: top-level managers with broad strategic knowledge who ensure alignment with organizational goals, and frontline managers with specialized task-specific knowledge who assess practical feasibility. Our study is instantiated on a product cross-selling ideation task at Sam’s Club warehouse stores in China. The results show a 29.8% increase in sales after GenAI adoption. Interestingly, frontline managers using GenAI ideation significantly outperform top-level managers (54% vs. 16% in sales improvement), and their cross-selling ideas receive higher panel evaluations across most metrics except for innovativeness. This suggests that as GenAI democratizes idea generation at minimal cost, the traditional advantage of broad strategic knowledge becomes less critical while local, operational knowledge becomes increasingly valuable for effective implementation. The findings contribute to our understanding of how AI technologies reshape organizational innovation processes, while providing practical insights for firms adopting GenAI for ideation, suggesting that successful integration requires balancing AI capabilities with appropriate managerial oversight and local operational knowledge.

Bio: 

Jianwei Liu is an associate professor in the Department of Management Science and Engineering, School of Economics and Management at Dalian University of Technology. He received his Ph.D. degree from the same department at Harbin Institute of Technology in 2019. His research interests include platform economy, generative AI customization and application, and online community.