384 / 2024-04-24 21:33:08

To be a superstar or not: Assessing the influence of experience in innovation contests

Innovation contests,Superstars,Experiential learning,Hidden Markov model,Machine learning

Innovation contests have emerged as a new crowd-based operational practice for sourcing managers to expand firms’ innovation boundaries, tap into the creativity and wisdom outside their organizations, and accelerate new product development by leveraging the competition within the crowd. As this practice becomes more and more popular in firms’ innovation processes, managers tend to rely on the participation of superstars, solvers who have exceptional performance records in a crowdsourcing platform, to effectively solve crowdsourced tasks and problems. Empirical evidence from multiple crowdsourcing platforms, such as Topcoder, Kaggle, and InnoCentive, shows that superstars are in the great minority and some of them even remain inactive from time to time. Statistics also show that it takes a long time for a solver to grow to be a superstar in the context of innovation contests. As such, managers at crowdsourcing platforms face a practical challenge on how to track the growth process of solvers and help them grow to be superstars.

Although the growth of solvers remains an important issue for managers, academic research on this issue seems to be underdeveloped because the current innovation contest literature mainly focuses on examining the performance implications of superstars (Archak, 2010; Zhang et al., 2019) and their influence of other solvers’ participation (Bockstedt et al., 2022; Liu et al., 2024). Empirical studies show that solvers’ participation experience is significantly related to individuals’ performance in innovation contests (Menon et al., 2022). However, research that examines whether and how solvers’ participation experience affects their chance to be superstars is in great scarcity. According to experiential learning theory (Argote & Miron-Spektor, 2011), we believe that solvers can improve their abilities and performance by learning from various channels, such as direct experience, indirect experience, and multitasking status, and thereby increase their probability of becoming a superstar. As such, we propose that solvers’ direct and indirect participation experience and multitasking are positively related to the level and duration of their problem-solving abilities, while solvers’ experience interval is negatively related to the level and duration of their problem-solving abilities because of the decay of knowledge and experience. Meanwhile, we also propose that the level and duration of solvers’ problem-solving abilities positively affect their performance in a competition and the chance to be a superstar.

We use secondary data from Kaggle, one of the largest crowdsourcing platforms that organize data science contests to test our hypotheses. Our data includes 235 competition tasks and 34,146 pairs of solver-contest observations that occurred between April 2010 and August 2023. Our reduced-form regression analyses show that solvers’ direct experience, indirect experience, and multitasking status have a significant positive relationship with the solution quality and the probability of becoming a superstar. However, the longer the experience interval is, the lower the solution quality is and the higher the probability of becoming a superstar. We then construct a Variable-Duration Hidden Markov Model (VD-HMM), which includes the two latent ability variables (i.e., the level and the duration of solvers’ problem-solving abilities) to explore the empirical relationships in our research framework, which include (1) the impact of solver ability on performance; (2) the impact of different dimensions of solver experience, contest settings on solvers’ abilities; (3) how the ability levels of the solver dynamically evolve; (4) whether the solver ability can predict the probability to become the superstars.

We then use the Markov Chain Monte Carlo (i.e., MCMC) method based on the Kaggle competition scenario to estimate the parameters for the level and duration of solvers’ problem-solving abilities. Empirical results suggest that: (1) solvers have three ability levels that can be identified: high, medium, and low; (2) different experience dimensions have divergent marginal effects on solvers with different ability levels: for solvers with high-level abilities and medium-level abilities, three dimensions of experience (i.e., direct, indirect, multitasking) are beneficial for them to maintain their current ability levels. In particular, the indirect experience seems to have the greatest effect. However, for solvers with a low ability level, direct experience and multitasking are more helpful in improving their ability levels, but indirect experience and experience intervals will increase the probability of maintaining a low ability level; (3) the ability level of the solvers is extremely sticky, and the probability of participants transitioning to higher abilities is higher than the probability of transitioning to lower abilities, indicating that solvers in the Kaggle community can obtain sustained learning effects; (4) high and medium ability levels have a significant positive impact on the probability of becoming a superstar, while low ability levels have a significant negative impact.

Our research makes several meaningful contributions to the innovation contest literature. First, this study examines the impact of different dimensions of prior experience on solver ability, which enriches the shortcomings of existing research and effectively manages superstars. Second, this study extends the assumption of individual static ability and can precisely track the dynamics of solvers’ ability to provide some insights into customized strategies for crowdsourcing platform managers. Finally, this research provides a tool for crowdsourcing platforms to track the dynamic ability of solvers to improve solvers’ growth and overall sustainable development of the solver community nested in crowdsourcing platforms.