In order to meet the demand of measuring the phenotype of Chaetomium in edible mushroom factory production, we propose a deep learning-based algorithm for measuring the phenotype of Chaetomium and apply the algorithm to a virtual simulation scenario. The camera is used to detect the process of growing the tea mushroom in real time and to obtain the phenotypic parameters during the growth of the tea mushroom. At the same time, the growth regulation model of edible mushrooms was established by analyzing the influence of environmental factors on the growth of Chaffinella mushrooms at different stages of growth and development. This study will obtain the optimal environmental factors and phenotypic parameters for the growth and development of the mushroom at different stages, which will improve the productivity of the mushroom, reduce the labor cost and increase the yield of the mushroom.