Human visual system (HVS) can detect the target utilizing the contrast of the target and the surrounding background under a low SNR, it is not only sensitive to the local contrast in space domain, but also in time domain. The existing infrared small target detection methods based on HVS are mainly based on the local contrast in space domain, while the false alarm caused by the background clutter could be serious. In consideration of this problem, an infrared small target detection method based on visual spatio-temporal contrast measure (STLCM) is proposed. At the first stage, a temporal local contrast measure (TLCM) was presented. At the second stage, visual spatio-temporal contrast measure was derived by the fusion of the local contrast in time and space domain, then an adaptive threshold was adopted in the saliency map of STLCM to segment the target. Experimental results demonstrate that the proposed method is superior in compressing the false alarm and efficient in different conditions.