Cathodoluminescence (CL) is defined as the luminescence from a material in response to excitation by incident electrons. As one of the non-invasive high-resolution detection methods, CL spectroscopy breaks the optical diffraction limit with precise control of the electron impact position, and it has realized electromagnetic field investigation at deep subwavelength scales and even revealed some quantum effects. Based on ultrafast electron microscopy and angle-resolved CL detection, a new spectroscopy characterization method has been established to achieve simultaneous space-angular-polarization-time-energy resolution for the study of material and optical-field dynamics. With the help of this unique approach, we successfully modulate diverse optical effects including chirality of electromagnetic modes, optical spin hall effect, valley polarization and so on. The analysis of the angular distribution of the light emission directly reflects optical mode symmetries and photonic band structures. Our approach not only provides a versatile platform to explore interaction mechanism among electrons, matter and light, but also opens new prospects at the interface of nanophotonics and quantum optics.