Through their exposure to predictable environments, microbes have evolved specialized regulatory networks to anticipate change. For example, the marine cyanobacterium Prochlorococcus orchestrates the up- and down-regulation of most of its proteome over the daily light cycle - repairing damage from the day prior, redirecting metabolic fluxes and preparing for the next day. Currently, optimization approaches to modeling microbial metabolic dynamics are predominantly reflexive - they see only the current state of the environment - and are unable to reproduce sub-optimal behaviors that create fitness advantages over time. We developed a Reinforcement Learning framework for Flux Balance Analysis (RL-FBA), whereby an agent learns an optimal regulatory policy on metabolic fluxes through enzyme allocation, improving fitness outcomes through repeated training. The framework is also able to reproduce known regulatory dynamics through Imitation Learning, using quantitative proteomics and transcriptomics time-series data, and providing a benchmark for trained policy comparisons. I will present the current state of development of RL-FBA through demonstrations with a toy model as well as applications to data-driven genome-scale model simulations of the Prochlorococcus diel cycle. I will also discuss generalizations of Reinforcement Learning policies to inform proteome allocation models, which are today being implemented in ocean biogeochemical models.