Ambulance offload delays (AODs) arise when emergency medical service (EMS) personnel are unable to promptly transfer a patient to an overwhelmed emergency department. This situation impacts both the patient, by delaying necessary care, and the EMS system, as it prevents the ambulance from responding to new emergencies. This study aims to establish a real-time, multi-priority patient transfer policy to reduce these delays. We modeled the patient transfer problem as a Markov decision process based on post-decision states, and developed approximate dynamic programming approaches to overcome the curse of dimensionality. We derived a lower bound for the optimal solution using information relaxation and formulated each relaxed inner problem as a mixed integer quadratic program. We evaluated the proposed method using real EMS data from St. Paul, Minnesota. For patients, our policy reduced hospital wait times by roughly 80\%. For hospitals, our policies helped alleviate congestion, and we found that even partial compatibility leads to notable system-wide improvements. For EMS, ambulance utilization was reduced by more than 30\%, and ambulances could handle more calls per day, with a reduction in lost calls exceeding 80\%.