Object categorization and scene understanding have long been a central goal of computer vision research. Changes in lighting, viewpoint, and pose, as well as intra-class differences, lead to enormous appearance variation, making the problem highly challenging. While advances in machine learning and image feature representations have led to great progress in 2D pattern recognition approaches, recent work suggests that large gains can be made by acknowledging that objects live in a physical, three-dimensional world. When modeling scenes, objects and their relations in 3D, we must answer several fundamental questions. How can we effectively learn 3D object representations from images or video? What level of supervision is required? How can we infer spatial knowledge of the scene and use it to aid in recognition? How can both depth sensors and RGB data be used to enable more descriptive representations for scenes and objects? After the success of the 3dRR workshop during the past ICCV07, ICCV09, and ICCV11, we are pleased to organize a fourth edition of 3dRR in conjunction with ICCV 2013. This workshop would represent a great opportunity to bring together experts from multiple areas of computer vision and provide an arena for stimulating debate. We believe the complementary viewpoint offered by studies in human vision can provide additional insight on this fundamental problem.