Recent developments in neural networks (or deep learning) for visual recognition have attracted the interest of internet search engines and social media sites. This interest has been driven by a desire to efficiently analyze visual content in images to help generate searchable tags enabling automated classification of the images. The biological and biomedical science communities are rife with examples of research and clinical image data that needs to be manually classified, ranging from segmenting and annotating digital pathology images to tracking masses in digital breast tomosynthesis. A large percentage of time and effort of highly trained medical professionals is spent on manual or semi-automated identification of regions of interest. There is an urgent need to develop unsupervised or minimally supervised approaches to identifying various regions of interest within biomedical images and to the classification and generation of metadata for archived images. Besides the obvious advantages of saving time and effort, such approaches can enable the development of a biomedical visual search engine which will allow researchers and clinicians to scan through large datasets and find appropriate sets of images of interest.