Currently, state and local governments monitor the roadways trusted to their care with trained inspectors and specialized equipment, but recent computer-vision research demonstrates the existence of machine-learning algorithms capable of automatically detecting and classifying road surface defects using only consumer grade cameras. Many of these solutions, however, remain strictly focused on detecting and identifying cracks from extensively annotated input data, and cracks only represent one aspect of the roads overall distress. This work addresses the larger, qualitative label annotators provide the entire surface given all the subtle aberrations apparent in the input image. It further introduces a deep learning network, trained from these qualitative data, capable of automatically producing a quantitative measure for new input images reflecting their overall distress level.