Deep neural networks have achieved state-of-the-art performance in various fields. Recent works observe that a class of widely used neural networks can be viewed as the Euler method of numerical discretization. In the numerical discretization perspective, strong stability preserving (SSP) methods are more advanced techniques than the Euler method and produce both accurate and stable solutions. Motivated by the SSP scheme, we proposed Strong Stability Preserving networks (SSP networks) which improve robustness against adversarial attacks. We demonstrate empirically that the proposed SSP networks improve the robustness against adversarial examples without any defensive methods. Further, our SSP networks are complementary with a state-of-the art adversarial training scheme. Lastly, we propose a novel network architecture that simultaneously improves adversarial robustness without sacrificing natural accuracy. Our results open up a way to study robust architectures of neural networks leveraging rich knowledge from numerical discretization literature.
