Deep learning, and deep neural networks (DNNs) in particular, has recently demonstrated impressive results for the design of artificial electromagnetic materials (AEMs). In this context AEMs are materials whose properties derive primarily from their structure, and includes metamaterials, metasurfaces, plasmonics, and photonic crystals. We discuss two major paradigms that are commonly employed for AEM design: forward design and inverse design. In each case we formulate the design problem, describe major fundamental challenges of design within that paradigm, and how DNNs have recently been used to overcome these challenges. We also discuss the DNN models typically employed for forward and inverse design, respectively, and a comparison of their performance and limitations is discussed. We conclude by detailing some important outstanding issues of DNN design of AEMs and present an outlook of this exciting field.