A large number of algorithms have been proposed for automatic buried threat detection (BTD) in ground penetrating radar (GPR) data. Convolutional neural networks (CNNs) have recently achieved groundbreaking results on many recognition tasks. This success is due, in part, to their ability to automatically infer effective data representations (i.e., features) using training data. This capability however results in a high capacity model (i.e., many free parameters) that is difficult to train, and more prone to overfitting, than models employing hand-crafted feature designs. This drawback is pronounced when training data is relatively scarce, as is the case with GPR BTD. In this work we propose to combine the relative advantages of hand-crafted features, and CNNs, by constructing CNN architectures that closely emulate successful hand-crafted feature designs for GPR BTD. This makes it possible to apply supervised training to traditional hand-crafted features, allowing them to adapt to the unique characteristics of the GPR BTD problem. Simultaneously, this approach yields a much lower capacity CNN model that incorporates substantial prior research knowledge, making the model much easier to train. We demonstrate the feasibility and effectiveness of this approach by designing a "neural" implementation of the popular histogram of oriented gradient (HOG) feature. The resulting neural HOG (NHOG) implementation is much smaller and easier to train than standard CNN architectures, and achieves superior detection performance compared to the un-trained HOG feature. In theory, neural implementations can be developed for many existing successful GPR BTD algorithms, potentially yielding similar benefits.