Conference on Cognitive Computational Neuroscience

fMRI enables non-invasive neuroimaging with high spatial resolution, but analyzing fMRI’s hemodynamic data is challenging due to its complex relationship to the underlying neural activity. Deep learning’s power to find non-linear relationships makes it particularly suitable for fMRI analyses, with many successful applications in classification of resting-state data. However, few have explored deep learning’s potential to generate continuous cross-modal predictions. If deep learning can translate fMRI data to neurophysiological EEG, it could become a promising method for uncovering relationships between hemodynamic changes and neural activity. Here, we demonstrate a proof-of-concept that deep recurrent neural networks can predict sleep EEG delta power from resting-state fMRI on a datapoint-to-datapoint basis, even for out-of-sample subjects, with predictions primarily driven by cortical fMRI dynamics. Supporting the idea that these predictions leverage non-linear information, a cross-correlation analysis revealed that our model outperformed simple linear methods. These results highlight the potential of deep learning to identify complex relationships between hemodynamic fMRI and EEG neural activity that cannot be detected with traditional linear analyses.