In this work, we focus on the construction of statistical surrogates for concurrent multiscale modeling in structures comprising nonlinear random materials. We specifically address the case of highly stochastic, highly nonlinear (and potentially, non-local) homogenized constitutive models, and use probabilistic learning to approximate the mapping between deformation and stress variables. The approximation problem is formulated using conditional statistics, and probabilistic learning on manifolds is used to draw samples of the nonlinear constitutive model at mesoscale. Two applications, relevant to inverse problem solving and forward propagation, are presented in the context of nonlinear elasticity. It is shown that the proposed framework enables accurate predictions, despite the small data regime and the very high levels of nonlinearity and stochasticity exhibited by the system.