Many physics models feature terms that are either partially unknown or too expensive to simulate. Discovering effective equations that represent such terms is a fundamental challenge in computational science. Multi-scale models are a prominent example: the large-scale behaviour is of main interest, but this cannot be obtained without resolving the fine scales. A well-known example occurs in climate models, which rely on the effect of clouds for accurate forecasts, but simulating clouds individually is computationally intractable. We propose a new software framework to extend generic physics models with data-driven neural networks (NNs) that represent the effect of small scales on large scales. The framework will use differentiable programming, allowing to couple multi-scale models and NNs while embedded in a learning environment. We test our framework on turbulent fluid flows. In particular, we develop new differentiable wind-turbine wake models, to be used for optimal control of wind farms.