Due to the many potential applications of Big Data, the expectations are high. However, there are some fundamental objections on the straightforward use of Big Data outcomes. In this paper, we take a philosophical view on the Big Data approach and discuss these objections. Formally, Big Data induces models from very large data sets, which are nevertheless incomplete. In many cases these data sets might be skewed as well. This gives rise to the question to what extent induced models represent the real world adequately, and therefore are sufficiently grounded to base new policies on. We argue that caution is needed in interpreting these models and well thought through strategies are required for using the models in practice in a responsible way. We discuss two strategies that may be used.