Because many biological processes related to the dynamics of infectious diseases are caused by complex interactions between the environment, the host(s) and the agent(s), the necessity to address the methodological implications of this inherent complexity has recently emerged in epidemiology. Most epidemiologists now acknowledge that most human infectious diseases are likely to have complex dynamics. However, this knowledge still percolates with difficulty in their statistical "modus operandi". Indeed, for the study of complex systems, the traditional first-line statistical toolbox of epidemiologists (mainly built around the Generalized Linear Model family), despite its undeniable practicality and robustness, has structural limitations deprecating its usefulness. Three major sources of complexity neglected or not taken into account by this first-line statistical toolbox and having deep statistical implications are the multi-level organization of data, the non-linear relationships between variables and the complex interactions between variables. Three promising candidates to incorporate along with traditional tools for a new first-line statistical toolbox more suitable to apprehend these sources of complexity are the generalized linear mixed models, the generalized additive models, and the structural equation models. The aforementioned methodologies have the advantage to be generalizations of GLM models and are relatively easy to implement. Their assimilation and implementation would thus be greatly facilitated for epidemiologists. More globally, this text underlines that an improved use of other methods as such described here compared to traditional ones has to be performed to better understand the complexity challenging epidemiologists every day. This is particularly true in the field of infectious diseases for which major public health challenges will have to be addressed in the coming decades.