While I do spend lots of time pushing the idea of linear approximations on my students, we do need nonlinear functions to describe most interesting physical phenomena.

This article reviews work on the planning of experiments with response functions nonlinear in some parameters. Apart from older work on dilution series experiments and quantal bioassays, this field is ...

For this, you also need non-linear thresholding functions that adjust the weights between the layers of neurons.