4 Transfer Learning

Transfer learning is one of the most useful and underrated deep learning tools. It allows us to take a model which was trained on a large data set, and “fine tune” it (more on what that means later) to work well for our specific use-case. It gets its name from the idea of learned information being transferred from one use case to another.

You might be asking yourself, why is this so powerful? Well, during the learning process, a deep learning model has to figure out things like what an edge looks like in the case of computer vision. This might seem like an abstract idea but it is a critical step for the model to learn how to distinguish between multiple objects. When we use transfer learning, many of these ideas have already been learned by the existing model we are starting with. This means that we do not need to spend as long training since already know some info about objects, even though the pre-trained model might never has seen data similar to the data you will be feeding into it.

Here is a simple example to try and illustrate why transfer learning works so well: imagine you are trying to teach someone what a car is. This person has never seen a car nor knows what it does. This person has seen a bicycle before and in fact uses one everyday. You can now explain to the person what a car is in terms of how it relates to a bike. The transfer learning process is much like this. Use the existing info that the model has learned and build off of that for some specific situation.

CC BY-NC-SA 4.0 Logan Kilpatrick