Transfer Learning - Self-taught Learning

Subscribe Send me a message home page tags


Transfer Learning

The objective of transfer learning is to transfer the learning experience from one domain to another and it usually involve at least two tasks. The task the allow us to get learn experience is called source task and the task that we want to apply the learned experience to is called target task.

A learning task consists of four components:

There are different techniques of transfer learning:

They correspond to the following learning scenarios

In this post, we will talk about self-taught learning.

Self-taught Leaning

Self-taught learning can be used when we have large volume of source data without label and relatively small number of labeled target data.

When we train a model (for the target task), the model needs to first "understand" the data. In technical term, the model needs to learn the latent representation of the input data. This latent representation is then used to perform tasks such as classification. Notice that learning the latent representation of input data does not require labels. There are many unsupervised learning algorithms such as Autoencoder that are designed for this task. The idea of self-taught learning is to first apply a unsupervised learning algorithms to the source data that don't have label and get the latent representation of the input data and then fine tune the network to perform the target task.


Step 1: Use unsupervised learning algorithms to learn the latent representation of the source data.
self-taught-learning-1.png
Step 2: Fine tune the model using target data
self-taught-learning-2.png

----- END -----

Welcome to join reddit self-learning community.
Send me a message Subscribe to blog updates

Want some fun stuff?

/static/shopping_demo.png