Which deep learning algorithm to use?

Artificial Intelligence Startups Technology Innovation

Posted by Enrico on Mar 29, 2017 3230

Unsupervised networks

Unsupervised learning can be used to extract patterns and features from unlabeled data.

  • Restricted Boltzmann machine (RBM)
    Takes a set of typically unlabeled inputs and, after encoding them, tries to contract them as accurately as possible. As a result, the net must decide which data feature is important, essentially acting as a feature extracting tool.
  • Autoencoder
    Autoencoders are very similar to RBMs. Trained with back-propagation using a matrix called “loss” as opposed to “cost”, it measures the amount of information that was lost when the net tried to reconstruct the input. These tools are extremely useful for dimensionality reduction.

Supervised networks

Supervised learning can be used where we have labeled data.

  • Text processing
    • Recursive Neural Tensor Network (RNTN)
      RNTNs are the best fit when one needs to discover the hierarchy of the structure of a set of data. They were first developed for sentiment analysis. The complexity of the network comes from the way data flow through the net. These networks are trained using back-propagation by comparing the predicted sentence structure with the actual sentence structure. They are used in natural language processes, to parse images, etc.
    • Recurrent Neural Network (RNN)
      These neural networks are used when data change over time.
      RNNs can receive a sequence of values as input and produce a sequence of values as output. This opens up a variety of applications, such as classifying videos frame by frame. They are typically used for prediction applications. Another example is speech recognition, which uses back-propagation for training. This runs into training issues.
  • Image recognition
    • Deep Belief Network (DBN)
      The training is performed between the visible layer and the closest hidden layer. The hidden layer will subsequently become the visible layer in the subsequent step of the forward-propagation and the result of the previous step will be its input. This is repeated until the output layer . DBN does not require much initial labeled data.
    • Convolutional Neural Network (CNN)
      A machine was able to beat humans at categorizing pictures for the first time using a CNN.
      Although they are very powerful, CNNs need a large number of labeled data to be successfully trained.
  • Object recognition
    • Recursive Neural Tensor Network (RNTN)
      See above.
    • Convolutional Neural Network (CNN)
      See above.
  • Speech recognition
    • Recurrent Neural Network (RNN)
      See above.

Share this post on social media:

Contact me if you'd like to become a contributor.

  • Thumb img 2844

    Enrico Tam

    MBA, PhD, tech entrepreneur, maker

    Hi, I’m Enrico and I started hacking at 9 years old back when it was Visual Basic. After trying to become a professional tennis player I somehow got entangled in a PhD in engineering, an MBA programme and a big consulting fir... (continued)

Join the discussion

Never miss a post!

I’m Enrico, I write to learn and to share my adventures :)

Don't hesitate to write to me on twitter!

Popular bloggers

Popular posts

See all posts

Cookies help us deliver our services. By using our services, you agree to our use of cookies.