Current Issues

Regular Industry Development Updates, Opinions and Talking Points relating to Manufacturing, the Supply Chain and Logistics.

Understanding the Power of Deep Learning: An Introductory Look at How this Technology Works

11-Dec-2018
Understanding the Power of Deep Learning: An Introductory Look at How this Technology Works
One reason why deep learning is a confusing concept to many is that it is often used alongside the terms machine learning (ML) and artificial intelligence (AI). Deep learning (DL) is a subset of ML, which is itself a subset of AI. A DL algorithm is able to learn hidden patterns from the data by itself, combine them together, and build much more efficient decision rules.

Deep learning works best as the amount of data scales, making it popular in industries that collect massive amounts of data. These industries include manufacturing, automotive, hospitality, healthcare, banking, agriculture, entertainment, IT/Security, retail, and supply chain and logistics.

The controversy surrounding deep learning (as well as surrounding AI) is the fear of the “black box.” That is, how can anyone base a service or product on deep learning and trust the decisions being made if no one knows how they’re being made?

A deeper look at deep learning
One of the key reasons deep learning is more powerful than classical machine learning is that it creates transferable solutions. Deep learning algorithms are able to create transferable solutions through neural networks: that is, layers of neurons/units. A neuron takes input and outputs a number that assigns the input to a class (group). The output is determined the way you would make a decision. A neuron similarly takes multiple inputs, each with a corresponding weight (importance). The inputs are passed through an activation function which gives the final output.

Deep learning problems boil down to classification - whether binary (e.g., is this image a cat, or not a cat?) or multiclass (e.g., is this image a cat, a dog, a bird, etc.). So finding the optimal features (variables) and parameters (weights) are key. A model can be built with a single layer of neurons, and adding layers lets the computer create more and more specific features that lead to a more complex final output.

Understanding Gradient Descent
Understanding gradient descent is helpful for understanding deep learning because it’s one of the most popular - if not the most popular - strategy for optimizing a model during training (that is, making sure it's “learning” correctly).

Remember that in deep learning, it's the algorithm that finds the features for the most accurate classification (instead of the human, as is the case in machine learning), so the computer needs a way to determine the optimal features and weights - that is, ones that lead to the most accurate final classification.

The Details of Gradient Descent
This happens through choosing the features and weights that minimize some error/cost function. The error/cost function is the sum of loss functions (predicted value of a point - actual value of a point) + a regularization term. The regularization term penalizes models with many features to prevent overfitting (being accurate for a specific dataset but failing to generalize).

To minimize our error function, we use gradient descent: the computer chooses certain parameters (features and weights) and takes the negative gradient (gradient is the rate of greatest increase, so the negative gradient is the rate of greatest decrease) of the error function until it finds the parameters that lead a gradient of 0 (corresponding to a minimum of the error function). It works like getting to the lowest point on a mountain as quickly as possible: you walk in the direction of steepest decrease until you hit a minimum. For example, here we keep adjusting the line until we have minimized the classification error (larger dots correspond to larger errors).

Types of Neural Networks
There are countless types of neural networks. Here is an overview of some of the most relevant types:
  • Feed Forward - Used in computer vision and speech recognition when classifying the target classes are complicated. Responsive to noisy data and easy to maintain. 
  • Radial Basis - Considers the distance of a point with respect to the center. Used for power restoration systems which are notoriously complicated. 
  • Kohonen - Recognizes patterns in data. Used medical analysis to cluster data into different categories ( a Kohonen network was able to classify patients with a diseased glomerular vs. a healthy one) 
  • Recurrent - Feeds the output of a layer back as input. Good for predicting the next word in a body of text, but harder to maintain. 
  • Modular - Collection of different networks work independently and contribute towards the final output. Increases computation speed (through the breakdown of a complicated computational process into simpler computations), but processing time is subject to the number of neurons. 

There is also the increasingly popular convolutional neural networks (CNN), which we discuss more in depth in this guide.

Real-Life Applications & The Future of Deep Learning
The future of deep learning is bright because of its open source community and accessible platforms. Increasingly, leading corporations such as Apple, Facebook, and Google, are making their technology accessible to the public. In the near future, deep learning will significantly improve voice command systems (such as Siri and Alexa), as well as healthcare and image identification.

Deep learning has applications across numerous industries, which is why experts think that this technology is the future of almost everything. There are truly cutting-edge deep learning technologies such as Google’s very human-like talking AI, a new theory that cracks the “black box” of deep learning, and various budding ideas like this one about why human forgetting might be the key to AI.

The new theory is especially important given the uptick in data regulation worldwide. General Data Protection Regulation (GDPR), especially, has made global companies think about how and where to use deep learning.

For those who want to know more about how deep learning works, download the whitepaper An Introduction to Deep Learning: Understanding The Basics of How (and Why) it Works.

Stephen Cameron
Business Development Director, SWRnewstar
Primark's move to 100 per cent sustainably sourced cotton should be commended
Marc Corriveau
Account Manager, Loftware
Changes in packaging result in ROI for F&B products
John Perry
Managing Director, SCALA
Why an extension is best for British businesses
David Jinks MILT
Head of Consumer Research, ParcelHero
Booking a courier? No printer required, says ParcelCompare
Graham Parker
CEO, Gravity Supply Chain Solutions
Mitigating The Risks Of Trade Wars And Tariffs
Sam Ireland
International Marketing Manager, Loftware
How Brexit Will Impact Supply Chain Labeling?
Sid Holian
Managing Director, Bis Henderson Consulting
Three critical steps to a hi-tech, agile supply chain
Manu Tyagi
Associate Partner, Retail and Consumer Goods, Infosys Consulting
M&S and Ocado have signed a £750 million deal to take M&S groceries into the home delivery market
Olivier Frère
Serialisaton Expert, Zetes
Understanding the Tobacco Products Directive
Paul Heiden
Senior Vice President of Product Management, Ultimaker
6 overlooked benefits of 3D printing for your supply chain
Dave Locke
EMEA Chief Technology Officer, WWT
Merged pharmaceutical firms seek fast-track route to savings
Johannes Panzer
Head of Industry Strategy for Ecommerce, Descartes
How to make international ecommerce transparent, efficient and cost-effective
Craig Summers
UK Managing Director, Manhattan Associates
Putting the WOW in the checkout experience
Richard Parfect
Fund Manager, Seneca Investment Managers
The cessation of Airbus A380 production
John Perry
Managing Director, SCALA
What next for British business?
Chris Jones
EVP Marketing & Services, Descartes
5 Key Logistics Trends and Technology Implications for 2019
Craig Summers
UK Managing Director, Manhattan Associates
What is the Checkout of the Future?
Andres Richter
CEO, Priority Software
UK Manufacturing: The Productivity Conundrum
Martin Meacock
Director, Product Management, Descartes
Brexit: only 10 weeks to implement if changes are ultimately needed
John Perry
Managing Director, SCALA
What does May's defeat mean for the supply chain?
Martin Meacock
Director, Product Management, Descartes
Get Ready for CDS
Don Brenchley
Director Industry Strategy, LLamasoft EMEA
Don't Let Soft Skills Be Forgotten In Your Supply Chain
Robert Broström
Director Of Service And Support, Handheld Group
Handheld Promotes Robert Broström To Director Of Service And Support
Dan Willmer
Professional Services Leader, MercuryGate
MercuryGate Hires Dan Willmer as New Professional Services Leader
Bob Brown
Manufacturing Industry Specialist, Loftware
Ideal Enterprise Labeling Systems Include Support and Service
Jason Chester
Director of Channel Programs, InfinityQS
2019 will see half of manufacturers invest in data collection technologies
Craig Summers
UK Managing Director, Manhattan Associates
Integrating Man And Machine: A Fresh Approach For Modern Grocery Warehouses
Dustin Maxey
Director of Product Marketing, Ping Identity
Security from the web to the loading bay
David Jinks MILT
Head of Consumer Research, Parcelhero
Parcel Senders Must Face Up to Their Christmas Duties
David Jonker
Vice President, Thought Leadership, SAP
Your Customer, the AI
Kelly Feehan
Service Director, CABA
Workplace wellbeing predictions for 2019
Mike Thornton
Head of Manufacturing, RSM
Brexit uncertainty bites for manufacturers
John Perry
Managing Director, SCALA
Stockpiling decision should not be taken lightly
Tony Dobson
Managing Director, Snapfulfil
Brexit Uncertainty Boosts Sales For Synergy
David Feakins
Founder and CEO, Modus Brands
How has technology transformed the distribution industry?
Nicole Hardin
Director of Product Management, Sage
The business case for reducing food waste in the supply chain