Step by Step Implementation: 3D Convolutional Neural Network in Keras
https://towardsdatascience.com/step-by-step-implementation-3d-convolutional-neural-network-in-keras-12efbdd7b130

Step by Step Implementation: 3D Convolutional Neural Network in Keras

One way to picture it is by using the following image:


How does 3d datas look like?

1- IRM data processing and therefore the inference

2- Autonomous vehicles

3- Distance estimation

Applications of 3D CNN

2d Maxpool Layers (2x2 filter) is about taking the maximum element of a small 2x2 square that we delimitate from the input.
Now in a 3d Maxpool (2x2x2), we look for the maximum element in a width 2 cube. This cube represents the space delimited by the 2x2x2 zone from the input

3d MaxPool Layers

A 2D convolution layer performs an entry per entry multiplication between the input and the filter, where the input and the filter are 2D matrices. 

In a 3D convolution layer, the same operation is performed on 3D inputs and the filters, such that:
- We multiply corresponding entries in the input and filter cubes and add up the results to determine the output entry.
- In addition to shifting (striding) the filter on two dimensions of the input matrix to perform the convolution operation, we also shift it in the third dimension.

For example, if we have a 4 x 4 x 4 input and a 3 x 3 x 3 filter, because we can shift the filter for only one entry per dimension, the output will be 2 x 2 x 2.

3D Convolution Layers

You are building a network layer whose input volume has size 32x32x32x16 (this volume has 16 channels), and applies convolutions with 32 filters of dimension 3x3x3 (no padding, stride 1).

You are working with 3D data. Find the resulting output volume using the information below.

A 3D CNN is similar to a 2D CNN except that it differs in (non-exhaustive listing):
1- 3D Convolution Layers

2- 3D MaxPool Layers

University of Michigan - Ann Arbor

Convolutional networks, also known as convolutional neural networks, or CNNs, are a specialized kind of neural network for processing data that has a known grid-like topology. Examples include time-series data, which can be thought of as a 1-D grid taking samples at regular time intervals, and image data, which can be thought of as a 2-D grid of pixels. 

The network employs a mathematical operation called convolution. Convolution is a specialized kind of linear operation. Convolutional networks are simply neural networks that use convolution in place of general matrix multiplication in at least one of their layers.

The CNN gets its power from the following concepts:
- Sparse connectivity
- Parameter sharing

Convolutional Neural Network (CNN)

A helpful website for understanding how CNN works:
https://towardsdatascience.com/a-comprehensive-guide-to-convolutional-neural-networks-the-eli5-way-3bd2b1164a53

CNN Reference

CNNs are currently being applied to several applications, including:

 - Computer vision—medical image analysis, image recognition, face detection and video classification
 - Natural Language Processing (NLP)—semantic parsing, sentence modeling and search query retrieval
 - Drug discovery—the discovery of chemical features and prediction of medicinal benefits

Applications of Convolutional Neural Networks

CNNs and RNNs are not mutually exclusive, as both can perform classification of image and text inputs, creating an opportunity to combine the two network types for increased effectiveness. This is especially true if the input to be classified is visually complex with added temporal characteristics that a CNN alone would be unable to process.

Typically, when these two network types are combined, sometimes referred to as a CRNN, inputs are first processed by CNN layers whose outputs are then fed to RNN layers. CNN Long Short-Term Memory (LSTM) architectures are particularly promising, as they facilitate analysis of inputs over longer periods than could be achieved with lower-level RNN architecture types.

Currently, these hybrid architectures are being explored for use in applications like video scene labeling, emotion detection or gesture recognition, video identification or gait recognition, and DNA sequence prediction.

Hybrid of Convolutional and Recurrent Neural Network

- Python: conv_forward
- TensorFlow: tf.nn.conv2d
- Keras: Conv2D
- Pytorch: torch.nn.Conv2d
- Caffe: message ConvolutionParameter
- Theano: theano.tensor.nnet.conv.conv2d

Methods to Calculate Convolution in Python

CNNs are made up of three-layer types—convolutional, pooling and fully-connected (FC).

In the convolutional layers, the input matrix is passed through a set of filters that output a feature map. This output is then sent to a pooling layer, which reduces the size of the feature map. This helps reduce the processing time by condensing the map to its most essential information.

The convolutional and pooling processes are repeated several times, with the number of repeats depending on the network, after which the condensed feature map outputs are sent to a series of FC layers. These FC layers then flatten the maps together and compare the probabilities of each feature occurring in conjunction with the others, until the best classification is determined.

Convolutional Neural Networks Architecture

3D Convolutional Neural Network

Visualizing and Understanding Convolutional Networks Paper

- Convolutional neural network can be used for obtaining high- dimensional images rather than being used in class prediction for logistic regression  or value prediction for regression problem

- Output of the CNN model can be a tensor S(i,j,k) which represent the pixel(j,k) of ith class, such a collaborative  effort can be used to precisely form a perfect image as the output.

Strategies to get output dimension same as the input are 

- Pooling is completely avoided or can use pooling with unit stride.

- Pixel-wise labelling involves initial guess of image pixels and refining this guess by interacting with neighboring pixels by parameter sharing.

Structured Output from CNN

A Convolutional Recurrent Neural Network (CRNN) is a combination of a Convolutional Neural Network (CNN) and a Recurrent Neural Network (RNN). 
-CNN are specialized networks for processing data that use a specialized, linear mathematical operation, also known as a convolution. 
-RNN is specialized for processing a sequence of inputs $x^{(1)}, . . . , x^{(τ)}$ and each time adds additional layers of comprehension on top of the previous inputs. 
-The goal of a CRNN is to take advantage of CNNs for local feature extraction and RNNs for temporal summarization of the extracted features.


Learn Before

Related

Learn After