I study computational and quantitative biology with a focus on network aging. This site is to serve as my note-book and to effectively communicate with my students and collaborators. Every now and then, a blog may be of interests to other researchers or teachers. Views in this blog are my own. All rights of research results and findings on this blog are reserved. See also http://youtube.com/c/hongqin
@hongqin

"Currently, the most popular models are trained end-
to-end in a supervised fashion, greatly simplifying
the training process. The most popular architectures
are convolutional neural networks (CNNs) and recur-
rent neural networks (RNNs). CNNs are currently
most widely used in (medical) image analysis, although
RNNs are gaining popularity. "

The second key difference between CNNs and MLPs,
is the typical incorporation of pooling layers in CNNs,
where pixel values of neighborhoods are aggregated using a permutation invariant function, typically the max
or mean operation. This induces a certain amount of
translation invariance and again reduces the amount of
parameters in the network. At the end of the convo-
lutional stream of the network, fully-connected layers
(i.e. regular neural network layers) are usually added,
where weights are no longer shared. Similar to MLPs,
a distribution over classes is generated by feeding the
activations in the final layer through a softmax function
and the network is trained using maximum likelihood.

https://en.wikipedia.org/wiki/Softmax_functionIn mathematics, the softmax function, ornormalized exponential function,[1]:198 is a generalization of the logistic function that "squashes" a K-dimensional vector {\displaystyle \mathbf {z} } of arbitrary real values to a K-dimensional vector {\displaystyle \sigma (\mathbf {z} )} of real values, where each entry is in the range (0, 1], and all the entries add up to 1.