Why do we get a three-dimensional output after a convolutional layer?

During a search for an optimal convolution kernel via gradient descent or some other method there must be at least one additional dimension to represent trials. It is most often one. If the input is in $\mathcal{R}^n$ space, then the output of the convolution operation is $\mathcal{R}^{n+1}$ space. However, this is not often (or ever) the output of the learning system using a convolutional layer, since the final layer in the conventional deep network designs used today is not the convolution layer.

In the case in this question, $m$ represents the number of discrete kernels tried, usually in rapid succession in most algorithms and hardware acceleration scenarios. It is using the results in this $\mathcal{R}^{n+1}$ space that the corrective mechanism requires to converge on an optimum efficiently.

By corrective mechanism is meant whatever corrects the assumptions made for the next set of kernels to be tried. This mechanism often involves gradient descent and back propagation in an artificial network design. It is the learning algorithm or, in the case of hardware acceleration, the learning circuit.

The value of $m$ is not arbitrary, but is largely problem dependent and based on hardware and execution environment. If $m$ is too large, then too much convolution work is performed before the correction is made. If $m$ is too small, then whatever efficiency is gained by grouping kernel tries together in time is lost. There may be a formula to find $m$, but it will be based on platform dependent metrics and can be found by trying several $m$ values and determining the $m$ providing the lowest convergence time.

The value of $m$ can also affect accuracy and reliability of convergence. Predicting this effect is not straightforward. Such prediction, which would allow the automated selection of hyper-parameters like $m$, is of interest to many researchers for obvious reasons. It has been and will probably continue to be an objective of AI development to remove the need for human intervention in AI system applications.

Welcome to ai.se.... Questions very similar to this have already been asked many times on this site. I suggest you go through some of them and then come back if you still require clarification. – DuttaA – 2019-08-16T06:14:39.483

Hi Prabu! Please, ask one question per post. I will edit your post to leave only the first question, otherwise, your post should be considered too broad, IMHO. Ask the other question in their separate posts. – nbro – 2019-08-16T08:12:20.120