There are scenarios where deep learning tasks demand combining the output or the intermediate results of separately trained neural networks. A few of these applications are Ensemble learning, learning from two different kinds of input for the same task, etc. There are two ways to combine feature vectors into a neural network.

One of the most common and easiest ways is to concatenate the feature vectors and train the network using this concatenated vector. A major drawback in this method is the length of the resultant vector which is the sum of individual vectors leading to a huge increase in dimensionality. Using higher dimension input features to train, networks are more susceptible to overfitting and hence will require increasing the training samples. Another method is to combine the two feature vectors into one by using averaging or weighted averaging.

To perform either operation, both vectors must be of the same dimension, this can be achieved by passing them through a layer of equal number of neurons before averaging. During weighted averaging the weights associated with each vector can be learned during the training process, this further provides additional information such as importance scores and measures of the contribution of individual feature vectors. The drawback of this approach is the loss of information that is caused as a result of averaging operations. The choice of combination layer strategy depends on the kind of problem, input datasets, and architectural and computation limitations.