Moving a Neuron from an Object to a 1D array. Notes

[hbyte]

In order to have a neural network running on a gpu you need to create a set of 1d arrays that usually hold data specific to each layer. Formerly Neural Networks were written inside FOR loops. Like this one:

Code: Select all

double sum=0;
double x=0;
    for(int i = 0; i < numInputs; i++){

        for(int k = 0; k < numHidden; k++){
   
       x =   net->errH1[k] * LR_IH;
       x = x * net->InputsNow[i];
      double weightChange = x;
       weightChange = (weightChange + Mntem*net->predltIH[i][k] - dcay*net->prewgtIH[i][k])/t_;
            net->weightsIH[i][k] = net->weightsIH[i][k] + weightChange;
     
     if(reg==1){       
       if (net->weightsIH[i][k] < -0.5){
            net->weightsIH[i][k] = -0.5;
        }else if (net->weightsIH[i][k] > 0.5){
            net->weightsIH[i][k] = 0.5;
        }
}
       
   net->prewgtIH[i][k] = net->weightsIH[i][k];
   net->predltIH[i][k] = weightChange;
   
   }
    }

In a neuron centered design I created a neural network in which every neuron contained its own data inside a c++ object class.

https://neuron-centered-architecture-mlp.blogspot.com/2016/11/my-neuron-centered-mlp-is-now.html

This can still be acheived using 1d arrays by using a LUT (LookupTable) that points to such things as neurons, their weights - inummerated, and their activation functions. This LUT can be an object retrieving data from the 1d arrays that encode the whole neural network suitable for a GPU.