1.4 KiB
Things I have learned about PyTorch and Neural networks.
Building models
All model building in Pytorch is based on the following three steps
- start by creating an object that extends the nn.Module base class
- define layers as class attributes (sequential wrapper for ease of use)
- implement the
.forward()method
Each layer is just a predefined 'function'. Really, they are objects that extend the nn.Module base class. Thus each NN can act as a layer in another NN. For example, I reimplemented an upscaling layer in BasicNeuralNet2. (I picked up a lot of this info here.)[https://deeplizard.com/learn/video/k4jY9L8H89U]
Also, neural networks can return more than just a single output as long as the loss function that is used for optimization can consume both of them. Thus I could write two separate neural networks (such as for launch and partials), and then write a third NN that binds the two together.
Notes on functions
ReLU is a linear rectifier, it does not have any training involved. This makes it good for working as a final cleanup of the launch function. This also makes it not so good for the partial derivatives.
Linear is a good but basic network type.
Upscaling allows you to create more features. Downscaling reduces the number of features (by throwing data away?). Instead of downscaling, use a linear function to change the dimensions.
Remaining Questions
- How do you set it up to run over a set of variables, i.e. batches?