Breaking News: Grepper is joining You.com. Read the official announcement!

neural network

Add Answer

Innocent Iguana answered on June 22, 2021 Popularity 8/10 Helpfulness 4/10

answer neural network

related how does a neural network work

related neural network

neural network

Comment

- Representation learning : The subsequent layers build increasingly sophisticated representations of raw data (first layers find simple patterns like lines, later layers combine those simple patterns to find more complex geometric shapes)
- partially replace the need for feature engineering
- In linear regression or other simple models, we assume the independent variables have minimal or no interaction with each other.
- In deep learning, we take for granted that there are always some level of interaction among independent variables.
    - The nodes in the hidden layers are the aggregation of information of independent variables that adds the models ability to capture the interaction
    - The more nodes we have, the more interaction we capture
- Forward propagation:
    - Used to calculate values of hidden layers' nodes
    - Creates a nework of nodes that process data by passing the data from input to output layer for prediction
    - All nodes from previous layer is connected to each node of the next layer with a line
    - The line has a weight that indicates how the previous nodes affect the next layer's hidden node
    - The value of the hidden node is the added multiplication of weights and previous node's values
    - Activation function is used for transformation of calculated value
- Activation function:
    - Allows the model to capture non-linear pattern (Anything that is not a straight line)
    - It takes the value of the hidden layer's node and transforms that value into a meaningful value. (eg: tanh, relu)
- Loss function:
    - Aggregates errors in predictions from many data points into single score
    - Measure of model's predictive performance (lower loss = better model)
    - Goal : Find the weights that give the lowest value for the loss function (gradient descent)
    - We use slope to find the minimum loss. (As long as slope is not close to zero, keep going downhill or opposite to slope)
    - learning rate : how much do we update the weight after each step (Update each weight by subtracting learning rate * slope from current value)
- Back propagation:
    - Change in adjusted weights (value of edges) improve model performance from initial random assigned weights
- which activation function to use:
	- ReLU are a good first choice
	- Sigmoids not recommended for deep models
	- Tune with experimentation
- multiclass without softmax?
	- use 'sigmoid' activation function in the output layer. This will act as one-vs-rest classification (2-class classification) problem
	- Since this becomes a two-class classification problem now, use 'binary_crossentropy' as the loss function
	- You can use 'adam' as optimizer
	- This approach is preferable for multi-label classification problem instead of multi-class classification problem
- Batch:
	- batch = training set
	- mini-batch = subset of training set
	- for each mini-batch, weight in each epoch is updated once
	- helps faster training, less ram usage
	- noise helps to reduce error and escape local minima
	- Disadvantages: more iterations, needs adjusted batch size
- Batch normalization:
	- helps avoid problems of activation functions and gradients
	- makes sure inputs of the next layers are normalized
	- Improves gradient .ow
	- Allows higher learning rates
	- Reduces dependence on weight initializations
	- Acts as an unintended form of regularization
	- Limits internal covariate shift
- Tensors:
	- an array of numbers
	- consists of n-dimensions
	- eg : a rank-2 tensor means a 2-D array

Popularity 8/10 Helpfulness 4/10 Language whatever

Source: Grepper

Tags: neural-network whatever

Link to this answer
Share Copy Link

Contributed on Apr 15 2024

Innocent Iguana

0 Answers Avg Quality 2/10

Closely Related Answers

how does a neural network work

Comment

HOW A NEURAL NET WORKS STEP BY STEP:

There is a matrix of nodes like this:

0 0 0
0 0 0
0 0 0

Inputs are fed in from the left. like this:

(.1) 0 0
(.24) 0 0		(inputs in parenthesis)
(0) 0 0

the next nodes data is calculated by multiplying A weight to each node 
in the layer before and then adding bias. And then putting in an 
activation function. Like this:

.1  ----(weight: 1)--> .1
.24 ----(weight: .5)--> .12  ----> .1 + .12 + 0 = .22 + bias
0   ----(weight: 2)-->  0


Each node has a bias, for this node lets say its .3

.22 + .3(bias) = .52

After all that you put it in an activation function.

tanh(.52) ---> data for the next node

Everything i showed above was just the caclulation for the 
first node in the second layer though so the matrix
would look like this
after all that.

.1 tanh(.52) 0
.24 0 0
0 0 0

Then you keep doing that for each layer until you reach the end.

Hope this helped?

(Q A)

Does each node have a weight?

No. Each connection has a weight. Or i guess you could say each node
has lots of weights each corresponding to a node in the last layer.


Does each node have a bias?
Yes each node has only one bias.

Popularity 10/10 Helpfulness 2/10 Language python

Source: Grepper

Tags: neural-network neur

Link to this answer
Share Copy Link

Contributed on Jun 22 2021

Aggressive Anaconda

0 Answers Avg Quality 2/10

neural network

Comment

neural network model that recognizes handwritten digits from MNIST dataset

import tensorflow as tf

mnist = tf.keras.datasets.mnist
(X_train, y_train), (X_test, y_test) = mnist.load_data()
X_train, X_test = X_train / 255.0, X_test / 255.0
model = tf.keras.models.Sequential([
  tf.keras.layers.Flatten(input_shape=(28, 28)),
  tf.keras.layers.Dense(128, activation='relu'),
  tf.keras.layers.Dropout(0.2),
  tf.keras.layers.Dense(10)
])
predictions = model(X_train[:1]).numpy()
tf.nn.softmax(predictions).numpy()

Popularity 10/10 Helpfulness 1/10 Language whatever

Source: python.plainenglish.io

Tags: neural-network neur

Link to this answer
Share Copy Link

Contributed on Feb 24 2023

Outrageous Oryx

0 Answers Avg Quality 2/10

neural network

Contents

More Related Answers

neural network

Closely Related Answers

how does a neural network work

neural network

Grepper

Documentation

Social

Legal

Contact

Oops, You will need to install Grepper and log-in to perform this action.