In [ ]:
# Install necessary libraries if not already present in Colab environment
!pip install numpy tensorflow matplotlib scikit-learn
Requirement already satisfied: numpy in /usr/local/lib/python3.11/dist-packages (2.0.2) Requirement already satisfied: tensorflow in /usr/local/lib/python3.11/dist-packages (2.18.0) Requirement already satisfied: matplotlib in /usr/local/lib/python3.11/dist-packages (3.10.0) Requirement already satisfied: scikit-learn in /usr/local/lib/python3.11/dist-packages (1.6.1) Requirement already satisfied: absl-py>=1.0.0 in /usr/local/lib/python3.11/dist-packages (from tensorflow) (1.4.0) Requirement already satisfied: astunparse>=1.6.0 in /usr/local/lib/python3.11/dist-packages (from tensorflow) (1.6.3) Requirement already satisfied: flatbuffers>=24.3.25 in /usr/local/lib/python3.11/dist-packages (from tensorflow) (25.2.10) Requirement already satisfied: gast!=0.5.0,!=0.5.1,!=0.5.2,>=0.2.1 in /usr/local/lib/python3.11/dist-packages (from tensorflow) (0.6.0) Requirement already satisfied: google-pasta>=0.1.1 in /usr/local/lib/python3.11/dist-packages (from tensorflow) (0.2.0) Requirement already satisfied: libclang>=13.0.0 in /usr/local/lib/python3.11/dist-packages (from tensorflow) (18.1.1) Requirement already satisfied: opt-einsum>=2.3.2 in /usr/local/lib/python3.11/dist-packages (from tensorflow) (3.4.0) Requirement already satisfied: packaging in /usr/local/lib/python3.11/dist-packages (from tensorflow) (24.2) Requirement already satisfied: protobuf!=4.21.0,!=4.21.1,!=4.21.2,!=4.21.3,!=4.21.4,!=4.21.5,<6.0.0dev,>=3.20.3 in /usr/local/lib/python3.11/dist-packages (from tensorflow) (5.29.5) Requirement already satisfied: requests<3,>=2.21.0 in /usr/local/lib/python3.11/dist-packages (from tensorflow) (2.32.3) Requirement already satisfied: setuptools in /usr/local/lib/python3.11/dist-packages (from tensorflow) (75.2.0) Requirement already satisfied: six>=1.12.0 in /usr/local/lib/python3.11/dist-packages (from tensorflow) (1.17.0) Requirement already satisfied: termcolor>=1.1.0 in /usr/local/lib/python3.11/dist-packages (from tensorflow) (3.1.0) Requirement already satisfied: typing-extensions>=3.6.6 in /usr/local/lib/python3.11/dist-packages (from tensorflow) (4.14.1) Requirement already satisfied: wrapt>=1.11.0 in /usr/local/lib/python3.11/dist-packages (from tensorflow) (1.17.2) Requirement already satisfied: grpcio<2.0,>=1.24.3 in /usr/local/lib/python3.11/dist-packages (from tensorflow) (1.73.1) Requirement already satisfied: tensorboard<2.19,>=2.18 in /usr/local/lib/python3.11/dist-packages (from tensorflow) (2.18.0) Requirement already satisfied: keras>=3.5.0 in /usr/local/lib/python3.11/dist-packages (from tensorflow) (3.8.0) Requirement already satisfied: h5py>=3.11.0 in /usr/local/lib/python3.11/dist-packages (from tensorflow) (3.14.0) Requirement already satisfied: ml-dtypes<0.5.0,>=0.4.0 in /usr/local/lib/python3.11/dist-packages (from tensorflow) (0.4.1) Requirement already satisfied: tensorflow-io-gcs-filesystem>=0.23.1 in /usr/local/lib/python3.11/dist-packages (from tensorflow) (0.37.1) Requirement already satisfied: contourpy>=1.0.1 in /usr/local/lib/python3.11/dist-packages (from matplotlib) (1.3.2) Requirement already satisfied: cycler>=0.10 in /usr/local/lib/python3.11/dist-packages (from matplotlib) (0.12.1) Requirement already satisfied: fonttools>=4.22.0 in /usr/local/lib/python3.11/dist-packages (from matplotlib) (4.58.5) Requirement already satisfied: kiwisolver>=1.3.1 in /usr/local/lib/python3.11/dist-packages (from matplotlib) (1.4.8) Requirement already satisfied: pillow>=8 in /usr/local/lib/python3.11/dist-packages (from matplotlib) (11.2.1) Requirement already satisfied: pyparsing>=2.3.1 in /usr/local/lib/python3.11/dist-packages (from matplotlib) (3.2.3) Requirement already satisfied: python-dateutil>=2.7 in /usr/local/lib/python3.11/dist-packages (from matplotlib) (2.9.0.post0) Requirement already satisfied: scipy>=1.6.0 in /usr/local/lib/python3.11/dist-packages (from scikit-learn) (1.15.3) Requirement already satisfied: joblib>=1.2.0 in /usr/local/lib/python3.11/dist-packages (from scikit-learn) (1.5.1) Requirement already satisfied: threadpoolctl>=3.1.0 in /usr/local/lib/python3.11/dist-packages (from scikit-learn) (3.6.0) Requirement already satisfied: wheel<1.0,>=0.23.0 in /usr/local/lib/python3.11/dist-packages (from astunparse>=1.6.0->tensorflow) (0.45.1) Requirement already satisfied: rich in /usr/local/lib/python3.11/dist-packages (from keras>=3.5.0->tensorflow) (13.9.4) Requirement already satisfied: namex in /usr/local/lib/python3.11/dist-packages (from keras>=3.5.0->tensorflow) (0.1.0) Requirement already satisfied: optree in /usr/local/lib/python3.11/dist-packages (from keras>=3.5.0->tensorflow) (0.16.0) Requirement already satisfied: charset-normalizer<4,>=2 in /usr/local/lib/python3.11/dist-packages (from requests<3,>=2.21.0->tensorflow) (3.4.2) Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.11/dist-packages (from requests<3,>=2.21.0->tensorflow) (3.10) Requirement already satisfied: urllib3<3,>=1.21.1 in /usr/local/lib/python3.11/dist-packages (from requests<3,>=2.21.0->tensorflow) (2.4.0) Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.11/dist-packages (from requests<3,>=2.21.0->tensorflow) (2025.6.15) Requirement already satisfied: markdown>=2.6.8 in /usr/local/lib/python3.11/dist-packages (from tensorboard<2.19,>=2.18->tensorflow) (3.8.2) Requirement already satisfied: tensorboard-data-server<0.8.0,>=0.7.0 in /usr/local/lib/python3.11/dist-packages (from tensorboard<2.19,>=2.18->tensorflow) (0.7.2) Requirement already satisfied: werkzeug>=1.0.1 in /usr/local/lib/python3.11/dist-packages (from tensorboard<2.19,>=2.18->tensorflow) (3.1.3) Requirement already satisfied: MarkupSafe>=2.1.1 in /usr/local/lib/python3.11/dist-packages (from werkzeug>=1.0.1->tensorboard<2.19,>=2.18->tensorflow) (3.0.2) Requirement already satisfied: markdown-it-py>=2.2.0 in /usr/local/lib/python3.11/dist-packages (from rich->keras>=3.5.0->tensorflow) (3.0.0) Requirement already satisfied: pygments<3.0.0,>=2.13.0 in /usr/local/lib/python3.11/dist-packages (from rich->keras>=3.5.0->tensorflow) (2.19.2) Requirement already satisfied: mdurl~=0.1 in /usr/local/lib/python3.11/dist-packages (from markdown-it-py>=2.2.0->rich->keras>=3.5.0->tensorflow) (0.1.2)
In [ ]:
# Import libraries
import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns
# TensorFlow and Keras
import tensorflow as tf
from tensorflow import keras
from tensorflow.keras import layers
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense
from tensorflow.keras.utils import to_categorical
# Scikit-learn for data generation and preprocessing
from sklearn.datasets import make_moons, make_classification
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler, LabelEncoder
from sklearn.metrics import accuracy_score, log_loss
# Suppress warnings for cleaner output
import warnings
warnings.filterwarnings('ignore')
print(f"NumPy Version: {np.__version__}")
print(f"TensorFlow Version: {tf.__version__}")
NumPy Version: 2.0.2 TensorFlow Version: 2.18.0
Part 1: Revisiting Forward Propagation¶
Before diving into backpropagation, let's ensure a solid understanding of how data flows forward through a simple neural network to produce an output. We'll implement a basic two-layer network (one input, one hidden, one output) using NumPy.
Tasks:
- Define activation functions (Sigmoid, ReLU, Softmax) and their derivatives.
- Implement a forward pass for a small, manually defined neural network.
- Calculate the initial output given some input and random weights/biases.
For interactive visualizations Link
For more reading Link
In [ ]:
# --- Activation Functions and their Derivatives ---
def sigmoid(x):
return 1 / (1 + np.exp(-x))
def sigmoid_derivative(x):
s = sigmoid(x)
return s * (1 - s)
def relu(x):
return np.maximum(0, x)
def relu_derivative(x):
return (x > 0).astype(float)
def softmax(x):
exp_x = np.exp(x - np.max(x, axis=-1, keepdims=True)) # Subtract max for numerical stability
return exp_x / np.sum(exp_x, axis=-1, keepdims=True)
# --- Define Network Parameters (Manual for simplicity) ---
# Input layer (2 features) -> Hidden layer (3 neurons) -> Output layer (2 classes)
input_size = 2
hidden_size = 3
output_size = 2
# Initialize weights and biases (randomly, typically small values)
np.random.seed(42) # for reproducibility
# Weights from input to hidden layer
W1 = np.random.randn(input_size, hidden_size) * 0.01 # shape (2, 3)
b1 = np.zeros((1, hidden_size)) # shape (1, 3)
# Weights from hidden to output layer
W2 = np.random.randn(hidden_size, output_size) * 0.01 # shape (3, 2)
b2 = np.zeros((1, output_size)) # shape (1, 2)
print(f"W1 shape: {W1.shape}, b1 shape: {b1.shape}")
print(f"W2 shape: {W2.shape}, b2 shape: {b2.shape}")
W1 shape: (2, 3), b1 shape: (1, 3) W2 shape: (3, 2), b2 shape: (1, 2)
In [ ]:
# --- Visualization of Activation Functions ---
x_vals = np.linspace(-10, 10, 400)
fig, axs = plt.subplots(2, 2, figsize=(12, 8))
# Sigmoid
axs[0, 0].plot(x_vals, sigmoid(x_vals), label="Sigmoid", color='blue')
axs[0, 0].set_title("Sigmoid Function")
axs[0, 0].grid(True)
# Sigmoid Derivative
axs[0, 1].plot(x_vals, sigmoid_derivative(x_vals), label="Sigmoid Derivative", color='red')
axs[0, 1].set_title("Sigmoid Derivative")
axs[0, 1].grid(True)
# ReLU
axs[1, 0].plot(x_vals, relu(x_vals), label="ReLU", color='green')
axs[1, 0].set_title("ReLU Function")
axs[1, 0].grid(True)
# ReLU Derivative
axs[1, 1].plot(x_vals, relu_derivative(x_vals), label="ReLU Derivative", color='orange')
axs[1, 1].set_title("ReLU Derivative")
axs[1, 1].grid(True)
plt.tight_layout()
plt.show()
In [ ]:
# --- Forward Pass Implementation ---
def forward_pass(X, W1, b1, W2, b2):
# Layer 1 (Hidden Layer)
Z1 = np.dot(X, W1) + b1 # Linear combination
A1 = sigmoid(Z1) # Activation
# Layer 2 (Output Layer)
Z2 = np.dot(A1, W2) + b2 # Linear combination
A2 = softmax(Z2) # Activation (probabilities for output)
# Store intermediate values needed for backpropagation
cache = (X, Z1, A1, Z2, A2)
return (A2, cache) # Explicitly return a tuple of two items
# Example Input (1 sample with 2 features)
X_sample = np.array([[0.5, 1.2]])
print(f"\nSample Input (X_sample):\n{X_sample}")
# Perform forward pass
output_probabilities, cache = forward_pass(X_sample, W1, b1, W2, b2)
print(f"\nInitial Output Probabilities (A2):\n{output_probabilities}")
print(f"Predicted class: {np.argmax(output_probabilities)}")
# Unpack cache for inspection
X_val, Z1_val, A1_val, Z2_val, A2_val = cache
print(f"\nZ1 (pre-activation hidden): {Z1_val}")
print(f"A1 (activated hidden): {A1_val}")
print(f"Z2 (pre-activation output): {Z2_val}")
Sample Input (X_sample): [[0.5 1.2]] Initial Output Probabilities (A2): [[0.49976532 0.50023468]] Predicted class: 1 Z1 (pre-activation hidden): [[ 0.02075993 -0.00350116 0.0004288 ]] A1 (activated hidden): [[0.5051898 0.49912471 0.5001072 ]] Z2 (pre-activation output): [[0.00331717 0.00425591]]
Discussion Point:¶
- Trace the dimensions of the matrices (input, weights, biases) as they flow through the
forward_passfunction. How do the shapes align for matrix multiplication? - What is the purpose of the
cachevariable? Why is it important to store these intermediate values?
Part 2: Implementing Backpropagation from Scratch¶
This is the core of the lab. We will implement the backward pass to calculate gradients, which are then used to update weights and biases. We'll use a simple dataset and a very basic training loop.
Tasks:
- Define a loss function (e.g., Categorical Cross-Entropy) and its derivative.
- Implement the
backward_passfunction to calculate gradients forW1,b1,W2, andb2. - Integrate forward and backward passes into a simple training loop to update weights.
- Observe how the loss decreases over epochs.
In [ ]:
# --- Loss Function and its Derivative ---
def cross_entropy_loss(y_pred, y_true):
# Ensure numerical stability
y_pred = np.clip(y_pred, 1e-12, 1 - 1e-12)
return -np.sum(y_true * np.log(y_pred)) / y_true.shape[0]
def cross_entropy_derivative(y_pred, y_true):
return (y_pred - y_true) / y_true.shape[0] # Derivative for softmax + cross-entropy
# --- Backward Pass Implementation ---
def backward_pass(X, y_true, cache, W2):
X_val, Z1_val, A1_val, Z2_val, A2_val = cache
m = X.shape[0] # Number of samples
# Output Layer Gradients
# dZ2: Gradient of loss wrt Z2 (pre-activation output)
# This specific derivative dL/dZ2 for softmax + cross-entropy simplifies to (A2 - y_true)
dZ2 = cross_entropy_derivative(A2_val, y_true) # shape (m, output_size)
# dW2: Gradient of loss wrt W2 (weights from hidden to output)
dW2 = np.dot(A1_val.T, dZ2) # shape (hidden_size, output_size)
# db2: Gradient of loss wrt b2 (bias for output layer)
db2 = np.sum(dZ2, axis=0, keepdims=True) # shape (1, output_size)
# Hidden Layer Gradients
# dA1: Gradient of loss wrt A1 (activated hidden)
dA1 = np.dot(dZ2, W2.T) # shape (m, hidden_size)
# dZ1: Gradient of loss wrt Z1 (pre-activation hidden)
# Apply derivative of activation function here
dZ1 = dA1 * sigmoid_derivative(Z1_val) # shape (m, hidden_size)
# dW1: Gradient of loss wrt W1 (weights from input to hidden)
dW1 = np.dot(X_val.T, dZ1) # shape (input_size, hidden_size)
# db1: Gradient of loss wrt b1 (bias for hidden layer)
db1 = np.sum(dZ1, axis=0, keepdims=True) # shape (1, hidden_size)
gradients = {"dW1": dW1, "db1": db1, "dW2": dW2, "db2": db2}
return gradients
In [ ]:
# --- Generate a simple dataset for binary classification ---
# Using make_classification for a slightly more complex but still separable dataset
X_data, y_data = make_classification(n_samples=100, n_features=2, n_redundant=0, n_informative=2,
n_clusters_per_class=1, random_state=42, n_classes=2)
# Scale features
scaler_bp = StandardScaler()
X_data_scaled = scaler_bp.fit_transform(X_data)
# One-hot encode target for 2 classes
y_data_oh = to_categorical(y_data, num_classes=2)
print(f"\nGenerated data shape: {X_data_scaled.shape}, Target one-hot shape: {y_data_oh.shape}")
Generated data shape: (100, 2), Target one-hot shape: (100, 2)
In [ ]:
# --- Training Loop with Backpropagation ---
epochs = 1000
learning_rate = 0.1
losses = []
# Re-initialize weights for training loop
np.random.seed(42)
W1_train = np.random.randn(input_size, hidden_size) * 0.01
b1_train = np.zeros((1, hidden_size))
W2_train = np.random.randn(hidden_size, output_size) * 0.01
b2_train = np.zeros((1, output_size))
print("\nStarting training with custom backpropagation...")
for epoch in range(epochs):
# Forward Pass
# Print shapes for debugging
print(f"Epoch {epoch}: X_data_scaled shape: {X_data_scaled.shape}")
print(f"Epoch {epoch}: W1_train shape: {W1_train.shape}")
print(f"Epoch {epoch}: b1_train shape: {b1_train.shape}")
print(f"Epoch {epoch}: W2_train shape: {W2_train.shape}")
print(f"Epoch {epoch}: b2_train shape: {b2_train.shape}")
y_pred, cache = forward_pass(X_data_scaled, W1_train, b1_train, W2_train, b2_train)
# Calculate Loss
loss = cross_entropy_loss(y_pred, y_data_oh)
losses.append(loss)
# Backward Pass (Calculate Gradients)
gradients = backward_pass(X_data_scaled, y_data_oh, cache, W2_train)
dW1, db1, dW2, db2 = gradients["dW1"], gradients["db1"], gradients["dW2"], gradients["db2"]
# Update Weights and Biases
W1_train -= learning_rate * dW1
b1_train -= learning_rate * db1
W2_train -= learning_rate * dW2
b2_train -= learning_rate * db2
if epoch % 100 == 0:
print(f"Epoch {epoch}/{epochs}, Loss: {loss:.4f}")
print("\nTraining complete.")
print(f"Final Loss: {losses[-1]:.4f}")
# Plotting the loss over epochs
plt.figure(figsize=(10, 6))
plt.plot(losses)
plt.title('Loss Reduction Over Epochs (Custom Backpropagation)')
plt.xlabel('Epoch')
plt.ylabel('Loss')
plt.grid(True, linestyle='--', alpha=0.6)
plt.show()
Starting training with custom backpropagation... Epoch 0: X_data_scaled shape: (100, 2) Epoch 0: W1_train shape: (2, 3) Epoch 0: b1_train shape: (1, 3) Epoch 0: W2_train shape: (3, 2) Epoch 0: b2_train shape: (1, 2) Epoch 0/1000, Loss: 5.2983 Epoch 1: X_data_scaled shape: (100, 2) Epoch 1: W1_train shape: (2, 3) Epoch 1: b1_train shape: (1, 3) Epoch 1: W2_train shape: (3, 2) Epoch 1: b2_train shape: (1, 2) Epoch 2: X_data_scaled shape: (100, 2) Epoch 2: W1_train shape: (2, 3) Epoch 2: b1_train shape: (1, 3) Epoch 2: W2_train shape: (3, 2) Epoch 2: b2_train shape: (1, 2) Epoch 3: X_data_scaled shape: (100, 2) Epoch 3: W1_train shape: (2, 3) Epoch 3: b1_train shape: (1, 3) Epoch 3: W2_train shape: (3, 2) Epoch 3: b2_train shape: (1, 2) Epoch 4: X_data_scaled shape: (100, 2) Epoch 4: W1_train shape: (2, 3) Epoch 4: b1_train shape: (1, 3) Epoch 4: W2_train shape: (3, 2) Epoch 4: b2_train shape: (1, 2) Epoch 5: X_data_scaled shape: (100, 2) Epoch 5: W1_train shape: (2, 3) Epoch 5: b1_train shape: (1, 3) Epoch 5: W2_train shape: (3, 2) Epoch 5: b2_train shape: (1, 2) Epoch 6: X_data_scaled shape: (100, 2) Epoch 6: W1_train shape: (2, 3) Epoch 6: b1_train shape: (1, 3) Epoch 6: W2_train shape: (3, 2) Epoch 6: b2_train shape: (1, 2) Epoch 7: X_data_scaled shape: (100, 2) Epoch 7: W1_train shape: (2, 3) Epoch 7: b1_train shape: (1, 3) Epoch 7: W2_train shape: (3, 2) Epoch 7: b2_train shape: (1, 2) Epoch 8: X_data_scaled shape: (100, 2) Epoch 8: W1_train shape: (2, 3) Epoch 8: b1_train shape: (1, 3) Epoch 8: W2_train shape: (3, 2) Epoch 8: b2_train shape: (1, 2) Epoch 9: X_data_scaled shape: (100, 2) Epoch 9: W1_train shape: (2, 3) Epoch 9: b1_train shape: (1, 3) Epoch 9: W2_train shape: (3, 2) Epoch 9: b2_train shape: (1, 2) Epoch 10: X_data_scaled shape: (100, 2) Epoch 10: W1_train shape: (2, 3) Epoch 10: b1_train shape: (1, 3) Epoch 10: W2_train shape: (3, 2) Epoch 10: b2_train shape: (1, 2) Epoch 11: X_data_scaled shape: (100, 2) Epoch 11: W1_train shape: (2, 3) Epoch 11: b1_train shape: (1, 3) Epoch 11: W2_train shape: (3, 2) Epoch 11: b2_train shape: (1, 2) Epoch 12: X_data_scaled shape: (100, 2) Epoch 12: W1_train shape: (2, 3) Epoch 12: b1_train shape: (1, 3) Epoch 12: W2_train shape: (3, 2) Epoch 12: b2_train shape: (1, 2) Epoch 13: X_data_scaled shape: (100, 2) Epoch 13: W1_train shape: (2, 3) Epoch 13: b1_train shape: (1, 3) Epoch 13: W2_train shape: (3, 2) Epoch 13: b2_train shape: (1, 2) Epoch 14: X_data_scaled shape: (100, 2) Epoch 14: W1_train shape: (2, 3) Epoch 14: b1_train shape: (1, 3) Epoch 14: W2_train shape: (3, 2) Epoch 14: b2_train shape: (1, 2) Epoch 15: X_data_scaled shape: (100, 2) Epoch 15: W1_train shape: (2, 3) Epoch 15: b1_train shape: (1, 3) Epoch 15: W2_train shape: (3, 2) Epoch 15: b2_train shape: (1, 2) Epoch 16: X_data_scaled shape: (100, 2) Epoch 16: W1_train shape: (2, 3) Epoch 16: b1_train shape: (1, 3) Epoch 16: W2_train shape: (3, 2) Epoch 16: b2_train shape: (1, 2) Epoch 17: X_data_scaled shape: (100, 2) Epoch 17: W1_train shape: (2, 3) Epoch 17: b1_train shape: (1, 3) Epoch 17: W2_train shape: (3, 2) Epoch 17: b2_train shape: (1, 2) Epoch 18: X_data_scaled shape: (100, 2) Epoch 18: W1_train shape: (2, 3) Epoch 18: b1_train shape: (1, 3) Epoch 18: W2_train shape: (3, 2) Epoch 18: b2_train shape: (1, 2) Epoch 19: X_data_scaled shape: (100, 2) Epoch 19: W1_train shape: (2, 3) Epoch 19: b1_train shape: (1, 3) Epoch 19: W2_train shape: (3, 2) Epoch 19: b2_train shape: (1, 2) Epoch 20: X_data_scaled shape: (100, 2) Epoch 20: W1_train shape: (2, 3) Epoch 20: b1_train shape: (1, 3) Epoch 20: W2_train shape: (3, 2) Epoch 20: b2_train shape: (1, 2) Epoch 21: X_data_scaled shape: (100, 2) Epoch 21: W1_train shape: (2, 3) Epoch 21: b1_train shape: (1, 3) Epoch 21: W2_train shape: (3, 2) Epoch 21: b2_train shape: (1, 2) Epoch 22: X_data_scaled shape: (100, 2) Epoch 22: W1_train shape: (2, 3) Epoch 22: b1_train shape: (1, 3) Epoch 22: W2_train shape: (3, 2) Epoch 22: b2_train shape: (1, 2) Epoch 23: X_data_scaled shape: (100, 2) Epoch 23: W1_train shape: (2, 3) Epoch 23: b1_train shape: (1, 3) Epoch 23: W2_train shape: (3, 2) Epoch 23: b2_train shape: (1, 2) Epoch 24: X_data_scaled shape: (100, 2) Epoch 24: W1_train shape: (2, 3) Epoch 24: b1_train shape: (1, 3) Epoch 24: W2_train shape: (3, 2) Epoch 24: b2_train shape: (1, 2) Epoch 25: X_data_scaled shape: (100, 2) Epoch 25: W1_train shape: (2, 3) Epoch 25: b1_train shape: (1, 3) Epoch 25: W2_train shape: (3, 2) Epoch 25: b2_train shape: (1, 2) Epoch 26: X_data_scaled shape: (100, 2) Epoch 26: W1_train shape: (2, 3) Epoch 26: b1_train shape: (1, 3) Epoch 26: W2_train shape: (3, 2) Epoch 26: b2_train shape: (1, 2) Epoch 27: X_data_scaled shape: (100, 2) Epoch 27: W1_train shape: (2, 3) Epoch 27: b1_train shape: (1, 3) Epoch 27: W2_train shape: (3, 2) Epoch 27: b2_train shape: (1, 2) Epoch 28: X_data_scaled shape: (100, 2) Epoch 28: W1_train shape: (2, 3) Epoch 28: b1_train shape: (1, 3) Epoch 28: W2_train shape: (3, 2) Epoch 28: b2_train shape: (1, 2) Epoch 29: X_data_scaled shape: (100, 2) Epoch 29: W1_train shape: (2, 3) Epoch 29: b1_train shape: (1, 3) Epoch 29: W2_train shape: (3, 2) Epoch 29: b2_train shape: (1, 2) Epoch 30: X_data_scaled shape: (100, 2) Epoch 30: W1_train shape: (2, 3) Epoch 30: b1_train shape: (1, 3) Epoch 30: W2_train shape: (3, 2) Epoch 30: b2_train shape: (1, 2) Epoch 31: X_data_scaled shape: (100, 2) Epoch 31: W1_train shape: (2, 3) Epoch 31: b1_train shape: (1, 3) Epoch 31: W2_train shape: (3, 2) Epoch 31: b2_train shape: (1, 2) Epoch 32: X_data_scaled shape: (100, 2) Epoch 32: W1_train shape: (2, 3) Epoch 32: b1_train shape: (1, 3) Epoch 32: W2_train shape: (3, 2) Epoch 32: b2_train shape: (1, 2) Epoch 33: X_data_scaled shape: (100, 2) Epoch 33: W1_train shape: (2, 3) Epoch 33: b1_train shape: (1, 3) Epoch 33: W2_train shape: (3, 2) Epoch 33: b2_train shape: (1, 2) Epoch 34: X_data_scaled shape: (100, 2) Epoch 34: W1_train shape: (2, 3) Epoch 34: b1_train shape: (1, 3) Epoch 34: W2_train shape: (3, 2) Epoch 34: b2_train shape: (1, 2) Epoch 35: X_data_scaled shape: (100, 2) Epoch 35: W1_train shape: (2, 3) Epoch 35: b1_train shape: (1, 3) Epoch 35: W2_train shape: (3, 2) Epoch 35: b2_train shape: (1, 2) Epoch 36: X_data_scaled shape: (100, 2) Epoch 36: W1_train shape: (2, 3) Epoch 36: b1_train shape: (1, 3) Epoch 36: W2_train shape: (3, 2) Epoch 36: b2_train shape: (1, 2) Epoch 37: X_data_scaled shape: (100, 2) Epoch 37: W1_train shape: (2, 3) Epoch 37: b1_train shape: (1, 3) Epoch 37: W2_train shape: (3, 2) Epoch 37: b2_train shape: (1, 2) Epoch 38: X_data_scaled shape: (100, 2) Epoch 38: W1_train shape: (2, 3) Epoch 38: b1_train shape: (1, 3) Epoch 38: W2_train shape: (3, 2) Epoch 38: b2_train shape: (1, 2) Epoch 39: X_data_scaled shape: (100, 2) Epoch 39: W1_train shape: (2, 3) Epoch 39: b1_train shape: (1, 3) Epoch 39: W2_train shape: (3, 2) Epoch 39: b2_train shape: (1, 2) Epoch 40: X_data_scaled shape: (100, 2) Epoch 40: W1_train shape: (2, 3) Epoch 40: b1_train shape: (1, 3) Epoch 40: W2_train shape: (3, 2) Epoch 40: b2_train shape: (1, 2) Epoch 41: X_data_scaled shape: (100, 2) Epoch 41: W1_train shape: (2, 3) Epoch 41: b1_train shape: (1, 3) Epoch 41: W2_train shape: (3, 2) Epoch 41: b2_train shape: (1, 2) Epoch 42: X_data_scaled shape: (100, 2) Epoch 42: W1_train shape: (2, 3) Epoch 42: b1_train shape: (1, 3) Epoch 42: W2_train shape: (3, 2) Epoch 42: b2_train shape: (1, 2) Epoch 43: X_data_scaled shape: (100, 2) Epoch 43: W1_train shape: (2, 3) Epoch 43: b1_train shape: (1, 3) Epoch 43: W2_train shape: (3, 2) Epoch 43: b2_train shape: (1, 2) Epoch 44: X_data_scaled shape: (100, 2) Epoch 44: W1_train shape: (2, 3) Epoch 44: b1_train shape: (1, 3) Epoch 44: W2_train shape: (3, 2) Epoch 44: b2_train shape: (1, 2) Epoch 45: X_data_scaled shape: (100, 2) Epoch 45: W1_train shape: (2, 3) Epoch 45: b1_train shape: (1, 3) Epoch 45: W2_train shape: (3, 2) Epoch 45: b2_train shape: (1, 2) Epoch 46: X_data_scaled shape: (100, 2) Epoch 46: W1_train shape: (2, 3) Epoch 46: b1_train shape: (1, 3) Epoch 46: W2_train shape: (3, 2) Epoch 46: b2_train shape: (1, 2) Epoch 47: X_data_scaled shape: (100, 2) Epoch 47: W1_train shape: (2, 3) Epoch 47: b1_train shape: (1, 3) Epoch 47: W2_train shape: (3, 2) Epoch 47: b2_train shape: (1, 2) Epoch 48: X_data_scaled shape: (100, 2) Epoch 48: W1_train shape: (2, 3) Epoch 48: b1_train shape: (1, 3) Epoch 48: W2_train shape: (3, 2) Epoch 48: b2_train shape: (1, 2) Epoch 49: X_data_scaled shape: (100, 2) Epoch 49: W1_train shape: (2, 3) Epoch 49: b1_train shape: (1, 3) Epoch 49: W2_train shape: (3, 2) Epoch 49: b2_train shape: (1, 2) Epoch 50: X_data_scaled shape: (100, 2) Epoch 50: W1_train shape: (2, 3) Epoch 50: b1_train shape: (1, 3) Epoch 50: W2_train shape: (3, 2) Epoch 50: b2_train shape: (1, 2) Epoch 51: X_data_scaled shape: (100, 2) Epoch 51: W1_train shape: (2, 3) Epoch 51: b1_train shape: (1, 3) Epoch 51: W2_train shape: (3, 2) Epoch 51: b2_train shape: (1, 2) Epoch 52: X_data_scaled shape: (100, 2) Epoch 52: W1_train shape: (2, 3) Epoch 52: b1_train shape: (1, 3) Epoch 52: W2_train shape: (3, 2) Epoch 52: b2_train shape: (1, 2) Epoch 53: X_data_scaled shape: (100, 2) Epoch 53: W1_train shape: (2, 3) Epoch 53: b1_train shape: (1, 3) Epoch 53: W2_train shape: (3, 2) Epoch 53: b2_train shape: (1, 2) Epoch 54: X_data_scaled shape: (100, 2) Epoch 54: W1_train shape: (2, 3) Epoch 54: b1_train shape: (1, 3) Epoch 54: W2_train shape: (3, 2) Epoch 54: b2_train shape: (1, 2) Epoch 55: X_data_scaled shape: (100, 2) Epoch 55: W1_train shape: (2, 3) Epoch 55: b1_train shape: (1, 3) Epoch 55: W2_train shape: (3, 2) Epoch 55: b2_train shape: (1, 2) Epoch 56: X_data_scaled shape: (100, 2) Epoch 56: W1_train shape: (2, 3) Epoch 56: b1_train shape: (1, 3) Epoch 56: W2_train shape: (3, 2) Epoch 56: b2_train shape: (1, 2) Epoch 57: X_data_scaled shape: (100, 2) Epoch 57: W1_train shape: (2, 3) Epoch 57: b1_train shape: (1, 3) Epoch 57: W2_train shape: (3, 2) Epoch 57: b2_train shape: (1, 2) Epoch 58: X_data_scaled shape: (100, 2) Epoch 58: W1_train shape: (2, 3) Epoch 58: b1_train shape: (1, 3) Epoch 58: W2_train shape: (3, 2) Epoch 58: b2_train shape: (1, 2) Epoch 59: X_data_scaled shape: (100, 2) Epoch 59: W1_train shape: (2, 3) Epoch 59: b1_train shape: (1, 3) Epoch 59: W2_train shape: (3, 2) Epoch 59: b2_train shape: (1, 2) Epoch 60: X_data_scaled shape: (100, 2) Epoch 60: W1_train shape: (2, 3) Epoch 60: b1_train shape: (1, 3) Epoch 60: W2_train shape: (3, 2) Epoch 60: b2_train shape: (1, 2) Epoch 61: X_data_scaled shape: (100, 2) Epoch 61: W1_train shape: (2, 3) Epoch 61: b1_train shape: (1, 3) Epoch 61: W2_train shape: (3, 2) Epoch 61: b2_train shape: (1, 2) Epoch 62: X_data_scaled shape: (100, 2) Epoch 62: W1_train shape: (2, 3) Epoch 62: b1_train shape: (1, 3) Epoch 62: W2_train shape: (3, 2) Epoch 62: b2_train shape: (1, 2) Epoch 63: X_data_scaled shape: (100, 2) Epoch 63: W1_train shape: (2, 3) Epoch 63: b1_train shape: (1, 3) Epoch 63: W2_train shape: (3, 2) Epoch 63: b2_train shape: (1, 2) Epoch 64: X_data_scaled shape: (100, 2) Epoch 64: W1_train shape: (2, 3) Epoch 64: b1_train shape: (1, 3) Epoch 64: W2_train shape: (3, 2) Epoch 64: b2_train shape: (1, 2) Epoch 65: X_data_scaled shape: (100, 2) Epoch 65: W1_train shape: (2, 3) Epoch 65: b1_train shape: (1, 3) Epoch 65: W2_train shape: (3, 2) Epoch 65: b2_train shape: (1, 2) Epoch 66: X_data_scaled shape: (100, 2) Epoch 66: W1_train shape: (2, 3) Epoch 66: b1_train shape: (1, 3) Epoch 66: W2_train shape: (3, 2) Epoch 66: b2_train shape: (1, 2) Epoch 67: X_data_scaled shape: (100, 2) Epoch 67: W1_train shape: (2, 3) Epoch 67: b1_train shape: (1, 3) Epoch 67: W2_train shape: (3, 2) Epoch 67: b2_train shape: (1, 2) Epoch 68: X_data_scaled shape: (100, 2) Epoch 68: W1_train shape: (2, 3) Epoch 68: b1_train shape: (1, 3) Epoch 68: W2_train shape: (3, 2) Epoch 68: b2_train shape: (1, 2) Epoch 69: X_data_scaled shape: (100, 2) Epoch 69: W1_train shape: (2, 3) Epoch 69: b1_train shape: (1, 3) Epoch 69: W2_train shape: (3, 2) Epoch 69: b2_train shape: (1, 2) Epoch 70: X_data_scaled shape: (100, 2) Epoch 70: W1_train shape: (2, 3) Epoch 70: b1_train shape: (1, 3) Epoch 70: W2_train shape: (3, 2) Epoch 70: b2_train shape: (1, 2) Epoch 71: X_data_scaled shape: (100, 2) Epoch 71: W1_train shape: (2, 3) Epoch 71: b1_train shape: (1, 3) Epoch 71: W2_train shape: (3, 2) Epoch 71: b2_train shape: (1, 2) Epoch 72: X_data_scaled shape: (100, 2) Epoch 72: W1_train shape: (2, 3) Epoch 72: b1_train shape: (1, 3) Epoch 72: W2_train shape: (3, 2) Epoch 72: b2_train shape: (1, 2) Epoch 73: X_data_scaled shape: (100, 2) Epoch 73: W1_train shape: (2, 3) Epoch 73: b1_train shape: (1, 3) Epoch 73: W2_train shape: (3, 2) Epoch 73: b2_train shape: (1, 2) Epoch 74: X_data_scaled shape: (100, 2) Epoch 74: W1_train shape: (2, 3) Epoch 74: b1_train shape: (1, 3) Epoch 74: W2_train shape: (3, 2) Epoch 74: b2_train shape: (1, 2) Epoch 75: X_data_scaled shape: (100, 2) Epoch 75: W1_train shape: (2, 3) Epoch 75: b1_train shape: (1, 3) Epoch 75: W2_train shape: (3, 2) Epoch 75: b2_train shape: (1, 2) Epoch 76: X_data_scaled shape: (100, 2) Epoch 76: W1_train shape: (2, 3) Epoch 76: b1_train shape: (1, 3) Epoch 76: W2_train shape: (3, 2) Epoch 76: b2_train shape: (1, 2) Epoch 77: X_data_scaled shape: (100, 2) Epoch 77: W1_train shape: (2, 3) Epoch 77: b1_train shape: (1, 3) Epoch 77: W2_train shape: (3, 2) Epoch 77: b2_train shape: (1, 2) Epoch 78: X_data_scaled shape: (100, 2) Epoch 78: W1_train shape: (2, 3) Epoch 78: b1_train shape: (1, 3) Epoch 78: W2_train shape: (3, 2) Epoch 78: b2_train shape: (1, 2) Epoch 79: X_data_scaled shape: (100, 2) Epoch 79: W1_train shape: (2, 3) Epoch 79: b1_train shape: (1, 3) Epoch 79: W2_train shape: (3, 2) Epoch 79: b2_train shape: (1, 2) Epoch 80: X_data_scaled shape: (100, 2) Epoch 80: W1_train shape: (2, 3) Epoch 80: b1_train shape: (1, 3) Epoch 80: W2_train shape: (3, 2) Epoch 80: b2_train shape: (1, 2) Epoch 81: X_data_scaled shape: (100, 2) Epoch 81: W1_train shape: (2, 3) Epoch 81: b1_train shape: (1, 3) Epoch 81: W2_train shape: (3, 2) Epoch 81: b2_train shape: (1, 2) Epoch 82: X_data_scaled shape: (100, 2) Epoch 82: W1_train shape: (2, 3) Epoch 82: b1_train shape: (1, 3) Epoch 82: W2_train shape: (3, 2) Epoch 82: b2_train shape: (1, 2) Epoch 83: X_data_scaled shape: (100, 2) Epoch 83: W1_train shape: (2, 3) Epoch 83: b1_train shape: (1, 3) Epoch 83: W2_train shape: (3, 2) Epoch 83: b2_train shape: (1, 2) Epoch 84: X_data_scaled shape: (100, 2) Epoch 84: W1_train shape: (2, 3) Epoch 84: b1_train shape: (1, 3) Epoch 84: W2_train shape: (3, 2) Epoch 84: b2_train shape: (1, 2) Epoch 85: X_data_scaled shape: (100, 2) Epoch 85: W1_train shape: (2, 3) Epoch 85: b1_train shape: (1, 3) Epoch 85: W2_train shape: (3, 2) Epoch 85: b2_train shape: (1, 2) Epoch 86: X_data_scaled shape: (100, 2) Epoch 86: W1_train shape: (2, 3) Epoch 86: b1_train shape: (1, 3) Epoch 86: W2_train shape: (3, 2) Epoch 86: b2_train shape: (1, 2) Epoch 87: X_data_scaled shape: (100, 2) Epoch 87: W1_train shape: (2, 3) Epoch 87: b1_train shape: (1, 3) Epoch 87: W2_train shape: (3, 2) Epoch 87: b2_train shape: (1, 2) Epoch 88: X_data_scaled shape: (100, 2) Epoch 88: W1_train shape: (2, 3) Epoch 88: b1_train shape: (1, 3) Epoch 88: W2_train shape: (3, 2) Epoch 88: b2_train shape: (1, 2) Epoch 89: X_data_scaled shape: (100, 2) Epoch 89: W1_train shape: (2, 3) Epoch 89: b1_train shape: (1, 3) Epoch 89: W2_train shape: (3, 2) Epoch 89: b2_train shape: (1, 2) Epoch 90: X_data_scaled shape: (100, 2) Epoch 90: W1_train shape: (2, 3) Epoch 90: b1_train shape: (1, 3) Epoch 90: W2_train shape: (3, 2) Epoch 90: b2_train shape: (1, 2) Epoch 91: X_data_scaled shape: (100, 2) Epoch 91: W1_train shape: (2, 3) Epoch 91: b1_train shape: (1, 3) Epoch 91: W2_train shape: (3, 2) Epoch 91: b2_train shape: (1, 2) Epoch 92: X_data_scaled shape: (100, 2) Epoch 92: W1_train shape: (2, 3) Epoch 92: b1_train shape: (1, 3) Epoch 92: W2_train shape: (3, 2) Epoch 92: b2_train shape: (1, 2) Epoch 93: X_data_scaled shape: (100, 2) Epoch 93: W1_train shape: (2, 3) Epoch 93: b1_train shape: (1, 3) Epoch 93: W2_train shape: (3, 2) Epoch 93: b2_train shape: (1, 2) Epoch 94: X_data_scaled shape: (100, 2) Epoch 94: W1_train shape: (2, 3) Epoch 94: b1_train shape: (1, 3) Epoch 94: W2_train shape: (3, 2) Epoch 94: b2_train shape: (1, 2) Epoch 95: X_data_scaled shape: (100, 2) Epoch 95: W1_train shape: (2, 3) Epoch 95: b1_train shape: (1, 3) Epoch 95: W2_train shape: (3, 2) Epoch 95: b2_train shape: (1, 2) Epoch 96: X_data_scaled shape: (100, 2) Epoch 96: W1_train shape: (2, 3) Epoch 96: b1_train shape: (1, 3) Epoch 96: W2_train shape: (3, 2) Epoch 96: b2_train shape: (1, 2) Epoch 97: X_data_scaled shape: (100, 2) Epoch 97: W1_train shape: (2, 3) Epoch 97: b1_train shape: (1, 3) Epoch 97: W2_train shape: (3, 2) Epoch 97: b2_train shape: (1, 2) Epoch 98: X_data_scaled shape: (100, 2) Epoch 98: W1_train shape: (2, 3) Epoch 98: b1_train shape: (1, 3) Epoch 98: W2_train shape: (3, 2) Epoch 98: b2_train shape: (1, 2) Epoch 99: X_data_scaled shape: (100, 2) Epoch 99: W1_train shape: (2, 3) Epoch 99: b1_train shape: (1, 3) Epoch 99: W2_train shape: (3, 2) Epoch 99: b2_train shape: (1, 2) Epoch 100: X_data_scaled shape: (100, 2) Epoch 100: W1_train shape: (2, 3) Epoch 100: b1_train shape: (1, 3) Epoch 100: W2_train shape: (3, 2) Epoch 100: b2_train shape: (1, 2) Epoch 100/1000, Loss: 5.2984 Epoch 101: X_data_scaled shape: (100, 2) Epoch 101: W1_train shape: (2, 3) Epoch 101: b1_train shape: (1, 3) Epoch 101: W2_train shape: (3, 2) Epoch 101: b2_train shape: (1, 2) Epoch 102: X_data_scaled shape: (100, 2) Epoch 102: W1_train shape: (2, 3) Epoch 102: b1_train shape: (1, 3) Epoch 102: W2_train shape: (3, 2) Epoch 102: b2_train shape: (1, 2) Epoch 103: X_data_scaled shape: (100, 2) Epoch 103: W1_train shape: (2, 3) Epoch 103: b1_train shape: (1, 3) Epoch 103: W2_train shape: (3, 2) Epoch 103: b2_train shape: (1, 2) Epoch 104: X_data_scaled shape: (100, 2) Epoch 104: W1_train shape: (2, 3) Epoch 104: b1_train shape: (1, 3) Epoch 104: W2_train shape: (3, 2) Epoch 104: b2_train shape: (1, 2) Epoch 105: X_data_scaled shape: (100, 2) Epoch 105: W1_train shape: (2, 3) Epoch 105: b1_train shape: (1, 3) Epoch 105: W2_train shape: (3, 2) Epoch 105: b2_train shape: (1, 2) Epoch 106: X_data_scaled shape: (100, 2) Epoch 106: W1_train shape: (2, 3) Epoch 106: b1_train shape: (1, 3) Epoch 106: W2_train shape: (3, 2) Epoch 106: b2_train shape: (1, 2) Epoch 107: X_data_scaled shape: (100, 2) Epoch 107: W1_train shape: (2, 3) Epoch 107: b1_train shape: (1, 3) Epoch 107: W2_train shape: (3, 2) Epoch 107: b2_train shape: (1, 2) Epoch 108: X_data_scaled shape: (100, 2) Epoch 108: W1_train shape: (2, 3) Epoch 108: b1_train shape: (1, 3) Epoch 108: W2_train shape: (3, 2) Epoch 108: b2_train shape: (1, 2) Epoch 109: X_data_scaled shape: (100, 2) Epoch 109: W1_train shape: (2, 3) Epoch 109: b1_train shape: (1, 3) Epoch 109: W2_train shape: (3, 2) Epoch 109: b2_train shape: (1, 2) Epoch 110: X_data_scaled shape: (100, 2) Epoch 110: W1_train shape: (2, 3) Epoch 110: b1_train shape: (1, 3) Epoch 110: W2_train shape: (3, 2) Epoch 110: b2_train shape: (1, 2) Epoch 111: X_data_scaled shape: (100, 2) Epoch 111: W1_train shape: (2, 3) Epoch 111: b1_train shape: (1, 3) Epoch 111: W2_train shape: (3, 2) Epoch 111: b2_train shape: (1, 2) Epoch 112: X_data_scaled shape: (100, 2) Epoch 112: W1_train shape: (2, 3) Epoch 112: b1_train shape: (1, 3) Epoch 112: W2_train shape: (3, 2) Epoch 112: b2_train shape: (1, 2) Epoch 113: X_data_scaled shape: (100, 2) Epoch 113: W1_train shape: (2, 3) Epoch 113: b1_train shape: (1, 3) Epoch 113: W2_train shape: (3, 2) Epoch 113: b2_train shape: (1, 2) Epoch 114: X_data_scaled shape: (100, 2) Epoch 114: W1_train shape: (2, 3) Epoch 114: b1_train shape: (1, 3) Epoch 114: W2_train shape: (3, 2) Epoch 114: b2_train shape: (1, 2) Epoch 115: X_data_scaled shape: (100, 2) Epoch 115: W1_train shape: (2, 3) Epoch 115: b1_train shape: (1, 3) Epoch 115: W2_train shape: (3, 2) Epoch 115: b2_train shape: (1, 2) Epoch 116: X_data_scaled shape: (100, 2) Epoch 116: W1_train shape: (2, 3) Epoch 116: b1_train shape: (1, 3) Epoch 116: W2_train shape: (3, 2) Epoch 116: b2_train shape: (1, 2) Epoch 117: X_data_scaled shape: (100, 2) Epoch 117: W1_train shape: (2, 3) Epoch 117: b1_train shape: (1, 3) Epoch 117: W2_train shape: (3, 2) Epoch 117: b2_train shape: (1, 2) Epoch 118: X_data_scaled shape: (100, 2) Epoch 118: W1_train shape: (2, 3) Epoch 118: b1_train shape: (1, 3) Epoch 118: W2_train shape: (3, 2) Epoch 118: b2_train shape: (1, 2) Epoch 119: X_data_scaled shape: (100, 2) Epoch 119: W1_train shape: (2, 3) Epoch 119: b1_train shape: (1, 3) Epoch 119: W2_train shape: (3, 2) Epoch 119: b2_train shape: (1, 2) Epoch 120: X_data_scaled shape: (100, 2) Epoch 120: W1_train shape: (2, 3) Epoch 120: b1_train shape: (1, 3) Epoch 120: W2_train shape: (3, 2) Epoch 120: b2_train shape: (1, 2) Epoch 121: X_data_scaled shape: (100, 2) Epoch 121: W1_train shape: (2, 3) Epoch 121: b1_train shape: (1, 3) Epoch 121: W2_train shape: (3, 2) Epoch 121: b2_train shape: (1, 2) Epoch 122: X_data_scaled shape: (100, 2) Epoch 122: W1_train shape: (2, 3) Epoch 122: b1_train shape: (1, 3) Epoch 122: W2_train shape: (3, 2) Epoch 122: b2_train shape: (1, 2) Epoch 123: X_data_scaled shape: (100, 2) Epoch 123: W1_train shape: (2, 3) Epoch 123: b1_train shape: (1, 3) Epoch 123: W2_train shape: (3, 2) Epoch 123: b2_train shape: (1, 2) Epoch 124: X_data_scaled shape: (100, 2) Epoch 124: W1_train shape: (2, 3) Epoch 124: b1_train shape: (1, 3) Epoch 124: W2_train shape: (3, 2) Epoch 124: b2_train shape: (1, 2) Epoch 125: X_data_scaled shape: (100, 2) Epoch 125: W1_train shape: (2, 3) Epoch 125: b1_train shape: (1, 3) Epoch 125: W2_train shape: (3, 2) Epoch 125: b2_train shape: (1, 2) Epoch 126: X_data_scaled shape: (100, 2) Epoch 126: W1_train shape: (2, 3) Epoch 126: b1_train shape: (1, 3) Epoch 126: W2_train shape: (3, 2) Epoch 126: b2_train shape: (1, 2) Epoch 127: X_data_scaled shape: (100, 2) Epoch 127: W1_train shape: (2, 3) Epoch 127: b1_train shape: (1, 3) Epoch 127: W2_train shape: (3, 2) Epoch 127: b2_train shape: (1, 2) Epoch 128: X_data_scaled shape: (100, 2) Epoch 128: W1_train shape: (2, 3) Epoch 128: b1_train shape: (1, 3) Epoch 128: W2_train shape: (3, 2) Epoch 128: b2_train shape: (1, 2) Epoch 129: X_data_scaled shape: (100, 2) Epoch 129: W1_train shape: (2, 3) Epoch 129: b1_train shape: (1, 3) Epoch 129: W2_train shape: (3, 2) Epoch 129: b2_train shape: (1, 2) Epoch 130: X_data_scaled shape: (100, 2) Epoch 130: W1_train shape: (2, 3) Epoch 130: b1_train shape: (1, 3) Epoch 130: W2_train shape: (3, 2) Epoch 130: b2_train shape: (1, 2) Epoch 131: X_data_scaled shape: (100, 2) Epoch 131: W1_train shape: (2, 3) Epoch 131: b1_train shape: (1, 3) Epoch 131: W2_train shape: (3, 2) Epoch 131: b2_train shape: (1, 2) Epoch 132: X_data_scaled shape: (100, 2) Epoch 132: W1_train shape: (2, 3) Epoch 132: b1_train shape: (1, 3) Epoch 132: W2_train shape: (3, 2) Epoch 132: b2_train shape: (1, 2) Epoch 133: X_data_scaled shape: (100, 2) Epoch 133: W1_train shape: (2, 3) Epoch 133: b1_train shape: (1, 3) Epoch 133: W2_train shape: (3, 2) Epoch 133: b2_train shape: (1, 2) Epoch 134: X_data_scaled shape: (100, 2) Epoch 134: W1_train shape: (2, 3) Epoch 134: b1_train shape: (1, 3) Epoch 134: W2_train shape: (3, 2) Epoch 134: b2_train shape: (1, 2) Epoch 135: X_data_scaled shape: (100, 2) Epoch 135: W1_train shape: (2, 3) Epoch 135: b1_train shape: (1, 3) Epoch 135: W2_train shape: (3, 2) Epoch 135: b2_train shape: (1, 2) Epoch 136: X_data_scaled shape: (100, 2) Epoch 136: W1_train shape: (2, 3) Epoch 136: b1_train shape: (1, 3) Epoch 136: W2_train shape: (3, 2) Epoch 136: b2_train shape: (1, 2) Epoch 137: X_data_scaled shape: (100, 2) Epoch 137: W1_train shape: (2, 3) Epoch 137: b1_train shape: (1, 3) Epoch 137: W2_train shape: (3, 2) Epoch 137: b2_train shape: (1, 2) Epoch 138: X_data_scaled shape: (100, 2) Epoch 138: W1_train shape: (2, 3) Epoch 138: b1_train shape: (1, 3) Epoch 138: W2_train shape: (3, 2) Epoch 138: b2_train shape: (1, 2) Epoch 139: X_data_scaled shape: (100, 2) Epoch 139: W1_train shape: (2, 3) Epoch 139: b1_train shape: (1, 3) Epoch 139: W2_train shape: (3, 2) Epoch 139: b2_train shape: (1, 2) Epoch 140: X_data_scaled shape: (100, 2) Epoch 140: W1_train shape: (2, 3) Epoch 140: b1_train shape: (1, 3) Epoch 140: W2_train shape: (3, 2) Epoch 140: b2_train shape: (1, 2) Epoch 141: X_data_scaled shape: (100, 2) Epoch 141: W1_train shape: (2, 3) Epoch 141: b1_train shape: (1, 3) Epoch 141: W2_train shape: (3, 2) Epoch 141: b2_train shape: (1, 2) Epoch 142: X_data_scaled shape: (100, 2) Epoch 142: W1_train shape: (2, 3) Epoch 142: b1_train shape: (1, 3) Epoch 142: W2_train shape: (3, 2) Epoch 142: b2_train shape: (1, 2) Epoch 143: X_data_scaled shape: (100, 2) Epoch 143: W1_train shape: (2, 3) Epoch 143: b1_train shape: (1, 3) Epoch 143: W2_train shape: (3, 2) Epoch 143: b2_train shape: (1, 2) Epoch 144: X_data_scaled shape: (100, 2) Epoch 144: W1_train shape: (2, 3) Epoch 144: b1_train shape: (1, 3) Epoch 144: W2_train shape: (3, 2) Epoch 144: b2_train shape: (1, 2) Epoch 145: X_data_scaled shape: (100, 2) Epoch 145: W1_train shape: (2, 3) Epoch 145: b1_train shape: (1, 3) Epoch 145: W2_train shape: (3, 2) Epoch 145: b2_train shape: (1, 2) Epoch 146: X_data_scaled shape: (100, 2) Epoch 146: W1_train shape: (2, 3) Epoch 146: b1_train shape: (1, 3) Epoch 146: W2_train shape: (3, 2) Epoch 146: b2_train shape: (1, 2) Epoch 147: X_data_scaled shape: (100, 2) Epoch 147: W1_train shape: (2, 3) Epoch 147: b1_train shape: (1, 3) Epoch 147: W2_train shape: (3, 2) Epoch 147: b2_train shape: (1, 2) Epoch 148: X_data_scaled shape: (100, 2) Epoch 148: W1_train shape: (2, 3) Epoch 148: b1_train shape: (1, 3) Epoch 148: W2_train shape: (3, 2) Epoch 148: b2_train shape: (1, 2) Epoch 149: X_data_scaled shape: (100, 2) Epoch 149: W1_train shape: (2, 3) Epoch 149: b1_train shape: (1, 3) Epoch 149: W2_train shape: (3, 2) Epoch 149: b2_train shape: (1, 2) Epoch 150: X_data_scaled shape: (100, 2) Epoch 150: W1_train shape: (2, 3) Epoch 150: b1_train shape: (1, 3) Epoch 150: W2_train shape: (3, 2) Epoch 150: b2_train shape: (1, 2) Epoch 151: X_data_scaled shape: (100, 2) Epoch 151: W1_train shape: (2, 3) Epoch 151: b1_train shape: (1, 3) Epoch 151: W2_train shape: (3, 2) Epoch 151: b2_train shape: (1, 2) Epoch 152: X_data_scaled shape: (100, 2) Epoch 152: W1_train shape: (2, 3) Epoch 152: b1_train shape: (1, 3) Epoch 152: W2_train shape: (3, 2) Epoch 152: b2_train shape: (1, 2) Epoch 153: X_data_scaled shape: (100, 2) Epoch 153: W1_train shape: (2, 3) Epoch 153: b1_train shape: (1, 3) Epoch 153: W2_train shape: (3, 2) Epoch 153: b2_train shape: (1, 2) Epoch 154: X_data_scaled shape: (100, 2) Epoch 154: W1_train shape: (2, 3) Epoch 154: b1_train shape: (1, 3) Epoch 154: W2_train shape: (3, 2) Epoch 154: b2_train shape: (1, 2) Epoch 155: X_data_scaled shape: (100, 2) Epoch 155: W1_train shape: (2, 3) Epoch 155: b1_train shape: (1, 3) Epoch 155: W2_train shape: (3, 2) Epoch 155: b2_train shape: (1, 2) Epoch 156: X_data_scaled shape: (100, 2) Epoch 156: W1_train shape: (2, 3) Epoch 156: b1_train shape: (1, 3) Epoch 156: W2_train shape: (3, 2) Epoch 156: b2_train shape: (1, 2) Epoch 157: X_data_scaled shape: (100, 2) Epoch 157: W1_train shape: (2, 3) Epoch 157: b1_train shape: (1, 3) Epoch 157: W2_train shape: (3, 2) Epoch 157: b2_train shape: (1, 2) Epoch 158: X_data_scaled shape: (100, 2) Epoch 158: W1_train shape: (2, 3) Epoch 158: b1_train shape: (1, 3) Epoch 158: W2_train shape: (3, 2) Epoch 158: b2_train shape: (1, 2) Epoch 159: X_data_scaled shape: (100, 2) Epoch 159: W1_train shape: (2, 3) Epoch 159: b1_train shape: (1, 3) Epoch 159: W2_train shape: (3, 2) Epoch 159: b2_train shape: (1, 2) Epoch 160: X_data_scaled shape: (100, 2) Epoch 160: W1_train shape: (2, 3) Epoch 160: b1_train shape: (1, 3) Epoch 160: W2_train shape: (3, 2) Epoch 160: b2_train shape: (1, 2) Epoch 161: X_data_scaled shape: (100, 2) Epoch 161: W1_train shape: (2, 3) Epoch 161: b1_train shape: (1, 3) Epoch 161: W2_train shape: (3, 2) Epoch 161: b2_train shape: (1, 2) Epoch 162: X_data_scaled shape: (100, 2) Epoch 162: W1_train shape: (2, 3) Epoch 162: b1_train shape: (1, 3) Epoch 162: W2_train shape: (3, 2) Epoch 162: b2_train shape: (1, 2) Epoch 163: X_data_scaled shape: (100, 2) Epoch 163: W1_train shape: (2, 3) Epoch 163: b1_train shape: (1, 3) Epoch 163: W2_train shape: (3, 2) Epoch 163: b2_train shape: (1, 2) Epoch 164: X_data_scaled shape: (100, 2) Epoch 164: W1_train shape: (2, 3) Epoch 164: b1_train shape: (1, 3) Epoch 164: W2_train shape: (3, 2) Epoch 164: b2_train shape: (1, 2) Epoch 165: X_data_scaled shape: (100, 2) Epoch 165: W1_train shape: (2, 3) Epoch 165: b1_train shape: (1, 3) Epoch 165: W2_train shape: (3, 2) Epoch 165: b2_train shape: (1, 2) Epoch 166: X_data_scaled shape: (100, 2) Epoch 166: W1_train shape: (2, 3) Epoch 166: b1_train shape: (1, 3) Epoch 166: W2_train shape: (3, 2) Epoch 166: b2_train shape: (1, 2) Epoch 167: X_data_scaled shape: (100, 2) Epoch 167: W1_train shape: (2, 3) Epoch 167: b1_train shape: (1, 3) Epoch 167: W2_train shape: (3, 2) Epoch 167: b2_train shape: (1, 2) Epoch 168: X_data_scaled shape: (100, 2) Epoch 168: W1_train shape: (2, 3) Epoch 168: b1_train shape: (1, 3) Epoch 168: W2_train shape: (3, 2) Epoch 168: b2_train shape: (1, 2) Epoch 169: X_data_scaled shape: (100, 2) Epoch 169: W1_train shape: (2, 3) Epoch 169: b1_train shape: (1, 3) Epoch 169: W2_train shape: (3, 2) Epoch 169: b2_train shape: (1, 2) Epoch 170: X_data_scaled shape: (100, 2) Epoch 170: W1_train shape: (2, 3) Epoch 170: b1_train shape: (1, 3) Epoch 170: W2_train shape: (3, 2) Epoch 170: b2_train shape: (1, 2) Epoch 171: X_data_scaled shape: (100, 2) Epoch 171: W1_train shape: (2, 3) Epoch 171: b1_train shape: (1, 3) Epoch 171: W2_train shape: (3, 2) Epoch 171: b2_train shape: (1, 2) Epoch 172: X_data_scaled shape: (100, 2) Epoch 172: W1_train shape: (2, 3) Epoch 172: b1_train shape: (1, 3) Epoch 172: W2_train shape: (3, 2) Epoch 172: b2_train shape: (1, 2) Epoch 173: X_data_scaled shape: (100, 2) Epoch 173: W1_train shape: (2, 3) Epoch 173: b1_train shape: (1, 3) Epoch 173: W2_train shape: (3, 2) Epoch 173: b2_train shape: (1, 2) Epoch 174: X_data_scaled shape: (100, 2) Epoch 174: W1_train shape: (2, 3) Epoch 174: b1_train shape: (1, 3) Epoch 174: W2_train shape: (3, 2) Epoch 174: b2_train shape: (1, 2) Epoch 175: X_data_scaled shape: (100, 2) Epoch 175: W1_train shape: (2, 3) Epoch 175: b1_train shape: (1, 3) Epoch 175: W2_train shape: (3, 2) Epoch 175: b2_train shape: (1, 2) Epoch 176: X_data_scaled shape: (100, 2) Epoch 176: W1_train shape: (2, 3) Epoch 176: b1_train shape: (1, 3) Epoch 176: W2_train shape: (3, 2) Epoch 176: b2_train shape: (1, 2) Epoch 177: X_data_scaled shape: (100, 2) Epoch 177: W1_train shape: (2, 3) Epoch 177: b1_train shape: (1, 3) Epoch 177: W2_train shape: (3, 2) Epoch 177: b2_train shape: (1, 2) Epoch 178: X_data_scaled shape: (100, 2) Epoch 178: W1_train shape: (2, 3) Epoch 178: b1_train shape: (1, 3) Epoch 178: W2_train shape: (3, 2) Epoch 178: b2_train shape: (1, 2) Epoch 179: X_data_scaled shape: (100, 2) Epoch 179: W1_train shape: (2, 3) Epoch 179: b1_train shape: (1, 3) Epoch 179: W2_train shape: (3, 2) Epoch 179: b2_train shape: (1, 2) Epoch 180: X_data_scaled shape: (100, 2) Epoch 180: W1_train shape: (2, 3) Epoch 180: b1_train shape: (1, 3) Epoch 180: W2_train shape: (3, 2) Epoch 180: b2_train shape: (1, 2) Epoch 181: X_data_scaled shape: (100, 2) Epoch 181: W1_train shape: (2, 3) Epoch 181: b1_train shape: (1, 3) Epoch 181: W2_train shape: (3, 2) Epoch 181: b2_train shape: (1, 2) Epoch 182: X_data_scaled shape: (100, 2) Epoch 182: W1_train shape: (2, 3) Epoch 182: b1_train shape: (1, 3) Epoch 182: W2_train shape: (3, 2) Epoch 182: b2_train shape: (1, 2) Epoch 183: X_data_scaled shape: (100, 2) Epoch 183: W1_train shape: (2, 3) Epoch 183: b1_train shape: (1, 3) Epoch 183: W2_train shape: (3, 2) Epoch 183: b2_train shape: (1, 2) Epoch 184: X_data_scaled shape: (100, 2) Epoch 184: W1_train shape: (2, 3) Epoch 184: b1_train shape: (1, 3) Epoch 184: W2_train shape: (3, 2) Epoch 184: b2_train shape: (1, 2) Epoch 185: X_data_scaled shape: (100, 2) Epoch 185: W1_train shape: (2, 3) Epoch 185: b1_train shape: (1, 3) Epoch 185: W2_train shape: (3, 2) Epoch 185: b2_train shape: (1, 2) Epoch 186: X_data_scaled shape: (100, 2) Epoch 186: W1_train shape: (2, 3) Epoch 186: b1_train shape: (1, 3) Epoch 186: W2_train shape: (3, 2) Epoch 186: b2_train shape: (1, 2) Epoch 187: X_data_scaled shape: (100, 2) Epoch 187: W1_train shape: (2, 3) Epoch 187: b1_train shape: (1, 3) Epoch 187: W2_train shape: (3, 2) Epoch 187: b2_train shape: (1, 2) Epoch 188: X_data_scaled shape: (100, 2) Epoch 188: W1_train shape: (2, 3) Epoch 188: b1_train shape: (1, 3) Epoch 188: W2_train shape: (3, 2) Epoch 188: b2_train shape: (1, 2) Epoch 189: X_data_scaled shape: (100, 2) Epoch 189: W1_train shape: (2, 3) Epoch 189: b1_train shape: (1, 3) Epoch 189: W2_train shape: (3, 2) Epoch 189: b2_train shape: (1, 2) Epoch 190: X_data_scaled shape: (100, 2) Epoch 190: W1_train shape: (2, 3) Epoch 190: b1_train shape: (1, 3) Epoch 190: W2_train shape: (3, 2) Epoch 190: b2_train shape: (1, 2) Epoch 191: X_data_scaled shape: (100, 2) Epoch 191: W1_train shape: (2, 3) Epoch 191: b1_train shape: (1, 3) Epoch 191: W2_train shape: (3, 2) Epoch 191: b2_train shape: (1, 2) Epoch 192: X_data_scaled shape: (100, 2) Epoch 192: W1_train shape: (2, 3) Epoch 192: b1_train shape: (1, 3) Epoch 192: W2_train shape: (3, 2) Epoch 192: b2_train shape: (1, 2) Epoch 193: X_data_scaled shape: (100, 2) Epoch 193: W1_train shape: (2, 3) Epoch 193: b1_train shape: (1, 3) Epoch 193: W2_train shape: (3, 2) Epoch 193: b2_train shape: (1, 2) Epoch 194: X_data_scaled shape: (100, 2) Epoch 194: W1_train shape: (2, 3) Epoch 194: b1_train shape: (1, 3) Epoch 194: W2_train shape: (3, 2) Epoch 194: b2_train shape: (1, 2) Epoch 195: X_data_scaled shape: (100, 2) Epoch 195: W1_train shape: (2, 3) Epoch 195: b1_train shape: (1, 3) Epoch 195: W2_train shape: (3, 2) Epoch 195: b2_train shape: (1, 2) Epoch 196: X_data_scaled shape: (100, 2) Epoch 196: W1_train shape: (2, 3) Epoch 196: b1_train shape: (1, 3) Epoch 196: W2_train shape: (3, 2) Epoch 196: b2_train shape: (1, 2) Epoch 197: X_data_scaled shape: (100, 2) Epoch 197: W1_train shape: (2, 3) Epoch 197: b1_train shape: (1, 3) Epoch 197: W2_train shape: (3, 2) Epoch 197: b2_train shape: (1, 2) Epoch 198: X_data_scaled shape: (100, 2) Epoch 198: W1_train shape: (2, 3) Epoch 198: b1_train shape: (1, 3) Epoch 198: W2_train shape: (3, 2) Epoch 198: b2_train shape: (1, 2) Epoch 199: X_data_scaled shape: (100, 2) Epoch 199: W1_train shape: (2, 3) Epoch 199: b1_train shape: (1, 3) Epoch 199: W2_train shape: (3, 2) Epoch 199: b2_train shape: (1, 2) Epoch 200: X_data_scaled shape: (100, 2) Epoch 200: W1_train shape: (2, 3) Epoch 200: b1_train shape: (1, 3) Epoch 200: W2_train shape: (3, 2) Epoch 200: b2_train shape: (1, 2) Epoch 200/1000, Loss: 5.2984 Epoch 201: X_data_scaled shape: (100, 2) Epoch 201: W1_train shape: (2, 3) Epoch 201: b1_train shape: (1, 3) Epoch 201: W2_train shape: (3, 2) Epoch 201: b2_train shape: (1, 2) Epoch 202: X_data_scaled shape: (100, 2) Epoch 202: W1_train shape: (2, 3) Epoch 202: b1_train shape: (1, 3) Epoch 202: W2_train shape: (3, 2) Epoch 202: b2_train shape: (1, 2) Epoch 203: X_data_scaled shape: (100, 2) Epoch 203: W1_train shape: (2, 3) Epoch 203: b1_train shape: (1, 3) Epoch 203: W2_train shape: (3, 2) Epoch 203: b2_train shape: (1, 2) Epoch 204: X_data_scaled shape: (100, 2) Epoch 204: W1_train shape: (2, 3) Epoch 204: b1_train shape: (1, 3) Epoch 204: W2_train shape: (3, 2) Epoch 204: b2_train shape: (1, 2) Epoch 205: X_data_scaled shape: (100, 2) Epoch 205: W1_train shape: (2, 3) Epoch 205: b1_train shape: (1, 3) Epoch 205: W2_train shape: (3, 2) Epoch 205: b2_train shape: (1, 2) Epoch 206: X_data_scaled shape: (100, 2) Epoch 206: W1_train shape: (2, 3) Epoch 206: b1_train shape: (1, 3) Epoch 206: W2_train shape: (3, 2) Epoch 206: b2_train shape: (1, 2) Epoch 207: X_data_scaled shape: (100, 2) Epoch 207: W1_train shape: (2, 3) Epoch 207: b1_train shape: (1, 3) Epoch 207: W2_train shape: (3, 2) Epoch 207: b2_train shape: (1, 2) Epoch 208: X_data_scaled shape: (100, 2) Epoch 208: W1_train shape: (2, 3) Epoch 208: b1_train shape: (1, 3) Epoch 208: W2_train shape: (3, 2) Epoch 208: b2_train shape: (1, 2) Epoch 209: X_data_scaled shape: (100, 2) Epoch 209: W1_train shape: (2, 3) Epoch 209: b1_train shape: (1, 3) Epoch 209: W2_train shape: (3, 2) Epoch 209: b2_train shape: (1, 2) Epoch 210: X_data_scaled shape: (100, 2) Epoch 210: W1_train shape: (2, 3) Epoch 210: b1_train shape: (1, 3) Epoch 210: W2_train shape: (3, 2) Epoch 210: b2_train shape: (1, 2) Epoch 211: X_data_scaled shape: (100, 2) Epoch 211: W1_train shape: (2, 3) Epoch 211: b1_train shape: (1, 3) Epoch 211: W2_train shape: (3, 2) Epoch 211: b2_train shape: (1, 2) Epoch 212: X_data_scaled shape: (100, 2) Epoch 212: W1_train shape: (2, 3) Epoch 212: b1_train shape: (1, 3) Epoch 212: W2_train shape: (3, 2) Epoch 212: b2_train shape: (1, 2) Epoch 213: X_data_scaled shape: (100, 2) Epoch 213: W1_train shape: (2, 3) Epoch 213: b1_train shape: (1, 3) Epoch 213: W2_train shape: (3, 2) Epoch 213: b2_train shape: (1, 2) Epoch 214: X_data_scaled shape: (100, 2) Epoch 214: W1_train shape: (2, 3) Epoch 214: b1_train shape: (1, 3) Epoch 214: W2_train shape: (3, 2) Epoch 214: b2_train shape: (1, 2) Epoch 215: X_data_scaled shape: (100, 2) Epoch 215: W1_train shape: (2, 3) Epoch 215: b1_train shape: (1, 3) Epoch 215: W2_train shape: (3, 2) Epoch 215: b2_train shape: (1, 2) Epoch 216: X_data_scaled shape: (100, 2) Epoch 216: W1_train shape: (2, 3) Epoch 216: b1_train shape: (1, 3) Epoch 216: W2_train shape: (3, 2) Epoch 216: b2_train shape: (1, 2) Epoch 217: X_data_scaled shape: (100, 2) Epoch 217: W1_train shape: (2, 3) Epoch 217: b1_train shape: (1, 3) Epoch 217: W2_train shape: (3, 2) Epoch 217: b2_train shape: (1, 2) Epoch 218: X_data_scaled shape: (100, 2) Epoch 218: W1_train shape: (2, 3) Epoch 218: b1_train shape: (1, 3) Epoch 218: W2_train shape: (3, 2) Epoch 218: b2_train shape: (1, 2) Epoch 219: X_data_scaled shape: (100, 2) Epoch 219: W1_train shape: (2, 3) Epoch 219: b1_train shape: (1, 3) Epoch 219: W2_train shape: (3, 2) Epoch 219: b2_train shape: (1, 2) Epoch 220: X_data_scaled shape: (100, 2) Epoch 220: W1_train shape: (2, 3) Epoch 220: b1_train shape: (1, 3) Epoch 220: W2_train shape: (3, 2) Epoch 220: b2_train shape: (1, 2) Epoch 221: X_data_scaled shape: (100, 2) Epoch 221: W1_train shape: (2, 3) Epoch 221: b1_train shape: (1, 3) Epoch 221: W2_train shape: (3, 2) Epoch 221: b2_train shape: (1, 2) Epoch 222: X_data_scaled shape: (100, 2) Epoch 222: W1_train shape: (2, 3) Epoch 222: b1_train shape: (1, 3) Epoch 222: W2_train shape: (3, 2) Epoch 222: b2_train shape: (1, 2) Epoch 223: X_data_scaled shape: (100, 2) Epoch 223: W1_train shape: (2, 3) Epoch 223: b1_train shape: (1, 3) Epoch 223: W2_train shape: (3, 2) Epoch 223: b2_train shape: (1, 2) Epoch 224: X_data_scaled shape: (100, 2) Epoch 224: W1_train shape: (2, 3) Epoch 224: b1_train shape: (1, 3) Epoch 224: W2_train shape: (3, 2) Epoch 224: b2_train shape: (1, 2) Epoch 225: X_data_scaled shape: (100, 2) Epoch 225: W1_train shape: (2, 3) Epoch 225: b1_train shape: (1, 3) Epoch 225: W2_train shape: (3, 2) Epoch 225: b2_train shape: (1, 2) Epoch 226: X_data_scaled shape: (100, 2) Epoch 226: W1_train shape: (2, 3) Epoch 226: b1_train shape: (1, 3) Epoch 226: W2_train shape: (3, 2) Epoch 226: b2_train shape: (1, 2) Epoch 227: X_data_scaled shape: (100, 2) Epoch 227: W1_train shape: (2, 3) Epoch 227: b1_train shape: (1, 3) Epoch 227: W2_train shape: (3, 2) Epoch 227: b2_train shape: (1, 2) Epoch 228: X_data_scaled shape: (100, 2) Epoch 228: W1_train shape: (2, 3) Epoch 228: b1_train shape: (1, 3) Epoch 228: W2_train shape: (3, 2) Epoch 228: b2_train shape: (1, 2) Epoch 229: X_data_scaled shape: (100, 2) Epoch 229: W1_train shape: (2, 3) Epoch 229: b1_train shape: (1, 3) Epoch 229: W2_train shape: (3, 2) Epoch 229: b2_train shape: (1, 2) Epoch 230: X_data_scaled shape: (100, 2) Epoch 230: W1_train shape: (2, 3) Epoch 230: b1_train shape: (1, 3) Epoch 230: W2_train shape: (3, 2) Epoch 230: b2_train shape: (1, 2) Epoch 231: X_data_scaled shape: (100, 2) Epoch 231: W1_train shape: (2, 3) Epoch 231: b1_train shape: (1, 3) Epoch 231: W2_train shape: (3, 2) Epoch 231: b2_train shape: (1, 2) Epoch 232: X_data_scaled shape: (100, 2) Epoch 232: W1_train shape: (2, 3) Epoch 232: b1_train shape: (1, 3) Epoch 232: W2_train shape: (3, 2) Epoch 232: b2_train shape: (1, 2) Epoch 233: X_data_scaled shape: (100, 2) Epoch 233: W1_train shape: (2, 3) Epoch 233: b1_train shape: (1, 3) Epoch 233: W2_train shape: (3, 2) Epoch 233: b2_train shape: (1, 2) Epoch 234: X_data_scaled shape: (100, 2) Epoch 234: W1_train shape: (2, 3) Epoch 234: b1_train shape: (1, 3) Epoch 234: W2_train shape: (3, 2) Epoch 234: b2_train shape: (1, 2) Epoch 235: X_data_scaled shape: (100, 2) Epoch 235: W1_train shape: (2, 3) Epoch 235: b1_train shape: (1, 3) Epoch 235: W2_train shape: (3, 2) Epoch 235: b2_train shape: (1, 2) Epoch 236: X_data_scaled shape: (100, 2) Epoch 236: W1_train shape: (2, 3) Epoch 236: b1_train shape: (1, 3) Epoch 236: W2_train shape: (3, 2) Epoch 236: b2_train shape: (1, 2) Epoch 237: X_data_scaled shape: (100, 2) Epoch 237: W1_train shape: (2, 3) Epoch 237: b1_train shape: (1, 3) Epoch 237: W2_train shape: (3, 2) Epoch 237: b2_train shape: (1, 2) Epoch 238: X_data_scaled shape: (100, 2) Epoch 238: W1_train shape: (2, 3) Epoch 238: b1_train shape: (1, 3) Epoch 238: W2_train shape: (3, 2) Epoch 238: b2_train shape: (1, 2) Epoch 239: X_data_scaled shape: (100, 2) Epoch 239: W1_train shape: (2, 3) Epoch 239: b1_train shape: (1, 3) Epoch 239: W2_train shape: (3, 2) Epoch 239: b2_train shape: (1, 2) Epoch 240: X_data_scaled shape: (100, 2) Epoch 240: W1_train shape: (2, 3) Epoch 240: b1_train shape: (1, 3) Epoch 240: W2_train shape: (3, 2) Epoch 240: b2_train shape: (1, 2) Epoch 241: X_data_scaled shape: (100, 2) Epoch 241: W1_train shape: (2, 3) Epoch 241: b1_train shape: (1, 3) Epoch 241: W2_train shape: (3, 2) Epoch 241: b2_train shape: (1, 2) Epoch 242: X_data_scaled shape: (100, 2) Epoch 242: W1_train shape: (2, 3) Epoch 242: b1_train shape: (1, 3) Epoch 242: W2_train shape: (3, 2) Epoch 242: b2_train shape: (1, 2) Epoch 243: X_data_scaled shape: (100, 2) Epoch 243: W1_train shape: (2, 3) Epoch 243: b1_train shape: (1, 3) Epoch 243: W2_train shape: (3, 2) Epoch 243: b2_train shape: (1, 2) Epoch 244: X_data_scaled shape: (100, 2) Epoch 244: W1_train shape: (2, 3) Epoch 244: b1_train shape: (1, 3) Epoch 244: W2_train shape: (3, 2) Epoch 244: b2_train shape: (1, 2) Epoch 245: X_data_scaled shape: (100, 2) Epoch 245: W1_train shape: (2, 3) Epoch 245: b1_train shape: (1, 3) Epoch 245: W2_train shape: (3, 2) Epoch 245: b2_train shape: (1, 2) Epoch 246: X_data_scaled shape: (100, 2) Epoch 246: W1_train shape: (2, 3) Epoch 246: b1_train shape: (1, 3) Epoch 246: W2_train shape: (3, 2) Epoch 246: b2_train shape: (1, 2) Epoch 247: X_data_scaled shape: (100, 2) Epoch 247: W1_train shape: (2, 3) Epoch 247: b1_train shape: (1, 3) Epoch 247: W2_train shape: (3, 2) Epoch 247: b2_train shape: (1, 2) Epoch 248: X_data_scaled shape: (100, 2) Epoch 248: W1_train shape: (2, 3) Epoch 248: b1_train shape: (1, 3) Epoch 248: W2_train shape: (3, 2) Epoch 248: b2_train shape: (1, 2) Epoch 249: X_data_scaled shape: (100, 2) Epoch 249: W1_train shape: (2, 3) Epoch 249: b1_train shape: (1, 3) Epoch 249: W2_train shape: (3, 2) Epoch 249: b2_train shape: (1, 2) Epoch 250: X_data_scaled shape: (100, 2) Epoch 250: W1_train shape: (2, 3) Epoch 250: b1_train shape: (1, 3) Epoch 250: W2_train shape: (3, 2) Epoch 250: b2_train shape: (1, 2) Epoch 251: X_data_scaled shape: (100, 2) Epoch 251: W1_train shape: (2, 3) Epoch 251: b1_train shape: (1, 3) Epoch 251: W2_train shape: (3, 2) Epoch 251: b2_train shape: (1, 2) Epoch 252: X_data_scaled shape: (100, 2) Epoch 252: W1_train shape: (2, 3) Epoch 252: b1_train shape: (1, 3) Epoch 252: W2_train shape: (3, 2) Epoch 252: b2_train shape: (1, 2) Epoch 253: X_data_scaled shape: (100, 2) Epoch 253: W1_train shape: (2, 3) Epoch 253: b1_train shape: (1, 3) Epoch 253: W2_train shape: (3, 2) Epoch 253: b2_train shape: (1, 2) Epoch 254: X_data_scaled shape: (100, 2) Epoch 254: W1_train shape: (2, 3) Epoch 254: b1_train shape: (1, 3) Epoch 254: W2_train shape: (3, 2) Epoch 254: b2_train shape: (1, 2) Epoch 255: X_data_scaled shape: (100, 2) Epoch 255: W1_train shape: (2, 3) Epoch 255: b1_train shape: (1, 3) Epoch 255: W2_train shape: (3, 2) Epoch 255: b2_train shape: (1, 2) Epoch 256: X_data_scaled shape: (100, 2) Epoch 256: W1_train shape: (2, 3) Epoch 256: b1_train shape: (1, 3) Epoch 256: W2_train shape: (3, 2) Epoch 256: b2_train shape: (1, 2) Epoch 257: X_data_scaled shape: (100, 2) Epoch 257: W1_train shape: (2, 3) Epoch 257: b1_train shape: (1, 3) Epoch 257: W2_train shape: (3, 2) Epoch 257: b2_train shape: (1, 2) Epoch 258: X_data_scaled shape: (100, 2) Epoch 258: W1_train shape: (2, 3) Epoch 258: b1_train shape: (1, 3) Epoch 258: W2_train shape: (3, 2) Epoch 258: b2_train shape: (1, 2) Epoch 259: X_data_scaled shape: (100, 2) Epoch 259: W1_train shape: (2, 3) Epoch 259: b1_train shape: (1, 3) Epoch 259: W2_train shape: (3, 2) Epoch 259: b2_train shape: (1, 2) Epoch 260: X_data_scaled shape: (100, 2) Epoch 260: W1_train shape: (2, 3) Epoch 260: b1_train shape: (1, 3) Epoch 260: W2_train shape: (3, 2) Epoch 260: b2_train shape: (1, 2) Epoch 261: X_data_scaled shape: (100, 2) Epoch 261: W1_train shape: (2, 3) Epoch 261: b1_train shape: (1, 3) Epoch 261: W2_train shape: (3, 2) Epoch 261: b2_train shape: (1, 2) Epoch 262: X_data_scaled shape: (100, 2) Epoch 262: W1_train shape: (2, 3) Epoch 262: b1_train shape: (1, 3) Epoch 262: W2_train shape: (3, 2) Epoch 262: b2_train shape: (1, 2) Epoch 263: X_data_scaled shape: (100, 2) Epoch 263: W1_train shape: (2, 3) Epoch 263: b1_train shape: (1, 3) Epoch 263: W2_train shape: (3, 2) Epoch 263: b2_train shape: (1, 2) Epoch 264: X_data_scaled shape: (100, 2) Epoch 264: W1_train shape: (2, 3) Epoch 264: b1_train shape: (1, 3) Epoch 264: W2_train shape: (3, 2) Epoch 264: b2_train shape: (1, 2) Epoch 265: X_data_scaled shape: (100, 2) Epoch 265: W1_train shape: (2, 3) Epoch 265: b1_train shape: (1, 3) Epoch 265: W2_train shape: (3, 2) Epoch 265: b2_train shape: (1, 2) Epoch 266: X_data_scaled shape: (100, 2) Epoch 266: W1_train shape: (2, 3) Epoch 266: b1_train shape: (1, 3) Epoch 266: W2_train shape: (3, 2) Epoch 266: b2_train shape: (1, 2) Epoch 267: X_data_scaled shape: (100, 2) Epoch 267: W1_train shape: (2, 3) Epoch 267: b1_train shape: (1, 3) Epoch 267: W2_train shape: (3, 2) Epoch 267: b2_train shape: (1, 2) Epoch 268: X_data_scaled shape: (100, 2) Epoch 268: W1_train shape: (2, 3) Epoch 268: b1_train shape: (1, 3) Epoch 268: W2_train shape: (3, 2) Epoch 268: b2_train shape: (1, 2) Epoch 269: X_data_scaled shape: (100, 2) Epoch 269: W1_train shape: (2, 3) Epoch 269: b1_train shape: (1, 3) Epoch 269: W2_train shape: (3, 2) Epoch 269: b2_train shape: (1, 2) Epoch 270: X_data_scaled shape: (100, 2) Epoch 270: W1_train shape: (2, 3) Epoch 270: b1_train shape: (1, 3) Epoch 270: W2_train shape: (3, 2) Epoch 270: b2_train shape: (1, 2) Epoch 271: X_data_scaled shape: (100, 2) Epoch 271: W1_train shape: (2, 3) Epoch 271: b1_train shape: (1, 3) Epoch 271: W2_train shape: (3, 2) Epoch 271: b2_train shape: (1, 2) Epoch 272: X_data_scaled shape: (100, 2) Epoch 272: W1_train shape: (2, 3) Epoch 272: b1_train shape: (1, 3) Epoch 272: W2_train shape: (3, 2) Epoch 272: b2_train shape: (1, 2) Epoch 273: X_data_scaled shape: (100, 2) Epoch 273: W1_train shape: (2, 3) Epoch 273: b1_train shape: (1, 3) Epoch 273: W2_train shape: (3, 2) Epoch 273: b2_train shape: (1, 2) Epoch 274: X_data_scaled shape: (100, 2) Epoch 274: W1_train shape: (2, 3) Epoch 274: b1_train shape: (1, 3) Epoch 274: W2_train shape: (3, 2) Epoch 274: b2_train shape: (1, 2) Epoch 275: X_data_scaled shape: (100, 2) Epoch 275: W1_train shape: (2, 3) Epoch 275: b1_train shape: (1, 3) Epoch 275: W2_train shape: (3, 2) Epoch 275: b2_train shape: (1, 2) Epoch 276: X_data_scaled shape: (100, 2) Epoch 276: W1_train shape: (2, 3) Epoch 276: b1_train shape: (1, 3) Epoch 276: W2_train shape: (3, 2) Epoch 276: b2_train shape: (1, 2) Epoch 277: X_data_scaled shape: (100, 2) Epoch 277: W1_train shape: (2, 3) Epoch 277: b1_train shape: (1, 3) Epoch 277: W2_train shape: (3, 2) Epoch 277: b2_train shape: (1, 2) Epoch 278: X_data_scaled shape: (100, 2) Epoch 278: W1_train shape: (2, 3) Epoch 278: b1_train shape: (1, 3) Epoch 278: W2_train shape: (3, 2) Epoch 278: b2_train shape: (1, 2) Epoch 279: X_data_scaled shape: (100, 2) Epoch 279: W1_train shape: (2, 3) Epoch 279: b1_train shape: (1, 3) Epoch 279: W2_train shape: (3, 2) Epoch 279: b2_train shape: (1, 2) Epoch 280: X_data_scaled shape: (100, 2) Epoch 280: W1_train shape: (2, 3) Epoch 280: b1_train shape: (1, 3) Epoch 280: W2_train shape: (3, 2) Epoch 280: b2_train shape: (1, 2) Epoch 281: X_data_scaled shape: (100, 2) Epoch 281: W1_train shape: (2, 3) Epoch 281: b1_train shape: (1, 3) Epoch 281: W2_train shape: (3, 2) Epoch 281: b2_train shape: (1, 2) Epoch 282: X_data_scaled shape: (100, 2) Epoch 282: W1_train shape: (2, 3) Epoch 282: b1_train shape: (1, 3) Epoch 282: W2_train shape: (3, 2) Epoch 282: b2_train shape: (1, 2) Epoch 283: X_data_scaled shape: (100, 2) Epoch 283: W1_train shape: (2, 3) Epoch 283: b1_train shape: (1, 3) Epoch 283: W2_train shape: (3, 2) Epoch 283: b2_train shape: (1, 2) Epoch 284: X_data_scaled shape: (100, 2) Epoch 284: W1_train shape: (2, 3) Epoch 284: b1_train shape: (1, 3) Epoch 284: W2_train shape: (3, 2) Epoch 284: b2_train shape: (1, 2) Epoch 285: X_data_scaled shape: (100, 2) Epoch 285: W1_train shape: (2, 3) Epoch 285: b1_train shape: (1, 3) Epoch 285: W2_train shape: (3, 2) Epoch 285: b2_train shape: (1, 2) Epoch 286: X_data_scaled shape: (100, 2) Epoch 286: W1_train shape: (2, 3) Epoch 286: b1_train shape: (1, 3) Epoch 286: W2_train shape: (3, 2) Epoch 286: b2_train shape: (1, 2) Epoch 287: X_data_scaled shape: (100, 2) Epoch 287: W1_train shape: (2, 3) Epoch 287: b1_train shape: (1, 3) Epoch 287: W2_train shape: (3, 2) Epoch 287: b2_train shape: (1, 2) Epoch 288: X_data_scaled shape: (100, 2) Epoch 288: W1_train shape: (2, 3) Epoch 288: b1_train shape: (1, 3) Epoch 288: W2_train shape: (3, 2) Epoch 288: b2_train shape: (1, 2) Epoch 289: X_data_scaled shape: (100, 2) Epoch 289: W1_train shape: (2, 3) Epoch 289: b1_train shape: (1, 3) Epoch 289: W2_train shape: (3, 2) Epoch 289: b2_train shape: (1, 2) Epoch 290: X_data_scaled shape: (100, 2) Epoch 290: W1_train shape: (2, 3) Epoch 290: b1_train shape: (1, 3) Epoch 290: W2_train shape: (3, 2) Epoch 290: b2_train shape: (1, 2) Epoch 291: X_data_scaled shape: (100, 2) Epoch 291: W1_train shape: (2, 3) Epoch 291: b1_train shape: (1, 3) Epoch 291: W2_train shape: (3, 2) Epoch 291: b2_train shape: (1, 2) Epoch 292: X_data_scaled shape: (100, 2) Epoch 292: W1_train shape: (2, 3) Epoch 292: b1_train shape: (1, 3) Epoch 292: W2_train shape: (3, 2) Epoch 292: b2_train shape: (1, 2) Epoch 293: X_data_scaled shape: (100, 2) Epoch 293: W1_train shape: (2, 3) Epoch 293: b1_train shape: (1, 3) Epoch 293: W2_train shape: (3, 2) Epoch 293: b2_train shape: (1, 2) Epoch 294: X_data_scaled shape: (100, 2) Epoch 294: W1_train shape: (2, 3) Epoch 294: b1_train shape: (1, 3) Epoch 294: W2_train shape: (3, 2) Epoch 294: b2_train shape: (1, 2) Epoch 295: X_data_scaled shape: (100, 2) Epoch 295: W1_train shape: (2, 3) Epoch 295: b1_train shape: (1, 3) Epoch 295: W2_train shape: (3, 2) Epoch 295: b2_train shape: (1, 2) Epoch 296: X_data_scaled shape: (100, 2) Epoch 296: W1_train shape: (2, 3) Epoch 296: b1_train shape: (1, 3) Epoch 296: W2_train shape: (3, 2) Epoch 296: b2_train shape: (1, 2) Epoch 297: X_data_scaled shape: (100, 2) Epoch 297: W1_train shape: (2, 3) Epoch 297: b1_train shape: (1, 3) Epoch 297: W2_train shape: (3, 2) Epoch 297: b2_train shape: (1, 2) Epoch 298: X_data_scaled shape: (100, 2) Epoch 298: W1_train shape: (2, 3) Epoch 298: b1_train shape: (1, 3) Epoch 298: W2_train shape: (3, 2) Epoch 298: b2_train shape: (1, 2) Epoch 299: X_data_scaled shape: (100, 2) Epoch 299: W1_train shape: (2, 3) Epoch 299: b1_train shape: (1, 3) Epoch 299: W2_train shape: (3, 2) Epoch 299: b2_train shape: (1, 2) Epoch 300: X_data_scaled shape: (100, 2) Epoch 300: W1_train shape: (2, 3) Epoch 300: b1_train shape: (1, 3) Epoch 300: W2_train shape: (3, 2) Epoch 300: b2_train shape: (1, 2) Epoch 300/1000, Loss: 5.2983 Epoch 301: X_data_scaled shape: (100, 2) Epoch 301: W1_train shape: (2, 3) Epoch 301: b1_train shape: (1, 3) Epoch 301: W2_train shape: (3, 2) Epoch 301: b2_train shape: (1, 2) Epoch 302: X_data_scaled shape: (100, 2) Epoch 302: W1_train shape: (2, 3) Epoch 302: b1_train shape: (1, 3) Epoch 302: W2_train shape: (3, 2) Epoch 302: b2_train shape: (1, 2) Epoch 303: X_data_scaled shape: (100, 2) Epoch 303: W1_train shape: (2, 3) Epoch 303: b1_train shape: (1, 3) Epoch 303: W2_train shape: (3, 2) Epoch 303: b2_train shape: (1, 2) Epoch 304: X_data_scaled shape: (100, 2) Epoch 304: W1_train shape: (2, 3) Epoch 304: b1_train shape: (1, 3) Epoch 304: W2_train shape: (3, 2) Epoch 304: b2_train shape: (1, 2) Epoch 305: X_data_scaled shape: (100, 2) Epoch 305: W1_train shape: (2, 3) Epoch 305: b1_train shape: (1, 3) Epoch 305: W2_train shape: (3, 2) Epoch 305: b2_train shape: (1, 2) Epoch 306: X_data_scaled shape: (100, 2) Epoch 306: W1_train shape: (2, 3) Epoch 306: b1_train shape: (1, 3) Epoch 306: W2_train shape: (3, 2) Epoch 306: b2_train shape: (1, 2) Epoch 307: X_data_scaled shape: (100, 2) Epoch 307: W1_train shape: (2, 3) Epoch 307: b1_train shape: (1, 3) Epoch 307: W2_train shape: (3, 2) Epoch 307: b2_train shape: (1, 2) Epoch 308: X_data_scaled shape: (100, 2) Epoch 308: W1_train shape: (2, 3) Epoch 308: b1_train shape: (1, 3) Epoch 308: W2_train shape: (3, 2) Epoch 308: b2_train shape: (1, 2) Epoch 309: X_data_scaled shape: (100, 2) Epoch 309: W1_train shape: (2, 3) Epoch 309: b1_train shape: (1, 3) Epoch 309: W2_train shape: (3, 2) Epoch 309: b2_train shape: (1, 2) Epoch 310: X_data_scaled shape: (100, 2) Epoch 310: W1_train shape: (2, 3) Epoch 310: b1_train shape: (1, 3) Epoch 310: W2_train shape: (3, 2) Epoch 310: b2_train shape: (1, 2) Epoch 311: X_data_scaled shape: (100, 2) Epoch 311: W1_train shape: (2, 3) Epoch 311: b1_train shape: (1, 3) Epoch 311: W2_train shape: (3, 2) Epoch 311: b2_train shape: (1, 2) Epoch 312: X_data_scaled shape: (100, 2) Epoch 312: W1_train shape: (2, 3) Epoch 312: b1_train shape: (1, 3) Epoch 312: W2_train shape: (3, 2) Epoch 312: b2_train shape: (1, 2) Epoch 313: X_data_scaled shape: (100, 2) Epoch 313: W1_train shape: (2, 3) Epoch 313: b1_train shape: (1, 3) Epoch 313: W2_train shape: (3, 2) Epoch 313: b2_train shape: (1, 2) Epoch 314: X_data_scaled shape: (100, 2) Epoch 314: W1_train shape: (2, 3) Epoch 314: b1_train shape: (1, 3) Epoch 314: W2_train shape: (3, 2) Epoch 314: b2_train shape: (1, 2) Epoch 315: X_data_scaled shape: (100, 2) Epoch 315: W1_train shape: (2, 3) Epoch 315: b1_train shape: (1, 3) Epoch 315: W2_train shape: (3, 2) Epoch 315: b2_train shape: (1, 2) Epoch 316: X_data_scaled shape: (100, 2) Epoch 316: W1_train shape: (2, 3) Epoch 316: b1_train shape: (1, 3) Epoch 316: W2_train shape: (3, 2) Epoch 316: b2_train shape: (1, 2) Epoch 317: X_data_scaled shape: (100, 2) Epoch 317: W1_train shape: (2, 3) Epoch 317: b1_train shape: (1, 3) Epoch 317: W2_train shape: (3, 2) Epoch 317: b2_train shape: (1, 2) Epoch 318: X_data_scaled shape: (100, 2) Epoch 318: W1_train shape: (2, 3) Epoch 318: b1_train shape: (1, 3) Epoch 318: W2_train shape: (3, 2) Epoch 318: b2_train shape: (1, 2) Epoch 319: X_data_scaled shape: (100, 2) Epoch 319: W1_train shape: (2, 3) Epoch 319: b1_train shape: (1, 3) Epoch 319: W2_train shape: (3, 2) Epoch 319: b2_train shape: (1, 2) Epoch 320: X_data_scaled shape: (100, 2) Epoch 320: W1_train shape: (2, 3) Epoch 320: b1_train shape: (1, 3) Epoch 320: W2_train shape: (3, 2) Epoch 320: b2_train shape: (1, 2) Epoch 321: X_data_scaled shape: (100, 2) Epoch 321: W1_train shape: (2, 3) Epoch 321: b1_train shape: (1, 3) Epoch 321: W2_train shape: (3, 2) Epoch 321: b2_train shape: (1, 2) Epoch 322: X_data_scaled shape: (100, 2) Epoch 322: W1_train shape: (2, 3) Epoch 322: b1_train shape: (1, 3) Epoch 322: W2_train shape: (3, 2) Epoch 322: b2_train shape: (1, 2) Epoch 323: X_data_scaled shape: (100, 2) Epoch 323: W1_train shape: (2, 3) Epoch 323: b1_train shape: (1, 3) Epoch 323: W2_train shape: (3, 2) Epoch 323: b2_train shape: (1, 2) Epoch 324: X_data_scaled shape: (100, 2) Epoch 324: W1_train shape: (2, 3) Epoch 324: b1_train shape: (1, 3) Epoch 324: W2_train shape: (3, 2) Epoch 324: b2_train shape: (1, 2) Epoch 325: X_data_scaled shape: (100, 2) Epoch 325: W1_train shape: (2, 3) Epoch 325: b1_train shape: (1, 3) Epoch 325: W2_train shape: (3, 2) Epoch 325: b2_train shape: (1, 2) Epoch 326: X_data_scaled shape: (100, 2) Epoch 326: W1_train shape: (2, 3) Epoch 326: b1_train shape: (1, 3) Epoch 326: W2_train shape: (3, 2) Epoch 326: b2_train shape: (1, 2) Epoch 327: X_data_scaled shape: (100, 2) Epoch 327: W1_train shape: (2, 3) Epoch 327: b1_train shape: (1, 3) Epoch 327: W2_train shape: (3, 2) Epoch 327: b2_train shape: (1, 2) Epoch 328: X_data_scaled shape: (100, 2) Epoch 328: W1_train shape: (2, 3) Epoch 328: b1_train shape: (1, 3) Epoch 328: W2_train shape: (3, 2) Epoch 328: b2_train shape: (1, 2) Epoch 329: X_data_scaled shape: (100, 2) Epoch 329: W1_train shape: (2, 3) Epoch 329: b1_train shape: (1, 3) Epoch 329: W2_train shape: (3, 2) Epoch 329: b2_train shape: (1, 2) Epoch 330: X_data_scaled shape: (100, 2) Epoch 330: W1_train shape: (2, 3) Epoch 330: b1_train shape: (1, 3) Epoch 330: W2_train shape: (3, 2) Epoch 330: b2_train shape: (1, 2) Epoch 331: X_data_scaled shape: (100, 2) Epoch 331: W1_train shape: (2, 3) Epoch 331: b1_train shape: (1, 3) Epoch 331: W2_train shape: (3, 2) Epoch 331: b2_train shape: (1, 2) Epoch 332: X_data_scaled shape: (100, 2) Epoch 332: W1_train shape: (2, 3) Epoch 332: b1_train shape: (1, 3) Epoch 332: W2_train shape: (3, 2) Epoch 332: b2_train shape: (1, 2) Epoch 333: X_data_scaled shape: (100, 2) Epoch 333: W1_train shape: (2, 3) Epoch 333: b1_train shape: (1, 3) Epoch 333: W2_train shape: (3, 2) Epoch 333: b2_train shape: (1, 2) Epoch 334: X_data_scaled shape: (100, 2) Epoch 334: W1_train shape: (2, 3) Epoch 334: b1_train shape: (1, 3) Epoch 334: W2_train shape: (3, 2) Epoch 334: b2_train shape: (1, 2) Epoch 335: X_data_scaled shape: (100, 2) Epoch 335: W1_train shape: (2, 3) Epoch 335: b1_train shape: (1, 3) Epoch 335: W2_train shape: (3, 2) Epoch 335: b2_train shape: (1, 2) Epoch 336: X_data_scaled shape: (100, 2) Epoch 336: W1_train shape: (2, 3) Epoch 336: b1_train shape: (1, 3) Epoch 336: W2_train shape: (3, 2) Epoch 336: b2_train shape: (1, 2) Epoch 337: X_data_scaled shape: (100, 2) Epoch 337: W1_train shape: (2, 3) Epoch 337: b1_train shape: (1, 3) Epoch 337: W2_train shape: (3, 2) Epoch 337: b2_train shape: (1, 2) Epoch 338: X_data_scaled shape: (100, 2) Epoch 338: W1_train shape: (2, 3) Epoch 338: b1_train shape: (1, 3) Epoch 338: W2_train shape: (3, 2) Epoch 338: b2_train shape: (1, 2) Epoch 339: X_data_scaled shape: (100, 2) Epoch 339: W1_train shape: (2, 3) Epoch 339: b1_train shape: (1, 3) Epoch 339: W2_train shape: (3, 2) Epoch 339: b2_train shape: (1, 2) Epoch 340: X_data_scaled shape: (100, 2) Epoch 340: W1_train shape: (2, 3) Epoch 340: b1_train shape: (1, 3) Epoch 340: W2_train shape: (3, 2) Epoch 340: b2_train shape: (1, 2) Epoch 341: X_data_scaled shape: (100, 2) Epoch 341: W1_train shape: (2, 3) Epoch 341: b1_train shape: (1, 3) Epoch 341: W2_train shape: (3, 2) Epoch 341: b2_train shape: (1, 2) Epoch 342: X_data_scaled shape: (100, 2) Epoch 342: W1_train shape: (2, 3) Epoch 342: b1_train shape: (1, 3) Epoch 342: W2_train shape: (3, 2) Epoch 342: b2_train shape: (1, 2) Epoch 343: X_data_scaled shape: (100, 2) Epoch 343: W1_train shape: (2, 3) Epoch 343: b1_train shape: (1, 3) Epoch 343: W2_train shape: (3, 2) Epoch 343: b2_train shape: (1, 2) Epoch 344: X_data_scaled shape: (100, 2) Epoch 344: W1_train shape: (2, 3) Epoch 344: b1_train shape: (1, 3) Epoch 344: W2_train shape: (3, 2) Epoch 344: b2_train shape: (1, 2) Epoch 345: X_data_scaled shape: (100, 2) Epoch 345: W1_train shape: (2, 3) Epoch 345: b1_train shape: (1, 3) Epoch 345: W2_train shape: (3, 2) Epoch 345: b2_train shape: (1, 2) Epoch 346: X_data_scaled shape: (100, 2) Epoch 346: W1_train shape: (2, 3) Epoch 346: b1_train shape: (1, 3) Epoch 346: W2_train shape: (3, 2) Epoch 346: b2_train shape: (1, 2) Epoch 347: X_data_scaled shape: (100, 2) Epoch 347: W1_train shape: (2, 3) Epoch 347: b1_train shape: (1, 3) Epoch 347: W2_train shape: (3, 2) Epoch 347: b2_train shape: (1, 2) Epoch 348: X_data_scaled shape: (100, 2) Epoch 348: W1_train shape: (2, 3) Epoch 348: b1_train shape: (1, 3) Epoch 348: W2_train shape: (3, 2) Epoch 348: b2_train shape: (1, 2) Epoch 349: X_data_scaled shape: (100, 2) Epoch 349: W1_train shape: (2, 3) Epoch 349: b1_train shape: (1, 3) Epoch 349: W2_train shape: (3, 2) Epoch 349: b2_train shape: (1, 2) Epoch 350: X_data_scaled shape: (100, 2) Epoch 350: W1_train shape: (2, 3) Epoch 350: b1_train shape: (1, 3) Epoch 350: W2_train shape: (3, 2) Epoch 350: b2_train shape: (1, 2) Epoch 351: X_data_scaled shape: (100, 2) Epoch 351: W1_train shape: (2, 3) Epoch 351: b1_train shape: (1, 3) Epoch 351: W2_train shape: (3, 2) Epoch 351: b2_train shape: (1, 2) Epoch 352: X_data_scaled shape: (100, 2) Epoch 352: W1_train shape: (2, 3) Epoch 352: b1_train shape: (1, 3) Epoch 352: W2_train shape: (3, 2) Epoch 352: b2_train shape: (1, 2) Epoch 353: X_data_scaled shape: (100, 2) Epoch 353: W1_train shape: (2, 3) Epoch 353: b1_train shape: (1, 3) Epoch 353: W2_train shape: (3, 2) Epoch 353: b2_train shape: (1, 2) Epoch 354: X_data_scaled shape: (100, 2) Epoch 354: W1_train shape: (2, 3) Epoch 354: b1_train shape: (1, 3) Epoch 354: W2_train shape: (3, 2) Epoch 354: b2_train shape: (1, 2) Epoch 355: X_data_scaled shape: (100, 2) Epoch 355: W1_train shape: (2, 3) Epoch 355: b1_train shape: (1, 3) Epoch 355: W2_train shape: (3, 2) Epoch 355: b2_train shape: (1, 2) Epoch 356: X_data_scaled shape: (100, 2) Epoch 356: W1_train shape: (2, 3) Epoch 356: b1_train shape: (1, 3) Epoch 356: W2_train shape: (3, 2) Epoch 356: b2_train shape: (1, 2) Epoch 357: X_data_scaled shape: (100, 2) Epoch 357: W1_train shape: (2, 3) Epoch 357: b1_train shape: (1, 3) Epoch 357: W2_train shape: (3, 2) Epoch 357: b2_train shape: (1, 2) Epoch 358: X_data_scaled shape: (100, 2) Epoch 358: W1_train shape: (2, 3) Epoch 358: b1_train shape: (1, 3) Epoch 358: W2_train shape: (3, 2) Epoch 358: b2_train shape: (1, 2) Epoch 359: X_data_scaled shape: (100, 2) Epoch 359: W1_train shape: (2, 3) Epoch 359: b1_train shape: (1, 3) Epoch 359: W2_train shape: (3, 2) Epoch 359: b2_train shape: (1, 2) Epoch 360: X_data_scaled shape: (100, 2) Epoch 360: W1_train shape: (2, 3) Epoch 360: b1_train shape: (1, 3) Epoch 360: W2_train shape: (3, 2) Epoch 360: b2_train shape: (1, 2) Epoch 361: X_data_scaled shape: (100, 2) Epoch 361: W1_train shape: (2, 3) Epoch 361: b1_train shape: (1, 3) Epoch 361: W2_train shape: (3, 2) Epoch 361: b2_train shape: (1, 2) Epoch 362: X_data_scaled shape: (100, 2) Epoch 362: W1_train shape: (2, 3) Epoch 362: b1_train shape: (1, 3) Epoch 362: W2_train shape: (3, 2) Epoch 362: b2_train shape: (1, 2) Epoch 363: X_data_scaled shape: (100, 2) Epoch 363: W1_train shape: (2, 3) Epoch 363: b1_train shape: (1, 3) Epoch 363: W2_train shape: (3, 2) Epoch 363: b2_train shape: (1, 2) Epoch 364: X_data_scaled shape: (100, 2) Epoch 364: W1_train shape: (2, 3) Epoch 364: b1_train shape: (1, 3) Epoch 364: W2_train shape: (3, 2) Epoch 364: b2_train shape: (1, 2) Epoch 365: X_data_scaled shape: (100, 2) Epoch 365: W1_train shape: (2, 3) Epoch 365: b1_train shape: (1, 3) Epoch 365: W2_train shape: (3, 2) Epoch 365: b2_train shape: (1, 2) Epoch 366: X_data_scaled shape: (100, 2) Epoch 366: W1_train shape: (2, 3) Epoch 366: b1_train shape: (1, 3) Epoch 366: W2_train shape: (3, 2) Epoch 366: b2_train shape: (1, 2) Epoch 367: X_data_scaled shape: (100, 2) Epoch 367: W1_train shape: (2, 3) Epoch 367: b1_train shape: (1, 3) Epoch 367: W2_train shape: (3, 2) Epoch 367: b2_train shape: (1, 2) Epoch 368: X_data_scaled shape: (100, 2) Epoch 368: W1_train shape: (2, 3) Epoch 368: b1_train shape: (1, 3) Epoch 368: W2_train shape: (3, 2) Epoch 368: b2_train shape: (1, 2) Epoch 369: X_data_scaled shape: (100, 2) Epoch 369: W1_train shape: (2, 3) Epoch 369: b1_train shape: (1, 3) Epoch 369: W2_train shape: (3, 2) Epoch 369: b2_train shape: (1, 2) Epoch 370: X_data_scaled shape: (100, 2) Epoch 370: W1_train shape: (2, 3) Epoch 370: b1_train shape: (1, 3) Epoch 370: W2_train shape: (3, 2) Epoch 370: b2_train shape: (1, 2) Epoch 371: X_data_scaled shape: (100, 2) Epoch 371: W1_train shape: (2, 3) Epoch 371: b1_train shape: (1, 3) Epoch 371: W2_train shape: (3, 2) Epoch 371: b2_train shape: (1, 2) Epoch 372: X_data_scaled shape: (100, 2) Epoch 372: W1_train shape: (2, 3) Epoch 372: b1_train shape: (1, 3) Epoch 372: W2_train shape: (3, 2) Epoch 372: b2_train shape: (1, 2) Epoch 373: X_data_scaled shape: (100, 2) Epoch 373: W1_train shape: (2, 3) Epoch 373: b1_train shape: (1, 3) Epoch 373: W2_train shape: (3, 2) Epoch 373: b2_train shape: (1, 2) Epoch 374: X_data_scaled shape: (100, 2) Epoch 374: W1_train shape: (2, 3) Epoch 374: b1_train shape: (1, 3) Epoch 374: W2_train shape: (3, 2) Epoch 374: b2_train shape: (1, 2) Epoch 375: X_data_scaled shape: (100, 2) Epoch 375: W1_train shape: (2, 3) Epoch 375: b1_train shape: (1, 3) Epoch 375: W2_train shape: (3, 2) Epoch 375: b2_train shape: (1, 2) Epoch 376: X_data_scaled shape: (100, 2) Epoch 376: W1_train shape: (2, 3) Epoch 376: b1_train shape: (1, 3) Epoch 376: W2_train shape: (3, 2) Epoch 376: b2_train shape: (1, 2) Epoch 377: X_data_scaled shape: (100, 2) Epoch 377: W1_train shape: (2, 3) Epoch 377: b1_train shape: (1, 3) Epoch 377: W2_train shape: (3, 2) Epoch 377: b2_train shape: (1, 2) Epoch 378: X_data_scaled shape: (100, 2) Epoch 378: W1_train shape: (2, 3) Epoch 378: b1_train shape: (1, 3) Epoch 378: W2_train shape: (3, 2) Epoch 378: b2_train shape: (1, 2) Epoch 379: X_data_scaled shape: (100, 2) Epoch 379: W1_train shape: (2, 3) Epoch 379: b1_train shape: (1, 3) Epoch 379: W2_train shape: (3, 2) Epoch 379: b2_train shape: (1, 2) Epoch 380: X_data_scaled shape: (100, 2) Epoch 380: W1_train shape: (2, 3) Epoch 380: b1_train shape: (1, 3) Epoch 380: W2_train shape: (3, 2) Epoch 380: b2_train shape: (1, 2) Epoch 381: X_data_scaled shape: (100, 2) Epoch 381: W1_train shape: (2, 3) Epoch 381: b1_train shape: (1, 3) Epoch 381: W2_train shape: (3, 2) Epoch 381: b2_train shape: (1, 2) Epoch 382: X_data_scaled shape: (100, 2) Epoch 382: W1_train shape: (2, 3) Epoch 382: b1_train shape: (1, 3) Epoch 382: W2_train shape: (3, 2) Epoch 382: b2_train shape: (1, 2) Epoch 383: X_data_scaled shape: (100, 2) Epoch 383: W1_train shape: (2, 3) Epoch 383: b1_train shape: (1, 3) Epoch 383: W2_train shape: (3, 2) Epoch 383: b2_train shape: (1, 2) Epoch 384: X_data_scaled shape: (100, 2) Epoch 384: W1_train shape: (2, 3) Epoch 384: b1_train shape: (1, 3) Epoch 384: W2_train shape: (3, 2) Epoch 384: b2_train shape: (1, 2) Epoch 385: X_data_scaled shape: (100, 2) Epoch 385: W1_train shape: (2, 3) Epoch 385: b1_train shape: (1, 3) Epoch 385: W2_train shape: (3, 2) Epoch 385: b2_train shape: (1, 2) Epoch 386: X_data_scaled shape: (100, 2) Epoch 386: W1_train shape: (2, 3) Epoch 386: b1_train shape: (1, 3) Epoch 386: W2_train shape: (3, 2) Epoch 386: b2_train shape: (1, 2) Epoch 387: X_data_scaled shape: (100, 2) Epoch 387: W1_train shape: (2, 3) Epoch 387: b1_train shape: (1, 3) Epoch 387: W2_train shape: (3, 2) Epoch 387: b2_train shape: (1, 2) Epoch 388: X_data_scaled shape: (100, 2) Epoch 388: W1_train shape: (2, 3) Epoch 388: b1_train shape: (1, 3) Epoch 388: W2_train shape: (3, 2) Epoch 388: b2_train shape: (1, 2) Epoch 389: X_data_scaled shape: (100, 2) Epoch 389: W1_train shape: (2, 3) Epoch 389: b1_train shape: (1, 3) Epoch 389: W2_train shape: (3, 2) Epoch 389: b2_train shape: (1, 2) Epoch 390: X_data_scaled shape: (100, 2) Epoch 390: W1_train shape: (2, 3) Epoch 390: b1_train shape: (1, 3) Epoch 390: W2_train shape: (3, 2) Epoch 390: b2_train shape: (1, 2) Epoch 391: X_data_scaled shape: (100, 2) Epoch 391: W1_train shape: (2, 3) Epoch 391: b1_train shape: (1, 3) Epoch 391: W2_train shape: (3, 2) Epoch 391: b2_train shape: (1, 2) Epoch 392: X_data_scaled shape: (100, 2) Epoch 392: W1_train shape: (2, 3) Epoch 392: b1_train shape: (1, 3) Epoch 392: W2_train shape: (3, 2) Epoch 392: b2_train shape: (1, 2) Epoch 393: X_data_scaled shape: (100, 2) Epoch 393: W1_train shape: (2, 3) Epoch 393: b1_train shape: (1, 3) Epoch 393: W2_train shape: (3, 2) Epoch 393: b2_train shape: (1, 2) Epoch 394: X_data_scaled shape: (100, 2) Epoch 394: W1_train shape: (2, 3) Epoch 394: b1_train shape: (1, 3) Epoch 394: W2_train shape: (3, 2) Epoch 394: b2_train shape: (1, 2) Epoch 395: X_data_scaled shape: (100, 2) Epoch 395: W1_train shape: (2, 3) Epoch 395: b1_train shape: (1, 3) Epoch 395: W2_train shape: (3, 2) Epoch 395: b2_train shape: (1, 2) Epoch 396: X_data_scaled shape: (100, 2) Epoch 396: W1_train shape: (2, 3) Epoch 396: b1_train shape: (1, 3) Epoch 396: W2_train shape: (3, 2) Epoch 396: b2_train shape: (1, 2) Epoch 397: X_data_scaled shape: (100, 2) Epoch 397: W1_train shape: (2, 3) Epoch 397: b1_train shape: (1, 3) Epoch 397: W2_train shape: (3, 2) Epoch 397: b2_train shape: (1, 2) Epoch 398: X_data_scaled shape: (100, 2) Epoch 398: W1_train shape: (2, 3) Epoch 398: b1_train shape: (1, 3) Epoch 398: W2_train shape: (3, 2) Epoch 398: b2_train shape: (1, 2) Epoch 399: X_data_scaled shape: (100, 2) Epoch 399: W1_train shape: (2, 3) Epoch 399: b1_train shape: (1, 3) Epoch 399: W2_train shape: (3, 2) Epoch 399: b2_train shape: (1, 2) Epoch 400: X_data_scaled shape: (100, 2) Epoch 400: W1_train shape: (2, 3) Epoch 400: b1_train shape: (1, 3) Epoch 400: W2_train shape: (3, 2) Epoch 400: b2_train shape: (1, 2) Epoch 400/1000, Loss: 5.2983 Epoch 401: X_data_scaled shape: (100, 2) Epoch 401: W1_train shape: (2, 3) Epoch 401: b1_train shape: (1, 3) Epoch 401: W2_train shape: (3, 2) Epoch 401: b2_train shape: (1, 2) Epoch 402: X_data_scaled shape: (100, 2) Epoch 402: W1_train shape: (2, 3) Epoch 402: b1_train shape: (1, 3) Epoch 402: W2_train shape: (3, 2) Epoch 402: b2_train shape: (1, 2) Epoch 403: X_data_scaled shape: (100, 2) Epoch 403: W1_train shape: (2, 3) Epoch 403: b1_train shape: (1, 3) Epoch 403: W2_train shape: (3, 2) Epoch 403: b2_train shape: (1, 2) Epoch 404: X_data_scaled shape: (100, 2) Epoch 404: W1_train shape: (2, 3) Epoch 404: b1_train shape: (1, 3) Epoch 404: W2_train shape: (3, 2) Epoch 404: b2_train shape: (1, 2) Epoch 405: X_data_scaled shape: (100, 2) Epoch 405: W1_train shape: (2, 3) Epoch 405: b1_train shape: (1, 3) Epoch 405: W2_train shape: (3, 2) Epoch 405: b2_train shape: (1, 2) Epoch 406: X_data_scaled shape: (100, 2) Epoch 406: W1_train shape: (2, 3) Epoch 406: b1_train shape: (1, 3) Epoch 406: W2_train shape: (3, 2) Epoch 406: b2_train shape: (1, 2) Epoch 407: X_data_scaled shape: (100, 2) Epoch 407: W1_train shape: (2, 3) Epoch 407: b1_train shape: (1, 3) Epoch 407: W2_train shape: (3, 2) Epoch 407: b2_train shape: (1, 2) Epoch 408: X_data_scaled shape: (100, 2) Epoch 408: W1_train shape: (2, 3) Epoch 408: b1_train shape: (1, 3) Epoch 408: W2_train shape: (3, 2) Epoch 408: b2_train shape: (1, 2) Epoch 409: X_data_scaled shape: (100, 2) Epoch 409: W1_train shape: (2, 3) Epoch 409: b1_train shape: (1, 3) Epoch 409: W2_train shape: (3, 2) Epoch 409: b2_train shape: (1, 2) Epoch 410: X_data_scaled shape: (100, 2) Epoch 410: W1_train shape: (2, 3) Epoch 410: b1_train shape: (1, 3) Epoch 410: W2_train shape: (3, 2) Epoch 410: b2_train shape: (1, 2) Epoch 411: X_data_scaled shape: (100, 2) Epoch 411: W1_train shape: (2, 3) Epoch 411: b1_train shape: (1, 3) Epoch 411: W2_train shape: (3, 2) Epoch 411: b2_train shape: (1, 2) Epoch 412: X_data_scaled shape: (100, 2) Epoch 412: W1_train shape: (2, 3) Epoch 412: b1_train shape: (1, 3) Epoch 412: W2_train shape: (3, 2) Epoch 412: b2_train shape: (1, 2) Epoch 413: X_data_scaled shape: (100, 2) Epoch 413: W1_train shape: (2, 3) Epoch 413: b1_train shape: (1, 3) Epoch 413: W2_train shape: (3, 2) Epoch 413: b2_train shape: (1, 2) Epoch 414: X_data_scaled shape: (100, 2) Epoch 414: W1_train shape: (2, 3) Epoch 414: b1_train shape: (1, 3) Epoch 414: W2_train shape: (3, 2) Epoch 414: b2_train shape: (1, 2) Epoch 415: X_data_scaled shape: (100, 2) Epoch 415: W1_train shape: (2, 3) Epoch 415: b1_train shape: (1, 3) Epoch 415: W2_train shape: (3, 2) Epoch 415: b2_train shape: (1, 2) Epoch 416: X_data_scaled shape: (100, 2) Epoch 416: W1_train shape: (2, 3) Epoch 416: b1_train shape: (1, 3) Epoch 416: W2_train shape: (3, 2) Epoch 416: b2_train shape: (1, 2) Epoch 417: X_data_scaled shape: (100, 2) Epoch 417: W1_train shape: (2, 3) Epoch 417: b1_train shape: (1, 3) Epoch 417: W2_train shape: (3, 2) Epoch 417: b2_train shape: (1, 2) Epoch 418: X_data_scaled shape: (100, 2) Epoch 418: W1_train shape: (2, 3) Epoch 418: b1_train shape: (1, 3) Epoch 418: W2_train shape: (3, 2) Epoch 418: b2_train shape: (1, 2) Epoch 419: X_data_scaled shape: (100, 2) Epoch 419: W1_train shape: (2, 3) Epoch 419: b1_train shape: (1, 3) Epoch 419: W2_train shape: (3, 2) Epoch 419: b2_train shape: (1, 2) Epoch 420: X_data_scaled shape: (100, 2) Epoch 420: W1_train shape: (2, 3) Epoch 420: b1_train shape: (1, 3) Epoch 420: W2_train shape: (3, 2) Epoch 420: b2_train shape: (1, 2) Epoch 421: X_data_scaled shape: (100, 2) Epoch 421: W1_train shape: (2, 3) Epoch 421: b1_train shape: (1, 3) Epoch 421: W2_train shape: (3, 2) Epoch 421: b2_train shape: (1, 2) Epoch 422: X_data_scaled shape: (100, 2) Epoch 422: W1_train shape: (2, 3) Epoch 422: b1_train shape: (1, 3) Epoch 422: W2_train shape: (3, 2) Epoch 422: b2_train shape: (1, 2) Epoch 423: X_data_scaled shape: (100, 2) Epoch 423: W1_train shape: (2, 3) Epoch 423: b1_train shape: (1, 3) Epoch 423: W2_train shape: (3, 2) Epoch 423: b2_train shape: (1, 2) Epoch 424: X_data_scaled shape: (100, 2) Epoch 424: W1_train shape: (2, 3) Epoch 424: b1_train shape: (1, 3) Epoch 424: W2_train shape: (3, 2) Epoch 424: b2_train shape: (1, 2) Epoch 425: X_data_scaled shape: (100, 2) Epoch 425: W1_train shape: (2, 3) Epoch 425: b1_train shape: (1, 3) Epoch 425: W2_train shape: (3, 2) Epoch 425: b2_train shape: (1, 2) Epoch 426: X_data_scaled shape: (100, 2) Epoch 426: W1_train shape: (2, 3) Epoch 426: b1_train shape: (1, 3) Epoch 426: W2_train shape: (3, 2) Epoch 426: b2_train shape: (1, 2) Epoch 427: X_data_scaled shape: (100, 2) Epoch 427: W1_train shape: (2, 3) Epoch 427: b1_train shape: (1, 3) Epoch 427: W2_train shape: (3, 2) Epoch 427: b2_train shape: (1, 2) Epoch 428: X_data_scaled shape: (100, 2) Epoch 428: W1_train shape: (2, 3) Epoch 428: b1_train shape: (1, 3) Epoch 428: W2_train shape: (3, 2) Epoch 428: b2_train shape: (1, 2) Epoch 429: X_data_scaled shape: (100, 2) Epoch 429: W1_train shape: (2, 3) Epoch 429: b1_train shape: (1, 3) Epoch 429: W2_train shape: (3, 2) Epoch 429: b2_train shape: (1, 2) Epoch 430: X_data_scaled shape: (100, 2) Epoch 430: W1_train shape: (2, 3) Epoch 430: b1_train shape: (1, 3) Epoch 430: W2_train shape: (3, 2) Epoch 430: b2_train shape: (1, 2) Epoch 431: X_data_scaled shape: (100, 2) Epoch 431: W1_train shape: (2, 3) Epoch 431: b1_train shape: (1, 3) Epoch 431: W2_train shape: (3, 2) Epoch 431: b2_train shape: (1, 2) Epoch 432: X_data_scaled shape: (100, 2) Epoch 432: W1_train shape: (2, 3) Epoch 432: b1_train shape: (1, 3) Epoch 432: W2_train shape: (3, 2) Epoch 432: b2_train shape: (1, 2) Epoch 433: X_data_scaled shape: (100, 2) Epoch 433: W1_train shape: (2, 3) Epoch 433: b1_train shape: (1, 3) Epoch 433: W2_train shape: (3, 2) Epoch 433: b2_train shape: (1, 2) Epoch 434: X_data_scaled shape: (100, 2) Epoch 434: W1_train shape: (2, 3) Epoch 434: b1_train shape: (1, 3) Epoch 434: W2_train shape: (3, 2) Epoch 434: b2_train shape: (1, 2) Epoch 435: X_data_scaled shape: (100, 2) Epoch 435: W1_train shape: (2, 3) Epoch 435: b1_train shape: (1, 3) Epoch 435: W2_train shape: (3, 2) Epoch 435: b2_train shape: (1, 2) Epoch 436: X_data_scaled shape: (100, 2) Epoch 436: W1_train shape: (2, 3) Epoch 436: b1_train shape: (1, 3) Epoch 436: W2_train shape: (3, 2) Epoch 436: b2_train shape: (1, 2) Epoch 437: X_data_scaled shape: (100, 2) Epoch 437: W1_train shape: (2, 3) Epoch 437: b1_train shape: (1, 3) Epoch 437: W2_train shape: (3, 2) Epoch 437: b2_train shape: (1, 2) Epoch 438: X_data_scaled shape: (100, 2) Epoch 438: W1_train shape: (2, 3) Epoch 438: b1_train shape: (1, 3) Epoch 438: W2_train shape: (3, 2) Epoch 438: b2_train shape: (1, 2) Epoch 439: X_data_scaled shape: (100, 2) Epoch 439: W1_train shape: (2, 3) Epoch 439: b1_train shape: (1, 3) Epoch 439: W2_train shape: (3, 2) Epoch 439: b2_train shape: (1, 2) Epoch 440: X_data_scaled shape: (100, 2) Epoch 440: W1_train shape: (2, 3) Epoch 440: b1_train shape: (1, 3) Epoch 440: W2_train shape: (3, 2) Epoch 440: b2_train shape: (1, 2) Epoch 441: X_data_scaled shape: (100, 2) Epoch 441: W1_train shape: (2, 3) Epoch 441: b1_train shape: (1, 3) Epoch 441: W2_train shape: (3, 2) Epoch 441: b2_train shape: (1, 2) Epoch 442: X_data_scaled shape: (100, 2) Epoch 442: W1_train shape: (2, 3) Epoch 442: b1_train shape: (1, 3) Epoch 442: W2_train shape: (3, 2) Epoch 442: b2_train shape: (1, 2) Epoch 443: X_data_scaled shape: (100, 2) Epoch 443: W1_train shape: (2, 3) Epoch 443: b1_train shape: (1, 3) Epoch 443: W2_train shape: (3, 2) Epoch 443: b2_train shape: (1, 2) Epoch 444: X_data_scaled shape: (100, 2) Epoch 444: W1_train shape: (2, 3) Epoch 444: b1_train shape: (1, 3) Epoch 444: W2_train shape: (3, 2) Epoch 444: b2_train shape: (1, 2) Epoch 445: X_data_scaled shape: (100, 2) Epoch 445: W1_train shape: (2, 3) Epoch 445: b1_train shape: (1, 3) Epoch 445: W2_train shape: (3, 2) Epoch 445: b2_train shape: (1, 2) Epoch 446: X_data_scaled shape: (100, 2) Epoch 446: W1_train shape: (2, 3) Epoch 446: b1_train shape: (1, 3) Epoch 446: W2_train shape: (3, 2) Epoch 446: b2_train shape: (1, 2) Epoch 447: X_data_scaled shape: (100, 2) Epoch 447: W1_train shape: (2, 3) Epoch 447: b1_train shape: (1, 3) Epoch 447: W2_train shape: (3, 2) Epoch 447: b2_train shape: (1, 2) Epoch 448: X_data_scaled shape: (100, 2) Epoch 448: W1_train shape: (2, 3) Epoch 448: b1_train shape: (1, 3) Epoch 448: W2_train shape: (3, 2) Epoch 448: b2_train shape: (1, 2) Epoch 449: X_data_scaled shape: (100, 2) Epoch 449: W1_train shape: (2, 3) Epoch 449: b1_train shape: (1, 3) Epoch 449: W2_train shape: (3, 2) Epoch 449: b2_train shape: (1, 2) Epoch 450: X_data_scaled shape: (100, 2) Epoch 450: W1_train shape: (2, 3) Epoch 450: b1_train shape: (1, 3) Epoch 450: W2_train shape: (3, 2) Epoch 450: b2_train shape: (1, 2) Epoch 451: X_data_scaled shape: (100, 2) Epoch 451: W1_train shape: (2, 3) Epoch 451: b1_train shape: (1, 3) Epoch 451: W2_train shape: (3, 2) Epoch 451: b2_train shape: (1, 2) Epoch 452: X_data_scaled shape: (100, 2) Epoch 452: W1_train shape: (2, 3) Epoch 452: b1_train shape: (1, 3) Epoch 452: W2_train shape: (3, 2) Epoch 452: b2_train shape: (1, 2) Epoch 453: X_data_scaled shape: (100, 2) Epoch 453: W1_train shape: (2, 3) Epoch 453: b1_train shape: (1, 3) Epoch 453: W2_train shape: (3, 2) Epoch 453: b2_train shape: (1, 2) Epoch 454: X_data_scaled shape: (100, 2) Epoch 454: W1_train shape: (2, 3) Epoch 454: b1_train shape: (1, 3) Epoch 454: W2_train shape: (3, 2) Epoch 454: b2_train shape: (1, 2) Epoch 455: X_data_scaled shape: (100, 2) Epoch 455: W1_train shape: (2, 3) Epoch 455: b1_train shape: (1, 3) Epoch 455: W2_train shape: (3, 2) Epoch 455: b2_train shape: (1, 2) Epoch 456: X_data_scaled shape: (100, 2) Epoch 456: W1_train shape: (2, 3) Epoch 456: b1_train shape: (1, 3) Epoch 456: W2_train shape: (3, 2) Epoch 456: b2_train shape: (1, 2) Epoch 457: X_data_scaled shape: (100, 2) Epoch 457: W1_train shape: (2, 3) Epoch 457: b1_train shape: (1, 3) Epoch 457: W2_train shape: (3, 2) Epoch 457: b2_train shape: (1, 2) Epoch 458: X_data_scaled shape: (100, 2) Epoch 458: W1_train shape: (2, 3) Epoch 458: b1_train shape: (1, 3) Epoch 458: W2_train shape: (3, 2) Epoch 458: b2_train shape: (1, 2) Epoch 459: X_data_scaled shape: (100, 2) Epoch 459: W1_train shape: (2, 3) Epoch 459: b1_train shape: (1, 3) Epoch 459: W2_train shape: (3, 2) Epoch 459: b2_train shape: (1, 2) Epoch 460: X_data_scaled shape: (100, 2) Epoch 460: W1_train shape: (2, 3) Epoch 460: b1_train shape: (1, 3) Epoch 460: W2_train shape: (3, 2) Epoch 460: b2_train shape: (1, 2) Epoch 461: X_data_scaled shape: (100, 2) Epoch 461: W1_train shape: (2, 3) Epoch 461: b1_train shape: (1, 3) Epoch 461: W2_train shape: (3, 2) Epoch 461: b2_train shape: (1, 2) Epoch 462: X_data_scaled shape: (100, 2) Epoch 462: W1_train shape: (2, 3) Epoch 462: b1_train shape: (1, 3) Epoch 462: W2_train shape: (3, 2) Epoch 462: b2_train shape: (1, 2) Epoch 463: X_data_scaled shape: (100, 2) Epoch 463: W1_train shape: (2, 3) Epoch 463: b1_train shape: (1, 3) Epoch 463: W2_train shape: (3, 2) Epoch 463: b2_train shape: (1, 2) Epoch 464: X_data_scaled shape: (100, 2) Epoch 464: W1_train shape: (2, 3) Epoch 464: b1_train shape: (1, 3) Epoch 464: W2_train shape: (3, 2) Epoch 464: b2_train shape: (1, 2) Epoch 465: X_data_scaled shape: (100, 2) Epoch 465: W1_train shape: (2, 3) Epoch 465: b1_train shape: (1, 3) Epoch 465: W2_train shape: (3, 2) Epoch 465: b2_train shape: (1, 2) Epoch 466: X_data_scaled shape: (100, 2) Epoch 466: W1_train shape: (2, 3) Epoch 466: b1_train shape: (1, 3) Epoch 466: W2_train shape: (3, 2) Epoch 466: b2_train shape: (1, 2) Epoch 467: X_data_scaled shape: (100, 2) Epoch 467: W1_train shape: (2, 3) Epoch 467: b1_train shape: (1, 3) Epoch 467: W2_train shape: (3, 2) Epoch 467: b2_train shape: (1, 2) Epoch 468: X_data_scaled shape: (100, 2) Epoch 468: W1_train shape: (2, 3) Epoch 468: b1_train shape: (1, 3) Epoch 468: W2_train shape: (3, 2) Epoch 468: b2_train shape: (1, 2) Epoch 469: X_data_scaled shape: (100, 2) Epoch 469: W1_train shape: (2, 3) Epoch 469: b1_train shape: (1, 3) Epoch 469: W2_train shape: (3, 2) Epoch 469: b2_train shape: (1, 2) Epoch 470: X_data_scaled shape: (100, 2) Epoch 470: W1_train shape: (2, 3) Epoch 470: b1_train shape: (1, 3) Epoch 470: W2_train shape: (3, 2) Epoch 470: b2_train shape: (1, 2) Epoch 471: X_data_scaled shape: (100, 2) Epoch 471: W1_train shape: (2, 3) Epoch 471: b1_train shape: (1, 3) Epoch 471: W2_train shape: (3, 2) Epoch 471: b2_train shape: (1, 2) Epoch 472: X_data_scaled shape: (100, 2) Epoch 472: W1_train shape: (2, 3) Epoch 472: b1_train shape: (1, 3) Epoch 472: W2_train shape: (3, 2) Epoch 472: b2_train shape: (1, 2) Epoch 473: X_data_scaled shape: (100, 2) Epoch 473: W1_train shape: (2, 3) Epoch 473: b1_train shape: (1, 3) Epoch 473: W2_train shape: (3, 2) Epoch 473: b2_train shape: (1, 2) Epoch 474: X_data_scaled shape: (100, 2) Epoch 474: W1_train shape: (2, 3) Epoch 474: b1_train shape: (1, 3) Epoch 474: W2_train shape: (3, 2) Epoch 474: b2_train shape: (1, 2) Epoch 475: X_data_scaled shape: (100, 2) Epoch 475: W1_train shape: (2, 3) Epoch 475: b1_train shape: (1, 3) Epoch 475: W2_train shape: (3, 2) Epoch 475: b2_train shape: (1, 2) Epoch 476: X_data_scaled shape: (100, 2) Epoch 476: W1_train shape: (2, 3) Epoch 476: b1_train shape: (1, 3) Epoch 476: W2_train shape: (3, 2) Epoch 476: b2_train shape: (1, 2) Epoch 477: X_data_scaled shape: (100, 2) Epoch 477: W1_train shape: (2, 3) Epoch 477: b1_train shape: (1, 3) Epoch 477: W2_train shape: (3, 2) Epoch 477: b2_train shape: (1, 2) Epoch 478: X_data_scaled shape: (100, 2) Epoch 478: W1_train shape: (2, 3) Epoch 478: b1_train shape: (1, 3) Epoch 478: W2_train shape: (3, 2) Epoch 478: b2_train shape: (1, 2) Epoch 479: X_data_scaled shape: (100, 2) Epoch 479: W1_train shape: (2, 3) Epoch 479: b1_train shape: (1, 3) Epoch 479: W2_train shape: (3, 2) Epoch 479: b2_train shape: (1, 2) Epoch 480: X_data_scaled shape: (100, 2) Epoch 480: W1_train shape: (2, 3) Epoch 480: b1_train shape: (1, 3) Epoch 480: W2_train shape: (3, 2) Epoch 480: b2_train shape: (1, 2) Epoch 481: X_data_scaled shape: (100, 2) Epoch 481: W1_train shape: (2, 3) Epoch 481: b1_train shape: (1, 3) Epoch 481: W2_train shape: (3, 2) Epoch 481: b2_train shape: (1, 2) Epoch 482: X_data_scaled shape: (100, 2) Epoch 482: W1_train shape: (2, 3) Epoch 482: b1_train shape: (1, 3) Epoch 482: W2_train shape: (3, 2) Epoch 482: b2_train shape: (1, 2) Epoch 483: X_data_scaled shape: (100, 2) Epoch 483: W1_train shape: (2, 3) Epoch 483: b1_train shape: (1, 3) Epoch 483: W2_train shape: (3, 2) Epoch 483: b2_train shape: (1, 2) Epoch 484: X_data_scaled shape: (100, 2) Epoch 484: W1_train shape: (2, 3) Epoch 484: b1_train shape: (1, 3) Epoch 484: W2_train shape: (3, 2) Epoch 484: b2_train shape: (1, 2) Epoch 485: X_data_scaled shape: (100, 2) Epoch 485: W1_train shape: (2, 3) Epoch 485: b1_train shape: (1, 3) Epoch 485: W2_train shape: (3, 2) Epoch 485: b2_train shape: (1, 2) Epoch 486: X_data_scaled shape: (100, 2) Epoch 486: W1_train shape: (2, 3) Epoch 486: b1_train shape: (1, 3) Epoch 486: W2_train shape: (3, 2) Epoch 486: b2_train shape: (1, 2) Epoch 487: X_data_scaled shape: (100, 2) Epoch 487: W1_train shape: (2, 3) Epoch 487: b1_train shape: (1, 3) Epoch 487: W2_train shape: (3, 2) Epoch 487: b2_train shape: (1, 2) Epoch 488: X_data_scaled shape: (100, 2) Epoch 488: W1_train shape: (2, 3) Epoch 488: b1_train shape: (1, 3) Epoch 488: W2_train shape: (3, 2) Epoch 488: b2_train shape: (1, 2) Epoch 489: X_data_scaled shape: (100, 2) Epoch 489: W1_train shape: (2, 3) Epoch 489: b1_train shape: (1, 3) Epoch 489: W2_train shape: (3, 2) Epoch 489: b2_train shape: (1, 2) Epoch 490: X_data_scaled shape: (100, 2) Epoch 490: W1_train shape: (2, 3) Epoch 490: b1_train shape: (1, 3) Epoch 490: W2_train shape: (3, 2) Epoch 490: b2_train shape: (1, 2) Epoch 491: X_data_scaled shape: (100, 2) Epoch 491: W1_train shape: (2, 3) Epoch 491: b1_train shape: (1, 3) Epoch 491: W2_train shape: (3, 2) Epoch 491: b2_train shape: (1, 2) Epoch 492: X_data_scaled shape: (100, 2) Epoch 492: W1_train shape: (2, 3) Epoch 492: b1_train shape: (1, 3) Epoch 492: W2_train shape: (3, 2) Epoch 492: b2_train shape: (1, 2) Epoch 493: X_data_scaled shape: (100, 2) Epoch 493: W1_train shape: (2, 3) Epoch 493: b1_train shape: (1, 3) Epoch 493: W2_train shape: (3, 2) Epoch 493: b2_train shape: (1, 2) Epoch 494: X_data_scaled shape: (100, 2) Epoch 494: W1_train shape: (2, 3) Epoch 494: b1_train shape: (1, 3) Epoch 494: W2_train shape: (3, 2) Epoch 494: b2_train shape: (1, 2) Epoch 495: X_data_scaled shape: (100, 2) Epoch 495: W1_train shape: (2, 3) Epoch 495: b1_train shape: (1, 3) Epoch 495: W2_train shape: (3, 2) Epoch 495: b2_train shape: (1, 2) Epoch 496: X_data_scaled shape: (100, 2) Epoch 496: W1_train shape: (2, 3) Epoch 496: b1_train shape: (1, 3) Epoch 496: W2_train shape: (3, 2) Epoch 496: b2_train shape: (1, 2) Epoch 497: X_data_scaled shape: (100, 2) Epoch 497: W1_train shape: (2, 3) Epoch 497: b1_train shape: (1, 3) Epoch 497: W2_train shape: (3, 2) Epoch 497: b2_train shape: (1, 2) Epoch 498: X_data_scaled shape: (100, 2) Epoch 498: W1_train shape: (2, 3) Epoch 498: b1_train shape: (1, 3) Epoch 498: W2_train shape: (3, 2) Epoch 498: b2_train shape: (1, 2) Epoch 499: X_data_scaled shape: (100, 2) Epoch 499: W1_train shape: (2, 3) Epoch 499: b1_train shape: (1, 3) Epoch 499: W2_train shape: (3, 2) Epoch 499: b2_train shape: (1, 2) Epoch 500: X_data_scaled shape: (100, 2) Epoch 500: W1_train shape: (2, 3) Epoch 500: b1_train shape: (1, 3) Epoch 500: W2_train shape: (3, 2) Epoch 500: b2_train shape: (1, 2) Epoch 500/1000, Loss: 5.2983 Epoch 501: X_data_scaled shape: (100, 2) Epoch 501: W1_train shape: (2, 3) Epoch 501: b1_train shape: (1, 3) Epoch 501: W2_train shape: (3, 2) Epoch 501: b2_train shape: (1, 2) Epoch 502: X_data_scaled shape: (100, 2) Epoch 502: W1_train shape: (2, 3) Epoch 502: b1_train shape: (1, 3) Epoch 502: W2_train shape: (3, 2) Epoch 502: b2_train shape: (1, 2) Epoch 503: X_data_scaled shape: (100, 2) Epoch 503: W1_train shape: (2, 3) Epoch 503: b1_train shape: (1, 3) Epoch 503: W2_train shape: (3, 2) Epoch 503: b2_train shape: (1, 2) Epoch 504: X_data_scaled shape: (100, 2) Epoch 504: W1_train shape: (2, 3) Epoch 504: b1_train shape: (1, 3) Epoch 504: W2_train shape: (3, 2) Epoch 504: b2_train shape: (1, 2) Epoch 505: X_data_scaled shape: (100, 2) Epoch 505: W1_train shape: (2, 3) Epoch 505: b1_train shape: (1, 3) Epoch 505: W2_train shape: (3, 2) Epoch 505: b2_train shape: (1, 2) Epoch 506: X_data_scaled shape: (100, 2) Epoch 506: W1_train shape: (2, 3) Epoch 506: b1_train shape: (1, 3) Epoch 506: W2_train shape: (3, 2) Epoch 506: b2_train shape: (1, 2) Epoch 507: X_data_scaled shape: (100, 2) Epoch 507: W1_train shape: (2, 3) Epoch 507: b1_train shape: (1, 3) Epoch 507: W2_train shape: (3, 2) Epoch 507: b2_train shape: (1, 2) Epoch 508: X_data_scaled shape: (100, 2) Epoch 508: W1_train shape: (2, 3) Epoch 508: b1_train shape: (1, 3) Epoch 508: W2_train shape: (3, 2) Epoch 508: b2_train shape: (1, 2) Epoch 509: X_data_scaled shape: (100, 2) Epoch 509: W1_train shape: (2, 3) Epoch 509: b1_train shape: (1, 3) Epoch 509: W2_train shape: (3, 2) Epoch 509: b2_train shape: (1, 2) Epoch 510: X_data_scaled shape: (100, 2) Epoch 510: W1_train shape: (2, 3) Epoch 510: b1_train shape: (1, 3) Epoch 510: W2_train shape: (3, 2) Epoch 510: b2_train shape: (1, 2) Epoch 511: X_data_scaled shape: (100, 2) Epoch 511: W1_train shape: (2, 3) Epoch 511: b1_train shape: (1, 3) Epoch 511: W2_train shape: (3, 2) Epoch 511: b2_train shape: (1, 2) Epoch 512: X_data_scaled shape: (100, 2) Epoch 512: W1_train shape: (2, 3) Epoch 512: b1_train shape: (1, 3) Epoch 512: W2_train shape: (3, 2) Epoch 512: b2_train shape: (1, 2) Epoch 513: X_data_scaled shape: (100, 2) Epoch 513: W1_train shape: (2, 3) Epoch 513: b1_train shape: (1, 3) Epoch 513: W2_train shape: (3, 2) Epoch 513: b2_train shape: (1, 2) Epoch 514: X_data_scaled shape: (100, 2) Epoch 514: W1_train shape: (2, 3) Epoch 514: b1_train shape: (1, 3) Epoch 514: W2_train shape: (3, 2) Epoch 514: b2_train shape: (1, 2) Epoch 515: X_data_scaled shape: (100, 2) Epoch 515: W1_train shape: (2, 3) Epoch 515: b1_train shape: (1, 3) Epoch 515: W2_train shape: (3, 2) Epoch 515: b2_train shape: (1, 2) Epoch 516: X_data_scaled shape: (100, 2) Epoch 516: W1_train shape: (2, 3) Epoch 516: b1_train shape: (1, 3) Epoch 516: W2_train shape: (3, 2) Epoch 516: b2_train shape: (1, 2) Epoch 517: X_data_scaled shape: (100, 2) Epoch 517: W1_train shape: (2, 3) Epoch 517: b1_train shape: (1, 3) Epoch 517: W2_train shape: (3, 2) Epoch 517: b2_train shape: (1, 2) Epoch 518: X_data_scaled shape: (100, 2) Epoch 518: W1_train shape: (2, 3) Epoch 518: b1_train shape: (1, 3) Epoch 518: W2_train shape: (3, 2) Epoch 518: b2_train shape: (1, 2) Epoch 519: X_data_scaled shape: (100, 2) Epoch 519: W1_train shape: (2, 3) Epoch 519: b1_train shape: (1, 3) Epoch 519: W2_train shape: (3, 2) Epoch 519: b2_train shape: (1, 2) Epoch 520: X_data_scaled shape: (100, 2) Epoch 520: W1_train shape: (2, 3) Epoch 520: b1_train shape: (1, 3) Epoch 520: W2_train shape: (3, 2) Epoch 520: b2_train shape: (1, 2) Epoch 521: X_data_scaled shape: (100, 2) Epoch 521: W1_train shape: (2, 3) Epoch 521: b1_train shape: (1, 3) Epoch 521: W2_train shape: (3, 2) Epoch 521: b2_train shape: (1, 2) Epoch 522: X_data_scaled shape: (100, 2) Epoch 522: W1_train shape: (2, 3) Epoch 522: b1_train shape: (1, 3) Epoch 522: W2_train shape: (3, 2) Epoch 522: b2_train shape: (1, 2) Epoch 523: X_data_scaled shape: (100, 2) Epoch 523: W1_train shape: (2, 3) Epoch 523: b1_train shape: (1, 3) Epoch 523: W2_train shape: (3, 2) Epoch 523: b2_train shape: (1, 2) Epoch 524: X_data_scaled shape: (100, 2) Epoch 524: W1_train shape: (2, 3) Epoch 524: b1_train shape: (1, 3) Epoch 524: W2_train shape: (3, 2) Epoch 524: b2_train shape: (1, 2) Epoch 525: X_data_scaled shape: (100, 2) Epoch 525: W1_train shape: (2, 3) Epoch 525: b1_train shape: (1, 3) Epoch 525: W2_train shape: (3, 2) Epoch 525: b2_train shape: (1, 2) Epoch 526: X_data_scaled shape: (100, 2) Epoch 526: W1_train shape: (2, 3) Epoch 526: b1_train shape: (1, 3) Epoch 526: W2_train shape: (3, 2) Epoch 526: b2_train shape: (1, 2) Epoch 527: X_data_scaled shape: (100, 2) Epoch 527: W1_train shape: (2, 3) Epoch 527: b1_train shape: (1, 3) Epoch 527: W2_train shape: (3, 2) Epoch 527: b2_train shape: (1, 2) Epoch 528: X_data_scaled shape: (100, 2) Epoch 528: W1_train shape: (2, 3) Epoch 528: b1_train shape: (1, 3) Epoch 528: W2_train shape: (3, 2) Epoch 528: b2_train shape: (1, 2) Epoch 529: X_data_scaled shape: (100, 2) Epoch 529: W1_train shape: (2, 3) Epoch 529: b1_train shape: (1, 3) Epoch 529: W2_train shape: (3, 2) Epoch 529: b2_train shape: (1, 2) Epoch 530: X_data_scaled shape: (100, 2) Epoch 530: W1_train shape: (2, 3) Epoch 530: b1_train shape: (1, 3) Epoch 530: W2_train shape: (3, 2) Epoch 530: b2_train shape: (1, 2) Epoch 531: X_data_scaled shape: (100, 2) Epoch 531: W1_train shape: (2, 3) Epoch 531: b1_train shape: (1, 3) Epoch 531: W2_train shape: (3, 2) Epoch 531: b2_train shape: (1, 2) Epoch 532: X_data_scaled shape: (100, 2) Epoch 532: W1_train shape: (2, 3) Epoch 532: b1_train shape: (1, 3) Epoch 532: W2_train shape: (3, 2) Epoch 532: b2_train shape: (1, 2) Epoch 533: X_data_scaled shape: (100, 2) Epoch 533: W1_train shape: (2, 3) Epoch 533: b1_train shape: (1, 3) Epoch 533: W2_train shape: (3, 2) Epoch 533: b2_train shape: (1, 2) Epoch 534: X_data_scaled shape: (100, 2) Epoch 534: W1_train shape: (2, 3) Epoch 534: b1_train shape: (1, 3) Epoch 534: W2_train shape: (3, 2) Epoch 534: b2_train shape: (1, 2) Epoch 535: X_data_scaled shape: (100, 2) Epoch 535: W1_train shape: (2, 3) Epoch 535: b1_train shape: (1, 3) Epoch 535: W2_train shape: (3, 2) Epoch 535: b2_train shape: (1, 2) Epoch 536: X_data_scaled shape: (100, 2) Epoch 536: W1_train shape: (2, 3) Epoch 536: b1_train shape: (1, 3) Epoch 536: W2_train shape: (3, 2) Epoch 536: b2_train shape: (1, 2) Epoch 537: X_data_scaled shape: (100, 2) Epoch 537: W1_train shape: (2, 3) Epoch 537: b1_train shape: (1, 3) Epoch 537: W2_train shape: (3, 2) Epoch 537: b2_train shape: (1, 2) Epoch 538: X_data_scaled shape: (100, 2) Epoch 538: W1_train shape: (2, 3) Epoch 538: b1_train shape: (1, 3) Epoch 538: W2_train shape: (3, 2) Epoch 538: b2_train shape: (1, 2) Epoch 539: X_data_scaled shape: (100, 2) Epoch 539: W1_train shape: (2, 3) Epoch 539: b1_train shape: (1, 3) Epoch 539: W2_train shape: (3, 2) Epoch 539: b2_train shape: (1, 2) Epoch 540: X_data_scaled shape: (100, 2) Epoch 540: W1_train shape: (2, 3) Epoch 540: b1_train shape: (1, 3) Epoch 540: W2_train shape: (3, 2) Epoch 540: b2_train shape: (1, 2) Epoch 541: X_data_scaled shape: (100, 2) Epoch 541: W1_train shape: (2, 3) Epoch 541: b1_train shape: (1, 3) Epoch 541: W2_train shape: (3, 2) Epoch 541: b2_train shape: (1, 2) Epoch 542: X_data_scaled shape: (100, 2) Epoch 542: W1_train shape: (2, 3) Epoch 542: b1_train shape: (1, 3) Epoch 542: W2_train shape: (3, 2) Epoch 542: b2_train shape: (1, 2) Epoch 543: X_data_scaled shape: (100, 2) Epoch 543: W1_train shape: (2, 3) Epoch 543: b1_train shape: (1, 3) Epoch 543: W2_train shape: (3, 2) Epoch 543: b2_train shape: (1, 2) Epoch 544: X_data_scaled shape: (100, 2) Epoch 544: W1_train shape: (2, 3) Epoch 544: b1_train shape: (1, 3) Epoch 544: W2_train shape: (3, 2) Epoch 544: b2_train shape: (1, 2) Epoch 545: X_data_scaled shape: (100, 2) Epoch 545: W1_train shape: (2, 3) Epoch 545: b1_train shape: (1, 3) Epoch 545: W2_train shape: (3, 2) Epoch 545: b2_train shape: (1, 2) Epoch 546: X_data_scaled shape: (100, 2) Epoch 546: W1_train shape: (2, 3) Epoch 546: b1_train shape: (1, 3) Epoch 546: W2_train shape: (3, 2) Epoch 546: b2_train shape: (1, 2) Epoch 547: X_data_scaled shape: (100, 2) Epoch 547: W1_train shape: (2, 3) Epoch 547: b1_train shape: (1, 3) Epoch 547: W2_train shape: (3, 2) Epoch 547: b2_train shape: (1, 2) Epoch 548: X_data_scaled shape: (100, 2) Epoch 548: W1_train shape: (2, 3) Epoch 548: b1_train shape: (1, 3) Epoch 548: W2_train shape: (3, 2) Epoch 548: b2_train shape: (1, 2) Epoch 549: X_data_scaled shape: (100, 2) Epoch 549: W1_train shape: (2, 3) Epoch 549: b1_train shape: (1, 3) Epoch 549: W2_train shape: (3, 2) Epoch 549: b2_train shape: (1, 2) Epoch 550: X_data_scaled shape: (100, 2) Epoch 550: W1_train shape: (2, 3) Epoch 550: b1_train shape: (1, 3) Epoch 550: W2_train shape: (3, 2) Epoch 550: b2_train shape: (1, 2) Epoch 551: X_data_scaled shape: (100, 2) Epoch 551: W1_train shape: (2, 3) Epoch 551: b1_train shape: (1, 3) Epoch 551: W2_train shape: (3, 2) Epoch 551: b2_train shape: (1, 2) Epoch 552: X_data_scaled shape: (100, 2) Epoch 552: W1_train shape: (2, 3) Epoch 552: b1_train shape: (1, 3) Epoch 552: W2_train shape: (3, 2) Epoch 552: b2_train shape: (1, 2) Epoch 553: X_data_scaled shape: (100, 2) Epoch 553: W1_train shape: (2, 3) Epoch 553: b1_train shape: (1, 3) Epoch 553: W2_train shape: (3, 2) Epoch 553: b2_train shape: (1, 2) Epoch 554: X_data_scaled shape: (100, 2) Epoch 554: W1_train shape: (2, 3) Epoch 554: b1_train shape: (1, 3) Epoch 554: W2_train shape: (3, 2) Epoch 554: b2_train shape: (1, 2) Epoch 555: X_data_scaled shape: (100, 2) Epoch 555: W1_train shape: (2, 3) Epoch 555: b1_train shape: (1, 3) Epoch 555: W2_train shape: (3, 2) Epoch 555: b2_train shape: (1, 2) Epoch 556: X_data_scaled shape: (100, 2) Epoch 556: W1_train shape: (2, 3) Epoch 556: b1_train shape: (1, 3) Epoch 556: W2_train shape: (3, 2) Epoch 556: b2_train shape: (1, 2) Epoch 557: X_data_scaled shape: (100, 2) Epoch 557: W1_train shape: (2, 3) Epoch 557: b1_train shape: (1, 3) Epoch 557: W2_train shape: (3, 2) Epoch 557: b2_train shape: (1, 2) Epoch 558: X_data_scaled shape: (100, 2) Epoch 558: W1_train shape: (2, 3) Epoch 558: b1_train shape: (1, 3) Epoch 558: W2_train shape: (3, 2) Epoch 558: b2_train shape: (1, 2) Epoch 559: X_data_scaled shape: (100, 2) Epoch 559: W1_train shape: (2, 3) Epoch 559: b1_train shape: (1, 3) Epoch 559: W2_train shape: (3, 2) Epoch 559: b2_train shape: (1, 2) Epoch 560: X_data_scaled shape: (100, 2) Epoch 560: W1_train shape: (2, 3) Epoch 560: b1_train shape: (1, 3) Epoch 560: W2_train shape: (3, 2) Epoch 560: b2_train shape: (1, 2) Epoch 561: X_data_scaled shape: (100, 2) Epoch 561: W1_train shape: (2, 3) Epoch 561: b1_train shape: (1, 3) Epoch 561: W2_train shape: (3, 2) Epoch 561: b2_train shape: (1, 2) Epoch 562: X_data_scaled shape: (100, 2) Epoch 562: W1_train shape: (2, 3) Epoch 562: b1_train shape: (1, 3) Epoch 562: W2_train shape: (3, 2) Epoch 562: b2_train shape: (1, 2) Epoch 563: X_data_scaled shape: (100, 2) Epoch 563: W1_train shape: (2, 3) Epoch 563: b1_train shape: (1, 3) Epoch 563: W2_train shape: (3, 2) Epoch 563: b2_train shape: (1, 2) Epoch 564: X_data_scaled shape: (100, 2) Epoch 564: W1_train shape: (2, 3) Epoch 564: b1_train shape: (1, 3) Epoch 564: W2_train shape: (3, 2) Epoch 564: b2_train shape: (1, 2) Epoch 565: X_data_scaled shape: (100, 2) Epoch 565: W1_train shape: (2, 3) Epoch 565: b1_train shape: (1, 3) Epoch 565: W2_train shape: (3, 2) Epoch 565: b2_train shape: (1, 2) Epoch 566: X_data_scaled shape: (100, 2) Epoch 566: W1_train shape: (2, 3) Epoch 566: b1_train shape: (1, 3) Epoch 566: W2_train shape: (3, 2) Epoch 566: b2_train shape: (1, 2) Epoch 567: X_data_scaled shape: (100, 2) Epoch 567: W1_train shape: (2, 3) Epoch 567: b1_train shape: (1, 3) Epoch 567: W2_train shape: (3, 2) Epoch 567: b2_train shape: (1, 2) Epoch 568: X_data_scaled shape: (100, 2) Epoch 568: W1_train shape: (2, 3) Epoch 568: b1_train shape: (1, 3) Epoch 568: W2_train shape: (3, 2) Epoch 568: b2_train shape: (1, 2) Epoch 569: X_data_scaled shape: (100, 2) Epoch 569: W1_train shape: (2, 3) Epoch 569: b1_train shape: (1, 3) Epoch 569: W2_train shape: (3, 2) Epoch 569: b2_train shape: (1, 2) Epoch 570: X_data_scaled shape: (100, 2) Epoch 570: W1_train shape: (2, 3) Epoch 570: b1_train shape: (1, 3) Epoch 570: W2_train shape: (3, 2) Epoch 570: b2_train shape: (1, 2) Epoch 571: X_data_scaled shape: (100, 2) Epoch 571: W1_train shape: (2, 3) Epoch 571: b1_train shape: (1, 3) Epoch 571: W2_train shape: (3, 2) Epoch 571: b2_train shape: (1, 2) Epoch 572: X_data_scaled shape: (100, 2) Epoch 572: W1_train shape: (2, 3) Epoch 572: b1_train shape: (1, 3) Epoch 572: W2_train shape: (3, 2) Epoch 572: b2_train shape: (1, 2) Epoch 573: X_data_scaled shape: (100, 2) Epoch 573: W1_train shape: (2, 3) Epoch 573: b1_train shape: (1, 3) Epoch 573: W2_train shape: (3, 2) Epoch 573: b2_train shape: (1, 2) Epoch 574: X_data_scaled shape: (100, 2) Epoch 574: W1_train shape: (2, 3) Epoch 574: b1_train shape: (1, 3) Epoch 574: W2_train shape: (3, 2) Epoch 574: b2_train shape: (1, 2) Epoch 575: X_data_scaled shape: (100, 2) Epoch 575: W1_train shape: (2, 3) Epoch 575: b1_train shape: (1, 3) Epoch 575: W2_train shape: (3, 2) Epoch 575: b2_train shape: (1, 2) Epoch 576: X_data_scaled shape: (100, 2) Epoch 576: W1_train shape: (2, 3) Epoch 576: b1_train shape: (1, 3) Epoch 576: W2_train shape: (3, 2) Epoch 576: b2_train shape: (1, 2) Epoch 577: X_data_scaled shape: (100, 2) Epoch 577: W1_train shape: (2, 3) Epoch 577: b1_train shape: (1, 3) Epoch 577: W2_train shape: (3, 2) Epoch 577: b2_train shape: (1, 2) Epoch 578: X_data_scaled shape: (100, 2) Epoch 578: W1_train shape: (2, 3) Epoch 578: b1_train shape: (1, 3) Epoch 578: W2_train shape: (3, 2) Epoch 578: b2_train shape: (1, 2) Epoch 579: X_data_scaled shape: (100, 2) Epoch 579: W1_train shape: (2, 3) Epoch 579: b1_train shape: (1, 3) Epoch 579: W2_train shape: (3, 2) Epoch 579: b2_train shape: (1, 2) Epoch 580: X_data_scaled shape: (100, 2) Epoch 580: W1_train shape: (2, 3) Epoch 580: b1_train shape: (1, 3) Epoch 580: W2_train shape: (3, 2) Epoch 580: b2_train shape: (1, 2) Epoch 581: X_data_scaled shape: (100, 2) Epoch 581: W1_train shape: (2, 3) Epoch 581: b1_train shape: (1, 3) Epoch 581: W2_train shape: (3, 2) Epoch 581: b2_train shape: (1, 2) Epoch 582: X_data_scaled shape: (100, 2) Epoch 582: W1_train shape: (2, 3) Epoch 582: b1_train shape: (1, 3) Epoch 582: W2_train shape: (3, 2) Epoch 582: b2_train shape: (1, 2) Epoch 583: X_data_scaled shape: (100, 2) Epoch 583: W1_train shape: (2, 3) Epoch 583: b1_train shape: (1, 3) Epoch 583: W2_train shape: (3, 2) Epoch 583: b2_train shape: (1, 2) Epoch 584: X_data_scaled shape: (100, 2) Epoch 584: W1_train shape: (2, 3) Epoch 584: b1_train shape: (1, 3) Epoch 584: W2_train shape: (3, 2) Epoch 584: b2_train shape: (1, 2) Epoch 585: X_data_scaled shape: (100, 2) Epoch 585: W1_train shape: (2, 3) Epoch 585: b1_train shape: (1, 3) Epoch 585: W2_train shape: (3, 2) Epoch 585: b2_train shape: (1, 2) Epoch 586: X_data_scaled shape: (100, 2) Epoch 586: W1_train shape: (2, 3) Epoch 586: b1_train shape: (1, 3) Epoch 586: W2_train shape: (3, 2) Epoch 586: b2_train shape: (1, 2) Epoch 587: X_data_scaled shape: (100, 2) Epoch 587: W1_train shape: (2, 3) Epoch 587: b1_train shape: (1, 3) Epoch 587: W2_train shape: (3, 2) Epoch 587: b2_train shape: (1, 2) Epoch 588: X_data_scaled shape: (100, 2) Epoch 588: W1_train shape: (2, 3) Epoch 588: b1_train shape: (1, 3) Epoch 588: W2_train shape: (3, 2) Epoch 588: b2_train shape: (1, 2) Epoch 589: X_data_scaled shape: (100, 2) Epoch 589: W1_train shape: (2, 3) Epoch 589: b1_train shape: (1, 3) Epoch 589: W2_train shape: (3, 2) Epoch 589: b2_train shape: (1, 2) Epoch 590: X_data_scaled shape: (100, 2) Epoch 590: W1_train shape: (2, 3) Epoch 590: b1_train shape: (1, 3) Epoch 590: W2_train shape: (3, 2) Epoch 590: b2_train shape: (1, 2) Epoch 591: X_data_scaled shape: (100, 2) Epoch 591: W1_train shape: (2, 3) Epoch 591: b1_train shape: (1, 3) Epoch 591: W2_train shape: (3, 2) Epoch 591: b2_train shape: (1, 2) Epoch 592: X_data_scaled shape: (100, 2) Epoch 592: W1_train shape: (2, 3) Epoch 592: b1_train shape: (1, 3) Epoch 592: W2_train shape: (3, 2) Epoch 592: b2_train shape: (1, 2) Epoch 593: X_data_scaled shape: (100, 2) Epoch 593: W1_train shape: (2, 3) Epoch 593: b1_train shape: (1, 3) Epoch 593: W2_train shape: (3, 2) Epoch 593: b2_train shape: (1, 2) Epoch 594: X_data_scaled shape: (100, 2) Epoch 594: W1_train shape: (2, 3) Epoch 594: b1_train shape: (1, 3) Epoch 594: W2_train shape: (3, 2) Epoch 594: b2_train shape: (1, 2) Epoch 595: X_data_scaled shape: (100, 2) Epoch 595: W1_train shape: (2, 3) Epoch 595: b1_train shape: (1, 3) Epoch 595: W2_train shape: (3, 2) Epoch 595: b2_train shape: (1, 2) Epoch 596: X_data_scaled shape: (100, 2) Epoch 596: W1_train shape: (2, 3) Epoch 596: b1_train shape: (1, 3) Epoch 596: W2_train shape: (3, 2) Epoch 596: b2_train shape: (1, 2) Epoch 597: X_data_scaled shape: (100, 2) Epoch 597: W1_train shape: (2, 3) Epoch 597: b1_train shape: (1, 3) Epoch 597: W2_train shape: (3, 2) Epoch 597: b2_train shape: (1, 2) Epoch 598: X_data_scaled shape: (100, 2) Epoch 598: W1_train shape: (2, 3) Epoch 598: b1_train shape: (1, 3) Epoch 598: W2_train shape: (3, 2) Epoch 598: b2_train shape: (1, 2) Epoch 599: X_data_scaled shape: (100, 2) Epoch 599: W1_train shape: (2, 3) Epoch 599: b1_train shape: (1, 3) Epoch 599: W2_train shape: (3, 2) Epoch 599: b2_train shape: (1, 2) Epoch 600: X_data_scaled shape: (100, 2) Epoch 600: W1_train shape: (2, 3) Epoch 600: b1_train shape: (1, 3) Epoch 600: W2_train shape: (3, 2) Epoch 600: b2_train shape: (1, 2) Epoch 600/1000, Loss: 5.2983 Epoch 601: X_data_scaled shape: (100, 2) Epoch 601: W1_train shape: (2, 3) Epoch 601: b1_train shape: (1, 3) Epoch 601: W2_train shape: (3, 2) Epoch 601: b2_train shape: (1, 2) Epoch 602: X_data_scaled shape: (100, 2) Epoch 602: W1_train shape: (2, 3) Epoch 602: b1_train shape: (1, 3) Epoch 602: W2_train shape: (3, 2) Epoch 602: b2_train shape: (1, 2) Epoch 603: X_data_scaled shape: (100, 2) Epoch 603: W1_train shape: (2, 3) Epoch 603: b1_train shape: (1, 3) Epoch 603: W2_train shape: (3, 2) Epoch 603: b2_train shape: (1, 2) Epoch 604: X_data_scaled shape: (100, 2) Epoch 604: W1_train shape: (2, 3) Epoch 604: b1_train shape: (1, 3) Epoch 604: W2_train shape: (3, 2) Epoch 604: b2_train shape: (1, 2) Epoch 605: X_data_scaled shape: (100, 2) Epoch 605: W1_train shape: (2, 3) Epoch 605: b1_train shape: (1, 3) Epoch 605: W2_train shape: (3, 2) Epoch 605: b2_train shape: (1, 2) Epoch 606: X_data_scaled shape: (100, 2) Epoch 606: W1_train shape: (2, 3) Epoch 606: b1_train shape: (1, 3) Epoch 606: W2_train shape: (3, 2) Epoch 606: b2_train shape: (1, 2) Epoch 607: X_data_scaled shape: (100, 2) Epoch 607: W1_train shape: (2, 3) Epoch 607: b1_train shape: (1, 3) Epoch 607: W2_train shape: (3, 2) Epoch 607: b2_train shape: (1, 2) Epoch 608: X_data_scaled shape: (100, 2) Epoch 608: W1_train shape: (2, 3) Epoch 608: b1_train shape: (1, 3) Epoch 608: W2_train shape: (3, 2) Epoch 608: b2_train shape: (1, 2) Epoch 609: X_data_scaled shape: (100, 2) Epoch 609: W1_train shape: (2, 3) Epoch 609: b1_train shape: (1, 3) Epoch 609: W2_train shape: (3, 2) Epoch 609: b2_train shape: (1, 2) Epoch 610: X_data_scaled shape: (100, 2) Epoch 610: W1_train shape: (2, 3) Epoch 610: b1_train shape: (1, 3) Epoch 610: W2_train shape: (3, 2) Epoch 610: b2_train shape: (1, 2) Epoch 611: X_data_scaled shape: (100, 2) Epoch 611: W1_train shape: (2, 3) Epoch 611: b1_train shape: (1, 3) Epoch 611: W2_train shape: (3, 2) Epoch 611: b2_train shape: (1, 2) Epoch 612: X_data_scaled shape: (100, 2) Epoch 612: W1_train shape: (2, 3) Epoch 612: b1_train shape: (1, 3) Epoch 612: W2_train shape: (3, 2) Epoch 612: b2_train shape: (1, 2) Epoch 613: X_data_scaled shape: (100, 2) Epoch 613: W1_train shape: (2, 3) Epoch 613: b1_train shape: (1, 3) Epoch 613: W2_train shape: (3, 2) Epoch 613: b2_train shape: (1, 2) Epoch 614: X_data_scaled shape: (100, 2) Epoch 614: W1_train shape: (2, 3) Epoch 614: b1_train shape: (1, 3) Epoch 614: W2_train shape: (3, 2) Epoch 614: b2_train shape: (1, 2) Epoch 615: X_data_scaled shape: (100, 2) Epoch 615: W1_train shape: (2, 3) Epoch 615: b1_train shape: (1, 3) Epoch 615: W2_train shape: (3, 2) Epoch 615: b2_train shape: (1, 2) Epoch 616: X_data_scaled shape: (100, 2) Epoch 616: W1_train shape: (2, 3) Epoch 616: b1_train shape: (1, 3) Epoch 616: W2_train shape: (3, 2) Epoch 616: b2_train shape: (1, 2) Epoch 617: X_data_scaled shape: (100, 2) Epoch 617: W1_train shape: (2, 3) Epoch 617: b1_train shape: (1, 3) Epoch 617: W2_train shape: (3, 2) Epoch 617: b2_train shape: (1, 2) Epoch 618: X_data_scaled shape: (100, 2) Epoch 618: W1_train shape: (2, 3) Epoch 618: b1_train shape: (1, 3) Epoch 618: W2_train shape: (3, 2) Epoch 618: b2_train shape: (1, 2) Epoch 619: X_data_scaled shape: (100, 2) Epoch 619: W1_train shape: (2, 3) Epoch 619: b1_train shape: (1, 3) Epoch 619: W2_train shape: (3, 2) Epoch 619: b2_train shape: (1, 2) Epoch 620: X_data_scaled shape: (100, 2) Epoch 620: W1_train shape: (2, 3) Epoch 620: b1_train shape: (1, 3) Epoch 620: W2_train shape: (3, 2) Epoch 620: b2_train shape: (1, 2) Epoch 621: X_data_scaled shape: (100, 2) Epoch 621: W1_train shape: (2, 3) Epoch 621: b1_train shape: (1, 3) Epoch 621: W2_train shape: (3, 2) Epoch 621: b2_train shape: (1, 2) Epoch 622: X_data_scaled shape: (100, 2) Epoch 622: W1_train shape: (2, 3) Epoch 622: b1_train shape: (1, 3) Epoch 622: W2_train shape: (3, 2) Epoch 622: b2_train shape: (1, 2) Epoch 623: X_data_scaled shape: (100, 2) Epoch 623: W1_train shape: (2, 3) Epoch 623: b1_train shape: (1, 3) Epoch 623: W2_train shape: (3, 2) Epoch 623: b2_train shape: (1, 2) Epoch 624: X_data_scaled shape: (100, 2) Epoch 624: W1_train shape: (2, 3) Epoch 624: b1_train shape: (1, 3) Epoch 624: W2_train shape: (3, 2) Epoch 624: b2_train shape: (1, 2) Epoch 625: X_data_scaled shape: (100, 2) Epoch 625: W1_train shape: (2, 3) Epoch 625: b1_train shape: (1, 3) Epoch 625: W2_train shape: (3, 2) Epoch 625: b2_train shape: (1, 2) Epoch 626: X_data_scaled shape: (100, 2) Epoch 626: W1_train shape: (2, 3) Epoch 626: b1_train shape: (1, 3) Epoch 626: W2_train shape: (3, 2) Epoch 626: b2_train shape: (1, 2) Epoch 627: X_data_scaled shape: (100, 2) Epoch 627: W1_train shape: (2, 3) Epoch 627: b1_train shape: (1, 3) Epoch 627: W2_train shape: (3, 2) Epoch 627: b2_train shape: (1, 2) Epoch 628: X_data_scaled shape: (100, 2) Epoch 628: W1_train shape: (2, 3) Epoch 628: b1_train shape: (1, 3) Epoch 628: W2_train shape: (3, 2) Epoch 628: b2_train shape: (1, 2) Epoch 629: X_data_scaled shape: (100, 2) Epoch 629: W1_train shape: (2, 3) Epoch 629: b1_train shape: (1, 3) Epoch 629: W2_train shape: (3, 2) Epoch 629: b2_train shape: (1, 2) Epoch 630: X_data_scaled shape: (100, 2) Epoch 630: W1_train shape: (2, 3) Epoch 630: b1_train shape: (1, 3) Epoch 630: W2_train shape: (3, 2) Epoch 630: b2_train shape: (1, 2) Epoch 631: X_data_scaled shape: (100, 2) Epoch 631: W1_train shape: (2, 3) Epoch 631: b1_train shape: (1, 3) Epoch 631: W2_train shape: (3, 2) Epoch 631: b2_train shape: (1, 2) Epoch 632: X_data_scaled shape: (100, 2) Epoch 632: W1_train shape: (2, 3) Epoch 632: b1_train shape: (1, 3) Epoch 632: W2_train shape: (3, 2) Epoch 632: b2_train shape: (1, 2) Epoch 633: X_data_scaled shape: (100, 2) Epoch 633: W1_train shape: (2, 3) Epoch 633: b1_train shape: (1, 3) Epoch 633: W2_train shape: (3, 2) Epoch 633: b2_train shape: (1, 2) Epoch 634: X_data_scaled shape: (100, 2) Epoch 634: W1_train shape: (2, 3) Epoch 634: b1_train shape: (1, 3) Epoch 634: W2_train shape: (3, 2) Epoch 634: b2_train shape: (1, 2) Epoch 635: X_data_scaled shape: (100, 2) Epoch 635: W1_train shape: (2, 3) Epoch 635: b1_train shape: (1, 3) Epoch 635: W2_train shape: (3, 2) Epoch 635: b2_train shape: (1, 2) Epoch 636: X_data_scaled shape: (100, 2) Epoch 636: W1_train shape: (2, 3) Epoch 636: b1_train shape: (1, 3) Epoch 636: W2_train shape: (3, 2) Epoch 636: b2_train shape: (1, 2) Epoch 637: X_data_scaled shape: (100, 2) Epoch 637: W1_train shape: (2, 3) Epoch 637: b1_train shape: (1, 3) Epoch 637: W2_train shape: (3, 2) Epoch 637: b2_train shape: (1, 2) Epoch 638: X_data_scaled shape: (100, 2) Epoch 638: W1_train shape: (2, 3) Epoch 638: b1_train shape: (1, 3) Epoch 638: W2_train shape: (3, 2) Epoch 638: b2_train shape: (1, 2) Epoch 639: X_data_scaled shape: (100, 2) Epoch 639: W1_train shape: (2, 3) Epoch 639: b1_train shape: (1, 3) Epoch 639: W2_train shape: (3, 2) Epoch 639: b2_train shape: (1, 2) Epoch 640: X_data_scaled shape: (100, 2) Epoch 640: W1_train shape: (2, 3) Epoch 640: b1_train shape: (1, 3) Epoch 640: W2_train shape: (3, 2) Epoch 640: b2_train shape: (1, 2) Epoch 641: X_data_scaled shape: (100, 2) Epoch 641: W1_train shape: (2, 3) Epoch 641: b1_train shape: (1, 3) Epoch 641: W2_train shape: (3, 2) Epoch 641: b2_train shape: (1, 2) Epoch 642: X_data_scaled shape: (100, 2) Epoch 642: W1_train shape: (2, 3) Epoch 642: b1_train shape: (1, 3) Epoch 642: W2_train shape: (3, 2) Epoch 642: b2_train shape: (1, 2) Epoch 643: X_data_scaled shape: (100, 2) Epoch 643: W1_train shape: (2, 3) Epoch 643: b1_train shape: (1, 3) Epoch 643: W2_train shape: (3, 2) Epoch 643: b2_train shape: (1, 2) Epoch 644: X_data_scaled shape: (100, 2) Epoch 644: W1_train shape: (2, 3) Epoch 644: b1_train shape: (1, 3) Epoch 644: W2_train shape: (3, 2) Epoch 644: b2_train shape: (1, 2) Epoch 645: X_data_scaled shape: (100, 2) Epoch 645: W1_train shape: (2, 3) Epoch 645: b1_train shape: (1, 3) Epoch 645: W2_train shape: (3, 2) Epoch 645: b2_train shape: (1, 2) Epoch 646: X_data_scaled shape: (100, 2) Epoch 646: W1_train shape: (2, 3) Epoch 646: b1_train shape: (1, 3) Epoch 646: W2_train shape: (3, 2) Epoch 646: b2_train shape: (1, 2) Epoch 647: X_data_scaled shape: (100, 2) Epoch 647: W1_train shape: (2, 3) Epoch 647: b1_train shape: (1, 3) Epoch 647: W2_train shape: (3, 2) Epoch 647: b2_train shape: (1, 2) Epoch 648: X_data_scaled shape: (100, 2) Epoch 648: W1_train shape: (2, 3) Epoch 648: b1_train shape: (1, 3) Epoch 648: W2_train shape: (3, 2) Epoch 648: b2_train shape: (1, 2) Epoch 649: X_data_scaled shape: (100, 2) Epoch 649: W1_train shape: (2, 3) Epoch 649: b1_train shape: (1, 3) Epoch 649: W2_train shape: (3, 2) Epoch 649: b2_train shape: (1, 2) Epoch 650: X_data_scaled shape: (100, 2) Epoch 650: W1_train shape: (2, 3) Epoch 650: b1_train shape: (1, 3) Epoch 650: W2_train shape: (3, 2) Epoch 650: b2_train shape: (1, 2) Epoch 651: X_data_scaled shape: (100, 2) Epoch 651: W1_train shape: (2, 3) Epoch 651: b1_train shape: (1, 3) Epoch 651: W2_train shape: (3, 2) Epoch 651: b2_train shape: (1, 2) Epoch 652: X_data_scaled shape: (100, 2) Epoch 652: W1_train shape: (2, 3) Epoch 652: b1_train shape: (1, 3) Epoch 652: W2_train shape: (3, 2) Epoch 652: b2_train shape: (1, 2) Epoch 653: X_data_scaled shape: (100, 2) Epoch 653: W1_train shape: (2, 3) Epoch 653: b1_train shape: (1, 3) Epoch 653: W2_train shape: (3, 2) Epoch 653: b2_train shape: (1, 2) Epoch 654: X_data_scaled shape: (100, 2) Epoch 654: W1_train shape: (2, 3) Epoch 654: b1_train shape: (1, 3) Epoch 654: W2_train shape: (3, 2) Epoch 654: b2_train shape: (1, 2) Epoch 655: X_data_scaled shape: (100, 2) Epoch 655: W1_train shape: (2, 3) Epoch 655: b1_train shape: (1, 3) Epoch 655: W2_train shape: (3, 2) Epoch 655: b2_train shape: (1, 2) Epoch 656: X_data_scaled shape: (100, 2) Epoch 656: W1_train shape: (2, 3) Epoch 656: b1_train shape: (1, 3) Epoch 656: W2_train shape: (3, 2) Epoch 656: b2_train shape: (1, 2) Epoch 657: X_data_scaled shape: (100, 2) Epoch 657: W1_train shape: (2, 3) Epoch 657: b1_train shape: (1, 3) Epoch 657: W2_train shape: (3, 2) Epoch 657: b2_train shape: (1, 2) Epoch 658: X_data_scaled shape: (100, 2) Epoch 658: W1_train shape: (2, 3) Epoch 658: b1_train shape: (1, 3) Epoch 658: W2_train shape: (3, 2) Epoch 658: b2_train shape: (1, 2) Epoch 659: X_data_scaled shape: (100, 2) Epoch 659: W1_train shape: (2, 3) Epoch 659: b1_train shape: (1, 3) Epoch 659: W2_train shape: (3, 2) Epoch 659: b2_train shape: (1, 2) Epoch 660: X_data_scaled shape: (100, 2) Epoch 660: W1_train shape: (2, 3) Epoch 660: b1_train shape: (1, 3) Epoch 660: W2_train shape: (3, 2) Epoch 660: b2_train shape: (1, 2) Epoch 661: X_data_scaled shape: (100, 2) Epoch 661: W1_train shape: (2, 3) Epoch 661: b1_train shape: (1, 3) Epoch 661: W2_train shape: (3, 2) Epoch 661: b2_train shape: (1, 2) Epoch 662: X_data_scaled shape: (100, 2) Epoch 662: W1_train shape: (2, 3) Epoch 662: b1_train shape: (1, 3) Epoch 662: W2_train shape: (3, 2) Epoch 662: b2_train shape: (1, 2) Epoch 663: X_data_scaled shape: (100, 2) Epoch 663: W1_train shape: (2, 3) Epoch 663: b1_train shape: (1, 3) Epoch 663: W2_train shape: (3, 2) Epoch 663: b2_train shape: (1, 2) Epoch 664: X_data_scaled shape: (100, 2) Epoch 664: W1_train shape: (2, 3) Epoch 664: b1_train shape: (1, 3) Epoch 664: W2_train shape: (3, 2) Epoch 664: b2_train shape: (1, 2) Epoch 665: X_data_scaled shape: (100, 2) Epoch 665: W1_train shape: (2, 3) Epoch 665: b1_train shape: (1, 3) Epoch 665: W2_train shape: (3, 2) Epoch 665: b2_train shape: (1, 2) Epoch 666: X_data_scaled shape: (100, 2) Epoch 666: W1_train shape: (2, 3) Epoch 666: b1_train shape: (1, 3) Epoch 666: W2_train shape: (3, 2) Epoch 666: b2_train shape: (1, 2) Epoch 667: X_data_scaled shape: (100, 2) Epoch 667: W1_train shape: (2, 3) Epoch 667: b1_train shape: (1, 3) Epoch 667: W2_train shape: (3, 2) Epoch 667: b2_train shape: (1, 2) Epoch 668: X_data_scaled shape: (100, 2) Epoch 668: W1_train shape: (2, 3) Epoch 668: b1_train shape: (1, 3) Epoch 668: W2_train shape: (3, 2) Epoch 668: b2_train shape: (1, 2) Epoch 669: X_data_scaled shape: (100, 2) Epoch 669: W1_train shape: (2, 3) Epoch 669: b1_train shape: (1, 3) Epoch 669: W2_train shape: (3, 2) Epoch 669: b2_train shape: (1, 2) Epoch 670: X_data_scaled shape: (100, 2) Epoch 670: W1_train shape: (2, 3) Epoch 670: b1_train shape: (1, 3) Epoch 670: W2_train shape: (3, 2) Epoch 670: b2_train shape: (1, 2) Epoch 671: X_data_scaled shape: (100, 2) Epoch 671: W1_train shape: (2, 3) Epoch 671: b1_train shape: (1, 3) Epoch 671: W2_train shape: (3, 2) Epoch 671: b2_train shape: (1, 2) Epoch 672: X_data_scaled shape: (100, 2) Epoch 672: W1_train shape: (2, 3) Epoch 672: b1_train shape: (1, 3) Epoch 672: W2_train shape: (3, 2) Epoch 672: b2_train shape: (1, 2) Epoch 673: X_data_scaled shape: (100, 2) Epoch 673: W1_train shape: (2, 3) Epoch 673: b1_train shape: (1, 3) Epoch 673: W2_train shape: (3, 2) Epoch 673: b2_train shape: (1, 2) Epoch 674: X_data_scaled shape: (100, 2) Epoch 674: W1_train shape: (2, 3) Epoch 674: b1_train shape: (1, 3) Epoch 674: W2_train shape: (3, 2) Epoch 674: b2_train shape: (1, 2) Epoch 675: X_data_scaled shape: (100, 2) Epoch 675: W1_train shape: (2, 3) Epoch 675: b1_train shape: (1, 3) Epoch 675: W2_train shape: (3, 2) Epoch 675: b2_train shape: (1, 2) Epoch 676: X_data_scaled shape: (100, 2) Epoch 676: W1_train shape: (2, 3) Epoch 676: b1_train shape: (1, 3) Epoch 676: W2_train shape: (3, 2) Epoch 676: b2_train shape: (1, 2) Epoch 677: X_data_scaled shape: (100, 2) Epoch 677: W1_train shape: (2, 3) Epoch 677: b1_train shape: (1, 3) Epoch 677: W2_train shape: (3, 2) Epoch 677: b2_train shape: (1, 2) Epoch 678: X_data_scaled shape: (100, 2) Epoch 678: W1_train shape: (2, 3) Epoch 678: b1_train shape: (1, 3) Epoch 678: W2_train shape: (3, 2) Epoch 678: b2_train shape: (1, 2) Epoch 679: X_data_scaled shape: (100, 2) Epoch 679: W1_train shape: (2, 3) Epoch 679: b1_train shape: (1, 3) Epoch 679: W2_train shape: (3, 2) Epoch 679: b2_train shape: (1, 2) Epoch 680: X_data_scaled shape: (100, 2) Epoch 680: W1_train shape: (2, 3) Epoch 680: b1_train shape: (1, 3) Epoch 680: W2_train shape: (3, 2) Epoch 680: b2_train shape: (1, 2) Epoch 681: X_data_scaled shape: (100, 2) Epoch 681: W1_train shape: (2, 3) Epoch 681: b1_train shape: (1, 3) Epoch 681: W2_train shape: (3, 2) Epoch 681: b2_train shape: (1, 2) Epoch 682: X_data_scaled shape: (100, 2) Epoch 682: W1_train shape: (2, 3) Epoch 682: b1_train shape: (1, 3) Epoch 682: W2_train shape: (3, 2) Epoch 682: b2_train shape: (1, 2) Epoch 683: X_data_scaled shape: (100, 2) Epoch 683: W1_train shape: (2, 3) Epoch 683: b1_train shape: (1, 3) Epoch 683: W2_train shape: (3, 2) Epoch 683: b2_train shape: (1, 2) Epoch 684: X_data_scaled shape: (100, 2) Epoch 684: W1_train shape: (2, 3) Epoch 684: b1_train shape: (1, 3) Epoch 684: W2_train shape: (3, 2) Epoch 684: b2_train shape: (1, 2) Epoch 685: X_data_scaled shape: (100, 2) Epoch 685: W1_train shape: (2, 3) Epoch 685: b1_train shape: (1, 3) Epoch 685: W2_train shape: (3, 2) Epoch 685: b2_train shape: (1, 2) Epoch 686: X_data_scaled shape: (100, 2) Epoch 686: W1_train shape: (2, 3) Epoch 686: b1_train shape: (1, 3) Epoch 686: W2_train shape: (3, 2) Epoch 686: b2_train shape: (1, 2) Epoch 687: X_data_scaled shape: (100, 2) Epoch 687: W1_train shape: (2, 3) Epoch 687: b1_train shape: (1, 3) Epoch 687: W2_train shape: (3, 2) Epoch 687: b2_train shape: (1, 2) Epoch 688: X_data_scaled shape: (100, 2) Epoch 688: W1_train shape: (2, 3) Epoch 688: b1_train shape: (1, 3) Epoch 688: W2_train shape: (3, 2) Epoch 688: b2_train shape: (1, 2) Epoch 689: X_data_scaled shape: (100, 2) Epoch 689: W1_train shape: (2, 3) Epoch 689: b1_train shape: (1, 3) Epoch 689: W2_train shape: (3, 2) Epoch 689: b2_train shape: (1, 2) Epoch 690: X_data_scaled shape: (100, 2) Epoch 690: W1_train shape: (2, 3) Epoch 690: b1_train shape: (1, 3) Epoch 690: W2_train shape: (3, 2) Epoch 690: b2_train shape: (1, 2) Epoch 691: X_data_scaled shape: (100, 2) Epoch 691: W1_train shape: (2, 3) Epoch 691: b1_train shape: (1, 3) Epoch 691: W2_train shape: (3, 2) Epoch 691: b2_train shape: (1, 2) Epoch 692: X_data_scaled shape: (100, 2) Epoch 692: W1_train shape: (2, 3) Epoch 692: b1_train shape: (1, 3) Epoch 692: W2_train shape: (3, 2) Epoch 692: b2_train shape: (1, 2) Epoch 693: X_data_scaled shape: (100, 2) Epoch 693: W1_train shape: (2, 3) Epoch 693: b1_train shape: (1, 3) Epoch 693: W2_train shape: (3, 2) Epoch 693: b2_train shape: (1, 2) Epoch 694: X_data_scaled shape: (100, 2) Epoch 694: W1_train shape: (2, 3) Epoch 694: b1_train shape: (1, 3) Epoch 694: W2_train shape: (3, 2) Epoch 694: b2_train shape: (1, 2) Epoch 695: X_data_scaled shape: (100, 2) Epoch 695: W1_train shape: (2, 3) Epoch 695: b1_train shape: (1, 3) Epoch 695: W2_train shape: (3, 2) Epoch 695: b2_train shape: (1, 2) Epoch 696: X_data_scaled shape: (100, 2) Epoch 696: W1_train shape: (2, 3) Epoch 696: b1_train shape: (1, 3) Epoch 696: W2_train shape: (3, 2) Epoch 696: b2_train shape: (1, 2) Epoch 697: X_data_scaled shape: (100, 2) Epoch 697: W1_train shape: (2, 3) Epoch 697: b1_train shape: (1, 3) Epoch 697: W2_train shape: (3, 2) Epoch 697: b2_train shape: (1, 2) Epoch 698: X_data_scaled shape: (100, 2) Epoch 698: W1_train shape: (2, 3) Epoch 698: b1_train shape: (1, 3) Epoch 698: W2_train shape: (3, 2) Epoch 698: b2_train shape: (1, 2) Epoch 699: X_data_scaled shape: (100, 2) Epoch 699: W1_train shape: (2, 3) Epoch 699: b1_train shape: (1, 3) Epoch 699: W2_train shape: (3, 2) Epoch 699: b2_train shape: (1, 2) Epoch 700: X_data_scaled shape: (100, 2) Epoch 700: W1_train shape: (2, 3) Epoch 700: b1_train shape: (1, 3) Epoch 700: W2_train shape: (3, 2) Epoch 700: b2_train shape: (1, 2) Epoch 700/1000, Loss: 5.2983 Epoch 701: X_data_scaled shape: (100, 2) Epoch 701: W1_train shape: (2, 3) Epoch 701: b1_train shape: (1, 3) Epoch 701: W2_train shape: (3, 2) Epoch 701: b2_train shape: (1, 2) Epoch 702: X_data_scaled shape: (100, 2) Epoch 702: W1_train shape: (2, 3) Epoch 702: b1_train shape: (1, 3) Epoch 702: W2_train shape: (3, 2) Epoch 702: b2_train shape: (1, 2) Epoch 703: X_data_scaled shape: (100, 2) Epoch 703: W1_train shape: (2, 3) Epoch 703: b1_train shape: (1, 3) Epoch 703: W2_train shape: (3, 2) Epoch 703: b2_train shape: (1, 2) Epoch 704: X_data_scaled shape: (100, 2) Epoch 704: W1_train shape: (2, 3) Epoch 704: b1_train shape: (1, 3) Epoch 704: W2_train shape: (3, 2) Epoch 704: b2_train shape: (1, 2) Epoch 705: X_data_scaled shape: (100, 2) Epoch 705: W1_train shape: (2, 3) Epoch 705: b1_train shape: (1, 3) Epoch 705: W2_train shape: (3, 2) Epoch 705: b2_train shape: (1, 2) Epoch 706: X_data_scaled shape: (100, 2) Epoch 706: W1_train shape: (2, 3) Epoch 706: b1_train shape: (1, 3) Epoch 706: W2_train shape: (3, 2) Epoch 706: b2_train shape: (1, 2) Epoch 707: X_data_scaled shape: (100, 2) Epoch 707: W1_train shape: (2, 3) Epoch 707: b1_train shape: (1, 3) Epoch 707: W2_train shape: (3, 2) Epoch 707: b2_train shape: (1, 2) Epoch 708: X_data_scaled shape: (100, 2) Epoch 708: W1_train shape: (2, 3) Epoch 708: b1_train shape: (1, 3) Epoch 708: W2_train shape: (3, 2) Epoch 708: b2_train shape: (1, 2) Epoch 709: X_data_scaled shape: (100, 2) Epoch 709: W1_train shape: (2, 3) Epoch 709: b1_train shape: (1, 3) Epoch 709: W2_train shape: (3, 2) Epoch 709: b2_train shape: (1, 2) Epoch 710: X_data_scaled shape: (100, 2) Epoch 710: W1_train shape: (2, 3) Epoch 710: b1_train shape: (1, 3) Epoch 710: W2_train shape: (3, 2) Epoch 710: b2_train shape: (1, 2) Epoch 711: X_data_scaled shape: (100, 2) Epoch 711: W1_train shape: (2, 3) Epoch 711: b1_train shape: (1, 3) Epoch 711: W2_train shape: (3, 2) Epoch 711: b2_train shape: (1, 2) Epoch 712: X_data_scaled shape: (100, 2) Epoch 712: W1_train shape: (2, 3) Epoch 712: b1_train shape: (1, 3) Epoch 712: W2_train shape: (3, 2) Epoch 712: b2_train shape: (1, 2) Epoch 713: X_data_scaled shape: (100, 2) Epoch 713: W1_train shape: (2, 3) Epoch 713: b1_train shape: (1, 3) Epoch 713: W2_train shape: (3, 2) Epoch 713: b2_train shape: (1, 2) Epoch 714: X_data_scaled shape: (100, 2) Epoch 714: W1_train shape: (2, 3) Epoch 714: b1_train shape: (1, 3) Epoch 714: W2_train shape: (3, 2) Epoch 714: b2_train shape: (1, 2) Epoch 715: X_data_scaled shape: (100, 2) Epoch 715: W1_train shape: (2, 3) Epoch 715: b1_train shape: (1, 3) Epoch 715: W2_train shape: (3, 2) Epoch 715: b2_train shape: (1, 2) Epoch 716: X_data_scaled shape: (100, 2) Epoch 716: W1_train shape: (2, 3) Epoch 716: b1_train shape: (1, 3) Epoch 716: W2_train shape: (3, 2) Epoch 716: b2_train shape: (1, 2) Epoch 717: X_data_scaled shape: (100, 2) Epoch 717: W1_train shape: (2, 3) Epoch 717: b1_train shape: (1, 3) Epoch 717: W2_train shape: (3, 2) Epoch 717: b2_train shape: (1, 2) Epoch 718: X_data_scaled shape: (100, 2) Epoch 718: W1_train shape: (2, 3) Epoch 718: b1_train shape: (1, 3) Epoch 718: W2_train shape: (3, 2) Epoch 718: b2_train shape: (1, 2) Epoch 719: X_data_scaled shape: (100, 2) Epoch 719: W1_train shape: (2, 3) Epoch 719: b1_train shape: (1, 3) Epoch 719: W2_train shape: (3, 2) Epoch 719: b2_train shape: (1, 2) Epoch 720: X_data_scaled shape: (100, 2) Epoch 720: W1_train shape: (2, 3) Epoch 720: b1_train shape: (1, 3) Epoch 720: W2_train shape: (3, 2) Epoch 720: b2_train shape: (1, 2) Epoch 721: X_data_scaled shape: (100, 2) Epoch 721: W1_train shape: (2, 3) Epoch 721: b1_train shape: (1, 3) Epoch 721: W2_train shape: (3, 2) Epoch 721: b2_train shape: (1, 2) Epoch 722: X_data_scaled shape: (100, 2) Epoch 722: W1_train shape: (2, 3) Epoch 722: b1_train shape: (1, 3) Epoch 722: W2_train shape: (3, 2) Epoch 722: b2_train shape: (1, 2) Epoch 723: X_data_scaled shape: (100, 2) Epoch 723: W1_train shape: (2, 3) Epoch 723: b1_train shape: (1, 3) Epoch 723: W2_train shape: (3, 2) Epoch 723: b2_train shape: (1, 2) Epoch 724: X_data_scaled shape: (100, 2) Epoch 724: W1_train shape: (2, 3) Epoch 724: b1_train shape: (1, 3) Epoch 724: W2_train shape: (3, 2) Epoch 724: b2_train shape: (1, 2) Epoch 725: X_data_scaled shape: (100, 2) Epoch 725: W1_train shape: (2, 3) Epoch 725: b1_train shape: (1, 3) Epoch 725: W2_train shape: (3, 2) Epoch 725: b2_train shape: (1, 2) Epoch 726: X_data_scaled shape: (100, 2) Epoch 726: W1_train shape: (2, 3) Epoch 726: b1_train shape: (1, 3) Epoch 726: W2_train shape: (3, 2) Epoch 726: b2_train shape: (1, 2) Epoch 727: X_data_scaled shape: (100, 2) Epoch 727: W1_train shape: (2, 3) Epoch 727: b1_train shape: (1, 3) Epoch 727: W2_train shape: (3, 2) Epoch 727: b2_train shape: (1, 2) Epoch 728: X_data_scaled shape: (100, 2) Epoch 728: W1_train shape: (2, 3) Epoch 728: b1_train shape: (1, 3) Epoch 728: W2_train shape: (3, 2) Epoch 728: b2_train shape: (1, 2) Epoch 729: X_data_scaled shape: (100, 2) Epoch 729: W1_train shape: (2, 3) Epoch 729: b1_train shape: (1, 3) Epoch 729: W2_train shape: (3, 2) Epoch 729: b2_train shape: (1, 2) Epoch 730: X_data_scaled shape: (100, 2) Epoch 730: W1_train shape: (2, 3) Epoch 730: b1_train shape: (1, 3) Epoch 730: W2_train shape: (3, 2) Epoch 730: b2_train shape: (1, 2) Epoch 731: X_data_scaled shape: (100, 2) Epoch 731: W1_train shape: (2, 3) Epoch 731: b1_train shape: (1, 3) Epoch 731: W2_train shape: (3, 2) Epoch 731: b2_train shape: (1, 2) Epoch 732: X_data_scaled shape: (100, 2) Epoch 732: W1_train shape: (2, 3) Epoch 732: b1_train shape: (1, 3) Epoch 732: W2_train shape: (3, 2) Epoch 732: b2_train shape: (1, 2) Epoch 733: X_data_scaled shape: (100, 2) Epoch 733: W1_train shape: (2, 3) Epoch 733: b1_train shape: (1, 3) Epoch 733: W2_train shape: (3, 2) Epoch 733: b2_train shape: (1, 2) Epoch 734: X_data_scaled shape: (100, 2) Epoch 734: W1_train shape: (2, 3) Epoch 734: b1_train shape: (1, 3) Epoch 734: W2_train shape: (3, 2) Epoch 734: b2_train shape: (1, 2) Epoch 735: X_data_scaled shape: (100, 2) Epoch 735: W1_train shape: (2, 3) Epoch 735: b1_train shape: (1, 3) Epoch 735: W2_train shape: (3, 2) Epoch 735: b2_train shape: (1, 2) Epoch 736: X_data_scaled shape: (100, 2) Epoch 736: W1_train shape: (2, 3) Epoch 736: b1_train shape: (1, 3) Epoch 736: W2_train shape: (3, 2) Epoch 736: b2_train shape: (1, 2) Epoch 737: X_data_scaled shape: (100, 2) Epoch 737: W1_train shape: (2, 3) Epoch 737: b1_train shape: (1, 3) Epoch 737: W2_train shape: (3, 2) Epoch 737: b2_train shape: (1, 2) Epoch 738: X_data_scaled shape: (100, 2) Epoch 738: W1_train shape: (2, 3) Epoch 738: b1_train shape: (1, 3) Epoch 738: W2_train shape: (3, 2) Epoch 738: b2_train shape: (1, 2) Epoch 739: X_data_scaled shape: (100, 2) Epoch 739: W1_train shape: (2, 3) Epoch 739: b1_train shape: (1, 3) Epoch 739: W2_train shape: (3, 2) Epoch 739: b2_train shape: (1, 2) Epoch 740: X_data_scaled shape: (100, 2) Epoch 740: W1_train shape: (2, 3) Epoch 740: b1_train shape: (1, 3) Epoch 740: W2_train shape: (3, 2) Epoch 740: b2_train shape: (1, 2) Epoch 741: X_data_scaled shape: (100, 2) Epoch 741: W1_train shape: (2, 3) Epoch 741: b1_train shape: (1, 3) Epoch 741: W2_train shape: (3, 2) Epoch 741: b2_train shape: (1, 2) Epoch 742: X_data_scaled shape: (100, 2) Epoch 742: W1_train shape: (2, 3) Epoch 742: b1_train shape: (1, 3) Epoch 742: W2_train shape: (3, 2) Epoch 742: b2_train shape: (1, 2) Epoch 743: X_data_scaled shape: (100, 2) Epoch 743: W1_train shape: (2, 3) Epoch 743: b1_train shape: (1, 3) Epoch 743: W2_train shape: (3, 2) Epoch 743: b2_train shape: (1, 2) Epoch 744: X_data_scaled shape: (100, 2) Epoch 744: W1_train shape: (2, 3) Epoch 744: b1_train shape: (1, 3) Epoch 744: W2_train shape: (3, 2) Epoch 744: b2_train shape: (1, 2) Epoch 745: X_data_scaled shape: (100, 2) Epoch 745: W1_train shape: (2, 3) Epoch 745: b1_train shape: (1, 3) Epoch 745: W2_train shape: (3, 2) Epoch 745: b2_train shape: (1, 2) Epoch 746: X_data_scaled shape: (100, 2) Epoch 746: W1_train shape: (2, 3) Epoch 746: b1_train shape: (1, 3) Epoch 746: W2_train shape: (3, 2) Epoch 746: b2_train shape: (1, 2) Epoch 747: X_data_scaled shape: (100, 2) Epoch 747: W1_train shape: (2, 3) Epoch 747: b1_train shape: (1, 3) Epoch 747: W2_train shape: (3, 2) Epoch 747: b2_train shape: (1, 2) Epoch 748: X_data_scaled shape: (100, 2) Epoch 748: W1_train shape: (2, 3) Epoch 748: b1_train shape: (1, 3) Epoch 748: W2_train shape: (3, 2) Epoch 748: b2_train shape: (1, 2) Epoch 749: X_data_scaled shape: (100, 2) Epoch 749: W1_train shape: (2, 3) Epoch 749: b1_train shape: (1, 3) Epoch 749: W2_train shape: (3, 2) Epoch 749: b2_train shape: (1, 2) Epoch 750: X_data_scaled shape: (100, 2) Epoch 750: W1_train shape: (2, 3) Epoch 750: b1_train shape: (1, 3) Epoch 750: W2_train shape: (3, 2) Epoch 750: b2_train shape: (1, 2) Epoch 751: X_data_scaled shape: (100, 2) Epoch 751: W1_train shape: (2, 3) Epoch 751: b1_train shape: (1, 3) Epoch 751: W2_train shape: (3, 2) Epoch 751: b2_train shape: (1, 2) Epoch 752: X_data_scaled shape: (100, 2) Epoch 752: W1_train shape: (2, 3) Epoch 752: b1_train shape: (1, 3) Epoch 752: W2_train shape: (3, 2) Epoch 752: b2_train shape: (1, 2) Epoch 753: X_data_scaled shape: (100, 2) Epoch 753: W1_train shape: (2, 3) Epoch 753: b1_train shape: (1, 3) Epoch 753: W2_train shape: (3, 2) Epoch 753: b2_train shape: (1, 2) Epoch 754: X_data_scaled shape: (100, 2) Epoch 754: W1_train shape: (2, 3) Epoch 754: b1_train shape: (1, 3) Epoch 754: W2_train shape: (3, 2) Epoch 754: b2_train shape: (1, 2) Epoch 755: X_data_scaled shape: (100, 2) Epoch 755: W1_train shape: (2, 3) Epoch 755: b1_train shape: (1, 3) Epoch 755: W2_train shape: (3, 2) Epoch 755: b2_train shape: (1, 2) Epoch 756: X_data_scaled shape: (100, 2) Epoch 756: W1_train shape: (2, 3) Epoch 756: b1_train shape: (1, 3) Epoch 756: W2_train shape: (3, 2) Epoch 756: b2_train shape: (1, 2) Epoch 757: X_data_scaled shape: (100, 2) Epoch 757: W1_train shape: (2, 3) Epoch 757: b1_train shape: (1, 3) Epoch 757: W2_train shape: (3, 2) Epoch 757: b2_train shape: (1, 2) Epoch 758: X_data_scaled shape: (100, 2) Epoch 758: W1_train shape: (2, 3) Epoch 758: b1_train shape: (1, 3) Epoch 758: W2_train shape: (3, 2) Epoch 758: b2_train shape: (1, 2) Epoch 759: X_data_scaled shape: (100, 2) Epoch 759: W1_train shape: (2, 3) Epoch 759: b1_train shape: (1, 3) Epoch 759: W2_train shape: (3, 2) Epoch 759: b2_train shape: (1, 2) Epoch 760: X_data_scaled shape: (100, 2) Epoch 760: W1_train shape: (2, 3) Epoch 760: b1_train shape: (1, 3) Epoch 760: W2_train shape: (3, 2) Epoch 760: b2_train shape: (1, 2) Epoch 761: X_data_scaled shape: (100, 2) Epoch 761: W1_train shape: (2, 3) Epoch 761: b1_train shape: (1, 3) Epoch 761: W2_train shape: (3, 2) Epoch 761: b2_train shape: (1, 2) Epoch 762: X_data_scaled shape: (100, 2) Epoch 762: W1_train shape: (2, 3) Epoch 762: b1_train shape: (1, 3) Epoch 762: W2_train shape: (3, 2) Epoch 762: b2_train shape: (1, 2) Epoch 763: X_data_scaled shape: (100, 2) Epoch 763: W1_train shape: (2, 3) Epoch 763: b1_train shape: (1, 3) Epoch 763: W2_train shape: (3, 2) Epoch 763: b2_train shape: (1, 2) Epoch 764: X_data_scaled shape: (100, 2) Epoch 764: W1_train shape: (2, 3) Epoch 764: b1_train shape: (1, 3) Epoch 764: W2_train shape: (3, 2) Epoch 764: b2_train shape: (1, 2) Epoch 765: X_data_scaled shape: (100, 2) Epoch 765: W1_train shape: (2, 3) Epoch 765: b1_train shape: (1, 3) Epoch 765: W2_train shape: (3, 2) Epoch 765: b2_train shape: (1, 2) Epoch 766: X_data_scaled shape: (100, 2) Epoch 766: W1_train shape: (2, 3) Epoch 766: b1_train shape: (1, 3) Epoch 766: W2_train shape: (3, 2) Epoch 766: b2_train shape: (1, 2) Epoch 767: X_data_scaled shape: (100, 2) Epoch 767: W1_train shape: (2, 3) Epoch 767: b1_train shape: (1, 3) Epoch 767: W2_train shape: (3, 2) Epoch 767: b2_train shape: (1, 2) Epoch 768: X_data_scaled shape: (100, 2) Epoch 768: W1_train shape: (2, 3) Epoch 768: b1_train shape: (1, 3) Epoch 768: W2_train shape: (3, 2) Epoch 768: b2_train shape: (1, 2) Epoch 769: X_data_scaled shape: (100, 2) Epoch 769: W1_train shape: (2, 3) Epoch 769: b1_train shape: (1, 3) Epoch 769: W2_train shape: (3, 2) Epoch 769: b2_train shape: (1, 2) Epoch 770: X_data_scaled shape: (100, 2) Epoch 770: W1_train shape: (2, 3) Epoch 770: b1_train shape: (1, 3) Epoch 770: W2_train shape: (3, 2) Epoch 770: b2_train shape: (1, 2) Epoch 771: X_data_scaled shape: (100, 2) Epoch 771: W1_train shape: (2, 3) Epoch 771: b1_train shape: (1, 3) Epoch 771: W2_train shape: (3, 2) Epoch 771: b2_train shape: (1, 2) Epoch 772: X_data_scaled shape: (100, 2) Epoch 772: W1_train shape: (2, 3) Epoch 772: b1_train shape: (1, 3) Epoch 772: W2_train shape: (3, 2) Epoch 772: b2_train shape: (1, 2) Epoch 773: X_data_scaled shape: (100, 2) Epoch 773: W1_train shape: (2, 3) Epoch 773: b1_train shape: (1, 3) Epoch 773: W2_train shape: (3, 2) Epoch 773: b2_train shape: (1, 2) Epoch 774: X_data_scaled shape: (100, 2) Epoch 774: W1_train shape: (2, 3) Epoch 774: b1_train shape: (1, 3) Epoch 774: W2_train shape: (3, 2) Epoch 774: b2_train shape: (1, 2) Epoch 775: X_data_scaled shape: (100, 2) Epoch 775: W1_train shape: (2, 3) Epoch 775: b1_train shape: (1, 3) Epoch 775: W2_train shape: (3, 2) Epoch 775: b2_train shape: (1, 2) Epoch 776: X_data_scaled shape: (100, 2) Epoch 776: W1_train shape: (2, 3) Epoch 776: b1_train shape: (1, 3) Epoch 776: W2_train shape: (3, 2) Epoch 776: b2_train shape: (1, 2) Epoch 777: X_data_scaled shape: (100, 2) Epoch 777: W1_train shape: (2, 3) Epoch 777: b1_train shape: (1, 3) Epoch 777: W2_train shape: (3, 2) Epoch 777: b2_train shape: (1, 2) Epoch 778: X_data_scaled shape: (100, 2) Epoch 778: W1_train shape: (2, 3) Epoch 778: b1_train shape: (1, 3) Epoch 778: W2_train shape: (3, 2) Epoch 778: b2_train shape: (1, 2) Epoch 779: X_data_scaled shape: (100, 2) Epoch 779: W1_train shape: (2, 3) Epoch 779: b1_train shape: (1, 3) Epoch 779: W2_train shape: (3, 2) Epoch 779: b2_train shape: (1, 2) Epoch 780: X_data_scaled shape: (100, 2) Epoch 780: W1_train shape: (2, 3) Epoch 780: b1_train shape: (1, 3) Epoch 780: W2_train shape: (3, 2) Epoch 780: b2_train shape: (1, 2) Epoch 781: X_data_scaled shape: (100, 2) Epoch 781: W1_train shape: (2, 3) Epoch 781: b1_train shape: (1, 3) Epoch 781: W2_train shape: (3, 2) Epoch 781: b2_train shape: (1, 2) Epoch 782: X_data_scaled shape: (100, 2) Epoch 782: W1_train shape: (2, 3) Epoch 782: b1_train shape: (1, 3) Epoch 782: W2_train shape: (3, 2) Epoch 782: b2_train shape: (1, 2) Epoch 783: X_data_scaled shape: (100, 2) Epoch 783: W1_train shape: (2, 3) Epoch 783: b1_train shape: (1, 3) Epoch 783: W2_train shape: (3, 2) Epoch 783: b2_train shape: (1, 2) Epoch 784: X_data_scaled shape: (100, 2) Epoch 784: W1_train shape: (2, 3) Epoch 784: b1_train shape: (1, 3) Epoch 784: W2_train shape: (3, 2) Epoch 784: b2_train shape: (1, 2) Epoch 785: X_data_scaled shape: (100, 2) Epoch 785: W1_train shape: (2, 3) Epoch 785: b1_train shape: (1, 3) Epoch 785: W2_train shape: (3, 2) Epoch 785: b2_train shape: (1, 2) Epoch 786: X_data_scaled shape: (100, 2) Epoch 786: W1_train shape: (2, 3) Epoch 786: b1_train shape: (1, 3) Epoch 786: W2_train shape: (3, 2) Epoch 786: b2_train shape: (1, 2) Epoch 787: X_data_scaled shape: (100, 2) Epoch 787: W1_train shape: (2, 3) Epoch 787: b1_train shape: (1, 3) Epoch 787: W2_train shape: (3, 2) Epoch 787: b2_train shape: (1, 2) Epoch 788: X_data_scaled shape: (100, 2) Epoch 788: W1_train shape: (2, 3) Epoch 788: b1_train shape: (1, 3) Epoch 788: W2_train shape: (3, 2) Epoch 788: b2_train shape: (1, 2) Epoch 789: X_data_scaled shape: (100, 2) Epoch 789: W1_train shape: (2, 3) Epoch 789: b1_train shape: (1, 3) Epoch 789: W2_train shape: (3, 2) Epoch 789: b2_train shape: (1, 2) Epoch 790: X_data_scaled shape: (100, 2) Epoch 790: W1_train shape: (2, 3) Epoch 790: b1_train shape: (1, 3) Epoch 790: W2_train shape: (3, 2) Epoch 790: b2_train shape: (1, 2) Epoch 791: X_data_scaled shape: (100, 2) Epoch 791: W1_train shape: (2, 3) Epoch 791: b1_train shape: (1, 3) Epoch 791: W2_train shape: (3, 2) Epoch 791: b2_train shape: (1, 2) Epoch 792: X_data_scaled shape: (100, 2) Epoch 792: W1_train shape: (2, 3) Epoch 792: b1_train shape: (1, 3) Epoch 792: W2_train shape: (3, 2) Epoch 792: b2_train shape: (1, 2) Epoch 793: X_data_scaled shape: (100, 2) Epoch 793: W1_train shape: (2, 3) Epoch 793: b1_train shape: (1, 3) Epoch 793: W2_train shape: (3, 2) Epoch 793: b2_train shape: (1, 2) Epoch 794: X_data_scaled shape: (100, 2) Epoch 794: W1_train shape: (2, 3) Epoch 794: b1_train shape: (1, 3) Epoch 794: W2_train shape: (3, 2) Epoch 794: b2_train shape: (1, 2) Epoch 795: X_data_scaled shape: (100, 2) Epoch 795: W1_train shape: (2, 3) Epoch 795: b1_train shape: (1, 3) Epoch 795: W2_train shape: (3, 2) Epoch 795: b2_train shape: (1, 2) Epoch 796: X_data_scaled shape: (100, 2) Epoch 796: W1_train shape: (2, 3) Epoch 796: b1_train shape: (1, 3) Epoch 796: W2_train shape: (3, 2) Epoch 796: b2_train shape: (1, 2) Epoch 797: X_data_scaled shape: (100, 2) Epoch 797: W1_train shape: (2, 3) Epoch 797: b1_train shape: (1, 3) Epoch 797: W2_train shape: (3, 2) Epoch 797: b2_train shape: (1, 2) Epoch 798: X_data_scaled shape: (100, 2) Epoch 798: W1_train shape: (2, 3) Epoch 798: b1_train shape: (1, 3) Epoch 798: W2_train shape: (3, 2) Epoch 798: b2_train shape: (1, 2) Epoch 799: X_data_scaled shape: (100, 2) Epoch 799: W1_train shape: (2, 3) Epoch 799: b1_train shape: (1, 3) Epoch 799: W2_train shape: (3, 2) Epoch 799: b2_train shape: (1, 2) Epoch 800: X_data_scaled shape: (100, 2) Epoch 800: W1_train shape: (2, 3) Epoch 800: b1_train shape: (1, 3) Epoch 800: W2_train shape: (3, 2) Epoch 800: b2_train shape: (1, 2) Epoch 800/1000, Loss: 5.2983 Epoch 801: X_data_scaled shape: (100, 2) Epoch 801: W1_train shape: (2, 3) Epoch 801: b1_train shape: (1, 3) Epoch 801: W2_train shape: (3, 2) Epoch 801: b2_train shape: (1, 2) Epoch 802: X_data_scaled shape: (100, 2) Epoch 802: W1_train shape: (2, 3) Epoch 802: b1_train shape: (1, 3) Epoch 802: W2_train shape: (3, 2) Epoch 802: b2_train shape: (1, 2) Epoch 803: X_data_scaled shape: (100, 2) Epoch 803: W1_train shape: (2, 3) Epoch 803: b1_train shape: (1, 3) Epoch 803: W2_train shape: (3, 2) Epoch 803: b2_train shape: (1, 2) Epoch 804: X_data_scaled shape: (100, 2) Epoch 804: W1_train shape: (2, 3) Epoch 804: b1_train shape: (1, 3) Epoch 804: W2_train shape: (3, 2) Epoch 804: b2_train shape: (1, 2) Epoch 805: X_data_scaled shape: (100, 2) Epoch 805: W1_train shape: (2, 3) Epoch 805: b1_train shape: (1, 3) Epoch 805: W2_train shape: (3, 2) Epoch 805: b2_train shape: (1, 2) Epoch 806: X_data_scaled shape: (100, 2) Epoch 806: W1_train shape: (2, 3) Epoch 806: b1_train shape: (1, 3) Epoch 806: W2_train shape: (3, 2) Epoch 806: b2_train shape: (1, 2) Epoch 807: X_data_scaled shape: (100, 2) Epoch 807: W1_train shape: (2, 3) Epoch 807: b1_train shape: (1, 3) Epoch 807: W2_train shape: (3, 2) Epoch 807: b2_train shape: (1, 2) Epoch 808: X_data_scaled shape: (100, 2) Epoch 808: W1_train shape: (2, 3) Epoch 808: b1_train shape: (1, 3) Epoch 808: W2_train shape: (3, 2) Epoch 808: b2_train shape: (1, 2) Epoch 809: X_data_scaled shape: (100, 2) Epoch 809: W1_train shape: (2, 3) Epoch 809: b1_train shape: (1, 3) Epoch 809: W2_train shape: (3, 2) Epoch 809: b2_train shape: (1, 2) Epoch 810: X_data_scaled shape: (100, 2) Epoch 810: W1_train shape: (2, 3) Epoch 810: b1_train shape: (1, 3) Epoch 810: W2_train shape: (3, 2) Epoch 810: b2_train shape: (1, 2) Epoch 811: X_data_scaled shape: (100, 2) Epoch 811: W1_train shape: (2, 3) Epoch 811: b1_train shape: (1, 3) Epoch 811: W2_train shape: (3, 2) Epoch 811: b2_train shape: (1, 2) Epoch 812: X_data_scaled shape: (100, 2) Epoch 812: W1_train shape: (2, 3) Epoch 812: b1_train shape: (1, 3) Epoch 812: W2_train shape: (3, 2) Epoch 812: b2_train shape: (1, 2) Epoch 813: X_data_scaled shape: (100, 2) Epoch 813: W1_train shape: (2, 3) Epoch 813: b1_train shape: (1, 3) Epoch 813: W2_train shape: (3, 2) Epoch 813: b2_train shape: (1, 2) Epoch 814: X_data_scaled shape: (100, 2) Epoch 814: W1_train shape: (2, 3) Epoch 814: b1_train shape: (1, 3) Epoch 814: W2_train shape: (3, 2) Epoch 814: b2_train shape: (1, 2) Epoch 815: X_data_scaled shape: (100, 2) Epoch 815: W1_train shape: (2, 3) Epoch 815: b1_train shape: (1, 3) Epoch 815: W2_train shape: (3, 2) Epoch 815: b2_train shape: (1, 2) Epoch 816: X_data_scaled shape: (100, 2) Epoch 816: W1_train shape: (2, 3) Epoch 816: b1_train shape: (1, 3) Epoch 816: W2_train shape: (3, 2) Epoch 816: b2_train shape: (1, 2) Epoch 817: X_data_scaled shape: (100, 2) Epoch 817: W1_train shape: (2, 3) Epoch 817: b1_train shape: (1, 3) Epoch 817: W2_train shape: (3, 2) Epoch 817: b2_train shape: (1, 2) Epoch 818: X_data_scaled shape: (100, 2) Epoch 818: W1_train shape: (2, 3) Epoch 818: b1_train shape: (1, 3) Epoch 818: W2_train shape: (3, 2) Epoch 818: b2_train shape: (1, 2) Epoch 819: X_data_scaled shape: (100, 2) Epoch 819: W1_train shape: (2, 3) Epoch 819: b1_train shape: (1, 3) Epoch 819: W2_train shape: (3, 2) Epoch 819: b2_train shape: (1, 2) Epoch 820: X_data_scaled shape: (100, 2) Epoch 820: W1_train shape: (2, 3) Epoch 820: b1_train shape: (1, 3) Epoch 820: W2_train shape: (3, 2) Epoch 820: b2_train shape: (1, 2) Epoch 821: X_data_scaled shape: (100, 2) Epoch 821: W1_train shape: (2, 3) Epoch 821: b1_train shape: (1, 3) Epoch 821: W2_train shape: (3, 2) Epoch 821: b2_train shape: (1, 2) Epoch 822: X_data_scaled shape: (100, 2) Epoch 822: W1_train shape: (2, 3) Epoch 822: b1_train shape: (1, 3) Epoch 822: W2_train shape: (3, 2) Epoch 822: b2_train shape: (1, 2) Epoch 823: X_data_scaled shape: (100, 2) Epoch 823: W1_train shape: (2, 3) Epoch 823: b1_train shape: (1, 3) Epoch 823: W2_train shape: (3, 2) Epoch 823: b2_train shape: (1, 2) Epoch 824: X_data_scaled shape: (100, 2) Epoch 824: W1_train shape: (2, 3) Epoch 824: b1_train shape: (1, 3) Epoch 824: W2_train shape: (3, 2) Epoch 824: b2_train shape: (1, 2) Epoch 825: X_data_scaled shape: (100, 2) Epoch 825: W1_train shape: (2, 3) Epoch 825: b1_train shape: (1, 3) Epoch 825: W2_train shape: (3, 2) Epoch 825: b2_train shape: (1, 2) Epoch 826: X_data_scaled shape: (100, 2) Epoch 826: W1_train shape: (2, 3) Epoch 826: b1_train shape: (1, 3) Epoch 826: W2_train shape: (3, 2) Epoch 826: b2_train shape: (1, 2) Epoch 827: X_data_scaled shape: (100, 2) Epoch 827: W1_train shape: (2, 3) Epoch 827: b1_train shape: (1, 3) Epoch 827: W2_train shape: (3, 2) Epoch 827: b2_train shape: (1, 2) Epoch 828: X_data_scaled shape: (100, 2) Epoch 828: W1_train shape: (2, 3) Epoch 828: b1_train shape: (1, 3) Epoch 828: W2_train shape: (3, 2) Epoch 828: b2_train shape: (1, 2) Epoch 829: X_data_scaled shape: (100, 2) Epoch 829: W1_train shape: (2, 3) Epoch 829: b1_train shape: (1, 3) Epoch 829: W2_train shape: (3, 2) Epoch 829: b2_train shape: (1, 2) Epoch 830: X_data_scaled shape: (100, 2) Epoch 830: W1_train shape: (2, 3) Epoch 830: b1_train shape: (1, 3) Epoch 830: W2_train shape: (3, 2) Epoch 830: b2_train shape: (1, 2) Epoch 831: X_data_scaled shape: (100, 2) Epoch 831: W1_train shape: (2, 3) Epoch 831: b1_train shape: (1, 3) Epoch 831: W2_train shape: (3, 2) Epoch 831: b2_train shape: (1, 2) Epoch 832: X_data_scaled shape: (100, 2) Epoch 832: W1_train shape: (2, 3) Epoch 832: b1_train shape: (1, 3) Epoch 832: W2_train shape: (3, 2) Epoch 832: b2_train shape: (1, 2) Epoch 833: X_data_scaled shape: (100, 2) Epoch 833: W1_train shape: (2, 3) Epoch 833: b1_train shape: (1, 3) Epoch 833: W2_train shape: (3, 2) Epoch 833: b2_train shape: (1, 2) Epoch 834: X_data_scaled shape: (100, 2) Epoch 834: W1_train shape: (2, 3) Epoch 834: b1_train shape: (1, 3) Epoch 834: W2_train shape: (3, 2) Epoch 834: b2_train shape: (1, 2) Epoch 835: X_data_scaled shape: (100, 2) Epoch 835: W1_train shape: (2, 3) Epoch 835: b1_train shape: (1, 3) Epoch 835: W2_train shape: (3, 2) Epoch 835: b2_train shape: (1, 2) Epoch 836: X_data_scaled shape: (100, 2) Epoch 836: W1_train shape: (2, 3) Epoch 836: b1_train shape: (1, 3) Epoch 836: W2_train shape: (3, 2) Epoch 836: b2_train shape: (1, 2) Epoch 837: X_data_scaled shape: (100, 2) Epoch 837: W1_train shape: (2, 3) Epoch 837: b1_train shape: (1, 3) Epoch 837: W2_train shape: (3, 2) Epoch 837: b2_train shape: (1, 2) Epoch 838: X_data_scaled shape: (100, 2) Epoch 838: W1_train shape: (2, 3) Epoch 838: b1_train shape: (1, 3) Epoch 838: W2_train shape: (3, 2) Epoch 838: b2_train shape: (1, 2) Epoch 839: X_data_scaled shape: (100, 2) Epoch 839: W1_train shape: (2, 3) Epoch 839: b1_train shape: (1, 3) Epoch 839: W2_train shape: (3, 2) Epoch 839: b2_train shape: (1, 2) Epoch 840: X_data_scaled shape: (100, 2) Epoch 840: W1_train shape: (2, 3) Epoch 840: b1_train shape: (1, 3) Epoch 840: W2_train shape: (3, 2) Epoch 840: b2_train shape: (1, 2) Epoch 841: X_data_scaled shape: (100, 2) Epoch 841: W1_train shape: (2, 3) Epoch 841: b1_train shape: (1, 3) Epoch 841: W2_train shape: (3, 2) Epoch 841: b2_train shape: (1, 2) Epoch 842: X_data_scaled shape: (100, 2) Epoch 842: W1_train shape: (2, 3) Epoch 842: b1_train shape: (1, 3) Epoch 842: W2_train shape: (3, 2) Epoch 842: b2_train shape: (1, 2) Epoch 843: X_data_scaled shape: (100, 2) Epoch 843: W1_train shape: (2, 3) Epoch 843: b1_train shape: (1, 3) Epoch 843: W2_train shape: (3, 2) Epoch 843: b2_train shape: (1, 2) Epoch 844: X_data_scaled shape: (100, 2) Epoch 844: W1_train shape: (2, 3) Epoch 844: b1_train shape: (1, 3) Epoch 844: W2_train shape: (3, 2) Epoch 844: b2_train shape: (1, 2) Epoch 845: X_data_scaled shape: (100, 2) Epoch 845: W1_train shape: (2, 3) Epoch 845: b1_train shape: (1, 3) Epoch 845: W2_train shape: (3, 2) Epoch 845: b2_train shape: (1, 2) Epoch 846: X_data_scaled shape: (100, 2) Epoch 846: W1_train shape: (2, 3) Epoch 846: b1_train shape: (1, 3) Epoch 846: W2_train shape: (3, 2) Epoch 846: b2_train shape: (1, 2) Epoch 847: X_data_scaled shape: (100, 2) Epoch 847: W1_train shape: (2, 3) Epoch 847: b1_train shape: (1, 3) Epoch 847: W2_train shape: (3, 2) Epoch 847: b2_train shape: (1, 2) Epoch 848: X_data_scaled shape: (100, 2) Epoch 848: W1_train shape: (2, 3) Epoch 848: b1_train shape: (1, 3) Epoch 848: W2_train shape: (3, 2) Epoch 848: b2_train shape: (1, 2) Epoch 849: X_data_scaled shape: (100, 2) Epoch 849: W1_train shape: (2, 3) Epoch 849: b1_train shape: (1, 3) Epoch 849: W2_train shape: (3, 2) Epoch 849: b2_train shape: (1, 2) Epoch 850: X_data_scaled shape: (100, 2) Epoch 850: W1_train shape: (2, 3) Epoch 850: b1_train shape: (1, 3) Epoch 850: W2_train shape: (3, 2) Epoch 850: b2_train shape: (1, 2) Epoch 851: X_data_scaled shape: (100, 2) Epoch 851: W1_train shape: (2, 3) Epoch 851: b1_train shape: (1, 3) Epoch 851: W2_train shape: (3, 2) Epoch 851: b2_train shape: (1, 2) Epoch 852: X_data_scaled shape: (100, 2) Epoch 852: W1_train shape: (2, 3) Epoch 852: b1_train shape: (1, 3) Epoch 852: W2_train shape: (3, 2) Epoch 852: b2_train shape: (1, 2) Epoch 853: X_data_scaled shape: (100, 2) Epoch 853: W1_train shape: (2, 3) Epoch 853: b1_train shape: (1, 3) Epoch 853: W2_train shape: (3, 2) Epoch 853: b2_train shape: (1, 2) Epoch 854: X_data_scaled shape: (100, 2) Epoch 854: W1_train shape: (2, 3) Epoch 854: b1_train shape: (1, 3) Epoch 854: W2_train shape: (3, 2) Epoch 854: b2_train shape: (1, 2) Epoch 855: X_data_scaled shape: (100, 2) Epoch 855: W1_train shape: (2, 3) Epoch 855: b1_train shape: (1, 3) Epoch 855: W2_train shape: (3, 2) Epoch 855: b2_train shape: (1, 2) Epoch 856: X_data_scaled shape: (100, 2) Epoch 856: W1_train shape: (2, 3) Epoch 856: b1_train shape: (1, 3) Epoch 856: W2_train shape: (3, 2) Epoch 856: b2_train shape: (1, 2) Epoch 857: X_data_scaled shape: (100, 2) Epoch 857: W1_train shape: (2, 3) Epoch 857: b1_train shape: (1, 3) Epoch 857: W2_train shape: (3, 2) Epoch 857: b2_train shape: (1, 2) Epoch 858: X_data_scaled shape: (100, 2) Epoch 858: W1_train shape: (2, 3) Epoch 858: b1_train shape: (1, 3) Epoch 858: W2_train shape: (3, 2) Epoch 858: b2_train shape: (1, 2) Epoch 859: X_data_scaled shape: (100, 2) Epoch 859: W1_train shape: (2, 3) Epoch 859: b1_train shape: (1, 3) Epoch 859: W2_train shape: (3, 2) Epoch 859: b2_train shape: (1, 2) Epoch 860: X_data_scaled shape: (100, 2) Epoch 860: W1_train shape: (2, 3) Epoch 860: b1_train shape: (1, 3) Epoch 860: W2_train shape: (3, 2) Epoch 860: b2_train shape: (1, 2) Epoch 861: X_data_scaled shape: (100, 2) Epoch 861: W1_train shape: (2, 3) Epoch 861: b1_train shape: (1, 3) Epoch 861: W2_train shape: (3, 2) Epoch 861: b2_train shape: (1, 2) Epoch 862: X_data_scaled shape: (100, 2) Epoch 862: W1_train shape: (2, 3) Epoch 862: b1_train shape: (1, 3) Epoch 862: W2_train shape: (3, 2) Epoch 862: b2_train shape: (1, 2) Epoch 863: X_data_scaled shape: (100, 2) Epoch 863: W1_train shape: (2, 3) Epoch 863: b1_train shape: (1, 3) Epoch 863: W2_train shape: (3, 2) Epoch 863: b2_train shape: (1, 2) Epoch 864: X_data_scaled shape: (100, 2) Epoch 864: W1_train shape: (2, 3) Epoch 864: b1_train shape: (1, 3) Epoch 864: W2_train shape: (3, 2) Epoch 864: b2_train shape: (1, 2) Epoch 865: X_data_scaled shape: (100, 2) Epoch 865: W1_train shape: (2, 3) Epoch 865: b1_train shape: (1, 3) Epoch 865: W2_train shape: (3, 2) Epoch 865: b2_train shape: (1, 2) Epoch 866: X_data_scaled shape: (100, 2) Epoch 866: W1_train shape: (2, 3) Epoch 866: b1_train shape: (1, 3) Epoch 866: W2_train shape: (3, 2) Epoch 866: b2_train shape: (1, 2) Epoch 867: X_data_scaled shape: (100, 2) Epoch 867: W1_train shape: (2, 3) Epoch 867: b1_train shape: (1, 3) Epoch 867: W2_train shape: (3, 2) Epoch 867: b2_train shape: (1, 2) Epoch 868: X_data_scaled shape: (100, 2) Epoch 868: W1_train shape: (2, 3) Epoch 868: b1_train shape: (1, 3) Epoch 868: W2_train shape: (3, 2) Epoch 868: b2_train shape: (1, 2) Epoch 869: X_data_scaled shape: (100, 2) Epoch 869: W1_train shape: (2, 3) Epoch 869: b1_train shape: (1, 3) Epoch 869: W2_train shape: (3, 2) Epoch 869: b2_train shape: (1, 2) Epoch 870: X_data_scaled shape: (100, 2) Epoch 870: W1_train shape: (2, 3) Epoch 870: b1_train shape: (1, 3) Epoch 870: W2_train shape: (3, 2) Epoch 870: b2_train shape: (1, 2) Epoch 871: X_data_scaled shape: (100, 2) Epoch 871: W1_train shape: (2, 3) Epoch 871: b1_train shape: (1, 3) Epoch 871: W2_train shape: (3, 2) Epoch 871: b2_train shape: (1, 2) Epoch 872: X_data_scaled shape: (100, 2) Epoch 872: W1_train shape: (2, 3) Epoch 872: b1_train shape: (1, 3) Epoch 872: W2_train shape: (3, 2) Epoch 872: b2_train shape: (1, 2) Epoch 873: X_data_scaled shape: (100, 2) Epoch 873: W1_train shape: (2, 3) Epoch 873: b1_train shape: (1, 3) Epoch 873: W2_train shape: (3, 2) Epoch 873: b2_train shape: (1, 2) Epoch 874: X_data_scaled shape: (100, 2) Epoch 874: W1_train shape: (2, 3) Epoch 874: b1_train shape: (1, 3) Epoch 874: W2_train shape: (3, 2) Epoch 874: b2_train shape: (1, 2) Epoch 875: X_data_scaled shape: (100, 2) Epoch 875: W1_train shape: (2, 3) Epoch 875: b1_train shape: (1, 3) Epoch 875: W2_train shape: (3, 2) Epoch 875: b2_train shape: (1, 2) Epoch 876: X_data_scaled shape: (100, 2) Epoch 876: W1_train shape: (2, 3) Epoch 876: b1_train shape: (1, 3) Epoch 876: W2_train shape: (3, 2) Epoch 876: b2_train shape: (1, 2) Epoch 877: X_data_scaled shape: (100, 2) Epoch 877: W1_train shape: (2, 3) Epoch 877: b1_train shape: (1, 3) Epoch 877: W2_train shape: (3, 2) Epoch 877: b2_train shape: (1, 2) Epoch 878: X_data_scaled shape: (100, 2) Epoch 878: W1_train shape: (2, 3) Epoch 878: b1_train shape: (1, 3) Epoch 878: W2_train shape: (3, 2) Epoch 878: b2_train shape: (1, 2) Epoch 879: X_data_scaled shape: (100, 2) Epoch 879: W1_train shape: (2, 3) Epoch 879: b1_train shape: (1, 3) Epoch 879: W2_train shape: (3, 2) Epoch 879: b2_train shape: (1, 2) Epoch 880: X_data_scaled shape: (100, 2) Epoch 880: W1_train shape: (2, 3) Epoch 880: b1_train shape: (1, 3) Epoch 880: W2_train shape: (3, 2) Epoch 880: b2_train shape: (1, 2) Epoch 881: X_data_scaled shape: (100, 2) Epoch 881: W1_train shape: (2, 3) Epoch 881: b1_train shape: (1, 3) Epoch 881: W2_train shape: (3, 2) Epoch 881: b2_train shape: (1, 2) Epoch 882: X_data_scaled shape: (100, 2) Epoch 882: W1_train shape: (2, 3) Epoch 882: b1_train shape: (1, 3) Epoch 882: W2_train shape: (3, 2) Epoch 882: b2_train shape: (1, 2) Epoch 883: X_data_scaled shape: (100, 2) Epoch 883: W1_train shape: (2, 3) Epoch 883: b1_train shape: (1, 3) Epoch 883: W2_train shape: (3, 2) Epoch 883: b2_train shape: (1, 2) Epoch 884: X_data_scaled shape: (100, 2) Epoch 884: W1_train shape: (2, 3) Epoch 884: b1_train shape: (1, 3) Epoch 884: W2_train shape: (3, 2) Epoch 884: b2_train shape: (1, 2) Epoch 885: X_data_scaled shape: (100, 2) Epoch 885: W1_train shape: (2, 3) Epoch 885: b1_train shape: (1, 3) Epoch 885: W2_train shape: (3, 2) Epoch 885: b2_train shape: (1, 2) Epoch 886: X_data_scaled shape: (100, 2) Epoch 886: W1_train shape: (2, 3) Epoch 886: b1_train shape: (1, 3) Epoch 886: W2_train shape: (3, 2) Epoch 886: b2_train shape: (1, 2) Epoch 887: X_data_scaled shape: (100, 2) Epoch 887: W1_train shape: (2, 3) Epoch 887: b1_train shape: (1, 3) Epoch 887: W2_train shape: (3, 2) Epoch 887: b2_train shape: (1, 2) Epoch 888: X_data_scaled shape: (100, 2) Epoch 888: W1_train shape: (2, 3) Epoch 888: b1_train shape: (1, 3) Epoch 888: W2_train shape: (3, 2) Epoch 888: b2_train shape: (1, 2) Epoch 889: X_data_scaled shape: (100, 2) Epoch 889: W1_train shape: (2, 3) Epoch 889: b1_train shape: (1, 3) Epoch 889: W2_train shape: (3, 2) Epoch 889: b2_train shape: (1, 2) Epoch 890: X_data_scaled shape: (100, 2) Epoch 890: W1_train shape: (2, 3) Epoch 890: b1_train shape: (1, 3) Epoch 890: W2_train shape: (3, 2) Epoch 890: b2_train shape: (1, 2) Epoch 891: X_data_scaled shape: (100, 2) Epoch 891: W1_train shape: (2, 3) Epoch 891: b1_train shape: (1, 3) Epoch 891: W2_train shape: (3, 2) Epoch 891: b2_train shape: (1, 2) Epoch 892: X_data_scaled shape: (100, 2) Epoch 892: W1_train shape: (2, 3) Epoch 892: b1_train shape: (1, 3) Epoch 892: W2_train shape: (3, 2) Epoch 892: b2_train shape: (1, 2) Epoch 893: X_data_scaled shape: (100, 2) Epoch 893: W1_train shape: (2, 3) Epoch 893: b1_train shape: (1, 3) Epoch 893: W2_train shape: (3, 2) Epoch 893: b2_train shape: (1, 2) Epoch 894: X_data_scaled shape: (100, 2) Epoch 894: W1_train shape: (2, 3) Epoch 894: b1_train shape: (1, 3) Epoch 894: W2_train shape: (3, 2) Epoch 894: b2_train shape: (1, 2) Epoch 895: X_data_scaled shape: (100, 2) Epoch 895: W1_train shape: (2, 3) Epoch 895: b1_train shape: (1, 3) Epoch 895: W2_train shape: (3, 2) Epoch 895: b2_train shape: (1, 2) Epoch 896: X_data_scaled shape: (100, 2) Epoch 896: W1_train shape: (2, 3) Epoch 896: b1_train shape: (1, 3) Epoch 896: W2_train shape: (3, 2) Epoch 896: b2_train shape: (1, 2) Epoch 897: X_data_scaled shape: (100, 2) Epoch 897: W1_train shape: (2, 3) Epoch 897: b1_train shape: (1, 3) Epoch 897: W2_train shape: (3, 2) Epoch 897: b2_train shape: (1, 2) Epoch 898: X_data_scaled shape: (100, 2) Epoch 898: W1_train shape: (2, 3) Epoch 898: b1_train shape: (1, 3) Epoch 898: W2_train shape: (3, 2) Epoch 898: b2_train shape: (1, 2) Epoch 899: X_data_scaled shape: (100, 2) Epoch 899: W1_train shape: (2, 3) Epoch 899: b1_train shape: (1, 3) Epoch 899: W2_train shape: (3, 2) Epoch 899: b2_train shape: (1, 2) Epoch 900: X_data_scaled shape: (100, 2) Epoch 900: W1_train shape: (2, 3) Epoch 900: b1_train shape: (1, 3) Epoch 900: W2_train shape: (3, 2) Epoch 900: b2_train shape: (1, 2) Epoch 900/1000, Loss: 5.2983 Epoch 901: X_data_scaled shape: (100, 2) Epoch 901: W1_train shape: (2, 3) Epoch 901: b1_train shape: (1, 3) Epoch 901: W2_train shape: (3, 2) Epoch 901: b2_train shape: (1, 2) Epoch 902: X_data_scaled shape: (100, 2) Epoch 902: W1_train shape: (2, 3) Epoch 902: b1_train shape: (1, 3) Epoch 902: W2_train shape: (3, 2) Epoch 902: b2_train shape: (1, 2) Epoch 903: X_data_scaled shape: (100, 2) Epoch 903: W1_train shape: (2, 3) Epoch 903: b1_train shape: (1, 3) Epoch 903: W2_train shape: (3, 2) Epoch 903: b2_train shape: (1, 2) Epoch 904: X_data_scaled shape: (100, 2) Epoch 904: W1_train shape: (2, 3) Epoch 904: b1_train shape: (1, 3) Epoch 904: W2_train shape: (3, 2) Epoch 904: b2_train shape: (1, 2) Epoch 905: X_data_scaled shape: (100, 2) Epoch 905: W1_train shape: (2, 3) Epoch 905: b1_train shape: (1, 3) Epoch 905: W2_train shape: (3, 2) Epoch 905: b2_train shape: (1, 2) Epoch 906: X_data_scaled shape: (100, 2) Epoch 906: W1_train shape: (2, 3) Epoch 906: b1_train shape: (1, 3) Epoch 906: W2_train shape: (3, 2) Epoch 906: b2_train shape: (1, 2) Epoch 907: X_data_scaled shape: (100, 2) Epoch 907: W1_train shape: (2, 3) Epoch 907: b1_train shape: (1, 3) Epoch 907: W2_train shape: (3, 2) Epoch 907: b2_train shape: (1, 2) Epoch 908: X_data_scaled shape: (100, 2) Epoch 908: W1_train shape: (2, 3) Epoch 908: b1_train shape: (1, 3) Epoch 908: W2_train shape: (3, 2) Epoch 908: b2_train shape: (1, 2) Epoch 909: X_data_scaled shape: (100, 2) Epoch 909: W1_train shape: (2, 3) Epoch 909: b1_train shape: (1, 3) Epoch 909: W2_train shape: (3, 2) Epoch 909: b2_train shape: (1, 2) Epoch 910: X_data_scaled shape: (100, 2) Epoch 910: W1_train shape: (2, 3) Epoch 910: b1_train shape: (1, 3) Epoch 910: W2_train shape: (3, 2) Epoch 910: b2_train shape: (1, 2) Epoch 911: X_data_scaled shape: (100, 2) Epoch 911: W1_train shape: (2, 3) Epoch 911: b1_train shape: (1, 3) Epoch 911: W2_train shape: (3, 2) Epoch 911: b2_train shape: (1, 2) Epoch 912: X_data_scaled shape: (100, 2) Epoch 912: W1_train shape: (2, 3) Epoch 912: b1_train shape: (1, 3) Epoch 912: W2_train shape: (3, 2) Epoch 912: b2_train shape: (1, 2) Epoch 913: X_data_scaled shape: (100, 2) Epoch 913: W1_train shape: (2, 3) Epoch 913: b1_train shape: (1, 3) Epoch 913: W2_train shape: (3, 2) Epoch 913: b2_train shape: (1, 2) Epoch 914: X_data_scaled shape: (100, 2) Epoch 914: W1_train shape: (2, 3) Epoch 914: b1_train shape: (1, 3) Epoch 914: W2_train shape: (3, 2) Epoch 914: b2_train shape: (1, 2) Epoch 915: X_data_scaled shape: (100, 2) Epoch 915: W1_train shape: (2, 3) Epoch 915: b1_train shape: (1, 3) Epoch 915: W2_train shape: (3, 2) Epoch 915: b2_train shape: (1, 2) Epoch 916: X_data_scaled shape: (100, 2) Epoch 916: W1_train shape: (2, 3) Epoch 916: b1_train shape: (1, 3) Epoch 916: W2_train shape: (3, 2) Epoch 916: b2_train shape: (1, 2) Epoch 917: X_data_scaled shape: (100, 2) Epoch 917: W1_train shape: (2, 3) Epoch 917: b1_train shape: (1, 3) Epoch 917: W2_train shape: (3, 2) Epoch 917: b2_train shape: (1, 2) Epoch 918: X_data_scaled shape: (100, 2) Epoch 918: W1_train shape: (2, 3) Epoch 918: b1_train shape: (1, 3) Epoch 918: W2_train shape: (3, 2) Epoch 918: b2_train shape: (1, 2) Epoch 919: X_data_scaled shape: (100, 2) Epoch 919: W1_train shape: (2, 3) Epoch 919: b1_train shape: (1, 3) Epoch 919: W2_train shape: (3, 2) Epoch 919: b2_train shape: (1, 2) Epoch 920: X_data_scaled shape: (100, 2) Epoch 920: W1_train shape: (2, 3) Epoch 920: b1_train shape: (1, 3) Epoch 920: W2_train shape: (3, 2) Epoch 920: b2_train shape: (1, 2) Epoch 921: X_data_scaled shape: (100, 2) Epoch 921: W1_train shape: (2, 3) Epoch 921: b1_train shape: (1, 3) Epoch 921: W2_train shape: (3, 2) Epoch 921: b2_train shape: (1, 2) Epoch 922: X_data_scaled shape: (100, 2) Epoch 922: W1_train shape: (2, 3) Epoch 922: b1_train shape: (1, 3) Epoch 922: W2_train shape: (3, 2) Epoch 922: b2_train shape: (1, 2) Epoch 923: X_data_scaled shape: (100, 2) Epoch 923: W1_train shape: (2, 3) Epoch 923: b1_train shape: (1, 3) Epoch 923: W2_train shape: (3, 2) Epoch 923: b2_train shape: (1, 2) Epoch 924: X_data_scaled shape: (100, 2) Epoch 924: W1_train shape: (2, 3) Epoch 924: b1_train shape: (1, 3) Epoch 924: W2_train shape: (3, 2) Epoch 924: b2_train shape: (1, 2) Epoch 925: X_data_scaled shape: (100, 2) Epoch 925: W1_train shape: (2, 3) Epoch 925: b1_train shape: (1, 3) Epoch 925: W2_train shape: (3, 2) Epoch 925: b2_train shape: (1, 2) Epoch 926: X_data_scaled shape: (100, 2) Epoch 926: W1_train shape: (2, 3) Epoch 926: b1_train shape: (1, 3) Epoch 926: W2_train shape: (3, 2) Epoch 926: b2_train shape: (1, 2) Epoch 927: X_data_scaled shape: (100, 2) Epoch 927: W1_train shape: (2, 3) Epoch 927: b1_train shape: (1, 3) Epoch 927: W2_train shape: (3, 2) Epoch 927: b2_train shape: (1, 2) Epoch 928: X_data_scaled shape: (100, 2) Epoch 928: W1_train shape: (2, 3) Epoch 928: b1_train shape: (1, 3) Epoch 928: W2_train shape: (3, 2) Epoch 928: b2_train shape: (1, 2) Epoch 929: X_data_scaled shape: (100, 2) Epoch 929: W1_train shape: (2, 3) Epoch 929: b1_train shape: (1, 3) Epoch 929: W2_train shape: (3, 2) Epoch 929: b2_train shape: (1, 2) Epoch 930: X_data_scaled shape: (100, 2) Epoch 930: W1_train shape: (2, 3) Epoch 930: b1_train shape: (1, 3) Epoch 930: W2_train shape: (3, 2) Epoch 930: b2_train shape: (1, 2) Epoch 931: X_data_scaled shape: (100, 2) Epoch 931: W1_train shape: (2, 3) Epoch 931: b1_train shape: (1, 3) Epoch 931: W2_train shape: (3, 2) Epoch 931: b2_train shape: (1, 2) Epoch 932: X_data_scaled shape: (100, 2) Epoch 932: W1_train shape: (2, 3) Epoch 932: b1_train shape: (1, 3) Epoch 932: W2_train shape: (3, 2) Epoch 932: b2_train shape: (1, 2) Epoch 933: X_data_scaled shape: (100, 2) Epoch 933: W1_train shape: (2, 3) Epoch 933: b1_train shape: (1, 3) Epoch 933: W2_train shape: (3, 2) Epoch 933: b2_train shape: (1, 2) Epoch 934: X_data_scaled shape: (100, 2) Epoch 934: W1_train shape: (2, 3) Epoch 934: b1_train shape: (1, 3) Epoch 934: W2_train shape: (3, 2) Epoch 934: b2_train shape: (1, 2) Epoch 935: X_data_scaled shape: (100, 2) Epoch 935: W1_train shape: (2, 3) Epoch 935: b1_train shape: (1, 3) Epoch 935: W2_train shape: (3, 2) Epoch 935: b2_train shape: (1, 2) Epoch 936: X_data_scaled shape: (100, 2) Epoch 936: W1_train shape: (2, 3) Epoch 936: b1_train shape: (1, 3) Epoch 936: W2_train shape: (3, 2) Epoch 936: b2_train shape: (1, 2) Epoch 937: X_data_scaled shape: (100, 2) Epoch 937: W1_train shape: (2, 3) Epoch 937: b1_train shape: (1, 3) Epoch 937: W2_train shape: (3, 2) Epoch 937: b2_train shape: (1, 2) Epoch 938: X_data_scaled shape: (100, 2) Epoch 938: W1_train shape: (2, 3) Epoch 938: b1_train shape: (1, 3) Epoch 938: W2_train shape: (3, 2) Epoch 938: b2_train shape: (1, 2) Epoch 939: X_data_scaled shape: (100, 2) Epoch 939: W1_train shape: (2, 3) Epoch 939: b1_train shape: (1, 3) Epoch 939: W2_train shape: (3, 2) Epoch 939: b2_train shape: (1, 2) Epoch 940: X_data_scaled shape: (100, 2) Epoch 940: W1_train shape: (2, 3) Epoch 940: b1_train shape: (1, 3) Epoch 940: W2_train shape: (3, 2) Epoch 940: b2_train shape: (1, 2) Epoch 941: X_data_scaled shape: (100, 2) Epoch 941: W1_train shape: (2, 3) Epoch 941: b1_train shape: (1, 3) Epoch 941: W2_train shape: (3, 2) Epoch 941: b2_train shape: (1, 2) Epoch 942: X_data_scaled shape: (100, 2) Epoch 942: W1_train shape: (2, 3) Epoch 942: b1_train shape: (1, 3) Epoch 942: W2_train shape: (3, 2) Epoch 942: b2_train shape: (1, 2) Epoch 943: X_data_scaled shape: (100, 2) Epoch 943: W1_train shape: (2, 3) Epoch 943: b1_train shape: (1, 3) Epoch 943: W2_train shape: (3, 2) Epoch 943: b2_train shape: (1, 2) Epoch 944: X_data_scaled shape: (100, 2) Epoch 944: W1_train shape: (2, 3) Epoch 944: b1_train shape: (1, 3) Epoch 944: W2_train shape: (3, 2) Epoch 944: b2_train shape: (1, 2) Epoch 945: X_data_scaled shape: (100, 2) Epoch 945: W1_train shape: (2, 3) Epoch 945: b1_train shape: (1, 3) Epoch 945: W2_train shape: (3, 2) Epoch 945: b2_train shape: (1, 2) Epoch 946: X_data_scaled shape: (100, 2) Epoch 946: W1_train shape: (2, 3) Epoch 946: b1_train shape: (1, 3) Epoch 946: W2_train shape: (3, 2) Epoch 946: b2_train shape: (1, 2) Epoch 947: X_data_scaled shape: (100, 2) Epoch 947: W1_train shape: (2, 3) Epoch 947: b1_train shape: (1, 3) Epoch 947: W2_train shape: (3, 2) Epoch 947: b2_train shape: (1, 2) Epoch 948: X_data_scaled shape: (100, 2) Epoch 948: W1_train shape: (2, 3) Epoch 948: b1_train shape: (1, 3) Epoch 948: W2_train shape: (3, 2) Epoch 948: b2_train shape: (1, 2) Epoch 949: X_data_scaled shape: (100, 2) Epoch 949: W1_train shape: (2, 3) Epoch 949: b1_train shape: (1, 3) Epoch 949: W2_train shape: (3, 2) Epoch 949: b2_train shape: (1, 2) Epoch 950: X_data_scaled shape: (100, 2) Epoch 950: W1_train shape: (2, 3) Epoch 950: b1_train shape: (1, 3) Epoch 950: W2_train shape: (3, 2) Epoch 950: b2_train shape: (1, 2) Epoch 951: X_data_scaled shape: (100, 2) Epoch 951: W1_train shape: (2, 3) Epoch 951: b1_train shape: (1, 3) Epoch 951: W2_train shape: (3, 2) Epoch 951: b2_train shape: (1, 2) Epoch 952: X_data_scaled shape: (100, 2) Epoch 952: W1_train shape: (2, 3) Epoch 952: b1_train shape: (1, 3) Epoch 952: W2_train shape: (3, 2) Epoch 952: b2_train shape: (1, 2) Epoch 953: X_data_scaled shape: (100, 2) Epoch 953: W1_train shape: (2, 3) Epoch 953: b1_train shape: (1, 3) Epoch 953: W2_train shape: (3, 2) Epoch 953: b2_train shape: (1, 2) Epoch 954: X_data_scaled shape: (100, 2) Epoch 954: W1_train shape: (2, 3) Epoch 954: b1_train shape: (1, 3) Epoch 954: W2_train shape: (3, 2) Epoch 954: b2_train shape: (1, 2) Epoch 955: X_data_scaled shape: (100, 2) Epoch 955: W1_train shape: (2, 3) Epoch 955: b1_train shape: (1, 3) Epoch 955: W2_train shape: (3, 2) Epoch 955: b2_train shape: (1, 2) Epoch 956: X_data_scaled shape: (100, 2) Epoch 956: W1_train shape: (2, 3) Epoch 956: b1_train shape: (1, 3) Epoch 956: W2_train shape: (3, 2) Epoch 956: b2_train shape: (1, 2) Epoch 957: X_data_scaled shape: (100, 2) Epoch 957: W1_train shape: (2, 3) Epoch 957: b1_train shape: (1, 3) Epoch 957: W2_train shape: (3, 2) Epoch 957: b2_train shape: (1, 2) Epoch 958: X_data_scaled shape: (100, 2) Epoch 958: W1_train shape: (2, 3) Epoch 958: b1_train shape: (1, 3) Epoch 958: W2_train shape: (3, 2) Epoch 958: b2_train shape: (1, 2) Epoch 959: X_data_scaled shape: (100, 2) Epoch 959: W1_train shape: (2, 3) Epoch 959: b1_train shape: (1, 3) Epoch 959: W2_train shape: (3, 2) Epoch 959: b2_train shape: (1, 2) Epoch 960: X_data_scaled shape: (100, 2) Epoch 960: W1_train shape: (2, 3) Epoch 960: b1_train shape: (1, 3) Epoch 960: W2_train shape: (3, 2) Epoch 960: b2_train shape: (1, 2) Epoch 961: X_data_scaled shape: (100, 2) Epoch 961: W1_train shape: (2, 3) Epoch 961: b1_train shape: (1, 3) Epoch 961: W2_train shape: (3, 2) Epoch 961: b2_train shape: (1, 2) Epoch 962: X_data_scaled shape: (100, 2) Epoch 962: W1_train shape: (2, 3) Epoch 962: b1_train shape: (1, 3) Epoch 962: W2_train shape: (3, 2) Epoch 962: b2_train shape: (1, 2) Epoch 963: X_data_scaled shape: (100, 2) Epoch 963: W1_train shape: (2, 3) Epoch 963: b1_train shape: (1, 3) Epoch 963: W2_train shape: (3, 2) Epoch 963: b2_train shape: (1, 2) Epoch 964: X_data_scaled shape: (100, 2) Epoch 964: W1_train shape: (2, 3) Epoch 964: b1_train shape: (1, 3) Epoch 964: W2_train shape: (3, 2) Epoch 964: b2_train shape: (1, 2) Epoch 965: X_data_scaled shape: (100, 2) Epoch 965: W1_train shape: (2, 3) Epoch 965: b1_train shape: (1, 3) Epoch 965: W2_train shape: (3, 2) Epoch 965: b2_train shape: (1, 2) Epoch 966: X_data_scaled shape: (100, 2) Epoch 966: W1_train shape: (2, 3) Epoch 966: b1_train shape: (1, 3) Epoch 966: W2_train shape: (3, 2) Epoch 966: b2_train shape: (1, 2) Epoch 967: X_data_scaled shape: (100, 2) Epoch 967: W1_train shape: (2, 3) Epoch 967: b1_train shape: (1, 3) Epoch 967: W2_train shape: (3, 2) Epoch 967: b2_train shape: (1, 2) Epoch 968: X_data_scaled shape: (100, 2) Epoch 968: W1_train shape: (2, 3) Epoch 968: b1_train shape: (1, 3) Epoch 968: W2_train shape: (3, 2) Epoch 968: b2_train shape: (1, 2) Epoch 969: X_data_scaled shape: (100, 2) Epoch 969: W1_train shape: (2, 3) Epoch 969: b1_train shape: (1, 3) Epoch 969: W2_train shape: (3, 2) Epoch 969: b2_train shape: (1, 2) Epoch 970: X_data_scaled shape: (100, 2) Epoch 970: W1_train shape: (2, 3) Epoch 970: b1_train shape: (1, 3) Epoch 970: W2_train shape: (3, 2) Epoch 970: b2_train shape: (1, 2) Epoch 971: X_data_scaled shape: (100, 2) Epoch 971: W1_train shape: (2, 3) Epoch 971: b1_train shape: (1, 3) Epoch 971: W2_train shape: (3, 2) Epoch 971: b2_train shape: (1, 2) Epoch 972: X_data_scaled shape: (100, 2) Epoch 972: W1_train shape: (2, 3) Epoch 972: b1_train shape: (1, 3) Epoch 972: W2_train shape: (3, 2) Epoch 972: b2_train shape: (1, 2) Epoch 973: X_data_scaled shape: (100, 2) Epoch 973: W1_train shape: (2, 3) Epoch 973: b1_train shape: (1, 3) Epoch 973: W2_train shape: (3, 2) Epoch 973: b2_train shape: (1, 2) Epoch 974: X_data_scaled shape: (100, 2) Epoch 974: W1_train shape: (2, 3) Epoch 974: b1_train shape: (1, 3) Epoch 974: W2_train shape: (3, 2) Epoch 974: b2_train shape: (1, 2) Epoch 975: X_data_scaled shape: (100, 2) Epoch 975: W1_train shape: (2, 3) Epoch 975: b1_train shape: (1, 3) Epoch 975: W2_train shape: (3, 2) Epoch 975: b2_train shape: (1, 2) Epoch 976: X_data_scaled shape: (100, 2) Epoch 976: W1_train shape: (2, 3) Epoch 976: b1_train shape: (1, 3) Epoch 976: W2_train shape: (3, 2) Epoch 976: b2_train shape: (1, 2) Epoch 977: X_data_scaled shape: (100, 2) Epoch 977: W1_train shape: (2, 3) Epoch 977: b1_train shape: (1, 3) Epoch 977: W2_train shape: (3, 2) Epoch 977: b2_train shape: (1, 2) Epoch 978: X_data_scaled shape: (100, 2) Epoch 978: W1_train shape: (2, 3) Epoch 978: b1_train shape: (1, 3) Epoch 978: W2_train shape: (3, 2) Epoch 978: b2_train shape: (1, 2) Epoch 979: X_data_scaled shape: (100, 2) Epoch 979: W1_train shape: (2, 3) Epoch 979: b1_train shape: (1, 3) Epoch 979: W2_train shape: (3, 2) Epoch 979: b2_train shape: (1, 2) Epoch 980: X_data_scaled shape: (100, 2) Epoch 980: W1_train shape: (2, 3) Epoch 980: b1_train shape: (1, 3) Epoch 980: W2_train shape: (3, 2) Epoch 980: b2_train shape: (1, 2) Epoch 981: X_data_scaled shape: (100, 2) Epoch 981: W1_train shape: (2, 3) Epoch 981: b1_train shape: (1, 3) Epoch 981: W2_train shape: (3, 2) Epoch 981: b2_train shape: (1, 2) Epoch 982: X_data_scaled shape: (100, 2) Epoch 982: W1_train shape: (2, 3) Epoch 982: b1_train shape: (1, 3) Epoch 982: W2_train shape: (3, 2) Epoch 982: b2_train shape: (1, 2) Epoch 983: X_data_scaled shape: (100, 2) Epoch 983: W1_train shape: (2, 3) Epoch 983: b1_train shape: (1, 3) Epoch 983: W2_train shape: (3, 2) Epoch 983: b2_train shape: (1, 2) Epoch 984: X_data_scaled shape: (100, 2) Epoch 984: W1_train shape: (2, 3) Epoch 984: b1_train shape: (1, 3) Epoch 984: W2_train shape: (3, 2) Epoch 984: b2_train shape: (1, 2) Epoch 985: X_data_scaled shape: (100, 2) Epoch 985: W1_train shape: (2, 3) Epoch 985: b1_train shape: (1, 3) Epoch 985: W2_train shape: (3, 2) Epoch 985: b2_train shape: (1, 2) Epoch 986: X_data_scaled shape: (100, 2) Epoch 986: W1_train shape: (2, 3) Epoch 986: b1_train shape: (1, 3) Epoch 986: W2_train shape: (3, 2) Epoch 986: b2_train shape: (1, 2) Epoch 987: X_data_scaled shape: (100, 2) Epoch 987: W1_train shape: (2, 3) Epoch 987: b1_train shape: (1, 3) Epoch 987: W2_train shape: (3, 2) Epoch 987: b2_train shape: (1, 2) Epoch 988: X_data_scaled shape: (100, 2) Epoch 988: W1_train shape: (2, 3) Epoch 988: b1_train shape: (1, 3) Epoch 988: W2_train shape: (3, 2) Epoch 988: b2_train shape: (1, 2) Epoch 989: X_data_scaled shape: (100, 2) Epoch 989: W1_train shape: (2, 3) Epoch 989: b1_train shape: (1, 3) Epoch 989: W2_train shape: (3, 2) Epoch 989: b2_train shape: (1, 2) Epoch 990: X_data_scaled shape: (100, 2) Epoch 990: W1_train shape: (2, 3) Epoch 990: b1_train shape: (1, 3) Epoch 990: W2_train shape: (3, 2) Epoch 990: b2_train shape: (1, 2) Epoch 991: X_data_scaled shape: (100, 2) Epoch 991: W1_train shape: (2, 3) Epoch 991: b1_train shape: (1, 3) Epoch 991: W2_train shape: (3, 2) Epoch 991: b2_train shape: (1, 2) Epoch 992: X_data_scaled shape: (100, 2) Epoch 992: W1_train shape: (2, 3) Epoch 992: b1_train shape: (1, 3) Epoch 992: W2_train shape: (3, 2) Epoch 992: b2_train shape: (1, 2) Epoch 993: X_data_scaled shape: (100, 2) Epoch 993: W1_train shape: (2, 3) Epoch 993: b1_train shape: (1, 3) Epoch 993: W2_train shape: (3, 2) Epoch 993: b2_train shape: (1, 2) Epoch 994: X_data_scaled shape: (100, 2) Epoch 994: W1_train shape: (2, 3) Epoch 994: b1_train shape: (1, 3) Epoch 994: W2_train shape: (3, 2) Epoch 994: b2_train shape: (1, 2) Epoch 995: X_data_scaled shape: (100, 2) Epoch 995: W1_train shape: (2, 3) Epoch 995: b1_train shape: (1, 3) Epoch 995: W2_train shape: (3, 2) Epoch 995: b2_train shape: (1, 2) Epoch 996: X_data_scaled shape: (100, 2) Epoch 996: W1_train shape: (2, 3) Epoch 996: b1_train shape: (1, 3) Epoch 996: W2_train shape: (3, 2) Epoch 996: b2_train shape: (1, 2) Epoch 997: X_data_scaled shape: (100, 2) Epoch 997: W1_train shape: (2, 3) Epoch 997: b1_train shape: (1, 3) Epoch 997: W2_train shape: (3, 2) Epoch 997: b2_train shape: (1, 2) Epoch 998: X_data_scaled shape: (100, 2) Epoch 998: W1_train shape: (2, 3) Epoch 998: b1_train shape: (1, 3) Epoch 998: W2_train shape: (3, 2) Epoch 998: b2_train shape: (1, 2) Epoch 999: X_data_scaled shape: (100, 2) Epoch 999: W1_train shape: (2, 3) Epoch 999: b1_train shape: (1, 3) Epoch 999: W2_train shape: (3, 2) Epoch 999: b2_train shape: (1, 2) Training complete. Final Loss: 5.2983
In [ ]:
# --- Evaluate the trained model (custom) ---
final_pred_proba, _ = forward_pass(X_data_scaled, W1_train, b1_train, W2_train, b2_train)
final_pred_labels = np.argmax(final_pred_proba, axis=1)
accuracy = accuracy_score(y_data, final_pred_labels)
print(f"\nFinal Accuracy of Custom Model: {accuracy:.4f}")
# Plot decision boundary (optional, for 2D data)
plt.figure(figsize=(10, 7))
h = .02 # step size in the mesh
x_min, x_max = X_data_scaled[:, 0].min() - 1, X_data_scaled[:, 0].max() + 1
y_min, y_max = X_data_scaled[:, 1].min() - 1, X_data_scaled[:, 1].max() + 1
xx, yy = np.meshgrid(np.arange(x_min, x_max, h),
np.arange(y_min, y_max, h))
Z = np.argmax(forward_pass(np.c_[xx.ravel(), yy.ravel()], W1_train, b1_train, W2_train, b2_train)[0], axis=1)
Z = Z.reshape(xx.shape)
plt.contourf(xx, yy, Z, cmap=plt.cm.RdBu, alpha=0.8)
plt.scatter(X_data_scaled[:, 0], X_data_scaled[:, 1], c=y_data, cmap=plt.cm.RdBu, edgecolor='k', s=80)
plt.title('Custom Neural Network Decision Boundary')
plt.xlabel('Feature 1 (Scaled)')
plt.ylabel('Feature 2 (Scaled)')
plt.show()
Final Accuracy of Custom Model: 0.5000
Discussion Points:¶
- Explain the role of the chain rule in the
backward_passfunction, especially when calculatingdZ1. - What does the loss plot tell you about the training process? If the loss wasn't decreasing, what might be wrong?
- How does the
learning_rateimpact the training process? What happens if it's too high or too low?
Part 3: Backpropagation in Practice (Using TensorFlow/Keras)¶
While implementing backpropagation from scratch provides deep understanding, in practice, deep learning frameworks like TensorFlow/Keras automate this complex process. They use highly optimized C++/CUDA implementations of backpropagation, often with automatic differentiation.
Tasks:
- Build the same network architecture (2 input -> 3 hidden -> 2 output) using Keras's Sequential API.
- Train the Keras model on the same
X_data_scaledandy_data_oh. - Compare the performance and ease of implementation to the "from scratch" version.
In [ ]:
# Define the Keras model
keras_model = Sequential([
Dense(units=3, activation='sigmoid', input_shape=(input_size,)), # Hidden layer
Dense(units=output_size, activation='softmax') # Output layer
])
# Compile the model
keras_model.compile(optimizer='adam',
loss='categorical_crossentropy',
metrics=['accuracy'])
keras_model.summary()
Model: "sequential"
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓ ┃ Layer (type) ┃ Output Shape ┃ Param # ┃ ┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩ │ dense (Dense) │ (None, 3) │ 9 │ ├─────────────────────────────────┼────────────────────────┼───────────────┤ │ dense_1 (Dense) │ (None, 2) │ 8 │ └─────────────────────────────────┴────────────────────────┴───────────────┘
Total params: 17 (68.00 B)
Trainable params: 17 (68.00 B)
Non-trainable params: 0 (0.00 B)
In [ ]:
# Train the Keras model
print("\nStarting training with Keras (using built-in backpropagation)...")
history_keras = keras_model.fit(X_data_scaled, y_data_oh,
epochs=1000,
batch_size=32, # Keras often uses larger batch sizes by default
verbose=0) # Set verbose to 0 to suppress per-epoch output
print("\nKeras training complete.")
print(f"Final Keras Loss: {history_keras.history['loss'][-1]:.4f}")
Starting training with Keras (using built-in backpropagation)... Keras training complete. Final Keras Loss: 0.0132
In [ ]:
# Evaluate the Keras model
loss_keras, accuracy_keras = keras_model.evaluate(X_data_scaled, y_data_oh, verbose=0)
print(f"Final Keras Model Accuracy: {accuracy_keras:.4f}")
# Plot Keras training loss
plt.figure(figsize=(10, 6))
plt.plot(history_keras.history['loss'])
plt.title('Keras Model Loss Over Epochs')
plt.xlabel('Epoch')
plt.ylabel('Loss')
plt.grid(True, linestyle='--', alpha=0.6)
plt.show()
# Plot Keras decision boundary
plt.figure(figsize=(10, 7))
h = .02
x_min, x_max = X_data_scaled[:, 0].min() - 1, X_data_scaled[:, 0].max() + 1
y_min, y_max = X_data_scaled[:, 1].min() - 1, X_data_scaled[:, 1].max() + 1
xx, yy = np.meshgrid(np.arange(x_min, x_max, h),
np.arange(y_min, y_max, h))
Z_keras = np.argmax(keras_model.predict(np.c_[xx.ravel(), yy.ravel()]), axis=1)
Z_keras = Z_keras.reshape(xx.shape)
plt.contourf(xx, yy, Z_keras, cmap=plt.cm.RdBu, alpha=0.8)
plt.scatter(X_data_scaled[:, 0], X_data_scaled[:, 1], c=y_data, cmap=plt.cm.RdBu, edgecolor='k', s=80)
plt.title('Keras Neural Network Decision Boundary')
plt.xlabel('Feature 1 (Scaled)')
plt.ylabel('Feature 2 (Scaled)')
plt.show()
Final Keras Model Accuracy: 1.0000
3394/3394 ━━━━━━━━━━━━━━━━━━━━ 6s 2ms/step
Discussion Points:¶
- Compare the final loss and accuracy of your custom NumPy implementation with the Keras implementation. What factors might account for any differences?
- How does Keras simplify the process of implementing and training a neural network compared to doing it from scratch? What specific aspects does it abstract away?
Part 4: Advanced Topics & Discussion:¶
This section is for broader discussion and conceptual understanding, extending beyond direct coding.
Discussion Topics:
- Vanishing/Exploding Gradients: What are these problems, why do they occur in deep networks, and what techniques are used to mitigate them?
- Computational Graph: Explain the concept of a computational graph in the context of backpropagation.
- Automatic Differentiation: How do modern deep learning frameworks (like TensorFlow/PyTorch) handle differentiation, making backpropagation seamless for users?
- Beyond Gradient Descent: Briefly discuss how more advanced optimizers (e.g., Adam, RMSprop) improve upon basic Gradient Descent in the context of backpropagation.
- Limitations of Backpropagation: Are there scenarios where backpropagation might not be suitable or faces inherent challenges?
Vanishing/Exploding Gradients:
- Vanishing Gradients: This occurs in deep neural networks when the gradients become extremely small as they propagate backward through many layers (especially with activation functions like Sigmoid or Tanh, whose derivatives are small). As gradients shrink, the updates to the weights and biases in earlier layers become negligible, making those layers learn very slowly or stop learning altogether.
- Exploding Gradients: Conversely, exploding gradients happen when gradients become excessively large during backpropagation. This leads to very large weight updates, causing the training process to become unstable, oscillating wildly or even diverging.
- Why they occur: Primarily due to repeated multiplication of gradients through many layers. If the values of the derivatives are consistently less than 1 (vanishing) or consistently greater than 1 (exploding), the product accumulates to a very small or very large number, respectively.
- Mitigation Techniques:
- Vanishing:
- ReLU-family Activation Functions: ReLU, Leaky ReLU, ELU, Swish have derivatives that are either 0 or 1 (or a small constant), preventing the product from shrinking too rapidly.
- Proper Weight Initialization: Techniques like He initialization (for ReLU) or Xavier/Glorot initialization (for Sigmoid/Tanh) ensure that initial weights are in a range that prevents activations from being pushed to the extremes.
- Batch Normalization: Normalizes the inputs of each layer, preventing small changes in previous layers from amplifying into large changes in subsequent layers, thus stabilizing gradients.
- Residual Connections (ResNets): Allow gradients to bypass layers directly, helping them flow more easily through very deep networks.
- Exploding:
- Gradient Clipping: If the gradient norm exceeds a certain threshold, it is rescaled back down to that threshold. This prevents individual large gradients from disrupting the learning process.
- Smaller Learning Rates: A straightforward approach, though it can slow down convergence.
- Vanishing:
Computational Graph:
- A computational graph (or dataflow graph) is a way to represent mathematical operations as a graph. In the context of neural networks and backpropagation, each node in the graph represents an operation (e.g., matrix multiplication, addition, activation function), and the edges represent the flow of data (tensors).
- How it relates to Backpropagation: During the forward pass, data flows forward through this graph, computing intermediate results and the final output. During backpropagation, the gradients are calculated by traversing this graph in reverse (backward pass), applying the chain rule at each node to compute the partial derivative of the loss with respect to that node's inputs and parameters. Modern deep learning frameworks construct and manage this graph automatically.
Automatic Differentiation:
- Automatic differentiation (AutoDiff) is a set of techniques used by deep learning frameworks (like TensorFlow's
tf.GradientTapeor PyTorch's autograd) to efficiently and accurately compute derivatives of complex functions. It's not symbolic differentiation (which computes formulas) or numerical differentiation (which approximates derivatives using finite differences). - How it works: AutoDiff leverages the chain rule. It records the sequence of operations performed during the forward pass. Then, to compute gradients, it traverses this recorded "tape" or computational graph backward, applying the chain rule step by step to propagate gradients. This automation is what makes it feasible for users to build and train incredibly complex neural networks without manually deriving gradients for every single weight and bias.
- Automatic differentiation (AutoDiff) is a set of techniques used by deep learning frameworks (like TensorFlow's
Beyond Gradient Descent:
- Basic Gradient Descent (GD) updates weights using the gradient of the entire dataset. Stochastic Gradient Descent (SGD) uses one sample at a time, and Mini-Batch Gradient Descent uses a small batch. While fundamental, these can be slow or get stuck in local minima.
- Advanced Optimizers (e.g., Adam, RMSprop, Adagrad): These optimizers build upon GD by incorporating concepts like momentum (which helps accelerate GD in the relevant direction and dampens oscillations) and adaptive learning rates (which adjust the learning rate for each parameter individually based on past gradients).
- How they improve: They help in:
- Faster Convergence: By taking larger steps in flatter regions and smaller steps near minima.
- Better Generalization: Often find better minima that generalize well to unseen data.
- Handling Sparsity: Can perform better with sparse data by adapting learning rates.
- Robustness: Less sensitive to the initial learning rate choice compared to vanilla GD.
- Example (Adam): Adam combines the advantages of RMSprop (adaptive learning rates based on squared gradients) and momentum (adaptive learning rates based on past gradients). It's often the default choice in many deep learning applications due to its efficiency and good performance.
Limitations of Backpropagation:
- Vanishing/Exploding Gradients: As discussed, this is a major challenge for very deep networks, though techniques exist to mitigate it.
- Local Minima and Saddle Points: Gradient-based optimization can get stuck in local minima or saddle points (where the gradient is zero, but it's not a true minimum), preventing the model from reaching the global optimum. Advanced optimizers and good initialization help but don't entirely eliminate this.
- Computational Cost: For very large models and datasets, backpropagation (and the overall training process) is computationally intensive, requiring significant hardware resources (GPUs/TPUs).
- Requires Differentiable Functions: Backpropagation relies on the differentiability of activation functions and loss functions. Non-differentiable operations cannot be directly used.
- Black Box Nature: While backpropagation allows training, it doesn't inherently provide interpretability into why the network makes certain decisions. The weights and biases themselves don't have direct human-understandable meaning.
- Data Hunger: Deep learning models trained with backpropagation often require massive amounts of labeled data to perform well, which can be expensive or impossible to acquire for certain domains.
Prepared By
Md. Atikuzzaman
Lecturer
Department of Computer Science and Engineering
Green University of Bangladesh
Email: atik@cse.green.edu.bd