Time-Series Prediction

A time series is a sequence where order carries the signal, and the honest baseline is "predict the last value". Beating that baseline is the task. The standard framing turns the series into a supervised learning problem with a sliding window.

The idea

• Windowing — from x₁…xₙ, build inputs (x_{t-w}…x_{t-1}) and target x_t. A w-wide window converts forecasting into regression.
• Single-step versus multi-step — predicting the next value is easy to frame; predicting h steps ahead needs either h outputs or autoregressive roll-out, which accumulates error.
• Stationarity — level and variance should be stable. Differencing x_t − x_{t−1} removes trend; a log transform stabilizes growing variance. Predict the difference, then invert it.
• Baselines — naive (last value) and seasonal naive (value one period ago). Report MAE, not just RMSE, and always place the baseline next to the model.
• Splits — never shuffle. Split chronologically and fit scalers on the training portion only.

Worked example

Hourly temperature, w = 24 hours in, 1 step out.

• Train/validation split at 80% by time.
• Scaler fit on the first 80% only, then applied to both.
• A 24 → 1 window produces n − 24 examples.
• Baseline MAE is the mean absolute error of "tomorrow equals today".
• Model: Dense(64, relu) then Dense(1), or an LSTM(32) when patterns run longer.
• Early stopping on val_loss with patience=5 halts before the model memorizes the training tail.

An LSTM(32) beats the naive baseline on smooth series; on noisy series it often does not, and that is a legitimate result to report.

In code

import numpy as np, tensorflow as tf

series = (np.arange(2000, dtype="float32") % 24
          + np.random.randn(2000).astype("float32"))

w, split = 24, int(2000 * 0.8)
windows = np.stack([series[i:i + w] for i in range(len(series) - w)])
targets = series[w:]                       # value right after each window

x_train, x_val = windows[:split], windows[split:]
y_train, y_val = targets[:split], targets[split:]

model = tf.keras.Sequential([
    tf.keras.layers.Dense(64, activation="relu", input_shape=(w,)),
    tf.keras.layers.Dense(1),
])
model.compile(optimizer="adam", loss="mae")
model.fit(x_train, y_train, validation_data=(x_val, y_val), epochs=50,
          callbacks=[tf.keras.callbacks.EarlyStopping(
              patience=5, restore_best_weights=True)])

Check yourself

1. Why must the scaler be fit on the training period only?
2. Why is MAE usually more interpretable than RMSE for a forecasting error?
3. What does a model with lower validation MAE than the naive baseline actually prove?

Key takeaways

• A sliding window turns forecasting into plain regression with tabular inputs.
• Chronological splits and train-only scalers prevent look-ahead leakage.
• A forecast is only meaningful relative to a naive baseline on the same split.