L0 · From fit to held-out validation
Fitting minimizes an error on the observations given to the estimator. That error alone is not enough: a flexible or badly structured model can match one trajectory and fail elsewhere.
L0 therefore has two explicit splits:
| Split | Input | Purpose |
|---|---|---|
| Fit | chirp | estimate J and b |
| Validation | held-out multisine | test prediction on a different motion |
What the inputs mean
The chirp is one sinusoid whose frequency changes continuously from low to high over the fitting window. It exposes both slow motion and fast changes, so the estimator sees the acceleration-sensitive and velocity-sensitive parts of the equation.
The held-out multisine is a separate signal made by adding several fixed frequency sinusoids. This motion is reserved in advance and never used to estimate or adjust the parameters. Good validation performance means the model predicts this different combination of frequencies.
The report gives mean absolute error (MAE) for both position q and velocity
qd. Compare the Initial-model rows with the Identified-model rows. The important
pattern is not that every number is zero; it is that the identified model
improves on the untouched validation motion as well as on the fit motion.
The residual panels show identified - observed. A residual close to zero and
without an obvious time pattern is consistent with this matched, idealized
example. It does not establish that the parameters are identifiable under all
inputs or that the same model will work on hardware.
Questions to answer
- Why would reporting only fit MAE make the experiment weaker?
- What evidence in the validation columns supports the identified model?
- If fit error is low but validation error is high, would you first blame the optimizer or the model structure? What extra experiment would distinguish those explanations?
Suggested answers: fit-only scoring cannot expose overfitting or a model that only matches one excitation. Here the identified validation errors are much smaller than the nominal errors. A high validation error needs diagnosis: repeat fitting from another start to test optimization, then inspect residuals and add a missing effect only when the evidence supports it.