Post-hoc explanation vs structural model control: what actually changes?

Deep learning’s power comes partly from the complexity that makes its internal decisions difficult to inspect.

Traditional explainability approaches usually start after the predictive model has already been trained. They ask:

“How can we explain what this black box just did?”

That is a legitimate and useful question.

Xpdeep starts from a different one:

“What changes if the learned structure itself can be read?”

The distinction matters because explanation can serve two very different purposes:

1. documenting or inspecting a prediction after it happened;
2. making the model understandable enough to design, operate and evolve differently.

This article explains where post-hoc explanation remains useful, where structural model control differs, and why the difference matters in consequential machine decisions.

Methods such as SHAP, LIME, integrated gradients, Grad-CAM and saliency techniques analyse a predictive model after it has already been trained.

They can be valuable for:

• inspection
• debugging
• model testing
• documentation
• communicating why a prediction may have occurred

But the predictive model itself remains the object being analysed.

The explanation is produced by a separate analytical procedure, gradient-based, perturbation-based or surrogate, depending on the method, around or after the model inference.

Different post-hoc methods have different capabilities. This article compares architectural roles, not every method individually.

Xpdeep operates at model level.

Its objective is not only to explain a black box from the outside.

It reads learned structure from the model itself and uses that structure as part of the model lifecycle.

That distinction appears differently in Native and Retrofit mode.

Native: readability is built into the model.

In Native mode, prediction and structural explanation are learned jointly. Readability does not cost accuracy: on time-series data, accuracy is at least equivalent to standard deep learning and frequently superior.

The model is trained so that the learned structure itself can be read and edited.

This enables:

• structural explanation
• design-time diagnosis
• input reduction
• KPI alignment
• targeted local editing
• controlled model evolution

Retrofit: open an existing model structurally.

In Retrofit mode, the customer’s existing PyTorch predictive model remains frozen.

Its architecture, weights and predictions remain unchanged.

Structural analysis requires read access to the PyTorch model’s architecture and weights. Input/output-only analysis is not offered.

Xpdeep learns explanations from the model’s structure and integrates them into one artefact that produces prediction and structural explanation together.

Retrofit is structural.

It is not ante-hoc, because the predictive model was trained before and without the explanation objective.

It is also not conventional post-hoc explanation, because the explanation is learned from the model structure and becomes part of the inference artefact.

Targeted structural editing and controlled evolution require Native mode.

Post-hoc explanations can support diagnosis and debugging. They do not make the predictive structure itself editable.

There is no separate post-inference explanation procedure in the Xpdeep inference path.

Prediction and structural explanation are produced together in the same forward pass.

The model can also return an attributed domain state: a graded distance to the trained domain, with the responsible variables and temporal windows identified.

On temporal and tabular stages, prescription is a separate inference, run only when an intervention is useful.

It computes a minimal feasible input change under stated constraints and returns the structural justification of why that intervention changes the predicted outcome.

Adjacent monitoring or optimization tools can add other functions around a model. The Xpdeep distinction is that these primitives are integrated at model level.

The distinction is not simply “more explanation”.

Xpdeep changes the model output from a prediction into a richer decision object.

This is what allows the model to participate in a bounded operational decision rather than merely produce a score for a human to interpret.

What is distinctive is the integration: readable structure, inference-time explanation and domain state in one model, and, in Native mode, structure that engineers can edit.

On temporal data, many conventional explanation views focus on identifying which input variables matter.

Some post-hoc methods also provide temporal attribution and support diagnosis and engineering work. The Xpdeep difference is where the localization comes from: the model’s own structure, at inference time, together with its domain state.

Xpdeep can localize the model’s learned decision on the signal itself. It can identify which variable, which time window, which signal morphology and which learned dependency carried the model output.

The model localizes the learned driver. The engineer identifies the physical cause.

Saliency and heatmap methods estimate how spatial regions of the input influenced a model’s output.

Xpdeep asks what learned structure produced the output and whether the image remains inside the model’s learned domain.

Image-level prescription: not available; under study.

These are the three loops Xpdeep closes at model level.

Explore the Three Loops on Technology →

If the objective is primarily:

• audit documentation
• retrospective inspection
• occasional local explanation
• validation of a model that must remain unchanged

a post-hoc method may be entirely appropriate.

Xpdeep is relevant when the problem extends beyond explanation:

• the model must be designed differently
• explanation must travel with every inference
• the model must expose whether it remains inside known conditions
• a bounded intervention may need to be computed
• the model may need targeted structural correction over time

In design, readable structure lets engineers diagnose and, in Native mode, edit the model rather than retrain by trial and error. In operation, it supplies the explanation and domain state a bounded decision needs, with execution still governed by the customer’s system rules. In controlled evolution, it supports targeted, validated correction between operating periods in Native mode.

The more consequential the decision, the more valuable it becomes to understand the model from the inside.