Regression: inference vs prediction

I see it questioned here quite often about what the users aim is when performing a regression analysis: prediction or inference. How would your approach differ based on whether you were modeling for the purpose of inference or prediction?

This is my rough draft of the differences, but there's always exceptions to the rules. Moreover, one is often interested in both inference and prediction.

Inference:

Use parametric models and subject matter and theoretical knowledge when building the model(s) and when doing covariate and model selection. Things of interest include treatment effects/effect sizes and the uncertainty in the estimates.

Prediction:

A more data driven and exploratory approach that might include strange and unintuitive feature engineering. Semi or non-parametric methods such as boosting which enable easy investigation of feature interactions. Things of interest mostly related to computation speed and prediction performance.

Prediction is inference, and is just as principled. Out-of-sample prediction accuracy has to be estimated from the data, and we can study the same properties (bias, variance, efficiency, etc) of estimators like LOO-CV or k-fold CV as we can in more conventional inferential statistics.

This is not true in practice. Out of sample prediction accuracy might be estimated for eg the next week.training for last 6 months. '(and then retrain). Model could be very unstable from week to week and perform great for prediction, bad for ( causal) inference

How does that contradict what I said? Prediction and causal inference (more generally, explanatory power) are two goals we might like to accomplish with a statistical model, but as long as predictive models require parameter estimates, they will also require inference (parameter estimation is inference).

This is a subreddit for discussion on all things dealing with statistical theory, software, and application.
We welcome all researchers, students, professionals, and enthusiasts looking to be apart of an online Statistics Community.