Question

How do I decide if I should use mean-squared-error (MSE) or root-MSE for minimizing the error of a regression?

Answer 1

This is a question quite overlooked by most students. I am delighted that you are even thinking about it.

What you SHOULD use is the loss function that you actually need for your work. MSE and root-MSE often are selected ONLY because they are simple from the perspective of the difficulty of the calculation and have nothing to do with what actually is needed.

Anyway, MSE emphasizes larger errors a little more than root-MSE. Said another way, with MSE, a larger error makes you pay a heavier price than the price you will pay based on root-MSE. Which do you think is appropriate?

Answer 2

@tkhunny I guess I would then use the MSE?

I'm programming a neural network and am now at the part where I have to calculate errors based on the expected and actual outputs

Answer 3

Maybe. If you're doing this for real, not just for an assignment, you may wish to consider whether Squared Error is even appropriate. Maybe Absolute error would be better. Like I said, squared error emphasizes larger errors. Absolute error would not do that. root-MSE partially compensates the squared emphasis. Absolute might be a middle ground of could land on the other side of the root-MSE.

Further, would normalized error even be okay? Maybe overstating something is FAR WORSE than understating. In that case, you wouldn't want absolute or squared.

Seriously, if you are doing this for real, a standard loss function is very unlikely to be most appropriate. It's a design consideration you probably had not included in your timeline. It's important.