Question

When choosing a cost function for machine learning, why is convexity generally a desirable property?

Answer 1

A little bit of heuristic reasoning; if we had a machine that is built for observing a series of transformations, and had to replicate it, it would do so by finding the "optimal" or cheapest transformation energy-wise. The text says that desirable properties, like convexity, usually define these cost functions.

Here I am wondering what that means.

Answer 2

Because I can't fathom any reason why convexity is inherently useful for cost functions, or even relevant, as a property.