Question

what determines whether or not work is being done?

Answer 1

Mathematically,
\[work = \int\limits F.x\]
Hence, whenever the RHS (integral dot product) evaluates to be zero; no work is done.

Physically,
Work is defined as the transformation of ordered energy into unordered energy. So, if no transformation took place, no work is done.

Finally, you may simply use the work-energy theorem -
Work done = Change in Kinetic Energy