Question

what is the difference between a mill and a factory

Answer 1

A mill is:
"A building equipped with machinery for grinding grain into flour."

A factory is:
"A building or group of buildings where goods are manufactured or assembled chiefly by machine."

Answer 2

What about "lumber mill" then? "Sawmill"?

Answer 3

Mill (I think) is generally specific and relating to the break down of a raw product, when you just say 'mill' you would be referring to a grain mill. I think lumber and saw mill's are fairly self explanatory. Factories can involve anything that's in a building and is manufacturing something via machine (by peep's definition). So factories can describe many processes.