How do I know whether automation is actually saving money?

Compare the full cost of the old process with the automated one, including staff time, errors, delays, maintenance, and exception handling. Measure over enough time to capture normal volume and support work, not just the first successful run.