How do I know if a pilot was actually successful?

Define success before the pilot begins and compare the result with the old process using time, cost, errors, response time, completion, or another relevant measure. Record failures and human intervention too. A pilot is successful when the result is repeatable enough to justify the next step.