I wouldn't change them back - just work off a copy that you can throw away.

You can make things easier by adding 1 to the times on Saturday and 2 to the times on Sunday and 3 to the times on Monday. Do the same for the criteria and use an if formula to trim the times accordingly before calculating the difference.