I've tried adding a "Difference From" field to the pivot table, but then I'm not able to use that field in a calculated item.

Any other ideas?