You either have to copy and paste it as values to remove the pivot, or stop using RefreshAll.