Use Conditional Formatting When Reference Cell Not Blank

I am using conditional formatting to change the fill color of a cell if the value does not equal a reference cell. The problem I am having is that the reference cell will be blank after data is entered into the cell with the conditional formatting. Obviously, when the data is entered the two cells are no longer equal and it changes color based on the condition being true (blank<>data entered). How can I keep it from changing color if the reference cell is blank or only allow it to change color if the reference cell has data entered in it?