COUNTBLANK formula of a COUNTA formula

So I am trying to figure out how to make a formula to fit. I have a COUNTA formula for Column A (Total Employees) and I have figured out how to do formulas for Dates of Hire Column. But, I cannot seem to figure out how to Create a formula for Column E, which represents the employees not appraised. I did the COUNTBLANK(E2:E) But that I counting all of the blanks, including the rows without any employees. How do I set it up to only count the blanks of the amount of rows being used, which may vary quite often?