row(a4) just yields the result of 4 - you could use any cell in row 4. when this formula is copied down it yields 5, 6, etc. my test worked just fine.

I assume you pasted it in the correct row???