If you have Sheet1!A1 copied down a few times...
=sheet1!A1
=sheet1!A2
=sheet1!A3
=sheet1!A4
and then insert a row after row 3 in sheet1, the result will look like this...
=sheet1!A1
=sheet1!A2
=sheet1!A3
=sheet1!A5

How would excel know you wanted to reference (the new) row 4?

The only way to keep those references, if you insert/delete rows, is to use INDIRECT()...
=INDIRECT("Sheet1!A"&rows($1:1)
where $1 would represent the row number your formulas start omn