So, if I ask this question again:
Just one column or all columns on that sheet?
Your answer is "just column A", right?

In that case, the code is:
Private Sub Worksheet_Change(ByVal Target As Range)

    If Target.Column > 1 Then Exit Sub
    If Target.Cells.Count > 1 Then Exit Sub
    If Target.Value = "" Then Target.Delete Shift:=xlUp

End Sub
Or, if I am still confused ... draw me a picture. I am slow sometimes (I mean it!)