Excel 2010 has a simple to use REMOVE DUPLICATES option on the DATA tab.
ActiveSheet.Range("$A:$B").RemoveDuplicates Columns:=2, Header:=xlNo

Also, in your original macro, here's a way to remove all the 7681 rows without looping line by line:
ActiveSheet.AutoFilterMode = False
Rows(1).AutoFilter
Rows(1).AutoFilter 1, "7681"
LastRow = Cells(Rows.Count, "B").End(xlUp).Row

If LastRow > 1 Then Range("A2:A" & LastRow).EntireRow.Delete xlShiftUp
ActiveSheet.AutoFilterMode = False