You're looping cell by cell to find the part number instead of just jumping right to it? Also your use of Activecell makes it look like you're selecting cells over and over again like a human might do. VBA has no need to select.

Dim PartFOUND as Range

On Error Resume Next
Set PartFOUND = Sheets("Inventory").Range("A:A").Find(TagID.Text, LookIn:=xlValues, LookAt:=xlWhole)

If Not PartFOUND is Nothing Then 
    PartFOUND.Offset(,1).Value = PartFOUND.Offset(,1).Value - Qty.Text
    Set PartFOUND = Nothing
End If