VBA to color data points in a Chart based on non-graphed column values

Instead of data labels, I sometimes go in and manually change data point colors based on values in another column (like yield values that are graphed and color changes based on input material lot number). What is annoying about this is if the data range of the graph is changed, the colors that I put in manually are no longer correct.

I am trying to figure out a VBA way to color the data points
Here is my first shot.
I am stuck though. See the ** comments in the code.

Sub LoopPoints()
   **snipped**
End Sub