I have been playing around with this and just figured out something. Maybe it will help others as well.

I turned of calculation [set it to manual] just before entering this code and turned it back on after exiting.
Made the code much faster. Seems like it was re-calculating after every step of the loop.

I still welcome any other tips I can get.