Nested IF/OR with VLOOKUP

I'm trying to build this formula and I'm only half way through it and it already isn't working.

Column D - can be 1, 2, 3, or 4
Column E - can be any number

If column D is 1 or 3, then the VLOOKUP should look up column E and point to the answer in column 4 on another worksheet (Companies in this case).

If column D is 2 or 5, then the VLOOKUP should look up column E and point to the answer in column 5 on another worksheet (Companies in this case)..

This is what I have so far, just trying to get it to recognize if it is a 1 or 2:

=IF(OR(D2=1,VLOOKUP(E2,Companies!A:G,4)),IF(D2=2,VLOOKUP(E2,Companies!A:G,5)))

What is making this not work? I'm getting a false. But before I added the OR, it worked for the D2=1.

Thanks.

Alisa