Can you create a formula to define the table array in a vlookup?

A B C
1 100m sprint
2 Male Female Points
3 14 16 1
4 13 15 2
5 12 14 3
6 11 13 4
7 10 12 5

The table above is a (massively simplified) table to return a number of points for a decathlon/heptathlon event. I know how write a vlookup formula for either male =vlookup(REF,$A$3:$C$7,3)
or female =vlookup(REF,$B$3:$C$7,2)

Is there a way to write a vlookup formula which would select $A$3:$C$7 as the table_array for any male athletes and $B$3:$C$7 for female?