this would be INDIRECT
I'd personally use it here together with CHR function. If your cell containing "how many columns right from O:O" is A1:

=COUNTIF(INDIRECT(CHAR(78+A1)&":"&CHAR(78+A1)),"Yes")
of course it works only until Z column

If its too little - use ADDRESS to get column name