SUMIF with Dynamic Criteria

I am trying to do:

=sumif(a:a,"=$c3&d$2",b:b)

Basically, I have a box I want to fill, with a row of names and a column of names along the left side and top. A:A has all of the different combinations of all of the different names (concatenated). So, I would like to drag that formula through the box and have all of the cells fill with the proper sum of the concatenated row and column header. Sorry if this is a little convoluted.

It only seems to want to match $c3&d$2 and not actually match what the concatenation is. I looked online and it looks like sumif only works with concrete things, like "dog" or "5", not with formulas that can change.

Does anyone know how to get the above to work or a different formula that would do the same thing?

Thanks!