Speed up a 100k AverageIf formulas

The first quick-win in these scenarios is to remove the requirement for the multi conditional test in the first instance.

Using your sample file - if you concatenate the fields of interest to a "key" on Sheet 1 then you can revert to AVERAGEIF using a single test based on the key.

'1'!H3
=$A3&"@"&$B3
copied down for all rows

You can then revert your formulae to:

F4:
=AVERAGEIF('1'!$H:$H;$A4&"@"&LOOKUP(REPT("Z";255);$F$2:F$2);Mxx)
copy across but modify the final named range in each column (avoid using INDIRECT for obvious reasons)

In terms of performance - on my machine your original matrix took around 4.5 seconds to execute whereas utilising the above the calc time drops to around 0.6 seconds.

If the performance gains remaining insufficient then the next thing to do would be to look into sorting your data on Sheet 1 (appropriately).
Sorting permits use of Binary Search based calculations which on large sets invariably offer some efficiencies and also has other advantages too regards helper calcs to reduce burden.