Faster alternative to sumifs and averageifs needed

Morning all,

Hopefully somebody will be able to help me with this.

I am currently building a call centre dashboard to track call handler stats.

I have used sumifs and averagesifs to calculate data between 2 dates for a selected handler, but its already quite slow with roughly 20000 rows of data, it will expand up to about 1000000 rows.

I am unable to attach the live dashboard, but I have attached a basic sheet to show where the data comes from and how the averageifs/sumifs work.
Sheet1 is where the raw data extract is, and sheet 2 holds my formula's

Any help will be appreciated and i'll answer any questions ASAP.

Thanks
Dan