This is due to the Exchange CalendarRepair

This happens when the user who left is the meeting organiser

You can disable this per use via : 

Set-mailbox <User> -CalendarRepairDisabled $true