Former Conservative cabinet minister Peter MacKay isn't pointing the finger at Pierre Poilievre for the party's election loss, but he does think members of his team need to go.