"Mrs Patmore has been loyal to this house and this house must be loyal to her."