Any person duly elected and qualified as Sergeant at Arms of the House of Representatives shall continue in said office until his successor is chosen and qualified subject, however, to removal by the House of Representatives.