This study proposes a nonlinear cable model for the cable-bracing inerter system (CBIS). In a CBIS, cables are introduced to connect inerter systems and the structure for translation-to-rotation conversion. This CBIS employs an inerter element, a nonlinear cable bracing element and an additional damping element to utilize their synergy benefits. This paper aims to investigate the control effect of the nonlinear CBIS for high-rise buildings that are represented as bending-shear type models. First, a nonlinear inerter system is incorporated into a single-degree-of-freedom (SDOF) system and the mechanical model is proposed. An optimum design method is then developed for a high-rise building system equipped with a CBIS and the time-history analyses are conducted to validate the control effect of the CBIS. It is concluded that the employment of a CBIS can substantially improve the structural performance. A genetic algorithm can be used to obtain optimal parameters of a CBIS, thereby more effectively reducing the dynamic response of high-rise buildings.