During the 14th and 15th centuries, it was gradually rebuilt and expanded into a large castle complex, which became the representative seat of the Lords of Rýzmberk. Rabí retained its residential function until the end of the 17th century. The destruction of the castle occurred in the first half of the 18th century, when a fire destroyed the roofs and wooden parts. In addition, the abandoned castle grounds became a source of building material for the inhabitants of the lower castle and the wider neighbourhood.
Rabí has been the property of state since the 1950s and, in 1978, it was included in the list of National Cultural Monuments. Since 2013, it has been under the care of the National Heritage Institute, the Territorial Heritage Administration in České Budějovice.