The castle came to be when the Venetians landed on the island and Marco Sanudo established the Duchy of the Aegean (or Archipelago) in 1207. Interestingly, the tower was one of the first buildings to repurpose material from the ancient acropolis.

Later, Latin nobles took up residence in the castle. This is the only Venetian castle in Greece that has been continuously inhabited since its construction. The castle originally had three entrances: the Trani Porta, the Paraporti, and the Piso Paraporti. The latter no longer stands.