The first castle built by the Order of the Sword Brothers around 1209 and originally named Wittenstein Castle ("made of white stone"). The castle was destroyed by townsfolk in the civil war. Townspeople built a new castle for the Livonian Order at the site of the Holy Spirit hospital under the condition of dismantling part of the town's fortification wall and opening the townfrom the castle side. The tower of the Holy Spirit and a fragment of the fortification wall were added to the castle that was built under the supervision of Master D.Kriege in 1330?1353. In 1484 the citizens destroyed this building as well, but this time they were forced to rebuild it and the reconstruction was completed by 1515. In the 16th and 17th centuries the castle was incorporated into the town's fortification system. In the 18th cent. several wings were added and the castle was used as a public building by Polish and Swedish officials, the moat surrounding the castle was filled up and the square near the castle was constructed. In the 18th century it became a residence for Russian Governor General.
Nowadays the castle is the President's residence.
