The Castle of San Sebastián is a 17th century fortress located in the city of Vigo, which was built at the same time as the town began to be walled in 1656, during the reign of Philip IV in Spain with the aim of protecting Vigo from attacks by the English and Portuguese.

On the site where the castle was built there was a chapel dedicated to San Sebastián, and the new wall would close up to this chapel. Juan de Villarroel y Prado designed the works for both the castle of San Sebastián and the Fortress of El Castro, located a few hundred metres up the hill. In 1665, Portugal besieged the town for six days, when the military engineers Carlos and Fernando de Grunemberg joined the two fortifications with a wall, which would be finished in 1667. However, both the castles and the wall were built very quickly and with little means. In 1719 the English invaded Vigo and the castle was left in ruins and was used as a place for livestock, and the town was defended from the Castro Fortress.

In the 19th century it was used as a hospital and underwent some modifications to accommodate this use.

In the 1920s, the mayor, Mauro Alonso Giménez-Cuenca, considered the recovery of the two castles. The San Sebastián castle remained a military barracks until 1964, when it became the property of Vigo City Council, thanks to the aldermen Alberto Varela Grandal and later José Ramón Fontán.

In 1970, Mayor Antonio Ramilo Fernández-Areal decided to build the city's new town hall, until then located in a building in the Plaza de la Constitución, for which some parts of the castle wall were demolished, and the new town hall was inaugurated by the King and Queen of Spain in 1976 when Mayor Joaquín García Picher was mayor. In 1983 the demolition of the part of the castle still standing was considered, as well as the Castrelos Bridge, but in the end these plans were never carried out. In the 1990s, the Plaza del Rey square was remodelled, which further degraded the castle's surroundings.