The construction of the Aragonese Castle of Venosa began in 1470 at the behest of Duke Pirro del Balzo in view of a larger fortification project.

It is an imposing building with a square plan and four cylindrical towers. The Del Balzo coat of arms, the radiant sun, is visible on the west tower. The construction of the castle and the excavation of the moat in accordance with the new fortified doctrines involved the demolition of the Romanesque cathedral and the neighborhood that surrounded it. A new cathedral was built in an expansion of the town on the lower part of the plateau where the city stands. It was transformed from a fortress into a noble residence by Carlo and Emanuele Gesualdo, with the addition of the internal loggia, the north-west wing and the redoubts at the base of the towers, and from 1612 it housed the Accademia dei Rinascenti.

The four cylindrical towers at the corners are supported by redoubts that constitute the shoe of the moat, used as prisons, and the whole building, which is accessed via a drawbridge, is surrounded by a deep moat. Inside there is a large courtyard surrounded by a Renaissance loggia. In front of the castle there is a porticoed square and a monumental fountain granted to Venosa by Carlo d'Angiò. Inside, in the rooms created in the basement of the towers, the castle houses the National Museum of Venosa, which preserves especially the rich evidence of the Roman colony of Venusia.