The first references to the castellany of Oświęcim are from when it was annexed to the Duchy of Opole (1177–1179).  The rebuilding and defensive fortification of the castle is attributed to Mieszko the Tanglefoot's son, Prince Casimir. Archaeological discoveries near the tower can attest to this, confirming the existence of a masonry structure on the castle hill in the first half of the 13th century. Based on the surviving vetting documents of the castle, it is possible to describe the furnishings and appearance of the rooms. To enter the castle, one had to cross two wooden bridges; the rooms where newcomers were received were located on the first floor. It consisted of a large courtroom, two rooms and an additional hall. On the other side of the castle, also on the first floor, there were several chambers and a chapel, with a chancellery and 'shops' (well-secured rooms, perhaps storing the castle books) below them. The tower had two or three storeys connected internally by a staircase. Climbing to a considerable height required the attachment of a ladder. A wooden external staircase was added in the 16th century, but it has not survived to the present day.  The lower part, which was not accessible from the outside, housed the prison. There was a clock on the facade of the tower. In addition to the living quarters, the building adjacent to the tower had a bakery, larders, vaulted cellars, a kitchen, a brewery and a malt house. In the armoury at the first gate, hooks, crossbows, lance cannons, transoms, darts (short spears with an iron blade), spears, gunpowder, iron balls and crossbow arrowswere stored.

In 1241, the castle and its surroundings were sacked by the Tartars. After this event, the town was rebuilt and surrounded by walls, and the erection of a square stone tower began on the hill. The executioner, the ‘master of holy justice’, performed sentences near this tower after Oświęcim was granted a judicial privilege in 1291. In 1314 or (1315), the castle became the seat of the first ruler of the Duchy of Oświęcim, Prince Władysław I. Between 1414 and 1433, Duke Casimir I, son of Przemysław, resided in the castle. In 1453, the castle survived a siege by its former owner, John IV, who, unable to come to terms with the loss of the residence, sought to regain it. In 1454, the castle, together with the Duchy of Oświęcim, was given as a tribute to Casimir Jagiellon, and three years later, on 21 February, it was sold. Under the Treaty of Głogów (1462), the castle, as part of the Oświęcim-Zator land, became part of the Crown. Mikołaj Słop became the first castellan. The king ordered the restoration of the castle and city walls, strengthening the defence system, and deepening the moats. The work was financed by the inhabitants of the Oświęcim and Kraków areas with a one-off voluntary tribute. In 1503, a fire consumed the wooden buildings; only the tower and foundations survived. Reconstruction took many years, and a brickyard was opened for the castle in 1512 and a quarry in 1525. The work was completed in 1534. This was the last major reconstruction of the castle and its surroundings. In later years, the building steadily declined and lost its defensive character. A fire caused by the Swedes in 1656 sealed the castle's demise. All the buildings except the tower and the south building of the castle were destroyed. Floods in 1805 and 1813 washed away the hill and destroyed the defensive walls and farm buildings. The last reconstructions from 1904–1906 and 1929–1931 finally changed the character of the castle. Only the Gothic tower bore witness to its former purpose and defence.

During the Second World War, the castle was the headquarters of the German occupation authorities.

The castle edifices hosted numerous prominent personalities: Elisabeth Rakuszanka (future wife of King Casimir IV Jagiellon), the Czech King Ladislaus Jagiellon or Bona Sforza d'Aragona (future wife of King Sigismund the Old).

The castle is now owned by the Oświęcim City Hall. In 2007, the first phase of renovation co-financed by the European Union was completed. The second phase will see the restoration of the Gothic tower.

The Oświęcim Castle Museum with its historical and ethnographic collections was located in the castle building. In the permanent exhibition, visitors can learn about the history of the Galician town, see the bourgeois rooms, which are an attempt to reconstruct the décor of the houses before the war and see objects found during archaeological work, such as axes, hatchets, and pots.  In turn, two tunnels extend under the walls. The older one was built before 1914, the second was hollowed out by the Germans during the war – it served as an air raid shelter.