Its location is linked by legend to the bringing of St Florian's relics to Poland. According to tradition, oxen carrying the saint's remains stood on this spot in 1184 and refused to budge until the decision was taken to build a church dedicated to this martyr on the spot where the oxen stood. When it was built, it was the most important church of Kleparz (the former city and now a district of Kraków). Further work was carried out between 1755 and 1779, with the addition of the Chapel of the Holy Cross and Chapel of Our Lady of Sorrows in the west façade along with side altars and interior fittings. In the chancel are late Baroque stalls with paintings from the late 18th century and a main altarpiece with a 17th-century painting depicting St Florian with the city of Kleparz in the background. In the treasury of this church is a Gothic reliquary in the shape of a miniature castle turret, taken as war booty during the Battle of Grunwald, which was donated to the church by King Ladislaus the Short. It was made in 1365 for a Teutonic Commander.

In 1999, Pope John Paul II elevated the church to Basilica Minor. The future Pope Karol Wojtyła exercised his priestly ministry here as a curate between 1949 and 1951. He lived in the parish building at the time, just behind the church. He quickly made a name for himself in Kraków as an academic pastor, excellent preacher and confessor. At St Florian's Church, he organised passion mysteries, meetings to sing carols, and services for the sick.

While curating at this parish, he also organised numerous outings for young people, during which he celebrated field masses. As Pope, he visited his former parish only once, during his last pilgrimage to his homeland in 2002.

His priestly ministry in the basilica is commemorated by two plaques -– one inside the church and the other on the vicarage, under the window of the room where he lived.