The College of San Andrés was the first foundation carried out by the Jesuits in Bilbao. It was built to the will of Domingo de Gorgolla, the religious administrator of the Cardinal of Toledo. He died in Alcalá de Henares in 1600 and left a legacy estimated at 1.500 gold ducats for the construction of a Jesuit School in Bilbao. Francisco de Borja had previously tried to establish a School in 1552 at the request of the city's nobility but the town council opposed. On the 15th of November 1604, this amount was used for the building's construction. In 1630, Antonia de Zamudio Martiartu y Guecho, the widow of Ochoa de Urquiza, the "Contador Mayor" and Seville's "Juez de la Contratación" gave her legacy too. The construction of these buildings did not finish until the 17th century. Most part of the works were carried out between 1610 and 1630. The School assembled around the cloister, a sober and austere element formed by 4 galleries and three floors, where there were the typical rooms of a teaching institution. The School, the first Latinity chairs and the Christian doctrine were created when the Jesuits established in the new location. They progressively enlarged until 1764-65, when they counted on the chairs of Theology, Moral, Logic, Metaphysics, Physics, Latinity and Grammar. In 1767, due to Charles III's Order, the Jesuits were expelled from Spain and the Order's goods were delivered to the Crown. By the Royal Order issued on 1769 and the local government of Bilbao's request as the owner of the parish churches of the city, the ruined parish of the Santos Juanes, located in Atxuri, was transferred to the College's church at the end of 1770. The College and the Church were separated by a wall that impeded the communication between both buildings and deprived the church from the cloisters. In the same year (1769), part of the Jesuit School served as a school of basic education, Latinity and Rhetoric. It counted on bedrooms and classrooms for teachers and students.

Part of the building was taken up by the "Santa Casa de Misericordia" for a hundred years. It lodged the poor, the sick, the orphans, and the disabled and taught them some jobs among which the pottery stood out. This led to the creation of an industry of ordinary and fine pottery. When the "Casa de la Misericordia" was cleared in 1872, the main building of the old college turned into the headquarters of several public institutions. In 1879, the "Escuela de Artes y Oficios" was created and settled in it, where it remained until 1910-11. In 1914, the local government permitted the cloister's transfer for the installation of the Archeological Museum of Vizcaya.