The Society of Jesus arrived in Jaen at the end of the 16th century because the canonry and some important families in the city considered that it was necessary that the Order took charge of the educational and doctrinal activities. Between 1590 and 1614, the missionary activities carried out by the Jesuits in Jaen were not fixed. This favored the knowledge and dedication on their work with the residents. When other houses with their own colleges were opened in the cities nearby, as it had happened in Úbeda in 1590, the positive influence that they applied over the city's education and habits led to the creation of another college that same year. They tried to do it several times because of the lack of resources and the opposition of some canonry's members that used their influences to impede its construction. On the other hand, other orders already established in the city also opposed because the Society of Jesus was their rival. Jaen's bishop Sancho Dávila finished with the opposition to the Society's establishment in the city. He was not only its protector but also its benefactor: he donated some houses between the Maestra and the Escobar Street. It was a privileged place for the settlement of the Order. According to some historians of the time, on the 7th of April 1604, the Jesuit temple was consecrated and dedicated to San Eufrasio to honor the patron of the Diocese in Jaen. The priests' work and the spread of the supposed miracles that occurred in the temple favoured the enthusiasm for the Order and its establishment in Jaen. However, the donations and inheritances given by important figures such as Bernardo Ortega y Gómez in 1639 permitted the construction of some schools in the houses that the priests had acquired together with the ones of Sancho Dávila. They were enlarged in 1689 thanks to the donation of Lucas de Ledesma and were transformed into a college. The college was built as the rest of the Jesuit constructions: it had a temple, a residence and classrooms throughout its two floors. The creator of this work is still unknown. Some researchers expose that it was made by Andrés Cortés. The college possessed pictorial works produced by important Baroque artists as Sebastián Martínez or Francisco Pancorbo, which shows the institution's relevance. In its beginnings, they taught grammar, reading and writing. Then, they included theology and philosophy. It was called "Seminario Menor". There are no more data about the training and cultural work of the college. It became inoperative in 1767 due to the Pragmatic Sanction published by Charles III (1759-1788).

After the Jesuits' expulsion, the building became the "Monte de Piedad y Reales Estudios de la Concordia". Later, it transformed into a convent of Augustins and, then, into the "Colegio de Humanidades de Nuestra Señora de la Capilla". Finally, in the 19th century, it became a high school, a teacher training institute, a painting museum and a public library. It is nowadays an official music school.