The orphanage was built in the place of the former Hotel Korona in 1749; it was further extended with the use of the town walls in 1781. The construction works were completed by Menyhért Hefele.

The Converted Orphanage in Kőszeg was founded in 1741 and was the first Catholic orphanage in the country. Two decades later other towns followed suit including Selmec (Banská Štiavnica), Sopron, Nagyszombat (Trnava), and some time later in Nagyvárad (Oradea), Veszprém, Kolozsvár (Cluj) and Vác. Its establishment was very early, even by European standards. The next similar institute was set up in Austria only two years later, in 1743, and in Vienna as late as 1766. Children without parental care or those minors who had lost either or both of their parents received careful attention in the 18th century. During the reign of Habsburg Empress Maria Theresa a large number of orphanages were established. In this respect, Kőszeg was clearly at the forefront; at the initiative of Chief Judge József Szvetics, the town management made it clear that they regarded the conversion of Protestant children to Catholicism and, to achieve this end, they are willing to make sacrifices. In light of the above, it can be said that the establishment of the orphanage was a key issue in the religious warfare between Catholics and Protestants. The successful setting up of the institution attracted some enthusiastic supporters such as Jesuit Missionary Imre Kelcz, a relative of József Szvetics, who wholeheartedly endorsed the initiative, moreover, he swapped the original plans with more ambition ones.

The institution began its operation in the new building, and they received orphans and half-orphaned children not only from the immediate vicinity but also from the whole of Vas County. As the Jesuits already had a secondary school up and running in Kőszeg, so entrusting the Jesuits with the operation of the orphanage seemed an ideal solution.They primarily accepted the abandoned children of local citizens, and if there was any vacancy, they also accepted minors from lower social classes such as the orphans of farmers or peasants. Those with noble origins were provided with better quality boarding but when it came to studying, achievement mattered over background.

From the letters the board received it seems obvious that there were more applicants than vacancies, and it is also clear that many children could not have afforded schooling, had it not been for the orphanage. Talented children were given a lot of attention; outstanding pupils were encouraged to go on to study, and they helped the rest of boys to learn some sort of a trade.