The fascinating ornaments of the exterior made the visit worthwhile