The villa was originally the summer residence for Tolomeo Gallio, Cardinal of Como. It was designed by architect Pellegrino Tibaldi, who built the villa on the site of a former nunnery.
At that time, the building was named Villa del Garovo, due to the Garovo stream that flows in the area. During the cardinal’s lifetime, the villa welcomed politicians, intellectuals, and ecclesiastics.
Upon Gallio’s death, the villa passed to his family that, over the years, led the building into a state of decay. In the following decades, the villa passed from hand to hand several times. It first came the Jesuits, then it was acquired by Count Mario Odescalchi, then again by Austrian colonel Count Marliani.
In 1784 the villa became property of the Calderari family from Milan. The family undertook a major restoration project and created a new Italian garden.