The house and surrounding land was originally owned by the Henry St John, 1st Viscount Bolingbroke. The family owned the house for several centuries until the cost of caring for the house proved too high, and they were forced to sell.
The house is now owned by Swindon Borough Council who remodelled it and opened it up as a museum for the public. The surrounding garden Lydiard Park is also owned by the council and has been opened as a public park.
The architecture of the buildings is mainly palladian]] and contains a great many Georgian elements.