The Castle occupies a spectacular setting in a highly elevated position overlooking the Firth of Forth. While the earliest house, a medieval Castle belonging to the MacDuffs, Thanes of Fife, was built to take advantage of the defensive, coastal site, `Castlehill,` a small 18th century mansion house, was built further east to exploit the site's picturesque qualities. The Blaw family owned the estate from the 16th century until 1830, when it was sold to Lady Margaret Keith of Tulliallan. Shortly afterwards the small mansion was incorporated into a castellated villa (1839-45) for Mrs Magdalene Sharpe Erskine (1787-1872), sister and heir to Sir John Drummond Erskine of Torrie (1776-1836).