Foundation Park is a small parcel of land on the top of a sandstone cliff behind the Argyle Terrace, which contains the last remaining foundations of eight former houses. These houses were very compact and built on different levels between 1874 and 1878, when the area was densely populated and people would grab what little land they could. Since they were also built cheaply, the houses were ultimately demolished in the 1938. In 1972 the remaining foundations of the houses and small spot of land they were sitting on were made a public park.