The Presbyterian Church was built in 1890 by John Hunter and cost $14,000 to build. The church was built for people called Presbyterians, and an extra part was added to the back in 1894. This was not the first Presbyterian church in Milton. Their first church was built in 1846 farther west on Main Street, and another stone church was built in 1855 on Martin Street before this church was built.