The area around Lake Swannanoa began as a Native American settlement, transitioning later into mostly farmland until the Ringling Brothers Circus acquired the property near the turn of the 19th century.

The lake enlarged (two lakes combined), and a 28-room manor house with auxiliary structures was built in the rustic neoclassical style to accommodate the winter home of the Circus, until the death of Alfred T. Ringling in 1919 brought an end to this era.

The Ringling buildings are on the National Register of Historic Places. And the manor house is now privately owned by the Order of Friars Minor Capuchin, Roman Catholic Church.

In 1925, the Ringling estate was purchased by Arthur D. McAllister, who renamed the area Lake Swannanoa. Plans were created to develop a ritzy golf and country club, the original prospectus calling for elegant high society balls, and racial/ethnic restrictions. But the great depression hit and this plan never came to fruition. Instead, in the 1930s, it was developed as a residential community by Ringling Estates Inc. Log cabin-style houses were built on small lots as primarily summer homes, and were used as such throughout the 1950s. Although there are a few weekend residents today, most houses are now occupied full time year-round. Many of the current houses are situated on multiple original lots.

The name Swannanoa is reported to be derived from the Cherokee word Suwali-Nunna, meaning "trail of the Suwali tribe." However the name was used in this case under the impression that the translation is actually "the beautiful trail." The street names within the development are all named after Native American tribes.

There are local legends about the ghosts of circus animals haunting the area. A human ghost is also said to haunt the retirement home located on the lake.

Today Lake Swannanoa (the lake, beach, and shoreline property, not the entire development) is privately owned and operated by the Lake Swannanoa Homeowners Association, formerly known as the Swannanoa Sentinel Society, a private non-profit corporation. Membership is by deeded covenant. There is a fee to join the covenant, and yearly dues are assessed. Access to and use of the lake and property is by permit and restricted to members in good standing.