By the 14th and 15th centuries Swaffham had a flourishing sheep and wool industryAs a result of this prosperity, the town has a large market place. The Market Cross here was built by George Walpole, 3rd Earl of Orford and presented to the town in 1783.  On the top is the statue of Ceres, the Roman goddess of the harvest.

About 8 km to the north of Swaffham can be found the ruins of the formerly important Castle Acre Priory and Castle Acre Castle.

On the west side of Swaffham Market Place are several old buildings which for many years housed the historic Hamond’s Grammar School, as a plaque on the wall of the main building explains. The Hamond’s Grammar School building latterly came to serve as the sixth form for theHamond’s High School, but that use has since ceased. Harry Carter, the Grammar School’s art teacher of the 1960s, was responsible for a great number of the carved village signs that are now found in many of Norfolk’s towns and villages, most notably perhaps Swaffham’s own sign commemorating the legendary Pedlar of Swaffham,