The first building for the Avenue Market, originally known as the Lafayette Market, was built in 1871. In the twentieth century, the market and the Old West Baltimore neighborhood thrived as the Pennsylvania Avenue became a center of Baltimore culture. When the wooden market burned to the ground in 1953, merchants banded together to rebuild it.
The market survived as many of the neighborhood's historic buildings were abandoned, and in the 1970s, demolished in the name of urban renewal. Enthusiasm for urban renewal in the 1970s waned, and by the 1990s, the Lafayette Market was in desperate need of a makeover. It closed in 1994 for renovations and reopened in 1996 as the Avenue Market, an homage to Pennsylvania Avenue, the cultural heart of the neighborhood. The market was again renovated in 2012.