On Monday night, dozens of defenders of the emblem gathered outside the courthouse in Greeneville. Tensions between them and those opposed to the rebel flag were calmed by local law enforcement, CBS reported.
The anti-flag side erupted into applause and cheers when the vote came down.
After his measure’s defeat, Randolph insisted the emblem did not have a racist meaning.
"I didn't figure the resolution would pass," Randolph told CBS on Monday night.
"I wish it had went through because it is a part of history and it ain't got anything to do with race or anything like that."
The flag controversy came to the fore after the murder of nine black worshippers at a traditionally black church in Charleston, South Carolina on June 17. The 19-year-old suspect, Dylann Roof, had posed for pictures with the flag in photos posted online. Now in custody, he faces capital murder charges.
In the wake of the mass shooting, the South Carolina legislature voted to remove the flag from the grounds of the statehouse in Columbia. The flag came down July 10 before a cheering crowd.
During the Civil War, Greene County in eastern Tennessee was disputed territory. It changed hands between Union and Confederacy troops multiple times over the course of the war.
"When you'd get up in the morning you'd look out the window and see what flag was flying," Betty Fletcher, director of the Greene County History Museum, told WBIR. "That's how you'd know which troops was occupying the town."