There are several bazaars in town but the biggest is on Shevchenko and Frunzey in the center. The biggest sections are clothing and food (the meat building is not for the faint of heart). You can also find toys, hardware, books, movies, flowers, pet supplies, electronics and just about anything else you might want at low prices. Largely outside with hundreds of individual sellers, it is an authentic Ukrainian experience.
Down a side alley you can find Soviet lore (such as war medals) for a fraction of the price in Kyiv.
Bazaars in Kyiv are expensive; bazaars in villages are tiny. Poltava has a good size one with low prices.