We have learned from intersectionality that people can be oppressed along more than one axis, such as women of color or trans people of color. But intersectionality also means that people can be both oppressed and oppressors at the same time, along different axes. For example, a white woman has white privilege but may be disadvantaged by her gender. A poor white man may be white and male, but he may be disadvantaged by his class status. An overweight middle class white man may face fat phobia. The word cannot neatly be divided into oppressors and the oppressed.
If we want to dismantle certain structures of oppression (say, gender), we need to not reinforce other structures of oppression in the process (say, appearance, or class). We need to take down all the structures of oppression, whether based on gender, race, class, sexual orientation, gender preference, or appearance.