You generally dismantle enough to get the speaker covers off so that they are on their own or with only plastic on the surrounding areas, then simply use some water in a spray bottle to rinse the cloth, and let dry. That removes most of the dirt, and if there still is then another rinse and some scrubbing lightly with a new paint brush will remove more. after that you start to get aggressive, starting with another wash with the brush and a drop of dishwashing liquid in the spray bottle, then a thorough rinsing and let dry. that will get rid of most light stains and not harm the by now fragile cloth too much.
If the cloth is torn then go to the haberdashery and get some black T shirt material and recover it, stretching the material to make a smooth surface.