Way I see it, as long as you can keep each capacitor's plates from touching each other (you'd have to use vacuum as a dielectric, of course), and not get the thing hot enough to start thermionic emission, everything should work just as in the "all normal matter" case. Electrostatically, a positron being pulled closer is the same as an electron being pushed away. And as someone mentioned before, electrons (or positrons) won't just fly off the metal into space by themselves.
Quite how you would go about keeping the plates separate is the real question, since in a capacitor the plates will attract each other when charged. In a real vacuum capacitor, the plates are secured against the same mechanical assembly, so they ARE touching each other, just not through anything conductive. This is not an option here. One side being made of antimatter pretty much means it must be kept floating in a void vis a vis the other side. For a moment I thought you could stabilize things with a geometry like this one:
-------| |--------------| |------
| ^antimatter^ |
| |
--------(~)---------[R]----------
With all wires being rigid. But this is an unstable equilibrium since the force between the plates increases quadratically. Only way you could keep this system in one piece is by dynamically adjusting the position of one side of the system with some rocket (anti-rocket?) system, or by pushing it with photons. All seems very impractical just to charge a capacitor...