Its a cute idea.
Lets put aside technological hurdles like bus connections etc etc.... Lets just open up today's very hard to open cellphone or itouch like device.
What do we find? A very small integrated board with SoC, DDR ram, Nand flash and power converter. All on a board that is about ~1" x ~2" give or take. Splitting that up won't be feasible other than maybe the nand (make it an sdcard connection) given routing and matching and speed of SoC to DDR ram. Then we have a camera board, battery and screen.
Lets put the camera on an SD card like board, okay now we have a modular camera , only constrained by bus topology... so lets make it USB, that should give us some flexibility and "upgrade ability".
We could do something similar for the cell modem. caveat would be bus interface.
Next wrap the battery and add a connector so it plugs into the exo frame.
To make the unit thinner and lighter, bond the exo frame to the screen.
So now you can change those modules...
The problem is going to be the s/w... See Windows/Linux/etc, see OS run on loads of different hardware... See loads of forums for having to configure OS to run correctly. Of course we could setup a certification authority...
It is an interesting idea overall..