Mr CM300's Pandora's box:
Upon a vibration proof optics table, set up a vacuum chamber. Within the vacuum chamber should be a magnetically levitated cube with a solar panel for powering it from high intensity LED lights mounted within the chamber.
The cube has a mechanical assembly affixed to it with a high speed, highly sensitive gyroscope / accelerometer that detects the slightest movements of it, triggering data corruption.
The cube holds the cryptographic processor which is a custom ASIC containing, amongst other methods of protection, optical logic gates, MEMS logic gates and possibly some chemical / electrochemical methods of data handling.
Several key parts of the data should rely on external optical delay lines and internal mercury delay lines.
Naturally have the security cube reconfigure the positions of some of the protections using randomized numbers generated from a radio-isotope or heat noise random number generator.
Data to be encrypted / decrypted is transferred to and from the cube with differential light beams using interferometry to ensure the beams are not being extended for listening.
You could also have security lasers exiting the box to go around secure areas of the building, if the beams are broken then it wipes the keys and destructs itself through the release of HF acid onto it's circuit (or just a high voltage spike / overheating)
A thermal sensor would watch it's own temperature to ensure that it isn't being supercooled.
Possibly add computer vision to the security cube (IR, Visual light, X-Ray & Gamma to prevent X-Ray / Gamma visualization of internals)
...... I'll eat my sock if anyone can see a way to get past that. (And record myself doing so.)
You have violated one of the stated requirements - that physical access is not prevented. But in general you have put your finger on the problem. You have made the system extremely expensive, and extremely difficult to use, and have not eliminated all possible attacks. I will leave it to your cleverness to uncover the weaknesses that I (not an expert in the field) would use to start an exploit. I will grant that you have made it very difficult.
Not sure where it was said that physical access needed to be available to the actual board.
I also still can't see any way for you to get access to the keys....
The other idea was to put the board, with optical lines and a power line, into a holding frame, then mount a laser galvometer on the board.
An FPGA on the board with various ADCs and DACs has several 100 outputs that connect via thin wires to 2 copper-plated plastic hemispheres that are placed surrounding the ball.
The board is then supplied with a YAG laser beam to ablate the copper foil with a heat-noise generated random pattern, the ADCs then pass various signals through the wires and foil at constantly varying frequencies and waveforms, the moment a change in characteristics is detected it will erase the keys.
An X-Ray and radiation detector will be put on the board too to detect if it's being observed via X-ray style techniques to reverse engineer it to open it. A temperature sensor detects and erases the keys should it be attempted to cool it beyond 0*C
Keys are wiped should power be lost. (naturally)
Uses a tritium / radioactive battery to supply power to erase / re-write / jumble any data to ensure no memory is kept.
Data is passed in and out to be encrypted or decrypted via fibre connections to an outside holding frame.