Making AI work outside it's training data.

[paulca]

As a pet project, at home I set out to:  "Take something which Claude cannot do, does not know enough about to work on and fix that to get useful output even with that the case."

I picked an odd back corner of a back corner for it.  "Stationeers" the game.

Stationeers is an engineering/physics game wrapped in disguise of an almost cartoonish, lo-fi, 3D graphics, survival game.  Opening scene for every game is you landing on Moon/Mars/Europa etc.  In a space suit and a few crates of basic building supplies.  Your first task is to create a shelter you can pressurise with a breathable atmosphere before your hydration levels drop and you die.  You can't drink in your space suit without opening the visor with obvious consequences in hard vacuum.

Challenges for claude.  It's a close source code base.  It has some out of date info from Reddit and can search Steam Community, but other than that it's mostly clueless.  The only real visibility I can give it, currently, is the save game files, which are zips containing 30Mb XML files.  From prior attempts, AI reading that file is painful, slow and expensive.  So my new attack was to get claude to write it's own python interface to the save game file.

That moved to me asking claude about things in my game, could it find named devices?  Could it correlate pipe networks and cable networks and their attached devices.  Each time claude would "process" and "churn" on something to produce an answer, I would follow with... "Now lets push what you just did down into the Python API".

Then we moved to the next challenge.  Game physics.  The game uses "simplified" thermodynamics.  It uses custom phase change coefficients and boundaries.  Gravity does not effect fluids.  And so on.  This is a special challenge as now I have to override something claude already knows.  Basic gas laws.

Claude at this point did it's thing of realising the way things were going and started to become "pro-active".  I find this part a lot of fun. Every time I asked a new question it would give me a mostly wrong answer.  I would explain about the game physics not being the same as reality.  I was able to give it an online MathLab style plot for the gas phase change parameters, which it pulled the in game phase change data it needed from.  Without being asked to, but knowing the direction, it created (after asking permission of course) "thermodynamics.py" and "gamemechanics.md".  The python script has half a dozen methods (expanding as we go) which gives claude (or me) a set of functions to do gas law calculations with, which use the game physics.

Eventually.  I got to the "wrap up test" for the evening.  "Designing, prototyping and testing" a "Mars atmosphere heat sink", using the martian atmosphere and compression to liquidfy CO2 and store it.  The heat of condensation is carried through and exhausted only the liquid is stored.  In the obvious expansion side the liquid CO2 is metered into the expansion vessel by a regulator.  Heat is then "dumped" via various consumers of the cooler into the expansion vessel.  The expansion vessel has a small amount 1MPa of back pressure to stop the exhausting CO2 from freezing in the exhaust pipe.  That later part neither me nor clause predicted.  Although it did predict the pipe burst when it seen the dry ice in the same game and -56C, 1C below the tripple point of CO2 in game.  Claude was also correct that when balanced at the triple point, which it will if exposed directly to the martian temperatures and pressures, the game will show all three states with simultaneous evap, condensation, sublimation and degas... then the pipe exploded.  The back pressure was claudes idea.  Exhaust it well above the tripple point, at 1Mpa.  (I also had to add a liquid drain to the exhaust as the game does not like liquid in gas pipes).

While working through this, claude was giving me actual values for things testable in game to be "close enough all things considered".  The game is way too complex to get a 100% accuracy, without doing the full recursive gas propogation calculations it will always be out by a few %.  It did at one point offer to start a python script to do just that, but I aborted it, as I know better.  Recursive gas flow and heat propogation across 1000s of grid cubes?  No chance.  C maybe.

The next phase is to step it up a bit.  This "Save game" as the only shared state is a bit rubbish.  So maybe this evening, maybe tomorrow Im going to get down and dirty with the in game DLLs, and pull a REST API interface out for the game state the python scripts are using in the game file.  Then claude can have realtime "pull" and "put" access to  game entities.

The subject, aka, the game is not the important part of whats going on here.  What I'm doing is learning how to "up skill" an existing AI through carefully constructed lenses for it to see, aka "agentic tools".#

EDIT:  As an aside.  One often requested 'claude' feature in the game space is to "Write/Modify and review" IC10 code.  "IC10" is an in game microcontroller.  It is how you do any real automation on the logic network in game.  Read senors, switch values, automate equipment etc.  It uses, because the game devs are a cruel bunch who know how we think,  MIPs based assembler.  MIPS 'based'.  Looks nothing like real MIPs except the parts of the instruction set.  Yet, when I ask it for a code review on some IC10 code, it does so and it's pretty damn close.  A few more additions into a "IC10 guide for claude" markdown file and that will fix those corner cases and assumptions.

[paulca]

In case anyone is intrigued by the game itself, this is what the setup looks like in game.


Or this view which shows the pipes and gubbins under the floor.