If security is requested in the data request, but is not enabled in the stack configuration, then the flag in the request will be ignored and the data will be sent in plain text. And device that has security enabled can always receive plain text data in addition to the encrypted data, of course.
In the NWK_DataInd_t structure there is a flag that tells whether data was received encrypted or not.
But something else was wrong with your previous test anyway. It does not matter that the key was short, some leftover garbage was used as the remainder of the key. If the keys were different on both devices, they should not have been able to communicate.