3.5mm and SMA connectors look very much alike. In fact, they mate just fine. However, when SMA is mated to 3.5mm, there is a chance it may damage expensive 3.5mm side. Note, adapter you showed have two sides. The side that's smaller, is likely be SMA. The one that faces outside is 3.5mm. 3.5mm is a precision connector. It goes higher in frequency than SMA. But it is easily damaged.
My understanding is, you CAN exchange N type to 3.5mm type and do a self-cal. Nothing else. I'm sure your specAn won't autodetect change in connectors. There has to be a way to change it but I don't know that one.
But big question is WHY. Do you really need to reach 26.5GHz and maintain 0.x dB precision? You can find 26.5GHz rated cable with N connectors on eBay. SCOFLEX is one of them. Regular SMA to SMA cables typically don't go to 26.5GHz. By going 3.5mm, only thing that'll really change is the cavity effect around 20GHz.