That project looks interesting, did you use any antenna design software? It would be interesting to learn how to design a Vivaldi antenna based on antenna requirements (swr, gain, etc).
I can neither confirm nor deny the use of any software. I am slowly trying to learn OpenEMS, which should be capable of this kind of simulation.
That said, the process of optimising for Gain/SWR is the same for both EM software and handcrafted prototyping. Generally the EM software takes a long time to calculate solutions over such large frequency sweeps, so doing it experimentally by hand is not a great deal slower so long as you have a good VNA.
First, you can divide the problem up into the feed and the antenna exponential and the corrugations.
FeedTo do this you can first start with a SMA to microstrip to SMA PCB (or copper tape on suitable thickness FR4). Then with a VNA connected you can cut out the ground plane such that it transitions to a broadside couples twinline. Experiment with different tapers and see how this affects the S11 and S22 (you can make do with an S11 measurement and a 50ohm terminator). There will be an upper limit where the S12 starts to drop off and the S11 increases. I have attached a picture of roughly what this test board will look like.
Once this is done, you have nailed down half of your variables and can be confident that the feed works.
AntennaThe exponential opening of the antenna is defined by a formula with an expansion coefficient.
I have uploaded an excel sheet that can generate these curves here
https://hexandflex.files.wordpress.com/2018/03/vivaldi-curves-hexandflex.xlsx. The value to adjust is alpha(I have spelt is aplha in my sheet).
You will need to guess at the rear cutout of the antenna. I just used an arbitrary oval shape. It is less critical than other factors and you can play with this later if needed.
You can then build and measure various opening coefficients (alpha). Either using a new antenna for each design or starting with the highest alpha first and trimming the antenna down each time (document all measurements, so you can go back).
CorrugationsOnce you have picked your optimal opening coefficient you can they experiment with corrugations on the arm edges to try and improve gain/reduce sidelobes. The pattern for the original palm tree antenna IEEE paper and my version seem to work well. Perhaps, you can find a better pattern for these cutouts and create a new category of Vivaldi antenna. The only examples I have seen in literature are rectangular slots and these palm tree cutouts.
If you do give this a go, I would love to see the results.