http://www.digikey.com/product-detail/en/SFH11-PBPC-D20-ST-BK/S9200-ND/1990093
http://www.digikey.com/product-detail/en/SBH11-PBPC-D20-ST-BK/S9175-ND/1990068
1-2$ a piece depending if it's female or male connectors. You can probably find them cheaper.
Those will fit the bill nicely, but is there a right-angle male connector to go with it? I didn't happen to see one. I saw one by another manufacturer, but it was like $4.70 a piece.
Digikey is full of connectors and pins, you can waste hours just searching.
Look up the connectors sections, filter by pitch 1.27mm and 2.54 mm, number of rows and pins, even right angle or not..
http://goo.gl/kz0jhThe plain unshrouded pins are very cheap, under 2$ for a 40 pin set... but you could be smart and play with volume prices even though you need just a few. Instead of getting 40 pin, get 20 pin or 10 pin and buy in volume, price may actually be better if you get 100 pieces of 10 pin compared to 25 x 40 pin connectors.
http://goo.gl/euaaz(scroll down, 40 pin connectors start from around page 3... the link above filters to show only straight, surface and through hole, with 1.27 and 2.54 mm pitch/row spacing)
And digikey isn't even the cheapest, but they're close to canada, much like mouser.com is or ca.newark.com
the ones that are male unshrouded you may be able to get from eBay really cheap (it's the same spacing as the breadboard holes so you can find them as breadboard accessories) but you don't have the guarantee that the pins have gold plating on them, or you have female + male from the same alloy... it's little things but it can matter.
I think it would be cheaper to put a straight male pins on the base board and a female at vertical board, the regular straight pins are cheaper