Ten or twelve years back, I rather enjoyed working with the ATmega128. It's a 64 pin package so it has a ton of IO, the C compiler is full featured (as opposed to what was available for the PIC 16Fxxx) and programming the device with AVRdude was easy to incorporate right into the makefiles.
The ATmega128 is still my favorite 8 bit chip.
But the original question for this thread was where to start and I took that to mean absolutely zero experience in the arena. The only correct answer today is Arduino because everything is handed to the user on a plate. For the ATMega128, I had to write every single line of code. If I wanted to use a peripheral, I had to do all the User Manual research, set up all the registers and do all the debugging. There was no application library.
Still is was doable. It helped that I had been working with uCs since '75, the earliest days of the 8080, Z80 and 8085. I understood what I was reading. I'm not so sure how it would have worked out if I were starting cold. Or, maybe I do!
I transitioned to the ARM 7TDMI and started cold. It took a while to get up to speed with the startup code. In that version of the ARM core, the startup code is written in assembly language and it takes a few hours to get the hang of it. And more reading of the User Manual. If I were to start fresh with the ARM processors, the original mbed (LPC1768) and the online toolchain/library would be my platform of choice. Again, much of the peripheral code is provided. Networking (lwIP) is provided. There is still plenty to write but there is a base set of library code and a lot of documentation.
I would think that building a project with a bare iron AVR would be many times more difficult than doing it with an Arduino. MANY times more difficult.