The lexer is the easiest part of a compiler. Parsing expressions is easy. The most impressive I have ever seen is two mutually recursive function with a loop each. With that few lines, all the precedence and associativity issues were taken care of.
The hard part of parsing is the forward declaration issues, and any grammar that are LR-easy but LL-hard - this is why compiler parser generators are used. Otherwise, a top down parser wins everytime for cleanliness and ease of design/implementation. You also have to take care of the mechanics of maintaining name spaces and scopes, but those are trivial.
Then comes the semantic analysis.
Writing a toy compiler is easy. Writing a production compiler is hard.