Everything was running smoothly, until I tried to upload a program using the serial port.
So the bootloader went in OK? Which bootloader did you install?
I notice that you haven't installed any capacitors. You should have 0.1uF from Vcc to GND close to the chip, and 18-22pf from each crystal pin to GND. Without these caps the Atmega may run erratically.
Your first screenshot shows avrdude sending STK_GET_SYNCH, Sync_CRC_EOP (Hex bytes [30] [20]) 3 times, receiving the correct response STK_INSYNC, STK_OK ([14] [10]). However the third screenshot shows no response from command STK_GET_PARAMETER, STK_HW_VER ([41] [80] [20]). This indicates that the 1284p either didn't receive the second command correctly, didn't understand it, or went to sleep since replying to the first command. This could be due to mismatched baud rates (wrong oscillator frequency? incorrect bootloader?) commands not being sent fast enough (causing watchdog time out), noisy power supply or bad wiring.
Arduino ISP doesn't seem to work either.
Put the Uno back to stock and check that it still works properly (particularly the pins used for ISP). Then install the recommended capacitors on the 1284p and try reinstalling the bootloader into it. If that doesn't work then either the crystal or the 1284p itself may be faulty.
I remember reading somewhere that Arduino FTDI chips can't address AVR devices with large memory capacity and that I should buy a dedicated USB to Serial chip. Is that correct? Can anyone help out? Screenshots of my connections are also posted below.
The Arduino Uno does not have an FTDI chip. It uses an ATMEGA8U to create the USB to Serial bridge.