If you can put your IC a bit higher to have straight lines going to it.
At these speeds and and distances the difference between the length of the traces is negligible.
The speed of the Serial port is not important, what is important is the speed of the USB connection, which
I would assume that it is full speed and not high speed.
At the end, if you are so concerned with trace length matching, then put the IC pependicular to the connector and
not parallel, like it is now.
I would be much more concerned to have ESD protection, if it is not provided by the converter IC,
termination resistors if not provided by the IC and a common mode choke for the lines if you have to pass
emissions test ( not always needed but it helps with conducted ).