You can open the library file directly, and copy symbols out of it.
Or you can place them on your schematics or PCB, and generate a library from that.
(There are no lib-merge tools built in, unfortunately, so you can't dump a sch's components into an existing library, for example. There might be a script that does something handy like that.)
The built-in libraries are kind of weird, mostly having badly proportioned shapes, ugly labels, and non-compliant footprints (with minimal mechanical features, e.g. courtyard, 3D model). It's a good plan to edit them anyway.
Tim