I suspect you are missing the switch transitions while processing floating point arithmetic for the LCD and I don't really understand why you are initializing the PWM on every pass through the while(1) loop.
Here's a microchip app note for rotary encoders and it doesn't use interrupts. However, the handler code needs to be called often (multiple times in the while(1) loop, after each significant step) or there will be slippage. So create a function to handle it and just call if from various places.
http://www.engr.mun.ca/~dpeters/6806/postings/RotaryEncoder.pdfYou could also use a timer interrupt handler to process the encoder at some rate. Use the ideas from the app note about XORing the current switch values versus the previous.
http://www.hobbytronics.co.uk/rotary-encoder-tutorialNone of these will ever work as well as using the pin change interrupts. Once you study the transition sequence, you will see how to use pin change interrupts to increment or decrement the value. All of this will happen in the background and the encoder count just changes whenever the user wants, not when the program eventually gets around to reading the pins.
I found an Instructable but it doesn't solve the entire problem:
http://www.instructables.com/id/How-to-Program-a-PIC-Microcontroller-Read-an-Encod/?ALLSTEPSYou need to search the Internet for 'PIC encoder interrupt'.
At startup, you note the existing condition of the encoder pins and set up the interrupt on change such that you will see an interrupt on the very next transition of either. Once you get the interrupt, you note which pin (A or B) has changed and from your knowledge of the transition sequence, you increment or decrement the counter. Then you set the register to interrupt on the alternate transition of that bit. If you just got a falling edge interrupt, the next interrupt for that pin is a rising edge.
In one direction, let's call it increment, the transitions are A=0, B=0 to A=1, B=0 to A=1, B=1 to A=0, B=1 to A=0, B=0. There are four transitions and in my application I added a count at each transition. So my count went up by 4. For knob encoders, the count should probably just go up by one after a complete set of transitions. That gives one count per detent.
In the other direction the transitions are A=0, B=0 to A=0, B=1 to A=1, B=1 to A=1, B=0 to A=0, B=0. Again, four transitions.
The tricky part is to be able to get, say, two of the forward transitions, one of the reverse transitions followed by three of the forward transitions and still get the right count. You get that odd reverse transition from switch bounce. When you build your state transition table, you need to consider this problem. That's why the quadrature encoder was invented.
I did this on an AVR several years ago and it worked great! Unfortunately, I can't get to the code right now and, anyway, it isn't for a PIC.
ADD: My code was for a rotary encoder on a shaft and I needed to measure distance so I might have an initial condition that was like A=1, B=0. For a knob encoder, the detent position will either be A=0,B=0 or A=1, B=1. Knowing which will make it easy to set up the pin transition interrupt. On the AVR, I could select whether I wanted rising or falling edges as my next interrupt. I'm not sure the PIC works that way, you may have to fiddle around to determine which transition just occurred.