DraStic

If your phone has a "performance mode" or "game mode", try turning that on.
There's a chance that the CPU governor thinks DraStic is a lighter load than it really is, meaning that it only keeps the clock speeds high enough while fast forwarding. I've noticed it more on octa-core phones recently ...