Hi Okami !
I'm surprised that your Vet was so dismissive of the possibility of an allergy.
Cats are particularly susceptible to allergens - often their immune systems react to a transient allergen, the cat will scratch at the itchiness, the immune system will react some more...which causes more itchiness....which causes the cat to scratch some more....and on and on it goes! It can be a never-ending cycle.
Now, there are many things beside food to cause such a reaction....and, once initiated, the reaction can continue long after the allergen has gone away!
You mentioned scabs....a competent feline Vet will know to take some of the scab material and send it off for a histopathological exam at a lab....a competent feline Vet would also know that a simple bloodtest can assess for an immune reaction to an allergy.
Perhaps it's time to find a competent cat-ONLY Vet?
Anyways, here's one thought of what might be going on:
An eosinophil is a type of white blood cell that is commonly associated with allergic responses or with parasitism. Eosinophil counts will go up on a blood test when a pet has fleas or worms or when an allergy is flaring up. Eosinophils can circulate in the blood or they can infiltrate tissue. They are part of the immune system and are “on patrol” for biochemical signals from tissue (calls for help, if you will) telling them that a parasite has invaded. Eosinophils home to the signal and release chemicals to attack the parasite. Unfortunately, they can be tricked into thinking that some sort of benign materials (pollens, dust, etc) are attempting invasion. In this instance (allergy) they release their inflammatory chemicals inappropriately, creating the sensations of itching, swelling, redness and other symptoms of allergy....
These conditions are felt to have an underlying allergic basis though it is not always possible to determine what that allergic basis might be. The presence of any of the three above conditions does not definitively imply any specific type of allergy. In fact, there is some evidence that some cases begin as simple allergic reaction to an external substance but when internal skin proteins are released by scratching, the reaction continues to involve these “self” proteins as well.
Personally, I would have it followed up because, from personal experience, I know that these things are much easier cleared up earlier than later.