The magical energies contained in the brain could cause side effects. When the wizard acts as a conduit there are risks. As these are third level side-effects, the results should be noticeable.
This makes wizards somewhat more unique based on the spells they know and can prepare. It also can make them more useful or somewhat of a liability.
Here is a listing of those side effects:
Blink: Bits of the caster phase out of reality. He must make a saving throw versus Wands every six turns or a small item he is carrying drops to the ground unnoticed.
Clairaudience: Casters hearing is slightly better, +1 on hear noise checks, but -4 on any saves versus sonic effects
Clairvoyance: Caster's vision is better, can detect secret doors with a +1, but -4 on any saves versus illusionary figments or other gaze or visual attacks
Dispel Magic: While memorized, any magic item used by the caster has a 25% failure chance
Explosive runes: Caster has flaming rune appear beneath the skin on his palm.
Feign Death: Caster is narcoleptic while this spell is memorized. During any non-active activity (sitting, reading) there is a 2 in 6 chance the caster will fall asleep.
Fireball: Caster can shoot a bolt of flame out to 15'. It requires a normal to hit roll and does 1d6 damage. This can be done once a round.
Flame Arrow: Temperature is raised 15 degrees around the caster
Fly: The casters feet no longer touch the ground. Any falls are affected as if the first 10' are covered by feather fall. The caster gets a -4 save versus any attacks or saves that move or push around the caster and gets knocked back a foot for every point of damage they take
Gust of wind: Caster is followed by gusting winds that blow hats off, knock papers around and generally make a mess of things
Haste: Casters speech becomes pressured and he begins exhibiting signs of mania
Hold Person: Parts of the casters body grow numb at inconvenient times. -2 on all Dexterity related checks.
Infravision: While memorized, casters eyes glow red and they have low-light vision, doubling the range they can see of light.
Invisibility 10' Radius: The casters skin is now transparent, and objects held or worn by the caster are translucent.
Leomund's Tiny Hut: The caster feels comfortable in any temperature range between 0 and 105 degrees. His clothing and items are impervious to rain and water, repelling it for several millimeters.
Lightning Bolt: The caster can shock any adjacent creature without a to-hit roll and do 1-3 damage.
Monster Summoning I: Pentagrams float on the surface of the wizards clothing.
Phantasmal Force: The wizard's clothing becomes more fancy in appearance, and his voice and footsteps echo slightly, and subtle sparkles appear in the light around the caster. If this spell is frequently memorized, the caster will begin to gain a halo, light shining from behind his head.
Protection from Evil 10' Radius: Silver filings, chalk dust and miniscule blue flames swirl around the caster, forming shapes of protective circles and pentagrams.
Protection from Normal Missiles: Anyone who approaches the caster feels a slight resistance. +1 to armor class.
Slow: The caster always acts last in the round. His speech is also slightly slow and uneducated observers might believe he was depressed.
Suggestion: The casters voice is hypnotic. +1 on reaction rolls.
Tongues: The casters accent becomes flawless on any language he actually speaks
Water Breathing: The caster drools uncontrollably.