It's always tricky, that balance between getting off a psych med slowly enough that you don't end up hurting yourself, and yet getting off of it as fast as you can since it's poisoning you.
Just be careful. Never CT a psych med, always taper. If you're getting a bad reaction, hold for a while or slow down the taper. Observe your body's reactions. You're the expert on you.
All these meds interact because all neurotransmitter systems interact in our brains via complex chemical feedback loops. So the way trazodone affected you before may not be the way it affects you now, if you've gone through a benzo taper or withdrawal.
Just pay attention and make any changes slowly. And maybe try posting on an antidepressant forum (trazodone is technically an antidepressant).