I had the same issues on my first flight. An increasing drift requiring more cyclic stick to correct and eventually i lost control.
If the unit is very sensitive to small vibrations I'm unsure ill be able to determine and correctly resolve them.
For example a tiny imbalance in the tail rotor will be very hard to fix?

I desperately want this to work!

Well, Spirit units are the most tolerant units out there. So if there is some issue caused by vibrations, they are pretty high.
Unfortunately with smaller models frequency/RPM increases so it might look like a small vibration.
Vibrations where you can see spikes along the plot are resonant and are the most problematic. They can be caused by things that are not holding well at the model. Even by connectors, nuts, cables. If they are produced just by motor, I would focus on these things around. Motor itself is looking good, because no high specific spike is visible. You can try to hold parts of the model and observe the plot while motor itself is running.

The most tolerant setting is:
Advanced/Stabi Correction OFF
Advanced/Signal procesing: Extended

Please note you can turn off Stabilisation or any function at any moment. So when you are in normal flight mode no drifting should happen. This mean if you will observe anything odd, it is enough to turn off the feature and continue in normal flight without any problem to prevent loss of control.

_________________
Spirit System developer