On Mental Health Treatment and Charity
"I like X" and "X is actually helpful" are two very different statements
Does UChicago offer good mental health care?
If we restrict our attention to people in exactly my situation, my instinct is to say yes: I’ve never had trouble getting an appointment, my psychiatrist is nice and gives me medicine, and I only have a $10 copay for my therapy appointments.
And these are all good! All else being equal, I do want a nice psychiatrist, and I'm glad I can afford the copays.
But in some very real sense, I have no idea how good our mental health care is because I have no idea if it's actually making me feel any better.
In some ways I'm feeling better than when I started treatment, and in other ways worse. Are the ways I'm feeling better because the medicine is working, or would that have happened anyways? Are the ways I'm feeling worse bad side effects, or random flukes?
My point isn't to say that mental healthcare is pointless - I think the overall evidence for psychiatric treatment is quite strong. My point is simply that as a patient, the things I look for in mental treatment are not the same things that make for a "good" program.
This is true in a lot more generality: if I follow the same metrics I'm using to fit my belief that I'm getting good mental health care, I'd probably be satisfied with various "alternative" or homeopathic medicines. If I follow them looking for healthy foods, I'll probably end up buying vitamin water (it tastes good AND says it's healthy and makes it easy to see what vitamins are inside!). If I follow them when choosing a religion, I'd probably end up with some sort of new-agey feel-good nonsense belief system.
I think charity works the same way.
We want to give money to charity for a fairly basic reason: because we want to help people.
But it's hard to measure how helped a person was by a charity. Sure, we can give someone water - but did they need the water? Did they have another source of water? Did we save them from dehydration, or save them the time of obtaining water, or not save them anything at all because they had water and what they really needed was food?
So instead we turn to other metrics: do I feel an emotional connection with this cause? Is it famous? Were its employees nice to me? Did I go on a missions trip "there" once? How much is spent on "overhead"? (See Charity Overhead is Not Evil for a fair discussion of the last point.)
Even Charity Navigator, which touts itself as "Your Guide to Intelligent Giving", judges on two main criteria: Financial Health and Accountability & Transparency.
And, once again, these are both important! A charity needs to be in good financial health to continue helping people, and it's important to know what your money is being spent on to avoid being scammed.
But neither of these tell you how much you're actually helping people, which was the whole thing we set out to do!
Let's say I have a charity handing out fladoodles to homeless people, and I want to know how much this actually helps. In a perfect world, I could wave my arms and shout "quantum mechanics" just loudly enough, and both give-and-not-give each person a fladoodle to see which will have the better outcome.
Since we're in an imperfect world, we'll have to settle for a randomized controlled trial (RCT): if we have 500 fladoodles, we can find 1000ish people qualified to receive them, and randomly assign each person to either receive or not receive a fladoodle. Since we did everything randomly, the only significant differences between the two populations should be the effects of receiving a fladoodle. If this makes people have much better life outcomes, then my charity is effective! If not, then I should give up on fladoodles and try something else. (Like malaria nets!)
Most of us don't have time to run RCTs on every charity we're interested in and compare the results. Fortunately, at least for secular outcomes, there are sites like GiveWell to do this for you! If you're looking for specifically Christian charities there doesn't seem to be the infrastructure to do this yet :(
But the moral of the story is that what you're trying to do and the easy metrics of how your experience was don't have to align, and I'm not entirely sure what to do with this information.