First off, what does it mean to ask how things would have turned out in a counterfactual? Any basic understanding of chaos theory and quantum indeterminacy should quickly disabuse you of the idea that there would be a single possible result. The instant you create the new timeline, an uncountable number of fundamentally random differences start building on each other. Create the same new timeline, different results. None of them would be any more real or correct than the other. So, on this level, counterfactuals are a pretty meaningless concept. Even if you had access to them somehow, comparing the prime reality to any one possible alternate would be meaningless.
We have to give up the idea of a single alternate timeline. The question cannot be "what would have happened?" but "what is the distribution of results that would have happened?". It's meaningful to think about the range of results that could have been. Assuming access to the other timelines, you could build up an understanding in terms of mean outcomes. On average, what would my life have been like if X? (Or, if the distribution is multimodal, a more sophisticated analysis than simple mean, of course.) That would give something I could really compare myself to, the me I most likely would have been if X. That would be the most meaningful, most real alternate version of myself to think about.
Great! Except... I can turn those same tools onto myself. Just as there is a cloud of almost infinite different timelines around every historical change we could make, prime reality has the same cloud. By the arguments above, if we re-ran history from a given point, even not making any changes at all, we'd get different results. I'm surrounded by other possible versions of myself, even without the ability to make changes! And that set of mes has its own distribution, showing the most likely me. Which raises the question, how do I compare to that me? How likely am I? And if I find I'm not very likely, that I'm living far out on the tail somewhere, what does that mean? I was quite happy previously to define the maximum a posteriori estimation of myself in alternate histories as the most meaningful version. The most real. I might be very far from the most real version of myself. I might not be very like myself at all.
I'm not sure what to do with this realization.