All of science is based on it…But I don't really know why it works

If I were to flip a coin a million times I’d be damn sure I wasn’t going to get all heads. I’m not a betting man but I’d be so sure that I’d bet my life or my soul. I’d even go the whole way and bet a year’s salary. I’m absolutely certain the laws of large numbers—probability theory—will work and protect me. All of science is based on it. But, I can’t prove it and I don’t really know why it works. That may be the reason why Einstein said, ‘God doesn’t play dice.’ It probably is.
— Physicist Leonard Susskind, Stanford University, in his 2005 essay for the collection “What do you believe is true even though you cannot prove it?”

Failure, with great precision

This measurement ruled out our whole inflation-based story with 99.999…% confidence, where there are a hundred million trillion trillion trillion nines after the decimal point. Not good.
Physicist Max Tegmark, on learning that data from the COBE satellite contradicted his theory about space/time inflation. From Our Mathematical Universe.

The Hubble volume

The Hubble volume is the sphere of space visible to the Hubble telescope—i.e. everything that’s not receding from us at a rate greater than the speed of light due to the expansion of the universe. The Hubble volume is an unfathomably large 1031 cubic light years.
That’s 846,800,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000 cubic kilometers.

You are often confident that something is true long before you have an airtight proof for it (this happens especially often in geometry). The main reason is that you have a large catalogue of connections between concepts, and you can quickly intuit that if X were to be false, that would create tensions with other things you know to be true.
What is it like to have an understanding of very advanced mathematics? - Quora
Understanding something abstract or proving that something is true becomes a task a lot like building something. You think: ‘First I will lay this foundation, then I will build this framework using these familiar pieces, but leave the walls to fill in later, then I will test the beams…’ All these steps have mathematical analogues, and structuring things in a modular way allows you to spend several days thinking about something you do not understand without feeling lost or frustrated.
What is it like to have an understanding of very advanced mathematics? - Quora
You are comfortable with feeling like you have no deep understanding of the problem you are studying. Indeed, when you do have a deep understanding, you have solved the problem and it is time to do something else. This makes the total time you spend in life reveling in your mastery of something quite brief. One of the main skills of research scientists of any type is knowing how to work comfortably and productively in a state of confusion.
What is it like to have an understanding of very advanced mathematics? - Quora