We mine it, but once we mine what’s there, it will take either hundreds of millions of years to make more, or we need to find a new source of Helium, such as through mastering terrestrial nuclear fusion or perhaps mining the Moon.
In the meantime, we should be aware that every time we fill a Helium balloon, we’re taking something that it took the entire natural history of the Earth to create and basically banishing it from our planet.
On running out of helium
Gravitropic 