The problem with the insights of spiritual teachers is that they are irregular, often imprecise and mostly unpredictable. The Perennial Philosophy, which dates back to the Renaissance, can certainly be found in most if not all of them, but the devil is in the details, and these are too often inconsistent. Especially when it comes to vital ones such as where the revelations come from: Secret Masters, Angels, ETs, super-intelligent shades of the colour blue, transdimensional entities - they've all been suggested. There is also the problem that for every A. C. Bhaktivedanta Swami - a man of conviction- there are ten Aleister Crowleys -sociopathic con-men. Scientific charlatans are more easily exposed than spiritual ones. People like something orderly and predictable. Something they can trust and rely on. Thus far metaphysics and philosophy have failed to supply this - at any rate in terms the aforementioned man on the Clapham omnibus can easily grasp or apply.
Science does not have or carry a commitment to atheism. Many scientists are believers, many agnostics, many atheists. Only a vocal and egotistical few are loudly anti-theist. Science, in its purest form, is a toolkit for the investigation of phenomena. It was never intended to probe purpose and meaning, just to find out what things are and how events occur. Thus far, material things have been the most susceptible to this kind of investigation. The non-material may require different tools and methods, but we should demand that these are of equal objectivity and rigour to those of science.
I take an interest in such questions because, while I am content to live my life as I find it (now, it wasn't always so), the thoughts of others show me a lot about people and how they think and see things. This matters because, as far as we know, if we are to find solutions to the problems that face us as a species, they are going to have to be human solutions. So we need to be aware of who and what we are. Will the search for transcendence unlock human potential, or is it a blind alley distracting us from material answers? Or is everything part of a whole we currenty only see part of?
Whatever the answer, it will require knowledge rather than faith, action rather than prayer and the acceptance that whatever entities may exist out there, they are probably, in their way, as flawed and human as we are.