Suppose you spent your career measuring the rate at which something was declining — the trajectory of a thrown stone, the cooling of a fire, the slowing of a universe. You have equations. You have instruments. You have a model of how things end, and the model, while not warm exactly, has a certain grandeur: the cosmos began in fire, expanded, and would eventually exhaust itself, the galaxies drawing back toward one another through the long gravity of time, everything reconciling in a final, unimaginable compression. There is a tragic beauty to that picture. There is dignity in a universe that knows how to close.
In 1998, two teams of astronomers — the High-Z Supernova Search Team, led by Brian Schmidt, and the Supernova Cosmology Project, led by Saul Perlmutter — were doing exactly this work. They were measuring Type Ia supernovae, a class of stellar explosion so consistent in its peak brightness that astronomers use them as “standard candles” — known-luminosity objects whose distance can be calculated from how faint they appear. By measuring many such supernovae at great distances, the teams expected to determine the rate at which the universe’s expansion was decelerating. The deceleration parameter. It was a number the cosmological models already knew must exist. Both teams were competing to measure it first and measure it precisely.
They measured it. They got a number. The number was wrong in the most profound possible way: it was not a deceleration. The universe was not slowing down. It was accelerating.
The Terror of the Result
Adam Riess, then a postdoctoral researcher working with the High-Z team, recalls reviewing the data and running the analysis multiple times, convinced there was an error. There was no error. Schmidt has described the period following the first clear signal as one of the most disorienting of his scientific life — the sensation, he has said, of standing in front of a result that cannot be right and cannot be wrong simultaneously. Perlmutter’s team, working independently, arrived at the same conclusion through different measurements and a different analysis chain.
Two teams. One answer. The universe is not slowing.
The discovery required the resurrection of a term Einstein had introduced into his field equations in 1917 and then spent the rest of his life regretting: the cosmological constant, a kind of repulsive energy built into the fabric of space itself, working against gravity on the largest scales. Einstein had added it to produce a static universe — the universe of 1917, before Hubble’s observation of recession velocities made the expanding universe unavoidable. Once expansion was confirmed, Einstein reportedly called the cosmological constant his greatest blunder and removed it from the equations. The removal left a clean set of equations that described a universe that would expand and then, eventually, recollapse.
The 1998 data put it back. Whatever dark energy is — and no one knows what it is, which is the source of the philosophical wound that will not close — it behaves like Einstein’s cosmological constant. It is a property of space itself, a repulsive pressure that does not dilute as the universe expands, that grows more powerful as the volume of space grows, driving the acceleration of expansion in a feedback that becomes more pronounced with every passing eon. The universe is not merely expanding. It is running away from itself. Every galaxy is fleeing every other galaxy, and the velocity of that flight is increasing, and will continue to increase, indefinitely, until the distances between galaxies become so vast that no signal, no light, no gravitational wave, no evidence of any other galaxy’s existence can ever reach any other.
This is not, in any ordinary sense, a comfortable thing to know.
What the Underground Man Sees
Dostoevsky understood something about the psychic cost of absolute knowledge. His underground man does not suffer from ignorance — he suffers from an excess of awareness. He sees too clearly, thinks too carefully, and is therefore unable to act with the blunt confidence of a man who has not thought enough. The cosmologists of 1998 did not suffer from ignorance either. They had wanted a number, and they had found a number, and the number disclosed a universe that did not behave the way any of the intellectual frameworks governing their lives had suggested it should.
What is dark energy? It is the name we give to our ignorance. The phrase “dark energy” was coined by cosmologist Michael Turner in 1998, almost immediately after the supernova results were announced, as a placeholder for whatever was causing the acceleration. It is not dark in the sense of being hidden — it is dark in the sense of being opaque to understanding. We know what it does. We measure its effects with high precision. The cosmological constant value derived from these measurements is consistent across multiple independent lines of evidence: the cosmic microwave background, the large-scale structure of the universe, weak gravitational lensing surveys. The number holds. What it represents — what kind of physical entity could possess these properties — we do not know.
The theoretical predictions are, if anything, more troubling than the observational data. Quantum field theory, the most successful framework physics has for describing the fundamental forces, predicts a vacuum energy — an energy density of empty space arising from quantum fluctuations — that should be, depending on the calculation, somewhere between forty and one hundred and twenty orders of magnitude larger than the value actually observed. This discrepancy — between what quantum mechanics says dark energy should be and what cosmology observes it to be — is sometimes called the worst prediction in the history of physics. The two most successful theories in science produce numbers that are incompatible with each other by a margin so vast it makes the word “discrepancy” seem inadequate. “Catastrophe” is closer.
The Competing Teams and the Nobel
The Nobel Prize in Physics for 2011 was awarded jointly to Saul Perlmutter, Brian Schmidt, and Adam Riess for the discovery of the accelerating expansion of the universe through observations of distant supernovae. In their Nobel lectures, all three men spoke with characteristic precision and candor. Perlmutter discussed the carefulness of the analysis, the need to account for systematic errors, the months of checking before the teams were willing to commit to the result. Schmidt described the emotional texture of the period: his awareness that the result would be controversial, his own resistance to believing it.
What neither lecture addresses directly, because it cannot be addressed directly without stepping outside the conventions of scientific discourse, is the existential weight of the discovery for the people who made it. To return, after GW150914, to Monday morning lectures on introductory cosmology; to teach undergraduates the standard model of the universe while knowing that a major component of that model — dark energy, roughly 68% of the universe’s total energy content — is entirely opaque; to produce pedagogically adequate descriptions of a universe that the models of physics cannot explain: this is a particular condition. It is not despair. These are not men given to despair. But it is a condition of sustained intellectual tension that the broader culture does not usually attribute to scientists, preferring to imagine science as the place where questions find their answers, rather than the place where answers produce more grievous questions.
The Hubble Tension: The Wound That Opened Wider
The 1998 discovery created a standard cosmological model — ΛCDM, for Lambda (the cosmological constant) Cold Dark Matter — that has proven durable and well-confirmed over the subsequent decades. But it has also, in recent years, produced a specific anomaly that keeps cosmologists awake.
The Hubble constant — the rate at which the universe is currently expanding — can be measured two different ways. It can be inferred from the cosmic microwave background, the afterglow of the early universe, using the predictions of the standard model. And it can be measured directly, using distance ladder techniques: Cepheid variable stars, Type Ia supernovae, and other distance indicators that establish, rung by rung, the actual recession velocities of nearby galaxies. These two measurements consistently disagree. The CMB-derived value is lower. The direct measurements give a higher number. The discrepancy is small in absolute terms — roughly eight percent — but in the precision cosmology of 2024, it is statistically significant. It is not noise. It is not an error that more careful analysis will resolve.
Wendy Freedman at the University of Chicago has spent years working to refine the direct measurement side of the tension, exploring whether systematic errors in the Cepheid calibration could explain the discrepancy. Her work with the James Webb Space Telescope has suggested that some of the tension may be reduced by better calibration — but has not eliminated it. The tension persists. Whether it represents a systematic measurement error, a flaw in the standard cosmological model, or an indication of new physics beyond the cosmological constant — that question remains open.
The underground man of Dostoevsky does not receive resolution. He circles. He examines his situation from every angle, finds the irony in every escape route, and returns to his corner. The Hubble tension is, for cosmology, something like this: a problem that has been examined carefully from every angle, that resists simple resolution, that continues to suggest — without yet confirming — that the picture of the universe we assembled in 1998 is not quite right. Perhaps the cosmological constant is not truly constant. Perhaps dark energy evolves with time. Perhaps the standard model of cosmology, like Einstein’s blunder before it, contains something that will one day require revision.
Acceleration Into Loneliness
Here is what the 1998 discovery ultimately means for the very long future, and it is worth stating plainly, without anesthetic.
In the far future — over timescales of tens of billions of years — the accelerating expansion of the universe will carry the galaxies of the local group beyond the cosmological horizon of any other galaxy cluster. Light from those distant galaxies will never reach us. They will become, effectively, unreachable. A civilization in the far future, looking outward with the finest instruments conceivable, will see only local galaxies — the remnant of the local group — surrounded by an apparent void. The evidence for the Big Bang, for the cosmic microwave background, for the large-scale structure of the universe, will have redshifted into inaccessibility. A cosmologist a trillion years from now, reasoning from available evidence, would have no way to reconstruct the universe’s true history.
They would be, cosmologically speaking, alone in a way that no one alive today is alone. The universe’s acceleration is making the evidence of its own origin disappear. It is not merely expanding. It is erasing.
Richard Feynman once described the universe as containing two kinds of knowledge: the map of how things are, and the map of why things are. Physics has been building the first map for centuries. The second map — why the cosmological constant has the value it has, why dark energy behaves as it does, what its ultimate nature is — remains blank. We have measured the acceleration of a universe we do not understand into a future we cannot inhabit. The cosmologists who found it were not broken by the discovery, not precisely. They went on. They taught on Monday. They published, verified, refined, and eventually accepted the prizes.
But they carry the knowledge with them. We all do, now. The universe is not moving toward warmth and reunion. It is fleeing. It is running from itself, and us with it, into an accelerating silence that no future telescope will have the range to illuminate. That is what the data say. That is the testimony the standard candles gave, in 1998, when two competing teams looked for a deceleration and found instead the face of something they had no name for yet.
They named it dark energy. The naming did not help.
For more on the philosophical dimensions of cosmology and the mathematics of universal structure, see the review of Just Six Numbers by Martin Rees, which addresses, with characteristic sobriety, the cosmological constants on which the observable universe depends. And the E=mc² review traces the groundwork from which all of this follows.
You Might Also Like
- The Fabric of Reality by David Deutsch — Knowledge Has No Ceiling, and That Should Change Everything
- Relativity: The Special and the General Theory by Albert Einstein — A Review
Sources
- Perlmutter, S. et al. “Measurements of Omega and Lambda from 42 High-Redshift Supernovae.” The Astrophysical Journal 517 (1999): 565–586. https://iopscience.iop.org/article/10.1086/307221
- Riess, A.G. et al. “Observational Evidence from Supernovae for an Accelerating Universe and a Cosmological Constant.” The Astronomical Journal 116 (1998): 1009–1038.
- Nobel Prize Committee. “The Nobel Prize in Physics 2011.” Awarded to Saul Perlmutter, Brian P. Schmidt, and Adam G. Riess. Nobel Lecture transcripts. https://www.nobelprize.org/prizes/physics/2011/summary/
- Freedman, Wendy L. “Measurements of the Hubble Constant: Tensions in Perspective.” The Astrophysical Journal 919 (2021): 16.
- Turner, Michael S. “Dark Energy and the New Cosmology.” Contribution to the SAAS-Fee Lectures, 2001. https://arxiv.org/abs/astro-ph/0108103
- Planck Collaboration. “Planck 2018 results: Cosmological parameters.” Astronomy & Astrophysics, 2020. https://www.aanda.org/articles/aa/abs/2020/09/aa33910-18/aa33910-18.html
