In the chapter “Beyond this horizon,” Sean Carroll discusses two related problems involving properties of empty space. Before discussing the vacuum energy problem, he profiles the so-called hierarchy problem in the cosmic energy scale. It’s about the effects of virtual particles.

The energy scale that characterizes the weak interactions (the Higgs field value, 246 GeV) and the one that characterizes gravity (the Planck scale, 10^18 GeV) are extremely different numbers; that’s the hierarchy we’re referring to. This would be weird enough on its own, but we need to remember that quantum-mechanical effects of virtual particles want to drive the weak scale up to the Planck scale. — Carroll, Sean (2012-11-13). The Particle at the End of the Universe: How the Hunt for the Higgs Boson Leads Us to the Edge of a New World (Kindle Locations 3585-3588). Penguin Publishing Group. Kindle Edition.

To specify a theory like the Standard Model, you have to give a list of the fields involved (quarks, leptons, gauge bosons, Higgs), but also the values of the various numbers that serve as parameters of the theory. … Essentially, we need to add up different contributions from various kinds of virtual particles to get the final answer. … If we measure a parameter to be much smaller than we expect, we declare there is a fine-tuning problem, and we say that the theory is “unnatural.” … For the most part, the parameters of the Standard Model are pretty natural. There are two glaring exceptions: the value of the Higgs field in empty space, and the energy density of empty space, also known as the “vacuum energy.” Both are much smaller than they have any right to be. … This giant difference between the expected value of the Higgs field in empty space and its observed value is known as the “hierarchy problem.” — Ibid (Kindle Locations 3557-3585).

Wiki describes the problem this way:

A hierarchy problem occurs when the fundamental value of some physical parameter, such as a coupling constant or a mass, in some Lagrangian is vastly different from its effective value, which is the value that gets measured in an experiment. This happens because the effective value is related to the fundamental value by a prescription known as renormalization, which applies corrections to it. Typically the renormalized value of parameters are close to their fundamental values, but in some cases, it appears that there has been a delicate cancellation between the fundamental quantity and the quantum corrections. Hierarchy problems are related to fine-tuning problems and problems of naturalness.

In particle physics, the most important hierarchy problem is the question that asks why the weak force is 10^24 times as strong as gravity. … More technically, the question is why the Higgs boson is so much lighter than the Planck mass (or the grand unification energy, or a heavy neutrino mass scale) … In a sense, the problem amounts to the worry that a future theory of fundamental particles, in which the Higgs boson mass will be calculable, should not have excessive fine-tunings.

A fascinating situation — the need to essentially “hand code” (manually tune) values of (input) parameters in mathematical models in order to obtain practical solutions. [1] The problem arises from the use of perturbation theory and renormalization.

Wiki:

Renormalization is a collection of techniques in quantum field theory, the statistical mechanics of fields, and the theory of self-similar geometric structures, that are used to treat infinities arising in calculated quantities by altering values of quantities to compensate for effects of their self-interactions.

For example, a theory of the electron may begin by postulating a mass and charge. However, in quantum field theory this electron is surrounded by a cloud of possibilities of other virtual particles such as photons, which interact with the original electron. Taking these interactions into account shows that the electron-system in fact behaves as if it had a different mass and charge. Renormalization replaces the originally postulated mass and charge with new numbers such that the observed mass and charge matches those originally postulated.

Renormalization specifies relationships between parameters in the theory when the parameters describing large distance scales differ from the parameters describing small distances. Physically, the pileup of contributions from an infinity of scales involved in a problem may then result in infinities. … Renormalization procedures are based on the requirement that certain physical quantities are equal to the observed values.

Today, the point of view has shifted: on the basis of the breakthrough renormalization group insights of Nikolay Bogolyubov and Kenneth Wilson, the focus is on variation of physical quantities across contiguous scales, while distant scales are related to each other through “effective” descriptions. All scales are linked in a broadly systematic way, and the actual physics pertinent to each is extracted with the suitable specific computational techniques appropriate for each. Wilson clarified which variables of a system are crucial and which are redundant. [Cf. Sean Carroll’s perspective.]

When developing quantum electrodynamics in the 1930s, Max Born, Werner Heisenberg, Pascual Jordan, and Paul Dirac discovered that in perturbative corrections many integrals were divergent.

One way of describing the perturbation theory corrections’ divergences was discovered in the 1946–49 by Hans Kramers, Julian Schwinger, Richard Feynman, and Shin’ichiro Tomonaga, and systematized by Freeman Dyson in 1949. The divergences appear in radiative corrections involving Feynman diagrams [see Figure 1 in article] with closed loops of virtual particles in them. [Which Lederman discusses in his book — see Note 3.]

So, I’m more attentive to terms like coupling (coupling strength) and binding, as well as remarks about complex dynamics of valence and virtual particles. There’s another hill to climb: quantum corrections.

Wiki:

In quantum mechanics, perturbation theory is a set of approximation schemes directly related to mathematical perturbation for describing a complicated quantum system in terms of a simpler one. The idea is to start with a simple system for which a mathematical solution is known, and add an additional “perturbing” Hamiltonian representing a weak disturbance to the system. If the disturbance is not too large, the various physical quantities associated with the perturbed system (e.g. its energy levels and eigenstates) can be expressed as “corrections” to those of the simple system. These corrections, being small compared to the size of the quantities themselves, can be calculated using approximate methods such as asymptotic series. The complicated system can therefore be studied based on knowledge of the simpler one.

[1] There is a similar need in many types of engineering problems — specifying boundary conditions (constraints) and properties of materials for each context.

[2] Wiki

Perturbation theory is an important tool for describing real quantum systems, as it turns out to be very difficult to find exact solutions to the Schrödinger equation for Hamiltonians of even moderate complexity. The Hamiltonians to which we know exact solutions, such as the hydrogen atom, the quantum harmonic oscillator and the particle in a box, are too idealized to adequately describe most systems. Using perturbation theory, we can use the known solutions of these simple Hamiltonians to generate solutions for a range of more complicated systems.

The expressions produced by perturbation theory are not exact, but they can lead to accurate results as long as the expansion parameter, say α, is very small. Typically, the results are expressed in terms of finite power series in α that seem to converge to the exact values when summed to higher order. After a certain order n ~ 1/α however, the results become increasingly worse since the series are usually divergent (being asymptotic series).

In the theory of quantum electrodynamics (QED), in which the electron–photon interaction is treated perturbatively, the calculation of the electron’s magnetic moment has been found to agree with experiment to eleven decimal places. In QED and other quantum field theories, special calculation techniques known as Feynman diagrams are used to systematically sum the power series terms.

[3] Lederman, Leon M.; Hill, Christopher T. (2011-11-29). Symmetry and the Beautiful Universe. Prometheus Books. Kindle Edition.

The real power of Feynman diagrams is that we can systematically compute physical processes in relativistic quantum theories to a high degree of precision. This comes from what we call the quantum corrections, or the so-called higher-order processes. In figure 29, we show the second-order quantum corrections to the scattering problem of two electrons. This is a set of diagrams, each of which must be computed in detail and then added together including the previous diagram in figure 27, to get the final total result for the T-matrix. (The T-matrix, as noted above, is essentially the quantum version of the potential energy between the electrons and describes the scattering process.) This gives the total T-matrix to a precision of about 1/ 10,000. We can then go to the third order of higher quantum corrections to try to get even more precise agreement with experiment. Third-order calculations, however, are extremely difficult and very tiring for theoretical physicists. — Ibid (p. 252).

In the second-order processes of figure 29 we now see the appearance of the “loop diagrams.” The first diagram contains a loop representing a particle and antiparticle being spontaneously produced and then reannihilating. They contain a looping flow of the particle’s momentum and energy. Here we must sum up all possible momenta and energies that can occur in the loops, such that the overall incoming and outgoing energy and momentum is conserved. The Feynman loops present us with a new problem that has bothered physicists in many different ways for many years: put simply, when we compute the loop sums for certain loop diagrams, we get infinity! The processes we compute seem to become nonsensical. The theory seems to crash and burn. However, as the loop momenta become larger and larger, the loop is physically occupying a smaller and smaller volume of space and time, by the quantum inverse relationship between wavelength (size) and momentum. So in fact, we can only sum up the loop momenta to some large scale, or correspondingly, down to some small distance scale of space, for which we still trust the structure of our theory. … Interpreted properly, the loop diagrams actually tell us how to examine the physics at different distance scales, as though we have a theoretical microscope with a variable magnification power. — Ibid (pp. 253-254). [Cf. Carroll’s and Wilson’s use of effective theory.]