Previously, I have written two blog articles (part 1 about thermal ensembles and part 2 about non-thermal ensembles) about a formalism of statistical ensembles. I will be using it as the formalism of classical statistical mechanics in this article.

In that formalism, the space of microstates of a system is a measure space $\mcal M$, and the physical meaning of the measure is the number of microstates. A macrostate is described by the extensive quantities, which are a function of the microstate, so designating a macrostate restricts the microstates that can realize it to a subset of $\mcal M$.

A state of the system is a probability density function $p$ on $\mcal M$, whose physical meaning is an ensemble of microstates. The macroscopically measured extensive quantities of the system are defined to be the ensemble average, i.e., the measured value of $A:\mcal M\to\bR$ is $\int_{\mcal M}pA$. Generally, any probability density function is a perfectly valid state, but the most important ones are those that are thermal equilibrium states, including the microcanonical ensembles, the thermal ensembles, and the non-thermal ensembles. The term about an ensemble being thermal or non-thermal is made up by me, but for most practical reasons, we only need to focus on thermal ensembles (because both canonical ensembles and grand canonical ensembles are thermal ensembles).

To avoid subtleties about measure theory and topology, in this article, we will only use counting measure and discrete spaces for the space of microstates and the space of extensive quantities.

However, in quantum mechanics, things get different because of the introduction of superpositions of states. For the superpositions to make sense, the space of microstates must be endowed with a vector space structure. By principles in quantum mechanics, it is the projective space of a separable Hilbert space $\mcal H$. A state of the system is then a density operator $\rho$ on $\mcal H$, which can be any positive semi-definite self-adjoint operator with trace $1$. This is quite different from a state in the classical case because we cannot simply interpret a density operator as an ensemble of microstates. Generally, we can have different ensembles that realize the same density operator. All those different ensembles are just equally physically valid (without further contexts) due to the Schrödinger–HJW theorem.

Extensive quantities are self-adjoint operators on $\mcal H$. This leads to a key difference between classical and quantum statistical mechanics: in quantum statistical mechanics, a microstate generally does not have a definite macrostate, except for the case when it is an eigenstate of all the extensive quantities. However, we can still define macroscopically measured extensive quantities for any state of the system, being $\Tr\rho A$ for any self-adjoint operator $A$.

The fact that only the states in the eigenspace of all the extensive quantities have a definite macrostate imposes a challenge on defining the microcanonical ensemble (to clarify, I am referring to the density operator, which does not define a particular ensemble, but I am still using “microcanonical ensemble” to refer to that state). It may not be possible to define a microcanonical ensemble for every possible combinations of values of the extensive quantities (in their spectra). In practice, one would restrict to only consider mutually commuting operators as the extensive quantities. Then, the microcanonical ensemble density operator is the projection operator onto the common eigenspace (properly normalized to have trace $1$). The statistical mechanics is then actually equivalent to the classical statistical mechanics (namely taking the eigenbasis of the extensive quantities as the classical space of microstates)! Unfortunately, this is not the way typically used in practice because it is not always practical to find the eigenbasis.

To avoid mathematical subtleties, we will mostly only consider finite-dimensional Hilbert spaces.

Here is a summary table:

Many-body systems

We then want to ask: if the space of microstates for one particle is $\mcal M$ (or $\mcal H$), what is the space of microstates for many particles? The answer depends on whether the particles are distinguishable particles, indistinguishable fermions, or indistinguishable bosons.

There are two aspects in which fermions and bosons contrasts with each other. One is their symmetry properties: fermions are antisymmetric under exchange of particles, and bosons are symmetric. The other is their statistical properties: fermions obey the Pauli exclusion principle, and the bosons do not. The second property naturally leads us to work with Fock states, which can be derived from the first property after second quantization. In this article, a third kind of particles, distinguishable particles, will also be considered. They are neither symmetric nor antisymmetric under exchange of particles, but exchanging particles actually gives a new state.

The whole idea of these different kinds of particles is very easy to describe in quantum mechanics. If the microstates of each particle live on $\mcal H$, then the microstates of many distinguishable particles live on the tensor algebra $\fc T{\mcal H}$; those of many bosons live on the symmetric algebra $\fc S{\mcal H}$; and those of many fermions live on the exterior algebra $\fc\bigwedge{\mcal H}$. Those spaces are called Fock spaces. They are naturally graded, so the particle number operator can be defined by defining the $N$-grade subspace of the Fock space to be the eigenspace associated with the eigenvalue $N$.

Taking ideas from the Fock basis in quantum mechanics, we can similarly discuss those different kinds of particles in classical statistical mechanics. If the microstates of each particle are $\mcal M$, then the microstates of many distinguishable particles are tuples $\bigcup_{N\in\bN}\mcal M^N$; those of many bosons are finite multisets in the universe $\mcal M$, i.e., $\set{m:\mcal M\to\bN}{\sum m<\infty}$; and those of many fermions are finite subsets of $\mcal M$, i.e., $\fc{\mcal P_{<\aleph_0}}{\mcal M}$. Those concepts, namely tuple, multiset, and set, are actually common mathematical constructs used in combinatorics. They all have a natural notion of size, which we define the number of particles to be.

I previously stated that there is an equivalence between quantum and classical statistical mechanics. Here, necessarily for the equivalence to hold, the dimension of the $N$-particle subspace in the Fock space (when it is finite) must be the same as the cardinality of the $N$-particle microstates in the classical case, and this is indeed the case. Assume that $\dim\mcal H=\card\mcal M=M$, then both the dimension of the subspace of $N$ distinguishable particles and the number of classical microstates of $N$ distinguishable particles are $M^N$. This number for bosons is $M^{\overline N}/N!$, and that for fermions is $M^{\underline N}/N!$, where $$M^{\overline N}\ceq\prod_{M\le k<M+N}k,\quad M^{\underline N}\ceq\prod_{M-N<k\le M}k$$ are called the rising factorial power and the falling factorial power respectively. These are the number of ways to put $N$ balls into $M$ boxes under three different rules.

Here is a summary table:

Gibbs factor and entropy

Gibbs put the famous factor of $1/N!$ in front of the phase space integral of the ideal gas to make the entropy asymptotically linear in $N$. People often interpret this as accounting for the indistinguishability of particles so that the result of classical treatment can match with the quantum treatment.

Actually, the effect of the Gibbs factor may not be as important as you imagined. In the microcanonical and the canonical ensemble, the Gibbs factor is just an overall factor for the partition function. The only effect is that the chemical potential would not be intensive and that the entropy would not be extensive without it, but there is no actual physical consequence of this because we cannot measure the entropy and the chemical potential in experiments anyway. In the grand canonical ensemble, the distribution of the number of particles is expected to be different with or without the Gibbs factor. However, at least for the ideal gas example (or more generally, for models with a quadratic Hamiltonian), the equipartition theorem and the ideal gas law would still hold without the Gibbs factor. Consider the grand canonical partition function of the ideal gas, whether we include the Gibbs factor or not: $$\Xi_1\ceq\sum_N\fr1{N!}\p{\fr{\e^{\beta\mu}V}{\lmd^d}}^N,\quad \Xi_2\ceq\sum_N\p{\fr{\e^{\beta\mu}V}{\lmd^d}}^N,$$ where $\lmd\ceq\sqrt{\beta h^2/2\pi m}$. If you spend the time to actually do the calculation, you can get the desired $pV=\a N/\beta$ and $\a E=d\a N/2\beta$, whether you include the Gibbs factor or not. The entropy and the chemical potential would indeed change drastically with the introduction of the Gibbs factor, but they are not actually measurable quantities in experiments.

This then raises questions. Does the entropy have to be linear in $N$? In other words, does the entropy need to meet the traditional sense of extensivity? Does physics actually care about our definition of the entropy? The answer to these questions is actually no. The entropy is not something that we can directly measure in experiments, and there are some freedom in the definition of the entropy that does not affect any physical outcomes.

Now, recall that the Gibbs factor accounts for the indistinguishability of particles. This would mean that whether the particles are actually distinguishable or not does not matter the actual physics. Gas particles in real life may well be distinguishable. For example, chlorine has two stable isotopes that naturally occur with considerable abundance, and that does not make it substantially different from, say, fluorine, which has only one stable isotope. Maybe people will also find observable features in fluorine molecules that would make them distinguishable, who knows? That would not deny any of the experimentally tested thermodynamic theories that can be applied to fluorine today.

Therefore, the Gibbs factor should not be introduced in the sole purpose of accounting for the indistinguishability of particles. It is introduced to make the entropy traditionally extensive. However, as I already stated, it is not necessary for the actual physics, so why is it important to make the entropy extensive? The answer is that, otherwise, the free energy (be it the Helmholtz energy or the Gibbs energy) would not be extensive. The free energy measures the work that can be extracted from the system, and by this nature it must be extensive because energy is additive. Therefore, only when we define the entropy in a way such that it is extensive, can it possibly make the derived free energy be able to measure the extractable work.

Having the idea that the free energy measures the amount of work that can be extracted from the system, we would then think we are able to extract some work out of the process of mixing two distinguishable gases. This is because distinguishability gives rise to a mixing entropy, which is the whole reason why it makes the entropy fail to be traditionally extensive. On the other hand, as I stated, whether we regard the two gases distinguishable or not in theory, it does not matter the actual physics. However, the amount of work that can be extracted from the process of mixing two gases is very physical by any means. To resolve this, the take is that, if it is possible to extract work from mixing them in one’s theory, then it should also be possible to distinguish the gases in their theory. On the other hand, if the two gases are indistinguishable in one’s theory, then it is impossible to extract work from mixing them in their theory. Therefore, it actually does not matter whether the gases are “in reality” distinguishable or not, the theory would be able to make itself consistent. The texts about the mixing paradox on Wikipedia explain this idea, which is a gist of the paper (which unfortunately did not talk about the grand canonical ensemble in detail).

Another importance for the entropy to be extensive is that only then can different ensembles be thermodynamically equivalent. The thermodynamical equivalence is the property that the thermodynamic properties determined from the characteristic functions (e.g., entropy, Helmholtz energy, and grand potential) of different statistical ensembles are the same in the thermodynamic limit. This is not a sufficient condition, though, because we also need to require that the entropy is a concave function of the extensive quantities. There is a good paper that explains the equivalence and nonequivalence of ensembles in detail, assuming the characteristic functions are always extensive. The main idea is that, for any statistical ensemble, the probability measure on the space of macrostates, parametrized by the particle number $N$, satisfies the large deviation principle with the rate function being the characteristic function. With the concavity condition, using a generalization of Laplace’s method, it can then be proven that the characteristic functions of different ensembles are related as being the Legendre transform of each other.

Gibbs factor and indistinguishability

Why can the introductiong of the Gibbs factor account for the indistinguishability?

Define $\fc{\Omg^0}{M,N}\ceq M^N$ to be the number of microstates of $N$ distinguishable particles with $M$ single-particle microstates. Then, define $\fc{\Omg_0}{M,N}\ceq\fc{\Omg^0}{M,N}/N!$ to be the version with the Gibbs factor. Also, define $\fc{\Omg_\pm}{M,N}\ceq M^{\overline{\underline N}}/N!$ for bosons and fermions, where $M^{\overline{\underline N}}$ is $M^{\overline N}$ or $M^{\underline N}$ corresponding to $+$ or $-$ in the notation “$\pm$” respectively.

If we make the distinguishable particles indistinguishable, we have to characterize them as either bosons or fermions. However, $\Omg_0$ and $\Omg_\pm$ are not exactly the same, This discrepancy can be resolved in the large $M$ limit. We have $$\begin{align*} \fc{\Omg_\pm}{M,N}&=\fr1{N!}\prod_{k=0}^{N-1}\p{M\pm k} =\fr{M^N}{N!}\prod_{k=0}^{N-1}\p{1\pm\fr kM}\\ &=\fc{\Omg_0}{M,N}\p{1\pm\fr{N\p{N-1}}{2M}+\order{M^{-2}}}, \end{align*}$$ where the big-O notation is understood as $N$ fixed and $M\to\infty$. Therefore, to have $\Omg_0\approx\Omg_\pm$, loosely speaking, we need $M\gg N^2$. In this limit, there is no difference between boson statistics and fermion statistics, and both of them are the same as distinguishable particles with the Gibbs factor.

Intuitively, if $M$ is very large, then in most of the microstates, each single-particle microstate is occupied by at most one particle, which renders boson statistics and fermion statistics the same. Particularly, if there are infinitely many single-particle microstates, then $M$ is effectively infinite, so $\Omg_0=\Omg_\pm$ is strictly true for any $M$ in this case. This is why the result for classical ideal gas is exact: there are so many single-particle microstates that the probability for two particles to occupy the same microstate is exactly zero, i.e., such microstates have zero measure.

Classical Fermi gas

I previously said that, to make things simple, the measure on $\mcal M$ would be the counting measure. One big reason behind that is the difficulty of a purely classical description of the Fermi gas.

In the classical description of a gas, the microstates of each particle are points in the $2d$-dimensional phase space, which is a region in $\bR^{2d}$, and the measure is just the usual Lebesgue measure (or any other practically equivalent measure, for math nerds). Therefore, naturally, the microstates of many particles with particle number $N$ would be a region in $\bR^{2dN}$, also equipped with the usual Lebesgue measure.

If the gas consists of fermions, then in any microstate, two particles cannot be in the same single-particle microstate. However, the set of all microstates that have two such particles has zero measure in the $2dN$-dimensional phase space. Therefore, it just would not matter at all whether the particles are fermions or not in the classical description.

However, we know that is not the actual case. In practice, we divide the single-particle phase space into cells of size $h^{d}$, where $h$ is the Planck constant, which we put here by hand. No two particles can reside in the same cell. Therefore, any “bulky” region in the single-particle phase space with volume $\Omg$ cannot contain more than $\Omg/h^{d}$ particles.

This all sounds fine, except that we did not define what a “bulky” region is. Of course, the Fermi sea is a bulky region, but what about a tube that is long enough to connect any specified discrete points in the single-particle phase space but is thin enough to have volume even smaller than $h^{d}$? In fact, by constructing regions with not-so-exotic shapes, we can make any distributino of particles in the single-particle phase space seem like it is violating the Pauli exclusion principle or not arbitrarily. Just shown in the figure below, particles that reasonably distribute in different cells may be regarded as being in one cell, while particles that reasonably occupy the same cell may be regarded as being in different cells.

There are some possible ways to resolve this issue. One naive way is to stipulate that the cell arranges in some lattice structure such as the simple cube lattice. However, this will break the rotational symmetry in the phase space so that the Fermi sea will not be strictly isotropic anymore. Also, the introduction of the lattice structure changes the physics of the system if it is far from the thermodynamic limit. Only in the thermodynamic limit will the particular choice of lattice structure be irrelevant to the physics.

Another way is to consider a phase space density functional theory, where a function $\rho$ is defined on the single-particle phase space, representing the number of particles in unit volume in the phase space. The measure of the number of microstates for the many-body system is then the functional integral of this density function. The Pauli exclusion principle can then be translated into the constraint that the value of $\rho$ must not exceed $1/h^{d}$ anywhere, which prevents the number of particles in any region of size $h^{d}$ from exceeding $1$. It can also describe bosons by removing this constraint. I have not explored this approach myself, but I doubt it would be a good idea because it seems like an overkill to the problem and will introduce even more mathematical subtleties with the functional integral. Also, more careful analysis must be done to devise the proper measure on the functional space to match the usual sense of number of microstates. Another issue is that it defies the classical notion of particles as clear points but instead treats them as cloudy distributions just like quantum mechanics, and by this very reason it is not capable of being generalized to describe distinguishable particles.

When $M$ is not very large, using the Gibbs factor is then not a correct way to account for indistinguishability. However, it can be corrected, as long as we use $\fc{\Omg^\pm}{M,N}\ceq M^{\overline{\underline N}}$ instead of $\fc{\Omg^0}{M,N}$. Then, we would have $\fc{\Omg^\pm}{M,N}/N!=\fc{\Omg_\pm}{M,N}$ exactly, corresponding to boson statistics and fermion statistics. There are indeed combanitorics problems of putting distinguishable balls into boxes that results in $\fc{\Omg^\pm}{M,N}$. Actually, $\fc{\Omg^+}{M,N}$ is the number of ways to put $N$ distinguishable balls into $M$ boxes with the balls in each box being ordered; $\fc{\Omg^-}{M,N}$ is the number of ways to put $N$ distinguishable balls into $M$ exclusive boxes (“exclusive” means that each box cannot contain more than one ball).

Now, $\Omg^\pm$ represents two more different rules under which we put balls into boxes. Together with $\Omg_0$ and $\Omg_\pm$, there are five different rules in total. We can summarize them into a table:

These are all common enumerative problems of putting balls into boxes in combinatorics. One can extend this table by including more different enumerative problems. There is such a table called the twentyfold way that lists 20 different enumerative problems.

If there are 200 typographical errors randomly distributed in a 500 page manuscript, find the probability that a given page contains exactly 3 errors.

We can abstract this type of problems as follows:

Suppose there are $n$ distinguishable boxes and $k$ indistinguishable balls. Now, we randomly put the balls into the boxes. For each of the boxes, what is the probability that it contains $m$ balls?

For example, if the first page contains 3 errors, the second page contains 197 errors, and the rest of the pages contain no errors, then the situation corresponds to the situation where the first box contains 3 balls, the second box contains 197 balls, and the rest of the boxes contain no balls. The balls are indistinguishable because we can only determine how many errors are on each page but not which errors are on the page.

To deal with the problem, we simply need to find these two numbers:

• the number of ways to put $k$ indistinguishable balls into $n$ distinguishable boxes, and
• the number of ways to put $k-m$ indistinguishable balls into $n-1$ distinguishable boxes.

The latter corresponds to the number of ways to put the balls into the boxes provided that we already know that the given box contains $m$ balls. After we find these two numbers, their ratio is the probability in question.

To find the number of ways to put $k$ indistinguishable balls into $n$ distinguishable boxes, we can use the stars and bars method. To see this, we write a special example. Here is an example of $n=4$ and $k=6$: $${}|{}\star{}\star{}|{}\star{}|{}\star{}\star{}\star{},$$ which corresponds to the distribution $0,2,1,3$. We can see that there are $n-1$ bars and $k$ stars. Therefore, the number of ways to put the balls is the same as the number of ways to choose the $k$ positions of the stars among $n+k-1$ positions. Therefore, the number of ways is $$N_{n,k}=\binom{n+k-1}{k}=\frac{\left(n+k-1\right)!}{k!\left(n-1\right)!}.$$ Therefore, the final probability of the given box containing $m$ balls is $$P_{n,k}(m)=\frac{N_{n-1,k-m}}{N_{n,k}} =\frac{\left(n-1\right)k!\left(n+k-m-2\right)!}{\left(k-m\right)!\left(n+k-1\right)!}.$$

Another easy way to derive this result is by using the generating function. The number $N_{n,k}$ is just the coefficient of $x^k$ in the expansion of the generating function $\left(1+x+x^2+\cdots\right)^n$. The generating function is just $\left(1-x\right)^{-n}$, which can be easily expanded by using the binomial theorem.

We are now interested in the limit $n,k\to\infty$ with $\lambda\coloneqq k/n$ fixed. By Stirling’s approximation, we have $$P_{n,k}(m)\sim\left(n-1\right) \frac{k^{k+1/2}\left(n+k-m-2\right)^{n+k-m-2+1/2}}{\left(k-m\right)^{k-m+1/2}\left(n+k-1\right)^{n+k-1+1/2} } \mathrm e^{k-m+n+k-1-k-n-k+m+2}.$$ The $1/2$’s in the exponents can just be dropped because you may find that if we extract the $1/2$’s, the factor tends to unity. The exponential is just constant $\mathrm e$. Therefore, we have $$\begin{align*} P_{n,k}(m)&\sim\left(n-1\right) \frac{\left(\lambda n\right)^{\lambda n}\left(n+\lambda n-m-2\right)^{n+\lambda n-m-2} } {\left(\lambda n-m\right)^{\lambda n-m}\left(n+\lambda n-1\right)^{n+\lambda n-1}}\mathrm e\\ &=\left(\tfrac{n+\lambda n-m-2}{n+\lambda n-1}\right)^n \left(\tfrac{\left(n+\lambda n-m-2\right)\lambda n}{\left(\lambda n-m\right)\left(n+\lambda n-1\right)}\right)^{\lambda n} \left(\tfrac{\lambda n-m}{n+\lambda n-m-2}\right)^m \tfrac{\left(n-1\right)\left(n+\lambda n-1\right)}{\left(n+\lambda n-m-2\right)^2}\mathrm e\\ &\to\mathrm e^{-\frac{m+1}{\lambda+1}}\,\mathrm e^m\, \mathrm e^{-\frac{m+1}{\lambda+1}\lambda}\left(\tfrac\lambda{\lambda+1}\right)^m\tfrac1{\lambda+1}\mathrm e\\ &=\left(\tfrac\lambda{\lambda+1}\right)^m\tfrac1{\lambda+1}. \end{align*}$$ This is just the geometric distribution with parameter $p=1/(\lambda+1)=n/(k+n)$.

If you want to simulate the number of balls in a box, here is a simple way to do this. First, because each box is the same, we can just focus on the first box without loss of generality. Then, we just need to randomly generate the positions of the $n-1$ bars among the $n+k-1$ positions, and then return the index of the first bar (which is the number of balls in the first box).

We can then write the following Ruby code to simulate the number of balls in the first box:

          
            def simulate n, k

              (n-1).times.inject(npkm1 = n+k-1) { |bar, i| [rand(npkm1 - i), bar].min }

            end

          
        

Compare the simulated result with the theoretical result:

          
            def frequency m, n, k, trials

              trials.times.count { simulate(n, k) == m } / trials.to_f

            end

            

            def truth m, n, k

              (n-1) * (k-m+1..k).reduce(1,:*) / (n+k-m-1..n+k-1).reduce(1,:*).to_f

            end

            

            def approx m, n, k

              n*k**m / ((n+k)**(m+1)).to_f

            end

            

            srand 1108

            m, n, k = 3, 5000, 8000

            p frequency m, n, k, 10000 # => 0.0902

            p truth m, n, k # => 0.08965012972626446

            p approx m, n, k # => 0.08963271594131858

          
        

Definition 1. Let $$f_n\!\left(z\right)\coloneqq\left(1-z\right)^{n+1}\sum_{k=1}^\infty k^nz^k,$$ where $n$ is a positive integer.

Our goal is to prove that $f_n\!\left(z\right)$ is a polynomial of degree $n$ w.r.t. $z$, and the sum of its coefficients is $n!$.

Lemma 1. $$f_{n+1}\!\left(z\right)=z\,\left(1-z\right)^{n+2}\frac{\mathrm d}{\mathrm dz}\left(\frac{f_n\!\left(z\right)}{\left(1-z\right)^{n+1}}\right).$$

Proof. $$\begin{align*} \frac{\mathrm d}{\mathrm dz}\left(\frac{f_n\!\left(z\right)}{\left(1-z\right)^{n+1}}\right) &=\frac{\mathrm d}{\mathrm dz}\sum_{k=1}^\infty k^nz^k\\ &=\sum_{k=1}^\infty k^{n+1}z^{k-1}\\ &=\frac{f_{n+1}\!\left(z\right)}{z\,\left(1-z\right)^{n+2}}. \end{align*}$$ $\square$

Definition 2 (Eulerian numbers). $$\left<\begin{matrix}n\\k\end{matrix}\right>\coloneqq\sum_{j=0}^{k+1}\left(-1\right)^j\binom{n+1}j\left(k-j+1\right)^n.$$

Lemma 2. $$\left<\begin{matrix}n+1\\k+1\end{matrix}\right>=\left(n-k\right)\left<\begin{matrix}n\\k\end{matrix}\right>+\left(k+2\right)\left<\begin{matrix}n\\k+1\end{matrix}\right>.$$

Proof. $$\begin{align*} &\phantom{=}~\,\left(n-k\right)\left<\begin{matrix}n\\k\end{matrix}\right>+\left(k+2\right)\left<\begin{matrix}n\\k+1\end{matrix}\right>\\ &=\left(n-k\right)\sum_{j=0}^{k+1}\left(-1\right)^j\binom{n+1}j\left(k-j+1\right)^n+\left(k+2\right)\sum_{j=0}^{k+2}\left(-1\right)^j\binom{n+1}j\left(k-j+2\right)^n\\ &=\left(n-k\right)\sum_{j=0}^{k+2}\left(-1\right)^{j-1}\binom{n+1}{j-1}\left(k-j+2\right)^n+\left(k+2\right)\sum_{j=0}^{k+2}\left(-1\right)^j\binom{n+1}j\left(k-j+2\right)^n\\ &=\sum_{j=0}^{k+2}\left(-1\right)^j\left(k-j+2\right)^n\left(\left(k-n\right)\binom{n+1}{j-1}+\left(k+2\right)\binom{n+1}j\right)\\ &=\sum_{j=0}^{k+2}\left(-1\right)^j\left(k-j+2\right)^n\left(\left(k-n\right)\frac{\left(n+1\right)!}{\left(j-1\right)!\left(n-j+2\right)!}+\left(k+2\right)\frac{\left(n+1\right)!}{j!\left(n-j+1\right)!}\right)\\ &=\sum_{j=0}^{k+2}\left(-1\right)^j\left(k-j+2\right)^n\frac{\left(n+1\right)!}{j!\left(n-j+2\right)}\left(\left(k-n\right)j+\left(k+2\right)\left(n-j+2\right)\right)\\ &=\sum_{j=0}^{k+2}\left(-1\right)^j\left(k-j+2\right)^n\frac{\left(n+1\right)!}{j!\left(n-j+2\right)}\left(n+2\right)\left(k-j+2\right)\\ &=\sum_{j=0}^{k+2}\left(-1\right)^j\left(k-j+2\right)^{n+1}\frac{\left(n+2\right)!}{j!\left(n-j+2\right)}\\ &=\sum_{j=0}^{k+2}\left(-1\right)^j\binom{n+2}j\left(k-j+2\right)^{n+1}\\ &=\left<\begin{matrix}n+1\\k+1\end{matrix}\right>. \end{align*}$$ $\square$

Lemma 3. $$\left<\begin{matrix}n\\0\end{matrix}\right>=1,\quad\left<\begin{matrix}n\\n\end{matrix}\right>=0.$$

Brief proof. Easily proved by Definition 2. $\square$

Lemma 4. $$f_n\!\left(z\right)=\sum_{k=1}^n\left<\begin{matrix}n\\n-k\end{matrix}\right>z^k.$$

Proof. By mathematical induction. When $n=1$, $$f_n\left(z\right)=z=\sum_{k=1}^n\left<\begin{matrix}n\\n-k\end{matrix}\right>z^k,$$ the result holds.

Suppose the result holds when $n=n_0$, and then when $n=n_0+1$, $$\begin{align*} &\phantom{=}~\,f_n\!\left(z\right)\\ &=z\,\left(1-z\right)^{n_0+2}\frac{\mathrm d}{\mathrm dz}\left(\frac{f_{n_0}\left(z\right)}{\left(1-z\right)^{n_0+1}}\right) &\text{(Lemma 1)}\\ &=z\,\left(1-z\right)^{n_0+2}\frac{\frac{\mathrm df_{n_0}\left(z\right)}{\mathrm dz}\left(1-z\right)^{n_0+1}-f_{n_0}\!\left(z\right)\frac{\mathrm d\left(\left(1-z\right)^{n_0+1}\right)}{\mathrm dz}}{\left(1-z\right)^{2n_0+2}}\\ &=z\,\left(1-z\right)^{n_0+2}\frac{\frac{\mathrm df_{n_0}\left(z\right)}{\mathrm dz}\left(1-z\right)^{n_0+1}+\left(n_0+1\right)f_{n_0}\!\left(z\right)\left(1-z\right)^{n_0}}{\left(1-z\right)^{2n_0+2}}\\ &=z\left(\frac{\mathrm df_{n_0}\!\left(z\right)}{\mathrm dz}\left(1-z\right)+\left(n_0+1\right)f_{n_0}\left(z\right)\right)\\ &=z\left(\left(1-z\right)\frac{\mathrm d}{\mathrm dz}\sum_{k=1}^{n_0}\left<\begin{matrix}n_0\\n_0-k\end{matrix}\right>z^k+\left(n_0+1\right)\sum_{k=1}^{n_0}\left<\begin{matrix}n_0\\n_0-k\end{matrix}\right>z^k\right) &\text{(supposed)}\\ &=z\left(\left(1-z\right)\sum_{k=1}^{n_0}\left<\begin{matrix}n_0\\n_0-k\end{matrix}\right>kz^{k-1}+\left(n_0+1\right)\sum_{k=1}^{n_0}\left<\begin{matrix}n_0\\n_0-k\end{matrix}\right>z^k\right)\\ &=z\left(\sum_{k=1}^{n_0}\left<\begin{matrix}n_0\\n_0-k\end{matrix}\right>kz^{k-1}-\sum_{k=1}^{n_0}\left<\begin{matrix}n_0\\n_0-k\end{matrix}\right>kz^k+\left(n_0+1\right)\sum_{k=1}^{n_0}\left<\begin{matrix}n_0\\n_0-k\end{matrix}\right>z^k\right)\\ &=z\left(\sum_{k=0}^{n_0}\left<\begin{matrix}n_0\\n_0-k-1\end{matrix}\right>\left(k+1\right)z^k-\sum_{k=0}^{n_0}\left<\begin{matrix}n_0\\n_0-k\end{matrix}\right>kz^k+\left(n_0+1\right)\sum_{k=0}^{n_0}\left<\begin{matrix}n_0\\n_0-k\end{matrix}\right>z^k\right) &\text{(Lemma 3)}\\ &=z\sum_{k=0}^{n_0}\left(\left<\begin{matrix}n_0\\n_0-k-1\end{matrix}\right>\left(k+1\right)z^k-\left<\begin{matrix}n_0\\n_0-k\end{matrix}\right>kz^k+\left(n_0+1\right)\left<\begin{matrix}n_0\\n_0-k\end{matrix}\right>z^k\right)\\ &=\sum_{k=0}^{n_0}\left(\left<\begin{matrix}n_0\\n_0-k-1\end{matrix}\right>\left(k+1\right)-\left<\begin{matrix}n_0\\n_0-k\end{matrix}\right>k+\left(n_0+1\right)\left<\begin{matrix}n_0\\n_0-k\end{matrix}\right>\right)z^{k+1}\\ &=\sum_{k=1}^{n_0+1}\left(k\left<\begin{matrix}n_0\\n_0-k\end{matrix}\right>+\left(n_0-k+2\right)\left<\begin{matrix}n_0\\n_0-k+1\end{matrix}\right>\right)z^k\\ &=\sum_{k=1}^{n_0+1}\left<\begin{matrix}n_0+1\\n_0-k+1\end{matrix}\right>z^k &\text{(Lemma 2)}\\ &=\sum_{k=1}^n\left<\begin{matrix}n\\n-k\end{matrix}\right>z^k. \end{align*}$$ Then we can derive that the result is true by mathematical induction. $\square$

Lemma 5. $$\sum_{j=0}^n\left(-1\right)^{n-j}\binom njj^n=n!.$$

Proof. Because $\mathrm e^x-1\sim x$ (in terms of infinitesimal quantity), $\left(\mathrm e^x-1\right)^n\sim x^n$, i.e. $$\left(\mathrm e^x-1\right)^n=x^n+o\!\left(x^n\right)$$ (where $o$ denotes higher order of infinitesimal quantity).

Apply Newton’s binomial theorem to the left-hand side, and we have $$\sum_{j=0}^n\left(-1\right)^{n-j}\binom nj\mathrm e^{jx}=x^n+o\!\left(x^n\right).$$ Take $n$th derivative of the equation, and we have $$\sum_{j=0}^n\left(-1\right)^{n-j}\binom njj^n\mathrm e^{jx}=n!+o\!\left(1\right).$$ Take $x=0$, and we have $$\sum_{j=0}^n\left(-1\right)^{n-j}\binom njj^n=n!.$$ $\square$

Lemma 6. $$\sum_{j=0}^n\left(-1\right)^{n-j}\binom nj\left(j+1\right)^n=n!.$$

Proof. $$\begin{align*} \left(n+1\right)!&=\sum_{j=0}^n\left(-1\right)^{n-j+1}\binom{n+1}jj^{n+1}& \text{(Lemma 5)}\\ &=\sum_{j=1}^n\left(-1\right)^{n-j+1}\binom{n+1}jj^{n+1}\\ &=\sum_{j=1}^n\left(-1\right)^{n-j+1}\frac{\left(n+1\right)!}{j!\left(n-j+1\right)!}j^{n+1} \\ &=\sum_{j=1}^n\left(-1\right)^{n-j+1}\frac{\left(n+1\right)n!}{\left(j-1\right)!\left(n-j+1\right)!}j^n\\ &=\sum_{j=0}^n\left(-1\right)^{n-j}\frac{\left(n+1\right)n!}{j!\left(n-j\right)!}\left(j+1\right)^n\\ &=\left(n+1\right)\sum_{j=0}^n\left(-1\right)^{n-j}\binom nj\left(j+1\right)^n. \end{align*}$$ $\square$