The culmination of the observations of the stars and increased understanding of the distances and brightness of stars lead to the relation between brightness of stars and their colors as described by the Hertzsprung-Russell (HR) diagram. The color of stars is related to their temperatures, and hence the temperature of the star is related to its brightness. Of course the detailed understanding of the HR diagram took a very long time. And the process is still continuing. The detailed physical processes that explain various features of the diagram are fairly complicated and for the most part not entirely understood. What follows is a whirlwind, no frills tour of the HR diagram. We will return to each piece in more detail later.
A star is born from a collapsing cloud of gas. Forget for the moment
where this gas came from and how it got to the stage where it could
collapse, we will get into that later. Let us say we walked into the
hall when the film showing star formation has already rolled to that
time. In the solar system the Sun is trying to pull all the planets to
itself. They on the other hand manage to stay out of the Sun by
constantly keeping moving. However if they move too fast the Sun will
no longer be able to tie them down and they'd escape its clutches
altogether. If they move just fast enough they can keep circulating
the Sun indefinitely. That is what is happening in a ball of gas. The
densest part of the gas cloud in its center acts like the Sun trying
to pull the rest of the cloud to the middle and make the whole cloud
smaller. But the rest of the cloud is desperately trying to keep
moving and so out the clutches of the center of the cloud. In the case
of the solar system the planets can move unassailed because they move
through empty space. In the gas cloud on the other hand huge numbers
of atoms, molecules, nuclei and electrons are crowded together and
constantly run into each other. This means that the constant fast
motion of these ingredients appears as a random motion of the
particles. Temperature is a measure of the energy, also called heat,
in this random motion of the particles of the gas. So a gas (or any
fluid) with its constituent particles in fast random motion is hot
where as a fluid with its constituent particles in slow random motion
is cold. From experience we know that hot objects want to expand. If
the random motion is fast, i.e., the gas is hot, the constituent
particles are more likely to get further away from each other than
when it is cold. Within a gas cloud thus there is a constant struggle
between the forces of gravity, that are trying to make the cloud
smaller and denser, and the forces of heat that are trying to make the
cloud bigger and less dense. If the cloud gets too massive the forces
of gravity can overcome the forces of heat and can force the cloud to
collapse. As the cloud collapses it heats up. If the mass of the cloud
is not too high, this increased temperature can arrest the
collapse. However if the mass of the cloud is above a certain critical
mass, called the Jeans' mass (after Sir James Jeans) its collapse
becomes irresistible. The collapse continues and the heat builds up
within the cloud, which by now is a proto-star. This frustrated heat,
overcome by gravity, makes the temperature in the center of the cloud
rise until it becomes so dense and hot that it can start burning
hydrogen. At this moment a star is born. At the stage of a proto-star,
it is much too cool to radiate in the visible range. And this is why
new infra-red detectors were needed before proto-stars were
discovered, as that is where they are most prominent. Because they are
large compared to stars they are bright even though they are
cool. Remember the luminosity of a star goes as its radius to the
second power and its temperature to the fourth power, 
. As the star collapses it moves left in the HR
diagram, as it gets hotter. It also rises in luminosity and hence
moves up in HR diagram. At some point the center gets so hot that the
outer layers get very thin and stop falling, and even get blown off in
case of heavier stars. At this point the proto-star drops in the HR
diagram growing dimmer (in case of heavier stars drops and then moves
sideways first growing dimmer than then getting hotter at the same
brightness, as the center starts to produce copious amounts of energy)
and settles onto what is called the zero-age Main Sequence (ZAMS)
where they become bona-fide stars generating energy by nuclear
fusion. The time scale for this is a few hundred thousand years.
All stars burn hydrogen in their centers to produce helium when they first start out in the ZAMS. As they continuously burn up hydrogen in the center and leave helium ash in the center the burning layer moves outward. As this happens stars of all masses gradually move above the ZAMS. At some stage there is an event that differentiates light stars from the heavier ones. Here heavier stars mean stars with mass greater than that of the Sun and lighter stars are those with mass equal to Sun's or less. This event is the nature of the start of helium burning. The heavier stars start burning helium after about 20 to a 100 million years of burning hydrogen. The heavier the star, the quicker it starts burning helium. Just before this the heavier stars move rapidly (in about half to a couple million years) to the right of the HR diagram at the same brightness as the hydrogen in the center is exhausted and hydrogen burns in a shell around the central helium ash. This burning hydrogen shell squeezes the central helium core increasing its temperature and density. At some point the central helium gets hot enough to ignite. The star burns helium (with the hydrogen burning proceeding as before in a shell outside the center burning hydrogen), and moves left and right in the HR diagram at the same brightness for about 10 million years. The number of loops depends on the mass of the star.
In case of lower mass stars the central hydrogen burning lasts much longer, about 7 billion years. As the central hydrogen is exhausted and the burning moves out in a shell the center doesn't get hot enough to burn helium left behind by the hydrogen burning. So hydrogen burns in the shell around the helium ash in the middle. As the shell moves out, like the heavier stars the lighter stars move right in the HR diagram. But unlike them the lighter stars can't start burning helium. Instead the hydrogen burning shell keeps moving out heating the gas surrounding the shell causing it to expand. This rapidly increases the size of the star and makes it go shooting up the HR diagram as it gets brighter. Eventually as enough helium piles up at the center it starts to burn. This starts suddenly, hence it is called a Helium flash. And the star moves onto a second zero-age Main Sequence, this time burning helium in the core instead of hydrogen. The region of the HR diagram that the star settles into its second zero-age Main Sequence is called the horizontal branch.
In case of the lighter stars after about 10 million years of burning helium in the center, a carbon and oxygen core forms and the star evolves like it did before from the hydrogen ZAMS, upwards and rightward in the HR diagram towards the asymptotic giant branch (AGB) which merges with the region occupied by giants in the HR diagram. At this stage the outer hydrogen burning proceeds sporadically producing pulses (of timescales of half a million years). These pulses remove some of the envelope of the star. At some point the envelope shrinks very rapidly, in a few ten thousand years, and the star moves rapidly to the left of the HR diagram producing a planetary nebula and settles down to a white dwarf moving below the main sequence. Thus a relatively quiet and sedate end awaits a star like the Sun.
The end for large stars is substantially more dramatic. In them the core of carbon and oxygen left behind by helium is hot and large enough to start burning. And its ashes on its turn also burns. So the star has a onion skin structure with shells of different material all burning, adding to the lower shell. This proceeds faster and faster, through neon, magnesium, silicon, sulphur, argon and calcium until iron is produced. Iron is a stable nucleus that doesn't burn like the previous nuclei. So the Iron core just keeps on growing as the Calcium shell outside it is burning, until it becomes too heavy to support itself and it collapses. The core is so hot that the Iron atoms are fully ionized and have lost all their electrons. The core consists of Iron nuclei floating in a sea of electrons. It keeps collapsing until the iron nuclei are crushed together so hard they loose their identity and the core becomes one giant nucleus. The core can't be crushed any further and it bounces back crashing into the outer layers following the collapsing core sending them exploding out in a spectacular display called a supernova. The core remains to form an incredibly dense object called a neutron star and slowly cools while the erstwhile outer shells expand out from the old site of the star in a supernova remnant.
Remember the difference of ages of light and heavy stars at the point when they start burning helium? This is one of the things that make the HR diagram an incredibly important tool in astronomy. Imagine a cluster of stars that formed out of a single huge cloud of gas at about the same time. We can observe such clusters in the sky. Clearly the cluster will have light as well as heavy stars. Now as the stars are all at about the same distance from us apparent magnitude is the same as actual magnitude. So it is very simple to plot the HR diagram for these stars. All we have to do is to plot the color-apparent magnitude diagram and shift it up and down until the main sequence matches the main-sequence of the HR diagrams of stars of known distance. And then we can find out the distance to the cluster.
R. P. Trumpler used these clusters to find out the size and shape of our galaxy in 1930. He measured the distances to the clusters using the method described. He then measured the angular diameters of these clusters. From the known distance and angular diameter he could then calculate the linear diameter of the clusters. He found that there was a relationship between the linear diameters of clusters and their richness (no. of stars), or the degree of concentratedness to the center. He then checked this for the more distant clusters. There turned out to be a large discrepancy. The diameters he surmised from the appearance (richness and concentration) of the distant clusters was systematically smaller than that he got by using distances measured using the HR diagram. He correctly assumed that this was because of material present between us and the clusters. This material absorbed light causing the stars to look dimmer than what they would be from just remoteness. So the HR method gives a larger distance than is true. How then can we rely on any distance measurement by the HR diagram? Fortunately for us, the dimming by obscuring material in front of the clusters is color dependant. So obscured stars also appear redder than stars of the same spectral type do otherwise. We can check for this reddening and correct for any obscuration and still get correct distances using the HR diagram. This is wonderful example of a very important scientific truism. It is extremely important to have to independent ways of measuring a quantity. In this case Trumpler used two independant ways of measuring the the distance to the cluster, and comparison of the discrepant results got in the two ways immediately led to a new discovery.
The HR diagrams of all clusters are of a distinctive type. This has to do with the age of the cluster. Now imagine the cluster is of a certain age. Remember that the time for which a star burns hydrogen in the center ranges from 2 million years for heavy stars to 7 billion years for light stars. So there will be stars in the cluster of a weight such that their hydrogen burning time equals the age of the cluster. All stars heavier than that weight in the cluster will have evolved off the main sequence while all stars lighter than that will still be on the main sequence. In other words, from the point at which the main sequence of the cluster starts to turn off we can tell the age of the cluster!
Both these measurements of distances and ages of clusters have great significance in the theories of the Universe as we will discover later on.