Back to : Text, Bytes and Videotape Main Page

Text, Bytes and Videotape

Task 2: Supporting notes

Go to calculating logs
Go to bookmakers' odds

Formula for calculating entropy

This formula can be rewritten in the form:

$\begin{displaymath}H = -(p(1) log_2p(1) + p(2) log_2p(2) + p(3) log_2p(3) + ... + p(r) log_2p(r)) \end{displaymath}$

Start by making sure you see how this expanded form relates to the formula above. Remember the $\Sigma$ sign means add up all the terms that follow it. The s = 1 below the $\Sigma$ and the r above it tell you which term you start the adding at, and which you finish at. So in this case we start at the first term, where s = 1, and finish at the final term, labelled with an r.

The first situation you are going to find the entropy for is the coin toss. Here we only have two events, so we just need to consider s = 1 and s = 2. The term p(s) is the probability of term s, so for the coin toss this means the probability of the two possible events, getting a head or a tail. This means that p(1) = 1/2 and p(2) = 1/2 also.

We then have to work out log₂ p(s) for each outcome, so at this point, we need to consider what the 'log' terms mean. Return to Task 3 to work through a question to help you with this, and use the answers to Task 3 to check what you get for this first section of Task 3, if this is new to you.

Return to top

Finding logs to the base of 2

log 100 or log₁₀100 (which means the same thing) is the power to which 10 has to be raised to give 100, ie. 2. So we can say log₁₀100 = 2, or, equivalently 10² = 100. The log function is the inverse of the function which raises a base number to a power.
So log₂32 = 5, or equivalently 2⁵ = 32, and log₂1/2 = -1, or equivalently 2^-1 = 1/2.

This is fine where we want to find the logs corresponding to exact powers of 2, but it only gives us a starting point for finding things like log₂2/3 or log₂1/3, as you have to do in question 5 of Task 3. We can get a starting point for both these by noting that log₂1 = 0 (since 2⁰ = 1), log₂1/2 = -1 and log₂1/4 = -2. This tells us that log₂2/3 is between -1 and 0, and that log₂1/3 is between -2 and -1. We can use a spreadsheet to get a more exact value for these

To do this, set up a column of x values running from -2 to 0, in intervals of 0.1. Now in the next column set up values of 2^x where x is the value in the first column, as in this image:

picture of columns for x and 2 to the power x from a spreadsheet

If you read the x value corresponding to the 2^x value as close to 2/3 as possible you can see that log₂2/3 is around -0.6, and similarly that log₂1/3 is around -1.6. You can get more accurate values by decreasing the intervals between x values.

An exact way to find values for logs to base 2, is to use the properties of relationships involving powers.
In general, x = log_a bⁿ is equivalent to saying that a^x = bⁿ. (Check that you agree with this, using the equivalences above for logs to the base of 10 and 2 and their corresponding power relationships).
a^x = bⁿ means that a^x/n = b, so log_aa^x/n = log_a b.
Now log_aa^x/n just means the power to which a has to be raised to make a^x/n, ie. x/n, and so x/n = log_a b, or x = n log_a b. (Again, check you understand this. The most difficult bit is probably seeing that log_aa^x/n = x/n, so check this out with numbers, eg. log₁₀10^4/2 = log₁₀10² = log₁₀100 = 2 = 4/2).

We can use this relationship to change the base of a log.

$\begin{displaymath}If log_2y = x then 2^x = y \end{displaymath}$

$\begin{displaymath}so log_{10} (2^x) = log_{10} y \end{displaymath}$

$\begin{displaymath}and x log_{10} 2 = log_{10} y \end{displaymath}$

This gives us:

$\begin{displaymath}x = log_2y = (log_{10} y)/(log_{10} 2) \end{displaymath}$

These are both values which can be found using a calculator. Check your values of log₂2/3 and log₂1/3 using this method.

Return to top

Bookmakers' odds

Odds of 1/2 on an outcome mean that the bookmaker thinks the ratio of the probability that it won't happen to the probability that it will happen is 1:2. This means that he thinks there is a 1/3 chance it won't happen and a 2/3 chance it will. Odds of 3/2 mean the bookmaker thinks the probability it will happen is 2/5, and the probability that it won't happen is 3/5.

Return to top
Return to Task 3

Return to Text, Bytes, and Videotape homepage.

top of page

Back to : Text, Bytes and Videotape Main Page