Information Theory

[](http://2.bp.blogspot.com/_UYUaEitRq54/SSLNE- nuuQI/AAAAAAAAAR0/Tux8D1HF4lg/s1600-h/infin.png)

This is a very highly theoretical problem. What do you call an "Information"? To know that you have been informed. What does that mean? Analyst are always tyring to find the information from a mathematical functions and applying a probabilistic approach, from the trend or clusters of data.

But, what is information and how can you measure the value of the information. Just recently I had posted a article on modeling where, I talked about the Information Value. I might have skipped by saying that it has a mathematically rigorous process and needs in depth understanding of Information Theory. Well, let me try to explain it in as many few words as possible.

Information is something you don't know. It is something which you don't expect to happen. For example your future- so if someone tells your future, its an information. Is that confusing??

Actually, its quite simple. If someone tells you something you already know, then its not an information, cause it doesn't have any value, as you already know it.

Therefore, even in a collection of data and in the correlation process, we try to figure out the best possible sets of variables that have the capability to predict the outcome variables.

Lets take another example:

Lets say, you know that your schedule for train is at 12pm at night. Cause its in the ticket.

Now if someone says, that your departure time is 12pm....hmmm is that an information.

But, if someone comes to you and says, your train has been rescheduled due to the delay. Then that is an information- an information, that is useful. Anything not useful is not an information. (From what I have explained)

Mathematically, Information is inversely proportional to the probability of the occurrence of an event.

If p(E) is zero than it contains INFINITEinformation.

if p(E) is 1, which is the largest, then it contains ZERO information.

I hope that was simple enough.

Remember that the information that I am talking about is not the collection of a data. It is something that has value in terms of its existence.