The Truth

"The Truth is Out There." Obviously, it is there and everyday we seek to find it. But being an analyst, I find myself searching it more harder then others. Every time you look into graphs and numbers, you seem to see certain pattern in it and it almost becomes an obsession. But time and again, I keep asking myself, if there is anything like an absolute truth, I mean THE TRUTH.

The hidden truth is always waiting to be revealed, in form of numbers or graphs. It is an amazing feeling that you get, when you certainly figure out the truth. You say, "AAh! So that is why?"

This small enlightened gives a certain amount of happiness in you. That is the nature of Truth. But why am I saying all these things, what does it has to do with an engineering problem in analytics. Well a lot, actually.

Long time back, when I was attending a presentation on Design of Complex Circuit by one of the lead designer of HP, he told us that the engineers working in the field of MIXER design is never going to be unemployed. The reason was that the MIXER design is full of problems that will never be solved completely. The field of Analytics is similar, no matter how many truths or patterns we find in our data, we are never going to have the complete truth. Thereafter, Analyst will also never be out of job. MMM! thats cool. But that's not my point.

There is more to be said. I am not concerned about the job opportunities, what I am pointing is that the analysis made by an analyst have a very short life. Therefore, every time a new data has to be processed to find a new trend and thus make a new model based on these new observation. But why does that happen?

The main reason is that the variable we choose has uncountable factors affecting them and when we analyse it, only a few are chosen. The nature of the behaviour that we see in graph is impossible to be explained with 100% confidence level. Otherwise, till now, we would have all truth and we would have been able to predict almost everything, but NO, we haven't, and we won't, but we will keep trying. May be that Super or Grand model is waiting to be revealed. This is the similar case as with the grand M-theory in Physics.

One of my favourite icon Richard Feynman, a physicist, had told that searching for a truth is like watching a chess game but with out knowing the rules of the game, so with each new move, you find new rule and a new truth. So after observing for a long time, you finally know the game, but in real world the rules keep on changing and the move is rather very complex. Therefore, for each new truth or rule to be discovered, an event has to occur, an event whose probability of occurrence is almost ZERO. This makes the thing even more interesting because we are almost uncertain if there is any new truth to be discovered.

Did I sound very complex, I hope not. I was just trying to explain the nature of analytics and nothing more. Lets me know how I did?