Conscious, Sub-conscious and un-conscious sides of data science
Thanks to Sigmund Fraud for all the fascinating discoveries in human Psychology. I have always been fascinated about how human psychology plays a vital role in decisions we make and how it is inevitable to be understand how human psychology works before you get into improving customer experience and talking about machines replacing human thinking.
One of the major discoveries which Fraud has is around the different states of mind, namely, conscious, sub-conscious and unconscious. From Psychology perspective, our conscious mind is responsible for interactions and decisions we make, sub-conscious is responsible for processing of recent and active information, and unconscious is the dormant state of mind. Ironically what looks the most active is only about 10% of our brain’s capabilities, sub-conscious is around 50–60% and unconscious is about 30–40%. Conscious is like a Screen, sub-conscious is like a processing unit and conscious is like a storage unit.
I see a relevant analogy to this with the way data and the science around it works. Let’s relate the conscious part of analytics to be as ‘Output: Visualization’, sub-conscious as ‘Analytical processing: Algorithms’ and unconscious as ‘Storage: Historical data’. Just that I think we should tweak the % composition a bit. How about this?
Let’s talk about this:
[Conscious] ‘Output: Visualization’:
Conscious part of data science has more of ‘finding answers’ perspective. This is much more agile and interactive. Conscious state is all about how your outputs look, what the insights are and what is not usual? The conscious part is more aware, it can make comparisons. In this stage, data science is about providing intuitive cues to the consumer, facilitating answers for obvious questions and convincing about the recommendations which challenge the environment around of the perception driven thinking. The conscious data science is agile and has breadth, it is not too deep, and it is descriptive and inquisitive largely
[Sub-Conscious] ‘Analytical processing: Algorithms’:
Sub-conscious works on the recent load of information which can still be processed quickly and unknowingly supports what our conscious says. So, sub-conscious data science is not about volume of data , it is more about variety and velocity of data. This part of data science guides the way we form hypotheses, calls for challenging intuitive findings and looks for insights. It is below the surface. Sub-conscious data science allows for experimentation and validation. This is about creating proof of concepts of how data changes decision making process . At this stage data science is descriptive, inquisitive and can be prescriptive too. Conscious working with sub-conscious helps focus on problem solving and building algorithms , based on rather recent understanding of business or domain
[Unconscious] ‘Storage: data’:
Unconscious is huge source of information. It is large data sitting in our data warehouses, it is historical understanding of why something works why something doesn’t . Discovering unconscious part of data science is about finding answers to difficult questions and making highly involved decisions. This is about volume and variety of data. Digging into unconscious of data is big data. Processing of this size of data is stressful and demands resources, it like our mind getting stressed with all things in our past and putting a plan of action for our future, living through present. This is predictive too.
An ideal state is to solve a problem using all the learning from our past (Historical data, unconscious), building robust algorithms using all the recent and self-adjusting knowledge (business & domain, Sub-conscious) and adding the agile movements around to it (Context and representation, Conscious).
Data Science is a mind’s thing!
