Miscellaneous Ramblings Today on Data and Analytics

Here are some ideas off the top of my head:

1. The Big Data Analytics industry – vendors, journalists, industry analysts – have flooded the market with messages as if no one ever used quantitative methods before

2. Because most of the content you see is generated by people who don’t actually use quantitative methods, it is:
– focused on technology
– full of the same use cases such as up-sell/cross-sell, churn, fraud, etc.

3. The real opportunity with Big Data and its attendant technologies is to get a richer understanding of those phenomena that are important to you

4. The rise of Data Science and Scientists is the invention of practitioners from the digital giants and not terribly relevant to most companies

5. Ultimately the benefit of Big Data Analytics will be better decisions born of better decision-making processes, not  just informing people of findings. This was the weak point of BI, it was too passive. Operational Intelligence and Decision Automation are key

6. All of this is possible because of the radically different analytical architectures and open source tools that are available in a variety of cloud-based topologies

7. Many business analysts have the background to use advanced analytical tools, provided the tools get better at guiding and advising.

8. The industry can’t continue without better tools. Big Data is a giant time sink. We’re seeing lots of interesting products emerge, many are open-source, to lubricate the whole data management and analytic spectrum

9. As always, finding a way for business units and IT to cooperate and work productivly is still a problem.

10. Existing operational systems are either based on relational databases technology or even older systems written in COBOL and other 2nd-generation languages. Capturing information in these systems is like fitting a square peg in a round hole. New database systems, the so-called NoSQL tools offer abundant opportunities to capture and use rich information. One example, graph databases, are brilliant at finding hidden relationships to expose concentration risk or fraud for example.

11. I’ve built a few Bayesian Belief Networks recently. What I learned is that they can get computationally expensive, perform poorly on high dimensional data and models can be hard to interpret. On the other hand is the ability to get to causation, not just correlation. Better to build from data and/or simulation

This entry was posted in Uncategorized. Bookmark the permalink.

2 Responses to Miscellaneous Ramblings Today on Data and Analytics

  1. Ted Cuzzillo says:

    I’m curious about what you say in point number five, that the benefit of big data analytics will derive from “better decisions born of better decision-making processes …” You’ve been interested in storytelling, which is usually associated only with presentation. But couldn’t storytelling also be among those “better decision-making processes”? Decisions, after all, are choices of stories. We decide to believe one story instead of others. Let’s say we believe that X data reflects Y story, and Y story normally leads to Z story, and that of course leads to Q decision. The discussion among decision makers is all about dueling stories.

    What do you think?

  2. waltika says:

    Reblogged this on Waltika and commented:
    I must say that there is lot of wisdom in this article.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s