Big Data Critical in IoT to the Progression from Connected to Autonomous Things

There is a progression in the maturity of solutions around the Internet of Things, from Connected, to Smart, and eventually to Autonomous.


Connected ThingsConnected

As connectivity has expanded, things have become more connected –to the internet and to one another. Connectivity enables them to communicate their state or to retrieve information, enabling visibility, and creating a more connected world.

Connectivity enables data from sensors or device inputs to flow into software solutions, opening up new applications and innovation.

Smart ThingsSmart

The next phase of maturity comes with the introduction of enhanced processing power embedded in things or via “bot” programs that can be called from the connecting devices, enabling IoT Applications with embedded business logic and interactivity. This is the world of “Smart Things.”

Smart things also enable  applications which can have “situational awareness” such as taking into consideration location, weather and other factors.

Autonomous ThingsAutonomous

As processing power and network connectivity expands,  we see new opportunities for autonomous behavior. However, there is another necessary ingredient for truly autonomous capabilities – Machine Learning. Machine Learning enables the creation of sophisticated algorithms to drive behaviors that don’t require pre-defined business logic.

Data Is Key to Progressing in IoT Maturity

Data has been essential in addressing challenges at each level of maturity, as well as moving solutions ahead in maturity.

Connected Things require processing IoT data for visibility, to enable  asset management, remote monitoring, and aggregation of data from thousands of devices, as well as enriching that data with other sources to provide comprehensive analytics applications. Another factor that drives the growth of data from connected things is the sampling rate, as there is considerable time-series data. If you need to retain this data for long windows, that can drive considerable growth. Discard it, or only record the deltas, and you may be losing important sources of future insights.

Without their Data, Things simply aren’t Connected.

Smart Things require even more data to drive interactivity, to manage more a more complex deployed device landscape, and to tie IoT data together with Enterprise, Customer, and Situational Data to provide the context needed for complete insights and to drive more sophisticated embedded logic. But, Smart things also need that data in near-real-time, so now we also have to deal with greater Velocity of data, and more compute capacity to handle that. In addition, as you start to add predictive use-cases using time-series data, you will require retention of complete time-series data, and over long enough windows to allow factoring in seasonality or other time-period-related into your models.

Bottom line, Data helps makes Things Smarter.

Autonomous Things require advanced algorithms developed using Machine Learning. Data is used to create these algorithms that drive autonomous capabilities, and you more need data for Model Training, and Scoring, and Testing. It also requires requires a large Variety of data, because you are trying to emulate and human-like decision making capabilities, including image, video, sound, and language processing. For perspective, it has been estimated one human brain stores and processes 10s of terabytes.. To emulate this, you need to have data from not one device, but from hundreds or thousands. You also need to enrich this with Contextual and Situational Data. Thus, it requires truly massive storage and compute capacity to generate and continuously improve the algorithms needed for Autonomous Devices.

Data is the raw material for the “Algorithm Factory” that is powered by Machine Learning.

I’ll close with a quote from one of my favorite fictional characters, Sherlock Holmes:

“Data, Data, Data, I can’t make bricks without clay!”

The post Big Data Critical in IoT to the Progression from Connected to Autonomous Things appeared first on Cloudera VISION.

Leave a Comment

Your email address will not be published. Required fields are marked *