The rise of the Internet of Things has thrown up many curious challenges, not least of which is where on earth all of that extra data will sit, and how it can be deployed effectively.
Cue the arrival of data lakes - huge repositories that feed data to users on as needed basis (think data as a utility). Enterprises have been increasingly struggling to handle the ever-growing streams of new data. Data lakes may be the answer.
The concept of data lakes has been around for a while - in fact Jed Mole of Acxiom was writing about them over a year ago. But it appears to be a growing topic of interest for larger enterprises.
This recent Network World piece expands on the principle.
Data lakes, storage repositories that hold extremely large amounts of raw data in its native format until the data is needed by users, are becoming increasingly popular within enterprises. Helping to fuel interest in data lakes are the digital transformation efforts underway at many enterprises, spurred by the emergence of the Internet of Things (IoT). The connected objects in the IoT will generate huge volumes of data. As more products, assets, vehicles and other “things” are instrumented and data ingested, it’s important that IoT data sets be aggregated in a single place, where they can be easily analyzed and correlated with other relevant data sets using big data processing capabilities. Doing so is critical to generating the most leverage and insight from IoT data.