Architecting and implementing serverless application with streaming sensor data: Part 5

Command and control in ACME Industries application ACME Industries is a fictional industrial company which has one facility that runs short 10 minute processes as part of its overall production…

Continue ReadingArchitecting and implementing serverless application with streaming sensor data: Part 5

Architecting and implementing serverless application with streaming sensor data: Part 4

In Part 3 I explain how the cumulative daily statistics of a running process is implemented in ACME Industries application. I also cover how statistics of a completed process is calculated,…

Continue ReadingArchitecting and implementing serverless application with streaming sensor data: Part 4

Architecting and implementing serverless application with streaming sensor data: Part 3

In Part 2, I provided detailed architecture and implementation of the ingestion layer which includes the AWS IoT Core, and Amazon Kinesis Data Streams services.In this post I explain how…

Continue ReadingArchitecting and implementing serverless application with streaming sensor data: Part 3

Architecting and implementing serverless application with streaming sensor data: Part 2

Ingestion layer design and implementation in ACME Industries example application In Part 1 I introduced ACME Industries application that streams sensor data from a running process of a facility. The facility…

Continue ReadingArchitecting and implementing serverless application with streaming sensor data: Part 2

Architecting and implementing serverless application with streaming sensor data: Part 1

We all have been witnessing a powerful  trend where more and more devices are able to communicate via the Internet, and this growing trend is considered as a part of the Internet of the future. In industrial processes these devices are capable of streaming real-time process data. Data is ubiquitous in businesses today, and the volume and speed of incoming data are constantly increasing. However, collecting, processing, and analyzing such streaming data workloads presents a unique set of architectural challenges. It becomes even more challenging when all this must be done in real-time, or near real-time. In this series of blog posts, I will demonstrate how I architected, and implemented an end-to-end serverless application that collects, processes, analyzes, and presents data by using various AWS services.

Continue ReadingArchitecting and implementing serverless application with streaming sensor data: Part 1

Collecting and storing real-time data with Kinesis Data Streams, Kinesis Agent, Lambda, and DynamoDB

In this post I will demonstrate how to setup an end-to-end application that collects, processes, and stores real-time purchase order data at a fictitious web site - VarietyGifts.com. The purchase…

Continue ReadingCollecting and storing real-time data with Kinesis Data Streams, Kinesis Agent, Lambda, and DynamoDB