Architecting and implementing serverless application with streaming sensor data: Part 6

ACME Industries is a fictional industrial company which has one facility that runs short 10 minute processes as part of its overall production process. There are two types of frontend…

Continue ReadingArchitecting and implementing serverless application with streaming sensor data: Part 6

Architecting and implementing serverless application with streaming sensor data: Part 5

Command and control in ACME Industries application ACME Industries is a fictional industrial company which has one facility that runs short 10 minute processes as part of its overall production…

Continue ReadingArchitecting and implementing serverless application with streaming sensor data: Part 5

Architecting and implementing serverless application with streaming sensor data: Part 4

In Part 3 I explain how the cumulative daily statistics of a running process is implemented in ACME Industries application. I also cover how statistics of a completed process is calculated,…

Continue ReadingArchitecting and implementing serverless application with streaming sensor data: Part 4

Architecting and implementing serverless application with streaming sensor data: Part 3

In Part 2, I provided detailed architecture and implementation of the ingestion layer which includes the AWS IoT Core, and Amazon Kinesis Data Streams services.In this post I explain how…

Continue ReadingArchitecting and implementing serverless application with streaming sensor data: Part 3

Architecting and implementing serverless application with streaming sensor data: Part 2

Ingestion layer design and implementation in ACME Industries example application In Part 1 I introduced ACME Industries application that streams sensor data from a running process of a facility. The facility…

Continue ReadingArchitecting and implementing serverless application with streaming sensor data: Part 2

Architecting and implementing serverless application with streaming sensor data: Part 1

We all have been witnessing a powerful  trend where more and more devices are able to communicate via the Internet, and this growing trend is considered as a part of the Internet of the future. In industrial processes these devices are capable of streaming real-time process data. Data is ubiquitous in businesses today, and the volume and speed of incoming data are constantly increasing. However, collecting, processing, and analyzing such streaming data workloads presents a unique set of architectural challenges. It becomes even more challenging when all this must be done in real-time, or near real-time. In this series of blog posts, I will demonstrate how I architected, and implemented an end-to-end serverless application that collects, processes, analyzes, and presents data by using various AWS services.

Continue ReadingArchitecting and implementing serverless application with streaming sensor data: Part 1

Collecting and storing real-time data with Kinesis Data Streams, Kinesis Agent, Lambda, and DynamoDB

In this post I will demonstrate how to setup an end-to-end application that collects, processes, and stores real-time purchase order data at a fictitious web site - VarietyGifts.com. The purchase…

Continue ReadingCollecting and storing real-time data with Kinesis Data Streams, Kinesis Agent, Lambda, and DynamoDB