AWS Kinesis Delivers Engineering Savings & Performance Benefits

Imagine having thousands of cameras connected to your network simultaneously and continuously streaming video data to your server(s) throughout the day. How do you keep all of that information and access it in the future? AWS Kinesis is the best answer.

Storing & Retrieving Video Data the Old Way

It used to be that storing and retrieving vast amounts of video data was quite challenging.

Because video data takes a tremendous amount of space to store, organizations producing a lot of video would have to build up a network-attached storage server. From there, connected devices would capture video and send that stream data to the server. Decision-makers then had to determine what to do with all of that information before the server filled up, whether that entailed exchanging out hard drives or creating tape drives, writing the video out to that drive and putting the tape drive in a vault.

While it’s likely none of that stored video data will ever be used, it’s often crucial that it is retrievable. This is especially true if the organization is facing a lawsuit or something similar. Unfortunately, what frequently happens is organizations fill up their hard drives, then begin cycling old data out and throwing the information away. Then, if that data is needed down the road, it’s not there.

But today, tools that enable the easy collection, processing and analysis of video and data streams in real time exist.

Amazon Kinesis Video

One we often use is Amazon Web Services’ Amazon Kinesis. Offering capabilities to cost-effectively process streaming data at any scale, along with the flexibility to choose the tools that best suit the requirements of the application, Amazon’s solution completely simplifies video and audio data storage.

With Kinesis, cameras stream video to AWS using the tech giant’s software development kit (SDK). The tool automatically provisions and elastically scales all the infrastructure needed to ingest streaming video data from millions of devices, and it durably stores, encrypts and indexes video data in streams. By enabling organizations to ingest, buffer and process streaming data in real time, Amazon Kinesis Video Streams makes it easy to securely stream video from connected devices to AWS for analysis, storage, playback or other processing. This allows organizations to access stream data through easy-to-use application programming interfaces (APIs).

Even better, the tool truly provides engineering savings and performance benefits. AWS Kinesis is fully managed and runs streaming applications without requiring organizations to manage any infrastructure. As such, organizations don’t have to worry about configuration, software updates, failures or scaling infrastructure as the number of streams and consuming applications grows. They just stream it up to Kinesis and are done.

Plus, Kinesis can handle any amount of streaming data and process data from hundreds of thousands of sources, including edge devices, smartphones, security cameras and other data sources, such as radars, lidars, drones, satellites, dash cams and depth sensors, with very low latencies.

Perhaps best of all, the tool is secure. Amazon Kinesis Video Streams allows organizations to control access to their streams using AWS Identity and Access Management (IAM), helping protect data by automatically encrypting the data at rest using AWS Key Management Service (KMS) and in transit using the industry-standard Transport Layer Security (TLS) protocol. It also provides all of the testing tools organizations need for playback, as well as the industry-standard tools that make development both easier to debug and move to production.

By handling all of the administration and maintenance required to manage video streams, Kinesis empowers organizations to focus on their own business rather than the storage and processing of media streams. From higher video resolution to more cameras and lengthier archival periods, the various tiers of storage capacity Kinesis offers enable organizations to determine the right balance of video data quantity, quality and archiving required for a given application.

Comparing & Contrasting a Recent Video Surveillance Project with a Similar Project I Worked On Before AWS Kinesis

With extensive digital video systems expertise, we were recently tapped to assist with a video surveillance project. Our client wanted to feed all of the video from their store cameras (four cameras per store with hundreds of stores for a total of 14,000 cameras) 24/7 into the cloud, then allow playback through the web.

With Amazon Kinesis, every camera was able to just run the software and then send the stream up to Kinesis, with 100% availability, 100% storage and no issues. This solution removed all of our customer’s responsibility for storing video data and providing the network interface to get the video. Best of all, the engineering savings and performance benefits ensured I could be just a one-person team doing all the work throughout the project.

On the other project I worked on well before AWS Kinesis, the customer had thousands of video surveillance cameras. I was on a team of 10 engineers, and we had to build servers, write software, install it on the servers to manage the storage and processing of the data and continue to maintain the system after launch. At the time, its only algorithm was to throw the data away after a few weeks, otherwise, it would run out of space or we would have to try and upload it in the background when it started to fill up. However, this quickly became a nightmare and often didn’t work right because sometimes the network was up, sometimes it was down, sometimes we couldn’t get the video before it cycled out. And we could only really retrieve data effectively on-site because we just didn’t have the network interface outside of the LAN to support large-scale playback out of site, limiting most of the playback to on-site.

Without Amazon’s uptime, the team I was a part of had to run those network servers and make them always available, which was very difficult to do. Plus, we had to deal with the hardware part of it, too, and upgrades to the hardware. Managing all of that and maintaining the infrastructure of the storage and retrieval took engineering effort and a lot of it — both in software and in development and operations (DevOps) — to keep the networks up. And with thousands of cameras, that’s a lot of networks, which costs a lot of money.

AWS Kinesis Performance Takeaways

Beyond scale and ease of deployment, Amazon Kinesis Video Streams is going to solve your storage issues and most of your playback issues. The engineering savings and performance benefits the tool unlocks are incredibly beneficial, and it’s quite difficult to get Amazon’s network performance somewhere else. The tool enables you to:

  • Stream video from millions of devices.
  • Playback live and recorded video streams.
  • Retrieve archived media from durable, searchable storage.
  • Control access to your streams.
  • Deploy with no infrastructure to manage.

Leveraging extensive expertise in embedded hardware/software, DSP, cloud/mobile infrastructure, encoding/decoding and playback, as well as experience with both AWS and Kinesis (we are an AWS design partner), Cardinal Peak can solve any video/audio/multimedia storage and playback problems you may have. We’ve worked in the cable and video industry for decades and possess a unique specialty in sophisticated video processing.

For more information, check out our streaming case studies, including an Amazon Kinesis Video Streams and edge machine learning smart cat detector for real-time remote monitoring.

To discover how we can help you with your next video project, reach out to us today!