AWS Panorama - Developer Guide PDF Free Download

1y ago

45 Views

1 Downloads

1.46 MB

119 Pages

Report/dmca

Download PDF

Transcription

AWS PanoramaDeveloper Guide

AWS Panorama Developer GuideAWS Panorama: Developer GuideCopyright Amazon Web Services, Inc. and/or its aﬃliates. All rights reserved.Amazon's trademarks and trade dress may not be used in connection with any product or service that is notAmazon's, in any manner that is likely to cause confusion among customers, or in any manner that disparages ordiscredits Amazon. All other trademarks not owned by Amazon are the property of their respective owners, who mayor may not be aﬃliated with, connected to, or sponsored by Amazon.

AWS Panorama Developer GuideTable of ContentsWhat is AWS Panorama? . 1Getting started . 2Concepts . 3The AWS Panorama Appliance . 3Compatible devices . 3Applications . 3Nodes . 4Models . 4Setting up . 5Prerequisites . 5Register and conﬁgure the AWS Panorama Appliance . 6Upgrade the appliance software . 7Add a camera stream . 8Next steps . 8Deploying an application . 9Prerequisites . 9Import the sample application . 10Deploy the application . 10Enable the SDK for Python . 12Clean up . 12Next steps . 12Developing applications . 13The application manifest . 13Building with the sample application . 16Changing the computer vision model . 17Preprocessing images . 19Uploading metrics with the SDK for Python . 19Next steps . 21Supported models and cameras . 22Supported models . 22Supported cameras . 22Appliance speciﬁcations . 23Permissions . 24User policies . 25Service roles . 26Securing the appliance role . 26Use of other services . 27Application role . 29Appliance . 30Managing . 31Update the appliance software . 31Deregister an appliance . 31Network setup . 33Single network conﬁguration . 33Dual network conﬁguration . 33Conﬁguring internet access . 34Conﬁguring local network access . 34Private connectivity . 35Cameras . 36Removing a stream . 36Applications . 38Buttons and lights . 39Status light . 39Network light . 39iii

AWS Panorama Developer GuidePower and reset buttons .Managing applications .Managing .Deploy an application .Update or copy an application .Delete versions and applications .Packages .Application manifest .JSON schema .Nodes .Edges .Abstract nodes .Parameters .Overrides .Building applications .Models .Using models in code .Building a custom model .Packaging a model .Training models .Build an image .Specifying dependencies .Building image assets .AWS SDK .Using Amazon S3 .Using the AWS IoT MQTT topic .Application SDK .Adding text and boxes to output video .Running multiple threads .Serving inbound traﬃc .Conﬁguring inbound ports .Serving traﬃc .Using the GPU .Tutorial – Windows development environment .Prerequisites .Install WSL 2 and Ubuntu .Install Docker .Conﬁgure Ubuntu .Next steps .The AWS Panorama API .Automate device registration .Manage appliances .Automate application deployment .Build the container .Upload the container and register nodes .Deploy the application .Monitor the deployment .Using VPC endpoints .Creating a VPC endpoint .Connecting an appliance to a private subnet .Sample AWS CloudFormation templates .Samples .Sample applications .Utility scripts .AWS CloudFormation templates .More samples and tools .Monitoring 08081838383848485

AWS Panorama Developer GuideAWS Panorama console . 86Logs . 87Viewing device logs . 87Viewing application logs . 87Conﬁguring application logs . 88Viewing provisioning logs . 88CloudWatch metrics . 90Using device metrics . 90Using application metrics . 90Conﬁguring alarms . 91Troubleshooting . 92Provisioning . 92Appliance conﬁguration . 92Application conﬁguration . 92Camera streams . 93Security . 94Security features . 95Best practices . 96Data protection . 97Encryption in transit . 97AWS Panorama Appliance . 97Applications . 98Other services . 98Identity and access management . 99Audience . 99Authenticating with identities . 99Managing access using policies . 101How AWS Panorama works with IAM . 103Identity-based policy examples . 103AWS managed policies . 104Using service-linked roles . 106Cross-service confused deputy prevention . 107Troubleshooting . 108Compliance validation . 110Additional considerations for when people are present . 110Infrastructure security . 111Deploying the AWS Panorama Appliance in your datacenter . 111Runtime environment . 112Releases . 113v

AWS Panorama Developer GuideWhat is AWS Panorama?AWS Panorama is a service that brings computer vision to your on-premises camera network. You installthe AWS Panorama Appliance or another compatible device in your datacenter, register it with AWSPanorama, and deploy computer vision applications from the cloud. AWS Panorama works with yourexisting real time streaming protocol (RTSP) network cameras. The appliance runs secure computervision applications from AWS Partners, or applications that you build yourself with the AWS PanoramaApplication SDK.The AWS Panorama Appliance is a compact edge appliance that uses a powerful system-on-module(SOM) that is optimized for machine learning workloads. The appliance can run multiple computer visionmodels against multiple video streams in parallel and output the results in real time. It is designed foruse in commercial and industrial settings and is rated for dust and liquid protection (IP-62).The AWS Panorama Appliance enables you to run self-contained computer vision applications at theedge, without sending images to the AWS Cloud. By using the AWS SDK, you can integrate with otherAWS services and use them to track data from the application over time. By integrating with other AWSservices, you can use AWS Panorama to do the following: Analyze traﬃc patterns – Use the AWS SDK to record data for retail analytics in Amazon DynamoDB.Use a serverless application to analyze the collected data over time, detect anomalies in the data, andpredict future behavior. Receive site safety alerts – Monitor oﬀ-limits areas at an industrial site. When your application detectsa potentially unsafe situation, upload an image to Amazon Simple Storage Service (Amazon S3) andsend a notiﬁcation to an Amazon Simple Notiﬁcation Service (Amazon SNS) topic so recipients cantake corrective action. Improve quality control – Monitor an assembly line's output to identify parts that don't conform torequirements. Highlight images of nonconformant parts with text and a bounding box and displaythem on a monitor for review by your quality control team. Collect training and test data – Upload images of objects that your computer vision model couldn'tidentify, or where the model's conﬁdence in its guess was borderline. Use a serverless application tocreate a queue of images that need to be tagged. Tag the images and use them to retrain the model inAmazon SageMaker.AWS Panorama uses other AWS services to manage the AWS Panorama Appliance, access models andcode, and deploy applications. AWS Panorama does as much as possible without requiring you to interactwith other services, but a knowledge of the following services can help you understand how AWSPanorama works. SageMaker – You can use SageMaker to collect training data from cameras or sensors, build a machinelearning model, and train it for computer vision. AWS Panorama uses SageMaker Neo to optimizemodels to run on the AWS Panorama Appliance. Amazon S3 – You use Amazon S3 access points to stage application code, models, and conﬁgurationﬁles for deployment to an AWS Panorama Appliance. AWS IoT – AWS Panorama uses AWS IoT services to monitor the state of the AWS Panorama Appliance,manage software updates, and deploy applications. You don't need to use AWS IoT directly.To get started with the AWS Panorama Appliance and learn more about the service, continue to Gettingstarted with AWS Panorama (p. 2).1

AWS Panorama Developer GuideGetting started with AWS PanoramaTo get started with AWS Panorama, ﬁrst learn about the service's concepts (p. 3) and theterminology used in this guide. Then you can use the AWS Panorama console to register your AWSPanorama Appliance (p. 5) and create an application (p. 9). In about an hour, you can conﬁgurethe device, update its software, and deploy a sample application. To complete the tutorials in thissection, you use the AWS Panorama Appliance and a camera that streams video over a local network.NoteTo purchase an AWS Panorama Appliance, visit How to purchase.The AWS Panorama sample application (p. 13) demonstrates use of AWS Panorama features.It includes a model that has been trained with SageMaker and sample code that uses the AWSPanorama Application SDK to run inference and output video. The sample application include a AWSCloudFormation template and scripts that show how to automate development and deploymentworkﬂows from the command line.The ﬁnal two topics in this chapter detail requirements for models and cameras (p. 22), and thehardware speciﬁcations of the AWS Panorama Appliance (p. 23). If you haven't obtained an applianceand cameras yet, or plan on developing your own computer vision models, see these topics ﬁrst for moreinformation.Topics AWS Panorama concepts (p. 3) Setting up the AWS Panorama Appliance (p. 5) Deploying the AWS Panorama sample application (p. 9) Developing AWS Panorama applications (p. 13) Supported computer vision models and cameras (p. 22) AWS Panorama Appliance speciﬁcations (p. 23)2

AWS Panorama Developer GuideConceptsAWS Panorama conceptsIn AWS Panorama, you create computer vision applications and deploy them to the AWS PanoramaAppliance or a compatible device to analyze video streams from network cameras. You write applicationcode in Python and build application containers with Docker. You use the AWS Panorama ApplicationCLI to import machine learning models locally or from Amazon Simple Storage Service (Amazon S3).Applications use the AWS Panorama Application SDK to receive video input from a camera and interactwith a model.Concepts The AWS Panorama Appliance (p. 3) Compatible devices (p. 3) Applications (p. 3) Nodes (p. 4) Models (p. 4)The AWS Panorama ApplianceThe AWS Panorama Appliance is the hardware that runs your applications. You use the AWS Panoramaconsole to register an appliance, update its software, and deploy applications to it. The software on theAWS Panorama Appliance connects to camera streams, sends frames of video to your application, anddisplays video output on an attached display.The AWS Panorama Appliance is an edge device powered by Nvidia Jetson AGX Xavier (p. 23). Insteadof sending images to the AWS Cloud for processing, it runs applications locally on optimized hardware.This enables you to analyze video in real time and process the results locally. The appliance requiresan internet connection to report its status, to upload logs, and to perform software updates anddeployments.For more information, see Managing the AWS Panorama Appliance (p. 30).Compatible devicesIn addition to the AWS Panorama Appliance, AWS Panorama supports compatible devices from AWSPartners. Compatible devices support the same features as the AWS Panorama Appliance. You registerand manage compatible devices with the AWS Panorama console and API, and build and deployapplications in the same way. Lenovo ThinkEdge SE70 – Powered by Nvidia Jetson Xavier NXThe content and sample applications in this guide are developed with the AWS Panorama Appliance.For more information about speciﬁc hardware and software features for your device, refer to themanufacturer's documentation.ApplicationsApplications run on the AWS Panorama Appliance to perform computer vision tasks on video streams.You can build computer vision applications by combining Python code and machine learning models, anddeploy them to the AWS Panorama Appliance over the internet. Applications can send video to a display,or use the AWS SDK to send results to AWS services.3

AWS Panorama Developer GuideNodesTo build and deploy applications, you use the AWS Panorama Application CLI. The AWS PanoramaApplication CLI is a command-line tool that generates default application folders and conﬁguration ﬁles,builds containers with Docker, and uploads assets.For more information, see Managing AWS Panorama applications (p. 41).NodesAn application comprises multiple components called nodes, which represent inputs, outputs, models,and code. A node can be conﬁguration only (inputs and outputs), or include artifacts (models and code).An application's nodes are bundled in node packages that you upload to an Amazon S3 access point,where the AWS Panorama Appliance can access them. An application manifest is a conﬁguration ﬁle thatdeﬁnes connections between the nodes.For more information, see Application nodes (p. 47).ModelsA computer vision model is a machine learning network that is trained to process images. Computervision models can perform various tasks such as classiﬁcation, detection, segmentation, and tracking. Acomputer vision model takes an image as input and outputs information about the image or objects inthe image.AWS Panorama supports models built with PyTorch, Apache MXNet, and TensorFlow. You can buildmodels with Amazon SageMaker or in your development environment. For more information, see? (p. 53).4

AWS Panorama Developer GuideSetting upSetting up the AWS Panorama ApplianceTo get started using your AWS Panorama Appliance or compatible device (p. 3), register it in the AWSPanorama console and update its software. During the setup process, you create an appliance resource inAWS Panorama that represents the physical appliance, and copy ﬁles to the appliance with a USB drive.The appliance uses these certiﬁcates and conﬁguration ﬁles to connect to the AWS Panorama service.Then you use the AWS Panorama console to update the appliance's software and register cameras.Sections Prerequisites (p. 5) Register and conﬁgure the AWS Panorama Appliance (p. 6) Upgrade the appliance software (p. 7) Add a camera stream (p. 8) Next steps