Detecting Faces from Videos using Rekognition in Serverless Architecture

With the rise of Artificial Intelligence and Cognitive Services, it’s now pretty easy to achieve things we used to hear as fairy tales or as science fiction in childhood.

In a just matter of 10 years, we have seen servers shifting their places from delicate server rooms to clouds and now things are getting popular around serverless architecture.

The software engineers of my company Craftsmen always work with bleeding-edge technologies like this. In this article, I am going to share an interesting topic based on the following architecture.

We have few videos stored in S3 bucket. Also we have few images of people stored in another S3 bucket. Using some ‘magic’ we would like to know which people are being shown in which video and at what time.

Here is the magic we shall use the AWS Rekognition service. Relevant code snippets are written in Node.js.

I shall not show how to configure API Gateway with Lambda here as it can be easily done and if you find any difficulties please google it and you will find ample help from there :).

Suppose we have a bunch of images of different people uploaded in an S3 bucket. The Rekognition engine does the facial analysis and faces recognition based on a collection of its own. We need to register these images for this collection.

const rekognition = new AWS.Rekognition({ region: 'eu-west-1' });
const params = {
    CollectionId: 'rekognitionFaceCollectionId', // Id of Face Collection 
    Image: {
        S3Object: {
            Bucket: 'IAmABucketContainingImagesOfPerson',
            Name: 'Alex.jpg',
        },
    },
};
const result = await rekognition.indexFaces(params).promise();

The face collection can be created using the following AWS CLI command. Needless to say that this collection can be created programmatically but in order to keep this article I am ignoring this here.

aws rekognition create-collection --collection-id 
'rekognitionFaceCollectionId' --region eu-west-1

Let’s assume that we have one CognitiveEngineApi exposed which takes a video item reference as an input. The Rekognition engine is an independent system and it communicates back with its processing status through SNS. This can be seen in the architecture diagram above.

The following code snippet will trigger a face detection command in the CognitiveEngineApi.

const params = {
    Video: {
        S3Object: {
            Bucket: 'BucketOfStoredVideos',
            Name: 'RandomVideoWithPeoples.mp4',
        },
    },
    CollectionId: 'rekognitionFaceCollectionId',
    NotificationChannel: {
        RoleArn: 'ArnOfRekognitionRoleFromIAM',
        SNSTopicArn: 'ArnOfSNS',
    },
};
const rekognitionJob = await rekognition.startFaceSearch(params)  .promise();

Upon completion of the analysis process, the Rekognition Engine will notify through SNS. So we need to have one more lambda which will be integrated with this SNS. Here we call this lambda as CognitiveServiceTrigger as can be seen in the architectural diagram above.

If the analysis completed successfully then in the SNS topic it sends the RekognitionJobId through which we can query the result from the Rekongnition Engine.

const params = {
    JobId: 'rekognitionJobIdFromSns',
};
const faceDetectionResult = await rekognition.getFaceSearch(params).promise();

facedetectionResult is a JSON object which contains JobStatus and a map of detection person with relevant data. Details of which can be found at this link.

Now depending on your business need you can extract and process data from the returned result which can be stored in DynamoDB ( or even in S3).

So far I have discussed a pretty simple use case with the Rekognition engine but it’s possible to do incredible things with this engine like sentiment analysis, finding and reading text in images, and on video that is hard to see with the naked eyes, etc which we are doing at Craftsmen!

If you are still reading this line then I hope, at least you got an idea about a standard serverless architecture with AWS Rekognition engine to do different ‘cognitive’ magic

Thanks for your time!