Automatic plug-in cloud storage for IoT devices

So you got a good deal for a cloud object storage solution from one of the big cloud vendors? Every day your devices upload thousands of images to the cloud and you are very enthusiastic about the reduction in operating cost and want to move forward ASA. But, wait a minute, does the software running on your hundreds of IoT devices include a driver to interact with this new cloud storage? No worries, you will find out from the device’s software vendor. Oh, it doesn’t? Hmmm. What now then?

And… hopefully everything works out well.

If that sounds to you like a de facto vendor lock in, it’s probably exactly what it is. Now, let’s imagine that your IoT device could upload your files to any cloud storage backend you wish without needing to change its software. That sounds better for sure. But, can this really work?

Let me share with you how Sony’s EVP (Edge Virtualization Platform) enables cloud storage in a very easy and elegant way or as I like to call it – ‘Automatic plug-in cloud storage for IoT devices’.

Let’s rollback a bit before we dive into the details.

Uploading large amounts of data to a cloud storage while keeping the MQTT channel free for control only, is essential to many IoT based solutions. In the case of Sony’s Aitrios, for example, users are likely uploading images to the cloud for such reasons as camera setup and tuning, deeper inspection, or AI model training.

Using cloud storage solutions, such as AWS S3, Azure Blob storage, or GCP object storage, is very appealing to Sony’s Aitrios customers as they probably already have accounts with some of these cloud providers for other uses.

So why do you need EVP? Well, EVP allows you to choose which cloud storage to use without worrying about anything else that concerns your IoT deployments. With EVP, the only thing you need to do is to configure your cloud storage endpoint and credentials, all through a single API call. The credentials always stay safely on the EVP cloud side, so the devices never need to hold them. Since IoT devices are more likely to be compromised than the cloud service, with EVP you are storing your sensitive credentials in a more secure environment – IoT device security should not be completely trusted. You configure a ‘storage access service’ in EVP by specifying only a few parameters, for example storage end-point or connection-string.

Here is an example of a POST request body for configuring ‘storage access service’ in EVP, on an Azure Blob in this case:

{ “type”: “AZURE”,
“endpoint”: “http://someblob.core.windows.net”,
“connectionString”:
“DefaultEndpointsProtocol=https;AccountName=youraccount;AccountKey=keyforaccount;EndpointSuffix=core.windows.net”,
“headers”: {
“x-ms-blob-type”: “BlockBlob”
},
“certificate”: “cert-value”,
“expirationDuration”:”PT2M”
}

Backend

1. Create your cloud storage account in AWS, Azure, GCP, or similar.

2. Use EVP REST to configure a ‘storage access service’ in EVP backend. providing your cloud storage details.

3. EVP backend connects to the cloud storage you have configured to get and cache an access token.

4. EVP runtime, the part of EVP which runs on the device that provides an API to the applications running on the device, allocates a designated directory in the file system for every module (application) running on the device (not shown in the diagram).

Device

1. When a module wishes to upload a file to the cloud storage, it saves the file in its designated directory and calls EVP_blobOperation() providing the file-name to upload. (this method is part of the EVP runtime SDK).

2. EVP runtime sends a request to the EVP backend for all the information needed to upload the file to your cloud storage (see blue lines in previous diagram).

3. If required (for example if the storage token expired) the EVP backend interacts with the cloud storage to renew the access token.

4. Using the access token, the EVP backend generates the full URI including the POST request header and sends it back to the EVP runtime on the device.

5. If the EVP runtime does not have the right certificate to talk to the backend storage it requests one from the EVP backend and stores it for future iterations.

6. The EVP runtime uploads the file directly to your cloud storage by making a POST call and using the URI and the request header it got from the EVP backend.

7. When the operation completes, the EVP runtime notifies the requesting module by executing the module’s callback function.

As we can see, it is only the EVP backend part that has to know the storage details. Practically, these are the details (specifications) of how to interact with different backend storage types over REST. EVP, being a SaaS running on a cloud, makes it fast and easy to add support in the backend for any new backend storage type.

There are many advantages for this approach which bring value. I have listed some here (I bet you can think of more):

• No change is required in device code when changing cloud storage provider
• No cloud-storage-specific code is required on the device, meaning a small code footprint, something that is very important for devices with few resources
• Libraries provided by the storage vendors to abstract the interface with the actual storage are usually provided for high-level programming languages, such as Python, Java, or C#, which fit poorly with low-resource embedded devices
• It is relatively easy to add support for new storage type, achieved only on the backend side
• No security downside of holding storage-related keys/tokens on the end device
• You can change cloud storage very easily – no fleet operations required
• You get tighter control on device access to your storage, meaning you can use EVP backend API to revoke or block any specific device immediately

Now, if you go back to the title of this post, do you think it is very far from reality?