How to Download from Azure Blob Storage with Streams (using Express)

by William Andrew Griffin

While working on our React Native app–Mender–we needed a simple way to upload and download images. Our backend is hosted on Azure using Node.js and Express, so Azure Blob Storage was a natural fit for our image store.

A lot of great articles exist explaining how to upload to Azure Blob Storage. Unfortunately, few do a good job explaining the details necessary for downloads. Even Azure's documentation leaves a lot to be desired. The majority of the articles provide steps to download blobs directly to the filesystem. While this works in some use cases, we needed something that could easily transport images without having to write it to disk or load the entire image in memory before sending it to our mobile app.

That is where streams come in...

Streams allow us to read and write data in chunks, passing them to another process or system as we receive or send them. This makes our application memory efficient (only handling a small portion of the data at a time) and speed efficient (since there is no disk I/O or waiting for the data to be fully collected in memory).

Passing the data to another process, or piping it, we are able to continue using the benefit of streams further down in our application process. In our case, Mender is able to read chunks of image data from our backend, which itself is reading data chunks from Azure.

For a more detailed introduction to streams, check out Liz Parody's article, Understanding Streams in Node.js.

Implementing the Download Process

Over the next few sections, we will take you through the components we use on our Express server and React Native application to download images.  The application process flow is:

  • React Native app sends an HTTP get request to an endpoint on our backend server (requesting the specific image file)
  • Server uses the request parameters to start the image file download from Azure
  • Server pipes the download back to the HTTP response of the original get
  • React Native app processes the image file

Let's start with the code we use on our Express server and then move to the client side, our React Native application.

Connecting to Azure

To interface with Azure, we use their @azure/storage-blob client. We created an object with properties that handle the interactions we want to perform: delete, download, upload. These properties store asynchronous functions that manage details needed by the storage-blob client. Our object is used on the Express server, so we are able to securely load our Azure connection string as an environment variable.

The download function takes in the container and filename for the specific image. It returns a Promise that resolves to a data object that contains a Readable stream: readableStreamBody.

const { BlobServiceClient } = require('@azure/storage-blob');

const azureBlob = {
    const AZURE_STORAGE_CONNECTION_STRING = process.env.AZURE_STORAGE_CONNECTION_STRING;
    delete: async (containerName, fileName) => {
    },
    download: async (containerName, fileName) => {
        // Create the BlobServiceClient object which will be used to create a container client
        const blobServiceClient = await BlobServiceClient.fromConnectionString(AZURE_STORAGE_CONNECTION_STRING);
        // Get a reference to a container
        const containerClient = await blobServiceClient.getContainerClient(containerName);
        // Get a block blob client
        const blockBlobClient = containerClient.getBlockBlobClient(fileName);
        return blockBlobClient.download(0);
    },
    upload: async (containerName, fileName) => {
    },
}

module.exports = {
    azureBlob
};

Triggering the Download Function

To trigger the download, we use Express Router's get method. Our React Native application sends a get request to our /api/avatar/:container/:imgUrl endpoint on our backend server. We use the route parameters container and imgUrl, of the request URL, to specify the blob container and filename. These string variables are fed in to our azureBlob.download(container, fileName) function we presented in the previous section.

const router = require('express').Router();
const { azureBlob } = require('../azure');

// Get route for avatars
router.get('/avatar/:container/:imgUrl', (req, res, next) => {
    // Start blob download from Azure
    azureBlob.download(req.params.container, req.params.imgUrl)
        // Pipe download stream to response
        .then(downloadBlockBlobResponse => 
            downloadBlockBlobResponse.readableStreamBody.pipe(res))
        .catch(error => {
            next(error);
        })
})

module.exports = router;

The Trick

This line of code is the how we are able to benefit from streams:

azureBlob.download(req.params.container, req.params.imgUrl)
        .then(downloadBlockBlobResponse =>
            downloadBlockBlobResponse.readableStreamBody.pipe(res))

When we receive the blockBlobClient.download(0) object from our azureBlob.download function, we are receiving a Promise that contains a readable stream of data. Using .pipe(destination), we are then able to send these incoming chunks of data from Azure to our React Native application using the HTTP response object, res.


– Node.js official documentation –
The readable.pipe() method attaches a Writable stream to the readable, causing it to switch automatically into flowing mode and push all of its data to the attached Writable. The flow of data will be automatically managed so that the destination Writable stream is not overwhelmed by a faster Readable stream.

Starting the Download from React Native

When the app needs to update a user image, for example when a user changes their avatar, it starts an asynchronous download. Using Expo for our React Native development, we are able to use their FileSystem component to handle the data stream coming from our Express server. (A side note on Expo: it is powerful development and build tool that vastly streamlines these processes across Android and iOS.) The application starts the download with two variables: the remote URI to download and the local URI of the file to download to.

  1. remote URI example: "https://expressserver.company.com/api/img/avatar/avatars/photo-5ea10a1f3a7b30e38143ec00.jpeg"
  2. local URI example: "file://avatars/photo-5ea10a1f3a7b30e38143ec00.jpeg"

During the image upload process, the application selects a random, unique filename and the respective container associated with the image as it appears on Azure Blob Storage. This URI string is stored in mongoDB as the imgUri. To download an image from Azure the application first queries mongoDB for the imgUri.

  • imgUri example: "avatars/photo-5ea10a1f3a7b30e38143ec00.jpeg"

The following React Native function is used to start the download process. It takes two variables: the imgUri to download and the API endpoint on our Express server to use.

import Constants from 'expo-constants';
import * as FileSystem from 'expo-file-system';

// Express server URL
const SERVER_URL = Constants.manifest.extra.SERVER_URL;

// Image download function
const downloadImg = async (imgUri, imgEndpoint) => {
    return FileSystem.downloadAsync(
      SERVER_URL + imgEndpoint + imgUri,
      FileSystem.documentDirectory + imgUri
    )
    .then(({ uri }) => {
      console.log('Finished downloading to ', uri);
      return uri
    })
    .catch(error => console.log(error))
};

The FileSystem component handles the collection of the data chunks coming from our Express server stream, saving then to a specified location. By using the imgUri with the FileSystem.document directory, we are telling the downloadAsync function to use the container as the folder for the image. In the example above, the application would download to "file://avatars" folder with "photo-5ea10a1f3a7b30e38143ec00.jpeg" as the filename.

And that is it. We have used streams in Express to pipe data flowing from Azure Blog Storage to a local file on our mobile device. A lot more can be done with streams, so I highly recommend spending time learning more about them. If you have any questions or need some advice on your Node.js project, please feel free to reach out to us at info@appagetech.com.