First class responsive video support with the new Mux plugin

When building Sanity, we placed much emphasis on building out a super flexible image pipeline that can transform and re-crop your images for you. We have no ambition of doing the same for video.

When Mux reached out to us about integrating with their video streaming and analytics platform we had no doubts that it would be a good fit.

Mux was founded in 2015 by Jon Dahl, Steve Heffernan, Matt McClure, and Adam Brown. Jon and Steve were the co-founders of Zencoder, which got acquired by Brightcove. With a team of engineers from YouTube, Twitch, Brightcove, and Facebook, they're now leading one of the best cloud-based video services on the web.

Mux is an API-first video platform for streaming and analytics. What sets them apart is their self-optimizing encoding and delivery technology. The service provides high-quality streams for your users that adapt to device, and network conditions. What you also get is transparency on how your videos perform. Mux makes this easily accessible with Mux Data, which gives you detailed video performance metrics – in real-time.

The performance and functionality of Mux are excellent. Store your full resolution master file with Mux and request whatever format you need – HLS for streaming, MP4 for downloads. Even GIF if you need it. From your frontends, apps, and digital services you can request exactly the format you need.

The Mux plugin for Sanity gives you a way to upload and preview videos easily.

Run this command in your Sanity project folder:

sanity install mux-input

To use the Mux video input, you create a field as you otherwise would, and use mux.video for the type.

export default {
  name: 'catVideos',
  type: 'mux.video'
}

Register for Mux (if you tell them “sanity” they'll give you $50 worth in credits ✨). If you don’t yet have a project on Sanity - we’ve made a special offer also.

Once you're logged in to Mux, find the “Access Tokens” page in the settings menu. Generate new tokens with full access on Mux Video and Read on Mux Data, and have them handy for the next step.

Back in Sanity Studio, find the document where your video field appears, and click the plug-icon 🔌on the Paste your Access Token and Secret Key.

Uploading a video is easy: Use the select button to open the file explorer on your system, drag the file right into the input area, or paste the URL (like I do in the video) to the video in the field. Once it's done uploading, you can select the thumbnail you want for the preview.

In this example, we have added the Mux video as a custom block in rich text. We can then use GROQ to generate the URL for the stream on the frontend. Here we also use concatenation to get the streaming URL out of the box.

*[_type == "post"]{
  ...,
  text[]{
  	...,
	  _type == "muxVideo" => {
  		...,
	  	asset->{
  			...,
	  		"url": "https://stream.mux.com/" + playbackId
			}
		}
	}  
}

If we pick out the asset data from the response to this query, we'll get something like this:

{
  "_key": "121c8c30a649",
  "_type": "muxVideo",
  "asset": {
    "_createdAt": "2018-11-30T18:27:21Z",
    "_id": "066e45f9-e2e6-4537-8b40-05f8c0f334d9",
    "_rev": "0xLJdqcI4pgly0b1J2cj0o",
    "_type": "mux.videoAsset",
    "_updatedAt": "2018-11-30T18:27:27Z",
    "assetId": "KdUXsmAKryppWd1wPiAtNVhMIqc7cPmL",
    "data": {
      "aspect_ratio": "16:9",
      "created_at": "1543602441",
      "duration": 28.233333,
      "id": "KdUXsmAKryppWd1wPiAtNVhMIqc7cPmL",
      "master_access": "none",
      "max_stored_frame_rate": 30,
      "max_stored_resolution": "HD",
      "mp4_support": "none",
      "passthrough": "066e45f9-e2e6-4537-8b40-05f8c0f334d9",
      "playback_ids": [
        {
          "id": "NZQMBbYtVa6pDebOjB8wXRNQvao3RWrv",
          "policy": "public"
        }
      ],
      "status": "ready",
      "tracks": [
        {
          "duration": 28.142585,
          "id": "Tcte02pHV5iGVYDUqGX5hiT5XqgB8pMym",
          "max_channel_layout": "stereo",
          "max_channels": 2,
          "type": "audio"
        },
        {
          "duration": 28.166,
          "id": "ocguJveQvPh95zPcnuitsLLXvLYaTAMwPfgDoqFLD01Q",
          "max_frame_rate": 30,
          "max_height": 1080,
          "max_width": 1920,
          "type": "video"
        }
      ]
    },
    "filename": "upload video with mux.mp4",
    "playbackId": "NZQMBbYtVa6pDebOjB8wXRNQvao3RWrv",
    "status": "ready",
    "url": "https://stream.mux.com/NZQMBbYtVa6pDebOjB8wXRNQvao3RWrv"
  }
}

As with our image pipeline, we make sure to have the metadata readily available. This means that you also easily can query for video assets by all these keys. For example will the query *[_type == "mux.videoAsset" && data.aspect_ratio == "16:9"] return all uploaded videos with a letterboxed aspect ratio.

The final piece of the puzzle is to get your videos to your end users. You can integrate videos on Mux with all sorts of stuff, be it the web, apps, TV platforms, or even offline by providing your users with an mp4-download. We made a video player for React that is easy to integrate into your project. Check it out on GitHub or in the CodeSandbox beneath.

We're super happy that Mux reached out to us, and that we get to stand on their shoulders when it comes to video support. We are planning to improve the integration even further, and we're stoked to see what you will make with it. For a more technical and in-depth read about integrating Mux on web frontends, do check out the blog post by our friends in ZEIT, who just launched their new platform ZEIT TV using Mux.

And finally, a cat video. Because everyone should have at least one cat video in their video archive.