360pan suite

If you need to produce localised or spatial audio for 360 VR movies for YouTube, you need to closely pan sounds onto the objects in the video. You also need to output an ambisonics stream for upload, and you want to listen to it exactly the way it will sound on YouTube, interactively turning while seeing what you’ll see on YouTube. All this can very easily be done with just two Pro Tools AAX or VST plug-ins: 360pan and 360monitor.

Please note The current release is AAX and for Mac OS only. VST support will follow later, Windows support is not planned.

Buy the 360pan suite now

360pan

360pan AAX plug-in screenshot

360 ambisonics panner, operated directly from the Pro Tools video window.

360monitor

360monitor AAX plug-in screenshot

YouTube ambisonics to headphones, plus video, interactive turning, all from within Pro Tools.

News
360radar preview

360radar (AAX, VST) displays, onto your video window, where your ambisonics source sounds are, whether these sounds are artificially panned or recorded using an ambisonics microphone.

360radar will be part of the upcoming 360pan suite v2.

What is 360pan suite for ?

There are a lot of names for the type video that the 360pan suite can deliver spatial (or localized) audio for:

  • 360° video
  • VR video
  • Virtual Reality video
  • Panoramic video
  • Spherical video

to name the most common.
These type videos require a soundtrack that can turn when the head of the viewer with VR goggles on, or his phone, is turned.

There are a number of playback platforms for 360 VR videos.

  • Facebook 360
  • Oculus Video
  • YouTube
  • SamsungVR
  • Jaunt

to name a couple.

All of these chose Ambisonics as the technology that allows for the interactively turning mix. This means that audio-post folks need to produce ambisonics mixes. Ambisonics mixes are in a format called b-format, which consist of four channels. B-format comes in two channel orders: AmbiX or FuMa.
Youtube requires AmbiX b-format ambisonics soundtracks, so this is what 360pan does.
We have a couple of converters (in the making) to convert between channel ordering though.

360pan

A 360 video is delivered to you flattened to a rectangular (or equirectangular) video, that plays back in the Pro Tools video window. Ideally you could pan your sounds exactly onto the objects in the video.

This is what 360pan is made for.

360pan output ambisonics b-format AmbiX ordered stream, and offers a position indicator (a puck) in a video overlay window that appears on top of the video window of your DAW (i.e. Pro Tools).

Video overlay

The power of the 360pan suite is that you can operate most relevant parametres right from the Pro Tools Video Window itself. In any 360pan plugin switch on show Video Overlay.

On top of the Pro Tools Video Window the overlay appears. Many spherical videos come with a black bar at the top and bottom. It is advisable to adjust the size of the video overlay grid to fit the actual video part, to exclude the black bars. This is done by grabbing and dragging the thick grey rim of the overlay grid. When set properly no black holes are visible in the 360Monitor when you look straight down or up.

Each 360pan plug-in instance shows one or more pucks in the overlay. Several knobs at the top of the overlay govern the visibility of the pucks. Right clicking a puck in the overlay brings up a popup menu in which you can change the appearance of the puck.
Hold down shift while dragging a puck to move all pucks of that particular 360pan by the same amount. This way you can easily turn the four pucks of a four channel input 360pan.
Any dialogue or plug-in interface that appears on top of the video window appears between the video and the overlay. Therefore it will not receive mouse clicks. Click the top left hide-overlay button to make the controls of that interface available, as well as the right click in the Pro Tools video window to access the video controls within Pro Tools.

360pan AAX plug-in screenshot

360monitor

360monitor AAX plug-in screenshot

While you need the Ambisoncs b-format mix to bounce it to the 360 video for upload to YouTube, you can also preview what it will look and sound like using the 360monitor.
360monitor shows a typical interactive video screen that allows tilting and turning, and it turns the audio along with it, while rendering the ambisonics down to binaural stereo for delivery on headphones. The algorithm used is the exact YouTube algorithm. The video is grabbed from the open Pro Tools video window to be expanded.

Tip: 360monitor can be heavy on the CPU when it is visible. This is because it is transporting frames from the Pro Tools video window to the 360monitor user interface. The smaller the Pro Tools video window, the lighter 360monitor operates.

360monitor output

stereo
For regular use and monitoring the YouTube sound you need a 360monitor that inputs 4 channels (AmbiX b-format) and outputs stereo (binaural for headphone).

quad
Mixing on headphones can be a drag. This is why there is also a 360monitor for a quadrophonic (quad) speaker setup. This is the 360monitor with 4 input and 4 output channels. Connect 4 speakers L R Ls Rs in a square and sit in the centre to properly hear the placing of all pucks. Of course now all height information cannot be appreciated.

Converters

360fuma to ambix
360ambix to fuma

The two main flavors in b-format channels are:

  • FuMa
    channel ordering WXYZ with W 3dB lowered compared to AmbiX
  • AmbiX
    channel ordering WYZX

360pan suite, Facebook, Oculus Video, YouTube, SamsungVR and Oculus all use AmbiX b-format.

Currently some technologies use FuMa ordering as their upload requirement. If you will be using the 360pan suite to create audio for:

  • JauntVR VR player
  • Audiokinetic's WWise
  • TSL's Surround Zone
  • HarpexB

Then make sure you insert a 360ambix to fuma in the ambisonics channel that you bounce, in order to get the FuMa channel order (and normalization) that is required.

Specifications

360pan suite currently features four plug-ins:

  • 360pan
    pan your (mono/stereo) source to ambisonics using the easy video overlay.
  • 360monitor
    monitor ambisonics audio and 360 video as if played on YouTube.
  • 360fuma to ambix
    convert FuMa ambisonics audio to AmbiX.
  • 360ambix to fuma
    convert AmbiX ambisonics audio to FuMa.

The 360pan suite of plug-ins is protected using a single license. An iLok 2 USB key can be used but is not required, the license can also be activated on a machine (computer). An ilok.com account is required for this.

Requirements

  • Mac OS 10.8.5 or up
  • Pro Tools 11 or up
  • an iLok.com account (no ilok required)
  • Pro Tools HD (or capability to use quad tracks)

Please note: This initial release contains AAX for Mac OS only, other formats will follow.

Download

Request a link to the downloads page with the latest 360pan suite downloads.

Download 360pan suite

Version history

The 360pan suite 1.0.1 for Mac changes:
Added plugins for FuMa to AmbiX and AmbiX to FuMa conversions
Corrected height parameter in 360monitor
Improved w-channel scaling in panner
Puck colors are stored in session now
When opening new panner vertical video offset remains
Several UI improvements
The 360pan suite 1.0.0 for Mac release:
Initial release, AAX, for Mac OS.

Support

Installation and authorization

The 360pan suite of plug-ins is protected using a single license. An iLok 2 USB key can be used but is not required, the license can also be activated on a machine (computer). An ilok.com account is required for this.

There is only a single AAX plug-in which, once installed, will manifest itself as a number of plug-ins in Pro Tools.

Installation is done by unpacking the downloaded zip file, opening the resulting folder and dragging the 360pan suite.aaxplugin into the alias (shortcut) to your AAX Plug-Ins folder.

Please note: The initial release contains AAX for Mac OS, other formats will follow.

Manual

Read the online manual and start with downloading the example project.

Example project

Quick start: download & check out the example Pro Tools session.

Download plug-in

Request a link to the downloads page with the latest 360pan suite downloads.

EULA

The 360pan end user license agreement (eula).

Call for help

+31 30 244 6335

SUPPORT HOURS
MONDAY-FRIDAY 9AM-5PM CET

LOCAL TIME @AudioEase
TUESDAY 12:38 PM

Free goodies

Here we offer a bunch of free ‘spatial treats’ that we use at Audio Ease ourselves on a daily basis. There are no warranties, and almost all of it builds upon either Google's Omnitone or FFmpeg.

Offline 360 video player
with spatial audio!

Over here

Preview your 360 Video with immersive audio right from your own hard drive or network instantly. No upload, not public, no social media, no trickery, no jump inspector or android required.
Works in Firefox on Mac or Windows. Nudge forward and replay seems broken in Chrome, Safari won’t do a thing at all.

offline player

Go now

Because the video is nice and big it this sometimes preferred over previewing in our 360monitor plug-in.
AND the best part: you don’t need extra’s like jump, or an android phone. This page uses Omnitone, Google’s own 360 web API which means you are listening to the same ambisonics to binaural rendering as the android and desktop clients viewing YouTube use. It being Google’s also means it likes .webm. Of course none of the data compression that is going on might be happening when you actually do upload to YouTube. However this is a lot quicker (and safer!) than uploading.

All local

The files play locally from your (network) drive, after loading the page you can pull internet and it would remain working (just to prove that nothing goes into the internet).

Video webm

Point to a 360 video file. webm is preferred. You can convert to webm using one of the handy droplets

Audio

Optionally the site enables you to side-load a separate 4 channel wav audio file. Simply point to an interleaved First Order Ambisonics AmbiX .wav file (4 channels, WYXZ order), hit play, drag to turn, scroll wheel handles distance (field of view).

As hires webm conversion takes a bit of time, we like to make our webm once, and then, for each mix, we just bounce a wave file from Pro Tools and only load this separately.

Check it out here

Mac OS ffmpeg droplets
(including ffmpeg)

Arjen, sadly, has not yet been able to type a single ffmpeg command line right in one try, so Aram built him a few droplets that will perform common tasks on videos.
The result is saved next to the input video. The ffmpeg must remain next to the droplets for them to function.

Download ffmpeg droplets for Mac OS

Pro Tools video droplet

drop any video to convert to a quarter of the resolution to a format that makes ProTools happy and spiffy. (we convert samsung gear stitched 360 stuff with this).

webm droplet

drop a video with 4 channel AmbiX audio (like an exported mov from protools) to make a webm version for the offline 360 video Player

Crop stereoscopic video droplet

crop a stereoscopic video to the left side only, saves cpu load and screen room.

Do yourself a favor and download this. No fiddling with installing xcode or brew to get ffmpeg installed, no terminal embarrassments, simply drag n drop video files and wait...

And of course you are free to open the apps (droplets) with the Script Editor app to check what's going on and adapt if required by taste.

Manual

Example project

Quick start: download & check out the example session featuring a short 360 video, mono panning audio, binaural preview and export suggestion.

download Pro Tools example session

download VST example projects

Basic workflow

If you have a flattened spherical video to work with, import the video in your project and bring up the video window.

  • Give the mono in channel that you want to position a quad (4 channel) output.
  • Then insert a 360pan plug-in on this channel.
  • The 360pan is inserted as the last plug-in in your channel. After compression, eq and such.
360pan suite channel routing

The indicator from the 360pan plug-in (the puck) shows up in the video window (bearing the track’s name), and it can be operated from there as well, so you can close the 360pan plug-in window.
You can insert a number of 360pan plug-ins on different tracks in your project and all pucks will be visible and moveable in the video window.

Note: Top / bottom area mono
Please note that in a spherical flattened video image the zenith, the point right above your head, is stretched out horizontally over the entire top of the image. Moving a puck there will therefore always result in the sound moving to the (top) center. The same goes for the point right below you.

360pan 360speaker zones

Listening to the Ambisonic (b-format) pan output
The 4 channels that are now output of the 360pan are not meant to be listened to directly.
You should route these to a 4 channel master or bus, and in there insert a 360monitor. Use only a single instance of 360monitor, all the way at the end of the signal flow.


The 360monitor collapses the 4 channel ambisonics world to two channels for headphone listening. 360monitor does this using the same technology that YouTube uses when the viewer listens to the resulting 360 video. This step is called ambisonics to binaural conversion and while doing it, the view direction can be changed continuously: you can look ‘around’.
In the 360monitor this is done by dragging the video right inside of 360monitor plug-in window. The audio will turn along with your movement.

The 360monitor intends to give you a 100% truthful preview of what your end product will sound like on YouTube.
If you are disappointed by what you are hearing, than know that this is what your viewers are hearing, and the only way to eq your way towards a better sounding mix is by continuously monitoring using the particular ambisonics to binaural conversion from 360monitor / YouTube.

Preview/mix on four speakers
A 4 input 4 output 360monitor plug-in is also available. This performs an ambisonics to quad conversion for speaker listening. If you are tired of listening over headphones, go sit in between four speakers in a square (the L, R, Ls and Rs of your 5.1 or 7.1 setup for instance) and listen to your work that way.

Panning a stereo signal
A 2 channel input 360pan is available too. It will present two pucks and is intende to place stereo recordings in your mix. You can move the pucks individually or hold down shift to move both of them simultaneously.

Likewise a 4 channel input 360pan exists. This is intended to receive either 4 channel recordings or quad surround reverb.

Export
When you are done mixing, you bounce the 4 channels ambisonics stream to a file for inclusion in the YouTube 360 video. Read more on export below.

360pan automation

You play the video and track the movement of your 360pan pucks.
A circular parameter needs to pass over from max value (+179) to min value (-180) in the rear. This happens at the left and right edges of the video screen.
Audio Ease chose to make the left-right (azimuth) and up-down (height) parameters of type Stepped rather than type Line. Stepped is not the most convenient type when trying to draw automation, or when using Pro Tools’ Glide To All Enabled feature. For this a type Line would have been a lot more convenient.
However Line type automation creates a diagonal line between the maximal and minimal value (179 and -180), making the puck travel back to the other side rather than through the passover point. The length of this diagonal line is all up to Pro Tools and your automation thinning settings. And this is less steep when Pro Tools is more busy. In any event, this can cause glitches and this cannot be prevented.
The stepped parameter does not suffer from this and ensures a smooth pan at all times.

Mix tip: the W channel

Mixing non-turning audio, like Voice Overs, in the W channel.
The first channel of the ambisonics stream is the W channel, which goes to all virtual speakers in the ambisonics to binaural converter. This means you can place non-turning mono audio there, great for voice overs.

In order to use this you should go into I/O settings dialogue of Pro Tools and create a mono subpath in the quad bus you are using for the ambisonics signal.
The mono subpath’s I/O puck should be placed on the first channel of the quad bus. Most likely, by default, it will be named L for left. But this is in fact W in the ambisonics signal. Anything you mix here will not turn and sound from the center of the mix (in head).

Distance

We were looking at available ambisonics panners and thought it was too difficult to attach a mono sound to an object on screen so we made the overlay with the pucks.
This means we have a frontal view of the panning arena, just like the video window. Most other ambisonics panners have a birds-eye (from the top) view.
That is because other ambisonics panners want to be able to easily display and encode distance.

Encoding distance is a game-engine paradigm. In games everything needs to remain interactive, not just the viewing angle. So in games there is no way to provide any sound source with a fixed reverb, because in games you can walk up to the sound source, in which case it needs to sound dry, or move away form it, which would make the source sound wetter.

In a game you can decide to walk away form another guy halfway his sentence, which should make him become distant-sounding at a viewer/player defined moment.

In audio post for picture, which is what Audio Ease caters for, everything you get is linear, cinematic storytelling.
In 360 video, the only thing that needs to remain interactive is the viewing angle. The viewer does not get to decide where he is himself, nor how far away the other people are. So reverb levels can be balanced to perfection, dopplers too, early reflections as well. There are no cpu-load limits. The beauty of the mix, lets call it audio quality, can therefore be way higher than in games. Audio quality can be as high as in, you guessed it, audio post for picture.

In audio post for picture, which is again what Audio Ease caters for, panners do not do distance. Distance is handled by a volume fader.
In audio post mixers have their own methods to create distance in sound. Tricks like eq’ing and Pre-fader sends to make reverb stay at a fixed level while the dry sound is faded out, which creates a distance effect.
In audio Post people use higher quality reverbs, higher quality doppler plug-ins (if required at all), and most certainly higher quality early reflections than in games.

This is why we have chosen not encode distance at all in 360pan suite. And we think that is best for 360 audio post, or 3D Audio or Virtual Reality cinematic linear storytelling.

Export 360pan suite audio to YouTube

Follow these steps to upload 360 video with ambisonics audio to YouTube. So that the viewer gets interactive turning audio when looking around in the VR video.

Here's the official YouTube guide:
https://support.google.com/youtube/answer/6178631?hl=en

Here's our recipe:
Mix your audio and place (pan) it using the 360pan suite.
Bounce the resulting quad audio to:
- an interleaved file and ask your video producer to add this to the video
- or bounce directly to the quicktime video, using pro tools

In both situations bounce as quad (four channel) interleaved wave audio file (not compressed).

Then download the Spatial Media Metadata Injector app

Open your video with the quad ambisonics audio in the Spatial Media Metadata Injector app.
Check the first and last box (Video is spherical and audio is spatial, ambiX)

Spatial Media Metadata Injector app

Inject metadata, upload to YouTube, wait an hour for your 360 video to process. Watch in a modern browser (Chrome works for sure, Firefox and Safari might work too) to verify you get spatial audio (look around with headphones on!).