Falcon Onboarding: Captioning Your Live Streams | Webinar Replay

On May 14, 2020, EEG Video hosted a webinar to educate media professionals about our Falcon live streaming caption encoder.

Falcon Onboarding: Captioning Your Live Streams • June 25, 2020

In this 30-minute webinar, Elan Grau, Technical Support Specialist for EEG, and Sam Windwer, Lead Software Engineer for EEG, instructed event attendees on how to caption live streams using Falcon and provided a thorough walkthrough for setting up and managing the service.

Topics covered included:

• All about the Falcon RTMP caption encoder

• How to set up Falcon and navigate the dashboard

• Adding captions to live streams

• A live Q&A session

To find out about upcoming EEG webinars, as well as all other previously streamed installments, visit here!

 

 

Transcript

Elan: Hi everyone. Thank you for joining us today for our Falcon webinar, which is going to help you guys get your live streams captioned. My name is Elan Grau. I'm one of the members of the support team here at EEG, and with me I have Sam Windwer.

Sam: Hi, I'm the lead software engineer and I'm gonna help Elan at the end answer your technical questions.

Elan: So as your host here, I'm going to be walking through and discussing how to set up Falcon so you can deliver captions to a live stream, and how to optionally use Lexi, our live automatic captioning. We've seen a lot of growth in the Falcon customer base over the past few months, which can be attributed to the significant uptick in streaming. We see the slowing - we don't see the slowing down anytime soon. People in companies are adapting to these changes and finding ways to make the virtual experiences as powerful and seamless as possible.

Even as in-person meetings and events continue to move online, providing accessibility is still just as crucial. We want to show our current and future customers just how easy it is to caption these events, no matter what your workflow. EEG makes captioning capability possible for anyone who is streaming live content.

I just want to mention that today's webinar is being captioned by EEG's automatic cloud-hosted captioning service and is being recorded to be uploaded to both our website and our YouTube at some point during this week.

If at any point you have any questions, feel free to enter it into the Q&A section located at the bottom of your Zoom. Sam and I will work together at the end of the webinar to answer your questions. And if you have any follow-up after the webinar's ended, feel free to reach out to either our support email or our sales email.

OK. Throughout this webinar we're gonna discuss how Falcon works, setting up a Falcon stream, the Monitor tab, the dashboard, utilizing Falcon with Lexi, and the final setup and captioned stream, and then we're gonna go ahead into the Q&A. We're approaching this webinar with the assumption that you're either interested in using Falcon or you've already purchased Falcon and are just looking for some helpful - helpful hints or tips to get your live stream up and running with captions as easily as possible, but just a quick review of what the product does for anyone who might be new to Falcon.

OK. So Falcon is EEG's cloud-hosted software-as-a-service streaming video solution that adds real-time closed captions to your live RTMP or RTMPS streams.

Falcon Flowchart

So this is how Falcon works. Your live video source and program audio is uplinked to the cloud using RTMP through a streaming encoder, which could be AWS Elemental, Telestream Wirecast, Teradek Cube, or OBS, just to name a few. Your program audio is captioned either by a live captioner using EEG's iCap service or by Lexi, EEG's automatic captioning service, and live caption data is returned to Falcon right away and embedded into the stream.

Features

This process does not re-encode your video, is entirely private and only adds one second of latency. Any video hosting platform that supports RTMP input with embedded captions will be able to process, store, and display the captions that are created through Falcon. Falcon can support an input bandwidth of up to 10 megabits per second - er, RTMP stream, I'm sorry. Captions can be viewed in players on webpages, mobile devices, as well as VOD recordings.

Setting up Falcon

That's the general overview of what Falcon does, so let's move on and show you how to actually set this up. First, you're gonna need a Falcon subscription through eegcloud.tv. This is where the Falcon software is accessed and managed, as well as other cloud products that can be utilized in this workflow, such as Lexi, which we're going to cover a little bit later to show how to integrate automatic captioning directly into this workflow. Once you're logged in, you'd go over to the Falcon tab and then create a new Falcon.

You'll be directed to the Configuration page, where you'll be able to customize your stream name and add tags that are going to categorize Falcon using simple keywords to keep track of any instances that may be active or that you're monitoring.

The Configure Access Code section is where you tell Falcon how you will be captioning your stream. There are two ways to do this. First, you can create an iCap Access Code, which allows your authorized captioners to have access to the stream by adding them as a caption agency - sorry. EEG works with many captioning services and most of them should be accessible by entering the first few letters of their name. If they don't show up in this field, which you can select and load directly into the caption agency's field, feel free to contact our support and we'll be glad to make sure that they are added to the list.

If you have a Lexi subscription, you're going to want to add EEG Lexi ASR and that will act as the captioning agency for your Falcon stream. Second, you can link it to an existing iCap account if you've worked with us before in the past and already have an iCap Admin set up.

The output settings by default do not enable pull permissions on the stream and it's required that you push the caption RTMP stream to one or more destinations, such as Facebook Live, Wowza, Twitch, Ustream, or others. So if you're looking to enable pull permissions to allow this Falcon to host your stream so you can have an external server pulled from it, you would check off the corresponding box, which will bring up a new field where you can select caption format. Choosing a stream type is pretty straightforward.

You just click on the field, which will show you the common advanced and standard destinations utilized by most users. Once the destination is selected, you'll need to retrieve the stream key and possibly stream URL from your destination, depending on which one you've already selected. Each destination may vary where that information can be found, so be sure to take a look at that beforehand so you know where everything is in the configuration settings on their end. Once you have that information, you just copy and paste it into the corresponding fields in Falcon and click Add Destination. You can also add numerous destinations to the stream in order to push to multiple platforms at once.

Additional settings allow you to enable test captions and make sure everything is working up in your workflow and everything is set up properly. You can also add tags and geographic - sorry about that. You can also add tags and geographic region options in order to further customize your streams. Once you have everything set, you just click on the Launch button.

Monitor Tab and Settings

The Monitor tab and finalizing the settings here will bring you in a little bit more - I'm sorry, scratch that. We're gonna bring that in a little bit more in depth in a moment, but first we have a few more settings that we have to finish up with in order to get this finalized. Under the output information, there will be an RTMP ingest URL and stream key. These need to be copy and pasted into your corresponding fields on your streaming encoder, much like retrieving the stream key and URL from your destination, where this information is - is dependent on which streaming encoder you're using at the time.

We've updated our interface to show the iCap encoder username created if you linked Falcon - a Falcon stream to an existing iCap account. We've also added the ability to drop your publisher if you're running into any issues with streaming - with your streaming encoder. This will allow you to break the connection that might be hindering you from publishing to your - to your Falcon.

So the monitor portion of this tab is going to display your input and output video of your stream. You should be able to see your latency on your input video, and your output video should be displaying captions and sending audio. Both of these can be toggled on and off as required. You also have the option to monitor captions using the iCap PC Client or you can enter the captioner credentials, provided you have them, and the Access Code of your stream in order to view captions and supervise - supervise audio quality of the Falcon stream as it's being pushed to iCap.

Additional Dashboard Tabs

Alright, so let's go over the remaining tabs here. Settings tab is where you can quickly access and edit your configuration settings. The Status tab shows the stream's input and output statuses in detail. The Logs tab allows you to access events happening within your stream and you can easily filter for a specific event by utilizing the log level drop-down menu. The Download Captions tab makes it easy for the user to retrieve the captions to store them or to repurpose for later use.

Utilizing Falcon with Lexi

Earlier I had mentioned that Falcon and Lexi can work hand in hand for a fully automated captioning delivery. Here's how that setup works. If you've already signed up for Lexi - for a Lexi subscription and you've added Lexi as the captioning agency for your stream, on the EEG Cloud dashboard you're gonna want to go over to the Lexi tab and create a new Lexi job. If you're interested in a new Lexi subscription, feel free to contact our sales department.

Enter in the Access Code for the Falcon stream you created. You can adjust the settings as needed, such as language selection, inactivity timeout, caption service, and even the caption positioning.

One of our newer features through Lexi allows you to enable multi-speaker settings to assist Lexi with crosstalk and conversational material. With Lexi Vision, which reposition - repositions captions and will also be able to learn new vocabulary based on the on-screen graphics and text.

Once you have your settings finalized, you want to click on Start Live ASR. This will bring you to your Instance Monitor page and initiate your Lexi job.

And let's take a look at the end result. Let me just pull this up here. OK. So here we have the Falcon Monitor page. Here you can see your input and your output. These captions are being auto-generated by Lexi right now. As I said before, you'll be able to toggle the CC on and off, you can adjust the audio as needed, etc. So Falcon is essentially acting as a middleman between your streaming encoder and your endpoint, taking in the video and the audio, pushing that audio up to the cloud where it's being captioned, sent back down into your output video, and here we have the end result.

I do want to point out that the caption style and size, font, etc., that's dependent on the decoder itself. Falcon only does the encoding, so you might see different results based on which player you're using, so this might look different than, say, YouTube or Twitch.

Q&A

Here we go. Alright, so that pretty much wraps up how to utilize Falcon and to caption your streams, and we'd be happy to take any questions that you may have. So we do have one question right off the bat here.

So right now that was actively being captioned by Lexi, the automated captioning system. I had set up an instance previously using OBS with just a TED Talk video we had and running that video through Falcon with Lexi as the automated captioner. I don't know if Sam can really elaborate beyond that, but that's pretty much it -

Sam: Yeah, no, that's pretty much what's going on, yeah.

Elan: So Falcon - the next question we have, I'm sorry, is how many simultaneous destinations does Falcon support? It can support up to 10 right now. Sam, do you want to elaborate on that a little?

Sam: No, that's pretty - pretty straightforward. Yeah, you could send to just one or multiple or up to 10, and they all get the same caption stream that is coming out of Falcon.

Elan: OK, I have one here. Been successfully using RTMP in and out of Falcon. What is required to use RTMPS or some other method of a secure stream? I'm using video enterprise - Vimeo Enterprise and I'm interested in how to do this. Sam, would you mind taking lead on that one? I'm not 100% familiar with RTMPS.

Sam: Yeah, I could jump in on this. Yeah, so Falcon supports RTMPS, basically - actually Elan, if you could pull back up the Falcon window that you had–the Monitor page–because you could see there at the top the ingest URLs and see how there's a little drop-down that says RTMP, you could change that to RTMPS. And so if you select the RTMPS option there and enter - and that will basically change the protocol in the URL to RTMPS, and use that in your streaming media encoder, then the stream from your encoder to Falcon, it will be encrypted. But that's only one side of it.

Now on the other side if you want the stream from Falcon to your ultimate destination, like you say Vimeo, to be - to be encrypted, then you have to also make sure that when you're configuring that destination that you're configuring that with a corresponding RTMPS URL. And so that way we could have RTMPS in and RTMPS out.

Elan: So another question we got here is what is the best way to get Lexi to improve recognition?

We have an option called Topic Models where Lexi can essentially be trained to use phrases that you expect to see more frequently in your particular broadcast. You can upload direct text files that would include things like names, stuff like that, locations. You can also add webpages so that this way they can be scanned and Lexi can essentially learn and categorize frequent terms that are being mentioned on that website, same as with text document. Sam, I don't know if you have anything to add on Topic Models.

Sam: Yeah, basically the Topic Models is the main thing. I mean, the other thing I would say is, if this is possibly in the scope, is to try to give Lexi, like, a clean audio feed without a lot of background noise. Like, it does the best when they're in an environment where there's, like, a clean feed with a single speaker. And then, yeah, as - as Elan said, in order to improve it - improve its performance, proper nouns and, like, jargon and stuff, then that's where the Topic Models could really help.

Elan: So we got another one here. Is it possible to add subtitles or captioning in Zoom with Falcon?

It is possible. We currently - I believe this stream is being captioned right now and we are using Falcon. There is a - it's a little bit more complicated though, just because of the way Zoom handles audio. You'd have to use what we refer to as Classic Falcon, and in there we would assign a encoder username through a PC software called iCap Webcast Audio, and this will allow the audio to be pushed through a digital encoder and then pushed to Falcon Classic where it can get captioned with the video.

Sam: Right, yeah, because this - because Zoom is not RTMP in, RTMP out. So it wouldn't be like the same exact procedure that Elan was just demoing. But - but yeah, like - as he said, there is - it's a little more involved setup. You could probably get a hand by calling our support, but yeah there is -

Elan: Yeah, we'd be happy to help. Is Falcon able to have multi-channel, multi-language CC channels? It - Falcon–correct me if I'm wrong, Sam–but Falcon can support multi-language. It's more the end point and the end player that becomes an issue there.

Sam: Yes, Falcon is able to encode up to four concurrent languages, but the - the caveat is that the number of languages that will be available to the viewer is dependent on the - on the platform, and many only support, like, one or two. Actually, I'm not sure that I've ever seen one that supports more than two, so yeah, that's kind of - that's something that you want to check every step of your workflow that it would support it. But just from the caption encoding standpoint, yes, Falcon could encode up to four languages.

Elan: Does the video stream get re-encoded in the process?

Sam: Yeah, I'll take that one. I mean, the answer - the answer to that is no. So, we do not touch what I would call, like, the video picture or the video essence or, you know, insert your, you know, jargon term there. The captions are - are inserted into, like, the user data space of the H.264 if you want to get technical. So yeah, it does not re-encode the video essence.

Elan: Alright. If the captioner is disconnected from Falcon, will the stream path be disrupted or will the video just reach the destination without captions?

It would just reach your endpoint without captions. You would end up seeing a little kind of, like, caution notification on your output stream here that the stream is not being captioned, which is, you know, obviously not an ideal situation when your stream's not getting captioned, but at least it does let you know that your captioner has disconnected.

The other way to monitor that kind of activity if your concern is that you might lose the connection to your captioner is, like we were saying earlier, through the iCap PC Client. This will actually show you who the active captioner is at that time, along with the audio and a little feedback window with the - the captions that are currently being sent to the stream. And if they were to disconnect, it - you would get a little status message in the windows stating that X captioner has disconnected from the Access Code, but the stream will continue to at some point.

Sam: Yeah, that's kind of independent of the - the RTMP video path, like that'll still be pushed through either way.

Elan: What is the difference, pros or cons, in having the hardware encoder or this, cloud-based? Also, is it possible if we have audio only and not video, and there are multiple people participating in the meeting?

So if the - for the second part of your question, if it's a Zoom meeting and you don't have any video, if that's what you're referring to, then yes, it is possible to push the audio itself through Falcon since it's connecting to the encoder, rather than the video stream for Falcon. And that'll still caption it, whether the participants are hidden or not.

But again, really it - it all depends on your workflow. If this is just for Zoom meetings or if you're just pushing, you know, a live stream directly to YouTube or Facebook, then Falcon might be better. If this is going to a broadcast endpoint, as well as outbound to YouTube or Facebook, then a hardware encoder might better suit your needs.

Sam: Yeah, I guess one thing I'll add about the - the audio-only thing, I think that with - with this RTMP workflow that we have here, like RTMP in, RTMP out, I do not think that that would work if you're just sending it at an RTMP stream with only audio and no video. But yeah, like Elan was saying, you could go the more - the Falcon Classic route - perhaps with that. Because also we encode the caption data, like as I was mentioning with a previous question, in the user data of the H.264 stream. So yeah, from that standpoint, you do need video, but there are other options if you just have audio.

Elan: What is the added delay versus the original stream pre-Falcon? So, it's typically around one second, but that can be modified depending on what the key frame rate is. The caption delay is approximately four seconds, which is really no different than any other methodology.

Sam: Yeah, the caption delay, yeah - if you have experience with our hardware encoders, like, that would be roughly the same, however long it takes the human transcriptionist or - or Lexi. As far as just, though, the - how much did we delay the RTMP stream, as Elan was saying, it'll typically be about a second but it depends on - on the key frame rate because, like, as I was mentioning for a previous answer, we insert the captions in the H.264 user data and so, in order to do that, we need to reorder the frames into display order and then insert the captions, and then put them back in decode order.

And so that requires cross-buffering and processing an entire group of pictures at a time. And so the longer the group of pictures is, which is equivalent to saying the longer the key frame interval is, or the - the slower the key frame rate is, it's–those are all equivalent–then that would mean that we have to buffer more at a time. And so they incorporate more delay, but if you have a shorter key frame interval or, equivalently, a faster key frame rate, then we wouldn't have to do that as much.

Elan: Alright. Is there a standard method for exporting captions after the live captioning session has ended? This gentleman seems to be having some issues downloading captions directly from captions that were recorded into Vimeo. Was there better workflows than this to pull the post captions down? I'm gonna assume beyond the download option. Sam, do you know if CC Archive is something that's included with Falcon?

Sam: Yeah, so that's - if you're referring to even just the Download tab in the Falcon interface, that - that uses the - the Archive and you can download the captions after it. So I'm trying to - because it seems, though, like, with the questioner that they are aware of that feature.

Yeah, I almost feel like this is maybe a question that if - you should get in touch with our support. If you're having a very specific problem with this, we can work through it, but yeah, in a general sense, yes, you can download the captions after the fact in the Download tab and that would get you, like, a SCC file that has the as-run captions in the time range that you put, so -

Elan: I just changed the slide so you can have the support email up there, or you can contact us directly if you have our number. I use two Wowza servers. One locally, which I push Publish to the second one. With Falcon, would I set my encoder to go to Falcon in the Wowza configuration? I'm sorry - and in Wowza configure it to use Falcon like an incoming encoder? Sam, do you want to take that one?

Sam: Yes, so you're saying that the stream is originating in one Wowza server and then it's winding up in another Wowza server? Because yeah, so that's like - like, as Elan said, Falcon is like the middleman, and so it's kind of like - first, you imagine how would you set it up with Falcon if Falcon wasn't there?

So you would set your first Wowza server with the destination stream key for the second Wowza server. And now it's like, OK, but now we're adding Falcon in the middle of that. So you have to configure your first Wowza server with the URL and stream key given by Falcon and then configure your Falcon with the URL and stream key of the second Wowza server.

Yeah, so in other words, the - the Falcon is configured like as if it were your streaming media encoder, and for your original encoder, that's configured to push the Falcon as if it was your server, if that makes sense.

Elan: And it looks like that's all the questions we have so far. If you folks think of any other ones, you can feel free, again, to contact our support email. Or, alternatively, if you're interested in checking out a demo of either of these or purchasing Lexi if you already have used Falcon in the past, so and so forth, you can feel free to contact our sales department with the above email as well.

And like i said earlier, this - this was recorded and will be sent up to YouTube. So yeah, if you want to review this at a later date, of course it'll be available. I believe I was told within the week it should be up there, both on our website and on YouTube. So if that's all, thank you guys very much for coming and I hope it was helpful.

Sam: Yeah, thanks!