How to Automate FFmpeg and Bento4 With Bash Scripts
4. Run the Script.
You run the Bash script from Terminal using the syntax shown on the top line of Figure 3 (./crf.sh). As you see in the figure, however, if you try to run the script without first authorizing it, you’ll get the Permission denied error.
As shown on the next input line, you make the file executable using this command:
chmod +x crf.sh
Then, run the script again (./crf.sh), and it will start FFmpeg and iterate through all the files in the folder. You’ll have to make each Bash script executable using this command sequence.
Figure 3. Authorizing and running the batch file
The script ran through all 13 files in less than an hour. Figure 4 shows two of the CRF files compared to the original with a data rate reduction as high as 83% with no impact on quality. Not bad for essentially two lines of code.
Figure 4. The fruits of our Bash script: 13 files created with up to an 83% file size reduction with no impact on visual quality
Working With Bento4
Now that you’ve got the basics, let’s take it up a notch and encode and package output for HLS and DASH distribution using FFmpeg and Bento4. The script I’ll use is shown in Figure 5. As you can see on line 3, the script implements another "for" loop, so it processes all .mp4 files located in the folder within which it is run.
Figure 5. Here’s our second script, which is a bit more ambitious.
As an overview, the most complicated part of the script relates to naming the output files and then recalling them for later use. In terms of major sections, here’s what’s happening:
Lines 1–4. As before, nothing new.
Line 5. I’m creating a separate output folder for each input file, which is necessary for the proper operation of the watch folder function implemented next.
Create the folder using the mkdir command, and name it "${file%.*}", which is the source file name without the extension. This folder designation precedes all files created in lines 6–15, telling FFmpeg and Bento4 to save these files into that folder.
Lines 6–10. Here I’m creating an encoding ladder with three video resolutions, 1080p, 720p, and 360p, and a separate audio file using two-pass encoding. You can insert as many additional FFmpeg commands as you would like and add additional rungs as needed on separate lines. If you’re used to Windows, the /dev/null && \ ending for the first pass is a bit different, but pretty easy to mimic. Otherwise, the FFmpeg commands are straightforward.
Note that the script will run completely through for each file, so if you have five MP4 files in the folder when you run the script, you’ll end up with five output folders, all containing three-rung encoding ladders formatted for HLS/DASH distribution.
As above, the "$file" input designates each file for input, and the "$[file%.*}_*" .mp4at the end designates the output file name for each file. Getting the names right is critical because the files created in lines 6–10 feed into the fragmented MP4 files created in lines 11–14, which feed into the packaging created in line 15. Again, note that I’m sending all files into the folder created in line 5.
Lines 11–14. This series of commands uses the Bento4 mp4fragment utility to convert the MP4 files created by FFmpeg into fragmented MP4 files required for the mp4-dash utility used in line 15. The syntax is simple, just mp4fragment inputname outputname; the hard part is pointing to the files in the subfolder for both input and output. You can read more about the mp4fragment utility at go2sm.com/bentomp4.
Line 15. This line calls the mp4-dash application, includes the --hlsswitch to create HLS output, and uses the --output-dir switch to store the final files in the proper subfolder. Then you simply list the files created in lines 11–14 in the order you want them placed in the manifest file. You can read more about mp4-dash at go2sm.com/mp4dash. Note that the applications in Bento4 are very functional and can do a lot more, including adding captions and DRM into your HLS/DASH outputs; as with FFmpeg, you can add as many switches as you need to implement the required features.
Line 16. This moves the source file into the folder that I created, and it’s also necessary for the proper operation of the watch folder function implemented next.
Line 17. Done. This tells the script that this "for" loop is complete and to start the operation on line 5 for any additional files in the folder.
To run the script, save it to the folder containing the MP4 files and run the script as you learned above. Upload the file packaged output to your HTTP web server, and it should be immediately distributable to both DASH and HLS clients.
Adding Watch Folder Functionality
So, batch encoding is nice, but you have to be there to execute the script. Now we’ll learn how to automate the encoding performed above via a watch folder. Operationally, the script shown in Figure 5 (ffmpeg_d.sh) runs on all MP4 files located in the folder when the script is run. The script in Figure 6 (watch.sh) watches the folder and runs ffmpeg_d.sh when new files are copied into the folder.
Figure 6. Creating a watch folder
The watch folder script uses a function called inotifywait to watch the folder. To deploy the script, insert the folder to be watched in line 3 and the script to run in line 5. Line 6 closes the "while loop" created in line 3.
You run the script as normal, first authorizing the script and then running it (Figure 7). The script remains running until you cancel it using the CTRL + C keystroke combination.
Figure 7. Running the watch script and creating the watch folder
In my tests, the only major anomaly was that the ffmpeg_d.shscript didn’t encode files added to the folder when it was already running. So, if you added video A to the watch folder, and ffmpeg_d.sh started running, and then you added video B before the encoding was complete, Video B wouldn’t immediately start encoding. When you add video C to the folder, however, the watch folder script will execute ffmpeg_d.sh and it will encode both videos B and C.
Using these scripts, it should be relatively easy to create your own in-house encoding station simply by installing Ubuntu. In a separate tutorial in this edition, I detail how to start an Amazon Machine Image running Ubuntu in Amazon Web Services. These scripts should also run on those machines, allowing you to create an ad hoc encoding farm to encode any number of files quickly and cheaply.
Related Articles
FFmpeg was designed as a cross-platform solution for video and audio recording, conversion, and streaming using simple static command lines. Using variables and "for loops" in a command string simplifies the reuse of existing scripts and helps automate their operation. While you can't use these scripts in the Windows Command window, you can use Microsoft PowerShell in Windows and Bash on Linux and the Mac. In this tutorial, you'll learn how to create and run such scripts with PowerShell and Bash.
14 Mar 2023
Rather than focusing on random tasks, this tutorial will walk you through the fundamentals of encoding with the latest version of FFmpeg.
27 Mar 2022
Anyone who does performance or benchmark testing, please take a look: The six commands in this article help with essential tasks that crop up in any studio or encoding facility.
22 Jul 2019
AV1 delivers equivalent quality to HEVC, but with a lower data rate. For now, though, it's slow. A five-second clip took 23 hours and 46 minutes to encode.
27 Sep 2018
Building on previous DIY articles, this installment will walk you through capturing audio, encoding and packaging with FFmpeg, then playing out through the Icecast server you set up in the last article
15 Jun 2018
FFmpeg might be a free download, but there's still a cost that goes with it. Robert Reinhardt explains why this encoder isn't right for every company.
21 Jan 2014