How with OpenAI Whisper do I transcribe all mp4s in a folder?

Problem:

I'd like to transcribe all mp4 files in a folder on both Mac and Windows using Whisper.

I am familiar with scripting and command-line, and have already set up Whisper to work on my machine.

Windows example solution:

The following is an example solution for Windows. Save this in a .bat file and run it in the folder you'd like to process all mp4 files. Modify the script as you need to.

This example transcribes in English with Whisper set up to use CUDA. It skips over any already transcribed mp4 files. It is not recursive.

e.g. transcribe_all_mp4.bat:

for %%f in (*.mp4) do (
	echo %%~nf
	if not exist "%%~nf.txt" (
		echo will process %%~nf.mp4
		whisper "%%~nf.mp4" --language English --device cuda
	) else (
		echo will NOT process %%~nf.mp4
	)
)

Mac example solution:

The following example is a single Terminal command you can use in MacOS with Whisper already set up.

Note that this example command does not check for existing transcriptions. It does however specify Whisper to use the medium model. It is not recursive. Modify the command as you need to.

In Terminal in the folder where you want to transcribe your .mp4 files:

for f in *.mp4 ; do whisper $f --model medium ; done

Notes:

The above solutions in Windows and Mac were verified to work between February 2023 to June 2023 (for real-life conference and film work). As of this time the good folks over at Whisper haven't yet implemented batch processing of files.

Hope these examples helped you start to get past where you too got stuck :)

References:

• https://www.digitalocean.com/community/tutorials/workflow-loop-through-files-in-a-directory
• https://github.com/openai/whisper/discussions/153
• https://stackoverflow.com/questions/39615/how-to-loop-through-files-matching-wildcard-in-batch-file
• https://learn.microsoft.com/en-us/previous-versions/windows/it-pro/windows-xp/bb490909(v=technet.10)?redirectedfrom=MSDN
• https://github.com/openai/whisper/blob/main/README.md
• https://stackoverflow.com/questions/23735282/if-not-exist-command-in-batch-file

Find executables installed via apt

Problem:

I want to list all executables installed via apt (or apt-get). I know the name of the package installed and I am already comfortable with using shell commands.

Solution:

To list all executables installed for a package, in your shell use the following command while replacing "packagename" with the actual package name:

  dpkg -L packagename | xargs file | grep executable
  

Example:

For example, if I want to list all executables installed in the package "nullmailer":

  dpkg -L nullmailer | xargs file | grep executable
  

Other reminders:

To list all files installed for a package, in your shell and replacing "packagename" with your package name, use:

  dpkg -L packagename
  

To list all executables for a package (named "packagename") installed in the "opt" folder, use:

  dpkg -L packagename | xargs file | grep ^/opt | grep executable
  

References:

• https://man7.org/linux/man-pages/man1/dpkg.1.html
• https://wiki.debian.org/dpkg
• https://stackoverflow.com/questions/18937393/list-of-executables-installed-from-package
• https://man7.org/linux/man-pages/man1/xargs.1.html

bash scripting reminders of basics

Problem:

I want quick reminders for bash shell scripting because I don't write bash scripts often enough to memorize everything.

Notes:

The following reminders assume you're already familiar with the basics of bash shell scripting and simply need a reminder of more commonly used syntax, etc. These reminders are for myself but I hope you also find them useful.

Reminder shortcuts

As the number of reminders grow I hope this index will help you find what you need. (It certainly helps me.)

• References

• Top of script starts with #!/bin/bash
• Simple variables
• Concatenate strings
• String quotes of bash scripting
• Bash check if directory exists
• Bash for-loop
• Bash if-and, bash if-or
• Bash if directory does not exist
• Bash check if file exists
• For items in bash array
• Command line arguments in bash scripting
• How to check if git repo needs update from remote upstream with bash scripting
• How to check if script is on correct git branch for bash script

Reminders:

Top of script starts with #!/bin/bash

#!/bin/bash

See this tldp.org reference page for details.

Simple variables

Examples of simple variables

NAME="Delta"
echo "Hello $NAME!"

Concatenate strings

HELLO="Hello, "
NAME="Lily"

echo $HELLO$NAME
# output: Hello, Lily

HELLO="Hello, "
NAME="Alex"
GREET="${HELLO}${NAME}!"

echo "$GREET"
#output: Hello, Alex!

GREET="Hello, "
GREET+="Ninja"

echo "$GREET"
#output: Hello, Ninja

String quotes of bash scripting

DIRECTION="Right"
echo "Turn $DIRECTION"  # => Turn Right
echo 'Turn $DIRECTION'  # => Turn $DIRECTION

Bash for-loop

VAR=""
for NAME in 'alpha' 'bravo' 'charlie' 'delta'; do
  VAR+="${NAME} "
done

echo $VAR
# output: alpha bravo charlie delta 

Bash if-and, bash if-or

if [ $FILENAME == 'important.txt' ] && [ -f $FILENAME ]; then
  echo "this is an important file"
fi

if [ $NAME == 'Hector' ] || [ $NAME == 'Miguel' ]; then
  echo "this person is on a hero's journey"
fi

Bash check if directory exists

DIR="/usr/mydir/"

if [ -d "$DIR" ]; then
  echo "directory exists"
else
  echo "directory not found"
  exit 1
fi

DIR="/usr/mydir/"

[ "$DIR" == "" ] && { echo "directory string is empty"; exit 1; }
[ -d "${DIR}" ] && echo "directory exists" || echo "directory not found"

Bash if directory does not exist

DIR="/usr/mydir/"

if ! [ -d "$DIR" ]; then
  echo "directory does not exist"
else
  echo "directory exists"
fi

Bash check if file exists

if [ -f "/opt/myfile.txt" ]; then
  echo "file found"
else
  echo "file not found
fi

For items in bash array

DIR3="/opt/mydir3"
DIRTOCHECK=("/opt/mydir1" "/opt/mydir2" $DIR3)

for dir in "${DIRTOCHECK[@]}"
do
  if [ -d "${dir}" ]; then
    echo "found directory ${dir}"
  else
    echo "did not find directory ${dir}"
  fi
done

Command line arguments in bash scripting

# example: yourscript.sh arg1 2nd-arg

echo "All arguments values:" $@
# output: All arguments values: arg1 2nd-arg

echo "First argument:" ${1}
# output: First argument: arg1

echo "Second argument:" ${2}
# output: Second argument: 2nd-arg

echo "Total arguments:" $#
# output: Total arguments: 2

For more detailed information, please refer to this tecadmin.net tutorial

How to check if git repo needs update from remote upstream with bash scripting

This how-to assumes you already have a git repository installed, a remote upstream setup, and have navigated into the local repository via your script.

git fetch
if [ $(git rev-parse HEAD) == $(git rev-parse @{u}) ]; then
  # add logic for when no update is needed, e.g.
  echo 'already up to date.'
else
  # add logic for when update is needed, e.g.
  echo 'updating to latest.'
  git pull
fi

How to check if script is on correct git branch for bash script

The following example assumes master branch is desired and the script has already navigated to the local repository.

if ! [ $(git rev-parse --abbrev-ref HEAD) == "master" ]; then
  echo "NOT in master branch."
else
  echo "in master branch."
fi

Why these reminders?

This page is intended to remind myself quickly without requiring too much reading. I hope it's helped you out, too.

References:

• http://tldp.org/LDP/Bash-Beginners-Guide/html/
• https://www.linux.com/training-tutorials/writing-simple-bash-script/
• https://www.gnu.org/software/bash/manual/html_node/index.html
• https://linuxconfig.org/bash-scripting-tutorial-for-beginners

Test for shellshock bash bug vulnerability in OS X or Linux

Problem:

How do I quickly test if my computer is vulnerable to the shellshock Bash bug?

Solution:

One way to test the shellshock bug is to open a Terminal (in OS X, or any shell prompt in Linux, Cygwin, etc.), and try the following one-lined command:

x='() { :;}; echo VULNERABLE' bash -c :

If vulnerable, VULNERABLE should appear as part of the response. Otherwise if things are alright, you should get a response similar to the following:

bash: warning: x: ignoring function definition attempt
bash: error importing function definition for `x'

(Note that in some versions of fixed/patched bash shells, the above error won't show up but "VULNERABLE" will also not be printed to the terminal.)

Notes:

If you're unsure of any steps of the above instructions, see if you can find a friend who may be familiar with how to use bash shell and Terminal to try the above for you.

For more detail, check out the references below.

If your system happens to be vulnerable, and you're using Linux and don't know how to update your bash shell, try this post for instructions: http://linux.about.com/od/howtos/fl/How-To-Fix-The-BASH-Shellshock-Bug-On-Your-Linux-System.htm

NOTE: I won't be able to answer any questions about shellshock as I am not enough of an expert on this bug to comfortably respond. Sorry about that! I hope that this post still comes in handy to people out there.

References:

• http://security.stackexchange.com/questions/68168/is-there-a-short-command-to-test-if-my-server-is-secure-against-the-shellshock-b (contains a good, detailed technical explanation of the above command)
• http://web.nvd.nist.gov/view/vuln/detail?vulnId=CVE-2014-6271
• http://www.troyhunt.com/2014/09/everything-you-need-to-know-about.html (a very detailed explanation, for those curious)
• http://linux.about.com/od/howtos/fl/How-To-Fix-The-BASH-Shellshock-Bug-On-Your-Linux-System.htm (how to update bash shell on various distros of Linux)

Selectively and recursively copy all files of a single file type in OS X and Linux

Problem:

In OS X or Linux, how do I selectively copy only one type of file to a folder? For instance, how can I recursively copy all .mp3 files from a folder with many sub-folders into a single folder?

Solution:

Note: this solution assumes you have basic knowledge of using the Terminal and know how to avoid destructive commands.

To recursively check the paths of all files for instance all .mp3 in a folder and all its sub-folders, type the following into the Terminal (replacing the path and file extension to fit your need):

find /path/to/files/to/search -iname '*.mp3'

If everything looks alright, to copy the files use the following command (replacing the paths and the extension to fit your need - also note that this is a single line/command, but it's long so it may wrap to more than one line in your browser):

find /path/to/files/to/search -iname '*.mp3' -exec cp {} /path/to/destination/ ./mp3 \;

Important: make sure that the / is at the end of the destination path! (It will all be copied as a single file otherwise as it'll think that your destination is a file name and not a folder.)

Batch resize images to a fixed width and add watermarks

Problem:

I've got a bunch of images that I want to make a resized version of to post online. Additionally, I want to add a watermark to each of these resized images. I already have ImageMagick installed in my OS X or Linux computer.

Solution:

This is quite a specific problem with a very specific solution. This post assumes that you have read and understood the previous post found here. If not, the solution in this blog post should still be somewhat clear, if not a little fast-paced.

For the sake of this post, let's assume that the source images are a bunch of .JPG images. Additionally, let's assume that we want to resize our images to a width of 320 pixels while maintaining the aspect ratio of the original image. Finally, let's assume that we want to give our images the semi-transparent watermark "grammarofdev.blogspot.com", using the font located at "/Library/Fonts/Arial.ttf". To prevent overriding our original images, we will also create a new sub-folder called 'results'. Below is an example Terminal command of how to do all of this (with line-breaks added for legibility):

$ mkdir ./results
$ mogrify -font /Library/Fonts/Arial.ttf -pointsize 88 -verbose
          -draw "fill rgba(255,255,255,0.5) text 24,75
                 'grammarofdev.blogspot.com'"
          -path ./results -resize 320 -quality 86 -format jpg *.JPG

Phew, what an ugly, long command. What does it all mean?

The "-font /Library/Fonts/Arial.ttf -pointsize 88" selects the Arial TrueType font with a point-size of 88pt. The "-verbose" flag outputs verbose information to the terminal while the conversion is being done. The "-draw "fill rgba(255,255,255,0.5) text 30,80 'grammarofdev.blogspot.com'"" portion tells ImageMagick to render the text "grammarofdev.blogspot.com" with a fill colour of white and an alpha of 0.5, starting at the (x,y) coordinate of (24,75) of the original image.

The "-path ./results" indicates that the resized images should be written to the "results" sub-folder. Note that all files in that sub-folder with the same name will be written over without warning! Additionally, omitting this flag will write over all of your original images!

Next, "-resize 320 -quality 86 -format jpg" indicates that the images should be resized to have a width of 320 pixels, maintaining the original aspect-ratio, and then converted to jpeg with a quality of 86. Finally "*.JPG" indicates that this command should convert all images with the suffix .JPG (case-sensitive).

Note that the text is drawn onto the original image prior to the image being scaled down to the final desired size.

Let's see how well this worked. The following two images will show an image before this command is run, and the resulting image afterwards. Note that the original image dimensions are 3264x2448 pixels, however this image is not included in this post. (A slightly-larger version can be found here!)

Original image (scaled to width 300px):

Watermarked image:

Where can this be useful:

Imagine that you had a hundred images you wanted to automatically watermark and resize, or imagine that you have created a web app that requires re-sizing and watermarking previews of images. Using ImageMagick to perform this task is a quick way to achieve the desired results without needing to program anything complex, or use programs such as Photoshop. This post only touched the surface of what ImageMagick can do. For more details on resizing, or details on drawing on images automatically with ImageMagick, check out the following references.

References:

• http://www.imagemagick.org/www/command-line-options.html#draw
• http://dptnt.com/2007/09/batch-resize-images-using-imagemagick/

Bash shell script confirmation prompt

To prompt for user confirmation before performing a dangerous action in a bash shell script, you can use the following code:

read -p "Confirm (y/n)? " -n 1 -r
if [[ $REPLY =~ ^[Yy]$ ]]
then
  # get cursor on next line (optional)
  echo
  # do dangerous stuff here
fi