I'm not sure why this was done this way, but the default font size in the Paint object does not scale automatically when you run your app on different resolutions and screen densities.
If you want to keep the current default size but scale it according to the screen density, simply add this line of code.
paint.setTextSize(paint.getTextSize() * yourActivity.getApplicationContext().getResources().getDisplayMetrics().density);
This will slightly scale it up or down, depending on whether the device is running at low or high density.
If you want to have more control over the size of the text, then use a dimension resource. In your "res/values" folder, either add to or create an XML file.
Using the editor, create a new "Dimension" value in that XML file with whatever name you want. For this example, I'm going to give it a value of "16sp". This means that it will be "16 scaled pixels" big, scaled to the device density AND the user's font size settings.
To apply that dimension:
Kudos to Marcin Gil and Joseph Earl for the help, but it ended up being my own noobish mistake for not applying it to the right Paint instance.