So I let my clutch/brake fluid levels get too low on my 1986 Volvo 740 turbo. Low enough to be under the clutch fluid tube's collection line. My clutch pedal now has no pressure. I bought some DOT 4, and I am about to change it.

Because I figure air is now in the clutch pedal system, am I going to need to bleed the new liquid through? Or can I just pump it through?

Filling the clutch slave cylinder is very tricky because the clutch fluid resevoir wall is not much higher than the max line on the brake resevoir. It only takes one or two pumps on the clutch peddle and you'll start sucking air and creat the same problem all over again. Keep a close eye on it and you shouldn't have any problem but it sounds like you've got a clutch fluid leak that needs quick attention. Master or slave cylinder? If you want to keep the car much longer I'd suggest doing both.

Will you get the air out by just pumping it a few dozen times?
Most likely.
You should try to find your leak,repair it,then flush and bleed system.