There was this black rubber tube that I noticed hanging unconnected underneath the driver's side dash. I looked around to see where it should be connected and I noticed that there was this small nipple protruding from the plastic heater box that leads to the vent.

When I connected the loose tube to the nipple, it seemed to fit perfectly. However, the tube didn't reach far enough. So I pulled on it some more, and snap! I accidentally yanked the tube off the other end of where it was connected.

My question is, what does this 12-inch black rubber tube do and how important is it to be connected to where it's supposed to be?

Sounds like you ripped off a tube thats part of your ventilation system. What that tube is part of is a suction activated system which selects which 'doors' in your vent system are opened or closed so as to route the air to where you choose. There are seperate doors for both sides of the system, and they control if its fresh or recirculated, floor, front, or deprost. If you never use the function which you just rendered inoperable, which seems like it could be the floor vents, then who cares, the rest is working fine right? Otherwise, just search around for the nipple on the other side, and reconnect.