On the one hand it can be a good strategy to show non available products with the note "Product currently not available". On the other hand it can be frustrating for your customers to find a lot of interesting products which they are not able to purchase. In case you rather want to automatically hide products which are dropping under an inventory of 1 this is what you have to do:
Hit "settings > product management"
Tick "hide unavailable products"
Save and you're done :-)