The best thing to do is to add a cord (which you may need a knife to fashion) and then to cut the other tat away (which will also require a knife), so that there is one nice and clean redundant anchor on the horn. Clearing away the garbage at rappel stations provides great stewardship and it shows that you care about the crags where you climb.
Cutting cords and sling material is a common occurrence on long multi-pitch routes that don't see a lot of traffic. It is not at all uncommon to have to do some work to beef up anchors or to clean up old materials left years before. Additionally, a knife could be used to cut away damaged sections of rope, be used in a first aid situation, or even be used to trim materials for a makeshift shelter. There are a million uses for a knife, especially on long routes...
I alluded to the possibility that there was some controversy about carrying a knife. That is not at all the case. Every guide carries a knife. No, instead the controversy lies in what kind of knife you should carry and how you should carry it.
It is not uncommon for people to carry cheap "gas station" knives on cords hanging off their harnesses. Indeed, some people even carry more expensive knives the same way. The concern is that a knife might open and become dangerous, both from the possibility of getting cut as well as the possibility of it damaging gear. As such, there are some guide trainers that don't allow guides to carry knives on their harnesses. They prefer if they were in a pack.
There are a couple of popular harness knives available on the market that theoretically will not open on your harness. The Trango Piranah Climbing Knife (pictured above) is a very small knife that takes up very little space on your harness.
The Trango Sharktool (pictured above) is a nice hybrid between a nut-tool and a knife. It is a nice way to eliminate some of the extra baggage of the other knives described here. In other words, you will only need to have one carabiner for both the knife and your nut-tool.
The Petzl Spatha (pictured above) is a tried and true classic. I would say that I've seen this particular knife on more peoples harnesses than any of the others listed.
Certainly many climbers carry a multi-tool. This is especially useful if you are on an expedition or on a big alpine climb. Some will elect to carry their multi-tool on a harness, but most will stow it in a pack.
So to answer the original question, there are many uses for a knife. But if you start to see your knife as a companion or a close friend, then you should seriously consider therapy...
Jason D. Martin