Welcome to Pressing Questions, Fast Company’s work-life advice column. Every week, deputy editor Kathleen Davis, host of The New Way We Work podcast, will answer the biggest and most pressing workplace questions.

Q:  What should I do if I don’t want to be someone’s reference?

A: Talk about awkward workplace conversations! But like all awkward workplace conversations, a compassionate and direct approach is best. Your exact approach should vary depending on the reason why you don’t want to give a reference:

1. You don’t have anything to say. If the reason you don’t want to be someone’s reference is that you haven’t worked with them closely, you can tell them exactly that. Something like, “Thanks for asking me, but since we haven’t worked together very closely, I don’t think I’m the best person to speak to your strengths.” If you know of someone else who would be better suited, suggest them.

2. They haven’t done a good job. This is the most uncomfortable reason. If an exiting or former employee didn’t excel at their job—or worse didn’t even fulfill their job requirements—you can tactfully let them know that they won’t get what they are looking for from you. Here, you can offset the responsibility a bit.

Many companies advise managers that they can’t say anything negative about former employees due to legal concerns. You can say something along the lines of: “I can’t provide a recommendation on your performance, but we can have HR provide confirmation of your role.” Again, if there is another department or previous role where they perhaps did better work, you can advise them to seek out a manager there instead.

3. You just don’t have the time. I fully support setting boundaries. And if you get a lot of these requests, it’s understandable that it can feel like a burden. However, if it is someone you’ve worked with closely who has done good work, you should ask a few questions before turning them down.

There’s a big difference between holding a permanent spot on someone’s reference list for any job they apply for and being asked to give a reference for one or two specific opportunities. Find out what exactly they are looking for and gauge if it’s a commitment you can give. If it’s not, you can offer to write a reference letter or a public recommendation on LinkedIn.

Karma is a powerful force in the world of work, and you never know when your paths might cross again. If you don’t have anything nice to say, your best bet is to avoid hurting someone’s future employment chances. But if you do have something nice to say, don’t keep it to yourself.