Patients, patients everywhere! They’re riding on my bus in the seat across from me, they’re at restaurants at nearby tables, they’re the other parents at my kids’ school events and at the playground. I even saw one on my family vacation to Mexico. I can’t escape them.

I love my patients. In my practice, I’ve formed special relationships with people from a true cross section of all over Philadelphia. When I meet patients in the privacy of a clinic or hospital room, they are truly vulnerable — sometimes they may literally be naked in front of me. In this safe space, they may tell me things that their spouses, kids, or closest friends don’t know.

Where it gets tricky is outside this safe space. What happens when I see a patient out in the real world? How about on social media? Should I say hello?

It sounds harmless enough, and I want to be friendly, but even confirming publicly that someone is a patient under my care could potentially be a violation of their privacy. Though many of the diseases I treat are minor or lack stigma, others do not. For instance, I take care of many patients with complex medical histories, from HIV-positive and transgender patients to those waging difficult battles with cancer. You really can’t be too careful.

The doctor-patient confidentiality is one-sided protection. I cannot violate it except in extreme circumstances (child abuse, imminent danger of homicide, or suicide), but patients may break confidentiality any time they’d like. Patients can post about their care on social media or scream from the rooftops about how things are going.

Yet, believe it or not, the federal Health Insurance Portability and Accountability Act (HIPAA) specifies that even stating that I saw a 90-year-old patient on a specific day could be a violation because there are relatively few people in their 90s. Say I run into a patient who is walking with a friend on the street. I can’t acknowledge him because I may be putting him in a position to share that he was seeking medical care, or have to explain what his care was for.

One of my favorite things is taking care of an entire family — for instance, a husband and wife, and their parents. It’s fascinating to see how partners can be so different or so alike, how parents and children are different or similar, and to see how they complement each other in different ways. But it gets challenging when I have to navigate privacy. What if I tell a woman I saw her husband the previous week, but she hadn’t known? Even if it were for something minor, his failure to tell her could suggest secrecy or dishonesty or just absent-mindedness. Another example: Let’s say the wife’s mother has cancer, but it’s unclear whether everyone in the family knows about it or is up-to-date on recent developments. I don’t always have the presence of mind to ask everyone whether everything can be shared among the family. And I might have to remember to refrain from asking others in the family how the mother’s treatments are going.

One time I saw a friend as a patient. Later that day I saw her sister, also my friend, who greeted me by joking, “I heard you looked at my sister’s butt today!” Even though it was clear that these sisters had talked, I had to awkwardly smile and just nod, not wanting to accidentally share information that hadn’t been part of their conversation. I’m simply not capable of keeping track of who is allowed to know what — so when in doubt, I keep my mouth shut.

The same thing happens even within the supposed safe spaces. In the hospital, as a consulting doctor, I often walk into a patient’s room to find that family members are visiting. It’s so easy to accidentally answer family questions without checking to be certain it’s OK with the patient. Most of the time it’s fine — but all it would take is one slip to learn that sometimes it is not fine at all.

A family member might ask me, “How is she?” about a patient. I don’t want to be evasive or prevent families from knowing about their loved ones, but it’s the patient’s decision to state who can know about the treatment. I try my best to ask the patient, alone, “For your privacy’s sake, I need to confirm that it’s OK to share all of your medical information and history with your family. Is that all right?”

But even that’s not perfect. If the patient is unconscious, I have to navigate which family member gets to decide what can be shared. It’s not my place to reveal past medical history that might not have been disclosed. Even when a patient says it’s OK, I tread extra carefully — especially for patients with a condition that may carry significant stigma.

After being in clinical practice for almost seven years in Philadelphia, I love the fact that I constantly see my patients everywhere I go. I savor that I can’t even walk through the hospital without a nurse or doctor promising me they’ll make an appointment to see me soon. This reminds me that I’m a part of a community. Still, if you are a patient of mine reading this, please know that when I’ve seen you on the street, in the supermarket, the airport, I might not say hi at first. I might even act like I don’t know you, but it’s because I’m waiting for you to make the first move.

Jules Lipoff is an assistant professor of clinical dermatology at the University of Pennsylvania’s Perelman School of Medicine.