Though most New Yorkers don’t know it, much of the city’s public housing is actually privately owned. Tens of thousands of poor tenants have their rents subsidized by the federal Department of Housing and Urban Development under its Section 8 program.

But that precious stock of affordable housing is now under threat. Over the next several years, the contracts for nearly 600 properties with Section 8 subsidies will be expiring, according to data from the National Housing Trust. That adds up to about 70,000 low-cost apartments at risk. Each of the landlords of these buildings will be faced with a decision: They can renew their federal agreements for a few years and maintain the buildings as low-income housing, or they can opt out and rent out their apartments on the open market. What these landlords do–and what HUD does to keep them in the program–will determine whether poor renters can keep living in these properties.

The process of Section 8 expirations has already started. Through last September, 500 properties with more than 24,000 subsidized units had left the program nationwide. In New York City, 12 owners have opted out of Section 8, adding up to a loss of 1,505 units of affordable housing. Currently, Tenants & Neighbors is aware of at least another 8 properties in the city where the owner has opted out or indicated his intention to leave the program.

Cumulatively, these decisions will have a massive impact on the city’s stock of affordable housing. As all New Yorkers know, the real estate market is booming, and these building owners have a major economic incentive to convert to market-rate rentals, co-ops or condos. And because they are relatively new, none of the Section 8 buildings will be subject to the rent stabilization laws.

Some neighborhoods may be hit especially hard by this phenomenon, as is apparent in this map, prepared by the New York Public Interest Research Group’s Community Mapping Assistance Project. Each dot represents one of 306 New York City properties where landlords are currently getting less money in rent subsidies than they would be likely to get on the open market, and therefore have the biggest incentive to leave the Section 8 program.

Essentially, it’s a guide to vulnerability. Clearly, certain neighborhoods in Harlem and the Lower East Side are at risk. However, the problem is not limited to Manhattan. Nearly half of the opt-outs that have already occurred took place in the outer boroughs. As more and more tenants are priced out of Manhattan, it is likely that many other owners in the outer boroughs will choose to capitalize on the economic benefits of leaving the Section 8 program by converting their buildings to market rate.

The map is just a snapshot, since when a contract actually expires the real rental market may be very different. In particular, HUD’s estimates don’t reflect the changes that gentrification and rising real estate values have brought to many neighborhoods. The future of each property is determined on a case-by-case basis as each contract expires, creating fear and uncertainty for the tenants who reside there.

When owners do opt out, the tenants have some protection. Last year, HUD instituted “enhanced vouchers” that are supposed to pay for increased rents, enabling Section 8 tenants to stay in their homes even if the owner raises his prices. However, at this time it’s unclear whether owners are required to accept these vouchers, and it’s also unclear if these special vouchers will actually pay enough to cover the rent hikes in New York City. Plus, Congress must reauthorize funds for these vouchers every year, leaving them victim to the vagaries of funding.

In an attempt to address this pending crisis, Congress and HUD have developed programs that give landlords incentives to renew their Section 8 contracts and stay in the program. Most recently, Congress passed legislation that would increase payments to owners who are getting below-market rates.

It’s a move in the right direction. But participation is voluntary for owners, and the renewal contracts last only five years. After that time is up, the whole process of renegotiation and reconsidering starts over again, putting tenants back on the line.

Joe Heaphy is executive director of New York State Tenants & Neighbors, which has been organizing tenants in HUD- subsidized buildings.