Nine hundred and ninety-nine windows. That’s how many the original 1881 building had — not because anyone was trying to make a design statement, but because the watchmakers needed light. They were stamping out gold and silver watchcases by hand, stamping out the kind of small precise work that ruins your eyes if you do it in the dark. The factory was built to serve their vision. The windows were a tool, not a virtue.
Today, inside what is now called The Watchcase Factory, those windows frame views of church steeples, the Peconic Bay, and yachts. Fumed oak floors run the length of spaces where machinists once sweated through two world wars. Pine-beamed ceilings, original to the structure, hang over marble surfaces and Thermador appliances. The same brick that absorbed a century of factory noise now holds in the quiet of luxury living. The transformation is staggering, if you let yourself think about who built the place and why.
This is a story about what happens when industrial memory meets Hamptons money — and whether the brick that holds them together actually remembers anything at all.
Sag Harbor Before the Watch Factory
By the time Joseph Fahys moved his watchcase operation from Carlstadt, New Jersey to Sag Harbor in 1881, the village was already coming off one identity crisis and negotiating its way into a second. Sag Harbor had been a major American whaling port — one of the great ones, second only to Nantucket at its peak. I wrote about this in my piece on The Whaling Barons of Sag Harbor: this was a village that sent ships around the world, produced fortunes in whale oil, and then watched the whole industry evaporate under the weight of kerosene and petroleum. The docks that once launched whalers sat idle. The deep-water harbor that had made the village prosperous needed a new reason to matter.
The watch factory gave it one. Fahys incorporated the Fahys Watch Case Co. officially in 1881, building on a site that had previously held a cotton mill destroyed by fire. At its height, his company was the largest manufacturer of gold and silver watchcases in the United States — and also, remarkably, the biggest producer of silverware in the country. He founded the Jewellers Board of Trade in 1884. He was president of the Watchcase Manufacturers Association. For a French immigrant who had arrived in the United States in the 1840s, this was the kind of ascent that the 19th century made possible if you were relentless enough and lucky enough to catch the industrial current at the right moment.
Fahys earned a reputation for treating his workers well, which in the context of Gilded Age manufacturing was not the norm. Eventually, under growing financial pressure, he sold the Sag Harbor plant to Joseph Bulova — an Austrian watchmaker — who continued producing watchcases in the factory well into the 20th century. The factory kept running through two world wars, through the Depression, through the long postwar boom, until it finally shut down in the mid-1970s. A hundred years of production, stamped out in a building that still stands one block from Main Street.
What Thirty-Five Years of Abandonment Does to a Building
After the watchcase operation ceased, the factory sat empty for three and a half decades. Local kids knew the place as a kind of haunted house — the scale of the building, the hollow arched windows, the ivy creeping over red brick. The Sag Harbor Express assigned a staff photographer, Michael Heller, to document it before and during the renovation. He has described walking in for the first time — hard hat, insurance waiver, a private website login so the developers could monitor his access — and being floored by what he found: the brickwork, the wood beams, the volume of windows. Thousands of frames shot across 115 visits. He published the result as a book, Watchcase: The Story of a Rebirth, which now retails for $174.99 and is one of the few serious documentary records of what the place actually looked like before money came back in.
Thirty-five years of abandonment is not nothing. The building was in severe disrepair when Cape Advisors, a Manhattan developer, acquired it. The renovation work — a $40 million project designed by the architectural firm Beyer Blinder Belle with interiors by designer Steven Gambrel — began in November 2011. It was not smooth. In 2012, a floor gave way under a pile of accumulated bricks during early demolition work. The collapse happened in the early morning hours, and no one was hurt. The building required structural rebracing before any of the real work could begin. Enormous ceiling beams, weighing several hundred pounds each, had to be replaced.
An architectural conservationist was brought in specifically to match the colors of original brick to new brick — cornices, window frames, grout, mortar, all calibrated against what had been put there in 1881. The original walls were cleaned with walnut shells rather than harsher blasting methods, to preserve the integrity of the surface. The 700-plus windows — the building was designed around natural light for the precision work done inside — were replaced with custom Andersen E-Series units matched to the originals, which had been built on-site by carpenters when the factory was first constructed.
The result is a 64-unit complex: lofts, townhouses, garden residences, and bungalow condominiums. Amenities include a saltwater pool in a landscaped courtyard, a fitness center, a yoga studio, a Vault Bar and Inglenook Lounge with a double-sided fireplace, and a Victorian-era water tower converted into a penthouse pavilion with panoramic views of the Peconic and the village below. A penthouse unit at the top of an old watchcase factory. The machinists who worked those floors could not have imagined it.
The Architecture of Memory
What makes a building like the Watchcase Factory worth preserving rather than simply demolishing and starting fresh? The economics of Sag Harbor real estate in 2014 would have supported a new glass-and-steel luxury complex on that site just as readily. The land value was there. The demand for Hamptons-adjacent living was there. And yet Cape Advisors and Beyer Blinder Belle chose — at considerable additional cost and complexity — to work within the original structure rather than replace it.
Part of this is National Register status. The site is a historic property in a historic district, which creates both restrictions and incentives. Part of it is the design appeal of old industrial brick in a market where new construction has become predictable. Buyers who can afford six-figure Watchcase condominiums often have their choice of new construction elsewhere. What they cannot get elsewhere is what the 27east.com writers and local observers described as genuine antiquity — the mosaic tile floors being carefully removed, a piece quietly kept for the John Jermain Memorial Library, the brickwork that has held through two world wars and three decades of abandonment.
But there is a philosophical question underneath the practical one, and it is worth asking plainly: when you replace the floors, the windows, the interior infrastructure — when you insert fumed oak and marble where machinists once moved — does the building still remember? The brick does, in some sense. The original cornices do. The massing of the building, its relationship to the street it has faced since 1881, does. Whether that constitutes memory in any meaningful sense, or whether it’s just the useful appearance of memory deployed in the service of a premium price point, is a question that adaptive reuse projects never fully answer.
I lean toward thinking it matters, but not for sentimental reasons. What is preserved in a building like this is the scale of the labor that built it. The windows that let in light for watchmakers’ eyes. The floor space designed around machinery. The ceiling heights needed to run that machinery. These are physical facts about work that no plaque or museum exhibit captures the way the space itself does. When Pawli at Maison Pawli Realty talks about what drives certain buyers toward adaptive reuse properties over new construction, this is often part of it — not nostalgia, but a desire to inhabit a space with a human record. A room that cost something to build, where the building of it was the point.
Who Bought the Watchcase, and What They Paid For
The Watchcase opened to buyers in 2014. Units have ranged from high-end loft apartments to penthouse configurations with those sweeping views of the Peconic. The complex sits one block from Sag Harbor’s Main Street — Bay Street Theater is close, the American Hotel is close, the Sag Harbor Cinema Arts Center is close. For the Hamptons buyer who wants village walkability and something more architecturally particular than a new-build colonial, the Watchcase offered a specific proposition.
What they were buying, if you want to frame it honestly, is exactly the transaction described at the top: the brick memory of labor, polished to a price point that the workers who built the place would have found incomprehensible. The Fahys company had a reputation for caring about its employees. Bulova extended that care in some form. The men and women who ran those precision machines, who squinted through the 999 windows to stamp out watchcases for a century, could not have bought these units. That’s not a criticism — it’s the economics of historic preservation, which is expensive by nature and therefore tends to end up serving a clientele that didn’t build the thing being preserved.
Whether that’s a tragedy or a reasonable outcome for a building that would otherwise have continued to deteriorate is a judgment call. What is not a judgment call is the quality of the restoration itself, which is genuine and painstaking, and the fact that the building is still there, still facing Main Street, still the same brick that went up in 1881.
Sag Harbor’s Long Negotiation with Its Own History
Sag Harbor has always been a village that had to decide what to do with what it was. The whaling barons built it into an international port. The decline of whaling left it without an economic identity. The watch factory gave it one for a hundred years. The watch factory’s closure left it to negotiate a third identity in the Hamptons luxury market — a negotiation that has been ongoing since the 1980s and that the Watchcase project represents in a particularly visible way.
The village has managed it better than most. Main Street still functions as an actual commercial street rather than a pure boutique corridor. The Bay Street Theater, now one of the premier regional theaters in the country, draws serious work to the space. The cemetery, the Old Whalers’ Church, the John Jermain Memorial Library — the institutional fabric of a real town is still intact. And the Watchcase Factory, whatever you think about the economics of its conversion, is part of a fabric of historic preservation rather than replacement.
For buyers and residents who want that specific combination — walkable village, deep history, Peconic water views, and a unit that comes with a century of industrial memory baked into the walls — there is nowhere quite like it. Pawli at Maison Pawli Realty has worked with buyers in exactly this market and will tell you that the Hamptons-adjacent villages are increasingly drawing buyers who specifically want that texture that new construction cannot provide.
The brick remembers, even when the price tags have forgotten. Standing in one of those lofts with the fumed oak floors and the pine-beamed ceilings and the custom windows framing the Peconic, you can feel the weight of the place — the labor embedded in the mass of the walls, the precision of the original craft in the proportions of the rooms. That’s not nothing. It’s not the machinists’ story, exactly. But it is their building, still standing, still lit by the windows they needed to see by.
