With those words, a construction worker greeted me last Thursday as I approached the entrance to David Adjaye's Sugar Hill development, on the site of a former brownfield on West 155th and Saint Nicholas Avenue in West Harlem.

Some of New York City's poorest residents will start moving into the 13-story, neo-brutalist housing complex in August. It's a building that Mayor Bill de Blasio referred to as the "epitome of everything we want to do with housing across the spectrum of incomes.”

While many applaud the purpose behind the new building, which will also have an early-education center and a children's museum, few seem to think much of its design. "It has nothing to do with the rest of the neighborhood," said another worker on site. "Nobody likes it."

Bloggers, art critics, and armchair architects are calling Adjaye's eccentric design everything from "imposing" to something that "looms like a ruined bastion." Much of this dislike has to do with the look of the building itself, but a large share of the backlash springs from concern over how it relates to the surrounding area.

Added to the National Register of Historic Places in 2002, the Sugar Hill district came of age during the Harlem Renaissance of the 1920s, continuing to serve as a popular destination for New York's wealthy and intellectual African-American communities through the 1950s. That past can be seen in the ornate late 19th- and early 20th-century row houses that give the neighborhood its signature style. These were the places that men such as W. E. B. Du Bois, Thurgood Marshall, Willie Mays, and Duke Ellington once called home.

But today, 70 percent of children in the neighborhood are born into poverty. The imposing Polo Grounds towers, a violence-plagued public-housing behemoth built in 1968, stands at the bottom of the hill—a reminder of the problems that fracture much of the area today.

That more recent history is what nonprofit developer Broadway Housing Communities (BHC) is concerned with. Established in 1983, BHC oversees hundreds of housing units in six other buildings around Washington Heights and West Harlem. Each one provides not only a place to live, but also support services.

At the Sugar Hill development, where BHC will have a ninth-floor office, workers are putting the final touches on 124 apartments ranging from studios to three-bedroom units. Twenty-five of those will be occupied by people in the city's emergency shelter system. The rest will have rents adjusted to the low (and very low) incomes of the families moving in. The building received about 50,000 housing-lottery applications for the slots.

Funded by a combination of state and philanthropic funds, the $80 million project was celebrated at a press conference last month, where de Blasio spoke of it as an example of what he'd like to see more of in his city. Not only does it help the city reach the mayor's goal of building and preserving 200,000 affordable housing units in the next 10 years, but it also fits into his new, city-wide pre-kindergarten plan.

Walk through the public plaza along Saint Nicholas that slopes down to the building's first floor, and you'll find the education center, set to open in September. Inside, as many as 200 kids ages 5 and under will spend their school days surrounded by floor-to-ceiling glass windows that look out onto the neighborhood. They'll also have access to two interior courtyards in which to play.

The Children’s Museum of Art & Storytelling—which will have space for kids to make their own art, room for interactive exhibits, as well as performance space—will be located in the basement, accessible by a first-floor entrance. Once completed, windows from the ground floor will bring natural light into the underground space. Adjaye, who has a New York office but is based in London, refers to the feature as a "light chimney."

Illumination is clearly one of Adjaye's main design focuses throughout the building. Vertical lighting fixtures decorate the hallways on each floor, providing a surprisingly stylish touch to a part of residential architecture that often receives little attention. Inside the apartments, asymmetrical windows give personality to otherwise straightforward spaces. But depending on how they align from room to room, that can either be an exciting or frustrating feature. In person, it appears as the building's most notable architectural flaw.

Terraces also emerge on the second, third and ninth floors of the building, as well as on the roof, designed to appear as imperfectly stacked boxes. From the ninth-floor terrace, you can see Yankee Stadium, the Polo Grounds towers, and the Harlem River.

Competing with 26 other firms for the commission, the BHC's design committee unanimously selected Adjaye's proposal in 2010. A New Yorkerprofile of Adjaye from last September noted that, soon after his plan was first shown, angry letters came in to the BHC in response to his ultra-modern design.

In the same New Yorker piece, Adjaye said that in his Sugar Hill project, he wanted "a building that acknowledges its surroundings." But at first (and probably second, even third) glance, the tower appears to have little in common with its gothic revival neighbors. If you look hard enough, though, context appears. Up close, the building's precast concrete panels are as harsh as those of Paul Rudolph—one of America's leading modernist architects. When sunlight hits the facade at an angle, abstract references to the roses on the facades of nearby row houses come into focus.

The Polo Grounds Towers as reflected through the 9th floor windows of David Adjaye's Sugar Hill development. (Mark Byrnes)

Unimpressed, New York Magazine's Justin Davidson writes that compared with the stone-carved flowers Adajye references, "his digitally drawn roses, twining around cheap air conditioning units, seem crude, the product of an evening spent fiddling with Photoshop."

Inside, an art exhibit offers a sense of what BHC wants to be to the community once construction is done. Titled "If You Build It," the exhibit is run by No Longer Empty, a nonprofit that uses unconventional spaces to show art. Usually, that means setting up in an older, underused structure. In Sugar Hill, it means a construction zone and inescapable summer heat.

Some of the art on display, inside and out, for the "If You Build It" exhibit. (Mark Byrnes)

A visit is worth the sweat. The show takes up multiple floors, using rooms and common spaces to display art from more than a dozen artists exploring issues of race, commerce, and community. The exhibit, which runs through August 10, functions as an art show as much as the building's introduction to the city. On the ninth floor, Scherezade Garcia's gold-painted inner tubes and luggage tags provide a stunning representation of vulnerabilities among Spanish-speaking immigrants in the States. Down the hall, Nari Ward playfully addresses cultural appropriation with his cans of "Sugar Hill Smiles." Intrigued by the use of Harlem as a brand to market products not made in Harlem, Ward had locals smile into a can before sealing it and selling it for $10. (Proceeds go to BHC programming.)

From the art within the building to the architecture of the structure itself, everything in view is done with Sugar Hill's empowerment in mind. The facade may take some getting used to, but what's happening inside should soon turn the place into a neighbor that's easy to love.