Since the 1980s, academics have downplayed the role that Goree played in the Atlantic Slave Trade, arguing that it is unlikely that many slaves actually walked through the door, and that Goree itself was marginal to the Atlantic slave trade. Ndiaye and other Senegalese have always maintained that the site is more than a memorial and is an actual historic site in the transport of Africans to French, Spanish, Dutch, and Portuguese colonies of the Americas, and thus undercounted by Anglophone researchers.
Built around 1776, the building was the home in the early 19th century to one of a class wealthy colonial Senegalese Métis woman trader (the Signares), Anna Colas Pépin. Researchers argue that while the houseowner may have sold small numbers of slaves (kept in the now reconstructed basement cells) and kept a few domestic slaves, the actual point of departure was 300m away at a fort on the beach. The house has been restored since the 1970s. Despite the shrine-like status of Goree Island, historians have argued only 26,000 enslaved Africans were recorded having passed through the island, of the 12 million slaves that were abducted from Africa. Ndiaye and supporters have argued that there is evidence that the building itself was earlier built to hold large numbers of slaves, and that as many as 15 million people passed through this particular Door of No Return.
Academic accounts, such as the 1969 statistical work of historian Philip D. Curtin, argue that exports from Goree began around 1670 and continued until about 1810, at no time more than 200 to 300 a year in important years and none at all in others. Curtin's 1969 accounting of slave trade statistics records that between 1711 and 1810 180,000 enslaved Africans were transported from the French posts in Senegambia, most being transported from Saint-Louis, Senegal and James Fort in modern Gambia Curtin has been quoted that the actual doorway memorialised likely had no historical significance In response to these figures, popularly rejected by much of the Senegalese public, an African historical conference in 1998 claimed that records from the French trading houses of Nantes documented 103,000 slaves being from Goree on Nantes owned ships in a single year in the 18th century.
Despite the controversy, the Maison des Esclaves is a central part of the Goree Island UNESCO World Heritage site, named in 1978, and a major draw for foreign tourists to Senegal. Only 20 minutes by ferry from the city centre of Dakar, 200,000 visitors a year pass through the Museum here. Many, especially those descended from enslaved Africans, describe highly emotional reactions to the place, and the pervasive influence of Ndiaye's interpretation of the historical significance of the building: especially the Door of No Return through which Ndiaye argued millions of enslaved Africans left the continent for the last time. Before his death in 2008, Ndiaye would personally lead tours through basement cells, out through the Door of No Return, and hold up to tourists iron shackles, like those used to bind enslaved Africans. Since the publication of Alex Haley's Roots in the 1970s, African-American tourists from the United States have made the Museum a focal point, often a highly emotion laden one, of pilgrimages hoping to reconnect with their African heritage.  World leaders include the Maison des Esclaves on their state visits: Pope John Paul II, two Presidents of the United States, and Nelson Mandela have all made high profile stops. Mandela was reported to have broken away from a tour to sit alone in a basement cell for five minutes in silent reflection upon his visit here.