The first use of the land for military purposes was as the site of a blockhouse in 1663. During the American Revolution it became known as Flagstaff Fort; captured by the British in 1776, it remained in British hands until the war's end in 1783. It became the responsibility of New York State in 1806, and reverted to federal control during the War of 1812. Divided into several smaller units, including Fort Tompkins and Fort Richmond, its present name was adopted in 1864 to honor Brigadier General James Wadsworth, who had been killed in the Battle of the Wilderness during the Civil War.

In 1910, the fort fired a 21-gun salute to former President Theodore Roosevelt as his ship passed through the Narrows on his return from a nearly year-long trip to Africa and Europe. In 1913, ground was broken by President William Howard Taft for a proposed National American Indian Memorial that was to be built on the site of Fort Tompkins. The monument was to include a 165-foot-tall (50 m) statue of an American Indian on the bluff overlooking the Narrows, but difficulties in fundraising and the advent of World War I precluded fruition of the plan.

By 1924, Fort Wadsworth had become an infantry post, and from 1955 until 1974 it was the headquarters of the 52nd AAA Brigade. It then was the site of the United States Army Chaplain school before being turned over to the United States Navy in 1979[citation needed], which used it as the headquarters of its New York Naval Station. The property became part of Gateway National Recreation Area when the Navy left in 1995. Shortly after that, the United States Coast Guard became a tenant in some of the buildings and housing previously occupied by the Navy. As of 2007, they are occupied by the United States Coast Guard's Sector New York and a Maritime Safety and Security Team.

Historic structures include Battery Weed, directly on the harbor, and Fort Tompkins on the bluff above. Both were built in the mid-19th Century and are open to the public on guided tours only. There are several smaller early 20th Century coastal artillery batteries and an overlook with panoramic views of the Upper Bay, Brooklyn and Manhattan. The National Park Service maintains a visitors' center on site and offers ranger-led tours of the facilities Wednesdays through Sundays.

The name "Fort Wadsworth" is also sometimes used to denote the residential neighborhood surrounding the former fort, the neighborhood south of Rosebank, west of Shore Acres and north of South Beach. This neighborhood once had a station on the South Beach Branch of the Staten Island Railway; service on this branch ceased in 1953.

The comic book version of the elite United States military counterterrorism unit G.I. Joe was based in "The Pit," a secret underground base beneath the Motor Pool of the Army Chaplains' Assistants School in Fort Wadsworth. After Cobra discovered the location of The Pit and destroyed it in a surprise attack, G.I. Joe relocated its headquarters elsewhere. Although written in 1982, the book still depicted the fort as the home of the "Chaplain's Assistants School", due to Larry Hama's memories of the fort from his years in the service. Also, Ft. Wadsworth did not actually have a proper motor pool; its equipment was serviced at Fort Hamilton.

40°36′20″N 74°03′15″W / 40.605506°N 74.054163°W / 40.605506; -74.054163

Annadale  Arden Heights  Arlington  Arrochar  Bay Terrace  Bloomfield  Brighton Heights  Bulls Head  Castleton  Castleton Corners  Charleston  Clifton  Concord  Dongan Hills  East Shore  Egbertville  Elm Park  Eltingville  Emerson Hill  Farm Colony  Fort Wadsworth  Graniteville  Grant City  Grasmere  Great Kills  Greenridge  Grymes Hill  Hamilton Park  Heartland Village  Huguenot  Lighthouse Hill  Livingston  Manor Heights  Mariners Harbor  Meiers Corners  Mid-Island  Midland Beach  New Brighton  New Dorp  New Springville  North Shore  Oakwood  Ocean Breeze  Old Place  Old Town  Pleasant Plains  Port Ivory  Port Richmond  Prince's Bay  Randall Manor  Richmondtown  Richmond Valley  Rosebank  Rossville  St. George  Sandy Ground  Shore Acres  Silver Lake  Snug Harbor  South Beach  South Shore  Stapleton  Stapleton Heights  Sunnyside  Todt Hill  Tompkinsville  Tottenville  Tottenville Beach  Travis  Ward Hill  Westerleigh  West New Brighton  West Shore  Willowbrook  Woodrow