At Fort Monroe on May 24, 1861, Union Maj. Gen. Benjamin Butler told Confederate Maj. John B. Cary that slaves Frank Baker, Sheppard Mallory and James Townsend would not be returned to their owners. Said Butler, "I am under no constitutional obligations to a foreign country, which Virginia now claims to be. ... I shall hold these Negroes as contraband of war."
On the night of May 23, 1861, Frank Baker, Sheppard Mallory and James Townsend slipped into a skiff and rowed across Hampton Roads from Sewell's Point in Norfolk to Old Point Comfort.
Being out at night without permission was dangerous for slaves. That they didn't know what was ahead made their trip even more frightening.
But they knew what they were leaving: fortifications for a Confederate artillery battery that they helped build under the command of their master, Col. Charles K. Mallory.
And they knew that he was going to take them south for more war work, forcing them to leave their families in Hampton behind.
"Whatever they had, they knew they would get a better shake from the Northern troops," says Gerri Hollins of Hampton, keeper of the flame for the Contraband Historical Society, which has the lot of Baker, Mallory and Townsend as its genesis.
The contraband story is one of courage. It's one of hope.
And it's one of blind luck.
"They had no idea what they could expect at Fort Monroe," says Robert F. Engs, a history professor at the University of Pennsylvania. He's the author of "Freedom's First Generation," about blacks in Hampton from the Civil War through Reconstruction.
Had Baker, Mallory and Townsend escaped two days earlier, they probably would have been sent back by Fort Monroe's post commander, Col. Justin Dimick. The Fugitive Slave Act required it.
So did President Abraham Lincoln, who censured other officers for trying to free slaves. Lincoln was still trying to hold the union together.
Had Maj. Gen. Benjamin Butler not been sent to Fort Monroe on May 22 - "to keep him out of trouble," Engs says - the slaves probably would have been returned.
It had happened to others who sought refuge at the fort.
Had Hampton not voted on May 23 to endorse the ordinance of Virginia's secession, the slaves probably would have been returned.
But on that night, Virginia was no longer a part of the United States.
That confluence of events gave Butler, a lawyer by trade, the evidence he needed to tell Confederate Maj. John Cary - an emissary sent by Mallory to reclaim his property - that Baker, Mallory and Townsend would be staying at Fort Monroe.
It was a reunion of sorts.
Butler and Cary had met at the 1860 Democratic National Convention in Charleston, S.C., where both voted for Jefferson Davis to be the party's nominee to run against the Republicans' Lincoln. Stephen Douglas ultimately got the Democratic nod.
(In 1864, Butler turned down an invitation to run as Lincoln's vice president, saying he would do so only if the president promised "that within three months after his inauguration, he would die.")
In his memoirs, Butler writes that he told Cary on May 24, 1861: " 'I mean to take Virginia at her word, as declared in the ordinance of secession passed yesterday. I am under no constitutional obligations to a foreign country, which Virginia now claims to be.'
" 'But you say we cannot secede,' (Cary) answered, 'and so you cannot consistently detain the Negroes.' "
" 'But you say you have seceded, so you cannot consistently claim them. I shall hold these Negroes as contraband of war, since they are engaged in the construction of your battery and are claimed as your property.' "
Butler offered Cary a deal: If Mallory were to take an oath of allegiance to the United States, his property would be returned.
That wouldn't happen.
From that day, the term "contraband" became synonymous with "freedom" for slaves. It became a pejorative in the jargon of their masters.
And Fort Monroe - built with slave labor more than 30 years earlier - became the "freedom fort."
"Fort Monroe is a beacon of hope for our people," says Hollins, descended from a contraband. "It was the birth of freedom for our people en masse."
The day after Butler's edict, eight more slaves showed up at Fort Monroe.
The day after that, 47 were there, most camped near the chapel inside the walls of the fort.
By war's end, as many as 10,000 had appealed for contraband status at Fort Monroe and its companion, Fort Calhoun, which later became Fort Wool. They had spilled into Hampton, which had been burned and evacuated by its citizens to deny it to the Union troops.
"Think about it," says Mike Cobb, curator of the Hampton Museum. "Think about what Fort Monroe must have looked like then - some kind of monolithic structure, not only to slaves but to everyone.
"It had to be intimidating. Put yourself in the role of a slave in the field and approaching this great place, appealing for your freedom."
Baker, Mallory and Townsend went to work for the Union army for $8 a month plus rations. Many of those who followed were similarly employed - women at half pay.
But shortly after Butler left later that year for New Orleans, the quartermaster under his successor - Maj. Gen. John Wool - began cheating the contraband. Wages were withheld. So was food.
By then, the former slaves had created the Grand Contraband Camp, that area in Hampton that includes today's Union, Lincoln and Grant streets. Contraband moved in among the rubble of Hampton, building houses of roughed-out timber. They called the area "Slabtown."
"When you stop and think about it," says Engs, "they might have been living under worse conditions than they had been on the plantations."
Except at the end of the day.
"The floors of many of the houses were dirt, and they were rough houses," says Cobb. "It was a quasi-freedom, but when they shut the doors, they were free for the first time in their lives, living in a home of their own."
In 1863 residents of Slabtown and the Grand Contraband Camp gathered around a tree, now called Emancipation Oak on the campus of Hampton University. There they heard the Emancipation Proclamation read for the first time.
Reports are that they rejoiced.
"I don't think they realized that they were excepted from the Emancipation Proclamation," Engs says.
Well down the document - just past three paragraphs of flowery language - is written that Virginia is included "except the forty-eight counties designated as West Virginia, and also the counties of Accomac, Northampton, Elizabeth City (now Hampton), York, Princess Ann (now Virginia Beach) and Norfolk."
They were still contraband.
"Technically, they weren't freed until the 13th Amendment," Engs says. That was in 1865.
Hollins of the Contraband Historical Society takes a philosophical approach - much, she believes, as her ancestors did.
"If you are working for wages and believe you are free, you're free," she says.
The descendants of the contraband flourish in Hampton, but the symbols of their ancestors are vanishing.
There is no historical marker for the Grand Contraband Camp or Slabtown. Houses in the old neighborhoods are giving way to redevelopment.
"The story will continue, but people want to see something to remind them," says Frank Earnest, an officer in the Sons of the Confederate Veterans and member of the Contraband Historical Society. "They want to be able to look out at something tangible. It's hard to get when you look out and see another bank."
The Hampton Museum has a display cabinet dedicated to the contraband, but there's little in it, save for drawings and an ax that Cobb says symbolizes the building of the houses of Slabtown.
Fort Monroe is scheduled to vanish after a review by the Base Realignment and Closure commission.
"Fort Monroe won't be gone," Cobb insists. "It's always going to be there in some way."
Its Casemate Museum will probably survive in some form, says Dennis Mroczkowski, the executive director. But the Casemate has only a single window display dedicated to the contraband.
"You can't display what you don't have," he says.
Hollins rails at what she sees as her heritage vanishing and campaigns for a contraband museum.
"When your history becomes null and void, you become invisible," she says. "That's what is happening in Hampton."
Engs lived at Fort Monroe for two years and researched "Freedom's First Generation" while his father was stationed at Fort Eustis. He hoped that his book would help perpetuate the contraband story.
He also teaches it in his classes.
"It's a story that may be well-known in Hampton but not much anyplace else," he says, "and it needs to be told."
It's the story of three men who led a migration to Fort Monroe, taking a giant step toward freedom.