Mystery and science fiction are cousins, not strangers. To answer the question about how I wrote a mystery after publishing several SF novels, I'm going to focus on some of their shared characteristics and a few of the ways in which they differ.
But first I need to mention that the transition wasn't as great as it could be in my case because most of my SF novels already include mystery-suspense elements. They feature a private eye on a distant planet, an amnesia/murder tale aboard a space station, an undercover operative going back home again and finding an old flame in trouble, a starship hijacking, and an investigative reporter wondering why one news team is often the first to reach a new disaster.
One of the ways the fields differ is in emphasis. Mysteries typically spend more word-count on characters and clues. SF tends to focus more time on technology, ideas, and setting. But both can transport you to a whole new place. Raymond Chandler's mysteries take Southern California of the 40s and make you feel you grew up there and can't wait to get back. Jack Finney's SF makes Manhattan feel like a wonderful place to live even if you're a small-town lover.
Both genres have to introduce you to the story world, but explaining modern day Colorado, as in Pushback, requires fewer strokes than building a newly discovered frontier world on the outer fringes of a galactic federation.
If you make a Venn diagram of SF and mystery it might look like a graphic of Jupiter and Earth, overlapping enough to give a cat's eye of commonality, but SF can explore possible futures with thought experiments, provide cautionary tales, speculate about where trends might take us, or just veer way off into unmapped territory. In that big overlapping area, though, most mystery and SF rely on plain old storytelling, compelling characters, catharsis, and showing how good can triumph over evil.
Both genres are ultimately about people and how they operate under stress. SF looks for new environments and societal changes and technological revolutions to put new stresses on people, but people are pretty much the way they've been for tens of thousands of years. We get jealous, we lust for power, we want to be loved. And those factors of course drive characters in mysteries, too.
All fiction takes a character and puts him or her to the test and our hero either wins and we feel a sense of catharsis, or our hero loses in a way that illuminates truths about people, and about that one person's character.
In mysteries, as in SF, we take familiar characters and put them into the most stressful times in their lives. Both have to be convincing. People have to act like people do, or have a good reason. Made up stuff, whether in mystery or SF, needs to have the air of verisimilitude.
And both genres build on what we know. If our goal is to show our characters' lives twisted by an alien invasion, or show how confrontation with a technologically advanced civilization might play out, we still often summon or exaggerate images and experiences from past wars, or, for example, the arrival of Europeans in America.
One big area of difference is that many SF stories could not even exist without the SF element. Flowers for Algernon by Daniel Keyes is about a retarded man who undergoes experimental therapy to increase his intelligence, which then elevates him to the point where he can see the flaw in the experiment. But aside from the SF aspect, it's still a story about a person who faces a huge obstacle and works toward a desirable goal, only to find out some things can remain beyond his grasp. So, underneath it all, that story is hardly unique to SF.
Compartmentalization. Both genres have plenty of subgenres. SF includes time travel, alternate history, military, dystopian futures, etc. Mystery features PIs, amateur sleuths, police procedurals, political thrillers, legal thrillers, and more.
Clues. In both fields, the writer stages a series of clues about where the story is leading and where it will end. Clues can be about character motivations and actions, but in SF clues can also point to understanding how the world works. And SF adds the sense of wonder.
Stranger Things lends itself to illustrating the contrast. We have a small town with a divorced sheriff, some interesting kids who are just discovering who they are, and on the edge of town is a secretive government installation. When one of the kids goes missing, the story could have gone multiple directions. In one story it could have explored local politics and a powerful city leader with a long-hidden dark side.
In the upside-down story, it focuses on secret experiments, a monster, and superpowers. But both stories illuminate the relationships between the kids and adults and put the characters to the test. Both make you wonder what's going on here. What's going to happen next? What pattern I'm not seeing yet explains these events? How will our heroes get out of this mess?
In some books, the difference between mystery and SF can ride a fine line. Several of L. P. Davies' novels could be either mystery or SF until the last chapter. Isaac Asimov's The Caves of Steel has a futuristic setting, but it's essentially a detective story featuring an odd-couple pair of cops, where one is a robot.
So the short answer about how one moves from SF to mystery is that while some of the trappings and settings look a lot different, at the core of both genres are characters we can care about. Or, as Buckaroo Banzai says, no matter where you go, there you are.
John E Stith's PUSHBACK is out Nov 1st 2018.
John E. Stith's novels include REDSHIFT RENDEZVOUS (Nebula Award nominee from Ace Books), MANHATTAN TRANSFER (Hugo Award Honorable Mention from Tor Books), REUNION ON NEVEREND, and RECKONING INFINITY (on Science Fiction Chronicle's Best Science Fiction Novels list for its publication year, on the Nebula Award preliminary ballot).
His other novels are SCAPESCOPE, MEMORY BLANK, DEATH TOLLS, and DEEP QUARRY.