First off, I would like to point out that I have never been a big fan of the term "Survival Horror" for the sort of game I am about to describe. It technically fits, in that there is usually a horror theme to the game, and you usually have a main character whose primary goal is to survive (although there's a surprising number of highly notable exceptions there). Really though, dying is something you want to avoid in most games, and horror is a thematic genre label, with no bearing on gameplay. I'm not bothered though that it's too vague to really define the genre properly, you can say that about any game genre label really. What bothers me is that it fits well enough for new entrants to the genre to focus on fitting the label, rather than the proper gameplay style.
A more fitting label for the genre would honestly be Action/Adventure (probably with the word Horror tacked on the front to clarify that we aren't talking about the likes of Zelda or Prince of Persia). Predominantly we're dealing with Action games in terms of how the games are played, but with some strong Adventure game influences, mainly in the puzzle solving aspects, but also the general tone and plot stylings these games have. If those aren't in your game, your game doesn't really fit in this genre. Most likely it's just an action game with a horror theme to it.
Having gotten that off my chest, on to the subject at hand. Most games in this little genre here set out to do two things. 1- Be an enjoyable game (although there are deliberate exceptions to this) and 2- Be scary. I'm not going to get into the first of these two points in this article per se, but if you get the horror down right, you'll have a good head start on the rest.
As stated above, if we're doing this genre properly, we need some Action elements and some Adventure elements. Both are quite vital to the mix here, and I'll address them each in turn. I'll start with Adventure though, because that's where the industry in general has really been slacking off.
Horror and Adventure are a perfect match for each other. There's a few such pairings of theme and gameplay that click this way, and different reasons for each of them. The reason in this case is that effective horror is totally dependent on immersion. If I don't feel like I'm part of the story, I have no reason to be afraid of any of the things in the story. Adventure games meanwhile are, by nature, the most immersive story driven genre out there. With other games, having a good story that draws you in is something of a bonus. You want to have one if you can, but it isn't tied to the gameplay in any meaningful way, and it's possible to make a great game without one. This isn't the case for Adventure games though. If you're making an Adventure game, and you're doing your job right, the player won't be able to finish the game unless they can get inside the main character's head, think about things from their perspective, and work out a long term plan on what they need to do, as well as approach every puzzle from a viewpoint unique to their character's abilities and the unique quirks of the game's setting. Forming that kind of connection gives you a far greater emotional investment in things than you'd otherwise have, giving the horror elements that much more bite.
So, the better job you do of making an adventure game where you need to change your perspective, the better your immersion. There's a catch though. If you make puzzles too hard, the player's going to get stuck for a while, and the more time they spend staring at a puzzle, the more distant we're getting from the scary stuff, and the more we lose the mood. This is bad. So, we want to make sure puzzle solving, while in there, doesn't take up too much of our time with head scratching. There's a few ways we can do this.
First, we can make the puzzles really easy. This takes out the thought involved though, so we lose the added immersion we threw them in there for to begin with. You want to be easier than a straight-up Adventure game of course, but there's a limit to it. You could throw time limits around, either on the game as a whole, or per puzzle, to keep players sweating while they work these things out. Another option is to limit the possible solutions quite a bit, but punish people for wrong answers. Hurt'em, throw in more monsters later, or go the Shadowgate route and use some fun imaginative deaths. Kludging doesn't take long, but has nasty consequences. It should also go without saying, but looking at games from recent years, it needs extra stress. Make sure all the puzzles have the proper in-game context. If there's an arbitrary sliding tile puzzle in the middle of a monster filled mansion, you will once again lose that sense of immersion.
Then there's the action side of things. If the only thing that's scary about the monsters (or whatever I'm supposed to be afraid of) is how they look and sound, they aren't going to be scary for very long. Anything that doesn't pose a legitimate threat to my life in terms of game mechanics is ultimately harmless, and it doesn't matter how many gaping maws that horror from beyond has if no part of me is ever going to be in one. The typical solution to this is to have everything kill you pretty much instantly if it gets a hit in. That's a good start, but if you stop there, you haven't done your job.
The real trick to throwing horror into your gameplay is to account for some of the basic sources of fear. Sudden shock is easy, anyone can pull that off, and most people seriously overuse it. I'm talking about things bursting through windows here. Dark tight spaces is another easy one, narrow hallways and tight camera angles deliver nicely. The real payoff though is dread. All the A-list Survival Horror games play off dread in a big way. A lot of it is just through their visual style, but you can work it into the gameplay too. The best example that comes to mind for this is Resident Evil 2, so if you'll endulge me, I'll use it for a specific example.
Resident Evil 2 has three resources to make you juggle, basically. Inventory space, healing items, and ammo. You need a lot of inventory space to make sure you can hold all the puzzle items you need at any given time, otherwise you're going to have to backtrack through dangerous areas a lot. Healing items of course are needed when you can't avoid taking damage. Ammo meanwhile is for killing monsters, which ultimately prevents damage to begin with.
The real key to making this game works is how all three of these resources are severely restricted. You don't have enough inventory space to run around with all your puzzle items, decent armaments, and a few healing items, you generally always have to pick something to leave out for a while, or carry an anemic mix. Healing items are finite, so there's only so much damage you can take over the course of the game, and it's a heck of a lot less than if you never kill a thing and try to run blindly through everything. Then of course we have the biggie people always leave out, ammo is finite, to the extent that there is only enough of it to kill maybe every 3rd monster in the game, give or take.
At some point, it'll sink in to the player that they need to conserve these resources as much as possible. They need to make trade-offs, they need to avoid mistakes, they're now worrying about this stuff. If you really orchestrate things properly, at any given point, the player is going to be under the impression that they might not have enough stuff left to get through the next section of the game, or if they do, it'll be by the skin of their teeth. You have this constant dread established because they know the difficulty's bound to climb sooner or later. Tada. Your player and your main character are now worried about the same things. You've got even more added immersion, and the very mechanics of the game are kind of scary to boot. Plus, you've got the seed for some nice satisfyingly challenging gameplay.
Again of course, limits exist on all this. Make things too hard, and people spend too much time dying, which leads to replaying the same parts of the game over and over. Not only does this lessen the impact of the game's content after a while, it's just plain frustrating. The real secret is to trick the player into constantly believing they're underprepared, while secretly making sure they have enough of everything they need to squeak by.
It also is important of course to establish your horror in more traditional ways. Strange sounds, labored breathing, twitching misshapen things, and so forth, but again, this isn't an area that seems to give people problems.