It is my firm belief that the essence of any story can be summed up in one word - CONFLICT. Without conflict, there is no story.
To see what I mean by this, take any well-known piece of work and remove the conflict, and examine the result. Let's take the Harry Potter series as a example - a young boy from a happy, secure background of expert wizards is groomed from birth to be a great wizard; he goes to Hogwarts, studies hard, graduates with flying colours, and lives happily ever after. I think if JK Rowling had written it in this way, the series probably wouldn't have sold in the hundreds of millions!
This is because a story without conflict is not a story, it is a description of a slice of a character's life. A story needs to be something more than this for it to qualify as such - it needs to take the reader on a journey.
How to achieve this? Let's look at a writer who has a single protagonist. This character needs a goal - what is he or she trying to achieve? If there is no obvious answer, this will be a major problem in creating a powerful story.
So the writer has now given the main character a goal. Now what? The writer cannot simply describe how that goal is achieved, for the reader will feel there is something lacking. The writer therefore needs to wire CONFLICT into every aspect of that character's journey, if it is to be interesting.
One large part of this will be - for want of a better word - the 'baddie'. This point was drummed home to me a short while ago by my three-year-old son. I was making up a bed-time story for him and my daughter, and half-way through, he stopped me. 'But where's the baddie?' he asked me. And he was right. There was no baddie, and therefore there was no story.
Now the 'baddie' doesn't have to be a 'villain' in the normal sense, but there should be one major source of conflict in the story that makes it hard for the protagonist to achieve his or her goal. It could be a character (as in many action thrillers), nature (wild weather, a natural disaster, fierce seas, an animal, etc), society itself (the character wants to achieve something that goes against societal norms), technology (robots, computers, etc), the supernatural (as in many ghost and horror stories), and even the self (an internal source of conflict, as in when the character's desires are at odds with their own morals or values).
When the main source of conflict has been identified, the writer will then need to make sure that conflict does not arise sporadically, just in the presence of that single source, but in every single scene.
Let's say the character needs to get from one location to another. The writer might compose a beautifully written scene to achieve this, describing the movement from the kitchen to the garage, how the engine sounds when it starts up, how the character then takes the car gently through the suburbs before opening it up on the wide country roads, and on and on.
Another option would be for the character to get into the car, try and turn the ignition key, and then be faced with a car that won't start. This is low-level conflict - the character wants to achieve something (get from A to B), and there is an impediment in his or her way.
Or two armed men in balaclavas could be waiting in the garage, as a more concrete impediment to the character's desires.
So the writer has a choice to make. With the first option, no matter how beautiful or wonderful the language used by that writer might be, the scene achieves nothing in the context of story. And so the writer either has to insert some conflict, some impediment to the protagonist's progress, or cut the scene altogether. If there is nothing to get in the character's way during the journey, the scene might as well open in the new location and do away with the journey itself.
So for the overall story, a writer should try and establish conflict through not only a protagonist, but also an antagonist. And then conflict should be further woven into every aspect of the work, every chapter, every scene.
A novel without conflict is merely a description of events - it is not a true story.
So please - don't forget the 'baddie'!