The biggest problem plaguing the Fantastic Beasts movie series isn't J.K. Rowling’s scripting efforts – it's director David Yates. Fantastic Beasts: The Crimes of Grindelwald is currently the worst-reviewed movie in the Harry Potter franchise, with 2016's Fantastic Beasts and Where to Find Them being the second worst-reviewed movie in the series.

Critics have lambasted Fantastic Beasts 2's pacing and its plot, while fans have balked at many of The Crimes of Grindelwald's many retcons and plot holes. Indeed, the furor has expanded so quickly that series stalwart Ezra Miller called for fans to trust in J.K. Rowling. It's true that many of Fantastic Beasts 2's problems do stem from the franchise’s creator. After all J.K. Rowling has a considerable amount of creative control over all of the Wizarding World’s projects. But filmmaking is not the same as novel-writing. It’s a far more collaborative process.

Related: J.K. Rowling Is Turning Into George Lucas

That is why producers Steve Kloves (who wrote all the Harry Potter screenplays), David Heyman, and Lionel Wigram are also responsible – as is director David Yates. Certainly, Rowling is not the only party to blame for The Crimes of Grindelwald’s confusing narrative. The strength of a script may make or break a movie, but directors play a major part in relating that story to the audience, from pre-production to the overseeing editing on the final cut. The Crimes of Grindelwald’s convoluted structure is as much Yates’ responsibility as are its performances and visuals. But the main recurring problem with Yates' Harry Potter-related movies is the realism that permeates his oeuvre, which was suitable for the darker, more political Harry Potter movies. But nearly 10 years after he directed Harry Potter and the Order of the Phoenix, Yates’ directorial style evidently cannot fully accommodate the fantasy – or the scope – of the Wizarding World.

The Harry Potter novels detail the sights, sounds, and smells of the universe and its magic. It bursts with vibrancy, variety, and movement. Naturally, this is hard to translate to a film, but thanks to the various directors who worked on Harry Potter prior to Yates, the series achieved this, with a wide – if somewhat inconsistent – diversity of onscreen enchantments. In contrast, in all of the movies that followed Harry Potter and the Half Blood Prince, most of the onscreen spell work merely consists of red, green, or blue/white flashes with whip crack noises. Under Yates, the magic has become too mundane. And this directorial mismatch becomes clearer when watching all of the duels across the Harry Potter and Fantastic Beasts movies.

The Goblet of Fire’s climatic duel between Harry Potter and a resurrected Lord Voldemort still stands as one of the franchise's finest sequences – so much so that the filmmakers borrowed the Priori Incantatem effect for Dumbledore and Voldemort’s battle in Harry Potter and the Order of the Phoenix. Harry Potter and the Goblet of Fire's showdown doesn’t work just because of its heartrending score, or the emotional heft behind this battle. Moreover, as Harry and Voldemort’s wands connect, the ways that the camera and effects are utilized create a sense of visceral power in the sequence. It may be CGI, but the clash between these two enemies feels visceral and tangible.

There’s little comparison between this and when the disguised Grindelwald singlehandedly takes on all of MACUSA’s Aurors in Fantastic Beasts and Where to Find Them. Admittedly, by this point, audiences haven’t had nearly as much time to become as invested in these characters – and this conflict – as they are with Harry’s plight in Harry Potter and the Goblet of Fire. Nevertheless, the climax of Fantastic Beasts had a chance to show just how unusually powerful Grindelwald is. Harry Potter and the Goblet of Fire showed how impactful a two-person duel could be – Fantastic Beasts could have fittingly generated a corresponding sense of drama and power. Instead, Grindelwald's assault is over in less than a minute, and it's peppered by indistinct CGI.

Related: What Fantastic Beasts 2's Bad Reviews Are Missing

Instead of improving, The Crimes of Grindelwald has only continued this trend. The CGI of Grindelwald’s fiery creatures is particularly impressive near the Fantastic Beasts 2's ending. Yet the sequence feels oddly empty because Yates doesn’t attempt to convey the depths of danger that Newt Scamander and his friends are in.

Ultimately, this is somewhat worrying when considering where the Fantastic Beasts series is headed: the Dumbledore and Grindelwald duel in 1945. Throughout the Harry Potter novels and films, the spectacular battle has been referred to in reverent, almost mythic tones. Though its highly plausible that Rowling will be adding more twists and turns to this famous rivalry, the final confrontation between Dumbledore and Grindelwald should be a suitably epic wizards duel. Fans will, therefore, be concerned that Yates is likely to return for Fantastic Beasts 3. After all, they will now expect something similar similar to Harry and Voldemort’s flighty, flashy duel in Harry Potter and the Deathly Hallows - Part 2, which, in turn, could have been a taut and dramatically potent confrontation.

This is not to say that Yates is necessarily a bad director, or that the series needs a total makeover after Fantastic Beasts: The Crimes of Grindelwald. On the contrary, David Yates has produced some fine sequences and performances for the Harry Potter franchise. Plus, the amazing cast, sets, costume design, as well as James Newton Howard’s scores, are not only the saga’s strongest aspects, but they’re also among the best in modern blockbusters. All that’s needed is a new and strong authorial voice to tailor them to Fantastic Beasts’ narrative. Directors Mike Newell and Alfonso Cuarón may now be busy with other projects, so their filmic flair is unlikely to grace the halls of Hogwarts again. Yet another new creative behind the camera could ensure that the characters, the magic, and the alluring power of the Wizarding World can be restored once more.

More: Fantastic Beasts 3: Release Date, Story Details & Every Update