The 'Megan Rhys' series by C.L. Taylor, though more psychological suspense than straight police procedural, deserves a look. They're standalones, but her work consistently traps ordinary people in extraordinary, psychologically terrifying situations. The 'crime' is often interpersonal, and the suspense comes from the slow unraveling of a character's sanity or safety. It's a pure, claustrophobic blend where the crime and the psychology of the victim are one and the same.
Finally, for a series that blends psychological games with outright horror, checkout 'The Dark Half' or 'Misery' by Stephen King. While not a series with a single detective, King's recurring theme of the writer's psyche under threat is its own genre. The mind games in 'Misery' are a brutal, confined duel between a creator and his "number one fan." The complexity is in the raw, powerless terror and the exploration of addiction, creation, and captivity. It's a primal psychological thriller that gets right to the heart of fear.