I was searching for this exact thing a while ago, trying to find more books that gave off the same specific, heavy mood as something like 'The Secret History' or 'Piranesi'. Honestly, I had to get creative. Goodreads' 'Browse' section lets you filter by genre and then by user-created 'themes' lists—look for stuff like 'books with unreliable narrators' or 'dark academia'. It's clunky but works. Also, TV Tropes is a weirdly deep resource. You can start on a page for a trope like 'Cosmic Horror Story' and just click through the 'Literature' examples. It's not a direct search, but you'll find titles linked by specific narrative devices and atmospheric elements that pure plot summaries miss.
For a more direct approach, I've heard people recommend apps like StoryGraph, which has a 'Mood' and 'Pacing' filter. You can search for 'atmospheric', 'reflective', and 'dark' moods, for instance, and it'll pull up recommendations. It's not strictly searching by the plot events themselves, but it gets at the thematic feel, which is often what I'm after when a specific story vibe is stuck in my head. It's less about 'character inherits a house' and more about 'haunting, melancholic exploration of memory'.
Don't overlook old-school forums, either. Reddit's r/suggestmeabook or specific genre subreddits are goldmines for hyper-specific requests. Posting something like 'Looking for books with the theme of enforced conformity breaking down' will get you more nuanced, human-curated results than any algorithm right now. It's slower, but the suggestions often come with little explanations of why they fit, which is the best part.