Hide specific posts in Previous section?

Is it possible to remove irrelevant posts that have different tag variables in the Previous and Next post sections?

For instance, a post with the tag "dog" should not display any posts with the tag "cat" in the Previous section.

Here is my current solution:

{{#tagged.dog}}
{{^tagged.cat}}
{{previous}}
{{/tagged.cat}}
{{/tagged.dog}}

However, this only solves half of the problem. If the previous post had the "cat" tag, then there will be a blank space in the Previous section. This is because it is a linear filter rather than an iteration loop.

I'm looking for a better approach that not only blocks out irrelevant posts but also shows previous posts with the same tag.