RHS Balancer bearing "loose" inside engine case - CRF450

So I rebuilt the engine of my CRF450 (bought it broken, never ran it) and it runs perfectly. The current problem I have is a coolant seal leak which I have narrowed down to the fact the balancer bearing on the right engine crank case sits loosely inside the case. Now, I know you don't need to belt it in with a socket normally because it has those tabs so I expect to just push it in, but you can feel and see movement. You can also move the balancer shaft up and down.

Has anyone had this problem before? Any decent fixes? I've seen on other forums people boring it out a little bit and putting a spacer inside it. Good idea but I want the engine to stay together if I can help it because the gaskets are new. I put some bearing glue made by loctite to just hold it in place, but that didn't seem to help a great deal. I've replaced everything as well, including the water pump shaft, so I'm adamant it's the only thing it can be.

Any ideas?

EDIT: I'll mention that you can move it, but the slightest amount. Maybe like... 7 thou of an inch, but just enough to mess with the seals.