Grid Sorting with Table in a Grid Column

I’m having a problem with my field type in Grid. My field type produces its own table. When you drag a Grid row and that Grid row gets over the top of my table in my field, the Grid row will go into my table and can be dropped there. This causes all hell to break loose as you might imagine.

Is there a way Grid’s sorter can be tightened up with specific classes or targets or something?

The sorter I use in my fieldtype has options for specifying specific targets and selectors.

Here’s a quick video of the issue in action. I disabled my sorter for the demo to make sure the problem is not my sorter.

Video of Issue in Action