You can avoid this by never using the base email address, and having a filter that deletes anything sent to it.

You can further secure by not using meaningful tags and instead use my.address+randomstring@gmail.com and having a filter that moves things tagged with randomstring specifically to . Only you and the people you shared ‘randomstring’ with will then be able to get through your filters, neither deleting the + nor altering it will cause emails to land in your inbox.