It will be a very sensitive tough issue, for a lot of people. When it comes to any anime or OVA adaptions I think there can be a big degree of separation since like you said the man himself isn’t onscreen and it is not even live action itself. Plus when it comes to any filler then it isn’t really his work at that point.
In the cases of say Mike Tyson and Mel Gibson their redemption of sorts was achievable due in part to no cases of child porn and the like against them. That is a whole different level of criminal, for good reason of course.