Authority

Hi ladies,
I have a question..
It’s like this, my partner and I have had a baby for 4 weeks now.
My partner and I argue a lot and our relationship is going really badly at the moment.
He barely pays any attention to the baby, is constantly on his phone, and doesn’t even get out of bed.. He sets his alarm and puts it on the other side of the room. But when his alarm goes off, he lets it ring for an hour first, then turns it off and goes back to sleep.
We arranged joint custody right after the birth, but I want it to be changed so that he no longer has custody because if we break up now, I have no trust at all that he will take responsibility.
Does anyone have experience with this or know how I can change custody without him noticing or finding out?

Please help!